- developers - lists.mariadb.org

[Maria-developers] Rev 2740: options for CREATE TABLE (MWL#43) (version after first review). in file:///home/bell/maria/bzr/work-maria-5.2-createoptions/
by sanja＠askmonty.org 05 Feb '10

05 Feb '10

At file:///home/bell/maria/bzr/work-maria-5.2-createoptions/ ------------------------------------------------------------ revno: 2740 revision-id: sanja(a)askmonty.org-20100205170316-gg4nio1p81cpmjop parent: knielsen(a)knielsen-hq.org-20100201190519-b9uktnn90rwwiile committer: sanja(a)askmonty.org branch nick: work-maria-5.2-createoptions timestamp: Fri 2010-02-05 19:03:16 +0200 message: options for CREATE TABLE (MWL#43) (version after first review). Diff too large for email (1280 lines, the limit is 1000).

2 1

[Maria-developers] Updated (by Serg): Store in binlog text of statements that caused RBR events (47)
by worklog-noreply＠askmonty.org 05 Feb '10

05 Feb '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Store in binlog text of statements that caused RBR events CREATION DATE..: Sat, 15 Aug 2009, 23:48 SUPERVISOR.....: Monty IMPLEMENTOR....: COPIES TO......: Knielsen, Serg CATEGORY.......: Server-Sprint TASK ID........: 47 (http://askmonty.org/worklog/?tid=47) VERSION........: Server-9.x STATUS.........: In-Progress PRIORITY.......: 60 WORKED HOURS...: 20 ESTIMATE.......: 35 (hours remain) ORIG. ESTIMATE.: 35 PROGRESS NOTES: -=-=(Serg - Fri, 05 Feb 2010, 14:04)=-=- Observers changed: Knielsen,Serg -=-=(Guest - Fri, 05 Feb 2010, 13:40)=-=- Category updated. --- /tmp/wklog.47.old.9197 2010-02-05 13:40:36.000000000 +0200 +++ /tmp/wklog.47.new.9197 2010-02-05 13:40:36.000000000 +0200 @@ -1 +1 @@ -Server-RawIdeaBin +Server-Sprint -=-=(Guest - Fri, 05 Feb 2010, 13:40)=-=- Status updated. --- /tmp/wklog.47.old.9197 2010-02-05 13:40:36.000000000 +0200 +++ /tmp/wklog.47.new.9197 2010-02-05 13:40:36.000000000 +0200 @@ -1 +1 @@ -Un-Assigned +In-Progress -=-=(Alexi - Thu, 04 Feb 2010, 09:54)=-=- Low Level Design modified. --- /tmp/wklog.47.old.16174 2010-02-04 09:54:13.000000000 +0200 +++ /tmp/wklog.47.new.16174 2010-02-04 09:54:13.000000000 +0200 @@ -171,35 +171,20 @@ 5. How slave IO thread requests Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -When requesting an event, the slave should inform the master whether -it should send Annotate_rows events or not. To that end we add a new -BINLOG_SEND_ANNOTATE_ROWS_EVENT flag used when requesting an event: +If the replicate-annotate-rows-events option is not set on a slave, there +is no need for master to send Annotate_rows events to this slave. The slave +(or mysqlbinlog in remote case), before requesting binlog dump via the +COM_BINLOG_DUMP command, informs the master whether it should send these +events by executing the newly added COM_BINLOG_DUMP_OPTIONS_EXT server +command: + + case COM_BINLOG_DUMP_OPTIONS_EXT: + thd->binlog_dump_flags_ext= packet[0]; + my_ok(thd); + break; - #define BINLOG_DUMP_NON_BLOCK 1 - #define BINLOG_SEND_ANNOTATE_ROWS_EVENT 2 - - pthread_handler_t handle_slave_io(void *arg) - { ... - request_dump(mysql, ...); - ... - } - - int request_dump(MYSQL* mysql, ...) - { ... - if (opt_log_slave_updates && - mi->io_thd->variables.binlog_annotate_rows_events) - binlog_flags|= BINLOG_SEND_ANNOTATE_ROWS_EVENT; - ... - int2store(buf + 4, binlog_flags); - ... - simple_command(mysql, COM_BINLOG_DUMP, buf, ...); - ... - } - -NOTE. mysqlbinlog, when remotely requesting BINLOG_DUMP by calling the -simple_command() function, should also use this flag if it wants (in case -of the --print-annotate-rows-events option set) to recieve Annotate_rows -events. +Note. We add this new command and don't use COM_BINLOG_DUMP to avoid possible +conflicts with MySQL/Sun. 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -338,10 +323,4 @@ to internals(a)lists.mysql.com about this and suggesting to reserve the event number. -Also we should notice the introduction of the BINLOG_SEND_ANNOTATE_ROWS_EVENT -flag taking into account that MySQL/Sun may also introduce a flag with the -same value to be used in the request_dump-mysql_binlog_send interface. -But this is mainly the question of merging: if a conflict concerning this -flag occur, we may simply change the BINLOG_SEND_ANNOTATE_ROWS_EVENT value -(this does not require additional changes in the code). -=-=(Alexi - Sun, 20 Dec 2009, 16:00)=-=- Low Level Design modified. --- /tmp/wklog.47.old.19667 2009-12-20 14:00:56.000000000 +0000 +++ /tmp/wklog.47.new.19667 2009-12-20 14:00:56.000000000 +0000 @@ -196,6 +196,11 @@ ... } +NOTE. mysqlbinlog, when remotely requesting BINLOG_DUMP by calling the +simple_command() function, should also use this flag if it wants (in case +of the --print-annotate-rows-events option set) to recieve Annotate_rows +events. + 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -212,8 +217,7 @@ Log_event::read_log_event(&log, packet, ...); ... if ((*packet)[EVENT_TYPE_OFFSET + 1] != ANNOTATE_ROWS_EVENT || - flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT || - thd->server_id == 0 /* slave == mysqlbinlog */ ) + flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT) { my_net_write(net, packet->ptr(), packet->length()); } -=-=(Alexi - Sun, 20 Dec 2009, 13:14)=-=- Low Level Design modified. --- /tmp/wklog.47.old.11350 2009-12-20 13:14:04.000000000 +0200 +++ /tmp/wklog.47.new.11350 2009-12-20 13:14:04.000000000 +0200 @@ -282,23 +282,18 @@ Annotate_rows_log_event* m_annotate_event; }; -When the saved Annotate_rows object may be deleted? When all corresponding -Rows events will be processed, i.e. before processing the first non-Rows -event (note that Annotate_rows object resides in the binary log *after* -the (possible) 'BEGIN' Query event which accompanies the rows events; note -also that this deletion is adjusted with the case when some or all -corresponding Rows events are filtered out by replicate filter rules): +The saved Annotate_rows object should be deleted when all corresponding +Rows events will be processed: int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev= next_event(rli); ... - if (rli->get_annotate_event() && !IS_RBR_EVENT_TYPE(ev->get_type_code())) - rli->free_annotate_event(); - apply_event_and_update_pos(ev, ...); - if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) + if (rli->get_annotate_event() && is_last_rows_event(ev)) + rli->free_annotate_event(); + else if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) rli->set_annotate_event((Annotate_rows_log_event*) ev); else if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; @@ -307,10 +302,21 @@ where - #define IS_RBR_EVENT_TYPE(type) ( (type) == TABLE_MAP_EVENT || \ - (type) == WRITE_ROWS_EVENT || \ + bool is_last_rows_event(Log_event* ev) + { + Log_event_type type= ev->get_type_code(); + if (IS_ROWS_EVENT_TYPE(type)) + { + Rows_log_event* rows= (Rows_log_event*)ev; + return rows->get_flags(Rows_log_event::STMT_END_F); + } + + return 0; + } + + #define IS_ROWS_EVENT_TYPE(type) ((type) == WRITE_ROWS_EVENT || \ (type) == UPDATE_ROWS_EVENT || \ - (type) == DELETE_ROWS_EVENT ) + (type) == DELETE_ROWS_EVENT) 8. General remarks ~~~~~~~~~~~~~~~~~~ -=-=(Alexi - Sun, 20 Dec 2009, 09:29)=-=- Low Level Design modified. --- /tmp/wklog.47.old.32726 2009-12-20 07:29:56.000000000 +0000 +++ /tmp/wklog.47.new.32726 2009-12-20 07:29:56.000000000 +0000 @@ -56,7 +56,7 @@ 0000006A | 00 | 0 - RESERVED [27] ..... ... 00000081 | 00 | 0 - RESERVED [50] - 00000082 | 00 | 0 - ANNOTATE_RBR_EVENT [51] + 00000082 | 00 | 0 - ANNOTATE_ROWS_EVENT [51] ************************ 2. Outline of Annotate_rows event behavior -=-=(Alexi - Sat, 19 Dec 2009, 16:10)=-=- Low Level Design modified. --- /tmp/wklog.47.old.16051 2009-12-19 16:10:48.000000000 +0200 +++ /tmp/wklog.47.new.16051 2009-12-19 16:10:48.000000000 +0200 @@ -253,7 +253,7 @@ thd query to that of the described by the event, i.e. to the query which caused the subsequent Rows events (see "How Master writes Annotate_rows events to the binary log" to follow what happens further when the subsequent -Rows events is applied): +Rows events are applied): int Annotate_rows_log_event::do_apply_event(...) { -=-=(Alexi - Sat, 19 Dec 2009, 16:02)=-=- High-Level Specification modified. --- /tmp/wklog.47.old.15695 2009-12-19 16:02:33.000000000 +0200 +++ /tmp/wklog.47.new.15695 2009-12-19 16:02:33.000000000 +0200 @@ -12,7 +12,7 @@ post-header and contains the query text in its data part. Example: ************************ - ANNOTATE_RBR_EVENT + ANNOTATE_ROWS_EVENT ************************ 00000220 | B6 A0 2C 4B | time_when = 1261215926 00000224 | 33 | event_type = 51 -=-=(Alexi - Sat, 19 Dec 2009, 15:58)=-=- Low Level Design modified. --- /tmp/wklog.47.old.15437 2009-12-19 13:58:12.000000000 +0000 +++ /tmp/wklog.47.new.15437 2009-12-19 13:58:12.000000000 +0000 @@ -1 +1,337 @@ +Content +~~~~~~~ + 1. Annotate_rows event number + 2. Outline of Annotate_rows event behavior + 3. How Master writes Annotate_rows events to the binary log + 4. How slave treats replicate-annotate-rows-events option + 5. How slave IO thread requests Annotate_rows events + 6. How master executes the request + 7. How slave SQL thread processes Annotate_rows events + 8. General remarks + +1. Annotate_rows event number +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +To avoid possible event numbers conflict with MySQL/Sun, we leave a gap +between the last MySQL event number and the Annotate_rows event number: + + enum Log_event_type + { ... + INCIDENT_EVENT= 26, + // New MySQL event numbers are to be added here + MYSQL_EVENTS_END, + + MARIA_EVENTS_BEGIN= 51, + // New Maria event numbers start from here + ANNOTATE_ROWS_EVENT= 51, + + ENUM_END_EVENT + }; + +together with the corresponding extension of 'post_header_len' array in the +Format description event. (This extension does not affect the compatibility +of the binary log). Here is how Format description event looks like with +this extension: + + ************************ + FORMAT_DESCRIPTION_EVENT + ************************ + 00000004 | A1 A0 2C 4B | time_when = 1261215905 + 00000008 | 0F | event_type = 15 + 00000009 | 64 00 00 00 | server_id = 100 + 0000000D | 7F 00 00 00 | event_len = 127 + 00000011 | 83 00 00 00 | log_pos = 00000083 + 00000015 | 01 00 | flags = LOG_EVENT_BINLOG_IN_USE_F + ------------------------ + 00000017 | 04 00 | binlog_ver = 4 + 00000019 | 35 2E 32 2E | server_ver = 5.2.0-MariaDB-alpha-debug-log + ..... ... + 0000004B | A1 A0 2C 4B | time_created = 1261215905 + 0000004F | 13 | common_header_len = 19 + ------------------------ + post_header_len + ------------------------ + 00000050 | 38 | 56 - START_EVENT_V3 [1] + ..... ... + 00000069 | 02 | 2 - INCIDENT_EVENT [26] + 0000006A | 00 | 0 - RESERVED [27] + ..... ... + 00000081 | 00 | 0 - RESERVED [50] + 00000082 | 00 | 0 - ANNOTATE_RBR_EVENT [51] + ************************ + +2. Outline of Annotate_rows event behavior +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Each Annotate_rows_log_event object has two private members describing the +corresponding query: + + char *m_query_txt; + uint m_query_len; + +When the object is created for writing to a binary log, this query is taken +from 'thd' (for short, below we omit the 'Annotate_rows_log_event::' prefix +as well as other implementation details): + + Annotate_rows_log_event(THD *thd) + { + m_query_txt = thd->query(); + m_query_len = thd->query_length(); + } + +When the object is read from a binary log, the query is taken from the buffer +containing the binary log representation of the event (this buffer is allocated +in Log_event object from which all Log events are derived): + + Annotate_rows_log_event(char *buf, uint event_len, + Format_description_log_event *desc) + { + m_query_len = event_len - desc->common_header_len; + m_query_txt = buf + desc->common_header_len; + } + +The events are written to the binary log by the Log_event::write() member +which calls virtual write_data_header() and write_data_body() members +("data header" and "post header" are synonym in replication terminology). +In our case, data header is empty and data body is just the query: + + bool write_data_body(IO_CACHE *file) + { + return my_b_safe_write(file, (uchar*) m_query_txt, m_query_len); + } + +Printing the event is just printing the query: + + void Annotate_rows_log_event::print(FILE *file, PRINT_EVENT_INFO *pinfo) + { + my_b_printf(&pinfo->head_cache, "\tQuery: `%s`\n", m_query_txt); + } + +3. How Master writes Annotate_rows events to the binary log +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The event is written to the binary log just before the group of Table_map +events which precede corresponding Rows events (one query may generate +several Table map events in the binary log, but the corresponding +Annotate_rows event must be written only once before the first Table map +event; hence the boolean variable 'with_annotate' below): + + int write_locked_table_maps(THD *thd) + { ... + bool with_annotate= thd->variables.binlog_annotate_rows_events; + ... + for (uint i= 0; i < ... <number of tables> ...; ++i) + { ... + thd->binlog_write_table_map(table, ..., with_annotate); + with_annotate= 0; // write Annotate_event not more than once + ... + } + ... + } + + int THD::binlog_write_table_map(TABLE *table, ..., bool with_annotate) + { ... + Table_map_log_event the_event(...); + ... + if (with_annotate) + { + Annotate_rows_log_event anno(this); + mysql_bin_log.write(&anno); + } + + mysql_bin_log.write(&the_event); + ... + } + +4. How slave treats replicate-annotate-rows-events option +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The replicate-annotate-rows-events option is treated just as the session +value of the binlog_annotate_rows_events variable for the slave IO and +SQL threads. This setting is done during initialization of these threads: + + pthread_handler_t handle_slave_io(void *arg) + { + THD *thd= new THD; + ... + init_slave_thread(thd, SLAVE_THD_IO); + ... + } + + pthread_handler_t handle_slave_sql(void *arg) + { + THD *thd= new THD; + ... + init_slave_thread(thd, SLAVE_THD_SQL); + ... + } + + int init_slave_thread(THD* thd, SLAVE_THD_TYPE thd_type) + { ... + thd->variables.binlog_annotate_rows_events= + opt_replicate_annotate_rows_events; + ... + } + +5. How slave IO thread requests Annotate_rows events +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +When requesting an event, the slave should inform the master whether +it should send Annotate_rows events or not. To that end we add a new +BINLOG_SEND_ANNOTATE_ROWS_EVENT flag used when requesting an event: + + #define BINLOG_DUMP_NON_BLOCK 1 + #define BINLOG_SEND_ANNOTATE_ROWS_EVENT 2 + + pthread_handler_t handle_slave_io(void *arg) + { ... + request_dump(mysql, ...); + ... + } + + int request_dump(MYSQL* mysql, ...) + { ... + if (opt_log_slave_updates && + mi->io_thd->variables.binlog_annotate_rows_events) + binlog_flags|= BINLOG_SEND_ANNOTATE_ROWS_EVENT; + ... + int2store(buf + 4, binlog_flags); + ... + simple_command(mysql, COM_BINLOG_DUMP, buf, ...); + ... + } + +6. How master executes the request +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + case COM_BINLOG_DUMP: + { ... + flags= uint2korr(packet + 4); + ... + mysql_binlog_send(thd, ..., flags); + ... + } + + void mysql_binlog_send(THD* thd, ..., ushort flags) + { ... + Log_event::read_log_event(&log, packet, ...); + ... + if ((*packet)[EVENT_TYPE_OFFSET + 1] != ANNOTATE_ROWS_EVENT || + flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT || + thd->server_id == 0 /* slave == mysqlbinlog */ ) + { + my_net_write(net, packet->ptr(), packet->length()); + } + ... + } + +7. How slave SQL thread processes Annotate_rows events +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The slave processes each recieved event by "applying" it, i.e. by +calling the Log_event::apply_event() function which in turn calls +the virtual do_apply_event() member specific for each type of the +event. + + int exec_relay_log_event(THD* thd, Relay_log_info* rli) + { ... + Log_event *ev = next_event(rli); + ... + apply_event_and_update_pos(ev, ...); + + if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) + delete ev; + ... + } + + int apply_event_and_update_pos(Log_event *ev, ...) + { ... + ev->apply_event(...); + ... + } + + int Log_event::apply_event(...) + { + return do_apply_event(...); + } + +What does it mean to "apply" an Annotate_rows event? It means to set current +thd query to that of the described by the event, i.e. to the query which +caused the subsequent Rows events (see "How Master writes Annotate_rows +events to the binary log" to follow what happens further when the subsequent +Rows events is applied): + + int Annotate_rows_log_event::do_apply_event(...) + { + thd->set_query(m_query_txt, m_query_len); + } + +NOTE. I am not sure, but possibly current values of thd->query and +thd->query_length should be saved before calling set_query() and to be +restored on the Annotate_rows_log_event object deletion. +Is it really needed ? + +After calling this do_apply_event() function we may not delete the +Annotate_rows_log_event object immediatedly (see exec_relay_log_event() +above) because thd->query now points to the string inside this object. +We may keep the pointer to this object in the Relay_log_info: + + class Relay_log_info + { + public: + ... + void set_annotate_event(Annotate_rows_log_event*); + Annotate_rows_log_event* get_annotate_event(); + void free_annotate_event(); + ... + private: + Annotate_rows_log_event* m_annotate_event; + }; + +When the saved Annotate_rows object may be deleted? When all corresponding +Rows events will be processed, i.e. before processing the first non-Rows +event (note that Annotate_rows object resides in the binary log *after* +the (possible) 'BEGIN' Query event which accompanies the rows events; note +also that this deletion is adjusted with the case when some or all +corresponding Rows events are filtered out by replicate filter rules): + + int exec_relay_log_event(THD* thd, Relay_log_info* rli) + { ... + Log_event *ev= next_event(rli); + ... + if (rli->get_annotate_event() && !IS_RBR_EVENT_TYPE(ev->get_type_code())) + rli->free_annotate_event(); + + apply_event_and_update_pos(ev, ...); + + if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) + rli->set_annotate_event((Annotate_rows_log_event*) ev); + else if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) + delete ev; + ... + } + +where + + #define IS_RBR_EVENT_TYPE(type) ( (type) == TABLE_MAP_EVENT || \ + (type) == WRITE_ROWS_EVENT || \ + (type) == UPDATE_ROWS_EVENT || \ + (type) == DELETE_ROWS_EVENT ) + +8. General remarks +~~~~~~~~~~~~~~~~~~ +Kristian noticed that introducing new log event type should be coordinated +somehow with MySQL/Sun: + + Kristian: The numeric code for this event must be assigned carefully. + It should be coordinated with MySQL/Sun, otherwise we can get into a + situation where MySQL uses the same numeric code for one event that + MariaDB uses for ANNOTATE_ROWS_EVENT, which would make merging the two + impossible. + Alex: I reserved about 20 numbers not to have possible conflicts + with MySQL. + Kristian: Still, I think it would be appropriate to send a polite email + to internals(a)lists.mysql.com about this and suggesting to reserve the + event number. + +Also we should notice the introduction of the BINLOG_SEND_ANNOTATE_ROWS_EVENT +flag taking into account that MySQL/Sun may also introduce a flag with the +same value to be used in the request_dump-mysql_binlog_send interface. +But this is mainly the question of merging: if a conflict concerning this +flag occur, we may simply change the BINLOG_SEND_ANNOTATE_ROWS_EVENT value +(this does not require additional changes in the code). ------------------------------------------------------------ -=-=(View All Progress Notes, 23 total)=-=- http://askmonty.org/worklog/index.pl?tid=47&nolimit=1 DESCRIPTION: Store in binlog (and show in mysqlbinlog output) texts of statements that caused RBR events This is needed for (list from Monty): - Easier to understand why updates happened - Would make it easier to find out where in application things went wrong (as you can search for exact strings) - Allow one to filter things based on comments in the statement. The cost of this can be that the binlog will be approximately 2x in size (especially insert of big blob's would be a bit painful), so this should be an optional feature. HIGH-LEVEL SPECIFICATION: Content ~~~~~~~ 1. Annotate_rows_log_event 2. Server option: --binlog-annotate-rows-events 3. Server option: --replicate-annotate-rows-events 4. mysqlbinlog option: --print-annotate-rows-events 5. mysqlbinlog output 1. Annotate_rows_log_event [ ANNOTATE_ROWS_EVENT ] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Describes the query which caused the corresponding rows events. Has empty post-header and contains the query text in its data part. Example: ************************ ANNOTATE_ROWS_EVENT ************************ 00000220 | B6 A0 2C 4B | time_when = 1261215926 00000224 | 33 | event_type = 51 00000225 | 64 00 00 00 | server_id = 100 00000229 | 36 00 00 00 | event_len = 54 0000022D | 56 02 00 00 | log_pos = 00000256 00000231 | 00 00 | flags = <none> ------------------------ 00000233 | 49 4E 53 45 | query = "INSERT INTO t1 VALUES (1), (2), (3)" 00000237 | 52 54 20 49 | 0000023B | 4E 54 4F 20 | 0000023F | 74 31 20 56 | 00000243 | 41 4C 55 45 | 00000247 | 53 20 28 31 | 0000024B | 29 2C 20 28 | 0000024F | 32 29 2C 20 | 00000253 | 28 33 29 | ************************ In binary log, Annotate_rows event follows the (possible) 'BEGIN' Query event and precedes the first of Table map events which accompany the corresponding rows events. (See example in the "mysqlbinlog output" section below.) 2. Server option: --binlog-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the master to write Annotate_rows events to the binary log. * Variable Name: binlog_annotate_rows_events * Scope: Global & Session * Access Type: Dynamic * Data Type: bool * Default Value: OFF NOTE. Session values allows to annotate only some selected statements: ... SET SESSION binlog_annotate_rows_events=ON; ... statements to be annotated ... SET SESSION binlog_annotate_rows_events=OFF; ... statements not to be annotated ... 3. Server option: --replicate-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the slave to reproduce Annotate_rows events recieved from the master in its own binary log (sensible only in pair with log-slave-updates option). * Variable Name: replicate_annotate_rows_events * Scope: Global * Access Type: Read only * Data Type: bool * Default Value: OFF NOTE. Why do we additionally need this 'replicate' option? Why not to make the slave to reproduce this events when its binlog-annotate-rows-events global value is ON? Well, because, for example, we may want to configure the slave which should reproduce Annotate_rows events but has global binlog-annotate-rows-events = OFF meaning this to be the default value for the client threads (see also "How slave treats replicate-annotate-rows-events option" in LLD part). 4. mysqlbinlog option: --print-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ With this option, mysqlbinlog prints the content of Annotate_rows events (if the binary log does contain them). Without this option (i.e. by default), mysqlbinlog skips Annotate_rows events. 5. mysqlbinlog output ~~~~~~~~~~~~~~~~~~~~~ With --print-annotate-rows-events, mysqlbinlog outputs Annotate_rows events in a form like this: ... # at 1646 #091219 12:45:26 server id 100 end_log_pos 1714 Query thread_id=1 exec_time=0 error_code=0 SET TIMESTAMP=1261215926/*!*/; BEGIN /*!*/; # at 1714 # at 1812 # at 1853 # at 1894 # at 1938 #091219 12:45:26 server id 100 end_log_pos 1812 Query: `DELETE t1, t2 FROM t1 INNER JOIN t2 INNER JOIN t3 WHERE t1.a=t2.a AND t2.a=t3.a` #091219 12:45:26 server id 100 end_log_pos 1853 Table_map: `test`.`t1` mapped to number 16 #091219 12:45:26 server id 100 end_log_pos 1894 Table_map: `test`.`t2` mapped to number 17 #091219 12:45:26 server id 100 end_log_pos 1938 Delete_rows: table id 16 #091219 12:45:26 server id 100 end_log_pos 1982 Delete_rows: table id 17 flags: STMT_END_F ... LOW-LEVEL DESIGN: Content ~~~~~~~ 1. Annotate_rows event number 2. Outline of Annotate_rows event behavior 3. How Master writes Annotate_rows events to the binary log 4. How slave treats replicate-annotate-rows-events option 5. How slave IO thread requests Annotate_rows events 6. How master executes the request 7. How slave SQL thread processes Annotate_rows events 8. General remarks 1. Annotate_rows event number ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To avoid possible event numbers conflict with MySQL/Sun, we leave a gap between the last MySQL event number and the Annotate_rows event number: enum Log_event_type { ... INCIDENT_EVENT= 26, // New MySQL event numbers are to be added here MYSQL_EVENTS_END, MARIA_EVENTS_BEGIN= 51, // New Maria event numbers start from here ANNOTATE_ROWS_EVENT= 51, ENUM_END_EVENT }; together with the corresponding extension of 'post_header_len' array in the Format description event. (This extension does not affect the compatibility of the binary log). Here is how Format description event looks like with this extension: ************************ FORMAT_DESCRIPTION_EVENT ************************ 00000004 | A1 A0 2C 4B | time_when = 1261215905 00000008 | 0F | event_type = 15 00000009 | 64 00 00 00 | server_id = 100 0000000D | 7F 00 00 00 | event_len = 127 00000011 | 83 00 00 00 | log_pos = 00000083 00000015 | 01 00 | flags = LOG_EVENT_BINLOG_IN_USE_F ------------------------ 00000017 | 04 00 | binlog_ver = 4 00000019 | 35 2E 32 2E | server_ver = 5.2.0-MariaDB-alpha-debug-log ..... ... 0000004B | A1 A0 2C 4B | time_created = 1261215905 0000004F | 13 | common_header_len = 19 ------------------------ post_header_len ------------------------ 00000050 | 38 | 56 - START_EVENT_V3 [1] ..... ... 00000069 | 02 | 2 - INCIDENT_EVENT [26] 0000006A | 00 | 0 - RESERVED [27] ..... ... 00000081 | 00 | 0 - RESERVED [50] 00000082 | 00 | 0 - ANNOTATE_ROWS_EVENT [51] ************************ 2. Outline of Annotate_rows event behavior ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Each Annotate_rows_log_event object has two private members describing the corresponding query: char *m_query_txt; uint m_query_len; When the object is created for writing to a binary log, this query is taken from 'thd' (for short, below we omit the 'Annotate_rows_log_event::' prefix as well as other implementation details): Annotate_rows_log_event(THD *thd) { m_query_txt = thd->query(); m_query_len = thd->query_length(); } When the object is read from a binary log, the query is taken from the buffer containing the binary log representation of the event (this buffer is allocated in Log_event object from which all Log events are derived): Annotate_rows_log_event(char *buf, uint event_len, Format_description_log_event *desc) { m_query_len = event_len - desc->common_header_len; m_query_txt = buf + desc->common_header_len; } The events are written to the binary log by the Log_event::write() member which calls virtual write_data_header() and write_data_body() members ("data header" and "post header" are synonym in replication terminology). In our case, data header is empty and data body is just the query: bool write_data_body(IO_CACHE *file) { return my_b_safe_write(file, (uchar*) m_query_txt, m_query_len); } Printing the event is just printing the query: void Annotate_rows_log_event::print(FILE *file, PRINT_EVENT_INFO *pinfo) { my_b_printf(&pinfo->head_cache, "\tQuery: `%s`\n", m_query_txt); } 3. How Master writes Annotate_rows events to the binary log ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The event is written to the binary log just before the group of Table_map events which precede corresponding Rows events (one query may generate several Table map events in the binary log, but the corresponding Annotate_rows event must be written only once before the first Table map event; hence the boolean variable 'with_annotate' below): int write_locked_table_maps(THD *thd) { ... bool with_annotate= thd->variables.binlog_annotate_rows_events; ... for (uint i= 0; i < ... <number of tables> ...; ++i) { ... thd->binlog_write_table_map(table, ..., with_annotate); with_annotate= 0; // write Annotate_event not more than once ... } ... } int THD::binlog_write_table_map(TABLE *table, ..., bool with_annotate) { ... Table_map_log_event the_event(...); ... if (with_annotate) { Annotate_rows_log_event anno(this); mysql_bin_log.write(&anno); } mysql_bin_log.write(&the_event); ... } 4. How slave treats replicate-annotate-rows-events option ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The replicate-annotate-rows-events option is treated just as the session value of the binlog_annotate_rows_events variable for the slave IO and SQL threads. This setting is done during initialization of these threads: pthread_handler_t handle_slave_io(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_IO); ... } pthread_handler_t handle_slave_sql(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_SQL); ... } int init_slave_thread(THD* thd, SLAVE_THD_TYPE thd_type) { ... thd->variables.binlog_annotate_rows_events= opt_replicate_annotate_rows_events; ... } 5. How slave IO thread requests Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If the replicate-annotate-rows-events option is not set on a slave, there is no need for master to send Annotate_rows events to this slave. The slave (or mysqlbinlog in remote case), before requesting binlog dump via the COM_BINLOG_DUMP command, informs the master whether it should send these events by executing the newly added COM_BINLOG_DUMP_OPTIONS_EXT server command: case COM_BINLOG_DUMP_OPTIONS_EXT: thd->binlog_dump_flags_ext= packet[0]; my_ok(thd); break; Note. We add this new command and don't use COM_BINLOG_DUMP to avoid possible conflicts with MySQL/Sun. 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ case COM_BINLOG_DUMP: { ... flags= uint2korr(packet + 4); ... mysql_binlog_send(thd, ..., flags); ... } void mysql_binlog_send(THD* thd, ..., ushort flags) { ... Log_event::read_log_event(&log, packet, ...); ... if ((*packet)[EVENT_TYPE_OFFSET + 1] != ANNOTATE_ROWS_EVENT || flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT) { my_net_write(net, packet->ptr(), packet->length()); } ... } 7. How slave SQL thread processes Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The slave processes each recieved event by "applying" it, i.e. by calling the Log_event::apply_event() function which in turn calls the virtual do_apply_event() member specific for each type of the event. int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev = next_event(rli); ... apply_event_and_update_pos(ev, ...); if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } int apply_event_and_update_pos(Log_event *ev, ...) { ... ev->apply_event(...); ... } int Log_event::apply_event(...) { return do_apply_event(...); } What does it mean to "apply" an Annotate_rows event? It means to set current thd query to that of the described by the event, i.e. to the query which caused the subsequent Rows events (see "How Master writes Annotate_rows events to the binary log" to follow what happens further when the subsequent Rows events are applied): int Annotate_rows_log_event::do_apply_event(...) { thd->set_query(m_query_txt, m_query_len); } NOTE. I am not sure, but possibly current values of thd->query and thd->query_length should be saved before calling set_query() and to be restored on the Annotate_rows_log_event object deletion. Is it really needed ? After calling this do_apply_event() function we may not delete the Annotate_rows_log_event object immediatedly (see exec_relay_log_event() above) because thd->query now points to the string inside this object. We may keep the pointer to this object in the Relay_log_info: class Relay_log_info { public: ... void set_annotate_event(Annotate_rows_log_event*); Annotate_rows_log_event* get_annotate_event(); void free_annotate_event(); ... private: Annotate_rows_log_event* m_annotate_event; }; The saved Annotate_rows object should be deleted when all corresponding Rows events will be processed: int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev= next_event(rli); ... apply_event_and_update_pos(ev, ...); if (rli->get_annotate_event() && is_last_rows_event(ev)) rli->free_annotate_event(); else if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) rli->set_annotate_event((Annotate_rows_log_event*) ev); else if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } where bool is_last_rows_event(Log_event* ev) { Log_event_type type= ev->get_type_code(); if (IS_ROWS_EVENT_TYPE(type)) { Rows_log_event* rows= (Rows_log_event*)ev; return rows->get_flags(Rows_log_event::STMT_END_F); } return 0; } #define IS_ROWS_EVENT_TYPE(type) ((type) == WRITE_ROWS_EVENT || \ (type) == UPDATE_ROWS_EVENT || \ (type) == DELETE_ROWS_EVENT) 8. General remarks ~~~~~~~~~~~~~~~~~~ Kristian noticed that introducing new log event type should be coordinated somehow with MySQL/Sun: Kristian: The numeric code for this event must be assigned carefully. It should be coordinated with MySQL/Sun, otherwise we can get into a situation where MySQL uses the same numeric code for one event that MariaDB uses for ANNOTATE_ROWS_EVENT, which would make merging the two impossible. Alex: I reserved about 20 numbers not to have possible conflicts with MySQL. Kristian: Still, I think it would be appropriate to send a polite email to internals(a)lists.mysql.com about this and suggesting to reserve the event number. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Serg): Store in binlog text of statements that caused RBR events (47)
by worklog-noreply＠askmonty.org 05 Feb '10

05 Feb '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Store in binlog text of statements that caused RBR events CREATION DATE..: Sat, 15 Aug 2009, 23:48 SUPERVISOR.....: Monty IMPLEMENTOR....: COPIES TO......: Knielsen, Serg CATEGORY.......: Server-Sprint TASK ID........: 47 (http://askmonty.org/worklog/?tid=47) VERSION........: Server-9.x STATUS.........: In-Progress PRIORITY.......: 60 WORKED HOURS...: 20 ESTIMATE.......: 35 (hours remain) ORIG. ESTIMATE.: 35 PROGRESS NOTES: -=-=(Serg - Fri, 05 Feb 2010, 14:04)=-=- Observers changed: Knielsen,Serg -=-=(Guest - Fri, 05 Feb 2010, 13:40)=-=- Category updated. --- /tmp/wklog.47.old.9197 2010-02-05 13:40:36.000000000 +0200 +++ /tmp/wklog.47.new.9197 2010-02-05 13:40:36.000000000 +0200 @@ -1 +1 @@ -Server-RawIdeaBin +Server-Sprint -=-=(Guest - Fri, 05 Feb 2010, 13:40)=-=- Status updated. --- /tmp/wklog.47.old.9197 2010-02-05 13:40:36.000000000 +0200 +++ /tmp/wklog.47.new.9197 2010-02-05 13:40:36.000000000 +0200 @@ -1 +1 @@ -Un-Assigned +In-Progress -=-=(Alexi - Thu, 04 Feb 2010, 09:54)=-=- Low Level Design modified. --- /tmp/wklog.47.old.16174 2010-02-04 09:54:13.000000000 +0200 +++ /tmp/wklog.47.new.16174 2010-02-04 09:54:13.000000000 +0200 @@ -171,35 +171,20 @@ 5. How slave IO thread requests Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -When requesting an event, the slave should inform the master whether -it should send Annotate_rows events or not. To that end we add a new -BINLOG_SEND_ANNOTATE_ROWS_EVENT flag used when requesting an event: +If the replicate-annotate-rows-events option is not set on a slave, there +is no need for master to send Annotate_rows events to this slave. The slave +(or mysqlbinlog in remote case), before requesting binlog dump via the +COM_BINLOG_DUMP command, informs the master whether it should send these +events by executing the newly added COM_BINLOG_DUMP_OPTIONS_EXT server +command: + + case COM_BINLOG_DUMP_OPTIONS_EXT: + thd->binlog_dump_flags_ext= packet[0]; + my_ok(thd); + break; - #define BINLOG_DUMP_NON_BLOCK 1 - #define BINLOG_SEND_ANNOTATE_ROWS_EVENT 2 - - pthread_handler_t handle_slave_io(void *arg) - { ... - request_dump(mysql, ...); - ... - } - - int request_dump(MYSQL* mysql, ...) - { ... - if (opt_log_slave_updates && - mi->io_thd->variables.binlog_annotate_rows_events) - binlog_flags|= BINLOG_SEND_ANNOTATE_ROWS_EVENT; - ... - int2store(buf + 4, binlog_flags); - ... - simple_command(mysql, COM_BINLOG_DUMP, buf, ...); - ... - } - -NOTE. mysqlbinlog, when remotely requesting BINLOG_DUMP by calling the -simple_command() function, should also use this flag if it wants (in case -of the --print-annotate-rows-events option set) to recieve Annotate_rows -events. +Note. We add this new command and don't use COM_BINLOG_DUMP to avoid possible +conflicts with MySQL/Sun. 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -338,10 +323,4 @@ to internals(a)lists.mysql.com about this and suggesting to reserve the event number. -Also we should notice the introduction of the BINLOG_SEND_ANNOTATE_ROWS_EVENT -flag taking into account that MySQL/Sun may also introduce a flag with the -same value to be used in the request_dump-mysql_binlog_send interface. -But this is mainly the question of merging: if a conflict concerning this -flag occur, we may simply change the BINLOG_SEND_ANNOTATE_ROWS_EVENT value -(this does not require additional changes in the code). -=-=(Alexi - Sun, 20 Dec 2009, 16:00)=-=- Low Level Design modified. --- /tmp/wklog.47.old.19667 2009-12-20 14:00:56.000000000 +0000 +++ /tmp/wklog.47.new.19667 2009-12-20 14:00:56.000000000 +0000 @@ -196,6 +196,11 @@ ... } +NOTE. mysqlbinlog, when remotely requesting BINLOG_DUMP by calling the +simple_command() function, should also use this flag if it wants (in case +of the --print-annotate-rows-events option set) to recieve Annotate_rows +events. + 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -212,8 +217,7 @@ Log_event::read_log_event(&log, packet, ...); ... if ((*packet)[EVENT_TYPE_OFFSET + 1] != ANNOTATE_ROWS_EVENT || - flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT || - thd->server_id == 0 /* slave == mysqlbinlog */ ) + flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT) { my_net_write(net, packet->ptr(), packet->length()); } -=-=(Alexi - Sun, 20 Dec 2009, 13:14)=-=- Low Level Design modified. --- /tmp/wklog.47.old.11350 2009-12-20 13:14:04.000000000 +0200 +++ /tmp/wklog.47.new.11350 2009-12-20 13:14:04.000000000 +0200 @@ -282,23 +282,18 @@ Annotate_rows_log_event* m_annotate_event; }; -When the saved Annotate_rows object may be deleted? When all corresponding -Rows events will be processed, i.e. before processing the first non-Rows -event (note that Annotate_rows object resides in the binary log *after* -the (possible) 'BEGIN' Query event which accompanies the rows events; note -also that this deletion is adjusted with the case when some or all -corresponding Rows events are filtered out by replicate filter rules): +The saved Annotate_rows object should be deleted when all corresponding +Rows events will be processed: int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev= next_event(rli); ... - if (rli->get_annotate_event() && !IS_RBR_EVENT_TYPE(ev->get_type_code())) - rli->free_annotate_event(); - apply_event_and_update_pos(ev, ...); - if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) + if (rli->get_annotate_event() && is_last_rows_event(ev)) + rli->free_annotate_event(); + else if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) rli->set_annotate_event((Annotate_rows_log_event*) ev); else if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; @@ -307,10 +302,21 @@ where - #define IS_RBR_EVENT_TYPE(type) ( (type) == TABLE_MAP_EVENT || \ - (type) == WRITE_ROWS_EVENT || \ + bool is_last_rows_event(Log_event* ev) + { + Log_event_type type= ev->get_type_code(); + if (IS_ROWS_EVENT_TYPE(type)) + { + Rows_log_event* rows= (Rows_log_event*)ev; + return rows->get_flags(Rows_log_event::STMT_END_F); + } + + return 0; + } + + #define IS_ROWS_EVENT_TYPE(type) ((type) == WRITE_ROWS_EVENT || \ (type) == UPDATE_ROWS_EVENT || \ - (type) == DELETE_ROWS_EVENT ) + (type) == DELETE_ROWS_EVENT) 8. General remarks ~~~~~~~~~~~~~~~~~~ -=-=(Alexi - Sun, 20 Dec 2009, 09:29)=-=- Low Level Design modified. --- /tmp/wklog.47.old.32726 2009-12-20 07:29:56.000000000 +0000 +++ /tmp/wklog.47.new.32726 2009-12-20 07:29:56.000000000 +0000 @@ -56,7 +56,7 @@ 0000006A | 00 | 0 - RESERVED [27] ..... ... 00000081 | 00 | 0 - RESERVED [50] - 00000082 | 00 | 0 - ANNOTATE_RBR_EVENT [51] + 00000082 | 00 | 0 - ANNOTATE_ROWS_EVENT [51] ************************ 2. Outline of Annotate_rows event behavior -=-=(Alexi - Sat, 19 Dec 2009, 16:10)=-=- Low Level Design modified. --- /tmp/wklog.47.old.16051 2009-12-19 16:10:48.000000000 +0200 +++ /tmp/wklog.47.new.16051 2009-12-19 16:10:48.000000000 +0200 @@ -253,7 +253,7 @@ thd query to that of the described by the event, i.e. to the query which caused the subsequent Rows events (see "How Master writes Annotate_rows events to the binary log" to follow what happens further when the subsequent -Rows events is applied): +Rows events are applied): int Annotate_rows_log_event::do_apply_event(...) { -=-=(Alexi - Sat, 19 Dec 2009, 16:02)=-=- High-Level Specification modified. --- /tmp/wklog.47.old.15695 2009-12-19 16:02:33.000000000 +0200 +++ /tmp/wklog.47.new.15695 2009-12-19 16:02:33.000000000 +0200 @@ -12,7 +12,7 @@ post-header and contains the query text in its data part. Example: ************************ - ANNOTATE_RBR_EVENT + ANNOTATE_ROWS_EVENT ************************ 00000220 | B6 A0 2C 4B | time_when = 1261215926 00000224 | 33 | event_type = 51 -=-=(Alexi - Sat, 19 Dec 2009, 15:58)=-=- Low Level Design modified. --- /tmp/wklog.47.old.15437 2009-12-19 13:58:12.000000000 +0000 +++ /tmp/wklog.47.new.15437 2009-12-19 13:58:12.000000000 +0000 @@ -1 +1,337 @@ +Content +~~~~~~~ + 1. Annotate_rows event number + 2. Outline of Annotate_rows event behavior + 3. How Master writes Annotate_rows events to the binary log + 4. How slave treats replicate-annotate-rows-events option + 5. How slave IO thread requests Annotate_rows events + 6. How master executes the request + 7. How slave SQL thread processes Annotate_rows events + 8. General remarks + +1. Annotate_rows event number +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +To avoid possible event numbers conflict with MySQL/Sun, we leave a gap +between the last MySQL event number and the Annotate_rows event number: + + enum Log_event_type + { ... + INCIDENT_EVENT= 26, + // New MySQL event numbers are to be added here + MYSQL_EVENTS_END, + + MARIA_EVENTS_BEGIN= 51, + // New Maria event numbers start from here + ANNOTATE_ROWS_EVENT= 51, + + ENUM_END_EVENT + }; + +together with the corresponding extension of 'post_header_len' array in the +Format description event. (This extension does not affect the compatibility +of the binary log). Here is how Format description event looks like with +this extension: + + ************************ + FORMAT_DESCRIPTION_EVENT + ************************ + 00000004 | A1 A0 2C 4B | time_when = 1261215905 + 00000008 | 0F | event_type = 15 + 00000009 | 64 00 00 00 | server_id = 100 + 0000000D | 7F 00 00 00 | event_len = 127 + 00000011 | 83 00 00 00 | log_pos = 00000083 + 00000015 | 01 00 | flags = LOG_EVENT_BINLOG_IN_USE_F + ------------------------ + 00000017 | 04 00 | binlog_ver = 4 + 00000019 | 35 2E 32 2E | server_ver = 5.2.0-MariaDB-alpha-debug-log + ..... ... + 0000004B | A1 A0 2C 4B | time_created = 1261215905 + 0000004F | 13 | common_header_len = 19 + ------------------------ + post_header_len + ------------------------ + 00000050 | 38 | 56 - START_EVENT_V3 [1] + ..... ... + 00000069 | 02 | 2 - INCIDENT_EVENT [26] + 0000006A | 00 | 0 - RESERVED [27] + ..... ... + 00000081 | 00 | 0 - RESERVED [50] + 00000082 | 00 | 0 - ANNOTATE_RBR_EVENT [51] + ************************ + +2. Outline of Annotate_rows event behavior +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Each Annotate_rows_log_event object has two private members describing the +corresponding query: + + char *m_query_txt; + uint m_query_len; + +When the object is created for writing to a binary log, this query is taken +from 'thd' (for short, below we omit the 'Annotate_rows_log_event::' prefix +as well as other implementation details): + + Annotate_rows_log_event(THD *thd) + { + m_query_txt = thd->query(); + m_query_len = thd->query_length(); + } + +When the object is read from a binary log, the query is taken from the buffer +containing the binary log representation of the event (this buffer is allocated +in Log_event object from which all Log events are derived): + + Annotate_rows_log_event(char *buf, uint event_len, + Format_description_log_event *desc) + { + m_query_len = event_len - desc->common_header_len; + m_query_txt = buf + desc->common_header_len; + } + +The events are written to the binary log by the Log_event::write() member +which calls virtual write_data_header() and write_data_body() members +("data header" and "post header" are synonym in replication terminology). +In our case, data header is empty and data body is just the query: + + bool write_data_body(IO_CACHE *file) + { + return my_b_safe_write(file, (uchar*) m_query_txt, m_query_len); + } + +Printing the event is just printing the query: + + void Annotate_rows_log_event::print(FILE *file, PRINT_EVENT_INFO *pinfo) + { + my_b_printf(&pinfo->head_cache, "\tQuery: `%s`\n", m_query_txt); + } + +3. How Master writes Annotate_rows events to the binary log +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The event is written to the binary log just before the group of Table_map +events which precede corresponding Rows events (one query may generate +several Table map events in the binary log, but the corresponding +Annotate_rows event must be written only once before the first Table map +event; hence the boolean variable 'with_annotate' below): + + int write_locked_table_maps(THD *thd) + { ... + bool with_annotate= thd->variables.binlog_annotate_rows_events; + ... + for (uint i= 0; i < ... <number of tables> ...; ++i) + { ... + thd->binlog_write_table_map(table, ..., with_annotate); + with_annotate= 0; // write Annotate_event not more than once + ... + } + ... + } + + int THD::binlog_write_table_map(TABLE *table, ..., bool with_annotate) + { ... + Table_map_log_event the_event(...); + ... + if (with_annotate) + { + Annotate_rows_log_event anno(this); + mysql_bin_log.write(&anno); + } + + mysql_bin_log.write(&the_event); + ... + } + +4. How slave treats replicate-annotate-rows-events option +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The replicate-annotate-rows-events option is treated just as the session +value of the binlog_annotate_rows_events variable for the slave IO and +SQL threads. This setting is done during initialization of these threads: + + pthread_handler_t handle_slave_io(void *arg) + { + THD *thd= new THD; + ... + init_slave_thread(thd, SLAVE_THD_IO); + ... + } + + pthread_handler_t handle_slave_sql(void *arg) + { + THD *thd= new THD; + ... + init_slave_thread(thd, SLAVE_THD_SQL); + ... + } + + int init_slave_thread(THD* thd, SLAVE_THD_TYPE thd_type) + { ... + thd->variables.binlog_annotate_rows_events= + opt_replicate_annotate_rows_events; + ... + } + +5. How slave IO thread requests Annotate_rows events +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +When requesting an event, the slave should inform the master whether +it should send Annotate_rows events or not. To that end we add a new +BINLOG_SEND_ANNOTATE_ROWS_EVENT flag used when requesting an event: + + #define BINLOG_DUMP_NON_BLOCK 1 + #define BINLOG_SEND_ANNOTATE_ROWS_EVENT 2 + + pthread_handler_t handle_slave_io(void *arg) + { ... + request_dump(mysql, ...); + ... + } + + int request_dump(MYSQL* mysql, ...) + { ... + if (opt_log_slave_updates && + mi->io_thd->variables.binlog_annotate_rows_events) + binlog_flags|= BINLOG_SEND_ANNOTATE_ROWS_EVENT; + ... + int2store(buf + 4, binlog_flags); + ... + simple_command(mysql, COM_BINLOG_DUMP, buf, ...); + ... + } + +6. How master executes the request +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + case COM_BINLOG_DUMP: + { ... + flags= uint2korr(packet + 4); + ... + mysql_binlog_send(thd, ..., flags); + ... + } + + void mysql_binlog_send(THD* thd, ..., ushort flags) + { ... + Log_event::read_log_event(&log, packet, ...); + ... + if ((*packet)[EVENT_TYPE_OFFSET + 1] != ANNOTATE_ROWS_EVENT || + flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT || + thd->server_id == 0 /* slave == mysqlbinlog */ ) + { + my_net_write(net, packet->ptr(), packet->length()); + } + ... + } + +7. How slave SQL thread processes Annotate_rows events +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The slave processes each recieved event by "applying" it, i.e. by +calling the Log_event::apply_event() function which in turn calls +the virtual do_apply_event() member specific for each type of the +event. + + int exec_relay_log_event(THD* thd, Relay_log_info* rli) + { ... + Log_event *ev = next_event(rli); + ... + apply_event_and_update_pos(ev, ...); + + if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) + delete ev; + ... + } + + int apply_event_and_update_pos(Log_event *ev, ...) + { ... + ev->apply_event(...); + ... + } + + int Log_event::apply_event(...) + { + return do_apply_event(...); + } + +What does it mean to "apply" an Annotate_rows event? It means to set current +thd query to that of the described by the event, i.e. to the query which +caused the subsequent Rows events (see "How Master writes Annotate_rows +events to the binary log" to follow what happens further when the subsequent +Rows events is applied): + + int Annotate_rows_log_event::do_apply_event(...) + { + thd->set_query(m_query_txt, m_query_len); + } + +NOTE. I am not sure, but possibly current values of thd->query and +thd->query_length should be saved before calling set_query() and to be +restored on the Annotate_rows_log_event object deletion. +Is it really needed ? + +After calling this do_apply_event() function we may not delete the +Annotate_rows_log_event object immediatedly (see exec_relay_log_event() +above) because thd->query now points to the string inside this object. +We may keep the pointer to this object in the Relay_log_info: + + class Relay_log_info + { + public: + ... + void set_annotate_event(Annotate_rows_log_event*); + Annotate_rows_log_event* get_annotate_event(); + void free_annotate_event(); + ... + private: + Annotate_rows_log_event* m_annotate_event; + }; + +When the saved Annotate_rows object may be deleted? When all corresponding +Rows events will be processed, i.e. before processing the first non-Rows +event (note that Annotate_rows object resides in the binary log *after* +the (possible) 'BEGIN' Query event which accompanies the rows events; note +also that this deletion is adjusted with the case when some or all +corresponding Rows events are filtered out by replicate filter rules): + + int exec_relay_log_event(THD* thd, Relay_log_info* rli) + { ... + Log_event *ev= next_event(rli); + ... + if (rli->get_annotate_event() && !IS_RBR_EVENT_TYPE(ev->get_type_code())) + rli->free_annotate_event(); + + apply_event_and_update_pos(ev, ...); + + if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) + rli->set_annotate_event((Annotate_rows_log_event*) ev); + else if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) + delete ev; + ... + } + +where + + #define IS_RBR_EVENT_TYPE(type) ( (type) == TABLE_MAP_EVENT || \ + (type) == WRITE_ROWS_EVENT || \ + (type) == UPDATE_ROWS_EVENT || \ + (type) == DELETE_ROWS_EVENT ) + +8. General remarks +~~~~~~~~~~~~~~~~~~ +Kristian noticed that introducing new log event type should be coordinated +somehow with MySQL/Sun: + + Kristian: The numeric code for this event must be assigned carefully. + It should be coordinated with MySQL/Sun, otherwise we can get into a + situation where MySQL uses the same numeric code for one event that + MariaDB uses for ANNOTATE_ROWS_EVENT, which would make merging the two + impossible. + Alex: I reserved about 20 numbers not to have possible conflicts + with MySQL. + Kristian: Still, I think it would be appropriate to send a polite email + to internals(a)lists.mysql.com about this and suggesting to reserve the + event number. + +Also we should notice the introduction of the BINLOG_SEND_ANNOTATE_ROWS_EVENT +flag taking into account that MySQL/Sun may also introduce a flag with the +same value to be used in the request_dump-mysql_binlog_send interface. +But this is mainly the question of merging: if a conflict concerning this +flag occur, we may simply change the BINLOG_SEND_ANNOTATE_ROWS_EVENT value +(this does not require additional changes in the code). ------------------------------------------------------------ -=-=(View All Progress Notes, 23 total)=-=- http://askmonty.org/worklog/index.pl?tid=47&nolimit=1 DESCRIPTION: Store in binlog (and show in mysqlbinlog output) texts of statements that caused RBR events This is needed for (list from Monty): - Easier to understand why updates happened - Would make it easier to find out where in application things went wrong (as you can search for exact strings) - Allow one to filter things based on comments in the statement. The cost of this can be that the binlog will be approximately 2x in size (especially insert of big blob's would be a bit painful), so this should be an optional feature. HIGH-LEVEL SPECIFICATION: Content ~~~~~~~ 1. Annotate_rows_log_event 2. Server option: --binlog-annotate-rows-events 3. Server option: --replicate-annotate-rows-events 4. mysqlbinlog option: --print-annotate-rows-events 5. mysqlbinlog output 1. Annotate_rows_log_event [ ANNOTATE_ROWS_EVENT ] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Describes the query which caused the corresponding rows events. Has empty post-header and contains the query text in its data part. Example: ************************ ANNOTATE_ROWS_EVENT ************************ 00000220 | B6 A0 2C 4B | time_when = 1261215926 00000224 | 33 | event_type = 51 00000225 | 64 00 00 00 | server_id = 100 00000229 | 36 00 00 00 | event_len = 54 0000022D | 56 02 00 00 | log_pos = 00000256 00000231 | 00 00 | flags = <none> ------------------------ 00000233 | 49 4E 53 45 | query = "INSERT INTO t1 VALUES (1), (2), (3)" 00000237 | 52 54 20 49 | 0000023B | 4E 54 4F 20 | 0000023F | 74 31 20 56 | 00000243 | 41 4C 55 45 | 00000247 | 53 20 28 31 | 0000024B | 29 2C 20 28 | 0000024F | 32 29 2C 20 | 00000253 | 28 33 29 | ************************ In binary log, Annotate_rows event follows the (possible) 'BEGIN' Query event and precedes the first of Table map events which accompany the corresponding rows events. (See example in the "mysqlbinlog output" section below.) 2. Server option: --binlog-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the master to write Annotate_rows events to the binary log. * Variable Name: binlog_annotate_rows_events * Scope: Global & Session * Access Type: Dynamic * Data Type: bool * Default Value: OFF NOTE. Session values allows to annotate only some selected statements: ... SET SESSION binlog_annotate_rows_events=ON; ... statements to be annotated ... SET SESSION binlog_annotate_rows_events=OFF; ... statements not to be annotated ... 3. Server option: --replicate-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the slave to reproduce Annotate_rows events recieved from the master in its own binary log (sensible only in pair with log-slave-updates option). * Variable Name: replicate_annotate_rows_events * Scope: Global * Access Type: Read only * Data Type: bool * Default Value: OFF NOTE. Why do we additionally need this 'replicate' option? Why not to make the slave to reproduce this events when its binlog-annotate-rows-events global value is ON? Well, because, for example, we may want to configure the slave which should reproduce Annotate_rows events but has global binlog-annotate-rows-events = OFF meaning this to be the default value for the client threads (see also "How slave treats replicate-annotate-rows-events option" in LLD part). 4. mysqlbinlog option: --print-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ With this option, mysqlbinlog prints the content of Annotate_rows events (if the binary log does contain them). Without this option (i.e. by default), mysqlbinlog skips Annotate_rows events. 5. mysqlbinlog output ~~~~~~~~~~~~~~~~~~~~~ With --print-annotate-rows-events, mysqlbinlog outputs Annotate_rows events in a form like this: ... # at 1646 #091219 12:45:26 server id 100 end_log_pos 1714 Query thread_id=1 exec_time=0 error_code=0 SET TIMESTAMP=1261215926/*!*/; BEGIN /*!*/; # at 1714 # at 1812 # at 1853 # at 1894 # at 1938 #091219 12:45:26 server id 100 end_log_pos 1812 Query: `DELETE t1, t2 FROM t1 INNER JOIN t2 INNER JOIN t3 WHERE t1.a=t2.a AND t2.a=t3.a` #091219 12:45:26 server id 100 end_log_pos 1853 Table_map: `test`.`t1` mapped to number 16 #091219 12:45:26 server id 100 end_log_pos 1894 Table_map: `test`.`t2` mapped to number 17 #091219 12:45:26 server id 100 end_log_pos 1938 Delete_rows: table id 16 #091219 12:45:26 server id 100 end_log_pos 1982 Delete_rows: table id 17 flags: STMT_END_F ... LOW-LEVEL DESIGN: Content ~~~~~~~ 1. Annotate_rows event number 2. Outline of Annotate_rows event behavior 3. How Master writes Annotate_rows events to the binary log 4. How slave treats replicate-annotate-rows-events option 5. How slave IO thread requests Annotate_rows events 6. How master executes the request 7. How slave SQL thread processes Annotate_rows events 8. General remarks 1. Annotate_rows event number ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To avoid possible event numbers conflict with MySQL/Sun, we leave a gap between the last MySQL event number and the Annotate_rows event number: enum Log_event_type { ... INCIDENT_EVENT= 26, // New MySQL event numbers are to be added here MYSQL_EVENTS_END, MARIA_EVENTS_BEGIN= 51, // New Maria event numbers start from here ANNOTATE_ROWS_EVENT= 51, ENUM_END_EVENT }; together with the corresponding extension of 'post_header_len' array in the Format description event. (This extension does not affect the compatibility of the binary log). Here is how Format description event looks like with this extension: ************************ FORMAT_DESCRIPTION_EVENT ************************ 00000004 | A1 A0 2C 4B | time_when = 1261215905 00000008 | 0F | event_type = 15 00000009 | 64 00 00 00 | server_id = 100 0000000D | 7F 00 00 00 | event_len = 127 00000011 | 83 00 00 00 | log_pos = 00000083 00000015 | 01 00 | flags = LOG_EVENT_BINLOG_IN_USE_F ------------------------ 00000017 | 04 00 | binlog_ver = 4 00000019 | 35 2E 32 2E | server_ver = 5.2.0-MariaDB-alpha-debug-log ..... ... 0000004B | A1 A0 2C 4B | time_created = 1261215905 0000004F | 13 | common_header_len = 19 ------------------------ post_header_len ------------------------ 00000050 | 38 | 56 - START_EVENT_V3 [1] ..... ... 00000069 | 02 | 2 - INCIDENT_EVENT [26] 0000006A | 00 | 0 - RESERVED [27] ..... ... 00000081 | 00 | 0 - RESERVED [50] 00000082 | 00 | 0 - ANNOTATE_ROWS_EVENT [51] ************************ 2. Outline of Annotate_rows event behavior ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Each Annotate_rows_log_event object has two private members describing the corresponding query: char *m_query_txt; uint m_query_len; When the object is created for writing to a binary log, this query is taken from 'thd' (for short, below we omit the 'Annotate_rows_log_event::' prefix as well as other implementation details): Annotate_rows_log_event(THD *thd) { m_query_txt = thd->query(); m_query_len = thd->query_length(); } When the object is read from a binary log, the query is taken from the buffer containing the binary log representation of the event (this buffer is allocated in Log_event object from which all Log events are derived): Annotate_rows_log_event(char *buf, uint event_len, Format_description_log_event *desc) { m_query_len = event_len - desc->common_header_len; m_query_txt = buf + desc->common_header_len; } The events are written to the binary log by the Log_event::write() member which calls virtual write_data_header() and write_data_body() members ("data header" and "post header" are synonym in replication terminology). In our case, data header is empty and data body is just the query: bool write_data_body(IO_CACHE *file) { return my_b_safe_write(file, (uchar*) m_query_txt, m_query_len); } Printing the event is just printing the query: void Annotate_rows_log_event::print(FILE *file, PRINT_EVENT_INFO *pinfo) { my_b_printf(&pinfo->head_cache, "\tQuery: `%s`\n", m_query_txt); } 3. How Master writes Annotate_rows events to the binary log ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The event is written to the binary log just before the group of Table_map events which precede corresponding Rows events (one query may generate several Table map events in the binary log, but the corresponding Annotate_rows event must be written only once before the first Table map event; hence the boolean variable 'with_annotate' below): int write_locked_table_maps(THD *thd) { ... bool with_annotate= thd->variables.binlog_annotate_rows_events; ... for (uint i= 0; i < ... <number of tables> ...; ++i) { ... thd->binlog_write_table_map(table, ..., with_annotate); with_annotate= 0; // write Annotate_event not more than once ... } ... } int THD::binlog_write_table_map(TABLE *table, ..., bool with_annotate) { ... Table_map_log_event the_event(...); ... if (with_annotate) { Annotate_rows_log_event anno(this); mysql_bin_log.write(&anno); } mysql_bin_log.write(&the_event); ... } 4. How slave treats replicate-annotate-rows-events option ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The replicate-annotate-rows-events option is treated just as the session value of the binlog_annotate_rows_events variable for the slave IO and SQL threads. This setting is done during initialization of these threads: pthread_handler_t handle_slave_io(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_IO); ... } pthread_handler_t handle_slave_sql(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_SQL); ... } int init_slave_thread(THD* thd, SLAVE_THD_TYPE thd_type) { ... thd->variables.binlog_annotate_rows_events= opt_replicate_annotate_rows_events; ... } 5. How slave IO thread requests Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If the replicate-annotate-rows-events option is not set on a slave, there is no need for master to send Annotate_rows events to this slave. The slave (or mysqlbinlog in remote case), before requesting binlog dump via the COM_BINLOG_DUMP command, informs the master whether it should send these events by executing the newly added COM_BINLOG_DUMP_OPTIONS_EXT server command: case COM_BINLOG_DUMP_OPTIONS_EXT: thd->binlog_dump_flags_ext= packet[0]; my_ok(thd); break; Note. We add this new command and don't use COM_BINLOG_DUMP to avoid possible conflicts with MySQL/Sun. 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ case COM_BINLOG_DUMP: { ... flags= uint2korr(packet + 4); ... mysql_binlog_send(thd, ..., flags); ... } void mysql_binlog_send(THD* thd, ..., ushort flags) { ... Log_event::read_log_event(&log, packet, ...); ... if ((*packet)[EVENT_TYPE_OFFSET + 1] != ANNOTATE_ROWS_EVENT || flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT) { my_net_write(net, packet->ptr(), packet->length()); } ... } 7. How slave SQL thread processes Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The slave processes each recieved event by "applying" it, i.e. by calling the Log_event::apply_event() function which in turn calls the virtual do_apply_event() member specific for each type of the event. int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev = next_event(rli); ... apply_event_and_update_pos(ev, ...); if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } int apply_event_and_update_pos(Log_event *ev, ...) { ... ev->apply_event(...); ... } int Log_event::apply_event(...) { return do_apply_event(...); } What does it mean to "apply" an Annotate_rows event? It means to set current thd query to that of the described by the event, i.e. to the query which caused the subsequent Rows events (see "How Master writes Annotate_rows events to the binary log" to follow what happens further when the subsequent Rows events are applied): int Annotate_rows_log_event::do_apply_event(...) { thd->set_query(m_query_txt, m_query_len); } NOTE. I am not sure, but possibly current values of thd->query and thd->query_length should be saved before calling set_query() and to be restored on the Annotate_rows_log_event object deletion. Is it really needed ? After calling this do_apply_event() function we may not delete the Annotate_rows_log_event object immediatedly (see exec_relay_log_event() above) because thd->query now points to the string inside this object. We may keep the pointer to this object in the Relay_log_info: class Relay_log_info { public: ... void set_annotate_event(Annotate_rows_log_event*); Annotate_rows_log_event* get_annotate_event(); void free_annotate_event(); ... private: Annotate_rows_log_event* m_annotate_event; }; The saved Annotate_rows object should be deleted when all corresponding Rows events will be processed: int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev= next_event(rli); ... apply_event_and_update_pos(ev, ...); if (rli->get_annotate_event() && is_last_rows_event(ev)) rli->free_annotate_event(); else if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) rli->set_annotate_event((Annotate_rows_log_event*) ev); else if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } where bool is_last_rows_event(Log_event* ev) { Log_event_type type= ev->get_type_code(); if (IS_ROWS_EVENT_TYPE(type)) { Rows_log_event* rows= (Rows_log_event*)ev; return rows->get_flags(Rows_log_event::STMT_END_F); } return 0; } #define IS_ROWS_EVENT_TYPE(type) ((type) == WRITE_ROWS_EVENT || \ (type) == UPDATE_ROWS_EVENT || \ (type) == DELETE_ROWS_EVENT) 8. General remarks ~~~~~~~~~~~~~~~~~~ Kristian noticed that introducing new log event type should be coordinated somehow with MySQL/Sun: Kristian: The numeric code for this event must be assigned carefully. It should be coordinated with MySQL/Sun, otherwise we can get into a situation where MySQL uses the same numeric code for one event that MariaDB uses for ANNOTATE_ROWS_EVENT, which would make merging the two impossible. Alex: I reserved about 20 numbers not to have possible conflicts with MySQL. Kristian: Still, I think it would be appropriate to send a polite email to internals(a)lists.mysql.com about this and suggesting to reserve the event number. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Serg): Store in binlog text of statements that caused RBR events (47)
by worklog-noreply＠askmonty.org 05 Feb '10

05 Feb '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Store in binlog text of statements that caused RBR events CREATION DATE..: Sat, 15 Aug 2009, 23:48 SUPERVISOR.....: Monty IMPLEMENTOR....: COPIES TO......: Knielsen, Serg CATEGORY.......: Server-Sprint TASK ID........: 47 (http://askmonty.org/worklog/?tid=47) VERSION........: Server-9.x STATUS.........: In-Progress PRIORITY.......: 60 WORKED HOURS...: 20 ESTIMATE.......: 35 (hours remain) ORIG. ESTIMATE.: 35 PROGRESS NOTES: -=-=(Serg - Fri, 05 Feb 2010, 14:04)=-=- Observers changed: Knielsen,Serg -=-=(Guest - Fri, 05 Feb 2010, 13:40)=-=- Category updated. --- /tmp/wklog.47.old.9197 2010-02-05 13:40:36.000000000 +0200 +++ /tmp/wklog.47.new.9197 2010-02-05 13:40:36.000000000 +0200 @@ -1 +1 @@ -Server-RawIdeaBin +Server-Sprint -=-=(Guest - Fri, 05 Feb 2010, 13:40)=-=- Status updated. --- /tmp/wklog.47.old.9197 2010-02-05 13:40:36.000000000 +0200 +++ /tmp/wklog.47.new.9197 2010-02-05 13:40:36.000000000 +0200 @@ -1 +1 @@ -Un-Assigned +In-Progress -=-=(Alexi - Thu, 04 Feb 2010, 09:54)=-=- Low Level Design modified. --- /tmp/wklog.47.old.16174 2010-02-04 09:54:13.000000000 +0200 +++ /tmp/wklog.47.new.16174 2010-02-04 09:54:13.000000000 +0200 @@ -171,35 +171,20 @@ 5. How slave IO thread requests Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -When requesting an event, the slave should inform the master whether -it should send Annotate_rows events or not. To that end we add a new -BINLOG_SEND_ANNOTATE_ROWS_EVENT flag used when requesting an event: +If the replicate-annotate-rows-events option is not set on a slave, there +is no need for master to send Annotate_rows events to this slave. The slave +(or mysqlbinlog in remote case), before requesting binlog dump via the +COM_BINLOG_DUMP command, informs the master whether it should send these +events by executing the newly added COM_BINLOG_DUMP_OPTIONS_EXT server +command: + + case COM_BINLOG_DUMP_OPTIONS_EXT: + thd->binlog_dump_flags_ext= packet[0]; + my_ok(thd); + break; - #define BINLOG_DUMP_NON_BLOCK 1 - #define BINLOG_SEND_ANNOTATE_ROWS_EVENT 2 - - pthread_handler_t handle_slave_io(void *arg) - { ... - request_dump(mysql, ...); - ... - } - - int request_dump(MYSQL* mysql, ...) - { ... - if (opt_log_slave_updates && - mi->io_thd->variables.binlog_annotate_rows_events) - binlog_flags|= BINLOG_SEND_ANNOTATE_ROWS_EVENT; - ... - int2store(buf + 4, binlog_flags); - ... - simple_command(mysql, COM_BINLOG_DUMP, buf, ...); - ... - } - -NOTE. mysqlbinlog, when remotely requesting BINLOG_DUMP by calling the -simple_command() function, should also use this flag if it wants (in case -of the --print-annotate-rows-events option set) to recieve Annotate_rows -events. +Note. We add this new command and don't use COM_BINLOG_DUMP to avoid possible +conflicts with MySQL/Sun. 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -338,10 +323,4 @@ to internals(a)lists.mysql.com about this and suggesting to reserve the event number. -Also we should notice the introduction of the BINLOG_SEND_ANNOTATE_ROWS_EVENT -flag taking into account that MySQL/Sun may also introduce a flag with the -same value to be used in the request_dump-mysql_binlog_send interface. -But this is mainly the question of merging: if a conflict concerning this -flag occur, we may simply change the BINLOG_SEND_ANNOTATE_ROWS_EVENT value -(this does not require additional changes in the code). -=-=(Alexi - Sun, 20 Dec 2009, 16:00)=-=- Low Level Design modified. --- /tmp/wklog.47.old.19667 2009-12-20 14:00:56.000000000 +0000 +++ /tmp/wklog.47.new.19667 2009-12-20 14:00:56.000000000 +0000 @@ -196,6 +196,11 @@ ... } +NOTE. mysqlbinlog, when remotely requesting BINLOG_DUMP by calling the +simple_command() function, should also use this flag if it wants (in case +of the --print-annotate-rows-events option set) to recieve Annotate_rows +events. + 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -212,8 +217,7 @@ Log_event::read_log_event(&log, packet, ...); ... if ((*packet)[EVENT_TYPE_OFFSET + 1] != ANNOTATE_ROWS_EVENT || - flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT || - thd->server_id == 0 /* slave == mysqlbinlog */ ) + flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT) { my_net_write(net, packet->ptr(), packet->length()); } -=-=(Alexi - Sun, 20 Dec 2009, 13:14)=-=- Low Level Design modified. --- /tmp/wklog.47.old.11350 2009-12-20 13:14:04.000000000 +0200 +++ /tmp/wklog.47.new.11350 2009-12-20 13:14:04.000000000 +0200 @@ -282,23 +282,18 @@ Annotate_rows_log_event* m_annotate_event; }; -When the saved Annotate_rows object may be deleted? When all corresponding -Rows events will be processed, i.e. before processing the first non-Rows -event (note that Annotate_rows object resides in the binary log *after* -the (possible) 'BEGIN' Query event which accompanies the rows events; note -also that this deletion is adjusted with the case when some or all -corresponding Rows events are filtered out by replicate filter rules): +The saved Annotate_rows object should be deleted when all corresponding +Rows events will be processed: int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev= next_event(rli); ... - if (rli->get_annotate_event() && !IS_RBR_EVENT_TYPE(ev->get_type_code())) - rli->free_annotate_event(); - apply_event_and_update_pos(ev, ...); - if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) + if (rli->get_annotate_event() && is_last_rows_event(ev)) + rli->free_annotate_event(); + else if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) rli->set_annotate_event((Annotate_rows_log_event*) ev); else if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; @@ -307,10 +302,21 @@ where - #define IS_RBR_EVENT_TYPE(type) ( (type) == TABLE_MAP_EVENT || \ - (type) == WRITE_ROWS_EVENT || \ + bool is_last_rows_event(Log_event* ev) + { + Log_event_type type= ev->get_type_code(); + if (IS_ROWS_EVENT_TYPE(type)) + { + Rows_log_event* rows= (Rows_log_event*)ev; + return rows->get_flags(Rows_log_event::STMT_END_F); + } + + return 0; + } + + #define IS_ROWS_EVENT_TYPE(type) ((type) == WRITE_ROWS_EVENT || \ (type) == UPDATE_ROWS_EVENT || \ - (type) == DELETE_ROWS_EVENT ) + (type) == DELETE_ROWS_EVENT) 8. General remarks ~~~~~~~~~~~~~~~~~~ -=-=(Alexi - Sun, 20 Dec 2009, 09:29)=-=- Low Level Design modified. --- /tmp/wklog.47.old.32726 2009-12-20 07:29:56.000000000 +0000 +++ /tmp/wklog.47.new.32726 2009-12-20 07:29:56.000000000 +0000 @@ -56,7 +56,7 @@ 0000006A | 00 | 0 - RESERVED [27] ..... ... 00000081 | 00 | 0 - RESERVED [50] - 00000082 | 00 | 0 - ANNOTATE_RBR_EVENT [51] + 00000082 | 00 | 0 - ANNOTATE_ROWS_EVENT [51] ************************ 2. Outline of Annotate_rows event behavior -=-=(Alexi - Sat, 19 Dec 2009, 16:10)=-=- Low Level Design modified. --- /tmp/wklog.47.old.16051 2009-12-19 16:10:48.000000000 +0200 +++ /tmp/wklog.47.new.16051 2009-12-19 16:10:48.000000000 +0200 @@ -253,7 +253,7 @@ thd query to that of the described by the event, i.e. to the query which caused the subsequent Rows events (see "How Master writes Annotate_rows events to the binary log" to follow what happens further when the subsequent -Rows events is applied): +Rows events are applied): int Annotate_rows_log_event::do_apply_event(...) { -=-=(Alexi - Sat, 19 Dec 2009, 16:02)=-=- High-Level Specification modified. --- /tmp/wklog.47.old.15695 2009-12-19 16:02:33.000000000 +0200 +++ /tmp/wklog.47.new.15695 2009-12-19 16:02:33.000000000 +0200 @@ -12,7 +12,7 @@ post-header and contains the query text in its data part. Example: ************************ - ANNOTATE_RBR_EVENT + ANNOTATE_ROWS_EVENT ************************ 00000220 | B6 A0 2C 4B | time_when = 1261215926 00000224 | 33 | event_type = 51 -=-=(Alexi - Sat, 19 Dec 2009, 15:58)=-=- Low Level Design modified. --- /tmp/wklog.47.old.15437 2009-12-19 13:58:12.000000000 +0000 +++ /tmp/wklog.47.new.15437 2009-12-19 13:58:12.000000000 +0000 @@ -1 +1,337 @@ +Content +~~~~~~~ + 1. Annotate_rows event number + 2. Outline of Annotate_rows event behavior + 3. How Master writes Annotate_rows events to the binary log + 4. How slave treats replicate-annotate-rows-events option + 5. How slave IO thread requests Annotate_rows events + 6. How master executes the request + 7. How slave SQL thread processes Annotate_rows events + 8. General remarks + +1. Annotate_rows event number +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +To avoid possible event numbers conflict with MySQL/Sun, we leave a gap +between the last MySQL event number and the Annotate_rows event number: + + enum Log_event_type + { ... + INCIDENT_EVENT= 26, + // New MySQL event numbers are to be added here + MYSQL_EVENTS_END, + + MARIA_EVENTS_BEGIN= 51, + // New Maria event numbers start from here + ANNOTATE_ROWS_EVENT= 51, + + ENUM_END_EVENT + }; + +together with the corresponding extension of 'post_header_len' array in the +Format description event. (This extension does not affect the compatibility +of the binary log). Here is how Format description event looks like with +this extension: + + ************************ + FORMAT_DESCRIPTION_EVENT + ************************ + 00000004 | A1 A0 2C 4B | time_when = 1261215905 + 00000008 | 0F | event_type = 15 + 00000009 | 64 00 00 00 | server_id = 100 + 0000000D | 7F 00 00 00 | event_len = 127 + 00000011 | 83 00 00 00 | log_pos = 00000083 + 00000015 | 01 00 | flags = LOG_EVENT_BINLOG_IN_USE_F + ------------------------ + 00000017 | 04 00 | binlog_ver = 4 + 00000019 | 35 2E 32 2E | server_ver = 5.2.0-MariaDB-alpha-debug-log + ..... ... + 0000004B | A1 A0 2C 4B | time_created = 1261215905 + 0000004F | 13 | common_header_len = 19 + ------------------------ + post_header_len + ------------------------ + 00000050 | 38 | 56 - START_EVENT_V3 [1] + ..... ... + 00000069 | 02 | 2 - INCIDENT_EVENT [26] + 0000006A | 00 | 0 - RESERVED [27] + ..... ... + 00000081 | 00 | 0 - RESERVED [50] + 00000082 | 00 | 0 - ANNOTATE_RBR_EVENT [51] + ************************ + +2. Outline of Annotate_rows event behavior +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Each Annotate_rows_log_event object has two private members describing the +corresponding query: + + char *m_query_txt; + uint m_query_len; + +When the object is created for writing to a binary log, this query is taken +from 'thd' (for short, below we omit the 'Annotate_rows_log_event::' prefix +as well as other implementation details): + + Annotate_rows_log_event(THD *thd) + { + m_query_txt = thd->query(); + m_query_len = thd->query_length(); + } + +When the object is read from a binary log, the query is taken from the buffer +containing the binary log representation of the event (this buffer is allocated +in Log_event object from which all Log events are derived): + + Annotate_rows_log_event(char *buf, uint event_len, + Format_description_log_event *desc) + { + m_query_len = event_len - desc->common_header_len; + m_query_txt = buf + desc->common_header_len; + } + +The events are written to the binary log by the Log_event::write() member +which calls virtual write_data_header() and write_data_body() members +("data header" and "post header" are synonym in replication terminology). +In our case, data header is empty and data body is just the query: + + bool write_data_body(IO_CACHE *file) + { + return my_b_safe_write(file, (uchar*) m_query_txt, m_query_len); + } + +Printing the event is just printing the query: + + void Annotate_rows_log_event::print(FILE *file, PRINT_EVENT_INFO *pinfo) + { + my_b_printf(&pinfo->head_cache, "\tQuery: `%s`\n", m_query_txt); + } + +3. How Master writes Annotate_rows events to the binary log +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The event is written to the binary log just before the group of Table_map +events which precede corresponding Rows events (one query may generate +several Table map events in the binary log, but the corresponding +Annotate_rows event must be written only once before the first Table map +event; hence the boolean variable 'with_annotate' below): + + int write_locked_table_maps(THD *thd) + { ... + bool with_annotate= thd->variables.binlog_annotate_rows_events; + ... + for (uint i= 0; i < ... <number of tables> ...; ++i) + { ... + thd->binlog_write_table_map(table, ..., with_annotate); + with_annotate= 0; // write Annotate_event not more than once + ... + } + ... + } + + int THD::binlog_write_table_map(TABLE *table, ..., bool with_annotate) + { ... + Table_map_log_event the_event(...); + ... + if (with_annotate) + { + Annotate_rows_log_event anno(this); + mysql_bin_log.write(&anno); + } + + mysql_bin_log.write(&the_event); + ... + } + +4. How slave treats replicate-annotate-rows-events option +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The replicate-annotate-rows-events option is treated just as the session +value of the binlog_annotate_rows_events variable for the slave IO and +SQL threads. This setting is done during initialization of these threads: + + pthread_handler_t handle_slave_io(void *arg) + { + THD *thd= new THD; + ... + init_slave_thread(thd, SLAVE_THD_IO); + ... + } + + pthread_handler_t handle_slave_sql(void *arg) + { + THD *thd= new THD; + ... + init_slave_thread(thd, SLAVE_THD_SQL); + ... + } + + int init_slave_thread(THD* thd, SLAVE_THD_TYPE thd_type) + { ... + thd->variables.binlog_annotate_rows_events= + opt_replicate_annotate_rows_events; + ... + } + +5. How slave IO thread requests Annotate_rows events +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +When requesting an event, the slave should inform the master whether +it should send Annotate_rows events or not. To that end we add a new +BINLOG_SEND_ANNOTATE_ROWS_EVENT flag used when requesting an event: + + #define BINLOG_DUMP_NON_BLOCK 1 + #define BINLOG_SEND_ANNOTATE_ROWS_EVENT 2 + + pthread_handler_t handle_slave_io(void *arg) + { ... + request_dump(mysql, ...); + ... + } + + int request_dump(MYSQL* mysql, ...) + { ... + if (opt_log_slave_updates && + mi->io_thd->variables.binlog_annotate_rows_events) + binlog_flags|= BINLOG_SEND_ANNOTATE_ROWS_EVENT; + ... + int2store(buf + 4, binlog_flags); + ... + simple_command(mysql, COM_BINLOG_DUMP, buf, ...); + ... + } + +6. How master executes the request +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + case COM_BINLOG_DUMP: + { ... + flags= uint2korr(packet + 4); + ... + mysql_binlog_send(thd, ..., flags); + ... + } + + void mysql_binlog_send(THD* thd, ..., ushort flags) + { ... + Log_event::read_log_event(&log, packet, ...); + ... + if ((*packet)[EVENT_TYPE_OFFSET + 1] != ANNOTATE_ROWS_EVENT || + flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT || + thd->server_id == 0 /* slave == mysqlbinlog */ ) + { + my_net_write(net, packet->ptr(), packet->length()); + } + ... + } + +7. How slave SQL thread processes Annotate_rows events +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The slave processes each recieved event by "applying" it, i.e. by +calling the Log_event::apply_event() function which in turn calls +the virtual do_apply_event() member specific for each type of the +event. + + int exec_relay_log_event(THD* thd, Relay_log_info* rli) + { ... + Log_event *ev = next_event(rli); + ... + apply_event_and_update_pos(ev, ...); + + if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) + delete ev; + ... + } + + int apply_event_and_update_pos(Log_event *ev, ...) + { ... + ev->apply_event(...); + ... + } + + int Log_event::apply_event(...) + { + return do_apply_event(...); + } + +What does it mean to "apply" an Annotate_rows event? It means to set current +thd query to that of the described by the event, i.e. to the query which +caused the subsequent Rows events (see "How Master writes Annotate_rows +events to the binary log" to follow what happens further when the subsequent +Rows events is applied): + + int Annotate_rows_log_event::do_apply_event(...) + { + thd->set_query(m_query_txt, m_query_len); + } + +NOTE. I am not sure, but possibly current values of thd->query and +thd->query_length should be saved before calling set_query() and to be +restored on the Annotate_rows_log_event object deletion. +Is it really needed ? + +After calling this do_apply_event() function we may not delete the +Annotate_rows_log_event object immediatedly (see exec_relay_log_event() +above) because thd->query now points to the string inside this object. +We may keep the pointer to this object in the Relay_log_info: + + class Relay_log_info + { + public: + ... + void set_annotate_event(Annotate_rows_log_event*); + Annotate_rows_log_event* get_annotate_event(); + void free_annotate_event(); + ... + private: + Annotate_rows_log_event* m_annotate_event; + }; + +When the saved Annotate_rows object may be deleted? When all corresponding +Rows events will be processed, i.e. before processing the first non-Rows +event (note that Annotate_rows object resides in the binary log *after* +the (possible) 'BEGIN' Query event which accompanies the rows events; note +also that this deletion is adjusted with the case when some or all +corresponding Rows events are filtered out by replicate filter rules): + + int exec_relay_log_event(THD* thd, Relay_log_info* rli) + { ... + Log_event *ev= next_event(rli); + ... + if (rli->get_annotate_event() && !IS_RBR_EVENT_TYPE(ev->get_type_code())) + rli->free_annotate_event(); + + apply_event_and_update_pos(ev, ...); + + if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) + rli->set_annotate_event((Annotate_rows_log_event*) ev); + else if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) + delete ev; + ... + } + +where + + #define IS_RBR_EVENT_TYPE(type) ( (type) == TABLE_MAP_EVENT || \ + (type) == WRITE_ROWS_EVENT || \ + (type) == UPDATE_ROWS_EVENT || \ + (type) == DELETE_ROWS_EVENT ) + +8. General remarks +~~~~~~~~~~~~~~~~~~ +Kristian noticed that introducing new log event type should be coordinated +somehow with MySQL/Sun: + + Kristian: The numeric code for this event must be assigned carefully. + It should be coordinated with MySQL/Sun, otherwise we can get into a + situation where MySQL uses the same numeric code for one event that + MariaDB uses for ANNOTATE_ROWS_EVENT, which would make merging the two + impossible. + Alex: I reserved about 20 numbers not to have possible conflicts + with MySQL. + Kristian: Still, I think it would be appropriate to send a polite email + to internals(a)lists.mysql.com about this and suggesting to reserve the + event number. + +Also we should notice the introduction of the BINLOG_SEND_ANNOTATE_ROWS_EVENT +flag taking into account that MySQL/Sun may also introduce a flag with the +same value to be used in the request_dump-mysql_binlog_send interface. +But this is mainly the question of merging: if a conflict concerning this +flag occur, we may simply change the BINLOG_SEND_ANNOTATE_ROWS_EVENT value +(this does not require additional changes in the code). ------------------------------------------------------------ -=-=(View All Progress Notes, 23 total)=-=- http://askmonty.org/worklog/index.pl?tid=47&nolimit=1 DESCRIPTION: Store in binlog (and show in mysqlbinlog output) texts of statements that caused RBR events This is needed for (list from Monty): - Easier to understand why updates happened - Would make it easier to find out where in application things went wrong (as you can search for exact strings) - Allow one to filter things based on comments in the statement. The cost of this can be that the binlog will be approximately 2x in size (especially insert of big blob's would be a bit painful), so this should be an optional feature. HIGH-LEVEL SPECIFICATION: Content ~~~~~~~ 1. Annotate_rows_log_event 2. Server option: --binlog-annotate-rows-events 3. Server option: --replicate-annotate-rows-events 4. mysqlbinlog option: --print-annotate-rows-events 5. mysqlbinlog output 1. Annotate_rows_log_event [ ANNOTATE_ROWS_EVENT ] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Describes the query which caused the corresponding rows events. Has empty post-header and contains the query text in its data part. Example: ************************ ANNOTATE_ROWS_EVENT ************************ 00000220 | B6 A0 2C 4B | time_when = 1261215926 00000224 | 33 | event_type = 51 00000225 | 64 00 00 00 | server_id = 100 00000229 | 36 00 00 00 | event_len = 54 0000022D | 56 02 00 00 | log_pos = 00000256 00000231 | 00 00 | flags = <none> ------------------------ 00000233 | 49 4E 53 45 | query = "INSERT INTO t1 VALUES (1), (2), (3)" 00000237 | 52 54 20 49 | 0000023B | 4E 54 4F 20 | 0000023F | 74 31 20 56 | 00000243 | 41 4C 55 45 | 00000247 | 53 20 28 31 | 0000024B | 29 2C 20 28 | 0000024F | 32 29 2C 20 | 00000253 | 28 33 29 | ************************ In binary log, Annotate_rows event follows the (possible) 'BEGIN' Query event and precedes the first of Table map events which accompany the corresponding rows events. (See example in the "mysqlbinlog output" section below.) 2. Server option: --binlog-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the master to write Annotate_rows events to the binary log. * Variable Name: binlog_annotate_rows_events * Scope: Global & Session * Access Type: Dynamic * Data Type: bool * Default Value: OFF NOTE. Session values allows to annotate only some selected statements: ... SET SESSION binlog_annotate_rows_events=ON; ... statements to be annotated ... SET SESSION binlog_annotate_rows_events=OFF; ... statements not to be annotated ... 3. Server option: --replicate-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the slave to reproduce Annotate_rows events recieved from the master in its own binary log (sensible only in pair with log-slave-updates option). * Variable Name: replicate_annotate_rows_events * Scope: Global * Access Type: Read only * Data Type: bool * Default Value: OFF NOTE. Why do we additionally need this 'replicate' option? Why not to make the slave to reproduce this events when its binlog-annotate-rows-events global value is ON? Well, because, for example, we may want to configure the slave which should reproduce Annotate_rows events but has global binlog-annotate-rows-events = OFF meaning this to be the default value for the client threads (see also "How slave treats replicate-annotate-rows-events option" in LLD part). 4. mysqlbinlog option: --print-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ With this option, mysqlbinlog prints the content of Annotate_rows events (if the binary log does contain them). Without this option (i.e. by default), mysqlbinlog skips Annotate_rows events. 5. mysqlbinlog output ~~~~~~~~~~~~~~~~~~~~~ With --print-annotate-rows-events, mysqlbinlog outputs Annotate_rows events in a form like this: ... # at 1646 #091219 12:45:26 server id 100 end_log_pos 1714 Query thread_id=1 exec_time=0 error_code=0 SET TIMESTAMP=1261215926/*!*/; BEGIN /*!*/; # at 1714 # at 1812 # at 1853 # at 1894 # at 1938 #091219 12:45:26 server id 100 end_log_pos 1812 Query: `DELETE t1, t2 FROM t1 INNER JOIN t2 INNER JOIN t3 WHERE t1.a=t2.a AND t2.a=t3.a` #091219 12:45:26 server id 100 end_log_pos 1853 Table_map: `test`.`t1` mapped to number 16 #091219 12:45:26 server id 100 end_log_pos 1894 Table_map: `test`.`t2` mapped to number 17 #091219 12:45:26 server id 100 end_log_pos 1938 Delete_rows: table id 16 #091219 12:45:26 server id 100 end_log_pos 1982 Delete_rows: table id 17 flags: STMT_END_F ... LOW-LEVEL DESIGN: Content ~~~~~~~ 1. Annotate_rows event number 2. Outline of Annotate_rows event behavior 3. How Master writes Annotate_rows events to the binary log 4. How slave treats replicate-annotate-rows-events option 5. How slave IO thread requests Annotate_rows events 6. How master executes the request 7. How slave SQL thread processes Annotate_rows events 8. General remarks 1. Annotate_rows event number ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To avoid possible event numbers conflict with MySQL/Sun, we leave a gap between the last MySQL event number and the Annotate_rows event number: enum Log_event_type { ... INCIDENT_EVENT= 26, // New MySQL event numbers are to be added here MYSQL_EVENTS_END, MARIA_EVENTS_BEGIN= 51, // New Maria event numbers start from here ANNOTATE_ROWS_EVENT= 51, ENUM_END_EVENT }; together with the corresponding extension of 'post_header_len' array in the Format description event. (This extension does not affect the compatibility of the binary log). Here is how Format description event looks like with this extension: ************************ FORMAT_DESCRIPTION_EVENT ************************ 00000004 | A1 A0 2C 4B | time_when = 1261215905 00000008 | 0F | event_type = 15 00000009 | 64 00 00 00 | server_id = 100 0000000D | 7F 00 00 00 | event_len = 127 00000011 | 83 00 00 00 | log_pos = 00000083 00000015 | 01 00 | flags = LOG_EVENT_BINLOG_IN_USE_F ------------------------ 00000017 | 04 00 | binlog_ver = 4 00000019 | 35 2E 32 2E | server_ver = 5.2.0-MariaDB-alpha-debug-log ..... ... 0000004B | A1 A0 2C 4B | time_created = 1261215905 0000004F | 13 | common_header_len = 19 ------------------------ post_header_len ------------------------ 00000050 | 38 | 56 - START_EVENT_V3 [1] ..... ... 00000069 | 02 | 2 - INCIDENT_EVENT [26] 0000006A | 00 | 0 - RESERVED [27] ..... ... 00000081 | 00 | 0 - RESERVED [50] 00000082 | 00 | 0 - ANNOTATE_ROWS_EVENT [51] ************************ 2. Outline of Annotate_rows event behavior ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Each Annotate_rows_log_event object has two private members describing the corresponding query: char *m_query_txt; uint m_query_len; When the object is created for writing to a binary log, this query is taken from 'thd' (for short, below we omit the 'Annotate_rows_log_event::' prefix as well as other implementation details): Annotate_rows_log_event(THD *thd) { m_query_txt = thd->query(); m_query_len = thd->query_length(); } When the object is read from a binary log, the query is taken from the buffer containing the binary log representation of the event (this buffer is allocated in Log_event object from which all Log events are derived): Annotate_rows_log_event(char *buf, uint event_len, Format_description_log_event *desc) { m_query_len = event_len - desc->common_header_len; m_query_txt = buf + desc->common_header_len; } The events are written to the binary log by the Log_event::write() member which calls virtual write_data_header() and write_data_body() members ("data header" and "post header" are synonym in replication terminology). In our case, data header is empty and data body is just the query: bool write_data_body(IO_CACHE *file) { return my_b_safe_write(file, (uchar*) m_query_txt, m_query_len); } Printing the event is just printing the query: void Annotate_rows_log_event::print(FILE *file, PRINT_EVENT_INFO *pinfo) { my_b_printf(&pinfo->head_cache, "\tQuery: `%s`\n", m_query_txt); } 3. How Master writes Annotate_rows events to the binary log ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The event is written to the binary log just before the group of Table_map events which precede corresponding Rows events (one query may generate several Table map events in the binary log, but the corresponding Annotate_rows event must be written only once before the first Table map event; hence the boolean variable 'with_annotate' below): int write_locked_table_maps(THD *thd) { ... bool with_annotate= thd->variables.binlog_annotate_rows_events; ... for (uint i= 0; i < ... <number of tables> ...; ++i) { ... thd->binlog_write_table_map(table, ..., with_annotate); with_annotate= 0; // write Annotate_event not more than once ... } ... } int THD::binlog_write_table_map(TABLE *table, ..., bool with_annotate) { ... Table_map_log_event the_event(...); ... if (with_annotate) { Annotate_rows_log_event anno(this); mysql_bin_log.write(&anno); } mysql_bin_log.write(&the_event); ... } 4. How slave treats replicate-annotate-rows-events option ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The replicate-annotate-rows-events option is treated just as the session value of the binlog_annotate_rows_events variable for the slave IO and SQL threads. This setting is done during initialization of these threads: pthread_handler_t handle_slave_io(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_IO); ... } pthread_handler_t handle_slave_sql(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_SQL); ... } int init_slave_thread(THD* thd, SLAVE_THD_TYPE thd_type) { ... thd->variables.binlog_annotate_rows_events= opt_replicate_annotate_rows_events; ... } 5. How slave IO thread requests Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If the replicate-annotate-rows-events option is not set on a slave, there is no need for master to send Annotate_rows events to this slave. The slave (or mysqlbinlog in remote case), before requesting binlog dump via the COM_BINLOG_DUMP command, informs the master whether it should send these events by executing the newly added COM_BINLOG_DUMP_OPTIONS_EXT server command: case COM_BINLOG_DUMP_OPTIONS_EXT: thd->binlog_dump_flags_ext= packet[0]; my_ok(thd); break; Note. We add this new command and don't use COM_BINLOG_DUMP to avoid possible conflicts with MySQL/Sun. 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ case COM_BINLOG_DUMP: { ... flags= uint2korr(packet + 4); ... mysql_binlog_send(thd, ..., flags); ... } void mysql_binlog_send(THD* thd, ..., ushort flags) { ... Log_event::read_log_event(&log, packet, ...); ... if ((*packet)[EVENT_TYPE_OFFSET + 1] != ANNOTATE_ROWS_EVENT || flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT) { my_net_write(net, packet->ptr(), packet->length()); } ... } 7. How slave SQL thread processes Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The slave processes each recieved event by "applying" it, i.e. by calling the Log_event::apply_event() function which in turn calls the virtual do_apply_event() member specific for each type of the event. int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev = next_event(rli); ... apply_event_and_update_pos(ev, ...); if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } int apply_event_and_update_pos(Log_event *ev, ...) { ... ev->apply_event(...); ... } int Log_event::apply_event(...) { return do_apply_event(...); } What does it mean to "apply" an Annotate_rows event? It means to set current thd query to that of the described by the event, i.e. to the query which caused the subsequent Rows events (see "How Master writes Annotate_rows events to the binary log" to follow what happens further when the subsequent Rows events are applied): int Annotate_rows_log_event::do_apply_event(...) { thd->set_query(m_query_txt, m_query_len); } NOTE. I am not sure, but possibly current values of thd->query and thd->query_length should be saved before calling set_query() and to be restored on the Annotate_rows_log_event object deletion. Is it really needed ? After calling this do_apply_event() function we may not delete the Annotate_rows_log_event object immediatedly (see exec_relay_log_event() above) because thd->query now points to the string inside this object. We may keep the pointer to this object in the Relay_log_info: class Relay_log_info { public: ... void set_annotate_event(Annotate_rows_log_event*); Annotate_rows_log_event* get_annotate_event(); void free_annotate_event(); ... private: Annotate_rows_log_event* m_annotate_event; }; The saved Annotate_rows object should be deleted when all corresponding Rows events will be processed: int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev= next_event(rli); ... apply_event_and_update_pos(ev, ...); if (rli->get_annotate_event() && is_last_rows_event(ev)) rli->free_annotate_event(); else if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) rli->set_annotate_event((Annotate_rows_log_event*) ev); else if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } where bool is_last_rows_event(Log_event* ev) { Log_event_type type= ev->get_type_code(); if (IS_ROWS_EVENT_TYPE(type)) { Rows_log_event* rows= (Rows_log_event*)ev; return rows->get_flags(Rows_log_event::STMT_END_F); } return 0; } #define IS_ROWS_EVENT_TYPE(type) ((type) == WRITE_ROWS_EVENT || \ (type) == UPDATE_ROWS_EVENT || \ (type) == DELETE_ROWS_EVENT) 8. General remarks ~~~~~~~~~~~~~~~~~~ Kristian noticed that introducing new log event type should be coordinated somehow with MySQL/Sun: Kristian: The numeric code for this event must be assigned carefully. It should be coordinated with MySQL/Sun, otherwise we can get into a situation where MySQL uses the same numeric code for one event that MariaDB uses for ANNOTATE_ROWS_EVENT, which would make merging the two impossible. Alex: I reserved about 20 numbers not to have possible conflicts with MySQL. Kristian: Still, I think it would be appropriate to send a polite email to internals(a)lists.mysql.com about this and suggesting to reserve the event number. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Serg): Store in binlog text of statements that caused RBR events (47)
by worklog-noreply＠askmonty.org 05 Feb '10

05 Feb '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Store in binlog text of statements that caused RBR events CREATION DATE..: Sat, 15 Aug 2009, 23:48 SUPERVISOR.....: Monty IMPLEMENTOR....: COPIES TO......: Knielsen, Serg CATEGORY.......: Server-Sprint TASK ID........: 47 (http://askmonty.org/worklog/?tid=47) VERSION........: Server-9.x STATUS.........: In-Progress PRIORITY.......: 60 WORKED HOURS...: 20 ESTIMATE.......: 35 (hours remain) ORIG. ESTIMATE.: 35 PROGRESS NOTES: -=-=(Serg - Fri, 05 Feb 2010, 14:04)=-=- Observers changed: Knielsen,Serg -=-=(Guest - Fri, 05 Feb 2010, 13:40)=-=- Category updated. --- /tmp/wklog.47.old.9197 2010-02-05 13:40:36.000000000 +0200 +++ /tmp/wklog.47.new.9197 2010-02-05 13:40:36.000000000 +0200 @@ -1 +1 @@ -Server-RawIdeaBin +Server-Sprint -=-=(Guest - Fri, 05 Feb 2010, 13:40)=-=- Status updated. --- /tmp/wklog.47.old.9197 2010-02-05 13:40:36.000000000 +0200 +++ /tmp/wklog.47.new.9197 2010-02-05 13:40:36.000000000 +0200 @@ -1 +1 @@ -Un-Assigned +In-Progress -=-=(Alexi - Thu, 04 Feb 2010, 09:54)=-=- Low Level Design modified. --- /tmp/wklog.47.old.16174 2010-02-04 09:54:13.000000000 +0200 +++ /tmp/wklog.47.new.16174 2010-02-04 09:54:13.000000000 +0200 @@ -171,35 +171,20 @@ 5. How slave IO thread requests Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -When requesting an event, the slave should inform the master whether -it should send Annotate_rows events or not. To that end we add a new -BINLOG_SEND_ANNOTATE_ROWS_EVENT flag used when requesting an event: +If the replicate-annotate-rows-events option is not set on a slave, there +is no need for master to send Annotate_rows events to this slave. The slave +(or mysqlbinlog in remote case), before requesting binlog dump via the +COM_BINLOG_DUMP command, informs the master whether it should send these +events by executing the newly added COM_BINLOG_DUMP_OPTIONS_EXT server +command: + + case COM_BINLOG_DUMP_OPTIONS_EXT: + thd->binlog_dump_flags_ext= packet[0]; + my_ok(thd); + break; - #define BINLOG_DUMP_NON_BLOCK 1 - #define BINLOG_SEND_ANNOTATE_ROWS_EVENT 2 - - pthread_handler_t handle_slave_io(void *arg) - { ... - request_dump(mysql, ...); - ... - } - - int request_dump(MYSQL* mysql, ...) - { ... - if (opt_log_slave_updates && - mi->io_thd->variables.binlog_annotate_rows_events) - binlog_flags|= BINLOG_SEND_ANNOTATE_ROWS_EVENT; - ... - int2store(buf + 4, binlog_flags); - ... - simple_command(mysql, COM_BINLOG_DUMP, buf, ...); - ... - } - -NOTE. mysqlbinlog, when remotely requesting BINLOG_DUMP by calling the -simple_command() function, should also use this flag if it wants (in case -of the --print-annotate-rows-events option set) to recieve Annotate_rows -events. +Note. We add this new command and don't use COM_BINLOG_DUMP to avoid possible +conflicts with MySQL/Sun. 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -338,10 +323,4 @@ to internals(a)lists.mysql.com about this and suggesting to reserve the event number. -Also we should notice the introduction of the BINLOG_SEND_ANNOTATE_ROWS_EVENT -flag taking into account that MySQL/Sun may also introduce a flag with the -same value to be used in the request_dump-mysql_binlog_send interface. -But this is mainly the question of merging: if a conflict concerning this -flag occur, we may simply change the BINLOG_SEND_ANNOTATE_ROWS_EVENT value -(this does not require additional changes in the code). -=-=(Alexi - Sun, 20 Dec 2009, 16:00)=-=- Low Level Design modified. --- /tmp/wklog.47.old.19667 2009-12-20 14:00:56.000000000 +0000 +++ /tmp/wklog.47.new.19667 2009-12-20 14:00:56.000000000 +0000 @@ -196,6 +196,11 @@ ... } +NOTE. mysqlbinlog, when remotely requesting BINLOG_DUMP by calling the +simple_command() function, should also use this flag if it wants (in case +of the --print-annotate-rows-events option set) to recieve Annotate_rows +events. + 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -212,8 +217,7 @@ Log_event::read_log_event(&log, packet, ...); ... if ((*packet)[EVENT_TYPE_OFFSET + 1] != ANNOTATE_ROWS_EVENT || - flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT || - thd->server_id == 0 /* slave == mysqlbinlog */ ) + flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT) { my_net_write(net, packet->ptr(), packet->length()); } -=-=(Alexi - Sun, 20 Dec 2009, 13:14)=-=- Low Level Design modified. --- /tmp/wklog.47.old.11350 2009-12-20 13:14:04.000000000 +0200 +++ /tmp/wklog.47.new.11350 2009-12-20 13:14:04.000000000 +0200 @@ -282,23 +282,18 @@ Annotate_rows_log_event* m_annotate_event; }; -When the saved Annotate_rows object may be deleted? When all corresponding -Rows events will be processed, i.e. before processing the first non-Rows -event (note that Annotate_rows object resides in the binary log *after* -the (possible) 'BEGIN' Query event which accompanies the rows events; note -also that this deletion is adjusted with the case when some or all -corresponding Rows events are filtered out by replicate filter rules): +The saved Annotate_rows object should be deleted when all corresponding +Rows events will be processed: int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev= next_event(rli); ... - if (rli->get_annotate_event() && !IS_RBR_EVENT_TYPE(ev->get_type_code())) - rli->free_annotate_event(); - apply_event_and_update_pos(ev, ...); - if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) + if (rli->get_annotate_event() && is_last_rows_event(ev)) + rli->free_annotate_event(); + else if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) rli->set_annotate_event((Annotate_rows_log_event*) ev); else if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; @@ -307,10 +302,21 @@ where - #define IS_RBR_EVENT_TYPE(type) ( (type) == TABLE_MAP_EVENT || \ - (type) == WRITE_ROWS_EVENT || \ + bool is_last_rows_event(Log_event* ev) + { + Log_event_type type= ev->get_type_code(); + if (IS_ROWS_EVENT_TYPE(type)) + { + Rows_log_event* rows= (Rows_log_event*)ev; + return rows->get_flags(Rows_log_event::STMT_END_F); + } + + return 0; + } + + #define IS_ROWS_EVENT_TYPE(type) ((type) == WRITE_ROWS_EVENT || \ (type) == UPDATE_ROWS_EVENT || \ - (type) == DELETE_ROWS_EVENT ) + (type) == DELETE_ROWS_EVENT) 8. General remarks ~~~~~~~~~~~~~~~~~~ -=-=(Alexi - Sun, 20 Dec 2009, 09:29)=-=- Low Level Design modified. --- /tmp/wklog.47.old.32726 2009-12-20 07:29:56.000000000 +0000 +++ /tmp/wklog.47.new.32726 2009-12-20 07:29:56.000000000 +0000 @@ -56,7 +56,7 @@ 0000006A | 00 | 0 - RESERVED [27] ..... ... 00000081 | 00 | 0 - RESERVED [50] - 00000082 | 00 | 0 - ANNOTATE_RBR_EVENT [51] + 00000082 | 00 | 0 - ANNOTATE_ROWS_EVENT [51] ************************ 2. Outline of Annotate_rows event behavior -=-=(Alexi - Sat, 19 Dec 2009, 16:10)=-=- Low Level Design modified. --- /tmp/wklog.47.old.16051 2009-12-19 16:10:48.000000000 +0200 +++ /tmp/wklog.47.new.16051 2009-12-19 16:10:48.000000000 +0200 @@ -253,7 +253,7 @@ thd query to that of the described by the event, i.e. to the query which caused the subsequent Rows events (see "How Master writes Annotate_rows events to the binary log" to follow what happens further when the subsequent -Rows events is applied): +Rows events are applied): int Annotate_rows_log_event::do_apply_event(...) { -=-=(Alexi - Sat, 19 Dec 2009, 16:02)=-=- High-Level Specification modified. --- /tmp/wklog.47.old.15695 2009-12-19 16:02:33.000000000 +0200 +++ /tmp/wklog.47.new.15695 2009-12-19 16:02:33.000000000 +0200 @@ -12,7 +12,7 @@ post-header and contains the query text in its data part. Example: ************************ - ANNOTATE_RBR_EVENT + ANNOTATE_ROWS_EVENT ************************ 00000220 | B6 A0 2C 4B | time_when = 1261215926 00000224 | 33 | event_type = 51 -=-=(Alexi - Sat, 19 Dec 2009, 15:58)=-=- Low Level Design modified. --- /tmp/wklog.47.old.15437 2009-12-19 13:58:12.000000000 +0000 +++ /tmp/wklog.47.new.15437 2009-12-19 13:58:12.000000000 +0000 @@ -1 +1,337 @@ +Content +~~~~~~~ + 1. Annotate_rows event number + 2. Outline of Annotate_rows event behavior + 3. How Master writes Annotate_rows events to the binary log + 4. How slave treats replicate-annotate-rows-events option + 5. How slave IO thread requests Annotate_rows events + 6. How master executes the request + 7. How slave SQL thread processes Annotate_rows events + 8. General remarks + +1. Annotate_rows event number +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +To avoid possible event numbers conflict with MySQL/Sun, we leave a gap +between the last MySQL event number and the Annotate_rows event number: + + enum Log_event_type + { ... + INCIDENT_EVENT= 26, + // New MySQL event numbers are to be added here + MYSQL_EVENTS_END, + + MARIA_EVENTS_BEGIN= 51, + // New Maria event numbers start from here + ANNOTATE_ROWS_EVENT= 51, + + ENUM_END_EVENT + }; + +together with the corresponding extension of 'post_header_len' array in the +Format description event. (This extension does not affect the compatibility +of the binary log). Here is how Format description event looks like with +this extension: + + ************************ + FORMAT_DESCRIPTION_EVENT + ************************ + 00000004 | A1 A0 2C 4B | time_when = 1261215905 + 00000008 | 0F | event_type = 15 + 00000009 | 64 00 00 00 | server_id = 100 + 0000000D | 7F 00 00 00 | event_len = 127 + 00000011 | 83 00 00 00 | log_pos = 00000083 + 00000015 | 01 00 | flags = LOG_EVENT_BINLOG_IN_USE_F + ------------------------ + 00000017 | 04 00 | binlog_ver = 4 + 00000019 | 35 2E 32 2E | server_ver = 5.2.0-MariaDB-alpha-debug-log + ..... ... + 0000004B | A1 A0 2C 4B | time_created = 1261215905 + 0000004F | 13 | common_header_len = 19 + ------------------------ + post_header_len + ------------------------ + 00000050 | 38 | 56 - START_EVENT_V3 [1] + ..... ... + 00000069 | 02 | 2 - INCIDENT_EVENT [26] + 0000006A | 00 | 0 - RESERVED [27] + ..... ... + 00000081 | 00 | 0 - RESERVED [50] + 00000082 | 00 | 0 - ANNOTATE_RBR_EVENT [51] + ************************ + +2. Outline of Annotate_rows event behavior +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Each Annotate_rows_log_event object has two private members describing the +corresponding query: + + char *m_query_txt; + uint m_query_len; + +When the object is created for writing to a binary log, this query is taken +from 'thd' (for short, below we omit the 'Annotate_rows_log_event::' prefix +as well as other implementation details): + + Annotate_rows_log_event(THD *thd) + { + m_query_txt = thd->query(); + m_query_len = thd->query_length(); + } + +When the object is read from a binary log, the query is taken from the buffer +containing the binary log representation of the event (this buffer is allocated +in Log_event object from which all Log events are derived): + + Annotate_rows_log_event(char *buf, uint event_len, + Format_description_log_event *desc) + { + m_query_len = event_len - desc->common_header_len; + m_query_txt = buf + desc->common_header_len; + } + +The events are written to the binary log by the Log_event::write() member +which calls virtual write_data_header() and write_data_body() members +("data header" and "post header" are synonym in replication terminology). +In our case, data header is empty and data body is just the query: + + bool write_data_body(IO_CACHE *file) + { + return my_b_safe_write(file, (uchar*) m_query_txt, m_query_len); + } + +Printing the event is just printing the query: + + void Annotate_rows_log_event::print(FILE *file, PRINT_EVENT_INFO *pinfo) + { + my_b_printf(&pinfo->head_cache, "\tQuery: `%s`\n", m_query_txt); + } + +3. How Master writes Annotate_rows events to the binary log +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The event is written to the binary log just before the group of Table_map +events which precede corresponding Rows events (one query may generate +several Table map events in the binary log, but the corresponding +Annotate_rows event must be written only once before the first Table map +event; hence the boolean variable 'with_annotate' below): + + int write_locked_table_maps(THD *thd) + { ... + bool with_annotate= thd->variables.binlog_annotate_rows_events; + ... + for (uint i= 0; i < ... <number of tables> ...; ++i) + { ... + thd->binlog_write_table_map(table, ..., with_annotate); + with_annotate= 0; // write Annotate_event not more than once + ... + } + ... + } + + int THD::binlog_write_table_map(TABLE *table, ..., bool with_annotate) + { ... + Table_map_log_event the_event(...); + ... + if (with_annotate) + { + Annotate_rows_log_event anno(this); + mysql_bin_log.write(&anno); + } + + mysql_bin_log.write(&the_event); + ... + } + +4. How slave treats replicate-annotate-rows-events option +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The replicate-annotate-rows-events option is treated just as the session +value of the binlog_annotate_rows_events variable for the slave IO and +SQL threads. This setting is done during initialization of these threads: + + pthread_handler_t handle_slave_io(void *arg) + { + THD *thd= new THD; + ... + init_slave_thread(thd, SLAVE_THD_IO); + ... + } + + pthread_handler_t handle_slave_sql(void *arg) + { + THD *thd= new THD; + ... + init_slave_thread(thd, SLAVE_THD_SQL); + ... + } + + int init_slave_thread(THD* thd, SLAVE_THD_TYPE thd_type) + { ... + thd->variables.binlog_annotate_rows_events= + opt_replicate_annotate_rows_events; + ... + } + +5. How slave IO thread requests Annotate_rows events +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +When requesting an event, the slave should inform the master whether +it should send Annotate_rows events or not. To that end we add a new +BINLOG_SEND_ANNOTATE_ROWS_EVENT flag used when requesting an event: + + #define BINLOG_DUMP_NON_BLOCK 1 + #define BINLOG_SEND_ANNOTATE_ROWS_EVENT 2 + + pthread_handler_t handle_slave_io(void *arg) + { ... + request_dump(mysql, ...); + ... + } + + int request_dump(MYSQL* mysql, ...) + { ... + if (opt_log_slave_updates && + mi->io_thd->variables.binlog_annotate_rows_events) + binlog_flags|= BINLOG_SEND_ANNOTATE_ROWS_EVENT; + ... + int2store(buf + 4, binlog_flags); + ... + simple_command(mysql, COM_BINLOG_DUMP, buf, ...); + ... + } + +6. How master executes the request +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + case COM_BINLOG_DUMP: + { ... + flags= uint2korr(packet + 4); + ... + mysql_binlog_send(thd, ..., flags); + ... + } + + void mysql_binlog_send(THD* thd, ..., ushort flags) + { ... + Log_event::read_log_event(&log, packet, ...); + ... + if ((*packet)[EVENT_TYPE_OFFSET + 1] != ANNOTATE_ROWS_EVENT || + flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT || + thd->server_id == 0 /* slave == mysqlbinlog */ ) + { + my_net_write(net, packet->ptr(), packet->length()); + } + ... + } + +7. How slave SQL thread processes Annotate_rows events +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The slave processes each recieved event by "applying" it, i.e. by +calling the Log_event::apply_event() function which in turn calls +the virtual do_apply_event() member specific for each type of the +event. + + int exec_relay_log_event(THD* thd, Relay_log_info* rli) + { ... + Log_event *ev = next_event(rli); + ... + apply_event_and_update_pos(ev, ...); + + if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) + delete ev; + ... + } + + int apply_event_and_update_pos(Log_event *ev, ...) + { ... + ev->apply_event(...); + ... + } + + int Log_event::apply_event(...) + { + return do_apply_event(...); + } + +What does it mean to "apply" an Annotate_rows event? It means to set current +thd query to that of the described by the event, i.e. to the query which +caused the subsequent Rows events (see "How Master writes Annotate_rows +events to the binary log" to follow what happens further when the subsequent +Rows events is applied): + + int Annotate_rows_log_event::do_apply_event(...) + { + thd->set_query(m_query_txt, m_query_len); + } + +NOTE. I am not sure, but possibly current values of thd->query and +thd->query_length should be saved before calling set_query() and to be +restored on the Annotate_rows_log_event object deletion. +Is it really needed ? + +After calling this do_apply_event() function we may not delete the +Annotate_rows_log_event object immediatedly (see exec_relay_log_event() +above) because thd->query now points to the string inside this object. +We may keep the pointer to this object in the Relay_log_info: + + class Relay_log_info + { + public: + ... + void set_annotate_event(Annotate_rows_log_event*); + Annotate_rows_log_event* get_annotate_event(); + void free_annotate_event(); + ... + private: + Annotate_rows_log_event* m_annotate_event; + }; + +When the saved Annotate_rows object may be deleted? When all corresponding +Rows events will be processed, i.e. before processing the first non-Rows +event (note that Annotate_rows object resides in the binary log *after* +the (possible) 'BEGIN' Query event which accompanies the rows events; note +also that this deletion is adjusted with the case when some or all +corresponding Rows events are filtered out by replicate filter rules): + + int exec_relay_log_event(THD* thd, Relay_log_info* rli) + { ... + Log_event *ev= next_event(rli); + ... + if (rli->get_annotate_event() && !IS_RBR_EVENT_TYPE(ev->get_type_code())) + rli->free_annotate_event(); + + apply_event_and_update_pos(ev, ...); + + if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) + rli->set_annotate_event((Annotate_rows_log_event*) ev); + else if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) + delete ev; + ... + } + +where + + #define IS_RBR_EVENT_TYPE(type) ( (type) == TABLE_MAP_EVENT || \ + (type) == WRITE_ROWS_EVENT || \ + (type) == UPDATE_ROWS_EVENT || \ + (type) == DELETE_ROWS_EVENT ) + +8. General remarks +~~~~~~~~~~~~~~~~~~ +Kristian noticed that introducing new log event type should be coordinated +somehow with MySQL/Sun: + + Kristian: The numeric code for this event must be assigned carefully. + It should be coordinated with MySQL/Sun, otherwise we can get into a + situation where MySQL uses the same numeric code for one event that + MariaDB uses for ANNOTATE_ROWS_EVENT, which would make merging the two + impossible. + Alex: I reserved about 20 numbers not to have possible conflicts + with MySQL. + Kristian: Still, I think it would be appropriate to send a polite email + to internals(a)lists.mysql.com about this and suggesting to reserve the + event number. + +Also we should notice the introduction of the BINLOG_SEND_ANNOTATE_ROWS_EVENT +flag taking into account that MySQL/Sun may also introduce a flag with the +same value to be used in the request_dump-mysql_binlog_send interface. +But this is mainly the question of merging: if a conflict concerning this +flag occur, we may simply change the BINLOG_SEND_ANNOTATE_ROWS_EVENT value +(this does not require additional changes in the code). ------------------------------------------------------------ -=-=(View All Progress Notes, 23 total)=-=- http://askmonty.org/worklog/index.pl?tid=47&nolimit=1 DESCRIPTION: Store in binlog (and show in mysqlbinlog output) texts of statements that caused RBR events This is needed for (list from Monty): - Easier to understand why updates happened - Would make it easier to find out where in application things went wrong (as you can search for exact strings) - Allow one to filter things based on comments in the statement. The cost of this can be that the binlog will be approximately 2x in size (especially insert of big blob's would be a bit painful), so this should be an optional feature. HIGH-LEVEL SPECIFICATION: Content ~~~~~~~ 1. Annotate_rows_log_event 2. Server option: --binlog-annotate-rows-events 3. Server option: --replicate-annotate-rows-events 4. mysqlbinlog option: --print-annotate-rows-events 5. mysqlbinlog output 1. Annotate_rows_log_event [ ANNOTATE_ROWS_EVENT ] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Describes the query which caused the corresponding rows events. Has empty post-header and contains the query text in its data part. Example: ************************ ANNOTATE_ROWS_EVENT ************************ 00000220 | B6 A0 2C 4B | time_when = 1261215926 00000224 | 33 | event_type = 51 00000225 | 64 00 00 00 | server_id = 100 00000229 | 36 00 00 00 | event_len = 54 0000022D | 56 02 00 00 | log_pos = 00000256 00000231 | 00 00 | flags = <none> ------------------------ 00000233 | 49 4E 53 45 | query = "INSERT INTO t1 VALUES (1), (2), (3)" 00000237 | 52 54 20 49 | 0000023B | 4E 54 4F 20 | 0000023F | 74 31 20 56 | 00000243 | 41 4C 55 45 | 00000247 | 53 20 28 31 | 0000024B | 29 2C 20 28 | 0000024F | 32 29 2C 20 | 00000253 | 28 33 29 | ************************ In binary log, Annotate_rows event follows the (possible) 'BEGIN' Query event and precedes the first of Table map events which accompany the corresponding rows events. (See example in the "mysqlbinlog output" section below.) 2. Server option: --binlog-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the master to write Annotate_rows events to the binary log. * Variable Name: binlog_annotate_rows_events * Scope: Global & Session * Access Type: Dynamic * Data Type: bool * Default Value: OFF NOTE. Session values allows to annotate only some selected statements: ... SET SESSION binlog_annotate_rows_events=ON; ... statements to be annotated ... SET SESSION binlog_annotate_rows_events=OFF; ... statements not to be annotated ... 3. Server option: --replicate-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the slave to reproduce Annotate_rows events recieved from the master in its own binary log (sensible only in pair with log-slave-updates option). * Variable Name: replicate_annotate_rows_events * Scope: Global * Access Type: Read only * Data Type: bool * Default Value: OFF NOTE. Why do we additionally need this 'replicate' option? Why not to make the slave to reproduce this events when its binlog-annotate-rows-events global value is ON? Well, because, for example, we may want to configure the slave which should reproduce Annotate_rows events but has global binlog-annotate-rows-events = OFF meaning this to be the default value for the client threads (see also "How slave treats replicate-annotate-rows-events option" in LLD part). 4. mysqlbinlog option: --print-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ With this option, mysqlbinlog prints the content of Annotate_rows events (if the binary log does contain them). Without this option (i.e. by default), mysqlbinlog skips Annotate_rows events. 5. mysqlbinlog output ~~~~~~~~~~~~~~~~~~~~~ With --print-annotate-rows-events, mysqlbinlog outputs Annotate_rows events in a form like this: ... # at 1646 #091219 12:45:26 server id 100 end_log_pos 1714 Query thread_id=1 exec_time=0 error_code=0 SET TIMESTAMP=1261215926/*!*/; BEGIN /*!*/; # at 1714 # at 1812 # at 1853 # at 1894 # at 1938 #091219 12:45:26 server id 100 end_log_pos 1812 Query: `DELETE t1, t2 FROM t1 INNER JOIN t2 INNER JOIN t3 WHERE t1.a=t2.a AND t2.a=t3.a` #091219 12:45:26 server id 100 end_log_pos 1853 Table_map: `test`.`t1` mapped to number 16 #091219 12:45:26 server id 100 end_log_pos 1894 Table_map: `test`.`t2` mapped to number 17 #091219 12:45:26 server id 100 end_log_pos 1938 Delete_rows: table id 16 #091219 12:45:26 server id 100 end_log_pos 1982 Delete_rows: table id 17 flags: STMT_END_F ... LOW-LEVEL DESIGN: Content ~~~~~~~ 1. Annotate_rows event number 2. Outline of Annotate_rows event behavior 3. How Master writes Annotate_rows events to the binary log 4. How slave treats replicate-annotate-rows-events option 5. How slave IO thread requests Annotate_rows events 6. How master executes the request 7. How slave SQL thread processes Annotate_rows events 8. General remarks 1. Annotate_rows event number ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To avoid possible event numbers conflict with MySQL/Sun, we leave a gap between the last MySQL event number and the Annotate_rows event number: enum Log_event_type { ... INCIDENT_EVENT= 26, // New MySQL event numbers are to be added here MYSQL_EVENTS_END, MARIA_EVENTS_BEGIN= 51, // New Maria event numbers start from here ANNOTATE_ROWS_EVENT= 51, ENUM_END_EVENT }; together with the corresponding extension of 'post_header_len' array in the Format description event. (This extension does not affect the compatibility of the binary log). Here is how Format description event looks like with this extension: ************************ FORMAT_DESCRIPTION_EVENT ************************ 00000004 | A1 A0 2C 4B | time_when = 1261215905 00000008 | 0F | event_type = 15 00000009 | 64 00 00 00 | server_id = 100 0000000D | 7F 00 00 00 | event_len = 127 00000011 | 83 00 00 00 | log_pos = 00000083 00000015 | 01 00 | flags = LOG_EVENT_BINLOG_IN_USE_F ------------------------ 00000017 | 04 00 | binlog_ver = 4 00000019 | 35 2E 32 2E | server_ver = 5.2.0-MariaDB-alpha-debug-log ..... ... 0000004B | A1 A0 2C 4B | time_created = 1261215905 0000004F | 13 | common_header_len = 19 ------------------------ post_header_len ------------------------ 00000050 | 38 | 56 - START_EVENT_V3 [1] ..... ... 00000069 | 02 | 2 - INCIDENT_EVENT [26] 0000006A | 00 | 0 - RESERVED [27] ..... ... 00000081 | 00 | 0 - RESERVED [50] 00000082 | 00 | 0 - ANNOTATE_ROWS_EVENT [51] ************************ 2. Outline of Annotate_rows event behavior ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Each Annotate_rows_log_event object has two private members describing the corresponding query: char *m_query_txt; uint m_query_len; When the object is created for writing to a binary log, this query is taken from 'thd' (for short, below we omit the 'Annotate_rows_log_event::' prefix as well as other implementation details): Annotate_rows_log_event(THD *thd) { m_query_txt = thd->query(); m_query_len = thd->query_length(); } When the object is read from a binary log, the query is taken from the buffer containing the binary log representation of the event (this buffer is allocated in Log_event object from which all Log events are derived): Annotate_rows_log_event(char *buf, uint event_len, Format_description_log_event *desc) { m_query_len = event_len - desc->common_header_len; m_query_txt = buf + desc->common_header_len; } The events are written to the binary log by the Log_event::write() member which calls virtual write_data_header() and write_data_body() members ("data header" and "post header" are synonym in replication terminology). In our case, data header is empty and data body is just the query: bool write_data_body(IO_CACHE *file) { return my_b_safe_write(file, (uchar*) m_query_txt, m_query_len); } Printing the event is just printing the query: void Annotate_rows_log_event::print(FILE *file, PRINT_EVENT_INFO *pinfo) { my_b_printf(&pinfo->head_cache, "\tQuery: `%s`\n", m_query_txt); } 3. How Master writes Annotate_rows events to the binary log ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The event is written to the binary log just before the group of Table_map events which precede corresponding Rows events (one query may generate several Table map events in the binary log, but the corresponding Annotate_rows event must be written only once before the first Table map event; hence the boolean variable 'with_annotate' below): int write_locked_table_maps(THD *thd) { ... bool with_annotate= thd->variables.binlog_annotate_rows_events; ... for (uint i= 0; i < ... <number of tables> ...; ++i) { ... thd->binlog_write_table_map(table, ..., with_annotate); with_annotate= 0; // write Annotate_event not more than once ... } ... } int THD::binlog_write_table_map(TABLE *table, ..., bool with_annotate) { ... Table_map_log_event the_event(...); ... if (with_annotate) { Annotate_rows_log_event anno(this); mysql_bin_log.write(&anno); } mysql_bin_log.write(&the_event); ... } 4. How slave treats replicate-annotate-rows-events option ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The replicate-annotate-rows-events option is treated just as the session value of the binlog_annotate_rows_events variable for the slave IO and SQL threads. This setting is done during initialization of these threads: pthread_handler_t handle_slave_io(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_IO); ... } pthread_handler_t handle_slave_sql(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_SQL); ... } int init_slave_thread(THD* thd, SLAVE_THD_TYPE thd_type) { ... thd->variables.binlog_annotate_rows_events= opt_replicate_annotate_rows_events; ... } 5. How slave IO thread requests Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If the replicate-annotate-rows-events option is not set on a slave, there is no need for master to send Annotate_rows events to this slave. The slave (or mysqlbinlog in remote case), before requesting binlog dump via the COM_BINLOG_DUMP command, informs the master whether it should send these events by executing the newly added COM_BINLOG_DUMP_OPTIONS_EXT server command: case COM_BINLOG_DUMP_OPTIONS_EXT: thd->binlog_dump_flags_ext= packet[0]; my_ok(thd); break; Note. We add this new command and don't use COM_BINLOG_DUMP to avoid possible conflicts with MySQL/Sun. 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ case COM_BINLOG_DUMP: { ... flags= uint2korr(packet + 4); ... mysql_binlog_send(thd, ..., flags); ... } void mysql_binlog_send(THD* thd, ..., ushort flags) { ... Log_event::read_log_event(&log, packet, ...); ... if ((*packet)[EVENT_TYPE_OFFSET + 1] != ANNOTATE_ROWS_EVENT || flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT) { my_net_write(net, packet->ptr(), packet->length()); } ... } 7. How slave SQL thread processes Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The slave processes each recieved event by "applying" it, i.e. by calling the Log_event::apply_event() function which in turn calls the virtual do_apply_event() member specific for each type of the event. int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev = next_event(rli); ... apply_event_and_update_pos(ev, ...); if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } int apply_event_and_update_pos(Log_event *ev, ...) { ... ev->apply_event(...); ... } int Log_event::apply_event(...) { return do_apply_event(...); } What does it mean to "apply" an Annotate_rows event? It means to set current thd query to that of the described by the event, i.e. to the query which caused the subsequent Rows events (see "How Master writes Annotate_rows events to the binary log" to follow what happens further when the subsequent Rows events are applied): int Annotate_rows_log_event::do_apply_event(...) { thd->set_query(m_query_txt, m_query_len); } NOTE. I am not sure, but possibly current values of thd->query and thd->query_length should be saved before calling set_query() and to be restored on the Annotate_rows_log_event object deletion. Is it really needed ? After calling this do_apply_event() function we may not delete the Annotate_rows_log_event object immediatedly (see exec_relay_log_event() above) because thd->query now points to the string inside this object. We may keep the pointer to this object in the Relay_log_info: class Relay_log_info { public: ... void set_annotate_event(Annotate_rows_log_event*); Annotate_rows_log_event* get_annotate_event(); void free_annotate_event(); ... private: Annotate_rows_log_event* m_annotate_event; }; The saved Annotate_rows object should be deleted when all corresponding Rows events will be processed: int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev= next_event(rli); ... apply_event_and_update_pos(ev, ...); if (rli->get_annotate_event() && is_last_rows_event(ev)) rli->free_annotate_event(); else if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) rli->set_annotate_event((Annotate_rows_log_event*) ev); else if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } where bool is_last_rows_event(Log_event* ev) { Log_event_type type= ev->get_type_code(); if (IS_ROWS_EVENT_TYPE(type)) { Rows_log_event* rows= (Rows_log_event*)ev; return rows->get_flags(Rows_log_event::STMT_END_F); } return 0; } #define IS_ROWS_EVENT_TYPE(type) ((type) == WRITE_ROWS_EVENT || \ (type) == UPDATE_ROWS_EVENT || \ (type) == DELETE_ROWS_EVENT) 8. General remarks ~~~~~~~~~~~~~~~~~~ Kristian noticed that introducing new log event type should be coordinated somehow with MySQL/Sun: Kristian: The numeric code for this event must be assigned carefully. It should be coordinated with MySQL/Sun, otherwise we can get into a situation where MySQL uses the same numeric code for one event that MariaDB uses for ANNOTATE_ROWS_EVENT, which would make merging the two impossible. Alex: I reserved about 20 numbers not to have possible conflicts with MySQL. Kristian: Still, I think it would be appropriate to send a polite email to internals(a)lists.mysql.com about this and suggesting to reserve the event number. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Guest): Store in binlog text of statements that caused RBR events (47)
by worklog-noreply＠askmonty.org 05 Feb '10

05 Feb '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Store in binlog text of statements that caused RBR events CREATION DATE..: Sat, 15 Aug 2009, 23:48 SUPERVISOR.....: Monty IMPLEMENTOR....: COPIES TO......: Knielsen CATEGORY.......: Server-Sprint TASK ID........: 47 (http://askmonty.org/worklog/?tid=47) VERSION........: Server-9.x STATUS.........: In-Progress PRIORITY.......: 60 WORKED HOURS...: 20 ESTIMATE.......: 35 (hours remain) ORIG. ESTIMATE.: 35 PROGRESS NOTES: -=-=(Guest - Fri, 05 Feb 2010, 13:40)=-=- Category updated. --- /tmp/wklog.47.old.9197 2010-02-05 13:40:36.000000000 +0200 +++ /tmp/wklog.47.new.9197 2010-02-05 13:40:36.000000000 +0200 @@ -1 +1 @@ -Server-RawIdeaBin +Server-Sprint -=-=(Guest - Fri, 05 Feb 2010, 13:40)=-=- Status updated. --- /tmp/wklog.47.old.9197 2010-02-05 13:40:36.000000000 +0200 +++ /tmp/wklog.47.new.9197 2010-02-05 13:40:36.000000000 +0200 @@ -1 +1 @@ -Un-Assigned +In-Progress -=-=(Alexi - Thu, 04 Feb 2010, 09:54)=-=- Low Level Design modified. --- /tmp/wklog.47.old.16174 2010-02-04 09:54:13.000000000 +0200 +++ /tmp/wklog.47.new.16174 2010-02-04 09:54:13.000000000 +0200 @@ -171,35 +171,20 @@ 5. How slave IO thread requests Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -When requesting an event, the slave should inform the master whether -it should send Annotate_rows events or not. To that end we add a new -BINLOG_SEND_ANNOTATE_ROWS_EVENT flag used when requesting an event: +If the replicate-annotate-rows-events option is not set on a slave, there +is no need for master to send Annotate_rows events to this slave. The slave +(or mysqlbinlog in remote case), before requesting binlog dump via the +COM_BINLOG_DUMP command, informs the master whether it should send these +events by executing the newly added COM_BINLOG_DUMP_OPTIONS_EXT server +command: + + case COM_BINLOG_DUMP_OPTIONS_EXT: + thd->binlog_dump_flags_ext= packet[0]; + my_ok(thd); + break; - #define BINLOG_DUMP_NON_BLOCK 1 - #define BINLOG_SEND_ANNOTATE_ROWS_EVENT 2 - - pthread_handler_t handle_slave_io(void *arg) - { ... - request_dump(mysql, ...); - ... - } - - int request_dump(MYSQL* mysql, ...) - { ... - if (opt_log_slave_updates && - mi->io_thd->variables.binlog_annotate_rows_events) - binlog_flags|= BINLOG_SEND_ANNOTATE_ROWS_EVENT; - ... - int2store(buf + 4, binlog_flags); - ... - simple_command(mysql, COM_BINLOG_DUMP, buf, ...); - ... - } - -NOTE. mysqlbinlog, when remotely requesting BINLOG_DUMP by calling the -simple_command() function, should also use this flag if it wants (in case -of the --print-annotate-rows-events option set) to recieve Annotate_rows -events. +Note. We add this new command and don't use COM_BINLOG_DUMP to avoid possible +conflicts with MySQL/Sun. 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -338,10 +323,4 @@ to internals(a)lists.mysql.com about this and suggesting to reserve the event number. -Also we should notice the introduction of the BINLOG_SEND_ANNOTATE_ROWS_EVENT -flag taking into account that MySQL/Sun may also introduce a flag with the -same value to be used in the request_dump-mysql_binlog_send interface. -But this is mainly the question of merging: if a conflict concerning this -flag occur, we may simply change the BINLOG_SEND_ANNOTATE_ROWS_EVENT value -(this does not require additional changes in the code). -=-=(Alexi - Sun, 20 Dec 2009, 16:00)=-=- Low Level Design modified. --- /tmp/wklog.47.old.19667 2009-12-20 14:00:56.000000000 +0000 +++ /tmp/wklog.47.new.19667 2009-12-20 14:00:56.000000000 +0000 @@ -196,6 +196,11 @@ ... } +NOTE. mysqlbinlog, when remotely requesting BINLOG_DUMP by calling the +simple_command() function, should also use this flag if it wants (in case +of the --print-annotate-rows-events option set) to recieve Annotate_rows +events. + 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -212,8 +217,7 @@ Log_event::read_log_event(&log, packet, ...); ... if ((*packet)[EVENT_TYPE_OFFSET + 1] != ANNOTATE_ROWS_EVENT || - flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT || - thd->server_id == 0 /* slave == mysqlbinlog */ ) + flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT) { my_net_write(net, packet->ptr(), packet->length()); } -=-=(Alexi - Sun, 20 Dec 2009, 13:14)=-=- Low Level Design modified. --- /tmp/wklog.47.old.11350 2009-12-20 13:14:04.000000000 +0200 +++ /tmp/wklog.47.new.11350 2009-12-20 13:14:04.000000000 +0200 @@ -282,23 +282,18 @@ Annotate_rows_log_event* m_annotate_event; }; -When the saved Annotate_rows object may be deleted? When all corresponding -Rows events will be processed, i.e. before processing the first non-Rows -event (note that Annotate_rows object resides in the binary log *after* -the (possible) 'BEGIN' Query event which accompanies the rows events; note -also that this deletion is adjusted with the case when some or all -corresponding Rows events are filtered out by replicate filter rules): +The saved Annotate_rows object should be deleted when all corresponding +Rows events will be processed: int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev= next_event(rli); ... - if (rli->get_annotate_event() && !IS_RBR_EVENT_TYPE(ev->get_type_code())) - rli->free_annotate_event(); - apply_event_and_update_pos(ev, ...); - if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) + if (rli->get_annotate_event() && is_last_rows_event(ev)) + rli->free_annotate_event(); + else if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) rli->set_annotate_event((Annotate_rows_log_event*) ev); else if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; @@ -307,10 +302,21 @@ where - #define IS_RBR_EVENT_TYPE(type) ( (type) == TABLE_MAP_EVENT || \ - (type) == WRITE_ROWS_EVENT || \ + bool is_last_rows_event(Log_event* ev) + { + Log_event_type type= ev->get_type_code(); + if (IS_ROWS_EVENT_TYPE(type)) + { + Rows_log_event* rows= (Rows_log_event*)ev; + return rows->get_flags(Rows_log_event::STMT_END_F); + } + + return 0; + } + + #define IS_ROWS_EVENT_TYPE(type) ((type) == WRITE_ROWS_EVENT || \ (type) == UPDATE_ROWS_EVENT || \ - (type) == DELETE_ROWS_EVENT ) + (type) == DELETE_ROWS_EVENT) 8. General remarks ~~~~~~~~~~~~~~~~~~ -=-=(Alexi - Sun, 20 Dec 2009, 09:29)=-=- Low Level Design modified. --- /tmp/wklog.47.old.32726 2009-12-20 07:29:56.000000000 +0000 +++ /tmp/wklog.47.new.32726 2009-12-20 07:29:56.000000000 +0000 @@ -56,7 +56,7 @@ 0000006A | 00 | 0 - RESERVED [27] ..... ... 00000081 | 00 | 0 - RESERVED [50] - 00000082 | 00 | 0 - ANNOTATE_RBR_EVENT [51] + 00000082 | 00 | 0 - ANNOTATE_ROWS_EVENT [51] ************************ 2. Outline of Annotate_rows event behavior -=-=(Alexi - Sat, 19 Dec 2009, 16:10)=-=- Low Level Design modified. --- /tmp/wklog.47.old.16051 2009-12-19 16:10:48.000000000 +0200 +++ /tmp/wklog.47.new.16051 2009-12-19 16:10:48.000000000 +0200 @@ -253,7 +253,7 @@ thd query to that of the described by the event, i.e. to the query which caused the subsequent Rows events (see "How Master writes Annotate_rows events to the binary log" to follow what happens further when the subsequent -Rows events is applied): +Rows events are applied): int Annotate_rows_log_event::do_apply_event(...) { -=-=(Alexi - Sat, 19 Dec 2009, 16:02)=-=- High-Level Specification modified. --- /tmp/wklog.47.old.15695 2009-12-19 16:02:33.000000000 +0200 +++ /tmp/wklog.47.new.15695 2009-12-19 16:02:33.000000000 +0200 @@ -12,7 +12,7 @@ post-header and contains the query text in its data part. Example: ************************ - ANNOTATE_RBR_EVENT + ANNOTATE_ROWS_EVENT ************************ 00000220 | B6 A0 2C 4B | time_when = 1261215926 00000224 | 33 | event_type = 51 -=-=(Alexi - Sat, 19 Dec 2009, 15:58)=-=- Low Level Design modified. --- /tmp/wklog.47.old.15437 2009-12-19 13:58:12.000000000 +0000 +++ /tmp/wklog.47.new.15437 2009-12-19 13:58:12.000000000 +0000 @@ -1 +1,337 @@ +Content +~~~~~~~ + 1. Annotate_rows event number + 2. Outline of Annotate_rows event behavior + 3. How Master writes Annotate_rows events to the binary log + 4. How slave treats replicate-annotate-rows-events option + 5. How slave IO thread requests Annotate_rows events + 6. How master executes the request + 7. How slave SQL thread processes Annotate_rows events + 8. General remarks + +1. Annotate_rows event number +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +To avoid possible event numbers conflict with MySQL/Sun, we leave a gap +between the last MySQL event number and the Annotate_rows event number: + + enum Log_event_type + { ... + INCIDENT_EVENT= 26, + // New MySQL event numbers are to be added here + MYSQL_EVENTS_END, + + MARIA_EVENTS_BEGIN= 51, + // New Maria event numbers start from here + ANNOTATE_ROWS_EVENT= 51, + + ENUM_END_EVENT + }; + +together with the corresponding extension of 'post_header_len' array in the +Format description event. (This extension does not affect the compatibility +of the binary log). Here is how Format description event looks like with +this extension: + + ************************ + FORMAT_DESCRIPTION_EVENT + ************************ + 00000004 | A1 A0 2C 4B | time_when = 1261215905 + 00000008 | 0F | event_type = 15 + 00000009 | 64 00 00 00 | server_id = 100 + 0000000D | 7F 00 00 00 | event_len = 127 + 00000011 | 83 00 00 00 | log_pos = 00000083 + 00000015 | 01 00 | flags = LOG_EVENT_BINLOG_IN_USE_F + ------------------------ + 00000017 | 04 00 | binlog_ver = 4 + 00000019 | 35 2E 32 2E | server_ver = 5.2.0-MariaDB-alpha-debug-log + ..... ... + 0000004B | A1 A0 2C 4B | time_created = 1261215905 + 0000004F | 13 | common_header_len = 19 + ------------------------ + post_header_len + ------------------------ + 00000050 | 38 | 56 - START_EVENT_V3 [1] + ..... ... + 00000069 | 02 | 2 - INCIDENT_EVENT [26] + 0000006A | 00 | 0 - RESERVED [27] + ..... ... + 00000081 | 00 | 0 - RESERVED [50] + 00000082 | 00 | 0 - ANNOTATE_RBR_EVENT [51] + ************************ + +2. Outline of Annotate_rows event behavior +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Each Annotate_rows_log_event object has two private members describing the +corresponding query: + + char *m_query_txt; + uint m_query_len; + +When the object is created for writing to a binary log, this query is taken +from 'thd' (for short, below we omit the 'Annotate_rows_log_event::' prefix +as well as other implementation details): + + Annotate_rows_log_event(THD *thd) + { + m_query_txt = thd->query(); + m_query_len = thd->query_length(); + } + +When the object is read from a binary log, the query is taken from the buffer +containing the binary log representation of the event (this buffer is allocated +in Log_event object from which all Log events are derived): + + Annotate_rows_log_event(char *buf, uint event_len, + Format_description_log_event *desc) + { + m_query_len = event_len - desc->common_header_len; + m_query_txt = buf + desc->common_header_len; + } + +The events are written to the binary log by the Log_event::write() member +which calls virtual write_data_header() and write_data_body() members +("data header" and "post header" are synonym in replication terminology). +In our case, data header is empty and data body is just the query: + + bool write_data_body(IO_CACHE *file) + { + return my_b_safe_write(file, (uchar*) m_query_txt, m_query_len); + } + +Printing the event is just printing the query: + + void Annotate_rows_log_event::print(FILE *file, PRINT_EVENT_INFO *pinfo) + { + my_b_printf(&pinfo->head_cache, "\tQuery: `%s`\n", m_query_txt); + } + +3. How Master writes Annotate_rows events to the binary log +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The event is written to the binary log just before the group of Table_map +events which precede corresponding Rows events (one query may generate +several Table map events in the binary log, but the corresponding +Annotate_rows event must be written only once before the first Table map +event; hence the boolean variable 'with_annotate' below): + + int write_locked_table_maps(THD *thd) + { ... + bool with_annotate= thd->variables.binlog_annotate_rows_events; + ... + for (uint i= 0; i < ... <number of tables> ...; ++i) + { ... + thd->binlog_write_table_map(table, ..., with_annotate); + with_annotate= 0; // write Annotate_event not more than once + ... + } + ... + } + + int THD::binlog_write_table_map(TABLE *table, ..., bool with_annotate) + { ... + Table_map_log_event the_event(...); + ... + if (with_annotate) + { + Annotate_rows_log_event anno(this); + mysql_bin_log.write(&anno); + } + + mysql_bin_log.write(&the_event); + ... + } + +4. How slave treats replicate-annotate-rows-events option +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The replicate-annotate-rows-events option is treated just as the session +value of the binlog_annotate_rows_events variable for the slave IO and +SQL threads. This setting is done during initialization of these threads: + + pthread_handler_t handle_slave_io(void *arg) + { + THD *thd= new THD; + ... + init_slave_thread(thd, SLAVE_THD_IO); + ... + } + + pthread_handler_t handle_slave_sql(void *arg) + { + THD *thd= new THD; + ... + init_slave_thread(thd, SLAVE_THD_SQL); + ... + } + + int init_slave_thread(THD* thd, SLAVE_THD_TYPE thd_type) + { ... + thd->variables.binlog_annotate_rows_events= + opt_replicate_annotate_rows_events; + ... + } + +5. How slave IO thread requests Annotate_rows events +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +When requesting an event, the slave should inform the master whether +it should send Annotate_rows events or not. To that end we add a new +BINLOG_SEND_ANNOTATE_ROWS_EVENT flag used when requesting an event: + + #define BINLOG_DUMP_NON_BLOCK 1 + #define BINLOG_SEND_ANNOTATE_ROWS_EVENT 2 + + pthread_handler_t handle_slave_io(void *arg) + { ... + request_dump(mysql, ...); + ... + } + + int request_dump(MYSQL* mysql, ...) + { ... + if (opt_log_slave_updates && + mi->io_thd->variables.binlog_annotate_rows_events) + binlog_flags|= BINLOG_SEND_ANNOTATE_ROWS_EVENT; + ... + int2store(buf + 4, binlog_flags); + ... + simple_command(mysql, COM_BINLOG_DUMP, buf, ...); + ... + } + +6. How master executes the request +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + case COM_BINLOG_DUMP: + { ... + flags= uint2korr(packet + 4); + ... + mysql_binlog_send(thd, ..., flags); + ... + } + + void mysql_binlog_send(THD* thd, ..., ushort flags) + { ... + Log_event::read_log_event(&log, packet, ...); + ... + if ((*packet)[EVENT_TYPE_OFFSET + 1] != ANNOTATE_ROWS_EVENT || + flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT || + thd->server_id == 0 /* slave == mysqlbinlog */ ) + { + my_net_write(net, packet->ptr(), packet->length()); + } + ... + } + +7. How slave SQL thread processes Annotate_rows events +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The slave processes each recieved event by "applying" it, i.e. by +calling the Log_event::apply_event() function which in turn calls +the virtual do_apply_event() member specific for each type of the +event. + + int exec_relay_log_event(THD* thd, Relay_log_info* rli) + { ... + Log_event *ev = next_event(rli); + ... + apply_event_and_update_pos(ev, ...); + + if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) + delete ev; + ... + } + + int apply_event_and_update_pos(Log_event *ev, ...) + { ... + ev->apply_event(...); + ... + } + + int Log_event::apply_event(...) + { + return do_apply_event(...); + } + +What does it mean to "apply" an Annotate_rows event? It means to set current +thd query to that of the described by the event, i.e. to the query which +caused the subsequent Rows events (see "How Master writes Annotate_rows +events to the binary log" to follow what happens further when the subsequent +Rows events is applied): + + int Annotate_rows_log_event::do_apply_event(...) + { + thd->set_query(m_query_txt, m_query_len); + } + +NOTE. I am not sure, but possibly current values of thd->query and +thd->query_length should be saved before calling set_query() and to be +restored on the Annotate_rows_log_event object deletion. +Is it really needed ? + +After calling this do_apply_event() function we may not delete the +Annotate_rows_log_event object immediatedly (see exec_relay_log_event() +above) because thd->query now points to the string inside this object. +We may keep the pointer to this object in the Relay_log_info: + + class Relay_log_info + { + public: + ... + void set_annotate_event(Annotate_rows_log_event*); + Annotate_rows_log_event* get_annotate_event(); + void free_annotate_event(); + ... + private: + Annotate_rows_log_event* m_annotate_event; + }; + +When the saved Annotate_rows object may be deleted? When all corresponding +Rows events will be processed, i.e. before processing the first non-Rows +event (note that Annotate_rows object resides in the binary log *after* +the (possible) 'BEGIN' Query event which accompanies the rows events; note +also that this deletion is adjusted with the case when some or all +corresponding Rows events are filtered out by replicate filter rules): + + int exec_relay_log_event(THD* thd, Relay_log_info* rli) + { ... + Log_event *ev= next_event(rli); + ... + if (rli->get_annotate_event() && !IS_RBR_EVENT_TYPE(ev->get_type_code())) + rli->free_annotate_event(); + + apply_event_and_update_pos(ev, ...); + + if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) + rli->set_annotate_event((Annotate_rows_log_event*) ev); + else if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) + delete ev; + ... + } + +where + + #define IS_RBR_EVENT_TYPE(type) ( (type) == TABLE_MAP_EVENT || \ + (type) == WRITE_ROWS_EVENT || \ + (type) == UPDATE_ROWS_EVENT || \ + (type) == DELETE_ROWS_EVENT ) + +8. General remarks +~~~~~~~~~~~~~~~~~~ +Kristian noticed that introducing new log event type should be coordinated +somehow with MySQL/Sun: + + Kristian: The numeric code for this event must be assigned carefully. + It should be coordinated with MySQL/Sun, otherwise we can get into a + situation where MySQL uses the same numeric code for one event that + MariaDB uses for ANNOTATE_ROWS_EVENT, which would make merging the two + impossible. + Alex: I reserved about 20 numbers not to have possible conflicts + with MySQL. + Kristian: Still, I think it would be appropriate to send a polite email + to internals(a)lists.mysql.com about this and suggesting to reserve the + event number. + +Also we should notice the introduction of the BINLOG_SEND_ANNOTATE_ROWS_EVENT +flag taking into account that MySQL/Sun may also introduce a flag with the +same value to be used in the request_dump-mysql_binlog_send interface. +But this is mainly the question of merging: if a conflict concerning this +flag occur, we may simply change the BINLOG_SEND_ANNOTATE_ROWS_EVENT value +(this does not require additional changes in the code). -=-=(Alexi - Sat, 19 Dec 2009, 15:41)=-=- High-Level Specification modified. --- /tmp/wklog.47.old.14545 2009-12-19 15:41:21.000000000 +0200 +++ /tmp/wklog.47.new.14545 2009-12-19 15:41:21.000000000 +0200 @@ -1,122 +1,107 @@ -First suggestion: - -> I think for this we would actually need a new binlog event type -> (Comment_log_event?). Unless we want to log an empty statement Query_log_event -> containing only a comment (a bit of a hack). - -New server option -~~~~~~~~~~~~~~~~~ - --binlog-annotate-rows-events - -Setting this option makes RBR (rows-) events in the binary log to be -preceded by Annotate rows events (see below). The corresponding -'binlog_annotate_rows_events' system variable is dynamic and has both -global and session values. Default global value is OFF. - -Note. Session values are usefull to make it possible to annotate only - some selected statements: +Content +~~~~~~~ + 1. Annotate_rows_log_event + 2. Server option: --binlog-annotate-rows-events + 3. Server option: --replicate-annotate-rows-events + 4. mysqlbinlog option: --print-annotate-rows-events + 5. mysqlbinlog output + +1. Annotate_rows_log_event [ ANNOTATE_ROWS_EVENT ] +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Describes the query which caused the corresponding rows events. Has empty +post-header and contains the query text in its data part. Example: + + ************************ + ANNOTATE_RBR_EVENT + ************************ + 00000220 | B6 A0 2C 4B | time_when = 1261215926 + 00000224 | 33 | event_type = 51 + 00000225 | 64 00 00 00 | server_id = 100 + 00000229 | 36 00 00 00 | event_len = 54 + 0000022D | 56 02 00 00 | log_pos = 00000256 + 00000231 | 00 00 | flags = <none> + ------------------------ + 00000233 | 49 4E 53 45 | query = "INSERT INTO t1 VALUES (1), (2), (3)" + 00000237 | 52 54 20 49 | + 0000023B | 4E 54 4F 20 | + 0000023F | 74 31 20 56 | + 00000243 | 41 4C 55 45 | + 00000247 | 53 20 28 31 | + 0000024B | 29 2C 20 28 | + 0000024F | 32 29 2C 20 | + 00000253 | 28 33 29 | + ************************ + +In binary log, Annotate_rows event follows the (possible) 'BEGIN' Query event +and precedes the first of Table map events which accompany the corresponding +rows events. (See example in the "mysqlbinlog output" section below.) + +2. Server option: --binlog-annotate-rows-events +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Tells the master to write Annotate_rows events to the binary log. + + * Variable Name: binlog_annotate_rows_events + * Scope: Global & Session + * Access Type: Dynamic + * Data Type: bool + * Default Value: OFF +NOTE. Session values allows to annotate only some selected statements: ... SET SESSION binlog_annotate_rows_events=ON; ... statements to be annotated ... SET SESSION binlog_annotate_rows_events=OFF; ... statements not to be annotated ... -New binlog event type -~~~~~~~~~~~~~~~~~~~~~ - Annotate_rows_log_event [ ANNOTATE_ROWS_EVENT ] - -Describes the query which caused the corresponding rows event. In binary log, -precedes each Table_map_log_event. Contains empty post-header and the query -text in its data part. - -The numeric code for this event must be assigned carefully. It should be -coordinated with MySQL/Sun, otherwise we can get into a situation where MySQL -uses the same numeric code for one event that MariaDB uses for -ANNOTATE_ROWS_EVENT, which would make merging the two impossible. - -Example: - - ... - ************************ - ANNOTATE_ROWS_EVENT [51] - ************************ - 000000C7 | 54 1B 12 4B | time_when = 1259477844 - 000000CB | 33 | event_type = 51 - 000000CC | 64 00 00 00 | server_id = 100 - 000000D0 | 2C 00 00 00 | event_len = 44 - 000000D4 | F3 00 00 00 | log_pos = 000000F3 - 000000D8 | 00 00 | flags = <none> - ------------------------ - 000000DA | 69 6E 73 65 | query = "insert into t1 values (1)" - 000000DE | 72 74 20 69 | - 000000E2 | 6E 74 6F 20 | - 000000E6 | 74 31 20 76 | - 000000EA | 61 6C 75 65 | - 000000EE | 73 20 28 31 | - 000000F2 | 29 | - ************************ - TABLE_MAP_EVENT [19] - ************************ - 000000F3 | 54 1B 12 4B | time_when = 1259477844 - 000000F7 | 13 | event_type = 19 - 000000F8 | 64 00 00 00 | server_id = 100 - 000000FC | 29 00 00 00 | event_len = 41 - 00000100 | 1C 01 00 00 | log_pos = 0000011C - 00000104 | 00 00 | flags = <none> - ------------------------ - ... - ************************ - WRITE_ROWS_EVENT [23] - ************************ - 0000011C | 54 1B 12 4B | time_when = 1259477844 - 00000120 | 17 | event_type = 23 - 00000121 | 64 00 00 00 | server_id = 100 - 00000125 | 22 00 00 00 | event_len = 34 - 00000129 | 3E 01 00 00 | log_pos = 0000013E - 0000012D | 10 00 | flags = LOG_EVENT_UPDATE_TABLE_MAP_VERSION_F - ------------------------ - 0000012F | 0F 00 00 00 | table_id = 15 - ... - -New mysqlbinlog option -~~~~~~~~~~~~~~~~~~~~~~ - --print-annotate-rows-events - -With this option, mysqlbinlog prints the content of Annotate-rows -events (if the binary log does contain them). Without this option -(i.e. by default), mysqlbinlog skips Annotate rows events. - - -mysqlbinlog output -~~~~~~~~~~~~~~~~~~ -Something like this: +3. Server option: --replicate-annotate-rows-events +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Tells the slave to reproduce Annotate_rows events recieved from the master +in its own binary log (sensible only in pair with log-slave-updates option). + + * Variable Name: replicate_annotate_rows_events + * Scope: Global + * Access Type: Read only + * Data Type: bool + * Default Value: OFF + +NOTE. Why do we additionally need this 'replicate' option? Why not to make +the slave to reproduce this events when its binlog-annotate-rows-events +global value is ON? Well, because, for example, we may want to configure +the slave which should reproduce Annotate_rows events but has global +binlog-annotate-rows-events = OFF meaning this to be the default value for +the client threads (see also "How slave treats replicate-annotate-rows-events +option" in LLD part). + +4. mysqlbinlog option: --print-annotate-rows-events +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +With this option, mysqlbinlog prints the content of Annotate_rows events (if +the binary log does contain them). Without this option (i.e. by default), +mysqlbinlog skips Annotate_rows events. +5. mysqlbinlog output +~~~~~~~~~~~~~~~~~~~~~ +With --print-annotate-rows-events, mysqlbinlog outputs Annotate_rows events +in a form like this: ... - # at 199 - # at 243 - # at 284 - #091129 9:57:24 server id 100 end_log_pos 243 Query: `insert into t1 values -(1)` - #091129 9:57:24 server id 100 end_log_pos 284 Table_map: `test`.`t1` mapped -to number 15 - #091129 9:57:24 server id 100 end_log_pos 318 Write_rows: table id 15 + # at 1646 + #091219 12:45:26 server id 100 end_log_pos 1714 Query thread_id=1 +exec_time=0 error_code=0 + SET TIMESTAMP=1261215926/*!*/; + BEGIN + /*!*/; + # at 1714 + # at 1812 + # at 1853 + # at 1894 + # at 1938 + #091219 12:45:26 server id 100 end_log_pos 1812 Query: `DELETE t1, t2 FROM +t1 INNER JOIN t2 INNER JOIN t3 WHERE t1.a=t2.a AND t2.a=t3.a` + #091219 12:45:26 server id 100 end_log_pos 1853 Table_map: `test`.`t1` +mapped to number 16 + #091219 12:45:26 server id 100 end_log_pos 1894 Table_map: `test`.`t2` +mapped to number 17 + #091219 12:45:26 server id 100 end_log_pos 1938 Delete_rows: table id 16 + #091219 12:45:26 server id 100 end_log_pos 1982 Delete_rows: table id 17 flags: STMT_END_F - - BINLOG ' - VBsSSzNkAAAALAAAAPMAAAAAAGluc2VydCBpbnRvIHQxIHZhbHVlcyAoMSk= - VBsSSxNkAAAAKQAAABwBAAAAAA8AAAAAAAAABHRlc3QAAnQxAAEDAAE= - VBsSSxdkAAAAIgAAAD4BAAAQAA8AAAAAAAEAAf/+AQAAAA== - '/*!*/; - ### INSERT INTO test.t1 - ### SET - ### @1=1 /* INT meta=0 nullable=1 is_null=0 */ ... -When master sends Annotate rows events -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -1. Master always sends Annotate_rows events to mysqlbinlog (in - remote case). -2. Master sends Annotate_rows events to a slave only if the slave has - both log-slave-updates and binlog-annotate-rows-events options set. - ------------------------------------------------------------ -=-=(View All Progress Notes, 22 total)=-=- http://askmonty.org/worklog/index.pl?tid=47&nolimit=1 DESCRIPTION: Store in binlog (and show in mysqlbinlog output) texts of statements that caused RBR events This is needed for (list from Monty): - Easier to understand why updates happened - Would make it easier to find out where in application things went wrong (as you can search for exact strings) - Allow one to filter things based on comments in the statement. The cost of this can be that the binlog will be approximately 2x in size (especially insert of big blob's would be a bit painful), so this should be an optional feature. HIGH-LEVEL SPECIFICATION: Content ~~~~~~~ 1. Annotate_rows_log_event 2. Server option: --binlog-annotate-rows-events 3. Server option: --replicate-annotate-rows-events 4. mysqlbinlog option: --print-annotate-rows-events 5. mysqlbinlog output 1. Annotate_rows_log_event [ ANNOTATE_ROWS_EVENT ] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Describes the query which caused the corresponding rows events. Has empty post-header and contains the query text in its data part. Example: ************************ ANNOTATE_ROWS_EVENT ************************ 00000220 | B6 A0 2C 4B | time_when = 1261215926 00000224 | 33 | event_type = 51 00000225 | 64 00 00 00 | server_id = 100 00000229 | 36 00 00 00 | event_len = 54 0000022D | 56 02 00 00 | log_pos = 00000256 00000231 | 00 00 | flags = <none> ------------------------ 00000233 | 49 4E 53 45 | query = "INSERT INTO t1 VALUES (1), (2), (3)" 00000237 | 52 54 20 49 | 0000023B | 4E 54 4F 20 | 0000023F | 74 31 20 56 | 00000243 | 41 4C 55 45 | 00000247 | 53 20 28 31 | 0000024B | 29 2C 20 28 | 0000024F | 32 29 2C 20 | 00000253 | 28 33 29 | ************************ In binary log, Annotate_rows event follows the (possible) 'BEGIN' Query event and precedes the first of Table map events which accompany the corresponding rows events. (See example in the "mysqlbinlog output" section below.) 2. Server option: --binlog-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the master to write Annotate_rows events to the binary log. * Variable Name: binlog_annotate_rows_events * Scope: Global & Session * Access Type: Dynamic * Data Type: bool * Default Value: OFF NOTE. Session values allows to annotate only some selected statements: ... SET SESSION binlog_annotate_rows_events=ON; ... statements to be annotated ... SET SESSION binlog_annotate_rows_events=OFF; ... statements not to be annotated ... 3. Server option: --replicate-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the slave to reproduce Annotate_rows events recieved from the master in its own binary log (sensible only in pair with log-slave-updates option). * Variable Name: replicate_annotate_rows_events * Scope: Global * Access Type: Read only * Data Type: bool * Default Value: OFF NOTE. Why do we additionally need this 'replicate' option? Why not to make the slave to reproduce this events when its binlog-annotate-rows-events global value is ON? Well, because, for example, we may want to configure the slave which should reproduce Annotate_rows events but has global binlog-annotate-rows-events = OFF meaning this to be the default value for the client threads (see also "How slave treats replicate-annotate-rows-events option" in LLD part). 4. mysqlbinlog option: --print-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ With this option, mysqlbinlog prints the content of Annotate_rows events (if the binary log does contain them). Without this option (i.e. by default), mysqlbinlog skips Annotate_rows events. 5. mysqlbinlog output ~~~~~~~~~~~~~~~~~~~~~ With --print-annotate-rows-events, mysqlbinlog outputs Annotate_rows events in a form like this: ... # at 1646 #091219 12:45:26 server id 100 end_log_pos 1714 Query thread_id=1 exec_time=0 error_code=0 SET TIMESTAMP=1261215926/*!*/; BEGIN /*!*/; # at 1714 # at 1812 # at 1853 # at 1894 # at 1938 #091219 12:45:26 server id 100 end_log_pos 1812 Query: `DELETE t1, t2 FROM t1 INNER JOIN t2 INNER JOIN t3 WHERE t1.a=t2.a AND t2.a=t3.a` #091219 12:45:26 server id 100 end_log_pos 1853 Table_map: `test`.`t1` mapped to number 16 #091219 12:45:26 server id 100 end_log_pos 1894 Table_map: `test`.`t2` mapped to number 17 #091219 12:45:26 server id 100 end_log_pos 1938 Delete_rows: table id 16 #091219 12:45:26 server id 100 end_log_pos 1982 Delete_rows: table id 17 flags: STMT_END_F ... LOW-LEVEL DESIGN: Content ~~~~~~~ 1. Annotate_rows event number 2. Outline of Annotate_rows event behavior 3. How Master writes Annotate_rows events to the binary log 4. How slave treats replicate-annotate-rows-events option 5. How slave IO thread requests Annotate_rows events 6. How master executes the request 7. How slave SQL thread processes Annotate_rows events 8. General remarks 1. Annotate_rows event number ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To avoid possible event numbers conflict with MySQL/Sun, we leave a gap between the last MySQL event number and the Annotate_rows event number: enum Log_event_type { ... INCIDENT_EVENT= 26, // New MySQL event numbers are to be added here MYSQL_EVENTS_END, MARIA_EVENTS_BEGIN= 51, // New Maria event numbers start from here ANNOTATE_ROWS_EVENT= 51, ENUM_END_EVENT }; together with the corresponding extension of 'post_header_len' array in the Format description event. (This extension does not affect the compatibility of the binary log). Here is how Format description event looks like with this extension: ************************ FORMAT_DESCRIPTION_EVENT ************************ 00000004 | A1 A0 2C 4B | time_when = 1261215905 00000008 | 0F | event_type = 15 00000009 | 64 00 00 00 | server_id = 100 0000000D | 7F 00 00 00 | event_len = 127 00000011 | 83 00 00 00 | log_pos = 00000083 00000015 | 01 00 | flags = LOG_EVENT_BINLOG_IN_USE_F ------------------------ 00000017 | 04 00 | binlog_ver = 4 00000019 | 35 2E 32 2E | server_ver = 5.2.0-MariaDB-alpha-debug-log ..... ... 0000004B | A1 A0 2C 4B | time_created = 1261215905 0000004F | 13 | common_header_len = 19 ------------------------ post_header_len ------------------------ 00000050 | 38 | 56 - START_EVENT_V3 [1] ..... ... 00000069 | 02 | 2 - INCIDENT_EVENT [26] 0000006A | 00 | 0 - RESERVED [27] ..... ... 00000081 | 00 | 0 - RESERVED [50] 00000082 | 00 | 0 - ANNOTATE_ROWS_EVENT [51] ************************ 2. Outline of Annotate_rows event behavior ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Each Annotate_rows_log_event object has two private members describing the corresponding query: char *m_query_txt; uint m_query_len; When the object is created for writing to a binary log, this query is taken from 'thd' (for short, below we omit the 'Annotate_rows_log_event::' prefix as well as other implementation details): Annotate_rows_log_event(THD *thd) { m_query_txt = thd->query(); m_query_len = thd->query_length(); } When the object is read from a binary log, the query is taken from the buffer containing the binary log representation of the event (this buffer is allocated in Log_event object from which all Log events are derived): Annotate_rows_log_event(char *buf, uint event_len, Format_description_log_event *desc) { m_query_len = event_len - desc->common_header_len; m_query_txt = buf + desc->common_header_len; } The events are written to the binary log by the Log_event::write() member which calls virtual write_data_header() and write_data_body() members ("data header" and "post header" are synonym in replication terminology). In our case, data header is empty and data body is just the query: bool write_data_body(IO_CACHE *file) { return my_b_safe_write(file, (uchar*) m_query_txt, m_query_len); } Printing the event is just printing the query: void Annotate_rows_log_event::print(FILE *file, PRINT_EVENT_INFO *pinfo) { my_b_printf(&pinfo->head_cache, "\tQuery: `%s`\n", m_query_txt); } 3. How Master writes Annotate_rows events to the binary log ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The event is written to the binary log just before the group of Table_map events which precede corresponding Rows events (one query may generate several Table map events in the binary log, but the corresponding Annotate_rows event must be written only once before the first Table map event; hence the boolean variable 'with_annotate' below): int write_locked_table_maps(THD *thd) { ... bool with_annotate= thd->variables.binlog_annotate_rows_events; ... for (uint i= 0; i < ... <number of tables> ...; ++i) { ... thd->binlog_write_table_map(table, ..., with_annotate); with_annotate= 0; // write Annotate_event not more than once ... } ... } int THD::binlog_write_table_map(TABLE *table, ..., bool with_annotate) { ... Table_map_log_event the_event(...); ... if (with_annotate) { Annotate_rows_log_event anno(this); mysql_bin_log.write(&anno); } mysql_bin_log.write(&the_event); ... } 4. How slave treats replicate-annotate-rows-events option ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The replicate-annotate-rows-events option is treated just as the session value of the binlog_annotate_rows_events variable for the slave IO and SQL threads. This setting is done during initialization of these threads: pthread_handler_t handle_slave_io(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_IO); ... } pthread_handler_t handle_slave_sql(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_SQL); ... } int init_slave_thread(THD* thd, SLAVE_THD_TYPE thd_type) { ... thd->variables.binlog_annotate_rows_events= opt_replicate_annotate_rows_events; ... } 5. How slave IO thread requests Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If the replicate-annotate-rows-events option is not set on a slave, there is no need for master to send Annotate_rows events to this slave. The slave (or mysqlbinlog in remote case), before requesting binlog dump via the COM_BINLOG_DUMP command, informs the master whether it should send these events by executing the newly added COM_BINLOG_DUMP_OPTIONS_EXT server command: case COM_BINLOG_DUMP_OPTIONS_EXT: thd->binlog_dump_flags_ext= packet[0]; my_ok(thd); break; Note. We add this new command and don't use COM_BINLOG_DUMP to avoid possible conflicts with MySQL/Sun. 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ case COM_BINLOG_DUMP: { ... flags= uint2korr(packet + 4); ... mysql_binlog_send(thd, ..., flags); ... } void mysql_binlog_send(THD* thd, ..., ushort flags) { ... Log_event::read_log_event(&log, packet, ...); ... if ((*packet)[EVENT_TYPE_OFFSET + 1] != ANNOTATE_ROWS_EVENT || flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT) { my_net_write(net, packet->ptr(), packet->length()); } ... } 7. How slave SQL thread processes Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The slave processes each recieved event by "applying" it, i.e. by calling the Log_event::apply_event() function which in turn calls the virtual do_apply_event() member specific for each type of the event. int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev = next_event(rli); ... apply_event_and_update_pos(ev, ...); if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } int apply_event_and_update_pos(Log_event *ev, ...) { ... ev->apply_event(...); ... } int Log_event::apply_event(...) { return do_apply_event(...); } What does it mean to "apply" an Annotate_rows event? It means to set current thd query to that of the described by the event, i.e. to the query which caused the subsequent Rows events (see "How Master writes Annotate_rows events to the binary log" to follow what happens further when the subsequent Rows events are applied): int Annotate_rows_log_event::do_apply_event(...) { thd->set_query(m_query_txt, m_query_len); } NOTE. I am not sure, but possibly current values of thd->query and thd->query_length should be saved before calling set_query() and to be restored on the Annotate_rows_log_event object deletion. Is it really needed ? After calling this do_apply_event() function we may not delete the Annotate_rows_log_event object immediatedly (see exec_relay_log_event() above) because thd->query now points to the string inside this object. We may keep the pointer to this object in the Relay_log_info: class Relay_log_info { public: ... void set_annotate_event(Annotate_rows_log_event*); Annotate_rows_log_event* get_annotate_event(); void free_annotate_event(); ... private: Annotate_rows_log_event* m_annotate_event; }; The saved Annotate_rows object should be deleted when all corresponding Rows events will be processed: int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev= next_event(rli); ... apply_event_and_update_pos(ev, ...); if (rli->get_annotate_event() && is_last_rows_event(ev)) rli->free_annotate_event(); else if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) rli->set_annotate_event((Annotate_rows_log_event*) ev); else if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } where bool is_last_rows_event(Log_event* ev) { Log_event_type type= ev->get_type_code(); if (IS_ROWS_EVENT_TYPE(type)) { Rows_log_event* rows= (Rows_log_event*)ev; return rows->get_flags(Rows_log_event::STMT_END_F); } return 0; } #define IS_ROWS_EVENT_TYPE(type) ((type) == WRITE_ROWS_EVENT || \ (type) == UPDATE_ROWS_EVENT || \ (type) == DELETE_ROWS_EVENT) 8. General remarks ~~~~~~~~~~~~~~~~~~ Kristian noticed that introducing new log event type should be coordinated somehow with MySQL/Sun: Kristian: The numeric code for this event must be assigned carefully. It should be coordinated with MySQL/Sun, otherwise we can get into a situation where MySQL uses the same numeric code for one event that MariaDB uses for ANNOTATE_ROWS_EVENT, which would make merging the two impossible. Alex: I reserved about 20 numbers not to have possible conflicts with MySQL. Kristian: Still, I think it would be appropriate to send a polite email to internals(a)lists.mysql.com about this and suggesting to reserve the event number. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Guest): Store in binlog text of statements that caused RBR events (47)
by worklog-noreply＠askmonty.org 05 Feb '10

05 Feb '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Store in binlog text of statements that caused RBR events CREATION DATE..: Sat, 15 Aug 2009, 23:48 SUPERVISOR.....: Monty IMPLEMENTOR....: COPIES TO......: Knielsen CATEGORY.......: Server-Sprint TASK ID........: 47 (http://askmonty.org/worklog/?tid=47) VERSION........: Server-9.x STATUS.........: In-Progress PRIORITY.......: 60 WORKED HOURS...: 20 ESTIMATE.......: 35 (hours remain) ORIG. ESTIMATE.: 35 PROGRESS NOTES: -=-=(Guest - Fri, 05 Feb 2010, 13:40)=-=- Category updated. --- /tmp/wklog.47.old.9197 2010-02-05 13:40:36.000000000 +0200 +++ /tmp/wklog.47.new.9197 2010-02-05 13:40:36.000000000 +0200 @@ -1 +1 @@ -Server-RawIdeaBin +Server-Sprint -=-=(Guest - Fri, 05 Feb 2010, 13:40)=-=- Status updated. --- /tmp/wklog.47.old.9197 2010-02-05 13:40:36.000000000 +0200 +++ /tmp/wklog.47.new.9197 2010-02-05 13:40:36.000000000 +0200 @@ -1 +1 @@ -Un-Assigned +In-Progress -=-=(Alexi - Thu, 04 Feb 2010, 09:54)=-=- Low Level Design modified. --- /tmp/wklog.47.old.16174 2010-02-04 09:54:13.000000000 +0200 +++ /tmp/wklog.47.new.16174 2010-02-04 09:54:13.000000000 +0200 @@ -171,35 +171,20 @@ 5. How slave IO thread requests Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -When requesting an event, the slave should inform the master whether -it should send Annotate_rows events or not. To that end we add a new -BINLOG_SEND_ANNOTATE_ROWS_EVENT flag used when requesting an event: +If the replicate-annotate-rows-events option is not set on a slave, there +is no need for master to send Annotate_rows events to this slave. The slave +(or mysqlbinlog in remote case), before requesting binlog dump via the +COM_BINLOG_DUMP command, informs the master whether it should send these +events by executing the newly added COM_BINLOG_DUMP_OPTIONS_EXT server +command: + + case COM_BINLOG_DUMP_OPTIONS_EXT: + thd->binlog_dump_flags_ext= packet[0]; + my_ok(thd); + break; - #define BINLOG_DUMP_NON_BLOCK 1 - #define BINLOG_SEND_ANNOTATE_ROWS_EVENT 2 - - pthread_handler_t handle_slave_io(void *arg) - { ... - request_dump(mysql, ...); - ... - } - - int request_dump(MYSQL* mysql, ...) - { ... - if (opt_log_slave_updates && - mi->io_thd->variables.binlog_annotate_rows_events) - binlog_flags|= BINLOG_SEND_ANNOTATE_ROWS_EVENT; - ... - int2store(buf + 4, binlog_flags); - ... - simple_command(mysql, COM_BINLOG_DUMP, buf, ...); - ... - } - -NOTE. mysqlbinlog, when remotely requesting BINLOG_DUMP by calling the -simple_command() function, should also use this flag if it wants (in case -of the --print-annotate-rows-events option set) to recieve Annotate_rows -events. +Note. We add this new command and don't use COM_BINLOG_DUMP to avoid possible +conflicts with MySQL/Sun. 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -338,10 +323,4 @@ to internals(a)lists.mysql.com about this and suggesting to reserve the event number. -Also we should notice the introduction of the BINLOG_SEND_ANNOTATE_ROWS_EVENT -flag taking into account that MySQL/Sun may also introduce a flag with the -same value to be used in the request_dump-mysql_binlog_send interface. -But this is mainly the question of merging: if a conflict concerning this -flag occur, we may simply change the BINLOG_SEND_ANNOTATE_ROWS_EVENT value -(this does not require additional changes in the code). -=-=(Alexi - Sun, 20 Dec 2009, 16:00)=-=- Low Level Design modified. --- /tmp/wklog.47.old.19667 2009-12-20 14:00:56.000000000 +0000 +++ /tmp/wklog.47.new.19667 2009-12-20 14:00:56.000000000 +0000 @@ -196,6 +196,11 @@ ... } +NOTE. mysqlbinlog, when remotely requesting BINLOG_DUMP by calling the +simple_command() function, should also use this flag if it wants (in case +of the --print-annotate-rows-events option set) to recieve Annotate_rows +events. + 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -212,8 +217,7 @@ Log_event::read_log_event(&log, packet, ...); ... if ((*packet)[EVENT_TYPE_OFFSET + 1] != ANNOTATE_ROWS_EVENT || - flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT || - thd->server_id == 0 /* slave == mysqlbinlog */ ) + flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT) { my_net_write(net, packet->ptr(), packet->length()); } -=-=(Alexi - Sun, 20 Dec 2009, 13:14)=-=- Low Level Design modified. --- /tmp/wklog.47.old.11350 2009-12-20 13:14:04.000000000 +0200 +++ /tmp/wklog.47.new.11350 2009-12-20 13:14:04.000000000 +0200 @@ -282,23 +282,18 @@ Annotate_rows_log_event* m_annotate_event; }; -When the saved Annotate_rows object may be deleted? When all corresponding -Rows events will be processed, i.e. before processing the first non-Rows -event (note that Annotate_rows object resides in the binary log *after* -the (possible) 'BEGIN' Query event which accompanies the rows events; note -also that this deletion is adjusted with the case when some or all -corresponding Rows events are filtered out by replicate filter rules): +The saved Annotate_rows object should be deleted when all corresponding +Rows events will be processed: int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev= next_event(rli); ... - if (rli->get_annotate_event() && !IS_RBR_EVENT_TYPE(ev->get_type_code())) - rli->free_annotate_event(); - apply_event_and_update_pos(ev, ...); - if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) + if (rli->get_annotate_event() && is_last_rows_event(ev)) + rli->free_annotate_event(); + else if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) rli->set_annotate_event((Annotate_rows_log_event*) ev); else if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; @@ -307,10 +302,21 @@ where - #define IS_RBR_EVENT_TYPE(type) ( (type) == TABLE_MAP_EVENT || \ - (type) == WRITE_ROWS_EVENT || \ + bool is_last_rows_event(Log_event* ev) + { + Log_event_type type= ev->get_type_code(); + if (IS_ROWS_EVENT_TYPE(type)) + { + Rows_log_event* rows= (Rows_log_event*)ev; + return rows->get_flags(Rows_log_event::STMT_END_F); + } + + return 0; + } + + #define IS_ROWS_EVENT_TYPE(type) ((type) == WRITE_ROWS_EVENT || \ (type) == UPDATE_ROWS_EVENT || \ - (type) == DELETE_ROWS_EVENT ) + (type) == DELETE_ROWS_EVENT) 8. General remarks ~~~~~~~~~~~~~~~~~~ -=-=(Alexi - Sun, 20 Dec 2009, 09:29)=-=- Low Level Design modified. --- /tmp/wklog.47.old.32726 2009-12-20 07:29:56.000000000 +0000 +++ /tmp/wklog.47.new.32726 2009-12-20 07:29:56.000000000 +0000 @@ -56,7 +56,7 @@ 0000006A | 00 | 0 - RESERVED [27] ..... ... 00000081 | 00 | 0 - RESERVED [50] - 00000082 | 00 | 0 - ANNOTATE_RBR_EVENT [51] + 00000082 | 00 | 0 - ANNOTATE_ROWS_EVENT [51] ************************ 2. Outline of Annotate_rows event behavior -=-=(Alexi - Sat, 19 Dec 2009, 16:10)=-=- Low Level Design modified. --- /tmp/wklog.47.old.16051 2009-12-19 16:10:48.000000000 +0200 +++ /tmp/wklog.47.new.16051 2009-12-19 16:10:48.000000000 +0200 @@ -253,7 +253,7 @@ thd query to that of the described by the event, i.e. to the query which caused the subsequent Rows events (see "How Master writes Annotate_rows events to the binary log" to follow what happens further when the subsequent -Rows events is applied): +Rows events are applied): int Annotate_rows_log_event::do_apply_event(...) { -=-=(Alexi - Sat, 19 Dec 2009, 16:02)=-=- High-Level Specification modified. --- /tmp/wklog.47.old.15695 2009-12-19 16:02:33.000000000 +0200 +++ /tmp/wklog.47.new.15695 2009-12-19 16:02:33.000000000 +0200 @@ -12,7 +12,7 @@ post-header and contains the query text in its data part. Example: ************************ - ANNOTATE_RBR_EVENT + ANNOTATE_ROWS_EVENT ************************ 00000220 | B6 A0 2C 4B | time_when = 1261215926 00000224 | 33 | event_type = 51 -=-=(Alexi - Sat, 19 Dec 2009, 15:58)=-=- Low Level Design modified. --- /tmp/wklog.47.old.15437 2009-12-19 13:58:12.000000000 +0000 +++ /tmp/wklog.47.new.15437 2009-12-19 13:58:12.000000000 +0000 @@ -1 +1,337 @@ +Content +~~~~~~~ + 1. Annotate_rows event number + 2. Outline of Annotate_rows event behavior + 3. How Master writes Annotate_rows events to the binary log + 4. How slave treats replicate-annotate-rows-events option + 5. How slave IO thread requests Annotate_rows events + 6. How master executes the request + 7. How slave SQL thread processes Annotate_rows events + 8. General remarks + +1. Annotate_rows event number +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +To avoid possible event numbers conflict with MySQL/Sun, we leave a gap +between the last MySQL event number and the Annotate_rows event number: + + enum Log_event_type + { ... + INCIDENT_EVENT= 26, + // New MySQL event numbers are to be added here + MYSQL_EVENTS_END, + + MARIA_EVENTS_BEGIN= 51, + // New Maria event numbers start from here + ANNOTATE_ROWS_EVENT= 51, + + ENUM_END_EVENT + }; + +together with the corresponding extension of 'post_header_len' array in the +Format description event. (This extension does not affect the compatibility +of the binary log). Here is how Format description event looks like with +this extension: + + ************************ + FORMAT_DESCRIPTION_EVENT + ************************ + 00000004 | A1 A0 2C 4B | time_when = 1261215905 + 00000008 | 0F | event_type = 15 + 00000009 | 64 00 00 00 | server_id = 100 + 0000000D | 7F 00 00 00 | event_len = 127 + 00000011 | 83 00 00 00 | log_pos = 00000083 + 00000015 | 01 00 | flags = LOG_EVENT_BINLOG_IN_USE_F + ------------------------ + 00000017 | 04 00 | binlog_ver = 4 + 00000019 | 35 2E 32 2E | server_ver = 5.2.0-MariaDB-alpha-debug-log + ..... ... + 0000004B | A1 A0 2C 4B | time_created = 1261215905 + 0000004F | 13 | common_header_len = 19 + ------------------------ + post_header_len + ------------------------ + 00000050 | 38 | 56 - START_EVENT_V3 [1] + ..... ... + 00000069 | 02 | 2 - INCIDENT_EVENT [26] + 0000006A | 00 | 0 - RESERVED [27] + ..... ... + 00000081 | 00 | 0 - RESERVED [50] + 00000082 | 00 | 0 - ANNOTATE_RBR_EVENT [51] + ************************ + +2. Outline of Annotate_rows event behavior +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Each Annotate_rows_log_event object has two private members describing the +corresponding query: + + char *m_query_txt; + uint m_query_len; + +When the object is created for writing to a binary log, this query is taken +from 'thd' (for short, below we omit the 'Annotate_rows_log_event::' prefix +as well as other implementation details): + + Annotate_rows_log_event(THD *thd) + { + m_query_txt = thd->query(); + m_query_len = thd->query_length(); + } + +When the object is read from a binary log, the query is taken from the buffer +containing the binary log representation of the event (this buffer is allocated +in Log_event object from which all Log events are derived): + + Annotate_rows_log_event(char *buf, uint event_len, + Format_description_log_event *desc) + { + m_query_len = event_len - desc->common_header_len; + m_query_txt = buf + desc->common_header_len; + } + +The events are written to the binary log by the Log_event::write() member +which calls virtual write_data_header() and write_data_body() members +("data header" and "post header" are synonym in replication terminology). +In our case, data header is empty and data body is just the query: + + bool write_data_body(IO_CACHE *file) + { + return my_b_safe_write(file, (uchar*) m_query_txt, m_query_len); + } + +Printing the event is just printing the query: + + void Annotate_rows_log_event::print(FILE *file, PRINT_EVENT_INFO *pinfo) + { + my_b_printf(&pinfo->head_cache, "\tQuery: `%s`\n", m_query_txt); + } + +3. How Master writes Annotate_rows events to the binary log +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The event is written to the binary log just before the group of Table_map +events which precede corresponding Rows events (one query may generate +several Table map events in the binary log, but the corresponding +Annotate_rows event must be written only once before the first Table map +event; hence the boolean variable 'with_annotate' below): + + int write_locked_table_maps(THD *thd) + { ... + bool with_annotate= thd->variables.binlog_annotate_rows_events; + ... + for (uint i= 0; i < ... <number of tables> ...; ++i) + { ... + thd->binlog_write_table_map(table, ..., with_annotate); + with_annotate= 0; // write Annotate_event not more than once + ... + } + ... + } + + int THD::binlog_write_table_map(TABLE *table, ..., bool with_annotate) + { ... + Table_map_log_event the_event(...); + ... + if (with_annotate) + { + Annotate_rows_log_event anno(this); + mysql_bin_log.write(&anno); + } + + mysql_bin_log.write(&the_event); + ... + } + +4. How slave treats replicate-annotate-rows-events option +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The replicate-annotate-rows-events option is treated just as the session +value of the binlog_annotate_rows_events variable for the slave IO and +SQL threads. This setting is done during initialization of these threads: + + pthread_handler_t handle_slave_io(void *arg) + { + THD *thd= new THD; + ... + init_slave_thread(thd, SLAVE_THD_IO); + ... + } + + pthread_handler_t handle_slave_sql(void *arg) + { + THD *thd= new THD; + ... + init_slave_thread(thd, SLAVE_THD_SQL); + ... + } + + int init_slave_thread(THD* thd, SLAVE_THD_TYPE thd_type) + { ... + thd->variables.binlog_annotate_rows_events= + opt_replicate_annotate_rows_events; + ... + } + +5. How slave IO thread requests Annotate_rows events +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +When requesting an event, the slave should inform the master whether +it should send Annotate_rows events or not. To that end we add a new +BINLOG_SEND_ANNOTATE_ROWS_EVENT flag used when requesting an event: + + #define BINLOG_DUMP_NON_BLOCK 1 + #define BINLOG_SEND_ANNOTATE_ROWS_EVENT 2 + + pthread_handler_t handle_slave_io(void *arg) + { ... + request_dump(mysql, ...); + ... + } + + int request_dump(MYSQL* mysql, ...) + { ... + if (opt_log_slave_updates && + mi->io_thd->variables.binlog_annotate_rows_events) + binlog_flags|= BINLOG_SEND_ANNOTATE_ROWS_EVENT; + ... + int2store(buf + 4, binlog_flags); + ... + simple_command(mysql, COM_BINLOG_DUMP, buf, ...); + ... + } + +6. How master executes the request +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + case COM_BINLOG_DUMP: + { ... + flags= uint2korr(packet + 4); + ... + mysql_binlog_send(thd, ..., flags); + ... + } + + void mysql_binlog_send(THD* thd, ..., ushort flags) + { ... + Log_event::read_log_event(&log, packet, ...); + ... + if ((*packet)[EVENT_TYPE_OFFSET + 1] != ANNOTATE_ROWS_EVENT || + flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT || + thd->server_id == 0 /* slave == mysqlbinlog */ ) + { + my_net_write(net, packet->ptr(), packet->length()); + } + ... + } + +7. How slave SQL thread processes Annotate_rows events +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The slave processes each recieved event by "applying" it, i.e. by +calling the Log_event::apply_event() function which in turn calls +the virtual do_apply_event() member specific for each type of the +event. + + int exec_relay_log_event(THD* thd, Relay_log_info* rli) + { ... + Log_event *ev = next_event(rli); + ... + apply_event_and_update_pos(ev, ...); + + if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) + delete ev; + ... + } + + int apply_event_and_update_pos(Log_event *ev, ...) + { ... + ev->apply_event(...); + ... + } + + int Log_event::apply_event(...) + { + return do_apply_event(...); + } + +What does it mean to "apply" an Annotate_rows event? It means to set current +thd query to that of the described by the event, i.e. to the query which +caused the subsequent Rows events (see "How Master writes Annotate_rows +events to the binary log" to follow what happens further when the subsequent +Rows events is applied): + + int Annotate_rows_log_event::do_apply_event(...) + { + thd->set_query(m_query_txt, m_query_len); + } + +NOTE. I am not sure, but possibly current values of thd->query and +thd->query_length should be saved before calling set_query() and to be +restored on the Annotate_rows_log_event object deletion. +Is it really needed ? + +After calling this do_apply_event() function we may not delete the +Annotate_rows_log_event object immediatedly (see exec_relay_log_event() +above) because thd->query now points to the string inside this object. +We may keep the pointer to this object in the Relay_log_info: + + class Relay_log_info + { + public: + ... + void set_annotate_event(Annotate_rows_log_event*); + Annotate_rows_log_event* get_annotate_event(); + void free_annotate_event(); + ... + private: + Annotate_rows_log_event* m_annotate_event; + }; + +When the saved Annotate_rows object may be deleted? When all corresponding +Rows events will be processed, i.e. before processing the first non-Rows +event (note that Annotate_rows object resides in the binary log *after* +the (possible) 'BEGIN' Query event which accompanies the rows events; note +also that this deletion is adjusted with the case when some or all +corresponding Rows events are filtered out by replicate filter rules): + + int exec_relay_log_event(THD* thd, Relay_log_info* rli) + { ... + Log_event *ev= next_event(rli); + ... + if (rli->get_annotate_event() && !IS_RBR_EVENT_TYPE(ev->get_type_code())) + rli->free_annotate_event(); + + apply_event_and_update_pos(ev, ...); + + if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) + rli->set_annotate_event((Annotate_rows_log_event*) ev); + else if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) + delete ev; + ... + } + +where + + #define IS_RBR_EVENT_TYPE(type) ( (type) == TABLE_MAP_EVENT || \ + (type) == WRITE_ROWS_EVENT || \ + (type) == UPDATE_ROWS_EVENT || \ + (type) == DELETE_ROWS_EVENT ) + +8. General remarks +~~~~~~~~~~~~~~~~~~ +Kristian noticed that introducing new log event type should be coordinated +somehow with MySQL/Sun: + + Kristian: The numeric code for this event must be assigned carefully. + It should be coordinated with MySQL/Sun, otherwise we can get into a + situation where MySQL uses the same numeric code for one event that + MariaDB uses for ANNOTATE_ROWS_EVENT, which would make merging the two + impossible. + Alex: I reserved about 20 numbers not to have possible conflicts + with MySQL. + Kristian: Still, I think it would be appropriate to send a polite email + to internals(a)lists.mysql.com about this and suggesting to reserve the + event number. + +Also we should notice the introduction of the BINLOG_SEND_ANNOTATE_ROWS_EVENT +flag taking into account that MySQL/Sun may also introduce a flag with the +same value to be used in the request_dump-mysql_binlog_send interface. +But this is mainly the question of merging: if a conflict concerning this +flag occur, we may simply change the BINLOG_SEND_ANNOTATE_ROWS_EVENT value +(this does not require additional changes in the code). -=-=(Alexi - Sat, 19 Dec 2009, 15:41)=-=- High-Level Specification modified. --- /tmp/wklog.47.old.14545 2009-12-19 15:41:21.000000000 +0200 +++ /tmp/wklog.47.new.14545 2009-12-19 15:41:21.000000000 +0200 @@ -1,122 +1,107 @@ -First suggestion: - -> I think for this we would actually need a new binlog event type -> (Comment_log_event?). Unless we want to log an empty statement Query_log_event -> containing only a comment (a bit of a hack). - -New server option -~~~~~~~~~~~~~~~~~ - --binlog-annotate-rows-events - -Setting this option makes RBR (rows-) events in the binary log to be -preceded by Annotate rows events (see below). The corresponding -'binlog_annotate_rows_events' system variable is dynamic and has both -global and session values. Default global value is OFF. - -Note. Session values are usefull to make it possible to annotate only - some selected statements: +Content +~~~~~~~ + 1. Annotate_rows_log_event + 2. Server option: --binlog-annotate-rows-events + 3. Server option: --replicate-annotate-rows-events + 4. mysqlbinlog option: --print-annotate-rows-events + 5. mysqlbinlog output + +1. Annotate_rows_log_event [ ANNOTATE_ROWS_EVENT ] +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Describes the query which caused the corresponding rows events. Has empty +post-header and contains the query text in its data part. Example: + + ************************ + ANNOTATE_RBR_EVENT + ************************ + 00000220 | B6 A0 2C 4B | time_when = 1261215926 + 00000224 | 33 | event_type = 51 + 00000225 | 64 00 00 00 | server_id = 100 + 00000229 | 36 00 00 00 | event_len = 54 + 0000022D | 56 02 00 00 | log_pos = 00000256 + 00000231 | 00 00 | flags = <none> + ------------------------ + 00000233 | 49 4E 53 45 | query = "INSERT INTO t1 VALUES (1), (2), (3)" + 00000237 | 52 54 20 49 | + 0000023B | 4E 54 4F 20 | + 0000023F | 74 31 20 56 | + 00000243 | 41 4C 55 45 | + 00000247 | 53 20 28 31 | + 0000024B | 29 2C 20 28 | + 0000024F | 32 29 2C 20 | + 00000253 | 28 33 29 | + ************************ + +In binary log, Annotate_rows event follows the (possible) 'BEGIN' Query event +and precedes the first of Table map events which accompany the corresponding +rows events. (See example in the "mysqlbinlog output" section below.) + +2. Server option: --binlog-annotate-rows-events +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Tells the master to write Annotate_rows events to the binary log. + + * Variable Name: binlog_annotate_rows_events + * Scope: Global & Session + * Access Type: Dynamic + * Data Type: bool + * Default Value: OFF +NOTE. Session values allows to annotate only some selected statements: ... SET SESSION binlog_annotate_rows_events=ON; ... statements to be annotated ... SET SESSION binlog_annotate_rows_events=OFF; ... statements not to be annotated ... -New binlog event type -~~~~~~~~~~~~~~~~~~~~~ - Annotate_rows_log_event [ ANNOTATE_ROWS_EVENT ] - -Describes the query which caused the corresponding rows event. In binary log, -precedes each Table_map_log_event. Contains empty post-header and the query -text in its data part. - -The numeric code for this event must be assigned carefully. It should be -coordinated with MySQL/Sun, otherwise we can get into a situation where MySQL -uses the same numeric code for one event that MariaDB uses for -ANNOTATE_ROWS_EVENT, which would make merging the two impossible. - -Example: - - ... - ************************ - ANNOTATE_ROWS_EVENT [51] - ************************ - 000000C7 | 54 1B 12 4B | time_when = 1259477844 - 000000CB | 33 | event_type = 51 - 000000CC | 64 00 00 00 | server_id = 100 - 000000D0 | 2C 00 00 00 | event_len = 44 - 000000D4 | F3 00 00 00 | log_pos = 000000F3 - 000000D8 | 00 00 | flags = <none> - ------------------------ - 000000DA | 69 6E 73 65 | query = "insert into t1 values (1)" - 000000DE | 72 74 20 69 | - 000000E2 | 6E 74 6F 20 | - 000000E6 | 74 31 20 76 | - 000000EA | 61 6C 75 65 | - 000000EE | 73 20 28 31 | - 000000F2 | 29 | - ************************ - TABLE_MAP_EVENT [19] - ************************ - 000000F3 | 54 1B 12 4B | time_when = 1259477844 - 000000F7 | 13 | event_type = 19 - 000000F8 | 64 00 00 00 | server_id = 100 - 000000FC | 29 00 00 00 | event_len = 41 - 00000100 | 1C 01 00 00 | log_pos = 0000011C - 00000104 | 00 00 | flags = <none> - ------------------------ - ... - ************************ - WRITE_ROWS_EVENT [23] - ************************ - 0000011C | 54 1B 12 4B | time_when = 1259477844 - 00000120 | 17 | event_type = 23 - 00000121 | 64 00 00 00 | server_id = 100 - 00000125 | 22 00 00 00 | event_len = 34 - 00000129 | 3E 01 00 00 | log_pos = 0000013E - 0000012D | 10 00 | flags = LOG_EVENT_UPDATE_TABLE_MAP_VERSION_F - ------------------------ - 0000012F | 0F 00 00 00 | table_id = 15 - ... - -New mysqlbinlog option -~~~~~~~~~~~~~~~~~~~~~~ - --print-annotate-rows-events - -With this option, mysqlbinlog prints the content of Annotate-rows -events (if the binary log does contain them). Without this option -(i.e. by default), mysqlbinlog skips Annotate rows events. - - -mysqlbinlog output -~~~~~~~~~~~~~~~~~~ -Something like this: +3. Server option: --replicate-annotate-rows-events +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Tells the slave to reproduce Annotate_rows events recieved from the master +in its own binary log (sensible only in pair with log-slave-updates option). + + * Variable Name: replicate_annotate_rows_events + * Scope: Global + * Access Type: Read only + * Data Type: bool + * Default Value: OFF + +NOTE. Why do we additionally need this 'replicate' option? Why not to make +the slave to reproduce this events when its binlog-annotate-rows-events +global value is ON? Well, because, for example, we may want to configure +the slave which should reproduce Annotate_rows events but has global +binlog-annotate-rows-events = OFF meaning this to be the default value for +the client threads (see also "How slave treats replicate-annotate-rows-events +option" in LLD part). + +4. mysqlbinlog option: --print-annotate-rows-events +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +With this option, mysqlbinlog prints the content of Annotate_rows events (if +the binary log does contain them). Without this option (i.e. by default), +mysqlbinlog skips Annotate_rows events. +5. mysqlbinlog output +~~~~~~~~~~~~~~~~~~~~~ +With --print-annotate-rows-events, mysqlbinlog outputs Annotate_rows events +in a form like this: ... - # at 199 - # at 243 - # at 284 - #091129 9:57:24 server id 100 end_log_pos 243 Query: `insert into t1 values -(1)` - #091129 9:57:24 server id 100 end_log_pos 284 Table_map: `test`.`t1` mapped -to number 15 - #091129 9:57:24 server id 100 end_log_pos 318 Write_rows: table id 15 + # at 1646 + #091219 12:45:26 server id 100 end_log_pos 1714 Query thread_id=1 +exec_time=0 error_code=0 + SET TIMESTAMP=1261215926/*!*/; + BEGIN + /*!*/; + # at 1714 + # at 1812 + # at 1853 + # at 1894 + # at 1938 + #091219 12:45:26 server id 100 end_log_pos 1812 Query: `DELETE t1, t2 FROM +t1 INNER JOIN t2 INNER JOIN t3 WHERE t1.a=t2.a AND t2.a=t3.a` + #091219 12:45:26 server id 100 end_log_pos 1853 Table_map: `test`.`t1` +mapped to number 16 + #091219 12:45:26 server id 100 end_log_pos 1894 Table_map: `test`.`t2` +mapped to number 17 + #091219 12:45:26 server id 100 end_log_pos 1938 Delete_rows: table id 16 + #091219 12:45:26 server id 100 end_log_pos 1982 Delete_rows: table id 17 flags: STMT_END_F - - BINLOG ' - VBsSSzNkAAAALAAAAPMAAAAAAGluc2VydCBpbnRvIHQxIHZhbHVlcyAoMSk= - VBsSSxNkAAAAKQAAABwBAAAAAA8AAAAAAAAABHRlc3QAAnQxAAEDAAE= - VBsSSxdkAAAAIgAAAD4BAAAQAA8AAAAAAAEAAf/+AQAAAA== - '/*!*/; - ### INSERT INTO test.t1 - ### SET - ### @1=1 /* INT meta=0 nullable=1 is_null=0 */ ... -When master sends Annotate rows events -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -1. Master always sends Annotate_rows events to mysqlbinlog (in - remote case). -2. Master sends Annotate_rows events to a slave only if the slave has - both log-slave-updates and binlog-annotate-rows-events options set. - ------------------------------------------------------------ -=-=(View All Progress Notes, 22 total)=-=- http://askmonty.org/worklog/index.pl?tid=47&nolimit=1 DESCRIPTION: Store in binlog (and show in mysqlbinlog output) texts of statements that caused RBR events This is needed for (list from Monty): - Easier to understand why updates happened - Would make it easier to find out where in application things went wrong (as you can search for exact strings) - Allow one to filter things based on comments in the statement. The cost of this can be that the binlog will be approximately 2x in size (especially insert of big blob's would be a bit painful), so this should be an optional feature. HIGH-LEVEL SPECIFICATION: Content ~~~~~~~ 1. Annotate_rows_log_event 2. Server option: --binlog-annotate-rows-events 3. Server option: --replicate-annotate-rows-events 4. mysqlbinlog option: --print-annotate-rows-events 5. mysqlbinlog output 1. Annotate_rows_log_event [ ANNOTATE_ROWS_EVENT ] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Describes the query which caused the corresponding rows events. Has empty post-header and contains the query text in its data part. Example: ************************ ANNOTATE_ROWS_EVENT ************************ 00000220 | B6 A0 2C 4B | time_when = 1261215926 00000224 | 33 | event_type = 51 00000225 | 64 00 00 00 | server_id = 100 00000229 | 36 00 00 00 | event_len = 54 0000022D | 56 02 00 00 | log_pos = 00000256 00000231 | 00 00 | flags = <none> ------------------------ 00000233 | 49 4E 53 45 | query = "INSERT INTO t1 VALUES (1), (2), (3)" 00000237 | 52 54 20 49 | 0000023B | 4E 54 4F 20 | 0000023F | 74 31 20 56 | 00000243 | 41 4C 55 45 | 00000247 | 53 20 28 31 | 0000024B | 29 2C 20 28 | 0000024F | 32 29 2C 20 | 00000253 | 28 33 29 | ************************ In binary log, Annotate_rows event follows the (possible) 'BEGIN' Query event and precedes the first of Table map events which accompany the corresponding rows events. (See example in the "mysqlbinlog output" section below.) 2. Server option: --binlog-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the master to write Annotate_rows events to the binary log. * Variable Name: binlog_annotate_rows_events * Scope: Global & Session * Access Type: Dynamic * Data Type: bool * Default Value: OFF NOTE. Session values allows to annotate only some selected statements: ... SET SESSION binlog_annotate_rows_events=ON; ... statements to be annotated ... SET SESSION binlog_annotate_rows_events=OFF; ... statements not to be annotated ... 3. Server option: --replicate-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the slave to reproduce Annotate_rows events recieved from the master in its own binary log (sensible only in pair with log-slave-updates option). * Variable Name: replicate_annotate_rows_events * Scope: Global * Access Type: Read only * Data Type: bool * Default Value: OFF NOTE. Why do we additionally need this 'replicate' option? Why not to make the slave to reproduce this events when its binlog-annotate-rows-events global value is ON? Well, because, for example, we may want to configure the slave which should reproduce Annotate_rows events but has global binlog-annotate-rows-events = OFF meaning this to be the default value for the client threads (see also "How slave treats replicate-annotate-rows-events option" in LLD part). 4. mysqlbinlog option: --print-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ With this option, mysqlbinlog prints the content of Annotate_rows events (if the binary log does contain them). Without this option (i.e. by default), mysqlbinlog skips Annotate_rows events. 5. mysqlbinlog output ~~~~~~~~~~~~~~~~~~~~~ With --print-annotate-rows-events, mysqlbinlog outputs Annotate_rows events in a form like this: ... # at 1646 #091219 12:45:26 server id 100 end_log_pos 1714 Query thread_id=1 exec_time=0 error_code=0 SET TIMESTAMP=1261215926/*!*/; BEGIN /*!*/; # at 1714 # at 1812 # at 1853 # at 1894 # at 1938 #091219 12:45:26 server id 100 end_log_pos 1812 Query: `DELETE t1, t2 FROM t1 INNER JOIN t2 INNER JOIN t3 WHERE t1.a=t2.a AND t2.a=t3.a` #091219 12:45:26 server id 100 end_log_pos 1853 Table_map: `test`.`t1` mapped to number 16 #091219 12:45:26 server id 100 end_log_pos 1894 Table_map: `test`.`t2` mapped to number 17 #091219 12:45:26 server id 100 end_log_pos 1938 Delete_rows: table id 16 #091219 12:45:26 server id 100 end_log_pos 1982 Delete_rows: table id 17 flags: STMT_END_F ... LOW-LEVEL DESIGN: Content ~~~~~~~ 1. Annotate_rows event number 2. Outline of Annotate_rows event behavior 3. How Master writes Annotate_rows events to the binary log 4. How slave treats replicate-annotate-rows-events option 5. How slave IO thread requests Annotate_rows events 6. How master executes the request 7. How slave SQL thread processes Annotate_rows events 8. General remarks 1. Annotate_rows event number ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To avoid possible event numbers conflict with MySQL/Sun, we leave a gap between the last MySQL event number and the Annotate_rows event number: enum Log_event_type { ... INCIDENT_EVENT= 26, // New MySQL event numbers are to be added here MYSQL_EVENTS_END, MARIA_EVENTS_BEGIN= 51, // New Maria event numbers start from here ANNOTATE_ROWS_EVENT= 51, ENUM_END_EVENT }; together with the corresponding extension of 'post_header_len' array in the Format description event. (This extension does not affect the compatibility of the binary log). Here is how Format description event looks like with this extension: ************************ FORMAT_DESCRIPTION_EVENT ************************ 00000004 | A1 A0 2C 4B | time_when = 1261215905 00000008 | 0F | event_type = 15 00000009 | 64 00 00 00 | server_id = 100 0000000D | 7F 00 00 00 | event_len = 127 00000011 | 83 00 00 00 | log_pos = 00000083 00000015 | 01 00 | flags = LOG_EVENT_BINLOG_IN_USE_F ------------------------ 00000017 | 04 00 | binlog_ver = 4 00000019 | 35 2E 32 2E | server_ver = 5.2.0-MariaDB-alpha-debug-log ..... ... 0000004B | A1 A0 2C 4B | time_created = 1261215905 0000004F | 13 | common_header_len = 19 ------------------------ post_header_len ------------------------ 00000050 | 38 | 56 - START_EVENT_V3 [1] ..... ... 00000069 | 02 | 2 - INCIDENT_EVENT [26] 0000006A | 00 | 0 - RESERVED [27] ..... ... 00000081 | 00 | 0 - RESERVED [50] 00000082 | 00 | 0 - ANNOTATE_ROWS_EVENT [51] ************************ 2. Outline of Annotate_rows event behavior ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Each Annotate_rows_log_event object has two private members describing the corresponding query: char *m_query_txt; uint m_query_len; When the object is created for writing to a binary log, this query is taken from 'thd' (for short, below we omit the 'Annotate_rows_log_event::' prefix as well as other implementation details): Annotate_rows_log_event(THD *thd) { m_query_txt = thd->query(); m_query_len = thd->query_length(); } When the object is read from a binary log, the query is taken from the buffer containing the binary log representation of the event (this buffer is allocated in Log_event object from which all Log events are derived): Annotate_rows_log_event(char *buf, uint event_len, Format_description_log_event *desc) { m_query_len = event_len - desc->common_header_len; m_query_txt = buf + desc->common_header_len; } The events are written to the binary log by the Log_event::write() member which calls virtual write_data_header() and write_data_body() members ("data header" and "post header" are synonym in replication terminology). In our case, data header is empty and data body is just the query: bool write_data_body(IO_CACHE *file) { return my_b_safe_write(file, (uchar*) m_query_txt, m_query_len); } Printing the event is just printing the query: void Annotate_rows_log_event::print(FILE *file, PRINT_EVENT_INFO *pinfo) { my_b_printf(&pinfo->head_cache, "\tQuery: `%s`\n", m_query_txt); } 3. How Master writes Annotate_rows events to the binary log ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The event is written to the binary log just before the group of Table_map events which precede corresponding Rows events (one query may generate several Table map events in the binary log, but the corresponding Annotate_rows event must be written only once before the first Table map event; hence the boolean variable 'with_annotate' below): int write_locked_table_maps(THD *thd) { ... bool with_annotate= thd->variables.binlog_annotate_rows_events; ... for (uint i= 0; i < ... <number of tables> ...; ++i) { ... thd->binlog_write_table_map(table, ..., with_annotate); with_annotate= 0; // write Annotate_event not more than once ... } ... } int THD::binlog_write_table_map(TABLE *table, ..., bool with_annotate) { ... Table_map_log_event the_event(...); ... if (with_annotate) { Annotate_rows_log_event anno(this); mysql_bin_log.write(&anno); } mysql_bin_log.write(&the_event); ... } 4. How slave treats replicate-annotate-rows-events option ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The replicate-annotate-rows-events option is treated just as the session value of the binlog_annotate_rows_events variable for the slave IO and SQL threads. This setting is done during initialization of these threads: pthread_handler_t handle_slave_io(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_IO); ... } pthread_handler_t handle_slave_sql(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_SQL); ... } int init_slave_thread(THD* thd, SLAVE_THD_TYPE thd_type) { ... thd->variables.binlog_annotate_rows_events= opt_replicate_annotate_rows_events; ... } 5. How slave IO thread requests Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If the replicate-annotate-rows-events option is not set on a slave, there is no need for master to send Annotate_rows events to this slave. The slave (or mysqlbinlog in remote case), before requesting binlog dump via the COM_BINLOG_DUMP command, informs the master whether it should send these events by executing the newly added COM_BINLOG_DUMP_OPTIONS_EXT server command: case COM_BINLOG_DUMP_OPTIONS_EXT: thd->binlog_dump_flags_ext= packet[0]; my_ok(thd); break; Note. We add this new command and don't use COM_BINLOG_DUMP to avoid possible conflicts with MySQL/Sun. 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ case COM_BINLOG_DUMP: { ... flags= uint2korr(packet + 4); ... mysql_binlog_send(thd, ..., flags); ... } void mysql_binlog_send(THD* thd, ..., ushort flags) { ... Log_event::read_log_event(&log, packet, ...); ... if ((*packet)[EVENT_TYPE_OFFSET + 1] != ANNOTATE_ROWS_EVENT || flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT) { my_net_write(net, packet->ptr(), packet->length()); } ... } 7. How slave SQL thread processes Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The slave processes each recieved event by "applying" it, i.e. by calling the Log_event::apply_event() function which in turn calls the virtual do_apply_event() member specific for each type of the event. int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev = next_event(rli); ... apply_event_and_update_pos(ev, ...); if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } int apply_event_and_update_pos(Log_event *ev, ...) { ... ev->apply_event(...); ... } int Log_event::apply_event(...) { return do_apply_event(...); } What does it mean to "apply" an Annotate_rows event? It means to set current thd query to that of the described by the event, i.e. to the query which caused the subsequent Rows events (see "How Master writes Annotate_rows events to the binary log" to follow what happens further when the subsequent Rows events are applied): int Annotate_rows_log_event::do_apply_event(...) { thd->set_query(m_query_txt, m_query_len); } NOTE. I am not sure, but possibly current values of thd->query and thd->query_length should be saved before calling set_query() and to be restored on the Annotate_rows_log_event object deletion. Is it really needed ? After calling this do_apply_event() function we may not delete the Annotate_rows_log_event object immediatedly (see exec_relay_log_event() above) because thd->query now points to the string inside this object. We may keep the pointer to this object in the Relay_log_info: class Relay_log_info { public: ... void set_annotate_event(Annotate_rows_log_event*); Annotate_rows_log_event* get_annotate_event(); void free_annotate_event(); ... private: Annotate_rows_log_event* m_annotate_event; }; The saved Annotate_rows object should be deleted when all corresponding Rows events will be processed: int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev= next_event(rli); ... apply_event_and_update_pos(ev, ...); if (rli->get_annotate_event() && is_last_rows_event(ev)) rli->free_annotate_event(); else if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) rli->set_annotate_event((Annotate_rows_log_event*) ev); else if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } where bool is_last_rows_event(Log_event* ev) { Log_event_type type= ev->get_type_code(); if (IS_ROWS_EVENT_TYPE(type)) { Rows_log_event* rows= (Rows_log_event*)ev; return rows->get_flags(Rows_log_event::STMT_END_F); } return 0; } #define IS_ROWS_EVENT_TYPE(type) ((type) == WRITE_ROWS_EVENT || \ (type) == UPDATE_ROWS_EVENT || \ (type) == DELETE_ROWS_EVENT) 8. General remarks ~~~~~~~~~~~~~~~~~~ Kristian noticed that introducing new log event type should be coordinated somehow with MySQL/Sun: Kristian: The numeric code for this event must be assigned carefully. It should be coordinated with MySQL/Sun, otherwise we can get into a situation where MySQL uses the same numeric code for one event that MariaDB uses for ANNOTATE_ROWS_EVENT, which would make merging the two impossible. Alex: I reserved about 20 numbers not to have possible conflicts with MySQL. Kristian: Still, I think it would be appropriate to send a polite email to internals(a)lists.mysql.com about this and suggesting to reserve the event number. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Guest): Store in binlog text of statements that caused RBR events (47)
by worklog-noreply＠askmonty.org 05 Feb '10

05 Feb '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Store in binlog text of statements that caused RBR events CREATION DATE..: Sat, 15 Aug 2009, 23:48 SUPERVISOR.....: Monty IMPLEMENTOR....: COPIES TO......: Knielsen CATEGORY.......: Server-Sprint TASK ID........: 47 (http://askmonty.org/worklog/?tid=47) VERSION........: Server-9.x STATUS.........: In-Progress PRIORITY.......: 60 WORKED HOURS...: 20 ESTIMATE.......: 35 (hours remain) ORIG. ESTIMATE.: 35 PROGRESS NOTES: -=-=(Guest - Fri, 05 Feb 2010, 13:40)=-=- Category updated. --- /tmp/wklog.47.old.9197 2010-02-05 13:40:36.000000000 +0200 +++ /tmp/wklog.47.new.9197 2010-02-05 13:40:36.000000000 +0200 @@ -1 +1 @@ -Server-RawIdeaBin +Server-Sprint -=-=(Guest - Fri, 05 Feb 2010, 13:40)=-=- Status updated. --- /tmp/wklog.47.old.9197 2010-02-05 13:40:36.000000000 +0200 +++ /tmp/wklog.47.new.9197 2010-02-05 13:40:36.000000000 +0200 @@ -1 +1 @@ -Un-Assigned +In-Progress -=-=(Alexi - Thu, 04 Feb 2010, 09:54)=-=- Low Level Design modified. --- /tmp/wklog.47.old.16174 2010-02-04 09:54:13.000000000 +0200 +++ /tmp/wklog.47.new.16174 2010-02-04 09:54:13.000000000 +0200 @@ -171,35 +171,20 @@ 5. How slave IO thread requests Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -When requesting an event, the slave should inform the master whether -it should send Annotate_rows events or not. To that end we add a new -BINLOG_SEND_ANNOTATE_ROWS_EVENT flag used when requesting an event: +If the replicate-annotate-rows-events option is not set on a slave, there +is no need for master to send Annotate_rows events to this slave. The slave +(or mysqlbinlog in remote case), before requesting binlog dump via the +COM_BINLOG_DUMP command, informs the master whether it should send these +events by executing the newly added COM_BINLOG_DUMP_OPTIONS_EXT server +command: + + case COM_BINLOG_DUMP_OPTIONS_EXT: + thd->binlog_dump_flags_ext= packet[0]; + my_ok(thd); + break; - #define BINLOG_DUMP_NON_BLOCK 1 - #define BINLOG_SEND_ANNOTATE_ROWS_EVENT 2 - - pthread_handler_t handle_slave_io(void *arg) - { ... - request_dump(mysql, ...); - ... - } - - int request_dump(MYSQL* mysql, ...) - { ... - if (opt_log_slave_updates && - mi->io_thd->variables.binlog_annotate_rows_events) - binlog_flags|= BINLOG_SEND_ANNOTATE_ROWS_EVENT; - ... - int2store(buf + 4, binlog_flags); - ... - simple_command(mysql, COM_BINLOG_DUMP, buf, ...); - ... - } - -NOTE. mysqlbinlog, when remotely requesting BINLOG_DUMP by calling the -simple_command() function, should also use this flag if it wants (in case -of the --print-annotate-rows-events option set) to recieve Annotate_rows -events. +Note. We add this new command and don't use COM_BINLOG_DUMP to avoid possible +conflicts with MySQL/Sun. 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -338,10 +323,4 @@ to internals(a)lists.mysql.com about this and suggesting to reserve the event number. -Also we should notice the introduction of the BINLOG_SEND_ANNOTATE_ROWS_EVENT -flag taking into account that MySQL/Sun may also introduce a flag with the -same value to be used in the request_dump-mysql_binlog_send interface. -But this is mainly the question of merging: if a conflict concerning this -flag occur, we may simply change the BINLOG_SEND_ANNOTATE_ROWS_EVENT value -(this does not require additional changes in the code). -=-=(Alexi - Sun, 20 Dec 2009, 16:00)=-=- Low Level Design modified. --- /tmp/wklog.47.old.19667 2009-12-20 14:00:56.000000000 +0000 +++ /tmp/wklog.47.new.19667 2009-12-20 14:00:56.000000000 +0000 @@ -196,6 +196,11 @@ ... } +NOTE. mysqlbinlog, when remotely requesting BINLOG_DUMP by calling the +simple_command() function, should also use this flag if it wants (in case +of the --print-annotate-rows-events option set) to recieve Annotate_rows +events. + 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -212,8 +217,7 @@ Log_event::read_log_event(&log, packet, ...); ... if ((*packet)[EVENT_TYPE_OFFSET + 1] != ANNOTATE_ROWS_EVENT || - flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT || - thd->server_id == 0 /* slave == mysqlbinlog */ ) + flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT) { my_net_write(net, packet->ptr(), packet->length()); } -=-=(Alexi - Sun, 20 Dec 2009, 13:14)=-=- Low Level Design modified. --- /tmp/wklog.47.old.11350 2009-12-20 13:14:04.000000000 +0200 +++ /tmp/wklog.47.new.11350 2009-12-20 13:14:04.000000000 +0200 @@ -282,23 +282,18 @@ Annotate_rows_log_event* m_annotate_event; }; -When the saved Annotate_rows object may be deleted? When all corresponding -Rows events will be processed, i.e. before processing the first non-Rows -event (note that Annotate_rows object resides in the binary log *after* -the (possible) 'BEGIN' Query event which accompanies the rows events; note -also that this deletion is adjusted with the case when some or all -corresponding Rows events are filtered out by replicate filter rules): +The saved Annotate_rows object should be deleted when all corresponding +Rows events will be processed: int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev= next_event(rli); ... - if (rli->get_annotate_event() && !IS_RBR_EVENT_TYPE(ev->get_type_code())) - rli->free_annotate_event(); - apply_event_and_update_pos(ev, ...); - if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) + if (rli->get_annotate_event() && is_last_rows_event(ev)) + rli->free_annotate_event(); + else if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) rli->set_annotate_event((Annotate_rows_log_event*) ev); else if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; @@ -307,10 +302,21 @@ where - #define IS_RBR_EVENT_TYPE(type) ( (type) == TABLE_MAP_EVENT || \ - (type) == WRITE_ROWS_EVENT || \ + bool is_last_rows_event(Log_event* ev) + { + Log_event_type type= ev->get_type_code(); + if (IS_ROWS_EVENT_TYPE(type)) + { + Rows_log_event* rows= (Rows_log_event*)ev; + return rows->get_flags(Rows_log_event::STMT_END_F); + } + + return 0; + } + + #define IS_ROWS_EVENT_TYPE(type) ((type) == WRITE_ROWS_EVENT || \ (type) == UPDATE_ROWS_EVENT || \ - (type) == DELETE_ROWS_EVENT ) + (type) == DELETE_ROWS_EVENT) 8. General remarks ~~~~~~~~~~~~~~~~~~ -=-=(Alexi - Sun, 20 Dec 2009, 09:29)=-=- Low Level Design modified. --- /tmp/wklog.47.old.32726 2009-12-20 07:29:56.000000000 +0000 +++ /tmp/wklog.47.new.32726 2009-12-20 07:29:56.000000000 +0000 @@ -56,7 +56,7 @@ 0000006A | 00 | 0 - RESERVED [27] ..... ... 00000081 | 00 | 0 - RESERVED [50] - 00000082 | 00 | 0 - ANNOTATE_RBR_EVENT [51] + 00000082 | 00 | 0 - ANNOTATE_ROWS_EVENT [51] ************************ 2. Outline of Annotate_rows event behavior -=-=(Alexi - Sat, 19 Dec 2009, 16:10)=-=- Low Level Design modified. --- /tmp/wklog.47.old.16051 2009-12-19 16:10:48.000000000 +0200 +++ /tmp/wklog.47.new.16051 2009-12-19 16:10:48.000000000 +0200 @@ -253,7 +253,7 @@ thd query to that of the described by the event, i.e. to the query which caused the subsequent Rows events (see "How Master writes Annotate_rows events to the binary log" to follow what happens further when the subsequent -Rows events is applied): +Rows events are applied): int Annotate_rows_log_event::do_apply_event(...) { -=-=(Alexi - Sat, 19 Dec 2009, 16:02)=-=- High-Level Specification modified. --- /tmp/wklog.47.old.15695 2009-12-19 16:02:33.000000000 +0200 +++ /tmp/wklog.47.new.15695 2009-12-19 16:02:33.000000000 +0200 @@ -12,7 +12,7 @@ post-header and contains the query text in its data part. Example: ************************ - ANNOTATE_RBR_EVENT + ANNOTATE_ROWS_EVENT ************************ 00000220 | B6 A0 2C 4B | time_when = 1261215926 00000224 | 33 | event_type = 51 -=-=(Alexi - Sat, 19 Dec 2009, 15:58)=-=- Low Level Design modified. --- /tmp/wklog.47.old.15437 2009-12-19 13:58:12.000000000 +0000 +++ /tmp/wklog.47.new.15437 2009-12-19 13:58:12.000000000 +0000 @@ -1 +1,337 @@ +Content +~~~~~~~ + 1. Annotate_rows event number + 2. Outline of Annotate_rows event behavior + 3. How Master writes Annotate_rows events to the binary log + 4. How slave treats replicate-annotate-rows-events option + 5. How slave IO thread requests Annotate_rows events + 6. How master executes the request + 7. How slave SQL thread processes Annotate_rows events + 8. General remarks + +1. Annotate_rows event number +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +To avoid possible event numbers conflict with MySQL/Sun, we leave a gap +between the last MySQL event number and the Annotate_rows event number: + + enum Log_event_type + { ... + INCIDENT_EVENT= 26, + // New MySQL event numbers are to be added here + MYSQL_EVENTS_END, + + MARIA_EVENTS_BEGIN= 51, + // New Maria event numbers start from here + ANNOTATE_ROWS_EVENT= 51, + + ENUM_END_EVENT + }; + +together with the corresponding extension of 'post_header_len' array in the +Format description event. (This extension does not affect the compatibility +of the binary log). Here is how Format description event looks like with +this extension: + + ************************ + FORMAT_DESCRIPTION_EVENT + ************************ + 00000004 | A1 A0 2C 4B | time_when = 1261215905 + 00000008 | 0F | event_type = 15 + 00000009 | 64 00 00 00 | server_id = 100 + 0000000D | 7F 00 00 00 | event_len = 127 + 00000011 | 83 00 00 00 | log_pos = 00000083 + 00000015 | 01 00 | flags = LOG_EVENT_BINLOG_IN_USE_F + ------------------------ + 00000017 | 04 00 | binlog_ver = 4 + 00000019 | 35 2E 32 2E | server_ver = 5.2.0-MariaDB-alpha-debug-log + ..... ... + 0000004B | A1 A0 2C 4B | time_created = 1261215905 + 0000004F | 13 | common_header_len = 19 + ------------------------ + post_header_len + ------------------------ + 00000050 | 38 | 56 - START_EVENT_V3 [1] + ..... ... + 00000069 | 02 | 2 - INCIDENT_EVENT [26] + 0000006A | 00 | 0 - RESERVED [27] + ..... ... + 00000081 | 00 | 0 - RESERVED [50] + 00000082 | 00 | 0 - ANNOTATE_RBR_EVENT [51] + ************************ + +2. Outline of Annotate_rows event behavior +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Each Annotate_rows_log_event object has two private members describing the +corresponding query: + + char *m_query_txt; + uint m_query_len; + +When the object is created for writing to a binary log, this query is taken +from 'thd' (for short, below we omit the 'Annotate_rows_log_event::' prefix +as well as other implementation details): + + Annotate_rows_log_event(THD *thd) + { + m_query_txt = thd->query(); + m_query_len = thd->query_length(); + } + +When the object is read from a binary log, the query is taken from the buffer +containing the binary log representation of the event (this buffer is allocated +in Log_event object from which all Log events are derived): + + Annotate_rows_log_event(char *buf, uint event_len, + Format_description_log_event *desc) + { + m_query_len = event_len - desc->common_header_len; + m_query_txt = buf + desc->common_header_len; + } + +The events are written to the binary log by the Log_event::write() member +which calls virtual write_data_header() and write_data_body() members +("data header" and "post header" are synonym in replication terminology). +In our case, data header is empty and data body is just the query: + + bool write_data_body(IO_CACHE *file) + { + return my_b_safe_write(file, (uchar*) m_query_txt, m_query_len); + } + +Printing the event is just printing the query: + + void Annotate_rows_log_event::print(FILE *file, PRINT_EVENT_INFO *pinfo) + { + my_b_printf(&pinfo->head_cache, "\tQuery: `%s`\n", m_query_txt); + } + +3. How Master writes Annotate_rows events to the binary log +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The event is written to the binary log just before the group of Table_map +events which precede corresponding Rows events (one query may generate +several Table map events in the binary log, but the corresponding +Annotate_rows event must be written only once before the first Table map +event; hence the boolean variable 'with_annotate' below): + + int write_locked_table_maps(THD *thd) + { ... + bool with_annotate= thd->variables.binlog_annotate_rows_events; + ... + for (uint i= 0; i < ... <number of tables> ...; ++i) + { ... + thd->binlog_write_table_map(table, ..., with_annotate); + with_annotate= 0; // write Annotate_event not more than once + ... + } + ... + } + + int THD::binlog_write_table_map(TABLE *table, ..., bool with_annotate) + { ... + Table_map_log_event the_event(...); + ... + if (with_annotate) + { + Annotate_rows_log_event anno(this); + mysql_bin_log.write(&anno); + } + + mysql_bin_log.write(&the_event); + ... + } + +4. How slave treats replicate-annotate-rows-events option +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The replicate-annotate-rows-events option is treated just as the session +value of the binlog_annotate_rows_events variable for the slave IO and +SQL threads. This setting is done during initialization of these threads: + + pthread_handler_t handle_slave_io(void *arg) + { + THD *thd= new THD; + ... + init_slave_thread(thd, SLAVE_THD_IO); + ... + } + + pthread_handler_t handle_slave_sql(void *arg) + { + THD *thd= new THD; + ... + init_slave_thread(thd, SLAVE_THD_SQL); + ... + } + + int init_slave_thread(THD* thd, SLAVE_THD_TYPE thd_type) + { ... + thd->variables.binlog_annotate_rows_events= + opt_replicate_annotate_rows_events; + ... + } + +5. How slave IO thread requests Annotate_rows events +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +When requesting an event, the slave should inform the master whether +it should send Annotate_rows events or not. To that end we add a new +BINLOG_SEND_ANNOTATE_ROWS_EVENT flag used when requesting an event: + + #define BINLOG_DUMP_NON_BLOCK 1 + #define BINLOG_SEND_ANNOTATE_ROWS_EVENT 2 + + pthread_handler_t handle_slave_io(void *arg) + { ... + request_dump(mysql, ...); + ... + } + + int request_dump(MYSQL* mysql, ...) + { ... + if (opt_log_slave_updates && + mi->io_thd->variables.binlog_annotate_rows_events) + binlog_flags|= BINLOG_SEND_ANNOTATE_ROWS_EVENT; + ... + int2store(buf + 4, binlog_flags); + ... + simple_command(mysql, COM_BINLOG_DUMP, buf, ...); + ... + } + +6. How master executes the request +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + case COM_BINLOG_DUMP: + { ... + flags= uint2korr(packet + 4); + ... + mysql_binlog_send(thd, ..., flags); + ... + } + + void mysql_binlog_send(THD* thd, ..., ushort flags) + { ... + Log_event::read_log_event(&log, packet, ...); + ... + if ((*packet)[EVENT_TYPE_OFFSET + 1] != ANNOTATE_ROWS_EVENT || + flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT || + thd->server_id == 0 /* slave == mysqlbinlog */ ) + { + my_net_write(net, packet->ptr(), packet->length()); + } + ... + } + +7. How slave SQL thread processes Annotate_rows events +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The slave processes each recieved event by "applying" it, i.e. by +calling the Log_event::apply_event() function which in turn calls +the virtual do_apply_event() member specific for each type of the +event. + + int exec_relay_log_event(THD* thd, Relay_log_info* rli) + { ... + Log_event *ev = next_event(rli); + ... + apply_event_and_update_pos(ev, ...); + + if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) + delete ev; + ... + } + + int apply_event_and_update_pos(Log_event *ev, ...) + { ... + ev->apply_event(...); + ... + } + + int Log_event::apply_event(...) + { + return do_apply_event(...); + } + +What does it mean to "apply" an Annotate_rows event? It means to set current +thd query to that of the described by the event, i.e. to the query which +caused the subsequent Rows events (see "How Master writes Annotate_rows +events to the binary log" to follow what happens further when the subsequent +Rows events is applied): + + int Annotate_rows_log_event::do_apply_event(...) + { + thd->set_query(m_query_txt, m_query_len); + } + +NOTE. I am not sure, but possibly current values of thd->query and +thd->query_length should be saved before calling set_query() and to be +restored on the Annotate_rows_log_event object deletion. +Is it really needed ? + +After calling this do_apply_event() function we may not delete the +Annotate_rows_log_event object immediatedly (see exec_relay_log_event() +above) because thd->query now points to the string inside this object. +We may keep the pointer to this object in the Relay_log_info: + + class Relay_log_info + { + public: + ... + void set_annotate_event(Annotate_rows_log_event*); + Annotate_rows_log_event* get_annotate_event(); + void free_annotate_event(); + ... + private: + Annotate_rows_log_event* m_annotate_event; + }; + +When the saved Annotate_rows object may be deleted? When all corresponding +Rows events will be processed, i.e. before processing the first non-Rows +event (note that Annotate_rows object resides in the binary log *after* +the (possible) 'BEGIN' Query event which accompanies the rows events; note +also that this deletion is adjusted with the case when some or all +corresponding Rows events are filtered out by replicate filter rules): + + int exec_relay_log_event(THD* thd, Relay_log_info* rli) + { ... + Log_event *ev= next_event(rli); + ... + if (rli->get_annotate_event() && !IS_RBR_EVENT_TYPE(ev->get_type_code())) + rli->free_annotate_event(); + + apply_event_and_update_pos(ev, ...); + + if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) + rli->set_annotate_event((Annotate_rows_log_event*) ev); + else if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) + delete ev; + ... + } + +where + + #define IS_RBR_EVENT_TYPE(type) ( (type) == TABLE_MAP_EVENT || \ + (type) == WRITE_ROWS_EVENT || \ + (type) == UPDATE_ROWS_EVENT || \ + (type) == DELETE_ROWS_EVENT ) + +8. General remarks +~~~~~~~~~~~~~~~~~~ +Kristian noticed that introducing new log event type should be coordinated +somehow with MySQL/Sun: + + Kristian: The numeric code for this event must be assigned carefully. + It should be coordinated with MySQL/Sun, otherwise we can get into a + situation where MySQL uses the same numeric code for one event that + MariaDB uses for ANNOTATE_ROWS_EVENT, which would make merging the two + impossible. + Alex: I reserved about 20 numbers not to have possible conflicts + with MySQL. + Kristian: Still, I think it would be appropriate to send a polite email + to internals(a)lists.mysql.com about this and suggesting to reserve the + event number. + +Also we should notice the introduction of the BINLOG_SEND_ANNOTATE_ROWS_EVENT +flag taking into account that MySQL/Sun may also introduce a flag with the +same value to be used in the request_dump-mysql_binlog_send interface. +But this is mainly the question of merging: if a conflict concerning this +flag occur, we may simply change the BINLOG_SEND_ANNOTATE_ROWS_EVENT value +(this does not require additional changes in the code). -=-=(Alexi - Sat, 19 Dec 2009, 15:41)=-=- High-Level Specification modified. --- /tmp/wklog.47.old.14545 2009-12-19 15:41:21.000000000 +0200 +++ /tmp/wklog.47.new.14545 2009-12-19 15:41:21.000000000 +0200 @@ -1,122 +1,107 @@ -First suggestion: - -> I think for this we would actually need a new binlog event type -> (Comment_log_event?). Unless we want to log an empty statement Query_log_event -> containing only a comment (a bit of a hack). - -New server option -~~~~~~~~~~~~~~~~~ - --binlog-annotate-rows-events - -Setting this option makes RBR (rows-) events in the binary log to be -preceded by Annotate rows events (see below). The corresponding -'binlog_annotate_rows_events' system variable is dynamic and has both -global and session values. Default global value is OFF. - -Note. Session values are usefull to make it possible to annotate only - some selected statements: +Content +~~~~~~~ + 1. Annotate_rows_log_event + 2. Server option: --binlog-annotate-rows-events + 3. Server option: --replicate-annotate-rows-events + 4. mysqlbinlog option: --print-annotate-rows-events + 5. mysqlbinlog output + +1. Annotate_rows_log_event [ ANNOTATE_ROWS_EVENT ] +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Describes the query which caused the corresponding rows events. Has empty +post-header and contains the query text in its data part. Example: + + ************************ + ANNOTATE_RBR_EVENT + ************************ + 00000220 | B6 A0 2C 4B | time_when = 1261215926 + 00000224 | 33 | event_type = 51 + 00000225 | 64 00 00 00 | server_id = 100 + 00000229 | 36 00 00 00 | event_len = 54 + 0000022D | 56 02 00 00 | log_pos = 00000256 + 00000231 | 00 00 | flags = <none> + ------------------------ + 00000233 | 49 4E 53 45 | query = "INSERT INTO t1 VALUES (1), (2), (3)" + 00000237 | 52 54 20 49 | + 0000023B | 4E 54 4F 20 | + 0000023F | 74 31 20 56 | + 00000243 | 41 4C 55 45 | + 00000247 | 53 20 28 31 | + 0000024B | 29 2C 20 28 | + 0000024F | 32 29 2C 20 | + 00000253 | 28 33 29 | + ************************ + +In binary log, Annotate_rows event follows the (possible) 'BEGIN' Query event +and precedes the first of Table map events which accompany the corresponding +rows events. (See example in the "mysqlbinlog output" section below.) + +2. Server option: --binlog-annotate-rows-events +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Tells the master to write Annotate_rows events to the binary log. + + * Variable Name: binlog_annotate_rows_events + * Scope: Global & Session + * Access Type: Dynamic + * Data Type: bool + * Default Value: OFF +NOTE. Session values allows to annotate only some selected statements: ... SET SESSION binlog_annotate_rows_events=ON; ... statements to be annotated ... SET SESSION binlog_annotate_rows_events=OFF; ... statements not to be annotated ... -New binlog event type -~~~~~~~~~~~~~~~~~~~~~ - Annotate_rows_log_event [ ANNOTATE_ROWS_EVENT ] - -Describes the query which caused the corresponding rows event. In binary log, -precedes each Table_map_log_event. Contains empty post-header and the query -text in its data part. - -The numeric code for this event must be assigned carefully. It should be -coordinated with MySQL/Sun, otherwise we can get into a situation where MySQL -uses the same numeric code for one event that MariaDB uses for -ANNOTATE_ROWS_EVENT, which would make merging the two impossible. - -Example: - - ... - ************************ - ANNOTATE_ROWS_EVENT [51] - ************************ - 000000C7 | 54 1B 12 4B | time_when = 1259477844 - 000000CB | 33 | event_type = 51 - 000000CC | 64 00 00 00 | server_id = 100 - 000000D0 | 2C 00 00 00 | event_len = 44 - 000000D4 | F3 00 00 00 | log_pos = 000000F3 - 000000D8 | 00 00 | flags = <none> - ------------------------ - 000000DA | 69 6E 73 65 | query = "insert into t1 values (1)" - 000000DE | 72 74 20 69 | - 000000E2 | 6E 74 6F 20 | - 000000E6 | 74 31 20 76 | - 000000EA | 61 6C 75 65 | - 000000EE | 73 20 28 31 | - 000000F2 | 29 | - ************************ - TABLE_MAP_EVENT [19] - ************************ - 000000F3 | 54 1B 12 4B | time_when = 1259477844 - 000000F7 | 13 | event_type = 19 - 000000F8 | 64 00 00 00 | server_id = 100 - 000000FC | 29 00 00 00 | event_len = 41 - 00000100 | 1C 01 00 00 | log_pos = 0000011C - 00000104 | 00 00 | flags = <none> - ------------------------ - ... - ************************ - WRITE_ROWS_EVENT [23] - ************************ - 0000011C | 54 1B 12 4B | time_when = 1259477844 - 00000120 | 17 | event_type = 23 - 00000121 | 64 00 00 00 | server_id = 100 - 00000125 | 22 00 00 00 | event_len = 34 - 00000129 | 3E 01 00 00 | log_pos = 0000013E - 0000012D | 10 00 | flags = LOG_EVENT_UPDATE_TABLE_MAP_VERSION_F - ------------------------ - 0000012F | 0F 00 00 00 | table_id = 15 - ... - -New mysqlbinlog option -~~~~~~~~~~~~~~~~~~~~~~ - --print-annotate-rows-events - -With this option, mysqlbinlog prints the content of Annotate-rows -events (if the binary log does contain them). Without this option -(i.e. by default), mysqlbinlog skips Annotate rows events. - - -mysqlbinlog output -~~~~~~~~~~~~~~~~~~ -Something like this: +3. Server option: --replicate-annotate-rows-events +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Tells the slave to reproduce Annotate_rows events recieved from the master +in its own binary log (sensible only in pair with log-slave-updates option). + + * Variable Name: replicate_annotate_rows_events + * Scope: Global + * Access Type: Read only + * Data Type: bool + * Default Value: OFF + +NOTE. Why do we additionally need this 'replicate' option? Why not to make +the slave to reproduce this events when its binlog-annotate-rows-events +global value is ON? Well, because, for example, we may want to configure +the slave which should reproduce Annotate_rows events but has global +binlog-annotate-rows-events = OFF meaning this to be the default value for +the client threads (see also "How slave treats replicate-annotate-rows-events +option" in LLD part). + +4. mysqlbinlog option: --print-annotate-rows-events +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +With this option, mysqlbinlog prints the content of Annotate_rows events (if +the binary log does contain them). Without this option (i.e. by default), +mysqlbinlog skips Annotate_rows events. +5. mysqlbinlog output +~~~~~~~~~~~~~~~~~~~~~ +With --print-annotate-rows-events, mysqlbinlog outputs Annotate_rows events +in a form like this: ... - # at 199 - # at 243 - # at 284 - #091129 9:57:24 server id 100 end_log_pos 243 Query: `insert into t1 values -(1)` - #091129 9:57:24 server id 100 end_log_pos 284 Table_map: `test`.`t1` mapped -to number 15 - #091129 9:57:24 server id 100 end_log_pos 318 Write_rows: table id 15 + # at 1646 + #091219 12:45:26 server id 100 end_log_pos 1714 Query thread_id=1 +exec_time=0 error_code=0 + SET TIMESTAMP=1261215926/*!*/; + BEGIN + /*!*/; + # at 1714 + # at 1812 + # at 1853 + # at 1894 + # at 1938 + #091219 12:45:26 server id 100 end_log_pos 1812 Query: `DELETE t1, t2 FROM +t1 INNER JOIN t2 INNER JOIN t3 WHERE t1.a=t2.a AND t2.a=t3.a` + #091219 12:45:26 server id 100 end_log_pos 1853 Table_map: `test`.`t1` +mapped to number 16 + #091219 12:45:26 server id 100 end_log_pos 1894 Table_map: `test`.`t2` +mapped to number 17 + #091219 12:45:26 server id 100 end_log_pos 1938 Delete_rows: table id 16 + #091219 12:45:26 server id 100 end_log_pos 1982 Delete_rows: table id 17 flags: STMT_END_F - - BINLOG ' - VBsSSzNkAAAALAAAAPMAAAAAAGluc2VydCBpbnRvIHQxIHZhbHVlcyAoMSk= - VBsSSxNkAAAAKQAAABwBAAAAAA8AAAAAAAAABHRlc3QAAnQxAAEDAAE= - VBsSSxdkAAAAIgAAAD4BAAAQAA8AAAAAAAEAAf/+AQAAAA== - '/*!*/; - ### INSERT INTO test.t1 - ### SET - ### @1=1 /* INT meta=0 nullable=1 is_null=0 */ ... -When master sends Annotate rows events -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -1. Master always sends Annotate_rows events to mysqlbinlog (in - remote case). -2. Master sends Annotate_rows events to a slave only if the slave has - both log-slave-updates and binlog-annotate-rows-events options set. - ------------------------------------------------------------ -=-=(View All Progress Notes, 22 total)=-=- http://askmonty.org/worklog/index.pl?tid=47&nolimit=1 DESCRIPTION: Store in binlog (and show in mysqlbinlog output) texts of statements that caused RBR events This is needed for (list from Monty): - Easier to understand why updates happened - Would make it easier to find out where in application things went wrong (as you can search for exact strings) - Allow one to filter things based on comments in the statement. The cost of this can be that the binlog will be approximately 2x in size (especially insert of big blob's would be a bit painful), so this should be an optional feature. HIGH-LEVEL SPECIFICATION: Content ~~~~~~~ 1. Annotate_rows_log_event 2. Server option: --binlog-annotate-rows-events 3. Server option: --replicate-annotate-rows-events 4. mysqlbinlog option: --print-annotate-rows-events 5. mysqlbinlog output 1. Annotate_rows_log_event [ ANNOTATE_ROWS_EVENT ] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Describes the query which caused the corresponding rows events. Has empty post-header and contains the query text in its data part. Example: ************************ ANNOTATE_ROWS_EVENT ************************ 00000220 | B6 A0 2C 4B | time_when = 1261215926 00000224 | 33 | event_type = 51 00000225 | 64 00 00 00 | server_id = 100 00000229 | 36 00 00 00 | event_len = 54 0000022D | 56 02 00 00 | log_pos = 00000256 00000231 | 00 00 | flags = <none> ------------------------ 00000233 | 49 4E 53 45 | query = "INSERT INTO t1 VALUES (1), (2), (3)" 00000237 | 52 54 20 49 | 0000023B | 4E 54 4F 20 | 0000023F | 74 31 20 56 | 00000243 | 41 4C 55 45 | 00000247 | 53 20 28 31 | 0000024B | 29 2C 20 28 | 0000024F | 32 29 2C 20 | 00000253 | 28 33 29 | ************************ In binary log, Annotate_rows event follows the (possible) 'BEGIN' Query event and precedes the first of Table map events which accompany the corresponding rows events. (See example in the "mysqlbinlog output" section below.) 2. Server option: --binlog-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the master to write Annotate_rows events to the binary log. * Variable Name: binlog_annotate_rows_events * Scope: Global & Session * Access Type: Dynamic * Data Type: bool * Default Value: OFF NOTE. Session values allows to annotate only some selected statements: ... SET SESSION binlog_annotate_rows_events=ON; ... statements to be annotated ... SET SESSION binlog_annotate_rows_events=OFF; ... statements not to be annotated ... 3. Server option: --replicate-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the slave to reproduce Annotate_rows events recieved from the master in its own binary log (sensible only in pair with log-slave-updates option). * Variable Name: replicate_annotate_rows_events * Scope: Global * Access Type: Read only * Data Type: bool * Default Value: OFF NOTE. Why do we additionally need this 'replicate' option? Why not to make the slave to reproduce this events when its binlog-annotate-rows-events global value is ON? Well, because, for example, we may want to configure the slave which should reproduce Annotate_rows events but has global binlog-annotate-rows-events = OFF meaning this to be the default value for the client threads (see also "How slave treats replicate-annotate-rows-events option" in LLD part). 4. mysqlbinlog option: --print-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ With this option, mysqlbinlog prints the content of Annotate_rows events (if the binary log does contain them). Without this option (i.e. by default), mysqlbinlog skips Annotate_rows events. 5. mysqlbinlog output ~~~~~~~~~~~~~~~~~~~~~ With --print-annotate-rows-events, mysqlbinlog outputs Annotate_rows events in a form like this: ... # at 1646 #091219 12:45:26 server id 100 end_log_pos 1714 Query thread_id=1 exec_time=0 error_code=0 SET TIMESTAMP=1261215926/*!*/; BEGIN /*!*/; # at 1714 # at 1812 # at 1853 # at 1894 # at 1938 #091219 12:45:26 server id 100 end_log_pos 1812 Query: `DELETE t1, t2 FROM t1 INNER JOIN t2 INNER JOIN t3 WHERE t1.a=t2.a AND t2.a=t3.a` #091219 12:45:26 server id 100 end_log_pos 1853 Table_map: `test`.`t1` mapped to number 16 #091219 12:45:26 server id 100 end_log_pos 1894 Table_map: `test`.`t2` mapped to number 17 #091219 12:45:26 server id 100 end_log_pos 1938 Delete_rows: table id 16 #091219 12:45:26 server id 100 end_log_pos 1982 Delete_rows: table id 17 flags: STMT_END_F ... LOW-LEVEL DESIGN: Content ~~~~~~~ 1. Annotate_rows event number 2. Outline of Annotate_rows event behavior 3. How Master writes Annotate_rows events to the binary log 4. How slave treats replicate-annotate-rows-events option 5. How slave IO thread requests Annotate_rows events 6. How master executes the request 7. How slave SQL thread processes Annotate_rows events 8. General remarks 1. Annotate_rows event number ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To avoid possible event numbers conflict with MySQL/Sun, we leave a gap between the last MySQL event number and the Annotate_rows event number: enum Log_event_type { ... INCIDENT_EVENT= 26, // New MySQL event numbers are to be added here MYSQL_EVENTS_END, MARIA_EVENTS_BEGIN= 51, // New Maria event numbers start from here ANNOTATE_ROWS_EVENT= 51, ENUM_END_EVENT }; together with the corresponding extension of 'post_header_len' array in the Format description event. (This extension does not affect the compatibility of the binary log). Here is how Format description event looks like with this extension: ************************ FORMAT_DESCRIPTION_EVENT ************************ 00000004 | A1 A0 2C 4B | time_when = 1261215905 00000008 | 0F | event_type = 15 00000009 | 64 00 00 00 | server_id = 100 0000000D | 7F 00 00 00 | event_len = 127 00000011 | 83 00 00 00 | log_pos = 00000083 00000015 | 01 00 | flags = LOG_EVENT_BINLOG_IN_USE_F ------------------------ 00000017 | 04 00 | binlog_ver = 4 00000019 | 35 2E 32 2E | server_ver = 5.2.0-MariaDB-alpha-debug-log ..... ... 0000004B | A1 A0 2C 4B | time_created = 1261215905 0000004F | 13 | common_header_len = 19 ------------------------ post_header_len ------------------------ 00000050 | 38 | 56 - START_EVENT_V3 [1] ..... ... 00000069 | 02 | 2 - INCIDENT_EVENT [26] 0000006A | 00 | 0 - RESERVED [27] ..... ... 00000081 | 00 | 0 - RESERVED [50] 00000082 | 00 | 0 - ANNOTATE_ROWS_EVENT [51] ************************ 2. Outline of Annotate_rows event behavior ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Each Annotate_rows_log_event object has two private members describing the corresponding query: char *m_query_txt; uint m_query_len; When the object is created for writing to a binary log, this query is taken from 'thd' (for short, below we omit the 'Annotate_rows_log_event::' prefix as well as other implementation details): Annotate_rows_log_event(THD *thd) { m_query_txt = thd->query(); m_query_len = thd->query_length(); } When the object is read from a binary log, the query is taken from the buffer containing the binary log representation of the event (this buffer is allocated in Log_event object from which all Log events are derived): Annotate_rows_log_event(char *buf, uint event_len, Format_description_log_event *desc) { m_query_len = event_len - desc->common_header_len; m_query_txt = buf + desc->common_header_len; } The events are written to the binary log by the Log_event::write() member which calls virtual write_data_header() and write_data_body() members ("data header" and "post header" are synonym in replication terminology). In our case, data header is empty and data body is just the query: bool write_data_body(IO_CACHE *file) { return my_b_safe_write(file, (uchar*) m_query_txt, m_query_len); } Printing the event is just printing the query: void Annotate_rows_log_event::print(FILE *file, PRINT_EVENT_INFO *pinfo) { my_b_printf(&pinfo->head_cache, "\tQuery: `%s`\n", m_query_txt); } 3. How Master writes Annotate_rows events to the binary log ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The event is written to the binary log just before the group of Table_map events which precede corresponding Rows events (one query may generate several Table map events in the binary log, but the corresponding Annotate_rows event must be written only once before the first Table map event; hence the boolean variable 'with_annotate' below): int write_locked_table_maps(THD *thd) { ... bool with_annotate= thd->variables.binlog_annotate_rows_events; ... for (uint i= 0; i < ... <number of tables> ...; ++i) { ... thd->binlog_write_table_map(table, ..., with_annotate); with_annotate= 0; // write Annotate_event not more than once ... } ... } int THD::binlog_write_table_map(TABLE *table, ..., bool with_annotate) { ... Table_map_log_event the_event(...); ... if (with_annotate) { Annotate_rows_log_event anno(this); mysql_bin_log.write(&anno); } mysql_bin_log.write(&the_event); ... } 4. How slave treats replicate-annotate-rows-events option ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The replicate-annotate-rows-events option is treated just as the session value of the binlog_annotate_rows_events variable for the slave IO and SQL threads. This setting is done during initialization of these threads: pthread_handler_t handle_slave_io(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_IO); ... } pthread_handler_t handle_slave_sql(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_SQL); ... } int init_slave_thread(THD* thd, SLAVE_THD_TYPE thd_type) { ... thd->variables.binlog_annotate_rows_events= opt_replicate_annotate_rows_events; ... } 5. How slave IO thread requests Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If the replicate-annotate-rows-events option is not set on a slave, there is no need for master to send Annotate_rows events to this slave. The slave (or mysqlbinlog in remote case), before requesting binlog dump via the COM_BINLOG_DUMP command, informs the master whether it should send these events by executing the newly added COM_BINLOG_DUMP_OPTIONS_EXT server command: case COM_BINLOG_DUMP_OPTIONS_EXT: thd->binlog_dump_flags_ext= packet[0]; my_ok(thd); break; Note. We add this new command and don't use COM_BINLOG_DUMP to avoid possible conflicts with MySQL/Sun. 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ case COM_BINLOG_DUMP: { ... flags= uint2korr(packet + 4); ... mysql_binlog_send(thd, ..., flags); ... } void mysql_binlog_send(THD* thd, ..., ushort flags) { ... Log_event::read_log_event(&log, packet, ...); ... if ((*packet)[EVENT_TYPE_OFFSET + 1] != ANNOTATE_ROWS_EVENT || flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT) { my_net_write(net, packet->ptr(), packet->length()); } ... } 7. How slave SQL thread processes Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The slave processes each recieved event by "applying" it, i.e. by calling the Log_event::apply_event() function which in turn calls the virtual do_apply_event() member specific for each type of the event. int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev = next_event(rli); ... apply_event_and_update_pos(ev, ...); if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } int apply_event_and_update_pos(Log_event *ev, ...) { ... ev->apply_event(...); ... } int Log_event::apply_event(...) { return do_apply_event(...); } What does it mean to "apply" an Annotate_rows event? It means to set current thd query to that of the described by the event, i.e. to the query which caused the subsequent Rows events (see "How Master writes Annotate_rows events to the binary log" to follow what happens further when the subsequent Rows events are applied): int Annotate_rows_log_event::do_apply_event(...) { thd->set_query(m_query_txt, m_query_len); } NOTE. I am not sure, but possibly current values of thd->query and thd->query_length should be saved before calling set_query() and to be restored on the Annotate_rows_log_event object deletion. Is it really needed ? After calling this do_apply_event() function we may not delete the Annotate_rows_log_event object immediatedly (see exec_relay_log_event() above) because thd->query now points to the string inside this object. We may keep the pointer to this object in the Relay_log_info: class Relay_log_info { public: ... void set_annotate_event(Annotate_rows_log_event*); Annotate_rows_log_event* get_annotate_event(); void free_annotate_event(); ... private: Annotate_rows_log_event* m_annotate_event; }; The saved Annotate_rows object should be deleted when all corresponding Rows events will be processed: int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev= next_event(rli); ... apply_event_and_update_pos(ev, ...); if (rli->get_annotate_event() && is_last_rows_event(ev)) rli->free_annotate_event(); else if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) rli->set_annotate_event((Annotate_rows_log_event*) ev); else if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } where bool is_last_rows_event(Log_event* ev) { Log_event_type type= ev->get_type_code(); if (IS_ROWS_EVENT_TYPE(type)) { Rows_log_event* rows= (Rows_log_event*)ev; return rows->get_flags(Rows_log_event::STMT_END_F); } return 0; } #define IS_ROWS_EVENT_TYPE(type) ((type) == WRITE_ROWS_EVENT || \ (type) == UPDATE_ROWS_EVENT || \ (type) == DELETE_ROWS_EVENT) 8. General remarks ~~~~~~~~~~~~~~~~~~ Kristian noticed that introducing new log event type should be coordinated somehow with MySQL/Sun: Kristian: The numeric code for this event must be assigned carefully. It should be coordinated with MySQL/Sun, otherwise we can get into a situation where MySQL uses the same numeric code for one event that MariaDB uses for ANNOTATE_ROWS_EVENT, which would make merging the two impossible. Alex: I reserved about 20 numbers not to have possible conflicts with MySQL. Kristian: Still, I think it would be appropriate to send a polite email to internals(a)lists.mysql.com about this and suggesting to reserve the event number. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] FYI: askmonty.org/buildbot is moving to buildbot.askmonty.org/buildbot
by Daniel Bartholomew 04 Feb '10

04 Feb '10

All, This is an FYI for everyone that uses our buildbot web interface: Due to an upcoming server move I am in the process of moving our buildbot web interface from http://askmonty.org/buildbot to http://buildbot.askmonty.org/buildbot Both of the above currently work (and point at the same data), but as of Friday 5 Feb 2010 at approx 09:00 US Eastern time, http://askmonty.org/buildbot will turn into a redirect. The redirect will handle all of your old bookmarks (i.e. they should still work), but I thought I should at least let everyone know so that you aren't surprised by the move. I've already updated all of the links I could find on the askmonty.org wiki. And if you are wondering why I kept the /buildbot subdirectory in place on the new subdomain, well, it was easier and safer to keep it in place, so I did. If you connect directly to http://buildbot.askmonty.org you will automatically be redirected to the /buildbot subdir. Thanks! -- Daniel Bartholomew Monty Program - http://askmonty.org

1 0

[Maria-developers] Rev 157: saving uncommitted changes in /etc prior to apt run in file:///etc/
by timour＠askmonty.org 04 Feb '10

04 Feb '10

At file:///etc/ ------------------------------------------------------------ revno: 157 revision-id: timour(a)askmonty.org-20100204112537-3a6fyd3ibqzi73wc parent: root@lamia-20100203064317-whagbeui3pyl1bck committer: timour(a)askmonty.org branch nick: lamia /etc repository timestamp: Thu 2010-02-04 13:25:37 +0200 message: saving uncommitted changes in /etc prior to apt run === modified file 'cups/subscriptions.conf' --- a/cups/subscriptions.conf 2010-02-03 06:43:17 +0000 +++ b/cups/subscriptions.conf 2010-02-04 11:25:37 +0000 @@ -1,11 +1,11 @@ # Subscription configuration file for CUPS v1.4.1 -# Written by cupsd on 2010-02-02 08:18 -NextSubscriptionId 101 -<Subscription 100> +# Written by cupsd on 2010-02-04 13:19 +NextSubscriptionId 102 +<Subscription 101> Events printer-state-changed printer-restarted printer-shutdown printer-stopped printer-added printer-deleted job-state-changed job-created job-completed job-stopped job-progress Owner tsk LeaseDuration 86400 Interval 0 -ExpirationTime 1265177812 -NextEventId 8 +ExpirationTime 1265368739 +NextEventId 1 </Subscription> === modified file 'cups/subscriptions.conf.O' --- a/cups/subscriptions.conf.O 2010-02-03 06:43:17 +0000 +++ b/cups/subscriptions.conf.O 2010-02-04 11:25:37 +0000 @@ -1,11 +1,3 @@ # Subscription configuration file for CUPS v1.4.1 -# Written by cupsd on 2010-02-02 08:17 +# Written by cupsd on 2010-02-03 08:43 NextSubscriptionId 101 -<Subscription 100> -Events printer-state-changed printer-restarted printer-shutdown printer-stopped printer-added printer-deleted job-state-changed job-created job-completed job-stopped job-progress -Owner tsk -LeaseDuration 86400 -Interval 0 -ExpirationTime 1265177812 -NextEventId 7 -</Subscription> === modified file 'resolv.conf' --- a/resolv.conf 2010-02-01 11:56:20 +0000 +++ b/resolv.conf 2010-02-04 11:25:37 +0000 @@ -1,2 +1,3 @@ # Generated by NetworkManager -nameserver 192.168.2.1 +nameserver 212.50.0.10 +nameserver 212.50.10.50

1 0

[Maria-developers] bzr commit into file:///etc/ branch (timour:157)
by timour＠askmonty.org 04 Feb '10

04 Feb '10

#At file:///etc/ based on revid:root@lamia-20100203064317-whagbeui3pyl1bck 157 timour(a)askmonty.org 2010-02-04 saving uncommitted changes in /etc prior to apt run modified: cups/subscriptions.conf cups/subscriptions.conf.O resolv.conf === modified file 'cups/subscriptions.conf' --- a/cups/subscriptions.conf 2010-02-03 06:43:17 +0000 +++ b/cups/subscriptions.conf 2010-02-04 11:25:37 +0000 @@ -1,11 +1,11 @@ # Subscription configuration file for CUPS v1.4.1 -# Written by cupsd on 2010-02-02 08:18 -NextSubscriptionId 101 -<Subscription 100> +# Written by cupsd on 2010-02-04 13:19 +NextSubscriptionId 102 +<Subscription 101> Events printer-state-changed printer-restarted printer-shutdown printer-stopped printer-added printer-deleted job-state-changed job-created job-completed job-stopped job-progress Owner tsk LeaseDuration 86400 Interval 0 -ExpirationTime 1265177812 -NextEventId 8 +ExpirationTime 1265368739 +NextEventId 1 </Subscription> === modified file 'cups/subscriptions.conf.O' --- a/cups/subscriptions.conf.O 2010-02-03 06:43:17 +0000 +++ b/cups/subscriptions.conf.O 2010-02-04 11:25:37 +0000 @@ -1,11 +1,3 @@ # Subscription configuration file for CUPS v1.4.1 -# Written by cupsd on 2010-02-02 08:17 +# Written by cupsd on 2010-02-03 08:43 NextSubscriptionId 101 -<Subscription 100> -Events printer-state-changed printer-restarted printer-shutdown printer-stopped printer-added printer-deleted job-state-changed job-created job-completed job-stopped job-progress -Owner tsk -LeaseDuration 86400 -Interval 0 -ExpirationTime 1265177812 -NextEventId 7 -</Subscription> === modified file 'resolv.conf' --- a/resolv.conf 2010-02-01 11:56:20 +0000 +++ b/resolv.conf 2010-02-04 11:25:37 +0000 @@ -1,2 +1,3 @@ # Generated by NetworkManager -nameserver 192.168.2.1 +nameserver 212.50.0.10 +nameserver 212.50.10.50

1 0

[Maria-developers] Rev 8: Merge. in file:///Users/hakan/work/monty_program/mariadb-tools/
by Hakan Kuecuekyilmaz 04 Feb '10

04 Feb '10

At file:///Users/hakan/work/monty_program/mariadb-tools/ ------------------------------------------------------------ revno: 8 [merge] revision-id: hakan(a)askmonty.org-20100204111840-0d9u3nyio2h8tavi parent: hakan(a)askmonty.org-20100204011956-u74a4es6oogvd50w parent: knielsen@hasky-20100202151012-pjcblb2v3bqsp5eo committer: Hakan Kuecuekyilmaz <hakan(a)askmonty.org> branch nick: mariadb-tools timestamp: Thu 2010-02-04 12:18:40 +0100 message: Merge. modified: buildbot/maria-master.cfg mariamaster.cfg-20091218103450-cvifjz3i70oerkej-1 buildbot/runvm runvm-20091218082659-ept0jpsqa5e8jno6-2 === modified file 'buildbot/maria-master.cfg' --- a/buildbot/maria-master.cfg 2010-01-02 21:17:58 +0000 +++ b/buildbot/maria-master.cfg 2010-02-02 15:10:12 +0000 @@ -59,7 +59,6 @@ ,mkSlave("psergey-pylon-amd64", max_builds=1) ,mkSlave("psergey-foxhole-x86", max_builds=1) ,mkSlave("psergey-pslp2-x86", max_builds=1) - ,mkSlave("hakan-mac-g5", max_builds=1) ,mkSlave("work-opensuse-amd64", max_builds=1) ,mkSlave("psergey-win32box", max_builds=1) ,mkSlave("vm-win-1", max_builds=1) @@ -114,7 +113,7 @@ "lp:~maria-captains/maria/maria-5.2-merge-5.1" : "maria-5.2-merge-5.1", "lp:~maria-captains/maria/5.2-dsmrr" : "5.2-dsmrr", "lp:~maria-captains/maria/5.3" : "5.3", - "lp:~maria-captains/maria/5.3" : "5.3-sj-subqueries" + "lp:~maria-captains/maria/5.3-sj-subqueries" : "5.3-sj-subqueries" } mailSource = mail.BzrLaunchpadEmailMaildirSource("/var/lib/buildbot/Maildir", branchMap=myBranchMap) @@ -163,7 +162,7 @@ # "lenny-amd64-dbg", "adutko-alpha", "debian5-i386-fulltest", "jaunty-x86-valgrind", "jaunty-amd64-rel", - "gentoo-x86-dbg", "ubuntu-x86-dbg", "macosx-g5-dbg", + "gentoo-x86-dbg", "ubuntu-x86-dbg", "macosx-x86-bld", "work-amd64-valgrind", "winxp-x86-nmake", "win32-rel-nmake", "hardy-amd64-fulltest", "gentoo-amd64-sanja", "opensolaris-511-x86", @@ -180,6 +179,7 @@ branches=["mariadb-5.1-knielsen", "5.1-release", "5.1", "5.1.39-oqgraph", "5.1-merge", "mariadb-5.1-monty", + "5.2", "mariadb-5.2-monty", ], treeStableTimer=1, # 1 sec for bzr properties= {"bakebranch": "lp:~maria-captains/ourdelta/ourdelta-montyprogram-fixes"}, @@ -197,7 +197,9 @@ "kvm-deb-jaunty-amd64", "kvm-deb-jaunty-x86", "kvm-deb-karmic-amd64", "kvm-deb-karmic-x86", "kvm-deb-lucid-amd64", "kvm-deb-lucid-x86", - "kvm-bintar-hardy-amd64", "kvm-bintar-hardy-x86",])) + "kvm-bintar-hardy-amd64", "kvm-bintar-hardy-x86", + "opensolaris-511-bintar", + ])) ####### BUILDERS @@ -529,7 +531,7 @@ "slavename": "adutko-centos5-amd64", "builddir": "centos5-amd64-minimal", "factory": f_minimal, - "category": "experimental", + "category": "main", } f_win32_rel_nmake = factory.BuildFactory() @@ -699,7 +701,7 @@ "slavename": "adutko-ultrasparc3", "builddir": "adutko-ultrasparc3", "factory": f_sol_sparc_32, - "category": "new", + "category": "main", } f_dbg_alpha = factory.BuildFactory() @@ -771,23 +773,6 @@ "category": "experimental", } -f_dbg_g5 = factory.BuildFactory() -f_dbg_g5.addStep(bzr_shared_repo) -f_dbg_g5.addStep(bzr_checkout) -f_dbg_g5.addStep(getCompileStep(["BUILD/compile-ppc-max"])) -f_dbg_g5.addStep(getMTR( - test_type="nm", - test_info="Normal run, no --ps-protocol", - command=["sh", "-c", "cd mysql-test && exec perl mysql-test-run.pl --force --retry=3 --skip-ndb --testcase-timeout=45 --suite-timeout=1080 --mysqld=--skip-safemalloc"], - timeout=2700)) - -bld_mac_g5 = {'name': "macosx-g5-dbg", - 'slavename': "hakan-mac-g5", - 'builddir': "hakan-mac-g5", - 'factory': f_dbg_g5, - "category": "experimental", - } - f_mac_x86 = factory.BuildFactory() f_mac_x86.addStep(bzr_shared_repo) f_mac_x86.addStep(bzr_checkout) @@ -878,6 +863,36 @@ "category": "experimental", } +# The trees for which we save binary packages. +savedPackageBranches= ["5.1-release", "5.2-release", "5.3-release"] + +# Get a build step that will archive binary packages (or source tarball). +# Only the newest 3 builds are saved for each (branch, builder) combination. +# The packages are saved under the build number of the tarbake step, so it is +# easy to locate all packages for a given release. +def getPackageArchiveStep(source, tarbuildnum): + cmdText= ("TARBUILDNUM='" + tarbuildnum + "'\n" + + "SRC='" + source + "'\n" + """ +BUILDERNAME='%(buildername)s' +BRANCH='%(branch)s' +BASE="/archive/pack/$BRANCH" +DST="$BASE/build-$TARBUILDNUM/$BUILDERNAME" +set -ex +mkdir -p "$BASE" +rm -Rf "$DST" +(ls -td $BASE/build-*/"$BUILDERNAME" || : ) | (read DUMMY || exit 0; read DUMMY || exit 0; while read VICTIM; do rm -Rf "$VICTIM"; done) +rmdir $BASE/build-* 2>/dev/null || : +mkdir -p "$DST" +cp -r "$SRC" "$DST/" +""") + return ShellCommand( + doStepIf=(lambda(step): step.getProperty("branch") in savedPackageBranches), + description=["archiving"], + descriptionDone=["archive"], + command=["sh", "-c", WithProperties(cmdText)], + ) + + f_kvm_tarbake_jaunty_x86= factory.BuildFactory() f_kvm_tarbake_jaunty_x86.addStep(Compile( description=["making", "dist"], @@ -906,35 +921,38 @@ """ set -ex cd buildbot/build/$(cat buildbot/build/bakery.txt)/ -echo mariadb-*.tar.gz > ../distname.txt -mv $(cat ../distname.txt) ../ +basename mariadb-*.tar.gz .tar.gz > ../distdirname.txt +mv "$(cat ../distdirname.txt).tar.gz" ../ """, - "= scp -P 2223 buildbot@localhost:buildbot/build/distname.txt .", + "= scp -P 2223 buildbot@localhost:buildbot/build/distdirname.txt .", "= scp -P 2223 buildbot@localhost:buildbot/build/bakery.txt .", "= scp -P 2223 'buildbot@localhost:buildbot/build/mariadb-*.tar.gz' .", "= scp -P 2223 'buildbot@localhost:buildbot/build/bakery-*.tar.gz' .", ], )) f_kvm_tarbake_jaunty_x86.addStep(SetProperty( - property="distname", - command=["cat", "distname.txt"], + property="distdirname", + command=["cat", "distdirname.txt"], )) f_kvm_tarbake_jaunty_x86.addStep(SetProperty( property="bakery", command=["cat", "bakery.txt"], )) -f_kvm_tarbake_jaunty_x86.addStep(FileUpload(slavesrc=WithProperties("%(distname)s"), - masterdest=WithProperties("/var/lib/buildbot/OQ-tarballs/%(buildnumber)s:%(distname)s"))) +f_kvm_tarbake_jaunty_x86.addStep(getPackageArchiveStep("%(distdirname)s.tar.gz", "%(buildnumber)s")) +f_kvm_tarbake_jaunty_x86.addStep(FileUpload(slavesrc=WithProperties("%(distdirname)s.tar.gz"), + masterdest=WithProperties("/var/lib/buildbot/OQ-tarballs/%(buildnumber)s:%(distdirname)s.tar.gz"))) f_kvm_tarbake_jaunty_x86.addStep(FileUpload(slavesrc=WithProperties("%(bakery)s.tar.gz"), masterdest=WithProperties("/var/lib/buildbot/OQ-bakeries/%(buildnumber)s:%(bakery)s.tar.gz"))) f_kvm_tarbake_jaunty_x86.addStep(Trigger( schedulerNames=["kvm-ourdelta-sched"], waitForFinish=False, updateSourceStamp=True, - set_properties={ "tarballpath": WithProperties("/var/lib/buildbot/OQ-tarballs/%(buildnumber)s:%(distname)s"), + set_properties={ "tarballpath": WithProperties("/var/lib/buildbot/OQ-tarballs/%(buildnumber)s:%(distdirname)s.tar.gz"), "bakerypath": WithProperties("/var/lib/buildbot/OQ-bakeries/%(buildnumber)s:%(bakery)s.tar.gz"), + "distname": WithProperties("%(distdirname)s.tar.gz"), + "tarbuildnum": WithProperties("%(buildnumber)s"), }, - copy_properties=[ "distname", "bakery" ])) + copy_properties=[ "distdirname", "bakery" ])) bld_kvm_tarbake_jaunty_x86 = {"name": "kvm-tarbake-jaunty-x86", "slavename": "knielsen-kvm-x86", @@ -977,6 +995,7 @@ "= rm -Rf rpms && mkdir rpms", "= scp -P "+port+" 'buildbot@localhost:/usr/src/redhat/RPMS/"+arch+"/*.rpm' rpms/", ])) + rpm_fact.addStep(getPackageArchiveStep("rpms", "%(tarbuildnum)s")) rpm_fact.addStep(Test( description=["testing", "install"], descriptionDone=["test", "install"], @@ -1013,11 +1032,22 @@ deb_fact.addStep(FileDownload( mastersrc=WithProperties("%(bakerypath)s"), slavedest=WithProperties("%(bakery)s.tar.gz"))) + # Extract the compiler warning suppressions file from the source tarball. + deb_fact.addStep(ShellCommand( + description=["getting", ".supp"], + descriptionDone=["get", ".supp"], + command=["sh", "-c", WithProperties(""" +rm -f compiler_warnings.supp +tar zxf "%(distname)s" --strip 2 "$(basename %(distname)s .tar.gz)/support-files/compiler_warnings.supp" +exit 0 # best-effort, not fatal if no suppression file +""")])) deb_fact.addStep(Compile( description=["making", "debs"], descriptionDone=["make", "debs"], logfiles={"kernel": "kernel_"+port+".log"}, warningPattern=gccWarningPattern, + warningExtractor=Compile.warnExtractFromRegexpGroups, + suppressionFile=WithProperties("compiler_warnings.supp"), command=["runvm", "--base-image=/kvm/vms/"+kvm_image+"-build.qcow2"] + args +["--logfile=kernel_"+port+".log", "vm-tmp-"+port+".qcow2", "rm -Rf buildbot && mkdir buildbot", WithProperties("= scp -P "+port+" %(distname)s buildbot@localhost:buildbot/"), @@ -1039,6 +1069,7 @@ "= rm -Rf debs", "= scp -r -P "+port+" buildbot@localhost:buildbot/debs .", ])) + deb_fact.addStep(getPackageArchiveStep("debs", "%(tarbuildnum)s")) deb_fact.addStep(Test( description=["testing", "install"], descriptionDone=["test", "install"], @@ -1144,6 +1175,7 @@ property="bindistname", command=["cat", "bindistname.txt"], )) + bin_fact.addStep(getPackageArchiveStep("%(bindistname)s.tar.gz", "%(tarbuildnum)s")) bin_fact.addStep(Test( description=["testing", "bintar"], descriptionDone=["test", "bintar"], @@ -1178,14 +1210,55 @@ 'slavename': 'opensolaris-x86', 'builddir': 'opensolaris-511-x86', 'factory': f_dbg_sol_64, - "category": "new", + "category": "main", } +f_sol_64_bintar = factory.BuildFactory() +f_sol_64_bintar.addStep(FileDownload(mastersrc=WithProperties("%(tarballpath)s"), + slavedest=WithProperties("%(distname)s"))) +f_sol_64_bintar.addStep(ShellCommand( + description="untarring", descriptionDone="untar", + command=["sh", "-c", WithProperties("rm -Rf %(distdirname)s && tar zxf %(distname)s")])) +f_sol_64_bintar.addStep(getCompileStep(["sh", "-c", WithProperties("cd %(distdirname)s && BUILD/compile-solaris-amd64-debug")], + subdir="%(distdirname)s")) +f_sol_64_bintar.addStep(ShellCommand( + description="packaging", descriptionDone="package", + command=["sh", "-c", WithProperties(""" +set -ex +cd %(distdirname)s +scripts/make_binary_distribution +ORIG_TAR=$(echo mariadb-*.tar.gz) +ORIG_DIR=$(basename ${ORIG_TAR} .tar.gz) +NEW_DIR="$(echo "${ORIG_DIR}" | sed -e 's/-MariaDB//')" +NEW_TAR="${NEW_DIR}.tar.gz" +rm -Rf $ORIG_DIR +tar zxf $ORIG_TAR +mv $ORIG_DIR "${NEW_DIR}" +tar zcf "${NEW_TAR}" "${NEW_DIR}/" +echo "${NEW_TAR}" > ../bindistname.txt +""")])) +f_sol_64_bintar.addStep(SetProperty( + property="bindistname", + command=["cat", "bindistname.txt"], + )) +f_sol_64_bintar.addStep(getPackageArchiveStep("%(distdirname)s/%(bindistname)s", "%(tarbuildnum)s")) +f_sol_64_bintar.addStep(getMTR( + test_type="nm", + test_info="Normal run, no --ps-protocol", + command=["sh", "-c", WithProperties("cd %(distdirname)s/mysql-test && exec perl mysql-test-run.pl --force --retry=3 --skip-ndb --parallel=2")], + mtr_subdir=WithProperties("%(distdirname)s/mysql-test"))) + +bld_opensolaris_511_bintar = {'name': 'opensolaris-511-bintar', + 'slavename': 'opensolaris-x86', + 'builddir': 'opensolaris-511-bintar', + 'factory': f_sol_64_bintar, + "category": "package", + } + c['builders'] = [bld_fulltest, bld_fulltest2, bld_work_valgrind, bld_valgrind_32, bld_rel_amd64, bld_dbg_x86, bld_makedist, bld_archivist_amd64, bld_archivist_cnc, - bld_mac_g5, bld_dbg2_x86, bld_psergey_win32box, bld_win32_tarball, bld_win32_zip, bld_win32_rel_nmake, bld_mac_x86, @@ -1205,6 +1278,7 @@ bld_kvm_deb_karmic_amd64, bld_kvm_deb_karmic_x86, bld_kvm_deb_lucid_amd64, bld_kvm_deb_lucid_x86, bld_kvm_bintar_hardy_amd64, bld_kvm_bintar_hardy_x86, + bld_opensolaris_511_bintar, bld_opensolaris_511_x86, bld_sol_sparc_32, ] === modified file 'buildbot/runvm' --- a/buildbot/runvm 2010-01-14 13:43:51 +0000 +++ b/buildbot/runvm 2010-02-01 12:44:19 +0000 @@ -52,6 +52,19 @@ my $opt_baseimage= undef; my @user_cmd_opt; +# Disable host key checking for ssh. +# This is a bit convoluted due to OpenSSH's slight security-paranoia. +# Without this, we would get a login failure if using another VM image +# (with different host key) on the same port, which is annoying. +# An alternative would be to use CheckHostIP=no and HostKeyAlias=<img.qcow2> +# to get ssh to check a different key for each image. But that would still +# cause an error if re-generating an image (with new ssh host key), and it +# doesn't really give any additional security. +my @ssh_cmd_prefix= ($ssh_exec, + '-o', 'UserKnownHostsFile=/dev/null', + '-o', 'StrictHostKeyChecking=no', + '-o', 'LogLevel=ERROR'); + my $image; my $pidfile; @@ -177,17 +190,6 @@ return $res; } -sub exec_guest_cmd { - my ($cmd_and_args)= @_; - if (scalar(@$cmd_and_args) == 0 || $cmd_and_args->[0] eq '') - { - # Otherwise ssh will do an interactive login, which we do not want. - exit 0; - } - - exec_guest_cmd $ssh_exec, '-p', $opt_port, @user_cmd_opt, 'localhost', @$cmd_and_args; -} - sub is_port_used { socket(SOCK, PF_INET, SOCK_STREAM, getprotobyname('tcp')) or die "socket() failed: $!\n"; @@ -348,7 +350,7 @@ die "Fatal error: Cannot fork(): $!\n"; } elsif (!$pid) { # Child. - exec_with_print($ssh_exec, '-o', 'ConnectTimeout=4', '-p', $opt_port, + exec_with_print(@ssh_cmd_prefix, '-o', 'ConnectTimeout=4', '-p', $opt_port, @user_cmd_opt, 'localhost', 'sudo', '/sbin/shutdown', '-h', 'now'); } else { @@ -410,7 +412,7 @@ die "Fatal error: Cannot fork(): $!\n"; } elsif (!$pid) { # Child. - exec_with_print($ssh_exec, '-o', 'ConnectTimeout=4', '-p', $opt_port, + exec_with_print(@ssh_cmd_prefix, '-o', 'ConnectTimeout=4', '-p', $opt_port, @user_cmd_opt, 'localhost', '/bin/true'); } else { # Parent. @@ -506,7 +508,7 @@ $res= system(substr($arg, 1)); } else { print STDERR "+ $arg\n"; - $res= system($ssh_exec, '-p', $opt_port, @user_cmd_opt, 'localhost', $arg); + $res= system(@ssh_cmd_prefix, '-p', $opt_port, @user_cmd_opt, 'localhost', $arg); } if ($res < 0) { print STDERR "Could not spawn command: $!\n";

1 0

[Maria-developers] Updated (by Alexi): Store in binlog text of statements that caused RBR events (47)
by worklog-noreply＠askmonty.org 04 Feb '10

04 Feb '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Store in binlog text of statements that caused RBR events CREATION DATE..: Sat, 15 Aug 2009, 23:48 SUPERVISOR.....: Monty IMPLEMENTOR....: COPIES TO......: Knielsen CATEGORY.......: Server-RawIdeaBin TASK ID........: 47 (http://askmonty.org/worklog/?tid=47) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 20 ESTIMATE.......: 35 (hours remain) ORIG. ESTIMATE.: 35 PROGRESS NOTES: -=-=(Alexi - Thu, 04 Feb 2010, 09:54)=-=- Low Level Design modified. --- /tmp/wklog.47.old.16174 2010-02-04 09:54:13.000000000 +0200 +++ /tmp/wklog.47.new.16174 2010-02-04 09:54:13.000000000 +0200 @@ -171,35 +171,20 @@ 5. How slave IO thread requests Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -When requesting an event, the slave should inform the master whether -it should send Annotate_rows events or not. To that end we add a new -BINLOG_SEND_ANNOTATE_ROWS_EVENT flag used when requesting an event: +If the replicate-annotate-rows-events option is not set on a slave, there +is no need for master to send Annotate_rows events to this slave. The slave +(or mysqlbinlog in remote case), before requesting binlog dump via the +COM_BINLOG_DUMP command, informs the master whether it should send these +events by executing the newly added COM_BINLOG_DUMP_OPTIONS_EXT server +command: + + case COM_BINLOG_DUMP_OPTIONS_EXT: + thd->binlog_dump_flags_ext= packet[0]; + my_ok(thd); + break; - #define BINLOG_DUMP_NON_BLOCK 1 - #define BINLOG_SEND_ANNOTATE_ROWS_EVENT 2 - - pthread_handler_t handle_slave_io(void *arg) - { ... - request_dump(mysql, ...); - ... - } - - int request_dump(MYSQL* mysql, ...) - { ... - if (opt_log_slave_updates && - mi->io_thd->variables.binlog_annotate_rows_events) - binlog_flags|= BINLOG_SEND_ANNOTATE_ROWS_EVENT; - ... - int2store(buf + 4, binlog_flags); - ... - simple_command(mysql, COM_BINLOG_DUMP, buf, ...); - ... - } - -NOTE. mysqlbinlog, when remotely requesting BINLOG_DUMP by calling the -simple_command() function, should also use this flag if it wants (in case -of the --print-annotate-rows-events option set) to recieve Annotate_rows -events. +Note. We add this new command and don't use COM_BINLOG_DUMP to avoid possible +conflicts with MySQL/Sun. 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -338,10 +323,4 @@ to internals(a)lists.mysql.com about this and suggesting to reserve the event number. -Also we should notice the introduction of the BINLOG_SEND_ANNOTATE_ROWS_EVENT -flag taking into account that MySQL/Sun may also introduce a flag with the -same value to be used in the request_dump-mysql_binlog_send interface. -But this is mainly the question of merging: if a conflict concerning this -flag occur, we may simply change the BINLOG_SEND_ANNOTATE_ROWS_EVENT value -(this does not require additional changes in the code). -=-=(Alexi - Sun, 20 Dec 2009, 16:00)=-=- Low Level Design modified. --- /tmp/wklog.47.old.19667 2009-12-20 14:00:56.000000000 +0000 +++ /tmp/wklog.47.new.19667 2009-12-20 14:00:56.000000000 +0000 @@ -196,6 +196,11 @@ ... } +NOTE. mysqlbinlog, when remotely requesting BINLOG_DUMP by calling the +simple_command() function, should also use this flag if it wants (in case +of the --print-annotate-rows-events option set) to recieve Annotate_rows +events. + 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -212,8 +217,7 @@ Log_event::read_log_event(&log, packet, ...); ... if ((*packet)[EVENT_TYPE_OFFSET + 1] != ANNOTATE_ROWS_EVENT || - flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT || - thd->server_id == 0 /* slave == mysqlbinlog */ ) + flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT) { my_net_write(net, packet->ptr(), packet->length()); } -=-=(Alexi - Sun, 20 Dec 2009, 13:14)=-=- Low Level Design modified. --- /tmp/wklog.47.old.11350 2009-12-20 13:14:04.000000000 +0200 +++ /tmp/wklog.47.new.11350 2009-12-20 13:14:04.000000000 +0200 @@ -282,23 +282,18 @@ Annotate_rows_log_event* m_annotate_event; }; -When the saved Annotate_rows object may be deleted? When all corresponding -Rows events will be processed, i.e. before processing the first non-Rows -event (note that Annotate_rows object resides in the binary log *after* -the (possible) 'BEGIN' Query event which accompanies the rows events; note -also that this deletion is adjusted with the case when some or all -corresponding Rows events are filtered out by replicate filter rules): +The saved Annotate_rows object should be deleted when all corresponding +Rows events will be processed: int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev= next_event(rli); ... - if (rli->get_annotate_event() && !IS_RBR_EVENT_TYPE(ev->get_type_code())) - rli->free_annotate_event(); - apply_event_and_update_pos(ev, ...); - if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) + if (rli->get_annotate_event() && is_last_rows_event(ev)) + rli->free_annotate_event(); + else if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) rli->set_annotate_event((Annotate_rows_log_event*) ev); else if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; @@ -307,10 +302,21 @@ where - #define IS_RBR_EVENT_TYPE(type) ( (type) == TABLE_MAP_EVENT || \ - (type) == WRITE_ROWS_EVENT || \ + bool is_last_rows_event(Log_event* ev) + { + Log_event_type type= ev->get_type_code(); + if (IS_ROWS_EVENT_TYPE(type)) + { + Rows_log_event* rows= (Rows_log_event*)ev; + return rows->get_flags(Rows_log_event::STMT_END_F); + } + + return 0; + } + + #define IS_ROWS_EVENT_TYPE(type) ((type) == WRITE_ROWS_EVENT || \ (type) == UPDATE_ROWS_EVENT || \ - (type) == DELETE_ROWS_EVENT ) + (type) == DELETE_ROWS_EVENT) 8. General remarks ~~~~~~~~~~~~~~~~~~ -=-=(Alexi - Sun, 20 Dec 2009, 09:29)=-=- Low Level Design modified. --- /tmp/wklog.47.old.32726 2009-12-20 07:29:56.000000000 +0000 +++ /tmp/wklog.47.new.32726 2009-12-20 07:29:56.000000000 +0000 @@ -56,7 +56,7 @@ 0000006A | 00 | 0 - RESERVED [27] ..... ... 00000081 | 00 | 0 - RESERVED [50] - 00000082 | 00 | 0 - ANNOTATE_RBR_EVENT [51] + 00000082 | 00 | 0 - ANNOTATE_ROWS_EVENT [51] ************************ 2. Outline of Annotate_rows event behavior -=-=(Alexi - Sat, 19 Dec 2009, 16:10)=-=- Low Level Design modified. --- /tmp/wklog.47.old.16051 2009-12-19 16:10:48.000000000 +0200 +++ /tmp/wklog.47.new.16051 2009-12-19 16:10:48.000000000 +0200 @@ -253,7 +253,7 @@ thd query to that of the described by the event, i.e. to the query which caused the subsequent Rows events (see "How Master writes Annotate_rows events to the binary log" to follow what happens further when the subsequent -Rows events is applied): +Rows events are applied): int Annotate_rows_log_event::do_apply_event(...) { -=-=(Alexi - Sat, 19 Dec 2009, 16:02)=-=- High-Level Specification modified. --- /tmp/wklog.47.old.15695 2009-12-19 16:02:33.000000000 +0200 +++ /tmp/wklog.47.new.15695 2009-12-19 16:02:33.000000000 +0200 @@ -12,7 +12,7 @@ post-header and contains the query text in its data part. Example: ************************ - ANNOTATE_RBR_EVENT + ANNOTATE_ROWS_EVENT ************************ 00000220 | B6 A0 2C 4B | time_when = 1261215926 00000224 | 33 | event_type = 51 -=-=(Alexi - Sat, 19 Dec 2009, 15:58)=-=- Low Level Design modified. --- /tmp/wklog.47.old.15437 2009-12-19 13:58:12.000000000 +0000 +++ /tmp/wklog.47.new.15437 2009-12-19 13:58:12.000000000 +0000 @@ -1 +1,337 @@ +Content +~~~~~~~ + 1. Annotate_rows event number + 2. Outline of Annotate_rows event behavior + 3. How Master writes Annotate_rows events to the binary log + 4. How slave treats replicate-annotate-rows-events option + 5. How slave IO thread requests Annotate_rows events + 6. How master executes the request + 7. How slave SQL thread processes Annotate_rows events + 8. General remarks + +1. Annotate_rows event number +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +To avoid possible event numbers conflict with MySQL/Sun, we leave a gap +between the last MySQL event number and the Annotate_rows event number: + + enum Log_event_type + { ... + INCIDENT_EVENT= 26, + // New MySQL event numbers are to be added here + MYSQL_EVENTS_END, + + MARIA_EVENTS_BEGIN= 51, + // New Maria event numbers start from here + ANNOTATE_ROWS_EVENT= 51, + + ENUM_END_EVENT + }; + +together with the corresponding extension of 'post_header_len' array in the +Format description event. (This extension does not affect the compatibility +of the binary log). Here is how Format description event looks like with +this extension: + + ************************ + FORMAT_DESCRIPTION_EVENT + ************************ + 00000004 | A1 A0 2C 4B | time_when = 1261215905 + 00000008 | 0F | event_type = 15 + 00000009 | 64 00 00 00 | server_id = 100 + 0000000D | 7F 00 00 00 | event_len = 127 + 00000011 | 83 00 00 00 | log_pos = 00000083 + 00000015 | 01 00 | flags = LOG_EVENT_BINLOG_IN_USE_F + ------------------------ + 00000017 | 04 00 | binlog_ver = 4 + 00000019 | 35 2E 32 2E | server_ver = 5.2.0-MariaDB-alpha-debug-log + ..... ... + 0000004B | A1 A0 2C 4B | time_created = 1261215905 + 0000004F | 13 | common_header_len = 19 + ------------------------ + post_header_len + ------------------------ + 00000050 | 38 | 56 - START_EVENT_V3 [1] + ..... ... + 00000069 | 02 | 2 - INCIDENT_EVENT [26] + 0000006A | 00 | 0 - RESERVED [27] + ..... ... + 00000081 | 00 | 0 - RESERVED [50] + 00000082 | 00 | 0 - ANNOTATE_RBR_EVENT [51] + ************************ + +2. Outline of Annotate_rows event behavior +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Each Annotate_rows_log_event object has two private members describing the +corresponding query: + + char *m_query_txt; + uint m_query_len; + +When the object is created for writing to a binary log, this query is taken +from 'thd' (for short, below we omit the 'Annotate_rows_log_event::' prefix +as well as other implementation details): + + Annotate_rows_log_event(THD *thd) + { + m_query_txt = thd->query(); + m_query_len = thd->query_length(); + } + +When the object is read from a binary log, the query is taken from the buffer +containing the binary log representation of the event (this buffer is allocated +in Log_event object from which all Log events are derived): + + Annotate_rows_log_event(char *buf, uint event_len, + Format_description_log_event *desc) + { + m_query_len = event_len - desc->common_header_len; + m_query_txt = buf + desc->common_header_len; + } + +The events are written to the binary log by the Log_event::write() member +which calls virtual write_data_header() and write_data_body() members +("data header" and "post header" are synonym in replication terminology). +In our case, data header is empty and data body is just the query: + + bool write_data_body(IO_CACHE *file) + { + return my_b_safe_write(file, (uchar*) m_query_txt, m_query_len); + } + +Printing the event is just printing the query: + + void Annotate_rows_log_event::print(FILE *file, PRINT_EVENT_INFO *pinfo) + { + my_b_printf(&pinfo->head_cache, "\tQuery: `%s`\n", m_query_txt); + } + +3. How Master writes Annotate_rows events to the binary log +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The event is written to the binary log just before the group of Table_map +events which precede corresponding Rows events (one query may generate +several Table map events in the binary log, but the corresponding +Annotate_rows event must be written only once before the first Table map +event; hence the boolean variable 'with_annotate' below): + + int write_locked_table_maps(THD *thd) + { ... + bool with_annotate= thd->variables.binlog_annotate_rows_events; + ... + for (uint i= 0; i < ... <number of tables> ...; ++i) + { ... + thd->binlog_write_table_map(table, ..., with_annotate); + with_annotate= 0; // write Annotate_event not more than once + ... + } + ... + } + + int THD::binlog_write_table_map(TABLE *table, ..., bool with_annotate) + { ... + Table_map_log_event the_event(...); + ... + if (with_annotate) + { + Annotate_rows_log_event anno(this); + mysql_bin_log.write(&anno); + } + + mysql_bin_log.write(&the_event); + ... + } + +4. How slave treats replicate-annotate-rows-events option +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The replicate-annotate-rows-events option is treated just as the session +value of the binlog_annotate_rows_events variable for the slave IO and +SQL threads. This setting is done during initialization of these threads: + + pthread_handler_t handle_slave_io(void *arg) + { + THD *thd= new THD; + ... + init_slave_thread(thd, SLAVE_THD_IO); + ... + } + + pthread_handler_t handle_slave_sql(void *arg) + { + THD *thd= new THD; + ... + init_slave_thread(thd, SLAVE_THD_SQL); + ... + } + + int init_slave_thread(THD* thd, SLAVE_THD_TYPE thd_type) + { ... + thd->variables.binlog_annotate_rows_events= + opt_replicate_annotate_rows_events; + ... + } + +5. How slave IO thread requests Annotate_rows events +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +When requesting an event, the slave should inform the master whether +it should send Annotate_rows events or not. To that end we add a new +BINLOG_SEND_ANNOTATE_ROWS_EVENT flag used when requesting an event: + + #define BINLOG_DUMP_NON_BLOCK 1 + #define BINLOG_SEND_ANNOTATE_ROWS_EVENT 2 + + pthread_handler_t handle_slave_io(void *arg) + { ... + request_dump(mysql, ...); + ... + } + + int request_dump(MYSQL* mysql, ...) + { ... + if (opt_log_slave_updates && + mi->io_thd->variables.binlog_annotate_rows_events) + binlog_flags|= BINLOG_SEND_ANNOTATE_ROWS_EVENT; + ... + int2store(buf + 4, binlog_flags); + ... + simple_command(mysql, COM_BINLOG_DUMP, buf, ...); + ... + } + +6. How master executes the request +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + case COM_BINLOG_DUMP: + { ... + flags= uint2korr(packet + 4); + ... + mysql_binlog_send(thd, ..., flags); + ... + } + + void mysql_binlog_send(THD* thd, ..., ushort flags) + { ... + Log_event::read_log_event(&log, packet, ...); + ... + if ((*packet)[EVENT_TYPE_OFFSET + 1] != ANNOTATE_ROWS_EVENT || + flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT || + thd->server_id == 0 /* slave == mysqlbinlog */ ) + { + my_net_write(net, packet->ptr(), packet->length()); + } + ... + } + +7. How slave SQL thread processes Annotate_rows events +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The slave processes each recieved event by "applying" it, i.e. by +calling the Log_event::apply_event() function which in turn calls +the virtual do_apply_event() member specific for each type of the +event. + + int exec_relay_log_event(THD* thd, Relay_log_info* rli) + { ... + Log_event *ev = next_event(rli); + ... + apply_event_and_update_pos(ev, ...); + + if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) + delete ev; + ... + } + + int apply_event_and_update_pos(Log_event *ev, ...) + { ... + ev->apply_event(...); + ... + } + + int Log_event::apply_event(...) + { + return do_apply_event(...); + } + +What does it mean to "apply" an Annotate_rows event? It means to set current +thd query to that of the described by the event, i.e. to the query which +caused the subsequent Rows events (see "How Master writes Annotate_rows +events to the binary log" to follow what happens further when the subsequent +Rows events is applied): + + int Annotate_rows_log_event::do_apply_event(...) + { + thd->set_query(m_query_txt, m_query_len); + } + +NOTE. I am not sure, but possibly current values of thd->query and +thd->query_length should be saved before calling set_query() and to be +restored on the Annotate_rows_log_event object deletion. +Is it really needed ? + +After calling this do_apply_event() function we may not delete the +Annotate_rows_log_event object immediatedly (see exec_relay_log_event() +above) because thd->query now points to the string inside this object. +We may keep the pointer to this object in the Relay_log_info: + + class Relay_log_info + { + public: + ... + void set_annotate_event(Annotate_rows_log_event*); + Annotate_rows_log_event* get_annotate_event(); + void free_annotate_event(); + ... + private: + Annotate_rows_log_event* m_annotate_event; + }; + +When the saved Annotate_rows object may be deleted? When all corresponding +Rows events will be processed, i.e. before processing the first non-Rows +event (note that Annotate_rows object resides in the binary log *after* +the (possible) 'BEGIN' Query event which accompanies the rows events; note +also that this deletion is adjusted with the case when some or all +corresponding Rows events are filtered out by replicate filter rules): + + int exec_relay_log_event(THD* thd, Relay_log_info* rli) + { ... + Log_event *ev= next_event(rli); + ... + if (rli->get_annotate_event() && !IS_RBR_EVENT_TYPE(ev->get_type_code())) + rli->free_annotate_event(); + + apply_event_and_update_pos(ev, ...); + + if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) + rli->set_annotate_event((Annotate_rows_log_event*) ev); + else if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) + delete ev; + ... + } + +where + + #define IS_RBR_EVENT_TYPE(type) ( (type) == TABLE_MAP_EVENT || \ + (type) == WRITE_ROWS_EVENT || \ + (type) == UPDATE_ROWS_EVENT || \ + (type) == DELETE_ROWS_EVENT ) + +8. General remarks +~~~~~~~~~~~~~~~~~~ +Kristian noticed that introducing new log event type should be coordinated +somehow with MySQL/Sun: + + Kristian: The numeric code for this event must be assigned carefully. + It should be coordinated with MySQL/Sun, otherwise we can get into a + situation where MySQL uses the same numeric code for one event that + MariaDB uses for ANNOTATE_ROWS_EVENT, which would make merging the two + impossible. + Alex: I reserved about 20 numbers not to have possible conflicts + with MySQL. + Kristian: Still, I think it would be appropriate to send a polite email + to internals(a)lists.mysql.com about this and suggesting to reserve the + event number. + +Also we should notice the introduction of the BINLOG_SEND_ANNOTATE_ROWS_EVENT +flag taking into account that MySQL/Sun may also introduce a flag with the +same value to be used in the request_dump-mysql_binlog_send interface. +But this is mainly the question of merging: if a conflict concerning this +flag occur, we may simply change the BINLOG_SEND_ANNOTATE_ROWS_EVENT value +(this does not require additional changes in the code). -=-=(Alexi - Sat, 19 Dec 2009, 15:41)=-=- High-Level Specification modified. --- /tmp/wklog.47.old.14545 2009-12-19 15:41:21.000000000 +0200 +++ /tmp/wklog.47.new.14545 2009-12-19 15:41:21.000000000 +0200 @@ -1,122 +1,107 @@ -First suggestion: - -> I think for this we would actually need a new binlog event type -> (Comment_log_event?). Unless we want to log an empty statement Query_log_event -> containing only a comment (a bit of a hack). - -New server option -~~~~~~~~~~~~~~~~~ - --binlog-annotate-rows-events - -Setting this option makes RBR (rows-) events in the binary log to be -preceded by Annotate rows events (see below). The corresponding -'binlog_annotate_rows_events' system variable is dynamic and has both -global and session values. Default global value is OFF. - -Note. Session values are usefull to make it possible to annotate only - some selected statements: +Content +~~~~~~~ + 1. Annotate_rows_log_event + 2. Server option: --binlog-annotate-rows-events + 3. Server option: --replicate-annotate-rows-events + 4. mysqlbinlog option: --print-annotate-rows-events + 5. mysqlbinlog output + +1. Annotate_rows_log_event [ ANNOTATE_ROWS_EVENT ] +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Describes the query which caused the corresponding rows events. Has empty +post-header and contains the query text in its data part. Example: + + ************************ + ANNOTATE_RBR_EVENT + ************************ + 00000220 | B6 A0 2C 4B | time_when = 1261215926 + 00000224 | 33 | event_type = 51 + 00000225 | 64 00 00 00 | server_id = 100 + 00000229 | 36 00 00 00 | event_len = 54 + 0000022D | 56 02 00 00 | log_pos = 00000256 + 00000231 | 00 00 | flags = <none> + ------------------------ + 00000233 | 49 4E 53 45 | query = "INSERT INTO t1 VALUES (1), (2), (3)" + 00000237 | 52 54 20 49 | + 0000023B | 4E 54 4F 20 | + 0000023F | 74 31 20 56 | + 00000243 | 41 4C 55 45 | + 00000247 | 53 20 28 31 | + 0000024B | 29 2C 20 28 | + 0000024F | 32 29 2C 20 | + 00000253 | 28 33 29 | + ************************ + +In binary log, Annotate_rows event follows the (possible) 'BEGIN' Query event +and precedes the first of Table map events which accompany the corresponding +rows events. (See example in the "mysqlbinlog output" section below.) + +2. Server option: --binlog-annotate-rows-events +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Tells the master to write Annotate_rows events to the binary log. + + * Variable Name: binlog_annotate_rows_events + * Scope: Global & Session + * Access Type: Dynamic + * Data Type: bool + * Default Value: OFF +NOTE. Session values allows to annotate only some selected statements: ... SET SESSION binlog_annotate_rows_events=ON; ... statements to be annotated ... SET SESSION binlog_annotate_rows_events=OFF; ... statements not to be annotated ... -New binlog event type -~~~~~~~~~~~~~~~~~~~~~ - Annotate_rows_log_event [ ANNOTATE_ROWS_EVENT ] - -Describes the query which caused the corresponding rows event. In binary log, -precedes each Table_map_log_event. Contains empty post-header and the query -text in its data part. - -The numeric code for this event must be assigned carefully. It should be -coordinated with MySQL/Sun, otherwise we can get into a situation where MySQL -uses the same numeric code for one event that MariaDB uses for -ANNOTATE_ROWS_EVENT, which would make merging the two impossible. - -Example: - - ... - ************************ - ANNOTATE_ROWS_EVENT [51] - ************************ - 000000C7 | 54 1B 12 4B | time_when = 1259477844 - 000000CB | 33 | event_type = 51 - 000000CC | 64 00 00 00 | server_id = 100 - 000000D0 | 2C 00 00 00 | event_len = 44 - 000000D4 | F3 00 00 00 | log_pos = 000000F3 - 000000D8 | 00 00 | flags = <none> - ------------------------ - 000000DA | 69 6E 73 65 | query = "insert into t1 values (1)" - 000000DE | 72 74 20 69 | - 000000E2 | 6E 74 6F 20 | - 000000E6 | 74 31 20 76 | - 000000EA | 61 6C 75 65 | - 000000EE | 73 20 28 31 | - 000000F2 | 29 | - ************************ - TABLE_MAP_EVENT [19] - ************************ - 000000F3 | 54 1B 12 4B | time_when = 1259477844 - 000000F7 | 13 | event_type = 19 - 000000F8 | 64 00 00 00 | server_id = 100 - 000000FC | 29 00 00 00 | event_len = 41 - 00000100 | 1C 01 00 00 | log_pos = 0000011C - 00000104 | 00 00 | flags = <none> - ------------------------ - ... - ************************ - WRITE_ROWS_EVENT [23] - ************************ - 0000011C | 54 1B 12 4B | time_when = 1259477844 - 00000120 | 17 | event_type = 23 - 00000121 | 64 00 00 00 | server_id = 100 - 00000125 | 22 00 00 00 | event_len = 34 - 00000129 | 3E 01 00 00 | log_pos = 0000013E - 0000012D | 10 00 | flags = LOG_EVENT_UPDATE_TABLE_MAP_VERSION_F - ------------------------ - 0000012F | 0F 00 00 00 | table_id = 15 - ... - -New mysqlbinlog option -~~~~~~~~~~~~~~~~~~~~~~ - --print-annotate-rows-events - -With this option, mysqlbinlog prints the content of Annotate-rows -events (if the binary log does contain them). Without this option -(i.e. by default), mysqlbinlog skips Annotate rows events. - - -mysqlbinlog output -~~~~~~~~~~~~~~~~~~ -Something like this: +3. Server option: --replicate-annotate-rows-events +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Tells the slave to reproduce Annotate_rows events recieved from the master +in its own binary log (sensible only in pair with log-slave-updates option). + + * Variable Name: replicate_annotate_rows_events + * Scope: Global + * Access Type: Read only + * Data Type: bool + * Default Value: OFF + +NOTE. Why do we additionally need this 'replicate' option? Why not to make +the slave to reproduce this events when its binlog-annotate-rows-events +global value is ON? Well, because, for example, we may want to configure +the slave which should reproduce Annotate_rows events but has global +binlog-annotate-rows-events = OFF meaning this to be the default value for +the client threads (see also "How slave treats replicate-annotate-rows-events +option" in LLD part). + +4. mysqlbinlog option: --print-annotate-rows-events +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +With this option, mysqlbinlog prints the content of Annotate_rows events (if +the binary log does contain them). Without this option (i.e. by default), +mysqlbinlog skips Annotate_rows events. +5. mysqlbinlog output +~~~~~~~~~~~~~~~~~~~~~ +With --print-annotate-rows-events, mysqlbinlog outputs Annotate_rows events +in a form like this: ... - # at 199 - # at 243 - # at 284 - #091129 9:57:24 server id 100 end_log_pos 243 Query: `insert into t1 values -(1)` - #091129 9:57:24 server id 100 end_log_pos 284 Table_map: `test`.`t1` mapped -to number 15 - #091129 9:57:24 server id 100 end_log_pos 318 Write_rows: table id 15 + # at 1646 + #091219 12:45:26 server id 100 end_log_pos 1714 Query thread_id=1 +exec_time=0 error_code=0 + SET TIMESTAMP=1261215926/*!*/; + BEGIN + /*!*/; + # at 1714 + # at 1812 + # at 1853 + # at 1894 + # at 1938 + #091219 12:45:26 server id 100 end_log_pos 1812 Query: `DELETE t1, t2 FROM +t1 INNER JOIN t2 INNER JOIN t3 WHERE t1.a=t2.a AND t2.a=t3.a` + #091219 12:45:26 server id 100 end_log_pos 1853 Table_map: `test`.`t1` +mapped to number 16 + #091219 12:45:26 server id 100 end_log_pos 1894 Table_map: `test`.`t2` +mapped to number 17 + #091219 12:45:26 server id 100 end_log_pos 1938 Delete_rows: table id 16 + #091219 12:45:26 server id 100 end_log_pos 1982 Delete_rows: table id 17 flags: STMT_END_F - - BINLOG ' - VBsSSzNkAAAALAAAAPMAAAAAAGluc2VydCBpbnRvIHQxIHZhbHVlcyAoMSk= - VBsSSxNkAAAAKQAAABwBAAAAAA8AAAAAAAAABHRlc3QAAnQxAAEDAAE= - VBsSSxdkAAAAIgAAAD4BAAAQAA8AAAAAAAEAAf/+AQAAAA== - '/*!*/; - ### INSERT INTO test.t1 - ### SET - ### @1=1 /* INT meta=0 nullable=1 is_null=0 */ ... -When master sends Annotate rows events -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -1. Master always sends Annotate_rows events to mysqlbinlog (in - remote case). -2. Master sends Annotate_rows events to a slave only if the slave has - both log-slave-updates and binlog-annotate-rows-events options set. - -=-=(Bothorsen - Fri, 18 Dec 2009, 16:22)=-=- Add estimation time. Worked 5 hours and estimate 35 hours remain (original estimate increased by 5 hours). -=-=(Bothorsen - Fri, 18 Dec 2009, 16:16)=-=- This is the work done on this patch so far. Most of it done by Alex. Worked 15 hours and estimate 035 hours remain (original estimate increased by 50 hours). ------------------------------------------------------------ -=-=(View All Progress Notes, 20 total)=-=- http://askmonty.org/worklog/index.pl?tid=47&nolimit=1 DESCRIPTION: Store in binlog (and show in mysqlbinlog output) texts of statements that caused RBR events This is needed for (list from Monty): - Easier to understand why updates happened - Would make it easier to find out where in application things went wrong (as you can search for exact strings) - Allow one to filter things based on comments in the statement. The cost of this can be that the binlog will be approximately 2x in size (especially insert of big blob's would be a bit painful), so this should be an optional feature. HIGH-LEVEL SPECIFICATION: Content ~~~~~~~ 1. Annotate_rows_log_event 2. Server option: --binlog-annotate-rows-events 3. Server option: --replicate-annotate-rows-events 4. mysqlbinlog option: --print-annotate-rows-events 5. mysqlbinlog output 1. Annotate_rows_log_event [ ANNOTATE_ROWS_EVENT ] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Describes the query which caused the corresponding rows events. Has empty post-header and contains the query text in its data part. Example: ************************ ANNOTATE_ROWS_EVENT ************************ 00000220 | B6 A0 2C 4B | time_when = 1261215926 00000224 | 33 | event_type = 51 00000225 | 64 00 00 00 | server_id = 100 00000229 | 36 00 00 00 | event_len = 54 0000022D | 56 02 00 00 | log_pos = 00000256 00000231 | 00 00 | flags = <none> ------------------------ 00000233 | 49 4E 53 45 | query = "INSERT INTO t1 VALUES (1), (2), (3)" 00000237 | 52 54 20 49 | 0000023B | 4E 54 4F 20 | 0000023F | 74 31 20 56 | 00000243 | 41 4C 55 45 | 00000247 | 53 20 28 31 | 0000024B | 29 2C 20 28 | 0000024F | 32 29 2C 20 | 00000253 | 28 33 29 | ************************ In binary log, Annotate_rows event follows the (possible) 'BEGIN' Query event and precedes the first of Table map events which accompany the corresponding rows events. (See example in the "mysqlbinlog output" section below.) 2. Server option: --binlog-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the master to write Annotate_rows events to the binary log. * Variable Name: binlog_annotate_rows_events * Scope: Global & Session * Access Type: Dynamic * Data Type: bool * Default Value: OFF NOTE. Session values allows to annotate only some selected statements: ... SET SESSION binlog_annotate_rows_events=ON; ... statements to be annotated ... SET SESSION binlog_annotate_rows_events=OFF; ... statements not to be annotated ... 3. Server option: --replicate-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the slave to reproduce Annotate_rows events recieved from the master in its own binary log (sensible only in pair with log-slave-updates option). * Variable Name: replicate_annotate_rows_events * Scope: Global * Access Type: Read only * Data Type: bool * Default Value: OFF NOTE. Why do we additionally need this 'replicate' option? Why not to make the slave to reproduce this events when its binlog-annotate-rows-events global value is ON? Well, because, for example, we may want to configure the slave which should reproduce Annotate_rows events but has global binlog-annotate-rows-events = OFF meaning this to be the default value for the client threads (see also "How slave treats replicate-annotate-rows-events option" in LLD part). 4. mysqlbinlog option: --print-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ With this option, mysqlbinlog prints the content of Annotate_rows events (if the binary log does contain them). Without this option (i.e. by default), mysqlbinlog skips Annotate_rows events. 5. mysqlbinlog output ~~~~~~~~~~~~~~~~~~~~~ With --print-annotate-rows-events, mysqlbinlog outputs Annotate_rows events in a form like this: ... # at 1646 #091219 12:45:26 server id 100 end_log_pos 1714 Query thread_id=1 exec_time=0 error_code=0 SET TIMESTAMP=1261215926/*!*/; BEGIN /*!*/; # at 1714 # at 1812 # at 1853 # at 1894 # at 1938 #091219 12:45:26 server id 100 end_log_pos 1812 Query: `DELETE t1, t2 FROM t1 INNER JOIN t2 INNER JOIN t3 WHERE t1.a=t2.a AND t2.a=t3.a` #091219 12:45:26 server id 100 end_log_pos 1853 Table_map: `test`.`t1` mapped to number 16 #091219 12:45:26 server id 100 end_log_pos 1894 Table_map: `test`.`t2` mapped to number 17 #091219 12:45:26 server id 100 end_log_pos 1938 Delete_rows: table id 16 #091219 12:45:26 server id 100 end_log_pos 1982 Delete_rows: table id 17 flags: STMT_END_F ... LOW-LEVEL DESIGN: Content ~~~~~~~ 1. Annotate_rows event number 2. Outline of Annotate_rows event behavior 3. How Master writes Annotate_rows events to the binary log 4. How slave treats replicate-annotate-rows-events option 5. How slave IO thread requests Annotate_rows events 6. How master executes the request 7. How slave SQL thread processes Annotate_rows events 8. General remarks 1. Annotate_rows event number ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To avoid possible event numbers conflict with MySQL/Sun, we leave a gap between the last MySQL event number and the Annotate_rows event number: enum Log_event_type { ... INCIDENT_EVENT= 26, // New MySQL event numbers are to be added here MYSQL_EVENTS_END, MARIA_EVENTS_BEGIN= 51, // New Maria event numbers start from here ANNOTATE_ROWS_EVENT= 51, ENUM_END_EVENT }; together with the corresponding extension of 'post_header_len' array in the Format description event. (This extension does not affect the compatibility of the binary log). Here is how Format description event looks like with this extension: ************************ FORMAT_DESCRIPTION_EVENT ************************ 00000004 | A1 A0 2C 4B | time_when = 1261215905 00000008 | 0F | event_type = 15 00000009 | 64 00 00 00 | server_id = 100 0000000D | 7F 00 00 00 | event_len = 127 00000011 | 83 00 00 00 | log_pos = 00000083 00000015 | 01 00 | flags = LOG_EVENT_BINLOG_IN_USE_F ------------------------ 00000017 | 04 00 | binlog_ver = 4 00000019 | 35 2E 32 2E | server_ver = 5.2.0-MariaDB-alpha-debug-log ..... ... 0000004B | A1 A0 2C 4B | time_created = 1261215905 0000004F | 13 | common_header_len = 19 ------------------------ post_header_len ------------------------ 00000050 | 38 | 56 - START_EVENT_V3 [1] ..... ... 00000069 | 02 | 2 - INCIDENT_EVENT [26] 0000006A | 00 | 0 - RESERVED [27] ..... ... 00000081 | 00 | 0 - RESERVED [50] 00000082 | 00 | 0 - ANNOTATE_ROWS_EVENT [51] ************************ 2. Outline of Annotate_rows event behavior ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Each Annotate_rows_log_event object has two private members describing the corresponding query: char *m_query_txt; uint m_query_len; When the object is created for writing to a binary log, this query is taken from 'thd' (for short, below we omit the 'Annotate_rows_log_event::' prefix as well as other implementation details): Annotate_rows_log_event(THD *thd) { m_query_txt = thd->query(); m_query_len = thd->query_length(); } When the object is read from a binary log, the query is taken from the buffer containing the binary log representation of the event (this buffer is allocated in Log_event object from which all Log events are derived): Annotate_rows_log_event(char *buf, uint event_len, Format_description_log_event *desc) { m_query_len = event_len - desc->common_header_len; m_query_txt = buf + desc->common_header_len; } The events are written to the binary log by the Log_event::write() member which calls virtual write_data_header() and write_data_body() members ("data header" and "post header" are synonym in replication terminology). In our case, data header is empty and data body is just the query: bool write_data_body(IO_CACHE *file) { return my_b_safe_write(file, (uchar*) m_query_txt, m_query_len); } Printing the event is just printing the query: void Annotate_rows_log_event::print(FILE *file, PRINT_EVENT_INFO *pinfo) { my_b_printf(&pinfo->head_cache, "\tQuery: `%s`\n", m_query_txt); } 3. How Master writes Annotate_rows events to the binary log ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The event is written to the binary log just before the group of Table_map events which precede corresponding Rows events (one query may generate several Table map events in the binary log, but the corresponding Annotate_rows event must be written only once before the first Table map event; hence the boolean variable 'with_annotate' below): int write_locked_table_maps(THD *thd) { ... bool with_annotate= thd->variables.binlog_annotate_rows_events; ... for (uint i= 0; i < ... <number of tables> ...; ++i) { ... thd->binlog_write_table_map(table, ..., with_annotate); with_annotate= 0; // write Annotate_event not more than once ... } ... } int THD::binlog_write_table_map(TABLE *table, ..., bool with_annotate) { ... Table_map_log_event the_event(...); ... if (with_annotate) { Annotate_rows_log_event anno(this); mysql_bin_log.write(&anno); } mysql_bin_log.write(&the_event); ... } 4. How slave treats replicate-annotate-rows-events option ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The replicate-annotate-rows-events option is treated just as the session value of the binlog_annotate_rows_events variable for the slave IO and SQL threads. This setting is done during initialization of these threads: pthread_handler_t handle_slave_io(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_IO); ... } pthread_handler_t handle_slave_sql(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_SQL); ... } int init_slave_thread(THD* thd, SLAVE_THD_TYPE thd_type) { ... thd->variables.binlog_annotate_rows_events= opt_replicate_annotate_rows_events; ... } 5. How slave IO thread requests Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If the replicate-annotate-rows-events option is not set on a slave, there is no need for master to send Annotate_rows events to this slave. The slave (or mysqlbinlog in remote case), before requesting binlog dump via the COM_BINLOG_DUMP command, informs the master whether it should send these events by executing the newly added COM_BINLOG_DUMP_OPTIONS_EXT server command: case COM_BINLOG_DUMP_OPTIONS_EXT: thd->binlog_dump_flags_ext= packet[0]; my_ok(thd); break; Note. We add this new command and don't use COM_BINLOG_DUMP to avoid possible conflicts with MySQL/Sun. 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ case COM_BINLOG_DUMP: { ... flags= uint2korr(packet + 4); ... mysql_binlog_send(thd, ..., flags); ... } void mysql_binlog_send(THD* thd, ..., ushort flags) { ... Log_event::read_log_event(&log, packet, ...); ... if ((*packet)[EVENT_TYPE_OFFSET + 1] != ANNOTATE_ROWS_EVENT || flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT) { my_net_write(net, packet->ptr(), packet->length()); } ... } 7. How slave SQL thread processes Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The slave processes each recieved event by "applying" it, i.e. by calling the Log_event::apply_event() function which in turn calls the virtual do_apply_event() member specific for each type of the event. int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev = next_event(rli); ... apply_event_and_update_pos(ev, ...); if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } int apply_event_and_update_pos(Log_event *ev, ...) { ... ev->apply_event(...); ... } int Log_event::apply_event(...) { return do_apply_event(...); } What does it mean to "apply" an Annotate_rows event? It means to set current thd query to that of the described by the event, i.e. to the query which caused the subsequent Rows events (see "How Master writes Annotate_rows events to the binary log" to follow what happens further when the subsequent Rows events are applied): int Annotate_rows_log_event::do_apply_event(...) { thd->set_query(m_query_txt, m_query_len); } NOTE. I am not sure, but possibly current values of thd->query and thd->query_length should be saved before calling set_query() and to be restored on the Annotate_rows_log_event object deletion. Is it really needed ? After calling this do_apply_event() function we may not delete the Annotate_rows_log_event object immediatedly (see exec_relay_log_event() above) because thd->query now points to the string inside this object. We may keep the pointer to this object in the Relay_log_info: class Relay_log_info { public: ... void set_annotate_event(Annotate_rows_log_event*); Annotate_rows_log_event* get_annotate_event(); void free_annotate_event(); ... private: Annotate_rows_log_event* m_annotate_event; }; The saved Annotate_rows object should be deleted when all corresponding Rows events will be processed: int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev= next_event(rli); ... apply_event_and_update_pos(ev, ...); if (rli->get_annotate_event() && is_last_rows_event(ev)) rli->free_annotate_event(); else if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) rli->set_annotate_event((Annotate_rows_log_event*) ev); else if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } where bool is_last_rows_event(Log_event* ev) { Log_event_type type= ev->get_type_code(); if (IS_ROWS_EVENT_TYPE(type)) { Rows_log_event* rows= (Rows_log_event*)ev; return rows->get_flags(Rows_log_event::STMT_END_F); } return 0; } #define IS_ROWS_EVENT_TYPE(type) ((type) == WRITE_ROWS_EVENT || \ (type) == UPDATE_ROWS_EVENT || \ (type) == DELETE_ROWS_EVENT) 8. General remarks ~~~~~~~~~~~~~~~~~~ Kristian noticed that introducing new log event type should be coordinated somehow with MySQL/Sun: Kristian: The numeric code for this event must be assigned carefully. It should be coordinated with MySQL/Sun, otherwise we can get into a situation where MySQL uses the same numeric code for one event that MariaDB uses for ANNOTATE_ROWS_EVENT, which would make merging the two impossible. Alex: I reserved about 20 numbers not to have possible conflicts with MySQL. Kristian: Still, I think it would be appropriate to send a polite email to internals(a)lists.mysql.com about this and suggesting to reserve the event number. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Alexi): Store in binlog text of statements that caused RBR events (47)
by worklog-noreply＠askmonty.org 04 Feb '10

04 Feb '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Store in binlog text of statements that caused RBR events CREATION DATE..: Sat, 15 Aug 2009, 23:48 SUPERVISOR.....: Monty IMPLEMENTOR....: COPIES TO......: Knielsen CATEGORY.......: Server-RawIdeaBin TASK ID........: 47 (http://askmonty.org/worklog/?tid=47) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 20 ESTIMATE.......: 35 (hours remain) ORIG. ESTIMATE.: 35 PROGRESS NOTES: -=-=(Alexi - Thu, 04 Feb 2010, 09:54)=-=- Low Level Design modified. --- /tmp/wklog.47.old.16174 2010-02-04 09:54:13.000000000 +0200 +++ /tmp/wklog.47.new.16174 2010-02-04 09:54:13.000000000 +0200 @@ -171,35 +171,20 @@ 5. How slave IO thread requests Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -When requesting an event, the slave should inform the master whether -it should send Annotate_rows events or not. To that end we add a new -BINLOG_SEND_ANNOTATE_ROWS_EVENT flag used when requesting an event: +If the replicate-annotate-rows-events option is not set on a slave, there +is no need for master to send Annotate_rows events to this slave. The slave +(or mysqlbinlog in remote case), before requesting binlog dump via the +COM_BINLOG_DUMP command, informs the master whether it should send these +events by executing the newly added COM_BINLOG_DUMP_OPTIONS_EXT server +command: + + case COM_BINLOG_DUMP_OPTIONS_EXT: + thd->binlog_dump_flags_ext= packet[0]; + my_ok(thd); + break; - #define BINLOG_DUMP_NON_BLOCK 1 - #define BINLOG_SEND_ANNOTATE_ROWS_EVENT 2 - - pthread_handler_t handle_slave_io(void *arg) - { ... - request_dump(mysql, ...); - ... - } - - int request_dump(MYSQL* mysql, ...) - { ... - if (opt_log_slave_updates && - mi->io_thd->variables.binlog_annotate_rows_events) - binlog_flags|= BINLOG_SEND_ANNOTATE_ROWS_EVENT; - ... - int2store(buf + 4, binlog_flags); - ... - simple_command(mysql, COM_BINLOG_DUMP, buf, ...); - ... - } - -NOTE. mysqlbinlog, when remotely requesting BINLOG_DUMP by calling the -simple_command() function, should also use this flag if it wants (in case -of the --print-annotate-rows-events option set) to recieve Annotate_rows -events. +Note. We add this new command and don't use COM_BINLOG_DUMP to avoid possible +conflicts with MySQL/Sun. 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -338,10 +323,4 @@ to internals(a)lists.mysql.com about this and suggesting to reserve the event number. -Also we should notice the introduction of the BINLOG_SEND_ANNOTATE_ROWS_EVENT -flag taking into account that MySQL/Sun may also introduce a flag with the -same value to be used in the request_dump-mysql_binlog_send interface. -But this is mainly the question of merging: if a conflict concerning this -flag occur, we may simply change the BINLOG_SEND_ANNOTATE_ROWS_EVENT value -(this does not require additional changes in the code). -=-=(Alexi - Sun, 20 Dec 2009, 16:00)=-=- Low Level Design modified. --- /tmp/wklog.47.old.19667 2009-12-20 14:00:56.000000000 +0000 +++ /tmp/wklog.47.new.19667 2009-12-20 14:00:56.000000000 +0000 @@ -196,6 +196,11 @@ ... } +NOTE. mysqlbinlog, when remotely requesting BINLOG_DUMP by calling the +simple_command() function, should also use this flag if it wants (in case +of the --print-annotate-rows-events option set) to recieve Annotate_rows +events. + 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -212,8 +217,7 @@ Log_event::read_log_event(&log, packet, ...); ... if ((*packet)[EVENT_TYPE_OFFSET + 1] != ANNOTATE_ROWS_EVENT || - flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT || - thd->server_id == 0 /* slave == mysqlbinlog */ ) + flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT) { my_net_write(net, packet->ptr(), packet->length()); } -=-=(Alexi - Sun, 20 Dec 2009, 13:14)=-=- Low Level Design modified. --- /tmp/wklog.47.old.11350 2009-12-20 13:14:04.000000000 +0200 +++ /tmp/wklog.47.new.11350 2009-12-20 13:14:04.000000000 +0200 @@ -282,23 +282,18 @@ Annotate_rows_log_event* m_annotate_event; }; -When the saved Annotate_rows object may be deleted? When all corresponding -Rows events will be processed, i.e. before processing the first non-Rows -event (note that Annotate_rows object resides in the binary log *after* -the (possible) 'BEGIN' Query event which accompanies the rows events; note -also that this deletion is adjusted with the case when some or all -corresponding Rows events are filtered out by replicate filter rules): +The saved Annotate_rows object should be deleted when all corresponding +Rows events will be processed: int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev= next_event(rli); ... - if (rli->get_annotate_event() && !IS_RBR_EVENT_TYPE(ev->get_type_code())) - rli->free_annotate_event(); - apply_event_and_update_pos(ev, ...); - if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) + if (rli->get_annotate_event() && is_last_rows_event(ev)) + rli->free_annotate_event(); + else if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) rli->set_annotate_event((Annotate_rows_log_event*) ev); else if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; @@ -307,10 +302,21 @@ where - #define IS_RBR_EVENT_TYPE(type) ( (type) == TABLE_MAP_EVENT || \ - (type) == WRITE_ROWS_EVENT || \ + bool is_last_rows_event(Log_event* ev) + { + Log_event_type type= ev->get_type_code(); + if (IS_ROWS_EVENT_TYPE(type)) + { + Rows_log_event* rows= (Rows_log_event*)ev; + return rows->get_flags(Rows_log_event::STMT_END_F); + } + + return 0; + } + + #define IS_ROWS_EVENT_TYPE(type) ((type) == WRITE_ROWS_EVENT || \ (type) == UPDATE_ROWS_EVENT || \ - (type) == DELETE_ROWS_EVENT ) + (type) == DELETE_ROWS_EVENT) 8. General remarks ~~~~~~~~~~~~~~~~~~ -=-=(Alexi - Sun, 20 Dec 2009, 09:29)=-=- Low Level Design modified. --- /tmp/wklog.47.old.32726 2009-12-20 07:29:56.000000000 +0000 +++ /tmp/wklog.47.new.32726 2009-12-20 07:29:56.000000000 +0000 @@ -56,7 +56,7 @@ 0000006A | 00 | 0 - RESERVED [27] ..... ... 00000081 | 00 | 0 - RESERVED [50] - 00000082 | 00 | 0 - ANNOTATE_RBR_EVENT [51] + 00000082 | 00 | 0 - ANNOTATE_ROWS_EVENT [51] ************************ 2. Outline of Annotate_rows event behavior -=-=(Alexi - Sat, 19 Dec 2009, 16:10)=-=- Low Level Design modified. --- /tmp/wklog.47.old.16051 2009-12-19 16:10:48.000000000 +0200 +++ /tmp/wklog.47.new.16051 2009-12-19 16:10:48.000000000 +0200 @@ -253,7 +253,7 @@ thd query to that of the described by the event, i.e. to the query which caused the subsequent Rows events (see "How Master writes Annotate_rows events to the binary log" to follow what happens further when the subsequent -Rows events is applied): +Rows events are applied): int Annotate_rows_log_event::do_apply_event(...) { -=-=(Alexi - Sat, 19 Dec 2009, 16:02)=-=- High-Level Specification modified. --- /tmp/wklog.47.old.15695 2009-12-19 16:02:33.000000000 +0200 +++ /tmp/wklog.47.new.15695 2009-12-19 16:02:33.000000000 +0200 @@ -12,7 +12,7 @@ post-header and contains the query text in its data part. Example: ************************ - ANNOTATE_RBR_EVENT + ANNOTATE_ROWS_EVENT ************************ 00000220 | B6 A0 2C 4B | time_when = 1261215926 00000224 | 33 | event_type = 51 -=-=(Alexi - Sat, 19 Dec 2009, 15:58)=-=- Low Level Design modified. --- /tmp/wklog.47.old.15437 2009-12-19 13:58:12.000000000 +0000 +++ /tmp/wklog.47.new.15437 2009-12-19 13:58:12.000000000 +0000 @@ -1 +1,337 @@ +Content +~~~~~~~ + 1. Annotate_rows event number + 2. Outline of Annotate_rows event behavior + 3. How Master writes Annotate_rows events to the binary log + 4. How slave treats replicate-annotate-rows-events option + 5. How slave IO thread requests Annotate_rows events + 6. How master executes the request + 7. How slave SQL thread processes Annotate_rows events + 8. General remarks + +1. Annotate_rows event number +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +To avoid possible event numbers conflict with MySQL/Sun, we leave a gap +between the last MySQL event number and the Annotate_rows event number: + + enum Log_event_type + { ... + INCIDENT_EVENT= 26, + // New MySQL event numbers are to be added here + MYSQL_EVENTS_END, + + MARIA_EVENTS_BEGIN= 51, + // New Maria event numbers start from here + ANNOTATE_ROWS_EVENT= 51, + + ENUM_END_EVENT + }; + +together with the corresponding extension of 'post_header_len' array in the +Format description event. (This extension does not affect the compatibility +of the binary log). Here is how Format description event looks like with +this extension: + + ************************ + FORMAT_DESCRIPTION_EVENT + ************************ + 00000004 | A1 A0 2C 4B | time_when = 1261215905 + 00000008 | 0F | event_type = 15 + 00000009 | 64 00 00 00 | server_id = 100 + 0000000D | 7F 00 00 00 | event_len = 127 + 00000011 | 83 00 00 00 | log_pos = 00000083 + 00000015 | 01 00 | flags = LOG_EVENT_BINLOG_IN_USE_F + ------------------------ + 00000017 | 04 00 | binlog_ver = 4 + 00000019 | 35 2E 32 2E | server_ver = 5.2.0-MariaDB-alpha-debug-log + ..... ... + 0000004B | A1 A0 2C 4B | time_created = 1261215905 + 0000004F | 13 | common_header_len = 19 + ------------------------ + post_header_len + ------------------------ + 00000050 | 38 | 56 - START_EVENT_V3 [1] + ..... ... + 00000069 | 02 | 2 - INCIDENT_EVENT [26] + 0000006A | 00 | 0 - RESERVED [27] + ..... ... + 00000081 | 00 | 0 - RESERVED [50] + 00000082 | 00 | 0 - ANNOTATE_RBR_EVENT [51] + ************************ + +2. Outline of Annotate_rows event behavior +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Each Annotate_rows_log_event object has two private members describing the +corresponding query: + + char *m_query_txt; + uint m_query_len; + +When the object is created for writing to a binary log, this query is taken +from 'thd' (for short, below we omit the 'Annotate_rows_log_event::' prefix +as well as other implementation details): + + Annotate_rows_log_event(THD *thd) + { + m_query_txt = thd->query(); + m_query_len = thd->query_length(); + } + +When the object is read from a binary log, the query is taken from the buffer +containing the binary log representation of the event (this buffer is allocated +in Log_event object from which all Log events are derived): + + Annotate_rows_log_event(char *buf, uint event_len, + Format_description_log_event *desc) + { + m_query_len = event_len - desc->common_header_len; + m_query_txt = buf + desc->common_header_len; + } + +The events are written to the binary log by the Log_event::write() member +which calls virtual write_data_header() and write_data_body() members +("data header" and "post header" are synonym in replication terminology). +In our case, data header is empty and data body is just the query: + + bool write_data_body(IO_CACHE *file) + { + return my_b_safe_write(file, (uchar*) m_query_txt, m_query_len); + } + +Printing the event is just printing the query: + + void Annotate_rows_log_event::print(FILE *file, PRINT_EVENT_INFO *pinfo) + { + my_b_printf(&pinfo->head_cache, "\tQuery: `%s`\n", m_query_txt); + } + +3. How Master writes Annotate_rows events to the binary log +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The event is written to the binary log just before the group of Table_map +events which precede corresponding Rows events (one query may generate +several Table map events in the binary log, but the corresponding +Annotate_rows event must be written only once before the first Table map +event; hence the boolean variable 'with_annotate' below): + + int write_locked_table_maps(THD *thd) + { ... + bool with_annotate= thd->variables.binlog_annotate_rows_events; + ... + for (uint i= 0; i < ... <number of tables> ...; ++i) + { ... + thd->binlog_write_table_map(table, ..., with_annotate); + with_annotate= 0; // write Annotate_event not more than once + ... + } + ... + } + + int THD::binlog_write_table_map(TABLE *table, ..., bool with_annotate) + { ... + Table_map_log_event the_event(...); + ... + if (with_annotate) + { + Annotate_rows_log_event anno(this); + mysql_bin_log.write(&anno); + } + + mysql_bin_log.write(&the_event); + ... + } + +4. How slave treats replicate-annotate-rows-events option +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The replicate-annotate-rows-events option is treated just as the session +value of the binlog_annotate_rows_events variable for the slave IO and +SQL threads. This setting is done during initialization of these threads: + + pthread_handler_t handle_slave_io(void *arg) + { + THD *thd= new THD; + ... + init_slave_thread(thd, SLAVE_THD_IO); + ... + } + + pthread_handler_t handle_slave_sql(void *arg) + { + THD *thd= new THD; + ... + init_slave_thread(thd, SLAVE_THD_SQL); + ... + } + + int init_slave_thread(THD* thd, SLAVE_THD_TYPE thd_type) + { ... + thd->variables.binlog_annotate_rows_events= + opt_replicate_annotate_rows_events; + ... + } + +5. How slave IO thread requests Annotate_rows events +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +When requesting an event, the slave should inform the master whether +it should send Annotate_rows events or not. To that end we add a new +BINLOG_SEND_ANNOTATE_ROWS_EVENT flag used when requesting an event: + + #define BINLOG_DUMP_NON_BLOCK 1 + #define BINLOG_SEND_ANNOTATE_ROWS_EVENT 2 + + pthread_handler_t handle_slave_io(void *arg) + { ... + request_dump(mysql, ...); + ... + } + + int request_dump(MYSQL* mysql, ...) + { ... + if (opt_log_slave_updates && + mi->io_thd->variables.binlog_annotate_rows_events) + binlog_flags|= BINLOG_SEND_ANNOTATE_ROWS_EVENT; + ... + int2store(buf + 4, binlog_flags); + ... + simple_command(mysql, COM_BINLOG_DUMP, buf, ...); + ... + } + +6. How master executes the request +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + case COM_BINLOG_DUMP: + { ... + flags= uint2korr(packet + 4); + ... + mysql_binlog_send(thd, ..., flags); + ... + } + + void mysql_binlog_send(THD* thd, ..., ushort flags) + { ... + Log_event::read_log_event(&log, packet, ...); + ... + if ((*packet)[EVENT_TYPE_OFFSET + 1] != ANNOTATE_ROWS_EVENT || + flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT || + thd->server_id == 0 /* slave == mysqlbinlog */ ) + { + my_net_write(net, packet->ptr(), packet->length()); + } + ... + } + +7. How slave SQL thread processes Annotate_rows events +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The slave processes each recieved event by "applying" it, i.e. by +calling the Log_event::apply_event() function which in turn calls +the virtual do_apply_event() member specific for each type of the +event. + + int exec_relay_log_event(THD* thd, Relay_log_info* rli) + { ... + Log_event *ev = next_event(rli); + ... + apply_event_and_update_pos(ev, ...); + + if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) + delete ev; + ... + } + + int apply_event_and_update_pos(Log_event *ev, ...) + { ... + ev->apply_event(...); + ... + } + + int Log_event::apply_event(...) + { + return do_apply_event(...); + } + +What does it mean to "apply" an Annotate_rows event? It means to set current +thd query to that of the described by the event, i.e. to the query which +caused the subsequent Rows events (see "How Master writes Annotate_rows +events to the binary log" to follow what happens further when the subsequent +Rows events is applied): + + int Annotate_rows_log_event::do_apply_event(...) + { + thd->set_query(m_query_txt, m_query_len); + } + +NOTE. I am not sure, but possibly current values of thd->query and +thd->query_length should be saved before calling set_query() and to be +restored on the Annotate_rows_log_event object deletion. +Is it really needed ? + +After calling this do_apply_event() function we may not delete the +Annotate_rows_log_event object immediatedly (see exec_relay_log_event() +above) because thd->query now points to the string inside this object. +We may keep the pointer to this object in the Relay_log_info: + + class Relay_log_info + { + public: + ... + void set_annotate_event(Annotate_rows_log_event*); + Annotate_rows_log_event* get_annotate_event(); + void free_annotate_event(); + ... + private: + Annotate_rows_log_event* m_annotate_event; + }; + +When the saved Annotate_rows object may be deleted? When all corresponding +Rows events will be processed, i.e. before processing the first non-Rows +event (note that Annotate_rows object resides in the binary log *after* +the (possible) 'BEGIN' Query event which accompanies the rows events; note +also that this deletion is adjusted with the case when some or all +corresponding Rows events are filtered out by replicate filter rules): + + int exec_relay_log_event(THD* thd, Relay_log_info* rli) + { ... + Log_event *ev= next_event(rli); + ... + if (rli->get_annotate_event() && !IS_RBR_EVENT_TYPE(ev->get_type_code())) + rli->free_annotate_event(); + + apply_event_and_update_pos(ev, ...); + + if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) + rli->set_annotate_event((Annotate_rows_log_event*) ev); + else if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) + delete ev; + ... + } + +where + + #define IS_RBR_EVENT_TYPE(type) ( (type) == TABLE_MAP_EVENT || \ + (type) == WRITE_ROWS_EVENT || \ + (type) == UPDATE_ROWS_EVENT || \ + (type) == DELETE_ROWS_EVENT ) + +8. General remarks +~~~~~~~~~~~~~~~~~~ +Kristian noticed that introducing new log event type should be coordinated +somehow with MySQL/Sun: + + Kristian: The numeric code for this event must be assigned carefully. + It should be coordinated with MySQL/Sun, otherwise we can get into a + situation where MySQL uses the same numeric code for one event that + MariaDB uses for ANNOTATE_ROWS_EVENT, which would make merging the two + impossible. + Alex: I reserved about 20 numbers not to have possible conflicts + with MySQL. + Kristian: Still, I think it would be appropriate to send a polite email + to internals(a)lists.mysql.com about this and suggesting to reserve the + event number. + +Also we should notice the introduction of the BINLOG_SEND_ANNOTATE_ROWS_EVENT +flag taking into account that MySQL/Sun may also introduce a flag with the +same value to be used in the request_dump-mysql_binlog_send interface. +But this is mainly the question of merging: if a conflict concerning this +flag occur, we may simply change the BINLOG_SEND_ANNOTATE_ROWS_EVENT value +(this does not require additional changes in the code). -=-=(Alexi - Sat, 19 Dec 2009, 15:41)=-=- High-Level Specification modified. --- /tmp/wklog.47.old.14545 2009-12-19 15:41:21.000000000 +0200 +++ /tmp/wklog.47.new.14545 2009-12-19 15:41:21.000000000 +0200 @@ -1,122 +1,107 @@ -First suggestion: - -> I think for this we would actually need a new binlog event type -> (Comment_log_event?). Unless we want to log an empty statement Query_log_event -> containing only a comment (a bit of a hack). - -New server option -~~~~~~~~~~~~~~~~~ - --binlog-annotate-rows-events - -Setting this option makes RBR (rows-) events in the binary log to be -preceded by Annotate rows events (see below). The corresponding -'binlog_annotate_rows_events' system variable is dynamic and has both -global and session values. Default global value is OFF. - -Note. Session values are usefull to make it possible to annotate only - some selected statements: +Content +~~~~~~~ + 1. Annotate_rows_log_event + 2. Server option: --binlog-annotate-rows-events + 3. Server option: --replicate-annotate-rows-events + 4. mysqlbinlog option: --print-annotate-rows-events + 5. mysqlbinlog output + +1. Annotate_rows_log_event [ ANNOTATE_ROWS_EVENT ] +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Describes the query which caused the corresponding rows events. Has empty +post-header and contains the query text in its data part. Example: + + ************************ + ANNOTATE_RBR_EVENT + ************************ + 00000220 | B6 A0 2C 4B | time_when = 1261215926 + 00000224 | 33 | event_type = 51 + 00000225 | 64 00 00 00 | server_id = 100 + 00000229 | 36 00 00 00 | event_len = 54 + 0000022D | 56 02 00 00 | log_pos = 00000256 + 00000231 | 00 00 | flags = <none> + ------------------------ + 00000233 | 49 4E 53 45 | query = "INSERT INTO t1 VALUES (1), (2), (3)" + 00000237 | 52 54 20 49 | + 0000023B | 4E 54 4F 20 | + 0000023F | 74 31 20 56 | + 00000243 | 41 4C 55 45 | + 00000247 | 53 20 28 31 | + 0000024B | 29 2C 20 28 | + 0000024F | 32 29 2C 20 | + 00000253 | 28 33 29 | + ************************ + +In binary log, Annotate_rows event follows the (possible) 'BEGIN' Query event +and precedes the first of Table map events which accompany the corresponding +rows events. (See example in the "mysqlbinlog output" section below.) + +2. Server option: --binlog-annotate-rows-events +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Tells the master to write Annotate_rows events to the binary log. + + * Variable Name: binlog_annotate_rows_events + * Scope: Global & Session + * Access Type: Dynamic + * Data Type: bool + * Default Value: OFF +NOTE. Session values allows to annotate only some selected statements: ... SET SESSION binlog_annotate_rows_events=ON; ... statements to be annotated ... SET SESSION binlog_annotate_rows_events=OFF; ... statements not to be annotated ... -New binlog event type -~~~~~~~~~~~~~~~~~~~~~ - Annotate_rows_log_event [ ANNOTATE_ROWS_EVENT ] - -Describes the query which caused the corresponding rows event. In binary log, -precedes each Table_map_log_event. Contains empty post-header and the query -text in its data part. - -The numeric code for this event must be assigned carefully. It should be -coordinated with MySQL/Sun, otherwise we can get into a situation where MySQL -uses the same numeric code for one event that MariaDB uses for -ANNOTATE_ROWS_EVENT, which would make merging the two impossible. - -Example: - - ... - ************************ - ANNOTATE_ROWS_EVENT [51] - ************************ - 000000C7 | 54 1B 12 4B | time_when = 1259477844 - 000000CB | 33 | event_type = 51 - 000000CC | 64 00 00 00 | server_id = 100 - 000000D0 | 2C 00 00 00 | event_len = 44 - 000000D4 | F3 00 00 00 | log_pos = 000000F3 - 000000D8 | 00 00 | flags = <none> - ------------------------ - 000000DA | 69 6E 73 65 | query = "insert into t1 values (1)" - 000000DE | 72 74 20 69 | - 000000E2 | 6E 74 6F 20 | - 000000E6 | 74 31 20 76 | - 000000EA | 61 6C 75 65 | - 000000EE | 73 20 28 31 | - 000000F2 | 29 | - ************************ - TABLE_MAP_EVENT [19] - ************************ - 000000F3 | 54 1B 12 4B | time_when = 1259477844 - 000000F7 | 13 | event_type = 19 - 000000F8 | 64 00 00 00 | server_id = 100 - 000000FC | 29 00 00 00 | event_len = 41 - 00000100 | 1C 01 00 00 | log_pos = 0000011C - 00000104 | 00 00 | flags = <none> - ------------------------ - ... - ************************ - WRITE_ROWS_EVENT [23] - ************************ - 0000011C | 54 1B 12 4B | time_when = 1259477844 - 00000120 | 17 | event_type = 23 - 00000121 | 64 00 00 00 | server_id = 100 - 00000125 | 22 00 00 00 | event_len = 34 - 00000129 | 3E 01 00 00 | log_pos = 0000013E - 0000012D | 10 00 | flags = LOG_EVENT_UPDATE_TABLE_MAP_VERSION_F - ------------------------ - 0000012F | 0F 00 00 00 | table_id = 15 - ... - -New mysqlbinlog option -~~~~~~~~~~~~~~~~~~~~~~ - --print-annotate-rows-events - -With this option, mysqlbinlog prints the content of Annotate-rows -events (if the binary log does contain them). Without this option -(i.e. by default), mysqlbinlog skips Annotate rows events. - - -mysqlbinlog output -~~~~~~~~~~~~~~~~~~ -Something like this: +3. Server option: --replicate-annotate-rows-events +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Tells the slave to reproduce Annotate_rows events recieved from the master +in its own binary log (sensible only in pair with log-slave-updates option). + + * Variable Name: replicate_annotate_rows_events + * Scope: Global + * Access Type: Read only + * Data Type: bool + * Default Value: OFF + +NOTE. Why do we additionally need this 'replicate' option? Why not to make +the slave to reproduce this events when its binlog-annotate-rows-events +global value is ON? Well, because, for example, we may want to configure +the slave which should reproduce Annotate_rows events but has global +binlog-annotate-rows-events = OFF meaning this to be the default value for +the client threads (see also "How slave treats replicate-annotate-rows-events +option" in LLD part). + +4. mysqlbinlog option: --print-annotate-rows-events +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +With this option, mysqlbinlog prints the content of Annotate_rows events (if +the binary log does contain them). Without this option (i.e. by default), +mysqlbinlog skips Annotate_rows events. +5. mysqlbinlog output +~~~~~~~~~~~~~~~~~~~~~ +With --print-annotate-rows-events, mysqlbinlog outputs Annotate_rows events +in a form like this: ... - # at 199 - # at 243 - # at 284 - #091129 9:57:24 server id 100 end_log_pos 243 Query: `insert into t1 values -(1)` - #091129 9:57:24 server id 100 end_log_pos 284 Table_map: `test`.`t1` mapped -to number 15 - #091129 9:57:24 server id 100 end_log_pos 318 Write_rows: table id 15 + # at 1646 + #091219 12:45:26 server id 100 end_log_pos 1714 Query thread_id=1 +exec_time=0 error_code=0 + SET TIMESTAMP=1261215926/*!*/; + BEGIN + /*!*/; + # at 1714 + # at 1812 + # at 1853 + # at 1894 + # at 1938 + #091219 12:45:26 server id 100 end_log_pos 1812 Query: `DELETE t1, t2 FROM +t1 INNER JOIN t2 INNER JOIN t3 WHERE t1.a=t2.a AND t2.a=t3.a` + #091219 12:45:26 server id 100 end_log_pos 1853 Table_map: `test`.`t1` +mapped to number 16 + #091219 12:45:26 server id 100 end_log_pos 1894 Table_map: `test`.`t2` +mapped to number 17 + #091219 12:45:26 server id 100 end_log_pos 1938 Delete_rows: table id 16 + #091219 12:45:26 server id 100 end_log_pos 1982 Delete_rows: table id 17 flags: STMT_END_F - - BINLOG ' - VBsSSzNkAAAALAAAAPMAAAAAAGluc2VydCBpbnRvIHQxIHZhbHVlcyAoMSk= - VBsSSxNkAAAAKQAAABwBAAAAAA8AAAAAAAAABHRlc3QAAnQxAAEDAAE= - VBsSSxdkAAAAIgAAAD4BAAAQAA8AAAAAAAEAAf/+AQAAAA== - '/*!*/; - ### INSERT INTO test.t1 - ### SET - ### @1=1 /* INT meta=0 nullable=1 is_null=0 */ ... -When master sends Annotate rows events -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -1. Master always sends Annotate_rows events to mysqlbinlog (in - remote case). -2. Master sends Annotate_rows events to a slave only if the slave has - both log-slave-updates and binlog-annotate-rows-events options set. - -=-=(Bothorsen - Fri, 18 Dec 2009, 16:22)=-=- Add estimation time. Worked 5 hours and estimate 35 hours remain (original estimate increased by 5 hours). -=-=(Bothorsen - Fri, 18 Dec 2009, 16:16)=-=- This is the work done on this patch so far. Most of it done by Alex. Worked 15 hours and estimate 035 hours remain (original estimate increased by 50 hours). ------------------------------------------------------------ -=-=(View All Progress Notes, 20 total)=-=- http://askmonty.org/worklog/index.pl?tid=47&nolimit=1 DESCRIPTION: Store in binlog (and show in mysqlbinlog output) texts of statements that caused RBR events This is needed for (list from Monty): - Easier to understand why updates happened - Would make it easier to find out where in application things went wrong (as you can search for exact strings) - Allow one to filter things based on comments in the statement. The cost of this can be that the binlog will be approximately 2x in size (especially insert of big blob's would be a bit painful), so this should be an optional feature. HIGH-LEVEL SPECIFICATION: Content ~~~~~~~ 1. Annotate_rows_log_event 2. Server option: --binlog-annotate-rows-events 3. Server option: --replicate-annotate-rows-events 4. mysqlbinlog option: --print-annotate-rows-events 5. mysqlbinlog output 1. Annotate_rows_log_event [ ANNOTATE_ROWS_EVENT ] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Describes the query which caused the corresponding rows events. Has empty post-header and contains the query text in its data part. Example: ************************ ANNOTATE_ROWS_EVENT ************************ 00000220 | B6 A0 2C 4B | time_when = 1261215926 00000224 | 33 | event_type = 51 00000225 | 64 00 00 00 | server_id = 100 00000229 | 36 00 00 00 | event_len = 54 0000022D | 56 02 00 00 | log_pos = 00000256 00000231 | 00 00 | flags = <none> ------------------------ 00000233 | 49 4E 53 45 | query = "INSERT INTO t1 VALUES (1), (2), (3)" 00000237 | 52 54 20 49 | 0000023B | 4E 54 4F 20 | 0000023F | 74 31 20 56 | 00000243 | 41 4C 55 45 | 00000247 | 53 20 28 31 | 0000024B | 29 2C 20 28 | 0000024F | 32 29 2C 20 | 00000253 | 28 33 29 | ************************ In binary log, Annotate_rows event follows the (possible) 'BEGIN' Query event and precedes the first of Table map events which accompany the corresponding rows events. (See example in the "mysqlbinlog output" section below.) 2. Server option: --binlog-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the master to write Annotate_rows events to the binary log. * Variable Name: binlog_annotate_rows_events * Scope: Global & Session * Access Type: Dynamic * Data Type: bool * Default Value: OFF NOTE. Session values allows to annotate only some selected statements: ... SET SESSION binlog_annotate_rows_events=ON; ... statements to be annotated ... SET SESSION binlog_annotate_rows_events=OFF; ... statements not to be annotated ... 3. Server option: --replicate-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the slave to reproduce Annotate_rows events recieved from the master in its own binary log (sensible only in pair with log-slave-updates option). * Variable Name: replicate_annotate_rows_events * Scope: Global * Access Type: Read only * Data Type: bool * Default Value: OFF NOTE. Why do we additionally need this 'replicate' option? Why not to make the slave to reproduce this events when its binlog-annotate-rows-events global value is ON? Well, because, for example, we may want to configure the slave which should reproduce Annotate_rows events but has global binlog-annotate-rows-events = OFF meaning this to be the default value for the client threads (see also "How slave treats replicate-annotate-rows-events option" in LLD part). 4. mysqlbinlog option: --print-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ With this option, mysqlbinlog prints the content of Annotate_rows events (if the binary log does contain them). Without this option (i.e. by default), mysqlbinlog skips Annotate_rows events. 5. mysqlbinlog output ~~~~~~~~~~~~~~~~~~~~~ With --print-annotate-rows-events, mysqlbinlog outputs Annotate_rows events in a form like this: ... # at 1646 #091219 12:45:26 server id 100 end_log_pos 1714 Query thread_id=1 exec_time=0 error_code=0 SET TIMESTAMP=1261215926/*!*/; BEGIN /*!*/; # at 1714 # at 1812 # at 1853 # at 1894 # at 1938 #091219 12:45:26 server id 100 end_log_pos 1812 Query: `DELETE t1, t2 FROM t1 INNER JOIN t2 INNER JOIN t3 WHERE t1.a=t2.a AND t2.a=t3.a` #091219 12:45:26 server id 100 end_log_pos 1853 Table_map: `test`.`t1` mapped to number 16 #091219 12:45:26 server id 100 end_log_pos 1894 Table_map: `test`.`t2` mapped to number 17 #091219 12:45:26 server id 100 end_log_pos 1938 Delete_rows: table id 16 #091219 12:45:26 server id 100 end_log_pos 1982 Delete_rows: table id 17 flags: STMT_END_F ... LOW-LEVEL DESIGN: Content ~~~~~~~ 1. Annotate_rows event number 2. Outline of Annotate_rows event behavior 3. How Master writes Annotate_rows events to the binary log 4. How slave treats replicate-annotate-rows-events option 5. How slave IO thread requests Annotate_rows events 6. How master executes the request 7. How slave SQL thread processes Annotate_rows events 8. General remarks 1. Annotate_rows event number ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To avoid possible event numbers conflict with MySQL/Sun, we leave a gap between the last MySQL event number and the Annotate_rows event number: enum Log_event_type { ... INCIDENT_EVENT= 26, // New MySQL event numbers are to be added here MYSQL_EVENTS_END, MARIA_EVENTS_BEGIN= 51, // New Maria event numbers start from here ANNOTATE_ROWS_EVENT= 51, ENUM_END_EVENT }; together with the corresponding extension of 'post_header_len' array in the Format description event. (This extension does not affect the compatibility of the binary log). Here is how Format description event looks like with this extension: ************************ FORMAT_DESCRIPTION_EVENT ************************ 00000004 | A1 A0 2C 4B | time_when = 1261215905 00000008 | 0F | event_type = 15 00000009 | 64 00 00 00 | server_id = 100 0000000D | 7F 00 00 00 | event_len = 127 00000011 | 83 00 00 00 | log_pos = 00000083 00000015 | 01 00 | flags = LOG_EVENT_BINLOG_IN_USE_F ------------------------ 00000017 | 04 00 | binlog_ver = 4 00000019 | 35 2E 32 2E | server_ver = 5.2.0-MariaDB-alpha-debug-log ..... ... 0000004B | A1 A0 2C 4B | time_created = 1261215905 0000004F | 13 | common_header_len = 19 ------------------------ post_header_len ------------------------ 00000050 | 38 | 56 - START_EVENT_V3 [1] ..... ... 00000069 | 02 | 2 - INCIDENT_EVENT [26] 0000006A | 00 | 0 - RESERVED [27] ..... ... 00000081 | 00 | 0 - RESERVED [50] 00000082 | 00 | 0 - ANNOTATE_ROWS_EVENT [51] ************************ 2. Outline of Annotate_rows event behavior ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Each Annotate_rows_log_event object has two private members describing the corresponding query: char *m_query_txt; uint m_query_len; When the object is created for writing to a binary log, this query is taken from 'thd' (for short, below we omit the 'Annotate_rows_log_event::' prefix as well as other implementation details): Annotate_rows_log_event(THD *thd) { m_query_txt = thd->query(); m_query_len = thd->query_length(); } When the object is read from a binary log, the query is taken from the buffer containing the binary log representation of the event (this buffer is allocated in Log_event object from which all Log events are derived): Annotate_rows_log_event(char *buf, uint event_len, Format_description_log_event *desc) { m_query_len = event_len - desc->common_header_len; m_query_txt = buf + desc->common_header_len; } The events are written to the binary log by the Log_event::write() member which calls virtual write_data_header() and write_data_body() members ("data header" and "post header" are synonym in replication terminology). In our case, data header is empty and data body is just the query: bool write_data_body(IO_CACHE *file) { return my_b_safe_write(file, (uchar*) m_query_txt, m_query_len); } Printing the event is just printing the query: void Annotate_rows_log_event::print(FILE *file, PRINT_EVENT_INFO *pinfo) { my_b_printf(&pinfo->head_cache, "\tQuery: `%s`\n", m_query_txt); } 3. How Master writes Annotate_rows events to the binary log ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The event is written to the binary log just before the group of Table_map events which precede corresponding Rows events (one query may generate several Table map events in the binary log, but the corresponding Annotate_rows event must be written only once before the first Table map event; hence the boolean variable 'with_annotate' below): int write_locked_table_maps(THD *thd) { ... bool with_annotate= thd->variables.binlog_annotate_rows_events; ... for (uint i= 0; i < ... <number of tables> ...; ++i) { ... thd->binlog_write_table_map(table, ..., with_annotate); with_annotate= 0; // write Annotate_event not more than once ... } ... } int THD::binlog_write_table_map(TABLE *table, ..., bool with_annotate) { ... Table_map_log_event the_event(...); ... if (with_annotate) { Annotate_rows_log_event anno(this); mysql_bin_log.write(&anno); } mysql_bin_log.write(&the_event); ... } 4. How slave treats replicate-annotate-rows-events option ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The replicate-annotate-rows-events option is treated just as the session value of the binlog_annotate_rows_events variable for the slave IO and SQL threads. This setting is done during initialization of these threads: pthread_handler_t handle_slave_io(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_IO); ... } pthread_handler_t handle_slave_sql(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_SQL); ... } int init_slave_thread(THD* thd, SLAVE_THD_TYPE thd_type) { ... thd->variables.binlog_annotate_rows_events= opt_replicate_annotate_rows_events; ... } 5. How slave IO thread requests Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If the replicate-annotate-rows-events option is not set on a slave, there is no need for master to send Annotate_rows events to this slave. The slave (or mysqlbinlog in remote case), before requesting binlog dump via the COM_BINLOG_DUMP command, informs the master whether it should send these events by executing the newly added COM_BINLOG_DUMP_OPTIONS_EXT server command: case COM_BINLOG_DUMP_OPTIONS_EXT: thd->binlog_dump_flags_ext= packet[0]; my_ok(thd); break; Note. We add this new command and don't use COM_BINLOG_DUMP to avoid possible conflicts with MySQL/Sun. 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ case COM_BINLOG_DUMP: { ... flags= uint2korr(packet + 4); ... mysql_binlog_send(thd, ..., flags); ... } void mysql_binlog_send(THD* thd, ..., ushort flags) { ... Log_event::read_log_event(&log, packet, ...); ... if ((*packet)[EVENT_TYPE_OFFSET + 1] != ANNOTATE_ROWS_EVENT || flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT) { my_net_write(net, packet->ptr(), packet->length()); } ... } 7. How slave SQL thread processes Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The slave processes each recieved event by "applying" it, i.e. by calling the Log_event::apply_event() function which in turn calls the virtual do_apply_event() member specific for each type of the event. int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev = next_event(rli); ... apply_event_and_update_pos(ev, ...); if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } int apply_event_and_update_pos(Log_event *ev, ...) { ... ev->apply_event(...); ... } int Log_event::apply_event(...) { return do_apply_event(...); } What does it mean to "apply" an Annotate_rows event? It means to set current thd query to that of the described by the event, i.e. to the query which caused the subsequent Rows events (see "How Master writes Annotate_rows events to the binary log" to follow what happens further when the subsequent Rows events are applied): int Annotate_rows_log_event::do_apply_event(...) { thd->set_query(m_query_txt, m_query_len); } NOTE. I am not sure, but possibly current values of thd->query and thd->query_length should be saved before calling set_query() and to be restored on the Annotate_rows_log_event object deletion. Is it really needed ? After calling this do_apply_event() function we may not delete the Annotate_rows_log_event object immediatedly (see exec_relay_log_event() above) because thd->query now points to the string inside this object. We may keep the pointer to this object in the Relay_log_info: class Relay_log_info { public: ... void set_annotate_event(Annotate_rows_log_event*); Annotate_rows_log_event* get_annotate_event(); void free_annotate_event(); ... private: Annotate_rows_log_event* m_annotate_event; }; The saved Annotate_rows object should be deleted when all corresponding Rows events will be processed: int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev= next_event(rli); ... apply_event_and_update_pos(ev, ...); if (rli->get_annotate_event() && is_last_rows_event(ev)) rli->free_annotate_event(); else if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) rli->set_annotate_event((Annotate_rows_log_event*) ev); else if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } where bool is_last_rows_event(Log_event* ev) { Log_event_type type= ev->get_type_code(); if (IS_ROWS_EVENT_TYPE(type)) { Rows_log_event* rows= (Rows_log_event*)ev; return rows->get_flags(Rows_log_event::STMT_END_F); } return 0; } #define IS_ROWS_EVENT_TYPE(type) ((type) == WRITE_ROWS_EVENT || \ (type) == UPDATE_ROWS_EVENT || \ (type) == DELETE_ROWS_EVENT) 8. General remarks ~~~~~~~~~~~~~~~~~~ Kristian noticed that introducing new log event type should be coordinated somehow with MySQL/Sun: Kristian: The numeric code for this event must be assigned carefully. It should be coordinated with MySQL/Sun, otherwise we can get into a situation where MySQL uses the same numeric code for one event that MariaDB uses for ANNOTATE_ROWS_EVENT, which would make merging the two impossible. Alex: I reserved about 20 numbers not to have possible conflicts with MySQL. Kristian: Still, I think it would be appropriate to send a polite email to internals(a)lists.mysql.com about this and suggesting to reserve the event number. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Alexi): Store in binlog text of statements that caused RBR events (47)
by worklog-noreply＠askmonty.org 04 Feb '10

04 Feb '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Store in binlog text of statements that caused RBR events CREATION DATE..: Sat, 15 Aug 2009, 23:48 SUPERVISOR.....: Monty IMPLEMENTOR....: COPIES TO......: Knielsen CATEGORY.......: Server-RawIdeaBin TASK ID........: 47 (http://askmonty.org/worklog/?tid=47) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 20 ESTIMATE.......: 35 (hours remain) ORIG. ESTIMATE.: 35 PROGRESS NOTES: -=-=(Alexi - Thu, 04 Feb 2010, 09:54)=-=- Low Level Design modified. --- /tmp/wklog.47.old.16174 2010-02-04 09:54:13.000000000 +0200 +++ /tmp/wklog.47.new.16174 2010-02-04 09:54:13.000000000 +0200 @@ -171,35 +171,20 @@ 5. How slave IO thread requests Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -When requesting an event, the slave should inform the master whether -it should send Annotate_rows events or not. To that end we add a new -BINLOG_SEND_ANNOTATE_ROWS_EVENT flag used when requesting an event: +If the replicate-annotate-rows-events option is not set on a slave, there +is no need for master to send Annotate_rows events to this slave. The slave +(or mysqlbinlog in remote case), before requesting binlog dump via the +COM_BINLOG_DUMP command, informs the master whether it should send these +events by executing the newly added COM_BINLOG_DUMP_OPTIONS_EXT server +command: + + case COM_BINLOG_DUMP_OPTIONS_EXT: + thd->binlog_dump_flags_ext= packet[0]; + my_ok(thd); + break; - #define BINLOG_DUMP_NON_BLOCK 1 - #define BINLOG_SEND_ANNOTATE_ROWS_EVENT 2 - - pthread_handler_t handle_slave_io(void *arg) - { ... - request_dump(mysql, ...); - ... - } - - int request_dump(MYSQL* mysql, ...) - { ... - if (opt_log_slave_updates && - mi->io_thd->variables.binlog_annotate_rows_events) - binlog_flags|= BINLOG_SEND_ANNOTATE_ROWS_EVENT; - ... - int2store(buf + 4, binlog_flags); - ... - simple_command(mysql, COM_BINLOG_DUMP, buf, ...); - ... - } - -NOTE. mysqlbinlog, when remotely requesting BINLOG_DUMP by calling the -simple_command() function, should also use this flag if it wants (in case -of the --print-annotate-rows-events option set) to recieve Annotate_rows -events. +Note. We add this new command and don't use COM_BINLOG_DUMP to avoid possible +conflicts with MySQL/Sun. 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -338,10 +323,4 @@ to internals(a)lists.mysql.com about this and suggesting to reserve the event number. -Also we should notice the introduction of the BINLOG_SEND_ANNOTATE_ROWS_EVENT -flag taking into account that MySQL/Sun may also introduce a flag with the -same value to be used in the request_dump-mysql_binlog_send interface. -But this is mainly the question of merging: if a conflict concerning this -flag occur, we may simply change the BINLOG_SEND_ANNOTATE_ROWS_EVENT value -(this does not require additional changes in the code). -=-=(Alexi - Sun, 20 Dec 2009, 16:00)=-=- Low Level Design modified. --- /tmp/wklog.47.old.19667 2009-12-20 14:00:56.000000000 +0000 +++ /tmp/wklog.47.new.19667 2009-12-20 14:00:56.000000000 +0000 @@ -196,6 +196,11 @@ ... } +NOTE. mysqlbinlog, when remotely requesting BINLOG_DUMP by calling the +simple_command() function, should also use this flag if it wants (in case +of the --print-annotate-rows-events option set) to recieve Annotate_rows +events. + 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -212,8 +217,7 @@ Log_event::read_log_event(&log, packet, ...); ... if ((*packet)[EVENT_TYPE_OFFSET + 1] != ANNOTATE_ROWS_EVENT || - flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT || - thd->server_id == 0 /* slave == mysqlbinlog */ ) + flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT) { my_net_write(net, packet->ptr(), packet->length()); } -=-=(Alexi - Sun, 20 Dec 2009, 13:14)=-=- Low Level Design modified. --- /tmp/wklog.47.old.11350 2009-12-20 13:14:04.000000000 +0200 +++ /tmp/wklog.47.new.11350 2009-12-20 13:14:04.000000000 +0200 @@ -282,23 +282,18 @@ Annotate_rows_log_event* m_annotate_event; }; -When the saved Annotate_rows object may be deleted? When all corresponding -Rows events will be processed, i.e. before processing the first non-Rows -event (note that Annotate_rows object resides in the binary log *after* -the (possible) 'BEGIN' Query event which accompanies the rows events; note -also that this deletion is adjusted with the case when some or all -corresponding Rows events are filtered out by replicate filter rules): +The saved Annotate_rows object should be deleted when all corresponding +Rows events will be processed: int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev= next_event(rli); ... - if (rli->get_annotate_event() && !IS_RBR_EVENT_TYPE(ev->get_type_code())) - rli->free_annotate_event(); - apply_event_and_update_pos(ev, ...); - if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) + if (rli->get_annotate_event() && is_last_rows_event(ev)) + rli->free_annotate_event(); + else if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) rli->set_annotate_event((Annotate_rows_log_event*) ev); else if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; @@ -307,10 +302,21 @@ where - #define IS_RBR_EVENT_TYPE(type) ( (type) == TABLE_MAP_EVENT || \ - (type) == WRITE_ROWS_EVENT || \ + bool is_last_rows_event(Log_event* ev) + { + Log_event_type type= ev->get_type_code(); + if (IS_ROWS_EVENT_TYPE(type)) + { + Rows_log_event* rows= (Rows_log_event*)ev; + return rows->get_flags(Rows_log_event::STMT_END_F); + } + + return 0; + } + + #define IS_ROWS_EVENT_TYPE(type) ((type) == WRITE_ROWS_EVENT || \ (type) == UPDATE_ROWS_EVENT || \ - (type) == DELETE_ROWS_EVENT ) + (type) == DELETE_ROWS_EVENT) 8. General remarks ~~~~~~~~~~~~~~~~~~ -=-=(Alexi - Sun, 20 Dec 2009, 09:29)=-=- Low Level Design modified. --- /tmp/wklog.47.old.32726 2009-12-20 07:29:56.000000000 +0000 +++ /tmp/wklog.47.new.32726 2009-12-20 07:29:56.000000000 +0000 @@ -56,7 +56,7 @@ 0000006A | 00 | 0 - RESERVED [27] ..... ... 00000081 | 00 | 0 - RESERVED [50] - 00000082 | 00 | 0 - ANNOTATE_RBR_EVENT [51] + 00000082 | 00 | 0 - ANNOTATE_ROWS_EVENT [51] ************************ 2. Outline of Annotate_rows event behavior -=-=(Alexi - Sat, 19 Dec 2009, 16:10)=-=- Low Level Design modified. --- /tmp/wklog.47.old.16051 2009-12-19 16:10:48.000000000 +0200 +++ /tmp/wklog.47.new.16051 2009-12-19 16:10:48.000000000 +0200 @@ -253,7 +253,7 @@ thd query to that of the described by the event, i.e. to the query which caused the subsequent Rows events (see "How Master writes Annotate_rows events to the binary log" to follow what happens further when the subsequent -Rows events is applied): +Rows events are applied): int Annotate_rows_log_event::do_apply_event(...) { -=-=(Alexi - Sat, 19 Dec 2009, 16:02)=-=- High-Level Specification modified. --- /tmp/wklog.47.old.15695 2009-12-19 16:02:33.000000000 +0200 +++ /tmp/wklog.47.new.15695 2009-12-19 16:02:33.000000000 +0200 @@ -12,7 +12,7 @@ post-header and contains the query text in its data part. Example: ************************ - ANNOTATE_RBR_EVENT + ANNOTATE_ROWS_EVENT ************************ 00000220 | B6 A0 2C 4B | time_when = 1261215926 00000224 | 33 | event_type = 51 -=-=(Alexi - Sat, 19 Dec 2009, 15:58)=-=- Low Level Design modified. --- /tmp/wklog.47.old.15437 2009-12-19 13:58:12.000000000 +0000 +++ /tmp/wklog.47.new.15437 2009-12-19 13:58:12.000000000 +0000 @@ -1 +1,337 @@ +Content +~~~~~~~ + 1. Annotate_rows event number + 2. Outline of Annotate_rows event behavior + 3. How Master writes Annotate_rows events to the binary log + 4. How slave treats replicate-annotate-rows-events option + 5. How slave IO thread requests Annotate_rows events + 6. How master executes the request + 7. How slave SQL thread processes Annotate_rows events + 8. General remarks + +1. Annotate_rows event number +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +To avoid possible event numbers conflict with MySQL/Sun, we leave a gap +between the last MySQL event number and the Annotate_rows event number: + + enum Log_event_type + { ... + INCIDENT_EVENT= 26, + // New MySQL event numbers are to be added here + MYSQL_EVENTS_END, + + MARIA_EVENTS_BEGIN= 51, + // New Maria event numbers start from here + ANNOTATE_ROWS_EVENT= 51, + + ENUM_END_EVENT + }; + +together with the corresponding extension of 'post_header_len' array in the +Format description event. (This extension does not affect the compatibility +of the binary log). Here is how Format description event looks like with +this extension: + + ************************ + FORMAT_DESCRIPTION_EVENT + ************************ + 00000004 | A1 A0 2C 4B | time_when = 1261215905 + 00000008 | 0F | event_type = 15 + 00000009 | 64 00 00 00 | server_id = 100 + 0000000D | 7F 00 00 00 | event_len = 127 + 00000011 | 83 00 00 00 | log_pos = 00000083 + 00000015 | 01 00 | flags = LOG_EVENT_BINLOG_IN_USE_F + ------------------------ + 00000017 | 04 00 | binlog_ver = 4 + 00000019 | 35 2E 32 2E | server_ver = 5.2.0-MariaDB-alpha-debug-log + ..... ... + 0000004B | A1 A0 2C 4B | time_created = 1261215905 + 0000004F | 13 | common_header_len = 19 + ------------------------ + post_header_len + ------------------------ + 00000050 | 38 | 56 - START_EVENT_V3 [1] + ..... ... + 00000069 | 02 | 2 - INCIDENT_EVENT [26] + 0000006A | 00 | 0 - RESERVED [27] + ..... ... + 00000081 | 00 | 0 - RESERVED [50] + 00000082 | 00 | 0 - ANNOTATE_RBR_EVENT [51] + ************************ + +2. Outline of Annotate_rows event behavior +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Each Annotate_rows_log_event object has two private members describing the +corresponding query: + + char *m_query_txt; + uint m_query_len; + +When the object is created for writing to a binary log, this query is taken +from 'thd' (for short, below we omit the 'Annotate_rows_log_event::' prefix +as well as other implementation details): + + Annotate_rows_log_event(THD *thd) + { + m_query_txt = thd->query(); + m_query_len = thd->query_length(); + } + +When the object is read from a binary log, the query is taken from the buffer +containing the binary log representation of the event (this buffer is allocated +in Log_event object from which all Log events are derived): + + Annotate_rows_log_event(char *buf, uint event_len, + Format_description_log_event *desc) + { + m_query_len = event_len - desc->common_header_len; + m_query_txt = buf + desc->common_header_len; + } + +The events are written to the binary log by the Log_event::write() member +which calls virtual write_data_header() and write_data_body() members +("data header" and "post header" are synonym in replication terminology). +In our case, data header is empty and data body is just the query: + + bool write_data_body(IO_CACHE *file) + { + return my_b_safe_write(file, (uchar*) m_query_txt, m_query_len); + } + +Printing the event is just printing the query: + + void Annotate_rows_log_event::print(FILE *file, PRINT_EVENT_INFO *pinfo) + { + my_b_printf(&pinfo->head_cache, "\tQuery: `%s`\n", m_query_txt); + } + +3. How Master writes Annotate_rows events to the binary log +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The event is written to the binary log just before the group of Table_map +events which precede corresponding Rows events (one query may generate +several Table map events in the binary log, but the corresponding +Annotate_rows event must be written only once before the first Table map +event; hence the boolean variable 'with_annotate' below): + + int write_locked_table_maps(THD *thd) + { ... + bool with_annotate= thd->variables.binlog_annotate_rows_events; + ... + for (uint i= 0; i < ... <number of tables> ...; ++i) + { ... + thd->binlog_write_table_map(table, ..., with_annotate); + with_annotate= 0; // write Annotate_event not more than once + ... + } + ... + } + + int THD::binlog_write_table_map(TABLE *table, ..., bool with_annotate) + { ... + Table_map_log_event the_event(...); + ... + if (with_annotate) + { + Annotate_rows_log_event anno(this); + mysql_bin_log.write(&anno); + } + + mysql_bin_log.write(&the_event); + ... + } + +4. How slave treats replicate-annotate-rows-events option +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The replicate-annotate-rows-events option is treated just as the session +value of the binlog_annotate_rows_events variable for the slave IO and +SQL threads. This setting is done during initialization of these threads: + + pthread_handler_t handle_slave_io(void *arg) + { + THD *thd= new THD; + ... + init_slave_thread(thd, SLAVE_THD_IO); + ... + } + + pthread_handler_t handle_slave_sql(void *arg) + { + THD *thd= new THD; + ... + init_slave_thread(thd, SLAVE_THD_SQL); + ... + } + + int init_slave_thread(THD* thd, SLAVE_THD_TYPE thd_type) + { ... + thd->variables.binlog_annotate_rows_events= + opt_replicate_annotate_rows_events; + ... + } + +5. How slave IO thread requests Annotate_rows events +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +When requesting an event, the slave should inform the master whether +it should send Annotate_rows events or not. To that end we add a new +BINLOG_SEND_ANNOTATE_ROWS_EVENT flag used when requesting an event: + + #define BINLOG_DUMP_NON_BLOCK 1 + #define BINLOG_SEND_ANNOTATE_ROWS_EVENT 2 + + pthread_handler_t handle_slave_io(void *arg) + { ... + request_dump(mysql, ...); + ... + } + + int request_dump(MYSQL* mysql, ...) + { ... + if (opt_log_slave_updates && + mi->io_thd->variables.binlog_annotate_rows_events) + binlog_flags|= BINLOG_SEND_ANNOTATE_ROWS_EVENT; + ... + int2store(buf + 4, binlog_flags); + ... + simple_command(mysql, COM_BINLOG_DUMP, buf, ...); + ... + } + +6. How master executes the request +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + case COM_BINLOG_DUMP: + { ... + flags= uint2korr(packet + 4); + ... + mysql_binlog_send(thd, ..., flags); + ... + } + + void mysql_binlog_send(THD* thd, ..., ushort flags) + { ... + Log_event::read_log_event(&log, packet, ...); + ... + if ((*packet)[EVENT_TYPE_OFFSET + 1] != ANNOTATE_ROWS_EVENT || + flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT || + thd->server_id == 0 /* slave == mysqlbinlog */ ) + { + my_net_write(net, packet->ptr(), packet->length()); + } + ... + } + +7. How slave SQL thread processes Annotate_rows events +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The slave processes each recieved event by "applying" it, i.e. by +calling the Log_event::apply_event() function which in turn calls +the virtual do_apply_event() member specific for each type of the +event. + + int exec_relay_log_event(THD* thd, Relay_log_info* rli) + { ... + Log_event *ev = next_event(rli); + ... + apply_event_and_update_pos(ev, ...); + + if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) + delete ev; + ... + } + + int apply_event_and_update_pos(Log_event *ev, ...) + { ... + ev->apply_event(...); + ... + } + + int Log_event::apply_event(...) + { + return do_apply_event(...); + } + +What does it mean to "apply" an Annotate_rows event? It means to set current +thd query to that of the described by the event, i.e. to the query which +caused the subsequent Rows events (see "How Master writes Annotate_rows +events to the binary log" to follow what happens further when the subsequent +Rows events is applied): + + int Annotate_rows_log_event::do_apply_event(...) + { + thd->set_query(m_query_txt, m_query_len); + } + +NOTE. I am not sure, but possibly current values of thd->query and +thd->query_length should be saved before calling set_query() and to be +restored on the Annotate_rows_log_event object deletion. +Is it really needed ? + +After calling this do_apply_event() function we may not delete the +Annotate_rows_log_event object immediatedly (see exec_relay_log_event() +above) because thd->query now points to the string inside this object. +We may keep the pointer to this object in the Relay_log_info: + + class Relay_log_info + { + public: + ... + void set_annotate_event(Annotate_rows_log_event*); + Annotate_rows_log_event* get_annotate_event(); + void free_annotate_event(); + ... + private: + Annotate_rows_log_event* m_annotate_event; + }; + +When the saved Annotate_rows object may be deleted? When all corresponding +Rows events will be processed, i.e. before processing the first non-Rows +event (note that Annotate_rows object resides in the binary log *after* +the (possible) 'BEGIN' Query event which accompanies the rows events; note +also that this deletion is adjusted with the case when some or all +corresponding Rows events are filtered out by replicate filter rules): + + int exec_relay_log_event(THD* thd, Relay_log_info* rli) + { ... + Log_event *ev= next_event(rli); + ... + if (rli->get_annotate_event() && !IS_RBR_EVENT_TYPE(ev->get_type_code())) + rli->free_annotate_event(); + + apply_event_and_update_pos(ev, ...); + + if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) + rli->set_annotate_event((Annotate_rows_log_event*) ev); + else if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) + delete ev; + ... + } + +where + + #define IS_RBR_EVENT_TYPE(type) ( (type) == TABLE_MAP_EVENT || \ + (type) == WRITE_ROWS_EVENT || \ + (type) == UPDATE_ROWS_EVENT || \ + (type) == DELETE_ROWS_EVENT ) + +8. General remarks +~~~~~~~~~~~~~~~~~~ +Kristian noticed that introducing new log event type should be coordinated +somehow with MySQL/Sun: + + Kristian: The numeric code for this event must be assigned carefully. + It should be coordinated with MySQL/Sun, otherwise we can get into a + situation where MySQL uses the same numeric code for one event that + MariaDB uses for ANNOTATE_ROWS_EVENT, which would make merging the two + impossible. + Alex: I reserved about 20 numbers not to have possible conflicts + with MySQL. + Kristian: Still, I think it would be appropriate to send a polite email + to internals(a)lists.mysql.com about this and suggesting to reserve the + event number. + +Also we should notice the introduction of the BINLOG_SEND_ANNOTATE_ROWS_EVENT +flag taking into account that MySQL/Sun may also introduce a flag with the +same value to be used in the request_dump-mysql_binlog_send interface. +But this is mainly the question of merging: if a conflict concerning this +flag occur, we may simply change the BINLOG_SEND_ANNOTATE_ROWS_EVENT value +(this does not require additional changes in the code). -=-=(Alexi - Sat, 19 Dec 2009, 15:41)=-=- High-Level Specification modified. --- /tmp/wklog.47.old.14545 2009-12-19 15:41:21.000000000 +0200 +++ /tmp/wklog.47.new.14545 2009-12-19 15:41:21.000000000 +0200 @@ -1,122 +1,107 @@ -First suggestion: - -> I think for this we would actually need a new binlog event type -> (Comment_log_event?). Unless we want to log an empty statement Query_log_event -> containing only a comment (a bit of a hack). - -New server option -~~~~~~~~~~~~~~~~~ - --binlog-annotate-rows-events - -Setting this option makes RBR (rows-) events in the binary log to be -preceded by Annotate rows events (see below). The corresponding -'binlog_annotate_rows_events' system variable is dynamic and has both -global and session values. Default global value is OFF. - -Note. Session values are usefull to make it possible to annotate only - some selected statements: +Content +~~~~~~~ + 1. Annotate_rows_log_event + 2. Server option: --binlog-annotate-rows-events + 3. Server option: --replicate-annotate-rows-events + 4. mysqlbinlog option: --print-annotate-rows-events + 5. mysqlbinlog output + +1. Annotate_rows_log_event [ ANNOTATE_ROWS_EVENT ] +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Describes the query which caused the corresponding rows events. Has empty +post-header and contains the query text in its data part. Example: + + ************************ + ANNOTATE_RBR_EVENT + ************************ + 00000220 | B6 A0 2C 4B | time_when = 1261215926 + 00000224 | 33 | event_type = 51 + 00000225 | 64 00 00 00 | server_id = 100 + 00000229 | 36 00 00 00 | event_len = 54 + 0000022D | 56 02 00 00 | log_pos = 00000256 + 00000231 | 00 00 | flags = <none> + ------------------------ + 00000233 | 49 4E 53 45 | query = "INSERT INTO t1 VALUES (1), (2), (3)" + 00000237 | 52 54 20 49 | + 0000023B | 4E 54 4F 20 | + 0000023F | 74 31 20 56 | + 00000243 | 41 4C 55 45 | + 00000247 | 53 20 28 31 | + 0000024B | 29 2C 20 28 | + 0000024F | 32 29 2C 20 | + 00000253 | 28 33 29 | + ************************ + +In binary log, Annotate_rows event follows the (possible) 'BEGIN' Query event +and precedes the first of Table map events which accompany the corresponding +rows events. (See example in the "mysqlbinlog output" section below.) + +2. Server option: --binlog-annotate-rows-events +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Tells the master to write Annotate_rows events to the binary log. + + * Variable Name: binlog_annotate_rows_events + * Scope: Global & Session + * Access Type: Dynamic + * Data Type: bool + * Default Value: OFF +NOTE. Session values allows to annotate only some selected statements: ... SET SESSION binlog_annotate_rows_events=ON; ... statements to be annotated ... SET SESSION binlog_annotate_rows_events=OFF; ... statements not to be annotated ... -New binlog event type -~~~~~~~~~~~~~~~~~~~~~ - Annotate_rows_log_event [ ANNOTATE_ROWS_EVENT ] - -Describes the query which caused the corresponding rows event. In binary log, -precedes each Table_map_log_event. Contains empty post-header and the query -text in its data part. - -The numeric code for this event must be assigned carefully. It should be -coordinated with MySQL/Sun, otherwise we can get into a situation where MySQL -uses the same numeric code for one event that MariaDB uses for -ANNOTATE_ROWS_EVENT, which would make merging the two impossible. - -Example: - - ... - ************************ - ANNOTATE_ROWS_EVENT [51] - ************************ - 000000C7 | 54 1B 12 4B | time_when = 1259477844 - 000000CB | 33 | event_type = 51 - 000000CC | 64 00 00 00 | server_id = 100 - 000000D0 | 2C 00 00 00 | event_len = 44 - 000000D4 | F3 00 00 00 | log_pos = 000000F3 - 000000D8 | 00 00 | flags = <none> - ------------------------ - 000000DA | 69 6E 73 65 | query = "insert into t1 values (1)" - 000000DE | 72 74 20 69 | - 000000E2 | 6E 74 6F 20 | - 000000E6 | 74 31 20 76 | - 000000EA | 61 6C 75 65 | - 000000EE | 73 20 28 31 | - 000000F2 | 29 | - ************************ - TABLE_MAP_EVENT [19] - ************************ - 000000F3 | 54 1B 12 4B | time_when = 1259477844 - 000000F7 | 13 | event_type = 19 - 000000F8 | 64 00 00 00 | server_id = 100 - 000000FC | 29 00 00 00 | event_len = 41 - 00000100 | 1C 01 00 00 | log_pos = 0000011C - 00000104 | 00 00 | flags = <none> - ------------------------ - ... - ************************ - WRITE_ROWS_EVENT [23] - ************************ - 0000011C | 54 1B 12 4B | time_when = 1259477844 - 00000120 | 17 | event_type = 23 - 00000121 | 64 00 00 00 | server_id = 100 - 00000125 | 22 00 00 00 | event_len = 34 - 00000129 | 3E 01 00 00 | log_pos = 0000013E - 0000012D | 10 00 | flags = LOG_EVENT_UPDATE_TABLE_MAP_VERSION_F - ------------------------ - 0000012F | 0F 00 00 00 | table_id = 15 - ... - -New mysqlbinlog option -~~~~~~~~~~~~~~~~~~~~~~ - --print-annotate-rows-events - -With this option, mysqlbinlog prints the content of Annotate-rows -events (if the binary log does contain them). Without this option -(i.e. by default), mysqlbinlog skips Annotate rows events. - - -mysqlbinlog output -~~~~~~~~~~~~~~~~~~ -Something like this: +3. Server option: --replicate-annotate-rows-events +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Tells the slave to reproduce Annotate_rows events recieved from the master +in its own binary log (sensible only in pair with log-slave-updates option). + + * Variable Name: replicate_annotate_rows_events + * Scope: Global + * Access Type: Read only + * Data Type: bool + * Default Value: OFF + +NOTE. Why do we additionally need this 'replicate' option? Why not to make +the slave to reproduce this events when its binlog-annotate-rows-events +global value is ON? Well, because, for example, we may want to configure +the slave which should reproduce Annotate_rows events but has global +binlog-annotate-rows-events = OFF meaning this to be the default value for +the client threads (see also "How slave treats replicate-annotate-rows-events +option" in LLD part). + +4. mysqlbinlog option: --print-annotate-rows-events +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +With this option, mysqlbinlog prints the content of Annotate_rows events (if +the binary log does contain them). Without this option (i.e. by default), +mysqlbinlog skips Annotate_rows events. +5. mysqlbinlog output +~~~~~~~~~~~~~~~~~~~~~ +With --print-annotate-rows-events, mysqlbinlog outputs Annotate_rows events +in a form like this: ... - # at 199 - # at 243 - # at 284 - #091129 9:57:24 server id 100 end_log_pos 243 Query: `insert into t1 values -(1)` - #091129 9:57:24 server id 100 end_log_pos 284 Table_map: `test`.`t1` mapped -to number 15 - #091129 9:57:24 server id 100 end_log_pos 318 Write_rows: table id 15 + # at 1646 + #091219 12:45:26 server id 100 end_log_pos 1714 Query thread_id=1 +exec_time=0 error_code=0 + SET TIMESTAMP=1261215926/*!*/; + BEGIN + /*!*/; + # at 1714 + # at 1812 + # at 1853 + # at 1894 + # at 1938 + #091219 12:45:26 server id 100 end_log_pos 1812 Query: `DELETE t1, t2 FROM +t1 INNER JOIN t2 INNER JOIN t3 WHERE t1.a=t2.a AND t2.a=t3.a` + #091219 12:45:26 server id 100 end_log_pos 1853 Table_map: `test`.`t1` +mapped to number 16 + #091219 12:45:26 server id 100 end_log_pos 1894 Table_map: `test`.`t2` +mapped to number 17 + #091219 12:45:26 server id 100 end_log_pos 1938 Delete_rows: table id 16 + #091219 12:45:26 server id 100 end_log_pos 1982 Delete_rows: table id 17 flags: STMT_END_F - - BINLOG ' - VBsSSzNkAAAALAAAAPMAAAAAAGluc2VydCBpbnRvIHQxIHZhbHVlcyAoMSk= - VBsSSxNkAAAAKQAAABwBAAAAAA8AAAAAAAAABHRlc3QAAnQxAAEDAAE= - VBsSSxdkAAAAIgAAAD4BAAAQAA8AAAAAAAEAAf/+AQAAAA== - '/*!*/; - ### INSERT INTO test.t1 - ### SET - ### @1=1 /* INT meta=0 nullable=1 is_null=0 */ ... -When master sends Annotate rows events -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -1. Master always sends Annotate_rows events to mysqlbinlog (in - remote case). -2. Master sends Annotate_rows events to a slave only if the slave has - both log-slave-updates and binlog-annotate-rows-events options set. - -=-=(Bothorsen - Fri, 18 Dec 2009, 16:22)=-=- Add estimation time. Worked 5 hours and estimate 35 hours remain (original estimate increased by 5 hours). -=-=(Bothorsen - Fri, 18 Dec 2009, 16:16)=-=- This is the work done on this patch so far. Most of it done by Alex. Worked 15 hours and estimate 035 hours remain (original estimate increased by 50 hours). ------------------------------------------------------------ -=-=(View All Progress Notes, 20 total)=-=- http://askmonty.org/worklog/index.pl?tid=47&nolimit=1 DESCRIPTION: Store in binlog (and show in mysqlbinlog output) texts of statements that caused RBR events This is needed for (list from Monty): - Easier to understand why updates happened - Would make it easier to find out where in application things went wrong (as you can search for exact strings) - Allow one to filter things based on comments in the statement. The cost of this can be that the binlog will be approximately 2x in size (especially insert of big blob's would be a bit painful), so this should be an optional feature. HIGH-LEVEL SPECIFICATION: Content ~~~~~~~ 1. Annotate_rows_log_event 2. Server option: --binlog-annotate-rows-events 3. Server option: --replicate-annotate-rows-events 4. mysqlbinlog option: --print-annotate-rows-events 5. mysqlbinlog output 1. Annotate_rows_log_event [ ANNOTATE_ROWS_EVENT ] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Describes the query which caused the corresponding rows events. Has empty post-header and contains the query text in its data part. Example: ************************ ANNOTATE_ROWS_EVENT ************************ 00000220 | B6 A0 2C 4B | time_when = 1261215926 00000224 | 33 | event_type = 51 00000225 | 64 00 00 00 | server_id = 100 00000229 | 36 00 00 00 | event_len = 54 0000022D | 56 02 00 00 | log_pos = 00000256 00000231 | 00 00 | flags = <none> ------------------------ 00000233 | 49 4E 53 45 | query = "INSERT INTO t1 VALUES (1), (2), (3)" 00000237 | 52 54 20 49 | 0000023B | 4E 54 4F 20 | 0000023F | 74 31 20 56 | 00000243 | 41 4C 55 45 | 00000247 | 53 20 28 31 | 0000024B | 29 2C 20 28 | 0000024F | 32 29 2C 20 | 00000253 | 28 33 29 | ************************ In binary log, Annotate_rows event follows the (possible) 'BEGIN' Query event and precedes the first of Table map events which accompany the corresponding rows events. (See example in the "mysqlbinlog output" section below.) 2. Server option: --binlog-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the master to write Annotate_rows events to the binary log. * Variable Name: binlog_annotate_rows_events * Scope: Global & Session * Access Type: Dynamic * Data Type: bool * Default Value: OFF NOTE. Session values allows to annotate only some selected statements: ... SET SESSION binlog_annotate_rows_events=ON; ... statements to be annotated ... SET SESSION binlog_annotate_rows_events=OFF; ... statements not to be annotated ... 3. Server option: --replicate-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Tells the slave to reproduce Annotate_rows events recieved from the master in its own binary log (sensible only in pair with log-slave-updates option). * Variable Name: replicate_annotate_rows_events * Scope: Global * Access Type: Read only * Data Type: bool * Default Value: OFF NOTE. Why do we additionally need this 'replicate' option? Why not to make the slave to reproduce this events when its binlog-annotate-rows-events global value is ON? Well, because, for example, we may want to configure the slave which should reproduce Annotate_rows events but has global binlog-annotate-rows-events = OFF meaning this to be the default value for the client threads (see also "How slave treats replicate-annotate-rows-events option" in LLD part). 4. mysqlbinlog option: --print-annotate-rows-events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ With this option, mysqlbinlog prints the content of Annotate_rows events (if the binary log does contain them). Without this option (i.e. by default), mysqlbinlog skips Annotate_rows events. 5. mysqlbinlog output ~~~~~~~~~~~~~~~~~~~~~ With --print-annotate-rows-events, mysqlbinlog outputs Annotate_rows events in a form like this: ... # at 1646 #091219 12:45:26 server id 100 end_log_pos 1714 Query thread_id=1 exec_time=0 error_code=0 SET TIMESTAMP=1261215926/*!*/; BEGIN /*!*/; # at 1714 # at 1812 # at 1853 # at 1894 # at 1938 #091219 12:45:26 server id 100 end_log_pos 1812 Query: `DELETE t1, t2 FROM t1 INNER JOIN t2 INNER JOIN t3 WHERE t1.a=t2.a AND t2.a=t3.a` #091219 12:45:26 server id 100 end_log_pos 1853 Table_map: `test`.`t1` mapped to number 16 #091219 12:45:26 server id 100 end_log_pos 1894 Table_map: `test`.`t2` mapped to number 17 #091219 12:45:26 server id 100 end_log_pos 1938 Delete_rows: table id 16 #091219 12:45:26 server id 100 end_log_pos 1982 Delete_rows: table id 17 flags: STMT_END_F ... LOW-LEVEL DESIGN: Content ~~~~~~~ 1. Annotate_rows event number 2. Outline of Annotate_rows event behavior 3. How Master writes Annotate_rows events to the binary log 4. How slave treats replicate-annotate-rows-events option 5. How slave IO thread requests Annotate_rows events 6. How master executes the request 7. How slave SQL thread processes Annotate_rows events 8. General remarks 1. Annotate_rows event number ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To avoid possible event numbers conflict with MySQL/Sun, we leave a gap between the last MySQL event number and the Annotate_rows event number: enum Log_event_type { ... INCIDENT_EVENT= 26, // New MySQL event numbers are to be added here MYSQL_EVENTS_END, MARIA_EVENTS_BEGIN= 51, // New Maria event numbers start from here ANNOTATE_ROWS_EVENT= 51, ENUM_END_EVENT }; together with the corresponding extension of 'post_header_len' array in the Format description event. (This extension does not affect the compatibility of the binary log). Here is how Format description event looks like with this extension: ************************ FORMAT_DESCRIPTION_EVENT ************************ 00000004 | A1 A0 2C 4B | time_when = 1261215905 00000008 | 0F | event_type = 15 00000009 | 64 00 00 00 | server_id = 100 0000000D | 7F 00 00 00 | event_len = 127 00000011 | 83 00 00 00 | log_pos = 00000083 00000015 | 01 00 | flags = LOG_EVENT_BINLOG_IN_USE_F ------------------------ 00000017 | 04 00 | binlog_ver = 4 00000019 | 35 2E 32 2E | server_ver = 5.2.0-MariaDB-alpha-debug-log ..... ... 0000004B | A1 A0 2C 4B | time_created = 1261215905 0000004F | 13 | common_header_len = 19 ------------------------ post_header_len ------------------------ 00000050 | 38 | 56 - START_EVENT_V3 [1] ..... ... 00000069 | 02 | 2 - INCIDENT_EVENT [26] 0000006A | 00 | 0 - RESERVED [27] ..... ... 00000081 | 00 | 0 - RESERVED [50] 00000082 | 00 | 0 - ANNOTATE_ROWS_EVENT [51] ************************ 2. Outline of Annotate_rows event behavior ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Each Annotate_rows_log_event object has two private members describing the corresponding query: char *m_query_txt; uint m_query_len; When the object is created for writing to a binary log, this query is taken from 'thd' (for short, below we omit the 'Annotate_rows_log_event::' prefix as well as other implementation details): Annotate_rows_log_event(THD *thd) { m_query_txt = thd->query(); m_query_len = thd->query_length(); } When the object is read from a binary log, the query is taken from the buffer containing the binary log representation of the event (this buffer is allocated in Log_event object from which all Log events are derived): Annotate_rows_log_event(char *buf, uint event_len, Format_description_log_event *desc) { m_query_len = event_len - desc->common_header_len; m_query_txt = buf + desc->common_header_len; } The events are written to the binary log by the Log_event::write() member which calls virtual write_data_header() and write_data_body() members ("data header" and "post header" are synonym in replication terminology). In our case, data header is empty and data body is just the query: bool write_data_body(IO_CACHE *file) { return my_b_safe_write(file, (uchar*) m_query_txt, m_query_len); } Printing the event is just printing the query: void Annotate_rows_log_event::print(FILE *file, PRINT_EVENT_INFO *pinfo) { my_b_printf(&pinfo->head_cache, "\tQuery: `%s`\n", m_query_txt); } 3. How Master writes Annotate_rows events to the binary log ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The event is written to the binary log just before the group of Table_map events which precede corresponding Rows events (one query may generate several Table map events in the binary log, but the corresponding Annotate_rows event must be written only once before the first Table map event; hence the boolean variable 'with_annotate' below): int write_locked_table_maps(THD *thd) { ... bool with_annotate= thd->variables.binlog_annotate_rows_events; ... for (uint i= 0; i < ... <number of tables> ...; ++i) { ... thd->binlog_write_table_map(table, ..., with_annotate); with_annotate= 0; // write Annotate_event not more than once ... } ... } int THD::binlog_write_table_map(TABLE *table, ..., bool with_annotate) { ... Table_map_log_event the_event(...); ... if (with_annotate) { Annotate_rows_log_event anno(this); mysql_bin_log.write(&anno); } mysql_bin_log.write(&the_event); ... } 4. How slave treats replicate-annotate-rows-events option ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The replicate-annotate-rows-events option is treated just as the session value of the binlog_annotate_rows_events variable for the slave IO and SQL threads. This setting is done during initialization of these threads: pthread_handler_t handle_slave_io(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_IO); ... } pthread_handler_t handle_slave_sql(void *arg) { THD *thd= new THD; ... init_slave_thread(thd, SLAVE_THD_SQL); ... } int init_slave_thread(THD* thd, SLAVE_THD_TYPE thd_type) { ... thd->variables.binlog_annotate_rows_events= opt_replicate_annotate_rows_events; ... } 5. How slave IO thread requests Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If the replicate-annotate-rows-events option is not set on a slave, there is no need for master to send Annotate_rows events to this slave. The slave (or mysqlbinlog in remote case), before requesting binlog dump via the COM_BINLOG_DUMP command, informs the master whether it should send these events by executing the newly added COM_BINLOG_DUMP_OPTIONS_EXT server command: case COM_BINLOG_DUMP_OPTIONS_EXT: thd->binlog_dump_flags_ext= packet[0]; my_ok(thd); break; Note. We add this new command and don't use COM_BINLOG_DUMP to avoid possible conflicts with MySQL/Sun. 6. How master executes the request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ case COM_BINLOG_DUMP: { ... flags= uint2korr(packet + 4); ... mysql_binlog_send(thd, ..., flags); ... } void mysql_binlog_send(THD* thd, ..., ushort flags) { ... Log_event::read_log_event(&log, packet, ...); ... if ((*packet)[EVENT_TYPE_OFFSET + 1] != ANNOTATE_ROWS_EVENT || flags & BINLOG_SEND_ANNOTATE_ROWS_EVENT) { my_net_write(net, packet->ptr(), packet->length()); } ... } 7. How slave SQL thread processes Annotate_rows events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The slave processes each recieved event by "applying" it, i.e. by calling the Log_event::apply_event() function which in turn calls the virtual do_apply_event() member specific for each type of the event. int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev = next_event(rli); ... apply_event_and_update_pos(ev, ...); if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } int apply_event_and_update_pos(Log_event *ev, ...) { ... ev->apply_event(...); ... } int Log_event::apply_event(...) { return do_apply_event(...); } What does it mean to "apply" an Annotate_rows event? It means to set current thd query to that of the described by the event, i.e. to the query which caused the subsequent Rows events (see "How Master writes Annotate_rows events to the binary log" to follow what happens further when the subsequent Rows events are applied): int Annotate_rows_log_event::do_apply_event(...) { thd->set_query(m_query_txt, m_query_len); } NOTE. I am not sure, but possibly current values of thd->query and thd->query_length should be saved before calling set_query() and to be restored on the Annotate_rows_log_event object deletion. Is it really needed ? After calling this do_apply_event() function we may not delete the Annotate_rows_log_event object immediatedly (see exec_relay_log_event() above) because thd->query now points to the string inside this object. We may keep the pointer to this object in the Relay_log_info: class Relay_log_info { public: ... void set_annotate_event(Annotate_rows_log_event*); Annotate_rows_log_event* get_annotate_event(); void free_annotate_event(); ... private: Annotate_rows_log_event* m_annotate_event; }; The saved Annotate_rows object should be deleted when all corresponding Rows events will be processed: int exec_relay_log_event(THD* thd, Relay_log_info* rli) { ... Log_event *ev= next_event(rli); ... apply_event_and_update_pos(ev, ...); if (rli->get_annotate_event() && is_last_rows_event(ev)) rli->free_annotate_event(); else if (ev->get_type_code() == ANNOTATE_ROWS_EVENT) rli->set_annotate_event((Annotate_rows_log_event*) ev); else if (ev->get_type_code() != FORMAT_DESCRIPTION_EVENT) delete ev; ... } where bool is_last_rows_event(Log_event* ev) { Log_event_type type= ev->get_type_code(); if (IS_ROWS_EVENT_TYPE(type)) { Rows_log_event* rows= (Rows_log_event*)ev; return rows->get_flags(Rows_log_event::STMT_END_F); } return 0; } #define IS_ROWS_EVENT_TYPE(type) ((type) == WRITE_ROWS_EVENT || \ (type) == UPDATE_ROWS_EVENT || \ (type) == DELETE_ROWS_EVENT) 8. General remarks ~~~~~~~~~~~~~~~~~~ Kristian noticed that introducing new log event type should be coordinated somehow with MySQL/Sun: Kristian: The numeric code for this event must be assigned carefully. It should be coordinated with MySQL/Sun, otherwise we can get into a situation where MySQL uses the same numeric code for one event that MariaDB uses for ANNOTATE_ROWS_EVENT, which would make merging the two impossible. Alex: I reserved about 20 numbers not to have possible conflicts with MySQL. Kristian: Still, I think it would be appropriate to send a polite email to internals(a)lists.mysql.com about this and suggesting to reserve the event number. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] SHOW PROFILE enhancements for Windows
by Alex Budovski 04 Feb '10

04 Feb '10

Hi, I've implemented a few things for the Windows port of SHOW PROFILE ( IO read/writes, user/kernel times, page faults) which you may want to consider looking at. There's also a few miscellaneous fixes in there. The commit log has the details. See: https://launchpad.net/~abudovski/maria/robust Thanks.

2 7

[Maria-developers] Rev 7: Added make install and enabled system tables as requested by Monty. in file:///Users/hakan/work/monty_program/mariadb-tools/
by Hakan Kuecuekyilmaz 04 Feb '10

04 Feb '10

At file:///Users/hakan/work/monty_program/mariadb-tools/ ------------------------------------------------------------ revno: 7 revision-id: hakan(a)askmonty.org-20100204011956-u74a4es6oogvd50w parent: hakan(a)askmonty.org-20100115142922-bgc032k180y2i29r committer: Hakan Kuecuekyilmaz <hakan(a)askmonty.org> branch nick: mariadb-tools timestamp: Thu 2010-02-04 02:19:56 +0100 message: Added make install and enabled system tables as requested by Monty. === modified file 'sql-bench/run-sql-bench.sh' --- a/sql-bench/run-sql-bench.sh 2010-01-12 12:42:59 +0000 +++ b/sql-bench/run-sql-bench.sh 2010-02-04 01:19:56 +0000 @@ -51,7 +51,8 @@ # Binaries. # BZR='/usr/local/bin/bzr' -MYSQLADMIN='client/mysqladmin' +#BZR='/usr/bin/bzr' +MYSQLADMIN='bin/mysqladmin' # # Check system. @@ -139,7 +140,9 @@ exit 1 fi - ./configure $MARIADB_CONFIG + # We need --prefix for running make install. Otherwise + # mysql_install_db does not work properly. + ./configure $MARIADB_CONFIG --prefix=${TEMP_DIR}/install if [ $? != 0 ]; then echo "[ERROR]: ./configure $MARIADB_CONFIG failed." echo " Please check your MARIADB_CONFIG in $i." @@ -157,10 +160,24 @@ exit 1 fi + make install + if [ $? != 0 ]; then + echo '[ERROR]: make install.' + echo ' Please check your build logs.' + echo 'Exiting.' + + exit 1 + fi + + cd ${TEMP_DIR}/install + + # Install system tables. + bin/mysql_install_db --no-defaults --basedir=${TEMP_DIR}/install --datadir=${TEMP_DIR}/data + # Start mysqld. MARIADB_SOCKET="${TEMP_DIR}/mysql.sock" MARIADB_OPTIONS="$MARIADB_OPTIONS \ - --datadir=$TEMP_DIR \ + --datadir=${TEMP_DIR}/data \ --tmpdir=$TEMP_DIR \ --socket=$MARIADB_SOCKET" @@ -168,10 +185,10 @@ --socket=$MARIADB_SOCKET" # Determine mysqld version for result file naming. - MARIADB_VERSION=$(sql/mysqld --version | awk '{ print $3 }') + MARIADB_VERSION=$(libexec/mysqld --version | awk '{ print $3 }') SUFFIX="$SUFFIX"-"$MARIADB_VERSION" - sql/mysqld $MARIADB_OPTIONS & + libexec/mysqld $MARIADB_OPTIONS & j=0 STARTED=-1 @@ -196,15 +213,6 @@ exit 1 fi - $MYSQLADMIN $MYSQLADMIN_OPTIONS create test - if [ $? != 0 ]; then - echo '[ERROR]: Create schema test failed.' - echo ' Please check your mysqld error log.' - echo 'Exiting.' - - exit 1 - fi - # Run sql-bench. cd sql-bench COMMENTS="Revision used: $REVISION_ID \ === modified file 'sql-bench/sql-bench-configurations/sql-bench-default-config.inc' --- a/sql-bench/sql-bench-configurations/sql-bench-default-config.inc 2009-12-29 13:42:53 +0000 +++ b/sql-bench/sql-bench-configurations/sql-bench-default-config.inc 2010-02-04 01:19:56 +0000 @@ -16,11 +16,9 @@ # # Options for starting mysqld. # -# Attention: Do not set --datadir, --tmpdir, and --socket. +# Attention: Do not set --datadir, --tmpdir, nor --socket. # -export MARIADB_OPTIONS="--no-defaults \ - --skip-grant-tables \ - --language=./sql/share/english" +export MARIADB_OPTIONS="--no-defaults" # # Options for sql-bench.

1 0

[Maria-developers] Rev 7: Added make install and enabled system tables. in file:///Users/hakan/work/monty_program/mariadb-tools/
by Hakan Kuecuekyilmaz 04 Feb '10

04 Feb '10

At file:///Users/hakan/work/monty_program/mariadb-tools/ ------------------------------------------------------------ revno: 7 revision-id: hakan(a)askmonty.org-20100204011810-rqj1ikb41773p0au parent: hakan(a)askmonty.org-20100115142922-bgc032k180y2i29r committer: Hakan Kuecuekyilmaz <hakan(a)askmonty.org> branch nick: mariadb-tools timestamp: Thu 2010-02-04 02:18:10 +0100 message: Added make install and enabled system tables. === modified file 'sql-bench/run-sql-bench.sh' --- a/sql-bench/run-sql-bench.sh 2010-01-12 12:42:59 +0000 +++ b/sql-bench/run-sql-bench.sh 2010-02-04 01:18:10 +0000 @@ -50,8 +50,9 @@ # # Binaries. # -BZR='/usr/local/bin/bzr' -MYSQLADMIN='client/mysqladmin' +#BZR='/usr/local/bin/bzr' +BZR='/usr/bin/bzr' +MYSQLADMIN='bin/mysqladmin' # # Check system. @@ -139,7 +140,9 @@ exit 1 fi - ./configure $MARIADB_CONFIG + # We need --prefix for running make install. Otherwise + # mysql_install_db does not work properly. + ./configure $MARIADB_CONFIG --prefix=${TEMP_DIR}/install if [ $? != 0 ]; then echo "[ERROR]: ./configure $MARIADB_CONFIG failed." echo " Please check your MARIADB_CONFIG in $i." @@ -157,10 +160,24 @@ exit 1 fi + make install + if [ $? != 0 ]; then + echo '[ERROR]: make install.' + echo ' Please check your build logs.' + echo 'Exiting.' + + exit 1 + fi + + cd ${TEMP_DIR}/install + + # Install system tables. + bin/mysql_install_db --no-defaults --basedir=${TEMP_DIR}/install --datadir=${TEMP_DIR}/data + # Start mysqld. MARIADB_SOCKET="${TEMP_DIR}/mysql.sock" MARIADB_OPTIONS="$MARIADB_OPTIONS \ - --datadir=$TEMP_DIR \ + --datadir=${TEMP_DIR}/data \ --tmpdir=$TEMP_DIR \ --socket=$MARIADB_SOCKET" @@ -168,10 +185,10 @@ --socket=$MARIADB_SOCKET" # Determine mysqld version for result file naming. - MARIADB_VERSION=$(sql/mysqld --version | awk '{ print $3 }') + MARIADB_VERSION=$(libexec/mysqld --version | awk '{ print $3 }') SUFFIX="$SUFFIX"-"$MARIADB_VERSION" - sql/mysqld $MARIADB_OPTIONS & + libexec/mysqld $MARIADB_OPTIONS & j=0 STARTED=-1 @@ -196,15 +213,6 @@ exit 1 fi - $MYSQLADMIN $MYSQLADMIN_OPTIONS create test - if [ $? != 0 ]; then - echo '[ERROR]: Create schema test failed.' - echo ' Please check your mysqld error log.' - echo 'Exiting.' - - exit 1 - fi - # Run sql-bench. cd sql-bench COMMENTS="Revision used: $REVISION_ID \ === modified file 'sql-bench/sql-bench-configurations/sql-bench-default-config.inc' --- a/sql-bench/sql-bench-configurations/sql-bench-default-config.inc 2009-12-29 13:42:53 +0000 +++ b/sql-bench/sql-bench-configurations/sql-bench-default-config.inc 2010-02-04 01:18:10 +0000 @@ -16,11 +16,9 @@ # # Options for starting mysqld. # -# Attention: Do not set --datadir, --tmpdir, and --socket. +# Attention: Do not set --datadir, --tmpdir, nor --socket. # -export MARIADB_OPTIONS="--no-defaults \ - --skip-grant-tables \ - --language=./sql/share/english" +export MARIADB_OPTIONS="--no-defaults" # # Options for sql-bench.

1 0

[Maria-developers] [psergey@askmonty.org: Rev 2750: BUG#31480: Incorrect result for nested subquery when executed via semi join: in file:///home/psergey/dev/maria-5.3-subqueries-r3/]
by Sergey Petrunya 02 Feb '10

02 Feb '10

Hello Igor, I believe that the following patch has the property that item_subselect->update_used_tables() will handle table-bit re-assignments, i.e. it is what we have discussed earlier today. I've also pushed this into 5.3-sj-subqueries tree. ----- Forwarded message from Sergey Petrunya <psergey(a)askmonty.org> ----- From: Sergey Petrunya <psergey(a)askmonty.org> To: maria-developers(a)lists.launchpad.net X-Mailer: mail (GNU Mailutils 1.2) Date: Tue, 2 Feb 2010 23:00:49 +0300 (MSK) Subject: [Maria-developers] Rev 2750: BUG#31480: Incorrect result for nested subquery when executed via semi join: in file:///home/psergey/dev/maria-5.3-subqueries-r3/ At file:///home/psergey/dev/maria-5.3-subqueries-r3/ ------------------------------------------------------------ revno: 2750 revision-id: psergey(a)askmonty.org-20100202200045-13q0nb5dwzm739j6 parent: psergey(a)askmonty.org-20100128134833-9000udjp5wa3tsff committer: Sergey Petrunya <psergey(a)askmonty.org> branch nick: maria-5.3-subqueries-r3 timestamp: Tue 2010-02-02 23:00:45 +0300 message: BUG#31480: Incorrect result for nested subquery when executed via semi join: A mark-2 fix that can survive FROM subquery handling and has some code unification with table elimination: Each subquery predicate now stores a (flat) list of all references from inside to outside the subquery. We actually store (select, referred_item) pairs which allows Item_subselect::fix_after_pullout() to recalculate subquery predicate's attributes after a broad range of FROM- and IN-subselect flattening operations. === modified file 'mysql-test/r/subselect_sj.result' --- a/mysql-test/r/subselect_sj.result 2010-01-17 14:51:10 +0000 +++ b/mysql-test/r/subselect_sj.result 2010-02-02 20:00:45 +0000 @@ -779,3 +779,48 @@ 1 PRIMARY it2 ALL NULL NULL NULL NULL 20 Using where; End temporary DROP TABLE ot1, it1, it2; # End of BUG#38075 +# +# BUG#31480: Incorrect result for nested subquery when executed via semi join +# +create table t1 (a int not null, b int not null); +create table t2 (c int not null, d int not null); +create table t3 (e int not null); +insert into t1 values (1,10); +insert into t1 values (2,10); +insert into t1 values (1,20); +insert into t1 values (2,20); +insert into t1 values (3,20); +insert into t1 values (2,30); +insert into t1 values (4,40); +insert into t2 values (2,10); +insert into t2 values (2,20); +insert into t2 values (4,10); +insert into t2 values (5,10); +insert into t2 values (3,20); +insert into t2 values (2,40); +insert into t3 values (10); +insert into t3 values (30); +insert into t3 values (10); +insert into t3 values (20); +explain extended +select a from t1 +where a in (select c from t2 where d >= some(select e from t3 where b=e)); +id select_type table type possible_keys key key_len ref rows filtered Extra +1 PRIMARY t2 ALL NULL NULL NULL NULL 6 100.00 Start temporary +1 PRIMARY t1 ALL NULL NULL NULL NULL 7 100.00 Using where; End temporary; Using join buffer +3 DEPENDENT SUBQUERY t3 ALL NULL NULL NULL NULL 4 100.00 Using where +Warnings: +Note 1276 Field or reference 'test.t1.b' of SELECT #3 was resolved in SELECT #1 +Note 1003 select `test`.`t1`.`a` AS `a` from `test`.`t1` semi join (`test`.`t2`) where ((`test`.`t1`.`a` = `test`.`t2`.`c`) and <nop>(<in_optimizer>(`test`.`t2`.`d`,<exists>(select 1 AS `Not_used` from `test`.`t3` where ((`test`.`t1`.`b` = `test`.`t3`.`e`) and (<cache>(`test`.`t2`.`d`) >= `test`.`t3`.`e`)))))) +show warnings; +Level Code Message +Note 1276 Field or reference 'test.t1.b' of SELECT #3 was resolved in SELECT #1 +Note 1003 select `test`.`t1`.`a` AS `a` from `test`.`t1` semi join (`test`.`t2`) where ((`test`.`t1`.`a` = `test`.`t2`.`c`) and <nop>(<in_optimizer>(`test`.`t2`.`d`,<exists>(select 1 AS `Not_used` from `test`.`t3` where ((`test`.`t1`.`b` = `test`.`t3`.`e`) and (<cache>(`test`.`t2`.`d`) >= `test`.`t3`.`e`)))))) +select a from t1 +where a in (select c from t2 where d >= some(select e from t3 where b=e)); +a +2 +2 +3 +2 +drop table t1, t2, t3; === modified file 'mysql-test/r/subselect_sj_jcl6.result' --- a/mysql-test/r/subselect_sj_jcl6.result 2010-01-17 14:51:10 +0000 +++ b/mysql-test/r/subselect_sj_jcl6.result 2010-02-02 20:00:45 +0000 @@ -783,6 +783,51 @@ 1 PRIMARY it2 ALL NULL NULL NULL NULL 20 Using where; End temporary; Using join buffer DROP TABLE ot1, it1, it2; # End of BUG#38075 +# +# BUG#31480: Incorrect result for nested subquery when executed via semi join +# +create table t1 (a int not null, b int not null); +create table t2 (c int not null, d int not null); +create table t3 (e int not null); +insert into t1 values (1,10); +insert into t1 values (2,10); +insert into t1 values (1,20); +insert into t1 values (2,20); +insert into t1 values (3,20); +insert into t1 values (2,30); +insert into t1 values (4,40); +insert into t2 values (2,10); +insert into t2 values (2,20); +insert into t2 values (4,10); +insert into t2 values (5,10); +insert into t2 values (3,20); +insert into t2 values (2,40); +insert into t3 values (10); +insert into t3 values (30); +insert into t3 values (10); +insert into t3 values (20); +explain extended +select a from t1 +where a in (select c from t2 where d >= some(select e from t3 where b=e)); +id select_type table type possible_keys key key_len ref rows filtered Extra +1 PRIMARY t2 ALL NULL NULL NULL NULL 6 100.00 Start temporary +1 PRIMARY t1 ALL NULL NULL NULL NULL 7 100.00 Using where; End temporary; Using join buffer +3 DEPENDENT SUBQUERY t3 ALL NULL NULL NULL NULL 4 100.00 Using where +Warnings: +Note 1276 Field or reference 'test.t1.b' of SELECT #3 was resolved in SELECT #1 +Note 1003 select `test`.`t1`.`a` AS `a` from `test`.`t1` semi join (`test`.`t2`) where ((`test`.`t1`.`a` = `test`.`t2`.`c`) and <nop>(<in_optimizer>(`test`.`t2`.`d`,<exists>(select 1 AS `Not_used` from `test`.`t3` where ((`test`.`t1`.`b` = `test`.`t3`.`e`) and (<cache>(`test`.`t2`.`d`) >= `test`.`t3`.`e`)))))) +show warnings; +Level Code Message +Note 1276 Field or reference 'test.t1.b' of SELECT #3 was resolved in SELECT #1 +Note 1003 select `test`.`t1`.`a` AS `a` from `test`.`t1` semi join (`test`.`t2`) where ((`test`.`t1`.`a` = `test`.`t2`.`c`) and <nop>(<in_optimizer>(`test`.`t2`.`d`,<exists>(select 1 AS `Not_used` from `test`.`t3` where ((`test`.`t1`.`b` = `test`.`t3`.`e`) and (<cache>(`test`.`t2`.`d`) >= `test`.`t3`.`e`)))))) +select a from t1 +where a in (select c from t2 where d >= some(select e from t3 where b=e)); +a +2 +2 +3 +2 +drop table t1, t2, t3; set join_cache_level=default; show variables like 'join_cache_level'; Variable_name Value === modified file 'mysql-test/t/subselect_sj.test' --- a/mysql-test/t/subselect_sj.test 2010-01-17 14:51:10 +0000 +++ b/mysql-test/t/subselect_sj.test 2010-02-02 20:00:45 +0000 @@ -681,3 +681,41 @@ DROP TABLE ot1, it1, it2; --echo # End of BUG#38075 + +--echo # +--echo # BUG#31480: Incorrect result for nested subquery when executed via semi join +--echo # +create table t1 (a int not null, b int not null); +create table t2 (c int not null, d int not null); +create table t3 (e int not null); + +insert into t1 values (1,10); +insert into t1 values (2,10); +insert into t1 values (1,20); +insert into t1 values (2,20); +insert into t1 values (3,20); +insert into t1 values (2,30); +insert into t1 values (4,40); + +insert into t2 values (2,10); +insert into t2 values (2,20); +insert into t2 values (4,10); +insert into t2 values (5,10); +insert into t2 values (3,20); +insert into t2 values (2,40); + +insert into t3 values (10); +insert into t3 values (30); +insert into t3 values (10); +insert into t3 values (20); + +explain extended +select a from t1 +where a in (select c from t2 where d >= some(select e from t3 where b=e)); +show warnings; + +select a from t1 +where a in (select c from t2 where d >= some(select e from t3 where b=e)); + +drop table t1, t2, t3; + === modified file 'sql/item.cc' --- a/sql/item.cc 2010-01-17 14:55:08 +0000 +++ b/sql/item.cc 2010-02-02 20:00:45 +0000 @@ -3646,7 +3646,7 @@ substitution) */ -static void mark_as_dependent(THD *thd, SELECT_LEX *last, SELECT_LEX *current, +static bool mark_as_dependent(THD *thd, SELECT_LEX *last, SELECT_LEX *current, Item_ident *resolved_item, Item_ident *mark_item) { @@ -3657,7 +3657,8 @@ /* store pointer on SELECT_LEX from which item is dependent */ if (mark_item) mark_item->depended_from= last; - current->mark_as_dependent(last, resolved_item); + if (current->mark_as_dependent(thd, last, resolved_item)) + return TRUE; if (thd->lex->describe & DESCRIBE_EXTENDED) { push_warning_printf(thd, MYSQL_ERROR::WARN_LEVEL_NOTE, @@ -3667,6 +3668,7 @@ resolved_item->field_name, current->select_number, last->select_number); } + return FALSE; } @@ -4118,6 +4120,7 @@ ((ref_type == REF_ITEM || ref_type == FIELD_ITEM) ? (Item_ident*) (*reference) : 0)); + /* A reference to a view field had been found and we substituted it instead of this Item (find_field_in_tables @@ -6437,7 +6440,7 @@ if (depended_from == new_parent) { *ref= outer_ref; - outer_ref->fix_after_pullout(new_parent, ref); + (*ref)->fix_after_pullout(new_parent, ref); } } === modified file 'sql/item_subselect.cc' --- a/sql/item_subselect.cc 2010-01-28 13:48:33 +0000 +++ b/sql/item_subselect.cc 2010-02-02 20:00:45 +0000 @@ -39,8 +39,8 @@ Item_subselect::Item_subselect(): Item_result_field(), value_assigned(0), thd(0), substitution(0), engine(0), old_engine(0), used_tables_cache(0), have_to_be_excluded(0), - const_item_cache(1), in_fix_fields(0), engine_changed(0), changed(0), - is_correlated(FALSE) + const_item_cache(1), inside_first_fix_fields(0), done_first_fix_fields(FALSE), + engine_changed(0), changed(0), is_correlated(FALSE) { with_subselect= 1; reset(); @@ -167,18 +167,23 @@ DBUG_ASSERT(fixed == 0); engine->set_thd((thd= thd_param)); - if (!in_fix_fields) - refers_to.empty(); + if (!done_first_fix_fields) + { + done_first_fix_fields= TRUE; + inside_first_fix_fields= TRUE; + } + eliminated= FALSE; + parent_select= thd_param->lex->current_select; if (check_stack_overrun(thd, STACK_MIN_SIZE, (uchar*)&res)) return TRUE; - in_fix_fields++; res= engine->prepare(); // all transformation is done (used by prepared statements) changed= 1; + inside_first_fix_fields= FALSE; if (!res) { @@ -210,14 +215,12 @@ if (!(*ref)->fixed) ret= (*ref)->fix_fields(thd, ref); thd->where= save_where; - in_fix_fields--; return ret; } // Is it one field subselect? if (engine->cols() > max_columns) { my_error(ER_OPERAND_COLUMNS, MYF(0), 1); - in_fix_fields--; return TRUE; } fix_length_and_dec(); @@ -234,7 +237,6 @@ fixed= 1; err: - in_fix_fields--; thd->where= save_where; return res; } @@ -242,11 +244,12 @@ bool Item_subselect::enumerate_field_refs_processor(uchar *arg) { - List_iterator<Item> it(refers_to); - Item *item; - while ((item= it++)) + List_iterator<Ref_to_outside> it(upper_refs); + Ref_to_outside *upper; + + while ((upper= it++)) { - if (item->walk(&Item::enumerate_field_refs_processor, FALSE, arg)) + if (upper->item->walk(&Item::enumerate_field_refs_processor, FALSE, arg)) return TRUE; } return FALSE; @@ -258,6 +261,115 @@ return FALSE; } + +bool Item_subselect::mark_as_dependent(THD *thd, st_select_lex *select, + Item *item) +{ + if (inside_first_fix_fields) + { + is_correlated= TRUE; + Ref_to_outside *upper; + if (!(upper= new (thd->stmt_arena->mem_root) Ref_to_outside())) + return TRUE; + upper->select= select; + upper->item= item; + if (upper_refs.push_back(upper, thd->stmt_arena->mem_root)) + return TRUE; + } + return FALSE; +} + +/* + Adjust attributes after our parent select has been merged into grandparent + + DESCRIPTION + Subquery is a composite object which may be correlated, that is, it may + have + 1. references to tables of the parent select (i.e. one that has the clause + with the subquery predicate) + 2. references to tables of the grandparent select + 3. references to tables of further ancestors. + + Before the pullout, this item indicates: + - #1 with table bits in used_tables() + - #2 and #3 with OUTER_REF_TABLE_BIT. + + After parent has been merged with grandparent: + - references to parent and grandparent tables should be indicated with + table bits. + - references to greatgrandparent and further ancestors - with + OUTER_REF_TABLE_BIT. +*/ + +void Item_subselect::fix_after_pullout(st_select_lex *new_parent, Item **ref) +{ + recalc_used_tables(new_parent, TRUE); + parent_select= new_parent; +} + + +/* + Recalculate used_tables_cache +*/ + +void Item_subselect::recalc_used_tables(st_select_lex *new_parent, + bool after_pullout) +{ + List_iterator<Ref_to_outside> it(upper_refs); + Ref_to_outside *upper; + + used_tables_cache= 0; + while ((upper= it++)) + { + bool found= FALSE; + /* + Check if + 1. the upper reference refers to the new immediate parent select, or + 2. one of the further ancestors. + + We rely on the fact that the tree of selects is modified by some kind of + 'flattening', i.e. a process where child selects are merged into their + parents. + The merged selects are removed from the select tree but keep pointers to + their parents. + */ + for (st_select_lex *sel= upper->select; sel; sel= sel->outer_select()) + { + /* + If we've reached the new parent select by walking upwards from + reference's original select, this means that the reference is now + referring to the direct parent: + */ + if (sel == new_parent) + { + found= TRUE; + /* + upper->item may be NULL when we've referred to a grouping function, + in which case we don't care about what it's table_map really is, + because item->with_sum_func==1 will ensure correct placement of the + item. + */ + if (upper->item) + { + if (after_pullout) + upper->item->fix_after_pullout(new_parent, &(upper->item)); + upper->item->update_used_tables(); + used_tables_cache |= upper->item->used_tables(); + } + } + } + if (!found) + used_tables_cache|= OUTER_REF_TABLE_BIT; + } + /* + Don't update const_tables_cache yet as we don't yet know which of the + parent's tables are constant. Parent will call update_used_tables() after + he has done const table detection, and that will be our chance to update + const_tables_cache. + */ +} + + bool Item_subselect::walk(Item_processor processor, bool walk_subquery, uchar *argument) { @@ -397,6 +509,7 @@ void Item_subselect::update_used_tables() { + recalc_used_tables(parent_select, FALSE); if (!engine->uncacheable()) { // did all used tables become static? @@ -1843,6 +1956,18 @@ return result || Item_subselect::fix_fields(thd_arg, ref); } +void Item_in_subselect::fix_after_pullout(st_select_lex *new_parent, Item **ref) +{ + left_expr->fix_after_pullout(new_parent, &left_expr); + Item_subselect::fix_after_pullout(new_parent, ref); +} + +void Item_in_subselect::update_used_tables() +{ + Item_subselect::update_used_tables(); + left_expr->update_used_tables(); + used_tables_cache |= left_expr->used_tables(); +} /** Try to create an engine to compute the subselect via materialization, === modified file 'sql/item_subselect.h' --- a/sql/item_subselect.h 2010-01-28 13:48:33 +0000 +++ b/sql/item_subselect.h 2010-02-02 20:00:45 +0000 @@ -67,14 +67,32 @@ bool have_to_be_excluded; /* cache of constant state */ bool const_item_cache; - + + bool inside_first_fix_fields; + bool done_first_fix_fields; public: - /* - References from inside the subquery to the select that this predicate is - in. References to parent selects not included. + /* A reference from inside subquery predicate to somewhere outside of it */ + class Ref_to_outside : public Sql_alloc + { + public: + st_select_lex *select; /* Select where the reference is pointing to */ + /* + What is being referred. This may be NULL when we're referring to an + aggregate function. + */ + Item *item; + }; + /* + References from within this subquery to somewhere outside of it (i.e. to + parent select, grandparent select, etc) */ - List<Item> refers_to; - int in_fix_fields; + List<Ref_to_outside> upper_refs; + st_select_lex *parent_select; + + /* + TRUE<=>Table Elimination has made it redundant to evaluate this select + (and so it is not part of QEP, etc) + */ bool eliminated; /* changed engine indicator */ @@ -117,6 +135,9 @@ return null_value; } bool fix_fields(THD *thd, Item **ref); + bool mark_as_dependent(THD *thd, st_select_lex *select, Item *item); + void fix_after_pullout(st_select_lex *new_parent, Item **ref); + void recalc_used_tables(st_select_lex *new_parent, bool after_pullout); virtual bool exec(); virtual void fix_length_and_dec(); table_map used_tables() const; @@ -396,6 +417,8 @@ bool test_limit(st_select_lex_unit *unit); virtual void print(String *str, enum_query_type query_type); bool fix_fields(THD *thd, Item **ref); + void fix_after_pullout(st_select_lex *new_parent, Item **ref); + void update_used_tables(); bool setup_engine(); bool init_left_expr_cache(); bool is_expensive_processor(uchar *arg); === modified file 'sql/item_sum.cc' --- a/sql/item_sum.cc 2009-10-15 21:38:29 +0000 +++ b/sql/item_sum.cc 2010-02-02 20:00:45 +0000 @@ -350,7 +350,7 @@ sl= sl->master_unit()->outer_select() ) sl->master_unit()->item->with_sum_func= 1; } - thd->lex->current_select->mark_as_dependent(aggr_sel, NULL); + thd->lex->current_select->mark_as_dependent(thd, aggr_sel, NULL); return FALSE; } === modified file 'sql/sql_lex.cc' --- a/sql/sql_lex.cc 2010-01-28 13:48:33 +0000 +++ b/sql/sql_lex.cc 2010-02-02 20:00:45 +0000 @@ -1841,9 +1841,8 @@ 'last' should be reachable from this st_select_lex_node */ -void st_select_lex::mark_as_dependent(st_select_lex *last, Item *dependency) +bool st_select_lex::mark_as_dependent(THD *thd, st_select_lex *last, Item *dependency) { - SELECT_LEX *next_to_last; /* Mark all selects from resolved to 1 before select where was found table as depended (of select where was found table) @@ -1867,12 +1866,15 @@ sl->uncacheable|= UNCACHEABLE_UNITED; } } - next_to_last= s; + + Item_subselect *subquery_expr= s->master_unit()->item; + if (subquery_expr && subquery_expr->mark_as_dependent(thd, last, + dependency)) + return TRUE; } is_correlated= TRUE; this->master_unit()->item->is_correlated= TRUE; - if (dependency) - next_to_last->master_unit()->item->refers_to.push_back(dependency); + return FALSE; } bool st_select_lex_node::set_braces(bool value) { return 1; } === modified file 'sql/sql_lex.h' --- a/sql/sql_lex.h 2010-01-28 13:48:33 +0000 +++ b/sql/sql_lex.h 2010-02-02 20:00:45 +0000 @@ -747,7 +747,7 @@ return master_unit()->return_after_parsing(); } - void mark_as_dependent(st_select_lex *last, Item *dependency); + bool mark_as_dependent(THD *thd, st_select_lex *last, Item *dependency); bool set_braces(bool value); bool inc_in_sum_expr(); === modified file 'sql/sql_select.h' --- a/sql/sql_select.h 2010-01-28 13:48:33 +0000 +++ b/sql/sql_select.h 2010-02-02 20:00:45 +0000 @@ -282,13 +282,11 @@ } bool check_rowid_field() { -/* !!!NB igor: enable the code in this comment after backporting the SJ code if (keep_current_rowid && !used_rowid_fields) { used_rowid_fields= 1; used_fieldlength+= table->file->ref_length; } -*/ return test(used_rowid_fields); } bool is_inner_table_of_semi_join_with_first_match() _______________________________________________ Mailing list: https://launchpad.net/~maria-developers Post to : maria-developers(a)lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-developers More help : https://help.launchpad.net/ListHelp ----- End forwarded message ----- -- BR Sergey -- Sergey Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] Rev 2750: BUG#31480: Incorrect result for nested subquery when executed via semi join: in file:///home/psergey/dev/maria-5.3-subqueries-r3/
by Sergey Petrunya 02 Feb '10

02 Feb '10

At file:///home/psergey/dev/maria-5.3-subqueries-r3/ ------------------------------------------------------------ revno: 2750 revision-id: psergey(a)askmonty.org-20100202200045-13q0nb5dwzm739j6 parent: psergey(a)askmonty.org-20100128134833-9000udjp5wa3tsff committer: Sergey Petrunya <psergey(a)askmonty.org> branch nick: maria-5.3-subqueries-r3 timestamp: Tue 2010-02-02 23:00:45 +0300 message: BUG#31480: Incorrect result for nested subquery when executed via semi join: A mark-2 fix that can survive FROM subquery handling and has some code unification with table elimination: Each subquery predicate now stores a (flat) list of all references from inside to outside the subquery. We actually store (select, referred_item) pairs which allows Item_subselect::fix_after_pullout() to recalculate subquery predicate's attributes after a broad range of FROM- and IN-subselect flattening operations. === modified file 'mysql-test/r/subselect_sj.result' --- a/mysql-test/r/subselect_sj.result 2010-01-17 14:51:10 +0000 +++ b/mysql-test/r/subselect_sj.result 2010-02-02 20:00:45 +0000 @@ -779,3 +779,48 @@ 1 PRIMARY it2 ALL NULL NULL NULL NULL 20 Using where; End temporary DROP TABLE ot1, it1, it2; # End of BUG#38075 +# +# BUG#31480: Incorrect result for nested subquery when executed via semi join +# +create table t1 (a int not null, b int not null); +create table t2 (c int not null, d int not null); +create table t3 (e int not null); +insert into t1 values (1,10); +insert into t1 values (2,10); +insert into t1 values (1,20); +insert into t1 values (2,20); +insert into t1 values (3,20); +insert into t1 values (2,30); +insert into t1 values (4,40); +insert into t2 values (2,10); +insert into t2 values (2,20); +insert into t2 values (4,10); +insert into t2 values (5,10); +insert into t2 values (3,20); +insert into t2 values (2,40); +insert into t3 values (10); +insert into t3 values (30); +insert into t3 values (10); +insert into t3 values (20); +explain extended +select a from t1 +where a in (select c from t2 where d >= some(select e from t3 where b=e)); +id select_type table type possible_keys key key_len ref rows filtered Extra +1 PRIMARY t2 ALL NULL NULL NULL NULL 6 100.00 Start temporary +1 PRIMARY t1 ALL NULL NULL NULL NULL 7 100.00 Using where; End temporary; Using join buffer +3 DEPENDENT SUBQUERY t3 ALL NULL NULL NULL NULL 4 100.00 Using where +Warnings: +Note 1276 Field or reference 'test.t1.b' of SELECT #3 was resolved in SELECT #1 +Note 1003 select `test`.`t1`.`a` AS `a` from `test`.`t1` semi join (`test`.`t2`) where ((`test`.`t1`.`a` = `test`.`t2`.`c`) and <nop>(<in_optimizer>(`test`.`t2`.`d`,<exists>(select 1 AS `Not_used` from `test`.`t3` where ((`test`.`t1`.`b` = `test`.`t3`.`e`) and (<cache>(`test`.`t2`.`d`) >= `test`.`t3`.`e`)))))) +show warnings; +Level Code Message +Note 1276 Field or reference 'test.t1.b' of SELECT #3 was resolved in SELECT #1 +Note 1003 select `test`.`t1`.`a` AS `a` from `test`.`t1` semi join (`test`.`t2`) where ((`test`.`t1`.`a` = `test`.`t2`.`c`) and <nop>(<in_optimizer>(`test`.`t2`.`d`,<exists>(select 1 AS `Not_used` from `test`.`t3` where ((`test`.`t1`.`b` = `test`.`t3`.`e`) and (<cache>(`test`.`t2`.`d`) >= `test`.`t3`.`e`)))))) +select a from t1 +where a in (select c from t2 where d >= some(select e from t3 where b=e)); +a +2 +2 +3 +2 +drop table t1, t2, t3; === modified file 'mysql-test/r/subselect_sj_jcl6.result' --- a/mysql-test/r/subselect_sj_jcl6.result 2010-01-17 14:51:10 +0000 +++ b/mysql-test/r/subselect_sj_jcl6.result 2010-02-02 20:00:45 +0000 @@ -783,6 +783,51 @@ 1 PRIMARY it2 ALL NULL NULL NULL NULL 20 Using where; End temporary; Using join buffer DROP TABLE ot1, it1, it2; # End of BUG#38075 +# +# BUG#31480: Incorrect result for nested subquery when executed via semi join +# +create table t1 (a int not null, b int not null); +create table t2 (c int not null, d int not null); +create table t3 (e int not null); +insert into t1 values (1,10); +insert into t1 values (2,10); +insert into t1 values (1,20); +insert into t1 values (2,20); +insert into t1 values (3,20); +insert into t1 values (2,30); +insert into t1 values (4,40); +insert into t2 values (2,10); +insert into t2 values (2,20); +insert into t2 values (4,10); +insert into t2 values (5,10); +insert into t2 values (3,20); +insert into t2 values (2,40); +insert into t3 values (10); +insert into t3 values (30); +insert into t3 values (10); +insert into t3 values (20); +explain extended +select a from t1 +where a in (select c from t2 where d >= some(select e from t3 where b=e)); +id select_type table type possible_keys key key_len ref rows filtered Extra +1 PRIMARY t2 ALL NULL NULL NULL NULL 6 100.00 Start temporary +1 PRIMARY t1 ALL NULL NULL NULL NULL 7 100.00 Using where; End temporary; Using join buffer +3 DEPENDENT SUBQUERY t3 ALL NULL NULL NULL NULL 4 100.00 Using where +Warnings: +Note 1276 Field or reference 'test.t1.b' of SELECT #3 was resolved in SELECT #1 +Note 1003 select `test`.`t1`.`a` AS `a` from `test`.`t1` semi join (`test`.`t2`) where ((`test`.`t1`.`a` = `test`.`t2`.`c`) and <nop>(<in_optimizer>(`test`.`t2`.`d`,<exists>(select 1 AS `Not_used` from `test`.`t3` where ((`test`.`t1`.`b` = `test`.`t3`.`e`) and (<cache>(`test`.`t2`.`d`) >= `test`.`t3`.`e`)))))) +show warnings; +Level Code Message +Note 1276 Field or reference 'test.t1.b' of SELECT #3 was resolved in SELECT #1 +Note 1003 select `test`.`t1`.`a` AS `a` from `test`.`t1` semi join (`test`.`t2`) where ((`test`.`t1`.`a` = `test`.`t2`.`c`) and <nop>(<in_optimizer>(`test`.`t2`.`d`,<exists>(select 1 AS `Not_used` from `test`.`t3` where ((`test`.`t1`.`b` = `test`.`t3`.`e`) and (<cache>(`test`.`t2`.`d`) >= `test`.`t3`.`e`)))))) +select a from t1 +where a in (select c from t2 where d >= some(select e from t3 where b=e)); +a +2 +2 +3 +2 +drop table t1, t2, t3; set join_cache_level=default; show variables like 'join_cache_level'; Variable_name Value === modified file 'mysql-test/t/subselect_sj.test' --- a/mysql-test/t/subselect_sj.test 2010-01-17 14:51:10 +0000 +++ b/mysql-test/t/subselect_sj.test 2010-02-02 20:00:45 +0000 @@ -681,3 +681,41 @@ DROP TABLE ot1, it1, it2; --echo # End of BUG#38075 + +--echo # +--echo # BUG#31480: Incorrect result for nested subquery when executed via semi join +--echo # +create table t1 (a int not null, b int not null); +create table t2 (c int not null, d int not null); +create table t3 (e int not null); + +insert into t1 values (1,10); +insert into t1 values (2,10); +insert into t1 values (1,20); +insert into t1 values (2,20); +insert into t1 values (3,20); +insert into t1 values (2,30); +insert into t1 values (4,40); + +insert into t2 values (2,10); +insert into t2 values (2,20); +insert into t2 values (4,10); +insert into t2 values (5,10); +insert into t2 values (3,20); +insert into t2 values (2,40); + +insert into t3 values (10); +insert into t3 values (30); +insert into t3 values (10); +insert into t3 values (20); + +explain extended +select a from t1 +where a in (select c from t2 where d >= some(select e from t3 where b=e)); +show warnings; + +select a from t1 +where a in (select c from t2 where d >= some(select e from t3 where b=e)); + +drop table t1, t2, t3; + === modified file 'sql/item.cc' --- a/sql/item.cc 2010-01-17 14:55:08 +0000 +++ b/sql/item.cc 2010-02-02 20:00:45 +0000 @@ -3646,7 +3646,7 @@ substitution) */ -static void mark_as_dependent(THD *thd, SELECT_LEX *last, SELECT_LEX *current, +static bool mark_as_dependent(THD *thd, SELECT_LEX *last, SELECT_LEX *current, Item_ident *resolved_item, Item_ident *mark_item) { @@ -3657,7 +3657,8 @@ /* store pointer on SELECT_LEX from which item is dependent */ if (mark_item) mark_item->depended_from= last; - current->mark_as_dependent(last, resolved_item); + if (current->mark_as_dependent(thd, last, resolved_item)) + return TRUE; if (thd->lex->describe & DESCRIBE_EXTENDED) { push_warning_printf(thd, MYSQL_ERROR::WARN_LEVEL_NOTE, @@ -3667,6 +3668,7 @@ resolved_item->field_name, current->select_number, last->select_number); } + return FALSE; } @@ -4118,6 +4120,7 @@ ((ref_type == REF_ITEM || ref_type == FIELD_ITEM) ? (Item_ident*) (*reference) : 0)); + /* A reference to a view field had been found and we substituted it instead of this Item (find_field_in_tables @@ -6437,7 +6440,7 @@ if (depended_from == new_parent) { *ref= outer_ref; - outer_ref->fix_after_pullout(new_parent, ref); + (*ref)->fix_after_pullout(new_parent, ref); } } === modified file 'sql/item_subselect.cc' --- a/sql/item_subselect.cc 2010-01-28 13:48:33 +0000 +++ b/sql/item_subselect.cc 2010-02-02 20:00:45 +0000 @@ -39,8 +39,8 @@ Item_subselect::Item_subselect(): Item_result_field(), value_assigned(0), thd(0), substitution(0), engine(0), old_engine(0), used_tables_cache(0), have_to_be_excluded(0), - const_item_cache(1), in_fix_fields(0), engine_changed(0), changed(0), - is_correlated(FALSE) + const_item_cache(1), inside_first_fix_fields(0), done_first_fix_fields(FALSE), + engine_changed(0), changed(0), is_correlated(FALSE) { with_subselect= 1; reset(); @@ -167,18 +167,23 @@ DBUG_ASSERT(fixed == 0); engine->set_thd((thd= thd_param)); - if (!in_fix_fields) - refers_to.empty(); + if (!done_first_fix_fields) + { + done_first_fix_fields= TRUE; + inside_first_fix_fields= TRUE; + } + eliminated= FALSE; + parent_select= thd_param->lex->current_select; if (check_stack_overrun(thd, STACK_MIN_SIZE, (uchar*)&res)) return TRUE; - in_fix_fields++; res= engine->prepare(); // all transformation is done (used by prepared statements) changed= 1; + inside_first_fix_fields= FALSE; if (!res) { @@ -210,14 +215,12 @@ if (!(*ref)->fixed) ret= (*ref)->fix_fields(thd, ref); thd->where= save_where; - in_fix_fields--; return ret; } // Is it one field subselect? if (engine->cols() > max_columns) { my_error(ER_OPERAND_COLUMNS, MYF(0), 1); - in_fix_fields--; return TRUE; } fix_length_and_dec(); @@ -234,7 +237,6 @@ fixed= 1; err: - in_fix_fields--; thd->where= save_where; return res; } @@ -242,11 +244,12 @@ bool Item_subselect::enumerate_field_refs_processor(uchar *arg) { - List_iterator<Item> it(refers_to); - Item *item; - while ((item= it++)) + List_iterator<Ref_to_outside> it(upper_refs); + Ref_to_outside *upper; + + while ((upper= it++)) { - if (item->walk(&Item::enumerate_field_refs_processor, FALSE, arg)) + if (upper->item->walk(&Item::enumerate_field_refs_processor, FALSE, arg)) return TRUE; } return FALSE; @@ -258,6 +261,115 @@ return FALSE; } + +bool Item_subselect::mark_as_dependent(THD *thd, st_select_lex *select, + Item *item) +{ + if (inside_first_fix_fields) + { + is_correlated= TRUE; + Ref_to_outside *upper; + if (!(upper= new (thd->stmt_arena->mem_root) Ref_to_outside())) + return TRUE; + upper->select= select; + upper->item= item; + if (upper_refs.push_back(upper, thd->stmt_arena->mem_root)) + return TRUE; + } + return FALSE; +} + +/* + Adjust attributes after our parent select has been merged into grandparent + + DESCRIPTION + Subquery is a composite object which may be correlated, that is, it may + have + 1. references to tables of the parent select (i.e. one that has the clause + with the subquery predicate) + 2. references to tables of the grandparent select + 3. references to tables of further ancestors. + + Before the pullout, this item indicates: + - #1 with table bits in used_tables() + - #2 and #3 with OUTER_REF_TABLE_BIT. + + After parent has been merged with grandparent: + - references to parent and grandparent tables should be indicated with + table bits. + - references to greatgrandparent and further ancestors - with + OUTER_REF_TABLE_BIT. +*/ + +void Item_subselect::fix_after_pullout(st_select_lex *new_parent, Item **ref) +{ + recalc_used_tables(new_parent, TRUE); + parent_select= new_parent; +} + + +/* + Recalculate used_tables_cache +*/ + +void Item_subselect::recalc_used_tables(st_select_lex *new_parent, + bool after_pullout) +{ + List_iterator<Ref_to_outside> it(upper_refs); + Ref_to_outside *upper; + + used_tables_cache= 0; + while ((upper= it++)) + { + bool found= FALSE; + /* + Check if + 1. the upper reference refers to the new immediate parent select, or + 2. one of the further ancestors. + + We rely on the fact that the tree of selects is modified by some kind of + 'flattening', i.e. a process where child selects are merged into their + parents. + The merged selects are removed from the select tree but keep pointers to + their parents. + */ + for (st_select_lex *sel= upper->select; sel; sel= sel->outer_select()) + { + /* + If we've reached the new parent select by walking upwards from + reference's original select, this means that the reference is now + referring to the direct parent: + */ + if (sel == new_parent) + { + found= TRUE; + /* + upper->item may be NULL when we've referred to a grouping function, + in which case we don't care about what it's table_map really is, + because item->with_sum_func==1 will ensure correct placement of the + item. + */ + if (upper->item) + { + if (after_pullout) + upper->item->fix_after_pullout(new_parent, &(upper->item)); + upper->item->update_used_tables(); + used_tables_cache |= upper->item->used_tables(); + } + } + } + if (!found) + used_tables_cache|= OUTER_REF_TABLE_BIT; + } + /* + Don't update const_tables_cache yet as we don't yet know which of the + parent's tables are constant. Parent will call update_used_tables() after + he has done const table detection, and that will be our chance to update + const_tables_cache. + */ +} + + bool Item_subselect::walk(Item_processor processor, bool walk_subquery, uchar *argument) { @@ -397,6 +509,7 @@ void Item_subselect::update_used_tables() { + recalc_used_tables(parent_select, FALSE); if (!engine->uncacheable()) { // did all used tables become static? @@ -1843,6 +1956,18 @@ return result || Item_subselect::fix_fields(thd_arg, ref); } +void Item_in_subselect::fix_after_pullout(st_select_lex *new_parent, Item **ref) +{ + left_expr->fix_after_pullout(new_parent, &left_expr); + Item_subselect::fix_after_pullout(new_parent, ref); +} + +void Item_in_subselect::update_used_tables() +{ + Item_subselect::update_used_tables(); + left_expr->update_used_tables(); + used_tables_cache |= left_expr->used_tables(); +} /** Try to create an engine to compute the subselect via materialization, === modified file 'sql/item_subselect.h' --- a/sql/item_subselect.h 2010-01-28 13:48:33 +0000 +++ b/sql/item_subselect.h 2010-02-02 20:00:45 +0000 @@ -67,14 +67,32 @@ bool have_to_be_excluded; /* cache of constant state */ bool const_item_cache; - + + bool inside_first_fix_fields; + bool done_first_fix_fields; public: - /* - References from inside the subquery to the select that this predicate is - in. References to parent selects not included. + /* A reference from inside subquery predicate to somewhere outside of it */ + class Ref_to_outside : public Sql_alloc + { + public: + st_select_lex *select; /* Select where the reference is pointing to */ + /* + What is being referred. This may be NULL when we're referring to an + aggregate function. + */ + Item *item; + }; + /* + References from within this subquery to somewhere outside of it (i.e. to + parent select, grandparent select, etc) */ - List<Item> refers_to; - int in_fix_fields; + List<Ref_to_outside> upper_refs; + st_select_lex *parent_select; + + /* + TRUE<=>Table Elimination has made it redundant to evaluate this select + (and so it is not part of QEP, etc) + */ bool eliminated; /* changed engine indicator */ @@ -117,6 +135,9 @@ return null_value; } bool fix_fields(THD *thd, Item **ref); + bool mark_as_dependent(THD *thd, st_select_lex *select, Item *item); + void fix_after_pullout(st_select_lex *new_parent, Item **ref); + void recalc_used_tables(st_select_lex *new_parent, bool after_pullout); virtual bool exec(); virtual void fix_length_and_dec(); table_map used_tables() const; @@ -396,6 +417,8 @@ bool test_limit(st_select_lex_unit *unit); virtual void print(String *str, enum_query_type query_type); bool fix_fields(THD *thd, Item **ref); + void fix_after_pullout(st_select_lex *new_parent, Item **ref); + void update_used_tables(); bool setup_engine(); bool init_left_expr_cache(); bool is_expensive_processor(uchar *arg); === modified file 'sql/item_sum.cc' --- a/sql/item_sum.cc 2009-10-15 21:38:29 +0000 +++ b/sql/item_sum.cc 2010-02-02 20:00:45 +0000 @@ -350,7 +350,7 @@ sl= sl->master_unit()->outer_select() ) sl->master_unit()->item->with_sum_func= 1; } - thd->lex->current_select->mark_as_dependent(aggr_sel, NULL); + thd->lex->current_select->mark_as_dependent(thd, aggr_sel, NULL); return FALSE; } === modified file 'sql/sql_lex.cc' --- a/sql/sql_lex.cc 2010-01-28 13:48:33 +0000 +++ b/sql/sql_lex.cc 2010-02-02 20:00:45 +0000 @@ -1841,9 +1841,8 @@ 'last' should be reachable from this st_select_lex_node */ -void st_select_lex::mark_as_dependent(st_select_lex *last, Item *dependency) +bool st_select_lex::mark_as_dependent(THD *thd, st_select_lex *last, Item *dependency) { - SELECT_LEX *next_to_last; /* Mark all selects from resolved to 1 before select where was found table as depended (of select where was found table) @@ -1867,12 +1866,15 @@ sl->uncacheable|= UNCACHEABLE_UNITED; } } - next_to_last= s; + + Item_subselect *subquery_expr= s->master_unit()->item; + if (subquery_expr && subquery_expr->mark_as_dependent(thd, last, + dependency)) + return TRUE; } is_correlated= TRUE; this->master_unit()->item->is_correlated= TRUE; - if (dependency) - next_to_last->master_unit()->item->refers_to.push_back(dependency); + return FALSE; } bool st_select_lex_node::set_braces(bool value) { return 1; } === modified file 'sql/sql_lex.h' --- a/sql/sql_lex.h 2010-01-28 13:48:33 +0000 +++ b/sql/sql_lex.h 2010-02-02 20:00:45 +0000 @@ -747,7 +747,7 @@ return master_unit()->return_after_parsing(); } - void mark_as_dependent(st_select_lex *last, Item *dependency); + bool mark_as_dependent(THD *thd, st_select_lex *last, Item *dependency); bool set_braces(bool value); bool inc_in_sum_expr(); === modified file 'sql/sql_select.h' --- a/sql/sql_select.h 2010-01-28 13:48:33 +0000 +++ b/sql/sql_select.h 2010-02-02 20:00:45 +0000 @@ -282,13 +282,11 @@ } bool check_rowid_field() { -/* !!!NB igor: enable the code in this comment after backporting the SJ code if (keep_current_rowid && !used_rowid_fields) { used_rowid_fields= 1; used_fieldlength+= table->file->ref_length; } -*/ return test(used_rowid_fields); } bool is_inner_table_of_semi_join_with_first_match()

1 0

Re: [Maria-developers] Ideas for improving MariaDB/MySQL replication
by Alex Yurchenko 02 Feb '10

02 Feb '10

Hi! On Mon, 1 Feb 2010 11:06:22 +0100, Sergei Golubchik <serg(a)askmonty.org> wrote: > Hi, Alex! > > On Jan 27, Alex Yurchenko wrote: >> >> I'll take this opportunity to put forth some theory behind the global >> transaction IDs as we see it at Codership. >> >> 1. We have an abstract set of data subject to replication/logging. It >> can be a whole database, a schema, a table, a row. Lets call it a >> Replication Set (RS). >> >> 2. RS is undergoing changes in time which can be represented as a >> series of atomic changes. Let's call it RS History. That it is a >> _series_ is trivial but important - otherwise we can't reproduce >> historical RS state evolution. Each RS change is represented by a >> changeset. Since it is a series, RS changesets can be enumerated with >> a sequence of natural numbers without gaps within a given RS History. >> Here comes the first component of a global transaction ID: sequence >> number (seqno). > > Why should it be a sequence of natural numbers without gaps ? 1) Well, to begin with I didn't say that it "should", I said that it "can" ;). I was proposing a definition, so it depends on what we want to achieve. Obviously, "gaplessness" is a useful requirement. It allows global transaction IDs to not only be globally unique, but also unambiguously indicates the position of a changeset in the history of changes. By relaxing it we are loosing a natural way to check for gaps in the stream of events. E.g. you can't say if you can concatenate binlog files one of which ends at 10 and another starts at 12. When a node at position 100 joins a cluster, how will it know that the next event to process is 113? "No gaps" requirement allows us to take a single changeset, carry it around all we like and then apply consistently elsewhere without the need of any other context. Indeed, there are other ways to address this, but gapless seqno is obviously the simplest of them all. 2) It is not a limiting requirement at all. If we agree, that RS undergoes a _series_ of changes - change 1, change 2, change 3, change 4, change 5, etc...- nothing prevents us from enumerating them without gaps. Moreover, why and according to what algorithm are you going to introduce gaps in sequence numbers? I mean, there is additional work to be done to achieve gaps when enumerating a sequence. 3) It simplifies testing and debugging. Robert from Continuent also raised this question, but I didn't have a chance to respond to it in time. No, gapless seqnos is not a caprice of Galera developers ;), it is just a proposal for global trx ID based on our experience. Galera can maintain it internally anyways, but I believe everyone would benefit from it. >> 3. However there can be more than one RS. Moreover, the same RS can >> end up in different clusters and undergo different changes. So, to >> achieve truly global unambiguity each changeset, in addition to seqno, >> should be marked with a RS History ID. Obviously seqnos from different >> histories are logically incomparable. Therefore RS History ID can be >> any globally unique identifier, with no need for < or > operations. >> This is the second component of global transaction ID. > > Assuming we want to replicate just one table, do you mean that in a > replication cluster this Logical Table is a Replication Set, that is > all copies of this table on all nodes belong to the same RS ? Yes, with a small clarification, that strictly speaking, copies of the table don't _belong_ to RS, they _are_ RS, that is in your example Replication Set consists of a single table, not of multiple copies the table. It is probably more correct to speak about RS replicas on nodes, rather than copies of the table. Regards, Alex -- Alexey Yurchenko, Codership Oy, www.codership.com Skype: alexey.yurchenko, Phone: +358-400-516-011

1 0

[Maria-developers] Rev 2750: BUG#31480: Incorrect result for nested subquery when executed via semi join: in file:///home/psergey/dev/maria-5.3-subqueries-r3/
by Sergey Petrunya 02 Feb '10

02 Feb '10

At file:///home/psergey/dev/maria-5.3-subqueries-r3/ ------------------------------------------------------------ revno: 2750 revision-id: psergey(a)askmonty.org-20100202121522-fw6ckzcdfis4d21p parent: psergey(a)askmonty.org-20100128134833-9000udjp5wa3tsff committer: Sergey Petrunya <psergey(a)askmonty.org> branch nick: maria-5.3-subqueries-r3 timestamp: Tue 2010-02-02 15:15:22 +0300 message: BUG#31480: Incorrect result for nested subquery when executed via semi join: A mark-2 fix that can survive FROM subquery handling and has some code unification with table elimination: Each subquery predicate now stores a (flat) list of all references from inside to outside the subquery. We actually store (select, referred_item) pairs which allows Item_subselect::fix_after_pullout() to recalculate subquery predicate's attributes after a broad range of FROM- and IN-subselect flattening operations. === modified file 'mysql-test/r/subselect_sj.result' --- a/mysql-test/r/subselect_sj.result 2010-01-17 14:51:10 +0000 +++ b/mysql-test/r/subselect_sj.result 2010-02-02 12:15:22 +0000 @@ -779,3 +779,48 @@ 1 PRIMARY it2 ALL NULL NULL NULL NULL 20 Using where; End temporary DROP TABLE ot1, it1, it2; # End of BUG#38075 +# +# BUG#31480: Incorrect result for nested subquery when executed via semi join +# +create table t1 (a int not null, b int not null); +create table t2 (c int not null, d int not null); +create table t3 (e int not null); +insert into t1 values (1,10); +insert into t1 values (2,10); +insert into t1 values (1,20); +insert into t1 values (2,20); +insert into t1 values (3,20); +insert into t1 values (2,30); +insert into t1 values (4,40); +insert into t2 values (2,10); +insert into t2 values (2,20); +insert into t2 values (4,10); +insert into t2 values (5,10); +insert into t2 values (3,20); +insert into t2 values (2,40); +insert into t3 values (10); +insert into t3 values (30); +insert into t3 values (10); +insert into t3 values (20); +explain extended +select a from t1 +where a in (select c from t2 where d >= some(select e from t3 where b=e)); +id select_type table type possible_keys key key_len ref rows filtered Extra +1 PRIMARY t2 ALL NULL NULL NULL NULL 6 100.00 Start temporary +1 PRIMARY t1 ALL NULL NULL NULL NULL 7 100.00 Using where; End temporary; Using join buffer +3 DEPENDENT SUBQUERY t3 ALL NULL NULL NULL NULL 4 100.00 Using where +Warnings: +Note 1276 Field or reference 'test.t1.b' of SELECT #3 was resolved in SELECT #1 +Note 1003 select `test`.`t1`.`a` AS `a` from `test`.`t1` semi join (`test`.`t2`) where ((`test`.`t1`.`a` = `test`.`t2`.`c`) and <nop>(<in_optimizer>(`test`.`t2`.`d`,<exists>(select 1 AS `Not_used` from `test`.`t3` where ((`test`.`t1`.`b` = `test`.`t3`.`e`) and (<cache>(`test`.`t2`.`d`) >= `test`.`t3`.`e`)))))) +show warnings; +Level Code Message +Note 1276 Field or reference 'test.t1.b' of SELECT #3 was resolved in SELECT #1 +Note 1003 select `test`.`t1`.`a` AS `a` from `test`.`t1` semi join (`test`.`t2`) where ((`test`.`t1`.`a` = `test`.`t2`.`c`) and <nop>(<in_optimizer>(`test`.`t2`.`d`,<exists>(select 1 AS `Not_used` from `test`.`t3` where ((`test`.`t1`.`b` = `test`.`t3`.`e`) and (<cache>(`test`.`t2`.`d`) >= `test`.`t3`.`e`)))))) +select a from t1 +where a in (select c from t2 where d >= some(select e from t3 where b=e)); +a +2 +2 +3 +2 +drop table t1, t2, t3; === modified file 'mysql-test/r/subselect_sj_jcl6.result' --- a/mysql-test/r/subselect_sj_jcl6.result 2010-01-17 14:51:10 +0000 +++ b/mysql-test/r/subselect_sj_jcl6.result 2010-02-02 12:15:22 +0000 @@ -783,6 +783,51 @@ 1 PRIMARY it2 ALL NULL NULL NULL NULL 20 Using where; End temporary; Using join buffer DROP TABLE ot1, it1, it2; # End of BUG#38075 +# +# BUG#31480: Incorrect result for nested subquery when executed via semi join +# +create table t1 (a int not null, b int not null); +create table t2 (c int not null, d int not null); +create table t3 (e int not null); +insert into t1 values (1,10); +insert into t1 values (2,10); +insert into t1 values (1,20); +insert into t1 values (2,20); +insert into t1 values (3,20); +insert into t1 values (2,30); +insert into t1 values (4,40); +insert into t2 values (2,10); +insert into t2 values (2,20); +insert into t2 values (4,10); +insert into t2 values (5,10); +insert into t2 values (3,20); +insert into t2 values (2,40); +insert into t3 values (10); +insert into t3 values (30); +insert into t3 values (10); +insert into t3 values (20); +explain extended +select a from t1 +where a in (select c from t2 where d >= some(select e from t3 where b=e)); +id select_type table type possible_keys key key_len ref rows filtered Extra +1 PRIMARY t2 ALL NULL NULL NULL NULL 6 100.00 Start temporary +1 PRIMARY t1 ALL NULL NULL NULL NULL 7 100.00 Using where; End temporary; Using join buffer +3 DEPENDENT SUBQUERY t3 ALL NULL NULL NULL NULL 4 100.00 Using where +Warnings: +Note 1276 Field or reference 'test.t1.b' of SELECT #3 was resolved in SELECT #1 +Note 1003 select `test`.`t1`.`a` AS `a` from `test`.`t1` semi join (`test`.`t2`) where ((`test`.`t1`.`a` = `test`.`t2`.`c`) and <nop>(<in_optimizer>(`test`.`t2`.`d`,<exists>(select 1 AS `Not_used` from `test`.`t3` where ((`test`.`t1`.`b` = `test`.`t3`.`e`) and (<cache>(`test`.`t2`.`d`) >= `test`.`t3`.`e`)))))) +show warnings; +Level Code Message +Note 1276 Field or reference 'test.t1.b' of SELECT #3 was resolved in SELECT #1 +Note 1003 select `test`.`t1`.`a` AS `a` from `test`.`t1` semi join (`test`.`t2`) where ((`test`.`t1`.`a` = `test`.`t2`.`c`) and <nop>(<in_optimizer>(`test`.`t2`.`d`,<exists>(select 1 AS `Not_used` from `test`.`t3` where ((`test`.`t1`.`b` = `test`.`t3`.`e`) and (<cache>(`test`.`t2`.`d`) >= `test`.`t3`.`e`)))))) +select a from t1 +where a in (select c from t2 where d >= some(select e from t3 where b=e)); +a +2 +2 +3 +2 +drop table t1, t2, t3; set join_cache_level=default; show variables like 'join_cache_level'; Variable_name Value === modified file 'mysql-test/t/subselect_sj.test' --- a/mysql-test/t/subselect_sj.test 2010-01-17 14:51:10 +0000 +++ b/mysql-test/t/subselect_sj.test 2010-02-02 12:15:22 +0000 @@ -681,3 +681,41 @@ DROP TABLE ot1, it1, it2; --echo # End of BUG#38075 + +--echo # +--echo # BUG#31480: Incorrect result for nested subquery when executed via semi join +--echo # +create table t1 (a int not null, b int not null); +create table t2 (c int not null, d int not null); +create table t3 (e int not null); + +insert into t1 values (1,10); +insert into t1 values (2,10); +insert into t1 values (1,20); +insert into t1 values (2,20); +insert into t1 values (3,20); +insert into t1 values (2,30); +insert into t1 values (4,40); + +insert into t2 values (2,10); +insert into t2 values (2,20); +insert into t2 values (4,10); +insert into t2 values (5,10); +insert into t2 values (3,20); +insert into t2 values (2,40); + +insert into t3 values (10); +insert into t3 values (30); +insert into t3 values (10); +insert into t3 values (20); + +explain extended +select a from t1 +where a in (select c from t2 where d >= some(select e from t3 where b=e)); +show warnings; + +select a from t1 +where a in (select c from t2 where d >= some(select e from t3 where b=e)); + +drop table t1, t2, t3; + === modified file 'sql/item.cc' --- a/sql/item.cc 2010-01-17 14:55:08 +0000 +++ b/sql/item.cc 2010-02-02 12:15:22 +0000 @@ -3646,7 +3646,7 @@ substitution) */ -static void mark_as_dependent(THD *thd, SELECT_LEX *last, SELECT_LEX *current, +static bool mark_as_dependent(THD *thd, SELECT_LEX *last, SELECT_LEX *current, Item_ident *resolved_item, Item_ident *mark_item) { @@ -3657,7 +3657,8 @@ /* store pointer on SELECT_LEX from which item is dependent */ if (mark_item) mark_item->depended_from= last; - current->mark_as_dependent(last, resolved_item); + if (current->mark_as_dependent(thd, last, resolved_item)) + return TRUE; if (thd->lex->describe & DESCRIBE_EXTENDED) { push_warning_printf(thd, MYSQL_ERROR::WARN_LEVEL_NOTE, @@ -3667,6 +3668,7 @@ resolved_item->field_name, current->select_number, last->select_number); } + return FALSE; } @@ -4118,6 +4120,7 @@ ((ref_type == REF_ITEM || ref_type == FIELD_ITEM) ? (Item_ident*) (*reference) : 0)); + /* A reference to a view field had been found and we substituted it instead of this Item (find_field_in_tables @@ -6437,7 +6440,7 @@ if (depended_from == new_parent) { *ref= outer_ref; - outer_ref->fix_after_pullout(new_parent, ref); + (*ref)->fix_after_pullout(new_parent, ref); } } === modified file 'sql/item_subselect.cc' --- a/sql/item_subselect.cc 2010-01-28 13:48:33 +0000 +++ b/sql/item_subselect.cc 2010-02-02 12:15:22 +0000 @@ -39,8 +39,8 @@ Item_subselect::Item_subselect(): Item_result_field(), value_assigned(0), thd(0), substitution(0), engine(0), old_engine(0), used_tables_cache(0), have_to_be_excluded(0), - const_item_cache(1), in_fix_fields(0), engine_changed(0), changed(0), - is_correlated(FALSE) + const_item_cache(1), inside_first_fix_fields(0), done_first_fix_fields(FALSE), + engine_changed(0), changed(0), is_correlated(FALSE) { with_subselect= 1; reset(); @@ -167,18 +167,22 @@ DBUG_ASSERT(fixed == 0); engine->set_thd((thd= thd_param)); - if (!in_fix_fields) - refers_to.empty(); + if (!done_first_fix_fields) + { + done_first_fix_fields= TRUE; + inside_first_fix_fields= TRUE; + } + eliminated= FALSE; if (check_stack_overrun(thd, STACK_MIN_SIZE, (uchar*)&res)) return TRUE; - in_fix_fields++; res= engine->prepare(); // all transformation is done (used by prepared statements) changed= 1; + inside_first_fix_fields= FALSE; if (!res) { @@ -210,14 +214,12 @@ if (!(*ref)->fixed) ret= (*ref)->fix_fields(thd, ref); thd->where= save_where; - in_fix_fields--; return ret; } // Is it one field subselect? if (engine->cols() > max_columns) { my_error(ER_OPERAND_COLUMNS, MYF(0), 1); - in_fix_fields--; return TRUE; } fix_length_and_dec(); @@ -234,7 +236,6 @@ fixed= 1; err: - in_fix_fields--; thd->where= save_where; return res; } @@ -242,11 +243,12 @@ bool Item_subselect::enumerate_field_refs_processor(uchar *arg) { - List_iterator<Item> it(refers_to); - Item *item; - while ((item= it++)) + List_iterator<Ref_to_outside> it(upper_refs); + Ref_to_outside *upper; + + while ((upper= it++)) { - if (item->walk(&Item::enumerate_field_refs_processor, FALSE, arg)) + if (upper->item->walk(&Item::enumerate_field_refs_processor, FALSE, arg)) return TRUE; } return FALSE; @@ -258,6 +260,102 @@ return FALSE; } + +bool Item_subselect::mark_as_dependent(THD *thd, st_select_lex *select, + Item *item) +{ + if (inside_first_fix_fields) + { + is_correlated= TRUE; + Ref_to_outside *upper; + if (!(upper= new (thd->stmt_arena->mem_root) Ref_to_outside())) + return TRUE; + upper->select= select; + upper->item= item; + if (upper_refs.push_back(upper, thd->stmt_arena->mem_root)) + return TRUE; + } + return FALSE; +} + +/* + Adjust attributes after our parent select has been merged into grandparent + + DESCRIPTION + Subquery is a composite object which may be correlated, that is, it may + have + 1. references to tables of the parent select (i.e. one that has the clause + with the subquery predicate) + 2. references to tables of the grandparent select + 3. references to tables of further ancestors. + + Before the pullout, this item indicates: + - #1 with table bits in used_tables() + - #2 and #3 with OUTER_REF_TABLE_BIT. + + After parent has been merged with grandparent: + - references to parent and grandparent tables should be indicated with + table bits. + - references to greatgrandparent and further ancestors - with + OUTER_REF_TABLE_BIT. +*/ + +void Item_subselect::fix_after_pullout(st_select_lex *new_parent, Item **ref) +{ + List_iterator<Ref_to_outside> it(upper_refs); + Ref_to_outside *upper; + + used_tables_cache= 0; + while ((upper= it++)) + { + bool found= FALSE; + /* + Check if + 1. the upper reference refers to the new immediate parent select, or + 2. one of the further ancestors. + + We rely on the fact that the tree of selects is modified by some kind of + 'flattening', i.e. a process where child selects are merged into their + parents. + The merged selects are removed from the select tree but keep pointers to + their parents. + */ + for (st_select_lex *sel= upper->select; sel; sel= sel->outer_select()) + { + /* + If we've reached the new parent select by walking upwards from + reference's original select, this means that the reference is now + referring to the direct parent: + */ + if (sel == new_parent) + { + found= TRUE; + /* + upper->item may be NULL when we've referred to a grouping function, + in which case we don't care about what it's table_map really is, + because item->with_sum_func would ensure correct placement of the + item. + */ + if (upper->item) + { + upper->item->update_used_tables(); + upper->item->fix_after_pullout(new_parent, &(upper->item)); + used_tables_cache |= upper->item->used_tables(); + } + } + } + if (!found) + used_tables_cache|= OUTER_REF_TABLE_BIT; + } + /* + Don't update const_tables_cache yet as we don't yet know which of the + parent's tables are constant. Parent will call update_used_tables() after + he has done const table detection, and that will be our chance to update + const_tables_cache. + */ +} + + bool Item_subselect::walk(Item_processor processor, bool walk_subquery, uchar *argument) { === modified file 'sql/item_subselect.h' --- a/sql/item_subselect.h 2010-01-28 13:48:33 +0000 +++ b/sql/item_subselect.h 2010-02-02 12:15:22 +0000 @@ -67,14 +67,31 @@ bool have_to_be_excluded; /* cache of constant state */ bool const_item_cache; - + + bool inside_first_fix_fields; + bool done_first_fix_fields; public: - /* - References from inside the subquery to the select that this predicate is - in. References to parent selects not included. + /* A reference from inside subquery predicate to somewhere outside of it */ + class Ref_to_outside : public Sql_alloc + { + public: + st_select_lex *select; /* Select where the reference is pointing to */ + /* + What is being referred. This may be NULL when we're referring to an + aggregate function. + */ + Item *item; + }; + /* + References from within this subquery to somewhere outside of it (i.e. to + parent select, grandparent select, etc) */ - List<Item> refers_to; - int in_fix_fields; + List<Ref_to_outside> upper_refs; + + /* + TRUE<=>Table Elimination has made it redundant to evaluate this select + (and so it is not part of QEP, etc) + */ bool eliminated; /* changed engine indicator */ @@ -117,6 +134,8 @@ return null_value; } bool fix_fields(THD *thd, Item **ref); + bool mark_as_dependent(THD *thd, st_select_lex *select, Item *item); + void fix_after_pullout(st_select_lex *new_parent, Item **ref); virtual bool exec(); virtual void fix_length_and_dec(); table_map used_tables() const; === modified file 'sql/item_sum.cc' --- a/sql/item_sum.cc 2009-10-15 21:38:29 +0000 +++ b/sql/item_sum.cc 2010-02-02 12:15:22 +0000 @@ -350,7 +350,7 @@ sl= sl->master_unit()->outer_select() ) sl->master_unit()->item->with_sum_func= 1; } - thd->lex->current_select->mark_as_dependent(aggr_sel, NULL); + thd->lex->current_select->mark_as_dependent(thd, aggr_sel, NULL); return FALSE; } === modified file 'sql/sql_lex.cc' --- a/sql/sql_lex.cc 2010-01-28 13:48:33 +0000 +++ b/sql/sql_lex.cc 2010-02-02 12:15:22 +0000 @@ -1841,9 +1841,8 @@ 'last' should be reachable from this st_select_lex_node */ -void st_select_lex::mark_as_dependent(st_select_lex *last, Item *dependency) +bool st_select_lex::mark_as_dependent(THD *thd, st_select_lex *last, Item *dependency) { - SELECT_LEX *next_to_last; /* Mark all selects from resolved to 1 before select where was found table as depended (of select where was found table) @@ -1867,12 +1866,15 @@ sl->uncacheable|= UNCACHEABLE_UNITED; } } - next_to_last= s; + + Item_subselect *subquery_expr= s->master_unit()->item; + if (subquery_expr && subquery_expr->mark_as_dependent(thd, last, + dependency)) + return TRUE; } is_correlated= TRUE; this->master_unit()->item->is_correlated= TRUE; - if (dependency) - next_to_last->master_unit()->item->refers_to.push_back(dependency); + return FALSE; } bool st_select_lex_node::set_braces(bool value) { return 1; } === modified file 'sql/sql_lex.h' --- a/sql/sql_lex.h 2010-01-28 13:48:33 +0000 +++ b/sql/sql_lex.h 2010-02-02 12:15:22 +0000 @@ -747,7 +747,7 @@ return master_unit()->return_after_parsing(); } - void mark_as_dependent(st_select_lex *last, Item *dependency); + bool mark_as_dependent(THD *thd, st_select_lex *last, Item *dependency); bool set_braces(bool value); bool inc_in_sum_expr();

1 0

Re: [Maria-developers] really need call my_thread_init() in libmysqlclient_r?
by xiaobing jiang 02 Feb '10

02 Feb '10

hi, Sergei: > You cannot easily remove my_thread_init/my_thread_end, besides debugging > THR_KEY_mysys is also needed for my_errno, which is uses all over mysys. > thanks, but have another question: what about in one thread, I call mysql_init() twice to create 2 connection. they will share the THR_KEY_mysys thread local variable, right? the two connections will share my_error, is this will cause some 'bug'?

1 1

[Maria-developers] bzr commit into Mariadb 5.2, with Maria 2.0:maria/5.2 branch (knielsen:2739)
by knielsen＠knielsen-hq.org 01 Feb '10

01 Feb '10

#At lp:maria/5.2 2739 knielsen(a)knielsen-hq.org 2010-02-01 Fix bad 5.1->5.2 merge: timezone must now be set explicitly for test cases that depend on it. added: mysql-test/suite/binlog/t/binlog_row_mysqlbinlog_options-master.opt === added file 'mysql-test/suite/binlog/t/binlog_row_mysqlbinlog_options-master.opt' --- a/mysql-test/suite/binlog/t/binlog_row_mysqlbinlog_options-master.opt 1970-01-01 00:00:00 +0000 +++ b/mysql-test/suite/binlog/t/binlog_row_mysqlbinlog_options-master.opt 2010-02-01 19:05:19 +0000 @@ -0,0 +1 @@ +--timezone=GMT-3

1 0

Re: [Maria-developers] Support building with system libevent
by Kristian Nielsen 01 Feb '10

01 Feb '10

Sergei Golubchik <serg(a)askmonty.org> writes: > Surely, it is possible to install a libevent 1.4 deb on Debian <= 5 and > Ubuntu <= 9.04, right ? They are not in the default package repository - > is that a big deal ? MariaDB is not either. Possible yes, but only if one knows where to obtain such a .deb package. I don't know anywhere such a package is available for older Ubuntu/Debian? (One could try installing the package from a newer Ubuntu or Debian unstable, may or may not work. But in general in .deb world people expect to not have to manually go hunt for dependencies. Maybe .rpm is different?) MariaDB is in the "default" repository list if users put the OurDelta repository there, which is the recommended way to install .deb. - Kristian.

1 0

[Maria-developers] Rev 3769: MWL#68 Subquery optimization: Efficient NOT IN execution with NULLs in file:///home/tsk/mprog/src/mysql-6.0-mwl68/
by timour＠askmonty.org 01 Feb '10

01 Feb '10

At file:///home/tsk/mprog/src/mysql-6.0-mwl68/ ------------------------------------------------------------ revno: 3769 revision-id: timour(a)askmonty.org-20100201120948-mdt7gtwcz50q1dzp parent: timour(a)sun.com-20100122161805-8lgrisqabrlvc3nc committer: timour(a)askmonty.org branch nick: mysql-6.0-mwl68 timestamp: Mon 2010-02-01 14:09:48 +0200 message: MWL#68 Subquery optimization: Efficient NOT IN execution with NULLs Completed main coding of partial matching. The code compiles, but cannot run. Changes compared to the previous commit: - Completed creation and initialization of all objects needed for partial matching. - Adjusted the interfaces of multiple methods in order to pass the correct information needed for creation/initialization. - Added comparion functions needed for binary search and index sorting. - Fixed binary search in the value index. - Exposed the Arg_comparator of comparison predicates. Diff too large for email (1030 lines, the limit is 1000).

1 0

[Maria-developers] bzr commit into file:///home/tsk/mprog/src/mysql-6.0-mwl68/ branch (timour:3769)
by timour＠askmonty.org 01 Feb '10

01 Feb '10

#At file:///home/tsk/mprog/src/mysql-6.0-mwl68/ based on revid:timour@sun.com-20100122161805-8lgrisqabrlvc3nc 3769 timour(a)askmonty.org 2010-02-01 MWL#68 Subquery optimization: Efficient NOT IN execution with NULLs Completed main coding of partial matching. The code compiles, but cannot run. Changes compared to the previous commit: - Completed creation and initialization of all objects needed for partial matching. - Adjusted the interfaces of multiple methods in order to pass the correct information needed for creation/initialization. - Added comparion functions needed for binary search and index sorting. - Fixed binary search in the value index. - Exposed the Arg_comparator of comparison predicates. @ sql/item_cmpfunc.h Expose the Arg_comparator of a comparison predicate so that it is possible to get the comparison result. @ sql/item_subselect.cc - Completed creation and initialization of all objects needed for partial matching. - Adjusted the interfaces of multiple methods in order to pass the correct information needed for creation/initialization. - Added comparion functions needed for binary search and index sorting. - Fixed binary search in the value index. @ sql/item_subselect.h Completed creation and initialization of all objects needed for partial matching. @ sql/sql_class.h - added accessors for NULL statistics modified: sql/item_cmpfunc.h sql/item_subselect.cc sql/item_subselect.h sql/sql_class.h === modified file 'sql/item_cmpfunc.h' --- a/sql/item_cmpfunc.h 2009-12-04 07:48:05 +0000 +++ b/sql/item_cmpfunc.h 2010-02-01 12:09:48 +0000 @@ -355,6 +355,7 @@ public: CHARSET_INFO *compare_collation() { return cmp.cmp_collation.collation; } uint decimal_precision() const { return 1; } void top_level_item() { abort_on_null= TRUE; } + Arg_comparator *get_comparator() { return &cmp; } friend class Arg_comparator; }; === modified file 'sql/item_subselect.cc' --- a/sql/item_subselect.cc 2010-01-22 16:18:05 +0000 +++ b/sql/item_subselect.cc 2010-02-01 12:09:48 +0000 @@ -3102,32 +3102,21 @@ void subselect_hash_sj_engine::set_strat outer_col= item_in->left_expr->element_index(i); inner_col= inner_col_it++; - if (!inner_col->maybe_null) - { - if (!outer_col->maybe_null) - { - non_null_outer_cols.push_back(outer_col); - non_null_key_parts |= 1 << i; - } - else - { - non_null_late_key_parts |= 1 << i; - ++count_non_null_late_cols; - } - } + if (!inner_col->maybe_null && !outer_col->maybe_null) + bitmap_set_bit(&non_null_key_parts, i); else { - partial_match_key_parts |= 1 << i; - ++count_partial_match_cols; + bitmap_set_bit(&partial_match_key_parts, i); + ++count_partial_match_columns; } } } /* If no column contains NULLs use regular hash index lookups. */ - if (!(non_null_late_key_parts || partial_match_key_parts)) - strategy= COMPLETE_MATCH; - else + if (count_partial_match_columns) strategy= PARTIAL_MATCH; + else + strategy= COMPLETE_MATCH; DBUG_VOID_RETURN; } @@ -3145,7 +3134,6 @@ void subselect_hash_sj_engine::set_strat void subselect_hash_sj_engine::set_strategy_using_data() { Item_in_subselect *item_in= (Item_in_subselect *) item; - key_part_map cur_col= 0; select_materialize_with_stats *result_sink= (select_materialize_with_stats *) result; @@ -3154,77 +3142,45 @@ void subselect_hash_sj_engine::set_strat /* Call this procedure only if already selected partial matching. */ DBUG_ASSERT(strategy == PARTIAL_MATCH); - /* - TODO: uncomment after enabling index creation after materialization. - List_iterator<Item> inner_col_it(*item_in->unit->get_unit_column_types()); - Item *inner_col, *outer_col; - */ - for (uint i= 0; i < item_in->left_expr->cols(); i++) { - /* - TODO: uncomment after enabling index creation after materialization. - outer_col= item_in->left_expr->element_index(i); - inner_col= inner_col_it++; - */ - cur_col= 1 << i; - - if (!(cur_col & partial_match_key_parts)) + if (!bitmap_is_set(&partial_match_key_parts, i)) continue; - if (result_sink->get_column_null_count(i) == - tmp_table->file->stats.records) + if (result_sink->get_null_count_of_col(i) == 0) { - /* Column i of the temp table consists only of NULLs. */ - --count_partial_match_cols; - inner_partial_match= TRUE; - partial_match_key_parts &= ~cur_col; /* unset bit 'i' */ - } - else if (result_sink->get_column_null_count(i) == 0) - { - --count_partial_match_cols; - partial_match_key_parts &= ~cur_col; - /* - TODO - Column i of the temp table doesn't contain any NULLs. Currently we - cannot create/alter an index on an already populated internal - temporary table. As a result even if we detect that a column should - belong to the NON_NULL index, it is too late to alter that index. The - only thing we can do is change it from PARTIAL_MATCH to NON_NULL_LATE, - thus removing the "OR NULL" predicate during lookup. Once this - limitation is removed, use the commented code below instead of the - following two lines. - */ - ++count_non_null_late_cols; - non_null_late_key_parts |= cur_col; - /* - if (!inner_col->maybe_null) - { - non_null_key_parts |= cur_col; - non_null_outer_cols.push_back(outer_col); - } - else - { - ++count_non_null_late_cols; - non_null_late_key_parts |= cur_col; - } - */ + bitmap_clear_bit(&partial_match_key_parts, i); + bitmap_set_bit(&non_null_key_parts, i); + --count_partial_match_columns; } } - /* - if (non_null_outer_cols.elements > max number of key parts) - DBUG_RETURN(TRUE); - */ - /* If no column contains NULLs use regular hash index lookups. */ - if (!(non_null_late_key_parts || partial_match_key_parts)) + if (!count_partial_match_columns) strategy= COMPLETE_MATCH; DBUG_VOID_RETURN; } +/* + Initialize a MY_BITMAP with a buffer allocated on the current + memory root. +*/ + +static my_bool +bitmap_init_memroot(MY_BITMAP *map, uint n_bits, MEM_ROOT *mem_root) +{ + my_bitmap_map *bitmap_buf; + + if (!(bitmap_buf= (my_bitmap_map*) alloc_root(mem_root, + bitmap_buffer_size(n_bits))) || + bitmap_init(map, bitmap_buf, n_bits, FALSE)) + return TRUE; + return FALSE; +} + + /** Create all structures needed for IN execution that can live between PS reexecution. @@ -3253,6 +3209,11 @@ bool subselect_hash_sj_engine::init_perm DBUG_ENTER("subselect_hash_sj_engine::init_permanent"); + if (!(bitmap_init_memroot(&non_null_key_parts, tmp_columns->elements, + thd->mem_root)) || + !(bitmap_init_memroot(&partial_match_key_parts, tmp_columns->elements, + thd->mem_root))) + DBUG_RETURN(TRUE); set_strategy_using_schema(); /* @@ -3261,6 +3222,10 @@ bool subselect_hash_sj_engine::init_perm managed (created/filled/etc) internally by the interceptor. */ /* + TODO: + Select a more efficient result sink when we know there is no need to collect + data statistics. + if (strategy == COMPLETE_MATCH) { if (!(result= new select_union)) @@ -3314,13 +3279,6 @@ bool subselect_hash_sj_engine::init_perm if (make_semi_join_conds()) DBUG_RETURN(TRUE); - /* - A complete match is the best we can get, so we can immediately - create the enginte to be used for lookup. - */ - if (strategy == COMPLETE_MATCH && - !(lookup_engine= make_unique_engine())) - DBUG_RETURN(TRUE); DBUG_RETURN(FALSE); } @@ -3582,7 +3540,7 @@ int subselect_hash_sj_engine::exec() set_strategy_using_data(); /* A unique_engine is used both for complete and partial matching. */ - if (!lookup_engine && !(lookup_engine= make_unique_engine())) + if (!(lookup_engine= make_unique_engine())) { res= 1; goto err; @@ -3590,12 +3548,33 @@ int subselect_hash_sj_engine::exec() if (strategy == PARTIAL_MATCH) { - if (!(lookup_engine= new subselect_rowid_merge_engine(lookup_engine, - tmp_table))) + subselect_rowid_merge_engine *new_lookup_engine; + uint count_pm_keys; + MY_BITMAP *nn_key_parts; + /* Total number of keys needed for partial matching. */ + if (count_partial_match_columns < tmp_table->s->fields) { - res= 1; - goto err; + count_pm_keys= count_partial_match_columns + 1; + nn_key_parts= &non_null_key_parts; } + else + { + count_pm_keys= count_partial_match_columns; + nn_key_parts= NULL; + } + + if (!(new_lookup_engine= + new subselect_rowid_merge_engine(lookup_engine, + tmp_table, + count_pm_keys, + item, result)) || + new_lookup_engine->init(nn_key_parts, &partial_match_key_parts)) + { + delete new_lookup_engine; + strategy= PARTIAL_MATCH_SCAN; + /* TODO: setup execution structures for partial match via scanning. */ + } + strategy= PARTIAL_MATCH_INDEX; } item_in->change_engine(lookup_engine); @@ -3648,37 +3627,299 @@ bool subselect_hash_sj_engine::change_re } -bool Ordered_key::sort_keys() +Ordered_key::Ordered_key(uint key_idx_arg, TABLE *tbl_arg, + Item *search_key_arg, ha_rows null_count_arg, + ha_rows min_null_row_arg, ha_rows max_null_row_arg, + uchar *row_num_to_rowid_arg) + : key_idx(key_idx_arg), tbl(tbl_arg), search_key(search_key_arg), + row_num_to_rowid(row_num_to_rowid_arg), null_count(null_count_arg), + min_null_row(min_null_row_arg), max_null_row(max_null_row_arg) { - return TRUE; + key_column_count= search_key->cols(); + cur_row= HA_POS_ERROR; +} + + +/* + Initialize a multi-column index. +*/ + +bool Ordered_key::init(MY_BITMAP *columns_to_index) +{ + THD *thd= tbl->in_use; + uint cur_key_col= 0; + + DBUG_ENTER("Ordered_key::init"); + + DBUG_ASSERT(key_column_count == bitmap_bits_set(columns_to_index)); + + // TODO: check for mem allocation err, revert to scan + + key_columns= (Item_field**) thd->alloc(key_column_count * + sizeof(Item_field*)); + compare_pred= (Item_func_lt**) thd->alloc(key_column_count * + sizeof(Item_func_lt*)); + + for (uint i= 0; i < columns_to_index->n_bits; i++, cur_key_col++) + { + if (!bitmap_is_set(columns_to_index, i)) + continue; + key_columns[cur_key_col]= new Item_field(tbl->field[i]); + /* Create the predicate (tmp_column[i] < outer_ref[i]). */ + compare_pred[cur_key_col]= new Item_func_lt(key_columns[cur_key_col], + search_key->element_index(i)); + } + + if (alloc_keys_buffers()) + { + /* TODO revert to partial match via table scan. */ + DBUG_RETURN(TRUE); + } + DBUG_RETURN(FALSE); +} + + +/* + Initialize a single-column index. +*/ + +bool Ordered_key::init(int col_idx) +{ + THD *thd= tbl->in_use; + + DBUG_ENTER("Ordered_key::init"); + + DBUG_ASSERT(key_column_count == 1); + + // TODO: check for mem allocation err, revert to scan + + key_columns= (Item_field**) thd->alloc(sizeof(Item_field*)); + compare_pred= (Item_func_lt**) thd->alloc(sizeof(Item_func_lt*)); + + key_columns[0]= new Item_field(tbl->field[col_idx]); + /* Create the predicate (tmp_column[i] < outer_ref[i]). */ + compare_pred[0]= new Item_func_lt(key_columns[0], + search_key->element_index(col_idx)); + + if (alloc_keys_buffers()) + { + /* TODO revert to partial match via table scan. */ + DBUG_RETURN(TRUE); + } + DBUG_RETURN(FALSE); +} + + +bool Ordered_key::alloc_keys_buffers() +{ + THD *thd= tbl->in_use; + ha_rows row_count= tbl->file->stats.records; + + if (!(row_index= (ha_rows*) thd->alloc((row_count - null_count) * + sizeof(ha_rows)))) + return TRUE; + + /* + TODO: it is enough to create bitmaps with size + (max_null_row - min_null_row), and then use min_null_row as + lookup offset. + */ + if (!(bitmap_init_memroot(&null_key, max_null_row, + thd->mem_root))) + return TRUE; + + return FALSE; +} + + +/* + Quick sort comparison function that compares two rows of the same table + indentfied with their row numbers. +*/ + +int Ordered_key::cmp_rows_by_rownum(Ordered_key *key, ha_rows *a, ha_rows *b) +{ + uchar *rowid_a, *rowid_b; + int error, cmp_res; + TABLE *tbl= key->tbl; + /* The length in bytes of the rowids (positions) of tmp_table. */ + uint rowid_length= tbl->file->ref_length; + + DBUG_ENTER("Ordered_key::cmp_rows_by_rownum"); + if (a == b) + DBUG_RETURN(0); + /* Get the corresponding rowids. */ + rowid_a= key->row_num_to_rowid + (*a) * rowid_length; + rowid_b= key->row_num_to_rowid + (*b) * rowid_length; + /* Fetch the rows for comparison. */ + error= tbl->file->rnd_pos(tbl->record[0], rowid_a); + DBUG_ASSERT(!error); + error= tbl->file->rnd_pos(tbl->record[1], rowid_b); + DBUG_ASSERT(!error); + /* Compare the two rows. */ + for (Field **f_ptr= tbl->field; *f_ptr; f_ptr++) + { + if ((cmp_res= (*f_ptr)->cmp_offset(tbl->s->rec_buff_length))) + DBUG_RETURN(cmp_res); + } + DBUG_RETURN(0); +} + + +void Ordered_key::sort_keys() +{ + my_qsort(row_index, tbl->file->stats.records, sizeof(ha_rows), + (qsort_cmp) &cmp_rows_by_rownum); +} + + +/* + Compare the value(s) of the current key in 'search_key' with the + data of the current table record accessible via 'key_columns'. + + @notes The comparison result follows from the way compare_pred + is created in Ordered_key::init. Currently compare_pred compares + a field in of the current row with the corresponding Item that + contains the search key. + + @retval -1 if (current row < search_key) + @retval 0 if (current row == search_key) + @retval +1 if (current row > search_key) +*/ + +int Ordered_key::compare_row_with_key(ha_rows row_num) +{ + /* The length in bytes of the rowids (positions) of tmp_table. */ + uint rowid_length= tbl->file->ref_length; + uchar *cur_rowid= row_num_to_rowid + row_num * rowid_length; + int error, cmp_res; + + DBUG_ENTER("Ordered_key::compare"); + error= tbl->file->rnd_pos(tbl->record[0], cur_rowid); + DBUG_ASSERT(!error); + + for (uint i= 0; i < key_column_count; i++) + { + cmp_res= compare_pred[i]->get_comparator()->compare(); + /* Unlike Arg_comparator::compare_row() here there should be no NULLs. */ + DBUG_ASSERT(!compare_pred[i]->null_value); + if (cmp_res) + DBUG_RETURN(cmp_res); + } + DBUG_RETURN(0); } /* + Find a key in a sorted array of keys via binary search. + see create_subq_in_equalities() */ -bool Ordered_key::lookup(Item *search_key) +bool Ordered_key::lookup() { DBUG_ENTER("Ordered_key::lookup"); - DBUG_ASSERT(search_key->cols() == key_column_count); - for (uint i= 0; i < key_column_count; i++) + ha_rows lo= 0; + ha_rows hi= tbl->file->stats.records - 1; + ha_rows mid; + int cmp_res; + + while (lo <= hi) { - // j = corresponding colum at pos i - // compare(search_key->element_index(i), key_columns(j)) - ; + mid= lo + (hi - lo) / 2; + cmp_res= compare_row_with_key(mid); + + if (cmp_res == -1) + { + /* row[mid] < search_key */ + lo= mid + 1; + } + else if (cmp_res == 1) + { + /* row[mid] > search_key */ + hi= mid - 1; + } + else + { + /* row[mid] == search_key */ + cur_row= mid; + DBUG_RETURN(TRUE); + } } - DBUG_RETURN(TRUE); + + DBUG_RETURN(FALSE); } /* + @param non_null_key_parts + @param partial_match_key_parts A union of all single-column NULL key parts. + @param count_partial_match_columns Number of NULL keyparts (set bits above). */ -bool subselect_rowid_merge_engine::init() +bool +subselect_rowid_merge_engine::init(MY_BITMAP *non_null_key_parts, + MY_BITMAP *partial_match_key_parts) { - // TODO + /* The length in bytes of the rowids (positions) of tmp_table. */ + uint rowid_length= tmp_table->file->ref_length; + ha_rows row_count= tmp_table->file->stats.records; + select_materialize_with_stats *result_sink= + (select_materialize_with_stats *) result; + uint cur_key= 0; + + if (!(row_num_to_rowid= (uchar*) thd->alloc(row_count * rowid_length * + sizeof(uchar)))) + return TRUE; + + if (!(bitmap_init_memroot(&matching_keys, keys_count, thd->mem_root))) + return TRUE; + + merge_keys= (Ordered_key**) thd->alloc(keys_count * sizeof(Ordered_key*)); + /* Create the only non-NULL key if there is any. */ + if (non_null_key_parts) + { + non_null_key= new Ordered_key(cur_key, tmp_table, item, 0, 0, 0, + row_num_to_rowid); + if (non_null_key->init(non_null_key_parts)) + { + // TODO: revert to partial matching via scanning + return TRUE; + } + merge_keys[cur_key]= non_null_key; + non_null_key->sort_keys(); + ++cur_key; + } + /* + Create one single-column NULL-key for each column in + partial_match_key_parts. + */ + for (uint i= 0; i < partial_match_key_parts->n_bits; i++, cur_key++) + { + if (!bitmap_is_set(partial_match_key_parts, i)) + continue; + + merge_keys[cur_key]= new Ordered_key(cur_key, tmp_table, item, + result_sink->get_null_count_of_col(i), + result_sink->get_min_null_of_col(i), + result_sink->get_max_null_of_col(i), + row_num_to_rowid); + if (merge_keys[cur_key]->init(i)) + { + // TODO: revert to partial matching via scanning + return TRUE; + } + merge_keys[cur_key]->sort_keys(); + } + + if (init_queue(&pq, keys_count, 0, FALSE, + subselect_rowid_merge_engine::cmp_key_by_cur_row, NULL)) + { + // TODO: revert to partial matching via scanning + return TRUE; + } + return FALSE; } @@ -3690,6 +3931,41 @@ void subselect_rowid_merge_engine::clean /* +*/ + +int +subselect_rowid_merge_engine::cmp_key_by_null_selectivity(Ordered_key *a, + Ordered_key *b) +{ + double a_sel= a->null_selectivity(); + double b_sel= b->null_selectivity(); + if (a_sel == b_sel) + return 0; + if (a_sel > b_sel) + return 1; + return -1; +} + + +/* +*/ + +int +subselect_rowid_merge_engine::cmp_key_by_cur_row(void *arg, + uchar *k1, uchar *k2) +{ + ha_rows row1= ((Ordered_key*) k1)->current(); + ha_rows row2= ((Ordered_key*) k2)->current(); + + if (row1 > row2) + return 1; + if (row1 == row2) + return 0; + return -1; +} + + +/* Check if certain table row contains a NULL in all columns in all columns for which there is no value match. @@ -3704,13 +3980,11 @@ void subselect_rowid_merge_engine::clean bool subselect_rowid_merge_engine::test_null_row(ha_rows row_num) { - Ordered_key *cur_key= keys; - DBUG_ENTER("subselect_rowid_merge_engine::test_null_row"); - for (uint i = 0; i < keys_count; i++, cur_key++) + for (uint i = 0; i < keys_count; i++) { - if (bitmap_is_set(matching_keys, i)) + if (bitmap_is_set(&matching_keys, i)) { /* The key 'i' already matches a value in row 'row_num', thus we @@ -3718,7 +3992,7 @@ bool subselect_rowid_merge_engine::test_ */ continue; } - if (!cur_key->is_null(row_num)) + if (!merge_keys[i]->is_null(row_num)) DBUG_RETURN(FALSE); } DBUG_RETURN(TRUE); @@ -3736,14 +4010,13 @@ bool subselect_rowid_merge_engine::parti ha_rows min_row; /* Current row number of min_key. */ Ordered_key *cur_key; ha_rows cur_row; - Item_in_subselect *item_in= (Item_in_subselect *) item; DBUG_ENTER("subselect_rowid_merge_engine::partial_match"); /* If there is a non-NULL key, it must be the first key in the keys array. */ - DBUG_ASSERT(non_null_key && keys == non_null_key); + DBUG_ASSERT(non_null_key && merge_keys[0] == non_null_key); /* Check if there is a match for the columns of the only non-NULL key. */ - if (non_null_key && !non_null_key->lookup(item_in->left_expr)) + if (non_null_key && !non_null_key->lookup()) DBUG_RETURN(FALSE); if (non_null_key) queue_insert(&pq, (uchar *) non_null_key); @@ -3753,10 +4026,10 @@ bool subselect_rowid_merge_engine::parti non_null_key, since it was already processed above. */ uint i= non_null_key ? 1 : 0; /* Skip the non-NULL key, already processed. */ - for (cur_key= keys; i < keys_count; i++, cur_key++) + for (; i < keys_count; i++) { - if (cur_key->lookup(item_in->left_expr)) - queue_insert(&pq, (uchar *) cur_key); + if (merge_keys[i]->lookup()) + queue_insert(&pq, (uchar *) merge_keys[i]); } /* Not all value keys are empty, thus we don't have only NULL keys. If we had, @@ -3767,8 +4040,8 @@ bool subselect_rowid_merge_engine::parti DBUG_ASSERT(pq.elements > 1); min_key= (Ordered_key*) queue_remove(&pq, 0); min_row= min_key->current(); - bitmap_clear_all(matching_keys); - bitmap_set_bit(matching_keys, min_key->get_key_idx()); + bitmap_clear_all(&matching_keys); + bitmap_set_bit(&matching_keys, min_key->get_key_idx()); min_key->next(); if (!min_key->is_eof()) queue_insert(&pq, (uchar *) min_key); @@ -3780,9 +4053,9 @@ bool subselect_rowid_merge_engine::parti if (cur_row == min_row) { - bitmap_set_bit(matching_keys, cur_key->get_key_idx()); + bitmap_set_bit(&matching_keys, cur_key->get_key_idx()); /* There cannot be a complete match, as we already checked for one. */ - DBUG_ASSERT(bitmap_bits_set(matching_keys) < matching_keys->n_bits); + DBUG_ASSERT(bitmap_bits_set(&matching_keys) < matching_keys.n_bits); } else { @@ -3794,8 +4067,8 @@ bool subselect_rowid_merge_engine::parti { min_key= cur_key; min_row= cur_row; - bitmap_clear_all(matching_keys); - bitmap_set_bit(matching_keys, min_key->get_key_idx()); + bitmap_clear_all(&matching_keys); + bitmap_set_bit(&matching_keys, min_key->get_key_idx()); } } === modified file 'sql/item_subselect.h' --- a/sql/item_subselect.h 2010-01-22 16:18:05 +0000 +++ b/sql/item_subselect.h 2010-02-01 12:09:48 +0000 @@ -683,68 +683,70 @@ class Ordered_key { protected: /* - Index of the key in some array of keys. This index allows to + Index of the key in an array of keys. This index allows to construct (sub)sets of keys represented by bitmaps. */ uint key_idx; + /* The table being indexed. */ + TABLE *tbl; /* The columns being indexed. */ Item_field **key_columns; /* Number of elements in 'key_columns' (number of key parts). */ uint key_column_count; + /* + An expression, or sequence of expressions that forms the search key. + */ + Item *search_key; /* Value index related members. */ - /* The actual value index, consists of a sorted sequence of row numbers. */ + /* + The actual value index, consists of a sorted sequence of row numbers. + There are tbl->file->stats.records elements in this array. + */ ha_rows *row_index; /* Current element in 'row_index'. */ ha_rows cur_row; /* - TODO: define a quick sort comparison function. + Mapping from row numbers to row ids. The element row_num_to_rowid[i] + contains a buffer with the rowid for the row numbered 'i'. + The memory for this member is not maintanined by this class because + all Ordered_key indexes of the same table share the same mapping. + */ + uchar *row_num_to_rowid; + /* + A sequence of predicates to compare the search key with the corresponding + columns of a table row from the index. */ + Item_func_lt **compare_pred; /* Null index related members. */ MY_BITMAP null_key; /* Count of NULLs per column. */ ha_rows null_count; - /* The row number that contains the last NULL in a column. */ - ha_rows max_null_row; /* The row number that contains the first NULL in a column. */ ha_rows min_null_row; - /* - TODO: define a qsort comparison function to compare keys in order of - increasing bitmap selectivity. - */ + /* The row number that contains the last NULL in a column. */ + ha_rows max_null_row; protected: - ha_rows get_row_count() - { - /* Assume that file->info(HA_STATUS_VARIABLE) has been called. */ - return key_columns[0]->field->table->file->stats.records; - } + bool alloc_keys_buffers(); /* - Compute the index (position) of an indexed column in the table definition. - - @param i index in the 'key_columns' array. - - @returns The index of the corresponding indexed column in the TABLE::field - array of all table fields. + Quick sort comparison function that compares two rows of the same table + indentfied with their row numbers. */ - uint get_column_idx(uint i) - { - DBUG_ENTER("get_column_idx"); - DBUG_ASSERT(i < key_column_count); - /* All key_columns must be from the same table, so any one is fine. */ - //TABLE *tab= key_columns[0]->field->table; - //Field *col_i= columns->field + i; - //DBUG_RETURN(col_i - tab->field); - DBUG_RETURN(0); - } + static int cmp_rows_by_rownum(Ordered_key *key, ha_rows* a, ha_rows* b); + + int compare_row_with_key(ha_rows row_num); public: - Ordered_key(TABLE *tab) - { - /* TODO: init all Item_fields from the table columns. */ - } - bool init(ha_rows row_count); + Ordered_key(uint key_idx_arg, TABLE *tbl_arg, + Item *search_key_arg, ha_rows null_count_arg, + ha_rows min_null_row_arg, ha_rows max_null_row_arg, + uchar *row_num_to_rowid_arg); + /* Initialize a multi-column index. */ + bool init(MY_BITMAP *columns_to_index); + /* Initialize a single-column index. */ + bool init(int col_idx); uint get_column_count() { return key_column_count; } uint get_key_idx() { return key_idx; } @@ -753,21 +755,23 @@ public: row_index[cur_row]= row_num; ++cur_row; } - bool sort_keys(); + + void sort_keys(); + + double null_selectivity() { return (1 - null_count / null_key.n_bits); } + /* Position the current element at the first row that matches the key. - TODO: the argument here is the left IN argument, which is a sequence - of Items. We have to compare these Items with the corresponding Fields - of the temp table. To do this wrap each field in an Item_field, then - compare. See how it is done in create_subq_in_equalities(). + The key itself is propagated by evaluating the current value(s) of + this->search_key. */ - bool lookup(Item *search_key); + bool lookup(); /* Return the current index element. */ ha_rows current() { return row_index[cur_row]; } /* Move the current index cursor at the next match. */ bool next() { - if (cur_row < get_row_count()) + if (cur_row < tbl->file->stats.records) { ++cur_row; return TRUE; @@ -775,7 +779,7 @@ public: return FALSE; }; /* Return false if all matches are exhausted, true otherwise. */ - bool is_eof() { return cur_row == get_row_count(); } + bool is_eof() { return cur_row == tbl->file->stats.records; } void set_null(ha_rows row_num) { @@ -789,9 +793,9 @@ public: Their only initialized member is 'n_bits', which is equal to the number of temp table rows. */ - if (null_count == get_row_count()) + if (null_count == tbl->file->stats.records) { - DBUG_ASSERT(get_row_count() == null_key.n_bits); + DBUG_ASSERT(tbl->file->stats.records == null_key.n_bits); DBUG_RETURN(TRUE); } if (row_num > max_null_row || row_num < min_null_row) @@ -812,20 +816,16 @@ protected: FALSE and UNKNOWN. */ subselect_engine *lookup_engine; - - /* The length in bytes of the rowids (positions) of tmp_table. */ - uint rowid_length; /* - Mapping from row numbers to row ids. The element 'i' with lenght - 'rowid_length' - (row_num_to_rowid + i*rowid_length) contains - the rowid of row numbered 'i'. + Mapping from row numbers to row ids. The element row_num_to_rowid[i] + contains a buffer with the rowid for the row numbered 'i'. */ uchar *row_num_to_rowid; /* A subset of all the keys for which there is a match for the same row. Used during execution. Computed for each call to exec(). */ - MY_BITMAP *matching_keys; + MY_BITMAP matching_keys; /* Indexes of row numbers, sorted by <column_value, row_number>. If an index may contain NULLs, the NULLs are stored efficiently in a bitmap. @@ -834,7 +834,7 @@ protected: one with the fewer NULLs is first. Thus, if there is any index on non-NULL columns, it is contained in keys[0]. */ - Ordered_key *keys; + Ordered_key **merge_keys; /* The number of elements in keys. */ uint keys_count; /* @@ -849,33 +849,31 @@ protected: This queue is used by the partial match algorithm in method exec(). */ QUEUE pq; - +protected: /* - True if some column in the temp table consist of only NULLs. Then - any match is a partial match. + Comparison function to compare keys in order of increasing bitmap + selectivity. */ - bool inner_partial_match; - bool null_keypart; /* TRUE <=> constructed search tuple has a NULL */ + static int cmp_key_by_null_selectivity(Ordered_key *a, Ordered_key *b); /* - A conjunction of all the equality condtions between all pairs of expressions - that are arguments of an IN predicate. We need these to post-filter some - IN results because index lookups sometimes match values that are actually - not equal to the search key in SQL terms. + Comparison function used by the priority queue pq, the 'smaller' key + is the one with the smaller current row number. */ - Item_cond_and *semi_join_conds; -protected: + static int cmp_key_by_cur_row(void *arg, uchar *k1, uchar *k2); + bool test_null_row(ha_rows row_num); bool partial_match(); public: subselect_rowid_merge_engine(subselect_engine *lookup_engine_arg, - TABLE *tmp_table_arg) - :subselect_engine(NULL, NULL) - { - lookup_engine= lookup_engine_arg; - tmp_table= tmp_table_arg; - rowid_length= tmp_table->file->ref_length; - } - bool init(); + TABLE *tmp_table_arg, uint keys_count_arg, + Item_subselect *item_arg, + select_result_interceptor *result_arg) + :subselect_engine(item_arg, result_arg), + tmp_table(tmp_table_arg), lookup_engine(lookup_engine_arg), + keys_count(keys_count_arg) + {} + + bool init(MY_BITMAP *non_null_key_parts, MY_BITMAP *partial_match_key_parts); void cleanup(); int prepare() { return 0; } void fix_length_and_dec(Item_cache**) {} @@ -930,15 +928,11 @@ protected: */ bool has_null_row; - /* Keyparts of the only non-NULL composite index in a ror_intersect. */ - key_part_map non_null_key_parts; - List<Item> non_null_outer_cols; /* Corresponding non-NULL outer columns. */ - /* keyparts of the non-NULL single column indexes, one keypart per index. */ - key_part_map non_null_late_key_parts; - /* keyparts of the single column indexes with NULL, one keypart per index. */ - key_part_map partial_match_key_parts; - uint count_non_null_late_cols, count_partial_match_cols; - + /* Keyparts of the only non-NULL composite index in a rowid merge. */ + MY_BITMAP non_null_key_parts; + /* Keyparts of the single column indexes with NULL, one keypart per index. */ + MY_BITMAP partial_match_key_parts; + uint count_partial_match_columns; /* A conjunction of all the equality condtions between all pairs of expressions that are arguments of an IN predicate. We need these to post-filter some @@ -948,8 +942,10 @@ protected: Item *semi_join_conds; /* Possible execution strategies that can be used to compute hash semi-join.*/ enum exec_strategy { - COMPLETE_MATCH, /* Use plain index lookups. */ - PARTIAL_MATCH, /* Use partial matching. */ + COMPLETE_MATCH, /* Use regular index lookups. */ + PARTIAL_MATCH, /* Use some partial matching strategy. */ + PARTIAL_MATCH_INDEX, /* Use partial matching through index merging. */ + PARTIAL_MATCH_SCAN, /* Use partial matching through table scan. */ IMPOSSIBLE /* Subquery materialization is not applicable. */ }; /* The chosen execution strategy. Computed after materialization. */ @@ -965,14 +961,10 @@ public: subselect_single_select_engine *old_engine) :subselect_engine(in_predicate, NULL), tmp_table(NULL), is_materialized(FALSE), materialize_engine(old_engine), lookup_engine(NULL), - materialize_join(NULL), inner_partial_match(FALSE), - count_non_null_late_cols(0), count_partial_match_cols(0), + materialize_join(NULL), count_partial_match_columns(0), semi_join_conds(NULL) { set_thd(thd); - non_null_key_parts= (key_part_map) 0; - non_null_late_key_parts= (key_part_map) 0; - partial_match_key_parts= (key_part_map) 0; } ~subselect_hash_sj_engine(); === modified file 'sql/sql_class.h' --- a/sql/sql_class.h 2010-01-22 16:18:05 +0000 +++ b/sql/sql_class.h 2010-02-01 12:09:48 +0000 @@ -3079,11 +3079,21 @@ public: count_rows= 0; memset(col_stat, 0, table->s->fields * sizeof(Column_statistics)); } - ha_rows get_column_null_count(uint idx) + ha_rows get_null_count_of_col(uint idx) { DBUG_ASSERT(idx < table->s->fields); return col_stat[idx].null_count; } + ha_rows get_max_null_of_col(uint idx) + { + DBUG_ASSERT(idx < table->s->fields); + return col_stat[idx].max_null_row; + } + ha_rows get_min_null_of_col(uint idx) + { + DBUG_ASSERT(idx < table->s->fields); + return col_stat[idx].min_null_row; + } ha_rows get_null_record_count() { return null_record_count; } };

1 0

Re: [Maria-developers] [Branch ~maria-captains/maria/5.1] Rev 2809: Support building with system libevent
by Kristian Nielsen 01 Feb '10

01 Feb '10

Sergei Golubchik <serg(a)askmonty.org> writes: >> Then I also need to decide what to do for packages when system >> libevent is too old. I checked, and Debian 4+5 and Ubuntu up to 9.04 >> have only libevent 1.3. Ubuntu >= 9.10 has libevent 1.4. >> >> I think the best is to use bundled libevent on those older distros. We >> will then still suffer from >> https://bugs.launchpad.net/maria/+bug/509521 on those, but hopefully >> most people using plugins which require libevent will build mariadb >> themselves and/or use a newer distro. > > What about specifying a deb dependency that mariadb needs > libevent >= 1.4 ? > > Then package manager will handle the rest I don't understand what you mean? On Ubuntu 9.10 and up specifying a dependency on libevent >=1.4 will work, yes. But on Debian <= 5 and Ubuntu <= 9.04, a dependency on libevent >= 1.4 would make the package uninstallable, as libevent 1.4 is not available in those distros. Or did you mean we should provide libevent 1.4 as a package in the OurDelta repository? I think using system libevent on Ubuntu >= 9.10 and bundled on Ubuntu <= 9.04 and Debian should work ok. Most people installing something like the Pinba storage engine (which uses libevent) will probably in any case recompile mariadb. Whatever packages do will not affect them, so the important thing is that with your patch, they can now get MariaDB to use libevent 1.4 if they install it. - Kristian.

1 0

[Maria-developers] really need call my_thread_init() in libmysqlclient_r?
by xiaobing jiang 01 Feb '10

01 Feb '10

HI, all: There is a bug about mysql-python, https://sourceforge.net/tracker/?func=detail&aid=1464563&group_id=22307&ati…. After review the code of libmysqlclient_r , I find for every new thread to create connection, it will call my_thread_init to init some thread local variable and other things. but the client api seems not using this thread local variable almost, just for some debug information. Can using macro remove this when the code using by libmysqlclient_r.so? If can do that, to fix the bug is easy. sorry for my poor engish, and this maybe should post in mysql mail-list! Thanks!

1 0

[Maria-developers] Rev 2738: merge 5.1->5.2 in file:///home/bell/maria/bzr/work-maria-5.2-merge-5.1/
by sanja＠askmonty.org 01 Feb '10

01 Feb '10

At file:///home/bell/maria/bzr/work-maria-5.2-merge-5.1/ ------------------------------------------------------------ revno: 2738 [merge] revision-id: sanja(a)askmonty.org-20100201061412-xx7312ytg9d6qnbk parent: monty(a)askmonty.org-20100106192016-46to56kdmp1frlgl parent: monty(a)askmonty.org-20100129184222-45b3lr7hp2joa5mp committer: sanja(a)askmonty.org branch nick: work-maria-5.2-merge-5.1 timestamp: Mon 2010-02-01 08:14:12 +0200 message: merge 5.1->5.2 removed: mysql-test/suite/maria/t/maria2-master.opt maria2master.opt-20090215104617-4fkcge2h3ssmgmv0-1 storage/xtradb/handler/handler0vars.h handler0vars.h-20081203050234-edoolglm28lyejuc-3 storage/xtradb/handler/win_delay_loader.cc win_delay_loader.cc-20081203050234-edoolglm28lyejuc-4 storage/xtradb/win-plugin/ winplugin-20081203050234-edoolglm28lyejuc-2 storage/xtradb/win-plugin/README readme-20081203050234-edoolglm28lyejuc-15 storage/xtradb/win-plugin/win-plugin.diff winplugin.diff-20081203050234-edoolglm28lyejuc-16 added: BUILD/compile-bintar compilebintar-20100107101810-lelof47hh40zljzw-1 BUILD/util.sh util.sh-20100107105306-5523083hapn1b4n5-1 mysql-test/extra/rpl_tests/rpl_not_null.test rpl_not_null.test-20090929140808-ln24vq9he9qkn20x-1 mysql-test/r/bug47671.result bug47671.result-20091125061811-9f1u8eyhc4lhli89-1 mysql-test/r/innodb-consistent.result innodbconsistent.res-20100106115838-xm7yncpgr8fsekjt-2 mysql-test/r/innodb_bug44571.result innodb_bug44571.resu-20100106115838-xm7yncpgr8fsekjt-6 mysql-test/r/innodb_bug46676.result innodb_bug46676.resu-20100106115838-xm7yncpgr8fsekjt-10 mysql-test/r/innodb_bug47167.result innodb_bug47167.resu-20100106115838-xm7yncpgr8fsekjt-12 mysql-test/r/udf_query_cache.result udf_query_cache.resu-20100111125806-7lkv720m352b58ia-1 mysql-test/std_data/bug47012.ARM bug47012.arm-20091110122641-iqp1lul6n2ekz7m1-3 mysql-test/std_data/bug47012.ARZ bug47012.arz-20091110122641-iqp1lul6n2ekz7m1-2 mysql-test/std_data/bug47012.frm bug47012.frm-20091110122641-iqp1lul6n2ekz7m1-1 mysql-test/suite/innodb/r/innodb_bug46676.result innodb_bug46676.resu-20091130121246-w0etrydh59vqrhfk-1 mysql-test/suite/innodb/r/innodb_bug47167.result innodb_bug47167.resu-20091130114932-z1l0gqbz3zjhelcn-1 mysql-test/suite/innodb/t/innodb_bug46676.test innodb_bug46676.test-20091130121246-w0etrydh59vqrhfk-2 mysql-test/suite/innodb/t/innodb_bug47167.test innodb_bug47167.test-20091130114932-z1l0gqbz3zjhelcn-2 mysql-test/suite/rpl/r/rpl_loaddata_symlink.result rpl_loaddata_symlink-20091112200612-x5ek042xipzbdluy-1 mysql-test/suite/rpl/r/rpl_nondeterministic_functions.result rpl_nondeterministic-20091118142013-9zx8msk45pqp0kcn-1 mysql-test/suite/rpl/r/rpl_not_null_innodb.result rpl_not_null_innodb.-20090929140808-ln24vq9he9qkn20x-2 mysql-test/suite/rpl/r/rpl_not_null_myisam.result rpl_not_null_myisam.-20090929140808-ln24vq9he9qkn20x-3 mysql-test/suite/rpl/r/rpl_row_trunc_temp.result rpl_row_trunc_temp.r-20091117145658-biyfd3a42vxttw4g-1 mysql-test/suite/rpl/t/rpl_loaddata_symlink-master.opt rpl_loaddata_symlink-20091112200645-33lzxlcjtrz4ekex-1 mysql-test/suite/rpl/t/rpl_loaddata_symlink-master.sh rpl_loaddata_symlink-20091120052332-4iio0f9yuk8srq37-1 mysql-test/suite/rpl/t/rpl_loaddata_symlink-slave.opt rpl_loaddata_symlink-20091120052318-rey16d8e27esu4af-1 mysql-test/suite/rpl/t/rpl_loaddata_symlink-slave.sh rpl_loaddata_symlink-20091120052322-exs2kyw12kmns1tw-1 mysql-test/suite/rpl/t/rpl_loaddata_symlink.test rpl_loaddata_symlink-20091112200628-bv7esgx1vr7rhcky-1 mysql-test/suite/rpl/t/rpl_nondeterministic_functions.test rpl_nondeterministic-20091118141837-p0896pkizwxx60yu-1 mysql-test/suite/rpl/t/rpl_not_null_innodb.test rpl_not_null_innodb.-20090929140808-ln24vq9he9qkn20x-4 mysql-test/suite/rpl/t/rpl_not_null_myisam.test rpl_not_null_myisam.-20090929140808-ln24vq9he9qkn20x-5 mysql-test/suite/rpl/t/rpl_row_trunc_temp.test rpl_row_trunc_temp.t-20091117145658-biyfd3a42vxttw4g-2 mysql-test/t/bug47671-master.opt bug47671master.opt-20091125061855-277zbp2qtbff4akr-1 mysql-test/t/bug47671.test bug47671.test-20091125061804-u0cz6u7eeeta2ao3-1 mysql-test/t/innodb-consistent-master.opt innodbconsistentmast-20100106115838-xm7yncpgr8fsekjt-1 mysql-test/t/innodb-consistent.test innodbconsistent.tes-20100106115838-xm7yncpgr8fsekjt-3 mysql-test/t/innodb_bug44571.test innodb_bug44571.test-20100106115838-xm7yncpgr8fsekjt-7 mysql-test/t/innodb_bug46676.test innodb_bug46676.test-20100106115838-xm7yncpgr8fsekjt-11 mysql-test/t/innodb_bug47167.test innodb_bug47167.test-20100106115838-xm7yncpgr8fsekjt-13 mysql-test/t/mysqlbinlog2-master.opt mysqlbinlog2master.o-20100119103451-7nkhltk4tgr4fegm-1 mysql-test/t/mysqlbinlog_row-master.opt mysqlbinlog_rowmaste-20100127131139-b8fh6zh2p2j13cfj-1 mysql-test/t/mysqlbinlog_row_innodb-master.opt mysqlbinlog_row_inno-20100119125405-22esbb4wolnwm5h3-1 mysql-test/t/mysqlbinlog_row_myisam-master.opt mysqlbinlog_row_myis-20100119125405-22esbb4wolnwm5h3-2 mysql-test/t/mysqlbinlog_row_trans-master.opt mysqlbinlog_row_tran-20100127113200-cas1q0skxod0t0xi-2 mysql-test/t/udf_query_cache-master.opt udf_query_cachemaste-20100111125806-7lkv720m352b58ia-2 mysql-test/t/udf_query_cache.test udf_query_cache.test-20100111125806-7lkv720m352b58ia-3 storage/xtradb/ut/ut0auxconf_have_gcc_atomics.c ut0auxconf_have_gcc_-20100106115838-xm7yncpgr8fsekjt-16 renamed: mysql-test/suite/pbxt/t/load_unique_error1.inc => mysql-test/std_data/pbxt_load_unique_error1.inc load_unique_error1.i-20090407105731-jrdzpnlb2nlsfdp1-1 modified: .bzrignore sp1f-ignore-20001018235455-q4gxfbritt5f42nwix354ufpsvrf5ebj BUILD/Makefile.am sp1f-makefile.am-20020102192940-dza66ux2yxyklupzjz4q3km3hvye5rnj BUILD/SETUP.sh sp1f-setup.sh-20001218212418-itvzddls4bsqffggcsjklbawdmaxdhde BUILD/compile-solaris-amd64-debug-forte* compilesolarisamd64d-20090707110736-p2i53hs87u5tkgxs-1 BUILD/compile-solaris-x86-32* compilesolarisx8632-20090707110736-p2i53hs87u5tkgxs-2 BUILD/compile-solaris-x86-32-debug* compilesolarisx8632d-20090707110736-p2i53hs87u5tkgxs-3 BUILD/compile-solaris-x86-32-debug-forte* compilesolarisx8632d-20090707110736-p2i53hs87u5tkgxs-4 BUILD/compile-solaris-x86-forte-32* compilesolarisx86for-20090707110736-p2i53hs87u5tkgxs-5 Makefile.am sp1f-makefile.am-19700101030959-jbbpiygwpgybyqknlavdxxupbrjonu7h client/mysql.cc sp1f-mysql.cc-19700101030959-5sipizk7ehvbsi3tywrkdords5qy5zdl client/mysqlbinlog.cc sp1f-mysqlbinlog.cc-19700101030959-b3vgyo47ljent5mhbyj6ik33bi4bukad client/mysqlslap.c sp1f-mysqlslap.c-20051130000206-7t375hf5mtlqof5xd4nj76yckxvxykhv client/mysqltest.cc sp1f-mysqltest.c-20001010065317-ix4zw26srlev7yugcz455ux22zwyynyf config/ac-macros/plugins.m4 sp1f-plugins.m4-20060413204924-cltp6uagmyygsgdno6od3mamfizdhk3m configure.in sp1f-configure.in-19700101030959-mgdpoxtnh2ewmvusvfpkreuhwvffkcjw extra/comp_err.c sp1f-comp_err.c-19700101030959-xhnod5xbbwq5dckoic5y65at66d3sgik extra/libevent/devpoll.c devpoll.c-20090312215838-41pxaswf0zgarxu3-30 extra/libevent/evbuffer.c evbuffer.c-20090312215838-41pxaswf0zgarxu3-33 extra/yassl/src/yassl_error.cpp sp1f-yassl_error.cpp-20050428132311-uwd5s2khyebi5wkzp66p3hhvr4sh44f3 extra/yassl/taocrypt/include/asn.hpp sp1f-asn.hpp-20050428132312-5ijcjgxj7cy3t67jcpi4rg3rbr4nnfmn extra/yassl/taocrypt/src/asn.cpp sp1f-asn.cpp-20050428132318-okq6hllvtur6rcfg4gc5pbxebunf764v include/ft_global.h sp1f-ft_global.h-19700101030959-qzez255ofrojrptdc5z2oi3sfi3bemf7 include/maria.h sp1f-maria.h-20060411134400-ylx7cem3pcdf2jg6it2tuutxyzoljzvv include/my_pthread.h sp1f-my_pthread.h-19700101030959-z4yp3kljwx5fgmhlyvumtwxuw73xgrjn include/mysql.h sp1f-mysql.h-19700101030959-soi7hu6ji273nui3fm25jjf4m4362pcw include/mysql.h.pp mysql.h.pp-20080613094407-2m1760u4zdzt4dc7-1 include/violite.h sp1f-violite.h-19700101030959-jfyqeh5pmto4ncgcdcdf36bl5ininiqx libmysql/libmysql.c sp1f-libmysql.c-19700101030959-ba4gwsjdmik5puh2qyrfpvoflwer257l libmysql/libmysql.def sp1f-libmysql.def-20011019010014-dtnxyq3opddoys6ptxjmzhmaa2rlowmq libmysqld/libmysqld.def sp1f-libmysqld.def-20020109073846-c3x4ovkgky3uok5h6uiaxaevadsluq6k mysql-test/collections/default.experimental default.experimental-20090224104813-e52mxw708penxv44-1 mysql-test/extra/rpl_tests/rpl_extraSlave_Col.test sp1f-rpl_extraslave_col.t-20061103140340-egmkull7owd2wp7d4egg6itzef6p7g23 mysql-test/extra/rpl_tests/rpl_row_tabledefs.test sp1f-rpl_row_tabledefs.te-20051222053451-cr6a33nz4a4knerv7ws3ffszjgkqfet3 mysql-test/extra/rpl_tests/rpl_stm_000001.test sp1f-rpl000001.test-20001118063528-ailyrmllkfzwjx3qfvmu555ijzuk5yur mysql-test/include/mtr_warnings.sql sp1f-mtr_warnings.sql-20080408145123-lhtlr627ins6hwi3hxjrcytx4t27nyjr mysql-test/include/ps_conv.inc sp1f-ps_conv.inc-20040925150736-7yq4rnzrahaz656cmry5skpgvu5fjbet mysql-test/lib/mtr_cases.pm sp1f-mtr_cases.pl-20050203205008-rrteoawyobvgq6u7zeyce4tmuu334ayg mysql-test/mysql-test-run.pl sp1f-mysqltestrun.pl-20041230152716-xjnn5ndv4rr4by6ijmj5a4ysubxc7qh3 mysql-test/r/archive.result sp1f-archive.result-20040525194738-teb7vr2fyyav2vmvw55tdwgvu3h65flc mysql-test/r/delayed.result sp1f-delayed.result-20001228015633-d5brh5c3ulnb2qshtfvxu5cvvvxf4lsr mysql-test/r/delete.result sp1f-delete.result-20010928050551-vf5sxtd554vuepifylwowaaq7k3mbilw mysql-test/r/fulltext.result sp1f-fulltext.result-20001228015633-fi5pm63lvhgn665dsef6fjihfkijbrvt mysql-test/r/func_group.result sp1f-func_group.result-20001228015633-oe57bieiww3s6erojiyha7p26m5ul5ql mysql-test/r/func_misc.result sp1f-func_misc.result-20001228015633-4sy6dzzt7xcs4ubzcxloyguc7zhougbr mysql-test/r/grant.result sp1f-grant.result-20020905131705-2gfwpyej777fcllxzcvadzd6tqdxfho3 mysql-test/r/grant2.result sp1f-grant2.result-20030722200047-flh2uaxcbwah7yfj5uohcoxndutgaced mysql-test/r/group_min_max.result sp1f-group_min_max.result-20040827133611-aqzadxttbw23mkanmvdsiaambv2pcy27 mysql-test/r/information_schema.result sp1f-information_schema.r-20041113105544-waoxa2fjjsicturpothmjmi6jc3yrovn mysql-test/r/information_schema_all_engines.result information_schema_a-20090408133348-au36idguotknighe-2 mysql-test/r/innodb-autoinc.result innodbautoinc.result-20081201061010-zymrrwrczns2vrex-280 mysql-test/r/innodb-index.result innodbindex.result-20081201061010-zymrrwrczns2vrex-284 mysql-test/r/innodb-timeout.result innodbtimeout.result-20081203050234-edoolglm28lyejuc-7 mysql-test/r/innodb-zip.result innodbzip.result-20081201061010-zymrrwrczns2vrex-296 mysql-test/r/innodb.result innodb.result-20081201061010-zymrrwrczns2vrex-298 mysql-test/r/innodb_bug36169.result innodb_bug36169.resu-20081201061010-zymrrwrczns2vrex-306 mysql-test/r/innodb_bug44369.result innodb_bug44369.resu-20091005111405-nbp5t33h95jrqha2-1 mysql-test/r/innodb_file_format.result innodb_file_format.r-20090923000535-ke95wdd4zn27df71-18 mysql-test/r/innodb_lock_wait_timeout_1.result bug40113.result-20090619150423-w3im08cym6tyzn8f-3 mysql-test/r/innodb_mysql.result sp1f-innodb_mysql.result-20060426055153-bychbbfnqtvmvrwccwhn24i6yi46uqjv mysql-test/r/innodb_xtradb_bug317074.result innodb_xtradb_bug317-20090326061054-ylrdb8libxw6u7e9-8 mysql-test/r/mysql.result sp1f-mysql.result-20050517191330-5ywsflw7k6pttof273om5l2mb7pyiu22 mysql-test/r/mysqltest.result sp1f-mysqltest.result-20041022024801-dfor5httbrm4yhbhqtfjzpkst5hoejym mysql-test/r/olap.result sp1f-olap.result-20020720115150-egx2d46xkyxi5dgcpyjexyj4ri6wlcvb mysql-test/r/openssl_1.result sp1f-ssl.result-20010831211351-xa6w74zno32dlg3iwugerlalsvrsq5hn mysql-test/r/order_by.result sp1f-order_by.result-20001228015634-omkoitbok7pbz53pkfmplnhbifnrebic mysql-test/r/partition.result sp1f-partition.result-20050718113029-xlmjyugiq5h2b5wjp236ipsmkmej7i62 mysql-test/r/query_cache.result sp1f-query_cache.result-20011205230530-qf3qzwsqsgfi67vv5ijruxeci6cbkwjl mysql-test/r/query_cache_notembedded.result sp1f-query_cache_notembed-20050729121335-enxz2r7srcrudvsmkq357ior3n4nlqpq mysql-test/r/range.result sp1f-range.result-20001228015634-6hpoyn74lnc7irf4gop2jbowgpazbbae mysql-test/r/select.result sp1f-select.result-20010103001548-znkoalxem6wchsbxizfosjhpfmhfyxuk mysql-test/r/show_check.result sp1f-show_check.result-20001228015634-5hf7elb3nj3zmuz6tosvytmbu52bploi mysql-test/r/sp-destruct.result sp1f-spdestruct.result-20051026133447-inwbkiot3w72y54qgbh3r3qkm7io632d mysql-test/r/sp-security.result sp1f-spsecurity.result-20031213154048-xglie74lizlzappe5papku3ysbvrzg75 mysql-test/r/sp.result sp1f-sp.result-20030117133802-duandg3yzagzyv7zhhbbt2kcomcegpc7 mysql-test/r/sp_notembedded.result sp1f-sp_notembedded.resul-20060224163410-okgh2uh6w7jxcoszw5y4sk6pq3ngt5n6 mysql-test/r/trigger.result sp1f-trigger.result-20040907122911-6m6f5d2ijohoqspgy53ybn6kavj4zefi mysql-test/r/trigger_notembedded.result sp1f-triggergrant.result-20051110192455-2zus7d4a7l2y7ldnokefkk6ibykyn46y mysql-test/r/type_newdecimal.result sp1f-type_newdecimal.resu-20050208224936-kj6dxol5i7zbbfohybib53ffje5i63mk mysql-test/r/type_year.result sp1f-type_year.result-20001228015634-qnsjzaaz3ams6pb2etie5s3eleghfgp5 mysql-test/r/udf.result sp1f-udf.result-20060215161120-pm5l3nyny5gbznc2egfu4bhwgxbuc6wz mysql-test/r/variables.result sp1f-variables.result-20001228015635-w5m2doorn7gzhyyhpqrlqsupnwn6f6xh mysql-test/std_data/cacert.pem sp1f-cacert.pem-20010724060723-mzsvdgy4lyvjjx62aqycz65bca4q4ien mysql-test/std_data/client-cert.pem sp1f-clientcert.pem-20010724060723-x4ijt4gnmldkenng3te7iebn6qfarrjj mysql-test/std_data/client-key.pem sp1f-clientkey.pem-20010822071134-2mmefinjcnjzc2gu6vh3ord32xvhpbch mysql-test/std_data/server-cert.pem sp1f-servercert.pem-20010724060723-w7e2s2asnomtwtus3ncffn53f2qjifu6 mysql-test/std_data/server-key.pem sp1f-serverkey.pem-20010822071134-trmhdwb2jmf3nfahr76457fjthitvdm7 mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result sp1f-mix_innodb_myisam_bi-20030822133916-l72xhg2oxjthj6ichxowk55lrbjebfxa mysql-test/suite/binlog/r/binlog_stm_row.result binlog_stm_row.resul-20080929134451-58v46d7hr9wzyo6f-1 mysql-test/suite/binlog/r/binlog_unsafe.result sp1f-binlog_unsafe.result-20070514124535-jqttmp2p2jfelgeewle4swp5rb4j7pbi mysql-test/suite/binlog/t/binlog_killed.test sp1f-binlog_killed.test-20070528192019-wgkf3lpghurbledmqfyi43fg3mlrhxby mysql-test/suite/binlog/t/binlog_stm_mix_innodb_myisam.test sp1f-binlog_stm_mix_innod-20051222053459-semsiuc62fcxnhjqz663pgpzwfjexwci mysql-test/suite/binlog/t/binlog_stm_row.test binlog_stm_row.test-20080929134444-cku0pcqzy8apdoac-1 mysql-test/suite/binlog/t/binlog_unsafe.test sp1f-binlog_unsafe.test-20070514124535-pau2ov4yta3qsk5zbdwkywce3vhd54xr mysql-test/suite/funcs_1/datadict/processlist_val.inc sp1f-processlist_val.inc-20070815194641-3hfsmyh3jr2gjhvgms52iydpkamdbnoz mysql-test/suite/funcs_1/r/is_columns_is.result sp1f-is_columns_is.result-20080307163304-7bd6seaxklddmff6f3bb54inlyw6unpw mysql-test/suite/funcs_1/r/is_tables_is.result sp1f-is_tables_is.result-20080307163304-6xl5vbegso6wet3dzoehxb645vntxpig mysql-test/suite/innodb/r/innodb-index.result innodbindex.result-20090610132748-q9m60aph2eqy8zr6-10 mysql-test/suite/innodb/t/innodb-consistent-master.opt innodbconsistentmast-20091009132511-05q1yxchk8rz94rf-1 mysql-test/suite/innodb/t/innodb-index.test innodbindex.test-20090610132748-q9m60aph2eqy8zr6-24 mysql-test/suite/maria/r/maria-recover.result mariarecover.result-20080602174048-lgw3ipowzkym118b-1 mysql-test/suite/maria/t/maria-recover.test mariarecover.test-20080602174033-rnr5wg8wn2bqarwk-1 mysql-test/suite/parts/inc/partition_timestamp.inc sp1f-partition_timestamp.-20070206122236-ukxavk6zxnrewb3yuohuvwqpnkf3pfne mysql-test/suite/parts/r/partition_recover_myisam.result partition_repair_myi-20080609121315-mjya2e9ekn7bunzm-3 mysql-test/suite/parts/t/partition_alter1_2_innodb.test sp1f-partition_alter1_2_i-20080513231047-g6yx4e6tufcbky2wq463xgngz66cuqnu mysql-test/suite/parts/t/partition_alter2_1_innodb.test sp1f-partition_alter2_inn-20070206122238-btax2l7djjymz6hlnkmuvwdt2nbqbxed mysql-test/suite/parts/t/partition_alter2_2_innodb.test partition_alter2_2_i-20080908140941-884mge0s10lxgki2-3 mysql-test/suite/parts/t/partition_alter4_innodb.test sp1f-partition_alter4_inn-20070206122238-wzdnxx4co5gutrm6mnbees2dkz7bwsii mysql-test/suite/parts/t/partition_recover_myisam.test partition_repair_myi-20080609121315-mjya2e9ekn7bunzm-2 mysql-test/suite/pbxt/r/func_group.result func_group.result-20090402100035-4ilk9i91sh65vjcb-50 mysql-test/suite/pbxt/r/join_nested.result join_nested.result-20090402100035-4ilk9i91sh65vjcb-81 mysql-test/suite/pbxt/r/lock_multi.result lock_multi.result-20090402100035-4ilk9i91sh65vjcb-90 mysql-test/suite/pbxt/r/mysqlshow.result mysqlshow.result-20090402100035-4ilk9i91sh65vjcb-101 mysql-test/suite/pbxt/r/pbxt_bugs.result pbxt_bugs.result-20090402100035-4ilk9i91sh65vjcb-120 mysql-test/suite/pbxt/t/join_nested.test join_nested.test-20090402100035-4ilk9i91sh65vjcb-248 mysql-test/suite/pbxt/t/lock_multi.test lock_multi.test-20090402100035-4ilk9i91sh65vjcb-257 mysql-test/suite/pbxt/t/pbxt_bugs.test pbxt_bugs.test-20090402100035-4ilk9i91sh65vjcb-287 mysql-test/suite/pbxt/t/pbxt_locking.test pbxt_locking.test-20090402100035-4ilk9i91sh65vjcb-288 mysql-test/suite/pbxt/t/pbxt_transactions.test pbxt_transactions.te-20090402100035-4ilk9i91sh65vjcb-291 mysql-test/suite/pbxt/t/ps_1general.test ps_1general.test-20090402100035-4ilk9i91sh65vjcb-297 mysql-test/suite/rpl/r/rpl_err_ignoredtable.result sp1f-rpl_error_ignored_ta-20030708095945-wtbf3wai2clqedywdvwntfdfwmloumec mysql-test/suite/rpl/r/rpl_extraCol_innodb.result sp1f-rpl_extracol_innodb.-20061103140439-oyaqsdcl3ymjfl5y2wvwjz3cgb36dbj3 mysql-test/suite/rpl/r/rpl_extraCol_myisam.result sp1f-rpl_extracol_myisam.-20061103140439-ipxcnvlavhkichgny6fvkejbdgnvudtd mysql-test/suite/rpl/r/rpl_get_lock.result sp1f-rpl_get_lock.result-20010831215549-yggpnug7jmur7hftd3ln6ytpungqckpy mysql-test/suite/rpl/r/rpl_mixed_ddl_dml.result sp1f-rpl000002.result-20001118063528-dp4vigctbaz5p7s7r7cqtgabk25a5j3m mysql-test/suite/rpl/r/rpl_relayspace.result sp1f-rpl_relayspace.resul-20030317215153-kx422hojs2xkiqciwgt7jps2hdk376fb mysql-test/suite/rpl/r/rpl_row_create_table.result sp1f-rpl_row_create_table-20051222053452-uud3ktz3erqptqb64rkh7ftoo7bdbf6c mysql-test/suite/rpl/r/rpl_row_tabledefs_2myisam.result sp1f-rpl_row_tabledefs.re-20051222053457-perubbeq3fwsqe5phfwcpsstjqltqrnj mysql-test/suite/rpl/r/rpl_row_tabledefs_3innodb.result sp1f-rpl_row_tabledefs_3i-20060508180502-wvvscuvjv34fiuhqjuhi6yvk6cwbmrpz mysql-test/suite/rpl/r/rpl_stm_000001.result sp1f-rpl000001.result-20010116163624-seoa5zygxq5ibscm6kld7cneoimbmer4 mysql-test/suite/rpl/r/rpl_temporary.result sp1f-rpl_temporary.result-20021229214238-uwyas6jaay7ygaqsdwolzlcec6reyckw mysql-test/suite/rpl/r/rpl_trigger.result sp1f-rpl_trigger.result-20050815151505-durmghxr6fqgzgolaezf6gfotoeptwhz mysql-test/suite/rpl/t/disabled.def sp1f-disabled.def-20070627122758-vdqevwzhnizicdrxrmfy4w4afgprx46x mysql-test/suite/rpl/t/rpl_err_ignoredtable.test sp1f-rpl_error_ignored_ta-20030708095933-nrriw3pbfsfrugbgvjpriczjb3dwm4mn mysql-test/suite/rpl/t/rpl_get_lock.test sp1f-rpl_get_lock.test-20010831215549-mxwhygd7dfgxcx4dnnjjomunk7oojsj5 mysql-test/suite/rpl/t/rpl_mixed_ddl_dml.test sp1f-rpl000002.test-20001118063528-xtihamqla2qxwkn544mamd5mlt5pev33 mysql-test/suite/rpl/t/rpl_relayspace.test sp1f-rpl_relayspace.test-20030317215153-aincxws3k2fb4ojvtowjzgandznovi7b mysql-test/suite/rpl/t/rpl_rotate_logs.test sp1f-rpl000016.test-20001215004309-uqid5ejphbyjwielf3t6nd7523ynp353 mysql-test/suite/rpl/t/rpl_row_create_table.test sp1f-rpl_row_create_table-20051222053501-6a6pxustyjnj6swfd4bbsptzllxov5ao mysql-test/suite/rpl/t/rpl_temporary.test sp1f-rpl_temporary.test-20021229214239-nxqbr5fvrk5sm3d5xf2gwzhfim3xdy7k mysql-test/suite/rpl/t/rpl_trigger.test sp1f-rpl_trigger.test-20050815151506-5wtqt7aazfxwebrvi37du2prbglo4lmq mysql-test/suite/rpl_ndb/r/rpl_ndb_extraCol.result sp1f-rpl_ndb_extracol.res-20061103140449-ucihyswq7mtsamyjm2whggyjxyfekxeo mysql-test/t/archive.test sp1f-archive.test-20040525194738-qla5yawytktcj3tlbgrlhvf3thbo6ghq mysql-test/t/delayed.test sp1f-delayed.test-20001228015635-nfs6w3ic7qt55pnm6uld7wmmq4p73afq mysql-test/t/delete.test sp1f-delete.test-20001228015635-7lhk263y3s3wild7htgoaesssx5wdy4s mysql-test/t/disabled.def sp1f-disabled.def-20050315184020-inpdp4hiogithilv62snllppjz2dcing mysql-test/t/fulltext.test sp1f-fulltext.test-20001228015635-snfzkwn2snrqit5pagdg2vzhcoa56eea mysql-test/t/func_group.test sp1f-func_group.test-20001228015635-wkz277djccbddkitm63hibutxp7o4rb7 mysql-test/t/func_misc.test sp1f-func_misc.test-20001228015635-kayguwcdgtjnekzavvdzbsnqcdwfm36c mysql-test/t/grant.test sp1f-grant.test-20020905131705-iadu5zcjshnxgtjx7qpmfrs77bl75suy mysql-test/t/grant2.test sp1f-grant2.test-20030722200048-galnas2hib5h2ygo4rzcnpblby7awdow mysql-test/t/group_min_max.test sp1f-group_min_max.test-20040827133612-bbe7hj6l7byvtyxsg4iicylzflsgy6vj mysql-test/t/innodb-analyze.test innodbanalyze.test-20081203050234-edoolglm28lyejuc-6 mysql-test/t/innodb-autoinc.test innodbautoinc.test-20081201061010-zymrrwrczns2vrex-281 mysql-test/t/innodb-index.test innodbindex.test-20081201061010-zymrrwrczns2vrex-285 mysql-test/t/innodb-master.opt innodbmaster.opt-20081201061010-zymrrwrczns2vrex-290 mysql-test/t/innodb-semi-consistent-master.opt innodbsemiconsistent-20081201061010-zymrrwrczns2vrex-293 mysql-test/t/innodb-timeout.test innodbtimeout.test-20081203050234-edoolglm28lyejuc-8 mysql-test/t/innodb-use-sys-malloc-master.opt innodbusesysmallocma-20090326061054-ylrdb8libxw6u7e9-3 mysql-test/t/innodb-zip.test innodbzip.test-20081201061010-zymrrwrczns2vrex-297 mysql-test/t/innodb.test innodb.test-20081201061010-zymrrwrczns2vrex-299 mysql-test/t/innodb_bug34300.test innodb_bug34300.test-20081201061010-zymrrwrczns2vrex-303 mysql-test/t/innodb_bug36169.test innodb_bug36169.test-20081201061010-zymrrwrczns2vrex-307 mysql-test/t/innodb_bug36172.test innodb_bug36172.test-20081203050234-edoolglm28lyejuc-10 mysql-test/t/innodb_bug42101-nonzero-master.opt innodb_bug42101nonze-20090519075917-c0hbhca1f80pmx80-4 mysql-test/t/innodb_bug44369.test innodb_bug44369.test-20091005111405-nbp5t33h95jrqha2-2 mysql-test/t/innodb_file_format.test innodb_file_format.t-20090923000535-ke95wdd4zn27df71-19 mysql-test/t/innodb_information_schema.test innodb_information_s-20081201061010-zymrrwrczns2vrex-309 mysql-test/t/innodb_lock_wait_timeout_1.test bug40113.test-20090619150423-w3im08cym6tyzn8f-2 mysql-test/t/innodb_mysql.test sp1f-innodb_mysql.test-20060816102624-6ymo37d3nyhvbqyzqn5ohsfuydwo426k mysql-test/t/innodb_xtradb_bug317074.test innodb_xtradb_bug317-20090326061054-ylrdb8libxw6u7e9-9 mysql-test/t/mysql.test sp1f-mysql.test-20050517191330-gc7zxd3q7cgw4g3pdnswpnxwqnvqhwks mysql-test/t/mysqltest.test sp1f-mysqltest.test-20041022024800-v3hvkzs4236l6rpunai7xttdltot7rvz mysql-test/t/olap.test sp1f-olap.test-20020720115151-u3y5qjyyz4c7hufu5vftj74rijkr7rf2 mysql-test/t/openssl_1.test sp1f-ssl.test-20010831211355-mk47pipvythsqcor32yidzoopgdewdo6 mysql-test/t/order_by.test sp1f-order_by.test-20001228015636-nr7aml75ra7mdlruhoqo5dgbfv5tcesc mysql-test/t/partition.test sp1f-partition.test-20050718113034-pbo3ht3bf4gfa3mz44on3sqafyctwo35 mysql-test/t/partition_innodb_semi_consistent.test partition_innodb_sem-20081216114001-2cqkultf4k3xhbvc-2 mysql-test/t/query_cache.test sp1f-query_cache.test-20011205230530-yfwho76ujeasygr3magwlmssnvwsukio mysql-test/t/query_cache_debug.test sp1f-query_cache_debug.te-20080107200614-idvgytisf3mqftabyk43v42cynhijq5h mysql-test/t/query_cache_notembedded.test sp1f-query_cache_notembed-20050729121335-367lhbc36drodp262lkuott3pk25wcdt mysql-test/t/range.test sp1f-range.test-20001228015636-xfak6bsaw5p3ek36np7bznadjb3boh2q mysql-test/t/select.test sp1f-select.test-20010103001548-tbl2ff7qehzh43qnsmf4ejhjqe66f46n mysql-test/t/show_check-master.opt sp1f-show_checkmaster.opt-20061004031826-m2pj2wv7l6njctrnpaenfdqxhckyfxpz mysql-test/t/show_check.test sp1f-show_check.test-20001228015637-uv35wm2ryvpkyrr6ojhmi2nq6x6jgdod mysql-test/t/sp-destruct.test sp1f-spdestruct.test-20051026133448-bt2vu42upsulrlap6vytgz7hygx6a2hj mysql-test/t/sp-security.test sp1f-spsecurity.test-20031213154048-snbqkvepvo4c45wtxld2qrc3h35ap4ty mysql-test/t/sp.test sp1f-sp.test-20030117133803-b6pcfv2yscbqkur5fszep7acmdg7nf5k mysql-test/t/sp_notembedded.test sp1f-sp_notembedded.test-20060224163411-4bxzhibgkpu3fm3zoyvknrqo3zudvvfa mysql-test/t/trigger.test sp1f-trigger.test-20040907122911-eamsjnplirl554ohkncdnwi765xm2hbk mysql-test/t/trigger_notembedded.test sp1f-triggergrant.test-20051110192456-j6hwzoi4loitpk57ccqotlhkzrm6ucsv mysql-test/t/type_newdecimal.test sp1f-type_newdecimal.test-20050208224936-e244l5ugrk3oditjqp53n6izptkrteq2 mysql-test/t/type_year.test sp1f-type_year.test-20001228015637-j547qmpytndiwdwgn35oq34jgjduzo6l mysql-test/t/udf.test sp1f-udf.test-20060215161120-inrv7ph3327gnzcvcqk25vmihneybyhk mysql-test/t/variables.test sp1f-variables.test-20001228015637-u4toadkin7aellpwwz75e5h5zuutteid mysql-test/valgrind.supp sp1f-valgrind.supp-20050406142216-yg7xhezklqhgqlc3inx36vbghodhbovy mysys/my_file.c sp1f-my_file.c-20040219173304-xky3nl63gme3w2apldzfpufwgcfnq23x mysys/my_getopt.c sp1f-my_getopt.c-20020125212008-5ppwsdqmfhny46gxkjxph22zh3phetir mysys/my_sync.c sp1f-my_sync.c-20031102135456-o4s6sunug6w2ch4bok2p3auq37qgqzox mysys/my_thr_init.c sp1f-my_thr_init.c-19700101030959-siwg2eavxsdwdc4kkmwxvs42rp6ntkrm scripts/make_win_bin_dist sp1f-make_win_bin_dist-20060901123056-xnusgszvkfrrcxkqidb7zszax2ezpyto scripts/mysql_secure_installation.pl.in sp1f-mysql_secure_install-20071228215050-nnco3kgp33fxs7ja6zdy6xh56zszi2cc scripts/mysql_secure_installation.sh sp1f-mysql_secure_install-20020609041239-b4zztmtqycjs24aubuhwun6np5wuuesy sql-common/client.c sp1f-client.c-20030502160736-oraaciqy6jkfwygs6tqfoaxgjbi65yo7 sql/CMakeLists.txt sp1f-cmakelists.txt-20060831175237-esoeu5kpdtwjvehkghwy6fzbleniq2wy sql/event_db_repository.cc sp1f-event_db_repository.-20060627064838-k6rpjg72omnihtxhbubu6ht7wjvxggb7 sql/field.cc sp1f-field.cc-19700101030959-f4imaofclsea3n4fj4ow5m7havmyxa2r sql/field.h sp1f-field.h-19700101030959-3n6smzxcwkjl7bikm3wg4hfkjn66uvvp sql/ha_ndbcluster.cc sp1f-ha_ndbcluster.cc-20040414175836-rvqnoxrkqexyhfu3d62s4t345ip7rez2 sql/handler.h sp1f-handler.h-19700101030959-mumq2hpilkpgxuf22ftyv5kbilysnzvn sql/item.cc sp1f-item.cc-19700101030959-u7hxqopwpfly4kf5ctlyk2dvrq4l3dhn sql/item.h sp1f-item.h-19700101030959-rrkb43htudd62batmoteashkebcwykpa sql/item_cmpfunc.cc sp1f-item_cmpfunc.cc-19700101030959-hrk7pi2n6qpwxauufnkizirsoucdcx2e sql/item_cmpfunc.h sp1f-item_cmpfunc.h-19700101030959-pcvbjplo4e4ng7ibynfhcd6pjyem57gr sql/item_create.cc sp1f-item_create.cc-19700101030959-zdsezbi5r5xu5syntjdzqs2d2dswsojn sql/item_func.cc sp1f-item_func.cc-19700101030959-3wmsx76yvc25sroqpfrx2n77kqdxxn3y sql/item_func.h sp1f-item_func.h-19700101030959-fbjcbwkg66qubbzptqwh5w5evhnpukze sql/item_geofunc.cc sp1f-item_geofunc.cc-20030530102226-vdbf2bd6tpkrzoy6q2wdibkzd3bkv2io sql/item_strfunc.cc sp1f-item_strfunc.cc-19700101030959-yl2pwnrngmla3nmlgiuiwrztx3iu4ffl sql/item_subselect.cc sp1f-item_subselect.cc-20020512204640-qep43aqhsfrwkqmrobni6czc3fqj36oo sql/item_subselect.h sp1f-item_subselect.h-20020512204640-qdg77wil56cxyhtc2bjjdrppxq3wqgh3 sql/item_sum.cc sp1f-item_sum.cc-19700101030959-4woo23bi3am2t2zvsddqbpxk7xbttdkm sql/item_sum.h sp1f-item_sum.h-19700101030959-ecgohlekwm355wxl5fv4zzq3alalbwyl sql/item_timefunc.cc sp1f-item_timefunc.cc-19700101030959-rvvlgmw5b4ewpuuxuntrkiqimyrr5sw2 sql/item_xmlfunc.cc sp1f-item_xmlfunc.cc-20051221130500-wo5dgojvjjm6mmra7fay3ri7ud5ow3yl sql/log.cc sp1f-log.cc-19700101030959-r3hdfovek4kl6nd64ovoaknmirota6bq sql/log_event.cc sp1f-log_event.cc-19700101030959-msmqlflsngxosswid2hpzxly5vfqdddc sql/log_event.h sp1f-log_event.h-19700101030959-clq6ett55tcqbpys2i4cpfrdccq7j4om sql/mysqld.cc sp1f-mysqld.cc-19700101030959-zpswdvekpvixxzxf7gdtofzel7nywtfj sql/net_serv.cc sp1f-net_serv.cc-19700101030959-dp4s27g5nk64sph4g6g54dghekqozzmy sql/opt_range.cc sp1f-opt_range.cc-19700101030959-afe3wtevb7zwrg4xyibt35uamov5r7ds sql/repl_failsafe.cc sp1f-repl_failsafe.cc-20011010025623-k7zhoyc3smc7tbliyp7vaf3f4idq22so sql/rpl_record.cc sp1f-rpl_record.cc-20070413125523-wuthuk5jk7uxikuioz6esll6xakdngs4 sql/rpl_record.h sp1f-rpl_record.h-20070413125523-xvn32ub2xcvqged7y6ayilghjetpvkvg sql/rpl_rli.cc sp1f-rpl_rli.cc-20061031112305-25t7pxjrjm24qo5h65c7rml66xu3uw4p sql/rpl_tblmap.cc sp1f-rpl_tblmap.cc-20051222053448-sgowtys7fb4tdpjvmzwktjxmb5krm3cc sql/set_var.cc sp1f-set_var.cc-20020723153119-nwbpg2pwpz55pfw7yfzaxt7hsszzy7y3 sql/slave.cc sp1f-slave.cc-19700101030959-a636aj3mjxgu7fnznrg5kt77p3u2bvhh sql/sp.cc sp1f-sp.cc-20021212121421-6xwuvxq5bku2b4yv655kp2e5gsvautd5 sql/sp.h sp1f-sp.h-20021212121421-eh5y7kpcb3hkgy4wjuh3q3non36itye5 sql/sp_cache.cc sp1f-sp_cache.cc-20030703140129-ugsn54s2jpxh7hdznsgxn6ubwvbtj5hw sql/sp_head.cc sp1f-sp_head.cc-20021208185920-jtgc5wvyqdnu2gvcdus3gazrfhxbofxd sql/sp_head.h sp1f-sp_head.h-20021208185920-yrolg3rzamehfoejkbiai4q7njg5w6cd sql/sp_rcontext.cc sp1f-sp_rcontext.cc-20030916122605-vg62h2qkjnbj54b4zlijgqswudijiyaf sql/sql_acl.cc sp1f-sql_acl.cc-19700101030959-c4hku3uqxzujthqnndeprbrhamqy6a4i sql/sql_acl.h sp1f-sql_acl.h-19700101030959-byf4bn7yfbxu6wa6z76kqcuspjl67msj sql/sql_base.cc sp1f-sql_base.cc-19700101030959-w7tul2gb2n4jzayjwlslj3ybmf3uhk6a sql/sql_cache.cc sp1f-sql_cache.cc-19700101030959-74bsqwcnhboovijsogcenqana5inu6wo sql/sql_cache.h sp1f-sql_cache.h-20011202123401-gegktsz2a3er7fqwpmpoejydzpkeadeo sql/sql_class.cc sp1f-sql_class.cc-19700101030959-rpotnweaff2pikkozh3butrf7mv3oero sql/sql_connect.cc sp1f-sql_connect.cc-20070223111352-fhh5znxdfvzxuca7da3uu4olnwgkrm4n sql/sql_delete.cc sp1f-sql_delete.cc-19700101030959-ch2a6r6ushvc2vfwxt7ehcjuplelwthr sql/sql_insert.cc sp1f-sql_insert.cc-19700101030959-xgwqe5svnimxudzdcuitauljzz2zjk5g sql/sql_load.cc sp1f-sql_load.cc-19700101030959-hoqlay5we4yslrw23xqedulkejw6a3o5 sql/sql_parse.cc sp1f-sql_parse.cc-19700101030959-ehcre3rwhv5l3mlxqhaxg36ujenxnrcd sql/sql_partition.cc sp1f-sql_partition.cc-20050718113038-57h5bzswps6cel2y7k7qideue3ghbg3u sql/sql_plugin.cc sp1f-sql_plugin.cc-20051105112032-hrm64p6xfjq33ud6zy3uivpo7azm75a2 sql/sql_profile.cc sp1f-sql_profile.cc-20070222150305-yv5grcusm3k2b6rrcx3kkqggtm33i3z4 sql/sql_profile.h sp1f-sql_profile.h-20070222150305-tbdpljkisvi3e657yunhkvsnjw6ifjru sql/sql_select.cc sp1f-sql_select.cc-19700101030959-egb7whpkh76zzvikycs5nsnuviu4fdlb sql/sql_show.cc sp1f-sql_show.cc-19700101030959-umlljfnpplg452h7reeyqr4xnbmlkvfj sql/sql_table.cc sp1f-sql_table.cc-19700101030959-tzdkvgigezpuaxnldqh3fx2h7h2ggslu sql/sql_yacc.yy sp1f-sql_yacc.yy-19700101030959-wvn4qyy2drpmge7kaq3dysprbhlrv27j sql/table.cc sp1f-table.cc-19700101030959-nsxtem2adyqzwe6nz4cgrpcmts3o54v7 sql/table.h sp1f-table.h-19700101030959-dv72bajftxj5fbdjuajquappanuv2ija sql/udf_example.def sp1f-udf_example.def-20060922124240-fxmt4egcapnpdlbg5u4xlrq4bppsjcnw storage/archive/CMakeLists.txt* sp1f-cmakelists.txt-20060324221904-eeglvbzhfjgzaeenavdxsdvjzfjatvre storage/archive/azio.c sp1f-azio.c-20051223034959-gjinfrr75cpes7iyw4uasbps2bkn65rc storage/archive/azlib.h sp1f-azlib.h-20051223035000-23fs3st2s6whj3xl4hjzfxuzsjyzgnf6 storage/archive/ha_archive.cc sp1f-ha_archive.cc-20040521001938-uy57z43drkjeirpjafdzdpvfxruqho4q storage/federated/CMakeLists.txt* sp1f-cmakelists.txt-20060819155453-lvbabhbwyj6n6qzjroyxasefis7wczs5 storage/innobase/btr/btr0btr.c sp1f-btr0btr.c-20010217121858-mwrlor7vioqhw732pmqhbjjhpajttjyp storage/innobase/data/data0type.c sp1f-data0type.c-20010217121859-zfpknsetjkfoyjnhomi2z2ek3jda2wji storage/innobase/handler/ha_innodb.cc sp1f-ha_innobase.cc-20001205235417-rlet3ei56gdrss673dssnrqgug67lwno storage/innobase/include/ha_prototypes.h sp1f-ha_prototypes.h-20060601063356-7fsbop6gnnzwnq7pndelgmztdm2uygg2 storage/innobase/include/mach0data.h sp1f-mach0data.h-20010217121903-66grfy3b6skgudvdli4owdshxnvcjo2o storage/innobase/include/mach0data.ic sp1f-mach0data.ic-20010217121903-n2kjcumh2n3j5ydrwt5cllgns7c4kpyw storage/innobase/include/os0file.h sp1f-os0file.h-20010217121904-5jy6i4kwdivoji2tzlssz6tj3hmntl4c storage/innobase/include/trx0trx.h sp1f-trx0trx.h-20010217121908-in6xvc2qnxbrz74ytwthpviclfo53zls storage/innobase/lock/lock0lock.c sp1f-lock0lock.c-20010217121909-slawfq55wuitbo2ujz7h5doebhwxfmhe storage/innobase/os/os0file.c sp1f-os0file.c-20010217121911-3wbmvsjzaw7z42arqr46leh46lbamvcp storage/innobase/row/row0sel.c sp1f-row0sel.c-20010217121914-c6o7vqncdgzrorm4pko5tpdlfeyujhvq storage/innobase/trx/trx0trx.c sp1f-trx0trx.c-20010217121916-b5g7pmqxezfo2mktfpeiapibgdhogrv2 storage/innobase/ut/ut0ut.c sp1f-ut0ut.c-20010217121917-7nqugyaqeu7yecvgv73lm354njkx74zq storage/innodb_plugin/CMakeLists.txt cmakelists.txt-20090527093836-7v4wb2xxka10h4d0-2 storage/innodb_plugin/ChangeLog changelog-20090527093836-7v4wb2xxka10h4d0-4 storage/innodb_plugin/btr/btr0btr.c btr0btr.c-20090527093836-7v4wb2xxka10h4d0-46 storage/innodb_plugin/btr/btr0sea.c btr0sea.c-20090527093836-7v4wb2xxka10h4d0-49 storage/innodb_plugin/buf/buf0buf.c buf0buf.c-20090527093836-7v4wb2xxka10h4d0-51 storage/innodb_plugin/data/data0type.c data0type.c-20090527093836-7v4wb2xxka10h4d0-56 storage/innodb_plugin/dict/dict0dict.c dict0dict.c-20090527093836-7v4wb2xxka10h4d0-59 storage/innodb_plugin/fil/fil0fil.c fil0fil.c-20090527093836-7v4wb2xxka10h4d0-65 storage/innodb_plugin/handler/ha_innodb.cc ha_innodb.cc-20090527093836-7v4wb2xxka10h4d0-72 storage/innodb_plugin/handler/ha_innodb.h ha_innodb.h-20090527093836-7v4wb2xxka10h4d0-73 storage/innodb_plugin/handler/handler0alter.cc handler0alter.cc-20090527093836-7v4wb2xxka10h4d0-74 storage/innodb_plugin/ibuf/ibuf0ibuf.c ibuf0ibuf.c-20090527093836-7v4wb2xxka10h4d0-80 storage/innodb_plugin/include/btr0sea.h btr0sea.h-20090527093836-7v4wb2xxka10h4d0-87 storage/innodb_plugin/include/db0err.h db0err.h-20090527093836-7v4wb2xxka10h4d0-105 storage/innodb_plugin/include/dict0dict.h dict0dict.h-20090527093836-7v4wb2xxka10h4d0-110 storage/innodb_plugin/include/fil0fil.h fil0fil.h-20090527093836-7v4wb2xxka10h4d0-123 storage/innodb_plugin/include/ha_prototypes.h ha_prototypes.h-20090527093836-7v4wb2xxka10h4d0-134 storage/innodb_plugin/include/ibuf0ibuf.h ibuf0ibuf.h-20090527093836-7v4wb2xxka10h4d0-138 storage/innodb_plugin/include/lock0lock.h lock0lock.h-20090527093836-7v4wb2xxka10h4d0-142 storage/innodb_plugin/include/log0log.h log0log.h-20090527093836-7v4wb2xxka10h4d0-147 storage/innodb_plugin/include/log0recv.h log0recv.h-20090527093836-7v4wb2xxka10h4d0-149 storage/innodb_plugin/include/mem0mem.h mem0mem.h-20090527093836-7v4wb2xxka10h4d0-155 storage/innodb_plugin/include/mem0pool.h mem0pool.h-20090527093836-7v4wb2xxka10h4d0-157 storage/innodb_plugin/include/os0file.h os0file.h-20090527093836-7v4wb2xxka10h4d0-165 storage/innodb_plugin/include/pars0pars.h pars0pars.h-20090527093836-7v4wb2xxka10h4d0-182 storage/innodb_plugin/include/srv0srv.h srv0srv.h-20090527093836-7v4wb2xxka10h4d0-223 storage/innodb_plugin/include/thr0loc.h thr0loc.h-20090527093836-7v4wb2xxka10h4d0-233 storage/innodb_plugin/include/trx0i_s.h trx0i_s.h-20090527093836-7v4wb2xxka10h4d0-235 storage/innodb_plugin/include/trx0purge.h trx0purge.h-20090527093836-7v4wb2xxka10h4d0-236 storage/innodb_plugin/include/trx0rseg.h trx0rseg.h-20090527093836-7v4wb2xxka10h4d0-242 storage/innodb_plugin/include/trx0sys.h trx0sys.h-20090527093836-7v4wb2xxka10h4d0-244 storage/innodb_plugin/include/trx0trx.h trx0trx.h-20090527093836-7v4wb2xxka10h4d0-246 storage/innodb_plugin/include/trx0undo.h trx0undo.h-20090527093836-7v4wb2xxka10h4d0-249 storage/innodb_plugin/include/univ.i univ.i-20090527093836-7v4wb2xxka10h4d0-252 storage/innodb_plugin/include/usr0sess.h usr0sess.h-20090527093836-7v4wb2xxka10h4d0-253 storage/innodb_plugin/lock/lock0lock.c lock0lock.c-20090527093836-7v4wb2xxka10h4d0-274 storage/innodb_plugin/log/log0log.c log0log.c-20090527093836-7v4wb2xxka10h4d0-275 storage/innodb_plugin/log/log0recv.c log0recv.c-20090527093836-7v4wb2xxka10h4d0-276 storage/innodb_plugin/mem/mem0dbg.c mem0dbg.c-20090527093836-7v4wb2xxka10h4d0-278 storage/innodb_plugin/mem/mem0pool.c mem0pool.c-20090527093836-7v4wb2xxka10h4d0-280 storage/innodb_plugin/os/os0file.c os0file.c-20090527093836-7v4wb2xxka10h4d0-338 storage/innodb_plugin/os/os0sync.c os0sync.c-20090527093836-7v4wb2xxka10h4d0-340 storage/innodb_plugin/os/os0thread.c os0thread.c-20090527093836-7v4wb2xxka10h4d0-341 storage/innodb_plugin/pars/lexyy.c lexyy.c-20090527093836-7v4wb2xxka10h4d0-345 storage/innodb_plugin/pars/pars0lex.l pars0lex.l-20090527093836-7v4wb2xxka10h4d0-350 storage/innodb_plugin/que/que0que.c que0que.c-20090527093836-7v4wb2xxka10h4d0-354 storage/innodb_plugin/row/row0merge.c row0merge.c-20090527093836-7v4wb2xxka10h4d0-360 storage/innodb_plugin/row/row0mysql.c row0mysql.c-20090527093836-7v4wb2xxka10h4d0-361 storage/innodb_plugin/srv/srv0srv.c srv0srv.c-20090527093836-7v4wb2xxka10h4d0-373 storage/innodb_plugin/srv/srv0start.c srv0start.c-20090527093836-7v4wb2xxka10h4d0-374 storage/innodb_plugin/sync/sync0arr.c sync0arr.c-20090527093836-7v4wb2xxka10h4d0-375 storage/innodb_plugin/sync/sync0sync.c sync0sync.c-20090527093836-7v4wb2xxka10h4d0-377 storage/innodb_plugin/thr/thr0loc.c thr0loc.c-20090527093836-7v4wb2xxka10h4d0-378 storage/innodb_plugin/trx/trx0i_s.c trx0i_s.c-20090527093836-7v4wb2xxka10h4d0-379 storage/innodb_plugin/trx/trx0purge.c trx0purge.c-20090527093836-7v4wb2xxka10h4d0-380 storage/innodb_plugin/trx/trx0rseg.c trx0rseg.c-20090527093836-7v4wb2xxka10h4d0-383 storage/innodb_plugin/trx/trx0sys.c trx0sys.c-20090527093836-7v4wb2xxka10h4d0-384 storage/innodb_plugin/trx/trx0trx.c trx0trx.c-20090527093836-7v4wb2xxka10h4d0-385 storage/innodb_plugin/trx/trx0undo.c trx0undo.c-20090527093836-7v4wb2xxka10h4d0-386 storage/innodb_plugin/usr/usr0sess.c usr0sess.c-20090527093836-7v4wb2xxka10h4d0-387 storage/innodb_plugin/ut/ut0mem.c ut0mem.c-20090527093836-7v4wb2xxka10h4d0-395 storage/maria/ft_maria.c sp1f-ft_maria.c-20060411134407-c7zixlxmx36vm37l35blmgkjeq5e2zgv storage/maria/ma_blockrec.c sp1f-ma_blockrec.c-20070118193810-5wtbfa4irhu4voa3diiuus5km2j6jvlv storage/maria/ma_check.c sp1f-ma_check.c-20060411134408-m5d5jao4sr32xsjjkig2uhdndqm5cgba storage/maria/ma_close.c sp1f-ma_close.c-20060411134409-5c3eq7j6oloex4c4hrvcqrsuvz7xohev storage/maria/ma_create.c sp1f-ma_create.c-20060411134410-ozzigempkjj2kdgxfbasiwfjzwjejevd storage/maria/ma_loghandler.c sp1f-ma_loghandler.c-20070202074129-utpzp3km4lrxldm2tdhejae2zy6zlmhq storage/maria/ma_test3.c sp1f-ma_test3.c-20060411134447-llbsdlhu2zyxbt6taoa2lsts7snaic2j storage/myisam/ft_boolean_search.c sp1f-ft_boolean_search.c-20010411110351-pu6lfsyiumvnnewko2oqbyjz6g3q4xm3 storage/myisam/ft_myisam.c sp1f-ft_myisam.c-20060411134458-uct2l3bly6nrej2hilx4explav2i3kgv storage/myisam/ft_nlq_search.c sp1f-ft_nlq_search.c-20010411110351-a7dhoojgfpsydi5k4qawswaatmakqe7b storage/myisam/ft_parser.c sp1f-ft_parser.c-19700101030959-goim35zn24ujo7rbznobwhhw5r3lemab storage/myisam/ft_stopwords.c sp1f-ft_stopwords.c-19700101030959-vgask5ebyzpaoa7j37ybfnjhx4rkzm63 storage/myisam/ftdefs.h sp1f-ftdefs.h-19700101030959-c5sgpgnpbutzv5fvbe6a63x6up2niz2p storage/myisam/mi_check.c sp1f-mi_check.c-19700101030959-yzbhnjgzcmqdyj4zz5codhkkw5eedp6f storage/myisam/mi_create.c sp1f-mi_create.c-19700101030959-i6lazhpsyf7ggr2yjukf6xxybhraxup3 storage/myisam/mi_test3.c sp1f-mi_test3.c-19700101030959-3yn4wc53noyuhc4nphffr4hxesodgfyf storage/pbxt/ChangeLog changelog-20090326121724-x683v32twzr3fi0y-3 storage/pbxt/plug.in plug.in-20090326121724-x683v32twzr3fi0y-9 storage/pbxt/src/Makefile.am makefile.am-20090326121724-x683v32twzr3fi0y-13 storage/pbxt/src/discover_xt.cc discover_xt.cc-20090326121724-x683v32twzr3fi0y-26 storage/pbxt/src/ha_pbxt.cc ha_pbxt.cc-20090326121724-x683v32twzr3fi0y-30 storage/pbxt/src/restart_xt.cc restart_xt.cc-20090326121724-x683v32twzr3fi0y-53 storage/pbxt/src/strutil_xt.cc strutil_xt.cc-20090326121724-x683v32twzr3fi0y-59 storage/pbxt/src/table_xt.cc table_xt.cc-20090326121724-x683v32twzr3fi0y-65 storage/pbxt/src/thread_xt.cc thread_xt.cc-20090326121724-x683v32twzr3fi0y-67 storage/pbxt/src/thread_xt.h thread_xt.h-20090326121724-x683v32twzr3fi0y-68 storage/pbxt/src/trace_xt.cc trace_xt.cc-20090326121724-x683v32twzr3fi0y-69 storage/xtradb/CMakeLists.txt cmakelists.txt-20081201061010-zymrrwrczns2vrex-1 storage/xtradb/ChangeLog changelog-20081201061010-zymrrwrczns2vrex-3 storage/xtradb/Makefile.am makefile.am-20081201061010-zymrrwrczns2vrex-4 storage/xtradb/btr/btr0btr.c btr0btr.c-20081201061010-zymrrwrczns2vrex-44 storage/xtradb/btr/btr0sea.c btr0sea.c-20081201061010-zymrrwrczns2vrex-47 storage/xtradb/buf/buf0buddy.c buf0buddy.c-20081201061010-zymrrwrczns2vrex-48 storage/xtradb/buf/buf0buf.c buf0buf.c-20081201061010-zymrrwrczns2vrex-49 storage/xtradb/buf/buf0flu.c buf0flu.c-20081201061010-zymrrwrczns2vrex-50 storage/xtradb/buf/buf0lru.c buf0lru.c-20081201061010-zymrrwrczns2vrex-51 storage/xtradb/buf/buf0rea.c buf0rea.c-20081201061010-zymrrwrczns2vrex-52 storage/xtradb/data/data0type.c data0type.c-20081201061010-zymrrwrczns2vrex-54 storage/xtradb/dict/dict0crea.c dict0crea.c-20081201061010-zymrrwrczns2vrex-56 storage/xtradb/dict/dict0dict.c dict0dict.c-20081201061010-zymrrwrczns2vrex-57 storage/xtradb/fil/fil0fil.c fil0fil.c-20081201061010-zymrrwrczns2vrex-63 storage/xtradb/fsp/fsp0fsp.c fsp0fsp.c-20081201061010-zymrrwrczns2vrex-64 storage/xtradb/handler/ha_innodb.cc ha_innodb.cc-20081201061010-zymrrwrczns2vrex-70 storage/xtradb/handler/ha_innodb.h ha_innodb.h-20081201061010-zymrrwrczns2vrex-71 storage/xtradb/handler/handler0alter.cc handler0alter.cc-20081201061010-zymrrwrczns2vrex-72 storage/xtradb/handler/i_s.cc i_s.cc-20081201061010-zymrrwrczns2vrex-73 storage/xtradb/handler/i_s.h i_s.h-20081201061010-zymrrwrczns2vrex-74 storage/xtradb/handler/innodb_patch_info.h innodb_patch_info.h-20081206234022-hep6ryfeacyr0572-1 storage/xtradb/ibuf/ibuf0ibuf.c ibuf0ibuf.c-20081201061010-zymrrwrczns2vrex-76 storage/xtradb/include/btr0cur.h btr0cur.h-20081201061010-zymrrwrczns2vrex-79 storage/xtradb/include/btr0sea.h btr0sea.h-20081201061010-zymrrwrczns2vrex-83 storage/xtradb/include/buf0buf.h buf0buf.h-20081201061010-zymrrwrczns2vrex-88 storage/xtradb/include/buf0buf.ic buf0buf.ic-20081201061010-zymrrwrczns2vrex-89 storage/xtradb/include/buf0lru.h buf0lru.h-20081201061010-zymrrwrczns2vrex-92 storage/xtradb/include/buf0rea.h buf0rea.h-20081201061010-zymrrwrczns2vrex-94 storage/xtradb/include/buf0types.h buf0types.h-20081201061010-zymrrwrczns2vrex-95 storage/xtradb/include/db0err.h db0err.h-20081201061010-zymrrwrczns2vrex-101 storage/xtradb/include/dict0crea.h dict0crea.h-20081201061010-zymrrwrczns2vrex-104 storage/xtradb/include/dict0dict.h dict0dict.h-20081201061010-zymrrwrczns2vrex-106 storage/xtradb/include/dict0mem.h dict0mem.h-20081201061010-zymrrwrczns2vrex-110 storage/xtradb/include/fil0fil.h fil0fil.h-20081201061010-zymrrwrczns2vrex-119 storage/xtradb/include/fsp0fsp.h fsp0fsp.h-20081201061010-zymrrwrczns2vrex-120 storage/xtradb/include/ibuf0ibuf.h ibuf0ibuf.h-20081201061010-zymrrwrczns2vrex-134 storage/xtradb/include/lock0lock.h lock0lock.h-20081201061010-zymrrwrczns2vrex-138 storage/xtradb/include/log0log.h log0log.h-20081201061010-zymrrwrczns2vrex-143 storage/xtradb/include/log0log.ic log0log.ic-20081201061010-zymrrwrczns2vrex-144 storage/xtradb/include/log0recv.h log0recv.h-20081201061010-zymrrwrczns2vrex-145 storage/xtradb/include/mem0mem.h mem0mem.h-20081201061010-zymrrwrczns2vrex-151 storage/xtradb/include/mem0pool.h mem0pool.h-20081201061010-zymrrwrczns2vrex-153 storage/xtradb/include/mtr0mtr.h mtr0mtr.h-20081201061010-zymrrwrczns2vrex-157 storage/xtradb/include/os0file.h os0file.h-20081201061010-zymrrwrczns2vrex-161 storage/xtradb/include/os0sync.h os0sync.h-20081201061010-zymrrwrczns2vrex-164 storage/xtradb/include/page0page.h page0page.h-20081201061010-zymrrwrczns2vrex-170 storage/xtradb/include/page0page.ic page0page.ic-20081201061010-zymrrwrczns2vrex-171 storage/xtradb/include/page0zip.h page0zip.h-20081201061010-zymrrwrczns2vrex-173 storage/xtradb/include/pars0pars.h pars0pars.h-20081201061010-zymrrwrczns2vrex-178 storage/xtradb/include/rem0cmp.h rem0cmp.h-20081201061010-zymrrwrczns2vrex-189 storage/xtradb/include/rem0rec.ic rem0rec.ic-20081201061010-zymrrwrczns2vrex-192 storage/xtradb/include/row0ins.h row0ins.h-20081201061010-zymrrwrczns2vrex-196 storage/xtradb/include/row0mysql.h row0mysql.h-20081201061010-zymrrwrczns2vrex-199 storage/xtradb/include/srv0srv.h srv0srv.h-20081201061010-zymrrwrczns2vrex-219 storage/xtradb/include/sync0rw.h sync0rw.h-20081201061010-zymrrwrczns2vrex-224 storage/xtradb/include/sync0sync.h sync0sync.h-20081201061010-zymrrwrczns2vrex-226 storage/xtradb/include/thr0loc.h thr0loc.h-20081201061010-zymrrwrczns2vrex-229 storage/xtradb/include/trx0i_s.h trx0i_s.h-20081201061010-zymrrwrczns2vrex-231 storage/xtradb/include/trx0purge.h trx0purge.h-20081201061010-zymrrwrczns2vrex-232 storage/xtradb/include/trx0rec.h trx0rec.h-20081201061010-zymrrwrczns2vrex-234 storage/xtradb/include/trx0rec.ic trx0rec.ic-20081201061010-zymrrwrczns2vrex-235 storage/xtradb/include/trx0roll.h trx0roll.h-20081201061010-zymrrwrczns2vrex-236 storage/xtradb/include/trx0rseg.h trx0rseg.h-20081201061010-zymrrwrczns2vrex-238 storage/xtradb/include/trx0sys.h trx0sys.h-20081201061010-zymrrwrczns2vrex-240 storage/xtradb/include/trx0sys.ic trx0sys.ic-20081201061010-zymrrwrczns2vrex-241 storage/xtradb/include/trx0trx.h trx0trx.h-20081201061010-zymrrwrczns2vrex-242 storage/xtradb/include/trx0types.h trx0types.h-20081201061010-zymrrwrczns2vrex-244 storage/xtradb/include/trx0undo.h trx0undo.h-20081201061010-zymrrwrczns2vrex-245 storage/xtradb/include/univ.i univ.i-20081201061010-zymrrwrczns2vrex-248 storage/xtradb/include/usr0sess.h usr0sess.h-20081201061010-zymrrwrczns2vrex-249 storage/xtradb/include/ut0auxconf.h ut0auxconf.h-20090326061054-ylrdb8libxw6u7e9-2 storage/xtradb/include/ut0byte.h ut0byte.h-20081201061010-zymrrwrczns2vrex-252 storage/xtradb/include/ut0byte.ic ut0byte.ic-20081201061010-zymrrwrczns2vrex-253 storage/xtradb/include/ut0ut.h ut0ut.h-20081201061010-zymrrwrczns2vrex-263 storage/xtradb/lock/lock0lock.c lock0lock.c-20081201061010-zymrrwrczns2vrex-269 storage/xtradb/log/log0log.c log0log.c-20081201061010-zymrrwrczns2vrex-270 storage/xtradb/log/log0recv.c log0recv.c-20081201061010-zymrrwrczns2vrex-271 storage/xtradb/mem/mem0dbg.c mem0dbg.c-20081201061010-zymrrwrczns2vrex-273 storage/xtradb/mem/mem0mem.c mem0mem.c-20081201061010-zymrrwrczns2vrex-274 storage/xtradb/mem/mem0pool.c mem0pool.c-20081201061010-zymrrwrczns2vrex-275 storage/xtradb/mtr/mtr0mtr.c mtr0mtr.c-20081201061010-zymrrwrczns2vrex-277 storage/xtradb/os/os0file.c os0file.c-20081201061010-zymrrwrczns2vrex-313 storage/xtradb/os/os0proc.c os0proc.c-20081201061010-zymrrwrczns2vrex-314 storage/xtradb/os/os0sync.c os0sync.c-20081201061010-zymrrwrczns2vrex-315 storage/xtradb/os/os0thread.c os0thread.c-20081201061010-zymrrwrczns2vrex-316 storage/xtradb/page/page0cur.c page0cur.c-20081201061010-zymrrwrczns2vrex-317 storage/xtradb/page/page0page.c page0page.c-20081201061010-zymrrwrczns2vrex-318 storage/xtradb/page/page0zip.c page0zip.c-20081201061010-zymrrwrczns2vrex-319 storage/xtradb/pars/lexyy.c lexyy.c-20081201061010-zymrrwrczns2vrex-320 storage/xtradb/pars/pars0lex.l pars0lex.l-20081201061010-zymrrwrczns2vrex-325 storage/xtradb/plug.in plug.in-20081201061010-zymrrwrczns2vrex-31 storage/xtradb/que/que0que.c que0que.c-20081201061010-zymrrwrczns2vrex-329 storage/xtradb/rem/rem0cmp.c rem0cmp.c-20081201061010-zymrrwrczns2vrex-331 storage/xtradb/row/row0ins.c row0ins.c-20081201061010-zymrrwrczns2vrex-334 storage/xtradb/row/row0merge.c row0merge.c-20081201061010-zymrrwrczns2vrex-335 storage/xtradb/row/row0mysql.c row0mysql.c-20081201061010-zymrrwrczns2vrex-336 storage/xtradb/row/row0sel.c row0sel.c-20081201061010-zymrrwrczns2vrex-339 storage/xtradb/scripts/install_innodb_plugins.sql install_innodb_plugi-20081201061010-zymrrwrczns2vrex-345 storage/xtradb/scripts/install_innodb_plugins_win.sql install_innodb_plugi-20081203050234-edoolglm28lyejuc-14 storage/xtradb/srv/srv0srv.c srv0srv.c-20081201061010-zymrrwrczns2vrex-347 storage/xtradb/srv/srv0start.c srv0start.c-20081201061010-zymrrwrczns2vrex-348 storage/xtradb/sync/sync0arr.c sync0arr.c-20081201061010-zymrrwrczns2vrex-349 storage/xtradb/sync/sync0rw.c sync0rw.c-20081201061010-zymrrwrczns2vrex-350 storage/xtradb/sync/sync0sync.c sync0sync.c-20081201061010-zymrrwrczns2vrex-351 storage/xtradb/thr/thr0loc.c thr0loc.c-20081201061010-zymrrwrczns2vrex-352 storage/xtradb/trx/trx0i_s.c trx0i_s.c-20081201061010-zymrrwrczns2vrex-353 storage/xtradb/trx/trx0purge.c trx0purge.c-20081201061010-zymrrwrczns2vrex-354 storage/xtradb/trx/trx0rec.c trx0rec.c-20081201061010-zymrrwrczns2vrex-355 storage/xtradb/trx/trx0roll.c trx0roll.c-20081201061010-zymrrwrczns2vrex-356 storage/xtradb/trx/trx0rseg.c trx0rseg.c-20081201061010-zymrrwrczns2vrex-357 storage/xtradb/trx/trx0sys.c trx0sys.c-20081201061010-zymrrwrczns2vrex-358 storage/xtradb/trx/trx0trx.c trx0trx.c-20081201061010-zymrrwrczns2vrex-359 storage/xtradb/trx/trx0undo.c trx0undo.c-20081201061010-zymrrwrczns2vrex-360 storage/xtradb/usr/usr0sess.c usr0sess.c-20081201061010-zymrrwrczns2vrex-361 storage/xtradb/ut/ut0auxconf_atomic_pthread_t_solaris.c ut0auxconf_atomic_pt-20090923000535-ke95wdd4zn27df71-21 storage/xtradb/ut/ut0mem.c ut0mem.c-20081201061010-zymrrwrczns2vrex-365 storage/xtradb/ut/ut0ut.c ut0ut.c-20081201061010-zymrrwrczns2vrex-367 support-files/compiler_warnings.supp sp1f-disabled_compiler_wa-20070110170439-wzgdkamsch2nrkgvcp2hytmquqeorohi tests/mysql_client_test.c sp1f-client_test.c-20020614002636-eqy2zzksgelocknwbbogfuwxfwqy7q5x unittest/mysys/waiting_threads-t.c waiting_threadst.c-20080623170213-r8baqa2porlpxzq1-5 vio/vio.c sp1f-vio.c-20010520120430-aw76h22ssarmssof7rplhty5elqiexku vio/viosocket.c sp1f-viotcpip.c-20010520120437-u3pbzbt3fdfbclbmusalnzmuqh2y4nav vio/viossl.c sp1f-viossl.c-20010520120431-amywaj3niiokylabjhaly7w33kgdifl6 Diff too large for email (36932 lines, the limit is 1000).

1 0

[Maria-developers] [Branch ~maria-captains/maria/5.1] Rev 2812: Automerge MariaDB 5.1.42 release branch into main.
by noreply＠launchpad.net 31 Jan '10

31 Jan '10

Merge authors: Kristian Nielsen (knielsen) ------------------------------------------------------------ revno: 2812 [merge] committer: knielsen(a)knielsen-hq.org branch nick: work-5.1 timestamp: Sun 2010-01-31 16:36:03 +0100 message: Automerge MariaDB 5.1.42 release branch into main. modified: libmysqld/CMakeLists.txt -- lp:maria https://code.launchpad.net/~maria-captains/maria/5.1 Your team Maria developers is subscribed to branch lp:maria. To unsubscribe from this branch go to https://code.launchpad.net/~maria-captains/maria/5.1/+edit-subscription.

1 0

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (knielsen:2812)
by knielsen＠knielsen-hq.org 31 Jan '10

31 Jan '10

#At lp:maria 2812 knielsen(a)knielsen-hq.org 2010-01-31 [merge] Automerge MariaDB 5.1.42 release branch into main. modified: libmysqld/CMakeLists.txt === modified file 'libmysqld/CMakeLists.txt' --- a/libmysqld/CMakeLists.txt 2009-12-03 11:19:05 +0000 +++ b/libmysqld/CMakeLists.txt 2010-01-31 09:13:21 +0000 @@ -153,7 +153,7 @@ FILE(WRITE cmake_dummy.c " ") # custom targets to "sql/CMakeLists.txt" and reference them here. ADD_LIBRARY(mysqlserver STATIC ${LIBMYSQLD_SOURCES}) ADD_DEPENDENCIES(mysqlserver GenServerSource GenError) -TARGET_LINK_LIBRARIES(mysqlserver) +TARGET_LINK_LIBRARIES(mysqlserver psapi.lib) # Add any additional libraries requested by engine(s) FOREACH (ENGINE_LIB ${MYSQLD_STATIC_ENGINE_LIBS})

1 0

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (knielsen:2792)
by knielsen＠knielsen-hq.org 31 Jan '10

31 Jan '10

#At lp:maria 2792 knielsen(a)knielsen-hq.org 2010-01-31 Fix windows build of embedded server (forgotten dependency). modified: libmysqld/CMakeLists.txt === modified file 'libmysqld/CMakeLists.txt' --- a/libmysqld/CMakeLists.txt 2009-12-03 11:19:05 +0000 +++ b/libmysqld/CMakeLists.txt 2010-01-31 09:13:21 +0000 @@ -153,7 +153,7 @@ FILE(WRITE cmake_dummy.c " ") # custom targets to "sql/CMakeLists.txt" and reference them here. ADD_LIBRARY(mysqlserver STATIC ${LIBMYSQLD_SOURCES}) ADD_DEPENDENCIES(mysqlserver GenServerSource GenError) -TARGET_LINK_LIBRARIES(mysqlserver) +TARGET_LINK_LIBRARIES(mysqlserver psapi.lib) # Add any additional libraries requested by engine(s) FOREACH (ENGINE_LIB ${MYSQLD_STATIC_ENGINE_LIBS})

1 0

[Maria-developers] [Branch ~maria-captains/maria/5.1] Rev 2811: now we force at least libevent-1.4
by noreply＠launchpad.net 30 Jan '10

30 Jan '10

------------------------------------------------------------ revno: 2811 committer: Sergei Golubchik <sergii(a)pisem.net> branch nick: maria-5.1 timestamp: Sat 2010-01-30 13:39:39 +0100 message: now we force at least libevent-1.4 modified: config/ac-macros/libevent.m4 -- lp:maria https://code.launchpad.net/~maria-captains/maria/5.1 Your team Maria developers is subscribed to branch lp:maria. To unsubscribe from this branch go to https://code.launchpad.net/~maria-captains/maria/5.1/+edit-subscription.

1 0

[Maria-developers] [Branch ~maria-captains/maria/5.1] Rev 2809: Support building with system libevent
by noreply＠launchpad.net 30 Jan '10

30 Jan '10

------------------------------------------------------------ revno: 2809 committer: Sergei Golubchik <sergii(a)pisem.net> branch nick: maria-5.1 timestamp: Fri 2010-01-29 21:35:56 +0100 message: Support building with system libevent modified: config/ac-macros/libevent.m4 -- lp:maria https://code.launchpad.net/~maria-captains/maria/5.1 Your team Maria developers is subscribed to branch lp:maria. To unsubscribe from this branch go to https://code.launchpad.net/~maria-captains/maria/5.1/+edit-subscription.

2 2

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (knielsen:2791)
by knielsen＠knielsen-hq.org 30 Jan '10

30 Jan '10

#At lp:maria 2791 knielsen(a)knielsen-hq.org 2010-01-30 [merge] Automerge from main into MariaDB 5.1.42 release branch. removed: mysql-test/suite/maria/t/maria2-master.opt storage/xtradb/handler/handler0vars.h storage/xtradb/handler/win_delay_loader.cc storage/xtradb/win-plugin/ storage/xtradb/win-plugin/README storage/xtradb/win-plugin/win-plugin.diff added: mysql-test/extra/rpl_tests/rpl_not_null.test mysql-test/r/bug47671.result mysql-test/r/innodb-consistent.result mysql-test/r/innodb_bug44571.result mysql-test/r/innodb_bug46676.result mysql-test/r/innodb_bug47167.result mysql-test/r/udf_query_cache.result mysql-test/std_data/bug47012.ARM mysql-test/std_data/bug47012.ARZ mysql-test/std_data/bug47012.frm mysql-test/suite/innodb/r/innodb_bug46676.result mysql-test/suite/innodb/r/innodb_bug47167.result mysql-test/suite/innodb/t/innodb_bug46676.test mysql-test/suite/innodb/t/innodb_bug47167.test mysql-test/suite/rpl/r/rpl_loaddata_symlink.result mysql-test/suite/rpl/r/rpl_nondeterministic_functions.result mysql-test/suite/rpl/r/rpl_not_null_innodb.result mysql-test/suite/rpl/r/rpl_not_null_myisam.result mysql-test/suite/rpl/r/rpl_row_trunc_temp.result mysql-test/suite/rpl/t/rpl_loaddata_symlink-master.opt mysql-test/suite/rpl/t/rpl_loaddata_symlink-master.sh mysql-test/suite/rpl/t/rpl_loaddata_symlink-slave.opt mysql-test/suite/rpl/t/rpl_loaddata_symlink-slave.sh mysql-test/suite/rpl/t/rpl_loaddata_symlink.test mysql-test/suite/rpl/t/rpl_nondeterministic_functions.test mysql-test/suite/rpl/t/rpl_not_null_innodb.test mysql-test/suite/rpl/t/rpl_not_null_myisam.test mysql-test/suite/rpl/t/rpl_row_trunc_temp.test mysql-test/t/bug47671-master.opt mysql-test/t/bug47671.test mysql-test/t/innodb-consistent-master.opt mysql-test/t/innodb-consistent.test mysql-test/t/innodb_bug44571.test mysql-test/t/innodb_bug46676.test mysql-test/t/innodb_bug47167.test mysql-test/t/mysqlbinlog2-master.opt mysql-test/t/mysqlbinlog_row-master.opt mysql-test/t/mysqlbinlog_row_innodb-master.opt mysql-test/t/mysqlbinlog_row_myisam-master.opt mysql-test/t/mysqlbinlog_row_trans-master.opt mysql-test/t/udf_query_cache-master.opt mysql-test/t/udf_query_cache.test storage/xtradb/ut/ut0auxconf_have_gcc_atomics.c modified: .bzrignore BUILD/util.sh Makefile.am client/mysql.cc client/mysqlbinlog.cc client/mysqlslap.c client/mysqltest.cc configure.in extra/comp_err.c extra/libevent/devpoll.c extra/yassl/src/yassl_error.cpp extra/yassl/taocrypt/include/asn.hpp extra/yassl/taocrypt/src/asn.cpp include/ft_global.h include/maria.h include/my_pthread.h include/mysql.h include/mysql.h.pp include/violite.h libmysql/libmysql.c libmysql/libmysql.def libmysqld/libmysqld.def mysql-test/collections/default.experimental mysql-test/extra/rpl_tests/rpl_extraSlave_Col.test mysql-test/extra/rpl_tests/rpl_row_tabledefs.test mysql-test/extra/rpl_tests/rpl_stm_000001.test mysql-test/include/mtr_warnings.sql mysql-test/include/ps_conv.inc mysql-test/lib/mtr_cases.pm mysql-test/mysql-test-run.pl mysql-test/r/archive.result mysql-test/r/delayed.result mysql-test/r/delete.result mysql-test/r/fulltext.result mysql-test/r/func_group.result mysql-test/r/func_misc.result mysql-test/r/grant.result mysql-test/r/grant2.result mysql-test/r/group_min_max.result mysql-test/r/information_schema.result mysql-test/r/information_schema_all_engines.result mysql-test/r/innodb-autoinc.result mysql-test/r/innodb-index.result mysql-test/r/innodb-timeout.result mysql-test/r/innodb-zip.result mysql-test/r/innodb.result mysql-test/r/innodb_bug36169.result mysql-test/r/innodb_bug44369.result mysql-test/r/innodb_file_format.result mysql-test/r/innodb_lock_wait_timeout_1.result mysql-test/r/innodb_mysql.result mysql-test/r/innodb_xtradb_bug317074.result mysql-test/r/mysql.result mysql-test/r/mysqltest.result mysql-test/r/olap.result mysql-test/r/openssl_1.result mysql-test/r/order_by.result mysql-test/r/partition.result mysql-test/r/query_cache.result mysql-test/r/query_cache_notembedded.result mysql-test/r/range.result mysql-test/r/select.result mysql-test/r/show_check.result mysql-test/r/sp-destruct.result mysql-test/r/sp-security.result mysql-test/r/sp.result mysql-test/r/sp_notembedded.result mysql-test/r/trigger.result mysql-test/r/trigger_notembedded.result mysql-test/r/type_newdecimal.result mysql-test/r/type_year.result mysql-test/r/udf.result mysql-test/r/variables.result mysql-test/std_data/cacert.pem mysql-test/std_data/client-cert.pem mysql-test/std_data/client-key.pem mysql-test/std_data/server-cert.pem mysql-test/std_data/server-key.pem mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result mysql-test/suite/binlog/r/binlog_stm_row.result mysql-test/suite/binlog/r/binlog_unsafe.result mysql-test/suite/binlog/t/binlog_killed.test mysql-test/suite/binlog/t/binlog_stm_mix_innodb_myisam.test mysql-test/suite/binlog/t/binlog_stm_row.test mysql-test/suite/binlog/t/binlog_unsafe.test mysql-test/suite/funcs_1/datadict/processlist_val.inc mysql-test/suite/funcs_1/r/is_columns_is.result mysql-test/suite/funcs_1/r/is_tables_is.result mysql-test/suite/innodb/r/innodb-index.result mysql-test/suite/innodb/t/innodb-consistent-master.opt mysql-test/suite/innodb/t/innodb-index.test mysql-test/suite/maria/r/maria-recover.result mysql-test/suite/maria/t/maria-recover.test mysql-test/suite/parts/inc/partition_timestamp.inc mysql-test/suite/parts/r/partition_recover_myisam.result mysql-test/suite/parts/t/partition_alter1_2_innodb.test mysql-test/suite/parts/t/partition_alter2_1_innodb.test mysql-test/suite/parts/t/partition_alter2_2_innodb.test mysql-test/suite/parts/t/partition_alter4_innodb.test mysql-test/suite/parts/t/partition_recover_myisam.test mysql-test/suite/pbxt/r/func_group.result mysql-test/suite/pbxt/r/mysqlshow.result mysql-test/suite/rpl/r/rpl_err_ignoredtable.result mysql-test/suite/rpl/r/rpl_extraCol_innodb.result mysql-test/suite/rpl/r/rpl_extraCol_myisam.result mysql-test/suite/rpl/r/rpl_get_lock.result mysql-test/suite/rpl/r/rpl_mixed_ddl_dml.result mysql-test/suite/rpl/r/rpl_relayspace.result mysql-test/suite/rpl/r/rpl_row_create_table.result mysql-test/suite/rpl/r/rpl_row_tabledefs_2myisam.result mysql-test/suite/rpl/r/rpl_row_tabledefs_3innodb.result mysql-test/suite/rpl/r/rpl_stm_000001.result mysql-test/suite/rpl/r/rpl_temporary.result mysql-test/suite/rpl/r/rpl_trigger.result mysql-test/suite/rpl/t/disabled.def mysql-test/suite/rpl/t/rpl_err_ignoredtable.test mysql-test/suite/rpl/t/rpl_get_lock.test mysql-test/suite/rpl/t/rpl_mixed_ddl_dml.test mysql-test/suite/rpl/t/rpl_relayspace.test mysql-test/suite/rpl/t/rpl_rotate_logs.test mysql-test/suite/rpl/t/rpl_row_create_table.test mysql-test/suite/rpl/t/rpl_temporary.test mysql-test/suite/rpl/t/rpl_trigger.test mysql-test/suite/rpl_ndb/r/rpl_ndb_extraCol.result mysql-test/t/archive.test mysql-test/t/delayed.test mysql-test/t/delete.test mysql-test/t/disabled.def mysql-test/t/fulltext.test mysql-test/t/func_group.test mysql-test/t/func_misc.test mysql-test/t/grant.test mysql-test/t/grant2.test mysql-test/t/group_min_max.test mysql-test/t/innodb-analyze.test mysql-test/t/innodb-autoinc.test mysql-test/t/innodb-index.test mysql-test/t/innodb-master.opt mysql-test/t/innodb-semi-consistent-master.opt mysql-test/t/innodb-timeout.test mysql-test/t/innodb-use-sys-malloc-master.opt mysql-test/t/innodb-zip.test mysql-test/t/innodb.test mysql-test/t/innodb_bug34300.test mysql-test/t/innodb_bug36169.test mysql-test/t/innodb_bug36172.test mysql-test/t/innodb_bug42101-nonzero-master.opt mysql-test/t/innodb_bug44369.test mysql-test/t/innodb_file_format.test mysql-test/t/innodb_information_schema.test mysql-test/t/innodb_lock_wait_timeout_1.test mysql-test/t/innodb_mysql.test mysql-test/t/innodb_xtradb_bug317074.test mysql-test/t/mysql.test mysql-test/t/mysqltest.test mysql-test/t/olap.test mysql-test/t/openssl_1.test mysql-test/t/order_by.test mysql-test/t/partition.test mysql-test/t/partition_innodb_semi_consistent.test mysql-test/t/query_cache.test mysql-test/t/query_cache_debug.test mysql-test/t/query_cache_notembedded.test mysql-test/t/range.test mysql-test/t/select.test mysql-test/t/show_check-master.opt mysql-test/t/show_check.test mysql-test/t/sp-destruct.test mysql-test/t/sp-security.test mysql-test/t/sp.test mysql-test/t/sp_notembedded.test mysql-test/t/trigger.test mysql-test/t/trigger_notembedded.test mysql-test/t/type_newdecimal.test mysql-test/t/type_year.test mysql-test/t/udf.test mysql-test/t/variables.test mysql-test/valgrind.supp mysys/my_file.c mysys/my_getopt.c mysys/my_sync.c mysys/my_thr_init.c scripts/make_win_bin_dist scripts/mysql_secure_installation.pl.in scripts/mysql_secure_installation.sh sql-common/client.c sql/CMakeLists.txt sql/event_db_repository.cc sql/field.cc sql/field.h sql/handler.h sql/item.cc sql/item.h sql/item_cmpfunc.cc sql/item_cmpfunc.h sql/item_create.cc sql/item_func.cc sql/item_func.h sql/item_geofunc.cc sql/item_strfunc.cc sql/item_subselect.cc sql/item_subselect.h sql/item_sum.cc sql/item_sum.h sql/item_timefunc.cc sql/item_xmlfunc.cc sql/log.cc sql/log_event.cc sql/log_event.h sql/mysqld.cc sql/net_serv.cc sql/opt_range.cc sql/repl_failsafe.cc sql/rpl_record.cc sql/rpl_record.h sql/rpl_rli.cc sql/rpl_tblmap.cc sql/set_var.cc sql/slave.cc sql/sp.cc sql/sp.h sql/sp_cache.cc sql/sp_head.cc sql/sp_head.h sql/sp_rcontext.cc sql/sql_acl.cc sql/sql_acl.h sql/sql_base.cc sql/sql_cache.cc sql/sql_cache.h sql/sql_class.cc sql/sql_connect.cc sql/sql_delete.cc sql/sql_insert.cc sql/sql_load.cc sql/sql_parse.cc sql/sql_partition.cc sql/sql_profile.cc sql/sql_profile.h sql/sql_select.cc sql/sql_show.cc sql/sql_table.cc sql/sql_yacc.yy sql/table.cc sql/table.h sql/udf_example.def storage/archive/CMakeLists.txt* storage/archive/azio.c storage/archive/ha_archive.cc storage/federated/CMakeLists.txt* storage/innobase/btr/btr0btr.c storage/innobase/data/data0type.c storage/innobase/handler/ha_innodb.cc storage/innobase/include/ha_prototypes.h storage/innobase/include/mach0data.h storage/innobase/include/mach0data.ic storage/innobase/include/os0file.h storage/innobase/include/trx0trx.h storage/innobase/lock/lock0lock.c storage/innobase/os/os0file.c storage/innobase/row/row0sel.c storage/innobase/trx/trx0trx.c storage/innobase/ut/ut0ut.c storage/innodb_plugin/CMakeLists.txt storage/innodb_plugin/ChangeLog storage/innodb_plugin/btr/btr0btr.c storage/innodb_plugin/btr/btr0sea.c storage/innodb_plugin/buf/buf0buf.c storage/innodb_plugin/data/data0type.c storage/innodb_plugin/dict/dict0dict.c storage/innodb_plugin/fil/fil0fil.c storage/innodb_plugin/handler/ha_innodb.cc storage/innodb_plugin/handler/ha_innodb.h storage/innodb_plugin/handler/handler0alter.cc storage/innodb_plugin/ibuf/ibuf0ibuf.c storage/innodb_plugin/include/btr0sea.h storage/innodb_plugin/include/db0err.h storage/innodb_plugin/include/dict0dict.h storage/innodb_plugin/include/fil0fil.h storage/innodb_plugin/include/ha_prototypes.h storage/innodb_plugin/include/ibuf0ibuf.h storage/innodb_plugin/include/lock0lock.h storage/innodb_plugin/include/log0log.h storage/innodb_plugin/include/log0recv.h storage/innodb_plugin/include/mem0mem.h storage/innodb_plugin/include/mem0pool.h storage/innodb_plugin/include/os0file.h storage/innodb_plugin/include/pars0pars.h storage/innodb_plugin/include/srv0srv.h storage/innodb_plugin/include/thr0loc.h storage/innodb_plugin/include/trx0i_s.h storage/innodb_plugin/include/trx0purge.h storage/innodb_plugin/include/trx0rseg.h storage/innodb_plugin/include/trx0sys.h storage/innodb_plugin/include/trx0trx.h storage/innodb_plugin/include/trx0undo.h storage/innodb_plugin/include/univ.i storage/innodb_plugin/include/usr0sess.h storage/innodb_plugin/lock/lock0lock.c storage/innodb_plugin/log/log0log.c storage/innodb_plugin/log/log0recv.c storage/innodb_plugin/mem/mem0dbg.c storage/innodb_plugin/mem/mem0pool.c storage/innodb_plugin/os/os0file.c storage/innodb_plugin/os/os0sync.c storage/innodb_plugin/os/os0thread.c storage/innodb_plugin/pars/lexyy.c storage/innodb_plugin/pars/pars0lex.l storage/innodb_plugin/que/que0que.c storage/innodb_plugin/row/row0merge.c storage/innodb_plugin/row/row0mysql.c storage/innodb_plugin/srv/srv0srv.c storage/innodb_plugin/srv/srv0start.c storage/innodb_plugin/sync/sync0arr.c storage/innodb_plugin/sync/sync0sync.c storage/innodb_plugin/thr/thr0loc.c storage/innodb_plugin/trx/trx0i_s.c storage/innodb_plugin/trx/trx0purge.c storage/innodb_plugin/trx/trx0rseg.c storage/innodb_plugin/trx/trx0sys.c storage/innodb_plugin/trx/trx0trx.c storage/innodb_plugin/trx/trx0undo.c storage/innodb_plugin/usr/usr0sess.c storage/innodb_plugin/ut/ut0mem.c storage/maria/ft_maria.c storage/maria/ma_blockrec.c storage/maria/ma_check.c storage/maria/ma_close.c storage/maria/ma_create.c storage/myisam/ft_boolean_search.c storage/myisam/ft_myisam.c storage/myisam/ft_nlq_search.c storage/myisam/ft_parser.c storage/myisam/ft_stopwords.c storage/myisam/ftdefs.h storage/myisam/mi_check.c storage/myisam/mi_create.c storage/xtradb/CMakeLists.txt storage/xtradb/ChangeLog storage/xtradb/Makefile.am storage/xtradb/btr/btr0btr.c storage/xtradb/btr/btr0sea.c storage/xtradb/buf/buf0buddy.c storage/xtradb/buf/buf0buf.c storage/xtradb/buf/buf0flu.c storage/xtradb/buf/buf0lru.c storage/xtradb/buf/buf0rea.c storage/xtradb/data/data0type.c storage/xtradb/dict/dict0crea.c storage/xtradb/dict/dict0dict.c storage/xtradb/fil/fil0fil.c storage/xtradb/fsp/fsp0fsp.c storage/xtradb/handler/ha_innodb.cc storage/xtradb/handler/ha_innodb.h storage/xtradb/handler/handler0alter.cc storage/xtradb/handler/i_s.cc storage/xtradb/handler/i_s.h storage/xtradb/handler/innodb_patch_info.h storage/xtradb/ibuf/ibuf0ibuf.c storage/xtradb/include/btr0cur.h storage/xtradb/include/btr0sea.h storage/xtradb/include/buf0buf.h storage/xtradb/include/buf0buf.ic storage/xtradb/include/buf0lru.h storage/xtradb/include/buf0rea.h storage/xtradb/include/buf0types.h storage/xtradb/include/db0err.h storage/xtradb/include/dict0crea.h storage/xtradb/include/dict0dict.h storage/xtradb/include/dict0mem.h storage/xtradb/include/fil0fil.h storage/xtradb/include/fsp0fsp.h storage/xtradb/include/ibuf0ibuf.h storage/xtradb/include/lock0lock.h storage/xtradb/include/log0log.h storage/xtradb/include/log0log.ic storage/xtradb/include/log0recv.h storage/xtradb/include/mem0mem.h storage/xtradb/include/mem0pool.h storage/xtradb/include/mtr0mtr.h storage/xtradb/include/os0file.h storage/xtradb/include/os0sync.h storage/xtradb/include/page0page.h storage/xtradb/include/page0page.ic storage/xtradb/include/page0zip.h storage/xtradb/include/pars0pars.h storage/xtradb/include/rem0cmp.h storage/xtradb/include/rem0rec.ic storage/xtradb/include/row0ins.h storage/xtradb/include/row0mysql.h storage/xtradb/include/srv0srv.h storage/xtradb/include/sync0rw.h storage/xtradb/include/sync0sync.h storage/xtradb/include/thr0loc.h storage/xtradb/include/trx0i_s.h storage/xtradb/include/trx0purge.h storage/xtradb/include/trx0rec.h storage/xtradb/include/trx0rec.ic storage/xtradb/include/trx0roll.h storage/xtradb/include/trx0rseg.h storage/xtradb/include/trx0sys.h storage/xtradb/include/trx0sys.ic storage/xtradb/include/trx0trx.h storage/xtradb/include/trx0types.h storage/xtradb/include/trx0undo.h storage/xtradb/include/univ.i storage/xtradb/include/usr0sess.h storage/xtradb/include/ut0auxconf.h storage/xtradb/include/ut0byte.h storage/xtradb/include/ut0byte.ic storage/xtradb/include/ut0ut.h storage/xtradb/lock/lock0lock.c storage/xtradb/log/log0log.c storage/xtradb/log/log0recv.c storage/xtradb/mem/mem0dbg.c storage/xtradb/mem/mem0mem.c storage/xtradb/mem/mem0pool.c storage/xtradb/mtr/mtr0mtr.c storage/xtradb/os/os0file.c storage/xtradb/os/os0proc.c storage/xtradb/os/os0sync.c storage/xtradb/os/os0thread.c storage/xtradb/page/page0cur.c storage/xtradb/page/page0page.c storage/xtradb/page/page0zip.c storage/xtradb/pars/lexyy.c storage/xtradb/pars/pars0lex.l storage/xtradb/plug.in storage/xtradb/que/que0que.c storage/xtradb/rem/rem0cmp.c storage/xtradb/row/row0ins.c storage/xtradb/row/row0merge.c storage/xtradb/row/row0mysql.c storage/xtradb/row/row0sel.c storage/xtradb/scripts/install_innodb_plugins.sql storage/xtradb/scripts/install_innodb_plugins_win.sql storage/xtradb/srv/srv0srv.c storage/xtradb/srv/srv0start.c storage/xtradb/sync/sync0arr.c storage/xtradb/sync/sync0rw.c storage/xtradb/sync/sync0sync.c storage/xtradb/thr/thr0loc.c storage/xtradb/trx/trx0i_s.c storage/xtradb/trx/trx0purge.c storage/xtradb/trx/trx0rec.c storage/xtradb/trx/trx0roll.c storage/xtradb/trx/trx0rseg.c storage/xtradb/trx/trx0sys.c storage/xtradb/trx/trx0trx.c storage/xtradb/trx/trx0undo.c storage/xtradb/usr/usr0sess.c storage/xtradb/ut/ut0auxconf_atomic_pthread_t_solaris.c storage/xtradb/ut/ut0mem.c storage/xtradb/ut/ut0ut.c support-files/compiler_warnings.supp tests/mysql_client_test.c unittest/mysys/waiting_threads-t.c vio/vio.c vio/viosocket.c vio/viossl.c === modified file '.bzrignore' --- a/.bzrignore 2009-12-22 13:50:20 +0000 +++ b/.bzrignore 2010-01-29 18:42:22 +0000 @@ -40,15 +40,15 @@ *.dsp *.Po *.Plo -*/*.dir/* +*.dir/ */*_pure_*warnings */.deps */.libs/* */.pure -*/debug/* -*/minsizerel/* -*/release/* -*/relwithdebinfo/* +debug/ +MinSizeRel/ +Release/ +RelWithDebInfo/ *~ .*.swp ./CMakeCache.txt @@ -96,7 +96,7 @@ BitKeeper/tmp/gone BitKeeper/tmp BitKeeper/log BitKeeper/etc/SCCS -CMakeFiles/* +CMakeFiles/ COPYING COPYING.LIB Docs/#manual.texi# === modified file 'BUILD/util.sh' --- a/BUILD/util.sh 2010-01-07 12:02:18 +0000 +++ b/BUILD/util.sh 2010-01-28 22:34:22 +0000 @@ -16,6 +16,8 @@ # Setting cpu options. get_cpuopt () { + case "$(uname -o)" in + *Linux*) case "$(gcc -dumpmachine)" in x86_64-*) # gcc barfs on -march=... on x64 @@ -26,7 +28,13 @@ get_cpuopt () { CPUOPT="-m32 -march=i586 -mtune=generic" ;; esac - return 0 + ;; + *Solaris*) + # ToDo: handle 32-bit build? For now default to 64-bit. + CPUOPT="-D__sun -m64 -mtune=athlon64" + ;; + esac + return 0 } # Default to a parallel build, but only if AM_MAKEFLAGS is not set. === modified file 'Makefile.am' --- a/Makefile.am 2009-12-03 11:19:05 +0000 +++ b/Makefile.am 2010-01-15 15:27:55 +0000 @@ -208,10 +208,6 @@ test-bt-fast: -cd mysql-test ; MTR_BUILD_THREAD=auto \ @PERL@ ./mysql-test-run.pl $(MTR_EXTRA_OPTIONS) --force --comment=stress --suite=stress -test-bt-fast2: - -cd mysql-test ; MTR_BUILD_THREAD=auto \ - @PERL@ ./mysql-test-run.pl $(MTR_EXTRA_OPTIONS) --force --comment=ps --ps-protocol --report-features - test-bt-debug: -cd mysql-test ; MTR_BUILD_THREAD=auto \ @PERL@ ./mysql-test-run.pl $(MTR_EXTRA_OPTIONS) --comment=debug --force --timer \ === modified file 'client/mysql.cc' --- a/client/mysql.cc 2009-12-03 11:34:11 +0000 +++ b/client/mysql.cc 2010-01-15 15:27:55 +0000 @@ -4356,7 +4356,7 @@ com_status(String *buffer __attribute__( Don't remove "limit 1", it is protection againts SQL_SELECT_LIMIT=0 */ - if (mysql_store_result_for_lazy(&result)) + if (!mysql_store_result_for_lazy(&result)) { MYSQL_ROW cur=mysql_fetch_row(result); if (cur) @@ -4401,7 +4401,7 @@ com_status(String *buffer __attribute__( if (mysql_errno(&mysql) == CR_SERVER_GONE_ERROR) return 0; } - if (mysql_store_result_for_lazy(&result)) + if (!mysql_store_result_for_lazy(&result)) { MYSQL_ROW cur=mysql_fetch_row(result); if (cur) @@ -4496,9 +4496,7 @@ server_version_string(MYSQL *con) */ if (server_version == NULL) - { - server_version= strdup(mysql_get_server_info(con)); - } + server_version= my_strdup(mysql_get_server_info(con), MYF(MY_WME)); } return server_version ? server_version : ""; === modified file 'client/mysqlbinlog.cc' --- a/client/mysqlbinlog.cc 2009-12-03 11:19:05 +0000 +++ b/client/mysqlbinlog.cc 2010-01-09 09:04:51 +0000 @@ -1378,6 +1378,10 @@ static int parse_args(int *argc, char*** */ static Exit_status safe_connect() { + /* Close and old connections to MySQL */ + if (mysql) + mysql_close(mysql); + mysql= mysql_init(NULL); if (!mysql) === modified file 'client/mysqlslap.c' --- a/client/mysqlslap.c 2009-12-03 11:34:11 +0000 +++ b/client/mysqlslap.c 2010-01-29 18:42:22 +0000 @@ -292,6 +292,25 @@ static int gettimeofday(struct timeval * } #endif +void set_mysql_connect_options(MYSQL *mysql) +{ + if (opt_compress) + mysql_options(mysql,MYSQL_OPT_COMPRESS,NullS); +#ifdef HAVE_OPENSSL + if (opt_use_ssl) + mysql_ssl_set(mysql, opt_ssl_key, opt_ssl_cert, opt_ssl_ca, + opt_ssl_capath, opt_ssl_cipher); +#endif + if (opt_protocol) + mysql_options(mysql,MYSQL_OPT_PROTOCOL,(char*)&opt_protocol); +#ifdef HAVE_SMEM + if (shared_memory_base_name) + mysql_options(mysql,MYSQL_SHARED_MEMORY_BASE_NAME,shared_memory_base_name); +#endif + mysql_options(mysql, MYSQL_SET_CHARSET_NAME, default_charset); +} + + int main(int argc, char **argv) { MYSQL mysql; @@ -323,20 +342,7 @@ int main(int argc, char **argv) exit(1); } mysql_init(&mysql); - if (opt_compress) - mysql_options(&mysql,MYSQL_OPT_COMPRESS,NullS); -#ifdef HAVE_OPENSSL - if (opt_use_ssl) - mysql_ssl_set(&mysql, opt_ssl_key, opt_ssl_cert, opt_ssl_ca, - opt_ssl_capath, opt_ssl_cipher); -#endif - if (opt_protocol) - mysql_options(&mysql,MYSQL_OPT_PROTOCOL,(char*)&opt_protocol); -#ifdef HAVE_SMEM - if (shared_memory_base_name) - mysql_options(&mysql,MYSQL_SHARED_MEMORY_BASE_NAME,shared_memory_base_name); -#endif - mysql_options(&mysql, MYSQL_SET_CHARSET_NAME, default_charset); + set_mysql_connect_options(&mysql); if (!opt_only_print) { @@ -1815,6 +1821,7 @@ pthread_handler_t run_task(void *p) my_progname, mysql_error(mysql)); exit(0); } + set_mysql_connect_options(mysql); if (mysql_thread_init()) { @@ -1855,7 +1862,6 @@ limit_not_met: my_progname, mysql_error(mysql)); exit(0); } - if (slap_connect(mysql)) goto end; } @@ -2223,6 +2229,7 @@ slap_connect(MYSQL *mysql) int x, connect_error= 1; for (x= 0; x < 10; x++) { + set_mysql_connect_options(mysql); if (mysql_real_connect(mysql, host, user, opt_password, create_schema_string, opt_mysql_port, === modified file 'client/mysqltest.cc' --- a/client/mysqltest.cc 2009-12-27 13:54:41 +0000 +++ b/client/mysqltest.cc 2010-01-28 14:49:14 +0000 @@ -657,7 +657,9 @@ public: } while ((bytes= fread(buf, 1, sizeof(buf), m_file)) > 0) - fwrite(buf, 1, bytes, stderr); + if (fwrite(buf, 1, bytes, stderr)) + die("Failed to write to '%s', errno: %d", + m_file_name, errno); if (!lines) { === modified file 'configure.in' --- a/configure.in 2009-12-23 08:32:14 +0000 +++ b/configure.in 2010-01-29 10:42:31 +0000 @@ -9,15 +9,16 @@ AC_CANONICAL_SYSTEM # remember to also update version.c in ndb # # When changing major version number please also check switch statement -# in mysqlbinlog.cc / check_master_version(). -# -# When merging new MySQL releases, update the version number to match the -# MySQL version number. -# -# Note: the following line must be parseable by win/configure.js:GetVersion() -AM_INIT_AUTOMAKE(mysql, 5.1.41-MariaDB-rc) +# in mysqlbinlog::check_master_version(). +AM_INIT_AUTOMAKE(mysql, 5.1.42-MariaDB) AM_CONFIG_HEADER([include/config.h:config.h.in]) +# Request support for automake silent-rules if available. +# Default to verbose output. One can use the configure-time +# option --enable-silent-rules or make V=0 to activate +# silent rules. +m4_ifdef([AM_SILENT_RULES], [AM_SILENT_RULES([no])]) + PROTOCOL_VERSION=10 DOT_FRM_VERSION=6 # See the libtool docs for information on how to do shared lib versions. @@ -224,14 +225,6 @@ then GXX="no" fi -if test "$ac_cv_prog_gcc" = "yes" -then - AS="$CC -c" - AC_SUBST(AS) -else - AC_PATH_PROG(AS, as, as) -fi - # Still need ranlib for readline; local static use only so no libtool. AC_PROG_RANLIB # We use libtool @@ -688,7 +681,7 @@ AC_ARG_ENABLE(assembler, AC_MSG_CHECKING(if we should use assembler functions) # For now we only support assembler on i386 and sparc systems -AM_CONDITIONAL(ASSEMBLER_x86, test "$ENABLE_ASSEMBLER" = "yes" -a "$BASE_MACHINE_TYPE" = "i386" && $AS strings/strings-x86.s -o checkassembler >/dev/null 2>&1 && test -f checkassembler && (rm -f checkassembler; exit 0;)) +AM_CONDITIONAL(ASSEMBLER_x86, test "$ENABLE_ASSEMBLER" = "yes" -a "$BASE_MACHINE_TYPE" = "i386" && $CCAS $CCASFLAGS -c strings/strings-x86.s -o checkassembler >/dev/null 2>&1 && test -f checkassembler && (rm -f checkassembler; exit 0;)) AM_CONDITIONAL(ASSEMBLER_sparc32, test "$ENABLE_ASSEMBLER" = "yes" -a "$BASE_MACHINE_TYPE" = "sparc") AM_CONDITIONAL(ASSEMBLER_sparc64, test "$ENABLE_ASSEMBLER" = "yes" -a "$BASE_MACHINE_TYPE" = "sparcv9") AM_CONDITIONAL(ASSEMBLER, test "$ASSEMBLER_x86_TRUE" = "" -o "$ASSEMBLER_sparc32_TRUE" = "") === modified file 'extra/comp_err.c' --- a/extra/comp_err.c 2009-02-13 16:41:47 +0000 +++ b/extra/comp_err.c 2009-11-20 10:11:31 +0000 @@ -660,7 +660,7 @@ static ha_checksum checksum_format_speci case 'u': case 'x': case 's': - chksum= my_checksum(chksum, start, (uint) (p - start)); + chksum= my_checksum(chksum, start, (uint) (p + 1 - start)); start= 0; /* Not in format specifier anymore */ break; @@ -1030,8 +1030,10 @@ static char *parse_text_line(char *pos) { int i, nr; char *row= pos; + size_t len; DBUG_ENTER("parse_text_line"); + len= strlen (pos); while (*pos) { if (*pos == '\\') @@ -1039,11 +1041,11 @@ static char *parse_text_line(char *pos) switch (*++pos) { case '\\': case '"': - VOID(strmov(pos - 1, pos)); + VOID(memmove (pos - 1, pos, len - (row - pos))); break; case 'n': pos[-1]= '\n'; - VOID(strmov(pos, pos + 1)); + VOID(memmove (pos, pos + 1, len - (row - pos))); break; default: if (*pos >= '0' && *pos < '8') @@ -1053,10 +1055,10 @@ static char *parse_text_line(char *pos) nr= nr * 8 + (*(pos++) - '0'); pos -= i; pos[-1]= nr; - VOID(strmov(pos, pos + i)); + VOID(memmove (pos, pos + i, len - (row - pos))); } else if (*pos) - VOID(strmov(pos - 1, pos)); /* Remove '\' */ + VOID(memmove (pos - 1, pos, len - (row - pos))); /* Remove '\' */ } } else === modified file 'extra/libevent/devpoll.c' --- a/extra/libevent/devpoll.c 2010-01-07 13:00:06 +0000 +++ b/extra/libevent/devpoll.c 2010-01-28 11:35:10 +0000 @@ -140,7 +140,7 @@ devpoll_init(struct event_base *base) return (NULL); if (getrlimit(RLIMIT_NOFILE, &rl) == 0 && - rl.rlim_cur != RLIM_INFINITY) + (unsigned long long) rl.rlim_cur != (unsigned long long) RLIM_INFINITY) nfiles = rl.rlim_cur - 1; /* Initialize the kernel queue */ === modified file 'extra/yassl/src/yassl_error.cpp' --- a/extra/yassl/src/yassl_error.cpp 2008-11-18 16:45:44 +0000 +++ b/extra/yassl/src/yassl_error.cpp 2010-01-29 10:42:31 +0000 @@ -60,7 +60,7 @@ void SetErrorString(YasslError error, ch using namespace TaoCrypt; const int max = MAX_ERROR_SZ; // shorthand - switch (error) { + switch ((int) error) { // yaSSL proper errors case range_error : === modified file 'extra/yassl/taocrypt/include/asn.hpp' --- a/extra/yassl/taocrypt/include/asn.hpp 2007-01-29 15:54:40 +0000 +++ b/extra/yassl/taocrypt/include/asn.hpp 2010-01-27 10:38:29 +0000 @@ -305,6 +305,7 @@ private: bool ValidateSignature(SignerList*); bool ConfirmSignature(Source&); void GetKey(); + char* AddTag(char*, const char*, const char*, word32, word32); void GetName(NameType); void GetValidity(); void GetDate(DateType); === modified file 'extra/yassl/taocrypt/src/asn.cpp' --- a/extra/yassl/taocrypt/src/asn.cpp 2009-09-15 11:22:39 +0000 +++ b/extra/yassl/taocrypt/src/asn.cpp 2010-01-27 10:38:29 +0000 @@ -652,6 +652,25 @@ word32 CertDecoder::GetDigest() } +// memory length checked add tag to buffer +char* CertDecoder::AddTag(char* ptr, const char* buf_end, const char* tag_name, + word32 tag_name_length, word32 tag_value_length) +{ + if (ptr + tag_name_length + tag_value_length > buf_end) { + source_.SetError(CONTENT_E); + return 0; + } + + memcpy(ptr, tag_name, tag_name_length); + ptr += tag_name_length; + + memcpy(ptr, source_.get_current(), tag_value_length); + ptr += tag_value_length; + + return ptr; +} + + // process NAME, either issuer or subject void CertDecoder::GetName(NameType nt) { @@ -659,11 +678,22 @@ void CertDecoder::GetName(NameType nt) SHA sha; word32 length = GetSequence(); // length of all distinguished names - assert (length < ASN_NAME_MAX); + + if (length >= ASN_NAME_MAX) + return; length += source_.get_index(); - char* ptr = (nt == ISSUER) ? issuer_ : subject_; - word32 idx = 0; + char* ptr; + char* buf_end; + + if (nt == ISSUER) { + ptr = issuer_; + buf_end = ptr + sizeof(issuer_) - 1; // 1 byte for trailing 0 + } + else { + ptr = subject_; + buf_end = ptr + sizeof(subject_) - 1; // 1 byte for trailing 0 + } while (source_.get_index() < length) { GetSet(); @@ -685,47 +715,36 @@ void CertDecoder::GetName(NameType nt) byte id = source_.next(); b = source_.next(); // strType word32 strLen = GetLength(source_); - bool copy = false; - - if (id == COMMON_NAME) { - memcpy(&ptr[idx], "/CN=", 4); - idx += 4; - copy = true; - } - else if (id == SUR_NAME) { - memcpy(&ptr[idx], "/SN=", 4); - idx += 4; - copy = true; - } - else if (id == COUNTRY_NAME) { - memcpy(&ptr[idx], "/C=", 3); - idx += 3; - copy = true; - } - else if (id == LOCALITY_NAME) { - memcpy(&ptr[idx], "/L=", 3); - idx += 3; - copy = true; - } - else if (id == STATE_NAME) { - memcpy(&ptr[idx], "/ST=", 4); - idx += 4; - copy = true; - } - else if (id == ORG_NAME) { - memcpy(&ptr[idx], "/O=", 3); - idx += 3; - copy = true; - } - else if (id == ORGUNIT_NAME) { - memcpy(&ptr[idx], "/OU=", 4); - idx += 4; - copy = true; - } - if (copy) { - memcpy(&ptr[idx], source_.get_current(), strLen); - idx += strLen; + switch (id) { + case COMMON_NAME: + if (!(ptr = AddTag(ptr, buf_end, "/CN=", 4, strLen))) + return; + break; + case SUR_NAME: + if (!(ptr = AddTag(ptr, buf_end, "/SN=", 4, strLen))) + return; + break; + case COUNTRY_NAME: + if (!(ptr = AddTag(ptr, buf_end, "/C=", 3, strLen))) + return; + break; + case LOCALITY_NAME: + if (!(ptr = AddTag(ptr, buf_end, "/L=", 3, strLen))) + return; + break; + case STATE_NAME: + if (!(ptr = AddTag(ptr, buf_end, "/ST=", 4, strLen))) + return; + break; + case ORG_NAME: + if (!(ptr = AddTag(ptr, buf_end, "/O=", 3, strLen))) + return; + break; + case ORGUNIT_NAME: + if (!(ptr = AddTag(ptr, buf_end, "/OU=", 4, strLen))) + return; + break; } sha.Update(source_.get_current(), strLen); @@ -740,17 +759,15 @@ void CertDecoder::GetName(NameType nt) word32 length = GetLength(source_); if (email) { - memcpy(&ptr[idx], "/emailAddress=", 14); - idx += 14; - - memcpy(&ptr[idx], source_.get_current(), length); - idx += length; + if (!(ptr = AddTag(ptr, buf_end, "/emailAddress=", 14, length))) + return; } source_.advance(length); } } - ptr[idx++] = 0; + + *ptr = 0; if (nt == ISSUER) sha.Final(issuerHash_); === modified file 'include/ft_global.h' --- a/include/ft_global.h 2009-11-30 13:36:06 +0000 +++ b/include/ft_global.h 2010-01-27 21:53:08 +0000 @@ -62,7 +62,8 @@ void ft_free_stopwords(void); #define FT_SORTED 2 #define FT_EXPAND 4 /* query expansion */ -FT_INFO *ft_init_search(uint,void *, uint, uchar *, uint,CHARSET_INFO *, uchar *); +FT_INFO *ft_init_search(uint,void *, uint, uchar *, size_t, + CHARSET_INFO *, uchar *); my_bool ft_boolean_check_syntax_string(const uchar *); /* Internal symbols for fulltext between maria and MyISAM */ === modified file 'include/maria.h' --- a/include/maria.h 2009-02-19 09:01:25 +0000 +++ b/include/maria.h 2010-01-27 21:53:08 +0000 @@ -462,7 +462,7 @@ void maria_versioning(MARIA_HA *info, my void maria_ignore_trids(MARIA_HA *info); /* fulltext functions */ -FT_INFO *maria_ft_init_search(uint,void *, uint, uchar *, uint, +FT_INFO *maria_ft_init_search(uint,void *, uint, uchar *, size_t, CHARSET_INFO *, uchar *); /* 'Almost-internal' Maria functions */ === modified file 'include/my_pthread.h' --- a/include/my_pthread.h 2009-06-30 12:01:29 +0000 +++ b/include/my_pthread.h 2010-01-14 16:51:00 +0000 @@ -543,9 +543,9 @@ void safe_mutex_free_deadlock_data(safe_ #else #define my_pthread_mutex_init(A,B,C,D) pthread_mutex_init((A),(B)) #define my_pthread_mutex_lock(A,B) pthread_mutex_lock(A) -#define safe_mutex_assert_owner(mp) -#define safe_mutex_assert_not_owner(mp) -#define safe_mutex_free_deadlock_data(mp) +#define safe_mutex_assert_owner(mp) do {} while(0) +#define safe_mutex_assert_not_owner(mp) do {} while(0) +#define safe_mutex_free_deadlock_data(mp) do {} while(0) #endif /* SAFE_MUTEX */ #if defined(MY_PTHREAD_FASTMUTEX) && !defined(SAFE_MUTEX) === modified file 'include/mysql.h' --- a/include/mysql.h 2009-12-03 11:19:05 +0000 +++ b/include/mysql.h 2010-01-15 15:27:55 +0000 @@ -558,16 +558,6 @@ unsigned long STDCALL mysql_real_escape_ char *to,const char *from, unsigned long length); void STDCALL mysql_debug(const char *debug); -char * STDCALL mysql_odbc_escape_string(MYSQL *mysql, - char *to, - unsigned long to_length, - const char *from, - unsigned long from_length, - void *param, - char * - (*extend_buffer) - (void *, char *to, - unsigned long *length)); void STDCALL myodbc_remove_escape(MYSQL *mysql,char *name); unsigned int STDCALL mysql_thread_safe(void); my_bool STDCALL mysql_embedded(void); === modified file 'include/mysql.h.pp' --- a/include/mysql.h.pp 2009-12-03 11:19:05 +0000 +++ b/include/mysql.h.pp 2010-01-15 15:27:55 +0000 @@ -518,16 +518,6 @@ unsigned long mysql_real_escape_string(M char *to,const char *from, unsigned long length); void mysql_debug(const char *debug); -char * mysql_odbc_escape_string(MYSQL *mysql, - char *to, - unsigned long to_length, - const char *from, - unsigned long from_length, - void *param, - char * - (*extend_buffer) - (void *, char *to, - unsigned long *length)); void myodbc_remove_escape(MYSQL *mysql,char *name); unsigned int mysql_thread_safe(void); my_bool mysql_embedded(void); === modified file 'include/violite.h' --- a/include/violite.h 2009-12-03 11:19:05 +0000 +++ b/include/violite.h 2010-01-29 10:42:31 +0000 @@ -123,8 +123,8 @@ struct st_VioSSLFd SSL_CTX *ssl_context; }; -int sslaccept(struct st_VioSSLFd*, Vio *, long timeout); -int sslconnect(struct st_VioSSLFd*, Vio *, long timeout); +int sslaccept(struct st_VioSSLFd*, Vio *, long timeout, char *error_string); +int sslconnect(struct st_VioSSLFd*, Vio *, long timeout, char *error_string); struct st_VioSSLFd *new_VioSSLConnectorFd(const char *key_file, const char *cert_file, @@ -225,8 +225,8 @@ struct st_vio #endif /* HAVE_SMEM */ #ifdef _WIN32 OVERLAPPED pipe_overlapped; - DWORD read_timeout_millis; - DWORD write_timeout_millis; + DWORD read_timeout_ms; + DWORD write_timeout_ms; #endif }; #endif /* vio_violite_h_ */ === modified file 'libmysql/libmysql.c' --- a/libmysql/libmysql.c 2009-12-03 11:34:11 +0000 +++ b/libmysql/libmysql.c 2010-01-15 15:27:55 +0000 @@ -1642,20 +1642,6 @@ mysql_real_escape_string(MYSQL *mysql, c return (uint) escape_string_for_mysql(mysql->charset, to, 0, from, length); } - -char * STDCALL -mysql_odbc_escape_string(MYSQL *mysql __attribute__((unused)), - char *to __attribute__((unused)), - ulong to_length __attribute__((unused)), - const char *from __attribute__((unused)), - ulong from_length __attribute__((unused)), - void *param __attribute__((unused)), - char * (*extend_buffer)(void *, char *, ulong *) - __attribute__((unused))) -{ - return NULL; -} - void STDCALL myodbc_remove_escape(MYSQL *mysql,char *name) { === modified file 'libmysql/libmysql.def' --- a/libmysql/libmysql.def 2009-12-03 11:19:05 +0000 +++ b/libmysql/libmysql.def 2010-01-15 15:27:55 +0000 @@ -78,7 +78,6 @@ EXPORTS mysql_next_result mysql_num_fields mysql_num_rows - mysql_odbc_escape_string mysql_options mysql_stmt_param_count mysql_stmt_param_metadata === modified file 'libmysqld/libmysqld.def' --- a/libmysqld/libmysqld.def 2009-12-03 11:19:05 +0000 +++ b/libmysqld/libmysqld.def 2010-01-15 15:27:55 +0000 @@ -50,7 +50,6 @@ EXPORTS mysql_next_result mysql_num_fields mysql_num_rows - mysql_odbc_escape_string mysql_options mysql_ping mysql_query === modified file 'mysql-test/collections/default.experimental' --- a/mysql-test/collections/default.experimental 2009-10-26 12:33:03 +0000 +++ b/mysql-test/collections/default.experimental 2009-12-02 09:47:49 +0000 @@ -13,15 +13,13 @@ funcs_1.ndb* funcs_2.ndb_charset # joro : NDB tests marked as experimental as agreed with bochklin main.ctype_gbk_binlog @solaris # Bug#46010: main.ctype_gbk_binlog fails sporadically : Table 't2' already exists -main.innodb-autoinc* # Bug#47809 2009-10-04 joro innodb-autoinc.test fails with valgrind errors with the innodb plugin main.plugin_load @solaris # Bug#42144 ndb.* # joro : NDB tests marked as experimental as agreed with bochklin -rpl.rpl_cross_version* # Bug #43913 2009-10-26 joro rpl_cross_version can't pass on conflicts complainig clash with --slave-load-tm -rpl.rpl_get_master_version_and_clock* # Bug#46931 2009-08-26 alik rpl.rpl_get_master_version_and_clock fails on hpux11.31 +rpl.rpl_cross_version* # Bug#48340 2009-12-01 Daogang rpl_cross_version: Found warnings/errors in server log file! +rpl.rpl_get_master_version_and_clock* # Bug #49191 2009-12-01 Daogang rpl_get_master_version_and_clock failed on PB2: COM_REGISTER_SLAVE failed rpl.rpl_innodb_bug28430* @solaris # Bug#46029 -rpl.rpl_row_create_table* # Bug#45576: rpl_row_create_table fails on PB2 rpl.rpl_trigger* # Bug#47810 2009-10-04 joro rpl.rpl_trigger.test fails with valgrind errors with the innodb plugin rpl_ndb.* # joro : NDB tests marked as experimental as agreed with bochklin === modified file 'mysql-test/extra/rpl_tests/rpl_extraSlave_Col.test' --- a/mysql-test/extra/rpl_tests/rpl_extraSlave_Col.test 2009-08-28 14:13:27 +0000 +++ b/mysql-test/extra/rpl_tests/rpl_extraSlave_Col.test 2009-10-22 00:21:50 +0000 @@ -407,37 +407,57 @@ sync_slave_with_master; ########################################### # Bug#22234, Bug#23907 Extra Slave Col is not # erroring on extra col with no default values. -######################################################## +############################################################### +# Error reaction is up to sql_mode of the slave sql (bug#38173) #--echo *** Create t9 on slave *** -STOP SLAVE; -RESET SLAVE; -eval CREATE TABLE t9 (a INT KEY, b BLOB, c CHAR(5), - d TIMESTAMP, - e INT NOT NULL) ENGINE=$engine_type; - ---echo *** Create t9 on Master *** -connection master; -eval CREATE TABLE t9 (a INT PRIMARY KEY, b BLOB, c CHAR(5) +# Please, check BUG#47741 to see why you are not testing NDB. +if (`SELECT $engine_type != 'NDB'`) +{ + STOP SLAVE; + RESET SLAVE; + eval CREATE TABLE t9 (a INT KEY, b BLOB, c CHAR(5), + d TIMESTAMP, + e INT NOT NULL, + f text not null, + g text, + h blob not null, + i blob) ENGINE=$engine_type; + + --echo *** Create t9 on Master *** + connection master; + eval CREATE TABLE t9 (a INT PRIMARY KEY, b BLOB, c CHAR(5) ) ENGINE=$engine_type; -RESET MASTER; + RESET MASTER; ---echo *** Start Slave *** -connection slave; -START SLAVE; - ---echo *** Master Data Insert *** -connection master; -set @b1 = 'b1b1b1b1'; -set @b1 = concat(@b1,@b1); -INSERT INTO t9 () VALUES(1,@b1,'Kyle'),(2,@b1,'JOE'),(3,@b1,'QA'); - -connection slave; ---source include/wait_for_slave_sql_to_stop.inc ---replace_result $MASTER_MYPORT MASTER_PORT ---replace_column 1 # 4 # 7 # 8 # 9 # 16 # 22 # 23 # 33 # 35 # 36 # ---query_vertical SHOW SLAVE STATUS -SET GLOBAL SQL_SLAVE_SKIP_COUNTER=2; -START SLAVE; + --echo *** Start Slave *** + connection slave; + START SLAVE; + + --echo *** Master Data Insert *** + connection master; + set @b1 = 'b1b1b1b1'; + + set @b1 = concat(@b1,@b1); + INSERT INTO t9 () VALUES(1,@b1,'Kyle'),(2,@b1,'JOE'),(3,@b1,'QA'); + + # the test would stop slave if @@sql_mode for the sql thread + # was set to strict. Otherwise, as with this tests setup, + # the implicit defaults will be inserted into fields even though + # they are declared without DEFAULT clause. + + sync_slave_with_master; + select * from t9; + + # todo: fix Bug #43992 slave sql thread can't tune own sql_mode ... + # and add/restore waiting for stop test + + #--source include/wait_for_slave_sql_to_stop.inc + #--replace_result $MASTER_MYPORT MASTER_PORT + #--replace_column 1 # 4 # 7 # 8 # 9 # 16 # 22 # 23 # 33 # 35 # 36 # + #--query_vertical SHOW SLAVE STATUS + #SET GLOBAL SQL_SLAVE_SKIP_COUNTER=2; + #START SLAVE; +} #--echo *** Drop t9 *** #connection master; === added file 'mysql-test/extra/rpl_tests/rpl_not_null.test' --- a/mysql-test/extra/rpl_tests/rpl_not_null.test 1970-01-01 00:00:00 +0000 +++ b/mysql-test/extra/rpl_tests/rpl_not_null.test 2009-10-22 00:19:52 +0000 @@ -0,0 +1,364 @@ +################################################################################# +# This test checks if the replication between "null" fields to either "null" +# fields or "not null" fields works properly. In the first case, the execution +# should work fine. In the second case, it may fail according to the sql_mode +# being used. +# +# The test is devided in three main parts: +# +# 1 - NULL --> NULL (no failures) +# 2 - NULL --> NOT NULL ( sql-mode = STRICT and failures) +# 3 - NULL --> NOT NULL ( sql-mode != STRICT and no failures) +# +################################################################################# +connection master; + +SET SQL_LOG_BIN= 0; +eval CREATE TABLE t1(`a` INT, `b` DATE DEFAULT NULL, +`c` INT DEFAULT NULL, +PRIMARY KEY(`a`)) ENGINE=$engine DEFAULT CHARSET=LATIN1; + +eval CREATE TABLE t2(`a` INT, `b` DATE DEFAULT NULL, +PRIMARY KEY(`a`)) ENGINE=$engine DEFAULT CHARSET=LATIN1; + +eval CREATE TABLE t3(`a` INT, `b` DATE DEFAULT NULL, +PRIMARY KEY(`a`)) ENGINE=$engine DEFAULT CHARSET=LATIN1; + +eval CREATE TABLE t4(`a` INT, `b` DATE DEFAULT NULL, +`c` INT DEFAULT NULL, +PRIMARY KEY(`a`)) ENGINE=$engine DEFAULT CHARSET=LATIN1; +SET SQL_LOG_BIN= 1; + +connection slave; + +eval CREATE TABLE t1(`a` INT, `b` DATE DEFAULT NULL, +`c` INT DEFAULT NULL, +PRIMARY KEY(`a`)) ENGINE=$engine DEFAULT CHARSET=LATIN1; + +eval CREATE TABLE t2(`a` INT, `b` DATE DEFAULT NULL, +PRIMARY KEY(`a`)) ENGINE=$engine DEFAULT CHARSET=LATIN1; + +eval CREATE TABLE t3(`a` INT, `b` DATE DEFAULT '0000-00-00', +`c` INT DEFAULT 500, +PRIMARY KEY(`a`)) ENGINE=$engine DEFAULT CHARSET=LATIN1; + +eval CREATE TABLE t4(`a` INT, `b` DATE DEFAULT '0000-00-00', +PRIMARY KEY(`a`)) ENGINE=$engine DEFAULT CHARSET=LATIN1; + +--echo ************* EXECUTION WITH INSERTS ************* +connection master; +INSERT INTO t1(a,b,c) VALUES (1, null, 1); +INSERT INTO t1(a,b,c) VALUES (2,'1111-11-11', 2); +INSERT INTO t1(a,b) VALUES (3, null); +INSERT INTO t1(a,c) VALUES (4, 4); +INSERT INTO t1(a) VALUES (5); + +INSERT INTO t2(a,b) VALUES (1, null); +INSERT INTO t2(a,b) VALUES (2,'1111-11-11'); +INSERT INTO t2(a) VALUES (3); + +INSERT INTO t3(a,b) VALUES (1, null); +INSERT INTO t3(a,b) VALUES (2,'1111-11-11'); +INSERT INTO t3(a) VALUES (3); + +INSERT INTO t4(a,b,c) VALUES (1, null, 1); +INSERT INTO t4(a,b,c) VALUES (2,'1111-11-11', 2); +INSERT INTO t4(a,b) VALUES (3, null); +INSERT INTO t4(a,c) VALUES (4, 4); +INSERT INTO t4(a) VALUES (5); + +--echo ************* SHOWING THE RESULT SETS WITH INSERTS ************* +sync_slave_with_master; + +--echo TABLES t1 and t2 must be equal otherwise an error will be thrown. +let $diff_table_1=master:test.t1; +let $diff_table_2=slave:test.t1; +source include/diff_tables.inc; + +let $diff_table_1=master:test.t2; +let $diff_table_2=slave:test.t2; +source include/diff_tables.inc; + +--echo TABLES t2 and t3 must be different. +connection master; +SELECT * FROM t3 ORDER BY a; +connection slave; +SELECT * FROM t3 ORDER BY a; +connection master; +SELECT * FROM t4 ORDER BY a; +connection slave; +SELECT * FROM t4 ORDER BY a; + +--echo ************* EXECUTION WITH UPDATES and REPLACES ************* +connection master; +DELETE FROM t1; +INSERT INTO t1(a,b,c) VALUES (1,'1111-11-11', 1); +REPLACE INTO t1(a,b,c) VALUES (2,'1111-11-11', 2); +UPDATE t1 set b= NULL, c= 300 where a= 1; +REPLACE INTO t1(a,b,c) VALUES (2, NULL, 300); + +--echo ************* SHOWING THE RESULT SETS WITH UPDATES and REPLACES ************* +sync_slave_with_master; + +--echo TABLES t1 and t2 must be equal otherwise an error will be thrown. +let $diff_table_1=master:test.t1; +let $diff_table_2=slave:test.t1; +source include/diff_tables.inc; + +--echo ************* CLEANING ************* +connection master; + +DROP TABLE t1; +DROP TABLE t2; +DROP TABLE t3; +DROP TABLE t4; + +sync_slave_with_master; + +connection master; + +SET SQL_LOG_BIN= 0; +eval CREATE TABLE t1 (`a` INT, `b` BIT DEFAULT NULL, `c` BIT DEFAULT NULL, +PRIMARY KEY (`a`)) ENGINE= $engine; +SET SQL_LOG_BIN= 1; + +connection slave; + +eval CREATE TABLE t1 (`a` INT, `b` BIT DEFAULT b'01', `c` BIT DEFAULT NULL, +PRIMARY KEY (`a`)) ENGINE= $engine; + +--echo ************* EXECUTION WITH INSERTS ************* +connection master; +INSERT INTO t1(a,b,c) VALUES (1, null, b'01'); +INSERT INTO t1(a,b,c) VALUES (2,b'00', b'01'); +INSERT INTO t1(a,b) VALUES (3, null); +INSERT INTO t1(a,c) VALUES (4, b'01'); +INSERT INTO t1(a) VALUES (5); + +--echo ************* SHOWING THE RESULT SETS WITH INSERTS ************* +--echo TABLES t1 and t2 must be different. +sync_slave_with_master; +connection master; +SELECT a,b+0,c+0 FROM t1 ORDER BY a; +connection slave; +SELECT a,b+0,c+0 FROM t1 ORDER BY a; + +--echo ************* EXECUTION WITH UPDATES and REPLACES ************* +connection master; +DELETE FROM t1; +INSERT INTO t1(a,b,c) VALUES (1,b'00', b'01'); +REPLACE INTO t1(a,b,c) VALUES (2,b'00',b'01'); +UPDATE t1 set b= NULL, c= b'00' where a= 1; +REPLACE INTO t1(a,b,c) VALUES (2, NULL, b'00'); + +--echo ************* SHOWING THE RESULT SETS WITH UPDATES and REPLACES ************* +--echo TABLES t1 and t2 must be equal otherwise an error will be thrown. +sync_slave_with_master; +let $diff_table_1=master:test.t1; +let $diff_table_2=slave:test.t1; +source include/diff_tables.inc; + +connection master; + +DROP TABLE t1; + +sync_slave_with_master; + +--echo ################################################################################ +--echo # NULL ---> NOT NULL (STRICT MODE) +--echo # UNCOMMENT THIS AFTER FIXING BUG#43992 +--echo ################################################################################ +#connection slave; +#SET GLOBAL sql_mode="TRADITIONAL"; +# +#STOP SLAVE; +#--source include/wait_for_slave_to_stop.inc +#START SLAVE; +#--source include/wait_for_slave_to_start.inc +# +#let $y=0; +#while (`select $y < 6`) +#{ +# connection master; +# +# SET SQL_LOG_BIN= 0; +# eval CREATE TABLE t1(`a` INT NOT NULL, `b` INT, +# PRIMARY KEY(`a`)) ENGINE=$engine DEFAULT CHARSET=LATIN1; +# eval CREATE TABLE t2(`a` INT NOT NULL, `b` INT, +# PRIMARY KEY(`a`)) ENGINE=$engine DEFAULT CHARSET=LATIN1; +# eval CREATE TABLE t3(`a` INT NOT NULL, `b` INT, +# PRIMARY KEY(`a`)) ENGINE=$engine DEFAULT CHARSET=LATIN1; +# SET SQL_LOG_BIN= 1; +# +# connection slave; +# +# eval CREATE TABLE t1(`a` INT NOT NULL, `b` INT NOT NULL, +# `c` INT NOT NULL, +# PRIMARY KEY(`a`)) ENGINE=$engine DEFAULT CHARSET=LATIN1; +# eval CREATE TABLE t2(`a` INT NOT NULL, `b` INT NOT NULL, +# `c` INT, +# PRIMARY KEY(`a`)) ENGINE=$engine DEFAULT CHARSET=LATIN1; +# eval CREATE TABLE t3(`a` INT NOT NULL, `b` INT NOT NULL, +# `c` INT DEFAULT 500, +# PRIMARY KEY(`a`)) ENGINE=$engine DEFAULT CHARSET=LATIN1; +# +# if (`select $y=0`) +# { +# --echo ************* EXECUTION WITH INSERTS ************* +# connection master; +# INSERT INTO t1(a) VALUES (1); +# } +# +# if (`select $y=1`) +# { +# --echo ************* EXECUTION WITH INSERTS ************* +# connection master; +# INSERT INTO t1(a, b) VALUES (1, NULL); +# } +# +# if (`select $y=2`) +# { +# --echo ************* EXECUTION WITH UPDATES ************* +# connection master; +# INSERT INTO t3(a, b) VALUES (1, 1); +# INSERT INTO t3(a, b) VALUES (2, 1); +# UPDATE t3 SET b = NULL where a= 1; +# } +# +# if (`select $y=3`) +# { +# --echo ************* EXECUTION WITH INSERTS/REPLACES ************* +# connection master; +# REPLACE INTO t3(a, b) VALUES (1, null); +# } +# +# if (`select $y=4`) +# { +# --echo ************* EXECUTION WITH UPDATES/REPLACES ************* +# connection master; +# INSERT INTO t3(a, b) VALUES (1, 1); +# REPLACE INTO t3(a, b) VALUES (1, null); +# } +# +# if (`select $y=5`) +# { +# --echo ************* EXECUTION WITH MULTI-ROW INSERTS ************* +# connection master; +# +# SET SQL_LOG_BIN= 0; +# INSERT INTO t2(a, b) VALUES (1, 1); +# INSERT INTO t2(a, b) VALUES (2, 1); +# INSERT INTO t2(a, b) VALUES (3, null); +# INSERT INTO t2(a, b) VALUES (4, 1); +# INSERT INTO t2(a, b) VALUES (5, 1); +# SET SQL_LOG_BIN= 1; +# +# INSERT INTO t2 SELECT a + 10, b from t2; +# --echo The statement below is just executed to stop processing +# INSERT INTO t1(a) VALUES (1); +# } +# +# --echo ************* SHOWING THE RESULT SETS ************* +# connection slave; +# --source include/wait_for_slave_sql_to_stop.inc +# connection master; +# SELECT * FROM t1 ORDER BY a; +# connection slave; +# SELECT * FROM t1 ORDER BY a; +# connection master; +# SELECT * FROM t2 ORDER BY a; +# connection slave; +# SELECT * FROM t2 ORDER BY a; +# connection master; +# SELECT * FROM t3 ORDER BY a; +# connection slave; +# SELECT * FROM t3 ORDER BY a; +# --source include/reset_master_and_slave.inc +# +# connection master; +# +# DROP TABLE t1; +# DROP TABLE t2; +# DROP TABLE t3; +# +# sync_slave_with_master; +# +# inc $y; +#} +#connection slave; +#SET GLOBAL sql_mode=""; +# +#STOP SLAVE; +#source include/wait_for_slave_to_stop.inc; +#START SLAVE; +#--source include/wait_for_slave_to_start.inc + +--echo ################################################################################ +--echo # NULL ---> NOT NULL (NON-STRICT MODE) +--echo ################################################################################ +connection master; + +SET SQL_LOG_BIN= 0; +eval CREATE TABLE t1(`a` INT NOT NULL, `b` INT, +PRIMARY KEY(`a`)) ENGINE=$engine DEFAULT CHARSET=LATIN1; +eval CREATE TABLE t2(`a` INT NOT NULL, `b` INT, +PRIMARY KEY(`a`)) ENGINE=$engine DEFAULT CHARSET=LATIN1; +eval CREATE TABLE t3(`a` INT NOT NULL, `b` INT, +PRIMARY KEY(`a`)) ENGINE=$engine DEFAULT CHARSET=LATIN1; +SET SQL_LOG_BIN= 1; + +connection slave; + +eval CREATE TABLE t1(`a` INT NOT NULL, `b` INT NOT NULL, +`c` INT NOT NULL, +PRIMARY KEY(`a`)) ENGINE=$engine DEFAULT CHARSET=LATIN1; +eval CREATE TABLE t2(`a` INT NOT NULL, `b` INT NOT NULL, +`c` INT, +PRIMARY KEY(`a`)) ENGINE=$engine DEFAULT CHARSET=LATIN1; +eval CREATE TABLE t3(`a` INT NOT NULL, `b` INT NOT NULL, +`c` INT DEFAULT 500, +PRIMARY KEY(`a`)) ENGINE=$engine DEFAULT CHARSET=LATIN1; + +--echo ************* EXECUTION WITH INSERTS ************* +connection master; +INSERT INTO t1(a) VALUES (1); +INSERT INTO t1(a, b) VALUES (2, NULL); +INSERT INTO t1(a, b) VALUES (3, 1); + +INSERT INTO t2(a) VALUES (1); +INSERT INTO t2(a, b) VALUES (2, NULL); +INSERT INTO t2(a, b) VALUES (3, 1); + +INSERT INTO t3(a) VALUES (1); +INSERT INTO t3(a, b) VALUES (2, NULL); +INSERT INTO t3(a, b) VALUES (3, 1); +INSERT INTO t3(a, b) VALUES (4, 1); +REPLACE INTO t3(a, b) VALUES (5, null); + +REPLACE INTO t3(a, b) VALUES (3, null); +UPDATE t3 SET b = NULL where a = 4; + +--echo ************* SHOWING THE RESULT SETS ************* +connection master; +sync_slave_with_master; + +connection master; +SELECT * FROM t1 ORDER BY a; +connection slave; +SELECT * FROM t1 ORDER BY a; +connection master; +SELECT * FROM t2 ORDER BY a; +connection slave; +SELECT * FROM t2 ORDER BY a; +connection master; +SELECT * FROM t3 ORDER BY a; +connection slave; +SELECT * FROM t3 ORDER BY a; + +connection master; + +DROP TABLE t1; +DROP TABLE t2; +DROP TABLE t3; + +sync_slave_with_master; === modified file 'mysql-test/extra/rpl_tests/rpl_row_tabledefs.test' --- a/mysql-test/extra/rpl_tests/rpl_row_tabledefs.test 2008-03-14 20:02:52 +0000 +++ b/mysql-test/extra/rpl_tests/rpl_row_tabledefs.test 2009-10-22 00:10:42 +0000 @@ -111,21 +111,18 @@ SELECT a,b,x FROM t1_int ORDER BY a; SELECT a,b,HEX(x),HEX(y),HEX(z) FROM t1_bit ORDER BY a; SELECT a,b,x FROM t1_char ORDER BY a; -# Each of these inserts should generate an error and stop the slave - connection master; INSERT INTO t9 VALUES (2); sync_slave_with_master; # Now slave is guaranteed to be running connection master; INSERT INTO t1_nodef VALUES (1,2); -connection slave; ---source include/wait_for_slave_sql_to_stop.inc ---replace_result $MASTER_MYPORT MASTER_PORT ---replace_column 1 # 4 # 7 # 8 # 9 # 20 <Last_Error> 22 # 23 # 33 # 35 <Last_IO_Errno> 36 <Last_IO_Error> 38 <Last_SQL_Error> ---query_vertical SHOW SLAVE STATUS -SET GLOBAL SQL_SLAVE_SKIP_COUNTER=2; -START SLAVE; + +# Last insert on wider slave table succeeds while slave sql sql_mode permits. +# The previous version of the above test expected slave sql to stop. +# bug#38173 relaxed conditions to stop only with the strict mode. +sync_slave_with_master; +select count(*) from t1_nodef; # # Replicating to tables with fewer columns at the end works as of WL#3228 === modified file 'mysql-test/extra/rpl_tests/rpl_stm_000001.test' --- a/mysql-test/extra/rpl_tests/rpl_stm_000001.test 2009-10-20 18:00:07 +0000 +++ b/mysql-test/extra/rpl_tests/rpl_stm_000001.test 2009-11-18 14:50:31 +0000 @@ -1,6 +1,11 @@ --- source include/have_binlog_format_mixed_or_statement.inc +# Requires binlog_format=statement format since query involving +# get_lock() is logged in row format if binlog_format=mixed or row. +-- source include/have_binlog_format_statement.inc -- source include/master-slave.inc +CALL mtr.add_suppression("Statement may not be safe to log in statement format."); + +# Load some data into t1 create table t1 (word char(20) not null); load data infile '../../std_data/words.dat' into table t1; --replace_result $MYSQL_TEST_DIR MYSQL_TEST_DIR @@ -10,9 +15,7 @@ select * from t1 limit 10; # # Test slave with wrong password # -save_master_pos; -connection slave; -sync_with_master; +sync_slave_with_master; stop slave; connection master; set password for root@"localhost" = password('foo'); @@ -29,16 +32,12 @@ sleep 2; create table t3(n int); insert into t3 values(1),(2); -save_master_pos; -connection slave; -sync_with_master; +sync_slave_with_master; select * from t3; select sum(length(word)) from t1; connection master; drop table t1,t3; -save_master_pos; -connection slave; -sync_with_master; +sync_slave_with_master; # Test if the slave SQL thread can be more than 16K behind the slave # I/O thread (> IO_SIZE) @@ -77,12 +76,13 @@ unlock tables; connection master; create table t2(id int); insert into t2 values(connection_id()); -save_master_pos; connection master1; # Avoid generating result create temporary table t3(n int); +--disable_warnings insert into t3 select get_lock('crash_lock%20C', 1) from t2; +--enable_warnings connection master; send update t1 set n = n + get_lock('crash_lock%20C', 2); @@ -93,8 +93,11 @@ kill @id; # We don't drop t3 as this is a temporary table drop table t2; connection master; +# The get_lock function causes warning for unsafe statement. +--disable_warnings --error 1317,2013 reap; +--enable_warnings connection slave; # The SQL slave thread should now have stopped because the query was killed on # the master (so it has a non-zero error code in the binlog). @@ -117,16 +120,12 @@ insert into mysql.user (Host, User, Pass select select_priv,user from mysql.user where user = _binary'blafasel2'; update mysql.user set Select_priv = "Y" where User= _binary"blafasel2"; select select_priv,user from mysql.user where user = _binary'blafasel2'; -save_master_pos; -connection slave; -sync_with_master; +sync_slave_with_master; select n from t1; select select_priv,user from mysql.user where user = _binary'blafasel2'; connection master1; drop table t1; delete from mysql.user where user="blafasel2"; -save_master_pos; -connection slave; -sync_with_master; +sync_slave_with_master; # End of 4.1 tests === modified file 'mysql-test/include/mtr_warnings.sql' --- a/mysql-test/include/mtr_warnings.sql 2009-12-03 11:19:05 +0000 +++ b/mysql-test/include/mtr_warnings.sql 2010-01-15 15:27:55 +0000 @@ -175,6 +175,8 @@ INSERT INTO global_suppressions VALUES ("Can't find file: '.\\\\test\\\\\\?{8}.frm'"), ("Slave: Unknown table 't1' Error_code: 1051"), + /* Maria storage engine dependent tests */ + /* maria-recovery.test has warning about missing log file */ ("File '.*maria_log.000.*' not found \$Errcode: 2\$"), /* and about marked-corrupted table */ @@ -184,6 +186,14 @@ INSERT INTO global_suppressions VALUES ("Table '..mysqltest.t_corrupted2' is marked as crashed and should be"), ("Incorrect key file for table '..mysqltest.t_corrupted2.MAI'"), + /* + Transient network failures that cause warnings on reconnect. + BUG#47743 and BUG#47983. + */ + ("Slave I/O: Get master SERVER_ID failed with error:.*"), + ("Slave I/O: Get master clock failed with error:.*"), + ("Slave I/O: Get master COLLATION_SERVER failed with error:.*"), + ("Slave I/O: Get master TIME_ZONE failed with error:.*"), ("THE_LAST_SUPPRESSION")|| === modified file 'mysql-test/include/ps_conv.inc' --- a/mysql-test/include/ps_conv.inc 2007-07-04 09:20:53 +0000 +++ b/mysql-test/include/ps_conv.inc 2010-01-19 10:36:52 +0000 @@ -30,6 +30,9 @@ --disable_warnings drop table if exists t5 ; --enable_warnings +--disable_query_log +SET TIME_ZONE= '+03:00'; +--enable_query_log set @arg01= 8; set @arg02= 8.0; set @arg03= 80.00000000000e-1; === modified file 'mysql-test/lib/mtr_cases.pm' --- a/mysql-test/lib/mtr_cases.pm 2009-12-06 17:34:54 +0000 +++ b/mysql-test/lib/mtr_cases.pm 2010-01-15 15:27:55 +0000 @@ -524,6 +524,10 @@ sub collect_one_suite next if ($test->{'name'} eq 'sys_vars.innodb_thread_concurrency_basic'); # Can't work with InnoPlug. Test framework needs to be re-designed. next if ($test->{'name'} eq 'main.innodb_bug46000'); + # Fails with innodb plugin + next if ($test->{'name'} eq 'main.innodb-autoinc'); + # Fails with innodb plugin: r6185 Testcases changes not included + next if ($test->{'name'} eq 'main.innodb_bug44369'); # Copy test options my $new_test= My::Test->new(); while (my ($key, $value) = each(%$test)) === modified file 'mysql-test/mysql-test-run.pl' --- a/mysql-test/mysql-test-run.pl 2010-01-06 21:27:53 +0000 +++ b/mysql-test/mysql-test-run.pl 2010-01-29 10:42:31 +0000 @@ -413,7 +413,6 @@ sub main { # Not all tests completed, failure mtr_report(); mtr_report("Only ", int(@$completed), " of $num_tests completed."); - mtr_error("Not all tests completed"); } mtr_print_line(); @@ -425,6 +424,10 @@ sub main { mtr_report_stats($fail, $completed, $extra_warnings); + if ( @$completed != $num_tests) + { + mtr_error("Not all tests completed"); + } exit(0); } @@ -3425,7 +3428,7 @@ sub restart_forced_by_test # Return timezone value of tinfo or default value sub timezone { my ($tinfo)= @_; - return $tinfo->{timezone} || "GMT-3"; + return $tinfo->{timezone} || "DEFAULT"; } @@ -3455,7 +3458,11 @@ sub run_testcase ($$) { # Init variables that can change between each test case # ------------------------------------------------------- my $timezone= timezone($tinfo); - $ENV{'TZ'}= $timezone; + if ($timezone ne 'DEFAULT') { + $ENV{'TZ'}= $timezone; + } else { + delete($ENV{'TZ'}); + } mtr_verbose("Setting timezone: $timezone"); if ( ! using_extern() ) === modified file 'mysql-test/r/archive.result' --- a/mysql-test/r/archive.result 2009-09-10 06:58:13 +0000 +++ b/mysql-test/r/archive.result 2009-11-11 08:03:29 +0000 @@ -12717,3 +12717,14 @@ COUNT(t1.a) 729 DROP TABLE t1; SET @@join_buffer_size= @save_join_buffer_size; +SHOW CREATE TABLE t1; +ERROR HY000: Table upgrade required. Please do "REPAIR TABLE `t1`" or dump/reload to fix it! +SELECT * FROM t1; +ERROR HY000: Table upgrade required. Please do "REPAIR TABLE `t1`" or dump/reload to fix it! +INSERT INTO t1 (col1, col2) VALUES (1, "value"); +ERROR HY000: Table upgrade required. Please do "REPAIR TABLE `t1`" or dump/reload to fix it! +REPAIR TABLE t1; +Table Op Msg_type Msg_text +test.t1 repair Error Table upgrade required. Please do "REPAIR TABLE `t1`" or dump/reload to fix it! +test.t1 repair error Corrupt +DROP TABLE t1; === added file 'mysql-test/r/bug47671.result' --- a/mysql-test/r/bug47671.result 1970-01-01 00:00:00 +0000 +++ b/mysql-test/r/bug47671.result 2010-01-15 15:27:55 +0000 @@ -0,0 +1,14 @@ +# +# Bug#47671 - wrong character-set after upgrade from 5.1.34 to 5.1.39 +# +# Extract only charset information from 'status' command output using regex +-------------- + +Server: MariaDB +Server characterset: utf8 +Db characterset: utf8 +Client characterset: utf8 +Conn. characterset: utf8 + +-------------- + === modified file 'mysql-test/r/delayed.result' --- a/mysql-test/r/delayed.result 2009-03-11 15:32:42 +0000 +++ b/mysql-test/r/delayed.result 2010-01-15 15:27:55 +0000 @@ -314,4 +314,16 @@ a b 2 2 drop table t1; set global low_priority_updates = @old_delayed_updates; +# +# Bug #47682 strange behaviour of INSERT DELAYED +# +DROP TABLE IF EXISTS t1, t2; +CREATE TABLE t1 (f1 integer); +CREATE TABLE t2 (f1 integer); +FLUSH TABLES WITH READ LOCK; +LOCK TABLES t1 READ; +INSERT DELAYED INTO t2 VALUES (1); +Got one of the listed errors +UNLOCK TABLES; +DROP TABLE t1, t2; End of 5.1 tests === modified file 'mysql-test/r/delete.result' --- a/mysql-test/r/delete.result 2009-09-28 10:48:52 +0000 +++ b/mysql-test/r/delete.result 2009-11-18 09:32:03 +0000 @@ -324,3 +324,16 @@ a 1 2 DROP TABLE t1, t2, t3; +# +# Bug #46425 crash in Diagnostics_area::set_ok_status, +# empty statement, DELETE IGNORE +# +CREATE table t1 (i INTEGER); +INSERT INTO t1 VALUES (1); +CREATE TRIGGER tr1 AFTER DELETE ON t1 FOR EACH ROW +BEGIN +INSERT INTO t1 SELECT * FROM t1 AS A; +END | +DELETE IGNORE FROM t1; +ERROR HY000: Can't update table 't1' in stored function/trigger because it is already used by statement which invoked this stored function/trigger. +DROP TABLE t1; === modified file 'mysql-test/r/fulltext.result' --- a/mysql-test/r/fulltext.result 2009-09-07 20:50:10 +0000 +++ b/mysql-test/r/fulltext.result 2010-01-15 15:27:55 +0000 @@ -559,3 +559,42 @@ EXECUTE s; MATCH (col) AGAINST('findme') DEALLOCATE PREPARE s; DROP TABLE t1; +# +# Bug #47930: MATCH IN BOOLEAN MODE returns too many results +# inside subquery +# +CREATE TABLE t1 (a int); +INSERT INTO t1 VALUES (1), (2); +CREATE TABLE t2 (a int, b2 char(10), FULLTEXT KEY b2 (b2)); +INSERT INTO t2 VALUES (1,'Scargill'); +CREATE TABLE t3 (a int, b int); +INSERT INTO t3 VALUES (1,1), (2,1); +# t2 should use full text index +EXPLAIN +SELECT count(*) FROM t1 WHERE +not exists( +SELECT 1 FROM t2, t3 +WHERE t3.a=t1.a AND MATCH(b2) AGAINST('scargill' IN BOOLEAN MODE) +); +id select_type table type possible_keys key key_len ref rows Extra +1 PRIMARY t1 ALL NULL NULL NULL NULL 2 Using where +2 DEPENDENT SUBQUERY t2 fulltext b2 b2 0 1 Using where +2 DEPENDENT SUBQUERY t3 ALL NULL NULL NULL NULL 2 Using where +# should return 0 +SELECT count(*) FROM t1 WHERE +not exists( +SELECT 1 FROM t2, t3 +WHERE t3.a=t1.a AND MATCH(b2) AGAINST('scargill' IN BOOLEAN MODE) +); +count(*) +0 +# should return 0 +SELECT count(*) FROM t1 WHERE +not exists( +SELECT 1 FROM t2 IGNORE INDEX (b2), t3 +WHERE t3.a=t1.a AND MATCH(b2) AGAINST('scargill' IN BOOLEAN MODE) +); +count(*) +0 +DROP TABLE t1,t2,t3; +End of 5.1 tests === modified file 'mysql-test/r/func_group.result' --- a/mysql-test/r/func_group.result 2009-10-14 08:46:50 +0000 +++ b/mysql-test/r/func_group.result 2009-11-24 15:26:13 +0000 @@ -885,7 +885,7 @@ cast(sum(distinct df) as signed) 3 select cast(min(df) as signed) from t1; cast(min(df) as signed) -0 +1 select 1e8 * sum(distinct df) from t1; 1e8 * sum(distinct df) 330000000 @@ -1520,4 +1520,197 @@ max i # Cleanup # DROP TABLE t1; +# +# Bug#43668: Wrong comparison and MIN/MAX for YEAR(2) +# +create table t1 (f1 year(2), f2 year(4), f3 date, f4 datetime); +insert into t1 values +(98,1998,19980101,"1998-01-01 00:00:00"), +(00,2000,20000101,"2000-01-01 00:00:01"), +(02,2002,20020101,"2002-01-01 23:59:59"), +(60,2060,20600101,"2060-01-01 11:11:11"), +(70,1970,19700101,"1970-11-11 22:22:22"), +(NULL,NULL,NULL,NULL); +select min(f1),max(f1) from t1; +min(f1) max(f1) +70 60 +select min(f2),max(f2) from t1; +min(f2) max(f2) +1970 2060 +select min(f3),max(f3) from t1; +min(f3) max(f3) +1970-01-01 2060-01-01 +select min(f4),max(f4) from t1; +min(f4) max(f4) +1970-11-11 22:22:22 2060-01-01 11:11:11 +select a.f1 as a, b.f1 as b, a.f1 > b.f1 as gt, +a.f1 < b.f1 as lt, a.f1<=>b.f1 as eq +from t1 a, t1 b; +a b gt lt eq +98 98 0 0 1 +00 98 1 0 0 +02 98 1 0 0 +60 98 1 0 0 +70 98 0 1 0 +NULL 98 NULL NULL 0 +98 00 0 1 0 +00 00 0 0 1 +02 00 1 0 0 +60 00 1 0 0 +70 00 0 1 0 +NULL 00 NULL NULL 0 +98 02 0 1 0 +00 02 0 1 0 +02 02 0 0 1 +60 02 1 0 0 +70 02 0 1 0 +NULL 02 NULL NULL 0 +98 60 0 1 0 +00 60 0 1 0 +02 60 0 1 0 +60 60 0 0 1 +70 60 0 1 0 +NULL 60 NULL NULL 0 +98 70 1 0 0 +00 70 1 0 0 +02 70 1 0 0 +60 70 1 0 0 +70 70 0 0 1 +NULL 70 NULL NULL 0 +98 NULL NULL NULL 0 +00 NULL NULL NULL 0 +02 NULL NULL NULL 0 +60 NULL NULL NULL 0 +70 NULL NULL NULL 0 +NULL NULL NULL NULL 1 +select a.f1 as a, b.f2 as b, a.f1 > b.f2 as gt, +a.f1 < b.f2 as lt, a.f1<=>b.f2 as eq +from t1 a, t1 b; +a b gt lt eq +98 1998 0 0 1 +00 1998 1 0 0 +02 1998 1 0 0 +60 1998 1 0 0 +70 1998 0 1 0 +NULL 1998 NULL NULL 0 +98 2000 0 1 0 +00 2000 0 0 1 +02 2000 1 0 0 +60 2000 1 0 0 +70 2000 0 1 0 +NULL 2000 NULL NULL 0 +98 2002 0 1 0 +00 2002 0 1 0 +02 2002 0 0 1 +60 2002 1 0 0 +70 2002 0 1 0 +NULL 2002 NULL NULL 0 +98 2060 0 1 0 +00 2060 0 1 0 +02 2060 0 1 0 +60 2060 0 0 1 +70 2060 0 1 0 +NULL 2060 NULL NULL 0 +98 1970 1 0 0 +00 1970 1 0 0 +02 1970 1 0 0 +60 1970 1 0 0 +70 1970 0 0 1 +NULL 1970 NULL NULL 0 +98 NULL NULL NULL 0 +00 NULL NULL NULL 0 +02 NULL NULL NULL 0 +60 NULL NULL NULL 0 +70 NULL NULL NULL 0 +NULL NULL NULL NULL 1 +select a.f1 as a, b.f3 as b, a.f1 > b.f3 as gt, +a.f1 < b.f3 as lt, a.f1<=>b.f3 as eq +from t1 a, t1 b; +a b gt lt eq +98 1998-01-01 0 1 0 +00 1998-01-01 1 0 0 +02 1998-01-01 1 0 0 +60 1998-01-01 1 0 0 +70 1998-01-01 0 1 0 +NULL 1998-01-01 NULL NULL 0 +98 2000-01-01 0 1 0 +00 2000-01-01 0 1 0 +02 2000-01-01 1 0 0 +60 2000-01-01 1 0 0 +70 2000-01-01 0 1 0 +NULL 2000-01-01 NULL NULL 0 +98 2002-01-01 0 1 0 +00 2002-01-01 0 1 0 +02 2002-01-01 0 1 0 +60 2002-01-01 1 0 0 +70 2002-01-01 0 1 0 +NULL 2002-01-01 NULL NULL 0 +98 2060-01-01 0 1 0 +00 2060-01-01 0 1 0 +02 2060-01-01 0 1 0 +60 2060-01-01 0 1 0 +70 2060-01-01 0 1 0 +NULL 2060-01-01 NULL NULL 0 +98 1970-01-01 1 0 0 +00 1970-01-01 1 0 0 +02 1970-01-01 1 0 0 +60 1970-01-01 1 0 0 +70 1970-01-01 0 1 0 +NULL 1970-01-01 NULL NULL 0 +98 NULL NULL NULL 0 +00 NULL NULL NULL 0 +02 NULL NULL NULL 0 +60 NULL NULL NULL 0 +70 NULL NULL NULL 0 +NULL NULL NULL NULL 1 +select a.f1 as a, b.f4 as b, a.f1 > b.f4 as gt, +a.f1 < b.f4 as lt, a.f1<=>b.f4 as eq +from t1 a, t1 b; +a b gt lt eq +98 1998-01-01 00:00:00 0 1 0 +00 1998-01-01 00:00:00 1 0 0 +02 1998-01-01 00:00:00 1 0 0 +60 1998-01-01 00:00:00 1 0 0 +70 1998-01-01 00:00:00 0 1 0 +NULL 1998-01-01 00:00:00 NULL NULL 0 +98 2000-01-01 00:00:01 0 1 0 +00 2000-01-01 00:00:01 0 1 0 +02 2000-01-01 00:00:01 1 0 0 +60 2000-01-01 00:00:01 1 0 0 +70 2000-01-01 00:00:01 0 1 0 +NULL 2000-01-01 00:00:01 NULL NULL 0 +98 2002-01-01 23:59:59 0 1 0 +00 2002-01-01 23:59:59 0 1 0 +02 2002-01-01 23:59:59 0 1 0 +60 2002-01-01 23:59:59 1 0 0 +70 2002-01-01 23:59:59 0 1 0 +NULL 2002-01-01 23:59:59 NULL NULL 0 +98 2060-01-01 11:11:11 0 1 0 +00 2060-01-01 11:11:11 0 1 0 +02 2060-01-01 11:11:11 0 1 0 +60 2060-01-01 11:11:11 0 1 0 +70 2060-01-01 11:11:11 0 1 0 +NULL 2060-01-01 11:11:11 NULL NULL 0 +98 1970-11-11 22:22:22 1 0 0 +00 1970-11-11 22:22:22 1 0 0 +02 1970-11-11 22:22:22 1 0 0 +60 1970-11-11 22:22:22 1 0 0 +70 1970-11-11 22:22:22 0 1 0 +NULL 1970-11-11 22:22:22 NULL NULL 0 +98 NULL NULL NULL 0 +00 NULL NULL NULL 0 +02 NULL NULL NULL 0 +60 NULL NULL NULL 0 +70 NULL NULL NULL 0 +NULL NULL NULL NULL 1 +select *, f1 = f2 from t1; +f1 f2 f3 f4 f1 = f2 +98 1998 1998-01-01 1998-01-01 00:00:00 1 +00 2000 2000-01-01 2000-01-01 00:00:01 1 +02 2002 2002-01-01 2002-01-01 23:59:59 1 +60 2060 2060-01-01 2060-01-01 11:11:11 1 +70 1970 1970-01-01 1970-11-11 22:22:22 1 +NULL NULL NULL NULL NULL +drop table t1; +# End of 5.1 tests === modified file 'mysql-test/r/func_misc.result' --- a/mysql-test/r/func_misc.result 2009-10-28 07:52:34 +0000 +++ b/mysql-test/r/func_misc.result 2010-01-11 13:15:28 +0000 @@ -104,95 +104,6 @@ t1 CREATE TABLE `t1` ( `length(uuid())` int(10) NOT NULL DEFAULT '0' ) ENGINE=MyISAM DEFAULT CHARSET=latin1 drop table t1; -#------------------------------------------------------------------------ -# Tests for Bug#6760 and Bug#12689 -SET @row_count = 4; -SET @sleep_time_per_result_row = 1; -SET @max_acceptable_delay = 2; -SET @@global.query_cache_size = 1024 * 64; -DROP TEMPORARY TABLE IF EXISTS t_history; -DROP TABLE IF EXISTS t1; -CREATE TEMPORARY TABLE t_history (attempt SMALLINT, -start_ts DATETIME, end_ts DATETIME, -start_cached INTEGER, end_cached INTEGER); -CREATE TABLE t1 (f1 BIGINT); -INSERT INTO t_history -SET attempt = 4 - 4 + 1, start_ts = NOW(), -start_cached = 0; -SELECT *, SLEEP(@sleep_time_per_result_row) FROM t1; -f1 SLEEP(@sleep_time_per_result_row) -1 0 -1 0 -1 0 -1 0 -UPDATE t_history SET end_ts = NOW() -WHERE attempt = 4 - 4 + 1; -UPDATE t_history SET end_cached = 0 -WHERE attempt = 4 - 4 + 1; -INSERT INTO t_history -SET attempt = 4 - 3 + 1, start_ts = NOW(), -start_cached = 0; -SELECT *, SLEEP(@sleep_time_per_result_row) FROM t1; -f1 SLEEP(@sleep_time_per_result_row) -1 0 -1 0 -1 0 -1 0 -UPDATE t_history SET end_ts = NOW() -WHERE attempt = 4 - 3 + 1; -UPDATE t_history SET end_cached = 0 -WHERE attempt = 4 - 3 + 1; -INSERT INTO t_history -SET attempt = 4 - 2 + 1, start_ts = NOW(), -start_cached = 0; -SELECT *, SLEEP(@sleep_time_per_result_row) FROM t1; -f1 SLEEP(@sleep_time_per_result_row) -1 0 -1 0 -1 0 -1 0 -UPDATE t_history SET end_ts = NOW() -WHERE attempt = 4 - 2 + 1; -UPDATE t_history SET end_cached = 0 -WHERE attempt = 4 - 2 + 1; -INSERT INTO t_history -SET attempt = 4 - 1 + 1, start_ts = NOW(), -start_cached = 0; -SELECT *, SLEEP(@sleep_time_per_result_row) FROM t1; -f1 SLEEP(@sleep_time_per_result_row) -1 0 -1 0 -1 0 -1 0 -UPDATE t_history SET end_ts = NOW() -WHERE attempt = 4 - 1 + 1; -UPDATE t_history SET end_cached = 0 -WHERE attempt = 4 - 1 + 1; -# Test 1: Does the query with SLEEP need a reasonable time? -SELECT COUNT(*) >= 4 - 1 INTO @aux1 FROM t_history -WHERE TIMEDIFF(end_ts,start_ts) - @sleep_time_per_result_row * @row_count -BETWEEN 0 AND @max_acceptable_delay; -SELECT @aux1 AS "Expect 1"; -Expect 1 -1 -# Test 2: Does the query with SLEEP need a reasonable time even in case -# of the non first execution? -SELECT COUNT(*) >= 4 - 1 - 1 INTO @aux2 FROM t_history -WHERE TIMEDIFF(end_ts,start_ts) - @sleep_time_per_result_row * @row_count -BETWEEN 0 AND @max_acceptable_delay -AND attempt > 1; -SELECT @aux2 AS "Expect 1"; -Expect 1 -1 -# Test 3: The query with SLEEP must be not cached. -SELECT COUNT(*) = 4 INTO @aux3 FROM t_history -WHERE end_cached = start_cached; -SELECT @aux3 AS "Expect 1"; -Expect 1 -1 -DROP TABLE t1; -DROP TEMPORARY TABLE t_history; -SET @@global.query_cache_size = default; create table t1 select INET_ATON('255.255.0.1') as `a`; show create table t1; Table Create Table === modified file 'mysql-test/r/grant.result' --- a/mysql-test/r/grant.result 2009-10-27 10:09:36 +0000 +++ b/mysql-test/r/grant.result 2010-01-29 10:42:31 +0000 @@ -29,15 +29,15 @@ show grants for mysqltest_1@localhost; Grants for mysqltest_1@localhost GRANT USAGE ON *.* TO 'mysqltest_1'@'localhost' GRANT SELECT ON `mysqltest`.* TO 'mysqltest_1'@'localhost' -grant USAGE on mysqltest.* to mysqltest_1@localhost require cipher "EDH-RSA-DES-CBC3-SHA" AND SUBJECT "testsubject" ISSUER "MySQL AB"; +grant USAGE on mysqltest.* to mysqltest_1@localhost require cipher "EDH-RSA-DES-CBC3-SHA" AND SUBJECT "testsubject" ISSUER "Monty Program Ab"; show grants for mysqltest_1@localhost; Grants for mysqltest_1@localhost -GRANT USAGE ON *.* TO 'mysqltest_1'@'localhost' REQUIRE ISSUER 'MySQL AB' SUBJECT 'testsubject' CIPHER 'EDH-RSA-DES-CBC3-SHA' +GRANT USAGE ON *.* TO 'mysqltest_1'@'localhost' REQUIRE ISSUER 'Monty Program Ab' SUBJECT 'testsubject' CIPHER 'EDH-RSA-DES-CBC3-SHA' GRANT SELECT ON `mysqltest`.* TO 'mysqltest_1'@'localhost' revoke all privileges on mysqltest.* from mysqltest_1@localhost; show grants for mysqltest_1@localhost; Grants for mysqltest_1@localhost -GRANT USAGE ON *.* TO 'mysqltest_1'@'localhost' REQUIRE ISSUER 'MySQL AB' SUBJECT 'testsubject' CIPHER 'EDH-RSA-DES-CBC3-SHA' +GRANT USAGE ON *.* TO 'mysqltest_1'@'localhost' REQUIRE ISSUER 'Monty Program Ab' SUBJECT 'testsubject' CIPHER 'EDH-RSA-DES-CBC3-SHA' delete from mysql.user where user='mysqltest_1'; flush privileges; delete from mysql.user where user='mysqltest_1'; === modified file 'mysql-test/r/grant2.result' --- a/mysql-test/r/grant2.result 2009-02-27 08:03:47 +0000 +++ b/mysql-test/r/grant2.result 2009-10-30 05:06:10 +0000 @@ -443,3 +443,30 @@ DROP TABLE db1.t1, db1.t2; DROP USER mysqltest1@localhost; DROP DATABASE db1; End of 5.0 tests +USE mysql; +SELECT LEFT(CURRENT_USER(),INSTR(CURRENT_USER(),'@')-1) INTO @u; +SELECT MID(CURRENT_USER(),INSTR(CURRENT_USER(),'@')+1) INTO @h; +SELECT password FROM user WHERE user=@u AND host=@h INTO @pwd; +SELECT user,host,password,insert_priv FROM user WHERE user=@u AND host=@h; +user host password insert_priv +root localhost Y +UPDATE user SET insert_priv='N' WHERE user=@u AND host=@h; +SELECT user,host,password,insert_priv FROM user WHERE user=@u AND host=@h; +user host password insert_priv +root localhost N +GRANT INSERT ON *.* TO CURRENT_USER(); +SELECT user,host,password,insert_priv FROM user WHERE user=@u AND host=@h; +user host password insert_priv +root localhost Y +UPDATE user SET insert_priv='N' WHERE user=@u AND host=@h; +GRANT INSERT ON *.* TO CURRENT_USER() IDENTIFIED BY 'keksdose'; +SELECT user,host,password,insert_priv FROM user WHERE user=@u AND host=@h; +user host password insert_priv +root localhost *0BB7188CF0DE9B403BA66E9DD810D82652D002EB Y +UPDATE user SET password=@pwd WHERE user=@u AND host=@h; +SELECT user,host,password,insert_priv FROM user WHERE user=@u AND host=@h; +user host password insert_priv +root localhost Y +FLUSH PRIVILEGES; +USE test; +End of 5.1 tests === modified file 'mysql-test/r/group_min_max.result' --- a/mysql-test/r/group_min_max.result 2009-10-09 09:30:40 +0000 +++ b/mysql-test/r/group_min_max.result 2009-11-23 10:04:17 +0000 @@ -2501,6 +2501,17 @@ SELECT a, MAX(b) FROM t WHERE b > 0 AND a MAX(b) 2 1 DROP TABLE t; +# +# Bug #48472: Loose index scan inappropriately chosen for some WHERE +# conditions +# +CREATE TABLE t (a INT, b INT, INDEX (a,b)); +INSERT INTO t VALUES (2,0), (2,0), (2,1), (2,1); +INSERT INTO t SELECT * FROM t; +SELECT a, MAX(b) FROM t WHERE 0=b+0 GROUP BY a; +a MAX(b) +2 0 +DROP TABLE t; End of 5.0 tests # # Bug #46607: Assertion failed: (cond_type == Item::FUNC_ITEM) results in === modified file 'mysql-test/r/information_schema.result' --- a/mysql-test/r/information_schema.result 2009-09-29 20:19:43 +0000 +++ b/mysql-test/r/information_schema.result 2010-01-15 15:58:25 +0000 @@ -85,6 +85,7 @@ TABLE_PRIVILEGES TRIGGERS USER_PRIVILEGES VIEWS +XTRADB_ADMIN_COMMAND XTRADB_ENHANCEMENTS columns_priv db @@ -865,8 +866,8 @@ TABLE_CONSTRAINTS TABLE_NAME select TABLE_PRIVILEGES TABLE_NAME select VIEWS TABLE_NAME select INNODB_BUFFER_POOL_PAGES_INDEX table_name select -INNODB_INDEX_STATS table_name select INNODB_TABLE_STATS table_name select +INNODB_INDEX_STATS table_name select delete from mysql.user where user='mysqltest_4'; delete from mysql.db where user='mysqltest_4'; flush privileges; === modified file 'mysql-test/r/information_schema_all_engines.result' --- a/mysql-test/r/information_schema_all_engines.result 2009-08-03 20:09:53 +0000 +++ b/mysql-test/r/information_schema_all_engines.result 2010-01-15 15:58:25 +0000 @@ -35,7 +35,7 @@ INNODB_CMP INNODB_RSEG XTRADB_ENHANCEMENTS INNODB_BUFFER_POOL_PAGES_INDEX -INNODB_INDEX_STATS +XTRADB_ADMIN_COMMAND INNODB_TRX INNODB_CMP_RESET INNODB_LOCK_WAITS @@ -44,6 +44,7 @@ INNODB_LOCKS INNODB_CMPMEM INNODB_TABLE_STATS INNODB_BUFFER_POOL_PAGES_BLOB +INNODB_INDEX_STATS SELECT t.table_name, c1.column_name FROM information_schema.tables t INNER JOIN @@ -93,7 +94,7 @@ INNODB_CMP page_size INNODB_RSEG rseg_id XTRADB_ENHANCEMENTS name INNODB_BUFFER_POOL_PAGES_INDEX schema_name -INNODB_INDEX_STATS table_name +XTRADB_ADMIN_COMMAND result_message INNODB_TRX trx_id INNODB_CMP_RESET page_size INNODB_LOCK_WAITS requesting_trx_id @@ -102,6 +103,7 @@ INNODB_LOCKS lock_id INNODB_CMPMEM page_size INNODB_TABLE_STATS table_name INNODB_BUFFER_POOL_PAGES_BLOB space_id +INNODB_INDEX_STATS table_name SELECT t.table_name, c1.column_name FROM information_schema.tables t INNER JOIN @@ -151,7 +153,7 @@ INNODB_CMP page_size INNODB_RSEG rseg_id XTRADB_ENHANCEMENTS name INNODB_BUFFER_POOL_PAGES_INDEX schema_name -INNODB_INDEX_STATS table_name +XTRADB_ADMIN_COMMAND result_message INNODB_TRX trx_id INNODB_CMP_RESET page_size INNODB_LOCK_WAITS requesting_trx_id @@ -160,6 +162,7 @@ INNODB_LOCKS lock_id INNODB_CMPMEM page_size INNODB_TABLE_STATS table_name INNODB_BUFFER_POOL_PAGES_BLOB space_id +INNODB_INDEX_STATS table_name select 1 as f1 from information_schema.tables where "CHARACTER_SETS"= (select cast(table_name as char) from information_schema.tables order by table_name limit 1) limit 1; @@ -262,7 +265,7 @@ Database: information_schema | INNODB_RSEG | | XTRADB_ENHANCEMENTS | | INNODB_BUFFER_POOL_PAGES_INDEX | -| INNODB_INDEX_STATS | +| XTRADB_ADMIN_COMMAND | | INNODB_TRX | | INNODB_CMP_RESET | | INNODB_LOCK_WAITS | @@ -271,6 +274,7 @@ Database: information_schema | INNODB_CMPMEM | | INNODB_TABLE_STATS | | INNODB_BUFFER_POOL_PAGES_BLOB | +| INNODB_INDEX_STATS | +---------------------------------------+ Database: INFORMATION_SCHEMA +---------------------------------------+ @@ -310,7 +314,7 @@ Database: INFORMATION_SCHEMA | INNODB_RSEG | | XTRADB_ENHANCEMENTS | | INNODB_BUFFER_POOL_PAGES_INDEX | -| INNODB_INDEX_STATS | +| XTRADB_ADMIN_COMMAND | | INNODB_TRX | | INNODB_CMP_RESET | | INNODB_LOCK_WAITS | @@ -319,6 +323,7 @@ Database: INFORMATION_SCHEMA | INNODB_CMPMEM | | INNODB_TABLE_STATS | | INNODB_BUFFER_POOL_PAGES_BLOB | +| INNODB_INDEX_STATS | +---------------------------------------+ Wildcard: inf_rmation_schema +--------------------+ @@ -328,5 +333,5 @@ Wildcard: inf_rmation_schema +--------------------+ SELECT table_schema, count(*) FROM information_schema.TABLES WHERE table_schema IN ('mysql', 'INFORMATION_SCHEMA', 'test', 'mysqltest') AND table_name<>'ndb_binlog_index' AND table_name<>'ndb_apply_status' GROUP BY TABLE_SCHEMA; table_schema count(*) -information_schema 43 +information_schema 44 mysql 22 === modified file 'mysql-test/r/innodb-autoinc.result' --- a/mysql-test/r/innodb-autoinc.result 2009-12-03 11:34:11 +0000 +++ b/mysql-test/r/innodb-autoinc.result 2010-01-15 17:02:57 +0000 @@ -197,7 +197,7 @@ c1 c2 5 9 DROP TABLE t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=100, @@SESSION.AUTO_INCREMENT_OFFSET=10; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 100 auto_increment_offset 10 @@ -230,7 +230,7 @@ c1 DROP TABLE t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 1 auto_increment_offset 1 @@ -269,7 +269,7 @@ c1 DROP TABLE t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 1 auto_increment_offset 1 @@ -282,7 +282,7 @@ SELECT * FROM t1; c1 -1 SET @@SESSION.AUTO_INCREMENT_INCREMENT=100, @@SESSION.AUTO_INCREMENT_OFFSET=10; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 100 auto_increment_offset 10 @@ -315,7 +315,7 @@ c1 DROP TABLE t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 1 auto_increment_offset 1 @@ -330,7 +330,7 @@ SELECT * FROM t1; c1 1 SET @@SESSION.AUTO_INCREMENT_INCREMENT=100, @@SESSION.AUTO_INCREMENT_OFFSET=10; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 100 auto_increment_offset 10 @@ -370,7 +370,7 @@ c1 DROP TABLE t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 1 auto_increment_offset 1 @@ -385,7 +385,7 @@ SELECT * FROM t1; c1 1 SET @@SESSION.AUTO_INCREMENT_INCREMENT=100, @@SESSION.AUTO_INCREMENT_OFFSET=10; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 100 auto_increment_offset 10 @@ -419,7 +419,7 @@ c1 DROP TABLE t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 1 auto_increment_offset 1 @@ -434,7 +434,7 @@ c1 1 9223372036854775794 SET @@SESSION.AUTO_INCREMENT_INCREMENT=2, @@SESSION.AUTO_INCREMENT_OFFSET=10; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 2 auto_increment_offset 10 @@ -452,7 +452,7 @@ c1 DROP TABLE t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 1 auto_increment_offset 1 @@ -467,7 +467,7 @@ c1 1 18446744073709551603 SET @@SESSION.AUTO_INCREMENT_INCREMENT=2, @@SESSION.AUTO_INCREMENT_OFFSET=10; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 2 auto_increment_offset 10 @@ -485,7 +485,7 @@ c1 DROP TABLE t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 1 auto_increment_offset 1 @@ -500,7 +500,7 @@ c1 1 18446744073709551603 SET @@SESSION.AUTO_INCREMENT_INCREMENT=5, @@SESSION.AUTO_INCREMENT_OFFSET=7; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 5 auto_increment_offset 7 @@ -514,7 +514,7 @@ c1 DROP TABLE t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 1 auto_increment_offset 1 @@ -533,7 +533,7 @@ c1 -9223372036854775806 1 SET @@SESSION.AUTO_INCREMENT_INCREMENT=3, @@SESSION.AUTO_INCREMENT_OFFSET=3; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 3 auto_increment_offset 3 @@ -550,7 +550,7 @@ c1 DROP TABLE t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 1 auto_increment_offset 1 @@ -568,7 +568,7 @@ SET @@SESSION.AUTO_INCREMENT_INCREMENT=1 Warnings: Warning 1292 Truncated incorrect auto_increment_increment value: '1152921504606846976' Warning 1292 Truncated incorrect auto_increment_offset value: '1152921504606846976' -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 65535 auto_increment_offset 65535 @@ -581,7 +581,7 @@ c1 DROP TABLE t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 1 auto_increment_offset 1 @@ -875,11 +875,11 @@ ALTER TABLE t1 CHANGE c1 d1 INT NOT NULL SELECT * FROM t1; d1 1 -3 +2 SELECT * FROM t1; d1 1 -3 +2 INSERT INTO t1 VALUES(null); Got one of the listed errors ALTER TABLE t1 AUTO_INCREMENT = 3; @@ -888,16 +888,16 @@ Table Create Table t1 CREATE TABLE `t1` ( `d1` int(11) NOT NULL AUTO_INCREMENT, PRIMARY KEY (`d1`) -) ENGINE=InnoDB AUTO_INCREMENT=4 DEFAULT CHARSET=latin1 +) ENGINE=InnoDB AUTO_INCREMENT=3 DEFAULT CHARSET=latin1 INSERT INTO t1 VALUES(null); SELECT * FROM t1; d1 1 +2 3 -4 DROP TABLE t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 1 auto_increment_offset 1 @@ -1126,3 +1126,61 @@ SELECT * FROM T1; c1 c2 10 0 DROP TABLE T1; +CREATE TABLE T1(C1 DOUBLE AUTO_INCREMENT KEY, C2 CHAR(10)) ENGINE=InnoDB; +INSERT INTO T1(C1, C2) VALUES (1, 'innodb'), (3, 'innodb'); +INSERT INTO T1(C2) VALUES ('innodb'); +SHOW CREATE TABLE T1; +Table Create Table +T1 CREATE TABLE `T1` ( + `C1` double NOT NULL AUTO_INCREMENT, + `C2` char(10) DEFAULT NULL, + PRIMARY KEY (`C1`) +) ENGINE=InnoDB AUTO_INCREMENT=5 DEFAULT CHARSET=latin1 +DROP TABLE T1; +CREATE TABLE T1(C1 FLOAT AUTO_INCREMENT KEY, C2 CHAR(10)) ENGINE=InnoDB; +INSERT INTO T1(C1, C2) VALUES (1, 'innodb'), (3, 'innodb'); +INSERT INTO T1(C2) VALUES ('innodb'); +SHOW CREATE TABLE T1; +Table Create Table +T1 CREATE TABLE `T1` ( + `C1` float NOT NULL AUTO_INCREMENT, + `C2` char(10) DEFAULT NULL, + PRIMARY KEY (`C1`) +) ENGINE=InnoDB AUTO_INCREMENT=5 DEFAULT CHARSET=latin1 +DROP TABLE T1; +CREATE TABLE t1 (c1 INT AUTO_INCREMENT PRIMARY KEY) ENGINE=InnoDB; +INSERT INTO t1 SET c1 = 1; +SHOW CREATE TABLE t1; +Table Create Table +t1 CREATE TABLE `t1` ( + `c1` int(11) NOT NULL AUTO_INCREMENT, + PRIMARY KEY (`c1`) +) ENGINE=InnoDB AUTO_INCREMENT=2 DEFAULT CHARSET=latin1 +INSERT INTO t1 SET c1 = 2; +INSERT INTO t1 SET c1 = -1; +SELECT * FROM t1; +c1 +-1 +1 +2 +INSERT INTO t1 SET c1 = -1; +Got one of the listed errors +SHOW CREATE TABLE t1; +Table Create Table +t1 CREATE TABLE `t1` ( + `c1` int(11) NOT NULL AUTO_INCREMENT, + PRIMARY KEY (`c1`) +) ENGINE=InnoDB AUTO_INCREMENT=3 DEFAULT CHARSET=latin1 +REPLACE INTO t1 VALUES (-1); +SELECT * FROM t1; +c1 +-1 +1 +2 +SHOW CREATE TABLE t1; +Table Create Table +t1 CREATE TABLE `t1` ( + `c1` int(11) NOT NULL AUTO_INCREMENT, + PRIMARY KEY (`c1`) +) ENGINE=InnoDB AUTO_INCREMENT=3 DEFAULT CHARSET=latin1 +DROP TABLE t1; === added file 'mysql-test/r/innodb-consistent.result' --- a/mysql-test/r/innodb-consistent.result 1970-01-01 00:00:00 +0000 +++ b/mysql-test/r/innodb-consistent.result 2010-01-15 15:58:25 +0000 @@ -0,0 +1,35 @@ +drop table if exists t1; +set session transaction isolation level read committed; +create table t1(a int not null) engine=innodb DEFAULT CHARSET=latin1; +create table t2 like t1; +insert into t2 values (1),(2),(3),(4),(5),(6),(7); +set autocommit=0; +begin; +replace into t1 select * from t2; +set session transaction isolation level read committed; +set autocommit=0; +delete from t2 where a=5; +commit; +delete from t2; +commit; +commit; +begin; +insert into t1 select * from t2; +set session transaction isolation level read committed; +set autocommit=0; +delete from t2 where a=5; +commit; +delete from t2; +commit; +commit; +select * from t1; +a +1 +2 +3 +4 +5 +6 +7 +drop table t1; +drop table t2; === modified file 'mysql-test/r/innodb-index.result' --- a/mysql-test/r/innodb-index.result 2009-11-30 21:37:27 +0000 +++ b/mysql-test/r/innodb-index.result 2010-01-15 15:58:25 +0000 @@ -1,4 +1,3 @@ -SET @save_innodb_file_format_check=@@global.innodb_file_format_check; create table t1(a int not null, b int, c char(10) not null, d varchar(20)) engine = innodb; insert into t1 values (5,5,'oo','oo'),(4,4,'tr','tr'),(3,4,'ad','ad'),(2,3,'ak','ak'); commit; @@ -629,7 +628,7 @@ drop table t1; create table t1(a int not null, b int, c char(10), d varchar(20), primary key (a)) engine = innodb; insert into t1 values (1,1,'ab','ab'),(2,2,'ac','ac'),(3,3,'ac','ac'),(4,4,'afe','afe'),(5,4,'affe','affe'); alter table t1 add unique index (b), add unique index (c), add unique index (d); -ERROR 23000: Duplicate entry '4' for key 'b' +ERROR 23000: Duplicate entry 'ac' for key 'c' alter table t1 add unique index (c), add unique index (b), add index (d); ERROR 23000: Duplicate entry 'ac' for key 'c' show create table t1; @@ -970,6 +969,7 @@ create index t1u on t1 (u(1)); drop table t1; set global innodb_file_per_table=0; set global innodb_file_format=Antelope; +set global innodb_file_format_check=Antelope; SET @OLD_UNIQUE_CHECKS=@@UNIQUE_CHECKS, UNIQUE_CHECKS=0; SET @OLD_FOREIGN_KEY_CHECKS=@@FOREIGN_KEY_CHECKS, FOREIGN_KEY_CHECKS=0; CREATE TABLE t1( @@ -1171,4 +1171,3 @@ a b 3 a 3 b DROP TABLE t1; -SET GLOBAL innodb_file_format_check=@save_innodb_file_format_check; === modified file 'mysql-test/r/innodb-timeout.result' --- a/mysql-test/r/innodb-timeout.result 2009-06-09 13:19:13 +0000 +++ b/mysql-test/r/innodb-timeout.result 2010-01-18 12:56:10 +0000 @@ -23,6 +23,7 @@ select @@innodb_lock_wait_timeout; create table t1(a int primary key)engine=innodb; begin; insert into t1 values(1),(2),(3); +set innodb_lock_wait_timeout=5; select * from t1 for update; commit; a @@ -31,8 +32,15 @@ a 3 begin; insert into t1 values(4); +set innodb_lock_wait_timeout=2; +set @a= current_timestamp(); select * from t1 for update; -commit; ERROR HY000: Lock wait timeout exceeded; try restarting transaction +set @b= current_timestamp(); +set @c= timestampdiff(SECOND, @a, @b); +select if(@c >= 1 and @c <= 10, 'OK', concat("NOT OK, time passed=", @c)); +if(@c >= 1 and @c <= 10, 'OK', concat("NOT OK, time passed=", @c)) +OK +commit; drop table t1; set global innodb_lock_wait_timeout=50; === modified file 'mysql-test/r/innodb-zip.result' --- a/mysql-test/r/innodb-zip.result 2009-06-09 15:08:46 +0000 +++ b/mysql-test/r/innodb-zip.result 2010-01-15 15:58:25 +0000 @@ -196,15 +196,15 @@ drop table t1; set innodb_strict_mode = on; create table t1 (id int primary key) engine = innodb key_block_size = 0; ERROR HY000: Can't create table 'test.t1' (errno: 1478) -show errors; +show warnings; Level Code Message -Error 1478 InnoDB: invalid KEY_BLOCK_SIZE = 0. Valid values are [1, 2, 4, 8, 16] +Warning 1478 InnoDB: invalid KEY_BLOCK_SIZE = 0. Valid values are [1, 2, 4, 8, 16] Error 1005 Can't create table 'test.t1' (errno: 1478) create table t2 (id int primary key) engine = innodb key_block_size = 9; ERROR HY000: Can't create table 'test.t2' (errno: 1478) -show errors; +show warnings; Level Code Message -Error 1478 InnoDB: invalid KEY_BLOCK_SIZE = 9. Valid values are [1, 2, 4, 8, 16] +Warning 1478 InnoDB: invalid KEY_BLOCK_SIZE = 9. Valid values are [1, 2, 4, 8, 16] Error 1005 Can't create table 'test.t2' (errno: 1478) create table t3 (id int primary key) engine = innodb key_block_size = 1; create table t4 (id int primary key) engine = innodb key_block_size = 2; @@ -233,30 +233,30 @@ key_block_size = 8 row_format = compress create table t2 (id int primary key) engine = innodb key_block_size = 8 row_format = redundant; ERROR HY000: Can't create table 'test.t2' (errno: 1478) -show errors; +show warnings; Level Code Message -Error 1478 InnoDB: cannot specify ROW_FORMAT = REDUNDANT with KEY_BLOCK_SIZE. +Warning 1478 InnoDB: cannot specify ROW_FORMAT = REDUNDANT with KEY_BLOCK_SIZE. Error 1005 Can't create table 'test.t2' (errno: 1478) create table t3 (id int primary key) engine = innodb key_block_size = 8 row_format = compact; ERROR HY000: Can't create table 'test.t3' (errno: 1478) -show errors; +show warnings; Level Code Message -Error 1478 InnoDB: cannot specify ROW_FORMAT = COMPACT with KEY_BLOCK_SIZE. +Warning 1478 InnoDB: cannot specify ROW_FORMAT = COMPACT with KEY_BLOCK_SIZE. Error 1005 Can't create table 'test.t3' (errno: 1478) create table t4 (id int primary key) engine = innodb key_block_size = 8 row_format = dynamic; ERROR HY000: Can't create table 'test.t4' (errno: 1478) -show errors; +show warnings; Level Code Message -Error 1478 InnoDB: cannot specify ROW_FORMAT = DYNAMIC with KEY_BLOCK_SIZE. +Warning 1478 InnoDB: cannot specify ROW_FORMAT = DYNAMIC with KEY_BLOCK_SIZE. Error 1005 Can't create table 'test.t4' (errno: 1478) create table t5 (id int primary key) engine = innodb key_block_size = 8 row_format = default; ERROR HY000: Can't create table 'test.t5' (errno: 1478) -show errors; +show warnings; Level Code Message -Error 1478 InnoDB: cannot specify ROW_FORMAT = COMPACT with KEY_BLOCK_SIZE. +Warning 1478 InnoDB: cannot specify ROW_FORMAT = COMPACT with KEY_BLOCK_SIZE. Error 1005 Can't create table 'test.t5' (errno: 1478) SELECT table_schema, table_name, row_format FROM information_schema.tables WHERE engine='innodb'; @@ -266,26 +266,26 @@ drop table t1; create table t1 (id int primary key) engine = innodb key_block_size = 9 row_format = redundant; ERROR HY000: Can't create table 'test.t1' (errno: 1478) -show errors; +show warnings; Level Code Message -Error 1478 InnoDB: invalid KEY_BLOCK_SIZE = 9. Valid values are [1, 2, 4, 8, 16] -Error 1478 InnoDB: cannot specify ROW_FORMAT = REDUNDANT with KEY_BLOCK_SIZE. +Warning 1478 InnoDB: invalid KEY_BLOCK_SIZE = 9. Valid values are [1, 2, 4, 8, 16] +Warning 1478 InnoDB: cannot specify ROW_FORMAT = REDUNDANT with KEY_BLOCK_SIZE. Error 1005 Can't create table 'test.t1' (errno: 1478) create table t2 (id int primary key) engine = innodb key_block_size = 9 row_format = compact; ERROR HY000: Can't create table 'test.t2' (errno: 1478) -show errors; +show warnings; Level Code Message -Error 1478 InnoDB: invalid KEY_BLOCK_SIZE = 9. Valid values are [1, 2, 4, 8, 16] -Error 1478 InnoDB: cannot specify ROW_FORMAT = COMPACT with KEY_BLOCK_SIZE. +Warning 1478 InnoDB: invalid KEY_BLOCK_SIZE = 9. Valid values are [1, 2, 4, 8, 16] +Warning 1478 InnoDB: cannot specify ROW_FORMAT = COMPACT with KEY_BLOCK_SIZE. Error 1005 Can't create table 'test.t2' (errno: 1478) create table t2 (id int primary key) engine = innodb key_block_size = 9 row_format = dynamic; ERROR HY000: Can't create table 'test.t2' (errno: 1478) -show errors; +show warnings; Level Code Message -Error 1478 InnoDB: invalid KEY_BLOCK_SIZE = 9. Valid values are [1, 2, 4, 8, 16] -Error 1478 InnoDB: cannot specify ROW_FORMAT = DYNAMIC with KEY_BLOCK_SIZE. +Warning 1478 InnoDB: invalid KEY_BLOCK_SIZE = 9. Valid values are [1, 2, 4, 8, 16] +Warning 1478 InnoDB: cannot specify ROW_FORMAT = DYNAMIC with KEY_BLOCK_SIZE. Error 1005 Can't create table 'test.t2' (errno: 1478) SELECT table_schema, table_name, row_format FROM information_schema.tables WHERE engine='innodb'; @@ -293,45 +293,45 @@ table_schema table_name row_format set global innodb_file_per_table = off; create table t1 (id int primary key) engine = innodb key_block_size = 1; ERROR HY000: Can't create table 'test.t1' (errno: 1478) -show errors; +show warnings; Level Code Message -Error 1478 InnoDB: KEY_BLOCK_SIZE requires innodb_file_per_table. +Warning 1478 InnoDB: KEY_BLOCK_SIZE requires innodb_file_per_table. Error 1005 Can't create table 'test.t1' (errno: 1478) create table t2 (id int primary key) engine = innodb key_block_size = 2; ERROR HY000: Can't create table 'test.t2' (errno: 1478) -show errors; +show warnings; Level Code Message -Error 1478 InnoDB: KEY_BLOCK_SIZE requires innodb_file_per_table. +Warning 1478 InnoDB: KEY_BLOCK_SIZE requires innodb_file_per_table. Error 1005 Can't create table 'test.t2' (errno: 1478) create table t3 (id int primary key) engine = innodb key_block_size = 4; ERROR HY000: Can't create table 'test.t3' (errno: 1478) -show errors; +show warnings; Level Code Message -Error 1478 InnoDB: KEY_BLOCK_SIZE requires innodb_file_per_table. +Warning 1478 InnoDB: KEY_BLOCK_SIZE requires innodb_file_per_table. Error 1005 Can't create table 'test.t3' (errno: 1478) create table t4 (id int primary key) engine = innodb key_block_size = 8; ERROR HY000: Can't create table 'test.t4' (errno: 1478) -show errors; +show warnings; Level Code Message -Error 1478 InnoDB: KEY_BLOCK_SIZE requires innodb_file_per_table. +Warning 1478 InnoDB: KEY_BLOCK_SIZE requires innodb_file_per_table. Error 1005 Can't create table 'test.t4' (errno: 1478) create table t5 (id int primary key) engine = innodb key_block_size = 16; ERROR HY000: Can't create table 'test.t5' (errno: 1478) -show errors; +show warnings; Level Code Message -Error 1478 InnoDB: KEY_BLOCK_SIZE requires innodb_file_per_table. +Warning 1478 InnoDB: KEY_BLOCK_SIZE requires innodb_file_per_table. Error 1005 Can't create table 'test.t5' (errno: 1478) create table t6 (id int primary key) engine = innodb row_format = compressed; ERROR HY000: Can't create table 'test.t6' (errno: 1478) -show errors; +show warnings; Level Code Message -Error 1478 InnoDB: ROW_FORMAT=COMPRESSED requires innodb_file_per_table. +Warning 1478 InnoDB: ROW_FORMAT=COMPRESSED requires innodb_file_per_table. Error 1005 Can't create table 'test.t6' (errno: 1478) create table t7 (id int primary key) engine = innodb row_format = dynamic; ERROR HY000: Can't create table 'test.t7' (errno: 1478) -show errors; +show warnings; Level Code Message -Error 1478 InnoDB: ROW_FORMAT=DYNAMIC requires innodb_file_per_table. +Warning 1478 InnoDB: ROW_FORMAT=DYNAMIC requires innodb_file_per_table. Error 1005 Can't create table 'test.t7' (errno: 1478) create table t8 (id int primary key) engine = innodb row_format = compact; create table t9 (id int primary key) engine = innodb row_format = redundant; @@ -345,45 +345,45 @@ set global innodb_file_per_table = on; set global innodb_file_format = `0`; create table t1 (id int primary key) engine = innodb key_block_size = 1; ERROR HY000: Can't create table 'test.t1' (errno: 1478) -show errors; +show warnings; Level Code Message -Error 1478 InnoDB: KEY_BLOCK_SIZE requires innodb_file_format > Antelope. +Warning 1478 InnoDB: KEY_BLOCK_SIZE requires innodb_file_format > Antelope. Error 1005 Can't create table 'test.t1' (errno: 1478) create table t2 (id int primary key) engine = innodb key_block_size = 2; ERROR HY000: Can't create table 'test.t2' (errno: 1478) -show errors; +show warnings; Level Code Message -Error 1478 InnoDB: KEY_BLOCK_SIZE requires innodb_file_format > Antelope. +Warning 1478 InnoDB: KEY_BLOCK_SIZE requires innodb_file_format > Antelope. Error 1005 Can't create table 'test.t2' (errno: 1478) create table t3 (id int primary key) engine = innodb key_block_size = 4; ERROR HY000: Can't create table 'test.t3' (errno: 1478) -show errors; +show warnings; Level Code Message -Error 1478 InnoDB: KEY_BLOCK_SIZE requires innodb_file_format > Antelope. +Warning 1478 InnoDB: KEY_BLOCK_SIZE requires innodb_file_format > Antelope. Error 1005 Can't create table 'test.t3' (errno: 1478) create table t4 (id int primary key) engine = innodb key_block_size = 8; ERROR HY000: Can't create table 'test.t4' (errno: 1478) -show errors; +show warnings; Level Code Message -Error 1478 InnoDB: KEY_BLOCK_SIZE requires innodb_file_format > Antelope. +Warning 1478 InnoDB: KEY_BLOCK_SIZE requires innodb_file_format > Antelope. Error 1005 Can't create table 'test.t4' (errno: 1478) create table t5 (id int primary key) engine = innodb key_block_size = 16; ERROR HY000: Can't create table 'test.t5' (errno: 1478) -show errors; +show warnings; Level Code Message -Error 1478 InnoDB: KEY_BLOCK_SIZE requires innodb_file_format > Antelope. +Warning 1478 InnoDB: KEY_BLOCK_SIZE requires innodb_file_format > Antelope. Error 1005 Can't create table 'test.t5' (errno: 1478) create table t6 (id int primary key) engine = innodb row_format = compressed; ERROR HY000: Can't create table 'test.t6' (errno: 1478) -show errors; +show warnings; Level Code Message -Error 1478 InnoDB: ROW_FORMAT=COMPRESSED requires innodb_file_format > Antelope. +Warning 1478 InnoDB: ROW_FORMAT=COMPRESSED requires innodb_file_format > Antelope. Error 1005 Can't create table 'test.t6' (errno: 1478) create table t7 (id int primary key) engine = innodb row_format = dynamic; ERROR HY000: Can't create table 'test.t7' (errno: 1478) -show errors; +show warnings; Level Code Message -Error 1478 InnoDB: ROW_FORMAT=DYNAMIC requires innodb_file_format > Antelope. +Warning 1478 InnoDB: ROW_FORMAT=DYNAMIC requires innodb_file_format > Antelope. Error 1005 Can't create table 'test.t7' (errno: 1478) create table t8 (id int primary key) engine = innodb row_format = compact; create table t9 (id int primary key) engine = innodb row_format = redundant; === modified file 'mysql-test/r/innodb.result' --- a/mysql-test/r/innodb.result 2009-12-27 13:54:41 +0000 +++ b/mysql-test/r/innodb.result 2010-01-15 15:58:25 +0000 @@ -3090,7 +3090,7 @@ ERROR HY000: Lock wait timeout exceeded; commit; drop table t1, t2, t3, t5, t6, t8, t9; CREATE TABLE t1 (DB_ROW_ID int) engine=innodb; -ERROR HY000: Can't create table 'test.t1' (errno: -1) +ERROR 42000: Incorrect column name 'DB_ROW_ID' CREATE TABLE t1 ( a BIGINT(20) NOT NULL, PRIMARY KEY (a) === modified file 'mysql-test/r/innodb_bug36169.result' --- a/mysql-test/r/innodb_bug36169.result 2009-11-13 21:26:08 +0000 +++ b/mysql-test/r/innodb_bug36169.result 2010-01-15 15:58:25 +0000 @@ -1,4 +1,2 @@ -set @old_innodb_file_format=@@innodb_file_format; -set @old_innodb_file_per_table=@@innodb_file_per_table; SET GLOBAL innodb_file_format='Barracuda'; SET GLOBAL innodb_file_per_table=ON; === modified file 'mysql-test/r/innodb_bug44369.result' --- a/mysql-test/r/innodb_bug44369.result 2009-11-02 14:59:44 +0000 +++ b/mysql-test/r/innodb_bug44369.result 2010-01-15 15:58:25 +0000 @@ -1,14 +1,6 @@ create table bug44369 (DB_ROW_ID int) engine=innodb; -ERROR HY000: Can't create table 'test.bug44369' (errno: -1) +ERROR 42000: Incorrect column name 'DB_ROW_ID' create table bug44369 (db_row_id int) engine=innodb; -ERROR HY000: Can't create table 'test.bug44369' (errno: -1) -show warnings; -Level Code Message -Warning 1005 Error creating table 'test/bug44369' with column name 'db_row_id'. 'db_row_id' is a reserved name. Please try to re-create the table with a different column name. -Error 1005 Can't create table 'test.bug44369' (errno: -1) +ERROR 42000: Incorrect column name 'db_row_id' create table bug44369 (db_TRX_Id int) engine=innodb; -ERROR HY000: Can't create table 'test.bug44369' (errno: -1) -show warnings; -Level Code Message -Warning 1005 Error creating table 'test/bug44369' with column name 'db_TRX_Id'. 'db_TRX_Id' is a reserved name. Please try to re-create the table with a different column name. -Error 1005 Can't create table 'test.bug44369' (errno: -1) +ERROR 42000: Incorrect column name 'db_TRX_Id' === added file 'mysql-test/r/innodb_bug44571.result' --- a/mysql-test/r/innodb_bug44571.result 1970-01-01 00:00:00 +0000 +++ b/mysql-test/r/innodb_bug44571.result 2010-01-15 15:58:25 +0000 @@ -0,0 +1,7 @@ +CREATE TABLE bug44571 (foo INT) ENGINE=InnoDB; +ALTER TABLE bug44571 CHANGE foo bar INT; +ALTER TABLE bug44571 ADD INDEX bug44571b (foo); +ERROR 42000: Key column 'foo' doesn't exist in table +ALTER TABLE bug44571 ADD INDEX bug44571b (bar); +CREATE INDEX bug44571c ON bug44571 (bar); +DROP TABLE bug44571; === added file 'mysql-test/r/innodb_bug46676.result' --- a/mysql-test/r/innodb_bug46676.result 1970-01-01 00:00:00 +0000 +++ b/mysql-test/r/innodb_bug46676.result 2010-01-15 15:58:25 +0000 @@ -0,0 +1,9 @@ +SET foreign_key_checks=0; +CREATE TABLE t1 (id int, foreign key (id) references t2(id)) ENGINE=INNODB; +CREATE TABLE t2 (id int, foreign key (id) references t1(id)) ENGINE=INNODB; +SET foreign_key_checks=1; +SELECT COUNT(*) FROM information_schema.key_column_usage WHERE REFERENCED_TABLE_NAME in ('t1', 't2'); +COUNT(*) +2 +SET foreign_key_checks=0; +DROP TABLE t1, t2; === added file 'mysql-test/r/innodb_bug47167.result' --- a/mysql-test/r/innodb_bug47167.result 1970-01-01 00:00:00 +0000 +++ b/mysql-test/r/innodb_bug47167.result 2010-01-15 15:58:25 +0000 @@ -0,0 +1,24 @@ +set @old_innodb_file_format_check=@@innodb_file_format_check; +select @old_innodb_file_format_check; +@old_innodb_file_format_check +Antelope +set global innodb_file_format_check = Barracuda; +select @@innodb_file_format_check; +@@innodb_file_format_check +Barracuda +set global innodb_file_format_check = DEFAULT; +select @@innodb_file_format_check; +@@innodb_file_format_check +Barracuda +set global innodb_file_format_check = @old_innodb_file_format_check; +select @@innodb_file_format_check; +@@innodb_file_format_check +Antelope +set global innodb_file_format_check = cheetah; +ERROR HY000: Incorrect arguments to SET +set global innodb_file_format_check = Bear; +ERROR HY000: Incorrect arguments to SET +set global innodb_file_format_check = on; +ERROR HY000: Incorrect arguments to SET +set global innodb_file_format_check = off; +ERROR HY000: Incorrect arguments to SET === modified file 'mysql-test/r/innodb_file_format.result' --- a/mysql-test/r/innodb_file_format.result 2009-11-30 21:37:27 +0000 +++ b/mysql-test/r/innodb_file_format.result 2010-01-15 15:58:25 +0000 @@ -1,4 +1,3 @@ -set @old_innodb_file_format=@@innodb_file_format; call mtr.add_suppression("InnoDB: invalid innodb_file_format_check value"); select @@innodb_file_format; @@innodb_file_format @@ -32,8 +31,6 @@ select @@innodb_file_format_check; @@innodb_file_format_check Barracuda set global innodb_file_format_check=default; -Warnings: -Warning 1210 Ignoring SET innodb_file_format=on select @@innodb_file_format_check; @@innodb_file_format_check Barracuda @@ -44,5 +41,4 @@ ERROR HY000: Incorrect arguments to SET select @@innodb_file_format_check; @@innodb_file_format_check Barracuda -set global innodb_file_format=@old_innodb_file_format; -set global innodb_file_format_check=Antelope; +set global innodb_file_format_check=antelope; === modified file 'mysql-test/r/innodb_lock_wait_timeout_1.result' --- a/mysql-test/r/innodb_lock_wait_timeout_1.result 2009-11-03 17:45:52 +0000 +++ b/mysql-test/r/innodb_lock_wait_timeout_1.result 2009-11-12 11:43:33 +0000 @@ -48,6 +48,24 @@ commit; set autocommit=default; drop table t1; # +# Bug #37183 insert ignore into .. select ... hangs +# after deadlock was encountered +# +create table t1(id int primary key,v int)engine=innodb; +insert into t1 values (1,1),(2,2),(3,3),(4,4),(5,5),(6,6),(7,7); +create table t2 like t1; +begin; +update t1 set v=id*2 where id=1; +begin; +update t1 set v=id*2 where id=2; +update t1 set v=id*2 where id=2; +ERROR HY000: Lock wait timeout exceeded; try restarting transaction +insert ignore into t2 select * from t1 where id=1; +ERROR HY000: Lock wait timeout exceeded; try restarting transaction +rollback; +rollback; +drop table t1, t2; +# # Bug#41756 Strange error messages about locks from InnoDB # drop table if exists t1; === modified file 'mysql-test/r/innodb_mysql.result' --- a/mysql-test/r/innodb_mysql.result 2009-12-03 11:19:05 +0000 +++ b/mysql-test/r/innodb_mysql.result 2010-01-15 15:27:55 +0000 @@ -2251,4 +2251,26 @@ c >= '2009-10-09 00:00:00.001' AND c <= id select_type table type possible_keys key key_len ref rows Extra 1 SIMPLE NULL NULL NULL NULL NULL NULL NULL Impossible WHERE noticed after reading const tables DROP TABLE t1; +# +# Bug #46175: NULL read_view and consistent read assertion +# +CREATE TABLE t1(a CHAR(13),KEY(a)) ENGINE=innodb; +CREATE TABLE t2(b DATETIME,KEY(b)) ENGINE=innodb; +INSERT INTO t1 VALUES (),(); +INSERT INTO t2 VALUES (),(); +CREATE OR REPLACE VIEW v1 AS SELECT 1 FROM t2 +WHERE b =(SELECT a FROM t1 LIMIT 1); +CREATE PROCEDURE p1(num INT) +BEGIN +DECLARE i INT DEFAULT 0; +REPEAT +SHOW CREATE VIEW v1; +SET i:=i+1; +UNTIL i>num END REPEAT; +END| +# Should not crash +# Should not crash +DROP PROCEDURE p1; +DROP VIEW v1; +DROP TABLE t1,t2; End of 5.1 tests === modified file 'mysql-test/r/innodb_xtradb_bug317074.result' --- a/mysql-test/r/innodb_xtradb_bug317074.result 2009-10-28 07:52:34 +0000 +++ b/mysql-test/r/innodb_xtradb_bug317074.result 2010-01-15 15:58:25 +0000 @@ -1,6 +1,5 @@ SET @old_innodb_file_format=@@innodb_file_format; SET @old_innodb_file_per_table=@@innodb_file_per_table; -SET @old_innodb_file_format_check=@@innodb_file_format_check; SET GLOBAL innodb_file_format='Barracuda'; SET GLOBAL innodb_file_per_table=ON; DROP TABLE IF EXISTS `test1`; @@ -29,4 +28,4 @@ ALTER TABLE test1 ENGINE=MyISAM; DROP TABLE test1; SET GLOBAL innodb_file_format=@old_innodb_file_format; SET GLOBAL innodb_file_per_table=@old_innodb_file_per_table; -SET GLOBAL innodb_file_format_check=@old_innodb_file_format_check; +set global innodb_file_format_check=Antelope; === modified file 'mysql-test/r/mysql.result' --- a/mysql-test/r/mysql.result 2009-07-31 23:43:46 +0000 +++ b/mysql-test/r/mysql.result 2009-11-27 14:41:45 +0000 @@ -229,5 +229,4 @@ a: b </row> </resultset> drop table t1; - -End of tests +End of 5.0 tests === modified file 'mysql-test/r/mysqltest.result' --- a/mysql-test/r/mysqltest.result 2009-10-08 09:30:03 +0000 +++ b/mysql-test/r/mysqltest.result 2010-01-11 13:15:28 +0000 @@ -1,3 +1,4 @@ +SET GLOBAL max_connections = 1000; select 0 as "before_use_test" ; before_use_test 0 === modified file 'mysql-test/r/olap.result' --- a/mysql-test/r/olap.result 2009-10-30 15:59:06 +0000 +++ b/mysql-test/r/olap.result 2009-12-08 09:26:11 +0000 @@ -753,4 +753,16 @@ b 100 NULL DROP TABLE t1, t2; +# +# Bug #48475: DISTINCT is ignored with GROUP BY WITH ROLLUP +# and only const tables +CREATE TABLE t1 (a INT); +CREATE TABLE t2 (b INT); +INSERT INTO t1 VALUES (1); +INSERT INTO t2 VALUES (1); +SELECT DISTINCT b FROM t1, t2 GROUP BY a, b WITH ROLLUP; +b +1 +NULL +DROP TABLE t1, t2; End of 5.0 tests === modified file 'mysql-test/r/openssl_1.result' --- a/mysql-test/r/openssl_1.result 2009-05-25 13:00:18 +0000 +++ b/mysql-test/r/openssl_1.result 2010-01-29 10:42:31 +0000 @@ -3,8 +3,10 @@ create table t1(f1 int); insert into t1 values (5); grant select on test.* to ssl_user1@localhost require SSL; grant select on test.* to ssl_user2@localhost require cipher "DHE-RSA-AES256-SHA"; -grant select on test.* to ssl_user3@localhost require cipher "DHE-RSA-AES256-SHA" AND SUBJECT "/C=SE/ST=Uppsala/O=MySQL AB/emailAddress=abstract.mysql.developer(a)mysql.com"; -grant select on test.* to ssl_user4@localhost require cipher "DHE-RSA-AES256-SHA" AND SUBJECT "/C=SE/ST=Uppsala/O=MySQL AB/emailAddress=abstract.mysql.developer(a)mysql.com" ISSUER "/C=SE/ST=Uppsala/L=Uppsala/O=MySQL AB"; +grant select on test.* to ssl_user3@localhost require cipher +"DHE-RSA-AES256-SHA" AND SUBJECT "/C=FI/ST=Tuusula/O=Monty Program Ab/emailAddress=abstract.developer(a)askmonty.org"; +grant select on test.* to ssl_user4@localhost require cipher +"DHE-RSA-AES256-SHA" AND SUBJECT "/C=FI/ST=Tuusula/O=Monty Program Ab/emailAddress=abstract.developer(a)askmonty.org" ISSUER "/C=FI/ST=Tuusula/O=Monty Program Ab/emailAddress=abstract.developer(a)askmonty.org"; grant select on test.* to ssl_user5@localhost require cipher "DHE-RSA-AES256-SHA" AND SUBJECT "xxx"; flush privileges; connect(localhost,ssl_user5,,test,MASTER_PORT,MASTER_SOCKET); @@ -44,9 +46,9 @@ ERROR 42000: DELETE command denied to us drop user ssl_user1@localhost, ssl_user2@localhost, ssl_user3@localhost, ssl_user4@localhost, ssl_user5@localhost; drop table t1; -mysqltest: Could not open connection 'default': 2026 SSL connection error -mysqltest: Could not open connection 'default': 2026 SSL connection error -mysqltest: Could not open connection 'default': 2026 SSL connection error +mysqltest: Could not open connection 'default': 2026 SSL error: ASN: bad other signature confirmation +mysqltest: Could not open connection 'default': 2026 SSL error: ASN: bad other signature confirmation +mysqltest: Could not open connection 'default': 2026 SSL error: ASN: bad other signature confirmation SSL error: Unable to get private key from '' mysqltest: Could not open connection 'default': 2026 SSL connection error SSL error: Unable to get certificate from '' === modified file 'mysql-test/r/order_by.result' --- a/mysql-test/r/order_by.result 2009-10-14 14:30:39 +0000 +++ b/mysql-test/r/order_by.result 2009-11-10 08:58:43 +0000 @@ -1444,6 +1444,27 @@ FROM t3; 2 NULL DROP TABLE t1, t2, t3; +# +# Bug #42760: Select doesn't return desired results when we have null +# values +# +CREATE TABLE t1 ( +a INT, +c INT, +UNIQUE KEY a_c (a,c), +KEY (a)); +INSERT INTO t1 VALUES (1, 10), (2, NULL); +# Must use ref-or-null on the a_c index +EXPLAIN +SELECT 1 AS col FROM t1 WHERE a=2 AND (c=10 OR c IS NULL) ORDER BY c; +id select_type table type possible_keys key key_len ref rows Extra +1 SIMPLE t1 ref_or_null a_c,a a_c 10 const,const 1 Using where +# Must return 1 row +SELECT 1 AS col FROM t1 WHERE a=2 AND (c=10 OR c IS NULL) ORDER BY c; +col +1 +DROP TABLE t1; +End of 5.0 tests CREATE TABLE t2 (a varchar(32), b int(11), c float, d double, UNIQUE KEY a (a,b,c), KEY b (b), KEY c (c)); CREATE TABLE t1 (a varchar(32), b char(3), UNIQUE KEY a (a,b), KEY b (b)); === modified file 'mysql-test/r/partition.result' --- a/mysql-test/r/partition.result 2009-12-03 11:19:05 +0000 +++ b/mysql-test/r/partition.result 2010-01-15 15:27:55 +0000 @@ -1,4 +1,10 @@ drop table if exists t1, t2; +CREATE TABLE t1 (a INT, b INT) +PARTITION BY LIST (a) +SUBPARTITION BY HASH (b) +(PARTITION p1 VALUES IN (1)); +ALTER TABLE t1 ADD COLUMN c INT; +DROP TABLE t1; CREATE TABLE t1 ( a int NOT NULL, b int NOT NULL); @@ -50,6 +56,13 @@ t1 CREATE TABLE `t1` ( PARTITION p3 VALUES LESS THAN (733969) ENGINE = MyISAM, PARTITION pmax VALUES LESS THAN MAXVALUE ENGINE = MyISAM) */ DROP TABLE t1; +create table t1 (a int NOT NULL, b varchar(5) NOT NULL) +default charset=utf8 +partition by list (a) +subpartition by key (b) +(partition p0 values in (1), +partition p1 values in (2)); +drop table t1; create table t1 (a int, b int, key(a)) partition by list (a) ( partition p0 values in (1), @@ -2045,10 +2058,15 @@ DROP TABLE t1; # # Bug #45807: crash accessing partitioned table and sql_mode # contains ONLY_FULL_GROUP_BY +# Bug#46923: select count(*) from partitioned table fails with +# ONLY_FULL_GROUP_BY # SET SESSION SQL_MODE='ONLY_FULL_GROUP_BY'; CREATE TABLE t1(id INT,KEY(id)) ENGINE=MYISAM PARTITION BY HASH(id) PARTITIONS 2; +SELECT COUNT(*) FROM t1; +COUNT(*) +0 DROP TABLE t1; SET SESSION SQL_MODE=DEFAULT; # === modified file 'mysql-test/r/query_cache.result' --- a/mysql-test/r/query_cache.result 2009-12-03 11:19:05 +0000 +++ b/mysql-test/r/query_cache.result 2010-01-11 13:15:28 +0000 @@ -1302,6 +1302,15 @@ drop procedure f3; drop procedure f4; drop table t1; set GLOBAL query_cache_size=0; +set GLOBAL query_cache_size=100000; +set SESSION query_cache_size=10000; +ERROR HY000: Variable 'query_cache_size' is a GLOBAL variable and should be set with SET GLOBAL +set global query_cache_limit=100; +set global query_cache_size=100; +set global query_cache_type=demand; +set GLOBAL query_cache_type=default; +set GLOBAL query_cache_limit=default; +set GLOBAL query_cache_size=default; End of 4.1 tests SET GLOBAL query_cache_size=102400; create table t1(a int); @@ -1707,6 +1716,95 @@ Variable_name Value Qcache_hits 2 DROP TABLE t1; SET GLOBAL query_cache_size= default; +#------------------------------------------------------------------------ +# Tests for Bug#6760 and Bug#12689 +SET @row_count = 4; +SET @sleep_time_per_result_row = 1; +SET @max_acceptable_delay = 2; +SET @@global.query_cache_size = 1024 * 64; +DROP TEMPORARY TABLE IF EXISTS t_history; +DROP TABLE IF EXISTS t1; +CREATE TEMPORARY TABLE t_history (attempt SMALLINT, +start_ts DATETIME, end_ts DATETIME, +start_cached INTEGER, end_cached INTEGER); +CREATE TABLE t1 (f1 BIGINT); +INSERT INTO t_history +SET attempt = 4 - 4 + 1, start_ts = NOW(), +start_cached = 0; +SELECT *, SLEEP(@sleep_time_per_result_row) FROM t1; +f1 SLEEP(@sleep_time_per_result_row) +1 0 +1 0 +1 0 +1 0 +UPDATE t_history SET end_ts = NOW() +WHERE attempt = 4 - 4 + 1; +UPDATE t_history SET end_cached = 0 +WHERE attempt = 4 - 4 + 1; +INSERT INTO t_history +SET attempt = 4 - 3 + 1, start_ts = NOW(), +start_cached = 0; +SELECT *, SLEEP(@sleep_time_per_result_row) FROM t1; +f1 SLEEP(@sleep_time_per_result_row) +1 0 +1 0 +1 0 +1 0 +UPDATE t_history SET end_ts = NOW() +WHERE attempt = 4 - 3 + 1; +UPDATE t_history SET end_cached = 0 +WHERE attempt = 4 - 3 + 1; +INSERT INTO t_history +SET attempt = 4 - 2 + 1, start_ts = NOW(), +start_cached = 0; +SELECT *, SLEEP(@sleep_time_per_result_row) FROM t1; +f1 SLEEP(@sleep_time_per_result_row) +1 0 +1 0 +1 0 +1 0 +UPDATE t_history SET end_ts = NOW() +WHERE attempt = 4 - 2 + 1; +UPDATE t_history SET end_cached = 0 +WHERE attempt = 4 - 2 + 1; +INSERT INTO t_history +SET attempt = 4 - 1 + 1, start_ts = NOW(), +start_cached = 0; +SELECT *, SLEEP(@sleep_time_per_result_row) FROM t1; +f1 SLEEP(@sleep_time_per_result_row) +1 0 +1 0 +1 0 +1 0 +UPDATE t_history SET end_ts = NOW() +WHERE attempt = 4 - 1 + 1; +UPDATE t_history SET end_cached = 0 +WHERE attempt = 4 - 1 + 1; +# Test 1: Does the query with SLEEP need a reasonable time? +SELECT COUNT(*) >= 4 - 1 INTO @aux1 FROM t_history +WHERE TIMEDIFF(end_ts,start_ts) - @sleep_time_per_result_row * @row_count +BETWEEN 0 AND @max_acceptable_delay; +SELECT @aux1 AS "Expect 1"; +Expect 1 +1 +# Test 2: Does the query with SLEEP need a reasonable time even in case +# of the non first execution? +SELECT COUNT(*) >= 4 - 1 - 1 INTO @aux2 FROM t_history +WHERE TIMEDIFF(end_ts,start_ts) - @sleep_time_per_result_row * @row_count +BETWEEN 0 AND @max_acceptable_delay +AND attempt > 1; +SELECT @aux2 AS "Expect 1"; +Expect 1 +1 +# Test 3: The query with SLEEP must be not cached. +SELECT COUNT(*) = 4 INTO @aux3 FROM t_history +WHERE end_cached = start_cached; +SELECT @aux3 AS "Expect 1"; +Expect 1 +1 +DROP TABLE t1; +DROP TEMPORARY TABLE t_history; +SET @@global.query_cache_size = default; End of 5.0 tests SET GLOBAL query_cache_size=1024*1024*512; CREATE TABLE t1 (a ENUM('rainbow')); === modified file 'mysql-test/r/query_cache_notembedded.result' --- a/mysql-test/r/query_cache_notembedded.result 2009-02-13 19:32:24 +0000 +++ b/mysql-test/r/query_cache_notembedded.result 2010-01-11 13:15:28 +0000 @@ -382,3 +382,55 @@ set GLOBAL query_cache_type=default; set GLOBAL query_cache_limit=default; set GLOBAL query_cache_min_res_unit=default; set GLOBAL query_cache_size=default; +drop table if exists t1| +create table t1 ( +id char(16) not null default '', +data int not null +)| +drop procedure if exists bug3583| +drop procedure if exists bug3583| +create procedure bug3583() +begin +declare c int; +select * from t1; +select count(*) into c from t1; +select c; +end| +insert into t1 values ("x", 3), ("y", 5)| +set @x = @@query_cache_size| +set global query_cache_size = 10*1024*1024| +flush status| +flush query cache| +show status like 'Qcache_hits'| +Variable_name Value +Qcache_hits 0 +call bug3583()| +id data +x 3 +y 5 +c +2 +show status like 'Qcache_hits'| +Variable_name Value +Qcache_hits 0 +call bug3583()| +id data +x 3 +y 5 +c +2 +call bug3583()| +id data +x 3 +y 5 +c +2 +show status like 'Qcache_hits'| +Variable_name Value +Qcache_hits 2 +set global query_cache_size = @x| +flush status| +flush query cache| +delete from t1| +drop procedure bug3583| +drop table t1| === modified file 'mysql-test/r/range.result' --- a/mysql-test/r/range.result 2009-11-02 12:24:07 +0000 +++ b/mysql-test/r/range.result 2009-12-08 09:26:11 +0000 @@ -1603,4 +1603,54 @@ SELECT str_to_date('', '%Y-%m-%d'); str_to_date('', '%Y-%m-%d') 0000-00-00 DROP TABLE t1, t2; +# +# Bug#48459: valgrind errors with query using 'Range checked for each +# record' +# +CREATE TABLE t1 ( +a INT, +b CHAR(2), +c INT, +d INT, +KEY ( c ), +KEY ( d, a, b ( 2 ) ), +KEY ( b ( 1 ) ) +); +INSERT INTO t1 VALUES ( NULL, 'a', 1, 2 ), ( NULL, 'a', 1, 2 ), +( 1, 'a', 1, 2 ), ( 1, 'a', 1, 2 ); +CREATE TABLE t2 ( +a INT, +c INT, +e INT, +KEY ( e ) +); +INSERT INTO t2 VALUES ( 1, 1, NULL ), ( 1, 1, NULL ); +# Should not give Valgrind warnings +SELECT 1 +FROM t1, t2 +WHERE t1.d <> '1' AND t1.b > '1' +AND t1.a = t2.a AND t1.c = t2.c; +1 +1 +1 +1 +1 +DROP TABLE t1, t2; +# +# Bug #48665: sql-bench's insert test fails due to wrong result +# +CREATE TABLE t1 (a INT, b INT, PRIMARY KEY (a)); +INSERT INTO t1 VALUES (0,0), (1,1); +EXPLAIN +SELECT * FROM t1 FORCE INDEX (PRIMARY) +WHERE (a>=1 AND a<=2) OR (a>=4 AND a<=5) OR (a>=0 AND a <=10); +id select_type table type possible_keys key key_len ref rows Extra +@ @ @ range @ @ @ @ @ @ +# Should return 2 rows +SELECT * FROM t1 FORCE INDEX (PRIMARY) +WHERE (a>=1 AND a<=2) OR (a>=4 AND a<=5) OR (a>=0 AND a <=10); +a b +0 0 +1 1 +DROP TABLE t1; End of 5.1 tests === modified file 'mysql-test/r/select.result' --- a/mysql-test/r/select.result 2009-12-03 11:19:05 +0000 +++ b/mysql-test/r/select.result 2010-01-15 15:27:55 +0000 @@ -4426,6 +4426,20 @@ ROW(a,a) <=> ROW((SELECT 1 FROM t1 WHERE INTO @var0; ERROR 21000: Subquery returns more than 1 row DROP TABLE t1; +# +# Bug #48458: simple query tries to allocate enormous amount of +# memory +# +CREATE TABLE t1(a INT NOT NULL, b YEAR); +INSERT INTO t1 VALUES (); +Warnings: +Warning 1364 Field 'a' doesn't have a default value +CREATE TABLE t2(c INT); +# Should not err out because of out-of-memory +SELECT 1 FROM t2 JOIN t1 ON 1=1 +WHERE a != '1' AND NOT a >= b OR NOT ROW(b,a )<> ROW(a,a); +1 +DROP TABLE t1,t2; End of 5.0 tests create table t1(a INT, KEY (a)); INSERT INTO t1 VALUES (1),(2),(3),(4),(5); @@ -4576,4 +4590,47 @@ field2 15:13:38 drop table A,AA,B,BB; #end of test for bug#45266 +# +# BUG#48052: Valgrind warning - uninitialized value in init_read_record() +# +CREATE TABLE t1 ( +pk int(11) NOT NULL, +i int(11) DEFAULT NULL, +v varchar(1) DEFAULT NULL, +PRIMARY KEY (pk) +); +INSERT INTO t1 VALUES (2,7,'m'); +INSERT INTO t1 VALUES (3,9,'m'); +SELECT v +FROM t1 +WHERE NOT pk > 0 +HAVING v <= 't' +ORDER BY pk; +v +DROP TABLE t1; +# +# Bug#49489 Uninitialized cache led to a wrong result. +# +CREATE TABLE t1(c1 DOUBLE(5,4)); +INSERT INTO t1 VALUES (9.1234); +SELECT * FROM t1 WHERE c1 < 9.12345; +c1 +9.1234 +DROP TABLE t1; +# End of test for bug#49489. +# +# Bug #49517: Inconsistent behavior while using +# NULLable BIGINT and INT columns in comparison +# +CREATE TABLE t1(a BIGINT UNSIGNED NOT NULL, b BIGINT NULL, c INT NULL); +INSERT INTO t1 VALUES(105, NULL, NULL); +SELECT * FROM t1 WHERE b < 102; +a b c +SELECT * FROM t1 WHERE c < 102; +a b c +SELECT * FROM t1 WHERE 102 < b; +a b c +SELECT * FROM t1 WHERE 102 < c; +a b c +DROP TABLE t1; End of 5.1 tests === modified file 'mysql-test/r/show_check.result' --- a/mysql-test/r/show_check.result 2009-03-06 14:56:17 +0000 +++ b/mysql-test/r/show_check.result 2010-01-27 21:53:08 +0000 @@ -252,13 +252,11 @@ drop table t1; flush tables; show open tables; Database Table In_use Name_locked -mysql general_log 0 0 create table t1(n int); insert into t1 values (1); show open tables; Database Table In_use Name_locked test t1 0 0 -mysql general_log 0 0 drop table t1; create table t1 (a int not null, b VARCHAR(10), INDEX (b) ) AVG_ROW_LENGTH=10 CHECKSUM=1 COMMENT="test" ENGINE=MYISAM MIN_ROWS=10 MAX_ROWS=100 PACK_KEYS=1 DELAY_KEY_WRITE=1 ROW_FORMAT=fixed; show create table t1; @@ -1454,4 +1452,10 @@ GRANT PROCESS ON *.* TO test_u@localhost SHOW ENGINE MYISAM MUTEX; SHOW ENGINE MYISAM STATUS; DROP USER test_u@localhost; +# +# Bug #48985: show create table crashes if previous access to the table +# was killed +# +SHOW CREATE TABLE non_existent; +ERROR 70100: Query execution was interrupted End of 5.1 tests === modified file 'mysql-test/r/sp-destruct.result' --- a/mysql-test/r/sp-destruct.result 2008-04-08 14:51:26 +0000 +++ b/mysql-test/r/sp-destruct.result 2009-11-21 11:18:21 +0000 @@ -1,3 +1,4 @@ +call mtr.add_suppression("Column count of mysql.proc is wrong. Expected 20, found 19. The table is probably corrupted"); use test; drop procedure if exists bug14233; drop function if exists bug14233; @@ -11,11 +12,13 @@ create table t1 (id int); create trigger t1_ai after insert on t1 for each row call bug14233(); alter table mysql.proc drop type; call bug14233(); -ERROR HY000: Failed to load routine test.bug14233. The table mysql.proc is missing, corrupt, or contains bad data (internal code -5) +ERROR HY000: Column count of mysql.proc is wrong. Expected 20, found 19. The table is probably corrupted create view v1 as select bug14233_f(); -ERROR HY000: Failed to load routine test.bug14233_f. The table mysql.proc is missing, corrupt, or contains bad data (internal code -5) +ERROR HY000: Column count of mysql.proc is wrong. Expected 20, found 19. The table is probably corrupted insert into t1 values (0); -ERROR HY000: Failed to load routine test.bug14233. The table mysql.proc is missing, corrupt, or contains bad data (internal code -5) +ERROR HY000: Column count of mysql.proc is wrong. Expected 20, found 19. The table is probably corrupted +show procedure status; +ERROR HY000: Column count of mysql.proc is wrong. Expected 20, found 19. The table is probably corrupted flush table mysql.proc; call bug14233(); ERROR HY000: Incorrect information in file: './mysql/proc.frm' @@ -88,3 +91,28 @@ show procedure status where db=DATABASE( Db Name Type Definer Modified Created Security_type Comment character_set_client collation_connection Database Collation show function status where db=DATABASE(); Db Name Type Definer Modified Created Security_type Comment character_set_client collation_connection Database Collation +DROP TABLE IF EXISTS proc_backup; +DROP PROCEDURE IF EXISTS p1; +# Backup the proc table +RENAME TABLE mysql.proc TO proc_backup; +CREATE TABLE mysql.proc LIKE proc_backup; +FLUSH TABLE mysql.proc; +# Test with a valid table. +CREATE PROCEDURE p1() +SET @foo = 10; +CALL p1(); +SHOW PROCEDURE STATUS; +Db Name Type Definer Modified Created Security_type Comment character_set_client collation_connection Database Collation +test p1 PROCEDURE root@localhost 0000-00-00 00:00:00 0000-00-00 00:00:00 DEFINER latin1 latin1_swedish_ci latin1_swedish_ci +# Modify a field of the table. +ALTER TABLE mysql.proc MODIFY comment CHAR (32); +CREATE PROCEDURE p2() +SET @foo = 10; +ERROR HY000: Cannot load from mysql.proc. The table is probably corrupted +# Procedure loaded from the cache +CALL p1(); +SHOW PROCEDURE STATUS; +ERROR HY000: Cannot load from mysql.proc. The table is probably corrupted +DROP TABLE mysql.proc; +RENAME TABLE proc_backup TO mysql.proc; +FLUSH TABLE mysql.proc; === modified file 'mysql-test/r/sp-security.result' --- a/mysql-test/r/sp-security.result 2009-03-06 14:56:17 +0000 +++ b/mysql-test/r/sp-security.result 2009-11-27 16:10:28 +0000 @@ -510,4 +510,60 @@ DROP USER mysqltest_u1@localhost; DROP PROCEDURE p_suid; DROP FUNCTION f_suid; DROP TABLE t1; +# +# Bug #48872 : Privileges for stored functions ignored if function name +# is mixed case +# +CREATE DATABASE B48872; +USE B48872; +CREATE TABLE `TestTab` (id INT); +INSERT INTO `TestTab` VALUES (1),(2); +CREATE FUNCTION `f_Test`() RETURNS INT RETURN 123; +CREATE FUNCTION `f_Test_denied`() RETURNS INT RETURN 123; +CREATE USER 'tester'; +CREATE USER 'Tester'; +GRANT SELECT ON TABLE `TestTab` TO 'tester'; +GRANT EXECUTE ON FUNCTION `f_Test` TO 'tester'; +GRANT EXECUTE ON FUNCTION `f_Test_denied` TO 'Tester'; +SELECT f_Test(); +f_Test() +123 +SELECT * FROM TestTab; +id +1 +2 +SELECT * FROM TestTab; +id +1 +2 +SELECT `f_Test`(); +`f_Test`() +123 +SELECT `F_TEST`(); +`F_TEST`() +123 +SELECT f_Test(); +f_Test() +123 +SELECT F_TEST(); +F_TEST() +123 +SELECT * FROM TestTab; +SELECT `f_Test`(); +SELECT `F_TEST`(); +SELECT f_Test(); +SELECT F_TEST(); +SELECT `f_Test_denied`(); +`f_Test_denied`() +123 +SELECT `F_TEST_DENIED`(); +`F_TEST_DENIED`() +123 +DROP TABLE `TestTab`; +DROP FUNCTION `f_Test`; +DROP FUNCTION `f_Test_denied`; +USE test; +DROP USER 'tester'; +DROP USER 'Tester'; +DROP DATABASE B48872; End of 5.0 tests. === modified file 'mysql-test/r/sp.result' --- a/mysql-test/r/sp.result 2009-10-23 13:54:58 +0000 +++ b/mysql-test/r/sp.result 2009-11-13 01:03:26 +0000 @@ -6979,6 +6979,64 @@ CALL p1; ERROR 42S22: Unknown column 'A.b' in 'IN/ALL/ANY subquery' DROP PROCEDURE p1; DROP TABLE t1, t2; +# +# Bug#47627: SET @@{global.session}.local_variable in stored routine causes crash +# Bug#48626: Crash or lost connection using SET for declared variables with @@ +# +DROP PROCEDURE IF EXISTS p1; +DROP PROCEDURE IF EXISTS p2; +DROP PROCEDURE IF EXISTS p3; +CREATE PROCEDURE p1() +BEGIN +DECLARE v INT DEFAULT 0; +SET @@SESSION.v= 10; +END// +ERROR HY000: Unknown system variable 'v' +CREATE PROCEDURE p2() +BEGIN +DECLARE v INT DEFAULT 0; +SET v= 10; +END// +call p2()// +CREATE PROCEDURE p3() +BEGIN +DECLARE v INT DEFAULT 0; +SELECT @@SESSION.v; +END// +ERROR HY000: Unknown system variable 'v' +CREATE PROCEDURE p4() +BEGIN +DECLARE v INT DEFAULT 0; +SET @@GLOBAL.v= 10; +END// +ERROR HY000: Unknown system variable 'v' +CREATE PROCEDURE p5() +BEGIN +DECLARE init_connect INT DEFAULT 0; +SET init_connect= 10; +SET @@GLOBAL.init_connect= 'SELECT 1'; +SET @@SESSION.IDENTITY= 1; +SELECT @@SESSION.IDENTITY; +SELECT @@GLOBAL.init_connect; +SELECT init_connect; +END// +CREATE PROCEDURE p6() +BEGIN +DECLARE v INT DEFAULT 0; +SET @@v= 0; +END// +ERROR HY000: Unknown system variable 'v' +SET @old_init_connect= @@GLOBAL.init_connect; +CALL p5(); +@@SESSION.IDENTITY +1 +@@GLOBAL.init_connect +SELECT 1 +init_connect +10 +SET @@GLOBAL.init_connect= @old_init_connect; +DROP PROCEDURE p2; +DROP PROCEDURE p5; # ------------------------------------------------------------------ # -- End of 5.1 tests # ------------------------------------------------------------------ === modified file 'mysql-test/r/sp_notembedded.result' --- a/mysql-test/r/sp_notembedded.result 2009-10-13 18:21:42 +0000 +++ b/mysql-test/r/sp_notembedded.result 2010-01-11 13:15:28 +0000 @@ -25,58 +25,6 @@ call bug4902_2()| show warnings| Level Code Message drop procedure bug4902_2| -drop table if exists t1| -create table t1 ( -id char(16) not null default '', -data int not null -)| -drop procedure if exists bug3583| -drop procedure if exists bug3583| -create procedure bug3583() -begin -declare c int; -select * from t1; -select count(*) into c from t1; -select c; -end| -insert into t1 values ("x", 3), ("y", 5)| -set @x = @@query_cache_size| -set global query_cache_size = 10*1024*1024| -flush status| -flush query cache| -show status like 'Qcache_hits'| -Variable_name Value -Qcache_hits 0 -call bug3583()| -id data -x 3 -y 5 -c -2 -show status like 'Qcache_hits'| -Variable_name Value -Qcache_hits 0 -call bug3583()| -id data -x 3 -y 5 -c -2 -call bug3583()| -id data -x 3 -y 5 -c -2 -show status like 'Qcache_hits'| -Variable_name Value -Qcache_hits 2 -set global query_cache_size = @x| -flush status| -flush query cache| -delete from t1| -drop procedure bug3583| -drop table t1| drop procedure if exists bug6807| create procedure bug6807() begin === modified file 'mysql-test/r/trigger.result' --- a/mysql-test/r/trigger.result 2009-06-22 12:51:33 +0000 +++ b/mysql-test/r/trigger.result 2010-01-12 08:19:48 +0000 @@ -1448,33 +1448,6 @@ isave 1 2 drop table t1, t2, t3; -CREATE TABLE t1 (id INTEGER); -CREATE TABLE t2 (id INTEGER); -INSERT INTO t2 VALUES (1),(2); -CREATE TRIGGER t1_test AFTER INSERT ON t1 FOR EACH ROW -INSERT INTO t2 VALUES (new.id); -SELECT GET_LOCK('B26162',120); -GET_LOCK('B26162',120) -1 -SELECT 'rl_acquirer', GET_LOCK('B26162',120), id FROM t2 WHERE id = 1; -SET SESSION LOW_PRIORITY_UPDATES=1; -SET GLOBAL LOW_PRIORITY_UPDATES=1; -INSERT INTO t1 VALUES (5); -SELECT 'rl_contender', id FROM t2 WHERE id > 1; -SELECT RELEASE_LOCK('B26162'); -RELEASE_LOCK('B26162') -1 -rl_acquirer GET_LOCK('B26162',120) id -rl_acquirer 1 1 -SELECT RELEASE_LOCK('B26162'); -RELEASE_LOCK('B26162') -1 -rl_contender id -rl_contender 2 -DROP TRIGGER t1_test; -DROP TABLE t1,t2; -SET SESSION LOW_PRIORITY_UPDATES=DEFAULT; -SET GLOBAL LOW_PRIORITY_UPDATES=DEFAULT; Bug#28502 Triggers that update another innodb table will block on X lock unnecessarily === modified file 'mysql-test/r/trigger_notembedded.result' --- a/mysql-test/r/trigger_notembedded.result 2009-09-17 11:33:23 +0000 +++ b/mysql-test/r/trigger_notembedded.result 2010-01-12 08:19:48 +0000 @@ -445,6 +445,33 @@ DROP TABLE t2; DROP TABLE t1; DROP DATABASE mysqltest_db1; USE test; +CREATE TABLE t1 (id INTEGER); +CREATE TABLE t2 (id INTEGER); +INSERT INTO t2 VALUES (1),(2); +CREATE TRIGGER t1_test AFTER INSERT ON t1 FOR EACH ROW +INSERT INTO t2 VALUES (new.id); +SELECT GET_LOCK('B26162',120); +GET_LOCK('B26162',120) +1 +SELECT 'rl_acquirer', GET_LOCK('B26162',120), id FROM t2 WHERE id = 1; +SET SESSION LOW_PRIORITY_UPDATES=1; +SET GLOBAL LOW_PRIORITY_UPDATES=1; +INSERT INTO t1 VALUES (5); +SELECT 'rl_contender', id FROM t2 WHERE id > 1; +SELECT RELEASE_LOCK('B26162'); +RELEASE_LOCK('B26162') +1 +rl_acquirer GET_LOCK('B26162',120) id +rl_acquirer 1 1 +SELECT RELEASE_LOCK('B26162'); +RELEASE_LOCK('B26162') +1 +rl_contender id +rl_contender 2 +DROP TRIGGER t1_test; +DROP TABLE t1,t2; +SET SESSION LOW_PRIORITY_UPDATES=DEFAULT; +SET GLOBAL LOW_PRIORITY_UPDATES=DEFAULT; End of 5.0 tests. drop table if exists t1; create table t1 (i int); === modified file 'mysql-test/r/type_newdecimal.result' --- a/mysql-test/r/type_newdecimal.result 2009-11-02 11:21:39 +0000 +++ b/mysql-test/r/type_newdecimal.result 2009-12-08 09:26:11 +0000 @@ -1630,3 +1630,287 @@ SELECT my_col FROM t1; my_col 0.012345687012345687012345687012 DROP TABLE t1; +# +# Bug#45261: Crash, stored procedure + decimal +# +DROP TABLE IF EXISTS t1; +CREATE TABLE t1 SELECT +/* 81 */ 100000000000000000000000000000000000000000000000000000000000000000000000000000001 +AS c1; +Warnings: +Warning 1264 Out of range value for column 'c1' at row 1 +DESC t1; +Field Type Null Key Default Extra +c1 decimal(65,0) NO 0 +SELECT * FROM t1; +c1 +99999999999999999999999999999999999999999999999999999999999999999 +DROP TABLE t1; +CREATE TABLE t1 SELECT +/* 81 */ 100000000000000000000000000000000000000000000000000000000000000000000000000000001. +AS c1; +Warnings: +Warning 1264 Out of range value for column 'c1' at row 1 +DESC t1; +Field Type Null Key Default Extra +c1 decimal(65,0) NO 0 +SELECT * FROM t1; +c1 +99999999999999999999999999999999999999999999999999999999999999999 +DROP TABLE t1; +CREATE TABLE t1 SELECT +/* 81 */ 100000000000000000000000000000000000000000000000000000000000000000000000000000001.1 /* 1 */ +AS c1; +Warnings: +Warning 1264 Out of range value for column 'c1' at row 1 +DESC t1; +Field Type Null Key Default Extra +c1 decimal(65,0) NO 0 +SELECT * FROM t1; +c1 +99999999999999999999999999999999999999999999999999999999999999999 +DROP TABLE t1; +CREATE TABLE t1 SELECT +/* 82 */ 1000000000000000000000000000000000000000000000000000000000000000000000000000000001 +AS c1; +Warnings: +Error 1292 Truncated incorrect DECIMAL value: '' +DESC t1; +Field Type Null Key Default Extra +c1 decimal(65,0) NO 0 +SELECT * FROM t1; +c1 +99999999999999999999999999999999999999999999999999999999999999999 +DROP TABLE t1; +CREATE TABLE t1 SELECT +/* 40 */ 1000000000000000000000000000000000000001.1000000000000000000000000000000000000001 /* 40 */ +AS c1; +Warnings: +Warning 1264 Out of range value for column 'c1' at row 1 +DESC t1; +Field Type Null Key Default Extra +c1 decimal(65,30) NO 0.000000000000000000000000000000 +SELECT * FROM t1; +c1 +99999999999999999999999999999999999.999999999999999999999999999999 +DROP TABLE t1; +CREATE TABLE t1 SELECT +/* 1 */ 1.10000000000000000000000000000000000000000000000000000000000000000000000000000001 /* 80 */ +AS c1; +DESC t1; +Field Type Null Key Default Extra +c1 decimal(31,30) NO 0.000000000000000000000000000000 +SELECT * FROM t1; +c1 +1.100000000000000000000000000000 +DROP TABLE t1; +CREATE TABLE t1 SELECT +/* 1 */ 1.100000000000000000000000000000000000000000000000000000000000000000000000000000001 /* 81 */ +AS c1; +DESC t1; +Field Type Null Key Default Extra +c1 decimal(31,30) NO 0.000000000000000000000000000000 +SELECT * FROM t1; +c1 +1.100000000000000000000000000000 +DROP TABLE t1; +CREATE TABLE t1 SELECT +.100000000000000000000000000000000000000000000000000000000000000000000000000000001 /* 81 */ +AS c1; +Warnings: +Note 1265 Data truncated for column 'c1' at row 1 +DESC t1; +Field Type Null Key Default Extra +c1 decimal(30,30) NO 0.000000000000000000000000000000 +SELECT * FROM t1; +c1 +0.100000000000000000000000000000 +DROP TABLE t1; +CREATE TABLE t1 SELECT +/* 45 */ 123456789012345678901234567890123456789012345.123456789012345678901234567890123456789012345 /* 45 */ +AS c1; +Warnings: +Warning 1264 Out of range value for column 'c1' at row 1 +DESC t1; +Field Type Null Key Default Extra +c1 decimal(65,30) NO 0.000000000000000000000000000000 +SELECT * FROM t1; +c1 +99999999999999999999999999999999999.999999999999999999999999999999 +DROP TABLE t1; +CREATE TABLE t1 SELECT +/* 65 */ 12345678901234567890123456789012345678901234567890123456789012345.1 /* 1 */ +AS c1; +Warnings: +Warning 1264 Out of range value for column 'c1' at row 1 +DESC t1; +Field Type Null Key Default Extra +c1 decimal(65,1) NO 0.0 +SELECT * FROM t1; +c1 +9999999999999999999999999999999999999999999999999999999999999999.9 +DROP TABLE t1; +CREATE TABLE t1 SELECT +/* 66 */ 123456789012345678901234567890123456789012345678901234567890123456.1 /* 1 */ +AS c1; +Warnings: +Warning 1264 Out of range value for column 'c1' at row 1 +DESC t1; +Field Type Null Key Default Extra +c1 decimal(65,1) NO 0.0 +SELECT * FROM t1; +c1 +9999999999999999999999999999999999999999999999999999999999999999.9 +DROP TABLE t1; +CREATE TABLE t1 SELECT +.123456789012345678901234567890123456789012345678901234567890123456 /* 66 */ +AS c1; +Warnings: +Note 1265 Data truncated for column 'c1' at row 1 +DESC t1; +Field Type Null Key Default Extra +c1 decimal(30,30) NO 0.000000000000000000000000000000 +SELECT * FROM t1; +c1 +0.123456789012345678901234567890 +DROP TABLE t1; +CREATE TABLE t1 AS SELECT 123.1234567890123456789012345678901 /* 31 */ AS c1; +Warnings: +Note 1265 Data truncated for column 'c1' at row 1 +DESC t1; +Field Type Null Key Default Extra +c1 decimal(33,30) NO 0.000000000000000000000000000000 +SELECT * FROM t1; +c1 +123.123456789012345678901234567890 +DROP TABLE t1; +CREATE TABLE t1 SELECT 1.1 + CAST(1 AS DECIMAL(65,30)) AS c1; +DESC t1; +Field Type Null Key Default Extra +c1 decimal(65,30) NO 0.000000000000000000000000000000 +SELECT * FROM t1; +c1 +2.100000000000000000000000000000 +DROP TABLE t1; +# +# Test that the integer and decimal parts are properly calculated. +# +CREATE TABLE t1 (a DECIMAL(30,30)); +INSERT INTO t1 VALUES (0.1),(0.2),(0.3); +CREATE TABLE t2 SELECT MIN(a + 0.0000000000000000000000000000001) AS c1 FROM t1; +Warnings: +Note 1265 Data truncated for column 'c1' at row 3 +DESC t2; +Field Type Null Key Default Extra +c1 decimal(32,30) YES NULL +DROP TABLE t1,t2; +CREATE TABLE t1 (a DECIMAL(30,30)); +INSERT INTO t1 VALUES (0.1),(0.2),(0.3); +CREATE TABLE t2 SELECT IFNULL(a + 0.0000000000000000000000000000001, NULL) AS c1 FROM t1; +Warnings: +Note 1265 Data truncated for column 'c1' at row 1 +Note 1265 Data truncated for column 'c1' at row 2 +Note 1265 Data truncated for column 'c1' at row 3 +DESC t2; +Field Type Null Key Default Extra +c1 decimal(34,0) YES NULL +DROP TABLE t1,t2; +CREATE TABLE t1 (a DECIMAL(30,30)); +INSERT INTO t1 VALUES (0.1),(0.2),(0.3); +CREATE TABLE t2 SELECT CASE a WHEN 0.1 THEN 0.0000000000000000000000000000000000000000000000000000000000000000001 END AS c1 FROM t1; +Warnings: +Note 1265 Data truncated for column 'c1' at row 1 +DESC t2; +Field Type Null Key Default Extra +c1 decimal(65,30) YES NULL +DROP TABLE t1,t2; +# +# Test that variables get maximum precision. +# +SET @decimal= 1.1; +CREATE TABLE t1 SELECT @decimal AS c1; +DESC t1; +Field Type Null Key Default Extra +c1 decimal(65,30) YES NULL +SELECT * FROM t1; +c1 +1.100000000000000000000000000000 +DROP TABLE t1; +# +# Bug #45261 : Crash, stored procedure + decimal +# Original test by the reporter. +# +# should not crash +CREATE TABLE t1 +SELECT .123456789012345678901234567890123456789012345678901234567890123456 AS a; +Warnings: +Note 1265 Data truncated for column 'a' at row 1 +DROP TABLE t1; +CREATE PROCEDURE test_proc() +BEGIN +# The las non critical CUSER definition is: +# DECLARE mycursor CURSOR FOR SELECT 1 % +# .12345678912345678912345678912345678912345678912345678912345678912 AS my_col; +DECLARE mycursor CURSOR FOR +SELECT 1 % +.123456789123456789123456789123456789123456789123456789123456789123456789123456789 +AS my_col; +OPEN mycursor; +CLOSE mycursor; +END| +# should not crash +CALL test_proc(); +DROP PROCEDURE test_proc; +# +# Bug #48370 Absolutely wrong calculations with GROUP BY and +# decimal fields when using IF +# +CREATE TABLE currencies (id int, rate decimal(16,4), +PRIMARY KEY (id), KEY (rate)); +INSERT INTO currencies VALUES (11,0.7028); +INSERT INTO currencies VALUES (1,1); +CREATE TABLE payments ( +id int, +supplier_id int, +status int, +currency_id int, +vat decimal(7,4), +PRIMARY KEY (id), +KEY currency_id (currency_id), +KEY supplier_id (supplier_id) +); +INSERT INTO payments (id,status,vat,supplier_id,currency_id) VALUES +(3001,2,0.0000,344,11), (1,2,0.0000,1,1); +CREATE TABLE sub_tasks ( +id int, +currency_id int, +price decimal(16,4), +discount decimal(10,4), +payment_id int, +PRIMARY KEY (id), +KEY currency_id (currency_id), +KEY payment_id (payment_id) +) ; +INSERT INTO sub_tasks (id, price, discount, payment_id, currency_id) VALUES +(52, 12.60, 0, 3001, 11), (56, 14.58, 0, 3001, 11); +# should return 1 and the same values in col 2 and 3 +select STRAIGHT_JOIN +(1 + PAY.vat) AS mult, +SUM(ROUND((SUB.price - ROUND(ROUND(SUB.price, 2) * SUB.discount, 2)) * +CUR.rate / CUR.rate, 2) +) v_net_with_discount, +SUM(ROUND((SUB.price - ROUND(ROUND(SUB.price, 2) * SUB.discount, 1)) * +CUR.rate / CUR.rate , 2) +* (1 + PAY.vat) +) v_total +from +currencies CUR, payments PAY, sub_tasks SUB +where +SUB.payment_id = PAY.id and +PAY.currency_id = CUR.id and +PAY.id > 2 +group by PAY.id + 1; +mult v_net_with_discount v_total +1.0000 27.18 27.180000 +DROP TABLE currencies, payments, sub_tasks; +End of 5.1 tests === modified file 'mysql-test/r/type_year.result' --- a/mysql-test/r/type_year.result 2007-03-29 04:08:30 +0000 +++ b/mysql-test/r/type_year.result 2009-12-15 08:37:10 +0000 @@ -46,3 +46,267 @@ a 2001 drop table t1; End of 5.0 tests +# +# Bug #49480: WHERE using YEAR columns returns unexpected results +# +CREATE TABLE t2(yy YEAR(2), c2 CHAR(4)); +CREATE TABLE t4(yyyy YEAR(4), c4 CHAR(4)); +INSERT INTO t2 (c2) VALUES (NULL),(1970),(1999),(2000),(2001),(2069); +INSERT INTO t4 (c4) SELECT c2 FROM t2; +UPDATE t2 SET yy = c2; +UPDATE t4 SET yyyy = c4; +SELECT * FROM t2; +yy c2 +NULL NULL +70 1970 +99 1999 +00 2000 +01 2001 +69 2069 +SELECT * FROM t4; +yyyy c4 +NULL NULL +1970 1970 +1999 1999 +2000 2000 +2001 2001 +2069 2069 +# Comparison of YEAR(2) with YEAR(4) +SELECT * FROM t2, t4 WHERE yy = yyyy; +yy c2 yyyy c4 +70 1970 1970 1970 +99 1999 1999 1999 +00 2000 2000 2000 +01 2001 2001 2001 +69 2069 2069 2069 +SELECT * FROM t2, t4 WHERE yy <=> yyyy; +yy c2 yyyy c4 +NULL NULL NULL NULL +70 1970 1970 1970 +99 1999 1999 1999 +00 2000 2000 2000 +01 2001 2001 2001 +69 2069 2069 2069 +SELECT * FROM t2, t4 WHERE yy < yyyy; +yy c2 yyyy c4 +70 1970 1999 1999 +70 1970 2000 2000 +99 1999 2000 2000 +70 1970 2001 2001 +99 1999 2001 2001 +00 2000 2001 2001 +70 1970 2069 2069 +99 1999 2069 2069 +00 2000 2069 2069 +01 2001 2069 2069 +SELECT * FROM t2, t4 WHERE yy > yyyy; +yy c2 yyyy c4 +99 1999 1970 1970 +00 2000 1970 1970 +01 2001 1970 1970 +69 2069 1970 1970 +00 2000 1999 1999 +01 2001 1999 1999 +69 2069 1999 1999 +01 2001 2000 2000 +69 2069 2000 2000 +69 2069 2001 2001 +# Comparison of YEAR(2) with YEAR(2) +SELECT * FROM t2 a, t2 b WHERE a.yy = b.yy; +yy c2 yy c2 +70 1970 70 1970 +99 1999 99 1999 +00 2000 00 2000 +01 2001 01 2001 +69 2069 69 2069 +SELECT * FROM t2 a, t2 b WHERE a.yy <=> b.yy; +yy c2 yy c2 +NULL NULL NULL NULL +70 1970 70 1970 +99 1999 99 1999 +00 2000 00 2000 +01 2001 01 2001 +69 2069 69 2069 +SELECT * FROM t2 a, t2 b WHERE a.yy < b.yy; +yy c2 yy c2 +70 1970 99 1999 +70 1970 00 2000 +99 1999 00 2000 +70 1970 01 2001 +99 1999 01 2001 +00 2000 01 2001 +70 1970 69 2069 +99 1999 69 2069 +00 2000 69 2069 +01 2001 69 2069 +# Comparison of YEAR(4) with YEAR(4) +SELECT * FROM t4 a, t4 b WHERE a.yyyy = b.yyyy; +yyyy c4 yyyy c4 +1970 1970 1970 1970 +1999 1999 1999 1999 +2000 2000 2000 2000 +2001 2001 2001 2001 +2069 2069 2069 2069 +SELECT * FROM t4 a, t4 b WHERE a.yyyy <=> b.yyyy; +yyyy c4 yyyy c4 +NULL NULL NULL NULL +1970 1970 1970 1970 +1999 1999 1999 1999 +2000 2000 2000 2000 +2001 2001 2001 2001 +2069 2069 2069 2069 +SELECT * FROM t4 a, t4 b WHERE a.yyyy < b.yyyy; +yyyy c4 yyyy c4 +1970 1970 1999 1999 +1970 1970 2000 2000 +1999 1999 2000 2000 +1970 1970 2001 2001 +1999 1999 2001 2001 +2000 2000 2001 2001 +1970 1970 2069 2069 +1999 1999 2069 2069 +2000 2000 2069 2069 +2001 2001 2069 2069 +# Comparison with constants: +SELECT * FROM t2 WHERE yy = NULL; +yy c2 +SELECT * FROM t4 WHERE yyyy = NULL; +yyyy c4 +SELECT * FROM t2 WHERE yy <=> NULL; +yy c2 +NULL NULL +SELECT * FROM t4 WHERE yyyy <=> NULL; +yyyy c4 +NULL NULL +SELECT * FROM t2 WHERE yy < NULL; +yy c2 +SELECT * FROM t2 WHERE yy > NULL; +yy c2 +SELECT * FROM t2 WHERE yy = NOW(); +yy c2 +SELECT * FROM t4 WHERE yyyy = NOW(); +yyyy c4 +SELECT * FROM t2 WHERE yy = 99; +yy c2 +99 1999 +SELECT * FROM t2 WHERE 99 = yy; +yy c2 +99 1999 +SELECT * FROM t4 WHERE yyyy = 99; +yyyy c4 +1999 1999 +SELECT * FROM t2 WHERE yy = 'test'; +yy c2 +00 2000 +Warnings: +Warning 1292 Truncated incorrect DOUBLE value: 'test' +SELECT * FROM t4 WHERE yyyy = 'test'; +yyyy c4 +Warnings: +Warning 1292 Truncated incorrect DOUBLE value: 'test' +SELECT * FROM t2 WHERE yy = '1999'; +yy c2 +99 1999 +SELECT * FROM t4 WHERE yyyy = '1999'; +yyyy c4 +1999 1999 +SELECT * FROM t2 WHERE yy = 1999; +yy c2 +99 1999 +SELECT * FROM t4 WHERE yyyy = 1999; +yyyy c4 +1999 1999 +SELECT * FROM t2 WHERE yy = 1999.1; +yy c2 +99 1999 +SELECT * FROM t4 WHERE yyyy = 1999.1; +yyyy c4 +1999 1999 +SELECT * FROM t2 WHERE yy = 1998.9; +yy c2 +99 1999 +SELECT * FROM t4 WHERE yyyy = 1998.9; +yyyy c4 +1999 1999 +# Coverage tests for YEAR with zero/2000 constants: +SELECT * FROM t2 WHERE yy = 0; +yy c2 +00 2000 +SELECT * FROM t2 WHERE yy = '0'; +yy c2 +00 2000 +SELECT * FROM t2 WHERE yy = '0000'; +yy c2 +00 2000 +SELECT * FROM t2 WHERE yy = '2000'; +yy c2 +00 2000 +SELECT * FROM t2 WHERE yy = 2000; +yy c2 +00 2000 +SELECT * FROM t4 WHERE yyyy = 0; +yyyy c4 +SELECT * FROM t4 WHERE yyyy = '0'; +yyyy c4 +2000 2000 +SELECT * FROM t4 WHERE yyyy = '0000'; +yyyy c4 +SELECT * FROM t4 WHERE yyyy = '2000'; +yyyy c4 +2000 2000 +SELECT * FROM t4 WHERE yyyy = 2000; +yyyy c4 +2000 2000 +# Comparison with constants those are out of YEAR range +# (coverage test for backward compatibility) +SELECT COUNT(yy) FROM t2; +COUNT(yy) +5 +SELECT COUNT(yyyy) FROM t4; +COUNT(yyyy) +5 +SELECT COUNT(*) FROM t2 WHERE yy = -1; +COUNT(*) +0 +SELECT COUNT(*) FROM t4 WHERE yyyy > -1; +COUNT(*) +5 +SELECT COUNT(*) FROM t2 WHERE yy > -1000000000000000000; +COUNT(*) +5 +SELECT COUNT(*) FROM t4 WHERE yyyy > -1000000000000000000; +COUNT(*) +5 +SELECT COUNT(*) FROM t2 WHERE yy < 2156; +COUNT(*) +5 +SELECT COUNT(*) FROM t4 WHERE yyyy < 2156; +COUNT(*) +5 +SELECT COUNT(*) FROM t2 WHERE yy < 1000000000000000000; +COUNT(*) +5 +SELECT COUNT(*) FROM t4 WHERE yyyy < 1000000000000000000; +COUNT(*) +5 +SELECT * FROM t2 WHERE yy < 123; +yy c2 +70 1970 +99 1999 +00 2000 +01 2001 +69 2069 +SELECT * FROM t2 WHERE yy > 123; +yy c2 +SELECT * FROM t4 WHERE yyyy < 123; +yyyy c4 +SELECT * FROM t4 WHERE yyyy > 123; +yyyy c4 +1970 1970 +1999 1999 +2000 2000 +2001 2001 +2069 2069 +DROP TABLE t2, t4; +# +End of 5.1 tests === modified file 'mysql-test/r/udf.result' --- a/mysql-test/r/udf.result 2009-09-07 09:57:22 +0000 +++ b/mysql-test/r/udf.result 2010-01-22 21:19:21 +0000 @@ -38,8 +38,6 @@ ERROR HY000: Can't initialize function ' select reverse_lookup("127.0.0.1"); select reverse_lookup(127,0,0,1); select reverse_lookup("localhost"); -reverse_lookup("localhost") -NULL select avgcost(); ERROR HY000: Can't initialize function 'avgcost'; wrong number of arguments: AVGCOST() requires two arguments select avgcost(100,23.76); @@ -311,29 +309,6 @@ drop function f3; drop function metaphon; drop function myfunc_double; drop function myfunc_int; -CREATE FUNCTION metaphon RETURNS STRING SONAME "UDF_EXAMPLE_LIB"; -create table t1 (a char); -set GLOBAL query_cache_size=1355776; -reset query cache; -select metaphon('MySQL') from t1; -metaphon('MySQL') -show status like "Qcache_hits"; -Variable_name Value -Qcache_hits 0 -show status like "Qcache_queries_in_cache"; -Variable_name Value -Qcache_queries_in_cache 0 -select metaphon('MySQL') from t1; -metaphon('MySQL') -show status like "Qcache_hits"; -Variable_name Value -Qcache_hits 0 -show status like "Qcache_queries_in_cache"; -Variable_name Value -Qcache_queries_in_cache 0 -drop table t1; -drop function metaphon; -set GLOBAL query_cache_size=default; DROP DATABASE IF EXISTS mysqltest; CREATE DATABASE mysqltest; USE mysqltest; === added file 'mysql-test/r/udf_query_cache.result' --- a/mysql-test/r/udf_query_cache.result 1970-01-01 00:00:00 +0000 +++ b/mysql-test/r/udf_query_cache.result 2010-01-11 13:15:28 +0000 @@ -0,0 +1,24 @@ +drop table if exists t1; +CREATE FUNCTION metaphon RETURNS STRING SONAME "UDF_EXAMPLE_LIB"; +create table t1 (a char); +set GLOBAL query_cache_size=1355776; +reset query cache; +select metaphon('MySQL') from t1; +metaphon('MySQL') +show status like "Qcache_hits"; +Variable_name Value +Qcache_hits 0 +show status like "Qcache_queries_in_cache"; +Variable_name Value +Qcache_queries_in_cache 0 +select metaphon('MySQL') from t1; +metaphon('MySQL') +show status like "Qcache_hits"; +Variable_name Value +Qcache_hits 0 +show status like "Qcache_queries_in_cache"; +Variable_name Value +Qcache_queries_in_cache 0 +drop table t1; +drop function metaphon; +set GLOBAL query_cache_size=default; === modified file 'mysql-test/r/variables.result' --- a/mysql-test/r/variables.result 2009-09-15 10:46:35 +0000 +++ b/mysql-test/r/variables.result 2010-01-11 13:15:28 +0000 @@ -19,8 +19,6 @@ set @my_myisam_max_sort_file_size =@@glo set @my_net_buffer_length =@@global.net_buffer_length; set @my_net_write_timeout =@@global.net_write_timeout; set @my_net_read_timeout =@@global.net_read_timeout; -set @my_query_cache_limit =@@global.query_cache_limit; -set @my_query_cache_type =@@global.query_cache_type; set @my_rpl_recovery_rank =@@global.rpl_recovery_rank; set @my_server_id =@@global.server_id; set @my_slow_launch_time =@@global.slow_launch_time; @@ -215,7 +213,6 @@ storage_engine MRG_MYISAM select * from information_schema.global_variables where variable_name like 'storage_engine'; VARIABLE_NAME VARIABLE_VALUE STORAGE_ENGINE MRG_MYISAM -set GLOBAL query_cache_size=100000; set GLOBAL myisam_max_sort_file_size=2000000; show global variables like 'myisam_max_sort_file_size'; Variable_name Value @@ -423,8 +420,6 @@ ERROR 42000: Variable 'big_tables' can't show local variables like 'storage_engine'; Variable_name Value storage_engine MEMORY -set SESSION query_cache_size=10000; -ERROR HY000: Variable 'query_cache_size' is a GLOBAL variable and should be set with SET GLOBAL set GLOBAL storage_engine=DEFAULT; ERROR 42000: Variable 'storage_engine' doesn't have a default value set character_set_client=UNKNOWN_CHARACTER_SET; @@ -529,9 +524,6 @@ Warnings: Warning 1292 Truncated incorrect net_buffer_length value: '100' set net_read_timeout=100; set net_write_timeout=100; -set global query_cache_limit=100; -set global query_cache_size=100; -set global query_cache_type=demand; set read_buffer_size=100; Warnings: Warning 1292 Truncated incorrect read_buffer_size value: '100' @@ -1047,8 +1039,6 @@ set global myisam_max_sort_file_size =@m set global net_buffer_length =@my_net_buffer_length; set global net_write_timeout =@my_net_write_timeout; set global net_read_timeout =@my_net_read_timeout; -set global query_cache_limit =@my_query_cache_limit; -set global query_cache_type =@my_query_cache_type; set global rpl_recovery_rank =@my_rpl_recovery_rank; set global server_id =@my_server_id; set global slow_launch_time =@my_slow_launch_time; === added file 'mysql-test/std_data/bug47012.ARM' Binary files a/mysql-test/std_data/bug47012.ARM 1970-01-01 00:00:00 +0000 and b/mysql-test/std_data/bug47012.ARM 2009-11-11 08:03:29 +0000 differ === added file 'mysql-test/std_data/bug47012.ARZ' Binary files a/mysql-test/std_data/bug47012.ARZ 1970-01-01 00:00:00 +0000 and b/mysql-test/std_data/bug47012.ARZ 2009-11-11 08:03:29 +0000 differ === added file 'mysql-test/std_data/bug47012.frm' Binary files a/mysql-test/std_data/bug47012.frm 1970-01-01 00:00:00 +0000 and b/mysql-test/std_data/bug47012.frm 2009-11-11 08:03:29 +0000 differ === modified file 'mysql-test/std_data/cacert.pem' --- a/mysql-test/std_data/cacert.pem 2009-06-11 16:21:32 +0000 +++ b/mysql-test/std_data/cacert.pem 2010-01-29 10:42:31 +0000 @@ -1,17 +1,19 @@ -----BEGIN CERTIFICATE----- -MIICrTCCAhagAwIBAgIJAJXpePU0UOTVMA0GCSqGSIb3DQEBBQUAMEQxCzAJBgNV -BAYTAlNFMRAwDgYDVQQIEwdVcHBzYWxhMRAwDgYDVQQHEwdVcHBzYWxhMREwDwYD -VQQKEwhNeVNRTCBBQjAeFw0wOTAxMjgxMDQ5NDZaFw0xNDAxMjcxMDQ5NDZaMEQx -CzAJBgNVBAYTAlNFMRAwDgYDVQQIEwdVcHBzYWxhMRAwDgYDVQQHEwdVcHBzYWxh -MREwDwYDVQQKEwhNeVNRTCBBQjCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEA -4XQHAe5R1+TXC8noZtWf+d5E0v1C59FWpn9SWEUCBjE5UiIwuJvi4Y+7xWGOXLAI -/JzJx5gNXLBiTsE/zh0uX9fKlajLhxB0GN+QU0ZlpQ1BeYipEcNXeI/7cT499f6v -XWabnTflivdCgHSWUOQ20/Lzs6kP6/e6OoZd/DPSjPECAwEAAaOBpjCBozAdBgNV -HQ4EFgQU8uLqVWWkmuKsnZf1RWz294wRrd8wdAYDVR0jBG0wa4AU8uLqVWWkmuKs -nZf1RWz294wRrd+hSKRGMEQxCzAJBgNVBAYTAlNFMRAwDgYDVQQIEwdVcHBzYWxh -MRAwDgYDVQQHEwdVcHBzYWxhMREwDwYDVQQKEwhNeVNRTCBBQoIJAJXpePU0UOTV -MAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcNAQEFBQADgYEAMMTE5sDN+Z0ZlV7KvH3g -6+aKvql8dTpRT3hYukeQlWua0nq74WPGVw0c4e/M/vbiMwmJcCYpB9pd4+dHqzSw -aPyoenjY6UF8n7B4quWy3SIUk2LSHeJLW+kzJn2afN9gvipFhdVh/uU2TIyLGOur -Z/vmJX2W7hF1uqPnbfa8Lrw= +MIIDIjCCAougAwIBAgIJAJhuvLP+2mGwMA0GCSqGSIb3DQEBBQUAMGoxCzAJBgNV +BAYTAkZJMRAwDgYDVQQIEwdUdXVzdWxhMRkwFwYDVQQKExBNb250eSBQcm9ncmFt +IEFiMS4wLAYJKoZIhvcNAQkBFh9hYnN0cmFjdC5kZXZlbG9wZXJAYXNrbW9udHku +b3JnMB4XDTEwMDEyODIxNTcyNVoXDTEwMDIyNzIxNTcyNVowajELMAkGA1UEBhMC +RkkxEDAOBgNVBAgTB1R1dXN1bGExGTAXBgNVBAoTEE1vbnR5IFByb2dyYW0gQWIx +LjAsBgkqhkiG9w0BCQEWH2Fic3RyYWN0LmRldmVsb3BlckBhc2ttb250eS5vcmcw +gZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBAMIaY4pwbst50S32xJH3bSXhPGep +6gx1AWwZKsHTXL3VeMO6PHmC8zu5HM0zbOcrIJcXL3YVnpmE4b9OQxIiMSx1Yd+U +u8/sTkxgpsEKhCbIzECIwPhppyT/JP5aSXCadEvg+PSjikv8dOVkD68wVG4CcFIX +MFttsPebBVzEokZZAgMBAAGjgc8wgcwwHQYDVR0OBBYEFOCKaNHFFPrju8AwzWxS +f96IKfRwMIGcBgNVHSMEgZQwgZGAFOCKaNHFFPrju8AwzWxSf96IKfRwoW6kbDBq +MQswCQYDVQQGEwJGSTEQMA4GA1UECBMHVHV1c3VsYTEZMBcGA1UEChMQTW9udHkg +UHJvZ3JhbSBBYjEuMCwGCSqGSIb3DQEJARYfYWJzdHJhY3QuZGV2ZWxvcGVyQGFz +a21vbnR5Lm9yZ4IJAJhuvLP+2mGwMAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcNAQEF +BQADgYEAsmCX2/k9AInq2qhXtnkLip6cB0iOerLTNAzEijZc/aVf4wUjkL3cqhmC +kSTCwAHIOxp+ICwh6ky3xghXjoI9QnPFDVkRkzPT2tV0IoBaeQuI4e0CU2EY7L3P +XoDqp3oq1XtVcr9ZZdP68fBYUG/qcrWcXWk45ZFaBmBv3TotsGk= -----END CERTIFICATE----- === modified file 'mysql-test/std_data/client-cert.pem' --- a/mysql-test/std_data/client-cert.pem 2009-06-11 16:21:32 +0000 +++ b/mysql-test/std_data/client-cert.pem 2010-01-29 10:42:31 +0000 @@ -1,55 +1,60 @@ Certificate: Data: Version: 3 (0x2) - Serial Number: 3 (0x3) + Serial Number: 2 (0x2) Signature Algorithm: sha1WithRSAEncryption - Issuer: C=SE, ST=Uppsala, L=Uppsala, O=MySQL AB + Issuer: C=FI, ST=Tuusula, O=Monty Program Ab/emailAddress=abstract.developer(a)askmonty.org Validity - Not Before: Jan 28 11:04:39 2009 GMT - Not After : Jan 28 11:04:39 2010 GMT - Subject: C=SE, ST=Uppsala, O=MySQL AB/emailAddress=abstract.mysql.developer(a)mysql.com + Not Before: Jan 28 22:01:38 2010 GMT + Not After : Dec 7 22:01:38 2019 GMT + Subject: C=FI, ST=Tuusula, O=Monty Program Ab/emailAddress=abstract.developer(a)askmonty.org Subject Public Key Info: Public Key Algorithm: rsaEncryption - RSA Public Key: (512 bit) - Modulus (512 bit): - 00:e1:52:30:2c:d9:be:64:28:91:5d:7a:fd:d9:e9: - 14:35:7a:d2:94:4e:91:46:e0:db:9f:6b:79:f4:4c: - ac:6e:07:61:34:86:74:62:a7:a8:44:af:fa:87:87: - a8:7d:42:61:ff:ab:50:d4:7b:bf:75:fa:d5:d5:b3: - 74:fb:56:1e:37 + RSA Public Key: (1024 bit) + Modulus (1024 bit): + 00:bd:dc:3d:f8:3c:0b:d4:d2:c0:a3:9c:34:2d:e7: + 11:ff:4d:43:35:17:f6:0d:91:01:92:9e:4f:4d:c0: + 38:d5:62:03:55:33:db:66:a2:91:d3:f2:b6:23:34: + 95:53:50:3d:4f:e3:0c:d7:76:fd:f6:54:64:e6:f6: + dc:70:74:7c:6b:74:41:59:b0:19:5d:62:90:3c:a7: + c8:5e:21:8f:2b:22:6b:c7:43:9b:be:79:84:60:da: + 16:c9:ce:ee:fd:66:cb:54:81:e2:b5:1c:cf:f9:74: + de:38:2b:28:d4:31:33:55:d2:30:1c:a3:e4:c2:c7: + 31:46:43:d5:33:3d:8a:0b:47 Exponent: 65537 (0x10001) X509v3 extensions: X509v3 Basic Constraints: CA:FALSE + Netscape Comment: + OpenSSL Generated Certificate X509v3 Subject Key Identifier: - 58:30:B5:9B:2C:05:94:06:BA:3D:3C:F0:B2:CD:1D:67:65:E3:7F:85 + BE:E6:DB:19:8D:DB:72:9A:85:EE:B2:B8:5D:E7:FF:61:DF:09:08:AF X509v3 Authority Key Identifier: - keyid:F2:E2:EA:55:65:A4:9A:E2:AC:9D:97:F5:45:6C:F6:F7:8C:11:AD:DF - DirName:/C=SE/ST=Uppsala/L=Uppsala/O=MySQL AB - serial:95:E9:78:F5:34:50:E4:D5 + keyid:E0:8A:68:D1:C5:14:FA:E3:BB:C0:30:CD:6C:52:7F:DE:88:29:F4:70 Signature Algorithm: sha1WithRSAEncryption - 05:19:e3:13:14:fc:c5:28:bf:69:f8:00:b3:25:cb:bd:ca:9f: - 2f:4c:b3:a8:04:11:f0:74:27:bd:82:2c:b4:49:9b:a7:59:f0: - f7:87:d1:e0:ba:99:a2:fe:4b:1d:10:6f:e4:a2:b3:cd:7f:8b: - 68:31:46:ee:cd:9e:e2:47:e1:4c:fa:74:d1:e2:8b:cc:a0:4b: - a8:24:d1:a4:c3:6b:2a:c6:28:cd:41:e0:06:48:e6:cf:f2:3c: - ca:37:95:d7:29:64:6b:91:91:83:e7:ac:c8:0b:87:bc:da:a6: - aa:f1:44:43:c8:74:7b:15:26:91:2e:03:c4:71:50:6c:f8:68: - dc:8c + 41:95:6d:0a:a4:ee:af:68:cd:94:26:59:9a:18:b7:75:3c:c5: + 0f:22:d3:5c:31:9b:85:a0:93:b3:f0:50:29:ba:1e:d3:5a:43: + 0b:77:2d:98:87:a7:a7:39:0f:40:8d:03:d3:b3:67:43:77:bc: + 3c:51:c2:f9:9e:7a:2d:39:c4:5c:16:d7:70:d6:74:d1:6c:e1: + 6a:4d:fd:1f:10:af:64:3b:f4:64:e9:b2:b3:fb:c8:cd:c5:41: + cd:99:e0:ac:83:1d:81:2c:6b:99:ba:80:02:12:72:f7:3b:bb: + 93:72:00:da:ff:d3:87:75:d2:3a:a4:ca:4d:c1:8b:c1:21:50: + cb:57 -----BEGIN CERTIFICATE----- -MIICfzCCAeigAwIBAgIBAzANBgkqhkiG9w0BAQUFADBEMQswCQYDVQQGEwJTRTEQ -MA4GA1UECBMHVXBwc2FsYTEQMA4GA1UEBxMHVXBwc2FsYTERMA8GA1UEChMITXlT -UUwgQUIwHhcNMDkwMTI4MTEwNDM5WhcNMTAwMTI4MTEwNDM5WjBlMQswCQYDVQQG -EwJTRTEQMA4GA1UECBMHVXBwc2FsYTERMA8GA1UEChMITXlTUUwgQUIxMTAvBgkq -hkiG9w0BCQEWImFic3RyYWN0Lm15c3FsLmRldmVsb3BlckBteXNxbC5jb20wXDAN -BgkqhkiG9w0BAQEFAANLADBIAkEA4VIwLNm+ZCiRXXr92ekUNXrSlE6RRuDbn2t5 -9EysbgdhNIZ0YqeoRK/6h4eofUJh/6tQ1Hu/dfrV1bN0+1YeNwIDAQABo4GjMIGg -MAkGA1UdEwQCMAAwHQYDVR0OBBYEFFgwtZssBZQGuj088LLNHWdl43+FMHQGA1Ud -IwRtMGuAFPLi6lVlpJrirJ2X9UVs9veMEa3foUikRjBEMQswCQYDVQQGEwJTRTEQ -MA4GA1UECBMHVXBwc2FsYTEQMA4GA1UEBxMHVXBwc2FsYTERMA8GA1UEChMITXlT -UUwgQUKCCQCV6Xj1NFDk1TANBgkqhkiG9w0BAQUFAAOBgQAFGeMTFPzFKL9p+ACz -Jcu9yp8vTLOoBBHwdCe9giy0SZunWfD3h9Hgupmi/ksdEG/korPNf4toMUbuzZ7i -R+FM+nTR4ovMoEuoJNGkw2sqxijNQeAGSObP8jzKN5XXKWRrkZGD56zIC4e82qaq -8URDyHR7FSaRLgPEcVBs+GjcjA== +MIICxTCCAi6gAwIBAgIBAjANBgkqhkiG9w0BAQUFADBqMQswCQYDVQQGEwJGSTEQ +MA4GA1UECBMHVHV1c3VsYTEZMBcGA1UEChMQTW9udHkgUHJvZ3JhbSBBYjEuMCwG +CSqGSIb3DQEJARYfYWJzdHJhY3QuZGV2ZWxvcGVyQGFza21vbnR5Lm9yZzAeFw0x +MDAxMjgyMjAxMzhaFw0xOTEyMDcyMjAxMzhaMGoxCzAJBgNVBAYTAkZJMRAwDgYD +VQQIEwdUdXVzdWxhMRkwFwYDVQQKExBNb250eSBQcm9ncmFtIEFiMS4wLAYJKoZI +hvcNAQkBFh9hYnN0cmFjdC5kZXZlbG9wZXJAYXNrbW9udHkub3JnMIGfMA0GCSqG +SIb3DQEBAQUAA4GNADCBiQKBgQC93D34PAvU0sCjnDQt5xH/TUM1F/YNkQGSnk9N +wDjVYgNVM9tmopHT8rYjNJVTUD1P4wzXdv32VGTm9txwdHxrdEFZsBldYpA8p8he +IY8rImvHQ5u+eYRg2hbJzu79ZstUgeK1HM/5dN44KyjUMTNV0jAco+TCxzFGQ9Uz +PYoLRwIDAQABo3sweTAJBgNVHRMEAjAAMCwGCWCGSAGG+EIBDQQfFh1PcGVuU1NM +IEdlbmVyYXRlZCBDZXJ0aWZpY2F0ZTAdBgNVHQ4EFgQUvubbGY3bcpqF7rK4Xef/ +Yd8JCK8wHwYDVR0jBBgwFoAU4Ipo0cUU+uO7wDDNbFJ/3ogp9HAwDQYJKoZIhvcN +AQEFBQADgYEAQZVtCqTur2jNlCZZmhi3dTzFDyLTXDGbhaCTs/BQKboe01pDC3ct +mIenpzkPQI0D07NnQ3e8PFHC+Z56LTnEXBbXcNZ00Wzhak39HxCvZDv0ZOmys/vI +zcVBzZngrIMdgSxrmbqAAhJy9zu7k3IA2v/Th3XSOqTKTcGLwSFQy1c= -----END CERTIFICATE----- === modified file 'mysql-test/std_data/client-key.pem' --- a/mysql-test/std_data/client-key.pem 2009-06-11 16:21:32 +0000 +++ b/mysql-test/std_data/client-key.pem 2010-01-29 10:42:31 +0000 @@ -1,9 +1,15 @@ -----BEGIN RSA PRIVATE KEY----- -MIIBOQIBAAJBAOFSMCzZvmQokV16/dnpFDV60pROkUbg259refRMrG4HYTSGdGKn -qESv+oeHqH1CYf+rUNR7v3X61dWzdPtWHjcCAwEAAQJAXYooM8ZlcuEgj+VKU1ee -qyEFIMqJJxqcMk+E/nWCM96WxCP3zHNSrqNfSpI3ld7QzMwhdRz+gFLxT2gGNpIw -MQIhAPxzM/lDihe67X3ADYtDl9ZjA8Pm430x9sXlcxI17tCZAiEA5H1SyFl4mUee -9VnfSC2XGW7lwz72ZygfVX+b7tLWF08CIEh40gzW5MfXM+KLxdea+fXjyursV5ZT -R6KcMiKiNQLRAiAcmHqlzFzFgisotai2Fc6VRkXHG7gmzOSvBJt1VjmpDQIge6jf -2N7whTdvC4ferB+zUlgWQdyvx1c3T4gnt6PYdaY= +MIICXgIBAAKBgQC93D34PAvU0sCjnDQt5xH/TUM1F/YNkQGSnk9NwDjVYgNVM9tm +opHT8rYjNJVTUD1P4wzXdv32VGTm9txwdHxrdEFZsBldYpA8p8heIY8rImvHQ5u+ +eYRg2hbJzu79ZstUgeK1HM/5dN44KyjUMTNV0jAco+TCxzFGQ9UzPYoLRwIDAQAB +AoGBAJa2lprPT7UJ99Ho1aL6ota/RnKHKtNqII17DgjyZis9OtgP6kJ3GrvdF6iq +vT79my4nVrJTyxYXuGF/5U1/qqNjuPPBE1Xbu1ubQlFv8CT0kKYynQ7Z3ls8fAHC +B3VJXnUVlG+GHtUEFFG4FQVX1fn/Sga67ioJ6ivAiBlHKaPBAkEA5f2ToWlj4u9O +KgfRkN54wdIp4yu2c40pbhMfKGjGGsBAHk92+qSBpzEmxLcI6Ay+4/QysSR4jYmK +jCJuxiTu1QJBANNU3Hx8Il2SF/2BqGLcIh2SHxzKQIT5wAyD2jb+P2cHvbk6pKGR +VTmw5bibxXmYMS6J/L2zUF2xtFe+Svwz96sCQEnKYSqBqOWvyBFeLtPfPTlal8vm +Q4SxfuBtTCrn6t+8XRYcgt0KGPsunvSwkS/6nuh+eiExxWgMACLUDVyPjv0CQQC4 +sJJc7LOv6Oy0bWr2swHRrBEqvQsz63zOszCzHPHWHirNxZV5aiT8XT/2XZRwlvRs +gsVyGFLk/1fn0vN/g/8vAkEAxUdzUKvC1ZwjzGhgcz2bQU0tEZN4C9jBCiwOI2ud +BpAsPG0xAGGL2+hz0B0n88XiTHobiTZ1bg4Z41i4pXx2ZA== -----END RSA PRIVATE KEY----- === modified file 'mysql-test/std_data/server-cert.pem' --- a/mysql-test/std_data/server-cert.pem 2009-06-11 16:21:32 +0000 +++ b/mysql-test/std_data/server-cert.pem 2010-01-29 10:42:31 +0000 @@ -3,53 +3,59 @@ Certificate: Version: 3 (0x2) Serial Number: 1 (0x1) Signature Algorithm: sha1WithRSAEncryption - Issuer: C=SE, ST=Uppsala, L=Uppsala, O=MySQL AB + Issuer: C=FI, ST=Tuusula, O=Monty Program Ab/emailAddress=abstract.developer(a)askmonty.org Validity - Not Before: Jan 28 10:55:13 2009 GMT - Not After : Jan 28 10:55:13 2010 GMT - Subject: C=SE, ST=Uppsala, O=MySQL AB, CN=localhost/emailAddress=abstract.mysql.developer(a)mysql.com + Not Before: Jan 28 21:59:14 2010 GMT + Not After : Dec 7 21:59:14 2019 GMT + Subject: C=FI, ST=Tuusula, O=Monty Program Ab, CN=localhost/emailAddress=abstract.developer(a)askmonty.org Subject Public Key Info: Public Key Algorithm: rsaEncryption - RSA Public Key: (512 bit) - Modulus (512 bit): - 00:b6:8f:e5:b7:b4:86:83:13:8a:f9:bf:63:cb:64: - 2d:b9:51:d1:de:ab:7b:45:1f:aa:b5:66:73:13:f9: - a6:07:d5:ba:7c:fa:92:bd:37:e2:ad:87:db:3e:b6: - 6a:12:64:f8:ee:17:e3:15:06:2f:a8:82:68:bf:57: - 8d:c3:04:98:27 + RSA Public Key: (1024 bit) + Modulus (1024 bit): + 00:cc:79:74:2a:14:7e:77:06:b0:ec:1b:b6:da:70: + 4c:4f:0e:94:04:8a:e7:69:f4:4c:9c:67:22:41:56: + 3a:96:53:9e:95:9d:17:55:80:41:c0:13:d6:94:0f: + cd:2c:51:fe:a4:6e:f2:74:d8:5d:3b:3a:80:e3:85: + 5b:a5:bc:7d:5c:25:55:e5:40:77:fe:f3:cb:5b:cd: + da:a5:f4:36:64:30:a2:a0:7f:93:b3:c4:56:75:2a: + c0:f7:65:2a:d7:e6:ff:88:25:03:e0:b4:40:2e:74: + 4c:cb:46:95:73:cb:25:5c:87:0e:ec:0f:5a:19:c2: + b6:dc:9e:e8:f0:30:b1:9c:99 Exponent: 65537 (0x10001) X509v3 extensions: X509v3 Basic Constraints: CA:FALSE + Netscape Comment: + OpenSSL Generated Certificate X509v3 Subject Key Identifier: - D9:9A:B8:5F:22:EA:04:10:C8:25:7D:82:57:E6:2E:FD:19:29:E7:DA + 6D:13:3B:40:52:3C:AF:18:EA:33:D1:B7:56:21:1B:05:FE:0B:9E:38 X509v3 Authority Key Identifier: - keyid:F2:E2:EA:55:65:A4:9A:E2:AC:9D:97:F5:45:6C:F6:F7:8C:11:AD:DF - DirName:/C=SE/ST=Uppsala/L=Uppsala/O=MySQL AB - serial:95:E9:78:F5:34:50:E4:D5 + keyid:E0:8A:68:D1:C5:14:FA:E3:BB:C0:30:CD:6C:52:7F:DE:88:29:F4:70 Signature Algorithm: sha1WithRSAEncryption - 54:07:2d:21:0b:a5:af:3b:58:23:32:5e:56:7f:ab:58:63:48: - 91:aa:38:90:89:16:f9:cc:bf:a4:0e:78:2b:9f:c5:1b:58:a6: - e6:08:8f:2e:ae:97:03:21:9b:f1:cd:c0:26:8f:1d:d7:28:27: - a0:8e:81:09:1b:1c:0f:c9:a5:41:3a:2d:44:3f:9c:fa:87:ff: - c8:4c:2b:44:f7:1b:c1:3e:4f:01:7f:e9:26:cc:9f:1c:06:b5: - 0b:27:d1:10:90:be:93:0c:9c:e7:b0:d1:ea:27:99:4e:06:14: - 0c:7a:e9:c1:52:c5:33:68:bc:61:0d:db:81:3b:57:48:57:bf: - 42:9a + 97:db:65:23:7f:f1:15:3c:1e:83:ac:0e:0a:50:a0:0c:22:b8: + 45:d4:ca:21:05:47:3b:3d:03:b5:6c:4b:8d:bb:5f:57:c3:c7: + 4e:71:23:cf:33:a3:7f:a0:3d:bd:58:75:b8:37:22:16:2f:e9: + ed:ae:9b:94:29:81:6e:34:79:cf:41:bd:3d:8d:17:d7:22:1c: + 1b:58:c7:0f:79:13:56:1d:e8:d8:4e:e5:07:3f:79:1b:dd:c4: + 06:9b:c5:b6:02:34:43:c5:bf:e5:87:ad:f1:c1:8a:f2:be:c2: + 00:1d:d4:27:1f:87:c8:80:31:ec:6e:97:95:b4:84:40:d1:73: + 42:71 -----BEGIN CERTIFICATE----- -MIICkzCCAfygAwIBAgIBATANBgkqhkiG9w0BAQUFADBEMQswCQYDVQQGEwJTRTEQ -MA4GA1UECBMHVXBwc2FsYTEQMA4GA1UEBxMHVXBwc2FsYTERMA8GA1UEChMITXlT -UUwgQUIwHhcNMDkwMTI4MTA1NTEzWhcNMTAwMTI4MTA1NTEzWjB5MQswCQYDVQQG -EwJTRTEQMA4GA1UECBMHVXBwc2FsYTERMA8GA1UEChMITXlTUUwgQUIxEjAQBgNV -BAMTCWxvY2FsaG9zdDExMC8GCSqGSIb3DQEJARYiYWJzdHJhY3QubXlzcWwuZGV2 -ZWxvcGVyQG15c3FsLmNvbTBcMA0GCSqGSIb3DQEBAQUAA0sAMEgCQQC2j+W3tIaD -E4r5v2PLZC25UdHeq3tFH6q1ZnMT+aYH1bp8+pK9N+Kth9s+tmoSZPjuF+MVBi+o -gmi/V43DBJgnAgMBAAGjgaMwgaAwCQYDVR0TBAIwADAdBgNVHQ4EFgQU2Zq4XyLq -BBDIJX2CV+Yu/Rkp59owdAYDVR0jBG0wa4AU8uLqVWWkmuKsnZf1RWz294wRrd+h -SKRGMEQxCzAJBgNVBAYTAlNFMRAwDgYDVQQIEwdVcHBzYWxhMRAwDgYDVQQHEwdV -cHBzYWxhMREwDwYDVQQKEwhNeVNRTCBBQoIJAJXpePU0UOTVMA0GCSqGSIb3DQEB -BQUAA4GBAFQHLSELpa87WCMyXlZ/q1hjSJGqOJCJFvnMv6QOeCufxRtYpuYIjy6u -lwMhm/HNwCaPHdcoJ6COgQkbHA/JpUE6LUQ/nPqH/8hMK0T3G8E+TwF/6SbMnxwG -tQsn0RCQvpMMnOew0eonmU4GFAx66cFSxTNovGEN24E7V0hXv0Ka +MIIC2TCCAkKgAwIBAgIBATANBgkqhkiG9w0BAQUFADBqMQswCQYDVQQGEwJGSTEQ +MA4GA1UECBMHVHV1c3VsYTEZMBcGA1UEChMQTW9udHkgUHJvZ3JhbSBBYjEuMCwG +CSqGSIb3DQEJARYfYWJzdHJhY3QuZGV2ZWxvcGVyQGFza21vbnR5Lm9yZzAeFw0x +MDAxMjgyMTU5MTRaFw0xOTEyMDcyMTU5MTRaMH4xCzAJBgNVBAYTAkZJMRAwDgYD +VQQIEwdUdXVzdWxhMRkwFwYDVQQKExBNb250eSBQcm9ncmFtIEFiMRIwEAYDVQQD +Ewlsb2NhbGhvc3QxLjAsBgkqhkiG9w0BCQEWH2Fic3RyYWN0LmRldmVsb3BlckBh +c2ttb250eS5vcmcwgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBAMx5dCoUfncG +sOwbttpwTE8OlASK52n0TJxnIkFWOpZTnpWdF1WAQcAT1pQPzSxR/qRu8nTYXTs6 +gOOFW6W8fVwlVeVAd/7zy1vN2qX0NmQwoqB/k7PEVnUqwPdlKtfm/4glA+C0QC50 +TMtGlXPLJVyHDuwPWhnCttye6PAwsZyZAgMBAAGjezB5MAkGA1UdEwQCMAAwLAYJ +YIZIAYb4QgENBB8WHU9wZW5TU0wgR2VuZXJhdGVkIENlcnRpZmljYXRlMB0GA1Ud +DgQWBBRtEztAUjyvGOoz0bdWIRsF/gueODAfBgNVHSMEGDAWgBTgimjRxRT647vA +MM1sUn/eiCn0cDANBgkqhkiG9w0BAQUFAAOBgQCX22Ujf/EVPB6DrA4KUKAMIrhF +1MohBUc7PQO1bEuNu19Xw8dOcSPPM6N/oD29WHW4NyIWL+ntrpuUKYFuNHnPQb09 +jRfXIhwbWMcPeRNWHejYTuUHP3kb3cQGm8W2AjRDxb/lh63xwYryvsIAHdQnH4fI +gDHsbpeVtIRA0XNCcQ== -----END CERTIFICATE----- === modified file 'mysql-test/std_data/server-key.pem' --- a/mysql-test/std_data/server-key.pem 2009-06-11 16:21:32 +0000 +++ b/mysql-test/std_data/server-key.pem 2010-01-29 10:42:31 +0000 @@ -1,9 +1,15 @@ -----BEGIN RSA PRIVATE KEY----- -MIIBOQIBAAJBALaP5be0hoMTivm/Y8tkLblR0d6re0UfqrVmcxP5pgfVunz6kr03 -4q2H2z62ahJk+O4X4xUGL6iCaL9XjcMEmCcCAwEAAQJASA5VwgNb0CKHiPm0ntOk -hG+54SRX3DmafEy6gRjZIl/bZ/asSLhXUZ+CeohyrQh7BZgYWvykd8pRISL9eKsU -GQIhAOXkUrOtP/EtjyqNluEqZdG+RZi/7p61JS3Ce13Myu+LAiEAy0uMlV34AJpM -b40FPKqlHxw8DD/Dt1iKhNVAg8+LDVUCIFjv7fbJDbW2VG63/Cj8CAwOukoP5rbL -iaicVrHBKrllAiB9+MiaXeopZXNrxDS0jQFYr8Q9yt1aJVFgUkxx4Q9HKQIgZPs0 -KlF3NNNWw78INaAEkyf0IEssnLMsuoCWw0DIOak= +MIICXgIBAAKBgQDMeXQqFH53BrDsG7bacExPDpQEiudp9EycZyJBVjqWU56VnRdV +gEHAE9aUD80sUf6kbvJ02F07OoDjhVulvH1cJVXlQHf+88tbzdql9DZkMKKgf5Oz +xFZ1KsD3ZSrX5v+IJQPgtEAudEzLRpVzyyVchw7sD1oZwrbcnujwMLGcmQIDAQAB +AoGBAMdMYkNZsmJFbVDVOobzCg3Mgc1jrmeBrOKNS8AvUe+QFXRyp3m5B102eOHb +/PmD+hU/5qao9UZzoYkiRM/oRq45jrqJEYwWrX007bKK0F9hnErtC1ImM1nBFVhx +6+6cr+ShUkvtj8+wJ2d5bIccUzGCUfFR5tb5BnePTXK8IVoBAkEA7WGNxHAVKgjS +AzlpHr5fvpivA07hNVJizTwZdWGGYeETilZhkkuMRwREceeohF6ILMf0FTZdFSa/ +8EeLa3icIQJBANyDKFjynKwWy5pyRSz75mVwrEi+4eTQPsCPNWLkbpbEPwqPLYWJ +2VSFkISXF7b7Od48JkQWgiB8/kXqMDEdsXkCQQCzZvj3ryWvoP7nhOoXXBWMPGR4 +gZLe86bMKVGsTsp7CtnzwRj4sbQQr/7yfvvzHmaYQX4M0gtDQwfolomd7YdBAkEA +y24ETuqjNu9grf81aiaJipPDnOjcJOcovSRgr/blPxmUvv0Pld5yLNN7W5a4PgrO +fAMpmi7ZpXcqbP17sBQgoQJAWTDFKAmfHPVdDGZuCw4yceP5d+Tv7ABglZUvpPKx +kAvGN1WBASUuCQJDOIgzl6gvYX07S5p147i9mv7UBWOpvw== -----END RSA PRIVATE KEY----- === modified file 'mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result' --- a/mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result 2009-09-28 12:41:10 +0000 +++ b/mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result 2009-11-18 14:50:31 +0000 @@ -1,3 +1,4 @@ +CALL mtr.add_suppression("Statement may not be safe to log in statement format."); drop table if exists t1, t2; create table t1 (a int) engine=innodb; create table t2 (a int) engine=myisam; @@ -224,6 +225,8 @@ create table t0 (n int); insert t0 select * from t1; set autocommit=1; insert into t0 select GET_LOCK("lock1",null); +Warnings: +Note 1592 Statement may not be safe to log in statement format. set autocommit=0; create table t2 (n int) engine=innodb; insert into t2 values (3); === modified file 'mysql-test/suite/binlog/r/binlog_stm_row.result' --- a/mysql-test/suite/binlog/r/binlog_stm_row.result 2009-09-07 20:50:10 +0000 +++ b/mysql-test/suite/binlog/r/binlog_stm_row.result 2010-01-15 15:27:55 +0000 @@ -1,3 +1,4 @@ +CALL mtr.add_suppression("Statement may not be safe to log in statement format."); DROP TABLE IF EXISTS t1; DROP TABLE IF EXISTS t2; set @saved_global_binlog_format = @@global.binlog_format; @@ -29,6 +30,8 @@ SELECT RELEASE_LOCK('Bug#34306'); RELEASE_LOCK('Bug#34306') 1 # con2 +Warnings: +Note 1592 Statement may not be safe to log in statement format. SELECT RELEASE_LOCK('Bug#34306'); RELEASE_LOCK('Bug#34306') 1 === modified file 'mysql-test/suite/binlog/r/binlog_unsafe.result' --- a/mysql-test/suite/binlog/r/binlog_unsafe.result 2009-09-07 20:50:10 +0000 +++ b/mysql-test/suite/binlog/r/binlog_unsafe.result 2010-01-19 10:36:52 +0000 @@ -327,4 +327,87 @@ Warnings: Note 1592 Statement may not be safe to log in statement format. DROP TABLE t1, t2; SET @@SESSION.SQL_MODE = @save_sql_mode; +CREATE TABLE t1 (a VARCHAR(1000)); +INSERT INTO t1 VALUES (CURRENT_USER()); +Warnings: +Note 1592 Statement may not be safe to log in statement format. +INSERT INTO t1 VALUES (FOUND_ROWS()); +Warnings: +Note 1592 Statement may not be safe to log in statement format. +INSERT INTO t1 VALUES (GET_LOCK('tmp', 1)); +Warnings: +Note 1592 Statement may not be safe to log in statement format. +INSERT INTO t1 VALUES (IS_FREE_LOCK('tmp')); +Warnings: +Note 1592 Statement may not be safe to log in statement format. +INSERT INTO t1 VALUES (IS_USED_LOCK('tmp')); +Warnings: +Note 1592 Statement may not be safe to log in statement format. +INSERT INTO t1 VALUES (LOAD_FILE('../../std_data/words2.dat')); +Warnings: +Note 1592 Statement may not be safe to log in statement format. +INSERT INTO t1 VALUES (MASTER_POS_WAIT('dummy arg', 4711, 1)); +Warnings: +Note 1592 Statement may not be safe to log in statement format. +INSERT INTO t1 VALUES (RELEASE_LOCK('tmp')); +Warnings: +Note 1592 Statement may not be safe to log in statement format. +INSERT INTO t1 VALUES (ROW_COUNT()); +Warnings: +Note 1592 Statement may not be safe to log in statement format. +INSERT INTO t1 VALUES (SESSION_USER()); +Warnings: +Note 1592 Statement may not be safe to log in statement format. +INSERT INTO t1 VALUES (SLEEP(1)); +Warnings: +Note 1592 Statement may not be safe to log in statement format. +INSERT INTO t1 VALUES (SYSDATE()); +Warnings: +Note 1592 Statement may not be safe to log in statement format. +INSERT INTO t1 VALUES (SYSTEM_USER()); +Warnings: +Note 1592 Statement may not be safe to log in statement format. +INSERT INTO t1 VALUES (USER()); +Warnings: +Note 1592 Statement may not be safe to log in statement format. +INSERT INTO t1 VALUES (UUID()); +Warnings: +Note 1592 Statement may not be safe to log in statement format. +INSERT INTO t1 VALUES (UUID_SHORT()); +Warnings: +Note 1592 Statement may not be safe to log in statement format. +INSERT INTO t1 VALUES (VERSION()); +Warnings: +Note 1592 Statement may not be safe to log in statement format. +DELETE FROM t1; +SET TIME_ZONE= '+03:00'; +SET TIMESTAMP=1000000; +INSERT INTO t1 VALUES +(CURDATE()), +(CURRENT_DATE()), +(CURRENT_TIME()), +(CURRENT_TIMESTAMP()), +(CURTIME()), +(LOCALTIME()), +(LOCALTIMESTAMP()), +(NOW()), +(UNIX_TIMESTAMP()), +(UTC_DATE()), +(UTC_TIME()), +(UTC_TIMESTAMP()); +SELECT * FROM t1; +a +1970-01-12 +1970-01-12 +16:46:40 +1970-01-12 16:46:40 +16:46:40 +1970-01-12 16:46:40 +1970-01-12 16:46:40 +1970-01-12 16:46:40 +1000000 +1970-01-12 +13:46:40 +1970-01-12 13:46:40 +DROP TABLE t1; "End of tests" === modified file 'mysql-test/suite/binlog/t/binlog_killed.test' --- a/mysql-test/suite/binlog/t/binlog_killed.test 2008-10-23 19:27:09 +0000 +++ b/mysql-test/suite/binlog/t/binlog_killed.test 2009-11-18 14:50:31 +0000 @@ -1,5 +1,5 @@ -- source include/have_innodb.inc --- source include/have_binlog_format_mixed_or_statement.inc +-- source include/have_binlog_format_statement.inc # You cannot use `KILL' with the Embedded MySQL Server library, # because the embedded server merely runs inside the threads of the host === modified file 'mysql-test/suite/binlog/t/binlog_stm_mix_innodb_myisam.test' --- a/mysql-test/suite/binlog/t/binlog_stm_mix_innodb_myisam.test 2008-02-28 11:21:44 +0000 +++ b/mysql-test/suite/binlog/t/binlog_stm_mix_innodb_myisam.test 2009-11-18 14:50:31 +0000 @@ -2,6 +2,9 @@ # For both statement and row based bin logs 9/19/2005 [jbm] -- source include/have_binlog_format_statement.inc + +CALL mtr.add_suppression("Statement may not be safe to log in statement format."); + -- source extra/binlog_tests/mix_innodb_myisam_binlog.test set @@session.binlog_format=statement; === modified file 'mysql-test/suite/binlog/t/binlog_stm_row.test' --- a/mysql-test/suite/binlog/t/binlog_stm_row.test 2009-02-19 09:01:25 +0000 +++ b/mysql-test/suite/binlog/t/binlog_stm_row.test 2010-01-15 15:27:55 +0000 @@ -1,5 +1,8 @@ --source include/have_log_bin.inc ---source include/have_binlog_format_row_or_statement.inc +# Test sets its own binlog_format, so we restrict it to run only once +--source include/have_binlog_format_row.inc + +CALL mtr.add_suppression("Statement may not be safe to log in statement format."); # Get rid of previous tests binlog --disable_query_log === modified file 'mysql-test/suite/binlog/t/binlog_unsafe.test' --- a/mysql-test/suite/binlog/t/binlog_unsafe.test 2009-09-07 20:50:10 +0000 +++ b/mysql-test/suite/binlog/t/binlog_unsafe.test 2010-01-19 10:36:52 +0000 @@ -388,4 +388,57 @@ DELETE FROM t1 LIMIT 1; DROP TABLE t1, t2; SET @@SESSION.SQL_MODE = @save_sql_mode; + +# +# BUG#47995: Mark user functions as unsafe +# +# Test that the system functions that are supposed to be marked unsafe +# generate a warning. Each INSERT statement below should generate a +# warning. +# + +CREATE TABLE t1 (a VARCHAR(1000)); +INSERT INTO t1 VALUES (CURRENT_USER()); #marked unsafe before BUG#47995 +INSERT INTO t1 VALUES (FOUND_ROWS()); #marked unsafe before BUG#47995 +INSERT INTO t1 VALUES (GET_LOCK('tmp', 1)); +INSERT INTO t1 VALUES (IS_FREE_LOCK('tmp')); +INSERT INTO t1 VALUES (IS_USED_LOCK('tmp')); +INSERT INTO t1 VALUES (LOAD_FILE('../../std_data/words2.dat')); #marked unsafe before BUG#47995 +INSERT INTO t1 VALUES (MASTER_POS_WAIT('dummy arg', 4711, 1)); +INSERT INTO t1 VALUES (RELEASE_LOCK('tmp')); +INSERT INTO t1 VALUES (ROW_COUNT()); #marked unsafe before BUG#47995 +INSERT INTO t1 VALUES (SESSION_USER()); #marked unsafe before BUG#47995 +INSERT INTO t1 VALUES (SLEEP(1)); +INSERT INTO t1 VALUES (SYSDATE()); +INSERT INTO t1 VALUES (SYSTEM_USER()); #marked unsafe before BUG#47995 +INSERT INTO t1 VALUES (USER()); #marked unsafe before BUG#47995 +INSERT INTO t1 VALUES (UUID()); #marked unsafe before BUG#47995 +INSERT INTO t1 VALUES (UUID_SHORT()); #marked unsafe before BUG#47995 +INSERT INTO t1 VALUES (VERSION()); +DELETE FROM t1; + +# Since we replicate the TIMESTAMP variable, functions affected by the +# TIMESTAMP variable are safe to replicate. So we check that the +# following following functions depend on the TIMESTAMP variable and +# don't generate a warning. + +SET TIME_ZONE= '+03:00'; +SET TIMESTAMP=1000000; +INSERT INTO t1 VALUES + (CURDATE()), + (CURRENT_DATE()), + (CURRENT_TIME()), + (CURRENT_TIMESTAMP()), + (CURTIME()), + (LOCALTIME()), + (LOCALTIMESTAMP()), + (NOW()), + (UNIX_TIMESTAMP()), + (UTC_DATE()), + (UTC_TIME()), + (UTC_TIMESTAMP()); +SELECT * FROM t1; + +DROP TABLE t1; + --echo "End of tests" === modified file 'mysql-test/suite/funcs_1/datadict/processlist_val.inc' --- a/mysql-test/suite/funcs_1/datadict/processlist_val.inc 2009-10-10 09:59:06 +0000 +++ b/mysql-test/suite/funcs_1/datadict/processlist_val.inc 2010-01-11 13:15:28 +0000 @@ -238,7 +238,7 @@ echo # Poll till all connections of 'test_user' are in a state with COMMAND = 'Sleep' ; let $wait_condition= SELECT COUNT(*) = 2 FROM INFORMATION_SCHEMA.PROCESSLIST - WHERE USER = 'test_user' AND COMMAND = 'Sleep'; + WHERE USER = 'test_user' AND COMMAND = 'Sleep' AND STATE = ''; --source include/wait_condition.inc echo # ----- switch to connection con2 (user = test_user) ----- === modified file 'mysql-test/suite/funcs_1/r/is_columns_is.result' --- a/mysql-test/suite/funcs_1/r/is_columns_is.result 2009-10-10 09:59:06 +0000 +++ b/mysql-test/suite/funcs_1/r/is_columns_is.result 2010-01-16 05:12:57 +0000 @@ -127,7 +127,7 @@ NULL information_schema INNODB_BUFFER_PO NULL information_schema INNODB_BUFFER_POOL_PAGES_BLOB page_no 2 0 NO bigint NULL NULL 19 0 NULL NULL bigint(21) unsigned select NULL information_schema INNODB_BUFFER_POOL_PAGES_BLOB part_len 4 0 NO bigint NULL NULL 19 0 NULL NULL bigint(21) unsigned select NULL information_schema INNODB_BUFFER_POOL_PAGES_BLOB space_id 1 0 NO bigint NULL NULL 19 0 NULL NULL bigint(21) unsigned select -NULL information_schema INNODB_BUFFER_POOL_PAGES_INDEX accessed 9 0 NO bigint NULL NULL 19 0 NULL NULL bigint(21) unsigned select +NULL information_schema INNODB_BUFFER_POOL_PAGES_INDEX access_time 9 0 NO bigint NULL NULL 19 0 NULL NULL bigint(21) unsigned select NULL information_schema INNODB_BUFFER_POOL_PAGES_INDEX data_size 7 0 NO bigint NULL NULL 19 0 NULL NULL bigint(21) unsigned select NULL information_schema INNODB_BUFFER_POOL_PAGES_INDEX dirty 11 0 NO bigint NULL NULL 19 0 NULL NULL bigint(21) unsigned select NULL information_schema INNODB_BUFFER_POOL_PAGES_INDEX fix_count 14 0 NO bigint NULL NULL 19 0 NULL NULL bigint(21) unsigned select @@ -394,6 +394,7 @@ NULL information_schema VIEWS TABLE_CATA NULL information_schema VIEWS TABLE_NAME 3 NO varchar 64 192 NULL NULL utf8 utf8_general_ci varchar(64) select NULL information_schema VIEWS TABLE_SCHEMA 2 NO varchar 64 192 NULL NULL utf8 utf8_general_ci varchar(64) select NULL information_schema VIEWS VIEW_DEFINITION 4 NULL NO longtext 4294967295 4294967295 NULL NULL utf8 utf8_general_ci longtext select +NULL information_schema XTRADB_ADMIN_COMMAND result_message 1 NO varchar 1024 3072 NULL NULL utf8 utf8_general_ci varchar(1024) select NULL information_schema XTRADB_ENHANCEMENTS comment 3 NO varchar 100 300 NULL NULL utf8 utf8_general_ci varchar(100) select NULL information_schema XTRADB_ENHANCEMENTS description 2 NO varchar 255 765 NULL NULL utf8 utf8_general_ci varchar(255) select NULL information_schema XTRADB_ENHANCEMENTS link 4 NO varchar 255 765 NULL NULL utf8 utf8_general_ci varchar(255) select @@ -589,7 +590,7 @@ NULL information_schema INNODB_BUFFER_PO NULL information_schema INNODB_BUFFER_POOL_PAGES_INDEX n_recs bigint NULL NULL NULL NULL bigint(21) unsigned NULL information_schema INNODB_BUFFER_POOL_PAGES_INDEX data_size bigint NULL NULL NULL NULL bigint(21) unsigned NULL information_schema INNODB_BUFFER_POOL_PAGES_INDEX hashed bigint NULL NULL NULL NULL bigint(21) unsigned -NULL information_schema INNODB_BUFFER_POOL_PAGES_INDEX accessed bigint NULL NULL NULL NULL bigint(21) unsigned +NULL information_schema INNODB_BUFFER_POOL_PAGES_INDEX access_time bigint NULL NULL NULL NULL bigint(21) unsigned NULL information_schema INNODB_BUFFER_POOL_PAGES_INDEX modified bigint NULL NULL NULL NULL bigint(21) unsigned NULL information_schema INNODB_BUFFER_POOL_PAGES_INDEX dirty bigint NULL NULL NULL NULL bigint(21) unsigned NULL information_schema INNODB_BUFFER_POOL_PAGES_INDEX old bigint NULL NULL NULL NULL bigint(21) unsigned @@ -848,6 +849,7 @@ NULL information_schema TRIGGERS CREATED 3.0000 information_schema VIEWS SECURITY_TYPE varchar 7 21 utf8 utf8_general_ci varchar(7) 3.0000 information_schema VIEWS CHARACTER_SET_CLIENT varchar 32 96 utf8 utf8_general_ci varchar(32) 3.0000 information_schema VIEWS COLLATION_CONNECTION varchar 32 96 utf8 utf8_general_ci varchar(32) +3.0000 information_schema XTRADB_ADMIN_COMMAND result_message varchar 1024 3072 utf8 utf8_general_ci varchar(1024) 3.0000 information_schema XTRADB_ENHANCEMENTS name varchar 255 765 utf8 utf8_general_ci varchar(255) 3.0000 information_schema XTRADB_ENHANCEMENTS description varchar 255 765 utf8 utf8_general_ci varchar(255) 3.0000 information_schema XTRADB_ENHANCEMENTS comment varchar 100 300 utf8 utf8_general_ci varchar(100) === modified file 'mysql-test/suite/funcs_1/r/is_tables_is.result' --- a/mysql-test/suite/funcs_1/r/is_tables_is.result 2009-10-10 09:59:06 +0000 +++ b/mysql-test/suite/funcs_1/r/is_tables_is.result 2010-01-16 05:12:57 +0000 @@ -958,6 +958,29 @@ user_comment Separator ----------------------------------------------------- TABLE_CATALOG NULL TABLE_SCHEMA information_schema +TABLE_NAME XTRADB_ADMIN_COMMAND +TABLE_TYPE SYSTEM VIEW +ENGINE MEMORY +VERSION 10 +ROW_FORMAT Fixed +TABLE_ROWS #TBLR# +AVG_ROW_LENGTH #ARL# +DATA_LENGTH #DL# +MAX_DATA_LENGTH #MDL# +INDEX_LENGTH #IL# +DATA_FREE #DF# +AUTO_INCREMENT NULL +CREATE_TIME #CRT# +UPDATE_TIME #UT# +CHECK_TIME #CT# +TABLE_COLLATION utf8_general_ci +CHECKSUM NULL +CREATE_OPTIONS #CO# +TABLE_COMMENT #TC# +user_comment +Separator ----------------------------------------------------- +TABLE_CATALOG NULL +TABLE_SCHEMA information_schema TABLE_NAME XTRADB_ENHANCEMENTS TABLE_TYPE SYSTEM VIEW ENGINE MEMORY @@ -1941,6 +1964,29 @@ user_comment Separator ----------------------------------------------------- TABLE_CATALOG NULL TABLE_SCHEMA information_schema +TABLE_NAME XTRADB_ADMIN_COMMAND +TABLE_TYPE SYSTEM VIEW +ENGINE MEMORY +VERSION 10 +ROW_FORMAT Fixed +TABLE_ROWS #TBLR# +AVG_ROW_LENGTH #ARL# +DATA_LENGTH #DL# +MAX_DATA_LENGTH #MDL# +INDEX_LENGTH #IL# +DATA_FREE #DF# +AUTO_INCREMENT NULL +CREATE_TIME #CRT# +UPDATE_TIME #UT# +CHECK_TIME #CT# +TABLE_COLLATION utf8_general_ci +CHECKSUM NULL +CREATE_OPTIONS #CO# +TABLE_COMMENT #TC# +user_comment +Separator ----------------------------------------------------- +TABLE_CATALOG NULL +TABLE_SCHEMA information_schema TABLE_NAME XTRADB_ENHANCEMENTS TABLE_TYPE SYSTEM VIEW ENGINE MEMORY === modified file 'mysql-test/suite/innodb/r/innodb-index.result' --- a/mysql-test/suite/innodb/r/innodb-index.result 2009-06-10 13:51:20 +0000 +++ b/mysql-test/suite/innodb/r/innodb-index.result 2009-11-30 12:49:13 +0000 @@ -968,6 +968,7 @@ create index t1u on t1 (u(1)); drop table t1; set global innodb_file_per_table=0; set global innodb_file_format=Antelope; +set global innodb_file_format_check=Antelope; SET @OLD_UNIQUE_CHECKS=@@UNIQUE_CHECKS, UNIQUE_CHECKS=0; SET @OLD_FOREIGN_KEY_CHECKS=@@FOREIGN_KEY_CHECKS, FOREIGN_KEY_CHECKS=0; CREATE TABLE t1( === added file 'mysql-test/suite/innodb/r/innodb_bug46676.result' --- a/mysql-test/suite/innodb/r/innodb_bug46676.result 1970-01-01 00:00:00 +0000 +++ b/mysql-test/suite/innodb/r/innodb_bug46676.result 2009-11-30 12:24:54 +0000 @@ -0,0 +1,9 @@ +SET foreign_key_checks=0; +CREATE TABLE t1 (id int, foreign key (id) references t2(id)) ENGINE=INNODB; +CREATE TABLE t2 (id int, foreign key (id) references t1(id)) ENGINE=INNODB; +SET foreign_key_checks=1; +SELECT COUNT(*) FROM information_schema.key_column_usage WHERE REFERENCED_TABLE_NAME in ('t1', 't2'); +COUNT(*) +2 +SET foreign_key_checks=0; +DROP TABLE t1, t2; === added file 'mysql-test/suite/innodb/r/innodb_bug47167.result' --- a/mysql-test/suite/innodb/r/innodb_bug47167.result 1970-01-01 00:00:00 +0000 +++ b/mysql-test/suite/innodb/r/innodb_bug47167.result 2009-11-30 11:56:21 +0000 @@ -0,0 +1,24 @@ +set @old_innodb_file_format_check=@@innodb_file_format_check; +select @old_innodb_file_format_check; +@old_innodb_file_format_check +Antelope +set global innodb_file_format_check = Barracuda; +select @@innodb_file_format_check; +@@innodb_file_format_check +Barracuda +set global innodb_file_format_check = DEFAULT; +select @@innodb_file_format_check; +@@innodb_file_format_check +Barracuda +set global innodb_file_format_check = @old_innodb_file_format_check; +select @@innodb_file_format_check; +@@innodb_file_format_check +Antelope +set global innodb_file_format_check = cheetah; +ERROR HY000: Incorrect arguments to SET +set global innodb_file_format_check = Bear; +ERROR HY000: Incorrect arguments to SET +set global innodb_file_format_check = on; +ERROR HY000: Incorrect arguments to SET +set global innodb_file_format_check = off; +ERROR HY000: Incorrect arguments to SET === modified file 'mysql-test/suite/innodb/t/innodb-consistent-master.opt' --- a/mysql-test/suite/innodb/t/innodb-consistent-master.opt 2009-10-09 13:37:47 +0000 +++ b/mysql-test/suite/innodb/t/innodb-consistent-master.opt 2009-11-30 12:49:13 +0000 @@ -1 +1 @@ ---innodb_lock_wait_timeout=2 +--loose-innodb_lock_wait_timeout=2 === modified file 'mysql-test/suite/innodb/t/innodb-index.test' --- a/mysql-test/suite/innodb/t/innodb-index.test 2009-06-11 12:57:44 +0000 +++ b/mysql-test/suite/innodb/t/innodb-index.test 2009-11-30 12:49:13 +0000 @@ -1,5 +1,7 @@ -- source include/have_innodb.inc +let $innodb_file_format_check_orig=`select @@innodb_file_format_check`; + create table t1(a int not null, b int, c char(10) not null, d varchar(20)) engine = innodb; insert into t1 values (5,5,'oo','oo'),(4,4,'tr','tr'),(3,4,'ad','ad'),(2,3,'ak','ak'); commit; @@ -398,6 +400,7 @@ create index t1u on t1 (u(1)); drop table t1; eval set global innodb_file_per_table=$per_table; eval set global innodb_file_format=$format; +eval set global innodb_file_format_check=$format; # # Test to check whether CREATE INDEX handles implicit foreign key @@ -532,3 +535,10 @@ disconnect a; disconnect b; DROP TABLE t1; + +# +# restore environment to the state it was before this test execution +# + +-- disable_query_log +eval SET GLOBAL innodb_file_format_check=$innodb_file_format_check_orig; === added file 'mysql-test/suite/innodb/t/innodb_bug46676.test' --- a/mysql-test/suite/innodb/t/innodb_bug46676.test 1970-01-01 00:00:00 +0000 +++ b/mysql-test/suite/innodb/t/innodb_bug46676.test 2009-11-30 12:24:54 +0000 @@ -0,0 +1,16 @@ +# This is the test for bug 46676: mysqld got exception 0xc0000005 +# It is reproducible with InnoDB plugin 1.0.4 + MySQL 5.1.37. +# But no longer reproducible after MySQL 5.1.38 (with plugin 1.0.5). + +--source include/have_innodb.inc + +SET foreign_key_checks=0; +CREATE TABLE t1 (id int, foreign key (id) references t2(id)) ENGINE=INNODB; +CREATE TABLE t2 (id int, foreign key (id) references t1(id)) ENGINE=INNODB; +SET foreign_key_checks=1; + +# Server crashes +SELECT COUNT(*) FROM information_schema.key_column_usage WHERE REFERENCED_TABLE_NAME in ('t1', 't2'); + +SET foreign_key_checks=0; +DROP TABLE t1, t2; === added file 'mysql-test/suite/innodb/t/innodb_bug47167.test' --- a/mysql-test/suite/innodb/t/innodb_bug47167.test 1970-01-01 00:00:00 +0000 +++ b/mysql-test/suite/innodb/t/innodb_bug47167.test 2009-11-30 11:56:21 +0000 @@ -0,0 +1,46 @@ +# This is the unit test for bug *47167. +# It tests setting the global variable +# "innodb_file_format_check" with a +# user-Defined Variable. + +--source include/have_innodb.inc +-- source suite/innodb/include/have_innodb_plugin.inc + +# Save the value (Antelope) in 'innodb_file_format_check' to +# 'old_innodb_file_format_check' +set @old_innodb_file_format_check=@@innodb_file_format_check; + +# @old_innodb_file_format_check shall have the value of 'Antelope' +select @old_innodb_file_format_check; + +# Reset the value in 'innodb_file_format_check' to 'Barracuda' +set global innodb_file_format_check = Barracuda; + +select @@innodb_file_format_check; + +# Set 'innodb_file_format_check' to its default value, which +# is the latest file format supported in the current release. +set global innodb_file_format_check = DEFAULT; + +select @@innodb_file_format_check; + +# Put the saved value back to 'innodb_file_format_check' +set global innodb_file_format_check = @old_innodb_file_format_check; + +# Check whether 'innodb_file_format_check' get its original value. +select @@innodb_file_format_check; + +# Following are negative tests, all should fail. +--disable_warnings +--error ER_WRONG_ARGUMENTS +set global innodb_file_format_check = cheetah; + +--error ER_WRONG_ARGUMENTS +set global innodb_file_format_check = Bear; + +--error ER_WRONG_ARGUMENTS +set global innodb_file_format_check = on; + +--error ER_WRONG_ARGUMENTS +set global innodb_file_format_check = off; +--enable_warnings === modified file 'mysql-test/suite/maria/r/maria-recover.result' --- a/mysql-test/suite/maria/r/maria-recover.result 2009-02-19 09:01:25 +0000 +++ b/mysql-test/suite/maria/r/maria-recover.result 2010-01-27 17:41:05 +0000 @@ -1,6 +1,3 @@ -call mtr.add_suppression("Checking table: '.\/mysqltest\/t_corrupted2'"); -call mtr.add_suppression("Recovering table: '.\/mysqltest\/t_corrupted2'"); -call mtr.add_suppression("Table '.\/mysqltest\/t_corrupted2' is marked as crashed and should be repaired"); select @@global.maria_recover; @@global.maria_recover BACKUP === modified file 'mysql-test/suite/maria/t/maria-recover.test' --- a/mysql-test/suite/maria/t/maria-recover.test 2009-10-09 08:09:24 +0000 +++ b/mysql-test/suite/maria/t/maria-recover.test 2010-01-27 17:41:05 +0000 @@ -2,9 +2,13 @@ --source include/have_maria.inc -call mtr.add_suppression("Checking table: '.\/mysqltest\/t_corrupted2'"); -call mtr.add_suppression("Recovering table: '.\/mysqltest\/t_corrupted2'"); -call mtr.add_suppression("Table '.\/mysqltest\/t_corrupted2' is marked as crashed and should be repaired"); +--disable_query_log +# Note: \\. matches a single period. We use '.' as directory separator to +# account for Unix and Windows variation. +call mtr.add_suppression("Checking table: '\\..mysqltest.t_corrupted2'"); +call mtr.add_suppression("Recovering table: '\\..mysqltest.t_corrupted2'"); +call mtr.add_suppression("Table '\\..mysqltest.t_corrupted2' is marked as crashed and should be repaired"); +--enable_query_log # Note: we're setting an environment variable (not prefixing it by $), # so that the perl code below can access it. === removed file 'mysql-test/suite/maria/t/maria2-master.opt' --- a/mysql-test/suite/maria/t/maria2-master.opt 2009-02-15 10:58:34 +0000 +++ b/mysql-test/suite/maria/t/maria2-master.opt 1970-01-01 00:00:00 +0000 @@ -1,2 +0,0 @@ ---secure-file-priv="" - === modified file 'mysql-test/suite/parts/inc/partition_timestamp.inc' --- a/mysql-test/suite/parts/inc/partition_timestamp.inc 2009-10-28 07:52:34 +0000 +++ b/mysql-test/suite/parts/inc/partition_timestamp.inc 2010-01-27 17:41:05 +0000 @@ -48,6 +48,7 @@ show create table t3; let $count=12; --echo $count inserts; --disable_query_log +SET TIME_ZONE= '+03:00'; begin; while ($count) { === modified file 'mysql-test/suite/parts/r/partition_recover_myisam.result' --- a/mysql-test/suite/parts/r/partition_recover_myisam.result 2009-08-29 21:29:47 +0000 +++ b/mysql-test/suite/parts/r/partition_recover_myisam.result 2010-01-27 17:41:05 +0000 @@ -1,5 +1,3 @@ -call mtr.add_suppression("./test/t1_will_crash"); -call mtr.add_suppression("Got an error from unknown thread, ha_myisam.cc"); CREATE TABLE t1_will_crash (a INT, KEY (a)) ENGINE=MyISAM; INSERT INTO t1_will_crash VALUES (1), (2), (3), (4), (5), (6), (7), (8), (9), (10), (11); FLUSH TABLES; === modified file 'mysql-test/suite/parts/t/partition_alter1_2_innodb.test' --- a/mysql-test/suite/parts/t/partition_alter1_2_innodb.test 2009-10-09 13:08:09 +0000 +++ b/mysql-test/suite/parts/t/partition_alter1_2_innodb.test 2010-01-15 15:27:55 +0000 @@ -28,6 +28,8 @@ #------------------------------------------------------------------------------# # General not engine specific settings and requirements +--source include/big_test.inc + ##### Options, for debugging support ##### let $debug= 0; let $with_partitioning= 1; === modified file 'mysql-test/suite/parts/t/partition_alter2_1_innodb.test' --- a/mysql-test/suite/parts/t/partition_alter2_1_innodb.test 2009-10-09 13:08:09 +0000 +++ b/mysql-test/suite/parts/t/partition_alter2_1_innodb.test 2010-01-15 15:27:55 +0000 @@ -22,6 +22,8 @@ # any of the variables. # +--source include/big_test.inc + #------------------------------------------------------------------------------# # General not engine specific settings and requirements === modified file 'mysql-test/suite/parts/t/partition_alter2_2_innodb.test' --- a/mysql-test/suite/parts/t/partition_alter2_2_innodb.test 2009-10-09 13:08:09 +0000 +++ b/mysql-test/suite/parts/t/partition_alter2_2_innodb.test 2010-01-15 15:27:55 +0000 @@ -22,6 +22,8 @@ # any of the variables. # +--source include/big_test.inc + #------------------------------------------------------------------------------# # General not engine specific settings and requirements === modified file 'mysql-test/suite/parts/t/partition_alter4_innodb.test' --- a/mysql-test/suite/parts/t/partition_alter4_innodb.test 2009-10-09 13:08:09 +0000 +++ b/mysql-test/suite/parts/t/partition_alter4_innodb.test 2010-01-15 15:27:55 +0000 @@ -22,6 +22,8 @@ # any of the variables. # +--source include/big_test.inc + #------------------------------------------------------------------------------# # General not engine specific settings and requirements === modified file 'mysql-test/suite/parts/t/partition_recover_myisam.test' --- a/mysql-test/suite/parts/t/partition_recover_myisam.test 2009-12-03 11:19:05 +0000 +++ b/mysql-test/suite/parts/t/partition_recover_myisam.test 2010-01-27 17:41:05 +0000 @@ -1,7 +1,9 @@ # test the auto-recover (--myisam-recover) of partitioned myisam tables -call mtr.add_suppression("./test/t1_will_crash"); +--disable_query_log +call mtr.add_suppression("..test.t1_will_crash"); call mtr.add_suppression("Got an error from unknown thread, ha_myisam.cc"); +--enable_query_log --source include/have_partition.inc --disable_warnings === modified file 'mysql-test/suite/pbxt/r/func_group.result' --- a/mysql-test/suite/pbxt/r/func_group.result 2009-11-24 10:19:08 +0000 +++ b/mysql-test/suite/pbxt/r/func_group.result 2010-01-16 05:12:57 +0000 @@ -885,7 +885,7 @@ cast(sum(distinct df) as signed) 3 select cast(min(df) as signed) from t1; cast(min(df) as signed) -0 +1 select 1e8 * sum(distinct df) from t1; 1e8 * sum(distinct df) 330000000 === modified file 'mysql-test/suite/pbxt/r/mysqlshow.result' --- a/mysql-test/suite/pbxt/r/mysqlshow.result 2009-08-17 15:57:58 +0000 +++ b/mysql-test/suite/pbxt/r/mysqlshow.result 2010-01-16 05:12:57 +0000 @@ -113,7 +113,7 @@ Database: information_schema | INNODB_RSEG | | XTRADB_ENHANCEMENTS | | INNODB_BUFFER_POOL_PAGES_INDEX | -| INNODB_INDEX_STATS | +| XTRADB_ADMIN_COMMAND | | INNODB_TRX | | INNODB_CMP_RESET | | INNODB_LOCK_WAITS | @@ -122,6 +122,7 @@ Database: information_schema | INNODB_CMPMEM | | INNODB_TABLE_STATS | | INNODB_BUFFER_POOL_PAGES_BLOB | +| INNODB_INDEX_STATS | +---------------------------------------+ Database: INFORMATION_SCHEMA +---------------------------------------+ @@ -161,7 +162,7 @@ Database: INFORMATION_SCHEMA | INNODB_RSEG | | XTRADB_ENHANCEMENTS | | INNODB_BUFFER_POOL_PAGES_INDEX | -| INNODB_INDEX_STATS | +| XTRADB_ADMIN_COMMAND | | INNODB_TRX | | INNODB_CMP_RESET | | INNODB_LOCK_WAITS | @@ -170,6 +171,7 @@ Database: INFORMATION_SCHEMA | INNODB_CMPMEM | | INNODB_TABLE_STATS | | INNODB_BUFFER_POOL_PAGES_BLOB | +| INNODB_INDEX_STATS | +---------------------------------------+ Wildcard: inf_rmation_schema +--------------------+ === modified file 'mysql-test/suite/rpl/r/rpl_err_ignoredtable.result' --- a/mysql-test/suite/rpl/r/rpl_err_ignoredtable.result 2007-06-27 12:28:02 +0000 +++ b/mysql-test/suite/rpl/r/rpl_err_ignoredtable.result 2009-11-18 14:50:31 +0000 @@ -4,6 +4,7 @@ reset master; reset slave; drop table if exists t1,t2,t3,t4,t5,t6,t7,t8,t9; start slave; +CALL mtr.add_suppression("Statement may not be safe to log in statement format."); create table t1 (a int primary key); create table t4 (a int primary key); insert into t1 values (1),(1); === modified file 'mysql-test/suite/rpl/r/rpl_extraCol_innodb.result' --- a/mysql-test/suite/rpl/r/rpl_extraCol_innodb.result 2009-08-28 14:13:27 +0000 +++ b/mysql-test/suite/rpl/r/rpl_extraCol_innodb.result 2009-10-22 00:10:42 +0000 @@ -404,7 +404,11 @@ STOP SLAVE; RESET SLAVE; CREATE TABLE t9 (a INT KEY, b BLOB, c CHAR(5), d TIMESTAMP, -e INT NOT NULL) ENGINE='InnoDB'; +e INT NOT NULL, +f text not null, +g text, +h blob not null, +i blob) ENGINE='InnoDB'; *** Create t9 on Master *** CREATE TABLE t9 (a INT PRIMARY KEY, b BLOB, c CHAR(5) ) ENGINE='InnoDB'; @@ -415,47 +419,11 @@ START SLAVE; set @b1 = 'b1b1b1b1'; set @b1 = concat(@b1,@b1); INSERT INTO t9 () VALUES(1,@b1,'Kyle'),(2,@b1,'JOE'),(3,@b1,'QA'); -SHOW SLAVE STATUS; -Slave_IO_State # -Master_Host 127.0.0.1 -Master_User root -Master_Port # -Connect_Retry 1 -Master_Log_File master-bin.000001 -Read_Master_Log_Pos # -Relay_Log_File # -Relay_Log_Pos # -Relay_Master_Log_File master-bin.000001 -Slave_IO_Running Yes -Slave_SQL_Running No -Replicate_Do_DB -Replicate_Ignore_DB -Replicate_Do_Table -Replicate_Ignore_Table # -Replicate_Wild_Do_Table -Replicate_Wild_Ignore_Table -Last_Errno 1364 -Last_Error Could not execute Write_rows event on table test.t9; Field 'e' doesn't have a default value, Error_code: 1364; handler error HA_ERR_ROWS_EVENT_APPLY; the event's master log master-bin.000001, end_log_pos 330 -Skip_Counter 0 -Exec_Master_Log_Pos # -Relay_Log_Space # -Until_Condition None -Until_Log_File -Until_Log_Pos 0 -Master_SSL_Allowed No -Master_SSL_CA_File -Master_SSL_CA_Path -Master_SSL_Cert -Master_SSL_Cipher -Master_SSL_Key -Seconds_Behind_Master # -Master_SSL_Verify_Server_Cert No -Last_IO_Errno # -Last_IO_Error # -Last_SQL_Errno 1364 -Last_SQL_Error Could not execute Write_rows event on table test.t9; Field 'e' doesn't have a default value, Error_code: 1364; handler error HA_ERR_ROWS_EVENT_APPLY; the event's master log master-bin.000001, end_log_pos 330 -SET GLOBAL SQL_SLAVE_SKIP_COUNTER=2; -START SLAVE; +select * from t9; +a b c d e f g h i +1 b1b1b1b1b1b1b1b1 Kyle 0000-00-00 00:00:00 0 NULL NULL +2 b1b1b1b1b1b1b1b1 JOE 0000-00-00 00:00:00 0 NULL NULL +3 b1b1b1b1b1b1b1b1 QA 0000-00-00 00:00:00 0 NULL NULL *** Create t10 on slave *** STOP SLAVE; RESET SLAVE; === modified file 'mysql-test/suite/rpl/r/rpl_extraCol_myisam.result' --- a/mysql-test/suite/rpl/r/rpl_extraCol_myisam.result 2009-08-28 14:13:27 +0000 +++ b/mysql-test/suite/rpl/r/rpl_extraCol_myisam.result 2009-10-22 00:10:42 +0000 @@ -404,7 +404,11 @@ STOP SLAVE; RESET SLAVE; CREATE TABLE t9 (a INT KEY, b BLOB, c CHAR(5), d TIMESTAMP, -e INT NOT NULL) ENGINE='MyISAM'; +e INT NOT NULL, +f text not null, +g text, +h blob not null, +i blob) ENGINE='MyISAM'; *** Create t9 on Master *** CREATE TABLE t9 (a INT PRIMARY KEY, b BLOB, c CHAR(5) ) ENGINE='MyISAM'; @@ -415,47 +419,11 @@ START SLAVE; set @b1 = 'b1b1b1b1'; set @b1 = concat(@b1,@b1); INSERT INTO t9 () VALUES(1,@b1,'Kyle'),(2,@b1,'JOE'),(3,@b1,'QA'); -SHOW SLAVE STATUS; -Slave_IO_State # -Master_Host 127.0.0.1 -Master_User root -Master_Port # -Connect_Retry 1 -Master_Log_File master-bin.000001 -Read_Master_Log_Pos # -Relay_Log_File # -Relay_Log_Pos # -Relay_Master_Log_File master-bin.000001 -Slave_IO_Running Yes -Slave_SQL_Running No -Replicate_Do_DB -Replicate_Ignore_DB -Replicate_Do_Table -Replicate_Ignore_Table # -Replicate_Wild_Do_Table -Replicate_Wild_Ignore_Table -Last_Errno 1364 -Last_Error Could not execute Write_rows event on table test.t9; Field 'e' doesn't have a default value, Error_code: 1364; handler error HA_ERR_ROWS_EVENT_APPLY; the event's master log master-bin.000001, end_log_pos 330 -Skip_Counter 0 -Exec_Master_Log_Pos # -Relay_Log_Space # -Until_Condition None -Until_Log_File -Until_Log_Pos 0 -Master_SSL_Allowed No -Master_SSL_CA_File -Master_SSL_CA_Path -Master_SSL_Cert -Master_SSL_Cipher -Master_SSL_Key -Seconds_Behind_Master # -Master_SSL_Verify_Server_Cert No -Last_IO_Errno # -Last_IO_Error # -Last_SQL_Errno 1364 -Last_SQL_Error Could not execute Write_rows event on table test.t9; Field 'e' doesn't have a default value, Error_code: 1364; handler error HA_ERR_ROWS_EVENT_APPLY; the event's master log master-bin.000001, end_log_pos 330 -SET GLOBAL SQL_SLAVE_SKIP_COUNTER=2; -START SLAVE; +select * from t9; +a b c d e f g h i +1 b1b1b1b1b1b1b1b1 Kyle 0000-00-00 00:00:00 0 NULL NULL +2 b1b1b1b1b1b1b1b1 JOE 0000-00-00 00:00:00 0 NULL NULL +3 b1b1b1b1b1b1b1b1 QA 0000-00-00 00:00:00 0 NULL NULL *** Create t10 on slave *** STOP SLAVE; RESET SLAVE; === modified file 'mysql-test/suite/rpl/r/rpl_get_lock.result' --- a/mysql-test/suite/rpl/r/rpl_get_lock.result 2008-02-12 19:09:16 +0000 +++ b/mysql-test/suite/rpl/r/rpl_get_lock.result 2009-11-18 14:50:31 +0000 @@ -4,6 +4,7 @@ reset master; reset slave; drop table if exists t1,t2,t3,t4,t5,t6,t7,t8,t9; start slave; +CALL mtr.add_suppression("Statement may not be safe to log in statement format."); create table t1(n int); insert into t1 values(get_lock("lock",2)); select get_lock("lock",2); === added file 'mysql-test/suite/rpl/r/rpl_loaddata_symlink.result' --- a/mysql-test/suite/rpl/r/rpl_loaddata_symlink.result 1970-01-01 00:00:00 +0000 +++ b/mysql-test/suite/rpl/r/rpl_loaddata_symlink.result 2009-11-28 04:43:16 +0000 @@ -0,0 +1,17 @@ +stop slave; +drop table if exists t1,t2,t3,t4,t5,t6,t7,t8,t9; +reset master; +reset slave; +drop table if exists t1,t2,t3,t4,t5,t6,t7,t8,t9; +start slave; +create table t1(a int not null auto_increment, b int, primary key(a) ); +load data infile '../../std_data/rpl_loaddata.dat' into table t1; +select * from t1; +a b +1 10 +2 15 +select * from t1; +a b +1 10 +2 15 +drop table t1; === modified file 'mysql-test/suite/rpl/r/rpl_mixed_ddl_dml.result' --- a/mysql-test/suite/rpl/r/rpl_mixed_ddl_dml.result 2008-11-04 17:07:14 +0000 +++ b/mysql-test/suite/rpl/r/rpl_mixed_ddl_dml.result 2010-01-19 10:36:52 +0000 @@ -18,6 +18,7 @@ Server_id Host Port Rpl_recovery_rank Ma drop table t1; stop slave; create table t2(id int auto_increment primary key, created datetime); +SET TIME_ZONE= '+03:00'; set timestamp=12345; insert into t2 set created=now(); select * from t2; === added file 'mysql-test/suite/rpl/r/rpl_nondeterministic_functions.result' --- a/mysql-test/suite/rpl/r/rpl_nondeterministic_functions.result 1970-01-01 00:00:00 +0000 +++ b/mysql-test/suite/rpl/r/rpl_nondeterministic_functions.result 2009-11-18 14:50:31 +0000 @@ -0,0 +1,26 @@ +stop slave; +drop table if exists t1,t2,t3,t4,t5,t6,t7,t8,t9; +reset master; +reset slave; +drop table if exists t1,t2,t3,t4,t5,t6,t7,t8,t9; +start slave; +CREATE TABLE t1 (a VARCHAR(1000)); +INSERT INTO t1 VALUES (CONNECTION_ID()); +INSERT INTO t1 VALUES (CONNECTION_ID()); +INSERT INTO t1 VALUES +(CURDATE()), +(CURRENT_DATE()), +(CURRENT_TIME()), +(CURRENT_TIMESTAMP()), +(CURTIME()), +(LOCALTIME()), +(LOCALTIMESTAMP()), +(NOW()), +(UNIX_TIMESTAMP()), +(UTC_DATE()), +(UTC_TIME()), +(UTC_TIMESTAMP()); +INSERT INTO t1 VALUES (RAND()); +INSERT INTO t1 VALUES (LAST_INSERT_ID()); +Comparing tables master:test.t1 and slave:test.t1 +DROP TABLE t1; === added file 'mysql-test/suite/rpl/r/rpl_not_null_innodb.result' --- a/mysql-test/suite/rpl/r/rpl_not_null_innodb.result 1970-01-01 00:00:00 +0000 +++ b/mysql-test/suite/rpl/r/rpl_not_null_innodb.result 2009-10-22 00:19:52 +0000 @@ -0,0 +1,202 @@ +stop slave; +drop table if exists t1,t2,t3,t4,t5,t6,t7,t8,t9; +reset master; +reset slave; +drop table if exists t1,t2,t3,t4,t5,t6,t7,t8,t9; +start slave; +SET SQL_LOG_BIN= 0; +CREATE TABLE t1(`a` INT, `b` DATE DEFAULT NULL, +`c` INT DEFAULT NULL, +PRIMARY KEY(`a`)) ENGINE=Innodb DEFAULT CHARSET=LATIN1; +CREATE TABLE t2(`a` INT, `b` DATE DEFAULT NULL, +PRIMARY KEY(`a`)) ENGINE=Innodb DEFAULT CHARSET=LATIN1; +CREATE TABLE t3(`a` INT, `b` DATE DEFAULT NULL, +PRIMARY KEY(`a`)) ENGINE=Innodb DEFAULT CHARSET=LATIN1; +CREATE TABLE t4(`a` INT, `b` DATE DEFAULT NULL, +`c` INT DEFAULT NULL, +PRIMARY KEY(`a`)) ENGINE=Innodb DEFAULT CHARSET=LATIN1; +SET SQL_LOG_BIN= 1; +CREATE TABLE t1(`a` INT, `b` DATE DEFAULT NULL, +`c` INT DEFAULT NULL, +PRIMARY KEY(`a`)) ENGINE=Innodb DEFAULT CHARSET=LATIN1; +CREATE TABLE t2(`a` INT, `b` DATE DEFAULT NULL, +PRIMARY KEY(`a`)) ENGINE=Innodb DEFAULT CHARSET=LATIN1; +CREATE TABLE t3(`a` INT, `b` DATE DEFAULT '0000-00-00', +`c` INT DEFAULT 500, +PRIMARY KEY(`a`)) ENGINE=Innodb DEFAULT CHARSET=LATIN1; +CREATE TABLE t4(`a` INT, `b` DATE DEFAULT '0000-00-00', +PRIMARY KEY(`a`)) ENGINE=Innodb DEFAULT CHARSET=LATIN1; +************* EXECUTION WITH INSERTS ************* +INSERT INTO t1(a,b,c) VALUES (1, null, 1); +INSERT INTO t1(a,b,c) VALUES (2,'1111-11-11', 2); +INSERT INTO t1(a,b) VALUES (3, null); +INSERT INTO t1(a,c) VALUES (4, 4); +INSERT INTO t1(a) VALUES (5); +INSERT INTO t2(a,b) VALUES (1, null); +INSERT INTO t2(a,b) VALUES (2,'1111-11-11'); +INSERT INTO t2(a) VALUES (3); +INSERT INTO t3(a,b) VALUES (1, null); +INSERT INTO t3(a,b) VALUES (2,'1111-11-11'); +INSERT INTO t3(a) VALUES (3); +INSERT INTO t4(a,b,c) VALUES (1, null, 1); +INSERT INTO t4(a,b,c) VALUES (2,'1111-11-11', 2); +INSERT INTO t4(a,b) VALUES (3, null); +INSERT INTO t4(a,c) VALUES (4, 4); +INSERT INTO t4(a) VALUES (5); +************* SHOWING THE RESULT SETS WITH INSERTS ************* +TABLES t1 and t2 must be equal otherwise an error will be thrown. +Comparing tables master:test.t1 and slave:test.t1 +Comparing tables master:test.t2 and slave:test.t2 +TABLES t2 and t3 must be different. +SELECT * FROM t3 ORDER BY a; +a b +1 NULL +2 1111-11-11 +3 NULL +SELECT * FROM t3 ORDER BY a; +a b c +1 NULL 500 +2 1111-11-11 500 +3 NULL 500 +SELECT * FROM t4 ORDER BY a; +a b c +1 NULL 1 +2 1111-11-11 2 +3 NULL NULL +4 NULL 4 +5 NULL NULL +SELECT * FROM t4 ORDER BY a; +a b +1 NULL +2 1111-11-11 +3 NULL +4 NULL +5 NULL +************* EXECUTION WITH UPDATES and REPLACES ************* +DELETE FROM t1; +INSERT INTO t1(a,b,c) VALUES (1,'1111-11-11', 1); +REPLACE INTO t1(a,b,c) VALUES (2,'1111-11-11', 2); +UPDATE t1 set b= NULL, c= 300 where a= 1; +REPLACE INTO t1(a,b,c) VALUES (2, NULL, 300); +************* SHOWING THE RESULT SETS WITH UPDATES and REPLACES ************* +TABLES t1 and t2 must be equal otherwise an error will be thrown. +Comparing tables master:test.t1 and slave:test.t1 +************* CLEANING ************* +DROP TABLE t1; +DROP TABLE t2; +DROP TABLE t3; +DROP TABLE t4; +SET SQL_LOG_BIN= 0; +CREATE TABLE t1 (`a` INT, `b` BIT DEFAULT NULL, `c` BIT DEFAULT NULL, +PRIMARY KEY (`a`)) ENGINE= Innodb; +SET SQL_LOG_BIN= 1; +CREATE TABLE t1 (`a` INT, `b` BIT DEFAULT b'01', `c` BIT DEFAULT NULL, +PRIMARY KEY (`a`)) ENGINE= Innodb; +************* EXECUTION WITH INSERTS ************* +INSERT INTO t1(a,b,c) VALUES (1, null, b'01'); +INSERT INTO t1(a,b,c) VALUES (2,b'00', b'01'); +INSERT INTO t1(a,b) VALUES (3, null); +INSERT INTO t1(a,c) VALUES (4, b'01'); +INSERT INTO t1(a) VALUES (5); +************* SHOWING THE RESULT SETS WITH INSERTS ************* +TABLES t1 and t2 must be different. +SELECT a,b+0,c+0 FROM t1 ORDER BY a; +a b+0 c+0 +1 NULL 1 +2 0 1 +3 NULL NULL +4 NULL 1 +5 NULL NULL +SELECT a,b+0,c+0 FROM t1 ORDER BY a; +a b+0 c+0 +1 NULL 1 +2 0 1 +3 NULL NULL +4 NULL 1 +5 NULL NULL +************* EXECUTION WITH UPDATES and REPLACES ************* +DELETE FROM t1; +INSERT INTO t1(a,b,c) VALUES (1,b'00', b'01'); +REPLACE INTO t1(a,b,c) VALUES (2,b'00',b'01'); +UPDATE t1 set b= NULL, c= b'00' where a= 1; +REPLACE INTO t1(a,b,c) VALUES (2, NULL, b'00'); +************* SHOWING THE RESULT SETS WITH UPDATES and REPLACES ************* +TABLES t1 and t2 must be equal otherwise an error will be thrown. +Comparing tables master:test.t1 and slave:test.t1 +DROP TABLE t1; +################################################################################ +# NULL ---> NOT NULL (STRICT MODE) +# UNCOMMENT THIS AFTER FIXING BUG#43992 +################################################################################ +################################################################################ +# NULL ---> NOT NULL (NON-STRICT MODE) +################################################################################ +SET SQL_LOG_BIN= 0; +CREATE TABLE t1(`a` INT NOT NULL, `b` INT, +PRIMARY KEY(`a`)) ENGINE=Innodb DEFAULT CHARSET=LATIN1; +CREATE TABLE t2(`a` INT NOT NULL, `b` INT, +PRIMARY KEY(`a`)) ENGINE=Innodb DEFAULT CHARSET=LATIN1; +CREATE TABLE t3(`a` INT NOT NULL, `b` INT, +PRIMARY KEY(`a`)) ENGINE=Innodb DEFAULT CHARSET=LATIN1; +SET SQL_LOG_BIN= 1; +CREATE TABLE t1(`a` INT NOT NULL, `b` INT NOT NULL, +`c` INT NOT NULL, +PRIMARY KEY(`a`)) ENGINE=Innodb DEFAULT CHARSET=LATIN1; +CREATE TABLE t2(`a` INT NOT NULL, `b` INT NOT NULL, +`c` INT, +PRIMARY KEY(`a`)) ENGINE=Innodb DEFAULT CHARSET=LATIN1; +CREATE TABLE t3(`a` INT NOT NULL, `b` INT NOT NULL, +`c` INT DEFAULT 500, +PRIMARY KEY(`a`)) ENGINE=Innodb DEFAULT CHARSET=LATIN1; +************* EXECUTION WITH INSERTS ************* +INSERT INTO t1(a) VALUES (1); +INSERT INTO t1(a, b) VALUES (2, NULL); +INSERT INTO t1(a, b) VALUES (3, 1); +INSERT INTO t2(a) VALUES (1); +INSERT INTO t2(a, b) VALUES (2, NULL); +INSERT INTO t2(a, b) VALUES (3, 1); +INSERT INTO t3(a) VALUES (1); +INSERT INTO t3(a, b) VALUES (2, NULL); +INSERT INTO t3(a, b) VALUES (3, 1); +INSERT INTO t3(a, b) VALUES (4, 1); +REPLACE INTO t3(a, b) VALUES (5, null); +REPLACE INTO t3(a, b) VALUES (3, null); +UPDATE t3 SET b = NULL where a = 4; +************* SHOWING THE RESULT SETS ************* +SELECT * FROM t1 ORDER BY a; +a b +1 NULL +2 NULL +3 1 +SELECT * FROM t1 ORDER BY a; +a b c +1 0 0 +2 0 0 +3 1 0 +SELECT * FROM t2 ORDER BY a; +a b +1 NULL +2 NULL +3 1 +SELECT * FROM t2 ORDER BY a; +a b c +1 0 NULL +2 0 NULL +3 1 NULL +SELECT * FROM t3 ORDER BY a; +a b +1 NULL +2 NULL +3 NULL +4 NULL +5 NULL +SELECT * FROM t3 ORDER BY a; +a b c +1 0 500 +2 0 500 +3 0 500 +4 0 500 +5 0 500 +DROP TABLE t1; +DROP TABLE t2; +DROP TABLE t3; === added file 'mysql-test/suite/rpl/r/rpl_not_null_myisam.result' --- a/mysql-test/suite/rpl/r/rpl_not_null_myisam.result 1970-01-01 00:00:00 +0000 +++ b/mysql-test/suite/rpl/r/rpl_not_null_myisam.result 2009-10-22 00:19:52 +0000 @@ -0,0 +1,202 @@ +stop slave; +drop table if exists t1,t2,t3,t4,t5,t6,t7,t8,t9; +reset master; +reset slave; +drop table if exists t1,t2,t3,t4,t5,t6,t7,t8,t9; +start slave; +SET SQL_LOG_BIN= 0; +CREATE TABLE t1(`a` INT, `b` DATE DEFAULT NULL, +`c` INT DEFAULT NULL, +PRIMARY KEY(`a`)) ENGINE=MyISAM DEFAULT CHARSET=LATIN1; +CREATE TABLE t2(`a` INT, `b` DATE DEFAULT NULL, +PRIMARY KEY(`a`)) ENGINE=MyISAM DEFAULT CHARSET=LATIN1; +CREATE TABLE t3(`a` INT, `b` DATE DEFAULT NULL, +PRIMARY KEY(`a`)) ENGINE=MyISAM DEFAULT CHARSET=LATIN1; +CREATE TABLE t4(`a` INT, `b` DATE DEFAULT NULL, +`c` INT DEFAULT NULL, +PRIMARY KEY(`a`)) ENGINE=MyISAM DEFAULT CHARSET=LATIN1; +SET SQL_LOG_BIN= 1; +CREATE TABLE t1(`a` INT, `b` DATE DEFAULT NULL, +`c` INT DEFAULT NULL, +PRIMARY KEY(`a`)) ENGINE=MyISAM DEFAULT CHARSET=LATIN1; +CREATE TABLE t2(`a` INT, `b` DATE DEFAULT NULL, +PRIMARY KEY(`a`)) ENGINE=MyISAM DEFAULT CHARSET=LATIN1; +CREATE TABLE t3(`a` INT, `b` DATE DEFAULT '0000-00-00', +`c` INT DEFAULT 500, +PRIMARY KEY(`a`)) ENGINE=MyISAM DEFAULT CHARSET=LATIN1; +CREATE TABLE t4(`a` INT, `b` DATE DEFAULT '0000-00-00', +PRIMARY KEY(`a`)) ENGINE=MyISAM DEFAULT CHARSET=LATIN1; +************* EXECUTION WITH INSERTS ************* +INSERT INTO t1(a,b,c) VALUES (1, null, 1); +INSERT INTO t1(a,b,c) VALUES (2,'1111-11-11', 2); +INSERT INTO t1(a,b) VALUES (3, null); +INSERT INTO t1(a,c) VALUES (4, 4); +INSERT INTO t1(a) VALUES (5); +INSERT INTO t2(a,b) VALUES (1, null); +INSERT INTO t2(a,b) VALUES (2,'1111-11-11'); +INSERT INTO t2(a) VALUES (3); +INSERT INTO t3(a,b) VALUES (1, null); +INSERT INTO t3(a,b) VALUES (2,'1111-11-11'); +INSERT INTO t3(a) VALUES (3); +INSERT INTO t4(a,b,c) VALUES (1, null, 1); +INSERT INTO t4(a,b,c) VALUES (2,'1111-11-11', 2); +INSERT INTO t4(a,b) VALUES (3, null); +INSERT INTO t4(a,c) VALUES (4, 4); +INSERT INTO t4(a) VALUES (5); +************* SHOWING THE RESULT SETS WITH INSERTS ************* +TABLES t1 and t2 must be equal otherwise an error will be thrown. +Comparing tables master:test.t1 and slave:test.t1 +Comparing tables master:test.t2 and slave:test.t2 +TABLES t2 and t3 must be different. +SELECT * FROM t3 ORDER BY a; +a b +1 NULL +2 1111-11-11 +3 NULL +SELECT * FROM t3 ORDER BY a; +a b c +1 NULL 500 +2 1111-11-11 500 +3 NULL 500 +SELECT * FROM t4 ORDER BY a; +a b c +1 NULL 1 +2 1111-11-11 2 +3 NULL NULL +4 NULL 4 +5 NULL NULL +SELECT * FROM t4 ORDER BY a; +a b +1 NULL +2 1111-11-11 +3 NULL +4 NULL +5 NULL +************* EXECUTION WITH UPDATES and REPLACES ************* +DELETE FROM t1; +INSERT INTO t1(a,b,c) VALUES (1,'1111-11-11', 1); +REPLACE INTO t1(a,b,c) VALUES (2,'1111-11-11', 2); +UPDATE t1 set b= NULL, c= 300 where a= 1; +REPLACE INTO t1(a,b,c) VALUES (2, NULL, 300); +************* SHOWING THE RESULT SETS WITH UPDATES and REPLACES ************* +TABLES t1 and t2 must be equal otherwise an error will be thrown. +Comparing tables master:test.t1 and slave:test.t1 +************* CLEANING ************* +DROP TABLE t1; +DROP TABLE t2; +DROP TABLE t3; +DROP TABLE t4; +SET SQL_LOG_BIN= 0; +CREATE TABLE t1 (`a` INT, `b` BIT DEFAULT NULL, `c` BIT DEFAULT NULL, +PRIMARY KEY (`a`)) ENGINE= MyISAM; +SET SQL_LOG_BIN= 1; +CREATE TABLE t1 (`a` INT, `b` BIT DEFAULT b'01', `c` BIT DEFAULT NULL, +PRIMARY KEY (`a`)) ENGINE= MyISAM; +************* EXECUTION WITH INSERTS ************* +INSERT INTO t1(a,b,c) VALUES (1, null, b'01'); +INSERT INTO t1(a,b,c) VALUES (2,b'00', b'01'); +INSERT INTO t1(a,b) VALUES (3, null); +INSERT INTO t1(a,c) VALUES (4, b'01'); +INSERT INTO t1(a) VALUES (5); +************* SHOWING THE RESULT SETS WITH INSERTS ************* +TABLES t1 and t2 must be different. +SELECT a,b+0,c+0 FROM t1 ORDER BY a; +a b+0 c+0 +1 NULL 1 +2 0 1 +3 NULL NULL +4 NULL 1 +5 NULL NULL +SELECT a,b+0,c+0 FROM t1 ORDER BY a; +a b+0 c+0 +1 NULL 1 +2 0 1 +3 NULL NULL +4 NULL 1 +5 NULL NULL +************* EXECUTION WITH UPDATES and REPLACES ************* +DELETE FROM t1; +INSERT INTO t1(a,b,c) VALUES (1,b'00', b'01'); +REPLACE INTO t1(a,b,c) VALUES (2,b'00',b'01'); +UPDATE t1 set b= NULL, c= b'00' where a= 1; +REPLACE INTO t1(a,b,c) VALUES (2, NULL, b'00'); +************* SHOWING THE RESULT SETS WITH UPDATES and REPLACES ************* +TABLES t1 and t2 must be equal otherwise an error will be thrown. +Comparing tables master:test.t1 and slave:test.t1 +DROP TABLE t1; +################################################################################ +# NULL ---> NOT NULL (STRICT MODE) +# UNCOMMENT THIS AFTER FIXING BUG#43992 +################################################################################ +################################################################################ +# NULL ---> NOT NULL (NON-STRICT MODE) +################################################################################ +SET SQL_LOG_BIN= 0; +CREATE TABLE t1(`a` INT NOT NULL, `b` INT, +PRIMARY KEY(`a`)) ENGINE=MyISAM DEFAULT CHARSET=LATIN1; +CREATE TABLE t2(`a` INT NOT NULL, `b` INT, +PRIMARY KEY(`a`)) ENGINE=MyISAM DEFAULT CHARSET=LATIN1; +CREATE TABLE t3(`a` INT NOT NULL, `b` INT, +PRIMARY KEY(`a`)) ENGINE=MyISAM DEFAULT CHARSET=LATIN1; +SET SQL_LOG_BIN= 1; +CREATE TABLE t1(`a` INT NOT NULL, `b` INT NOT NULL, +`c` INT NOT NULL, +PRIMARY KEY(`a`)) ENGINE=MyISAM DEFAULT CHARSET=LATIN1; +CREATE TABLE t2(`a` INT NOT NULL, `b` INT NOT NULL, +`c` INT, +PRIMARY KEY(`a`)) ENGINE=MyISAM DEFAULT CHARSET=LATIN1; +CREATE TABLE t3(`a` INT NOT NULL, `b` INT NOT NULL, +`c` INT DEFAULT 500, +PRIMARY KEY(`a`)) ENGINE=MyISAM DEFAULT CHARSET=LATIN1; +************* EXECUTION WITH INSERTS ************* +INSERT INTO t1(a) VALUES (1); +INSERT INTO t1(a, b) VALUES (2, NULL); +INSERT INTO t1(a, b) VALUES (3, 1); +INSERT INTO t2(a) VALUES (1); +INSERT INTO t2(a, b) VALUES (2, NULL); +INSERT INTO t2(a, b) VALUES (3, 1); +INSERT INTO t3(a) VALUES (1); +INSERT INTO t3(a, b) VALUES (2, NULL); +INSERT INTO t3(a, b) VALUES (3, 1); +INSERT INTO t3(a, b) VALUES (4, 1); +REPLACE INTO t3(a, b) VALUES (5, null); +REPLACE INTO t3(a, b) VALUES (3, null); +UPDATE t3 SET b = NULL where a = 4; +************* SHOWING THE RESULT SETS ************* +SELECT * FROM t1 ORDER BY a; +a b +1 NULL +2 NULL +3 1 +SELECT * FROM t1 ORDER BY a; +a b c +1 0 0 +2 0 0 +3 1 0 +SELECT * FROM t2 ORDER BY a; +a b +1 NULL +2 NULL +3 1 +SELECT * FROM t2 ORDER BY a; +a b c +1 0 NULL +2 0 NULL +3 1 NULL +SELECT * FROM t3 ORDER BY a; +a b +1 NULL +2 NULL +3 NULL +4 NULL +5 NULL +SELECT * FROM t3 ORDER BY a; +a b c +1 0 500 +2 0 500 +3 0 500 +4 0 500 +5 0 500 +DROP TABLE t1; +DROP TABLE t2; +DROP TABLE t3; === modified file 'mysql-test/suite/rpl/r/rpl_relayspace.result' --- a/mysql-test/suite/rpl/r/rpl_relayspace.result 2007-06-27 12:28:02 +0000 +++ b/mysql-test/suite/rpl/r/rpl_relayspace.result 2010-01-18 12:56:10 +0000 @@ -14,6 +14,6 @@ start slave io_thread; stop slave io_thread; reset slave; start slave; -select master_pos_wait('master-bin.001',200,6)=-1; -master_pos_wait('master-bin.001',200,6)=-1 +select master_pos_wait('MASTER_LOG_FILE',200,30)=-1; +master_pos_wait('MASTER_LOG_FILE',200,30)=-1 0 === modified file 'mysql-test/suite/rpl/r/rpl_row_create_table.result' --- a/mysql-test/suite/rpl/r/rpl_row_create_table.result 2009-10-06 00:54:00 +0000 +++ b/mysql-test/suite/rpl/r/rpl_row_create_table.result 2009-11-27 13:34:39 +0000 @@ -476,4 +476,30 @@ master-bin.000001 # Table_map # # table_ master-bin.000001 # Write_rows # # table_id: # flags: STMT_END_F master-bin.000001 # Query # # COMMIT DROP DATABASE mysqltest1; +stop slave; +drop table if exists t1,t2,t3,t4,t5,t6,t7,t8,t9; +reset master; +reset slave; +drop table if exists t1,t2,t3,t4,t5,t6,t7,t8,t9; +start slave; +CREATE TEMPORARY TABLE t7(c1 INT); +CREATE TABLE t5(c1 INT); +CREATE TABLE t4(c1 INT); +CREATE VIEW bug48506_t1 AS SELECT 1; +CREATE VIEW bug48506_t2 AS SELECT * FROM t4; +CREATE VIEW bug48506_t3 AS SELECT t5.c1 AS A, t4.c1 AS B FROM t5, t4; +CREATE TABLE bug48506_t4(c1 INT); +DROP VIEW bug48506_t1, bug48506_t2, bug48506_t3; +DROP TABLE bug48506_t4; +CREATE TABLE IF NOT EXISTS bug48506_t1 LIKE t7; +CREATE TABLE IF NOT EXISTS bug48506_t2 LIKE t7; +CREATE TABLE IF NOT EXISTS bug48506_t3 LIKE t7; +CREATE TABLE IF NOT EXISTS bug48506_t4 LIKE t7; +SHOW TABLES LIKE 'bug48506%'; +Tables_in_test (bug48506%) +bug48506_t4 +DROP VIEW IF EXISTS bug48506_t1, bug48506_t2, bug48506_t3; +DROP TEMPORARY TABLES t7; +DROP TABLES t4, t5; +DROP TABLES IF EXISTS bug48506_t4; end of the tests === modified file 'mysql-test/suite/rpl/r/rpl_row_tabledefs_2myisam.result' --- a/mysql-test/suite/rpl/r/rpl_row_tabledefs_2myisam.result 2008-03-14 20:02:52 +0000 +++ b/mysql-test/suite/rpl/r/rpl_row_tabledefs_2myisam.result 2009-10-22 00:10:42 +0000 @@ -105,47 +105,9 @@ a b x 2 10 Foo is a bar INSERT INTO t9 VALUES (2); INSERT INTO t1_nodef VALUES (1,2); -SHOW SLAVE STATUS; -Slave_IO_State # -Master_Host 127.0.0.1 -Master_User root -Master_Port # -Connect_Retry 1 -Master_Log_File master-bin.000001 -Read_Master_Log_Pos # -Relay_Log_File # -Relay_Log_Pos # -Relay_Master_Log_File master-bin.000001 -Slave_IO_Running Yes -Slave_SQL_Running No -Replicate_Do_DB -Replicate_Ignore_DB -Replicate_Do_Table -Replicate_Ignore_Table -Replicate_Wild_Do_Table -Replicate_Wild_Ignore_Table -Last_Errno 1364 -Last_Error <Last_Error> -Skip_Counter 0 -Exec_Master_Log_Pos # -Relay_Log_Space # -Until_Condition None -Until_Log_File -Until_Log_Pos 0 -Master_SSL_Allowed No -Master_SSL_CA_File -Master_SSL_CA_Path -Master_SSL_Cert -Master_SSL_Cipher -Master_SSL_Key -Seconds_Behind_Master # -Master_SSL_Verify_Server_Cert No -Last_IO_Errno <Last_IO_Errno> -Last_IO_Error <Last_IO_Error> -Last_SQL_Errno 1364 -Last_SQL_Error <Last_SQL_Error> -SET GLOBAL SQL_SLAVE_SKIP_COUNTER=2; -START SLAVE; +select count(*) from t1_nodef; +count(*) +1 INSERT INTO t9 VALUES (2); **** On Master **** INSERT INTO t2 VALUES (2,4); === modified file 'mysql-test/suite/rpl/r/rpl_row_tabledefs_3innodb.result' --- a/mysql-test/suite/rpl/r/rpl_row_tabledefs_3innodb.result 2008-03-14 20:02:52 +0000 +++ b/mysql-test/suite/rpl/r/rpl_row_tabledefs_3innodb.result 2009-10-22 00:10:42 +0000 @@ -105,47 +105,9 @@ a b x 2 10 Foo is a bar INSERT INTO t9 VALUES (2); INSERT INTO t1_nodef VALUES (1,2); -SHOW SLAVE STATUS; -Slave_IO_State # -Master_Host 127.0.0.1 -Master_User root -Master_Port # -Connect_Retry 1 -Master_Log_File master-bin.000001 -Read_Master_Log_Pos # -Relay_Log_File # -Relay_Log_Pos # -Relay_Master_Log_File master-bin.000001 -Slave_IO_Running Yes -Slave_SQL_Running No -Replicate_Do_DB -Replicate_Ignore_DB -Replicate_Do_Table -Replicate_Ignore_Table -Replicate_Wild_Do_Table -Replicate_Wild_Ignore_Table -Last_Errno 1364 -Last_Error <Last_Error> -Skip_Counter 0 -Exec_Master_Log_Pos # -Relay_Log_Space # -Until_Condition None -Until_Log_File -Until_Log_Pos 0 -Master_SSL_Allowed No -Master_SSL_CA_File -Master_SSL_CA_Path -Master_SSL_Cert -Master_SSL_Cipher -Master_SSL_Key -Seconds_Behind_Master # -Master_SSL_Verify_Server_Cert No -Last_IO_Errno <Last_IO_Errno> -Last_IO_Error <Last_IO_Error> -Last_SQL_Errno 1364 -Last_SQL_Error <Last_SQL_Error> -SET GLOBAL SQL_SLAVE_SKIP_COUNTER=2; -START SLAVE; +select count(*) from t1_nodef; +count(*) +1 INSERT INTO t9 VALUES (2); **** On Master **** INSERT INTO t2 VALUES (2,4); === added file 'mysql-test/suite/rpl/r/rpl_row_trunc_temp.result' --- a/mysql-test/suite/rpl/r/rpl_row_trunc_temp.result 1970-01-01 00:00:00 +0000 +++ b/mysql-test/suite/rpl/r/rpl_row_trunc_temp.result 2009-11-22 05:10:33 +0000 @@ -0,0 +1,29 @@ +stop slave; +drop table if exists t1,t2,t3,t4,t5,t6,t7,t8,t9; +reset master; +reset slave; +drop table if exists t1,t2,t3,t4,t5,t6,t7,t8,t9; +start slave; +CREATE TEMPORARY TABLE t1(c1 INTEGER); +CREATE TABLE t2(c1 INTEGER); +CREATE TABLE t1(c1 INTEGER); +INSERT INTO t1 VALUES(1), (2); +INSERT INTO t2 VALUES(1), (2); +SELECT * FROM t1; +c1 +1 +2 +SELECT * FROM t2; +c1 +1 +2 +TRUNCATE t1; +TRUNCATE t2; +SELECT * FROM t1; +c1 +1 +2 +SELECT * FROM t2; +c1 +DROP TABLE t1; +DROP TABLE t2; === modified file 'mysql-test/suite/rpl/r/rpl_stm_000001.result' --- a/mysql-test/suite/rpl/r/rpl_stm_000001.result 2007-12-12 17:19:24 +0000 +++ b/mysql-test/suite/rpl/r/rpl_stm_000001.result 2009-11-18 14:50:31 +0000 @@ -4,6 +4,7 @@ reset master; reset slave; drop table if exists t1,t2,t3,t4,t5,t6,t7,t8,t9; start slave; +CALL mtr.add_suppression("Statement may not be safe to log in statement format."); create table t1 (word char(20) not null); load data infile '../../std_data/words.dat' into table t1; load data local infile 'MYSQL_TEST_DIR/std_data/words.dat' into table t1; === modified file 'mysql-test/suite/rpl/r/rpl_temporary.result' --- a/mysql-test/suite/rpl/r/rpl_temporary.result 2009-05-22 23:29:41 +0000 +++ b/mysql-test/suite/rpl/r/rpl_temporary.result 2010-01-11 13:15:28 +0000 @@ -4,7 +4,8 @@ reset master; reset slave; drop table if exists t1,t2,t3,t4,t5,t6,t7,t8,t9; start slave; -call mtr.add_suppression("Slave: Can\'t find record in \'user\' Error_code: 1032"); +SET sql_log_bin = 0; +SET sql_log_bin = 1; reset master; DROP TABLE IF EXISTS t1; CREATE TEMPORARY TABLE t1 (a char(1)); @@ -127,6 +128,8 @@ select * from t1; a 1 drop table t1; +SET sql_log_bin = 0; +SET sql_log_bin = 1; -- Bug#43748 -- make a user on the slave that can list but not kill system threads. FLUSH PRIVILEGES; === modified file 'mysql-test/suite/rpl/r/rpl_trigger.result' --- a/mysql-test/suite/rpl/r/rpl_trigger.result 2009-08-03 09:47:45 +0000 +++ b/mysql-test/suite/rpl/r/rpl_trigger.result 2009-11-18 14:50:31 +0000 @@ -4,6 +4,7 @@ reset master; reset slave; drop table if exists t1,t2,t3,t4,t5,t6,t7,t8,t9; start slave; +CALL mtr.add_suppression("Statement may not be safe to log in statement format."); DROP TABLE IF EXISTS t1; DROP TABLE IF EXISTS t2; DROP TABLE IF EXISTS t3; === modified file 'mysql-test/suite/rpl/t/disabled.def' --- a/mysql-test/suite/rpl/t/disabled.def 2009-09-27 10:12:58 +0000 +++ b/mysql-test/suite/rpl/t/disabled.def 2009-12-01 09:21:15 +0000 @@ -10,3 +10,4 @@ # ############################################################################## +rpl_row_create_table : Bug#45576 2009-12-01 joro rpl_row_create_table fails on PB2 === modified file 'mysql-test/suite/rpl/t/rpl_err_ignoredtable.test' --- a/mysql-test/suite/rpl/t/rpl_err_ignoredtable.test 2009-10-20 18:00:07 +0000 +++ b/mysql-test/suite/rpl/t/rpl_err_ignoredtable.test 2009-11-18 14:50:31 +0000 @@ -7,6 +7,8 @@ -- source include/master-slave.inc +CALL mtr.add_suppression("Statement may not be safe to log in statement format."); + connection master; create table t1 (a int primary key); create table t4 (a int primary key); @@ -14,19 +16,15 @@ create table t4 (a int primary key); --error 1022, ER_DUP_ENTRY insert into t1 values (1),(1); insert into t4 values (1),(2); -save_master_pos; -connection slave; # as the t1 table is ignored on the slave, the slave should be able to sync -sync_with_master; +sync_slave_with_master; # check that the table has been ignored, because otherwise the test is nonsense show tables like 't1'; show tables like 't4'; SELECT * FROM test.t4 ORDER BY a; connection master; drop table t1; -save_master_pos; -connection slave; -sync_with_master; +sync_slave_with_master; # Now test that even critical errors (connection killed) # are ignored if rules allow it. @@ -50,18 +48,17 @@ kill @id; drop table t2,t3; insert into t4 values (3),(4); connection master; +# The get_lock function causes warning for unsafe statement. +--disable_warnings --error 0,1317,2013 reap; +--enable_warnings connection master1; -save_master_pos; -connection slave; -sync_with_master; +sync_slave_with_master; SELECT * FROM test.t4 ORDER BY a; connection master1; DROP TABLE test.t4; -save_master_pos; -connection slave; -sync_with_master; +sync_slave_with_master; # End of 4.1 tests # Adding comment for force manual merge 5.0 -> wl1012. delete me if needed === modified file 'mysql-test/suite/rpl/t/rpl_get_lock.test' --- a/mysql-test/suite/rpl/t/rpl_get_lock.test 2007-06-27 12:28:02 +0000 +++ b/mysql-test/suite/rpl/t/rpl_get_lock.test 2009-11-18 14:50:31 +0000 @@ -1,7 +1,12 @@ source include/master-slave.inc; +CALL mtr.add_suppression("Statement may not be safe to log in statement format."); + create table t1(n int); +# Use of get_lock gives a warning for unsafeness if binlog_format=statement +--disable_warnings insert into t1 values(get_lock("lock",2)); +--enable_warnings dirty_close master; connection master1; select get_lock("lock",2); === added file 'mysql-test/suite/rpl/t/rpl_loaddata_symlink-master.opt' --- a/mysql-test/suite/rpl/t/rpl_loaddata_symlink-master.opt 1970-01-01 00:00:00 +0000 +++ b/mysql-test/suite/rpl/t/rpl_loaddata_symlink-master.opt 2009-11-28 04:43:16 +0000 @@ -0,0 +1 @@ +--secure-file-priv=$MYSQLTEST_VARDIR/std_data_master_link === added file 'mysql-test/suite/rpl/t/rpl_loaddata_symlink-master.sh' --- a/mysql-test/suite/rpl/t/rpl_loaddata_symlink-master.sh 1970-01-01 00:00:00 +0000 +++ b/mysql-test/suite/rpl/t/rpl_loaddata_symlink-master.sh 2009-11-28 04:43:16 +0000 @@ -0,0 +1 @@ +ln -s $MYSQLTEST_VARDIR/std_data $MYSQLTEST_VARDIR/std_data_master_link === added file 'mysql-test/suite/rpl/t/rpl_loaddata_symlink-slave.opt' --- a/mysql-test/suite/rpl/t/rpl_loaddata_symlink-slave.opt 1970-01-01 00:00:00 +0000 +++ b/mysql-test/suite/rpl/t/rpl_loaddata_symlink-slave.opt 2009-11-28 04:43:16 +0000 @@ -0,0 +1 @@ +--slave-load-tmpdir=$MYSQLTEST_VARDIR/std_data_slave_link === added file 'mysql-test/suite/rpl/t/rpl_loaddata_symlink-slave.sh' --- a/mysql-test/suite/rpl/t/rpl_loaddata_symlink-slave.sh 1970-01-01 00:00:00 +0000 +++ b/mysql-test/suite/rpl/t/rpl_loaddata_symlink-slave.sh 2009-11-28 04:43:16 +0000 @@ -0,0 +1 @@ +ln -s $MYSQLTEST_VARDIR/std_data $MYSQLTEST_VARDIR/std_data_slave_link === added file 'mysql-test/suite/rpl/t/rpl_loaddata_symlink.test' --- a/mysql-test/suite/rpl/t/rpl_loaddata_symlink.test 1970-01-01 00:00:00 +0000 +++ b/mysql-test/suite/rpl/t/rpl_loaddata_symlink.test 2009-11-28 04:43:16 +0000 @@ -0,0 +1,20 @@ +# +# BUG#43913 +# This test verifies if loading data infile will work fine +# if the path of the load data file is a symbolic link. +# +--source include/master-slave.inc +--source include/have_binlog_format_statement.inc + +create table t1(a int not null auto_increment, b int, primary key(a) ); +load data infile '../../std_data/rpl_loaddata.dat' into table t1; +select * from t1; + +sync_slave_with_master; +connection slave; +select * from t1; + +connection master; +drop table t1; +sync_slave_with_master; + === modified file 'mysql-test/suite/rpl/t/rpl_mixed_ddl_dml.test' --- a/mysql-test/suite/rpl/t/rpl_mixed_ddl_dml.test 2007-06-27 12:28:02 +0000 +++ b/mysql-test/suite/rpl/t/rpl_mixed_ddl_dml.test 2010-01-19 10:36:52 +0000 @@ -23,6 +23,7 @@ connection master; # Test replication of timestamp create table t2(id int auto_increment primary key, created datetime); +SET TIME_ZONE= '+03:00'; set timestamp=12345; insert into t2 set created=now(); select * from t2; === added file 'mysql-test/suite/rpl/t/rpl_nondeterministic_functions.test' --- a/mysql-test/suite/rpl/t/rpl_nondeterministic_functions.test 1970-01-01 00:00:00 +0000 +++ b/mysql-test/suite/rpl/t/rpl_nondeterministic_functions.test 2009-11-18 14:50:31 +0000 @@ -0,0 +1,53 @@ +# ==== Purpose ==== +# +# Test that nondeterministic system functions are correctly replicated. +# +# (Some functions are only correctly replicated if binlog_format=MIXED +# or ROW. See binlog_unsafe.test for a test that those variables are +# indeed unsafe.) +# +# ==== Implementation ==== +# +# We insert the values of each unsafe function into a table. Then we +# replicate and check that the table is identical on slave. +# +# ==== Related bugs ==== +# +# BUG#47995 + +--source include/master-slave.inc + +CREATE TABLE t1 (a VARCHAR(1000)); + +# We replicate the connection_id in the query_log_event +INSERT INTO t1 VALUES (CONNECTION_ID()); +--connection master1 +INSERT INTO t1 VALUES (CONNECTION_ID()); + +# We replicate the TIMESTAMP variable, so the following functions that +# are affected by the TIMESTAMP variable should be safe to replicate. +INSERT INTO t1 VALUES + (CURDATE()), + (CURRENT_DATE()), + (CURRENT_TIME()), + (CURRENT_TIMESTAMP()), + (CURTIME()), + (LOCALTIME()), + (LOCALTIMESTAMP()), + (NOW()), + (UNIX_TIMESTAMP()), + (UTC_DATE()), + (UTC_TIME()), + (UTC_TIMESTAMP()); + +# We replicate the random seed in a rand_log_event +INSERT INTO t1 VALUES (RAND()); +# We replicate the last_insert_id in an intvar_log_event +INSERT INTO t1 VALUES (LAST_INSERT_ID()); + +--sync_slave_with_master +--let $diff_table_1= master:test.t1 +--let $diff_table_2= slave:test.t1 +--source include/diff_tables.inc + +DROP TABLE t1; === added file 'mysql-test/suite/rpl/t/rpl_not_null_innodb.test' --- a/mysql-test/suite/rpl/t/rpl_not_null_innodb.test 1970-01-01 00:00:00 +0000 +++ b/mysql-test/suite/rpl/t/rpl_not_null_innodb.test 2009-10-22 00:15:45 +0000 @@ -0,0 +1,19 @@ +################################################################################# +# This test checks if the replication between "null" fields to either "null" +# fields or "not null" fields works properly. In the first case, the execution +# should work fine. In the second case, it may fail according to the sql_mode +# being used. +# +# The test is devided in three main parts: +# +# 1 - NULL --> NULL (no failures) +# 2 - NULL --> NOT NULL ( sql-mode = STRICT and failures) +# 3 - NULL --> NOT NULL ( sql-mode != STRICT and no failures) +# +################################################################################# +--source include/master-slave.inc +--source include/have_innodb.inc +--source include/have_binlog_format_row.inc + +let $engine=Innodb; +--source extra/rpl_tests/rpl_not_null.test === added file 'mysql-test/suite/rpl/t/rpl_not_null_myisam.test' --- a/mysql-test/suite/rpl/t/rpl_not_null_myisam.test 1970-01-01 00:00:00 +0000 +++ b/mysql-test/suite/rpl/t/rpl_not_null_myisam.test 2009-10-22 00:15:45 +0000 @@ -0,0 +1,18 @@ +################################################################################# +# This test checks if the replication between "null" fields to either "null" +# fields or "not null" fields works properly. In the first case, the execution +# should work fine. In the second case, it may fail according to the sql_mode +# being used. +# +# The test is devided in three main parts: +# +# 1 - NULL --> NULL (no failures) +# 2 - NULL --> NOT NULL ( sql-mode = STRICT and failures) +# 3 - NULL --> NOT NULL ( sql-mode != STRICT and no failures) +# +################################################################################# +--source include/master-slave.inc +--source include/have_binlog_format_row.inc + +let $engine=MyISAM; +--source extra/rpl_tests/rpl_not_null.test === modified file 'mysql-test/suite/rpl/t/rpl_relayspace.test' --- a/mysql-test/suite/rpl/t/rpl_relayspace.test 2008-02-28 11:36:14 +0000 +++ b/mysql-test/suite/rpl/t/rpl_relayspace.test 2010-01-18 12:56:10 +0000 @@ -2,8 +2,10 @@ # to force the deadlock after one event. source include/master-slave.inc; +let $master_log_file= query_get_value(SHOW MASTER STATUS, File, 1); connection slave; stop slave; +source include/wait_for_slave_to_stop.inc; connection master; # This will generate a master's binlog > 10 bytes create table t1 (a int); @@ -20,6 +22,7 @@ source include/wait_for_slave_param.inc; # A bug caused the I/O thread to refuse stopping. stop slave io_thread; +source include/wait_for_slave_io_to_stop.inc; reset slave; start slave; # The I/O thread stops filling the relay log when @@ -29,9 +32,11 @@ start slave; # So we should have a deadlock. # if it is not resolved automatically we'll detect # it with master_pos_wait that waits for farther than 1Ob; -# it will timeout after 10 seconds; +# it will timeout; # also the slave will probably not cooperate to shutdown # (as 2 threads are locked) -select master_pos_wait('master-bin.001',200,6)=-1; +--replace_result $master_log_file MASTER_LOG_FILE +eval select master_pos_wait('$master_log_file',200,30)=-1; + # End of 4.1 tests === modified file 'mysql-test/suite/rpl/t/rpl_rotate_logs.test' --- a/mysql-test/suite/rpl/t/rpl_rotate_logs.test 2009-03-12 09:48:41 +0000 +++ b/mysql-test/suite/rpl/t/rpl_rotate_logs.test 2010-01-27 21:53:08 +0000 @@ -153,7 +153,7 @@ disable_query_log; while ($1) { #eval means expand $ expressions - eval insert into t3 values($1 + 4); + eval insert HIGH_PRIORITY into t3 values($1 + 4); dec $1; } enable_query_log; === modified file 'mysql-test/suite/rpl/t/rpl_row_create_table.test' --- a/mysql-test/suite/rpl/t/rpl_row_create_table.test 2009-01-23 12:22:05 +0000 +++ b/mysql-test/suite/rpl/t/rpl_row_create_table.test 2009-11-27 13:34:39 +0000 @@ -292,4 +292,40 @@ connection master; DROP DATABASE mysqltest1; sync_slave_with_master; +# +# BUG#48506: crash in CREATE TABLE <existing_view> IF NOT EXISTS LIKE +# <tmp_tbl> with RBL +# + +source include/master-slave-reset.inc; + +connection master; +CREATE TEMPORARY TABLE t7(c1 INT); +CREATE TABLE t5(c1 INT); +CREATE TABLE t4(c1 INT); +CREATE VIEW bug48506_t1 AS SELECT 1; +CREATE VIEW bug48506_t2 AS SELECT * FROM t4; +CREATE VIEW bug48506_t3 AS SELECT t5.c1 AS A, t4.c1 AS B FROM t5, t4; +CREATE TABLE bug48506_t4(c1 INT); +--disable_warnings +sync_slave_with_master; +DROP VIEW bug48506_t1, bug48506_t2, bug48506_t3; +DROP TABLE bug48506_t4; + +connection master; +CREATE TABLE IF NOT EXISTS bug48506_t1 LIKE t7; +CREATE TABLE IF NOT EXISTS bug48506_t2 LIKE t7; +CREATE TABLE IF NOT EXISTS bug48506_t3 LIKE t7; +CREATE TABLE IF NOT EXISTS bug48506_t4 LIKE t7; +--enable_warnings +sync_slave_with_master; + +SHOW TABLES LIKE 'bug48506%'; + +connection master; +DROP VIEW IF EXISTS bug48506_t1, bug48506_t2, bug48506_t3; +DROP TEMPORARY TABLES t7; +DROP TABLES t4, t5; +DROP TABLES IF EXISTS bug48506_t4; +source include/master-slave-end.inc; --echo end of the tests === added file 'mysql-test/suite/rpl/t/rpl_row_trunc_temp.test' --- a/mysql-test/suite/rpl/t/rpl_row_trunc_temp.test 1970-01-01 00:00:00 +0000 +++ b/mysql-test/suite/rpl/t/rpl_row_trunc_temp.test 2009-11-22 05:10:33 +0000 @@ -0,0 +1,35 @@ +# +# Bug#48350 truncate temporary table crashes replication +# +# All statements operating on temporary tables should not be binlogged in RBR. +# However, before fix of bug#48350, 'TRUNCATE ...' statement on a temporary +# table was binlogged in RBR. +# + +--source include/master-slave.inc +--source include/have_binlog_format_row.inc + +#This statement is not binlogged in RBR. +CREATE TEMPORARY TABLE t1(c1 INTEGER); +CREATE TABLE t2(c1 INTEGER); +sync_slave_with_master; + +CREATE TABLE t1(c1 INTEGER); +INSERT INTO t1 VALUES(1), (2); +INSERT INTO t2 VALUES(1), (2); +SELECT * FROM t1; +SELECT * FROM t2; + +connection master; +TRUNCATE t1; +TRUNCATE t2; +sync_slave_with_master; +# t1 will have nothing, if 'TRUNCATE t1' has been replicate from master to +# slave. +SELECT * FROM t1; +SELECT * FROM t2; + +DROP TABLE t1; +connection master; +DROP TABLE t2; +--source include/master-slave-end.inc === modified file 'mysql-test/suite/rpl/t/rpl_temporary.test' --- a/mysql-test/suite/rpl/t/rpl_temporary.test 2009-05-22 23:29:41 +0000 +++ b/mysql-test/suite/rpl/t/rpl_temporary.test 2010-01-11 13:15:28 +0000 @@ -1,7 +1,10 @@ +-- source include/master-slave.inc + # Test need anonymous user when connection are made as "zedjzlcsjhd" +# But we only need it on the master, not the slave. +SET sql_log_bin = 0; source include/add_anonymous_users.inc; - --- source include/master-slave.inc +SET sql_log_bin = 1; # Clean up old slave's binlogs. # The slave is started with --log-slave-updates @@ -17,9 +20,6 @@ source include/add_anonymous_users.inc; save_master_pos; connection slave; -# Add suppression for expected warning(s) in slaves error log -call mtr.add_suppression("Slave: Can\'t find record in \'user\' Error_code: 1032"); - sync_with_master; reset master; @@ -291,7 +291,9 @@ drop table t1; --remove_file $MYSQLTEST_VARDIR/tmp/bug14157.sql # Delete the anonymous users +SET sql_log_bin = 0; source include/delete_anonymous_users.inc; +SET sql_log_bin = 1; === modified file 'mysql-test/suite/rpl/t/rpl_trigger.test' --- a/mysql-test/suite/rpl/t/rpl_trigger.test 2009-08-03 15:01:06 +0000 +++ b/mysql-test/suite/rpl/t/rpl_trigger.test 2009-11-18 14:50:31 +0000 @@ -5,6 +5,8 @@ --source include/have_binlog_format_mixed_or_statement.inc --source include/master-slave.inc +CALL mtr.add_suppression("Statement may not be safe to log in statement format."); + --disable_warnings DROP TABLE IF EXISTS t1; DROP TABLE IF EXISTS t2; @@ -89,7 +91,11 @@ end | delimiter ;| +# The trigger causes a warning for unsafe statement when +# binlog_format=statement since it uses get_lock. +--disable_warnings insert into t1 set a = now(); +--enable_warnings select a=b && a=c from t1; let $time=`select a from t1`; @@ -135,7 +141,11 @@ disconnect con2; truncate table t1; drop trigger t1_first; +# The trigger causes a warning for unsafe statement when +# binlog_format=statement since it uses get_lock. +--disable_warnings insert into t1 values ("2003-03-03","2003-03-03","2003-03-03"),(bug12480(),bug12480(),bug12480()),(now(),now(),now()); +--enable_warnings select a=b && a=c from t1; drop function bug12480; === modified file 'mysql-test/suite/rpl_ndb/r/rpl_ndb_extraCol.result' --- a/mysql-test/suite/rpl_ndb/r/rpl_ndb_extraCol.result 2009-08-29 08:30:59 +0000 +++ b/mysql-test/suite/rpl_ndb/r/rpl_ndb_extraCol.result 2009-10-22 00:21:50 +0000 @@ -400,62 +400,6 @@ set @b1 = concat(@b1,@b1); INSERT INTO t8 () VALUES(1,@b1,'Kyle'),(2,@b1,'JOE'),(3,@b1,'QA'); *** Drop t8 *** DROP TABLE t8; -STOP SLAVE; -RESET SLAVE; -CREATE TABLE t9 (a INT KEY, b BLOB, c CHAR(5), -d TIMESTAMP, -e INT NOT NULL) ENGINE='NDB'; -*** Create t9 on Master *** -CREATE TABLE t9 (a INT PRIMARY KEY, b BLOB, c CHAR(5) -) ENGINE='NDB'; -RESET MASTER; -*** Start Slave *** -START SLAVE; -*** Master Data Insert *** -set @b1 = 'b1b1b1b1'; -set @b1 = concat(@b1,@b1); -INSERT INTO t9 () VALUES(1,@b1,'Kyle'),(2,@b1,'JOE'),(3,@b1,'QA'); -SHOW SLAVE STATUS; -Slave_IO_State # -Master_Host 127.0.0.1 -Master_User root -Master_Port # -Connect_Retry 1 -Master_Log_File master-bin.000001 -Read_Master_Log_Pos # -Relay_Log_File # -Relay_Log_Pos # -Relay_Master_Log_File master-bin.000001 -Slave_IO_Running Yes -Slave_SQL_Running No -Replicate_Do_DB -Replicate_Ignore_DB -Replicate_Do_Table -Replicate_Ignore_Table # -Replicate_Wild_Do_Table -Replicate_Wild_Ignore_Table -Last_Errno 1364 -Last_Error Could not execute Write_rows event on table test.t9; Field 'e' doesn't have a default value, Error_code: 1364; handler error HA_ERR_ROWS_EVENT_APPLY; the event's master log master-bin.000001, end_log_pos 447 -Skip_Counter 0 -Exec_Master_Log_Pos # -Relay_Log_Space # -Until_Condition None -Until_Log_File -Until_Log_Pos 0 -Master_SSL_Allowed No -Master_SSL_CA_File -Master_SSL_CA_Path -Master_SSL_Cert -Master_SSL_Cipher -Master_SSL_Key -Seconds_Behind_Master # -Master_SSL_Verify_Server_Cert No -Last_IO_Errno # -Last_IO_Error # -Last_SQL_Errno 1364 -Last_SQL_Error Could not execute Write_rows event on table test.t9; Field 'e' doesn't have a default value, Error_code: 1364; handler error HA_ERR_ROWS_EVENT_APPLY; the event's master log master-bin.000001, end_log_pos 447 -SET GLOBAL SQL_SLAVE_SKIP_COUNTER=2; -START SLAVE; *** Create t10 on slave *** STOP SLAVE; RESET SLAVE; === modified file 'mysql-test/t/archive.test' --- a/mysql-test/t/archive.test 2009-12-03 11:19:05 +0000 +++ b/mysql-test/t/archive.test 2010-01-15 15:27:55 +0000 @@ -1625,3 +1625,24 @@ INSERT INTO t1 VALUES('aaaaaaaaaaaaaaaaa SELECT COUNT(t1.a) FROM t1, t1 a, t1 b, t1 c, t1 d, t1 e; DROP TABLE t1; SET @@join_buffer_size= @save_join_buffer_size; + +# +# BUG#47012 archive tables are not upgradeable, and server crashes on any access +# +let $MYSQLD_DATADIR= `SELECT @@datadir`; +copy_file std_data/bug47012.frm $MYSQLD_DATADIR/test/t1.frm; +copy_file std_data/bug47012.ARZ $MYSQLD_DATADIR/test/t1.ARZ; +copy_file std_data/bug47012.ARM $MYSQLD_DATADIR/test/t1.ARM; + +--error ER_TABLE_NEEDS_UPGRADE +SHOW CREATE TABLE t1; + +--error ER_TABLE_NEEDS_UPGRADE +SELECT * FROM t1; + +--error ER_TABLE_NEEDS_UPGRADE +INSERT INTO t1 (col1, col2) VALUES (1, "value"); + +REPAIR TABLE t1; +DROP TABLE t1; +remove_file $MYSQLD_DATADIR/test/t1.ARM; === added file 'mysql-test/t/bug47671-master.opt' --- a/mysql-test/t/bug47671-master.opt 1970-01-01 00:00:00 +0000 +++ b/mysql-test/t/bug47671-master.opt 2009-11-25 06:55:49 +0000 @@ -0,0 +1 @@ +--default-character-set=utf8 --skip-character-set-client-handshake === added file 'mysql-test/t/bug47671.test' --- a/mysql-test/t/bug47671.test 1970-01-01 00:00:00 +0000 +++ b/mysql-test/t/bug47671.test 2009-11-30 05:24:26 +0000 @@ -0,0 +1,9 @@ +# Embedded server doesn't support external clients +--source include/not_embedded.inc + +--echo # +--echo # Bug#47671 - wrong character-set after upgrade from 5.1.34 to 5.1.39 +--echo # +--echo # Extract only charset information from 'status' command output using regex +--replace_regex /.*mysql.*// /Connection.*// /Current.*// /SSL.*// /Using.*// /Server version.*// /Protocol.*// /UNIX.*// /Uptime.*// /Threads.*// /TCP.*// +--exec $MYSQL -e "status"; === modified file 'mysql-test/t/delayed.test' --- a/mysql-test/t/delayed.test 2009-03-11 15:32:42 +0000 +++ b/mysql-test/t/delayed.test 2010-01-15 15:27:55 +0000 @@ -341,4 +341,28 @@ drop table t1; set global low_priority_updates = @old_delayed_updates; + +--echo # +--echo # Bug #47682 strange behaviour of INSERT DELAYED +--echo # + +--disable_warnings +DROP TABLE IF EXISTS t1, t2; +--enable_warnings + +CREATE TABLE t1 (f1 integer); +CREATE TABLE t2 (f1 integer); + +FLUSH TABLES WITH READ LOCK; +LOCK TABLES t1 READ; + +# ER_CANT_UPDATE_WITH_READLOCK with normal execution +# ER_TABLE_NOT_LOCKED when executed as prepared statement +--error ER_CANT_UPDATE_WITH_READLOCK, ER_TABLE_NOT_LOCKED +INSERT DELAYED INTO t2 VALUES (1); + +UNLOCK TABLES; +DROP TABLE t1, t2; + + --echo End of 5.1 tests === modified file 'mysql-test/t/delete.test' --- a/mysql-test/t/delete.test 2009-09-28 10:48:52 +0000 +++ b/mysql-test/t/delete.test 2009-11-18 09:32:03 +0000 @@ -336,3 +336,25 @@ SELECT * FROM t2; SELECT * FROM t3; DROP TABLE t1, t2, t3; + +--echo # +--echo # Bug #46425 crash in Diagnostics_area::set_ok_status, +--echo # empty statement, DELETE IGNORE +--echo # + +CREATE table t1 (i INTEGER); + +INSERT INTO t1 VALUES (1); + +--delimiter | + +CREATE TRIGGER tr1 AFTER DELETE ON t1 FOR EACH ROW +BEGIN + INSERT INTO t1 SELECT * FROM t1 AS A; +END | + +--delimiter ; +--error ER_CANT_UPDATE_USED_TABLE_IN_SF_OR_TRG +DELETE IGNORE FROM t1; + +DROP TABLE t1; \ No newline at end of file === modified file 'mysql-test/t/disabled.def' --- a/mysql-test/t/disabled.def 2009-12-03 11:19:05 +0000 +++ b/mysql-test/t/disabled.def 2010-01-15 17:02:57 +0000 @@ -11,7 +11,4 @@ ############################################################################## kill : Bug#37780 2008-12-03 HHunger need some changes to be robust enough for pushbuild. query_cache_28249 : Bug#43861 2009-03-25 main.query_cache_28249 fails sporadically -partition_innodb_builtin : Bug#32430 2009-09-25 mattiasj Waiting for push of Innodb changes -partition_innodb_plugin : Bug#32430 2009-09-25 mattiasj Waiting for push of Innodb changes -innodb-autoinc : Bug#48482 2009-11-02 svoj innodb-autoinc.test fails with results difference rpl_killed_ddl : Bug#45520: rpl_killed_ddl fails sporadically in pb2 === modified file 'mysql-test/t/fulltext.test' --- a/mysql-test/t/fulltext.test 2009-12-27 13:54:41 +0000 +++ b/mysql-test/t/fulltext.test 2010-01-15 15:27:55 +0000 @@ -496,3 +496,44 @@ PREPARE s FROM EXECUTE s; DEALLOCATE PREPARE s; DROP TABLE t1; + +--echo # +--echo # Bug #47930: MATCH IN BOOLEAN MODE returns too many results +--echo # inside subquery +--echo # + +CREATE TABLE t1 (a int); +INSERT INTO t1 VALUES (1), (2); + +CREATE TABLE t2 (a int, b2 char(10), FULLTEXT KEY b2 (b2)); +INSERT INTO t2 VALUES (1,'Scargill'); + +CREATE TABLE t3 (a int, b int); +INSERT INTO t3 VALUES (1,1), (2,1); + +--echo # t2 should use full text index +EXPLAIN +SELECT count(*) FROM t1 WHERE + not exists( + SELECT 1 FROM t2, t3 + WHERE t3.a=t1.a AND MATCH(b2) AGAINST('scargill' IN BOOLEAN MODE) + ); + +--echo # should return 0 +SELECT count(*) FROM t1 WHERE + not exists( + SELECT 1 FROM t2, t3 + WHERE t3.a=t1.a AND MATCH(b2) AGAINST('scargill' IN BOOLEAN MODE) + ); + +--echo # should return 0 +SELECT count(*) FROM t1 WHERE + not exists( + SELECT 1 FROM t2 IGNORE INDEX (b2), t3 + WHERE t3.a=t1.a AND MATCH(b2) AGAINST('scargill' IN BOOLEAN MODE) + ); + +DROP TABLE t1,t2,t3; + + +--echo End of 5.1 tests === modified file 'mysql-test/t/func_group.test' --- a/mysql-test/t/func_group.test 2009-10-14 08:46:50 +0000 +++ b/mysql-test/t/func_group.test 2009-11-24 15:26:13 +0000 @@ -1053,4 +1053,35 @@ ORDER BY max; --echo # DROP TABLE t1; +--echo # +--echo # Bug#43668: Wrong comparison and MIN/MAX for YEAR(2) +--echo # +create table t1 (f1 year(2), f2 year(4), f3 date, f4 datetime); +insert into t1 values + (98,1998,19980101,"1998-01-01 00:00:00"), + (00,2000,20000101,"2000-01-01 00:00:01"), + (02,2002,20020101,"2002-01-01 23:59:59"), + (60,2060,20600101,"2060-01-01 11:11:11"), + (70,1970,19700101,"1970-11-11 22:22:22"), + (NULL,NULL,NULL,NULL); +select min(f1),max(f1) from t1; +select min(f2),max(f2) from t1; +select min(f3),max(f3) from t1; +select min(f4),max(f4) from t1; +select a.f1 as a, b.f1 as b, a.f1 > b.f1 as gt, + a.f1 < b.f1 as lt, a.f1<=>b.f1 as eq +from t1 a, t1 b; +select a.f1 as a, b.f2 as b, a.f1 > b.f2 as gt, + a.f1 < b.f2 as lt, a.f1<=>b.f2 as eq +from t1 a, t1 b; +select a.f1 as a, b.f3 as b, a.f1 > b.f3 as gt, + a.f1 < b.f3 as lt, a.f1<=>b.f3 as eq +from t1 a, t1 b; +select a.f1 as a, b.f4 as b, a.f1 > b.f4 as gt, + a.f1 < b.f4 as lt, a.f1<=>b.f4 as eq +from t1 a, t1 b; +select *, f1 = f2 from t1; +drop table t1; +--echo # --echo End of 5.1 tests + === modified file 'mysql-test/t/func_misc.test' --- a/mysql-test/t/func_misc.test 2009-10-28 07:52:34 +0000 +++ b/mysql-test/t/func_misc.test 2010-01-11 13:15:28 +0000 @@ -103,198 +103,6 @@ show create table t1; drop table t1; # -# Bug#6760: Add SLEEP() function (feature request) -# -# Logics of original test: -# Reveal that a query with SLEEP does not need less time than estimated. -# -# Bug#12689: SLEEP() gets incorrectly cached/optimized-away -# -# Description from bug report (slightly modified) -# -# Bug 1 (happened all time): -# SELECT * FROM t1 WHERE SLEEP(1) will only result in a sleep of 1 -# second, regardless of the number of rows in t1. -# Bug 2 (happened all time): -# Such a query will also get cached by the query cache, but should not. -# -# Notes (mleich, 2008-05) -# ======================= -# -# Experiments around -# Bug#36345 Test 'func_misc' fails on RHAS3 x86_64 -# showed that the tests for both bugs could produce in case of parallel -# artificial system time (like via ntpd) -# - decreases false alarm -# - increases false success -# -# We try here to circumvent these issues by reimplementation of the tests -# and sophisticated scripting, although the cause of the problems is a massive -# error within the setup of the testing environment. -# Tests relying on or checking derivates of the system time must never meet -# parallel manipulations of system time. -# -# Results of experiments with/without manipulation of system time, -# information_schema.processlist content, high load on testing box -# ---------------------------------------------------------------- -# Definition: Predicted_cumulative_sleep_time = -# #_of_result_rows * sleep_time_per_result_row -# -# 1. Total (real sleep time) ~= predicted_cumulative_sleep_time !! -# 2. The state of a session within the PROCESSLIST changes to 'User sleep' -# if the sessions runs a statement containing the sleep function and the -# processing of the statement is just within the phase where the sleep -# is done. (*) -# 3. NOW() and processlist.time behave "synchronous" to system time and -# show also the "jumps" caused by system time manipulations. (*) -# 4. processlist.time is unsigned, the "next" value below 0 is ~ 4G (*) -# 5. Current processlist.time ~= current real sleep time if the system time -# was not manipulated. (*) -# 6. High system load can cause delays of <= 2 seconds. -# 7. Thanks to Davi for excellent hints and ideas. -# -# (*) -# - information_schema.processlist is not available before MySQL 5.1. -# - Observation of processlist content requires a -# - "worker" session sending the query with "send" and pulling results -# with "reap" -# - session observing the processlist parallel to the worker session -# "send" and "reap" do not work in case of an embedded server. -# Conclusion: Tests based on processlist have too many restrictions. -# -# Solutions for subtests based on TIMEDIFF of values filled via NOW() -# ------------------------------------------------------------------- -# Run the following sequence three times -# 1. SELECT <start_time> -# 2. Query with SLEEP -# 3. SELECT <end_time> -# If TIMEDIFF(<end_time>,<start_time>) is at least two times within a -# reasonable range assume that we did not met errors we were looking for. -# -# It is extreme unlikely that we have two system time changes within the -# < 30 seconds runtime. Even if the unlikely happens, there are so -# frequent runs of this test on this or another testing box which will -# catch the problem. -# - ---echo #------------------------------------------------------------------------ ---echo # Tests for Bug#6760 and Bug#12689 -# Number of rows within the intended result set. -SET @row_count = 4; -# Parameter within SLEEP function -SET @sleep_time_per_result_row = 1; -# Maximum acceptable delay caused by high load on testing box -SET @max_acceptable_delay = 2; -# TIMEDIFF = time for query with sleep (mostly the time caused by SLEEP) -# + time for delays caused by high load on testing box -# Ensure that at least a reasonable fraction of TIMEDIFF belongs to the SLEEP -# by appropriate setting of variables. -# Ensure that any "judging" has a base of minimum three attempts. -# (Test 2 uses all attempts except the first one.) -if (!` SELECT (@sleep_time_per_result_row * @row_count - @max_acceptable_delay > - @sleep_time_per_result_row) AND (@row_count - 1 >= 3)`) -{ - --echo # Have to abort because of error in plausibility check - --echo ###################################################### - --vertical_results - SELECT @sleep_time_per_result_row * @row_count - @max_acceptable_delay > - @sleep_time_per_result_row AS must_be_1, - @row_count - 1 >= 3 AS must_be_also_1, - @sleep_time_per_result_row, @row_count, @max_acceptable_delay; - exit; -} -SET @@global.query_cache_size = 1024 * 64; ---disable_warnings -DROP TEMPORARY TABLE IF EXISTS t_history; -DROP TABLE IF EXISTS t1; ---enable_warnings -CREATE TEMPORARY TABLE t_history (attempt SMALLINT, -start_ts DATETIME, end_ts DATETIME, -start_cached INTEGER, end_cached INTEGER); -CREATE TABLE t1 (f1 BIGINT); -let $num = `SELECT @row_count`; ---disable_query_log -begin; -while ($num) -{ - INSERT INTO t1 VALUES (1); - dec $num; -} -commit; ---enable_query_log - -let $loops = 4; -let $num = $loops; -while ($num) -{ - let $Qcache_queries_in_cache = - query_get_value(SHOW STATUS LIKE 'Qcache_queries_in_cache', Value, 1); - eval - INSERT INTO t_history - SET attempt = $loops - $num + 1, start_ts = NOW(), - start_cached = $Qcache_queries_in_cache; - SELECT *, SLEEP(@sleep_time_per_result_row) FROM t1; - # - # Do not determine Qcache_queries_in_cache before updating end_ts. The SHOW - # might cost too much time on an overloaded box. - eval - UPDATE t_history SET end_ts = NOW() - WHERE attempt = $loops - $num + 1; - let $Qcache_queries_in_cache = - query_get_value(SHOW STATUS LIKE 'Qcache_queries_in_cache', Value, 1); - eval - UPDATE t_history SET end_cached = $Qcache_queries_in_cache - WHERE attempt = $loops - $num + 1; - # DEBUG eval SELECT * FROM t_history WHERE attempt = $loops - $num + 1; - dec $num; -} - -# 1. The majority of queries with SLEEP must need a reasonable time -# -> SLEEP has an impact on runtime -# = Replacement for original Bug#6760 test -# -> total runtime is clear more needed than for one result row needed -# = Replacement for one of the original Bug#12689 tests ---echo # Test 1: Does the query with SLEEP need a reasonable time? -eval SELECT COUNT(*) >= $loops - 1 INTO @aux1 FROM t_history -WHERE TIMEDIFF(end_ts,start_ts) - @sleep_time_per_result_row * @row_count - BETWEEN 0 AND @max_acceptable_delay; -SELECT @aux1 AS "Expect 1"; -# -# 2. The majority of queries (the first one must be ignored) with SLEEP must -# need a reasonable time -# -> If we assume that the result of a cached query will be sent back -# immediate, without any sleep, than the query with SLEEP cannot be cached -# (current and intended behaviour for queries with SLEEP). -# -> It could be also not excluded that the query was cached but the server -# honoured somehow the SLEEP. Such a behaviour would be also acceptable. -# = Replacement for one of the original Bug#12689 tests ---echo # Test 2: Does the query with SLEEP need a reasonable time even in case ---echo # of the non first execution? -eval SELECT COUNT(*) >= $loops - 1 - 1 INTO @aux2 FROM t_history -WHERE TIMEDIFF(end_ts,start_ts) - @sleep_time_per_result_row * @row_count - BETWEEN 0 AND @max_acceptable_delay - AND attempt > 1; -SELECT @aux2 AS "Expect 1"; -# -# 3. The query with SLEEP should be not cached. -# -> SHOW STATUS Qcache_queries_in_cache must be not incremented after -# the execution of the query with SLEEP ---echo # Test 3: The query with SLEEP must be not cached. -eval SELECT COUNT(*) = $loops INTO @aux3 FROM t_history -WHERE end_cached = start_cached; -SELECT @aux3 AS "Expect 1"; -# -# Dump the content of t_history if one of the tests failed. -if (`SELECT @aux1 + @aux2 + @aux3 <> 3`) -{ - --echo # Some tests failed, dumping the content of t_history - SELECT * FROM t_history; -} -DROP TABLE t1; -DROP TEMPORARY TABLE t_history; -SET @@global.query_cache_size = default; - -# # Bug #21466: INET_ATON() returns signed, not unsigned # === modified file 'mysql-test/t/grant.test' --- a/mysql-test/t/grant.test 2009-06-15 15:53:45 +0000 +++ b/mysql-test/t/grant.test 2010-01-29 10:42:31 +0000 @@ -35,7 +35,7 @@ revoke delete on mysqltest.* from mysqlt show grants for mysqltest_1@localhost; grant select on mysqltest.* to mysqltest_1@localhost require NONE; show grants for mysqltest_1@localhost; -grant USAGE on mysqltest.* to mysqltest_1@localhost require cipher "EDH-RSA-DES-CBC3-SHA" AND SUBJECT "testsubject" ISSUER "MySQL AB"; +grant USAGE on mysqltest.* to mysqltest_1@localhost require cipher "EDH-RSA-DES-CBC3-SHA" AND SUBJECT "testsubject" ISSUER "Monty Program Ab"; show grants for mysqltest_1@localhost; revoke all privileges on mysqltest.* from mysqltest_1@localhost; show grants for mysqltest_1@localhost; === modified file 'mysql-test/t/grant2.test' --- a/mysql-test/t/grant2.test 2009-02-27 08:03:47 +0000 +++ b/mysql-test/t/grant2.test 2009-10-30 05:06:10 +0000 @@ -632,5 +632,40 @@ DROP DATABASE db1; --echo End of 5.0 tests +# +# Bug #48319: Server crashes on "GRANT/REVOKE ... TO CURRENT_USER" +# + +# work out who we are. +USE mysql; +SELECT LEFT(CURRENT_USER(),INSTR(CURRENT_USER(),'@')-1) INTO @u; +SELECT MID(CURRENT_USER(),INSTR(CURRENT_USER(),'@')+1) INTO @h; +SELECT password FROM user WHERE user=@u AND host=@h INTO @pwd; + +# show current privs. +SELECT user,host,password,insert_priv FROM user WHERE user=@u AND host=@h; + +# toggle INSERT +UPDATE user SET insert_priv='N' WHERE user=@u AND host=@h; +SELECT user,host,password,insert_priv FROM user WHERE user=@u AND host=@h; + +# show that GRANT ... TO CURRENT_USER() no longer crashes +GRANT INSERT ON *.* TO CURRENT_USER(); +SELECT user,host,password,insert_priv FROM user WHERE user=@u AND host=@h; +UPDATE user SET insert_priv='N' WHERE user=@u AND host=@h; + +# show that GRANT ... TO CURRENT_USER() IDENTIFIED BY ... works now +GRANT INSERT ON *.* TO CURRENT_USER() IDENTIFIED BY 'keksdose'; +SELECT user,host,password,insert_priv FROM user WHERE user=@u AND host=@h; + +UPDATE user SET password=@pwd WHERE user=@u AND host=@h; +SELECT user,host,password,insert_priv FROM user WHERE user=@u AND host=@h; + +FLUSH PRIVILEGES; + +USE test; + +--echo End of 5.1 tests + # Wait till we reached the initial number of concurrent sessions --source include/wait_until_count_sessions.inc === modified file 'mysql-test/t/group_min_max.test' --- a/mysql-test/t/group_min_max.test 2009-08-30 07:03:37 +0000 +++ b/mysql-test/t/group_min_max.test 2009-11-23 10:04:17 +0000 @@ -1016,6 +1016,18 @@ SELECT a, MAX(b) FROM t WHERE b > 0 AND DROP TABLE t; +--echo # +--echo # Bug #48472: Loose index scan inappropriately chosen for some WHERE +--echo # conditions +--echo # + +CREATE TABLE t (a INT, b INT, INDEX (a,b)); +INSERT INTO t VALUES (2,0), (2,0), (2,1), (2,1); +INSERT INTO t SELECT * FROM t; + +SELECT a, MAX(b) FROM t WHERE 0=b+0 GROUP BY a; + +DROP TABLE t; --echo End of 5.0 tests === modified file 'mysql-test/t/innodb-analyze.test' --- a/mysql-test/t/innodb-analyze.test 2009-11-13 21:26:08 +0000 +++ b/mysql-test/t/innodb-analyze.test 2010-01-15 15:58:25 +0000 @@ -11,7 +11,7 @@ -- disable_result_log -- enable_warnings -SET @old_innodb_stats_sample_pages=@@innodb_stats_sample_pages; +let $sample_pages=`select @@innodb_stats_sample_pages`; SET GLOBAL innodb_stats_sample_pages=0; # check that the value has been adjusted to 1 @@ -62,4 +62,4 @@ SET GLOBAL innodb_stats_sample_pages=16; ANALYZE TABLE innodb_analyze; DROP TABLE innodb_analyze; -SET GLOBAL innodb_stats_sample_pages=@old_innodb_stats_sample_pages; +EVAL SET GLOBAL innodb_stats_sample_pages=$sample_pages; === modified file 'mysql-test/t/innodb-autoinc.test' --- a/mysql-test/t/innodb-autoinc.test 2009-12-03 11:34:11 +0000 +++ b/mysql-test/t/innodb-autoinc.test 2010-01-15 21:12:30 +0000 @@ -2,6 +2,8 @@ # embedded server ignores 'delayed', so skip this -- source include/not_embedded.inc +let $file_format_check=`select @@innodb_file_format_check`; + --disable_warnings drop table if exists t1; --enable_warnings @@ -156,7 +158,7 @@ DROP TABLE t1; # # Test changes to AUTOINC next value calculation SET @@SESSION.AUTO_INCREMENT_INCREMENT=100, @@SESSION.AUTO_INCREMENT_OFFSET=10; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; DROP TABLE IF EXISTS t1; CREATE TABLE t1 (c1 INT AUTO_INCREMENT, PRIMARY KEY(c1)) ENGINE=InnoDB; INSERT INTO t1 VALUES (NULL),(5),(NULL); @@ -173,7 +175,7 @@ DROP TABLE t1; # Reset the AUTOINC session variables SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; DROP TABLE IF EXISTS t1; CREATE TABLE t1 (c1 INT AUTO_INCREMENT, PRIMARY KEY(c1)) ENGINE=InnoDB; INSERT INTO t1 VALUES(0); @@ -193,13 +195,13 @@ DROP TABLE t1; # Reset the AUTOINC session variables SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; DROP TABLE IF EXISTS t1; CREATE TABLE t1 (c1 INT AUTO_INCREMENT, PRIMARY KEY(c1)) ENGINE=InnoDB; INSERT INTO t1 VALUES(-1); SELECT * FROM t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=100, @@SESSION.AUTO_INCREMENT_OFFSET=10; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; INSERT INTO t1 VALUES (-2), (NULL),(2),(NULL); INSERT INTO t1 VALUES (250),(NULL); SELECT * FROM t1; @@ -214,13 +216,13 @@ DROP TABLE t1; # Reset the AUTOINC session variables SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; DROP TABLE IF EXISTS t1; CREATE TABLE t1 (c1 INT UNSIGNED AUTO_INCREMENT, PRIMARY KEY(c1)) ENGINE=InnoDB; INSERT INTO t1 VALUES(-1); SELECT * FROM t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=100, @@SESSION.AUTO_INCREMENT_OFFSET=10; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; INSERT INTO t1 VALUES (-2); INSERT INTO t1 VALUES (NULL); INSERT INTO t1 VALUES (2); @@ -240,13 +242,13 @@ DROP TABLE t1; # Reset the AUTOINC session variables SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; DROP TABLE IF EXISTS t1; CREATE TABLE t1 (c1 INT UNSIGNED AUTO_INCREMENT, PRIMARY KEY(c1)) ENGINE=InnoDB; INSERT INTO t1 VALUES(-1); SELECT * FROM t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=100, @@SESSION.AUTO_INCREMENT_OFFSET=10; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; INSERT INTO t1 VALUES (-2),(NULL),(2),(NULL); INSERT INTO t1 VALUES (250),(NULL); SELECT * FROM t1; @@ -262,7 +264,7 @@ DROP TABLE t1; # Check for overflow handling when increment is > 1 SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; DROP TABLE IF EXISTS t1; CREATE TABLE t1 (c1 BIGINT AUTO_INCREMENT, PRIMARY KEY(c1)) ENGINE=InnoDB; # TODO: Fix the autoinc init code @@ -271,7 +273,7 @@ INSERT INTO t1 VALUES(NULL); INSERT INTO t1 VALUES (9223372036854775794); #-- 2^63 - 14 SELECT * FROM t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=2, @@SESSION.AUTO_INCREMENT_OFFSET=10; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; # This should just fit INSERT INTO t1 VALUES (NULL),(NULL),(NULL),(NULL),(NULL),(NULL); SELECT * FROM t1; @@ -281,7 +283,7 @@ DROP TABLE t1; # Check for overflow handling when increment and offser are > 1 SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; DROP TABLE IF EXISTS t1; CREATE TABLE t1 (c1 BIGINT UNSIGNED AUTO_INCREMENT, PRIMARY KEY(c1)) ENGINE=InnoDB; # TODO: Fix the autoinc init code @@ -290,7 +292,7 @@ INSERT INTO t1 VALUES(NULL); INSERT INTO t1 VALUES (18446744073709551603); #-- 2^64 - 13 SELECT * FROM t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=2, @@SESSION.AUTO_INCREMENT_OFFSET=10; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; # This should fail because of overflow but it doesn't, it seems to be # a MySQL server bug. It wraps around to 0 for the last value. # See MySQL Bug# 39828 @@ -313,7 +315,7 @@ DROP TABLE t1; # Check for overflow handling when increment and offset are odd numbers SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; DROP TABLE IF EXISTS t1; CREATE TABLE t1 (c1 BIGINT UNSIGNED AUTO_INCREMENT, PRIMARY KEY(c1)) ENGINE=InnoDB; # TODO: Fix the autoinc init code @@ -322,7 +324,7 @@ INSERT INTO t1 VALUES(NULL); INSERT INTO t1 VALUES (18446744073709551603); #-- 2^64 - 13 SELECT * FROM t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=5, @@SESSION.AUTO_INCREMENT_OFFSET=7; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; # This should fail because of overflow but it doesn't. It fails with # a duplicate entry message because of a MySQL server bug, it wraps # around. See MySQL Bug# 39828, once MySQL fix the bug we can replace @@ -344,7 +346,7 @@ DROP TABLE t1; # and check for large -ve numbers SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; DROP TABLE IF EXISTS t1; CREATE TABLE t1 (c1 BIGINT AUTO_INCREMENT, PRIMARY KEY(c1)) ENGINE=InnoDB; # TODO: Fix the autoinc init code @@ -355,7 +357,7 @@ INSERT INTO t1 VALUES(-92233720368547758 INSERT INTO t1 VALUES(-9223372036854775808); #-- -2^63 SELECT * FROM t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=3, @@SESSION.AUTO_INCREMENT_OFFSET=3; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; INSERT INTO t1 VALUES (NULL),(NULL), (NULL); SELECT * FROM t1; DROP TABLE t1; @@ -364,7 +366,7 @@ DROP TABLE t1; # large numbers 2^60 SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; DROP TABLE IF EXISTS t1; CREATE TABLE t1 (c1 BIGINT UNSIGNED AUTO_INCREMENT, PRIMARY KEY(c1)) ENGINE=InnoDB; # TODO: Fix the autoinc init code @@ -373,7 +375,7 @@ INSERT INTO t1 VALUES(NULL); INSERT INTO t1 VALUES (18446744073709551610); #-- 2^64 - 2 SELECT * FROM t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=1152921504606846976, @@SESSION.AUTO_INCREMENT_OFFSET=1152921504606846976; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; # This should fail because of overflow but it doesn't. It wraps around # and the autoinc values look bogus too. # See MySQL Bug# 39828, once MySQL fix the bug we can enable the error @@ -396,7 +398,7 @@ DROP TABLE t1; # SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; CREATE TABLE t1 (c1 DOUBLE NOT NULL AUTO_INCREMENT, c2 INT, PRIMARY KEY (c1)) ENGINE=InnoDB; INSERT INTO t1 VALUES(NULL, 1); INSERT INTO t1 VALUES(NULL, 2); @@ -508,7 +510,7 @@ DROP TABLE t1; # If the user has specified negative values for an AUTOINC column then # InnoDB should ignore those values when setting the table's max value. SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; # TINYINT CREATE TABLE t1 (c1 TINYINT PRIMARY KEY AUTO_INCREMENT, c2 VARCHAR(10)) ENGINE=InnoDB; INSERT INTO t1 VALUES (1, NULL); @@ -620,3 +622,42 @@ SHOW CREATE TABLE T1; INSERT INTO T1 (c2) values (0); SELECT * FROM T1; DROP TABLE T1; + +## +# 49032: Use the correct function to read the AUTOINC column value +# +CREATE TABLE T1(C1 DOUBLE AUTO_INCREMENT KEY, C2 CHAR(10)) ENGINE=InnoDB; +INSERT INTO T1(C1, C2) VALUES (1, 'innodb'), (3, 'innodb'); +# Restart the server +-- source include/restart_mysqld.inc +INSERT INTO T1(C2) VALUES ('innodb'); +SHOW CREATE TABLE T1; +DROP TABLE T1; +CREATE TABLE T1(C1 FLOAT AUTO_INCREMENT KEY, C2 CHAR(10)) ENGINE=InnoDB; +INSERT INTO T1(C1, C2) VALUES (1, 'innodb'), (3, 'innodb'); +# Restart the server +-- source include/restart_mysqld.inc +INSERT INTO T1(C2) VALUES ('innodb'); +SHOW CREATE TABLE T1; +DROP TABLE T1; + +## +# 47720: REPLACE INTO Autoincrement column with negative values +# +CREATE TABLE t1 (c1 INT AUTO_INCREMENT PRIMARY KEY) ENGINE=InnoDB; +INSERT INTO t1 SET c1 = 1; +SHOW CREATE TABLE t1; +INSERT INTO t1 SET c1 = 2; +INSERT INTO t1 SET c1 = -1; +SELECT * FROM t1; +-- error ER_DUP_ENTRY,1062 +INSERT INTO t1 SET c1 = -1; +SHOW CREATE TABLE t1; +REPLACE INTO t1 VALUES (-1); +SELECT * FROM t1; +SHOW CREATE TABLE t1; +DROP TABLE t1; + +--disable_query_log +EVAL SET GLOBAL innodb_file_format_check=$file_format_check; +--enable_query_log === added file 'mysql-test/t/innodb-consistent-master.opt' --- a/mysql-test/t/innodb-consistent-master.opt 1970-01-01 00:00:00 +0000 +++ b/mysql-test/t/innodb-consistent-master.opt 2010-01-15 15:58:25 +0000 @@ -0,0 +1 @@ +--loose-innodb_lock_wait_timeout=2 === added file 'mysql-test/t/innodb-consistent.test' --- a/mysql-test/t/innodb-consistent.test 1970-01-01 00:00:00 +0000 +++ b/mysql-test/t/innodb-consistent.test 2010-01-15 15:58:25 +0000 @@ -0,0 +1,58 @@ +-- source include/not_embedded.inc +-- source include/have_innodb.inc + +--disable_warnings +drop table if exists t1; +--enable_warnings + +# REPLACE INTO ... SELECT and INSERT INTO ... SELECT should do +# a consistent read of the source table. + +connect (a,localhost,root,,); +connect (b,localhost,root,,); +connection a; +set session transaction isolation level read committed; +create table t1(a int not null) engine=innodb DEFAULT CHARSET=latin1; +create table t2 like t1; +insert into t2 values (1),(2),(3),(4),(5),(6),(7); +set autocommit=0; + +# REPLACE INTO ... SELECT case +begin; +# this should not result in any locks on t2. +replace into t1 select * from t2; + +connection b; +set session transaction isolation level read committed; +set autocommit=0; +# should not cuase a lock wait. +delete from t2 where a=5; +commit; +delete from t2; +commit; +connection a; +commit; + +# INSERT INTO ... SELECT case +begin; +# this should not result in any locks on t2. +insert into t1 select * from t2; + +connection b; +set session transaction isolation level read committed; +set autocommit=0; +# should not cuase a lock wait. +delete from t2 where a=5; +commit; +delete from t2; +commit; +connection a; +commit; + +select * from t1; +drop table t1; +drop table t2; + +connection default; +disconnect a; +disconnect b; === modified file 'mysql-test/t/innodb-index.test' --- a/mysql-test/t/innodb-index.test 2009-11-30 21:37:27 +0000 +++ b/mysql-test/t/innodb-index.test 2010-01-15 15:58:25 +0000 @@ -1,6 +1,6 @@ -- source include/have_innodb.inc -SET @save_innodb_file_format_check=@@global.innodb_file_format_check; +let $innodb_file_format_check_orig=`select @@innodb_file_format_check`; create table t1(a int not null, b int, c char(10) not null, d varchar(20)) engine = innodb; insert into t1 values (5,5,'oo','oo'),(4,4,'tr','tr'),(3,4,'ad','ad'),(2,3,'ak','ak'); @@ -404,6 +404,7 @@ create index t1u on t1 (u(1)); drop table t1; eval set global innodb_file_per_table=$per_table; eval set global innodb_file_format=$format; +eval set global innodb_file_format_check=$format; # # Test to check whether CREATE INDEX handles implicit foreign key @@ -541,4 +542,9 @@ disconnect b; DROP TABLE t1; -SET GLOBAL innodb_file_format_check=@save_innodb_file_format_check; +# +# restore environment to the state it was before this test execution +# + +-- disable_query_log +eval SET GLOBAL innodb_file_format_check=$innodb_file_format_check_orig; === modified file 'mysql-test/t/innodb-master.opt' --- a/mysql-test/t/innodb-master.opt 2009-06-09 13:19:13 +0000 +++ b/mysql-test/t/innodb-master.opt 2010-01-15 15:58:25 +0000 @@ -1 +1 @@ ---binlog_cache_size=32768 --innodb_lock_wait_timeout=1 +--binlog_cache_size=32768 --loose_innodb_lock_wait_timeout=1 === modified file 'mysql-test/t/innodb-semi-consistent-master.opt' --- a/mysql-test/t/innodb-semi-consistent-master.opt 2009-06-09 13:19:13 +0000 +++ b/mysql-test/t/innodb-semi-consistent-master.opt 2010-01-15 15:58:25 +0000 @@ -1 +1 @@ ---innodb_lock_wait_timeout=2 +--loose-innodb_lock_wait_timeout=2 === modified file 'mysql-test/t/innodb-timeout.test' --- a/mysql-test/t/innodb-timeout.test 2009-06-09 13:19:13 +0000 +++ b/mysql-test/t/innodb-timeout.test 2010-01-18 12:56:10 +0000 @@ -30,6 +30,7 @@ begin; insert into t1 values(1),(2),(3); connection b; +set innodb_lock_wait_timeout=5; --send select * from t1 for update; @@ -44,16 +45,20 @@ begin; insert into t1 values(4); connection b; ---send +# Test that we get a lock timeout. +# We cannot reliably test that the timeout is exactly 1 seconds due to +# process scheduling differences on the host running the test suite. But we +# can test that we are within reasonable range. +set innodb_lock_wait_timeout=2; +set @a= current_timestamp(); +--error ER_LOCK_WAIT_TIMEOUT select * from t1 for update; +set @b= current_timestamp(); +set @c= timestampdiff(SECOND, @a, @b); +select if(@c >= 1 and @c <= 10, 'OK', concat("NOT OK, time passed=", @c)); connection a; -sleep 2; commit; - -connection b; ---error ER_LOCK_WAIT_TIMEOUT -reap; drop table t1; connection default; === modified file 'mysql-test/t/innodb-use-sys-malloc-master.opt' --- a/mysql-test/t/innodb-use-sys-malloc-master.opt 2009-06-09 13:19:13 +0000 +++ b/mysql-test/t/innodb-use-sys-malloc-master.opt 2010-01-15 15:58:25 +0000 @@ -1,2 +1 @@ ---innodb-use-sys-malloc=true ---innodb-use-sys-malloc=true +--loose-innodb-use-sys-malloc=true === modified file 'mysql-test/t/innodb-zip.test' --- a/mysql-test/t/innodb-zip.test 2009-06-09 15:08:46 +0000 +++ b/mysql-test/t/innodb-zip.test 2010-01-15 15:58:25 +0000 @@ -178,11 +178,11 @@ set innodb_strict_mode = on; --error ER_CANT_CREATE_TABLE create table t1 (id int primary key) engine = innodb key_block_size = 0; -show errors; +show warnings; --error ER_CANT_CREATE_TABLE create table t2 (id int primary key) engine = innodb key_block_size = 9; -show errors; +show warnings; create table t3 (id int primary key) engine = innodb key_block_size = 1; @@ -208,22 +208,22 @@ key_block_size = 8 row_format = compress --error ER_CANT_CREATE_TABLE create table t2 (id int primary key) engine = innodb key_block_size = 8 row_format = redundant; -show errors; +show warnings; --error ER_CANT_CREATE_TABLE create table t3 (id int primary key) engine = innodb key_block_size = 8 row_format = compact; -show errors; +show warnings; --error ER_CANT_CREATE_TABLE create table t4 (id int primary key) engine = innodb key_block_size = 8 row_format = dynamic; -show errors; +show warnings; --error ER_CANT_CREATE_TABLE create table t5 (id int primary key) engine = innodb key_block_size = 8 row_format = default; -show errors; +show warnings; SELECT table_schema, table_name, row_format FROM information_schema.tables WHERE engine='innodb'; @@ -233,17 +233,17 @@ drop table t1; --error ER_CANT_CREATE_TABLE create table t1 (id int primary key) engine = innodb key_block_size = 9 row_format = redundant; -show errors; +show warnings; --error ER_CANT_CREATE_TABLE create table t2 (id int primary key) engine = innodb key_block_size = 9 row_format = compact; -show errors; +show warnings; --error ER_CANT_CREATE_TABLE create table t2 (id int primary key) engine = innodb key_block_size = 9 row_format = dynamic; -show errors; +show warnings; SELECT table_schema, table_name, row_format FROM information_schema.tables WHERE engine='innodb'; @@ -253,25 +253,25 @@ set global innodb_file_per_table = off; --error ER_CANT_CREATE_TABLE create table t1 (id int primary key) engine = innodb key_block_size = 1; -show errors; +show warnings; --error ER_CANT_CREATE_TABLE create table t2 (id int primary key) engine = innodb key_block_size = 2; -show errors; +show warnings; --error ER_CANT_CREATE_TABLE create table t3 (id int primary key) engine = innodb key_block_size = 4; -show errors; +show warnings; --error ER_CANT_CREATE_TABLE create table t4 (id int primary key) engine = innodb key_block_size = 8; -show errors; +show warnings; --error ER_CANT_CREATE_TABLE create table t5 (id int primary key) engine = innodb key_block_size = 16; -show errors; +show warnings; --error ER_CANT_CREATE_TABLE create table t6 (id int primary key) engine = innodb row_format = compressed; -show errors; +show warnings; --error ER_CANT_CREATE_TABLE create table t7 (id int primary key) engine = innodb row_format = dynamic; -show errors; +show warnings; create table t8 (id int primary key) engine = innodb row_format = compact; create table t9 (id int primary key) engine = innodb row_format = redundant; @@ -285,25 +285,25 @@ set global innodb_file_format = `0`; --error ER_CANT_CREATE_TABLE create table t1 (id int primary key) engine = innodb key_block_size = 1; -show errors; +show warnings; --error ER_CANT_CREATE_TABLE create table t2 (id int primary key) engine = innodb key_block_size = 2; -show errors; +show warnings; --error ER_CANT_CREATE_TABLE create table t3 (id int primary key) engine = innodb key_block_size = 4; -show errors; +show warnings; --error ER_CANT_CREATE_TABLE create table t4 (id int primary key) engine = innodb key_block_size = 8; -show errors; +show warnings; --error ER_CANT_CREATE_TABLE create table t5 (id int primary key) engine = innodb key_block_size = 16; -show errors; +show warnings; --error ER_CANT_CREATE_TABLE create table t6 (id int primary key) engine = innodb row_format = compressed; -show errors; +show warnings; --error ER_CANT_CREATE_TABLE create table t7 (id int primary key) engine = innodb row_format = dynamic; -show errors; +show warnings; create table t8 (id int primary key) engine = innodb row_format = compact; create table t9 (id int primary key) engine = innodb row_format = redundant; === modified file 'mysql-test/t/innodb.test' --- a/mysql-test/t/innodb.test 2009-12-27 13:54:41 +0000 +++ b/mysql-test/t/innodb.test 2010-01-15 15:58:25 +0000 @@ -2270,7 +2270,7 @@ disconnect j; drop table t1, t2, t3, t5, t6, t8, t9; # bug 18934, "InnoDB crashes when table uses column names like DB_ROW_ID" ---error 1005 +--error ER_WRONG_COLUMN_NAME CREATE TABLE t1 (DB_ROW_ID int) engine=innodb; # === modified file 'mysql-test/t/innodb_bug34300.test' --- a/mysql-test/t/innodb_bug34300.test 2009-11-13 21:26:08 +0000 +++ b/mysql-test/t/innodb_bug34300.test 2010-01-15 15:58:25 +0000 @@ -9,7 +9,7 @@ -- disable_result_log # set packet size and reconnect -SET @save_max_allowed_packet=@@global.max_allowed_packet; +let $max_packet=`select @@global.max_allowed_packet`; SET @@global.max_allowed_packet=16777216; --connect (newconn, localhost, root,,) @@ -33,4 +33,4 @@ SELECT f4, f8 FROM bug34300; DROP TABLE bug34300; disconnect newconn; connection default; -SET @@global.max_allowed_packet=@save_max_allowed_packet; +EVAL SET @@global.max_allowed_packet=$max_packet; === modified file 'mysql-test/t/innodb_bug36169.test' --- a/mysql-test/t/innodb_bug36169.test 2009-11-13 21:26:08 +0000 +++ b/mysql-test/t/innodb_bug36169.test 2010-01-15 15:58:25 +0000 @@ -4,9 +4,9 @@ # -- source include/have_innodb.inc -set @old_innodb_file_format=@@innodb_file_format; -set @old_innodb_file_per_table=@@innodb_file_per_table; +let $file_format=`select @@innodb_file_format`; +let $file_per_table=`select @@innodb_file_per_table`; SET GLOBAL innodb_file_format='Barracuda'; SET GLOBAL innodb_file_per_table=ON; @@ -1155,5 +1155,5 @@ DROP TABLE IF EXISTS table4; DROP TABLE IF EXISTS table5; DROP TABLE IF EXISTS table6; -set global innodb_file_format=@old_innodb_file_format; -set global innodb_file_per_table=@old_innodb_file_per_table; +EVAL SET GLOBAL innodb_file_format=$file_format; +EVAL SET GLOBAL innodb_file_per_table=$file_per_table; === modified file 'mysql-test/t/innodb_bug36172.test' --- a/mysql-test/t/innodb_bug36172.test 2009-11-13 21:26:08 +0000 +++ b/mysql-test/t/innodb_bug36172.test 2010-01-15 15:58:25 +0000 @@ -13,9 +13,10 @@ SET storage_engine=InnoDB; -- disable_query_log -- disable_result_log -set @old_innodb_file_per_table=@@innodb_file_per_table; -set @old_innodb_file_format=@@innodb_file_format; +let $file_format=`select @@innodb_file_format`; +let $file_format_check=`select @@innodb_file_format_check`; +let $file_per_table=`select @@innodb_file_per_table`; SET GLOBAL innodb_file_format='Barracuda'; SET GLOBAL innodb_file_per_table=on; @@ -27,6 +28,6 @@ INSERT IGNORE INTO `table0` SET `col19` CHECK TABLE table0 EXTENDED; DROP TABLE table0; -set global innodb_file_per_table=@old_innodb_file_per_table; -set global innodb_file_format=@old_innodb_file_format; -set global innodb_file_format_check=Antelope; +EVAL SET GLOBAL innodb_file_format=$file_format; +EVAL SET GLOBAL innodb_file_format_check=$file_format_check; +EVAL SET GLOBAL innodb_file_per_table=$file_per_table; === modified file 'mysql-test/t/innodb_bug42101-nonzero-master.opt' --- a/mysql-test/t/innodb_bug42101-nonzero-master.opt 2009-05-19 08:20:28 +0000 +++ b/mysql-test/t/innodb_bug42101-nonzero-master.opt 2010-01-15 15:58:25 +0000 @@ -1 +1 @@ ---innodb_commit_concurrency=1 +--loose_innodb_commit_concurrency=1 === modified file 'mysql-test/t/innodb_bug44369.test' --- a/mysql-test/t/innodb_bug44369.test 2009-11-02 14:59:44 +0000 +++ b/mysql-test/t/innodb_bug44369.test 2010-01-15 15:58:25 +0000 @@ -6,16 +6,12 @@ --source include/have_innodb.inc # This create table operation should fail. ---error ER_CANT_CREATE_TABLE +--error ER_WRONG_COLUMN_NAME create table bug44369 (DB_ROW_ID int) engine=innodb; # This create should fail as well ---error ER_CANT_CREATE_TABLE +--error ER_WRONG_COLUMN_NAME create table bug44369 (db_row_id int) engine=innodb; -show warnings; - ---error ER_CANT_CREATE_TABLE +--error ER_WRONG_COLUMN_NAME create table bug44369 (db_TRX_Id int) engine=innodb; - -show warnings; === added file 'mysql-test/t/innodb_bug44571.test' --- a/mysql-test/t/innodb_bug44571.test 1970-01-01 00:00:00 +0000 +++ b/mysql-test/t/innodb_bug44571.test 2010-01-15 15:58:25 +0000 @@ -0,0 +1,13 @@ +# +# Bug#44571 InnoDB Plugin crashes on ADD INDEX +# http://bugs.mysql.com/44571 +# +-- source include/have_innodb.inc + +CREATE TABLE bug44571 (foo INT) ENGINE=InnoDB; +ALTER TABLE bug44571 CHANGE foo bar INT; +-- error ER_KEY_COLUMN_DOES_NOT_EXITS +ALTER TABLE bug44571 ADD INDEX bug44571b (foo); +ALTER TABLE bug44571 ADD INDEX bug44571b (bar); +CREATE INDEX bug44571c ON bug44571 (bar); +DROP TABLE bug44571; === added file 'mysql-test/t/innodb_bug46676.test' --- a/mysql-test/t/innodb_bug46676.test 1970-01-01 00:00:00 +0000 +++ b/mysql-test/t/innodb_bug46676.test 2010-01-15 15:58:25 +0000 @@ -0,0 +1,16 @@ +# This is the test for bug 46676: mysqld got exception 0xc0000005 +# It is reproducible with InnoDB plugin 1.0.4 + MySQL 5.1.37. +# But no longer reproducible after MySQL 5.1.38 (with plugin 1.0.5). + +--source include/have_innodb.inc + +SET foreign_key_checks=0; +CREATE TABLE t1 (id int, foreign key (id) references t2(id)) ENGINE=INNODB; +CREATE TABLE t2 (id int, foreign key (id) references t1(id)) ENGINE=INNODB; +SET foreign_key_checks=1; + +# Server crashes +SELECT COUNT(*) FROM information_schema.key_column_usage WHERE REFERENCED_TABLE_NAME in ('t1', 't2'); + +SET foreign_key_checks=0; +DROP TABLE t1, t2; === added file 'mysql-test/t/innodb_bug47167.test' --- a/mysql-test/t/innodb_bug47167.test 1970-01-01 00:00:00 +0000 +++ b/mysql-test/t/innodb_bug47167.test 2010-01-15 15:58:25 +0000 @@ -0,0 +1,45 @@ +# This is the unit test for bug *47167. +# It tests setting the global variable +# "innodb_file_format_check" with a +# user-Defined Variable. + +--source include/have_innodb.inc + +# Save the value (Antelope) in 'innodb_file_format_check' to +# 'old_innodb_file_format_check' +set @old_innodb_file_format_check=@@innodb_file_format_check; + +# @old_innodb_file_format_check shall have the value of 'Antelope' +select @old_innodb_file_format_check; + +# Reset the value in 'innodb_file_format_check' to 'Barracuda' +set global innodb_file_format_check = Barracuda; + +select @@innodb_file_format_check; + +# Set 'innodb_file_format_check' to its default value, which +# is the latest file format supported in the current release. +set global innodb_file_format_check = DEFAULT; + +select @@innodb_file_format_check; + +# Put the saved value back to 'innodb_file_format_check' +set global innodb_file_format_check = @old_innodb_file_format_check; + +# Check whether 'innodb_file_format_check' get its original value. +select @@innodb_file_format_check; + +# Following are negative tests, all should fail. +--disable_warnings +--error ER_WRONG_ARGUMENTS +set global innodb_file_format_check = cheetah; + +--error ER_WRONG_ARGUMENTS +set global innodb_file_format_check = Bear; + +--error ER_WRONG_ARGUMENTS +set global innodb_file_format_check = on; + +--error ER_WRONG_ARGUMENTS +set global innodb_file_format_check = off; +--enable_warnings === modified file 'mysql-test/t/innodb_file_format.test' --- a/mysql-test/t/innodb_file_format.test 2009-11-30 21:37:27 +0000 +++ b/mysql-test/t/innodb_file_format.test 2010-01-15 15:58:25 +0000 @@ -1,5 +1,4 @@ -- source include/have_innodb.inc -set @old_innodb_file_format=@@innodb_file_format; call mtr.add_suppression("InnoDB: invalid innodb_file_format_check value"); @@ -29,6 +28,4 @@ set global innodb_file_format=on; --error ER_WRONG_ARGUMENTS set global innodb_file_format=off; select @@innodb_file_format_check; - -set global innodb_file_format=@old_innodb_file_format; -set global innodb_file_format_check=Antelope; +set global innodb_file_format_check=antelope; === modified file 'mysql-test/t/innodb_information_schema.test' --- a/mysql-test/t/innodb_information_schema.test 2009-06-23 12:00:24 +0000 +++ b/mysql-test/t/innodb_information_schema.test 2010-01-15 15:58:25 +0000 @@ -109,19 +109,18 @@ SELECT * FROM ```t'\"_str` WHERE c1 = '3 -- send SELECT * FROM ```t'\"_str` WHERE c1 = '4' FOR UPDATE; +-- enable_result_log -- connection con_verify_innodb_locks - -# Loop, giving time for the above 2 queries to execute before continuing. -# Without this, it sometimes happens that the SELECT FROM innodb_locks +# Wait for the above queries to execute before continuing. +# Without this, it sometimes happens that the SELECT from innodb_locks # executes before some of them, resulting in less than expected number -# of rows being selected from innodb_locks. -SET @counter := 0; -while (`SELECT (@counter := @counter + 1) <= 50 AND COUNT(*) != 14 FROM INFORMATION_SCHEMA.INNODB_LOCKS`) -{ - sleep 0.1; -} - --- enable_result_log +# of rows being selected from innodb_locks. If there is a bug and there +# are no 14 rows in innodb_locks then this test will fail with timeout. +let $count = 14; +let $table = INFORMATION_SCHEMA.INNODB_LOCKS; +-- source include/wait_until_rows_count.inc +# the above enables the query log, re-disable it +-- disable_query_log SELECT lock_mode, lock_type, lock_table, lock_index, lock_rec, lock_data FROM INFORMATION_SCHEMA.INNODB_LOCKS ORDER BY lock_data; === modified file 'mysql-test/t/innodb_lock_wait_timeout_1.test' --- a/mysql-test/t/innodb_lock_wait_timeout_1.test 2009-11-03 17:45:52 +0000 +++ b/mysql-test/t/innodb_lock_wait_timeout_1.test 2009-11-12 11:43:33 +0000 @@ -71,6 +71,40 @@ set autocommit=default; drop table t1; --echo # +--echo # Bug #37183 insert ignore into .. select ... hangs +--echo # after deadlock was encountered +--echo # +connect (con1,localhost,root,,); +create table t1(id int primary key,v int)engine=innodb; +insert into t1 values (1,1),(2,2),(3,3),(4,4),(5,5),(6,6),(7,7); +create table t2 like t1; + +--connection con1 +begin; +update t1 set v=id*2 where id=1; + +--connection default +begin; +update t1 set v=id*2 where id=2; + +--connection con1 +--error 1205 +update t1 set v=id*2 where id=2; + +--connection default +--error 1205 +insert ignore into t2 select * from t1 where id=1; +rollback; + +--connection con1 +rollback; + +--connection default +disconnect con1; +drop table t1, t2; + + +--echo # --echo # Bug#41756 Strange error messages about locks from InnoDB --echo # --disable_warnings === modified file 'mysql-test/t/innodb_mysql.test' --- a/mysql-test/t/innodb_mysql.test 2009-12-03 11:19:05 +0000 +++ b/mysql-test/t/innodb_mysql.test 2010-01-15 15:27:55 +0000 @@ -491,5 +491,51 @@ EXPLAIN SELECT * FROM t1 WHERE a = 'TEST c >= '2009-10-09 00:00:00.001' AND c <= '2009-10-09 00:00:00.00'; DROP TABLE t1; +--echo # +--echo # Bug #46175: NULL read_view and consistent read assertion +--echo # + +CREATE TABLE t1(a CHAR(13),KEY(a)) ENGINE=innodb; +CREATE TABLE t2(b DATETIME,KEY(b)) ENGINE=innodb; +INSERT INTO t1 VALUES (),(); +INSERT INTO t2 VALUES (),(); +CREATE OR REPLACE VIEW v1 AS SELECT 1 FROM t2 + WHERE b =(SELECT a FROM t1 LIMIT 1); + +--disable_query_log +--disable_result_log +CONNECT (con1, localhost, root,,); +--enable_query_log +--enable_result_log +CONNECTION default; + +DELIMITER |; +CREATE PROCEDURE p1(num INT) +BEGIN + DECLARE i INT DEFAULT 0; + REPEAT + SHOW CREATE VIEW v1; + SET i:=i+1; + UNTIL i>num END REPEAT; +END| +DELIMITER ;| + +--echo # Should not crash +--disable_query_log +--disable_result_log +--send CALL p1(1000) +CONNECTION con1; +--echo # Should not crash +CALL p1(1000); + +CONNECTION default; +--reap +--enable_query_log +--enable_result_log + +DISCONNECT con1; +DROP PROCEDURE p1; +DROP VIEW v1; +DROP TABLE t1,t2; --echo End of 5.1 tests === modified file 'mysql-test/t/innodb_xtradb_bug317074.test' --- a/mysql-test/t/innodb_xtradb_bug317074.test 2009-10-28 07:52:34 +0000 +++ b/mysql-test/t/innodb_xtradb_bug317074.test 2010-01-15 15:58:25 +0000 @@ -2,7 +2,7 @@ SET @old_innodb_file_format=@@innodb_file_format; SET @old_innodb_file_per_table=@@innodb_file_per_table; -SET @old_innodb_file_format_check=@@innodb_file_format_check; +let $innodb_file_format_check_orig=`select @@innodb_file_format_check`; SET GLOBAL innodb_file_format='Barracuda'; SET GLOBAL innodb_file_per_table=ON; @@ -45,4 +45,4 @@ ALTER TABLE test1 ENGINE=MyISAM; DROP TABLE test1; SET GLOBAL innodb_file_format=@old_innodb_file_format; SET GLOBAL innodb_file_per_table=@old_innodb_file_per_table; -SET GLOBAL innodb_file_format_check=@old_innodb_file_format_check; +eval set global innodb_file_format_check=$innodb_file_format_check_orig; === modified file 'mysql-test/t/mysql.test' --- a/mysql-test/t/mysql.test 2009-10-05 13:22:23 +0000 +++ b/mysql-test/t/mysql.test 2010-01-15 15:27:55 +0000 @@ -386,10 +386,16 @@ drop tables t1, t2; # # Bug #27884: mysql --html does not quote HTML special characters in output # ---exec $MYSQL --html test -e "select '< & >' as '<'" +--write_file $MYSQLTEST_VARDIR/tmp/bug27884.sql +SELECT '< & >' AS `<`; +EOF +--exec $MYSQL --html test < $MYSQLTEST_VARDIR/tmp/bug27884.sql + +remove_file $MYSQLTEST_VARDIR/tmp/bug27884.sql; + # -# Bug #27884: mysql client + null byte +# Bug #28203: mysql client + null byte # create table t1 (a char(5)); insert into t1 values ('\0b\0'); @@ -402,5 +408,5 @@ insert into t1 values ('\0b\0'); --exec $MYSQL --xml test -e "select a from t1" drop table t1; ---echo ---echo End of tests + +--echo End of 5.0 tests === added file 'mysql-test/t/mysqlbinlog2-master.opt' --- a/mysql-test/t/mysqlbinlog2-master.opt 1970-01-01 00:00:00 +0000 +++ b/mysql-test/t/mysqlbinlog2-master.opt 2010-01-19 10:36:52 +0000 @@ -0,0 +1 @@ +--timezone=GMT-3 === added file 'mysql-test/t/mysqlbinlog_row-master.opt' --- a/mysql-test/t/mysqlbinlog_row-master.opt 1970-01-01 00:00:00 +0000 +++ b/mysql-test/t/mysqlbinlog_row-master.opt 2010-01-27 17:41:05 +0000 @@ -0,0 +1 @@ +--timezone=GMT-3 === added file 'mysql-test/t/mysqlbinlog_row_innodb-master.opt' --- a/mysql-test/t/mysqlbinlog_row_innodb-master.opt 1970-01-01 00:00:00 +0000 +++ b/mysql-test/t/mysqlbinlog_row_innodb-master.opt 2010-01-27 17:41:05 +0000 @@ -0,0 +1 @@ +--timezone=GMT-3 === added file 'mysql-test/t/mysqlbinlog_row_myisam-master.opt' --- a/mysql-test/t/mysqlbinlog_row_myisam-master.opt 1970-01-01 00:00:00 +0000 +++ b/mysql-test/t/mysqlbinlog_row_myisam-master.opt 2010-01-27 17:41:05 +0000 @@ -0,0 +1 @@ +--timezone=GMT-3 === added file 'mysql-test/t/mysqlbinlog_row_trans-master.opt' --- a/mysql-test/t/mysqlbinlog_row_trans-master.opt 1970-01-01 00:00:00 +0000 +++ b/mysql-test/t/mysqlbinlog_row_trans-master.opt 2010-01-27 17:41:05 +0000 @@ -0,0 +1 @@ +--timezone=GMT-3 === modified file 'mysql-test/t/mysqltest.test' --- a/mysql-test/t/mysqltest.test 2009-10-08 09:30:03 +0000 +++ b/mysql-test/t/mysqltest.test 2010-01-11 13:15:28 +0000 @@ -9,6 +9,14 @@ # Save the initial number of concurrent sessions --source include/count_sessions.inc +# Some tests below connect/disconnect rapidly in a loop. This causes a race +# where mysqld may not have time to register the previous disconnects before +# new connects, and eventually we run out of connections. So we need to +# increase the maximum. +let $saved_max_connections = `SELECT @@global.max_connections`; +SET GLOBAL max_connections = 1000; + + # ============================================================================ # # Test of mysqltest itself @@ -2319,3 +2327,7 @@ disconnect $y; connection default; # Wait till we reached the initial number of concurrent sessions --source include/wait_until_count_sessions.inc + +--disable_query_log +--eval SET GLOBAL max_connections = $saved_max_connections +--enable_query_log === modified file 'mysql-test/t/olap.test' --- a/mysql-test/t/olap.test 2009-10-30 15:54:53 +0000 +++ b/mysql-test/t/olap.test 2009-12-08 09:26:11 +0000 @@ -390,4 +390,17 @@ SELECT DISTINCT b FROM t1, t2 GROUP BY a DROP TABLE t1, t2; +--echo # +--echo # Bug #48475: DISTINCT is ignored with GROUP BY WITH ROLLUP +--echo # and only const tables + +CREATE TABLE t1 (a INT); +CREATE TABLE t2 (b INT); +INSERT INTO t1 VALUES (1); +INSERT INTO t2 VALUES (1); + +SELECT DISTINCT b FROM t1, t2 GROUP BY a, b WITH ROLLUP; + +DROP TABLE t1, t2; + --echo End of 5.0 tests === modified file 'mysql-test/t/openssl_1.test' --- a/mysql-test/t/openssl_1.test 2009-05-25 13:00:18 +0000 +++ b/mysql-test/t/openssl_1.test 2010-01-29 10:42:31 +0000 @@ -15,8 +15,10 @@ insert into t1 values (5); grant select on test.* to ssl_user1@localhost require SSL; grant select on test.* to ssl_user2@localhost require cipher "DHE-RSA-AES256-SHA"; -grant select on test.* to ssl_user3@localhost require cipher "DHE-RSA-AES256-SHA" AND SUBJECT "/C=SE/ST=Uppsala/O=MySQL AB/emailAddress=abstract.mysql.developer(a)mysql.com"; -grant select on test.* to ssl_user4@localhost require cipher "DHE-RSA-AES256-SHA" AND SUBJECT "/C=SE/ST=Uppsala/O=MySQL AB/emailAddress=abstract.mysql.developer(a)mysql.com" ISSUER "/C=SE/ST=Uppsala/L=Uppsala/O=MySQL AB"; +grant select on test.* to ssl_user3@localhost require cipher +"DHE-RSA-AES256-SHA" AND SUBJECT "/C=FI/ST=Tuusula/O=Monty Program Ab/emailAddress=abstract.developer(a)askmonty.org"; +grant select on test.* to ssl_user4@localhost require cipher +"DHE-RSA-AES256-SHA" AND SUBJECT "/C=FI/ST=Tuusula/O=Monty Program Ab/emailAddress=abstract.developer(a)askmonty.org" ISSUER "/C=FI/ST=Tuusula/O=Monty Program Ab/emailAddress=abstract.developer(a)askmonty.org"; grant select on test.* to ssl_user5@localhost require cipher "DHE-RSA-AES256-SHA" AND SUBJECT "xxx"; flush privileges; === modified file 'mysql-test/t/order_by.test' --- a/mysql-test/t/order_by.test 2009-12-03 11:19:05 +0000 +++ b/mysql-test/t/order_by.test 2010-01-15 15:27:55 +0000 @@ -869,6 +869,31 @@ SELECT DROP TABLE t1, t2, t3; +--echo # +--echo # Bug #42760: Select doesn't return desired results when we have null +--echo # values +--echo # + +CREATE TABLE t1 ( + a INT, + c INT, + UNIQUE KEY a_c (a,c), + KEY (a)); + +INSERT INTO t1 VALUES (1, 10), (2, NULL); + +--echo # Must use ref-or-null on the a_c index +EXPLAIN +SELECT 1 AS col FROM t1 WHERE a=2 AND (c=10 OR c IS NULL) ORDER BY c; +--echo # Must return 1 row +SELECT 1 AS col FROM t1 WHERE a=2 AND (c=10 OR c IS NULL) ORDER BY c; + +DROP TABLE t1; + + +--echo End of 5.0 tests + + # # Bug #35206: select query result different if the key is indexed or not # === modified file 'mysql-test/t/partition.test' --- a/mysql-test/t/partition.test 2009-12-03 11:19:05 +0000 +++ b/mysql-test/t/partition.test 2010-01-15 15:27:55 +0000 @@ -15,6 +15,15 @@ drop table if exists t1, t2; --enable_warnings # +# Bug#48276: can't add column if subpartition exists +CREATE TABLE t1 (a INT, b INT) +PARTITION BY LIST (a) +SUBPARTITION BY HASH (b) +(PARTITION p1 VALUES IN (1)); +ALTER TABLE t1 ADD COLUMN c INT; +DROP TABLE t1; + +# # Bug#46639: 1030 (HY000): Got error 124 from storage engine on # INSERT ... SELECT ... CREATE TABLE t1 ( @@ -62,6 +71,17 @@ SHOW CREATE TABLE t1; DROP TABLE t1; # +# Bug#45904: Error when CHARSET=utf8 and subpartitioning +# +create table t1 (a int NOT NULL, b varchar(5) NOT NULL) +default charset=utf8 +partition by list (a) +subpartition by key (b) +(partition p0 values in (1), + partition p1 values in (2)); +drop table t1; + +# # Bug#44059: rec_per_key on empty partition gives weird optimiser results # create table t1 (a int, b int, key(a)) @@ -2035,11 +2055,14 @@ DROP TABLE t1; --echo # --echo # Bug #45807: crash accessing partitioned table and sql_mode --echo # contains ONLY_FULL_GROUP_BY +--echo # Bug#46923: select count(*) from partitioned table fails with +--echo # ONLY_FULL_GROUP_BY --echo # SET SESSION SQL_MODE='ONLY_FULL_GROUP_BY'; CREATE TABLE t1(id INT,KEY(id)) ENGINE=MYISAM PARTITION BY HASH(id) PARTITIONS 2; +SELECT COUNT(*) FROM t1; DROP TABLE t1; SET SESSION SQL_MODE=DEFAULT; === modified file 'mysql-test/t/partition_innodb_semi_consistent.test' --- a/mysql-test/t/partition_innodb_semi_consistent.test 2009-01-13 22:12:16 +0000 +++ b/mysql-test/t/partition_innodb_semi_consistent.test 2010-01-28 14:49:14 +0000 @@ -187,6 +187,7 @@ SELECT * FROM t1; --echo # Switch to connection con2 connection con2; +--reap SELECT * FROM t1; connection default; === modified file 'mysql-test/t/query_cache.test' --- a/mysql-test/t/query_cache.test 2009-12-03 11:19:05 +0000 +++ b/mysql-test/t/query_cache.test 2010-01-11 13:15:28 +0000 @@ -882,6 +882,19 @@ drop procedure f4; drop table t1; set GLOBAL query_cache_size=0; +# Tests moved from main.variables due to needing query cache in server. +set GLOBAL query_cache_size=100000; +--error ER_GLOBAL_VARIABLE +set SESSION query_cache_size=10000; +set global query_cache_limit=100; +set global query_cache_size=100; +set global query_cache_type=demand; + +set GLOBAL query_cache_type=default; +set GLOBAL query_cache_limit=default; +set GLOBAL query_cache_size=default; + + --echo End of 4.1 tests # @@ -1288,6 +1301,198 @@ SHOW STATUS LIKE "Qcache_hits"; DROP TABLE t1; SET GLOBAL query_cache_size= default; +# +# Bug#6760: Add SLEEP() function (feature request) +# +# Logics of original test: +# Reveal that a query with SLEEP does not need less time than estimated. +# +# Bug#12689: SLEEP() gets incorrectly cached/optimized-away +# +# Description from bug report (slightly modified) +# +# Bug 1 (happened all time): +# SELECT * FROM t1 WHERE SLEEP(1) will only result in a sleep of 1 +# second, regardless of the number of rows in t1. +# Bug 2 (happened all time): +# Such a query will also get cached by the query cache, but should not. +# +# Notes (mleich, 2008-05) +# ======================= +# +# Experiments around +# Bug#36345 Test 'func_misc' fails on RHAS3 x86_64 +# showed that the tests for both bugs could produce in case of parallel +# artificial system time (like via ntpd) +# - decreases false alarm +# - increases false success +# +# We try here to circumvent these issues by reimplementation of the tests +# and sophisticated scripting, although the cause of the problems is a massive +# error within the setup of the testing environment. +# Tests relying on or checking derivates of the system time must never meet +# parallel manipulations of system time. +# +# Results of experiments with/without manipulation of system time, +# information_schema.processlist content, high load on testing box +# ---------------------------------------------------------------- +# Definition: Predicted_cumulative_sleep_time = +# #_of_result_rows * sleep_time_per_result_row +# +# 1. Total (real sleep time) ~= predicted_cumulative_sleep_time !! +# 2. The state of a session within the PROCESSLIST changes to 'User sleep' +# if the sessions runs a statement containing the sleep function and the +# processing of the statement is just within the phase where the sleep +# is done. (*) +# 3. NOW() and processlist.time behave "synchronous" to system time and +# show also the "jumps" caused by system time manipulations. (*) +# 4. processlist.time is unsigned, the "next" value below 0 is ~ 4G (*) +# 5. Current processlist.time ~= current real sleep time if the system time +# was not manipulated. (*) +# 6. High system load can cause delays of <= 2 seconds. +# 7. Thanks to Davi for excellent hints and ideas. +# +# (*) +# - information_schema.processlist is not available before MySQL 5.1. +# - Observation of processlist content requires a +# - "worker" session sending the query with "send" and pulling results +# with "reap" +# - session observing the processlist parallel to the worker session +# "send" and "reap" do not work in case of an embedded server. +# Conclusion: Tests based on processlist have too many restrictions. +# +# Solutions for subtests based on TIMEDIFF of values filled via NOW() +# ------------------------------------------------------------------- +# Run the following sequence three times +# 1. SELECT <start_time> +# 2. Query with SLEEP +# 3. SELECT <end_time> +# If TIMEDIFF(<end_time>,<start_time>) is at least two times within a +# reasonable range assume that we did not met errors we were looking for. +# +# It is extreme unlikely that we have two system time changes within the +# < 30 seconds runtime. Even if the unlikely happens, there are so +# frequent runs of this test on this or another testing box which will +# catch the problem. +# + +--echo #------------------------------------------------------------------------ +--echo # Tests for Bug#6760 and Bug#12689 +# Number of rows within the intended result set. +SET @row_count = 4; +# Parameter within SLEEP function +SET @sleep_time_per_result_row = 1; +# Maximum acceptable delay caused by high load on testing box +SET @max_acceptable_delay = 2; +# TIMEDIFF = time for query with sleep (mostly the time caused by SLEEP) +# + time for delays caused by high load on testing box +# Ensure that at least a reasonable fraction of TIMEDIFF belongs to the SLEEP +# by appropriate setting of variables. +# Ensure that any "judging" has a base of minimum three attempts. +# (Test 2 uses all attempts except the first one.) +if (!` SELECT (@sleep_time_per_result_row * @row_count - @max_acceptable_delay > + @sleep_time_per_result_row) AND (@row_count - 1 >= 3)`) +{ + --echo # Have to abort because of error in plausibility check + --echo ###################################################### + --vertical_results + SELECT @sleep_time_per_result_row * @row_count - @max_acceptable_delay > + @sleep_time_per_result_row AS must_be_1, + @row_count - 1 >= 3 AS must_be_also_1, + @sleep_time_per_result_row, @row_count, @max_acceptable_delay; + exit; +} +SET @@global.query_cache_size = 1024 * 64; +--disable_warnings +DROP TEMPORARY TABLE IF EXISTS t_history; +DROP TABLE IF EXISTS t1; +--enable_warnings +CREATE TEMPORARY TABLE t_history (attempt SMALLINT, +start_ts DATETIME, end_ts DATETIME, +start_cached INTEGER, end_cached INTEGER); +CREATE TABLE t1 (f1 BIGINT); +let $num = `SELECT @row_count`; +--disable_query_log +begin; +while ($num) +{ + INSERT INTO t1 VALUES (1); + dec $num; +} +commit; +--enable_query_log + +let $loops = 4; +let $num = $loops; +while ($num) +{ + let $Qcache_queries_in_cache = + query_get_value(SHOW STATUS LIKE 'Qcache_queries_in_cache', Value, 1); + eval + INSERT INTO t_history + SET attempt = $loops - $num + 1, start_ts = NOW(), + start_cached = $Qcache_queries_in_cache; + SELECT *, SLEEP(@sleep_time_per_result_row) FROM t1; + # + # Do not determine Qcache_queries_in_cache before updating end_ts. The SHOW + # might cost too much time on an overloaded box. + eval + UPDATE t_history SET end_ts = NOW() + WHERE attempt = $loops - $num + 1; + let $Qcache_queries_in_cache = + query_get_value(SHOW STATUS LIKE 'Qcache_queries_in_cache', Value, 1); + eval + UPDATE t_history SET end_cached = $Qcache_queries_in_cache + WHERE attempt = $loops - $num + 1; + # DEBUG eval SELECT * FROM t_history WHERE attempt = $loops - $num + 1; + dec $num; +} + +# 1. The majority of queries with SLEEP must need a reasonable time +# -> SLEEP has an impact on runtime +# = Replacement for original Bug#6760 test +# -> total runtime is clear more needed than for one result row needed +# = Replacement for one of the original Bug#12689 tests +--echo # Test 1: Does the query with SLEEP need a reasonable time? +eval SELECT COUNT(*) >= $loops - 1 INTO @aux1 FROM t_history +WHERE TIMEDIFF(end_ts,start_ts) - @sleep_time_per_result_row * @row_count + BETWEEN 0 AND @max_acceptable_delay; +SELECT @aux1 AS "Expect 1"; +# +# 2. The majority of queries (the first one must be ignored) with SLEEP must +# need a reasonable time +# -> If we assume that the result of a cached query will be sent back +# immediate, without any sleep, than the query with SLEEP cannot be cached +# (current and intended behaviour for queries with SLEEP). +# -> It could be also not excluded that the query was cached but the server +# honoured somehow the SLEEP. Such a behaviour would be also acceptable. +# = Replacement for one of the original Bug#12689 tests +--echo # Test 2: Does the query with SLEEP need a reasonable time even in case +--echo # of the non first execution? +eval SELECT COUNT(*) >= $loops - 1 - 1 INTO @aux2 FROM t_history +WHERE TIMEDIFF(end_ts,start_ts) - @sleep_time_per_result_row * @row_count + BETWEEN 0 AND @max_acceptable_delay + AND attempt > 1; +SELECT @aux2 AS "Expect 1"; +# +# 3. The query with SLEEP should be not cached. +# -> SHOW STATUS Qcache_queries_in_cache must be not incremented after +# the execution of the query with SLEEP +--echo # Test 3: The query with SLEEP must be not cached. +eval SELECT COUNT(*) = $loops INTO @aux3 FROM t_history +WHERE end_cached = start_cached; +SELECT @aux3 AS "Expect 1"; +# +# Dump the content of t_history if one of the tests failed. +if (`SELECT @aux1 + @aux2 + @aux3 <> 3`) +{ + --echo # Some tests failed, dumping the content of t_history + SELECT * FROM t_history; +} +DROP TABLE t1; +DROP TEMPORARY TABLE t_history; +SET @@global.query_cache_size = default; + --echo End of 5.0 tests # === modified file 'mysql-test/t/query_cache_debug.test' --- a/mysql-test/t/query_cache_debug.test 2009-09-07 20:50:10 +0000 +++ b/mysql-test/t/query_cache_debug.test 2010-01-29 10:42:31 +0000 @@ -1,5 +1,4 @@ --source include/not_embedded.inc ---source include/big_test.inc --source include/have_query_cache.inc --source include/have_debug.inc === modified file 'mysql-test/t/query_cache_notembedded.test' --- a/mysql-test/t/query_cache_notembedded.test 2009-04-25 09:04:38 +0000 +++ b/mysql-test/t/query_cache_notembedded.test 2010-01-11 13:15:28 +0000 @@ -274,5 +274,52 @@ set GLOBAL query_cache_limit=default; set GLOBAL query_cache_min_res_unit=default; set GLOBAL query_cache_size=default; +# +# Bug#3583 query cache doesn't work for stored procedures +# +delimiter |; +--disable_warnings +drop table if exists t1| +--enable_warnings +create table t1 ( + id char(16) not null default '', + data int not null +)| +--disable_warnings +drop procedure if exists bug3583| +--enable_warnings +--disable_warnings +drop procedure if exists bug3583| +--enable_warnings +create procedure bug3583() +begin + declare c int; + + select * from t1; + select count(*) into c from t1; + select c; +end| + +insert into t1 values ("x", 3), ("y", 5)| +set @x = @@query_cache_size| +set global query_cache_size = 10*1024*1024| + +flush status| +flush query cache| +show status like 'Qcache_hits'| +call bug3583()| +show status like 'Qcache_hits'| +call bug3583()| +call bug3583()| +show status like 'Qcache_hits'| + +set global query_cache_size = @x| +flush status| +flush query cache| +delete from t1| +drop procedure bug3583| +drop table t1| +delimiter ;| + # Wait till we reached the initial number of concurrent sessions --source include/wait_until_count_sessions.inc === modified file 'mysql-test/t/range.test' --- a/mysql-test/t/range.test 2009-11-02 12:24:07 +0000 +++ b/mysql-test/t/range.test 2009-12-08 09:26:11 +0000 @@ -1260,4 +1260,57 @@ SELECT str_to_date('', '%Y-%m-%d'); DROP TABLE t1, t2; +--echo # +--echo # Bug#48459: valgrind errors with query using 'Range checked for each +--echo # record' +--echo # +CREATE TABLE t1 ( + a INT, + b CHAR(2), + c INT, + d INT, + KEY ( c ), + KEY ( d, a, b ( 2 ) ), + KEY ( b ( 1 ) ) +); + +INSERT INTO t1 VALUES ( NULL, 'a', 1, 2 ), ( NULL, 'a', 1, 2 ), + ( 1, 'a', 1, 2 ), ( 1, 'a', 1, 2 ); + +CREATE TABLE t2 ( + a INT, + c INT, + e INT, + KEY ( e ) +); + +INSERT INTO t2 VALUES ( 1, 1, NULL ), ( 1, 1, NULL ); + +--echo # Should not give Valgrind warnings +SELECT 1 +FROM t1, t2 +WHERE t1.d <> '1' AND t1.b > '1' +AND t1.a = t2.a AND t1.c = t2.c; + +DROP TABLE t1, t2; + +--echo # +--echo # Bug #48665: sql-bench's insert test fails due to wrong result +--echo # + +CREATE TABLE t1 (a INT, b INT, PRIMARY KEY (a)); + +INSERT INTO t1 VALUES (0,0), (1,1); + +--replace_column 1 @ 2 @ 3 @ 5 @ 6 @ 7 @ 8 @ 9 @ 10 @ +EXPLAIN +SELECT * FROM t1 FORCE INDEX (PRIMARY) + WHERE (a>=1 AND a<=2) OR (a>=4 AND a<=5) OR (a>=0 AND a <=10); + +--echo # Should return 2 rows +SELECT * FROM t1 FORCE INDEX (PRIMARY) + WHERE (a>=1 AND a<=2) OR (a>=4 AND a<=5) OR (a>=0 AND a <=10); + +DROP TABLE t1; + --echo End of 5.1 tests === modified file 'mysql-test/t/select.test' --- a/mysql-test/t/select.test 2009-10-30 14:13:13 +0000 +++ b/mysql-test/t/select.test 2009-12-15 17:08:21 +0000 @@ -3772,6 +3772,19 @@ INTO @var0; DROP TABLE t1; +--echo # +--echo # Bug #48458: simple query tries to allocate enormous amount of +--echo # memory +--echo # + +CREATE TABLE t1(a INT NOT NULL, b YEAR); +INSERT INTO t1 VALUES (); +CREATE TABLE t2(c INT); +--echo # Should not err out because of out-of-memory +SELECT 1 FROM t2 JOIN t1 ON 1=1 + WHERE a != '1' AND NOT a >= b OR NOT ROW(b,a )<> ROW(a,a); +DROP TABLE t1,t2; + --echo End of 5.0 tests @@ -3918,4 +3931,60 @@ SELECT table1 .`time_key` field2 FROM B drop table A,AA,B,BB; --echo #end of test for bug#45266 + +--echo # +--echo # BUG#48052: Valgrind warning - uninitialized value in init_read_record() +--echo # + +# Needed in 6.0 codebase +#--echo # Disable Index condition pushdown +#--replace_column 1 # +#SELECT @old_icp:=@@engine_condition_pushdown; +#SET SESSION engine_condition_pushdown = 'OFF'; + +CREATE TABLE t1 ( + pk int(11) NOT NULL, + i int(11) DEFAULT NULL, + v varchar(1) DEFAULT NULL, + PRIMARY KEY (pk) +); + +INSERT INTO t1 VALUES (2,7,'m'); +INSERT INTO t1 VALUES (3,9,'m'); + +SELECT v +FROM t1 +WHERE NOT pk > 0 +HAVING v <= 't' +ORDER BY pk; + +# Needed in 6.0 codebase +#--echo # Restore old value for Index condition pushdown +#SET SESSION engine_condition_pushdown=@old_icp; + +DROP TABLE t1; + +--echo # +--echo # Bug#49489 Uninitialized cache led to a wrong result. +--echo # +CREATE TABLE t1(c1 DOUBLE(5,4)); +INSERT INTO t1 VALUES (9.1234); +SELECT * FROM t1 WHERE c1 < 9.12345; +DROP TABLE t1; +--echo # End of test for bug#49489. + + +--echo # +--echo # Bug #49517: Inconsistent behavior while using +--echo # NULLable BIGINT and INT columns in comparison +--echo # +CREATE TABLE t1(a BIGINT UNSIGNED NOT NULL, b BIGINT NULL, c INT NULL); +INSERT INTO t1 VALUES(105, NULL, NULL); +SELECT * FROM t1 WHERE b < 102; +SELECT * FROM t1 WHERE c < 102; +SELECT * FROM t1 WHERE 102 < b; +SELECT * FROM t1 WHERE 102 < c; +DROP TABLE t1; + + --echo End of 5.1 tests === modified file 'mysql-test/t/show_check-master.opt' --- a/mysql-test/t/show_check-master.opt 2009-12-03 11:19:05 +0000 +++ b/mysql-test/t/show_check-master.opt 2010-01-27 21:53:08 +0000 @@ -1 +1 @@ ---log-output=table --slow-query-log --log-long-format --log-queries-not-using-indexes --myisam-recover="" --general-log --general-log-file="" --slow-query-log-file="" +--log-output=file --slow-query-log --log-long-format --log-queries-not-using-indexes --myisam-recover="" --general-log --general-log-file="foo" --slow-query-log-file="" === modified file 'mysql-test/t/show_check.test' --- a/mysql-test/t/show_check.test 2009-03-06 14:56:17 +0000 +++ b/mysql-test/t/show_check.test 2009-12-15 09:03:24 +0000 @@ -1207,6 +1207,28 @@ connection default; DROP USER test_u@localhost; +--echo # +--echo # Bug #48985: show create table crashes if previous access to the table +--echo # was killed +--echo # + +connect(con1,localhost,root,,); +CONNECTION con1; +LET $ID= `SELECT connection_id()`; + +CONNECTION default; +--disable_query_log +eval KILL QUERY $ID; +--enable_query_log + +CONNECTION con1; +--error ER_QUERY_INTERRUPTED +SHOW CREATE TABLE non_existent; + +CONNECTION default; +DISCONNECT con1; + + --echo End of 5.1 tests # Wait till all disconnects are completed === modified file 'mysql-test/t/sp-destruct.test' --- a/mysql-test/t/sp-destruct.test 2008-04-08 14:51:26 +0000 +++ b/mysql-test/t/sp-destruct.test 2009-11-21 11:18:21 +0000 @@ -12,6 +12,9 @@ # mysqltest should be fixed to allow REPLACE_RESULT in error message -- source include/not_embedded.inc +# Supress warnings written to the log file +call mtr.add_suppression("Column count of mysql.proc is wrong. Expected 20, found 19. The table is probably corrupted"); + # Backup proc table let $MYSQLD_DATADIR= `select @@datadir`; --copy_file $MYSQLD_DATADIR/mysql/proc.frm $MYSQLTEST_VARDIR/tmp/proc.frm @@ -38,15 +41,14 @@ create trigger t1_ai after insert on t1 # Unsupported tampering with the mysql.proc definition alter table mysql.proc drop type; ---replace_result $MYSQL_TEST_DIR . ---error ER_SP_PROC_TABLE_CORRUPT +--error ER_COL_COUNT_DOESNT_MATCH_CORRUPTED call bug14233(); ---replace_result $MYSQL_TEST_DIR . ---error ER_SP_PROC_TABLE_CORRUPT +--error ER_COL_COUNT_DOESNT_MATCH_CORRUPTED create view v1 as select bug14233_f(); ---replace_result $MYSQL_TEST_DIR . ---error ER_SP_PROC_TABLE_CORRUPT +--error ER_COL_COUNT_DOESNT_MATCH_CORRUPTED insert into t1 values (0); +--error ER_COL_COUNT_DOESNT_MATCH_CORRUPTED +show procedure status; flush table mysql.proc; @@ -155,3 +157,43 @@ drop procedure bug14233_3; # Assert: These should show nothing. show procedure status where db=DATABASE(); show function status where db=DATABASE(); + +# +# Bug#41726 upgrade from 5.0 to 5.1.30 crashes if you didn't run mysql_upgrade +# + + +--disable_warnings +DROP TABLE IF EXISTS proc_backup; +DROP PROCEDURE IF EXISTS p1; +--enable_warnings + +--echo # Backup the proc table + +RENAME TABLE mysql.proc TO proc_backup; +CREATE TABLE mysql.proc LIKE proc_backup; +FLUSH TABLE mysql.proc; + +--echo # Test with a valid table. + +CREATE PROCEDURE p1() + SET @foo = 10; +CALL p1(); +--replace_column 5 '0000-00-00 00:00:00' 6 '0000-00-00 00:00:00' +SHOW PROCEDURE STATUS; + +--echo # Modify a field of the table. + +ALTER TABLE mysql.proc MODIFY comment CHAR (32); + +--error ER_CANNOT_LOAD_FROM_TABLE +CREATE PROCEDURE p2() + SET @foo = 10; +--echo # Procedure loaded from the cache +CALL p1(); +--error ER_CANNOT_LOAD_FROM_TABLE +SHOW PROCEDURE STATUS; + +DROP TABLE mysql.proc; +RENAME TABLE proc_backup TO mysql.proc; +FLUSH TABLE mysql.proc; === modified file 'mysql-test/t/sp-security.test' --- a/mysql-test/t/sp-security.test 2009-03-06 14:56:17 +0000 +++ b/mysql-test/t/sp-security.test 2009-11-27 16:10:28 +0000 @@ -865,6 +865,65 @@ DROP PROCEDURE p_suid; DROP FUNCTION f_suid; DROP TABLE t1; +--echo # +--echo # Bug #48872 : Privileges for stored functions ignored if function name +--echo # is mixed case +--echo # + +CREATE DATABASE B48872; +USE B48872; +CREATE TABLE `TestTab` (id INT); +INSERT INTO `TestTab` VALUES (1),(2); +CREATE FUNCTION `f_Test`() RETURNS INT RETURN 123; +CREATE FUNCTION `f_Test_denied`() RETURNS INT RETURN 123; +CREATE USER 'tester'; +CREATE USER 'Tester'; +GRANT SELECT ON TABLE `TestTab` TO 'tester'; +GRANT EXECUTE ON FUNCTION `f_Test` TO 'tester'; +GRANT EXECUTE ON FUNCTION `f_Test_denied` TO 'Tester'; + +SELECT f_Test(); +SELECT * FROM TestTab; + +CONNECT (con_tester,localhost,tester,,B48872); +CONNECT (con_tester_denied,localhost,Tester,,B48872); +CONNECTION con_tester; + +SELECT * FROM TestTab; +SELECT `f_Test`(); +SELECT `F_TEST`(); +SELECT f_Test(); +SELECT F_TEST(); + +CONNECTION con_tester_denied; + +--disable_result_log +--error ER_TABLEACCESS_DENIED_ERROR +SELECT * FROM TestTab; +--error ER_PROCACCESS_DENIED_ERROR +SELECT `f_Test`(); +--error ER_PROCACCESS_DENIED_ERROR +SELECT `F_TEST`(); +--error ER_PROCACCESS_DENIED_ERROR +SELECT f_Test(); +--error ER_PROCACCESS_DENIED_ERROR +SELECT F_TEST(); +--enable_result_log +SELECT `f_Test_denied`(); +SELECT `F_TEST_DENIED`(); + +CONNECTION default; +DISCONNECT con_tester; +DISCONNECT con_tester_denied; +DROP TABLE `TestTab`; +DROP FUNCTION `f_Test`; +DROP FUNCTION `f_Test_denied`; + +USE test; +DROP USER 'tester'; +DROP USER 'Tester'; +DROP DATABASE B48872; + --echo End of 5.0 tests. # Wait till all disconnects are completed === modified file 'mysql-test/t/sp.test' --- a/mysql-test/t/sp.test 2009-10-23 13:54:58 +0000 +++ b/mysql-test/t/sp.test 2009-11-13 01:03:26 +0000 @@ -8263,6 +8263,73 @@ CALL p1; DROP PROCEDURE p1; DROP TABLE t1, t2; +--echo # +--echo # Bug#47627: SET @@{global.session}.local_variable in stored routine causes crash +--echo # Bug#48626: Crash or lost connection using SET for declared variables with @@ +--echo # + +--disable_warnings +DROP PROCEDURE IF EXISTS p1; +DROP PROCEDURE IF EXISTS p2; +DROP PROCEDURE IF EXISTS p3; +--enable_warnings + +delimiter //; + +--error ER_UNKNOWN_SYSTEM_VARIABLE +CREATE PROCEDURE p1() +BEGIN + DECLARE v INT DEFAULT 0; + SET @@SESSION.v= 10; +END// + +CREATE PROCEDURE p2() +BEGIN + DECLARE v INT DEFAULT 0; + SET v= 10; +END// +call p2()// + +--error ER_UNKNOWN_SYSTEM_VARIABLE +CREATE PROCEDURE p3() +BEGIN + DECLARE v INT DEFAULT 0; + SELECT @@SESSION.v; +END// + +--error ER_UNKNOWN_SYSTEM_VARIABLE +CREATE PROCEDURE p4() +BEGIN + DECLARE v INT DEFAULT 0; + SET @@GLOBAL.v= 10; +END// + +CREATE PROCEDURE p5() +BEGIN + DECLARE init_connect INT DEFAULT 0; + SET init_connect= 10; + SET @@GLOBAL.init_connect= 'SELECT 1'; + SET @@SESSION.IDENTITY= 1; + SELECT @@SESSION.IDENTITY; + SELECT @@GLOBAL.init_connect; + SELECT init_connect; +END// + +--error ER_UNKNOWN_SYSTEM_VARIABLE +CREATE PROCEDURE p6() +BEGIN + DECLARE v INT DEFAULT 0; + SET @@v= 0; +END// + +delimiter ;// + +SET @old_init_connect= @@GLOBAL.init_connect; +CALL p5(); +SET @@GLOBAL.init_connect= @old_init_connect; + +DROP PROCEDURE p2; +DROP PROCEDURE p5; --echo # ------------------------------------------------------------------ --echo # -- End of 5.1 tests === modified file 'mysql-test/t/sp_notembedded.test' --- a/mysql-test/t/sp_notembedded.test 2009-10-13 18:21:42 +0000 +++ b/mysql-test/t/sp_notembedded.test 2010-01-11 13:15:28 +0000 @@ -56,52 +56,6 @@ show warnings| drop procedure bug4902_2| # -# Bug#3583 query cache doesn't work for stored procedures -# ---disable_warnings -drop table if exists t1| ---enable_warnings -create table t1 ( - id char(16) not null default '', - data int not null -)| ---disable_warnings -drop procedure if exists bug3583| ---enable_warnings ---disable_warnings -drop procedure if exists bug3583| ---enable_warnings -create procedure bug3583() -begin - declare c int; - - select * from t1; - select count(*) into c from t1; - select c; -end| - -insert into t1 values ("x", 3), ("y", 5)| -set @x = @@query_cache_size| -set global query_cache_size = 10*1024*1024| - -flush status| -flush query cache| -show status like 'Qcache_hits'| -call bug3583()| -show status like 'Qcache_hits'| -call bug3583()| -call bug3583()| -show status like 'Qcache_hits'| - -set global query_cache_size = @x| -flush status| -flush query cache| -delete from t1| -drop procedure bug3583| -drop table t1| - - -# # Bug#6807 Stored procedure crash if CREATE PROCEDURE ... KILL QUERY # --disable_warnings === modified file 'mysql-test/t/trigger.test' --- a/mysql-test/t/trigger.test 2009-06-22 12:51:33 +0000 +++ b/mysql-test/t/trigger.test 2010-01-12 08:19:48 +0000 @@ -1767,68 +1767,6 @@ drop table t1, t2, t3; disconnect addconroot1; disconnect addconroot2; disconnect addconwithoutdb; -# -# Bug #26162: Trigger DML ignores low_priority_updates setting -# -CREATE TABLE t1 (id INTEGER); -CREATE TABLE t2 (id INTEGER); - -INSERT INTO t2 VALUES (1),(2); - -# trigger that produces the high priority insert, but should be low, adding -# LOW_PRIORITY fixes this -CREATE TRIGGER t1_test AFTER INSERT ON t1 FOR EACH ROW - INSERT INTO t2 VALUES (new.id); - -CONNECT (rl_holder, localhost, root,,); -CONNECT (rl_acquirer, localhost, root,,); -CONNECT (wl_acquirer, localhost, root,,); -CONNECT (rl_contender, localhost, root,,); - -CONNECTION rl_holder; -SELECT GET_LOCK('B26162',120); - -CONNECTION rl_acquirer; ---send -SELECT 'rl_acquirer', GET_LOCK('B26162',120), id FROM t2 WHERE id = 1; - -CONNECTION wl_acquirer; -SET SESSION LOW_PRIORITY_UPDATES=1; -SET GLOBAL LOW_PRIORITY_UPDATES=1; -#need to wait for rl_acquirer to lock on the B26162 lock -sleep 2; ---send -INSERT INTO t1 VALUES (5); - -CONNECTION rl_contender; -# must not "see" the row inserted by the INSERT (as it must run before the -# INSERT) ---send -SELECT 'rl_contender', id FROM t2 WHERE id > 1; - -CONNECTION rl_holder; -#need to wait for wl_acquirer and rl_contender to lock on t2 -sleep 2; -SELECT RELEASE_LOCK('B26162'); - -CONNECTION rl_acquirer; ---reap -SELECT RELEASE_LOCK('B26162'); -CONNECTION wl_acquirer; ---reap -CONNECTION rl_contender; ---reap - -CONNECTION default; -DISCONNECT rl_acquirer; -DISCONNECT wl_acquirer; -DISCONNECT rl_contender; -DISCONNECT rl_holder; - -DROP TRIGGER t1_test; -DROP TABLE t1,t2; -SET SESSION LOW_PRIORITY_UPDATES=DEFAULT; -SET GLOBAL LOW_PRIORITY_UPDATES=DEFAULT; --echo --echo Bug#28502 Triggers that update another innodb table will block --echo on X lock unnecessarily === modified file 'mysql-test/t/trigger_notembedded.test' --- a/mysql-test/t/trigger_notembedded.test 2009-06-25 10:52:50 +0000 +++ b/mysql-test/t/trigger_notembedded.test 2010-01-12 08:19:48 +0000 @@ -875,6 +875,79 @@ DROP TABLE t1; DROP DATABASE mysqltest_db1; USE test; +# +# Bug #26162: Trigger DML ignores low_priority_updates setting +# +CREATE TABLE t1 (id INTEGER); +CREATE TABLE t2 (id INTEGER); + +INSERT INTO t2 VALUES (1),(2); + +# trigger that produces the high priority insert, but should be low, adding +# LOW_PRIORITY fixes this +CREATE TRIGGER t1_test AFTER INSERT ON t1 FOR EACH ROW + INSERT INTO t2 VALUES (new.id); + +CONNECT (rl_holder, localhost, root,,); +CONNECT (rl_acquirer, localhost, root,,); +CONNECT (wl_acquirer, localhost, root,,); +CONNECT (rl_contender, localhost, root,,); + +CONNECTION rl_holder; +SELECT GET_LOCK('B26162',120); + +CONNECTION rl_acquirer; +let $rl_acquirer_thread_id = `SELECT @@pseudo_thread_id`; +--send +SELECT 'rl_acquirer', GET_LOCK('B26162',120), id FROM t2 WHERE id = 1; + +CONNECTION wl_acquirer; +let $wl_acquirer_thread_id = `SELECT @@pseudo_thread_id`; +SET SESSION LOW_PRIORITY_UPDATES=1; +SET GLOBAL LOW_PRIORITY_UPDATES=1; +#need to wait for rl_acquirer to lock on the B26162 lock +let $wait_condition= + SELECT STATE = 'User lock' FROM INFORMATION_SCHEMA.PROCESSLIST + WHERE ID = $rl_acquirer_thread_id; +--source include/wait_condition.inc +--send +INSERT INTO t1 VALUES (5); + +CONNECTION rl_contender; +# Wait until wl_acquirer is waiting for the read lock on t2 to be released. +let $wait_condition= + SELECT STATE = 'Locked' FROM INFORMATION_SCHEMA.PROCESSLIST + WHERE ID = $wl_acquirer_thread_id; +--source include/wait_condition.inc +# must not "see" the row inserted by the INSERT (as it must run before the +# INSERT) +--send +SELECT 'rl_contender', id FROM t2 WHERE id > 1; + +CONNECTION rl_holder; +#need to wait for wl_acquirer and rl_contender to lock on t2 +sleep 2; +SELECT RELEASE_LOCK('B26162'); + +CONNECTION rl_acquirer; +--reap +SELECT RELEASE_LOCK('B26162'); +CONNECTION wl_acquirer; +--reap +CONNECTION rl_contender; +--reap + +CONNECTION default; +DISCONNECT rl_acquirer; +DISCONNECT wl_acquirer; +DISCONNECT rl_contender; +DISCONNECT rl_holder; + +DROP TRIGGER t1_test; +DROP TABLE t1,t2; +SET SESSION LOW_PRIORITY_UPDATES=DEFAULT; +SET GLOBAL LOW_PRIORITY_UPDATES=DEFAULT; + --echo End of 5.0 tests. # === modified file 'mysql-test/t/type_newdecimal.test' --- a/mysql-test/t/type_newdecimal.test 2009-11-02 11:21:39 +0000 +++ b/mysql-test/t/type_newdecimal.test 2009-12-08 09:26:11 +0000 @@ -1286,3 +1286,229 @@ CREATE TABLE t1 SELECT 1 % .123456789123 DESCRIBE t1; SELECT my_col FROM t1; DROP TABLE t1; + +--echo # +--echo # Bug#45261: Crash, stored procedure + decimal +--echo # + +--disable_warnings +DROP TABLE IF EXISTS t1; +--enable_warnings + +CREATE TABLE t1 SELECT + /* 81 */ 100000000000000000000000000000000000000000000000000000000000000000000000000000001 + AS c1; +DESC t1; +SELECT * FROM t1; +DROP TABLE t1; + +CREATE TABLE t1 SELECT + /* 81 */ 100000000000000000000000000000000000000000000000000000000000000000000000000000001. + AS c1; +DESC t1; +SELECT * FROM t1; +DROP TABLE t1; + +CREATE TABLE t1 SELECT + /* 81 */ 100000000000000000000000000000000000000000000000000000000000000000000000000000001.1 /* 1 */ + AS c1; +DESC t1; +SELECT * FROM t1; +DROP TABLE t1; + +CREATE TABLE t1 SELECT + /* 82 */ 1000000000000000000000000000000000000000000000000000000000000000000000000000000001 + AS c1; +DESC t1; +SELECT * FROM t1; +DROP TABLE t1; + +CREATE TABLE t1 SELECT + /* 40 */ 1000000000000000000000000000000000000001.1000000000000000000000000000000000000001 /* 40 */ + AS c1; +DESC t1; +SELECT * FROM t1; +DROP TABLE t1; + +CREATE TABLE t1 SELECT + /* 1 */ 1.10000000000000000000000000000000000000000000000000000000000000000000000000000001 /* 80 */ + AS c1; +DESC t1; +SELECT * FROM t1; +DROP TABLE t1; + +CREATE TABLE t1 SELECT + /* 1 */ 1.100000000000000000000000000000000000000000000000000000000000000000000000000000001 /* 81 */ + AS c1; +DESC t1; +SELECT * FROM t1; +DROP TABLE t1; + +CREATE TABLE t1 SELECT + .100000000000000000000000000000000000000000000000000000000000000000000000000000001 /* 81 */ + AS c1; +DESC t1; +SELECT * FROM t1; +DROP TABLE t1; + +CREATE TABLE t1 SELECT + /* 45 */ 123456789012345678901234567890123456789012345.123456789012345678901234567890123456789012345 /* 45 */ + AS c1; +DESC t1; +SELECT * FROM t1; +DROP TABLE t1; + +CREATE TABLE t1 SELECT + /* 65 */ 12345678901234567890123456789012345678901234567890123456789012345.1 /* 1 */ + AS c1; +DESC t1; +SELECT * FROM t1; +DROP TABLE t1; + +CREATE TABLE t1 SELECT + /* 66 */ 123456789012345678901234567890123456789012345678901234567890123456.1 /* 1 */ + AS c1; +DESC t1; +SELECT * FROM t1; +DROP TABLE t1; + +CREATE TABLE t1 SELECT + .123456789012345678901234567890123456789012345678901234567890123456 /* 66 */ + AS c1; +DESC t1; +SELECT * FROM t1; +DROP TABLE t1; + +CREATE TABLE t1 AS SELECT 123.1234567890123456789012345678901 /* 31 */ AS c1; +DESC t1; +SELECT * FROM t1; +DROP TABLE t1; + +CREATE TABLE t1 SELECT 1.1 + CAST(1 AS DECIMAL(65,30)) AS c1; +DESC t1; +SELECT * FROM t1; +DROP TABLE t1; + +--echo # +--echo # Test that the integer and decimal parts are properly calculated. +--echo # + +CREATE TABLE t1 (a DECIMAL(30,30)); +INSERT INTO t1 VALUES (0.1),(0.2),(0.3); +CREATE TABLE t2 SELECT MIN(a + 0.0000000000000000000000000000001) AS c1 FROM t1; +DESC t2; +DROP TABLE t1,t2; + +CREATE TABLE t1 (a DECIMAL(30,30)); +INSERT INTO t1 VALUES (0.1),(0.2),(0.3); +CREATE TABLE t2 SELECT IFNULL(a + 0.0000000000000000000000000000001, NULL) AS c1 FROM t1; +DESC t2; +DROP TABLE t1,t2; + +CREATE TABLE t1 (a DECIMAL(30,30)); +INSERT INTO t1 VALUES (0.1),(0.2),(0.3); +CREATE TABLE t2 SELECT CASE a WHEN 0.1 THEN 0.0000000000000000000000000000000000000000000000000000000000000000001 END AS c1 FROM t1; +DESC t2; +DROP TABLE t1,t2; + +--echo # +--echo # Test that variables get maximum precision. +--echo # + +SET @decimal= 1.1; +CREATE TABLE t1 SELECT @decimal AS c1; +DESC t1; +SELECT * FROM t1; +DROP TABLE t1; + +--echo # +--echo # Bug #45261 : Crash, stored procedure + decimal +--echo # Original test by the reporter. +--echo # + +--echo # should not crash +CREATE TABLE t1 +SELECT .123456789012345678901234567890123456789012345678901234567890123456 AS a; +DROP TABLE t1; + +delimiter |; +CREATE PROCEDURE test_proc() +BEGIN + # The las non critical CUSER definition is: + # DECLARE mycursor CURSOR FOR SELECT 1 % + # .12345678912345678912345678912345678912345678912345678912345678912 AS my_col; + DECLARE mycursor CURSOR FOR +SELECT 1 % +.123456789123456789123456789123456789123456789123456789123456789123456789123456789 + AS my_col; + + OPEN mycursor; + CLOSE mycursor; +END| +delimiter ;| +--echo # should not crash +CALL test_proc(); +DROP PROCEDURE test_proc; + +--echo # +--echo # Bug #48370 Absolutely wrong calculations with GROUP BY and +--echo # decimal fields when using IF +--echo # + +CREATE TABLE currencies (id int, rate decimal(16,4), + PRIMARY KEY (id), KEY (rate)); + +INSERT INTO currencies VALUES (11,0.7028); +INSERT INTO currencies VALUES (1,1); + +CREATE TABLE payments ( + id int, + supplier_id int, + status int, + currency_id int, + vat decimal(7,4), + PRIMARY KEY (id), + KEY currency_id (currency_id), + KEY supplier_id (supplier_id) +); + +INSERT INTO payments (id,status,vat,supplier_id,currency_id) VALUES +(3001,2,0.0000,344,11), (1,2,0.0000,1,1); + +CREATE TABLE sub_tasks ( + id int, + currency_id int, + price decimal(16,4), + discount decimal(10,4), + payment_id int, + PRIMARY KEY (id), + KEY currency_id (currency_id), + KEY payment_id (payment_id) +) ; + +INSERT INTO sub_tasks (id, price, discount, payment_id, currency_id) VALUES +(52, 12.60, 0, 3001, 11), (56, 14.58, 0, 3001, 11); + +--echo # should return 1 and the same values in col 2 and 3 +select STRAIGHT_JOIN + (1 + PAY.vat) AS mult, + SUM(ROUND((SUB.price - ROUND(ROUND(SUB.price, 2) * SUB.discount, 2)) * + CUR.rate / CUR.rate, 2) + ) v_net_with_discount, + + SUM(ROUND((SUB.price - ROUND(ROUND(SUB.price, 2) * SUB.discount, 1)) * + CUR.rate / CUR.rate , 2) + * (1 + PAY.vat) + ) v_total +from + currencies CUR, payments PAY, sub_tasks SUB +where + SUB.payment_id = PAY.id and + PAY.currency_id = CUR.id and + PAY.id > 2 +group by PAY.id + 1; + +DROP TABLE currencies, payments, sub_tasks; + + +--echo End of 5.1 tests === modified file 'mysql-test/t/type_year.test' --- a/mysql-test/t/type_year.test 2007-03-29 04:08:30 +0000 +++ b/mysql-test/t/type_year.test 2009-12-15 08:37:10 +0000 @@ -30,3 +30,109 @@ select * from t1; drop table t1; --echo End of 5.0 tests + +--echo # +--echo # Bug #49480: WHERE using YEAR columns returns unexpected results +--echo # + +CREATE TABLE t2(yy YEAR(2), c2 CHAR(4)); +CREATE TABLE t4(yyyy YEAR(4), c4 CHAR(4)); + +INSERT INTO t2 (c2) VALUES (NULL),(1970),(1999),(2000),(2001),(2069); +INSERT INTO t4 (c4) SELECT c2 FROM t2; +UPDATE t2 SET yy = c2; +UPDATE t4 SET yyyy = c4; + +SELECT * FROM t2; +SELECT * FROM t4; + +--echo # Comparison of YEAR(2) with YEAR(4) + +SELECT * FROM t2, t4 WHERE yy = yyyy; +SELECT * FROM t2, t4 WHERE yy <=> yyyy; +SELECT * FROM t2, t4 WHERE yy < yyyy; +SELECT * FROM t2, t4 WHERE yy > yyyy; + +--echo # Comparison of YEAR(2) with YEAR(2) + +SELECT * FROM t2 a, t2 b WHERE a.yy = b.yy; +SELECT * FROM t2 a, t2 b WHERE a.yy <=> b.yy; +SELECT * FROM t2 a, t2 b WHERE a.yy < b.yy; + +--echo # Comparison of YEAR(4) with YEAR(4) + +SELECT * FROM t4 a, t4 b WHERE a.yyyy = b.yyyy; +SELECT * FROM t4 a, t4 b WHERE a.yyyy <=> b.yyyy; +SELECT * FROM t4 a, t4 b WHERE a.yyyy < b.yyyy; + +--echo # Comparison with constants: + +SELECT * FROM t2 WHERE yy = NULL; +SELECT * FROM t4 WHERE yyyy = NULL; +SELECT * FROM t2 WHERE yy <=> NULL; +SELECT * FROM t4 WHERE yyyy <=> NULL; +SELECT * FROM t2 WHERE yy < NULL; +SELECT * FROM t2 WHERE yy > NULL; + +SELECT * FROM t2 WHERE yy = NOW(); +SELECT * FROM t4 WHERE yyyy = NOW(); + +SELECT * FROM t2 WHERE yy = 99; +SELECT * FROM t2 WHERE 99 = yy; +SELECT * FROM t4 WHERE yyyy = 99; + +SELECT * FROM t2 WHERE yy = 'test'; +SELECT * FROM t4 WHERE yyyy = 'test'; + +SELECT * FROM t2 WHERE yy = '1999'; +SELECT * FROM t4 WHERE yyyy = '1999'; + +SELECT * FROM t2 WHERE yy = 1999; +SELECT * FROM t4 WHERE yyyy = 1999; + +SELECT * FROM t2 WHERE yy = 1999.1; +SELECT * FROM t4 WHERE yyyy = 1999.1; + +SELECT * FROM t2 WHERE yy = 1998.9; +SELECT * FROM t4 WHERE yyyy = 1998.9; + +--echo # Coverage tests for YEAR with zero/2000 constants: + +SELECT * FROM t2 WHERE yy = 0; +SELECT * FROM t2 WHERE yy = '0'; +SELECT * FROM t2 WHERE yy = '0000'; +SELECT * FROM t2 WHERE yy = '2000'; +SELECT * FROM t2 WHERE yy = 2000; + +SELECT * FROM t4 WHERE yyyy = 0; +SELECT * FROM t4 WHERE yyyy = '0'; +SELECT * FROM t4 WHERE yyyy = '0000'; +SELECT * FROM t4 WHERE yyyy = '2000'; +SELECT * FROM t4 WHERE yyyy = 2000; + +--echo # Comparison with constants those are out of YEAR range +--echo # (coverage test for backward compatibility) + +SELECT COUNT(yy) FROM t2; +SELECT COUNT(yyyy) FROM t4; + +SELECT COUNT(*) FROM t2 WHERE yy = -1; +SELECT COUNT(*) FROM t4 WHERE yyyy > -1; +SELECT COUNT(*) FROM t2 WHERE yy > -1000000000000000000; +SELECT COUNT(*) FROM t4 WHERE yyyy > -1000000000000000000; + +SELECT COUNT(*) FROM t2 WHERE yy < 2156; +SELECT COUNT(*) FROM t4 WHERE yyyy < 2156; +SELECT COUNT(*) FROM t2 WHERE yy < 1000000000000000000; +SELECT COUNT(*) FROM t4 WHERE yyyy < 1000000000000000000; + +SELECT * FROM t2 WHERE yy < 123; +SELECT * FROM t2 WHERE yy > 123; +SELECT * FROM t4 WHERE yyyy < 123; +SELECT * FROM t4 WHERE yyyy > 123; + +DROP TABLE t2, t4; + +--echo # + +--echo End of 5.1 tests === modified file 'mysql-test/t/udf.test' --- a/mysql-test/t/udf.test 2009-09-07 09:57:22 +0000 +++ b/mysql-test/t/udf.test 2010-01-22 21:19:21 +0000 @@ -56,9 +56,9 @@ select reverse_lookup(); --disable_result_log select reverse_lookup("127.0.0.1"); select reverse_lookup(127,0,0,1); +select reverse_lookup("localhost"); --enable_result_log -select reverse_lookup("localhost"); --error ER_CANT_INITIALIZE_UDF select avgcost(); --error ER_CANT_INITIALIZE_UDF @@ -342,29 +342,6 @@ drop function myfunc_double; drop function myfunc_int; # -# Bug #28921: Queries containing UDF functions are cached -# - ---replace_result $UDF_EXAMPLE_LIB UDF_EXAMPLE_LIB -eval CREATE FUNCTION metaphon RETURNS STRING SONAME "$UDF_EXAMPLE_LIB"; -create table t1 (a char); - -set GLOBAL query_cache_size=1355776; -reset query cache; - -select metaphon('MySQL') from t1; -show status like "Qcache_hits"; -show status like "Qcache_queries_in_cache"; - -select metaphon('MySQL') from t1; -show status like "Qcache_hits"; -show status like "Qcache_queries_in_cache"; - -drop table t1; -drop function metaphon; -set GLOBAL query_cache_size=default; - -# # Bug#28318 CREATE FUNCTION (UDF) requires a schema # === added file 'mysql-test/t/udf_query_cache-master.opt' --- a/mysql-test/t/udf_query_cache-master.opt 1970-01-01 00:00:00 +0000 +++ b/mysql-test/t/udf_query_cache-master.opt 2010-01-11 13:15:28 +0000 @@ -0,0 +1 @@ +$UDF_EXAMPLE_LIB_OPT === added file 'mysql-test/t/udf_query_cache.test' --- a/mysql-test/t/udf_query_cache.test 1970-01-01 00:00:00 +0000 +++ b/mysql-test/t/udf_query_cache.test 2010-01-11 13:15:28 +0000 @@ -0,0 +1,35 @@ +--source include/have_udf.inc +--source include/have_query_cache.inc +# +# To run this tests the "sql/udf_example.c" need to be compiled into +# udf_example.so and LD_LIBRARY_PATH should be setup to point out where +# the library are. +# + +--disable_warnings +drop table if exists t1; +--enable_warnings + +# +# Bug #28921: Queries containing UDF functions are cached +# + +--replace_result $UDF_EXAMPLE_LIB UDF_EXAMPLE_LIB +eval CREATE FUNCTION metaphon RETURNS STRING SONAME "$UDF_EXAMPLE_LIB"; +create table t1 (a char); + +set GLOBAL query_cache_size=1355776; +reset query cache; + +select metaphon('MySQL') from t1; +show status like "Qcache_hits"; +show status like "Qcache_queries_in_cache"; + +select metaphon('MySQL') from t1; +show status like "Qcache_hits"; +show status like "Qcache_queries_in_cache"; + +drop table t1; +drop function metaphon; +set GLOBAL query_cache_size=default; + === modified file 'mysql-test/t/variables.test' --- a/mysql-test/t/variables.test 2009-09-07 20:50:10 +0000 +++ b/mysql-test/t/variables.test 2010-01-11 13:15:28 +0000 @@ -28,8 +28,6 @@ set @my_myisam_max_sort_file_size =@@glo set @my_net_buffer_length =@@global.net_buffer_length; set @my_net_write_timeout =@@global.net_write_timeout; set @my_net_read_timeout =@@global.net_read_timeout; -set @my_query_cache_limit =@@global.query_cache_limit; -set @my_query_cache_type =@@global.query_cache_type; set @my_rpl_recovery_rank =@@global.rpl_recovery_rank; set @my_server_id =@@global.server_id; set @my_slow_launch_time =@@global.slow_launch_time; @@ -138,7 +136,6 @@ show local variables like 'storage_engin select * from information_schema.session_variables where variable_name like 'storage_engine'; show global variables like 'storage_engine'; select * from information_schema.global_variables where variable_name like 'storage_engine'; -set GLOBAL query_cache_size=100000; set GLOBAL myisam_max_sort_file_size=2000000; show global variables like 'myisam_max_sort_file_size'; @@ -255,8 +252,6 @@ set storage_engine=UNKNOWN_TABLE_TYPE; --error ER_WRONG_VALUE_FOR_VAR set storage_engine=MERGE, big_tables=2; show local variables like 'storage_engine'; ---error ER_GLOBAL_VARIABLE -set SESSION query_cache_size=10000; --error ER_NO_DEFAULT set GLOBAL storage_engine=DEFAULT; --error ER_UNKNOWN_CHARACTER_SET @@ -334,9 +329,6 @@ set myisam_sort_buffer_size=100; set global net_buffer_length=100; set net_read_timeout=100; set net_write_timeout=100; -set global query_cache_limit=100; -set global query_cache_size=100; -set global query_cache_type=demand; set read_buffer_size=100; set read_rnd_buffer_size=100; set global rpl_recovery_rank=100; @@ -822,8 +814,6 @@ set global myisam_max_sort_file_size =@m set global net_buffer_length =@my_net_buffer_length; set global net_write_timeout =@my_net_write_timeout; set global net_read_timeout =@my_net_read_timeout; -set global query_cache_limit =@my_query_cache_limit; -set global query_cache_type =@my_query_cache_type; set global rpl_recovery_rank =@my_rpl_recovery_rank; set global server_id =@my_server_id; set global slow_launch_time =@my_slow_launch_time; === modified file 'mysql-test/valgrind.supp' --- a/mysql-test/valgrind.supp 2009-12-03 11:19:05 +0000 +++ b/mysql-test/valgrind.supp 2010-01-18 12:56:10 +0000 @@ -991,3 +991,15 @@ fun:_dl_allocate_tls fun:pthread_create* } + +# +# Bug in Glibc 2.9: http://sourceware.org/bugzilla/show_bug.cgi?id=10391 +# Fixed in latest Glibc, but suppressed here for running tests on hosts +# with older Glibc version. +# +{ + Glibc bug in __libc_res_nsend + Memcheck:Cond + fun:__libc_res_nsend + fun:__libc_res_nquery +} === modified file 'mysys/my_file.c' --- a/mysys/my_file.c 2006-12-23 19:20:40 +0000 +++ b/mysys/my_file.c 2010-01-28 11:35:10 +0000 @@ -51,7 +51,7 @@ static uint set_max_open_files(uint max_ DBUG_PRINT("info", ("rlim_cur: %u rlim_max: %u", (uint) rlimit.rlim_cur, (uint) rlimit.rlim_max)); - if (rlimit.rlim_cur == RLIM_INFINITY) + if ((ulonglong) rlimit.rlim_cur == (ulonglong) RLIM_INFINITY) rlimit.rlim_cur = max_file_limit; if (rlimit.rlim_cur >= max_file_limit) DBUG_RETURN(rlimit.rlim_cur); /* purecov: inspected */ === modified file 'mysys/my_getopt.c' --- a/mysys/my_getopt.c 2009-12-03 11:19:05 +0000 +++ b/mysys/my_getopt.c 2010-01-15 15:27:55 +0000 @@ -414,17 +414,11 @@ invalid value '%s'", (optp->var_type & GET_TYPE_MASK) == GET_ENUM)) { if (optend == disabled_my_option) - if ((optp->var_type & GET_TYPE_MASK) == GET_BOOL) - *((my_bool*) value)= (my_bool) 0; - else - *((ulong*) value)= (ulong) 0; + init_one_value(optp, value, 0); else { if (!optend) /* No argument -> enable option */ - if ((optp->var_type & GET_TYPE_MASK) == GET_BOOL) - *((my_bool*) value)= (my_bool) 1; - else - *((ulong*) value)= (ulong) 1; + init_one_value(optp, value, 1); else argument= optend; } === modified file 'mysys/my_sync.c' --- a/mysys/my_sync.c 2010-01-06 21:27:53 +0000 +++ b/mysys/my_sync.c 2010-01-15 15:27:55 +0000 @@ -104,11 +104,11 @@ int my_sync_dir(const char *dir_name __a myf my_flags __attribute__((unused))) { #ifdef NEED_EXPLICIT_SYNC_DIR - DBUG_ENTER("my_sync_dir"); - DBUG_PRINT("my",("Dir: '%s' my_flags: %d", dir_name, my_flags)); File dir_fd; int res= 0; const char *correct_dir_name; + DBUG_ENTER("my_sync_dir"); + DBUG_PRINT("my",("Dir: '%s' my_flags: %d", dir_name, my_flags)); /* Sometimes the path does not contain an explicit directory */ correct_dir_name= (dir_name[0] == 0) ? cur_dir_name : dir_name; /* === modified file 'mysys/my_thr_init.c' --- a/mysys/my_thr_init.c 2009-12-03 11:19:05 +0000 +++ b/mysys/my_thr_init.c 2010-01-29 18:42:22 +0000 @@ -317,7 +317,7 @@ my_bool my_thread_init(void) /* Skip initialization if the thread specific variable is already initialized */ - if (THR_KEY_mysys.id) + if (THR_KEY_mysys.init) goto end; tmp= &THR_KEY_mysys; #endif === modified file 'scripts/make_win_bin_dist' --- a/scripts/make_win_bin_dist 2009-12-03 11:19:05 +0000 +++ b/scripts/make_win_bin_dist 2010-01-15 15:27:55 +0000 @@ -352,7 +352,7 @@ mkdir $DESTDIR/mysql-test cp mysql-test/mysql-test-run.pl $DESTDIR/mysql-test/ cp mysql-test/mysql-stress-test.pl $DESTDIR/mysql-test/ cp mysql-test/README $DESTDIR/mysql-test/ -cp -R mysql-test/{t,r,include,suite,std_data,lib} $DESTDIR/mysql-test/ +cp -R mysql-test/{t,r,include,suite,std_data,lib,collections} $DESTDIR/mysql-test/ rm -rf $DESTDIR/mysql-test/lib/My/SafeProcess/my_safe_kill.{dir,vcproj} rm -rf $DESTDIR/mysql-test/lib/My/SafeProcess/my_safe_process.{dir,vcproj} === modified file 'scripts/mysql_secure_installation.pl.in' --- a/scripts/mysql_secure_installation.pl.in 2007-12-28 21:58:54 +0000 +++ b/scripts/mysql_secure_installation.pl.in 2009-11-03 21:34:01 +0000 @@ -17,16 +17,41 @@ # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA use Fcntl; +use File::Spec; +use if $^O eq 'MSWin32', 'Term::ReadKey' => qw/ReadMode/; use strict; my $config = ".my.cnf.$$"; my $command = ".mysql.$$"; my $hadpass = 0; +my $mysql; # How to call the mysql client +my $rootpass = ""; -# FIXME -# trap "interrupt" 2 -my $rootpass = ""; +$SIG{QUIT} = $SIG{INT} = sub { + print "\nAborting!\n\n"; + echo_on(); + cleanup(); + exit 1; +}; + + +END { + # Remove temporary files, even if exiting via die(), etc. + cleanup(); +} + + +sub read_without_echo { + my ($prompt) = @_; + print $prompt; + echo_off(); + my $answer = <STDIN>; + echo_on(); + print "\n"; + chomp($answer); + return $answer; +} sub echo_on { if ($^O eq 'MSWin32') { @@ -55,6 +80,25 @@ sub write_file { } sub prepare { + # Locate the mysql client; look in current directory first, then + # in path + our $SAVEERR; # Suppress Perl warning message + open SAVEERR, ">& STDERR"; + close STDERR; + for my $m (File::Spec->catfile('bin', 'mysql'), 'mysql') { + # mysql --version should always work + qx($m --no-defaults --version); + next unless $? == 0; + + $mysql = $m; + last; + } + open STDERR, ">& SAVEERR"; + + die "Can't find a 'mysql' client in PATH or ./bin\n" + unless $mysql; + + # Create safe files to avoid leaking info to other users foreach my $file ( $config, $command ) { next if -f $file; # Already exists local *FILE; @@ -64,30 +108,50 @@ sub prepare { } } +# Simple escape mechanism (\-escape any ' and \), suitable for two contexts: +# - single-quoted SQL strings +# - single-quoted option values on the right hand side of = in my.cnf +# +# These two contexts don't handle escapes identically. SQL strings allow +# quoting any character (\C => C, for any C), but my.cnf parsing allows +# quoting only \, ' or ". For example, password='a\b' quotes a 3-character +# string in my.cnf, but a 2-character string in SQL. +# +# This simple escape works correctly in both places. +sub basic_single_escape { + my ($str) = @_; + # Inside a character class, \ is not special; this escapes both \ and ' + $str =~ s/([\'])/\\$1/g; + return $str; +} + sub do_query { my $query = shift; write_file($command, $query); - system("mysql --defaults-file=$config < $command"); - return $?; + my $rv = system("$mysql --defaults-file=$config < $command"); + # system() returns -1 if exec fails (e.g., command not found, etc.); die + # in this case because nothing is going to work + die "Failed to execute mysql client '$mysql'\n" if $rv == -1; + # Return true if query executed OK, or false if there was some problem + # (for example, SQL error or wrong password) + return ($rv == 0 ? 1 : undef); } sub make_config { my $password = shift; + my $esc_pass = basic_single_escape($rootpass); write_file($config, "# mysql_secure_installation config file", "[mysql]", "user=root", - "password=$rootpass"); + "password='$esc_pass'"); } sub get_root_password { - my $status = 1; - while ( $status == 1 ) { - echo_off(); - print "Enter current password for root (enter for none): "; - my $password = <STDIN>; - echo_on(); + my $attempts = 3; + for (;;) { + my $password = read_without_echo("Enter current password for root (enter for none): "); if ( $password ) { $hadpass = 1; } else { @@ -95,64 +159,56 @@ sub get_root_password { } $rootpass = $password; make_config($rootpass); - do_query(""); - $status = $?; + last if do_query(""); + + die "Unable to connect to the server as root user, giving up.\n" + if --$attempts == 0; } print "OK, successfully used password, moving on...\n\n"; } sub set_root_password { - echo_off(); - print "New password: "; - my $password1 = <STDIN>; - print "\nRe-enter new password: "; - my $password2 = <STDIN>; - print "\n"; - echo_on(); - - if ( $password1 eq $password2 ) { - print "Sorry, passwords do not match.\n\n"; - return 1; - } + my $password1; + for (;;) { + $password1 = read_without_echo("New password: "); + + if ( !$password1 ) { + print "Sorry, you can't use an empty password here.\n\n"; + next; + } - if ( !$password1 ) { - print "Sorry, you can't use an empty password here.\n\n"; - return 1; - } + my $password2 = read_without_echo("Re-enter new password: "); - do_query("UPDATE mysql.user SET Password=PASSWORD('$password1') WHERE User='root';"); - if ( $? == 0 ) { - print "Password updated successfully!\n"; - print "Reloading privilege tables..\n"; - if ( !reload_privilege_tables() ) { - exit 1; + if ( $password1 ne $password2 ) { + print "Sorry, passwords do not match.\n\n"; + next; } - print "\n"; - $rootpass = $password1; - make_config($rootpass); - } else { - print "Password update failed!\n"; - exit 1; + + last; } - return 0; + my $esc_pass = basic_single_escape($password1); + do_query("UPDATE mysql.user SET Password=PASSWORD('$esc_pass') WHERE User='root';") + or die "Password update failed!\n"; + + print "Password updated successfully!\n"; + print "Reloading privilege tables..\n"; + reload_privilege_tables() + or die "Can not continue.\n"; + + print "\n"; + $rootpass = $password1; + make_config($rootpass); } sub remove_anonymous_users { - do_query("DELETE FROM mysql.user WHERE User='';"); - if ( $? == 0 ) { - print " ... Success!\n"; - } else { - print " ... Failed!\n"; - exit 1; - } - - return 0; + do_query("DELETE FROM mysql.user WHERE User='';") + or die print " ... Failed!\n"; + print " ... Success!\n"; } sub remove_remote_root { - do_query("DELETE FROM mysql.user WHERE User='root' AND Host!='localhost';"); - if ( $? == 0 ) { + if (do_query("DELETE FROM mysql.user WHERE User='root' AND Host!='localhost';")) { print " ... Success!\n"; } else { print " ... Failed!\n"; @@ -161,44 +217,31 @@ sub remove_remote_root { sub remove_test_database { print " - Dropping test database...\n"; - do_query("DROP DATABASE test;"); - if ( $? == 0 ) { + if (do_query("DROP DATABASE test;")) { print " ... Success!\n"; } else { print " ... Failed! Not critical, keep moving...\n"; } print " - Removing privileges on test database...\n"; - do_query("DELETE FROM mysql.db WHERE Db='test' OR Db='test\\_%'"); - if ( $? == 0 ) { + if (do_query("DELETE FROM mysql.db WHERE Db='test' OR Db='test\\_%'")) { print " ... Success!\n"; } else { print " ... Failed! Not critical, keep moving...\n"; } - - return 0; } sub reload_privilege_tables { - do_query("FLUSH PRIVILEGES;"); - if ( $? == 0 ) { + if (do_query("FLUSH PRIVILEGES;")) { print " ... Success!\n"; - return 0; + return 1; } else { print " ... Failed!\n"; - return 1; + return undef; } } -sub interrupt { - print "\nAborting!\n\n"; - cleanup(); - echo_on(); - exit 1; -} - sub cleanup { - print "Cleaning up...\n"; unlink($config,$command); } @@ -242,11 +285,7 @@ my $reply = <STDIN>; if ( $reply =~ /n/i ) { print " ... skipping.\n"; } else { - my $status = 1; - while ( $status == 1 ) { - set_root_password(); - $status = $?; - } + set_root_password(); } print "\n"; @@ -334,8 +373,6 @@ if ( $reply =~ /n/i ) { } print "\n"; -cleanup(); - print <<HERE; === modified file 'scripts/mysql_secure_installation.sh' --- a/scripts/mysql_secure_installation.sh 2009-10-23 16:48:54 +0000 +++ b/scripts/mysql_secure_installation.sh 2010-01-15 15:27:55 +0000 @@ -189,16 +189,39 @@ prepare() { } do_query() { - echo $1 >$command + echo "$1" >$command + #sed 's,^,> ,' < $command # Debugging $bindir/mysql --defaults-file=$config <$command return $? } +# Simple escape mechanism (\-escape any ' and \), suitable for two contexts: +# - single-quoted SQL strings +# - single-quoted option values on the right hand side of = in my.cnf +# +# These two contexts don't handle escapes identically. SQL strings allow +# quoting any character (\C => C, for any C), but my.cnf parsing allows +# quoting only \, ' or ". For example, password='a\b' quotes a 3-character +# string in my.cnf, but a 2-character string in SQL. +# +# This simple escape works correctly in both places. +basic_single_escape () { + # The quoting on this sed command is a bit complex. Single-quoted strings + # don't allow *any* escape mechanism, so they cannot contain a single + # quote. The string sed gets (as argv[1]) is: s/$['\]$/\\\1/g + # + # Inside a character class, \ and ' are not special, so the ['\] character + # class is balanced and contains two characters. + echo "$1" | sed 's/$['"'"'\]$/\\\1/g' +} + make_config() { echo "# mysql_secure_installation config file" >$config echo "[mysql]" >>$config echo "user=root" >>$config - echo "password=$rootpass" >>$config + esc_pass=`basic_single_escape "$rootpass"` + echo "password='$esc_pass'" >>$config + #sed 's,^,> ,' < $config # Debugging } get_root_password() { @@ -245,13 +268,12 @@ set_root_password() { return 1 fi - do_query "UPDATE mysql.user SET Password=PASSWORD('$password1') WHERE User='root';" + esc_pass=`basic_single_escape "$password1"` + do_query "UPDATE mysql.user SET Password=PASSWORD('$esc_pass') WHERE User='root';" if [ $? -eq 0 ]; then echo "Password updated successfully!" echo "Reloading privilege tables.." - if ! reload_privilege_tables; then - exit 1 - fi + reload_privilege_tables || exit 1 echo rootpass=$password1 make_config === modified file 'sql-common/client.c' --- a/sql-common/client.c 2009-12-03 15:26:54 +0000 +++ b/sql-common/client.c 2010-01-29 18:42:22 +0000 @@ -1863,6 +1863,7 @@ CLI_MYSQL_REAL_CONNECT(MYSQL *mysql,cons uint port, const char *unix_socket,ulong client_flag) { char buff[NAME_LEN+USERNAME_LENGTH+100]; + char error_string[1024]; char *end,*host_info= NULL; my_socket sock; in_addr_t ip_addr; @@ -1939,7 +1940,8 @@ CLI_MYSQL_REAL_CONNECT(MYSQL *mysql,cons #if defined(HAVE_SMEM) if ((!mysql->options.protocol || mysql->options.protocol == MYSQL_PROTOCOL_MEMORY) && - (!host || !strcmp(host,LOCAL_HOST))) + (!host || !strcmp(host,LOCAL_HOST)) && + mysql->options.shared_memory_base_name) { if ((create_shared_memory(mysql,net, mysql->options.connect_timeout)) == INVALID_HANDLE_VALUE) @@ -1948,7 +1950,7 @@ CLI_MYSQL_REAL_CONNECT(MYSQL *mysql,cons ("host: '%s' socket: '%s' shared memory: %s have_tcpip: %d", host ? host : "<null>", unix_socket ? unix_socket : "<null>", - (int) mysql->options.shared_memory_base_name, + mysql->options.shared_memory_base_name, (int) have_tcpip)); if (mysql->options.protocol == MYSQL_PROTOCOL_MEMORY) goto error; @@ -2331,9 +2333,14 @@ CLI_MYSQL_REAL_CONNECT(MYSQL *mysql,cons /* Connect to the server */ DBUG_PRINT("info", ("IO layer change in progress...")); if (sslconnect(ssl_fd, mysql->net.vio, - (long) (mysql->options.connect_timeout))) + (long) (mysql->options.connect_timeout), + error_string)) { - set_mysql_error(mysql, CR_SSL_CONNECTION_ERROR, unknown_sqlstate); + set_mysql_extended_error(mysql, CR_SSL_CONNECTION_ERROR, + unknown_sqlstate, + "SSL error: %s", + error_string[0] ? error_string : + ER(CR_SSL_CONNECTION_ERROR)); goto error; } DBUG_PRINT("info", ("IO layer change done!")); @@ -2746,6 +2753,13 @@ void mysql_detach_stmt_list(LIST **stmt_ } +/* + Close a MySQL connection and free all resources attached to it. + + This function is coded in such that it can be called multiple times + (As some clients call this after mysql_real_connect() fails) +*/ + void STDCALL mysql_close(MYSQL *mysql) { DBUG_ENTER("mysql_close"); @@ -2779,10 +2793,16 @@ void STDCALL mysql_close(MYSQL *mysql) } #endif if (mysql != mysql->master) + { mysql_close(mysql->master); + mysql->master= 0; + } #ifndef MYSQL_SERVER if (mysql->thd) + { (*mysql->methods->free_embedded_thd)(mysql); + mysql->thd= 0; + } #endif if (mysql->free_me) my_free((uchar*) mysql,MYF(0)); === modified file 'sql/CMakeLists.txt' --- a/sql/CMakeLists.txt 2009-12-03 11:19:05 +0000 +++ b/sql/CMakeLists.txt 2010-01-29 18:42:22 +0000 @@ -97,7 +97,7 @@ SET_TARGET_PROPERTIES(mysqld PROPERTIES SET (MYSQLD_CORE_LIBS mysys zlib dbug strings yassl taocrypt vio regex sql libevent) TARGET_LINK_LIBRARIES(mysqld ${MYSQLD_CORE_LIBS} ${MYSQLD_STATIC_ENGINE_LIBS}) -TARGET_LINK_LIBRARIES(mysqld ws2_32.lib) +TARGET_LINK_LIBRARIES(mysqld ws2_32.lib psapi.lib) IF(MSVC AND NOT WITHOUT_DYNAMIC_PLUGINS) === modified file 'sql/event_db_repository.cc' --- a/sql/event_db_repository.cc 2009-12-03 11:19:05 +0000 +++ b/sql/event_db_repository.cc 2010-01-15 15:27:55 +0000 @@ -26,7 +26,7 @@ */ static -const TABLE_FIELD_W_TYPE event_table_fields[ET_FIELD_COUNT] = +const TABLE_FIELD_TYPE event_table_fields[ET_FIELD_COUNT] = { { { C_STRING_WITH_LEN("db") }, @@ -151,6 +151,24 @@ const TABLE_FIELD_W_TYPE event_table_fie } }; +static const TABLE_FIELD_DEF + event_table_def= {ET_FIELD_COUNT, event_table_fields}; + +class Event_db_intact : public Table_check_intact +{ +protected: + void report_error(uint, const char *fmt, ...) + { + va_list args; + va_start(args, fmt); + error_log_print(ERROR_LEVEL, fmt, args); + va_end(args); + } +}; + +/** In case of an error, a message is printed to the error log. */ +static Event_db_intact table_intact; + /** Puts some data common to CREATE and ALTER EVENT into a row. @@ -1117,10 +1135,8 @@ Event_db_repository::check_system_tables } else { - if (table_check_intact(tables.table, MYSQL_DB_FIELD_COUNT, - mysql_db_table_fields)) + if (table_intact.check(tables.table, &mysql_db_table_def)) ret= 1; - /* in case of an error, the message is printed inside table_check_intact */ close_thread_tables(thd); } @@ -1154,9 +1170,8 @@ Event_db_repository::check_system_tables } else { - if (table_check_intact(tables.table, ET_FIELD_COUNT, event_table_fields)) + if (table_intact.check(tables.table, &event_table_def)) ret= 1; - /* in case of an error, the message is printed inside table_check_intact */ close_thread_tables(thd); } === modified file 'sql/field.cc' --- a/sql/field.cc 2009-12-03 11:19:05 +0000 +++ b/sql/field.cc 2010-01-15 15:27:55 +0000 @@ -2487,6 +2487,50 @@ Field_new_decimal::Field_new_decimal(uin } +Field *Field_new_decimal::create_from_item (Item *item) +{ + uint8 dec= item->decimals; + uint8 intg= item->decimal_precision() - dec; + uint32 len= item->max_length; + + DBUG_ASSERT (item->result_type() == DECIMAL_RESULT); + + /* + Trying to put too many digits overall in a DECIMAL(prec,dec) + will always throw a warning. We must limit dec to + DECIMAL_MAX_SCALE however to prevent an assert() later. + */ + + if (dec > 0) + { + signed int overflow; + + dec= min(dec, DECIMAL_MAX_SCALE); + + /* + If the value still overflows the field with the corrected dec, + we'll throw out decimals rather than integers. This is still + bad and of course throws a truncation warning. + +1: for decimal point + */ + + const int required_length= + my_decimal_precision_to_length(intg + dec, dec, + item->unsigned_flag); + + overflow= required_length - len; + + if (overflow > 0) + dec= max(0, dec - overflow); // too long, discard fract + else + /* Corrected value fits. */ + len= required_length; + } + return new Field_new_decimal(len, item->maybe_null, item->name, + dec, item->unsigned_flag); +} + + int Field_new_decimal::reset(void) { store_value(&decimal_zero); === modified file 'sql/field.h' --- a/sql/field.h 2009-12-03 11:19:05 +0000 +++ b/sql/field.h 2010-01-15 15:27:55 +0000 @@ -807,6 +807,7 @@ public: uint is_equal(Create_field *new_field); virtual const uchar *unpack(uchar* to, const uchar *from, uint param_data, bool low_byte_first); + static Field *create_from_item (Item *); }; === modified file 'sql/handler.h' --- a/sql/handler.h 2010-01-04 13:12:53 +0000 +++ b/sql/handler.h 2010-01-14 16:51:00 +0000 @@ -891,9 +891,9 @@ typedef struct { ulonglong delete_length; ha_rows records; ulong mean_rec_length; - ulong create_time; - ulong check_time; - ulong update_time; + time_t create_time; + time_t check_time; + time_t update_time; ulonglong check_sum; } PARTITION_INFO; @@ -1060,9 +1060,9 @@ public: ha_rows records; ha_rows deleted; /* Deleted records */ ulong mean_rec_length; /* physical reclength */ - ulong create_time; /* When table was created */ - ulong check_time; - ulong update_time; + time_t create_time; /* When table was created */ + time_t check_time; + time_t update_time; uint block_size; /* index block size */ ha_statistics(): === modified file 'sql/item.cc' --- a/sql/item.cc 2009-12-03 11:19:05 +0000 +++ b/sql/item.cc 2010-01-27 21:53:08 +0000 @@ -4908,9 +4908,7 @@ Field *Item::tmp_table_field_from_field_ switch (field_type()) { case MYSQL_TYPE_DECIMAL: case MYSQL_TYPE_NEWDECIMAL: - field= new Field_new_decimal((uchar*) 0, max_length, null_ptr, 0, - Field::NONE, name, decimals, 0, - unsigned_flag); + field= Field_new_decimal::create_from_item(this); break; case MYSQL_TYPE_TINY: field= new Field_tiny((uchar*) 0, max_length, null_ptr, 0, Field::NONE, @@ -6949,9 +6947,24 @@ int stored_field_cmp_to_item(THD *thd, F Item_cache* Item_cache::get_cache(const Item *item) { - switch (item->result_type()) { + return get_cache(item, item->result_type()); +} + + +/** + Get a cache item of given type. + + @param item value to be cached + @param type required type of cache + + @return cache item +*/ + +Item_cache* Item_cache::get_cache(const Item *item, const Item_result type) +{ + switch (type) { case INT_RESULT: - return new Item_cache_int(); + return new Item_cache_int(item->field_type()); case REAL_RESULT: return new Item_cache_real(); case DECIMAL_RESULT: @@ -6967,6 +6980,13 @@ Item_cache* Item_cache::get_cache(const } } +void Item_cache::store(Item *item) +{ + example= item; + if (!item) + null_value= TRUE; + value_cached= FALSE; +} void Item_cache::print(String *str, enum_query_type query_type) { @@ -6978,17 +6998,22 @@ void Item_cache::print(String *str, enum str->append(')'); } - -void Item_cache_int::store(Item *item) +bool Item_cache_int::cache_value() { - value= item->val_int_result(); - null_value= item->null_value; - unsigned_flag= item->unsigned_flag; + if (!example) + return FALSE; + value_cached= TRUE; + value= example->val_int_result(); + null_value= example->null_value; + unsigned_flag= example->unsigned_flag; + return TRUE; } void Item_cache_int::store(Item *item, longlong val_arg) { + /* An explicit values is given, save it. */ + value_cached= TRUE; value= val_arg; null_value= item->null_value; unsigned_flag= item->unsigned_flag; @@ -6998,6 +7023,8 @@ void Item_cache_int::store(Item *item, l String *Item_cache_int::val_str(String *str) { DBUG_ASSERT(fixed == 1); + if (!value_cached && !cache_value()) + return NULL; str->set(value, default_charset()); return str; } @@ -7006,21 +7033,52 @@ String *Item_cache_int::val_str(String * my_decimal *Item_cache_int::val_decimal(my_decimal *decimal_val) { DBUG_ASSERT(fixed == 1); + if (!value_cached && !cache_value()) + return NULL; int2my_decimal(E_DEC_FATAL_ERROR, value, unsigned_flag, decimal_val); return decimal_val; } +double Item_cache_int::val_real() +{ + DBUG_ASSERT(fixed == 1); + if (!value_cached && !cache_value()) + return 0.0; + return (double) value; +} -void Item_cache_real::store(Item *item) +longlong Item_cache_int::val_int() { - value= item->val_result(); - null_value= item->null_value; + DBUG_ASSERT(fixed == 1); + if (!value_cached && !cache_value()) + return 0; + return value; } +bool Item_cache_real::cache_value() +{ + if (!example) + return FALSE; + value_cached= TRUE; + value= example->val_result(); + null_value= example->null_value; + return TRUE; +} + + +double Item_cache_real::val_real() +{ + DBUG_ASSERT(fixed == 1); + if (!value_cached && !cache_value()) + return 0.0; + return value; +} longlong Item_cache_real::val_int() { DBUG_ASSERT(fixed == 1); + if (!value_cached && !cache_value()) + return 0; return (longlong) rint(value); } @@ -7028,6 +7086,8 @@ longlong Item_cache_real::val_int() String* Item_cache_real::val_str(String *str) { DBUG_ASSERT(fixed == 1); + if (!value_cached && !cache_value()) + return NULL; str->set_real(value, decimals, default_charset()); return str; } @@ -7036,22 +7096,30 @@ String* Item_cache_real::val_str(String my_decimal *Item_cache_real::val_decimal(my_decimal *decimal_val) { DBUG_ASSERT(fixed == 1); + if (!value_cached && !cache_value()) + return NULL; double2my_decimal(E_DEC_FATAL_ERROR, value, decimal_val); return decimal_val; } -void Item_cache_decimal::store(Item *item) +bool Item_cache_decimal::cache_value() { - my_decimal *val= item->val_decimal_result(&decimal_value); - if (!(null_value= item->null_value) && val != &decimal_value) + if (!example) + return FALSE; + value_cached= TRUE; + my_decimal *val= example->val_decimal_result(&decimal_value); + if (!(null_value= example->null_value) && val != &decimal_value) my_decimal2decimal(val, &decimal_value); + return TRUE; } double Item_cache_decimal::val_real() { DBUG_ASSERT(fixed); double res; + if (!value_cached && !cache_value()) + return 0.0; my_decimal2double(E_DEC_FATAL_ERROR, &decimal_value, &res); return res; } @@ -7060,6 +7128,8 @@ longlong Item_cache_decimal::val_int() { DBUG_ASSERT(fixed); longlong res; + if (!value_cached && !cache_value()) + return 0; my_decimal2int(E_DEC_FATAL_ERROR, &decimal_value, unsigned_flag, &res); return res; } @@ -7067,6 +7137,8 @@ longlong Item_cache_decimal::val_int() String* Item_cache_decimal::val_str(String *str) { DBUG_ASSERT(fixed); + if (!value_cached && !cache_value()) + return NULL; my_decimal_round(E_DEC_FATAL_ERROR, &decimal_value, decimals, FALSE, &decimal_value); my_decimal2string(E_DEC_FATAL_ERROR, &decimal_value, 0, 0, 0, str); @@ -7076,15 +7148,20 @@ String* Item_cache_decimal::val_str(Stri my_decimal *Item_cache_decimal::val_decimal(my_decimal *val) { DBUG_ASSERT(fixed); + if (!value_cached && !cache_value()) + return NULL; return &decimal_value; } -void Item_cache_str::store(Item *item) +bool Item_cache_str::cache_value() { - value_buff.set(buffer, sizeof(buffer), item->collation.collation); - value= item->str_result(&value_buff); - if ((null_value= item->null_value)) + if (!example) + return FALSE; + value_cached= TRUE; + value_buff.set(buffer, sizeof(buffer), example->collation.collation); + value= example->str_result(&value_buff); + if ((null_value= example->null_value)) value= 0; else if (value != &value_buff) { @@ -7099,6 +7176,7 @@ void Item_cache_str::store(Item *item) value_buff.copy(*value); value= &value_buff; } + return TRUE; } double Item_cache_str::val_real() @@ -7106,6 +7184,8 @@ double Item_cache_str::val_real() DBUG_ASSERT(fixed == 1); int err_not_used; char *end_not_used; + if (!value_cached && !cache_value()) + return 0.0; if (value) return my_strntod(value->charset(), (char*) value->ptr(), value->length(), &end_not_used, &err_not_used); @@ -7117,6 +7197,8 @@ longlong Item_cache_str::val_int() { DBUG_ASSERT(fixed == 1); int err; + if (!value_cached && !cache_value()) + return 0; if (value) return my_strntoll(value->charset(), value->ptr(), value->length(), 10, (char**) 0, &err); @@ -7124,9 +7206,21 @@ longlong Item_cache_str::val_int() return (longlong)0; } + +String* Item_cache_str::val_str(String *str) +{ + DBUG_ASSERT(fixed == 1); + if (!value_cached && !cache_value()) + return 0; + return value; +} + + my_decimal *Item_cache_str::val_decimal(my_decimal *decimal_val) { DBUG_ASSERT(fixed == 1); + if (!value_cached && !cache_value()) + return NULL; if (value) string2my_decimal(E_DEC_FATAL_ERROR, value, decimal_val); else @@ -7137,6 +7231,8 @@ my_decimal *Item_cache_str::val_decimal( int Item_cache_str::save_in_field(Field *field, bool no_conversions) { + if (!value_cached && !cache_value()) + return 0; int res= Item_cache::save_in_field(field, no_conversions); return (is_varbinary && field->type() == MYSQL_TYPE_STRING && value->length() < field->field_length) ? 1 : res; @@ -7171,13 +7267,30 @@ bool Item_cache_row::setup(Item * item) void Item_cache_row::store(Item * item) { + example= item; + if (!item) + { + null_value= TRUE; + return; + } + for (uint i= 0; i < item_count; i++) + values[i]->store(item->element_index(i)); +} + + +bool Item_cache_row::cache_value() +{ + if (!example) + return FALSE; + value_cached= TRUE; null_value= 0; - item->bring_value(); + example->bring_value(); for (uint i= 0; i < item_count; i++) { - values[i]->store(item->element_index(i)); + values[i]->cache_value(); null_value|= values[i]->null_value; } + return TRUE; } === modified file 'sql/item.h' --- a/sql/item.h 2009-12-03 11:19:05 +0000 +++ b/sql/item.h 2010-01-15 15:27:55 +0000 @@ -1053,7 +1053,11 @@ class sp_head; class Item_basic_constant :public Item { + table_map used_table_map; public: + Item_basic_constant(): Item(), used_table_map(0) {}; + void set_used_tables(table_map map) { used_table_map= map; } + table_map used_tables() const { return used_table_map; } /* to prevent drop fixed flag (no need parent cleanup call) */ void cleanup() { @@ -1065,7 +1069,6 @@ public: if (orig_name) name= orig_name; } - Item_basic_constant() {} /* Remove gcc warning */ }; @@ -2165,6 +2168,23 @@ public: save_in_field(result_field, no_conversions); } void cleanup(); + /* + This method is used for debug purposes to print the name of an + item to the debug log. The second use of this method is as + a helper function of print() and error messages, where it is + applicable. To suit both goals it should return a meaningful, + distinguishable and sintactically correct string. This method + should not be used for runtime type identification, use enum + {Sum}Functype and Item_func::functype()/Item_sum::sum_func() + instead. + Added here, to the parent class of both Item_func and Item_sum_func. + + NOTE: for Items inherited from Item_sum, func_name() return part of + function name till first argument (including '(') to make difference in + names for functions with 'distinct' clause and without 'distinct' and + also to make printing of items inherited from Item_sum uniform. + */ + virtual const char *func_name() const= 0; }; @@ -2924,15 +2944,25 @@ protected: */ Field *cached_field; enum enum_field_types cached_field_type; -public: - Item_cache(): - example(0), used_table_map(0), cached_field(0), cached_field_type(MYSQL_TYPE_STRING) + /* + TRUE <=> cache holds value of the last stored item (i.e actual value). + store() stores item to be cached and sets this flag to FALSE. + On the first call of val_xxx function if this flag is set to FALSE the + cache_value() will be called to actually cache value of saved item. + cache_value() will set this flag to TRUE. + */ + bool value_cached; +public: + Item_cache(): + example(0), used_table_map(0), cached_field(0), cached_field_type(MYSQL_TYPE_STRING), + value_cached(0) { fixed= 1; null_value= 1; } Item_cache(enum_field_types field_type_arg): - example(0), used_table_map(0), cached_field(0), cached_field_type(field_type_arg) + example(0), used_table_map(0), cached_field(0), cached_field_type(field_type_arg), + value_cached(0) { fixed= 1; null_value= 1; @@ -2952,10 +2982,10 @@ public: cached_field= ((Item_field *)item)->field; return 0; }; - virtual void store(Item *)= 0; enum Type type() const { return CACHE_ITEM; } enum_field_types field_type() const { return cached_field_type; } static Item_cache* get_cache(const Item *item); + static Item_cache* get_cache(const Item* item, const Item_result type); table_map used_tables() const { return used_table_map; } virtual void keep_array() {} virtual void print(String *str, enum_query_type query_type); @@ -2967,6 +2997,8 @@ public: { return this == item; } + virtual void store(Item *item); + virtual bool cache_value()= 0; }; @@ -2975,18 +3007,19 @@ class Item_cache_int: public Item_cache protected: longlong value; public: - Item_cache_int(): Item_cache(), value(0) {} + Item_cache_int(): Item_cache(), + value(0) {} Item_cache_int(enum_field_types field_type_arg): Item_cache(field_type_arg), value(0) {} - void store(Item *item); void store(Item *item, longlong val_arg); - double val_real() { DBUG_ASSERT(fixed == 1); return (double) value; } - longlong val_int() { DBUG_ASSERT(fixed == 1); return value; } + double val_real(); + longlong val_int(); String* val_str(String *str); my_decimal *val_decimal(my_decimal *); enum Item_result result_type() const { return INT_RESULT; } bool result_as_longlong() { return TRUE; } + bool cache_value(); }; @@ -2994,14 +3027,15 @@ class Item_cache_real: public Item_cache { double value; public: - Item_cache_real(): Item_cache(), value(0) {} + Item_cache_real(): Item_cache(), + value(0) {} - void store(Item *item); - double val_real() { DBUG_ASSERT(fixed == 1); return value; } + double val_real(); longlong val_int(); String* val_str(String *str); my_decimal *val_decimal(my_decimal *); enum Item_result result_type() const { return REAL_RESULT; } + bool cache_value(); }; @@ -3012,12 +3046,12 @@ protected: public: Item_cache_decimal(): Item_cache() {} - void store(Item *item); double val_real(); longlong val_int(); String* val_str(String *str); my_decimal *val_decimal(my_decimal *); enum Item_result result_type() const { return DECIMAL_RESULT; } + bool cache_value(); }; @@ -3035,14 +3069,14 @@ public: MYSQL_TYPE_VARCHAR && !((const Item_field *) item)->field->has_charset()) {} - void store(Item *item); double val_real(); longlong val_int(); - String* val_str(String *) { DBUG_ASSERT(fixed == 1); return value; } + String* val_str(String *); my_decimal *val_decimal(my_decimal *); enum Item_result result_type() const { return STRING_RESULT; } CHARSET_INFO *charset() const { return value->charset(); }; int save_in_field(Field *field, bool no_conversions); + bool cache_value(); }; class Item_cache_row: public Item_cache @@ -3052,7 +3086,8 @@ class Item_cache_row: public Item_cache bool save_array; public: Item_cache_row() - :Item_cache(), values(0), item_count(2), save_array(0) {} + :Item_cache(), values(0), item_count(2), + save_array(0) {} /* 'allocate' used only in row transformer, to preallocate space for row @@ -3110,6 +3145,7 @@ public: values= 0; DBUG_VOID_RETURN; } + bool cache_value(); }; === modified file 'sql/item_cmpfunc.cc' --- a/sql/item_cmpfunc.cc 2009-12-03 11:19:05 +0000 +++ b/sql/item_cmpfunc.cc 2010-01-15 15:27:55 +0000 @@ -30,6 +30,9 @@ #include "sql_select.h" static bool convert_constant_item(THD *, Item_field *, Item **); +static longlong +get_year_value(THD *thd, Item ***item_arg, Item **cache_arg, + Item *warn_item, bool *is_null); static Item_result item_store_type(Item_result a, Item *item, my_bool unsigned_flag) @@ -533,11 +536,12 @@ void Item_bool_func2::fix_length_and_dec } -int Arg_comparator::set_compare_func(Item_bool_func2 *item, Item_result type) +int Arg_comparator::set_compare_func(Item_result_field *item, Item_result type) { owner= item; func= comparator_matrix[type] - [test(owner->functype() == Item_func::EQUAL_FUNC)]; + [is_owner_equal_func()]; + switch (type) { case ROW_RESULT: { @@ -557,7 +561,8 @@ int Arg_comparator::set_compare_func(Ite my_error(ER_OPERAND_COLUMNS, MYF(0), (*a)->element_index(i)->cols()); return 1; } - if (comparators[i].set_cmp_func(owner, (*a)->addr(i), (*b)->addr(i))) + if (comparators[i].set_cmp_func(owner, (*a)->addr(i), (*b)->addr(i), + set_null)) return 1; } break; @@ -571,7 +576,8 @@ int Arg_comparator::set_compare_func(Ite if (cmp_collation.set((*a)->collation, (*b)->collation) || cmp_collation.derivation == DERIVATION_NONE) { - my_coll_agg_error((*a)->collation, (*b)->collation, owner->func_name()); + my_coll_agg_error((*a)->collation, (*b)->collation, + owner->func_name()); return 1; } if (cmp_collation.collation == &my_charset_bin) @@ -785,15 +791,21 @@ Arg_comparator::can_compare_as_dates(Ite if (cmp_type != CMP_DATE_DFLT) { + THD *thd= current_thd; /* Do not cache GET_USER_VAR() function as its const_item() may return TRUE for the current thread but it still may change during the execution. + Don't use cache while in the context analysis mode only (i.e. for + EXPLAIN/CREATE VIEW and similar queries). Cache is useless in such + cases and can cause problems. For example evaluating subqueries can + confuse storage engines since in context analysis mode tables + aren't locked. */ - if (cmp_type != CMP_DATE_WITH_DATE && str_arg->const_item() && + if (!thd->is_context_analysis_only() && + cmp_type != CMP_DATE_WITH_DATE && str_arg->const_item() && (str_arg->type() != Item::FUNC_ITEM || ((Item_func*)str_arg)->functype() != Item_func::GUSERVAR_FUNC)) { - THD *thd= current_thd; ulonglong value; bool error; String tmp, *str_val= 0; @@ -875,18 +887,20 @@ get_time_value(THD *thd, Item ***item_ar } -int Arg_comparator::set_cmp_func(Item_bool_func2 *owner_arg, +int Arg_comparator::set_cmp_func(Item_result_field *owner_arg, Item **a1, Item **a2, Item_result type) { ulonglong const_value= (ulonglong)-1; + thd= current_thd; + owner= owner_arg; + set_null= set_null && owner_arg; a= a1; b= a2; + thd= current_thd; if (can_compare_as_dates(*a, *b, &const_value)) { - thd= current_thd; - owner= owner_arg; a_type= (*a)->field_type(); b_type= (*b)->field_type(); a_cache= 0; @@ -894,6 +908,10 @@ int Arg_comparator::set_cmp_func(Item_bo if (const_value != (ulonglong)-1) { + /* + cache_converted_constant can't be used here because it can't + correctly convert a DATETIME value from string to int representation. + */ Item_cache_int *cache= new Item_cache_int(); /* Mark the cache as non-const to prevent re-caching. */ cache->set_used_tables(1); @@ -910,22 +928,22 @@ int Arg_comparator::set_cmp_func(Item_bo b= (Item **)&b_cache; } } - is_nulls_eq= test(owner && owner->functype() == Item_func::EQUAL_FUNC); + is_nulls_eq= is_owner_equal_func(); func= &Arg_comparator::compare_datetime; - get_value_func= &get_datetime_value; + get_value_a_func= &get_datetime_value; + get_value_b_func= &get_datetime_value; return 0; } else if (type == STRING_RESULT && (*a)->field_type() == MYSQL_TYPE_TIME && (*b)->field_type() == MYSQL_TYPE_TIME) { /* Compare TIME values as integers. */ - thd= current_thd; - owner= owner_arg; a_cache= 0; b_cache= 0; - is_nulls_eq= test(owner && owner->functype() == Item_func::EQUAL_FUNC); + is_nulls_eq= is_owner_equal_func(); func= &Arg_comparator::compare_datetime; - get_value_func= &get_time_value; + get_value_a_func= &get_time_value; + get_value_b_func= &get_time_value; return 0; } else if (type == STRING_RESULT && @@ -934,20 +952,97 @@ int Arg_comparator::set_cmp_func(Item_bo { DTCollation coll; coll.set((*a)->collation.collation); - if (agg_item_set_converter(coll, owner_arg->func_name(), + if (agg_item_set_converter(coll, owner->func_name(), b, 1, MY_COLL_CMP_CONV, 1)) return 1; } + else if (try_year_cmp_func(type)) + return 0; + a= cache_converted_constant(thd, a, &a_cache, type); + b= cache_converted_constant(thd, b, &b_cache, type); return set_compare_func(owner_arg, type); } -void Arg_comparator::set_datetime_cmp_func(Item **a1, Item **b1) +/* + Helper function to call from Arg_comparator::set_cmp_func() +*/ + +bool Arg_comparator::try_year_cmp_func(Item_result type) +{ + if (type == ROW_RESULT) + return FALSE; + + bool a_is_year= (*a)->field_type() == MYSQL_TYPE_YEAR; + bool b_is_year= (*b)->field_type() == MYSQL_TYPE_YEAR; + + if (!a_is_year && !b_is_year) + return FALSE; + + if (a_is_year && b_is_year) + { + get_value_a_func= &get_year_value; + get_value_b_func= &get_year_value; + } + else if (a_is_year && (*b)->is_datetime()) + { + get_value_a_func= &get_year_value; + get_value_b_func= &get_datetime_value; + } + else if (b_is_year && (*a)->is_datetime()) + { + get_value_b_func= &get_year_value; + get_value_a_func= &get_datetime_value; + } + else + return FALSE; + + is_nulls_eq= is_owner_equal_func(); + func= &Arg_comparator::compare_datetime; + + return TRUE; +} + +/** + Convert and cache a constant. + + @param value [in] An item to cache + @param cache_item [out] Placeholder for the cache item + @param type [in] Comparison type + + @details + When given item is a constant and its type differs from comparison type + then cache its value to avoid type conversion of this constant on each + evaluation. In this case the value is cached and the reference to the cache + is returned. + Original value is returned otherwise. + + @return cache item or original value. +*/ + +Item** Arg_comparator::cache_converted_constant(THD *thd, Item **value, + Item **cache_item, + Item_result type) +{ + /* Don't need cache if doing context analysis only. */ + if (!thd->is_context_analysis_only() && + (*value)->const_item() && type != (*value)->result_type()) + { + Item_cache *cache= Item_cache::get_cache(*value, type); + cache->setup(*value); + *cache_item= cache; + return cache_item; + } + return value; +} + + +void Arg_comparator::set_datetime_cmp_func(Item_result_field *owner_arg, + Item **a1, Item **b1) { thd= current_thd; - /* A caller will handle null values by itself. */ - owner= NULL; + owner= owner_arg; a= a1; b= b1; a_type= (*a)->field_type(); @@ -956,7 +1051,8 @@ void Arg_comparator::set_datetime_cmp_fu b_cache= 0; is_nulls_eq= FALSE; func= &Arg_comparator::compare_datetime; - get_value_func= &get_datetime_value; + get_value_a_func= &get_datetime_value; + get_value_b_func= &get_datetime_value; } @@ -1056,6 +1152,56 @@ get_datetime_value(THD *thd, Item ***ite return value; } + +/* + Retrieves YEAR value of 19XX-00-00 00:00:00 form from given item. + + SYNOPSIS + get_year_value() + thd thread handle + item_arg [in/out] item to retrieve YEAR value from + cache_arg [in/out] pointer to place to store the caching item to + warn_item [in] item for issuing the conversion warning + is_null [out] TRUE <=> the item_arg is null + + DESCRIPTION + Retrieves the YEAR value of 19XX form from given item for comparison by the + compare_datetime() function. + Converts year to DATETIME of form YYYY-00-00 00:00:00 for the compatibility + with the get_datetime_value function result. + + RETURN + obtained value +*/ + +static longlong +get_year_value(THD *thd, Item ***item_arg, Item **cache_arg, + Item *warn_item, bool *is_null) +{ + longlong value= 0; + Item *item= **item_arg; + + value= item->val_int(); + *is_null= item->null_value; + if (*is_null) + return ~(ulonglong) 0; + + /* + Coerce value to the 19XX form in order to correctly compare + YEAR(2) & YEAR(4) types. + */ + if (value < 70) + value+= 100; + if (value <= 1900) + value+= 1900; + + /* Convert year to DATETIME of form YYYY-00-00 00:00:00 (YYYY0000000000). */ + value*= 10000000000LL; + + return value; +} + + /* Compare items values as dates. @@ -1088,25 +1234,25 @@ int Arg_comparator::compare_datetime() longlong a_value, b_value; /* Get DATE/DATETIME/TIME value of the 'a' item. */ - a_value= (*get_value_func)(thd, &a, &a_cache, *b, &a_is_null); + a_value= (*get_value_a_func)(thd, &a, &a_cache, *b, &a_is_null); if (!is_nulls_eq && a_is_null) { - if (owner) + if (set_null) owner->null_value= 1; return -1; } /* Get DATE/DATETIME/TIME value of the 'b' item. */ - b_value= (*get_value_func)(thd, &b, &b_cache, *a, &b_is_null); + b_value= (*get_value_b_func)(thd, &b, &b_cache, *a, &b_is_null); if (a_is_null || b_is_null) { - if (owner) + if (set_null) owner->null_value= is_nulls_eq ? 0 : 1; return is_nulls_eq ? (a_is_null == b_is_null) : -1; } /* Here we have two not-NULL values. */ - if (owner) + if (set_null) owner->null_value= 0; /* Compare values. */ @@ -1119,15 +1265,17 @@ int Arg_comparator::compare_datetime() int Arg_comparator::compare_string() { String *res1,*res2; - if ((res1= (*a)->val_str(&owner->tmp_value1))) + if ((res1= (*a)->val_str(&value1))) { - if ((res2= (*b)->val_str(&owner->tmp_value2))) + if ((res2= (*b)->val_str(&value2))) { - owner->null_value= 0; + if (set_null) + owner->null_value= 0; return sortcmp(res1,res2,cmp_collation.collation); } } - owner->null_value= 1; + if (set_null) + owner->null_value= 1; return -1; } @@ -1146,18 +1294,20 @@ int Arg_comparator::compare_string() int Arg_comparator::compare_binary_string() { String *res1,*res2; - if ((res1= (*a)->val_str(&owner->tmp_value1))) + if ((res1= (*a)->val_str(&value1))) { - if ((res2= (*b)->val_str(&owner->tmp_value2))) + if ((res2= (*b)->val_str(&value2))) { - owner->null_value= 0; + if (set_null) + owner->null_value= 0; uint res1_length= res1->length(); uint res2_length= res2->length(); int cmp= memcmp(res1->ptr(), res2->ptr(), min(res1_length,res2_length)); return cmp ? cmp : (int) (res1_length - res2_length); } } - owner->null_value= 1; + if (set_null) + owner->null_value= 1; return -1; } @@ -1170,8 +1320,8 @@ int Arg_comparator::compare_binary_strin int Arg_comparator::compare_e_string() { String *res1,*res2; - res1= (*a)->val_str(&owner->tmp_value1); - res2= (*b)->val_str(&owner->tmp_value2); + res1= (*a)->val_str(&value1); + res2= (*b)->val_str(&value2); if (!res1 || !res2) return test(res1 == res2); return test(sortcmp(res1, res2, cmp_collation.collation) == 0); @@ -1181,8 +1331,8 @@ int Arg_comparator::compare_e_string() int Arg_comparator::compare_e_binary_string() { String *res1,*res2; - res1= (*a)->val_str(&owner->tmp_value1); - res2= (*b)->val_str(&owner->tmp_value2); + res1= (*a)->val_str(&value1); + res2= (*b)->val_str(&value2); if (!res1 || !res2) return test(res1 == res2); return test(stringcmp(res1, res2) == 0); @@ -1203,13 +1353,15 @@ int Arg_comparator::compare_real() val2= (*b)->val_real(); if (!(*b)->null_value) { - owner->null_value= 0; + if (set_null) + owner->null_value= 0; if (val1 < val2) return -1; if (val1 == val2) return 0; return 1; } } - owner->null_value= 1; + if (set_null) + owner->null_value= 1; return -1; } @@ -1223,11 +1375,13 @@ int Arg_comparator::compare_decimal() my_decimal *val2= (*b)->val_decimal(&value2); if (!(*b)->null_value) { - owner->null_value= 0; + if (set_null) + owner->null_value= 0; return my_decimal_cmp(val1, val2); } } - owner->null_value= 1; + if (set_null) + owner->null_value= 1; return -1; } @@ -1265,7 +1419,8 @@ int Arg_comparator::compare_real_fixed() val2= (*b)->val_real(); if (!(*b)->null_value) { - owner->null_value= 0; + if (set_null) + owner->null_value= 0; if (val1 == val2 || fabs(val1 - val2) < precision) return 0; if (val1 < val2) @@ -1273,7 +1428,8 @@ int Arg_comparator::compare_real_fixed() return 1; } } - owner->null_value= 1; + if (set_null) + owner->null_value= 1; return -1; } @@ -1296,13 +1452,15 @@ int Arg_comparator::compare_int_signed() longlong val2= (*b)->val_int(); if (!(*b)->null_value) { - owner->null_value= 0; + if (set_null) + owner->null_value= 0; if (val1 < val2) return -1; if (val1 == val2) return 0; return 1; } } - owner->null_value= 1; + if (set_null) + owner->null_value= 1; return -1; } @@ -1319,13 +1477,15 @@ int Arg_comparator::compare_int_unsigned ulonglong val2= (*b)->val_int(); if (!(*b)->null_value) { - owner->null_value= 0; + if (set_null) + owner->null_value= 0; if (val1 < val2) return -1; if (val1 == val2) return 0; return 1; } } - owner->null_value= 1; + if (set_null) + owner->null_value= 1; return -1; } @@ -1342,7 +1502,8 @@ int Arg_comparator::compare_int_signed_u ulonglong uval2= (ulonglong)(*b)->val_int(); if (!(*b)->null_value) { - owner->null_value= 0; + if (set_null) + owner->null_value= 0; if (sval1 < 0 || (ulonglong)sval1 < uval2) return -1; if ((ulonglong)sval1 == uval2) @@ -1350,7 +1511,8 @@ int Arg_comparator::compare_int_signed_u return 1; } } - owner->null_value= 1; + if (set_null) + owner->null_value= 1; return -1; } @@ -1367,7 +1529,8 @@ int Arg_comparator::compare_int_unsigned longlong sval2= (*b)->val_int(); if (!(*b)->null_value) { - owner->null_value= 0; + if (set_null) + owner->null_value= 0; if (sval2 < 0) return 1; if (uval1 < (ulonglong)sval2) @@ -1377,7 +1540,8 @@ int Arg_comparator::compare_int_unsigned return 1; } } - owner->null_value= 1; + if (set_null) + owner->null_value= 1; return -1; } @@ -1413,10 +1577,11 @@ int Arg_comparator::compare_row() for (uint i= 0; i<n; i++) { res= comparators[i].compare(); - if (owner->null_value) + /* Aggregate functions don't need special null handling. */ + if (owner->null_value && owner->type() == Item::FUNC_ITEM) { // NULL was compared - switch (owner->functype()) { + switch (((Item_func*)owner)->functype()) { case Item_func::NE_FUNC: break; // NE never aborts on NULL even if abort_on_null is set case Item_func::LT_FUNC: @@ -1425,7 +1590,7 @@ int Arg_comparator::compare_row() case Item_func::GE_FUNC: return -1; // <, <=, > and >= always fail on NULL default: // EQ_FUNC - if (owner->abort_on_null) + if (((Item_bool_func2*)owner)->abort_on_null) return -1; // We do not need correct NULL returning } was_null= 1; @@ -1581,6 +1746,7 @@ longlong Item_in_optimizer::val_int() bool tmp; DBUG_ASSERT(fixed == 1); cache->store(args[0]); + cache->cache_value(); if (cache->null_value) { @@ -1748,8 +1914,8 @@ longlong Item_func_lt::val_int() longlong Item_func_strcmp::val_int() { DBUG_ASSERT(fixed == 1); - String *a=args[0]->val_str(&tmp_value1); - String *b=args[1]->val_str(&tmp_value2); + String *a=args[0]->val_str(&cmp.value1); + String *b=args[1]->val_str(&cmp.value2); if (!a || !b) { null_value=1; @@ -2032,8 +2198,8 @@ void Item_func_between::fix_length_and_d if (compare_as_dates) { - ge_cmp.set_datetime_cmp_func(args, args + 1); - le_cmp.set_datetime_cmp_func(args, args + 2); + ge_cmp.set_datetime_cmp_func(this, args, args + 1); + le_cmp.set_datetime_cmp_func(this, args, args + 2); } else if (time_items_found == 3) { @@ -4370,13 +4536,13 @@ void Item_func_isnotnull::print(String * longlong Item_func_like::val_int() { DBUG_ASSERT(fixed == 1); - String* res = args[0]->val_str(&tmp_value1); + String* res = args[0]->val_str(&cmp.value1); if (args[0]->null_value) { null_value=1; return 0; } - String* res2 = args[1]->val_str(&tmp_value2); + String* res2 = args[1]->val_str(&cmp.value2); if (args[1]->null_value) { null_value=1; @@ -4400,7 +4566,7 @@ Item_func::optimize_type Item_func_like: { if (args[1]->const_item()) { - String* res2= args[1]->val_str((String *)&tmp_value2); + String* res2= args[1]->val_str((String *)&cmp.value2); if (!res2) return OPTIMIZE_NONE; @@ -4431,7 +4597,7 @@ bool Item_func_like::fix_fields(THD *thd if (escape_item->const_item()) { /* If we are on execution stage */ - String *escape_str= escape_item->val_str(&tmp_value1); + String *escape_str= escape_item->val_str(&cmp.value1); if (escape_str) { if (escape_used_in_parsing && ( @@ -4486,7 +4652,7 @@ bool Item_func_like::fix_fields(THD *thd if (args[1]->const_item() && !use_strnxfrm(collation.collation) && !(specialflag & SPECIAL_NO_NEW_FUNC)) { - String* res2 = args[1]->val_str(&tmp_value2); + String* res2 = args[1]->val_str(&cmp.value2); if (!res2) return FALSE; // Null argument === modified file 'sql/item_cmpfunc.h' --- a/sql/item_cmpfunc.h 2009-12-03 11:19:05 +0000 +++ b/sql/item_cmpfunc.h 2010-01-15 15:27:55 +0000 @@ -32,7 +32,7 @@ class Arg_comparator: public Sql_alloc { Item **a, **b; arg_cmp_func func; - Item_bool_func2 *owner; + Item_result_field *owner; Arg_comparator *comparators; // used only for compare_row() double precision; /* Fields used in DATE/DATETIME comparison. */ @@ -40,30 +40,40 @@ class Arg_comparator: public Sql_alloc enum_field_types a_type, b_type; // Types of a and b items Item *a_cache, *b_cache; // Cached values of a and b items bool is_nulls_eq; // TRUE <=> compare for the EQUAL_FUNC + bool set_null; // TRUE <=> set owner->null_value + // when one of arguments is NULL. enum enum_date_cmp_type { CMP_DATE_DFLT= 0, CMP_DATE_WITH_DATE, CMP_DATE_WITH_STR, CMP_STR_WITH_DATE }; - longlong (*get_value_func)(THD *thd, Item ***item_arg, Item **cache_arg, - Item *warn_item, bool *is_null); + longlong (*get_value_a_func)(THD *thd, Item ***item_arg, Item **cache_arg, + Item *warn_item, bool *is_null); + longlong (*get_value_b_func)(THD *thd, Item ***item_arg, Item **cache_arg, + Item *warn_item, bool *is_null); + bool try_year_cmp_func(Item_result type); public: DTCollation cmp_collation; + /* Allow owner function to use string buffers. */ + String value1, value2; - Arg_comparator(): thd(0), a_cache(0), b_cache(0) {}; + Arg_comparator(): thd(0), a_cache(0), b_cache(0), set_null(TRUE), + get_value_a_func(0), get_value_b_func(0) {}; Arg_comparator(Item **a1, Item **a2): a(a1), b(a2), thd(0), - a_cache(0), b_cache(0) {}; + a_cache(0), b_cache(0), set_null(TRUE), + get_value_a_func(0), get_value_b_func(0) {}; - int set_compare_func(Item_bool_func2 *owner, Item_result type); - inline int set_compare_func(Item_bool_func2 *owner_arg) + int set_compare_func(Item_result_field *owner, Item_result type); + inline int set_compare_func(Item_result_field *owner_arg) { return set_compare_func(owner_arg, item_cmp_type((*a)->result_type(), (*b)->result_type())); } - int set_cmp_func(Item_bool_func2 *owner_arg, + int set_cmp_func(Item_result_field *owner_arg, Item **a1, Item **a2, Item_result type); - inline int set_cmp_func(Item_bool_func2 *owner_arg, - Item **a1, Item **a2) + inline int set_cmp_func(Item_result_field *owner_arg, + Item **a1, Item **a2, bool set_null_arg) { + set_null= set_null_arg; return set_cmp_func(owner_arg, a1, a2, item_cmp_type((*a1)->result_type(), (*a2)->result_type())); @@ -93,8 +103,15 @@ public: static enum enum_date_cmp_type can_compare_as_dates(Item *a, Item *b, ulonglong *const_val_arg); - void set_datetime_cmp_func(Item **a1, Item **b1); + Item** cache_converted_constant(THD *thd, Item **value, Item **cache, + Item_result type); + void set_datetime_cmp_func(Item_result_field *owner_arg, Item **a1, Item **b1); static arg_cmp_func comparator_matrix [5][2]; + inline bool is_owner_equal_func() + { + return (owner->type() == Item::FUNC_ITEM && + ((Item_func*)owner)->functype() == Item_func::EQUAL_FUNC); + } friend class Item_func; }; @@ -324,7 +341,6 @@ class Item_bool_func2 :public Item_int_f { /* Bool with 2 string args */ protected: Arg_comparator cmp; - String tmp_value1,tmp_value2; bool abort_on_null; public: @@ -333,7 +349,7 @@ public: void fix_length_and_dec(); void set_cmp_func() { - cmp.set_cmp_func(this, tmp_arg, tmp_arg+1); + cmp.set_cmp_func(this, tmp_arg, tmp_arg+1, TRUE); } optimize_type select_optimize() const { return OPTIMIZE_OP; } virtual enum Functype rev_functype() const { return UNKNOWN_FUNC; } === modified file 'sql/item_create.cc' --- a/sql/item_create.cc 2009-10-15 21:38:29 +0000 +++ b/sql/item_create.cc 2010-01-15 15:27:55 +0000 @@ -3524,6 +3524,7 @@ Create_func_get_lock Create_func_get_loc Item* Create_func_get_lock::create(THD *thd, Item *arg1, Item *arg2) { + thd->lex->set_stmt_unsafe(); thd->lex->uncacheable(UNCACHEABLE_SIDEEFFECT); return new (thd->mem_root) Item_func_get_lock(arg1, arg2); } @@ -3635,6 +3636,7 @@ Create_func_is_free_lock Create_func_is_ Item* Create_func_is_free_lock::create(THD *thd, Item *arg1) { + thd->lex->set_stmt_unsafe(); thd->lex->uncacheable(UNCACHEABLE_SIDEEFFECT); return new (thd->mem_root) Item_func_is_free_lock(arg1); } @@ -3645,6 +3647,7 @@ Create_func_is_used_lock Create_func_is_ Item* Create_func_is_used_lock::create(THD *thd, Item *arg1) { + thd->lex->set_stmt_unsafe(); thd->lex->uncacheable(UNCACHEABLE_SIDEEFFECT); return new (thd->mem_root) Item_func_is_used_lock(arg1); } @@ -3961,6 +3964,8 @@ Create_func_master_pos_wait::create_nati Item *func= NULL; int arg_count= 0; + thd->lex->set_stmt_unsafe(); + if (item_list != NULL) arg_count= item_list->elements; @@ -4203,6 +4208,7 @@ Create_func_release_lock Create_func_rel Item* Create_func_release_lock::create(THD *thd, Item *arg1) { + thd->lex->set_stmt_unsafe(); thd->lex->uncacheable(UNCACHEABLE_SIDEEFFECT); return new (thd->mem_root) Item_func_release_lock(arg1); } @@ -4325,6 +4331,7 @@ Create_func_sleep Create_func_sleep::s_s Item* Create_func_sleep::create(THD *thd, Item *arg1) { + thd->lex->set_stmt_unsafe(); thd->lex->uncacheable(UNCACHEABLE_SIDEEFFECT); return new (thd->mem_root) Item_func_sleep(arg1); } @@ -4591,6 +4598,7 @@ Create_func_version Create_func_version: Item* Create_func_version::create(THD *thd) { + thd->lex->set_stmt_unsafe(); return new (thd->mem_root) Item_static_string_func("version()", server_version, (uint) strlen(server_version), === modified file 'sql/item_func.cc' --- a/sql/item_func.cc 2009-12-03 11:19:05 +0000 +++ b/sql/item_func.cc 2010-01-15 15:27:55 +0000 @@ -450,45 +450,8 @@ Field *Item_func::tmp_table_field(TABLE case STRING_RESULT: return make_string_field(table); case DECIMAL_RESULT: - { - uint8 dec= decimals; - uint8 intg= decimal_precision() - dec; - uint32 len= max_length; - - /* - Trying to put too many digits overall in a DECIMAL(prec,dec) - will always throw a warning. We must limit dec to - DECIMAL_MAX_SCALE however to prevent an assert() later. - */ - - if (dec > 0) - { - int overflow; - - dec= min(dec, DECIMAL_MAX_SCALE); - - /* - If the value still overflows the field with the corrected dec, - we'll throw out decimals rather than integers. This is still - bad and of course throws a truncation warning. - */ - - const int required_length= - my_decimal_precision_to_length(intg + dec, dec, - unsigned_flag); - - overflow= required_length - len; - - if (overflow > 0) - dec= max(0, dec - overflow); // too long, discard fract - else - /* Corrected value fits. */ - len= required_length; - } - - field= new Field_new_decimal(len, maybe_null, name, dec, unsigned_flag); + field= Field_new_decimal::create_from_item(this); break; - } case ROW_RESULT: default: // This case should never be chosen === modified file 'sql/item_func.h' --- a/sql/item_func.h 2009-12-03 11:19:05 +0000 +++ b/sql/item_func.h 2010-01-15 15:27:55 +0000 @@ -124,17 +124,6 @@ public: virtual optimize_type select_optimize() const { return OPTIMIZE_NONE; } virtual bool have_rev_func() const { return 0; } virtual Item *key_item() const { return args[0]; } - /* - This method is used for debug purposes to print the name of an - item to the debug log. The second use of this method is as - a helper function of print(), where it is applicable. - To suit both goals it should return a meaningful, - distinguishable and sintactically correct string. This method - should not be used for runtime type identification, use enum - {Sum}Functype and Item_func::functype()/Item_sum::sum_func() - instead. - */ - virtual const char *func_name() const= 0; virtual bool const_item() const { return const_item_cache; } inline Item **arguments() const { return args; } void set_arguments(List<Item> &list); === modified file 'sql/item_geofunc.cc' --- a/sql/item_geofunc.cc 2009-10-24 06:57:31 +0000 +++ b/sql/item_geofunc.cc 2009-12-08 09:26:11 +0000 @@ -511,8 +511,8 @@ err: longlong Item_func_spatial_rel::val_int() { DBUG_ASSERT(fixed == 1); - String *res1= args[0]->val_str(&tmp_value1); - String *res2= args[1]->val_str(&tmp_value2); + String *res1= args[0]->val_str(&cmp.value1); + String *res2= args[1]->val_str(&cmp.value2); Geometry_buffer buffer1, buffer2; Geometry *g1, *g2; MBR mbr1, mbr2; === modified file 'sql/item_strfunc.cc' --- a/sql/item_strfunc.cc 2009-12-03 11:19:05 +0000 +++ b/sql/item_strfunc.cc 2010-01-15 15:27:55 +0000 @@ -1828,8 +1828,9 @@ String *Item_func_database::val_str(Stri /** - @todo - make USER() replicate properly (currently it is replicated to "") + @note USER() is replicated correctly if binlog_format=ROW or (as of + BUG#28086) binlog_format=MIXED, but is incorrectly replicated to '' + if binlog_format=STATEMENT. */ bool Item_func_user::init(const char *user, const char *host) { === modified file 'sql/item_subselect.cc' --- a/sql/item_subselect.cc 2009-12-03 11:19:05 +0000 +++ b/sql/item_subselect.cc 2010-01-15 15:27:55 +0000 @@ -503,6 +503,7 @@ Item_singlerow_subselect::select_transfo void Item_singlerow_subselect::store(uint i, Item *item) { row[i]->store(item); + row[i]->cache_value(); } enum Item_result Item_singlerow_subselect::result_type() const @@ -1854,6 +1855,7 @@ void subselect_engine::set_row(List<Item if (!(row[i]= Item_cache::get_cache(sel_item))) return; row[i]->setup(sel_item); + row[i]->store(sel_item); } if (item_list.elements > 1) res_type= ROW_RESULT; === modified file 'sql/item_subselect.h' --- a/sql/item_subselect.h 2009-08-31 20:02:09 +0000 +++ b/sql/item_subselect.h 2010-01-15 15:27:55 +0000 @@ -142,6 +142,7 @@ public: @return the SELECT_LEX structure associated with this Item */ st_select_lex* get_select_lex(); + const char *func_name() const { DBUG_ASSERT(0); return "subselect"; } friend class select_subselect; friend class Item_in_optimizer; === modified file 'sql/item_sum.cc' --- a/sql/item_sum.cc 2009-12-03 11:19:05 +0000 +++ b/sql/item_sum.cc 2010-01-15 15:27:55 +0000 @@ -517,8 +517,7 @@ Field *Item_sum::create_tmp_field(bool g name, table->s, collation.collation); break; case DECIMAL_RESULT: - field= new Field_new_decimal(max_length, maybe_null, name, - decimals, unsigned_flag); + field= Field_new_decimal::create_from_item(this); break; case ROW_RESULT: default: @@ -610,35 +609,6 @@ Item_sum_num::fix_fields(THD *thd, Item } -Item_sum_hybrid::Item_sum_hybrid(THD *thd, Item_sum_hybrid *item) - :Item_sum(thd, item), value(item->value), hybrid_type(item->hybrid_type), - hybrid_field_type(item->hybrid_field_type), cmp_sign(item->cmp_sign), - was_values(item->was_values) -{ - /* copy results from old value */ - switch (hybrid_type) { - case INT_RESULT: - sum_int= item->sum_int; - break; - case DECIMAL_RESULT: - my_decimal2decimal(&item->sum_dec, &sum_dec); - break; - case REAL_RESULT: - sum= item->sum; - break; - case STRING_RESULT: - /* - This can happen with ROLLUP. Note that the value is already - copied at function call. - */ - break; - case ROW_RESULT: - default: - DBUG_ASSERT(0); - } - collation.set(item->collation); -} - bool Item_sum_hybrid::fix_fields(THD *thd, Item **ref) { @@ -658,15 +628,12 @@ Item_sum_hybrid::fix_fields(THD *thd, It switch (hybrid_type= item->result_type()) { case INT_RESULT: max_length= 20; - sum_int= 0; break; case DECIMAL_RESULT: max_length= item->max_length; - my_decimal_set_zero(&sum_dec); break; case REAL_RESULT: max_length= float_length(decimals); - sum= 0.0; break; case STRING_RESULT: max_length= item->max_length; @@ -675,10 +642,10 @@ Item_sum_hybrid::fix_fields(THD *thd, It default: DBUG_ASSERT(0); }; + setup(args[0], NULL); /* MIN/MAX can return NULL for empty set indepedent of the used column */ maybe_null= 1; unsigned_flag=item->unsigned_flag; - collation.set(item->collation); result_field=0; null_value=1; fix_length_and_dec(); @@ -696,6 +663,30 @@ Item_sum_hybrid::fix_fields(THD *thd, It return FALSE; } + +/** + MIN/MAX function setup. + + @param item argument of MIN/MAX function + @param value_arg calculated value of MIN/MAX function + + @details + Setup cache/comparator of MIN/MAX functions. When called by the + copy_or_same function value_arg parameter contains calculated value + of the original MIN/MAX object and it is saved in this object's cache. +*/ + +void Item_sum_hybrid::setup(Item *item, Item *value_arg) +{ + value= Item_cache::get_cache(item); + value->setup(item); + value->store(value_arg); + cmp= new Arg_comparator(); + cmp->set_cmp_func(this, args, (Item**)&value, FALSE); + collation.set(item->collation); +} + + Field *Item_sum_hybrid::create_tmp_field(bool group, TABLE *table, uint convert_blob_length) { @@ -1265,8 +1256,7 @@ Field *Item_sum_avg::create_tmp_field(bo 0, name, &my_charset_bin); } else if (hybrid_type == DECIMAL_RESULT) - field= new Field_new_decimal(max_length, maybe_null, name, - decimals, unsigned_flag); + field= Field_new_decimal::create_from_item(this); else field= new Field_double(max_length, maybe_null, name, decimals, TRUE); if (field) @@ -1587,19 +1577,7 @@ void Item_sum_variance::update_field() void Item_sum_hybrid::clear() { - switch (hybrid_type) { - case INT_RESULT: - sum_int= 0; - break; - case DECIMAL_RESULT: - my_decimal_set_zero(&sum_dec); - break; - case REAL_RESULT: - sum= 0.0; - break; - default: - value.length(0); - } + value->null_value= 1; null_value= 1; } @@ -1608,30 +1586,7 @@ double Item_sum_hybrid::val_real() DBUG_ASSERT(fixed == 1); if (null_value) return 0.0; - switch (hybrid_type) { - case STRING_RESULT: - { - char *end_not_used; - int err_not_used; - String *res; res=val_str(&str_value); - return (res ? my_strntod(res->charset(), (char*) res->ptr(), res->length(), - &end_not_used, &err_not_used) : 0.0); - } - case INT_RESULT: - if (unsigned_flag) - return ulonglong2double(sum_int); - return (double) sum_int; - case DECIMAL_RESULT: - my_decimal2double(E_DEC_FATAL_ERROR, &sum_dec, &sum); - return sum; - case REAL_RESULT: - return sum; - case ROW_RESULT: - default: - // This case should never be choosen - DBUG_ASSERT(0); - return 0; - } + return value->val_real(); } longlong Item_sum_hybrid::val_int() @@ -1639,18 +1594,7 @@ longlong Item_sum_hybrid::val_int() DBUG_ASSERT(fixed == 1); if (null_value) return 0; - switch (hybrid_type) { - case INT_RESULT: - return sum_int; - case DECIMAL_RESULT: - { - longlong result; - my_decimal2int(E_DEC_FATAL_ERROR, &sum_dec, unsigned_flag, &result); - return sum_int; - } - default: - return (longlong) rint(Item_sum_hybrid::val_real()); - } + return value->val_int(); } @@ -1659,26 +1603,7 @@ my_decimal *Item_sum_hybrid::val_decimal DBUG_ASSERT(fixed == 1); if (null_value) return 0; - switch (hybrid_type) { - case STRING_RESULT: - string2my_decimal(E_DEC_FATAL_ERROR, &value, val); - break; - case REAL_RESULT: - double2my_decimal(E_DEC_FATAL_ERROR, sum, val); - break; - case DECIMAL_RESULT: - val= &sum_dec; - break; - case INT_RESULT: - int2my_decimal(E_DEC_FATAL_ERROR, sum_int, unsigned_flag, val); - break; - case ROW_RESULT: - default: - // This case should never be choosen - DBUG_ASSERT(0); - break; - } - return val; // Keep compiler happy + return value->val_decimal(val); } @@ -1688,25 +1613,7 @@ Item_sum_hybrid::val_str(String *str) DBUG_ASSERT(fixed == 1); if (null_value) return 0; - switch (hybrid_type) { - case STRING_RESULT: - return &value; - case REAL_RESULT: - str->set_real(sum,decimals, &my_charset_bin); - break; - case DECIMAL_RESULT: - my_decimal2string(E_DEC_FATAL_ERROR, &sum_dec, 0, 0, 0, str); - return str; - case INT_RESULT: - str->set_int(sum_int, unsigned_flag, &my_charset_bin); - break; - case ROW_RESULT: - default: - // This case should never be choosen - DBUG_ASSERT(0); - break; - } - return str; // Keep compiler happy + return value->val_str(str); } @@ -1715,7 +1622,9 @@ void Item_sum_hybrid::cleanup() DBUG_ENTER("Item_sum_hybrid::cleanup"); Item_sum::cleanup(); forced_const= FALSE; - + if (cmp) + delete cmp; + cmp= 0; /* by default it is TRUE to avoid TRUE reporting by Item_func_not_all/Item_func_nop_all if this item was never called. @@ -1736,63 +1645,22 @@ void Item_sum_hybrid::no_rows_in_result( Item *Item_sum_min::copy_or_same(THD* thd) { - return new (thd->mem_root) Item_sum_min(thd, this); + Item_sum_min *item= new (thd->mem_root) Item_sum_min(thd, this); + item->setup(args[0], value); + return item; } bool Item_sum_min::add() { - switch (hybrid_type) { - case STRING_RESULT: + /* args[0] < value */ + int res= cmp->compare(); + if (!args[0]->null_value && + (null_value || res < 0)) { - String *result=args[0]->val_str(&tmp_value); - if (!args[0]->null_value && - (null_value || sortcmp(&value,result,collation.collation) > 0)) - { - value.copy(*result); - null_value=0; - } - } - break; - case INT_RESULT: - { - longlong nr=args[0]->val_int(); - if (!args[0]->null_value && (null_value || - (unsigned_flag && - (ulonglong) nr < (ulonglong) sum_int) || - (!unsigned_flag && nr < sum_int))) - { - sum_int=nr; - null_value=0; - } - } - break; - case DECIMAL_RESULT: - { - my_decimal value_buff, *val= args[0]->val_decimal(&value_buff); - if (!args[0]->null_value && - (null_value || (my_decimal_cmp(&sum_dec, val) > 0))) - { - my_decimal2decimal(val, &sum_dec); - null_value= 0; - } - } - break; - case REAL_RESULT: - { - double nr= args[0]->val_real(); - if (!args[0]->null_value && (null_value || nr < sum)) - { - sum=nr; - null_value=0; - } - } - break; - case ROW_RESULT: - default: - // This case should never be choosen - DBUG_ASSERT(0); - break; + value->store(args[0]); + value->cache_value(); + null_value= 0; } return 0; } @@ -1800,63 +1668,22 @@ bool Item_sum_min::add() Item *Item_sum_max::copy_or_same(THD* thd) { - return new (thd->mem_root) Item_sum_max(thd, this); + Item_sum_max *item= new (thd->mem_root) Item_sum_max(thd, this); + item->setup(args[0], value); + return item; } bool Item_sum_max::add() { - switch (hybrid_type) { - case STRING_RESULT: + /* args[0] > value */ + int res= cmp->compare(); + if (!args[0]->null_value && + (null_value || res > 0)) { - String *result=args[0]->val_str(&tmp_value); - if (!args[0]->null_value && - (null_value || sortcmp(&value,result,collation.collation) < 0)) - { - value.copy(*result); - null_value=0; - } - } - break; - case INT_RESULT: - { - longlong nr=args[0]->val_int(); - if (!args[0]->null_value && (null_value || - (unsigned_flag && - (ulonglong) nr > (ulonglong) sum_int) || - (!unsigned_flag && nr > sum_int))) - { - sum_int=nr; - null_value=0; - } - } - break; - case DECIMAL_RESULT: - { - my_decimal value_buff, *val= args[0]->val_decimal(&value_buff); - if (!args[0]->null_value && - (null_value || (my_decimal_cmp(val, &sum_dec) > 0))) - { - my_decimal2decimal(val, &sum_dec); - null_value= 0; - } - } - break; - case REAL_RESULT: - { - double nr= args[0]->val_real(); - if (!args[0]->null_value && (null_value || nr > sum)) - { - sum=nr; - null_value=0; - } - } - break; - case ROW_RESULT: - default: - // This case should never be choosen - DBUG_ASSERT(0); - break; + value->store(args[0]); + value->cache_value(); + null_value= 0; } return 0; } @@ -2221,14 +2048,15 @@ void Item_sum_hybrid::update_field() void Item_sum_hybrid::min_max_update_str_field() { - String *res_str=args[0]->val_str(&value); + DBUG_ASSERT(cmp); + String *res_str=args[0]->val_str(&cmp->value1); if (!args[0]->null_value) { - result_field->val_str(&tmp_value); + result_field->val_str(&cmp->value2); if (result_field->is_null() || - (cmp_sign * sortcmp(res_str,&tmp_value,collation.collation)) < 0) + (cmp_sign * sortcmp(res_str,&cmp->value2,collation.collation)) < 0) result_field->store(res_str->ptr(),res_str->length(),res_str->charset()); result_field->set_notnull(); } === modified file 'sql/item_sum.h' --- a/sql/item_sum.h 2009-09-15 10:46:35 +0000 +++ b/sql/item_sum.h 2010-01-15 15:27:55 +0000 @@ -329,22 +329,6 @@ public: virtual void update_field()=0; virtual bool keep_field_type(void) const { return 0; } virtual void fix_length_and_dec() { maybe_null=1; null_value=1; } - /* - This method is used for debug purposes to print the name of an - item to the debug log. The second use of this method is as - a helper function of print(), where it is applicable. - To suit both goals it should return a meaningful, - distinguishable and sintactically correct string. This method - should not be used for runtime type identification, use enum - {Sum}Functype and Item_func::functype()/Item_sum::sum_func() - instead. - - NOTE: for Items inherited from Item_sum, func_name() return part of - function name till first argument (including '(') to make difference in - names for functions with 'distinct' clause and without 'distinct' and - also to make printing of items inherited from Item_sum uniform. - */ - virtual const char *func_name() const= 0; virtual Item *result_item(Field *field) { return new Item_field(field); } /* @@ -679,6 +663,7 @@ public: } void fix_length_and_dec() {} enum Item_result result_type () const { return hybrid_type; } + const char *func_name() const { DBUG_ASSERT(0); return "avg_field"; } }; @@ -747,6 +732,7 @@ public: } void fix_length_and_dec() {} enum Item_result result_type () const { return hybrid_type; } + const char *func_name() const { DBUG_ASSERT(0); return "variance_field"; } }; @@ -822,6 +808,7 @@ public: my_decimal *val_decimal(my_decimal *); enum Item_result result_type () const { return REAL_RESULT; } enum_field_types field_type() const { return MYSQL_TYPE_DOUBLE;} + const char *func_name() const { DBUG_ASSERT(0); return "std_field"; } }; /* @@ -847,14 +834,13 @@ class Item_sum_std :public Item_sum_vari }; // This class is a string or number function depending on num_func - +class Arg_comparator; +class Item_cache; class Item_sum_hybrid :public Item_sum { protected: - String value,tmp_value; - double sum; - longlong sum_int; - my_decimal sum_dec; + Item_cache *value; + Arg_comparator *cmp; Item_result hybrid_type; enum_field_types hybrid_field_type; int cmp_sign; @@ -862,12 +848,17 @@ protected: public: Item_sum_hybrid(Item *item_par,int sign) - :Item_sum(item_par), sum(0.0), sum_int(0), + :Item_sum(item_par), value(0), cmp(0), hybrid_type(INT_RESULT), hybrid_field_type(MYSQL_TYPE_LONGLONG), cmp_sign(sign), was_values(TRUE) { collation.set(&my_charset_bin); } - Item_sum_hybrid(THD *thd, Item_sum_hybrid *item); + Item_sum_hybrid(THD *thd, Item_sum_hybrid *item) + :Item_sum(thd, item), value(item->value), hybrid_type(item->hybrid_type), + hybrid_field_type(item->hybrid_field_type), cmp_sign(item->cmp_sign), + was_values(item->was_values) + { } bool fix_fields(THD *, Item **); + void setup(Item *item, Item *value_arg); void clear(); double val_real(); longlong val_int(); === modified file 'sql/item_timefunc.cc' --- a/sql/item_timefunc.cc 2009-12-03 11:19:05 +0000 +++ b/sql/item_timefunc.cc 2010-01-15 15:27:55 +0000 @@ -391,7 +391,7 @@ static bool extract_date_time(DATE_TIME_ if (tmp - val > 6) tmp= (char*) val + 6; l_time->second_part= (int) my_strtoll10(val, &tmp, &error); - frac_part= 6 - (uint) (tmp - val); + frac_part= 6 - (int) (tmp - val); if (frac_part > 0) l_time->second_part*= (ulong) log_10_int[frac_part]; val= tmp; @@ -882,9 +882,9 @@ static bool get_interval_info(const char value= value*LL(10) + (longlong) (*str - '0'); if (transform_msec && i == count - 1) // microseconds always last { - long msec_length= 6 - (uint) (str - start); + int msec_length= 6 - (int)(str - start); if (msec_length > 0) - value*= (long) log_10_int[msec_length]; + value*= (long)log_10_int[msec_length]; } values[i]= value; while (str != end && !my_isdigit(cs,*str)) === modified file 'sql/item_xmlfunc.cc' --- a/sql/item_xmlfunc.cc 2009-12-03 11:19:05 +0000 +++ b/sql/item_xmlfunc.cc 2010-01-15 15:27:55 +0000 @@ -941,14 +941,16 @@ static Item *create_comparator(MY_XPATH in a loop through all of the nodes in the node set. */ - Item *fake= new Item_string("", 0, xpath->cs); + Item_string *fake= new Item_string("", 0, xpath->cs); + /* Don't cache fake because its value will be changed during comparison.*/ + fake->set_used_tables(RAND_TABLE_BIT); Item_nodeset_func *nodeset; Item *scalar, *comp; if (a->type() == Item::XPATH_NODESET) { nodeset= (Item_nodeset_func*) a; scalar= b; - comp= eq_func(oper, fake, scalar); + comp= eq_func(oper, (Item*)fake, scalar); } else { === modified file 'sql/log.cc' --- a/sql/log.cc 2010-01-04 18:25:29 +0000 +++ b/sql/log.cc 2010-01-15 15:27:55 +0000 @@ -5691,9 +5691,8 @@ int TC_LOG_BINLOG::recover(IO_CACHE *log Xid_log_event *xev=(Xid_log_event *)ev; uchar *x= (uchar *) memdup_root(&mem_root, (uchar*) &xev->xid, sizeof(xev->xid)); - if (! x) + if (!x || my_hash_insert(&xids, x)) goto err2; - my_hash_insert(&xids, x); } delete ev; } === modified file 'sql/log_event.cc' --- a/sql/log_event.cc 2009-12-03 11:34:11 +0000 +++ b/sql/log_event.cc 2010-01-15 15:27:55 +0000 @@ -8453,13 +8453,17 @@ Rows_log_event::write_row(const Relay_lo auto_afree_ptr<char> key(NULL); /* fill table->record[0] with default values */ - + bool abort_on_warnings= (rli->sql_thd->variables.sql_mode & + (MODE_STRICT_TRANS_TABLES | MODE_STRICT_ALL_TABLES)); if ((error= prepare_record(table, m_width, - TRUE /* check if columns have def. values */))) + table->file->ht->db_type != DB_TYPE_NDBCLUSTER, + abort_on_warnings, m_curr_row == m_rows_buf))) DBUG_RETURN(error); /* unpack row into table->record[0] */ - error= unpack_current_row(rli); // TODO: how to handle errors? + if ((error= unpack_current_row(rli, abort_on_warnings))) + DBUG_RETURN(error); + if (m_curr_row == m_rows_buf) { /* this is the first row to be inserted, we estimate the rows with @@ -9256,8 +9260,12 @@ Update_rows_log_event::do_exec_row(const store_record(m_table,record[1]); + bool abort_on_warnings= (rli->sql_thd->variables.sql_mode & + (MODE_STRICT_TRANS_TABLES | MODE_STRICT_ALL_TABLES)); m_curr_row= m_curr_row_end; - error= unpack_current_row(rli); // this also updates m_curr_row_end + /* this also updates m_curr_row_end */ + if ((error= unpack_current_row(rli, abort_on_warnings))) + return error; /* Now we have the right row to update. The old row (the one we're === modified file 'sql/log_event.h' --- a/sql/log_event.h 2009-12-03 11:19:05 +0000 +++ b/sql/log_event.h 2010-01-15 15:27:55 +0000 @@ -3541,12 +3541,16 @@ protected: int write_row(const Relay_log_info *const, const bool); // Unpack the current row into m_table->record[0] - int unpack_current_row(const Relay_log_info *const rli) + int unpack_current_row(const Relay_log_info *const rli, + const bool abort_on_warning= TRUE) { DBUG_ASSERT(m_table); + + bool first_row= (m_curr_row == m_rows_buf); ASSERT_OR_RETURN_ERROR(m_curr_row < m_rows_end, HA_ERR_CORRUPT_EVENT); int const result= ::unpack_row(rli, m_table, m_width, m_curr_row, &m_cols, - &m_curr_row_end, &m_master_reclength); + &m_curr_row_end, &m_master_reclength, + abort_on_warning, first_row); if (m_curr_row_end > m_rows_end) my_error(ER_SLAVE_CORRUPT_EVENT, MYF(0)); ASSERT_OR_RETURN_ERROR(m_curr_row_end <= m_rows_end, HA_ERR_CORRUPT_EVENT); === modified file 'sql/mysqld.cc' --- a/sql/mysqld.cc 2009-12-04 15:12:22 +0000 +++ b/sql/mysqld.cc 2010-01-29 18:42:22 +0000 @@ -1173,11 +1173,14 @@ static void __cdecl kill_server(int sig_ /* Send event to smem_event_connect_request for aborting */ - if (!SetEvent(smem_event_connect_request)) + if (opt_enable_shared_memory) { - DBUG_PRINT("error", - ("Got error: %ld from SetEvent of smem_event_connect_request", - GetLastError())); + if (!SetEvent(smem_event_connect_request)) + { + DBUG_PRINT("error", + ("Got error: %ld from SetEvent of smem_event_connect_request", + GetLastError())); + } } #endif @@ -2704,7 +2707,7 @@ static void init_signals(void) { /* Change limits so that we will get a core file */ STRUCT_RLIMIT rl; - rl.rlim_cur = rl.rlim_max = RLIM_INFINITY; + rl.rlim_cur = rl.rlim_max = (rlim_t) RLIM_INFINITY; if (setrlimit(RLIMIT_CORE, &rl) && global_system_variables.log_warnings) sql_print_warning("setrlimit could not change the size of core files to 'infinity'; We may not be able to generate a core file on signals"); } @@ -4005,6 +4008,27 @@ server."); if (opt_bin_log) { + /* Reports an error and aborts, if the --log-bin's path + is a directory.*/ + if (opt_bin_logname && + opt_bin_logname[strlen(opt_bin_logname) - 1] == FN_LIBCHAR) + { + sql_print_error("Path '%s' is a directory name, please specify \ +a file name for --log-bin option", opt_bin_logname); + unireg_abort(1); + } + + /* Reports an error and aborts, if the --log-bin-index's path + is a directory.*/ + if (opt_binlog_index_name && + opt_binlog_index_name[strlen(opt_binlog_index_name) - 1] + == FN_LIBCHAR) + { + sql_print_error("Path '%s' is a directory name, please specify \ +a file name for --log-bin-index option", opt_binlog_index_name); + unireg_abort(1); + } + char buf[FN_REFLEN]; const char *ln; ln= mysql_bin_log.generate_name(opt_bin_logname, "-bin", 1, buf); @@ -5355,12 +5379,16 @@ pthread_handler_t handle_connections_soc pthread_handler_t handle_connections_namedpipes(void *arg) { HANDLE hConnectedPipe; - OVERLAPPED connectOverlapped = {0}; + OVERLAPPED connectOverlapped= {0}; THD *thd; my_thread_init(); DBUG_ENTER("handle_connections_namedpipes"); - connectOverlapped.hEvent = CreateEvent(NULL, TRUE, FALSE, NULL); - + connectOverlapped.hEvent= CreateEvent(NULL, TRUE, FALSE, NULL); + if (!connectOverlapped.hEvent) + { + sql_print_error("Can't create event, last error=%u", GetLastError()); + unireg_abort(1); + } DBUG_PRINT("general",("Waiting for named pipe connections.")); while (!abort_loop) { @@ -5383,7 +5411,8 @@ pthread_handler_t handle_connections_nam { CloseHandle(hPipe); if ((hPipe= CreateNamedPipe(pipe_name, - PIPE_ACCESS_DUPLEX|FILE_FLAG_OVERLAPPED, + PIPE_ACCESS_DUPLEX | + FILE_FLAG_OVERLAPPED, PIPE_TYPE_BYTE | PIPE_READMODE_BYTE | PIPE_WAIT, @@ -5403,7 +5432,8 @@ pthread_handler_t handle_connections_nam hConnectedPipe = hPipe; /* create new pipe for new connection */ if ((hPipe = CreateNamedPipe(pipe_name, - PIPE_ACCESS_DUPLEX|FILE_FLAG_OVERLAPPED, + PIPE_ACCESS_DUPLEX | + FILE_FLAG_OVERLAPPED, PIPE_TYPE_BYTE | PIPE_READMODE_BYTE | PIPE_WAIT, @@ -8974,14 +9004,8 @@ static int fix_paths(void) pos[0]= FN_LIBCHAR; pos[1]= 0; } - convert_dirname(mysql_real_data_home,mysql_real_data_home,NullS); - my_realpath(mysql_unpacked_real_data_home, mysql_real_data_home, MYF(0)); - mysql_unpacked_real_data_home_len= strlen(mysql_unpacked_real_data_home); - if (mysql_unpacked_real_data_home[mysql_unpacked_real_data_home_len-1] == FN_LIBCHAR) - --mysql_unpacked_real_data_home_len; - - convert_dirname(language,language,NullS); + convert_dirname(mysql_real_data_home,mysql_real_data_home,NullS); (void) my_load_path(mysql_home,mysql_home,""); // Resolve current dir (void) my_load_path(mysql_real_data_home,mysql_real_data_home,mysql_home); (void) my_load_path(pidfile_name,pidfile_name,mysql_real_data_home); @@ -8989,6 +9013,12 @@ static int fix_paths(void) get_relative_path(PLUGINDIR), mysql_home); opt_plugin_dir_ptr= opt_plugin_dir; + my_realpath(mysql_unpacked_real_data_home, mysql_real_data_home, MYF(0)); + mysql_unpacked_real_data_home_len= + (int) strlen(mysql_unpacked_real_data_home); + if (mysql_unpacked_real_data_home[mysql_unpacked_real_data_home_len-1] == FN_LIBCHAR) + --mysql_unpacked_real_data_home_len; + char *sharedir=get_relative_path(SHAREDIR); if (test_if_hard_path(sharedir)) strmake(buff,sharedir,sizeof(buff)-1); /* purecov: tested */ @@ -9019,8 +9049,8 @@ static int fix_paths(void) /* Convert the secure-file-priv option to system format, allowing a quick strcmp to check if read or write is in an allowed dir - */ - if (opt_secure_file_priv) + */ + if (opt_secure_file_priv && opt_secure_file_priv[0]) { convert_dirname(buff, opt_secure_file_priv, NullS); my_free(opt_secure_file_priv, MYF(0)); === modified file 'sql/net_serv.cc' --- a/sql/net_serv.cc 2009-09-07 20:50:10 +0000 +++ b/sql/net_serv.cc 2010-01-29 10:42:31 +0000 @@ -219,7 +219,7 @@ my_bool net_realloc(NET *net, size_t len -1 Don't know if data is ready or not */ -#if !defined(EMBEDDED_LIBRARY) +#if !defined(EMBEDDED_LIBRARY) && defined(DBUG_OFF) static int net_data_is_ready(my_socket sd) { @@ -280,13 +280,18 @@ static int net_data_is_ready(my_socket s void net_clear(NET *net, my_bool clear_buffer) { -#if !defined(EMBEDDED_LIBRARY) +#if !defined(EMBEDDED_LIBRARY) && defined(DBUG_OFF) size_t count; int ready; #endif DBUG_ENTER("net_clear"); -#if !defined(EMBEDDED_LIBRARY) +/* + We don't do a clear in case of DBUG_OFF to catch bugs + in the protocol handling +*/ + +#if !defined(EMBEDDED_LIBRARY) && defined(DBUG_OFF) if (clear_buffer) { while ((ready= net_data_is_ready(net->vio->sd)) > 0) === modified file 'sql/opt_range.cc' --- a/sql/opt_range.cc 2009-12-03 11:19:05 +0000 +++ b/sql/opt_range.cc 2010-01-15 15:27:55 +0000 @@ -446,9 +446,9 @@ public: range_key, *range_key_flag); *range_key_flag|= key_tree->min_flag; if (key_tree->next_key_part && + key_tree->next_key_part->type == SEL_ARG::KEY_RANGE && key_tree->next_key_part->part == key_tree->part+1 && - !(*range_key_flag & (NO_MIN_RANGE | NEAR_MIN)) && - key_tree->next_key_part->type == SEL_ARG::KEY_RANGE) + !(*range_key_flag & (NO_MIN_RANGE | NEAR_MIN))) res+= key_tree->next_key_part->store_min_key(key, range_key, range_key_flag); return res; @@ -462,9 +462,9 @@ public: range_key, *range_key_flag); (*range_key_flag)|= key_tree->max_flag; if (key_tree->next_key_part && + key_tree->next_key_part->type == SEL_ARG::KEY_RANGE && key_tree->next_key_part->part == key_tree->part+1 && - !(*range_key_flag & (NO_MAX_RANGE | NEAR_MAX)) && - key_tree->next_key_part->type == SEL_ARG::KEY_RANGE) + !(*range_key_flag & (NO_MAX_RANGE | NEAR_MAX))) res+= key_tree->next_key_part->store_max_key(key, range_key, range_key_flag); return res; @@ -1700,6 +1700,7 @@ SEL_ARG *SEL_ARG::clone(RANGE_OPT_PARAM tmp->prev= *next_arg; // Link into next/prev chain (*next_arg)->next=tmp; (*next_arg)= tmp; + tmp->part= this->part; } else { @@ -6672,6 +6673,7 @@ key_or(RANGE_OPT_PARAM *param, SEL_ARG * else if ((cmp=tmp->cmp_max_to_min(key2)) < 0) { // Found tmp.max < key2.min SEL_ARG *next=tmp->next; + /* key1 on the left of key2 non-overlapping */ if (cmp == -2 && eq_tree(tmp->next_key_part,key2->next_key_part)) { // Join near ranges like tmp.max < 0 and key2.min >= 0 @@ -6700,6 +6702,7 @@ key_or(RANGE_OPT_PARAM *param, SEL_ARG * int tmp_cmp; if ((tmp_cmp=tmp->cmp_min_to_max(key2)) > 0) // if tmp.min > key2.max { + /* tmp is on the right of key2 non-overlapping */ if (tmp_cmp == 2 && eq_tree(tmp->next_key_part,key2->next_key_part)) { // ranges are connected tmp->copy_min_to_min(key2); @@ -6734,25 +6737,52 @@ key_or(RANGE_OPT_PARAM *param, SEL_ARG * } } - // tmp.max >= key2.min && tmp.min <= key.max (overlapping ranges) + /* + tmp.min >= key2.min && tmp.min <= key.max (overlapping ranges) + key2.min <= tmp.min <= key2.max + */ if (eq_tree(tmp->next_key_part,key2->next_key_part)) { if (tmp->is_same(key2)) { + /* + Found exact match of key2 inside key1. + Use the relevant range in key1. + */ tmp->merge_flags(key2); // Copy maybe flags key2->increment_use_count(-1); // Free not used tree } else { SEL_ARG *last=tmp; + SEL_ARG *first=tmp; + /* + Find the last range in tmp that overlaps key2 and has the same + condition on the rest of the keyparts. + */ while (last->next && last->next->cmp_min_to_max(key2) <= 0 && eq_tree(last->next->next_key_part,key2->next_key_part)) { + /* + We've found the last overlapping key1 range in last. + This means that the ranges between (and including) the + first overlapping range (tmp) and the last overlapping range + (last) are fully nested into the current range of key2 + and can safely be discarded. We just need the minimum endpoint + of the first overlapping range (tmp) so we can compare it with + the minimum endpoint of the enclosing key2 range. + */ SEL_ARG *save=last; last=last->next; key1=key1->tree_delete(save); } - last->copy_min(tmp); + /* + The tmp range (the first overlapping range) could have been discarded + by the previous loop. We should re-direct tmp to the new united range + that's taking its place. + */ + tmp= last; + last->copy_min(first); bool full_range= last->copy_min(key2); if (!full_range) { @@ -7262,27 +7292,25 @@ int test_rb_tree(SEL_ARG *element,SEL_AR } -/* - Count how many times SEL_ARG graph "root" refers to its part "key" +/** + Count how many times SEL_ARG graph "root" refers to its part "key" via + transitive closure. - SYNOPSIS - count_key_part_usage() - root An RB-Root node in a SEL_ARG graph. - key Another RB-Root node in that SEL_ARG graph. + @param root An RB-Root node in a SEL_ARG graph. + @param key Another RB-Root node in that SEL_ARG graph. - DESCRIPTION - The passed "root" node may refer to "key" node via root->next_key_part, - root->next->n + The passed "root" node may refer to "key" node via root->next_key_part, + root->next->n - This function counts how many times the node "key" is referred (via - SEL_ARG::next_key_part) by - - intervals of RB-tree pointed by "root", - - intervals of RB-trees that are pointed by SEL_ARG::next_key_part from - intervals of RB-tree pointed by "root", - - and so on. + This function counts how many times the node "key" is referred (via + SEL_ARG::next_key_part) by + - intervals of RB-tree pointed by "root", + - intervals of RB-trees that are pointed by SEL_ARG::next_key_part from + intervals of RB-tree pointed by "root", + - and so on. - Here is an example (horizontal links represent next_key_part pointers, - vertical links - next/prev prev pointers): + Here is an example (horizontal links represent next_key_part pointers, + vertical links - next/prev prev pointers): +----+ $ |root|-----------------+ @@ -7302,8 +7330,8 @@ int test_rb_tree(SEL_ARG *element,SEL_AR ... +---+ $ | | |------------+ +---+ $ - RETURN - Number of links to "key" from nodes reachable from "root". + @return + Number of links to "key" from nodes reachable from "root". */ static ulong count_key_part_usage(SEL_ARG *root, SEL_ARG *key) @@ -7558,8 +7586,8 @@ check_quick_keys(PARAM *param, uint idx, param->first_null_comp= key_tree->part+1; if (key_tree->next_key_part && - key_tree->next_key_part->part == key_tree->part+1 && - key_tree->next_key_part->type == SEL_ARG::KEY_RANGE) + key_tree->next_key_part->type == SEL_ARG::KEY_RANGE && + key_tree->next_key_part->part == key_tree->part+1) { // const key as prefix if (min_key_length == max_key_length && !memcmp(min_key, max_key, (uint) (tmp_max_key - max_key)) && @@ -7840,8 +7868,8 @@ get_quick_keys(PARAM *param,QUICK_RANGE_ &tmp_max_key,max_key_flag); if (key_tree->next_key_part && - key_tree->next_key_part->part == key_tree->part+1 && - key_tree->next_key_part->type == SEL_ARG::KEY_RANGE) + key_tree->next_key_part->type == SEL_ARG::KEY_RANGE && + key_tree->next_key_part->part == key_tree->part+1) { // const key as prefix if ((tmp_min_key - min_key) == (tmp_max_key - max_key) && memcmp(min_key, max_key, (uint)(tmp_max_key - max_key))==0 && @@ -9823,7 +9851,11 @@ check_group_min_max_predicates(COND *con } else if (cur_arg->const_item()) { - DBUG_RETURN(TRUE); + /* + For predicates of the form "const OP expr" we also have to check 'expr' + to make a decision. + */ + continue; } else DBUG_RETURN(FALSE); === modified file 'sql/repl_failsafe.cc' --- a/sql/repl_failsafe.cc 2009-09-23 13:10:23 +0000 +++ b/sql/repl_failsafe.cc 2009-11-20 15:18:01 +0000 @@ -559,7 +559,12 @@ HOSTS"; goto err; } si->server_id = log_server_id; - my_hash_insert(&slave_list, (uchar*)si); + if (my_hash_insert(&slave_list, (uchar*)si)) + { + error= "the slave is out of memory"; + pthread_mutex_unlock(&LOCK_slave_list); + goto err; + } } strmake(si->host, row[1], sizeof(si->host)-1); si->port = atoi(row[port_ind]); === modified file 'sql/rpl_record.cc' --- a/sql/rpl_record.cc 2009-03-05 19:54:53 +0000 +++ b/sql/rpl_record.cc 2010-01-28 11:35:10 +0000 @@ -180,7 +180,8 @@ int unpack_row(Relay_log_info const *rli, TABLE *table, uint const colcnt, uchar const *const row_data, MY_BITMAP const *cols, - uchar const **const row_end, ulong *const master_reclength) + uchar const **const row_end, ulong *const master_reclength, + const bool abort_on_warning, const bool first_row) { DBUG_ENTER("unpack_row"); DBUG_ASSERT(row_data); @@ -224,8 +225,35 @@ unpack_row(Relay_log_info const *rli, /* Field...::unpack() cannot return 0 */ DBUG_ASSERT(pack_ptr != NULL); - if ((null_bits & null_mask) && f->maybe_null()) - f->set_null(); + if (null_bits & null_mask) + { + if (f->maybe_null()) + { + DBUG_PRINT("debug", ("Was NULL; null mask: 0x%x; null bits: 0x%x", + null_mask, null_bits)); + f->set_null(); + } + else + { + MYSQL_ERROR::enum_warning_level error_type= + MYSQL_ERROR::WARN_LEVEL_NOTE; + if (abort_on_warning && (table->file->has_transactions() || + first_row)) + { + error = HA_ERR_ROWS_EVENT_APPLY; + error_type= MYSQL_ERROR::WARN_LEVEL_ERROR; + } + else + { + f->set_default(); + error_type= MYSQL_ERROR::WARN_LEVEL_WARN; + } + push_warning_printf(current_thd, error_type, + ER_BAD_NULL_ERROR, + ER(ER_BAD_NULL_ERROR), + f->field_name); + } + } else { f->set_notnull(); @@ -305,13 +333,17 @@ unpack_row(Relay_log_info const *rli, @param table Table whose record[0] buffer is prepared. @param skip Number of columns for which default/nullable check should be skipped. - @param check Indicates if errors should be raised when checking - default/nullable field properties. + @param check Specifies if lack of default error needs checking. + @param abort_on_warning + Controls how to react on lack of a field's default. + The parameter mimics the master side one for + @c check_that_all_fields_are_given_values. @returns 0 on success or a handler level error code */ int prepare_record(TABLE *const table, - const uint skip, const bool check) + const uint skip, const bool check, + const bool abort_on_warning, const bool first_row) { DBUG_ENTER("prepare_record"); @@ -326,17 +358,36 @@ int prepare_record(TABLE *const table, if (skip >= table->s->fields || !check) DBUG_RETURN(0); - /* Checking if exists default/nullable fields in the default values. */ - - for (Field **field_ptr= table->field+skip ; *field_ptr ; ++field_ptr) + /* + For fields the extra fields on the slave, we check if they have a default. + The check follows the same rules as the INSERT query without specifying an + explicit value for a field not having the explicit default + (@c check_that_all_fields_are_given_values()). + */ + for (Field **field_ptr= table->field+skip; *field_ptr; ++field_ptr) { - uint32 const mask= NOT_NULL_FLAG | NO_DEFAULT_VALUE_FLAG; Field *const f= *field_ptr; - - if (((f->flags & mask) == mask)) + if ((f->flags & NO_DEFAULT_VALUE_FLAG) && + (f->real_type() != MYSQL_TYPE_ENUM)) { - my_error(ER_NO_DEFAULT_FOR_FIELD, MYF(0), f->field_name); - error = HA_ERR_ROWS_EVENT_APPLY; + + MYSQL_ERROR::enum_warning_level error_type= + MYSQL_ERROR::WARN_LEVEL_NOTE; + if (abort_on_warning && (table->file->has_transactions() || + first_row)) + { + error= HA_ERR_ROWS_EVENT_APPLY; + error_type= MYSQL_ERROR::WARN_LEVEL_ERROR; + } + else + { + f->set_default(); + error_type= MYSQL_ERROR::WARN_LEVEL_WARN; + } + push_warning_printf(current_thd, error_type, + ER_NO_DEFAULT_FOR_FIELD, + ER(ER_NO_DEFAULT_FOR_FIELD), + f->field_name); } } === modified file 'sql/rpl_record.h' --- a/sql/rpl_record.h 2008-01-31 12:54:03 +0000 +++ b/sql/rpl_record.h 2009-10-22 00:15:45 +0000 @@ -27,10 +27,13 @@ size_t pack_row(TABLE* table, MY_BITMAP int unpack_row(Relay_log_info const *rli, TABLE *table, uint const colcnt, uchar const *const row_data, MY_BITMAP const *cols, - uchar const **const row_end, ulong *const master_reclength); + uchar const **const row_end, ulong *const master_reclength, + const bool abort_on_warning= TRUE, const bool first_row= TRUE); // Fill table's record[0] with default values. -int prepare_record(TABLE *const, const uint =0, const bool =FALSE); +int prepare_record(TABLE *const table, const uint skip, const bool check, + const bool abort_on_warning= TRUE, + const bool first_row= TRUE); #endif #endif === modified file 'sql/rpl_rli.cc' --- a/sql/rpl_rli.cc 2009-12-03 11:19:05 +0000 +++ b/sql/rpl_rli.cc 2010-01-15 15:27:55 +0000 @@ -100,7 +100,8 @@ int init_relay_log_info(Relay_log_info* rli->tables_to_lock_count= 0; char pattern[FN_REFLEN]; - if (fn_format(pattern, PREFIX_SQL_LOAD, slave_load_tmpdir, "", + (void) my_realpath(pattern, slave_load_tmpdir, 0); + if (fn_format(pattern, PREFIX_SQL_LOAD, pattern, "", MY_SAFE_PATH | MY_RETURN_REAL_PATH) == NullS) { pthread_mutex_unlock(&rli->data_lock); @@ -127,6 +128,29 @@ int init_relay_log_info(Relay_log_info* rli->relay_log.max_size (and mysql_bin_log.max_size). */ { + /* Reports an error and returns, if the --relay-log's path + is a directory.*/ + if (opt_relay_logname && + opt_relay_logname[strlen(opt_relay_logname) - 1] == FN_LIBCHAR) + { + pthread_mutex_unlock(&rli->data_lock); + sql_print_error("Path '%s' is a directory name, please specify \ +a file name for --relay-log option", opt_relay_logname); + DBUG_RETURN(1); + } + + /* Reports an error and returns, if the --relay-log-index's path + is a directory.*/ + if (opt_relaylog_index_name && + opt_relaylog_index_name[strlen(opt_relaylog_index_name) - 1] + == FN_LIBCHAR) + { + pthread_mutex_unlock(&rli->data_lock); + sql_print_error("Path '%s' is a directory name, please specify \ +a file name for --relay-log-index option", opt_relaylog_index_name); + DBUG_RETURN(1); + } + char buf[FN_REFLEN]; const char *ln; static bool name_warning_sent= 0; === modified file 'sql/rpl_tblmap.cc' --- a/sql/rpl_tblmap.cc 2008-08-20 14:06:31 +0000 +++ b/sql/rpl_tblmap.cc 2009-11-20 15:18:01 +0000 @@ -119,7 +119,13 @@ int table_mapping::set_table(ulong table } e->table_id= table_id; e->table= table; - my_hash_insert(&m_table_ids,(uchar *)e); + if (my_hash_insert(&m_table_ids,(uchar *)e)) + { + /* we add this entry to the chain of free (free for use) entries */ + e->next= m_free; + m_free= e; + DBUG_RETURN(ERR_MEMORY_ALLOCATION); + } DBUG_PRINT("info", ("tid %lu -> table 0x%lx (%s)", table_id, (long) e->table, === modified file 'sql/set_var.cc' --- a/sql/set_var.cc 2009-12-03 11:19:05 +0000 +++ b/sql/set_var.cc 2010-01-14 16:32:41 +0000 @@ -58,6 +58,9 @@ #include <my_getopt.h> #include <thr_alarm.h> #include <myisam.h> +#ifdef WITH_MARIA_STORAGE_ENGINE +#include <maria.h> +#endif #include <my_dir.h> #include <waiting_threads.h> #include "events.h" === modified file 'sql/slave.cc' --- a/sql/slave.cc 2009-12-06 17:51:48 +0000 +++ b/sql/slave.cc 2010-01-29 10:42:31 +0000 @@ -3803,10 +3803,11 @@ static int connect_to_master(THD* thd, M suppress_warnings= 0; mi->report(ERROR_LEVEL, last_errno, "error %s to master '%s@%s:%d'" - " - retry-time: %d retries: %lu", + " - retry-time: %d retries: %lu message: %s", (reconnect ? "reconnecting" : "connecting"), mi->user, mi->host, mi->port, - mi->connect_retry, master_retry_count); + mi->connect_retry, master_retry_count, + mysql_error(mysql)); } /* By default we try forever. The reason is that failure will trigger === modified file 'sql/sp.cc' --- a/sql/sp.cc 2009-10-16 10:29:42 +0000 +++ b/sql/sp.cc 2009-11-21 11:18:21 +0000 @@ -70,6 +70,122 @@ enum MYSQL_PROC_FIELD_COUNT }; +static const +TABLE_FIELD_TYPE proc_table_fields[MYSQL_PROC_FIELD_COUNT] = +{ + { + { C_STRING_WITH_LEN("db") }, + { C_STRING_WITH_LEN("char(64)") }, + { C_STRING_WITH_LEN("utf8") } + }, + { + { C_STRING_WITH_LEN("name") }, + { C_STRING_WITH_LEN("char(64)") }, + { C_STRING_WITH_LEN("utf8") } + }, + { + { C_STRING_WITH_LEN("type") }, + { C_STRING_WITH_LEN("enum('FUNCTION','PROCEDURE')") }, + { NULL, 0 } + }, + { + { C_STRING_WITH_LEN("specific_name") }, + { C_STRING_WITH_LEN("char(64)") }, + { C_STRING_WITH_LEN("utf8") } + }, + { + { C_STRING_WITH_LEN("language") }, + { C_STRING_WITH_LEN("enum('SQL')") }, + { NULL, 0 } + }, + { + { C_STRING_WITH_LEN("sql_data_access") }, + { C_STRING_WITH_LEN("enum('CONTAINS_SQL','NO_SQL','READS_SQL_DATA','MODIFIES_SQL_DATA')") }, + { NULL, 0 } + }, + { + { C_STRING_WITH_LEN("is_deterministic") }, + { C_STRING_WITH_LEN("enum('YES','NO')") }, + { NULL, 0 } + }, + { + { C_STRING_WITH_LEN("security_type") }, + { C_STRING_WITH_LEN("enum('INVOKER','DEFINER')") }, + { NULL, 0 } + }, + { + { C_STRING_WITH_LEN("param_list") }, + { C_STRING_WITH_LEN("blob") }, + { NULL, 0 } + }, + + { + { C_STRING_WITH_LEN("returns") }, + { C_STRING_WITH_LEN("longblob") }, + { NULL, 0 } + }, + { + { C_STRING_WITH_LEN("body") }, + { C_STRING_WITH_LEN("longblob") }, + { NULL, 0 } + }, + { + { C_STRING_WITH_LEN("definer") }, + { C_STRING_WITH_LEN("char(77)") }, + { C_STRING_WITH_LEN("utf8") } + }, + { + { C_STRING_WITH_LEN("created") }, + { C_STRING_WITH_LEN("timestamp") }, + { NULL, 0 } + }, + { + { C_STRING_WITH_LEN("modified") }, + { C_STRING_WITH_LEN("timestamp") }, + { NULL, 0 } + }, + { + { C_STRING_WITH_LEN("sql_mode") }, + { C_STRING_WITH_LEN("set('REAL_AS_FLOAT','PIPES_AS_CONCAT','ANSI_QUOTES'," + "'IGNORE_SPACE','NOT_USED','ONLY_FULL_GROUP_BY','NO_UNSIGNED_SUBTRACTION'," + "'NO_DIR_IN_CREATE','POSTGRESQL','ORACLE','MSSQL','DB2','MAXDB'," + "'NO_KEY_OPTIONS','NO_TABLE_OPTIONS','NO_FIELD_OPTIONS','MYSQL323','MYSQL40'," + "'ANSI','NO_AUTO_VALUE_ON_ZERO','NO_BACKSLASH_ESCAPES','STRICT_TRANS_TABLES'," + "'STRICT_ALL_TABLES','NO_ZERO_IN_DATE','NO_ZERO_DATE','INVALID_DATES'," + "'ERROR_FOR_DIVISION_BY_ZERO','TRADITIONAL','NO_AUTO_CREATE_USER'," + "'HIGH_NOT_PRECEDENCE','NO_ENGINE_SUBSTITUTION','PAD_CHAR_TO_FULL_LENGTH')") }, + { NULL, 0 } + }, + { + { C_STRING_WITH_LEN("comment") }, + { C_STRING_WITH_LEN("char(64)") }, + { C_STRING_WITH_LEN("utf8") } + }, + { + { C_STRING_WITH_LEN("character_set_client") }, + { C_STRING_WITH_LEN("char(32)") }, + { C_STRING_WITH_LEN("utf8") } + }, + { + { C_STRING_WITH_LEN("collation_connection") }, + { C_STRING_WITH_LEN("char(32)") }, + { C_STRING_WITH_LEN("utf8") } + }, + { + { C_STRING_WITH_LEN("db_collation") }, + { C_STRING_WITH_LEN("char(32)") }, + { C_STRING_WITH_LEN("utf8") } + }, + { + { C_STRING_WITH_LEN("body_utf8") }, + { C_STRING_WITH_LEN("longblob") }, + { NULL, 0 } + } +}; + +static const TABLE_FIELD_DEF + proc_table_def= {MYSQL_PROC_FIELD_COUNT, proc_table_fields}; + /*************************************************************************/ /** @@ -247,6 +363,50 @@ Stored_routine_creation_ctx::load_from_d /*************************************************************************/ +class Proc_table_intact : public Table_check_intact +{ +private: + bool m_print_once; + +public: + Proc_table_intact() : m_print_once(TRUE) {} + +protected: + void report_error(uint code, const char *fmt, ...); +}; + + +/** + Report failure to validate the mysql.proc table definition. + Print a message to the error log only once. +*/ + +void Proc_table_intact::report_error(uint code, const char *fmt, ...) +{ + va_list args; + char buf[512]; + + va_start(args, fmt); + my_vsnprintf(buf, sizeof(buf), fmt, args); + va_end(args); + + if (code) + my_message(code, buf, MYF(0)); + else + my_error(ER_CANNOT_LOAD_FROM_TABLE, MYF(0), "proc"); + + if (m_print_once) + { + m_print_once= FALSE; + sql_print_error("%s", buf); + } +}; + + +/** Single instance used to control printing to the error log. */ +static Proc_table_intact proc_table_intact; + + /** Open the mysql.proc table for read. @@ -266,15 +426,17 @@ TABLE *open_proc_table_for_read(THD *thd DBUG_ENTER("open_proc_table_for_read"); TABLE_LIST table; - bzero((char*) &table, sizeof(table)); - table.db= (char*) "mysql"; - table.table_name= table.alias= (char*)"proc"; - table.lock_type= TL_READ; + table.init_one_table("mysql", "proc", TL_READ); + + if (open_system_tables_for_read(thd, &table, backup)) + DBUG_RETURN(NULL); - if (!open_system_tables_for_read(thd, &table, backup)) + if (!proc_table_intact.check(table.table, &proc_table_def)) DBUG_RETURN(table.table); - else - DBUG_RETURN(0); + + close_system_tables(thd, backup); + + DBUG_RETURN(NULL); } @@ -296,13 +458,19 @@ static TABLE *open_proc_table_for_update { DBUG_ENTER("open_proc_table_for_update"); - TABLE_LIST table; - bzero((char*) &table, sizeof(table)); - table.db= (char*) "mysql"; - table.table_name= table.alias= (char*)"proc"; - table.lock_type= TL_WRITE; + TABLE *table; + TABLE_LIST table_list; + table_list.init_one_table("mysql", "proc", TL_WRITE); + + if (!(table= open_system_table_for_update(thd, &table_list))) + DBUG_RETURN(NULL); + + if (!proc_table_intact.check(table, &proc_table_def)) + DBUG_RETURN(table); + + close_thread_tables(thd); - DBUG_RETURN(open_system_table_for_update(thd, &table)); + DBUG_RETURN(NULL); } @@ -1506,7 +1674,8 @@ static bool add_used_routine(LEX *lex, Q rn->key.length= key->length; rn->key.str= (char *)rn + sizeof(Sroutine_hash_entry); memcpy(rn->key.str, key->str, key->length + 1); - my_hash_insert(&lex->sroutines, (uchar *)rn); + if (my_hash_insert(&lex->sroutines, (uchar *)rn)) + return FALSE; lex->sroutines_list.link_in_list((uchar *)rn, (uchar **)&rn->next); rn->belong_to_view= belong_to_view; return TRUE; @@ -1584,16 +1753,24 @@ void sp_remove_not_own_routines(LEX *lex dependant on time of life of elements from source hash. It also won't touch lists linking elements in source and destination hashes. + + @returns + @return TRUE Failure + @return FALSE Success */ -void sp_update_sp_used_routines(HASH *dst, HASH *src) +bool sp_update_sp_used_routines(HASH *dst, HASH *src) { for (uint i=0 ; i < src->records ; i++) { Sroutine_hash_entry *rt= (Sroutine_hash_entry *)hash_element(src, i); if (!hash_search(dst, (uchar *)rt->key.str, rt->key.length)) - my_hash_insert(dst, (uchar *)rt); + { + if (my_hash_insert(dst, (uchar *)rt)) + return TRUE; + } } + return FALSE; } === modified file 'sql/sp.h' --- a/sql/sp.h 2009-07-28 17:44:38 +0000 +++ b/sql/sp.h 2009-11-20 15:18:01 +0000 @@ -69,7 +69,7 @@ void sp_get_prelocking_info(THD *thd, bo void sp_add_used_routine(LEX *lex, Query_arena *arena, sp_name *rt, char rt_type); void sp_remove_not_own_routines(LEX *lex); -void sp_update_sp_used_routines(HASH *dst, HASH *src); +bool sp_update_sp_used_routines(HASH *dst, HASH *src); int sp_cache_routines_and_add_tables(THD *thd, LEX *lex, bool first_no_prelock); int sp_cache_routines_and_add_tables_for_view(THD *thd, LEX *lex, === modified file 'sql/sp_cache.cc' --- a/sql/sp_cache.cc 2008-12-02 22:02:52 +0000 +++ b/sql/sp_cache.cc 2010-01-15 15:27:55 +0000 @@ -36,10 +36,16 @@ public: sp_cache(); ~sp_cache(); - inline void insert(sp_head *sp) + /** + Inserts a sp_head object into a hash table. + + @returns Success status + @return TRUE Failure + @return FALSE Success + */ + inline bool insert(sp_head *sp) { - /* TODO: why don't we check return value? */ - my_hash_insert(&m_hashtable, (const uchar *)sp); + return my_hash_insert(&m_hashtable, (const uchar *)sp); } inline sp_head *lookup(char *name, uint namelen) === modified file 'sql/sp_head.cc' --- a/sql/sp_head.cc 2009-12-03 11:34:11 +0000 +++ b/sql/sp_head.cc 2010-01-15 15:27:55 +0000 @@ -2090,8 +2090,18 @@ sp_head::reset_lex(THD *thd) DBUG_RETURN(FALSE); } -/// Restore lex during parsing, after we have parsed a sub statement. -void + +/** + Restore lex during parsing, after we have parsed a sub statement. + + @param thd Thread handle + + @return + @retval TRUE failure + @retval FALSE success +*/ + +bool sp_head::restore_lex(THD *thd) { DBUG_ENTER("sp_head::restore_lex"); @@ -2102,7 +2112,7 @@ sp_head::restore_lex(THD *thd) oldlex= (LEX *)m_lex.pop(); if (! oldlex) - return; // Nothing to restore + DBUG_RETURN(FALSE); // Nothing to restore oldlex->trg_table_fields.push_back(&sublex->trg_table_fields); @@ -2118,7 +2128,8 @@ sp_head::restore_lex(THD *thd) Add routines which are used by statement to respective set for this routine. */ - sp_update_sp_used_routines(&m_sroutines, &sublex->sroutines); + if (sp_update_sp_used_routines(&m_sroutines, &sublex->sroutines)) + DBUG_RETURN(TRUE); /* Merge tables used by this statement (but not by its functions or procedures) to multiset of tables used by this routine. @@ -2130,7 +2141,7 @@ sp_head::restore_lex(THD *thd) delete sublex; } thd->lex= oldlex; - DBUG_VOID_RETURN; + DBUG_RETURN(FALSE); } /** @@ -3867,7 +3878,8 @@ sp_head::merge_table_list(THD *thd, TABL tab->lock_type= table->lock_type; tab->lock_count= tab->query_lock_count= 1; tab->trg_event_map= table->trg_event_map; - my_hash_insert(&m_sptabs, (uchar *)tab); + if (my_hash_insert(&m_sptabs, (uchar *)tab)) + return FALSE; } } return TRUE; === modified file 'sql/sp_head.h' --- a/sql/sp_head.h 2009-04-29 02:59:10 +0000 +++ b/sql/sp_head.h 2009-11-20 15:18:01 +0000 @@ -340,7 +340,7 @@ public: @todo Conflicting comment in sp_head.cc */ - void + bool restore_lex(THD *thd); /// Put the instruction on the backpatch list, associated with the label. === modified file 'sql/sp_rcontext.cc' --- a/sql/sp_rcontext.cc 2008-01-23 22:36:57 +0000 +++ b/sql/sp_rcontext.cc 2009-11-06 19:34:25 +0000 @@ -617,7 +617,7 @@ sp_rcontext::set_case_expr(THD *thd, int } m_case_expr_holders[case_expr_id]->store(case_expr_item); - + m_case_expr_holders[case_expr_id]->cache_value(); return FALSE; } === modified file 'sql/sql_acl.cc' --- a/sql/sql_acl.cc 2009-12-03 11:19:05 +0000 +++ b/sql/sql_acl.cc 2010-01-15 15:27:55 +0000 @@ -31,9 +31,8 @@ #include "sp_head.h" #include "sp.h" -time_t mysql_db_table_last_check= 0L; - -TABLE_FIELD_W_TYPE mysql_db_table_fields[MYSQL_DB_FIELD_COUNT] = { +static const +TABLE_FIELD_TYPE mysql_db_table_fields[MYSQL_DB_FIELD_COUNT] = { { { C_STRING_WITH_LEN("Host") }, { C_STRING_WITH_LEN("char(60)") }, @@ -146,6 +145,8 @@ TABLE_FIELD_W_TYPE mysql_db_table_fields } }; +const TABLE_FIELD_DEF + mysql_db_table_def= {MYSQL_DB_FIELD_COUNT, mysql_db_table_fields}; #ifndef NO_EMBEDDED_ACCESS_CHECKS @@ -2405,7 +2406,12 @@ GRANT_TABLE::GRANT_TABLE(TABLE *form, TA privs = cols = 0; /* purecov: deadcode */ return; /* purecov: deadcode */ } - my_hash_insert(&hash_columns, (uchar *) mem_check); + if (my_hash_insert(&hash_columns, (uchar *) mem_check)) + { + /* Invalidate this entry */ + privs= cols= 0; + return; + } } while (!col_privs->file->index_next(col_privs->record[0]) && !key_cmp_if_same(col_privs,key,0,key_prefix_len)); col_privs->file->ha_index_end(); @@ -2439,14 +2445,17 @@ static GRANT_NAME *name_hash_search(HASH const char *host,const char* ip, const char *db, const char *user, const char *tname, - bool exact) + bool exact, bool name_tolower) { - char helping [NAME_LEN*2+USERNAME_LENGTH+3]; + char helping [NAME_LEN*2+USERNAME_LENGTH+3], *name_ptr; uint len; GRANT_NAME *grant_name,*found=0; HASH_SEARCH_STATE state; - len = (uint) (strmov(strmov(strmov(helping,user)+1,db)+1,tname)-helping)+ 1; + name_ptr= strmov(strmov(helping, user) + 1, db) + 1; + len = (uint) (strmov(name_ptr, tname) - helping) + 1; + if (name_tolower) + my_casedn_str(files_charset_info, name_ptr); for (grant_name= (GRANT_NAME*) hash_first(name_hash, (uchar*) helping, len, &state); grant_name ; @@ -2479,7 +2488,7 @@ routine_hash_search(const char *host, co { return (GRANT_TABLE*) name_hash_search(proc ? &proc_priv_hash : &func_priv_hash, - host, ip, db, user, tname, exact); + host, ip, db, user, tname, exact, TRUE); } @@ -2488,7 +2497,7 @@ table_hash_search(const char *host, cons const char *user, const char *tname, bool exact) { return (GRANT_TABLE*) name_hash_search(&column_priv_hash, host, ip, db, - user, tname, exact); + user, tname, exact, FALSE); } @@ -2610,7 +2619,11 @@ static int replace_column_table(GRANT_TA goto end; /* purecov: inspected */ } grant_column= new GRANT_COLUMN(column->column,privileges); - my_hash_insert(&g_t->hash_columns,(uchar*) grant_column); + if (my_hash_insert(&g_t->hash_columns,(uchar*) grant_column)) + { + result= -1; + goto end; + } } } @@ -3135,12 +3148,12 @@ int mysql_table_grant(THD *thd, TABLE_LI Str->user.str, table_name, rights, column_priv); - if (!grant_table) // end of memory + if (!grant_table || + my_hash_insert(&column_priv_hash,(uchar*) grant_table)) { result= TRUE; /* purecov: deadcode */ continue; /* purecov: deadcode */ } - my_hash_insert(&column_priv_hash,(uchar*) grant_table); } /* If revoke_grant, calculate the new column privilege for tables_priv */ @@ -3344,12 +3357,13 @@ bool mysql_routine_grant(THD *thd, TABLE grant_name= new GRANT_NAME(Str->host.str, db_name, Str->user.str, table_name, rights, TRUE); - if (!grant_name) + if (!grant_name || + my_hash_insert(is_proc ? + &proc_priv_hash : &func_priv_hash,(uchar*) grant_name)) { result= TRUE; continue; } - my_hash_insert(is_proc ? &proc_priv_hash : &func_priv_hash,(uchar*) grant_name); } if (replace_routine_table(thd, grant_name, tables[1].table, *Str, @@ -3452,6 +3466,13 @@ bool mysql_grant(THD *thd, const char *d result= TRUE; continue; } + /* + No User, but a password? + They did GRANT ... TO CURRENT_USER() IDENTIFIED BY ... ! + Get the current user, and shallow-copy the new password to them! + */ + if (!tmp_Str->user.str && tmp_Str->password.str) + Str->password= tmp_Str->password; if (replace_user_table(thd, tables[0].table, *Str, (!db ? rights : 0), revoke_grant, create_new_users, test(thd->variables.sql_mode & === modified file 'sql/sql_acl.h' --- a/sql/sql_acl.h 2009-05-29 13:37:54 +0000 +++ b/sql/sql_acl.h 2009-11-21 11:18:21 +0000 @@ -159,8 +159,7 @@ enum mysql_db_table_field MYSQL_DB_FIELD_COUNT }; -extern TABLE_FIELD_W_TYPE mysql_db_table_fields[]; -extern time_t mysql_db_table_last_check; +extern const TABLE_FIELD_DEF mysql_db_table_def; /* Classes */ === modified file 'sql/sql_base.cc' --- a/sql/sql_base.cc 2009-12-04 15:12:22 +0000 +++ b/sql/sql_base.cc 2010-01-15 15:27:55 +0000 @@ -2938,7 +2938,12 @@ TABLE *open_table(THD *thd, TABLE_LIST * DBUG_PRINT("info", ("inserting table '%s'.'%s' 0x%lx into the cache", table->s->db.str, table->s->table_name.str, (long) table)); - VOID(my_hash_insert(&open_cache,(uchar*) table)); + if (my_hash_insert(&open_cache,(uchar*) table)) + { + my_free(table, MYF(0)); + VOID(pthread_mutex_unlock(&LOCK_open)); + DBUG_RETURN(NULL); + } } check_unused(); // Debugging call === modified file 'sql/sql_cache.cc' --- a/sql/sql_cache.cc 2009-12-03 11:19:05 +0000 +++ b/sql/sql_cache.cc 2010-01-29 10:42:31 +0000 @@ -377,9 +377,11 @@ TODO list: static void debug_wait_for_kill(const char *info) { - DBUG_ENTER("debug_wait_for_kill"); const char *prev_info; THD *thd; + char buff[1024]; + DBUG_ENTER("debug_wait_for_kill"); + thd= current_thd; prev_info= thd->proc_info; thd->proc_info= info; @@ -387,8 +389,16 @@ static void debug_wait_for_kill(const ch while(!thd->killed) my_sleep(1000); thd->killed= THD::NOT_KILLED; + /* + Remove the set debug variable, to ensure we don't get stuck on it again + This is needed as for MyISAM, invalidate_table() may be called twice + (Once from mysql_delete() and once from mi_update_status()) + */ + sprintf(buff, "-d,%s", info); + DBUG_SET(buff); sql_print_information("Exit debug_wait_for_kill"); thd->proc_info= prev_info; + DBUG_VOID_RETURN; } @@ -421,12 +431,16 @@ TYPELIB query_cache_type_typelib= effect by another thread. This enables a quick path in execution to skip waits when the outcome is known. + @param use_timeout TRUE if the lock can abort because of a timeout. + + @note use_timeout is optional and default value is FALSE. + @return @retval FALSE An exclusive lock was taken @retval TRUE The locking attempt failed */ -bool Query_cache::try_lock(void) +bool Query_cache::try_lock(bool use_timeout) { bool interrupt= FALSE; DBUG_ENTER("Query_cache::try_lock"); @@ -456,7 +470,26 @@ bool Query_cache::try_lock(void) else { DBUG_ASSERT(m_cache_lock_status == Query_cache::LOCKED); - pthread_cond_wait(&COND_cache_status_changed, &structure_guard_mutex); + /* + To prevent send_result_to_client() and query_cache_insert() from + blocking execution for too long a timeout is put on the lock. + */ + if (use_timeout) + { + struct timespec waittime; + set_timespec_nsec(waittime,(ulong)(50000000L)); /* Wait for 50 msec */ + int res= pthread_cond_timedwait(&COND_cache_status_changed, + &structure_guard_mutex,&waittime); + if (res == ETIMEDOUT) + { + interrupt= TRUE; + break; + } + } + else + { + pthread_cond_wait(&COND_cache_status_changed, &structure_guard_mutex); + } } } pthread_mutex_unlock(&structure_guard_mutex); @@ -891,15 +924,18 @@ void query_cache_insert(NET *net, const void query_cache_abort(NET *net) { + THD *thd; DBUG_ENTER("query_cache_abort"); - THD *thd= current_thd; /* See the comment on double-check locking usage above. */ if (net->query_cache_query == 0) DBUG_VOID_RETURN; if (query_cache.try_lock()) + { + net->query_cache_query = 0; DBUG_VOID_RETURN; + } /* While we were waiting another thread might have changed the status @@ -909,6 +945,7 @@ void query_cache_abort(NET *net) net->query_cache_query); if (query_block) { + thd= current_thd; thd_proc_info(thd, "storing result in query cache"); DUMP(&query_cache); BLOCK_LOCK_WR(query_block); @@ -918,6 +955,7 @@ void query_cache_abort(NET *net) DBUG_EXECUTE("check_querycache",query_cache.check_integrity(1);); } + DBUG_ASSERT(!net->query_cache_query); query_cache.unlock(); DBUG_VOID_RETURN; } @@ -947,8 +985,12 @@ void query_cache_end_of_result(THD *thd) #endif if (query_cache.try_lock()) + { + thd->net.query_cache_query= 0; DBUG_VOID_RETURN; + } + /* thd->net.query_cache_query may have changed during resize */ query_block= ((Query_cache_block*) thd->net.query_cache_query); if (query_block) { @@ -974,8 +1016,8 @@ void query_cache_end_of_result(THD *thd) to this function. In the release version that query should be ignored and removed from QC. */ - DBUG_ASSERT(0); query_cache.free_query(query_block); + thd->net.query_cache_query= 0; query_cache.unlock(); DBUG_VOID_RETURN; } @@ -1190,8 +1232,14 @@ def_week_frmt: %lu, in_trans: %d, autoco A table- or a full flush operation can potentially take a long time to finish. We choose not to wait for them and skip caching statements instead. + + In case the wait time can't be determined there is an upper limit which + causes try_lock() to abort with a time out. + + The 'TRUE' parameter indicate that the lock is allowed to timeout + */ - if (try_lock()) + if (try_lock(TRUE)) DBUG_VOID_RETURN; if (query_cache_size == 0) { @@ -1306,8 +1354,8 @@ end: to the user. RESULTS - 1 Query was not cached. - 0 The query was cached and user was sent the result. + 0 Query was not cached. + 1 The query was cached and user was sent the result. -1 The query was cached but we didn't have rights to use it. No error is sent to the client yet. @@ -1388,8 +1436,10 @@ Query_cache::send_result_to_client(THD * Try to obtain an exclusive lock on the query cache. If the cache is disabled or if a full cache flush is in progress, the attempt to get the lock is aborted. + + The 'TRUE' parameter indicate that the lock is allowed to timeout */ - if (try_lock()) + if (try_lock(TRUE)) goto err; if (query_cache_size == 0) === modified file 'sql/sql_cache.h' --- a/sql/sql_cache.h 2009-06-16 08:34:47 +0000 +++ b/sql/sql_cache.h 2009-11-20 12:49:06 +0000 @@ -485,7 +485,7 @@ protected: const char *name); my_bool in_blocks(Query_cache_block * point); - bool try_lock(void); + bool try_lock(bool use_timeout= FALSE); void lock(void); void lock_and_suspend(void); void unlock(void); === modified file 'sql/sql_class.cc' --- a/sql/sql_class.cc 2009-12-04 15:12:22 +0000 +++ b/sql/sql_class.cc 2010-01-15 15:27:55 +0000 @@ -379,6 +379,8 @@ char *thd_security_context(THD *thd, cha str.append(proc_info); } + pthread_mutex_lock(&thd->LOCK_thd_data); + if (thd->query()) { if (max_query_len < 1) @@ -388,6 +390,9 @@ char *thd_security_context(THD *thd, cha str.append('\n'); str.append(thd->query(), len); } + + pthread_mutex_unlock(&thd->LOCK_thd_data); + if (str.c_ptr_safe() == buffer) return buffer; === modified file 'sql/sql_connect.cc' --- a/sql/sql_connect.cc 2009-12-03 11:19:05 +0000 +++ b/sql/sql_connect.cc 2010-01-29 10:42:31 +0000 @@ -798,6 +798,7 @@ static int check_connection(THD *thd) DBUG_PRINT("info", ("client capabilities: %lu", thd->client_capabilities)); if (thd->client_capabilities & CLIENT_SSL) { + char error_string[1024]; /* Do the SSL layering. */ if (!ssl_acceptor_fd) { @@ -806,7 +807,7 @@ static int check_connection(THD *thd) return 1; } DBUG_PRINT("info", ("IO layer change in progress...")); - if (sslaccept(ssl_acceptor_fd, net->vio, net->read_timeout)) + if (sslaccept(ssl_acceptor_fd, net->vio, net->read_timeout, error_string)) { DBUG_PRINT("error", ("Failed to accept new SSL connection")); inc_host_errors(&thd->remote.sin_addr); === modified file 'sql/sql_delete.cc' --- a/sql/sql_delete.cc 2009-12-03 11:34:11 +0000 +++ b/sql/sql_delete.cc 2010-01-15 15:27:55 +0000 @@ -426,7 +426,8 @@ cleanup: } DBUG_ASSERT(transactional_table || !deleted || thd->transaction.stmt.modified_non_trans_table); free_underlaid_joins(thd, select_lex); - if (error < 0 || (thd->lex->ignore && !thd->is_fatal_error)) + if (error < 0 || + (thd->lex->ignore && !thd->is_error() && !thd->is_fatal_error)) { /* If a TRUNCATE TABLE was issued, the number of rows should be reported as @@ -1089,6 +1090,7 @@ bool mysql_truncate(THD *thd, TABLE_LIST TABLE *table; bool error; uint path_length; + bool is_temporary_table= false; DBUG_ENTER("mysql_truncate"); bzero((char*) &create_info,sizeof(create_info)); @@ -1101,6 +1103,8 @@ bool mysql_truncate(THD *thd, TABLE_LIST { TABLE_SHARE *share= table->s; handlerton *table_type= share->db_type(); + is_temporary_table= true; + if (!ha_check_storage_engine_flag(table_type, HTON_CAN_RECREATE)) goto trunc_by_del; @@ -1166,11 +1170,9 @@ end: { if (!error) { - /* - TRUNCATE must always be statement-based binlogged (not row-based) so - we don't test current_stmt_binlog_row_based. - */ - write_bin_log(thd, TRUE, thd->query(), thd->query_length()); + /* In RBR, the statement is not binlogged if the table is temporary. */ + if (!is_temporary_table || !thd->current_stmt_binlog_row_based) + write_bin_log(thd, TRUE, thd->query(), thd->query_length()); my_ok(thd); // This should return record count } VOID(pthread_mutex_lock(&LOCK_open)); === modified file 'sql/sql_insert.cc' --- a/sql/sql_insert.cc 2009-12-04 15:12:22 +0000 +++ b/sql/sql_insert.cc 2010-01-15 15:27:55 +0000 @@ -521,6 +521,22 @@ bool open_and_lock_for_insert_delayed(TH DBUG_ENTER("open_and_lock_for_insert_delayed"); #ifndef EMBEDDED_LIBRARY + if (thd->locked_tables && thd->global_read_lock) + { + /* + If this connection has the global read lock, the handler thread + will not be able to lock the table. It will wait for the global + read lock to go away, but this will never happen since the + connection thread will be stuck waiting for the handler thread + to open and lock the table. + If we are not in locked tables mode, INSERT will seek protection + against the global read lock (and fail), thus we will only get + to this point in locked tables mode. + */ + my_error(ER_CANT_UPDATE_WITH_READLOCK, MYF(0)); + DBUG_RETURN(TRUE); + } + if (delayed_get_table(thd, table_list)) DBUG_RETURN(TRUE); === modified file 'sql/sql_load.cc' --- a/sql/sql_load.cc 2009-12-03 11:19:05 +0000 +++ b/sql/sql_load.cc 2010-01-15 15:27:55 +0000 @@ -304,7 +304,8 @@ int mysql_load(THD *thd,sql_exchange *ex else { (void) fn_format(name, ex->file_name, mysql_real_data_home, "", - MY_RELATIVE_PATH | MY_UNPACK_FILENAME); + MY_RELATIVE_PATH | MY_UNPACK_FILENAME | + MY_RETURN_REAL_PATH); #if !defined(__WIN__) && ! defined(__NETWARE__) MY_STAT stat_info; if (!my_stat(name,&stat_info,MYF(MY_WME))) @@ -347,12 +348,16 @@ int mysql_load(THD *thd,sql_exchange *ex DBUG_ASSERT(FALSE); #endif } - else if (opt_secure_file_priv && - strncmp(opt_secure_file_priv, name, strlen(opt_secure_file_priv))) + else if (opt_secure_file_priv) { - /* Read only allowed from within dir specified by secure_file_priv */ - my_error(ER_OPTION_PREVENTS_STATEMENT, MYF(0), "--secure-file-priv"); - DBUG_RETURN(TRUE); + char secure_file_real_path[FN_REFLEN]; + (void) my_realpath(secure_file_real_path, opt_secure_file_priv, 0); + if (strncmp(secure_file_real_path, name, strlen(secure_file_real_path))) + { + /* Read only allowed from within dir specified by secure_file_priv */ + my_error(ER_OPTION_PREVENTS_STATEMENT, MYF(0), "--secure-file-priv"); + DBUG_RETURN(TRUE); + } } } === modified file 'sql/sql_parse.cc' --- a/sql/sql_parse.cc 2009-12-03 11:34:11 +0000 +++ b/sql/sql_parse.cc 2010-01-29 10:42:31 +0000 @@ -1252,12 +1252,14 @@ bool dispatch_command(enum enum_server_c ha_maria::implicit_commit(thd, FALSE); #endif - net_end_statement(thd); - query_cache_end_of_result(thd); /* Multiple queries exits, execute them individually */ close_thread_tables(thd); + + net_end_statement(thd); + query_cache_end_of_result(thd); + ulong length= (ulong)(packet_end - beginning_of_next_stmt); log_slow_statement(thd); @@ -1645,13 +1647,22 @@ bool dispatch_command(enum enum_server_c ha_maria::implicit_commit(thd, FALSE); #endif - net_end_statement(thd); - query_cache_end_of_result(thd); - + if (!(sql_command_flags[thd->lex->sql_command] & CF_CHANGES_DATA)) + { + /* No changes in data; We can send ok at once to the client */ + net_end_statement(thd); + query_cache_end_of_result(thd); + } thd->proc_info= "closing tables"; /* Free tables */ close_thread_tables(thd); + if (sql_command_flags[thd->lex->sql_command] & CF_CHANGES_DATA) + { + net_end_statement(thd); + query_cache_end_of_result(thd); + } + log_slow_statement(thd); thd_proc_info(thd, "cleaning up"); @@ -7632,6 +7643,9 @@ void get_default_definer(THD *thd, LEX_U definer->host.str= (char *) sctx->priv_host; definer->host.length= strlen(definer->host.str); + + definer->password.str= NULL; + definer->password.length= 0; } @@ -7683,6 +7697,8 @@ LEX_USER *create_definer(THD *thd, LEX_S definer->user= *user_name; definer->host= *host_name; + definer->password.str= NULL; + definer->password.length= 0; return definer; } === modified file 'sql/sql_partition.cc' --- a/sql/sql_partition.cc 2009-12-03 11:19:05 +0000 +++ b/sql/sql_partition.cc 2010-01-15 15:27:55 +0000 @@ -196,26 +196,27 @@ bool partition_default_handling(TABLE *t { DBUG_ENTER("partition_default_handling"); - if (part_info->use_default_no_partitions) + if (!is_create_table_ind) { - if (!is_create_table_ind && - table->file->get_no_parts(normalized_path, &part_info->no_parts)) + if (part_info->use_default_no_partitions) { - DBUG_RETURN(TRUE); + if (table->file->get_no_parts(normalized_path, &part_info->no_parts)) + { + DBUG_RETURN(TRUE); + } } - } - else if (part_info->is_sub_partitioned() && - part_info->use_default_no_subpartitions) - { - uint no_parts; - if (!is_create_table_ind && - (table->file->get_no_parts(normalized_path, &no_parts))) + else if (part_info->is_sub_partitioned() && + part_info->use_default_no_subpartitions) { - DBUG_RETURN(TRUE); + uint no_parts; + if (table->file->get_no_parts(normalized_path, &no_parts)) + { + DBUG_RETURN(TRUE); + } + DBUG_ASSERT(part_info->no_parts > 0); + DBUG_ASSERT((no_parts % part_info->no_parts) == 0); + part_info->no_subparts= no_parts / part_info->no_parts; } - DBUG_ASSERT(part_info->no_parts > 0); - part_info->no_subparts= no_parts / part_info->no_parts; - DBUG_ASSERT((no_parts % part_info->no_parts) == 0); } part_info->set_up_defaults_for_partitioning(table->file, (ulonglong)0, (uint)0); @@ -905,6 +906,8 @@ bool fix_fields_part_func(THD *thd, Item char* db_name; char db_name_string[FN_REFLEN]; bool save_use_only_table_context; + uint8 saved_full_group_by_flag; + nesting_map saved_allow_sum_func; DBUG_ENTER("fix_fields_part_func"); if (part_info->fixed) @@ -974,9 +977,19 @@ bool fix_fields_part_func(THD *thd, Item save_use_only_table_context= thd->lex->use_only_table_context; thd->lex->use_only_table_context= TRUE; thd->lex->current_select->cur_pos_in_select_list= UNDEF_POS; + saved_full_group_by_flag= thd->lex->current_select->full_group_by_flag; + saved_allow_sum_func= thd->lex->allow_sum_func; + thd->lex->allow_sum_func= 0; error= func_expr->fix_fields(thd, (Item**)&func_expr); + /* + Restore full_group_by_flag and allow_sum_func, + fix_fields should not affect mysql_select later, see Bug#46923. + */ + thd->lex->current_select->full_group_by_flag= saved_full_group_by_flag; + thd->lex->allow_sum_func= saved_allow_sum_func; + thd->lex->use_only_table_context= save_use_only_table_context; context->table_list= save_table_list; @@ -1679,7 +1692,7 @@ bool fix_partition_func(THD *thd, TABLE if (((part_info->part_type != HASH_PARTITION || part_info->list_of_part_fields == FALSE) && check_part_func_fields(part_info->part_field_array, TRUE)) || - (part_info->list_of_part_fields == FALSE && + (part_info->list_of_subpart_fields == FALSE && part_info->is_sub_partitioned() && check_part_func_fields(part_info->subpart_field_array, TRUE))) { === modified file 'sql/sql_profile.cc' --- a/sql/sql_profile.cc 2009-10-15 21:38:29 +0000 +++ b/sql/sql_profile.cc 2010-01-29 18:42:22 +0000 @@ -131,6 +131,23 @@ int make_profile_table_for_show(THD *thd #define RUSAGE_USEC(tv) ((tv).tv_sec*1000*1000 + (tv).tv_usec) #define RUSAGE_DIFF_USEC(tv1, tv2) (RUSAGE_USEC((tv1))-RUSAGE_USEC((tv2))) +#ifdef __WIN__ +inline ULONGLONG FileTimeToQuadWord(FILETIME *ft) +{ + ULONGLONG nrv = 0; + nrv |= ft->dwHighDateTime; + nrv <<= 32; + nrv |= ft->dwLowDateTime; + return nrv; +} + + +// Get time difference between to FILETIME objects in seconds. +inline double GetTimeDiffInSeconds(FILETIME *a, FILETIME *b) +{ + return ((FileTimeToQuadWord(a) - FileTimeToQuadWord(b)) / 1e7); +} +#endif /* __WIN__ */ PROF_MEASUREMENT::PROF_MEASUREMENT(QUERY_PROFILE *profile_arg, const char *status_arg) @@ -221,6 +238,11 @@ void PROF_MEASUREMENT::collect() time_usecs= (double) my_getsystime() / 10.0; /* 1 sec was 1e7, now is 1e6 */ #ifdef HAVE_GETRUSAGE getrusage(RUSAGE_SELF, &rusage); +#elif defined(__WIN__) + FILETIME ftDummy; + GetProcessTimes(GetCurrentProcess(), &ftDummy, &ftDummy, &ftKernel, &ftUser); + GetProcessIoCounters(GetCurrentProcess(), &io_count); + GetProcessMemoryInfo(GetCurrentProcess(), &mem_count, sizeof(mem_count)); #endif } @@ -590,6 +612,23 @@ int PROFILING::fill_statistics_info(THD table->field[5]->store_decimal(&cpu_stime_decimal); table->field[4]->set_notnull(); table->field[5]->set_notnull(); +#elif defined(__WIN__) + my_decimal cpu_utime_decimal, cpu_stime_decimal; + + double2my_decimal(E_DEC_FATAL_ERROR, + GetTimeDiffInSeconds(&entry->ftUser, + &previous->ftUser), + &cpu_utime_decimal); + double2my_decimal(E_DEC_FATAL_ERROR, + GetTimeDiffInSeconds(&entry->ftKernel, + &previous->ftKernel), + &cpu_stime_decimal); + + // Store the result. + table->field[4]->store_decimal(&cpu_utime_decimal); + table->field[5]->store_decimal(&cpu_stime_decimal); + table->field[4]->set_notnull(); + table->field[5]->set_notnull(); #else /* TODO: Add CPU-usage info for non-BSD systems */ #endif @@ -612,6 +651,17 @@ int PROFILING::fill_statistics_info(THD table->field[9]->store((uint32)(entry->rusage.ru_oublock - previous->rusage.ru_oublock)); table->field[9]->set_notnull(); +#elif defined(__WIN__) + ULONGLONG reads_delta = entry->io_count.ReadOperationCount - + previous->io_count.ReadOperationCount; + ULONGLONG writes_delta = entry->io_count.WriteOperationCount - + previous->io_count.WriteOperationCount; + + table->field[8]->store((uint32)reads_delta); + table->field[8]->set_notnull(); + + table->field[9]->store((uint32)writes_delta); + table->field[9]->set_notnull(); #else /* TODO: Add block IO info for non-BSD systems */ #endif @@ -634,6 +684,13 @@ int PROFILING::fill_statistics_info(THD table->field[13]->store((uint32)(entry->rusage.ru_minflt - previous->rusage.ru_minflt), true); table->field[13]->set_notnull(); +#elif defined(__WIN__) + /* Windows APIs don't easily distinguish between hard and soft page + faults, so we just fill the 'major' column and leave the second NULL. + */ + table->field[12]->store((uint32)(entry->mem_count.PageFaultCount - + previous->mem_count.PageFaultCount), true); + table->field[12]->set_notnull(); #else /* TODO: Add page fault info for non-BSD systems */ #endif === modified file 'sql/sql_profile.h' --- a/sql/sql_profile.h 2009-09-07 20:50:10 +0000 +++ b/sql/sql_profile.h 2010-01-29 18:42:22 +0000 @@ -36,6 +36,10 @@ int make_profile_table_for_show(THD *thd #if defined(ENABLED_PROFILING) && defined(COMMUNITY_SERVER) #include "mysql_priv.h" +#ifdef __WIN__ +#include <psapi.h> +#endif + #ifdef HAVE_SYS_RESOURCE_H #include <sys/resource.h> #endif @@ -165,6 +169,10 @@ private: char *status; #ifdef HAVE_GETRUSAGE struct rusage rusage; +#elif defined(__WIN__) + FILETIME ftKernel, ftUser; + IO_COUNTERS io_count; + PROCESS_MEMORY_COUNTERS mem_count; #endif char *function; === modified file 'sql/sql_select.cc' --- a/sql/sql_select.cc 2009-12-03 11:34:11 +0000 +++ b/sql/sql_select.cc 2010-01-15 15:27:55 +0000 @@ -992,14 +992,20 @@ JOIN::optimize() DBUG_RETURN(1); } - if (select_lex->olap == ROLLUP_TYPE && rollup_process_const_fields()) + if (rollup.state != ROLLUP::STATE_NONE) { - DBUG_PRINT("error", ("Error: rollup_process_fields() failed")); - DBUG_RETURN(1); + if (rollup_process_const_fields()) + { + DBUG_PRINT("error", ("Error: rollup_process_fields() failed")); + DBUG_RETURN(1); + } + } + else + { + /* Remove distinct if only const tables */ + select_distinct= select_distinct && (const_tables != tables); } - /* Remove distinct if only const tables */ - select_distinct= select_distinct && (const_tables != tables); thd_proc_info(thd, "preparing"); if (result->initialize_tables(this)) { @@ -1298,11 +1304,14 @@ JOIN::optimize() - We are using an ORDER BY or GROUP BY on fields not in the first table - We are using different ORDER BY and GROUP BY orders - The user wants us to buffer the result. + When the WITH ROLLUP modifier is present, we cannot skip temporary table + creation for the DISTINCT clause just because there are only const tables. */ - need_tmp= (const_tables != tables && + need_tmp= ((const_tables != tables && ((select_distinct || !simple_order || !simple_group) || (group_list && order) || - test(select_options & OPTION_BUFFER_RESULT))); + test(select_options & OPTION_BUFFER_RESULT))) || + (rollup.state != ROLLUP::STATE_NONE && select_distinct)); // No cache for MATCH make_join_readinfo(this, @@ -2144,17 +2153,13 @@ JOIN::exec() DBUG_VOID_RETURN; if (!curr_table->select->cond) curr_table->select->cond= sort_table_cond; - else // This should never happen + else { if (!(curr_table->select->cond= new Item_cond_and(curr_table->select->cond, sort_table_cond))) DBUG_VOID_RETURN; - /* - Item_cond_and do not need fix_fields for execution, its parameters - are fixed or do not need fix_fields, too - */ - curr_table->select->cond->quick_fix_field(); + curr_table->select->cond->fix_fields(thd, 0); } curr_table->select_cond= curr_table->select->cond; curr_table->select_cond->top_level_item(); @@ -6565,6 +6570,56 @@ void rr_unlock_row(st_join_table *tab) +/** + Pick the appropriate access method functions + + Sets the functions for the selected table access method + + @param tab Table reference to put access method +*/ + +static void +pick_table_access_method(JOIN_TAB *tab) +{ + switch (tab->type) + { + case JT_REF: + tab->read_first_record= join_read_always_key; + tab->read_record.read_record= join_read_next_same; + break; + + case JT_REF_OR_NULL: + tab->read_first_record= join_read_always_key_or_null; + tab->read_record.read_record= join_read_next_same_or_null; + break; + + case JT_CONST: + tab->read_first_record= join_read_const; + tab->read_record.read_record= join_no_more_records; + break; + + case JT_EQ_REF: + tab->read_first_record= join_read_key; + tab->read_record.read_record= join_no_more_records; + break; + + case JT_FT: + tab->read_first_record= join_ft_read_first; + tab->read_record.read_record= join_ft_read_next; + break; + + case JT_SYSTEM: + tab->read_first_record= join_read_system; + tab->read_record.read_record= join_no_more_records; + break; + + /* keep gcc happy */ + default: + break; + } +} + + static void make_join_readinfo(JOIN *join, ulonglong options) { @@ -6599,45 +6654,15 @@ make_join_readinfo(JOIN *join, ulonglong tab->sorted= sorted; sorted= 0; // only first must be sorted + table->status=STATUS_NO_RECORD; + pick_table_access_method (tab); + switch (tab->type) { - case JT_SYSTEM: // Only happens with left join - table->status=STATUS_NO_RECORD; - tab->read_first_record= join_read_system; - tab->read_record.read_record= join_no_more_records; - break; - case JT_CONST: // Only happens with left join - table->status=STATUS_NO_RECORD; - tab->read_first_record= join_read_const; - tab->read_record.read_record= join_no_more_records; - if (table->covering_keys.is_set(tab->ref.key) && - !table->no_keyread) - { - table->key_read=1; - table->file->extra(HA_EXTRA_KEYREAD); - } - break; case JT_EQ_REF: - table->status=STATUS_NO_RECORD; - if (tab->select) - { - delete tab->select->quick; - tab->select->quick=0; - } - delete tab->quick; - tab->quick=0; - tab->read_first_record= join_read_key; tab->read_record.unlock_row= join_read_key_unlock_row; - tab->read_record.read_record= join_no_more_records; - if (table->covering_keys.is_set(tab->ref.key) && - !table->no_keyread) - { - table->key_read=1; - table->file->extra(HA_EXTRA_KEYREAD); - } - break; + /* fall through */ case JT_REF_OR_NULL: case JT_REF: - table->status=STATUS_NO_RECORD; if (tab->select) { delete tab->select->quick; @@ -6645,34 +6670,20 @@ make_join_readinfo(JOIN *join, ulonglong } delete tab->quick; tab->quick=0; + /* fall through */ + case JT_CONST: // Only happens with left join if (table->covering_keys.is_set(tab->ref.key) && !table->no_keyread) { table->key_read=1; table->file->extra(HA_EXTRA_KEYREAD); } - if (tab->type == JT_REF) - { - tab->read_first_record= join_read_always_key; - tab->read_record.read_record= join_read_next_same; - } - else - { - tab->read_first_record= join_read_always_key_or_null; - tab->read_record.read_record= join_read_next_same_or_null; - } - break; - case JT_FT: - table->status=STATUS_NO_RECORD; - tab->read_first_record= join_ft_read_first; - tab->read_record.read_record= join_ft_read_next; break; case JT_ALL: /* If previous table use cache If the incoming data set is already sorted don't use cache. */ - table->status=STATUS_NO_RECORD; if (i != join->const_tables && !(options & SELECT_NO_JOIN_CACHE) && tab->use_quick != 2 && !tab->first_inner && !ordered_set) { @@ -6758,6 +6769,9 @@ make_join_readinfo(JOIN *join, ulonglong } } break; + case JT_FT: + case JT_SYSTEM: + break; default: DBUG_PRINT("error",("Table type %d found",tab->type)); /* purecov: deadcode */ break; /* purecov: deadcode */ @@ -7909,12 +7923,12 @@ static COND *build_equal_items_for_cond( { item_equal->fix_length_and_dec(); item_equal->update_used_tables(); + set_if_bigger(thd->lex->current_select->max_equal_elems, + item_equal->members()); + return item_equal; } - else - item_equal= (Item_equal *) eq_list.pop(); - set_if_bigger(thd->lex->current_select->max_equal_elems, - item_equal->members()); - return item_equal; + + return eq_list.pop(); } else { @@ -9552,47 +9566,8 @@ static Field *create_tmp_field_from_item new_field->set_derivation(item->collation.derivation); break; case DECIMAL_RESULT: - { - uint8 dec= item->decimals; - uint8 intg= ((Item_decimal *) item)->decimal_precision() - dec; - uint32 len= item->max_length; - - /* - Trying to put too many digits overall in a DECIMAL(prec,dec) - will always throw a warning. We must limit dec to - DECIMAL_MAX_SCALE however to prevent an assert() later. - */ - - if (dec > 0) - { - signed int overflow; - - dec= min(dec, DECIMAL_MAX_SCALE); - - /* - If the value still overflows the field with the corrected dec, - we'll throw out decimals rather than integers. This is still - bad and of course throws a truncation warning. - +1: for decimal point - */ - - const int required_length= - my_decimal_precision_to_length(intg + dec, dec, - item->unsigned_flag); - - overflow= required_length - len; - - if (overflow > 0) - dec= max(0, dec - overflow); // too long, discard fract - else - /* Corrected value fits. */ - len= required_length; - } - - new_field= new Field_new_decimal(len, maybe_null, item->name, - dec, item->unsigned_flag); + new_field= Field_new_decimal::create_from_item(item); break; - } case ROW_RESULT: default: // This case should never be choosen @@ -13496,6 +13471,8 @@ test_if_skip_sort_order(JOIN_TAB *tab,OR if (create_ref_for_key(tab->join, tab, keyuse, tab->join->const_table_map)) DBUG_RETURN(0); + + pick_table_access_method(tab); } else { @@ -14305,7 +14282,10 @@ static int remove_dup_with_hash_index(TH goto err; } else - (void) my_hash_insert(&hash, org_key_pos); + { + if (my_hash_insert(&hash, org_key_pos)) + goto err; + } key_pos+=extra_length; } my_free((char*) key_buffer,MYF(0)); === modified file 'sql/sql_show.cc' --- a/sql/sql_show.cc 2009-12-03 11:34:11 +0000 +++ b/sql/sql_show.cc 2010-01-15 15:27:55 +0000 @@ -721,7 +721,7 @@ mysqld_show_create(THD *thd, TABLE_LIST thd->push_internal_handler(&view_error_suppressor); bool error= open_normal_and_derived_tables(thd, table_list, 0); thd->pop_internal_handler(); - if (error && thd->main_da.is_error()) + if (error && (thd->killed || thd->main_da.is_error())) DBUG_RETURN(TRUE); } === modified file 'sql/sql_table.cc' --- a/sql/sql_table.cc 2009-12-03 11:19:05 +0000 +++ b/sql/sql_table.cc 2010-01-15 15:27:55 +0000 @@ -5428,12 +5428,20 @@ binlog: } VOID(pthread_mutex_unlock(&LOCK_open)); - IF_DBUG(int result=) - store_create_info(thd, table, &query, - create_info, FALSE /* show_database */); + /* + The condition avoids a crash as described in BUG#48506. Other + binlogging problems related to CREATE TABLE IF NOT EXISTS LIKE + when the existing object is a view will be solved by BUG 47442. + */ + if (!table->view) + { + IF_DBUG(int result=) + store_create_info(thd, table, &query, + create_info, FALSE /* show_database */); - DBUG_ASSERT(result == 0); // store_create_info() always return 0 - write_bin_log(thd, TRUE, query.ptr(), query.length()); + DBUG_ASSERT(result == 0); // store_create_info() always return 0 + write_bin_log(thd, TRUE, query.ptr(), query.length()); + } } else // Case 1 write_bin_log(thd, TRUE, thd->query(), thd->query_length()); === modified file 'sql/sql_yacc.yy' --- a/sql/sql_yacc.yy 2009-12-03 11:19:05 +0000 +++ b/sql/sql_yacc.yy 2010-01-17 17:22:46 +0000 @@ -389,6 +389,138 @@ void case_stmt_action_end_case(LEX *lex, lex->sphead->do_cont_backpatch(); } + +static bool +find_sys_var_null_base(THD *thd, struct sys_var_with_base *tmp) +{ + tmp->var= find_sys_var(thd, tmp->base_name.str, tmp->base_name.length); + + if (tmp->var == NULL) + my_error(ER_UNKNOWN_SYSTEM_VARIABLE, MYF(0), tmp->base_name.str); + else + tmp->base_name= null_lex_str; + + return thd->is_error(); +} + + +/** + Helper action for a SET statement. + Used to push a system variable into the assignment list. + + @param thd the current thread + @param tmp the system variable with base name + @param var_type the scope of the variable + @param val the value being assigned to the variable + + @return TRUE if error, FALSE otherwise. +*/ + +static bool +set_system_variable(THD *thd, struct sys_var_with_base *tmp, + enum enum_var_type var_type, Item *val) +{ + set_var *var; + LEX *lex= thd->lex; + + /* No AUTOCOMMIT from a stored function or trigger. */ + if (lex->spcont && tmp->var == &sys_autocommit) + lex->sphead->m_flags|= sp_head::HAS_SET_AUTOCOMMIT_STMT; + + if (! (var= new set_var(var_type, tmp->var, &tmp->base_name, val))) + return TRUE; + + return lex->var_list.push_back(var); +} + + +/** + Helper action for a SET statement. + Used to push a SP local variable into the assignment list. + + @param thd the current thread + @param var_type the SP local variable + @param val the value being assigned to the variable + + @return TRUE if error, FALSE otherwise. +*/ + +static bool +set_local_variable(THD *thd, sp_variable_t *spv, Item *val) +{ + Item *it; + LEX *lex= thd->lex; + sp_instr_set *sp_set; + + if (val) + it= val; + else if (spv->dflt) + it= spv->dflt; + else + { + it= new (thd->mem_root) Item_null(); + if (it == NULL) + return TRUE; + } + + sp_set= new sp_instr_set(lex->sphead->instructions(), lex->spcont, + spv->offset, it, spv->type, lex, TRUE); + + return (sp_set == NULL || lex->sphead->add_instr(sp_set)); +} + + +/** + Helper action for a SET statement. + Used to SET a field of NEW row. + + @param thd the current thread + @param name the field name + @param val the value being assigned to the row + + @return TRUE if error, FALSE otherwise. +*/ + +static bool +set_trigger_new_row(THD *thd, LEX_STRING *name, Item *val) +{ + LEX *lex= thd->lex; + Item_trigger_field *trg_fld; + sp_instr_set_trigger_field *sp_fld; + + /* QQ: Shouldn't this be field's default value ? */ + if (! val) + val= new Item_null(); + + DBUG_ASSERT(lex->trg_chistics.action_time == TRG_ACTION_BEFORE && + (lex->trg_chistics.event == TRG_EVENT_INSERT || + lex->trg_chistics.event == TRG_EVENT_UPDATE)); + + trg_fld= new (thd->mem_root) + Item_trigger_field(lex->current_context(), + Item_trigger_field::NEW_ROW, + name->str, UPDATE_ACL, FALSE); + + if (trg_fld == NULL) + return TRUE; + + sp_fld= new sp_instr_set_trigger_field(lex->sphead->instructions(), + lex->spcont, trg_fld, val, lex); + + if (sp_fld == NULL) + return TRUE; + + /* + Let us add this item to list of all Item_trigger_field + objects in trigger. + */ + lex->trg_table_fields.link_in_list((uchar *) trg_fld, + (uchar **) &trg_fld->next_trg_field); + + return lex->sphead->add_instr(sp_fld); +} + + /** Helper to resolve the SQL:2003 Syntax exception 1) in <in predicate>. See SQL:2003, Part 2, section 8.4 <in predicate>, Note 184, page 383. @@ -2335,8 +2467,8 @@ sp_decl: } pctx->declare_var_boundary(0); - lex->sphead->restore_lex(YYTHD); - + if (lex->sphead->restore_lex(YYTHD)) + MYSQL_YYABORT; $$.vars= $2; $$.conds= $$.hndlrs= $$.curs= 0; } @@ -2446,7 +2578,8 @@ sp_cursor_stmt: } lex->sp_lex_in_use= TRUE; $$= lex; - lex->sphead->restore_lex(YYTHD); + if (lex->sphead->restore_lex(YYTHD)) + MYSQL_YYABORT; } ; @@ -2665,7 +2798,8 @@ sp_proc_stmt_statement: sp->add_instr(i)) MYSQL_YYABORT; } - sp->restore_lex(thd); + if (sp->restore_lex(thd)) + MYSQL_YYABORT; } ; @@ -2693,7 +2827,8 @@ sp_proc_stmt_return: MYSQL_YYABORT; sp->m_flags|= sp_head::HAS_RETURN; } - sp->restore_lex(YYTHD); + if (sp->restore_lex(YYTHD)) + MYSQL_YYABORT; } ; @@ -2933,7 +3068,8 @@ sp_if: sp->add_cont_backpatch(i) || sp->add_instr(i)) MYSQL_YYABORT; - sp->restore_lex(YYTHD); + if (sp->restore_lex(YYTHD)) + MYSQL_YYABORT; } sp_proc_stmts1 { @@ -2979,7 +3115,9 @@ simple_case_stmt: if (case_stmt_action_expr(lex, $3)) MYSQL_YYABORT; - lex->sphead->restore_lex(YYTHD); /* For expr $3 */ + /* For expr $3 */ + if (lex->sphead->restore_lex(YYTHD)) + MYSQL_YYABORT; } simple_when_clause_list else_clause_opt @@ -3029,7 +3167,9 @@ simple_when_clause: LEX *lex= Lex; if (case_stmt_action_when(lex, $3, true)) MYSQL_YYABORT; - lex->sphead->restore_lex(YYTHD); /* For expr $3 */ + /* For expr $3 */ + if (lex->sphead->restore_lex(YYTHD)) + MYSQL_YYABORT; } THEN_SYM sp_proc_stmts1 @@ -3050,7 +3190,9 @@ searched_when_clause: LEX *lex= Lex; if (case_stmt_action_when(lex, $3, false)) MYSQL_YYABORT; - lex->sphead->restore_lex(YYTHD); /* For expr $3 */ + /* For expr $3 */ + if (lex->sphead->restore_lex(YYTHD)) + MYSQL_YYABORT; } THEN_SYM sp_proc_stmts1 @@ -3227,7 +3369,8 @@ sp_unlabeled_control: sp->new_cont_backpatch(i) || sp->add_instr(i)) MYSQL_YYABORT; - sp->restore_lex(YYTHD); + if (sp->restore_lex(YYTHD)) + MYSQL_YYABORT; } sp_proc_stmts1 END WHILE_SYM { @@ -3253,7 +3396,8 @@ sp_unlabeled_control: if (i == NULL || lex->sphead->add_instr(i)) MYSQL_YYABORT; - lex->sphead->restore_lex(YYTHD); + if (lex->sphead->restore_lex(YYTHD)) + MYSQL_YYABORT; /* We can shortcut the cont_backpatch here */ i->m_cont_dest= ip+1; } @@ -7539,6 +7683,14 @@ function_call_nonkeyword: } | SYSDATE optional_braces { + /* + Unlike other time-related functions, SYSDATE() is + replication-unsafe because it is not affected by the + TIMESTAMP variable. It is unsafe even if + sysdate_is_now=1, because the slave may have + sysdate_is_now=0. + */ + Lex->set_stmt_unsafe(); if (global_system_variables.sysdate_is_now == 0) $$= new (YYTHD->mem_root) Item_func_sysdate_local(); else @@ -9483,16 +9635,12 @@ replace: insert_lock_option: /* empty */ { -#ifdef HAVE_QUERY_CACHE /* - If it is SP we do not allow insert optimisation whan result of + If it is SP we do not allow insert optimisation when result of insert visible only after the table unlocking but everyone can read table. */ $$= (Lex->sphead ? TL_WRITE_DEFAULT : TL_WRITE_CONCURRENT_INSERT); -#else - $$= TL_WRITE_CONCURRENT_INSERT; -#endif } | LOW_PRIORITY { $$= TL_WRITE_LOW_PRIORITY; } | DELAYED_SYM { $$= TL_WRITE_DELAYED; } @@ -10515,15 +10663,11 @@ load_data_lock: /* empty */ { $$= TL_WRITE_DEFAULT; } | CONCURRENT { -#ifdef HAVE_QUERY_CACHE /* - Ignore this option in SP to avoid problem with query cache + Ignore this option in SP to avoid problem with query cache and + triggers with non default priority locks */ - if (Lex->sphead != 0) - $$= TL_WRITE_DEFAULT; - else -#endif - $$= TL_WRITE_CONCURRENT_INSERT; + $$= (Lex->sphead ? TL_WRITE_DEFAULT : TL_WRITE_CONCURRENT_INSERT); } | LOW_PRIORITY { $$= TL_WRITE_LOW_PRIORITY; } ; @@ -11791,7 +11935,8 @@ option_type_value: if (sp->add_instr(i)) MYSQL_YYABORT; } - lex->sphead->restore_lex(thd); + if (lex->sphead->restore_lex(thd)) + MYSQL_YYABORT; } } ; @@ -11831,98 +11976,42 @@ sys_option_value: option_type internal_variable_name equal set_expr_or_default { THD *thd= YYTHD; - LEX *lex=Lex; + LEX *lex= Lex; + LEX_STRING *name= &$2.base_name; if ($2.var == trg_new_row_fake_var) { /* We are in trigger and assigning value to field of new row */ - Item *it; - Item_trigger_field *trg_fld; - sp_instr_set_trigger_field *sp_fld; - LINT_INIT(sp_fld); if ($1) { my_parse_error(ER(ER_SYNTAX_ERROR)); MYSQL_YYABORT; } - if ($4) - it= $4; - else - { - /* QQ: Shouldn't this be field's default value ? */ - it= new Item_null(); - } - - DBUG_ASSERT(lex->trg_chistics.action_time == TRG_ACTION_BEFORE && - (lex->trg_chistics.event == TRG_EVENT_INSERT || - lex->trg_chistics.event == TRG_EVENT_UPDATE)); - - trg_fld= new (thd->mem_root) - Item_trigger_field(Lex->current_context(), - Item_trigger_field::NEW_ROW, - $2.base_name.str, - UPDATE_ACL, FALSE); - if (trg_fld == NULL) - MYSQL_YYABORT; - - sp_fld= new sp_instr_set_trigger_field(lex->sphead-> - instructions(), - lex->spcont, - trg_fld, - it, lex); - if (sp_fld == NULL) - MYSQL_YYABORT; - - /* - Let us add this item to list of all Item_trigger_field - objects in trigger. - */ - lex->trg_table_fields.link_in_list((uchar *)trg_fld, - (uchar **) &trg_fld-> - next_trg_field); - - if (lex->sphead->add_instr(sp_fld)) + if (set_trigger_new_row(YYTHD, name, $4)) MYSQL_YYABORT; } else if ($2.var) - { /* System variable */ + { if ($1) lex->option_type= $1; - set_var *var= new set_var(lex->option_type, $2.var, - &$2.base_name, $4); - if (var == NULL) + + /* It is a system variable. */ + if (set_system_variable(thd, &$2, lex->option_type, $4)) MYSQL_YYABORT; - lex->var_list.push_back(var); } else { - /* An SP local variable */ - sp_pcontext *ctx= lex->spcont; - sp_variable_t *spv; - sp_instr_set *sp_set; - Item *it; + sp_pcontext *spc= lex->spcont; + sp_variable_t *spv= spc->find_variable(name); + if ($1) { my_parse_error(ER(ER_SYNTAX_ERROR)); MYSQL_YYABORT; } - spv= ctx->find_variable(&$2.base_name); - - if ($4) - it= $4; - else if (spv->dflt) - it= spv->dflt; - else - { - it= new (thd->mem_root) Item_null(); - if (it == NULL) - MYSQL_YYABORT; - } - sp_set= new sp_instr_set(lex->sphead->instructions(), ctx, - spv->offset, it, spv->type, lex, TRUE); - if (sp_set == NULL || - lex->sphead->add_instr(sp_set)) + /* It is a local variable. */ + if (set_local_variable(thd, spv, $4)) MYSQL_YYABORT; } } @@ -11958,11 +12047,16 @@ option_value: } | '@' '@' opt_var_ident_type internal_variable_name equal set_expr_or_default { - LEX *lex=Lex; - set_var *var= new set_var($3, $4.var, &$4.base_name, $6); - if (var == NULL) + THD *thd= YYTHD; + struct sys_var_with_base tmp= $4; + /* Lookup if necessary: must be a system variable. */ + if (tmp.var == NULL) + { + if (find_sys_var_null_base(thd, &tmp)) + MYSQL_YYABORT; + } + if (set_system_variable(thd, &tmp, $3, $6)) MYSQL_YYABORT; - lex->var_list.push_back(var); } | charset old_or_new_charset_name_or_default { @@ -12055,31 +12149,26 @@ internal_variable_name: ident { THD *thd= YYTHD; - LEX *lex= thd->lex; - sp_pcontext *spc= lex->spcont; + sp_pcontext *spc= thd->lex->spcont; sp_variable_t *spv; - /* We have to lookup here since local vars can shadow sysvars */ + /* Best effort lookup for system variable. */ if (!spc || !(spv = spc->find_variable(&$1))) { + struct sys_var_with_base tmp= {NULL, $1}; + /* Not an SP local variable */ - sys_var *tmp=find_sys_var(thd, $1.str, $1.length); - if (!tmp) + if (find_sys_var_null_base(thd, &tmp)) MYSQL_YYABORT; - $$.var= tmp; - $$.base_name= null_lex_str; - if (spc && tmp == &sys_autocommit) - { - /* - We don't allow setting AUTOCOMMIT from a stored function - or trigger. - */ - lex->sphead->m_flags|= sp_head::HAS_SET_AUTOCOMMIT_STMT; - } + + $$= tmp; } else { - /* An SP local variable */ + /* + Possibly an SP local variable (or a shadowed sysvar). + Will depend on the context of the SET statement. + */ $$.var= NULL; $$.base_name= $1; } @@ -12237,12 +12326,7 @@ lock_option: | WRITE_SYM { $$= TL_WRITE_DEFAULT; } | WRITE_SYM CONCURRENT { -#ifdef HAVE_QUERY_CACHE - if (Lex->sphead != 0) - $$= TL_WRITE_DEFAULT; - else -#endif - $$= TL_WRITE_CONCURRENT_INSERT; + $$= (Lex->sphead ? TL_WRITE_DEFAULT : TL_WRITE_CONCURRENT_INSERT); } | LOW_PRIORITY WRITE_SYM { $$= TL_WRITE_LOW_PRIORITY; } === modified file 'sql/table.cc' --- a/sql/table.cc 2009-12-03 11:34:11 +0000 +++ b/sql/table.cc 2010-01-15 15:27:55 +0000 @@ -1316,8 +1316,16 @@ static int open_binary_frm(THD *thd, TAB share->timestamp_field_offset= i; if (use_hash) - (void) my_hash_insert(&share->name_hash, - (uchar*) field_ptr); // never fail + if (my_hash_insert(&share->name_hash, (uchar*) field_ptr) ) + { + /* + Set return code 8 here to indicate that an error has + occurred but that the error message already has been + sent (OOM). + */ + error= 8; + goto err; + } } *field_ptr=0; // End marker @@ -2804,34 +2812,38 @@ bool check_column_name(const char *name) and such errors never reach the user. */ -my_bool -table_check_intact(TABLE *table, const uint table_f_count, - const TABLE_FIELD_W_TYPE *table_def) +bool +Table_check_intact::check(TABLE *table, const TABLE_FIELD_DEF *table_def) { uint i; my_bool error= FALSE; - my_bool fields_diff_count; + const TABLE_FIELD_TYPE *field_def= table_def->field; DBUG_ENTER("table_check_intact"); DBUG_PRINT("info",("table: %s expected_count: %d", - table->alias, table_f_count)); + table->alias, table_def->count)); - fields_diff_count= (table->s->fields != table_f_count); - if (fields_diff_count) + /* Whether the table definition has already been validated. */ + if (table->s->table_field_def_cache == table_def) + DBUG_RETURN(FALSE); + + if (table->s->fields != table_def->count) { DBUG_PRINT("info", ("Column count has changed, checking the definition")); /* previous MySQL version */ if (MYSQL_VERSION_ID > table->s->mysql_version) { - sql_print_error(ER(ER_COL_COUNT_DOESNT_MATCH_PLEASE_UPDATE), - table->alias, table_f_count, table->s->fields, - table->s->mysql_version, MYSQL_VERSION_ID); + report_error(ER_COL_COUNT_DOESNT_MATCH_PLEASE_UPDATE, + ER(ER_COL_COUNT_DOESNT_MATCH_PLEASE_UPDATE), + table->alias, table_def->count, table->s->fields, + table->s->mysql_version, MYSQL_VERSION_ID); DBUG_RETURN(TRUE); } else if (MYSQL_VERSION_ID == table->s->mysql_version) { - sql_print_error(ER(ER_COL_COUNT_DOESNT_MATCH_CORRUPTED), table->alias, - table_f_count, table->s->fields); + report_error(ER_COL_COUNT_DOESNT_MATCH_CORRUPTED, + ER(ER_COL_COUNT_DOESNT_MATCH_CORRUPTED), table->alias, + table_def->count, table->s->fields); DBUG_RETURN(TRUE); } /* @@ -2843,7 +2855,7 @@ table_check_intact(TABLE *table, const u */ } char buffer[STRING_BUFFER_USUAL_SIZE]; - for (i=0 ; i < table_f_count; i++, table_def++) + for (i=0 ; i < table_def->count; i++, field_def++) { String sql_type(buffer, sizeof(buffer), system_charset_info); sql_type.length(0); @@ -2851,18 +2863,18 @@ table_check_intact(TABLE *table, const u { Field *field= table->field[i]; - if (strncmp(field->field_name, table_def->name.str, - table_def->name.length)) + if (strncmp(field->field_name, field_def->name.str, + field_def->name.length)) { /* Name changes are not fatal, we use ordinal numbers to access columns. Still this can be a sign of a tampered table, output an error to the error log. */ - sql_print_error("Incorrect definition of table %s.%s: " - "expected column '%s' at position %d, found '%s'.", - table->s->db.str, table->alias, table_def->name.str, i, - field->field_name); + report_error(0, "Incorrect definition of table %s.%s: " + "expected column '%s' at position %d, found '%s'.", + table->s->db.str, table->alias, field_def->name.str, i, + field->field_name); } field->sql_type(sql_type); /* @@ -2882,47 +2894,51 @@ table_check_intact(TABLE *table, const u the new table definition is backward compatible with the original one. */ - if (strncmp(sql_type.c_ptr_safe(), table_def->type.str, - table_def->type.length - 1)) + if (strncmp(sql_type.c_ptr_safe(), field_def->type.str, + field_def->type.length - 1)) { - sql_print_error("Incorrect definition of table %s.%s: " - "expected column '%s' at position %d to have type " - "%s, found type %s.", table->s->db.str, table->alias, - table_def->name.str, i, table_def->type.str, - sql_type.c_ptr_safe()); + report_error(0, "Incorrect definition of table %s.%s: " + "expected column '%s' at position %d to have type " + "%s, found type %s.", table->s->db.str, table->alias, + field_def->name.str, i, field_def->type.str, + sql_type.c_ptr_safe()); error= TRUE; } - else if (table_def->cset.str && !field->has_charset()) + else if (field_def->cset.str && !field->has_charset()) { - sql_print_error("Incorrect definition of table %s.%s: " - "expected the type of column '%s' at position %d " - "to have character set '%s' but the type has no " - "character set.", table->s->db.str, table->alias, - table_def->name.str, i, table_def->cset.str); + report_error(0, "Incorrect definition of table %s.%s: " + "expected the type of column '%s' at position %d " + "to have character set '%s' but the type has no " + "character set.", table->s->db.str, table->alias, + field_def->name.str, i, field_def->cset.str); error= TRUE; } - else if (table_def->cset.str && - strcmp(field->charset()->csname, table_def->cset.str)) + else if (field_def->cset.str && + strcmp(field->charset()->csname, field_def->cset.str)) { - sql_print_error("Incorrect definition of table %s.%s: " - "expected the type of column '%s' at position %d " - "to have character set '%s' but found " - "character set '%s'.", table->s->db.str, table->alias, - table_def->name.str, i, table_def->cset.str, - field->charset()->csname); + report_error(0, "Incorrect definition of table %s.%s: " + "expected the type of column '%s' at position %d " + "to have character set '%s' but found " + "character set '%s'.", table->s->db.str, table->alias, + field_def->name.str, i, field_def->cset.str, + field->charset()->csname); error= TRUE; } } else { - sql_print_error("Incorrect definition of table %s.%s: " - "expected column '%s' at position %d to have type %s " - " but the column is not found.", - table->s->db.str, table->alias, - table_def->name.str, i, table_def->type.str); + report_error(0, "Incorrect definition of table %s.%s: " + "expected column '%s' at position %d to have type %s " + " but the column is not found.", + table->s->db.str, table->alias, + field_def->name.str, i, field_def->type.str); error= TRUE; } } + + if (! error) + table->s->table_field_def_cache= table_def; + DBUG_RETURN(error); } === modified file 'sql/table.h' --- a/sql/table.h 2009-12-03 11:19:05 +0000 +++ b/sql/table.h 2010-01-15 15:27:55 +0000 @@ -285,6 +285,36 @@ typedef enum enum_table_category TABLE_C TABLE_CATEGORY get_table_category(const LEX_STRING *db, const LEX_STRING *name); + +typedef struct st_table_field_type +{ + LEX_STRING name; + LEX_STRING type; + LEX_STRING cset; +} TABLE_FIELD_TYPE; + + +typedef struct st_table_field_def +{ + uint count; + const TABLE_FIELD_TYPE *field; +} TABLE_FIELD_DEF; + + +class Table_check_intact +{ +protected: + virtual void report_error(uint code, const char *fmt, ...)= 0; + +public: + Table_check_intact() {} + virtual ~Table_check_intact() {} + + /** Checks whether a table is intact. */ + bool check(TABLE *table, const TABLE_FIELD_DEF *table_def); +}; + + /* This structure is shared between different table objects. There is one instance of table share per one table in the database. @@ -423,6 +453,18 @@ typedef struct st_table_share handlerton *default_part_db_type; #endif + /** + Cache the checked structure of this table. + + The pointer data is used to describe the structure that + a instance of the table must have. Each element of the + array specifies a field that must exist on the table. + + The pointer is cached in order to perform the check only + once -- when the table is loaded from the disk. + */ + const TABLE_FIELD_DEF *table_field_def_cache; + /** place to store storage engine specific data */ void *ha_data; @@ -1674,17 +1716,6 @@ typedef struct st_open_table_list{ uint32 in_use,locked; } OPEN_TABLE_LIST; -typedef struct st_table_field_w_type -{ - LEX_STRING name; - LEX_STRING type; - LEX_STRING cset; -} TABLE_FIELD_W_TYPE; - - -my_bool -table_check_intact(TABLE *table, const uint table_f_count, - const TABLE_FIELD_W_TYPE *table_def); static inline my_bitmap_map *tmp_use_all_columns(TABLE *table, MY_BITMAP *bitmap) === modified file 'sql/udf_example.def' --- a/sql/udf_example.def 2007-10-29 14:01:40 +0000 +++ b/sql/udf_example.def 2010-01-29 18:42:22 +0000 @@ -3,8 +3,10 @@ VERSION 1.0 EXPORTS lookup lookup_init + lookup_deinit reverse_lookup reverse_lookup_init + reverse_lookup_deinit metaphon_init metaphon_deinit metaphon === modified file 'storage/archive/CMakeLists.txt' (properties changed: -x to +x) --- a/storage/archive/CMakeLists.txt 2009-06-10 08:59:49 +0000 +++ b/storage/archive/CMakeLists.txt 2009-11-10 19:41:43 +0000 @@ -13,6 +13,9 @@ # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +SET(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} -DSAFEMALLOC -DSAFE_MUTEX") +SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} -DSAFEMALLOC -DSAFE_MUTEX") + INCLUDE("${PROJECT_SOURCE_DIR}/storage/mysql_storage_engine.cmake") SET(ARCHIVE_SOURCES azio.c ha_archive.cc ha_archive.h) MYSQL_STORAGE_ENGINE(ARCHIVE) === modified file 'storage/archive/azio.c' --- a/storage/archive/azio.c 2009-05-22 12:38:50 +0000 +++ b/storage/archive/azio.c 2010-01-15 15:27:55 +0000 @@ -71,7 +71,8 @@ int az_open (azio_stream *s, const char s->transparent = 0; s->mode = 'r'; s->version = (unsigned char)az_magic[1]; /* this needs to be a define to version */ - s->version = (unsigned char)az_magic[2]; /* minor version */ + s->minor_version= (unsigned char) az_magic[2]; /* minor version */ + s->dirty= AZ_STATE_CLEAN; /* We do our own version of append by nature. @@ -354,10 +355,19 @@ void read_header(azio_stream *s, unsigne s->comment_length= (unsigned int)uint4korr(buffer + AZ_COMMENT_LENGTH_POS); s->dirty= (unsigned int)buffer[AZ_DIRTY_POS]; } - else + else if (buffer[0] == gz_magic[0] && buffer[1] == gz_magic[1]) { - DBUG_ASSERT(buffer[0] == az_magic[0] && buffer[1] == az_magic[1]); - return; + /* + Set version number to previous version (2). + */ + s->version= (unsigned char) 2; + } else { + /* + Unknown version. + Most probably due to a corrupt archive. + */ + s->dirty= AZ_STATE_DIRTY; + s->z_err= Z_VERSION_ERROR; } } === modified file 'storage/archive/ha_archive.cc' --- a/storage/archive/ha_archive.cc 2009-12-03 11:19:05 +0000 +++ b/storage/archive/ha_archive.cc 2010-01-15 15:27:55 +0000 @@ -360,6 +360,12 @@ ARCHIVE_SHARE *ha_archive::get_share(con stats.auto_increment_value= archive_tmp.auto_increment + 1; share->rows_recorded= (ha_rows)archive_tmp.rows; share->crashed= archive_tmp.dirty; + /* + If archive version is less than 3, It should be upgraded before + use. + */ + if (archive_tmp.version < ARCHIVE_VERSION) + *rc= HA_ERR_TABLE_NEEDS_UPGRADE; azclose(&archive_tmp); VOID(my_hash_insert(&archive_open_tables, (uchar*) share)); @@ -491,7 +497,15 @@ int ha_archive::open(const char *name, i (open_options & HA_OPEN_FOR_REPAIR) ? "yes" : "no")); share= get_share(name, &rc); - if (rc == HA_ERR_CRASHED_ON_USAGE && !(open_options & HA_OPEN_FOR_REPAIR)) + /* + Allow open on crashed table in repair mode only. + Block open on 5.0 ARCHIVE table. Though we have almost all + routines to access these tables, they were not well tested. + For now we have to refuse to open such table to avoid + potential data loss. + */ + if ((rc == HA_ERR_CRASHED_ON_USAGE && !(open_options & HA_OPEN_FOR_REPAIR)) + || rc == HA_ERR_TABLE_NEEDS_UPGRADE) { /* purecov: begin inspected */ free_share(); === modified file 'storage/federated/CMakeLists.txt' (properties changed: -x to +x) --- a/storage/federated/CMakeLists.txt 2009-06-10 08:59:49 +0000 +++ b/storage/federated/CMakeLists.txt 2009-11-10 19:41:43 +0000 @@ -1,18 +1,21 @@ # Copyright (C) 2006 MySQL AB -# +# # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; version 2 of the License. -# +# # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. -# +# # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +SET(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} -DSAFEMALLOC -DSAFE_MUTEX") +SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} -DSAFEMALLOC -DSAFE_MUTEX") + INCLUDE("${PROJECT_SOURCE_DIR}/storage/mysql_storage_engine.cmake") SET(FEDERATED_SOURCES ha_federated.cc) MYSQL_STORAGE_ENGINE(FEDERATED) === modified file 'storage/innobase/btr/btr0btr.c' --- a/storage/innobase/btr/btr0btr.c 2007-07-10 14:34:21 +0000 +++ b/storage/innobase/btr/btr0btr.c 2009-11-30 08:50:08 +0000 @@ -709,8 +709,15 @@ btr_create( } else { /* It is a non-ibuf tree: create a file segment for leaf pages */ - fseg_create(space, page_no, PAGE_HEADER + PAGE_BTR_SEG_LEAF, - mtr); + if (!fseg_create(space, page_no, + PAGE_HEADER + PAGE_BTR_SEG_LEAF, mtr)) { + /* Not enough space for new segment, free root + segment before return. */ + btr_free_root(space, page_no, mtr); + + return(FIL_NULL); + } + /* The fseg create acquires a second latch on the page, therefore we must declare it: */ #ifdef UNIV_SYNC_DEBUG === modified file 'storage/innobase/data/data0type.c' --- a/storage/innobase/data/data0type.c 2007-07-10 11:37:43 +0000 +++ b/storage/innobase/data/data0type.c 2009-11-30 08:53:52 +0000 @@ -252,6 +252,22 @@ dtype_print( fputs("DATA_SYS", stderr); break; + case DATA_FLOAT: + fputs("DATA_FLOAT", stderr); + break; + + case DATA_DOUBLE: + fputs("DATA_DOUBLE", stderr); + break; + + case DATA_DECIMAL: + fputs("DATA_DECIMAL", stderr); + break; + + case DATA_VARMYSQL: + fputs("DATA_VARMYSQL", stderr); + break; + default: fprintf(stderr, "type %lu", (ulong) mtype); break; === modified file 'storage/innobase/handler/ha_innodb.cc' --- a/storage/innobase/handler/ha_innodb.cc 2009-12-03 11:19:05 +0000 +++ b/storage/innobase/handler/ha_innodb.cc 2010-01-15 15:27:55 +0000 @@ -662,6 +662,12 @@ convert_error_code_to_mysql( } else if (error == (int) DB_DUPLICATE_KEY) { + /* Be cautious with returning this error, since + mysql could re-enter the storage layer to get + duplicated key info, the operation requires a + valid table handle and/or transaction information, + which might not always be available in the error + handling stage. */ return(HA_ERR_FOUND_DUPP_KEY); } else if (error == (int) DB_FOREIGN_DUPLICATE_KEY) { @@ -765,35 +771,6 @@ convert_error_code_to_mysql( } /***************************************************************** -If you want to print a thd that is not associated with the current thread, -you must call this function before reserving the InnoDB kernel_mutex, to -protect MySQL from setting thd->query NULL. If you print a thd of the current -thread, we know that MySQL cannot modify thd->query, and it is not necessary -to call this. Call innobase_mysql_end_print_arbitrary_thd() after you release -the kernel_mutex. -NOTE that /mysql/innobase/lock/lock0lock.c must contain the prototype for this -function! */ -extern "C" -void -innobase_mysql_prepare_print_arbitrary_thd(void) -/*============================================*/ -{ - VOID(pthread_mutex_lock(&LOCK_thread_count)); -} - -/***************************************************************** -Releases the mutex reserved by innobase_mysql_prepare_print_arbitrary_thd(). -NOTE that /mysql/innobase/lock/lock0lock.c must contain the prototype for this -function! */ -extern "C" -void -innobase_mysql_end_print_arbitrary_thd(void) -/*========================================*/ -{ - VOID(pthread_mutex_unlock(&LOCK_thread_count)); -} - -/***************************************************************** Prints info of a THD object (== user session thread) to the given file. NOTE that /mysql/innobase/trx/trx0trx.c must contain the prototype for this function! */ @@ -1499,70 +1476,148 @@ innobase_invalidate_query_cache( #endif } -/********************************************************************* -Display an SQL identifier. */ -extern "C" -void -innobase_print_identifier( -/*======================*/ - FILE* f, /* in: output stream */ - trx_t* trx, /* in: transaction */ - ibool table_id,/* in: TRUE=print a table name, - FALSE=print other identifier */ - const char* name, /* in: name to print */ - ulint namelen)/* in: length of name */ -{ - const char* s = name; - char* qname = NULL; +/*****************************************************************//** +Convert an SQL identifier to the MySQL system_charset_info (UTF-8) +and quote it if needed. +@return pointer to the end of buf */ +static +char* +innobase_convert_identifier( +/*========================*/ + char* buf, /*!< out: buffer for converted identifier */ + ulint buflen, /*!< in: length of buf, in bytes */ + const char* id, /*!< in: identifier to convert */ + ulint idlen, /*!< in: length of id, in bytes */ + void* thd, /*!< in: MySQL connection thread, or NULL */ + ibool file_id)/*!< in: TRUE=id is a table or database name; + FALSE=id is an UTF-8 string */ +{ + char nz[NAME_LEN + 1]; +#if MYSQL_VERSION_ID >= 50141 + char nz2[NAME_LEN + 1 + EXPLAIN_FILENAME_MAX_EXTRA_LENGTH]; +#else /* MYSQL_VERSION_ID >= 50141 */ + char nz2[NAME_LEN + 1 + sizeof srv_mysql50_table_name_prefix]; +#endif /* MYSQL_VERSION_ID >= 50141 */ + + const char* s = id; int q; - if (table_id) { - /* Decode the table name. The filename_to_tablename() - function expects a NUL-terminated string. The input and - output strings buffers must not be shared. The function - only produces more output when the name contains other - characters than [0-9A-Z_a-z]. */ - char* temp_name = (char*) my_malloc((uint) namelen + 1, MYF(MY_WME)); - uint qnamelen = (uint) (namelen - + (1 + sizeof srv_mysql50_table_name_prefix)); - - if (temp_name) { - qname = (char*) my_malloc(qnamelen, MYF(MY_WME)); - if (qname) { - memcpy(temp_name, name, namelen); - temp_name[namelen] = 0; - s = qname; - namelen = filename_to_tablename(temp_name, - qname, qnamelen); - } - my_free(temp_name, MYF(0)); - } + if (file_id) { + /* Decode the table name. The MySQL function expects + a NUL-terminated string. The input and output strings + buffers must not be shared. */ + + if (UNIV_UNLIKELY(idlen > (sizeof nz) - 1)) { + idlen = (sizeof nz) - 1; + } + + memcpy(nz, id, idlen); + nz[idlen] = 0; + + s = nz2; +#if MYSQL_VERSION_ID >= 50141 + idlen = explain_filename((THD*) thd, nz, nz2, sizeof nz2, + EXPLAIN_PARTITIONS_AS_COMMENT); + goto no_quote; +#else /* MYSQL_VERSION_ID >= 50141 */ + idlen = filename_to_tablename(nz, nz2, sizeof nz2); +#endif /* MYSQL_VERSION_ID >= 50141 */ } - if (!trx || !trx->mysql_thd) { - + /* See if the identifier needs to be quoted. */ + if (UNIV_UNLIKELY(!thd)) { q = '"'; } else { - q = get_quote_char_for_identifier((THD*) trx->mysql_thd, - s, (int) namelen); + q = get_quote_char_for_identifier((THD*) thd, s, (int) idlen); } if (q == EOF) { - fwrite(s, 1, namelen, f); - } else { - const char* e = s + namelen; - putc(q, f); - while (s < e) { - int c = *s++; - if (c == q) { - putc(c, f); +#if MYSQL_VERSION_ID >= 50141 +no_quote: +#endif /* MYSQL_VERSION_ID >= 50141 */ + if (UNIV_UNLIKELY(idlen > buflen)) { + idlen = buflen; + } + memcpy(buf, s, idlen); + return(buf + idlen); + } + + /* Quote the identifier. */ + if (buflen < 2) { + return(buf); + } + + *buf++ = q; + buflen--; + + for (; idlen; idlen--) { + int c = *s++; + if (UNIV_UNLIKELY(c == q)) { + if (UNIV_UNLIKELY(buflen < 3)) { + break; + } + + *buf++ = c; + *buf++ = c; + buflen -= 2; + } else { + if (UNIV_UNLIKELY(buflen < 2)) { + break; } - putc(c, f); + + *buf++ = c; + buflen--; + } + } + + *buf++ = q; + return(buf); +} + +/*****************************************************************//** +Convert a table or index name to the MySQL system_charset_info (UTF-8) +and quote it if needed. +@return pointer to the end of buf */ +extern "C" +char* +innobase_convert_name( +/*==================*/ + char* buf, /*!< out: buffer for converted identifier */ + ulint buflen, /*!< in: length of buf, in bytes */ + const char* id, /*!< in: identifier to convert */ + ulint idlen, /*!< in: length of id, in bytes */ + void* thd, /*!< in: MySQL connection thread, or NULL */ + ibool table_id)/*!< in: TRUE=id is a table or database name; + FALSE=id is an index name */ +{ + char* s = buf; + const char* bufend = buf + buflen; + + if (table_id) { + const char* slash = (const char*) memchr(id, '/', idlen); + if (!slash) { + + goto no_db_name; } - putc(q, f); + + /* Print the database name and table name separately. */ + s = innobase_convert_identifier(s, bufend - s, id, slash - id, + thd, TRUE); + if (UNIV_LIKELY(s < bufend)) { + *s++ = '.'; + s = innobase_convert_identifier(s, bufend - s, + slash + 1, idlen + - (slash - id) - 1, + thd, TRUE); + } + } else { +no_db_name: + s = innobase_convert_identifier(buf, buflen, id, idlen, + thd, table_id); } - my_free(qname, MYF(MY_ALLOW_ZERO_PTR)); + return(s); + } /************************************************************************** @@ -3986,24 +4041,29 @@ no_commit: update the table upper limit. Note: last_value will be 0 if get_auto_increment() was not called.*/ - if (auto_inc <= col_max_value - && auto_inc >= prebuilt->autoinc_last_value) { + if (auto_inc >= prebuilt->autoinc_last_value) { set_max_autoinc: - ut_a(prebuilt->autoinc_increment > 0); - - ulonglong need; - ulonglong offset; - - offset = prebuilt->autoinc_offset; - need = prebuilt->autoinc_increment; - - auto_inc = innobase_next_autoinc( - auto_inc, need, offset, col_max_value); - - err = innobase_set_max_autoinc(auto_inc); - - if (err != DB_SUCCESS) { - error = err; + /* This should filter out the negative + values set explicitly by the user. */ + if (auto_inc <= col_max_value) { + ut_a(prebuilt->autoinc_increment > 0); + + ulonglong need; + ulonglong offset; + + offset = prebuilt->autoinc_offset; + need = prebuilt->autoinc_increment; + + auto_inc = innobase_next_autoinc( + auto_inc, + need, offset, col_max_value); + + err = innobase_set_max_autoinc( + auto_inc); + + if (err != DB_SUCCESS) { + error = err; + } } } break; @@ -5970,6 +6030,24 @@ ha_innobase::rename_table( innobase_commit_low(trx); trx_free_for_mysql(trx); + /* Add a special case to handle the Duplicated Key error + and return DB_ERROR instead. + This is to avoid a possible SIGSEGV error from mysql error + handling code. Currently, mysql handles the Duplicated Key + error by re-entering the storage layer and getting dup key + info by calling get_dup_key(). This operation requires a valid + table handle ('row_prebuilt_t' structure) which could no + longer be available in the error handling stage. The suggested + solution is to report a 'table exists' error message (since + the dup key error here is due to an existing table whose name + is the one we are trying to rename to) and return the generic + error code. */ + if (error == (int) DB_DUPLICATE_KEY) { + my_error(ER_TABLE_EXISTS_ERROR, MYF(0), to); + + error = DB_ERROR; + } + error = convert_error_code_to_mysql(error, NULL); DBUG_RETURN(error); @@ -8204,8 +8282,7 @@ innobase_xa_prepare( executing XA PREPARE and XA COMMIT commands. In this case we cannot know how many minutes or hours will be between XA PREPARE and XA COMMIT, and we don't want - to block for undefined period of time. - */ + to block for undefined period of time. */ pthread_mutex_lock(&prepare_commit_mutex); trx->active_trans = 2; } === modified file 'storage/innobase/include/ha_prototypes.h' --- a/storage/innobase/include/ha_prototypes.h 2008-12-14 19:28:19 +0000 +++ b/storage/innobase/include/ha_prototypes.h 2009-11-30 08:26:45 +0000 @@ -24,18 +24,21 @@ innobase_convert_string( CHARSET_INFO* from_cs, uint* errors); -/********************************************************************* -Display an SQL identifier. */ - -void -innobase_print_identifier( -/*======================*/ - FILE* f, /* in: output stream */ - trx_t* trx, /* in: transaction */ - ibool table_id,/* in: TRUE=print a table name, - FALSE=print other identifier */ - const char* name, /* in: name to print */ - ulint namelen);/* in: length of name */ +/*****************************************************************//** +Convert a table or index name to the MySQL system_charset_info (UTF-8) +and quote it if needed. +@return pointer to the end of buf */ + +char* +innobase_convert_name( +/*==================*/ + char* buf, /*!< out: buffer for converted identifier */ + ulint buflen, /*!< in: length of buf, in bytes */ + const char* id, /*!< in: identifier to convert */ + ulint idlen, /*!< in: length of id, in bytes */ + void* thd, /*!< in: MySQL connection thread, or NULL */ + ibool table_id);/*!< in: TRUE=id is a table or database name; + FALSE=id is an index name */ /********************************************************************** Returns true if the thread is the replication thread on the slave === modified file 'storage/innobase/include/mach0data.h' --- a/storage/innobase/include/mach0data.h 2008-08-20 00:37:41 +0000 +++ b/storage/innobase/include/mach0data.h 2009-11-30 09:41:38 +0000 @@ -266,8 +266,8 @@ UNIV_INLINE double mach_double_read( /*=============*/ - /* out: double read */ - byte* b); /* in: pointer to memory from where to read */ + /* out: double read */ + const byte* b); /* in: pointer to memory from where to read */ /************************************************************* Writes a double. It is stored in a little-endian format. */ UNIV_INLINE @@ -282,8 +282,8 @@ UNIV_INLINE float mach_float_read( /*============*/ - /* out: float read */ - byte* b); /* in: pointer to memory from where to read */ + /* out: float read */ + const byte* b); /* in: pointer to memory from where to read */ /************************************************************* Writes a float. It is stored in a little-endian format. */ UNIV_INLINE === modified file 'storage/innobase/include/mach0data.ic' --- a/storage/innobase/include/mach0data.ic 2008-08-20 00:37:41 +0000 +++ b/storage/innobase/include/mach0data.ic 2009-11-30 09:41:38 +0000 @@ -504,8 +504,8 @@ UNIV_INLINE double mach_double_read( /*=============*/ - /* out: double read */ - byte* b) /* in: pointer to memory from where to read */ + /* out: double read */ + const byte* b) /* in: pointer to memory from where to read */ { double d; ulint i; @@ -553,8 +553,8 @@ UNIV_INLINE float mach_float_read( /*============*/ - /* out: float read */ - byte* b) /* in: pointer to memory from where to read */ + /* out: float read */ + const byte* b) /* in: pointer to memory from where to read */ { float d; ulint i; === modified file 'storage/innobase/include/os0file.h' --- a/storage/innobase/include/os0file.h 2007-07-10 14:34:21 +0000 +++ b/storage/innobase/include/os0file.h 2009-11-30 08:40:31 +0000 @@ -96,6 +96,8 @@ log. */ to become available again */ #define OS_FILE_SHARING_VIOLATION 76 #define OS_FILE_ERROR_NOT_SPECIFIED 77 + /* 78 is used in the plugin */ +#define OS_FILE_OPERATION_ABORTED 79 /* Types for aio operations */ #define OS_FILE_READ 10 === modified file 'storage/innobase/include/trx0trx.h' --- a/storage/innobase/include/trx0trx.h 2009-07-10 23:12:13 +0000 +++ b/storage/innobase/include/trx0trx.h 2009-12-01 10:38:40 +0000 @@ -318,9 +318,7 @@ trx_commit_step( /************************************************************************** Prints info about a transaction to the given file. The caller must own the -kernel mutex and must have called -innobase_mysql_prepare_print_arbitrary_thd(), unless he knows that MySQL -or InnoDB cannot meanwhile change the info printed here. */ +kernel mutex. */ void trx_print( === modified file 'storage/innobase/lock/lock0lock.c' --- a/storage/innobase/lock/lock0lock.c 2009-07-10 23:12:13 +0000 +++ b/storage/innobase/lock/lock0lock.c 2009-12-01 10:38:40 +0000 @@ -22,31 +22,6 @@ Created 5/7/1996 Heikki Tuuri #include "trx0sys.h" -/* 2 function prototypes copied from ha_innodb.cc: */ - -/***************************************************************** -If you want to print a thd that is not associated with the current thread, -you must call this function before reserving the InnoDB kernel_mutex, to -protect MySQL from setting thd->query NULL. If you print a thd of the current -thread, we know that MySQL cannot modify thd->query, and it is not necessary -to call this. Call innobase_mysql_end_print_arbitrary_thd() after you release -the kernel_mutex. -NOTE that /mysql/innobase/lock/lock0lock.c must contain the prototype for this -function! */ - -void -innobase_mysql_prepare_print_arbitrary_thd(void); -/*============================================*/ - -/***************************************************************** -Relases the mutex reserved by innobase_mysql_prepare_print_arbitrary_thd(). -NOTE that /mysql/innobase/lock/lock0lock.c must contain the prototype for this -function! */ - -void -innobase_mysql_end_print_arbitrary_thd(void); -/*========================================*/ - /* Restricts the length of search we will do in the waits-for graph of transactions */ #define LOCK_MAX_N_STEPS_IN_DEADLOCK_CHECK 1000000 @@ -4222,11 +4197,6 @@ lock_print_info_summary( /*====================*/ FILE* file) /* in: file where to print */ { - /* We must protect the MySQL thd->query field with a MySQL mutex, and - because the MySQL mutex must be reserved before the kernel_mutex of - InnoDB, we call innobase_mysql_prepare_print_arbitrary_thd() here. */ - - innobase_mysql_prepare_print_arbitrary_thd(); lock_mutex_enter_kernel(); if (lock_deadlock_found) { @@ -4314,7 +4284,6 @@ loop: if (trx == NULL) { lock_mutex_exit_kernel(); - innobase_mysql_end_print_arbitrary_thd(); ut_ad(lock_validate()); @@ -4386,7 +4355,6 @@ loop: if (load_page_first) { lock_mutex_exit_kernel(); - innobase_mysql_end_print_arbitrary_thd(); mtr_start(&mtr); @@ -4397,7 +4365,6 @@ loop: load_page_first = FALSE; - innobase_mysql_prepare_print_arbitrary_thd(); lock_mutex_enter_kernel(); goto loop; === modified file 'storage/innobase/os/os0file.c' --- a/storage/innobase/os/os0file.c 2008-12-14 19:15:12 +0000 +++ b/storage/innobase/os/os0file.c 2009-11-30 08:40:31 +0000 @@ -257,6 +257,13 @@ os_file_get_last_error( " software or another instance\n" "InnoDB: of MySQL." " Please close it to get rid of this error.\n"); + } else if (err == ERROR_OPERATION_ABORTED) { + fprintf(stderr, + "InnoDB: The error means that the I/O" + " operation has been aborted\n" + "InnoDB: because of either a thread exit" + " or an application request.\n" + "InnoDB: Retry attempt is made.\n"); } else { fprintf(stderr, "InnoDB: Some operating system error numbers" @@ -278,6 +285,8 @@ os_file_get_last_error( } else if (err == ERROR_SHARING_VIOLATION || err == ERROR_LOCK_VIOLATION) { return(OS_FILE_SHARING_VIOLATION); + } else if (err == ERROR_OPERATION_ABORTED) { + return(OS_FILE_OPERATION_ABORTED); } else { return(100 + err); } @@ -402,6 +411,10 @@ os_file_handle_error_cond_exit( os_thread_sleep(10000000); /* 10 sec */ return(TRUE); + } else if (err == OS_FILE_OPERATION_ABORTED) { + + os_thread_sleep(100000); /* 100 ms */ + return(TRUE); } else { if (name) { fprintf(stderr, "InnoDB: File name %s\n", name); @@ -3692,6 +3705,7 @@ os_aio_windows_handle( ibool ret_val; BOOL ret; DWORD len; + BOOL retry = FALSE; if (segment == ULINT_UNDEFINED) { array = os_aio_sync_array; @@ -3745,14 +3759,52 @@ os_aio_windows_handle( ut_a(TRUE == os_file_flush(slot->file)); } # endif /* UNIV_DO_FLUSH */ + } else if (os_file_handle_error(slot->name, "Windows aio")) { + + retry = TRUE; } else { - os_file_handle_error(slot->name, "Windows aio"); ret_val = FALSE; } os_mutex_exit(array->mutex); + if (retry) { + /* retry failed read/write operation synchronously. + No need to hold array->mutex. */ + + switch (slot->type) { + case OS_FILE_WRITE: + ret = WriteFile(slot->file, slot->buf, + slot->len, &len, + &(slot->control)); + + break; + case OS_FILE_READ: + ret = ReadFile(slot->file, slot->buf, + slot->len, &len, + &(slot->control)); + + break; + default: + ut_error; + } + + if (!ret && GetLastError() == ERROR_IO_PENDING) { + /* aio was queued successfully! + We want a synchronous i/o operation on a + file where we also use async i/o: in Windows + we must use the same wait mechanism as for + async i/o */ + + ret = GetOverlappedResult(slot->file, + &(slot->control), + &len, TRUE); + } + + ret_val = ret && len == slot->len; + } + os_aio_array_free_slot(array, slot); return(ret_val); === modified file 'storage/innobase/row/row0sel.c' --- a/storage/innobase/row/row0sel.c 2009-07-10 23:12:13 +0000 +++ b/storage/innobase/row/row0sel.c 2009-11-30 09:41:38 +0000 @@ -4514,6 +4514,7 @@ row_search_autoinc_read_column( dict_index_t* index, /* in: index to read from */ const rec_t* rec, /* in: current rec */ ulint col_no, /* in: column number */ + ulint mtype, /*!< in: column main type */ ibool unsigned_type) /* in: signed or unsigned flag */ { ulint len; @@ -4535,9 +4536,26 @@ row_search_autoinc_read_column( data = rec_get_nth_field((rec_t*)rec, offsets, col_no, &len); ut_a(len != UNIV_SQL_NULL); - ut_a(len <= sizeof value); - value = mach_read_int_type(data, len, unsigned_type); + switch (mtype) { + case DATA_INT: + ut_a(len <= sizeof value); + value = mach_read_int_type(data, len, unsigned_type); + break; + + case DATA_FLOAT: + ut_a(len == sizeof(float)); + value = mach_float_read(data); + break; + + case DATA_DOUBLE: + ut_a(len == sizeof(double)); + value = mach_double_read(data); + break; + + default: + ut_error; + } if (UNIV_LIKELY_NULL(heap)) { mem_heap_free(heap); @@ -4625,7 +4643,8 @@ row_search_max_autoinc( dfield->col->prtype & DATA_UNSIGNED); *value = row_search_autoinc_read_column( - index, rec, i, unsigned_type); + index, rec, i, + dfield->col->mtype, unsigned_type); } } === modified file 'storage/innobase/trx/trx0trx.c' --- a/storage/innobase/trx/trx0trx.c 2009-07-10 23:12:13 +0000 +++ b/storage/innobase/trx/trx0trx.c 2009-12-01 10:38:40 +0000 @@ -1652,9 +1652,7 @@ trx_mark_sql_stat_end( /************************************************************************** Prints info about a transaction to the given file. The caller must own the -kernel mutex and must have called -innobase_mysql_prepare_print_arbitrary_thd(), unless he knows that MySQL -or InnoDB cannot meanwhile change the info printed here. */ +kernel mutex. */ void trx_print( === modified file 'storage/innobase/ut/ut0ut.c' --- a/storage/innobase/ut/ut0ut.c 2008-12-14 19:18:59 +0000 +++ b/storage/innobase/ut/ut0ut.c 2009-11-30 08:26:45 +0000 @@ -19,6 +19,7 @@ Created 5/11/1994 Heikki Tuuri #include "ut0sort.h" #include "trx0trx.h" #include "ha_prototypes.h" +#include "mysql_com.h" /* NAME_LEN */ ibool ut_always_false = FALSE; @@ -484,26 +485,17 @@ ut_print_namel( const char* name, /* in: name to print */ ulint namelen)/* in: length of name */ { -#ifdef UNIV_HOTBACKUP - fwrite(name, 1, namelen, f); -#else - if (table_id) { - char* slash = memchr(name, '/', namelen); - if (!slash) { - - goto no_db_name; - } + /* 2 * NAME_LEN for database and table name, + and some slack for the #mysql50# prefix and quotes */ + char buf[3 * NAME_LEN]; + const char* bufend; + + bufend = innobase_convert_name(buf, sizeof buf, + name, namelen, + trx ? trx->mysql_thd : NULL, + table_id); - /* Print the database name and table name separately. */ - innobase_print_identifier(f, trx, TRUE, name, slash - name); - putc('.', f); - innobase_print_identifier(f, trx, TRUE, slash + 1, - namelen - (slash - name) - 1); - } else { -no_db_name: - innobase_print_identifier(f, trx, table_id, name, namelen); - } -#endif + fwrite(buf, 1, bufend - buf, f); } /************************************************************************** === modified file 'storage/innodb_plugin/CMakeLists.txt' --- a/storage/innodb_plugin/CMakeLists.txt 2009-12-03 11:19:05 +0000 +++ b/storage/innodb_plugin/CMakeLists.txt 2010-01-15 15:27:55 +0000 @@ -83,4 +83,4 @@ SET(INNODB_PLUGIN_SOURCES btr/btr0btr.c ADD_DEFINITIONS(-DHAVE_WINDOWS_ATOMICS -DIB_HAVE_PAUSE_INSTRUCTION) #Disable storage engine, as we are using XtraDB -#MYSQL_STORAGE_ENGINE(INNODB_PLUGIN) +#MYSQL_STORAGE_ENGINE(INNOBASE) === modified file 'storage/innodb_plugin/ChangeLog' --- a/storage/innodb_plugin/ChangeLog 2009-11-03 10:34:03 +0000 +++ b/storage/innodb_plugin/ChangeLog 2009-11-30 13:42:26 +0000 @@ -1,3 +1,87 @@ +2009-11-20 The InnoDB Team + + * handler/ha_innodb.cc: + Add a workaround to prevent a crash due to Bug#45961 DDL on + partitioned innodb tables leaves data dictionary in an inconsistent + state + +2009-11-19 The InnoDB Team + + * btr/btr0btr.c: + Fix Bug#48469 when innodb tablespace is configured too small, crash + and corruption! + +2009-11-19 The InnoDB Team + + * data/data0type.c: + Fix Bug#48526 Data type for float and double is incorrectly reported + in InnoDB table monitor + +2009-11-19 The InnoDB Team + + * CMakeLists.txt: + Fix Bug#48317 cannot build innodb as static library + +2009-11-18 The InnoDB Team + + * handler/handler0alter.cc: + Fix Bug#48782 On lock wait timeout, CREATE INDEX (creating primary key) + attempts DROP TABLE + +2009-11-17 The InnoDB Team + + * handler/ha_innodb.cc, mysql-test/innodb.result, + mysql-test/innodb.test, mysql-test/innodb_bug44369.result, + mysql-test/innodb_bug44369.test, mysql-test/patches/innodb-index.diff, + row/row0mysql.c: + Report duplicate table names to the client connection, not to the + error log. + +2009-11-12 The InnoDB Team + + * handler/ha_innodb.cc, include/db0err.h, row/row0merge.c, + row/row0mysql.c: + Allow CREATE INDEX to be interrupted. + Also, when CHECK TABLE is interrupted, report ER_QUERY_INTERRUPTED. + +2009-11-11 The InnoDB Team + + * handler/ha_innodb.cc, mysql-test/innodb_bug47167.result, + mysql-test/innodb_bug47167.test, mysql-test/innodb_file_format.result: + Fix Bug#47167 "set global innodb_file_format_check" cannot set value + by User-Defined Variable + +2009-11-11 The InnoDB Team + + * include/os0file.h, os/os0file.c: + Fix Bug#3139 Mysql crashes: 'windows error 995' after several selects + on a large DB + +2009-11-04 The InnoDB Team + + * handler/ha_innodb.cc: + Fix Bug#32430 'show innodb status' causes errors + Invalid (old?) table or database name in logs + +2009-11-02 The InnoDB Team + + * btr/btr0sea.c, buf/buf0buf.c, dict/dict0dict.c, fil/fil0fil.c, + ibuf/ibuf0ibuf.c, include/btr0sea.h, include/dict0dict.h, + include/fil0fil.h, include/ibuf0ibuf.h, include/lock0lock.h, + include/log0log.h, include/log0recv.h, include/mem0mem.h, + include/mem0pool.h, include/os0file.h, include/pars0pars.h, + include/srv0srv.h, include/thr0loc.h, include/trx0i_s.h, + include/trx0purge.h, include/trx0rseg.h, include/trx0sys.h, + include/trx0undo.h, include/usr0sess.h, lock/lock0lock.c, + log/log0log.c, log/log0recv.c, mem/mem0dbg.c, mem/mem0pool.c, + os/os0file.c, os/os0sync.c, os/os0thread.c, pars/lexyy.c, + pars/pars0lex.l, que/que0que.c, srv/srv0srv.c, srv/srv0start.c, + sync/sync0arr.c, sync/sync0sync.c, thr/thr0loc.c, trx/trx0i_s.c, + trx/trx0purge.c, trx/trx0rseg.c, trx/trx0sys.c, trx/trx0undo.c, + usr/usr0sess.c, ut/ut0mem.c: + Fix Bug #45992 innodb memory not freed after shutdown + Fix Bug #46656 InnoDB plugin: memory leaks (Valgrind) + 2009-10-29 The InnoDB Team * handler/ha_innodb.cc, mysql-test/innodb-autoinc.result, @@ -66,6 +150,12 @@ Fix Bug#47058 Failure to compile innodb_plugin on solaris 10u7 + spro cc/CC 5.10 +2009-10-13 The InnoDB Team + + * buf/buf0flu.c: + Call fsync() on datafiles after a batch of pages is written to disk + even when skip_innodb_doublewrite is set. + 2009-10-05 The InnoDB Team * buf/buf0buf.c: === modified file 'storage/innodb_plugin/btr/btr0btr.c' --- a/storage/innodb_plugin/btr/btr0btr.c 2009-10-12 12:00:56 +0000 +++ b/storage/innodb_plugin/btr/btr0btr.c 2009-11-30 13:42:26 +0000 @@ -790,8 +790,15 @@ btr_create( } else { /* It is a non-ibuf tree: create a file segment for leaf pages */ - fseg_create(space, page_no, - PAGE_HEADER + PAGE_BTR_SEG_LEAF, mtr); + if (!fseg_create(space, page_no, + PAGE_HEADER + PAGE_BTR_SEG_LEAF, mtr)) { + /* Not enough space for new segment, free root + segment before return. */ + btr_free_root(space, zip_size, page_no, mtr); + + return(FIL_NULL); + } + /* The fseg create acquires a second latch on the page, therefore we must declare it: */ buf_block_dbg_add_level(block, SYNC_TREE_NODE_NEW); === modified file 'storage/innodb_plugin/btr/btr0sea.c' --- a/storage/innodb_plugin/btr/btr0sea.c 2009-10-08 11:28:37 +0000 +++ b/storage/innodb_plugin/btr/btr0sea.c 2009-11-30 11:32:05 +0000 @@ -175,6 +175,21 @@ btr_search_sys_create( btr_search_sys->hash_index = ha_create(hash_size, 0, 0); } +/*****************************************************************//** +Frees the adaptive search system at a database shutdown. */ +UNIV_INTERN +void +btr_search_sys_free(void) +/*=====================*/ +{ + mem_free(btr_search_latch_temp); + btr_search_latch_temp = NULL; + mem_heap_free(btr_search_sys->hash_index->heap); + hash_table_free(btr_search_sys->hash_index); + mem_free(btr_search_sys); + btr_search_sys = NULL; +} + /********************************************************************//** Disable the adaptive hash search system and empty the index. */ UNIV_INTERN === modified file 'storage/innodb_plugin/buf/buf0buf.c' --- a/storage/innodb_plugin/buf/buf0buf.c 2009-11-03 10:26:07 +0000 +++ b/storage/innodb_plugin/buf/buf0buf.c 2009-11-30 11:32:05 +0000 @@ -1020,7 +1020,11 @@ buf_pool_free(void) os_mem_free_large(chunk->mem, chunk->mem_size); } - buf_pool->n_chunks = 0; + mem_free(buf_pool->chunks); + hash_table_free(buf_pool->page_hash); + hash_table_free(buf_pool->zip_hash); + mem_free(buf_pool); + buf_pool = NULL; } /********************************************************************//** === modified file 'storage/innodb_plugin/data/data0type.c' --- a/storage/innodb_plugin/data/data0type.c 2009-07-30 12:42:56 +0000 +++ b/storage/innodb_plugin/data/data0type.c 2009-11-30 13:35:20 +0000 @@ -237,6 +237,22 @@ dtype_print( fputs("DATA_SYS", stderr); break; + case DATA_FLOAT: + fputs("DATA_FLOAT", stderr); + break; + + case DATA_DOUBLE: + fputs("DATA_DOUBLE", stderr); + break; + + case DATA_DECIMAL: + fputs("DATA_DECIMAL", stderr); + break; + + case DATA_VARMYSQL: + fputs("DATA_VARMYSQL", stderr); + break; + default: fprintf(stderr, "type %lu", (ulong) mtype); break; === modified file 'storage/innodb_plugin/dict/dict0dict.c' --- a/storage/innodb_plugin/dict/dict0dict.c 2009-10-09 12:52:18 +0000 +++ b/storage/innodb_plugin/dict/dict0dict.c 2009-11-30 11:42:51 +0000 @@ -1200,7 +1200,7 @@ dict_index_too_big_for_undo( = TRX_UNDO_PAGE_HDR - TRX_UNDO_PAGE_HDR_SIZE + 2 /* next record pointer */ + 1 /* type_cmpl */ - + 11 /* trx->undo_no */ - 11 /* table->id */ + + 11 /* trx->undo_no */ + 11 /* table->id */ + 1 /* rec_get_info_bits() */ + 11 /* DB_TRX_ID */ + 11 /* DB_ROLL_PTR */ @@ -4652,6 +4652,26 @@ dict_ind_init(void) dict_ind_redundant->cached = dict_ind_compact->cached = TRUE; } +/**********************************************************************//** +Frees dict_ind_redundant and dict_ind_compact. */ +static +void +dict_ind_free(void) +/*===============*/ +{ + dict_table_t* table; + + table = dict_ind_compact->table; + dict_mem_index_free(dict_ind_compact); + dict_ind_compact = NULL; + dict_mem_table_free(table); + + table = dict_ind_redundant->table; + dict_mem_index_free(dict_ind_redundant); + dict_ind_redundant = NULL; + dict_mem_table_free(table); +} + #ifndef UNIV_HOTBACKUP /**********************************************************************//** Get index by name @@ -4777,4 +4797,55 @@ dict_table_check_for_dup_indexes( } } #endif /* UNIV_DEBUG */ + +/************************************************************************** +Closes the data dictionary module. */ +UNIV_INTERN +void +dict_close(void) +/*============*/ +{ + ulint i; + + /* Free the hash elements. We don't remove them from the table + because we are going to destroy the table anyway. */ + for (i = 0; i < hash_get_n_cells(dict_sys->table_hash); i++) { + dict_table_t* table; + + table = HASH_GET_FIRST(dict_sys->table_hash, i); + + while (table) { + dict_table_t* prev_table = table; + + table = HASH_GET_NEXT(name_hash, prev_table); +#ifdef UNIV_DEBUG + ut_a(prev_table->magic_n == DICT_TABLE_MAGIC_N); +#endif + /* Acquire only because it's a pre-condition. */ + mutex_enter(&dict_sys->mutex); + + dict_table_remove_from_cache(prev_table); + + mutex_exit(&dict_sys->mutex); + } + } + + hash_table_free(dict_sys->table_hash); + + /* The elements are the same instance as in dict_sys->table_hash, + therefore we don't delete the individual elements. */ + hash_table_free(dict_sys->table_id_hash); + + dict_ind_free(); + + mutex_free(&dict_sys->mutex); + + rw_lock_free(&dict_operation_lock); + memset(&dict_operation_lock, 0x0, sizeof(dict_operation_lock)); + + mutex_free(&dict_foreign_err_mutex); + + mem_free(dict_sys); + dict_sys = NULL; +} #endif /* !UNIV_HOTBACKUP */ === modified file 'storage/innodb_plugin/fil/fil0fil.c' --- a/storage/innodb_plugin/fil/fil0fil.c 2009-11-03 10:24:21 +0000 +++ b/storage/innodb_plugin/fil/fil0fil.c 2009-11-30 11:32:05 +0000 @@ -321,6 +321,17 @@ fil_get_space_id_for_table( /*=======================*/ const char* name); /*!< in: table name in the standard 'databasename/tablename' format */ +/*******************************************************************//** +Frees a space object from the tablespace memory cache. Closes the files in +the chain but does not delete them. There must not be any pending i/o's or +flushes on the files. */ +static +ibool +fil_space_free( +/*===========*/ + /* out: TRUE if success */ + ulint id, /* in: space id */ + ibool own_mutex);/* in: TRUE if own system->mutex */ /********************************************************************//** Reads data from a space to a buffer. Remember that the possible incomplete blocks at the end of file are ignored: they are not taken into account when @@ -1144,7 +1155,7 @@ try_again: mutex_exit(&fil_system->mutex); - fil_space_free(namesake_id); + fil_space_free(namesake_id, FALSE); goto try_again; } @@ -1269,17 +1280,21 @@ Frees a space object from the tablespace the chain but does not delete them. There must not be any pending i/o's or flushes on the files. @return TRUE if success */ -UNIV_INTERN +static ibool fil_space_free( /*===========*/ - ulint id) /*!< in: space id */ + /* out: TRUE if success */ + ulint id, /* in: space id */ + ibool own_mutex) /* in: TRUE if own system->mutex */ { fil_space_t* space; fil_space_t* namespace; fil_node_t* fil_node; - mutex_enter(&fil_system->mutex); + if (!own_mutex) { + mutex_enter(&fil_system->mutex); + } space = fil_space_get_by_id(id); @@ -1326,7 +1341,9 @@ fil_space_free( ut_a(0 == UT_LIST_GET_LEN(space->chain)); - mutex_exit(&fil_system->mutex); + if (!own_mutex) { + mutex_exit(&fil_system->mutex); + } rw_lock_free(&(space->latch)); @@ -1586,6 +1603,8 @@ fil_close_all_files(void) space = UT_LIST_GET_FIRST(fil_system->space_list); while (space != NULL) { + fil_space_t* prev_space = space; + node = UT_LIST_GET_FIRST(space->chain); while (node != NULL) { @@ -1595,6 +1614,7 @@ fil_close_all_files(void) node = UT_LIST_GET_NEXT(chain, node); } space = UT_LIST_GET_NEXT(space_list, space); + fil_space_free(prev_space->id, TRUE); } mutex_exit(&fil_system->mutex); @@ -2226,7 +2246,7 @@ try_again: #endif /* printf("Deleting tablespace %s id %lu\n", space->name, id); */ - success = fil_space_free(id); + success = fil_space_free(id, FALSE); if (success) { success = os_file_delete(path); @@ -4753,3 +4773,26 @@ fil_page_get_type( return(mach_read_from_2(page + FIL_PAGE_TYPE)); } + +/******************************************************************** +Initializes the tablespace memory cache. */ +UNIV_INTERN +void +fil_close(void) +/*===========*/ +{ + /* The mutex should already have been freed. */ + ut_ad(fil_system->mutex.magic_n == 0); + + hash_table_free(fil_system->spaces); + + hash_table_free(fil_system->name_hash); + + ut_a(UT_LIST_GET_LEN(fil_system->LRU) == 0); + ut_a(UT_LIST_GET_LEN(fil_system->unflushed_spaces) == 0); + ut_a(UT_LIST_GET_LEN(fil_system->space_list) == 0); + + mem_free(fil_system); + + fil_system = NULL; +} === modified file 'storage/innodb_plugin/handler/ha_innodb.cc' --- a/storage/innodb_plugin/handler/ha_innodb.cc 2009-11-03 10:34:38 +0000 +++ b/storage/innodb_plugin/handler/ha_innodb.cc 2009-12-08 09:26:11 +0000 @@ -269,10 +269,10 @@ innobase_file_format_check_on_off( /************************************************************//** Validate the file format check config parameters, as a side effect it sets the srv_check_file_format_at_startup variable. -@return true if valid config value */ +@return the format_id if valid config value, otherwise, return -1 */ static -bool -innobase_file_format_check_validate( +int +innobase_file_format_validate_and_set( /*================================*/ const char* format_check); /*!< in: parameter value */ /****************************************************************//** @@ -785,11 +785,20 @@ convert_error_code_to_mysql( case DB_SUCCESS: return(0); + case DB_INTERRUPTED: + my_error(ER_QUERY_INTERRUPTED, MYF(0)); + /* fall through */ case DB_ERROR: default: return(-1); /* unspecified error */ case DB_DUPLICATE_KEY: + /* Be cautious with returning this error, since + mysql could re-enter the storage layer to get + duplicated key info, the operation requires a + valid table handle and/or transaction information, + which might not always be available in the error + handling stage. */ return(HA_ERR_FOUND_DUPP_KEY); case DB_FOREIGN_DUPLICATE_KEY: @@ -890,36 +899,6 @@ convert_error_code_to_mysql( } /*************************************************************//** -If you want to print a thd that is not associated with the current thread, -you must call this function before reserving the InnoDB kernel_mutex, to -protect MySQL from setting thd->query NULL. If you print a thd of the current -thread, we know that MySQL cannot modify thd->query, and it is not necessary -to call this. Call innobase_mysql_end_print_arbitrary_thd() after you release -the kernel_mutex. */ -extern "C" UNIV_INTERN -void -innobase_mysql_prepare_print_arbitrary_thd(void) -/*============================================*/ -{ - ut_ad(!mutex_own(&kernel_mutex)); - VOID(pthread_mutex_lock(&LOCK_thread_count)); -} - -/*************************************************************//** -Releases the mutex reserved by innobase_mysql_prepare_print_arbitrary_thd(). -In the InnoDB latching order, the mutex sits right above the -kernel_mutex. In debug builds, we assert that the kernel_mutex is -released before this function is invoked. */ -extern "C" UNIV_INTERN -void -innobase_mysql_end_print_arbitrary_thd(void) -/*========================================*/ -{ - ut_ad(!mutex_own(&kernel_mutex)); - VOID(pthread_mutex_unlock(&LOCK_thread_count)); -} - -/*************************************************************//** Prints info of a THD object (== user session thread) to the given file. */ extern "C" UNIV_INTERN void @@ -1707,15 +1686,19 @@ innobase_convert_identifier( FALSE=id is an UTF-8 string */ { char nz[NAME_LEN + 1]; +#if MYSQL_VERSION_ID >= 50141 + char nz2[NAME_LEN + 1 + EXPLAIN_FILENAME_MAX_EXTRA_LENGTH]; +#else /* MYSQL_VERSION_ID >= 50141 */ char nz2[NAME_LEN + 1 + sizeof srv_mysql50_table_name_prefix]; +#endif /* MYSQL_VERSION_ID >= 50141 */ const char* s = id; int q; if (file_id) { - /* Decode the table name. The filename_to_tablename() - function expects a NUL-terminated string. The input and - output strings buffers must not be shared. */ + /* Decode the table name. The MySQL function expects + a NUL-terminated string. The input and output strings + buffers must not be shared. */ if (UNIV_UNLIKELY(idlen > (sizeof nz) - 1)) { idlen = (sizeof nz) - 1; @@ -1725,7 +1708,13 @@ innobase_convert_identifier( nz[idlen] = 0; s = nz2; +#if MYSQL_VERSION_ID >= 50141 + idlen = explain_filename((THD*) thd, nz, nz2, sizeof nz2, + EXPLAIN_PARTITIONS_AS_COMMENT); + goto no_quote; +#else /* MYSQL_VERSION_ID >= 50141 */ idlen = filename_to_tablename(nz, nz2, sizeof nz2); +#endif /* MYSQL_VERSION_ID >= 50141 */ } /* See if the identifier needs to be quoted. */ @@ -1736,6 +1725,9 @@ innobase_convert_identifier( } if (q == EOF) { +#if MYSQL_VERSION_ID >= 50141 +no_quote: +#endif /* MYSQL_VERSION_ID >= 50141 */ if (UNIV_UNLIKELY(idlen > buflen)) { idlen = buflen; } @@ -2133,8 +2125,8 @@ mem_free_and_error: /* Did the user specify a format name that we support ? As a side effect it will update the variable srv_check_file_format_at_startup */ - if (!innobase_file_format_check_validate( - innobase_file_format_check)) { + if (innobase_file_format_validate_and_set( + innobase_file_format_check) < 0) { sql_print_error("InnoDB: invalid " "innodb_file_format_check value: " @@ -5225,8 +5217,10 @@ ha_innobase::change_active_index( prebuilt->index); if (UNIV_UNLIKELY(!prebuilt->index_usable)) { - sql_print_warning("InnoDB: insufficient history for index %u", - keynr); + push_warning_printf(user_thd, MYSQL_ERROR::WARN_LEVEL_WARN, + HA_ERR_TABLE_DEF_CHANGED, + "InnoDB: insufficient history for index %u", + keynr); /* The caller seems to ignore this. Thus, we must check this again in row_search_for_mysql(). */ DBUG_RETURN(2); @@ -5713,17 +5707,8 @@ create_table_def( /* First check whether the column to be added has a system reserved name. */ if (dict_col_name_is_reserved(field->field_name)){ - push_warning_printf( - (THD*) trx->mysql_thd, - MYSQL_ERROR::WARN_LEVEL_WARN, - ER_CANT_CREATE_TABLE, - "Error creating table '%s' with " - "column name '%s'. '%s' is a " - "reserved name. Please try to " - "re-create the table with a " - "different column name.", - table->name, (char*) field->field_name, - (char*) field->field_name); + my_error(ER_WRONG_COLUMN_NAME, MYF(0), + field->field_name); dict_mem_table_free(table); trx_commit_for_mysql(trx); @@ -5745,6 +5730,14 @@ create_table_def( error = row_create_table_for_mysql(table, trx); + if (error == DB_DUPLICATE_KEY) { + char buf[100]; + innobase_convert_identifier(buf, sizeof buf, + table_name, strlen(table_name), + trx->mysql_thd, TRUE); + my_error(ER_TABLE_EXISTS_ERROR, MYF(0), buf); + } + error_ret: error = convert_error_code_to_mysql(error, flags, NULL); @@ -6798,6 +6791,24 @@ ha_innobase::rename_table( innobase_commit_low(trx); trx_free_for_mysql(trx); + /* Add a special case to handle the Duplicated Key error + and return DB_ERROR instead. + This is to avoid a possible SIGSEGV error from mysql error + handling code. Currently, mysql handles the Duplicated Key + error by re-entering the storage layer and getting dup key + info by calling get_dup_key(). This operation requires a valid + table handle ('row_prebuilt_t' structure) which could no + longer be available in the error handling stage. The suggested + solution is to report a 'table exists' error message (since + the dup key error here is due to an existing table whose name + is the one we are trying to rename to) and return the generic + error code. */ + if (error == (int) DB_DUPLICATE_KEY) { + my_error(ER_TABLE_EXISTS_ERROR, MYF(0), to); + + error = DB_ERROR; + } + error = convert_error_code_to_mysql(error, 0, NULL); DBUG_RETURN(error); @@ -7348,11 +7359,15 @@ ha_innobase::check( ret = row_check_table_for_mysql(prebuilt); - if (ret == DB_SUCCESS) { + switch (ret) { + case DB_SUCCESS: return(HA_ADMIN_OK); + case DB_INTERRUPTED: + my_error(ER_QUERY_INTERRUPTED, MYF(0)); + return(-1); + default: + return(HA_ADMIN_CORRUPT); } - - return(HA_ADMIN_CORRUPT); } /*************************************************************//** @@ -7899,7 +7914,10 @@ ha_innobase::external_lock( ulong const tx_isolation = thd_tx_isolation(ha_thd()); if (tx_isolation <= ISO_READ_COMMITTED && binlog_format == BINLOG_FORMAT_STMT - && thd_binlog_filter_ok(thd)) +#if MYSQL_VERSION_ID > 50140 + && thd_binlog_filter_ok(thd) +#endif /* MYSQL_VERSION_ID > 50140 */ + ) { char buf[256]; my_snprintf(buf, sizeof(buf), @@ -9148,8 +9166,7 @@ innobase_xa_prepare( executing XA PREPARE and XA COMMIT commands. In this case we cannot know how many minutes or hours will be between XA PREPARE and XA COMMIT, and we don't want - to block for undefined period of time. - */ + to block for undefined period of time. */ pthread_mutex_lock(&prepare_commit_mutex); trx->active_trans = 2; } @@ -9491,25 +9508,24 @@ innobase_file_format_check_on_off( /************************************************************//** Validate the file format check config parameters, as a side effect it sets the srv_check_file_format_at_startup variable. -@return true if valid config value */ +@return the format_id if valid config value, otherwise, return -1 */ static -bool -innobase_file_format_check_validate( +int +innobase_file_format_validate_and_set( /*================================*/ const char* format_check) /*!< in: parameter value */ { uint format_id; - bool ret = true; format_id = innobase_file_format_name_lookup(format_check); if (format_id < DICT_TF_FORMAT_MAX + 1) { srv_check_file_format_at_startup = format_id; + + return((int) format_id); } else { - ret = false; + return(-1); } - - return(ret); } /*************************************************************//** @@ -9544,7 +9560,11 @@ innodb_file_format_name_validate( if (format_id <= DICT_TF_FORMAT_MAX) { - *static_cast<const char**>(save) = file_format_input; + /* Save a pointer to the name in the + 'file_format_name_map' constant array. */ + *static_cast<const char**>(save) = + trx_sys_file_format_id_to_name(format_id); + return(0); } } @@ -9607,6 +9627,7 @@ innodb_file_format_check_validate( const char* file_format_input; char buff[STRING_BUFFER_USUAL_SIZE]; int len = sizeof(buff); + int format_id; ut_a(save != NULL); ut_a(value != NULL); @@ -9619,24 +9640,35 @@ innodb_file_format_check_validate( message if they did so. */ if (innobase_file_format_check_on_off(file_format_input)) { - sql_print_warning( + push_warning_printf(thd, + MYSQL_ERROR::WARN_LEVEL_WARN, + ER_WRONG_ARGUMENTS, "InnoDB: invalid innodb_file_format_check " "value; on/off can only be set at startup or " "in the configuration file"); - } else if (innobase_file_format_check_validate( - file_format_input)) { + } else { + format_id = innobase_file_format_validate_and_set( + file_format_input); - *static_cast<const char**>(save) = file_format_input; + if (format_id >= 0) { + /* Save a pointer to the name in the + 'file_format_name_map' constant array. */ + *static_cast<const char**>(save) = + trx_sys_file_format_id_to_name( + (uint)format_id); - return(0); + return(0); - } else { - sql_print_warning( - "InnoDB: invalid innodb_file_format_check " - "value; can be any format up to %s " - "or its equivalent numeric id", - trx_sys_file_format_id_to_name( - DICT_TF_FORMAT_MAX)); + } else { + push_warning_printf(thd, + MYSQL_ERROR::WARN_LEVEL_WARN, + ER_WRONG_ARGUMENTS, + "InnoDB: invalid innodb_file_format_check " + "value; can be any format up to %s " + "or its equivalent numeric id", + trx_sys_file_format_id_to_name( + DICT_TF_FORMAT_MAX)); + } } } @@ -9906,12 +9938,15 @@ static MYSQL_SYSVAR_STR(file_format, inn innodb_file_format_name_validate, innodb_file_format_name_update, "Antelope"); +/* If a new file format is introduced, the file format +name needs to be updated accordingly. Please refer to +file_format_name_map[] defined in trx0sys.c for the next +file format name. */ static MYSQL_SYSVAR_STR(file_format_check, innobase_file_format_check, PLUGIN_VAR_OPCMDARG, "The highest file format in the tablespace.", innodb_file_format_check_validate, - innodb_file_format_check_update, - "on"); + innodb_file_format_check_update, "Barracuda"); static MYSQL_SYSVAR_ULONG(flush_log_at_trx_commit, srv_flush_log_at_trx_commit, PLUGIN_VAR_OPCMDARG, === modified file 'storage/innodb_plugin/handler/ha_innodb.h' --- a/storage/innodb_plugin/handler/ha_innodb.h 2009-11-03 10:07:51 +0000 +++ b/storage/innodb_plugin/handler/ha_innodb.h 2009-11-30 12:11:36 +0000 @@ -258,12 +258,14 @@ int thd_binlog_format(const MYSQL_THD th */ void thd_mark_transaction_to_rollback(MYSQL_THD thd, bool all); +#if MYSQL_VERSION_ID > 50140 /** Check if binary logging is filtered for thread's current db. @param thd Thread handle @retval 1 the query is not filtered, 0 otherwise. */ bool thd_binlog_filter_ok(const MYSQL_THD thd); +#endif /* MYSQL_VERSION_ID > 50140 */ } typedef struct trx_struct trx_t; === modified file 'storage/innodb_plugin/handler/handler0alter.cc' --- a/storage/innodb_plugin/handler/handler0alter.cc 2009-11-03 10:07:51 +0000 +++ b/storage/innodb_plugin/handler/handler0alter.cc 2009-11-30 13:42:26 +0000 @@ -765,10 +765,11 @@ err_exit: ut_ad(error == DB_SUCCESS); /* Commit the data dictionary transaction in order to release - the table locks on the system tables. Unfortunately, this - means that if MySQL crashes while creating a new primary key - inside row_merge_build_indexes(), indexed_table will not be - dropped on crash recovery. Thus, it will become orphaned. */ + the table locks on the system tables. This means that if + MySQL crashes while creating a new primary key inside + row_merge_build_indexes(), indexed_table will not be dropped + by trx_rollback_active(). It will have to be recovered or + dropped by the database administrator. */ trx_commit_for_mysql(trx); row_mysql_unlock_data_dictionary(trx); @@ -882,7 +883,9 @@ error: /* fall through */ default: if (new_primary) { - row_merge_drop_table(trx, indexed_table); + if (indexed_table != innodb_table) { + row_merge_drop_table(trx, indexed_table); + } } else { if (!dict_locked) { row_mysql_lock_data_dictionary(trx); === modified file 'storage/innodb_plugin/ibuf/ibuf0ibuf.c' --- a/storage/innodb_plugin/ibuf/ibuf0ibuf.c 2009-07-30 12:42:56 +0000 +++ b/storage/innodb_plugin/ibuf/ibuf0ibuf.c 2009-11-30 11:32:05 +0000 @@ -390,6 +390,27 @@ ibuf_count_set( #endif /******************************************************************//** +Closes insert buffer and frees the data structures. */ +UNIV_INTERN +void +ibuf_close(void) +/*============*/ +{ + mutex_free(&ibuf_pessimistic_insert_mutex); + memset(&ibuf_pessimistic_insert_mutex, + 0x0, sizeof(ibuf_pessimistic_insert_mutex)); + + mutex_free(&ibuf_mutex); + memset(&ibuf_mutex, 0x0, sizeof(ibuf_mutex)); + + mutex_free(&ibuf_bitmap_mutex); + memset(&ibuf_bitmap_mutex, 0x0, sizeof(ibuf_mutex)); + + mem_free(ibuf); + ibuf = NULL; +} + +/******************************************************************//** Updates the size information of the ibuf, assuming the segment size has not changed. */ static === modified file 'storage/innodb_plugin/include/btr0sea.h' --- a/storage/innodb_plugin/include/btr0sea.h 2009-05-27 09:45:59 +0000 +++ b/storage/innodb_plugin/include/btr0sea.h 2009-11-30 11:32:05 +0000 @@ -41,6 +41,12 @@ void btr_search_sys_create( /*==================*/ ulint hash_size); /*!< in: hash index hash table size */ +/*****************************************************************//** +Frees the adaptive search system at a database shutdown. */ +UNIV_INTERN +void +btr_search_sys_free(void); +/*=====================*/ /********************************************************************//** Disable the adaptive hash search system and empty the index. */ === modified file 'storage/innodb_plugin/include/db0err.h' --- a/storage/innodb_plugin/include/db0err.h 2009-05-27 09:45:59 +0000 +++ b/storage/innodb_plugin/include/db0err.h 2009-11-30 12:24:54 +0000 @@ -32,6 +32,7 @@ enum db_err { /* The following are error codes */ DB_ERROR, + DB_INTERRUPTED, DB_OUT_OF_MEMORY, DB_OUT_OF_FILE_SPACE, DB_LOCK_WAIT, === modified file 'storage/innodb_plugin/include/dict0dict.h' --- a/storage/innodb_plugin/include/dict0dict.h 2009-10-08 11:28:37 +0000 +++ b/storage/innodb_plugin/include/dict0dict.h 2009-11-30 11:32:05 +0000 @@ -1151,6 +1151,13 @@ void dict_ind_init(void); /*===============*/ +/**********************************************************************//** +Closes the data dictionary module. */ +UNIV_INTERN +void +dict_close(void); +/*============*/ + #ifndef UNIV_NONINL #include "dict0dict.ic" #endif === modified file 'storage/innodb_plugin/include/fil0fil.h' --- a/storage/innodb_plugin/include/fil0fil.h 2009-05-27 09:45:59 +0000 +++ b/storage/innodb_plugin/include/fil0fil.h 2009-11-30 11:32:05 +0000 @@ -224,15 +224,6 @@ fil_space_create( 0 for uncompressed tablespaces */ ulint purpose);/*!< in: FIL_TABLESPACE, or FIL_LOG if log */ /*******************************************************************//** -Frees a space object from a the tablespace memory cache. Closes the files in -the chain but does not delete them. -@return TRUE if success */ -UNIV_INTERN -ibool -fil_space_free( -/*===========*/ - ulint id); /*!< in: space id */ -/*******************************************************************//** Returns the size of the space in pages. The tablespace must be cached in the memory cache. @return space size, 0 if space not found */ @@ -278,6 +269,12 @@ fil_init( ulint hash_size, /*!< in: hash table size */ ulint max_n_open); /*!< in: max number of open files */ /*******************************************************************//** +Initializes the tablespace memory cache. */ +UNIV_INTERN +void +fil_close(void); +/*===========*/ +/*******************************************************************//** Opens all log files and system tablespace data files. They stay open until the database server shutdown. This should be called at a server startup after the space objects for the log and the system tablespace have been created. The === modified file 'storage/innodb_plugin/include/ha_prototypes.h' --- a/storage/innodb_plugin/include/ha_prototypes.h 2009-05-27 09:45:59 +0000 +++ b/storage/innodb_plugin/include/ha_prototypes.h 2009-12-01 10:38:40 +0000 @@ -153,28 +153,6 @@ get_innobase_type_from_mysql_type( const void* field) /*!< in: MySQL Field */ __attribute__((nonnull)); -/*************************************************************//** -If you want to print a thd that is not associated with the current thread, -you must call this function before reserving the InnoDB kernel_mutex, to -protect MySQL from setting thd->query NULL. If you print a thd of the current -thread, we know that MySQL cannot modify thd->query, and it is not necessary -to call this. Call innobase_mysql_end_print_arbitrary_thd() after you release -the kernel_mutex. */ -UNIV_INTERN -void -innobase_mysql_prepare_print_arbitrary_thd(void); -/*============================================*/ - -/*************************************************************//** -Releases the mutex reserved by innobase_mysql_prepare_print_arbitrary_thd(). -In the InnoDB latching order, the mutex sits right above the -kernel_mutex. In debug builds, we assert that the kernel_mutex is -released before this function is invoked. */ -UNIV_INTERN -void -innobase_mysql_end_print_arbitrary_thd(void); -/*========================================*/ - /******************************************************************//** Get the variable length bounds of the given character set. */ UNIV_INTERN === modified file 'storage/innodb_plugin/include/ibuf0ibuf.h' --- a/storage/innodb_plugin/include/ibuf0ibuf.h 2009-05-27 09:45:59 +0000 +++ b/storage/innodb_plugin/include/ibuf0ibuf.h 2009-11-30 11:32:05 +0000 @@ -356,6 +356,12 @@ void ibuf_print( /*=======*/ FILE* file); /*!< in: file where to print */ +/******************************************************************//** +Closes insert buffer and frees the data structures. */ +UNIV_INTERN +void +ibuf_close(void); +/*============*/ #define IBUF_HEADER_PAGE_NO FSP_IBUF_HEADER_PAGE_NO #define IBUF_TREE_ROOT_PAGE_NO FSP_IBUF_TREE_ROOT_PAGE_NO === modified file 'storage/innodb_plugin/include/lock0lock.h' --- a/storage/innodb_plugin/include/lock0lock.h 2009-10-08 12:18:19 +0000 +++ b/storage/innodb_plugin/include/lock0lock.h 2009-11-30 11:32:05 +0000 @@ -59,6 +59,12 @@ lock_sys_create( /*============*/ ulint n_cells); /*!< in: number of slots in lock hash table */ /*********************************************************************//** +Closes the lock system at database shutdown. */ +UNIV_INTERN +void +lock_sys_close(void); +/*================*/ +/*********************************************************************//** Checks if some transaction has an implicit x-lock on a record in a clustered index. @return transaction which has the x-lock, or NULL */ === modified file 'storage/innodb_plugin/include/log0log.h' --- a/storage/innodb_plugin/include/log0log.h 2009-10-08 12:18:19 +0000 +++ b/storage/innodb_plugin/include/log0log.h 2009-11-30 11:32:05 +0000 @@ -572,6 +572,18 @@ UNIV_INTERN void log_refresh_stats(void); /*===================*/ +/********************************************************** +Shutdown the log system but do not release all the memory. */ +UNIV_INTERN +void +log_shutdown(void); +/*==============*/ +/********************************************************** +Free the log system data structures. */ +UNIV_INTERN +void +log_mem_free(void); +/*==============*/ extern log_t* log_sys; @@ -584,7 +596,7 @@ extern log_t* log_sys; #define LOG_RECOVER 98887331 /* The counting of lsn's starts from this value: this must be non-zero */ -#define LOG_START_LSN ((ib_uint64_t) (16 * OS_FILE_LOG_BLOCK_SIZE)) +#define LOG_START_LSN ((ib_uint64_t) (16 * OS_FILE_LOG_BLOCK_SIZE)) #define LOG_BUFFER_SIZE (srv_log_buffer_size * UNIV_PAGE_SIZE) #define LOG_ARCHIVE_BUF_SIZE (srv_log_buffer_size * UNIV_PAGE_SIZE / 4) @@ -721,9 +733,12 @@ struct log_group_struct{ ulint lsn_offset; /*!< the offset of the above lsn */ ulint n_pending_writes;/*!< number of currently pending flush writes for this log group */ + byte** file_header_bufs_ptr;/*!< unaligned buffers */ byte** file_header_bufs;/*!< buffers for each file header in the group */ +#ifdef UNIV_LOG_ARCHIVE /*-----------------------------*/ + byte** archive_file_header_bufs_ptr;/*!< unaligned buffers */ byte** archive_file_header_bufs;/*!< buffers for each file header in the group */ ulint archive_space_id;/*!< file space which @@ -742,10 +757,12 @@ struct log_group_struct{ completion function then sets the new value to ..._file_no */ ulint next_archived_offset; /*!< like the preceding field */ +#endif /* UNIV_LOG_ARCHIVE */ /*-----------------------------*/ ib_uint64_t scanned_lsn; /*!< used only in recovery: recovery scan succeeded up to this lsn in this log group */ + byte* checkpoint_buf_ptr;/*!< unaligned checkpoint header */ byte* checkpoint_buf; /*!< checkpoint header is written from this buffer to the group */ UT_LIST_NODE_T(log_group_t) @@ -763,6 +780,7 @@ struct log_struct{ #ifndef UNIV_HOTBACKUP mutex_t mutex; /*!< mutex protecting the log */ #endif /* !UNIV_HOTBACKUP */ + byte* buf_ptr; /* unaligned log buffer */ byte* buf; /*!< log buffer */ ulint buf_size; /*!< log buffer size in bytes */ ulint max_buf_free; /*!< recommended maximum value of @@ -899,6 +917,7 @@ struct log_struct{ should wait for this without owning the log mutex */ #endif /* !UNIV_HOTBACKUP */ + byte* checkpoint_buf_ptr;/* unaligned checkpoint header */ byte* checkpoint_buf; /*!< checkpoint header is read to this buffer */ /* @} */ === modified file 'storage/innodb_plugin/include/log0recv.h' --- a/storage/innodb_plugin/include/log0recv.h 2009-10-09 14:13:15 +0000 +++ b/storage/innodb_plugin/include/log0recv.h 2009-11-30 11:32:05 +0000 @@ -239,6 +239,18 @@ UNIV_INTERN void recv_sys_create(void); /*=================*/ +/**********************************************************//** +Release recovery system mutexes. */ +UNIV_INTERN +void +recv_sys_close(void); +/*================*/ +/********************************************************//** +Frees the recovery system memory. */ +UNIV_INTERN +void +recv_sys_mem_free(void); +/*===================*/ /********************************************************//** Inits the recovery system for a recovery operation. */ UNIV_INTERN @@ -246,6 +258,12 @@ void recv_sys_init( /*==========*/ ulint available_memory); /*!< in: available memory in bytes */ +/********************************************************//** +Reset the state of the recovery system variables. */ +UNIV_INTERN +void +recv_sys_var_init(void); +/*===================*/ /*******************************************************************//** Empties the hash table of stored log records, applying them to appropriate pages. */ === modified file 'storage/innodb_plugin/include/mem0mem.h' --- a/storage/innodb_plugin/include/mem0mem.h 2009-07-30 12:42:56 +0000 +++ b/storage/innodb_plugin/include/mem0mem.h 2009-11-30 11:32:05 +0000 @@ -82,6 +82,13 @@ void mem_init( /*=====*/ ulint size); /*!< in: common pool size in bytes */ +/******************************************************************//** +Closes the memory system. */ +UNIV_INTERN +void +mem_close(void); +/*===========*/ + /**************************************************************//** Use this macro instead of the corresponding function! Macro for memory heap creation. */ === modified file 'storage/innodb_plugin/include/mem0pool.h' --- a/storage/innodb_plugin/include/mem0pool.h 2009-05-27 09:45:59 +0000 +++ b/storage/innodb_plugin/include/mem0pool.h 2009-11-30 11:32:05 +0000 @@ -62,6 +62,13 @@ mem_pool_create( /*============*/ ulint size); /*!< in: pool size in bytes */ /********************************************************************//** +Frees a memory pool. */ +UNIV_INTERN +void +mem_pool_free( +/*==========*/ + mem_pool_t* pool); /*!< in, own: memory pool */ +/********************************************************************//** Allocates memory from a pool. NOTE: This low-level function should only be used in mem0mem.*! @return own: allocated memory buffer */ === modified file 'storage/innodb_plugin/include/os0file.h' --- a/storage/innodb_plugin/include/os0file.h 2009-11-03 09:59:06 +0000 +++ b/storage/innodb_plugin/include/os0file.h 2009-11-30 12:04:09 +0000 @@ -158,6 +158,7 @@ log. */ #define OS_FILE_SHARING_VIOLATION 76 #define OS_FILE_ERROR_NOT_SPECIFIED 77 #define OS_FILE_INSUFFICIENT_RESOURCE 78 +#define OS_FILE_OPERATION_ABORTED 79 /* @} */ /** Types for aio operations @{ */ @@ -620,6 +621,13 @@ os_aio_init( ulint n_write_segs, /*<! in: number of writer threads */ ulint n_slots_sync); /*<! in: number of slots in the sync aio array */ +/*********************************************************************** +Frees the asynchronous io system. */ +UNIV_INTERN +void +os_aio_free(void); +/*=============*/ + /*******************************************************************//** Requests an asynchronous i/o operation. @return TRUE if request was queued successfully, FALSE if fail */ === modified file 'storage/innodb_plugin/include/pars0pars.h' --- a/storage/innodb_plugin/include/pars0pars.h 2009-05-27 09:45:59 +0000 +++ b/storage/innodb_plugin/include/pars0pars.h 2009-11-30 11:32:05 +0000 @@ -583,6 +583,12 @@ pars_info_get_bound_id( pars_info_t* info, /*!< in: info struct */ const char* name); /*!< in: bound id name to find */ +/******************************************************************//** +Release any resources used by the lexer. */ +UNIV_INTERN +void +pars_lexer_close(void); +/*==================*/ /** Extra information supplied for pars_sql(). */ struct pars_info_struct { === modified file 'storage/innodb_plugin/include/srv0srv.h' --- a/storage/innodb_plugin/include/srv0srv.h 2009-10-08 11:28:37 +0000 +++ b/storage/innodb_plugin/include/srv0srv.h 2009-11-30 11:32:05 +0000 @@ -411,7 +411,7 @@ void srv_init(void); /*==========*/ /*********************************************************************//** -Frees the OS fast mutex created in srv_boot(). */ +Frees the data structures created in srv_init(). */ UNIV_INTERN void srv_free(void); === modified file 'storage/innodb_plugin/include/thr0loc.h' --- a/storage/innodb_plugin/include/thr0loc.h 2009-05-27 09:45:59 +0000 +++ b/storage/innodb_plugin/include/thr0loc.h 2009-11-30 11:32:05 +0000 @@ -39,6 +39,12 @@ UNIV_INTERN void thr_local_init(void); /*================*/ + /****************************************************************//** +Close the thread local storage module. */ +UNIV_INTERN +void +thr_local_close(void); +/*=================*/ /*******************************************************************//** Creates a local storage struct for the calling new thread. */ UNIV_INTERN === modified file 'storage/innodb_plugin/include/trx0i_s.h' --- a/storage/innodb_plugin/include/trx0i_s.h 2009-05-27 09:45:59 +0000 +++ b/storage/innodb_plugin/include/trx0i_s.h 2009-11-30 11:32:05 +0000 @@ -141,6 +141,13 @@ void trx_i_s_cache_init( /*===============*/ trx_i_s_cache_t* cache); /*!< out: cache to init */ +/*******************************************************************//** +Free the INFORMATION SCHEMA trx related cache. */ +UNIV_INTERN +void +trx_i_s_cache_free( +/*===============*/ + trx_i_s_cache_t* cache); /*!< in/out: cache to free */ /*******************************************************************//** Issue a shared/read lock on the tables cache. */ === modified file 'storage/innodb_plugin/include/trx0purge.h' --- a/storage/innodb_plugin/include/trx0purge.h 2009-05-27 09:45:59 +0000 +++ b/storage/innodb_plugin/include/trx0purge.h 2009-11-30 11:32:05 +0000 @@ -71,6 +71,12 @@ void trx_purge_sys_create(void); /*======================*/ /********************************************************************//** +Frees the global purge system control structure. */ +UNIV_INTERN +void +trx_purge_sys_close(void); +/*======================*/ +/************************************************************************ Adds the update undo log as the first log in the history list. Removes the update undo log segment from the rseg slot if it is too big for reuse. */ UNIV_INTERN === modified file 'storage/innodb_plugin/include/trx0rseg.h' --- a/storage/innodb_plugin/include/trx0rseg.h 2009-05-27 09:45:59 +0000 +++ b/storage/innodb_plugin/include/trx0rseg.h 2009-11-30 11:32:05 +0000 @@ -125,6 +125,13 @@ trx_rseg_create( ulint max_size, /*!< in: max size in pages */ ulint* id, /*!< out: rseg id */ mtr_t* mtr); /*!< in: mtr */ +/*************************************************************************** +Free's an instance of the rollback segment in memory. */ +UNIV_INTERN +void +trx_rseg_mem_free( +/*==============*/ + trx_rseg_t* rseg); /* in, own: instance to free */ /* Number of undo log slots in a rollback segment file copy */ === modified file 'storage/innodb_plugin/include/trx0sys.h' --- a/storage/innodb_plugin/include/trx0sys.h 2009-07-30 12:42:56 +0000 +++ b/storage/innodb_plugin/include/trx0sys.h 2009-11-30 11:32:05 +0000 @@ -334,6 +334,12 @@ void trx_sys_file_format_tag_init(void); /*==============================*/ /*****************************************************************//** +Shutdown/Close the transaction system. */ +UNIV_INTERN +void +trx_sys_close(void); +/*===============*/ +/*****************************************************************//** Get the name representation of the file format from its id. @return pointer to the name */ UNIV_INTERN === modified file 'storage/innodb_plugin/include/trx0trx.h' --- a/storage/innodb_plugin/include/trx0trx.h 2009-10-08 13:05:59 +0000 +++ b/storage/innodb_plugin/include/trx0trx.h 2009-12-01 10:38:40 +0000 @@ -338,9 +338,7 @@ trx_commit_step( /**********************************************************************//** Prints info about a transaction to the given file. The caller must own the -kernel mutex and must have called -innobase_mysql_prepare_print_arbitrary_thd(), unless he knows that MySQL -or InnoDB cannot meanwhile change the info printed here. */ +kernel mutex. */ UNIV_INTERN void trx_print( === modified file 'storage/innodb_plugin/include/trx0undo.h' --- a/storage/innodb_plugin/include/trx0undo.h 2009-05-27 09:45:59 +0000 +++ b/storage/innodb_plugin/include/trx0undo.h 2009-11-30 11:32:05 +0000 @@ -333,6 +333,13 @@ trx_undo_parse_discard_latest( byte* end_ptr,/*!< in: buffer end */ page_t* page, /*!< in: page or NULL */ mtr_t* mtr); /*!< in: mtr or NULL */ +/************************************************************************ +Frees an undo log memory copy. */ +UNIV_INTERN +void +trx_undo_mem_free( +/*==============*/ + trx_undo_t* undo); /* in: the undo object to be freed */ /* Types of an undo log segment */ #define TRX_UNDO_INSERT 1 /* contains undo entries for inserts */ === modified file 'storage/innodb_plugin/include/univ.i' --- a/storage/innodb_plugin/include/univ.i 2009-11-03 10:26:39 +0000 +++ b/storage/innodb_plugin/include/univ.i 2009-11-30 13:13:34 +0000 @@ -46,7 +46,7 @@ Created 1/20/1994 Heikki Tuuri #define INNODB_VERSION_MAJOR 1 #define INNODB_VERSION_MINOR 0 -#define INNODB_VERSION_BUGFIX 5 +#define INNODB_VERSION_BUGFIX 6 /* The following is the InnoDB version as shown in SELECT plugin_version FROM information_schema.plugins; === modified file 'storage/innodb_plugin/include/usr0sess.h' --- a/storage/innodb_plugin/include/usr0sess.h 2009-05-27 09:45:59 +0000 +++ b/storage/innodb_plugin/include/usr0sess.h 2009-11-30 11:32:05 +0000 @@ -44,14 +44,12 @@ sess_t* sess_open(void); /*============*/ /*********************************************************************//** -Closes a session, freeing the memory occupied by it, if it is in a state -where it should be closed. -@return TRUE if closed */ +Closes a session, freeing the memory occupied by it. */ UNIV_INTERN -ibool -sess_try_close( -/*===========*/ - sess_t* sess); /*!< in, own: session object */ +void +sess_close( +/*=======*/ + sess_t* sess); /* in, own: session object */ /* The session handle. All fields are protected by the kernel mutex */ struct sess_struct{ === modified file 'storage/innodb_plugin/lock/lock0lock.c' --- a/storage/innodb_plugin/lock/lock0lock.c 2009-10-09 14:13:15 +0000 +++ b/storage/innodb_plugin/lock/lock0lock.c 2009-12-01 10:38:40 +0000 @@ -578,6 +578,23 @@ lock_sys_create( } /*********************************************************************//** +Closes the lock system at database shutdown. */ +UNIV_INTERN +void +lock_sys_close(void) +/*================*/ +{ + if (lock_latest_err_file != NULL) { + fclose(lock_latest_err_file); + lock_latest_err_file = NULL; + } + + hash_table_free(lock_sys->rec_hash); + mem_free(lock_sys); + lock_sys = NULL; +} + +/*********************************************************************//** Gets the size of a lock struct. @return size in bytes */ UNIV_INTERN @@ -4307,11 +4324,6 @@ lock_print_info_summary( /*====================*/ FILE* file) /*!< in: file where to print */ { - /* We must protect the MySQL thd->query field with a MySQL mutex, and - because the MySQL mutex must be reserved before the kernel_mutex of - InnoDB, we call innobase_mysql_prepare_print_arbitrary_thd() here. */ - - innobase_mysql_prepare_print_arbitrary_thd(); lock_mutex_enter_kernel(); if (lock_deadlock_found) { @@ -4394,7 +4406,6 @@ loop: if (trx == NULL) { lock_mutex_exit_kernel(); - innobase_mysql_end_print_arbitrary_thd(); ut_ad(lock_validate()); @@ -4478,7 +4489,6 @@ loop: } lock_mutex_exit_kernel(); - innobase_mysql_end_print_arbitrary_thd(); mtr_start(&mtr); @@ -4489,7 +4499,6 @@ loop: load_page_first = FALSE; - innobase_mysql_prepare_print_arbitrary_thd(); lock_mutex_enter_kernel(); goto loop; === modified file 'storage/innodb_plugin/log/log0log.c' --- a/storage/innodb_plugin/log/log0log.c 2009-10-09 14:13:15 +0000 +++ b/storage/innodb_plugin/log/log0log.c 2009-11-30 11:32:05 +0000 @@ -771,8 +771,6 @@ void log_init(void) /*==========*/ { - byte* buf; - log_sys = mem_alloc(sizeof(log_t)); mutex_create(&log_sys->mutex, SYNC_LOG); @@ -787,8 +785,8 @@ log_init(void) ut_a(LOG_BUFFER_SIZE >= 16 * OS_FILE_LOG_BLOCK_SIZE); ut_a(LOG_BUFFER_SIZE >= 4 * UNIV_PAGE_SIZE); - buf = mem_alloc(LOG_BUFFER_SIZE + OS_FILE_LOG_BLOCK_SIZE); - log_sys->buf = ut_align(buf, OS_FILE_LOG_BLOCK_SIZE); + log_sys->buf_ptr = mem_alloc(LOG_BUFFER_SIZE + OS_FILE_LOG_BLOCK_SIZE); + log_sys->buf = ut_align(log_sys->buf_ptr, OS_FILE_LOG_BLOCK_SIZE); log_sys->buf_size = LOG_BUFFER_SIZE; @@ -833,9 +831,9 @@ log_init(void) rw_lock_create(&log_sys->checkpoint_lock, SYNC_NO_ORDER_CHECK); - log_sys->checkpoint_buf - = ut_align(mem_alloc(2 * OS_FILE_LOG_BLOCK_SIZE), - OS_FILE_LOG_BLOCK_SIZE); + log_sys->checkpoint_buf_ptr = mem_alloc(2 * OS_FILE_LOG_BLOCK_SIZE); + log_sys->checkpoint_buf = ut_align(log_sys->checkpoint_buf_ptr, + OS_FILE_LOG_BLOCK_SIZE); memset(log_sys->checkpoint_buf, '\0', OS_FILE_LOG_BLOCK_SIZE); /*----------------------------*/ @@ -918,23 +916,33 @@ log_group_init( group->lsn_offset = LOG_FILE_HDR_SIZE; group->n_pending_writes = 0; + group->file_header_bufs_ptr = mem_alloc(sizeof(byte*) * n_files); group->file_header_bufs = mem_alloc(sizeof(byte*) * n_files); #ifdef UNIV_LOG_ARCHIVE + group->archive_file_header_bufs_ptr = mem_alloc( + sizeof(byte*) * n_files); group->archive_file_header_bufs = mem_alloc(sizeof(byte*) * n_files); #endif /* UNIV_LOG_ARCHIVE */ for (i = 0; i < n_files; i++) { - *(group->file_header_bufs + i) = ut_align( - mem_alloc(LOG_FILE_HDR_SIZE + OS_FILE_LOG_BLOCK_SIZE), + group->file_header_bufs_ptr[i] = mem_alloc( + LOG_FILE_HDR_SIZE + OS_FILE_LOG_BLOCK_SIZE); + + group->file_header_bufs[i] = ut_align( + group->file_header_bufs_ptr[i], OS_FILE_LOG_BLOCK_SIZE); memset(*(group->file_header_bufs + i), '\0', LOG_FILE_HDR_SIZE); #ifdef UNIV_LOG_ARCHIVE - *(group->archive_file_header_bufs + i) = ut_align( - mem_alloc(LOG_FILE_HDR_SIZE + OS_FILE_LOG_BLOCK_SIZE), + group->archive_file_header_bufs_ptr[i] = mem_alloc( + LOG_FILE_HDR_SIZE + OS_FILE_LOG_BLOCK_SIZE); + + group->archive_file_header_bufs[i] = ut_align( + group->archive_file_header_bufs_ptr[i], OS_FILE_LOG_BLOCK_SIZE); + memset(*(group->archive_file_header_bufs + i), '\0', LOG_FILE_HDR_SIZE); #endif /* UNIV_LOG_ARCHIVE */ @@ -947,8 +955,9 @@ log_group_init( group->archived_offset = 0; #endif /* UNIV_LOG_ARCHIVE */ - group->checkpoint_buf = ut_align( - mem_alloc(2 * OS_FILE_LOG_BLOCK_SIZE), OS_FILE_LOG_BLOCK_SIZE); + group->checkpoint_buf_ptr = mem_alloc(2 * OS_FILE_LOG_BLOCK_SIZE); + group->checkpoint_buf = ut_align(group->checkpoint_buf_ptr, + OS_FILE_LOG_BLOCK_SIZE); memset(group->checkpoint_buf, '\0', OS_FILE_LOG_BLOCK_SIZE); @@ -3364,4 +3373,95 @@ log_refresh_stats(void) log_sys->n_log_ios_old = log_sys->n_log_ios; log_sys->last_printout_time = time(NULL); } + +/********************************************************************** +Closes a log group. */ +static +void +log_group_close( +/*===========*/ + log_group_t* group) /* in,own: log group to close */ +{ + ulint i; + + for (i = 0; i < group->n_files; i++) { + mem_free(group->file_header_bufs_ptr[i]); +#ifdef UNIV_LOG_ARCHIVE + mem_free(group->archive_file_header_bufs_ptr[i]); +#endif /* UNIV_LOG_ARCHIVE */ + } + + mem_free(group->file_header_bufs_ptr); + mem_free(group->file_header_bufs); + +#ifdef UNIV_LOG_ARCHIVE + mem_free(group->archive_file_header_bufs_ptr); + mem_free(group->archive_file_header_bufs); +#endif /* UNIV_LOG_ARCHIVE */ + + mem_free(group->checkpoint_buf_ptr); + + mem_free(group); +} + +/********************************************************** +Shutdown the log system but do not release all the memory. */ +UNIV_INTERN +void +log_shutdown(void) +/*==============*/ +{ + log_group_t* group; + + group = UT_LIST_GET_FIRST(log_sys->log_groups); + + while (UT_LIST_GET_LEN(log_sys->log_groups) > 0) { + log_group_t* prev_group = group; + + group = UT_LIST_GET_NEXT(log_groups, group); + UT_LIST_REMOVE(log_groups, log_sys->log_groups, prev_group); + + log_group_close(prev_group); + } + + mem_free(log_sys->buf_ptr); + log_sys->buf_ptr = NULL; + log_sys->buf = NULL; + mem_free(log_sys->checkpoint_buf_ptr); + log_sys->checkpoint_buf_ptr = NULL; + log_sys->checkpoint_buf = NULL; + + os_event_free(log_sys->no_flush_event); + os_event_free(log_sys->one_flushed_event); + + rw_lock_free(&log_sys->checkpoint_lock); + + mutex_free(&log_sys->mutex); + +#ifdef UNIV_LOG_ARCHIVE + rw_lock_free(&log_sys->archive_lock); + os_event_create(log_sys->archiving_on); +#endif /* UNIV_LOG_ARCHIVE */ + +#ifdef UNIV_LOG_DEBUG + recv_sys_debug_free(); +#endif + + recv_sys_close(); +} + +/********************************************************** +Free the log system data structures. */ +UNIV_INTERN +void +log_mem_free(void) +/*==============*/ +{ + if (log_sys != NULL) { + recv_sys_mem_free(); + mem_free(log_sys); + + log_sys = NULL; + } +} #endif /* !UNIV_HOTBACKUP */ === modified file 'storage/innodb_plugin/log/log0recv.c' --- a/storage/innodb_plugin/log/log0recv.c 2009-10-09 14:13:15 +0000 +++ b/storage/innodb_plugin/log/log0recv.c 2009-11-30 11:32:05 +0000 @@ -69,15 +69,15 @@ UNIV_INTERN recv_sys_t* recv_sys = NULL; /** TRUE when applying redo log records during crash recovery; FALSE otherwise. Note that this is FALSE while a background thread is rolling back incomplete transactions. */ -UNIV_INTERN ibool recv_recovery_on = FALSE; +UNIV_INTERN ibool recv_recovery_on; #ifdef UNIV_LOG_ARCHIVE /** TRUE when applying redo log records from an archived log file */ -UNIV_INTERN ibool recv_recovery_from_backup_on = FALSE; +UNIV_INTERN ibool recv_recovery_from_backup_on; #endif /* UNIV_LOG_ARCHIVE */ #ifndef UNIV_HOTBACKUP /** TRUE when recv_init_crash_recovery() has been called. */ -UNIV_INTERN ibool recv_needed_recovery = FALSE; +UNIV_INTERN ibool recv_needed_recovery; # ifdef UNIV_DEBUG /** TRUE if writing to the redo log (mtr_commit) is forbidden. Protected by log_sys->mutex. */ @@ -87,7 +87,7 @@ UNIV_INTERN ibool recv_no_log_write = FA /** TRUE if buf_page_is_corrupted() should check if the log sequence number (FIL_PAGE_LSN) is in the future. Initially FALSE, and set by recv_recovery_from_checkpoint_start_func(). */ -UNIV_INTERN ibool recv_lsn_checks_on = FALSE; +UNIV_INTERN ibool recv_lsn_checks_on; /** There are two conditions under which we scan the logs, the first is normal startup and the second is when we do a recovery from an @@ -97,7 +97,7 @@ startup. If we find log entries that wer we know that the server was not cleanly shutdown. We must then initialize the crash recovery environment before attempting to store these entries in the log hash table. */ -static ibool recv_log_scan_is_startup_type = FALSE; +static ibool recv_log_scan_is_startup_type; /** If the following is TRUE, the buffer pool file pages must be invalidated after recovery and no ibuf operations are allowed; this becomes TRUE if @@ -108,7 +108,7 @@ buffer pool before the pages have been r TRUE means that recovery is running and no operations on the log files are allowed yet: the variable name is misleading. */ -UNIV_INTERN ibool recv_no_ibuf_operations = FALSE; +UNIV_INTERN ibool recv_no_ibuf_operations; /** TRUE when the redo log is being backed up */ # define recv_is_making_a_backup FALSE /** TRUE when recovering from a backed up redo log file */ @@ -116,30 +116,30 @@ UNIV_INTERN ibool recv_no_ibuf_operation #else /* !UNIV_HOTBACKUP */ # define recv_needed_recovery FALSE /** TRUE when the redo log is being backed up */ -UNIV_INTERN ibool recv_is_making_a_backup = FALSE; +UNIV_INTERN ibool recv_is_making_a_backup = FALSE; /** TRUE when recovering from a backed up redo log file */ UNIV_INTERN ibool recv_is_from_backup = FALSE; # define buf_pool_get_curr_size() (5 * 1024 * 1024) #endif /* !UNIV_HOTBACKUP */ /** The following counter is used to decide when to print info on log scan */ -static ulint recv_scan_print_counter = 0; +static ulint recv_scan_print_counter; /** The type of the previous parsed redo log record */ -static ulint recv_previous_parsed_rec_type = 999999; +static ulint recv_previous_parsed_rec_type; /** The offset of the previous parsed redo log record */ -static ulint recv_previous_parsed_rec_offset = 0; +static ulint recv_previous_parsed_rec_offset; /** The 'multi' flag of the previous parsed redo log record */ -static ulint recv_previous_parsed_rec_is_multi = 0; +static ulint recv_previous_parsed_rec_is_multi; /** Maximum page number encountered in the redo log */ -UNIV_INTERN ulint recv_max_parsed_page_no = 0; +UNIV_INTERN ulint recv_max_parsed_page_no; /** This many frames must be left free in the buffer pool when we scan the log and store the scanned log records in the buffer pool: we will use these free frames to read in pages when we start applying the log records to the database. */ -UNIV_INTERN ulint recv_n_pool_free_frames = 256; +UNIV_INTERN ulint recv_n_pool_free_frames; /** The maximum lsn we see for a page during the recovery process. If this is bigger than the lsn we are able to scan up to, that is an indication that @@ -170,7 +170,8 @@ recv_sys_create(void) return; } - recv_sys = mem_alloc(sizeof(recv_sys_t)); + recv_sys = mem_alloc(sizeof(*recv_sys)); + memset(recv_sys, 0x0, sizeof(*recv_sys)); mutex_create(&recv_sys->mutex, SYNC_RECV); @@ -179,6 +180,106 @@ recv_sys_create(void) } /********************************************************//** +Release recovery system mutexes. */ +UNIV_INTERN +void +recv_sys_close(void) +/*================*/ +{ + if (recv_sys != NULL) { + if (recv_sys->addr_hash != NULL) { + hash_table_free(recv_sys->addr_hash); + } + + if (recv_sys->heap != NULL) { + mem_heap_free(recv_sys->heap); + } + + if (recv_sys->buf != NULL) { + ut_free(recv_sys->buf); + } + + if (recv_sys->last_block_buf_start != NULL) { + mem_free(recv_sys->last_block_buf_start); + } + + mutex_free(&recv_sys->mutex); + + mem_free(recv_sys); + recv_sys = NULL; + } +} + +/********************************************************//** +Frees the recovery system memory. */ +UNIV_INTERN +void +recv_sys_mem_free(void) +/*===================*/ +{ + if (recv_sys != NULL) { + if (recv_sys->addr_hash != NULL) { + hash_table_free(recv_sys->addr_hash); + } + + if (recv_sys->heap != NULL) { + mem_heap_free(recv_sys->heap); + } + + if (recv_sys->buf != NULL) { + ut_free(recv_sys->buf); + } + + if (recv_sys->last_block_buf_start != NULL) { + mem_free(recv_sys->last_block_buf_start); + } + + mem_free(recv_sys); + recv_sys = NULL; + } +} + +/************************************************************ +Reset the state of the recovery system variables. */ +UNIV_INTERN +void +recv_sys_var_init(void) +/*===================*/ +{ + recv_lsn_checks_on = FALSE; + + recv_n_pool_free_frames = 256; + + recv_recovery_on = FALSE; + +#ifdef UNIV_LOG_ARCHIVE + recv_recovery_from_backup_on = FALSE; +#endif /* UNIV_LOG_ARCHIVE */ + + recv_needed_recovery = FALSE; + + recv_lsn_checks_on = FALSE; + + recv_log_scan_is_startup_type = FALSE; + + recv_no_ibuf_operations = FALSE; + + recv_scan_print_counter = 0; + + recv_previous_parsed_rec_type = 999999; + + recv_previous_parsed_rec_offset = 0; + + recv_previous_parsed_rec_is_multi = 0; + + recv_max_parsed_page_no = 0; + + recv_n_pool_free_frames = 256; + + recv_max_page_lsn = 0; +} + +/************************************************************ Inits the recovery system for a recovery operation. */ UNIV_INTERN void @@ -253,8 +354,8 @@ recv_sys_empty_hash(void) Frees the recovery system. */ static void -recv_sys_free(void) -/*===============*/ +recv_sys_debug_free(void) +/*=====================*/ { mutex_enter(&(recv_sys->mutex)); @@ -263,8 +364,10 @@ recv_sys_free(void) ut_free(recv_sys->buf); mem_free(recv_sys->last_block_buf_start); - recv_sys->addr_hash = NULL; + recv_sys->buf = NULL; recv_sys->heap = NULL; + recv_sys->addr_hash = NULL; + recv_sys->last_block_buf_start = NULL; mutex_exit(&(recv_sys->mutex)); } @@ -3149,7 +3252,7 @@ recv_recovery_from_checkpoint_finish(voi recv_recovery_on = FALSE; #ifndef UNIV_LOG_DEBUG - recv_sys_free(); + recv_sys_debug_free(); #endif /* Roll back any recovered data dictionary transactions, so that the data dictionary tables will be free of any locks. === modified file 'storage/innodb_plugin/mem/mem0dbg.c' --- a/storage/innodb_plugin/mem/mem0dbg.c 2009-05-27 09:45:59 +0000 +++ b/storage/innodb_plugin/mem/mem0dbg.c 2009-11-30 11:32:05 +0000 @@ -170,6 +170,17 @@ mem_init( mem_comm_pool = mem_pool_create(size); } + +/******************************************************************//** +Closes the memory system. */ +UNIV_INTERN +void +mem_close(void) +/*===========*/ +{ + mem_pool_free(mem_comm_pool); + mem_comm_pool = NULL; +} #endif /* !UNIV_HOTBACKUP */ #ifdef UNIV_MEM_DEBUG === modified file 'storage/innodb_plugin/mem/mem0pool.c' --- a/storage/innodb_plugin/mem/mem0pool.c 2009-05-27 09:45:59 +0000 +++ b/storage/innodb_plugin/mem/mem0pool.c 2009-11-30 11:32:05 +0000 @@ -261,6 +261,18 @@ mem_pool_create( } /********************************************************************//** +Frees a memory pool. */ +UNIV_INTERN +void +mem_pool_free( +/*==========*/ + mem_pool_t* pool) /*!< in, own: memory pool */ +{ + ut_free(pool->buf); + ut_free(pool); +} + +/********************************************************************//** Fills the specified free list. @return TRUE if we were able to insert a block to the free list */ static === modified file 'storage/innodb_plugin/os/os0file.c' --- a/storage/innodb_plugin/os/os0file.c 2009-11-03 09:59:31 +0000 +++ b/storage/innodb_plugin/os/os0file.c 2009-11-30 12:04:09 +0000 @@ -323,6 +323,13 @@ os_file_get_last_error( "InnoDB: The error means that there are no" " sufficient system resources or quota to" " complete the operation.\n"); + } else if (err == ERROR_OPERATION_ABORTED) { + fprintf(stderr, + "InnoDB: The error means that the I/O" + " operation has been aborted\n" + "InnoDB: because of either a thread exit" + " or an application request.\n" + "InnoDB: Retry attempt is made.\n"); } else { fprintf(stderr, "InnoDB: Some operating system error numbers" @@ -347,6 +354,8 @@ os_file_get_last_error( } else if (err == ERROR_WORKING_SET_QUOTA || err == ERROR_NO_SYSTEM_RESOURCES) { return(OS_FILE_INSUFFICIENT_RESOURCE); + } else if (err == ERROR_OPERATION_ABORTED) { + return(OS_FILE_OPERATION_ABORTED); } else { return(100 + err); } @@ -469,6 +478,10 @@ os_file_handle_error_cond_exit( os_thread_sleep(100000); /* 100 ms */ return(TRUE); + } else if (err == OS_FILE_OPERATION_ABORTED) { + + os_thread_sleep(100000); /* 100 ms */ + return(TRUE); } else { if (name) { fprintf(stderr, "InnoDB: File name %s\n", name); @@ -3029,6 +3042,34 @@ os_aio_array_create( return(array); } +/************************************************************************//** +Frees an aio wait array. */ +static +void +os_aio_array_free( +/*==============*/ + os_aio_array_t* array) /*!< in, own: array to free */ +{ +#ifdef WIN_ASYNC_IO + ulint i; + + for (i = 0; i < array->n_slots; i++) { + os_aio_slot_t* slot = os_aio_array_get_nth_slot(array, i); + os_event_free(slot->event); + } +#endif /* WIN_ASYNC_IO */ + +#ifdef __WIN__ + ut_free(array->native_events); +#endif /* __WIN__ */ + os_mutex_free(array->mutex); + os_event_free(array->not_full); + os_event_free(array->is_empty); + + ut_free(array->slots); + ut_free(array); +} + /*********************************************************************** Initializes the asynchronous io system. Creates one array each for ibuf and log i/o. Also creates one array each for read and write where each @@ -3099,6 +3140,35 @@ os_aio_init( } +/*********************************************************************** +Frees the asynchronous io system. */ +UNIV_INTERN +void +os_aio_free(void) +/*=============*/ +{ + ulint i; + + os_aio_array_free(os_aio_ibuf_array); + os_aio_ibuf_array = NULL; + os_aio_array_free(os_aio_log_array); + os_aio_log_array = NULL; + os_aio_array_free(os_aio_read_array); + os_aio_read_array = NULL; + os_aio_array_free(os_aio_write_array); + os_aio_write_array = NULL; + os_aio_array_free(os_aio_sync_array); + os_aio_sync_array = NULL; + + for (i = 0; i < os_aio_n_segments; i++) { + os_event_free(os_aio_segment_wait_events[i]); + } + + ut_free(os_aio_segment_wait_events); + os_aio_segment_wait_events = 0; + os_aio_n_segments = 0; +} + #ifdef WIN_ASYNC_IO /************************************************************************//** Wakes up all async i/o threads in the array in Windows async i/o at @@ -3709,6 +3779,7 @@ os_aio_windows_handle( ibool ret_val; BOOL ret; DWORD len; + BOOL retry = FALSE; if (segment == ULINT_UNDEFINED) { array = os_aio_sync_array; @@ -3762,14 +3833,52 @@ os_aio_windows_handle( ut_a(TRUE == os_file_flush(slot->file)); } #endif /* UNIV_DO_FLUSH */ + } else if (os_file_handle_error(slot->name, "Windows aio")) { + + retry = TRUE; } else { - os_file_handle_error(slot->name, "Windows aio"); ret_val = FALSE; } os_mutex_exit(array->mutex); + if (retry) { + /* retry failed read/write operation synchronously. + No need to hold array->mutex. */ + + switch (slot->type) { + case OS_FILE_WRITE: + ret = WriteFile(slot->file, slot->buf, + slot->len, &len, + &(slot->control)); + + break; + case OS_FILE_READ: + ret = ReadFile(slot->file, slot->buf, + slot->len, &len, + &(slot->control)); + + break; + default: + ut_error; + } + + if (!ret && GetLastError() == ERROR_IO_PENDING) { + /* aio was queued successfully! + We want a synchronous i/o operation on a + file where we also use async i/o: in Windows + we must use the same wait mechanism as for + async i/o */ + + ret = GetOverlappedResult(slot->file, + &(slot->control), + &len, TRUE); + } + + ret_val = ret && len == slot->len; + } + os_aio_array_free_slot(array, slot); return(ret_val); === modified file 'storage/innodb_plugin/os/os0sync.c' --- a/storage/innodb_plugin/os/os0sync.c 2009-07-30 12:42:56 +0000 +++ b/storage/innodb_plugin/os/os0sync.c 2009-11-30 11:32:05 +0000 @@ -86,6 +86,9 @@ os_sync_init(void) UT_LIST_INIT(os_event_list); UT_LIST_INIT(os_mutex_list); + os_sync_mutex = NULL; + os_sync_mutex_inited = FALSE; + os_sync_mutex = os_mutex_create(NULL); os_sync_mutex_inited = TRUE; @@ -713,6 +716,7 @@ os_fast_mutex_free( os_mutex_enter(os_sync_mutex); } + ut_ad(os_fast_mutex_count > 0); os_fast_mutex_count--; if (UNIV_LIKELY(os_sync_mutex_inited)) { === modified file 'storage/innodb_plugin/os/os0thread.c' --- a/storage/innodb_plugin/os/os0thread.c 2009-05-27 09:45:59 +0000 +++ b/storage/innodb_plugin/os/os0thread.c 2009-11-30 11:32:05 +0000 @@ -233,6 +233,7 @@ os_thread_exit( #ifdef __WIN__ ExitThread((DWORD)exit_value); #else + pthread_detach(pthread_self()); pthread_exit(exit_value); #endif } === modified file 'storage/innodb_plugin/pars/lexyy.c' --- a/storage/innodb_plugin/pars/lexyy.c 2009-05-27 09:45:59 +0000 +++ b/storage/innodb_plugin/pars/lexyy.c 2009-11-30 11:32:05 +0000 @@ -2778,3 +2778,16 @@ static void yyfree (void * ptr ) + +/********************************************************************** +Release any resources used by the lexer. */ +UNIV_INTERN +void +pars_lexer_close(void) +/*==================*/ +{ + yylex_destroy(); + free(stringbuf); + stringbuf = NULL; + stringbuf_len_alloc = stringbuf_len = 0; +} === modified file 'storage/innodb_plugin/pars/pars0lex.l' --- a/storage/innodb_plugin/pars/pars0lex.l 2009-05-27 09:45:59 +0000 +++ b/storage/innodb_plugin/pars/pars0lex.l 2009-11-30 11:32:05 +0000 @@ -661,3 +661,16 @@ In the state 'id', only two actions are } %% + +/********************************************************************** +Release any resources used by the lexer. */ +UNIV_INTERN +void +pars_lexer_close(void) +/*==================*/ +{ + yylex_destroy(); + free(stringbuf); + stringbuf = NULL; + stringbuf_len_alloc = stringbuf_len = 0; +} === modified file 'storage/innodb_plugin/que/que0que.c' --- a/storage/innodb_plugin/que/que0que.c 2009-07-30 12:42:56 +0000 +++ b/storage/innodb_plugin/que/que0que.c 2009-11-30 11:32:05 +0000 @@ -518,6 +518,7 @@ que_graph_free_recursive( upd_node_t* upd; tab_node_t* cre_tab; ind_node_t* cre_ind; + purge_node_t* purge; if (node == NULL) { @@ -579,6 +580,13 @@ que_graph_free_recursive( mem_heap_free(ins->entry_sys_heap); break; + case QUE_NODE_PURGE: + purge = node; + + mem_heap_free(purge->heap); + + break; + case QUE_NODE_UPDATE: upd = node; === modified file 'storage/innodb_plugin/row/row0merge.c' --- a/storage/innodb_plugin/row/row0merge.c 2009-10-09 14:13:15 +0000 +++ b/storage/innodb_plugin/row/row0merge.c 2009-11-30 12:24:54 +0000 @@ -1200,6 +1200,12 @@ row_merge_read_clustered_index( in order to release the latch on the old page. */ if (btr_pcur_is_after_last_on_page(&pcur)) { + if (UNIV_UNLIKELY(trx_is_interrupted(trx))) { + i = 0; + err = DB_INTERRUPTED; + goto err_exit; + } + btr_pcur_store_position(&pcur, &mtr); mtr_commit(&mtr); mtr_start(&mtr); @@ -1557,6 +1563,7 @@ static __attribute__((nonnull)) ulint row_merge( /*======*/ + trx_t* trx, /*!< in: transaction */ const dict_index_t* index, /*!< in: index being created */ merge_file_t* file, /*!< in/out: file containing index entries */ @@ -1590,6 +1597,10 @@ row_merge( for (; foffs0 < ihalf && foffs1 < file->offset; foffs0++, foffs1++) { ulint ahalf; /*!< arithmetic half the input file */ + if (UNIV_UNLIKELY(trx_is_interrupted(trx))) { + return(DB_INTERRUPTED); + } + error = row_merge_blocks(index, file, block, &foffs0, &foffs1, &of, table); @@ -1617,6 +1628,10 @@ row_merge( /* Copy the last blocks, if there are any. */ while (foffs0 < ihalf) { + if (UNIV_UNLIKELY(trx_is_interrupted(trx))) { + return(DB_INTERRUPTED); + } + if (!row_merge_blocks_copy(index, file, block, &foffs0, &of)) { return(DB_CORRUPTION); } @@ -1625,6 +1640,10 @@ row_merge( ut_ad(foffs0 == ihalf); while (foffs1 < file->offset) { + if (UNIV_UNLIKELY(trx_is_interrupted(trx))) { + return(DB_INTERRUPTED); + } + if (!row_merge_blocks_copy(index, file, block, &foffs1, &of)) { return(DB_CORRUPTION); } @@ -1653,6 +1672,7 @@ static ulint row_merge_sort( /*===========*/ + trx_t* trx, /*!< in: transaction */ const dict_index_t* index, /*!< in: index being created */ merge_file_t* file, /*!< in/out: file containing index entries */ @@ -1671,7 +1691,8 @@ row_merge_sort( do { ulint error; - error = row_merge(index, file, &half, block, tmpfd, table); + error = row_merge(trx, index, file, &half, + block, tmpfd, table); if (error != DB_SUCCESS) { return(error); @@ -2490,7 +2511,7 @@ row_merge_build_indexes( sorting and inserting. */ for (i = 0; i < n_indexes; i++) { - error = row_merge_sort(indexes[i], &merge_files[i], + error = row_merge_sort(trx, indexes[i], &merge_files[i], block, &tmpfd, table); if (error == DB_SUCCESS) { === modified file 'storage/innodb_plugin/row/row0mysql.c' --- a/storage/innodb_plugin/row/row0mysql.c 2009-11-03 10:32:33 +0000 +++ b/storage/innodb_plugin/row/row0mysql.c 2009-11-30 13:13:34 +0000 @@ -1880,6 +1880,8 @@ err_exit: if (UNIV_UNLIKELY(err != DB_SUCCESS)) { trx->error_state = DB_SUCCESS; trx_general_rollback_for_mysql(trx, NULL); + /* TO DO: free table? The code below will dereference + table->name, though. */ } switch (err) { @@ -1898,31 +1900,6 @@ err_exit: break; case DB_DUPLICATE_KEY: - ut_print_timestamp(stderr); - fputs(" InnoDB: Error: table ", stderr); - ut_print_name(stderr, trx, TRUE, table->name); - fputs(" already exists in InnoDB internal\n" - "InnoDB: data dictionary. Have you deleted" - " the .frm file\n" - "InnoDB: and not used DROP TABLE?" - " Have you used DROP DATABASE\n" - "InnoDB: for InnoDB tables in" - " MySQL version <= 3.23.43?\n" - "InnoDB: See the Restrictions section" - " of the InnoDB manual.\n" - "InnoDB: You can drop the orphaned table" - " inside InnoDB by\n" - "InnoDB: creating an InnoDB table with" - " the same name in another\n" - "InnoDB: database and copying the .frm file" - " to the current database.\n" - "InnoDB: Then MySQL thinks the table exists," - " and DROP TABLE will\n" - "InnoDB: succeed.\n" - "InnoDB: You can look for further help from\n" - "InnoDB: " REFMAN "innodb-troubleshooting.html\n", - stderr); - /* We may also get err == DB_ERROR if the .ibd file for the table already exists */ @@ -4157,6 +4134,7 @@ row_check_table_for_mysql( } if (trx_is_interrupted(prebuilt->trx)) { + ret = DB_INTERRUPTED; break; } === modified file 'storage/innodb_plugin/srv/srv0srv.c' --- a/storage/innodb_plugin/srv/srv0srv.c 2009-10-09 12:19:13 +0000 +++ b/storage/innodb_plugin/srv/srv0srv.c 2009-11-30 11:32:05 +0000 @@ -1006,13 +1006,26 @@ srv_init(void) } /*********************************************************************//** -Frees the OS fast mutex created in srv_init(). */ +Frees the data structures created in srv_init(). */ UNIV_INTERN void srv_free(void) /*==========*/ { os_fast_mutex_free(&srv_conc_mutex); + mem_free(srv_conc_slots); + srv_conc_slots = NULL; + + mem_free(srv_sys->threads); + mem_free(srv_sys); + srv_sys = NULL; + + mem_free(kernel_mutex_temp); + kernel_mutex_temp = NULL; + mem_free(srv_mysql_table); + srv_mysql_table = NULL; + + trx_i_s_cache_free(trx_i_s_cache); } /*********************************************************************//** @@ -1024,6 +1037,8 @@ srv_general_init(void) /*==================*/ { ut_mem_init(); + /* Reset the system variables in the recovery module. */ + recv_sys_var_init(); os_sync_init(); sync_init(); mem_init(srv_mem_pool_size); === modified file 'storage/innodb_plugin/srv/srv0start.c' --- a/storage/innodb_plugin/srv/srv0start.c 2009-11-03 10:23:22 +0000 +++ b/storage/innodb_plugin/srv/srv0start.c 2009-11-30 11:32:05 +0000 @@ -103,6 +103,7 @@ Created 2/16/1996 Heikki Tuuri # include "row0row.h" # include "row0mysql.h" # include "btr0pcur.h" +# include "thr0loc.h" # include "os0sync.h" /* for INNODB_RW_LOCKS_USE_ATOMICS */ /** Log sequence number immediately after startup */ @@ -495,6 +496,8 @@ io_handler_thread( mutex_exit(&ios_mutex); } + thr_local_free(os_thread_get_curr_id()); + /* We count the number of threads in os_thread_exit(). A created thread should always use that to exit and not use return() to exit. The thread actually never comes here because it is exited in an @@ -531,32 +534,6 @@ srv_normalize_path_for_win( #endif } -/*********************************************************************//** -Adds a slash or a backslash to the end of a string if it is missing -and the string is not empty. -@return string which has the separator if the string is not empty */ -UNIV_INTERN -char* -srv_add_path_separator_if_needed( -/*=============================*/ - char* str) /*!< in: null-terminated character string */ -{ - char* out_str; - ulint len = ut_strlen(str); - - if (len == 0 || str[len - 1] == SRV_PATH_SEPARATOR) { - - return(str); - } - - out_str = ut_malloc(len + 2); - memcpy(out_str, str, len); - out_str[len] = SRV_PATH_SEPARATOR; - out_str[len + 1] = 0; - - return(out_str); -} - #ifndef UNIV_HOTBACKUP /*********************************************************************//** Calculates the low 32 bits when a file size which is given as a number @@ -605,19 +582,24 @@ open_or_create_log_file( ulint size; ulint size_high; char name[10000]; + ulint dirnamelen; UT_NOT_USED(create_new_db); *log_file_created = FALSE; srv_normalize_path_for_win(srv_log_group_home_dirs[k]); - srv_log_group_home_dirs[k] = srv_add_path_separator_if_needed( - srv_log_group_home_dirs[k]); - ut_a(strlen(srv_log_group_home_dirs[k]) - < (sizeof name) - 10 - sizeof "ib_logfile"); - sprintf(name, "%s%s%lu", srv_log_group_home_dirs[k], - "ib_logfile", (ulong) i); + dirnamelen = strlen(srv_log_group_home_dirs[k]); + ut_a(dirnamelen < (sizeof name) - 10 - sizeof "ib_logfile"); + memcpy(name, srv_log_group_home_dirs[k], dirnamelen); + + /* Add a path separator if needed. */ + if (dirnamelen && name[dirnamelen - 1] != SRV_PATH_SEPARATOR) { + name[dirnamelen++] = SRV_PATH_SEPARATOR; + } + + sprintf(name + dirnamelen, "%s%lu", "ib_logfile", (ulong) i); files[i] = os_file_create(name, OS_FILE_CREATE, OS_FILE_NORMAL, OS_LOG_FILE, &ret); @@ -780,14 +762,22 @@ open_or_create_data_files( *create_new_db = FALSE; srv_normalize_path_for_win(srv_data_home); - srv_data_home = srv_add_path_separator_if_needed(srv_data_home); for (i = 0; i < srv_n_data_files; i++) { + ulint dirnamelen; + srv_normalize_path_for_win(srv_data_file_names[i]); + dirnamelen = strlen(srv_data_home); - ut_a(strlen(srv_data_home) + strlen(srv_data_file_names[i]) + ut_a(dirnamelen + strlen(srv_data_file_names[i]) < (sizeof name) - 1); - sprintf(name, "%s%s", srv_data_home, srv_data_file_names[i]); + memcpy(name, srv_data_home, dirnamelen); + /* Add a path separator if needed. */ + if (dirnamelen && name[dirnamelen - 1] != SRV_PATH_SEPARATOR) { + name[dirnamelen++] = SRV_PATH_SEPARATOR; + } + + strcpy(name + dirnamelen, srv_data_file_names[i]); if (srv_data_file_is_raw_partition[i] == 0) { @@ -1009,7 +999,7 @@ skip_size_check: return(DB_SUCCESS); } -/****************************************************************//** +/******************************************************************** Starts InnoDB and creates a new database if database files are not found and the user wants. @return DB_SUCCESS or error code */ @@ -1120,7 +1110,7 @@ innobase_start_or_create_for_mysql(void) if (srv_start_has_been_called) { fprintf(stderr, - "InnoDB: Error:startup called second time" + "InnoDB: Error: startup called second time" " during the process lifetime.\n" "InnoDB: In the MySQL Embedded Server Library" " you cannot call server_init()\n" @@ -1959,8 +1949,10 @@ innobase_shutdown_for_mysql(void) /* All the threads have exited or are just exiting; NOTE that the threads may not have completed their exit yet. Should we use pthread_join() to make sure - they have exited? Now we just sleep 0.1 seconds and - hope that is enough! */ + they have exited? If we did, we would have to + remove the pthread_detach() from + os_thread_exit(). Now we just sleep 0.1 + seconds and hope that is enough! */ os_mutex_exit(os_sync_mutex); @@ -1999,37 +1991,41 @@ innobase_shutdown_for_mysql(void) srv_misc_tmpfile = 0; } + /* This must be disabled before closing the buffer pool + and closing the data dictionary. */ + btr_search_disable(); + + ibuf_close(); + log_shutdown(); + lock_sys_close(); + thr_local_close(); trx_sys_file_format_close(); + trx_sys_close(); mutex_free(&srv_monitor_file_mutex); mutex_free(&srv_dict_tmpfile_mutex); mutex_free(&srv_misc_tmpfile_mutex); + dict_close(); + btr_search_sys_free(); /* 3. Free all InnoDB's own mutexes and the os_fast_mutexes inside them */ + os_aio_free(); sync_close(); + srv_free(); + fil_close(); /* 4. Free the os_conc_mutex and all os_events and os_mutexes */ - srv_free(); os_sync_free(); - /* Check that all read views are closed except read view owned - by a purge. */ - - if (UT_LIST_GET_LEN(trx_sys->view_list) > 1) { - fprintf(stderr, - "InnoDB: Error: all read views were not closed" - " before shutdown:\n" - "InnoDB: %lu read views open \n", - UT_LIST_GET_LEN(trx_sys->view_list) - 1); - } - - /* 5. Free all allocated memory and the os_fast_mutex created in - ut0mem.c */ + /* 5. Free all allocated memory */ + pars_lexer_close(); + log_mem_free(); buf_pool_free(); ut_free_all_mem(); + mem_close(); if (os_thread_count != 0 || os_event_count != 0 @@ -2060,6 +2056,7 @@ innobase_shutdown_for_mysql(void) } srv_was_started = FALSE; + srv_start_has_been_called = FALSE; return((int) DB_SUCCESS); } === modified file 'storage/innodb_plugin/sync/sync0arr.c' --- a/storage/innodb_plugin/sync/sync0arr.c 2009-05-27 09:45:59 +0000 +++ b/storage/innodb_plugin/sync/sync0arr.c 2009-11-30 11:32:05 +0000 @@ -227,24 +227,21 @@ sync_array_create( SYNC_ARRAY_MUTEX: determines the type of mutex protecting the data structure */ { + ulint sz; sync_array_t* arr; - sync_cell_t* cell_array; - sync_cell_t* cell; - ulint i; ut_a(n_cells > 0); /* Allocate memory for the data structures */ arr = ut_malloc(sizeof(sync_array_t)); + memset(arr, 0x0, sizeof(*arr)); - cell_array = ut_malloc(sizeof(sync_cell_t) * n_cells); + sz = sizeof(sync_cell_t) * n_cells; + arr->array = ut_malloc(sz); + memset(arr->array, 0x0, sz); arr->n_cells = n_cells; - arr->n_reserved = 0; - arr->array = cell_array; arr->protection = protection; - arr->sg_count = 0; - arr->res_count = 0; /* Then create the mutex to protect the wait array complex */ if (protection == SYNC_ARRAY_OS_MUTEX) { @@ -255,13 +252,6 @@ sync_array_create( ut_error; } - for (i = 0; i < n_cells; i++) { - cell = sync_array_get_nth_cell(arr, i); - cell->wait_object = NULL; - cell->waiting = FALSE; - cell->signal_count = 0; - } - return(arr); } === modified file 'storage/innodb_plugin/sync/sync0sync.c' --- a/storage/innodb_plugin/sync/sync0sync.c 2009-10-12 12:00:56 +0000 +++ b/storage/innodb_plugin/sync/sync0sync.c 2009-11-30 11:32:05 +0000 @@ -1377,7 +1377,12 @@ sync_close(void) mutex_free(&mutex_list_mutex); #ifdef UNIV_SYNC_DEBUG mutex_free(&sync_thread_mutex); + + /* Switch latching order checks on in sync0sync.c */ + sync_order_checks_on = FALSE; #endif /* UNIV_SYNC_DEBUG */ + + sync_initialized = FALSE; } /*******************************************************************//** === modified file 'storage/innodb_plugin/thr/thr0loc.c' --- a/storage/innodb_plugin/thr/thr0loc.c 2009-10-08 10:00:49 +0000 +++ b/storage/innodb_plugin/thr/thr0loc.c 2009-11-30 11:32:05 +0000 @@ -246,3 +246,34 @@ thr_local_init(void) mutex_create(&thr_local_mutex, SYNC_THR_LOCAL); } + +/******************************************************************** +Close the thread local storage module. */ +UNIV_INTERN +void +thr_local_close(void) +/*=================*/ +{ + ulint i; + + ut_a(thr_local_hash != NULL); + + /* Free the hash elements. We don't remove them from the table + because we are going to destroy the table anyway. */ + for (i = 0; i < hash_get_n_cells(thr_local_hash); i++) { + thr_local_t* local; + + local = HASH_GET_FIRST(thr_local_hash, i); + + while (local) { + thr_local_t* prev_local = local; + + local = HASH_GET_NEXT(hash, prev_local); + ut_a(prev_local->magic_n == THR_LOCAL_MAGIC_N); + mem_free(prev_local); + } + } + + hash_table_free(thr_local_hash); + thr_local_hash = NULL; +} === modified file 'storage/innodb_plugin/trx/trx0i_s.c' --- a/storage/innodb_plugin/trx/trx0i_s.c 2009-05-27 09:45:59 +0000 +++ b/storage/innodb_plugin/trx/trx0i_s.c 2009-12-01 10:38:40 +0000 @@ -60,7 +60,7 @@ Created July 17, 2007 Vasil Dimov /** @brief The maximum number of chunks to allocate for a table cache. The rows of a table cache are stored in a set of chunks. When a new -row is added a new chunk is allocated if necessary. Assuming that the +row is added a new chunk is allocated if necessary. Assuming that the first one is 1024 rows (TABLE_CACHE_INITIAL_ROWSNUM) and each subsequent is N/2 where N is the number of rows we have allocated till now, then 39th chunk would accommodate 1677416425 rows and all chunks @@ -238,6 +238,27 @@ table_cache_init( } /*******************************************************************//** +Frees a table cache. */ +static +void +table_cache_free( +/*=============*/ + i_s_table_cache_t* table_cache) /*!< in/out: table cache */ +{ + ulint i; + + for (i = 0; i < MEM_CHUNKS_IN_TABLE_CACHE; i++) { + + /* the memory is actually allocated in + table_cache_create_empty_row() */ + if (table_cache->chunks[i].base) { + mem_free(table_cache->chunks[i].base); + table_cache->chunks[i].base = NULL; + } + } +} + +/*******************************************************************//** Returns an empty row from a table cache. The row is allocated if no more empty rows are available. The number of used rows is incremented. If the memory limit is hit then NULL is returned and nothing is @@ -1184,9 +1205,6 @@ trx_i_s_possibly_fetch_data_into_cache( return(1); } - /* We are going to access trx->query in all transactions */ - innobase_mysql_prepare_print_arbitrary_thd(); - /* We need to read trx_sys and record/table lock queues */ mutex_enter(&kernel_mutex); @@ -1194,8 +1212,6 @@ trx_i_s_possibly_fetch_data_into_cache( mutex_exit(&kernel_mutex); - innobase_mysql_end_print_arbitrary_thd(); - return(0); } @@ -1252,6 +1268,22 @@ trx_i_s_cache_init( } /*******************************************************************//** +Free the INFORMATION SCHEMA trx related cache. */ +UNIV_INTERN +void +trx_i_s_cache_free( +/*===============*/ + trx_i_s_cache_t* cache) /*!< in, own: cache to free */ +{ + hash_table_free(cache->locks_hash); + ha_storage_free(cache->storage); + table_cache_free(&cache->innodb_trx); + table_cache_free(&cache->innodb_locks); + table_cache_free(&cache->innodb_lock_waits); + memset(cache, 0, sizeof *cache); +} + +/*******************************************************************//** Issue a shared/read lock on the tables cache. */ UNIV_INTERN void === modified file 'storage/innodb_plugin/trx/trx0purge.c' --- a/storage/innodb_plugin/trx/trx0purge.c 2009-07-30 12:42:56 +0000 +++ b/storage/innodb_plugin/trx/trx0purge.c 2009-11-30 11:32:05 +0000 @@ -249,6 +249,44 @@ trx_purge_sys_create(void) purge_sys->heap); } +/************************************************************************ +Frees the global purge system control structure. */ +UNIV_INTERN +void +trx_purge_sys_close(void) +/*======================*/ +{ + ut_ad(!mutex_own(&kernel_mutex)); + + que_graph_free(purge_sys->query); + + ut_a(purge_sys->sess->trx->is_purge); + purge_sys->sess->trx->conc_state = TRX_NOT_STARTED; + sess_close(purge_sys->sess); + purge_sys->sess = NULL; + + if (purge_sys->view != NULL) { + /* Because acquiring the kernel mutex is a pre-condition + of read_view_close(). We don't really need it here. */ + mutex_enter(&kernel_mutex); + + read_view_close(purge_sys->view); + purge_sys->view = NULL; + + mutex_exit(&kernel_mutex); + } + + trx_undo_arr_free(purge_sys->arr); + + rw_lock_free(&purge_sys->latch); + mutex_free(&purge_sys->mutex); + + mem_heap_free(purge_sys->heap); + mem_free(purge_sys); + + purge_sys = NULL; +} + /*================ UNDO LOG HISTORY LIST =============================*/ /********************************************************************//** === modified file 'storage/innodb_plugin/trx/trx0rseg.c' --- a/storage/innodb_plugin/trx/trx0rseg.c 2009-05-27 09:45:59 +0000 +++ b/storage/innodb_plugin/trx/trx0rseg.c 2009-11-30 11:32:05 +0000 @@ -132,6 +132,49 @@ trx_rseg_header_create( } /***********************************************************************//** +Free's an instance of the rollback segment in memory. */ +UNIV_INTERN +void +trx_rseg_mem_free( +/*==============*/ + trx_rseg_t* rseg) /* in, own: instance to free */ +{ + trx_undo_t* undo; + + mutex_free(&rseg->mutex); + + /* There can't be any active transactions. */ + ut_a(UT_LIST_GET_LEN(rseg->update_undo_list) == 0); + ut_a(UT_LIST_GET_LEN(rseg->insert_undo_list) == 0); + + undo = UT_LIST_GET_FIRST(rseg->update_undo_cached); + + while (undo != NULL) { + trx_undo_t* prev_undo = undo; + + undo = UT_LIST_GET_NEXT(undo_list, undo); + UT_LIST_REMOVE(undo_list, rseg->update_undo_cached, prev_undo); + + trx_undo_mem_free(prev_undo); + } + + undo = UT_LIST_GET_FIRST(rseg->insert_undo_cached); + + while (undo != NULL) { + trx_undo_t* prev_undo = undo; + + undo = UT_LIST_GET_NEXT(undo_list, undo); + UT_LIST_REMOVE(undo_list, rseg->insert_undo_cached, prev_undo); + + trx_undo_mem_free(prev_undo); + } + + trx_sys_set_nth_rseg(trx_sys, rseg->id, NULL); + + mem_free(rseg); +} + +/*************************************************************************** Creates and initializes a rollback segment object. The values for the fields are read from the header. The object is inserted to the rseg list of the trx system object and a pointer is inserted in the rseg === modified file 'storage/innodb_plugin/trx/trx0sys.c' --- a/storage/innodb_plugin/trx/trx0sys.c 2009-07-30 12:42:56 +0000 +++ b/storage/innodb_plugin/trx/trx0sys.c 2009-11-30 11:32:05 +0000 @@ -40,6 +40,7 @@ Created 3/26/1996 Heikki Tuuri #include "trx0purge.h" #include "log0log.h" #include "os0file.h" +#include "read0read.h" /** The file format tag structure with id and name. */ struct file_format_struct { @@ -1533,3 +1534,80 @@ trx_sys_file_format_id_to_name( } #endif /* !UNIV_HOTBACKUP */ + +/********************************************************************* +Shutdown/Close the transaction system. */ +UNIV_INTERN +void +trx_sys_close(void) +/*===============*/ +{ + trx_rseg_t* rseg; + read_view_t* view; + + ut_ad(trx_sys != NULL); + + /* Check that all read views are closed except read view owned + by a purge. */ + + if (UT_LIST_GET_LEN(trx_sys->view_list) > 1) { + fprintf(stderr, + "InnoDB: Error: all read views were not closed" + " before shutdown:\n" + "InnoDB: %lu read views open \n", + UT_LIST_GET_LEN(trx_sys->view_list) - 1); + } + + sess_close(trx_dummy_sess); + trx_dummy_sess = NULL; + + trx_purge_sys_close(); + + mutex_enter(&kernel_mutex); + + /* Free the double write data structures. */ + ut_a(trx_doublewrite != NULL); + ut_free(trx_doublewrite->write_buf_unaligned); + trx_doublewrite->write_buf_unaligned = NULL; + + mem_free(trx_doublewrite->buf_block_arr); + trx_doublewrite->buf_block_arr = NULL; + + mutex_free(&trx_doublewrite->mutex); + mem_free(trx_doublewrite); + trx_doublewrite = NULL; + + /* There can't be any active transactions. */ + rseg = UT_LIST_GET_FIRST(trx_sys->rseg_list); + + while (rseg != NULL) { + trx_rseg_t* prev_rseg = rseg; + + rseg = UT_LIST_GET_NEXT(rseg_list, prev_rseg); + UT_LIST_REMOVE(rseg_list, trx_sys->rseg_list, prev_rseg); + + trx_rseg_mem_free(prev_rseg); + } + + view = UT_LIST_GET_FIRST(trx_sys->view_list); + + while (view != NULL) { + read_view_t* prev_view = view; + + view = UT_LIST_GET_NEXT(view_list, prev_view); + + /* Views are allocated from the trx_sys->global_read_view_heap. + So, we simply remove the element here. */ + UT_LIST_REMOVE(view_list, trx_sys->view_list, prev_view); + } + + ut_a(UT_LIST_GET_LEN(trx_sys->trx_list) == 0); + ut_a(UT_LIST_GET_LEN(trx_sys->rseg_list) == 0); + ut_a(UT_LIST_GET_LEN(trx_sys->view_list) == 0); + ut_a(UT_LIST_GET_LEN(trx_sys->mysql_trx_list) == 0); + + mem_free(trx_sys); + + trx_sys = NULL; + mutex_exit(&kernel_mutex); +} === modified file 'storage/innodb_plugin/trx/trx0trx.c' --- a/storage/innodb_plugin/trx/trx0trx.c 2009-10-09 14:13:15 +0000 +++ b/storage/innodb_plugin/trx/trx0trx.c 2009-12-01 10:38:40 +0000 @@ -1636,9 +1636,7 @@ trx_mark_sql_stat_end( /**********************************************************************//** Prints info about a transaction to the given file. The caller must own the -kernel mutex and must have called -innobase_mysql_prepare_print_arbitrary_thd(), unless he knows that MySQL -or InnoDB cannot meanwhile change the info printed here. */ +kernel mutex. */ UNIV_INTERN void trx_print( === modified file 'storage/innodb_plugin/trx/trx0undo.c' --- a/storage/innodb_plugin/trx/trx0undo.c 2009-07-30 12:42:56 +0000 +++ b/storage/innodb_plugin/trx/trx0undo.c 2009-11-30 11:32:05 +0000 @@ -1522,7 +1522,7 @@ trx_undo_mem_init_for_reuse( /********************************************************************//** Frees an undo log memory copy. */ -static +UNIV_INTERN void trx_undo_mem_free( /*==============*/ === modified file 'storage/innodb_plugin/usr/usr0sess.c' --- a/storage/innodb_plugin/usr/usr0sess.c 2009-05-27 09:45:59 +0000 +++ b/storage/innodb_plugin/usr/usr0sess.c 2009-11-30 11:32:05 +0000 @@ -32,14 +32,6 @@ Created 6/25/1996 Heikki Tuuri #include "trx0trx.h" /*********************************************************************//** -Closes a session, freeing the memory occupied by it. */ -static -void -sess_close( -/*=======*/ - sess_t* sess); /*!< in, own: session object */ - -/*********************************************************************//** Opens a session. @return own: session object */ UNIV_INTERN @@ -64,35 +56,16 @@ sess_open(void) /*********************************************************************//** Closes a session, freeing the memory occupied by it. */ -static +UNIV_INTERN void sess_close( /*=======*/ sess_t* sess) /*!< in, own: session object */ { - ut_ad(mutex_own(&kernel_mutex)); - ut_ad(sess->trx == NULL); - - mem_free(sess); -} - -/*********************************************************************//** -Closes a session, freeing the memory occupied by it, if it is in a state -where it should be closed. -@return TRUE if closed */ -UNIV_INTERN -ibool -sess_try_close( -/*===========*/ - sess_t* sess) /*!< in, own: session object */ -{ - ut_ad(mutex_own(&kernel_mutex)); + ut_ad(!mutex_own(&kernel_mutex)); - if (UT_LIST_GET_LEN(sess->graphs) == 0) { - sess_close(sess); + ut_a(UT_LIST_GET_LEN(sess->graphs) == 0); - return(TRUE); - } - - return(FALSE); + trx_free_for_background(sess->trx); + mem_free(sess); } === modified file 'storage/innodb_plugin/ut/ut0mem.c' --- a/storage/innodb_plugin/ut/ut0mem.c 2009-07-30 12:42:56 +0000 +++ b/storage/innodb_plugin/ut/ut0mem.c 2009-11-30 11:32:05 +0000 @@ -433,6 +433,8 @@ ut_free_all_mem(void) " total allocated memory is %lu\n", (ulong) ut_total_allocated_memory); } + + ut_mem_block_list_inited = FALSE; } #endif /* !UNIV_HOTBACKUP */ === modified file 'storage/maria/ft_maria.c' --- a/storage/maria/ft_maria.c 2007-07-02 17:45:15 +0000 +++ b/storage/maria/ft_maria.c 2010-01-27 21:53:08 +0000 @@ -22,8 +22,8 @@ #include "ma_ftdefs.h" FT_INFO *maria_ft_init_search(uint flags, void *info, uint keynr, - uchar *query, uint query_len, CHARSET_INFO *cs, - uchar *record) + uchar *query, size_t query_len, + CHARSET_INFO *cs, uchar *record) { FT_INFO *res; if (flags & FT_BOOL) === modified file 'storage/maria/ma_blockrec.c' --- a/storage/maria/ma_blockrec.c 2010-01-06 21:27:53 +0000 +++ b/storage/maria/ma_blockrec.c 2010-01-28 11:35:10 +0000 @@ -1688,7 +1688,8 @@ static my_bool get_head_or_tail_page(MAR if (!page_link.changed) goto crashed; - DBUG_ASSERT((res->buff[PAGE_TYPE_OFFSET] & PAGE_TYPE_MASK) == page_type); + DBUG_ASSERT((uint) (res->buff[PAGE_TYPE_OFFSET] & PAGE_TYPE_MASK) == + page_type); if (!(dir= find_free_position(page_type == HEAD_PAGE ? info : 0, res->buff, block_size, &res->rownr, &res->length, &res->empty_space))) === modified file 'storage/maria/ma_check.c' --- a/storage/maria/ma_check.c 2009-11-29 23:08:56 +0000 +++ b/storage/maria/ma_check.c 2010-01-14 16:51:00 +0000 @@ -6018,7 +6018,7 @@ int maria_update_state_info(HA_CHECK *pa { if (update & UPDATE_TIME) { - share->state.check_time= (long) time((time_t*) 0); + share->state.check_time= time((time_t*) 0); if (!share->state.create_time) share->state.create_time= share->state.check_time; } === modified file 'storage/maria/ma_close.c' --- a/storage/maria/ma_close.c 2009-02-05 22:38:30 +0000 +++ b/storage/maria/ma_close.c 2010-01-29 18:42:22 +0000 @@ -177,6 +177,7 @@ int maria_close(register MARIA_HA *info) { (void) pthread_mutex_destroy(&share->intern_lock); (void) pthread_mutex_destroy(&share->close_lock); + (void) pthread_cond_destroy(&share->key_del_cond); my_free((uchar *)share, MYF(0)); /* If share cannot be freed, it's because checkpoint has previously === modified file 'storage/maria/ma_create.c' --- a/storage/maria/ma_create.c 2009-02-19 09:01:25 +0000 +++ b/storage/maria/ma_create.c 2010-01-14 16:51:00 +0000 @@ -772,7 +772,7 @@ int maria_create(const char *name, enum share.base.min_block_length= share.base.pack_reclength; if (! (flags & HA_DONT_TOUCH_DATA)) - share.state.create_time= (long) time((time_t*) 0); + share.state.create_time= time((time_t*) 0); pthread_mutex_lock(&THR_LOCK_maria); === modified file 'storage/myisam/ft_boolean_search.c' --- a/storage/myisam/ft_boolean_search.c 2009-11-30 13:36:06 +0000 +++ b/storage/myisam/ft_boolean_search.c 2010-01-28 14:49:14 +0000 @@ -299,7 +299,7 @@ static int ftb_parse_query_internal(MYSQ } -static int _ftb_parse_query(FTB *ftb, uchar *query, uint len, +static int _ftb_parse_query(FTB *ftb, uchar *query, mysql_ft_size_t len, struct st_mysql_ftparser *parser) { MYSQL_FTPARSER_PARAM *param; @@ -321,7 +321,7 @@ static int _ftb_parse_query(FTB *ftb, uc param->mysql_add_word= ftb_query_add_word; param->mysql_ftparam= (void *)&ftb_param; param->cs= ftb->charset; - param->doc= (char*) query; + param->doc= query; param->length= len; param->flags= 0; param->mode= MYSQL_FTPARSER_FULL_BOOLEAN_INFO; @@ -475,8 +475,7 @@ static void _ftb_init_index_search(FT_IN int i; FTB_WORD *ftbw; - if ((ftb->state != READY && ftb->state !=INDEX_DONE) || - ftb->keynr == NO_SUCH_KEY) + if (ftb->state == UNINITIALIZED || ftb->keynr == NO_SUCH_KEY) return; ftb->state=INDEX_SEARCH; @@ -541,7 +540,7 @@ static void _ftb_init_index_search(FT_IN FT_INFO * ft_init_boolean_search(MI_INFO *info, uint keynr, uchar *query, - uint query_len, CHARSET_INFO *cs) + mysql_ft_size_t query_len, CHARSET_INFO *cs) { FTB *ftb; FTB_EXPR *ftbe; @@ -680,8 +679,9 @@ static int ftb_check_phrase_internal(MYS -1 is returned if error occurs. */ -static int _ftb_check_phrase(FTB *ftb, const uchar *document, uint len, - FTB_EXPR *ftbe, struct st_mysql_ftparser *parser) +static int _ftb_check_phrase(FTB *ftb, const uchar *document, + mysql_ft_size_t len, + FTB_EXPR *ftbe, struct st_mysql_ftparser *parser) { MY_FTB_PHRASE_PARAM ftb_param; MYSQL_FTPARSER_PARAM *param; @@ -702,7 +702,7 @@ static int _ftb_check_phrase(FTB *ftb, c param->mysql_add_word= ftb_phrase_add_word; param->mysql_ftparam= (void *)&ftb_param; param->cs= ftb->charset; - param->doc= (char *) document; + param->doc= document; param->length= len; param->flags= 0; param->mode= MYSQL_FTPARSER_WITH_STOPWORDS; @@ -1000,7 +1000,7 @@ float ft_boolean_find_relevance(FT_INFO { if (!ftsi.pos) continue; - param->doc= (char *)ftsi.pos; + param->doc= ftsi.pos; param->length= ftsi.len; if (unlikely(parser->parse(param))) return 0; === modified file 'storage/myisam/ft_myisam.c' --- a/storage/myisam/ft_myisam.c 2007-07-02 17:45:15 +0000 +++ b/storage/myisam/ft_myisam.c 2010-01-27 21:53:08 +0000 @@ -23,8 +23,8 @@ #include "ftdefs.h" FT_INFO *ft_init_search(uint flags, void *info, uint keynr, - uchar *query, uint query_len, CHARSET_INFO *cs, - uchar *record) + uchar *query, size_t query_len, + CHARSET_INFO *cs, uchar *record) { FT_INFO *res; if (flags & FT_BOOL) === modified file 'storage/myisam/ft_nlq_search.c' --- a/storage/myisam/ft_nlq_search.c 2009-11-30 13:36:06 +0000 +++ b/storage/myisam/ft_nlq_search.c 2010-01-27 21:53:08 +0000 @@ -204,7 +204,8 @@ static int FT_DOC_cmp(void *unused __att FT_INFO *ft_init_nlq_search(MI_INFO *info, uint keynr, uchar *query, - uint query_len, uint flags, uchar *record) + mysql_ft_size_t query_len, uint flags, + uchar *record) { TREE wtree; ALL_IN_ONE aio; === modified file 'storage/myisam/ft_parser.c' --- a/storage/myisam/ft_parser.c 2009-11-30 13:36:06 +0000 +++ b/storage/myisam/ft_parser.c 2010-01-28 14:49:14 +0000 @@ -175,7 +175,7 @@ uchar ft_get_word(CHARSET_INFO *cs, cons if ((param->trunc=(doc<end && *doc == FTB_TRUNC))) doc++; - if (((length >= ft_min_word_len && !is_stopword((char*) word->pos, + if (((length >= ft_min_word_len && !is_stopword(word->pos, word->len)) || param->trunc) && length < ft_max_word_len) { @@ -298,13 +298,13 @@ static int ft_parse_internal(MYSQL_FTPAR DBUG_ENTER("ft_parse_internal"); while (ft_simple_get_word(wtree->custom_arg, &doc, end, &w, TRUE)) - if (param->mysql_add_word(param, (char*) w.pos, w.len, 0)) + if (param->mysql_add_word(param, w.pos, w.len, 0)) DBUG_RETURN(1); DBUG_RETURN(0); } -int ft_parse(TREE *wtree, const uchar *doc, int doclen, +int ft_parse(TREE *wtree, const uchar *doc, mysql_ft_size_t doclen, struct st_mysql_ftparser *parser, MYSQL_FTPARSER_PARAM *param, MEM_ROOT *mem_root) { @@ -319,7 +319,7 @@ int ft_parse(TREE *wtree, const uchar *d param->mysql_add_word= ft_add_word; param->mysql_ftparam= &my_param; param->cs= wtree->custom_arg; - param->doc= (char*) doc; + param->doc= doc; param->length= doclen; param->mode= MYSQL_FTPARSER_SIMPLE_MODE; DBUG_RETURN(parser->parse(param)); === modified file 'storage/myisam/ft_stopwords.c' --- a/storage/myisam/ft_stopwords.c 2009-11-29 23:08:56 +0000 +++ b/storage/myisam/ft_stopwords.c 2010-01-28 14:49:14 +0000 @@ -20,8 +20,8 @@ typedef struct st_ft_stopwords { - const char * pos; - uint len; + const uchar* pos; + size_t len; } FT_STOPWORD; static TREE *stopwords3=NULL; @@ -30,15 +30,15 @@ static int FT_STOPWORD_cmp(void* cmp_arg FT_STOPWORD *w1, FT_STOPWORD *w2) { return ha_compare_text(default_charset_info, - (uchar *)w1->pos,w1->len, - (uchar *)w2->pos,w2->len,0,0); + w1->pos, w1->len, + w2->pos, w2->len, 0, 0); } static void FT_STOPWORD_free(FT_STOPWORD *w, TREE_FREE action, void *arg __attribute__((unused))) { if (action == free_free) - my_free((uchar*) w->pos, MYF(0)); + my_free((void*) w->pos, MYF(0)); } static int ft_add_stopword(const char *w) === modified file 'storage/myisam/ftdefs.h' --- a/storage/myisam/ftdefs.h 2009-11-29 23:08:56 +0000 +++ b/storage/myisam/ftdefs.h 2010-01-27 21:53:08 +0000 @@ -97,8 +97,8 @@ typedef struct st_ft_word { const uchar *pos; - uint len; double weight; + size_t len; } FT_WORD; int is_stopword(const uchar *word, size_t len); @@ -111,7 +111,8 @@ uchar ft_simple_get_word(CHARSET_INFO *, FT_WORD *, my_bool); typedef struct _st_ft_seg_iterator { - uint num, len; + uint num; + mysql_ft_size_t len; HA_KEYSEG *seg; const uchar *rec, *pos; } FT_SEG_ITERATOR; @@ -128,8 +129,9 @@ FT_WORD * _mi_ft_parserecord(MI_INFO *, uint _mi_ft_parse(TREE *, MI_INFO *, uint, const uchar *, MYSQL_FTPARSER_PARAM *, MEM_ROOT *); -FT_INFO *ft_init_nlq_search(MI_INFO *, uint, uchar *, uint, uint, uchar *); -FT_INFO *ft_init_boolean_search(MI_INFO *, uint, uchar *, uint, CHARSET_INFO *); +FT_INFO *ft_init_nlq_search(MI_INFO *, uint, uchar *, mysql_ft_size_t, uint, + uchar *); +FT_INFO *ft_init_boolean_search(MI_INFO *, uint, uchar *, mysql_ft_size_t, CHARSET_INFO *); extern const struct _ft_vft _ft_vft_nlq; int ft_nlq_read_next(FT_INFO *, char *); === modified file 'storage/myisam/mi_check.c' --- a/storage/myisam/mi_check.c 2009-12-03 11:34:11 +0000 +++ b/storage/myisam/mi_check.c 2010-01-14 16:51:00 +0000 @@ -4447,7 +4447,7 @@ int update_state_info(HA_CHECK *param, M { if (update & UPDATE_TIME) { - share->state.check_time= (long) time((time_t*) 0); + share->state.check_time= time((time_t*) 0); if (!share->state.create_time) share->state.create_time=share->state.check_time; } === modified file 'storage/myisam/mi_create.c' --- a/storage/myisam/mi_create.c 2009-10-15 21:38:29 +0000 +++ b/storage/myisam/mi_create.c 2010-01-14 16:51:00 +0000 @@ -575,7 +575,7 @@ int mi_create(const char *name,uint keys max(share.base.pack_reclength,MI_MIN_BLOCK_LENGTH) : MI_EXTEND_BLOCK_LENGTH; if (! (flags & HA_DONT_TOUCH_DATA)) - share.state.create_time= (long) time((time_t*) 0); + share.state.create_time= time((time_t*) 0); pthread_mutex_lock(&THR_LOCK_myisam); === modified file 'storage/xtradb/CMakeLists.txt' --- a/storage/xtradb/CMakeLists.txt 2009-11-14 09:53:18 +0000 +++ b/storage/xtradb/CMakeLists.txt 2010-01-15 15:58:25 +0000 @@ -13,36 +13,41 @@ # along with this program; if not, write to the Free Software # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA -IF (CMAKE_SIZEOF_VOID_P MATCHES 8) - SET(WIN64 TRUE) -ENDIF (CMAKE_SIZEOF_VOID_P MATCHES 8) +# This is the CMakeLists for InnoDB Plugin -# Check type sizes -include(CheckTypeSize) - -# Currently, the checked results are not used. -CHECK_TYPE_SIZE(int SIZEOF_INT) -CHECK_TYPE_SIZE(long SIZEOF_LONG) -CHECK_TYPE_SIZE(void* SIZEOF_VOID_P) SET(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} -DSAFEMALLOC -DSAFE_MUTEX") SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} -DSAFEMALLOC -DSAFE_MUTEX") -INCLUDE("${PROJECT_SOURCE_DIR}/storage/mysql_storage_engine.cmake") + +# Starting at 5.1.38, MySQL CMake files are simplified. But the plugin +# CMakeLists.txt still needs to work with previous versions of MySQL. +IF (MYSQL_VERSION_ID GREATER "50137") + INCLUDE("${PROJECT_SOURCE_DIR}/storage/mysql_storage_engine.cmake") +ENDIF (MYSQL_VERSION_ID GREATER "50137") + +IF (CMAKE_SIZEOF_VOID_P MATCHES 8) + SET(WIN64 TRUE) +ENDIF (CMAKE_SIZEOF_VOID_P MATCHES 8) + ADD_DEFINITIONS(-D_WIN32 -D_LIB -DMYSQL_SERVER) -INCLUDE_DIRECTORIES(${CMAKE_SOURCE_DIR}/include ${CMAKE_SOURCE_DIR}/zlib - ${CMAKE_SOURCE_DIR}/storage/xtradb/include - ${CMAKE_SOURCE_DIR}/storage/xtradb/handler - ${CMAKE_SOURCE_DIR}/sql - ${CMAKE_SOURCE_DIR}/regex - ${CMAKE_SOURCE_DIR}/extra/yassl/include) +# Include directories under innobase +INCLUDE_DIRECTORIES(${CMAKE_SOURCE_DIR}/storage/xtradb/include + ${CMAKE_SOURCE_DIR}/storage/xtradb/handler) + +# Include directories under mysql +INCLUDE_DIRECTORIES(${CMAKE_SOURCE_DIR}/include + ${CMAKE_SOURCE_DIR}/sql + ${CMAKE_SOURCE_DIR}/regex + ${CMAKE_SOURCE_DIR}/zlib + ${CMAKE_SOURCE_DIR}/extra/yassl/include) # Removing compiler optimizations for innodb/mem/* files on 64-bit Windows # due to 64-bit compiler error, See MySQL Bug #19424, #36366, #34297 -IF(MSVC AND $(WIN64)) +IF (MSVC AND $(WIN64)) SET_SOURCE_FILES_PROPERTIES(mem/mem0mem.c mem/mem0pool.c PROPERTIES COMPILE_FLAGS -Od) -ENDIF(MSVC AND $(WIN64)) +ENDIF (MSVC AND $(WIN64)) SET(INNOBASE_SOURCES btr/btr0btr.c btr/btr0cur.c btr/btr0pcur.c btr/btr0sea.c buf/buf0buddy.c buf/buf0buf.c buf/buf0flu.c buf/buf0lru.c buf/buf0rea.c @@ -77,5 +82,20 @@ SET(INNOBASE_SOURCES btr/btr0btr.c btr/b usr/usr0sess.c ut/ut0byte.c ut/ut0dbg.c ut/ut0mem.c ut/ut0rnd.c ut/ut0ut.c ut/ut0vec.c ut/ut0list.c ut/ut0wqueue.c) +ADD_DEFINITIONS(-DHAVE_WINDOWS_ATOMICS -DIB_HAVE_PAUSE_INSTRUCTION) -MYSQL_STORAGE_ENGINE(INNOBASE) +IF (MYSQL_VERSION_ID GREATER "50137") + MYSQL_STORAGE_ENGINE(INNOBASE) + # Use ha_innodb for plugin name, if plugin is built + GET_TARGET_PROPERTY(LIB_LOCATION ha_innobase LOCATION) + IF(LIB_LOCATION) + SET_TARGET_PROPERTIES(ha_innobase PROPERTIES OUTPUT_NAME ha_innodb) + ENDIF(LIB_LOCATION) +ELSE (MYSQL_VERSION_ID GREATER "50137") + IF (NOT SOURCE_SUBLIBS) + ADD_DEFINITIONS(-D_WIN32 -DMYSQL_SERVER) + ADD_LIBRARY(innobase STATIC ${INNOBASE_SOURCES}) + # Require mysqld_error.h, which is built as part of the GenError + ADD_DEPENDENCIES(innobase GenError) + ENDIF (NOT SOURCE_SUBLIBS) +ENDIF (MYSQL_VERSION_ID GREATER "50137") === modified file 'storage/xtradb/ChangeLog' --- a/storage/xtradb/ChangeLog 2009-11-13 21:26:08 +0000 +++ b/storage/xtradb/ChangeLog 2010-01-15 15:58:25 +0000 @@ -1,3 +1,319 @@ +2009-11-20 The InnoDB Team + + * handler/ha_innodb.cc: + Add a workaround to prevent a crash due to Bug#45961 DDL on + partitioned innodb tables leaves data dictionary in an inconsistent + state + +2009-11-19 The InnoDB Team + + * btr/btr0btr.c: + Fix Bug#48469 when innodb tablespace is configured too small, crash + and corruption! + +2009-11-19 The InnoDB Team + + * data/data0type.c: + Fix Bug#48526 Data type for float and double is incorrectly reported + in InnoDB table monitor + +2009-11-19 The InnoDB Team + + * CMakeLists.txt: + Fix Bug#48317 cannot build innodb as static library + +2009-11-18 The InnoDB Team + + * handler/handler0alter.cc: + Fix Bug#48782 On lock wait timeout, CREATE INDEX (creating primary key) + attempts DROP TABLE + +2009-11-17 The InnoDB Team + + * handler/ha_innodb.cc, mysql-test/innodb.result, + mysql-test/innodb.test, mysql-test/innodb_bug44369.result, + mysql-test/innodb_bug44369.test, mysql-test/patches/innodb-index.diff, + row/row0mysql.c: + Report duplicate table names to the client connection, not to the + error log. + +2009-11-12 The InnoDB Team + + * handler/ha_innodb.cc, include/db0err.h, row/row0merge.c, + row/row0mysql.c: + Allow CREATE INDEX to be interrupted. + Also, when CHECK TABLE is interrupted, report ER_QUERY_INTERRUPTED. + +2009-11-11 The InnoDB Team + + * handler/ha_innodb.cc, mysql-test/innodb_bug47167.result, + mysql-test/innodb_bug47167.test, mysql-test/innodb_file_format.result: + Fix Bug#47167 "set global innodb_file_format_check" cannot set value + by User-Defined Variable + +2009-11-11 The InnoDB Team + + * include/os0file.h, os/os0file.c: + Fix Bug#3139 Mysql crashes: 'windows error 995' after several selects + on a large DB + +2009-11-04 The InnoDB Team + + * handler/ha_innodb.cc: + Fix Bug#32430 'show innodb status' causes errors + Invalid (old?) table or database name in logs + +2009-11-02 The InnoDB Team + + * btr/btr0sea.c, buf/buf0buf.c, dict/dict0dict.c, fil/fil0fil.c, + ibuf/ibuf0ibuf.c, include/btr0sea.h, include/dict0dict.h, + include/fil0fil.h, include/ibuf0ibuf.h, include/lock0lock.h, + include/log0log.h, include/log0recv.h, include/mem0mem.h, + include/mem0pool.h, include/os0file.h, include/pars0pars.h, + include/srv0srv.h, include/thr0loc.h, include/trx0i_s.h, + include/trx0purge.h, include/trx0rseg.h, include/trx0sys.h, + include/trx0undo.h, include/usr0sess.h, lock/lock0lock.c, + log/log0log.c, log/log0recv.c, mem/mem0dbg.c, mem/mem0pool.c, + os/os0file.c, os/os0sync.c, os/os0thread.c, pars/lexyy.c, + pars/pars0lex.l, que/que0que.c, srv/srv0srv.c, srv/srv0start.c, + sync/sync0arr.c, sync/sync0sync.c, thr/thr0loc.c, trx/trx0i_s.c, + trx/trx0purge.c, trx/trx0rseg.c, trx/trx0sys.c, trx/trx0undo.c, + usr/usr0sess.c, ut/ut0mem.c: + Fix Bug #45992 innodb memory not freed after shutdown + Fix Bug #46656 InnoDB plugin: memory leaks (Valgrind) + +2009-10-29 The InnoDB Team + + * handler/ha_innodb.cc, mysql-test/innodb-autoinc.result, + mysql-test/innodb-autoinc.test: + Fix Bug#47125 auto_increment start value is ignored if an index is + created and engine=innodb + +2009-10-29 The InnoDB Team + + * handler/ha_innodb.cc, mysql-test/innodb_bug47777.result, + mysql-test/innodb_bug47777.test: + Fix Bug#47777 innodb dies with spatial pk: Failing assertion: buf <= + original_buf + buf_len + +2009-10-29 The InnoDB Team + + * handler/ha_innodb.cc: + Fix Bug#38996 Race condition in ANALYZE TABLE + +2009-10-29 The InnoDB Team + + * handler/ha_innodb.cc: + Fix bug#42383: Can't create table 'test.bug39438' + +2009-10-29 The InnoDB Team + + * os/os0proc.c: + Fix Bug#48237 Error handling in os_mem_alloc_large appears to + be incorrect + +2009-10-29 The InnoDB Team + + * buf/buf0buf.c, buf/buf0lru.c, include/buf0buf.h, include/buf0buf.ic: + Fix corruption of the buf_pool->LRU_old list and improve debug + assertions. + +2009-10-28 The InnoDB Team + + * srv/srv0start.c: + Fix Bug#41490 After enlargement of InnoDB page size, the error message + become inaccurate + +2009-10-26 The InnoDB Team + + * row/row0ins.c: + When allocating a data tuple, zero out the system fields in order + to avoid Valgrind warnings about uninitialized fields in + dtuple_validate(). + +2009-10-22 The InnoDB Team + + * handler/ha_innodb.cc, mysql-test/innodb-zip.result, + mysql-test/innodb-zip.test, mysql-test/innodb_bug44369.result, + mysql-test/innodb_bug44369.test: + Fix Bug#47233 Innodb calls push_warning(MYSQL_ERROR::WARN_LEVEL_ERROR) + +2009-10-19 The InnoDB Team + + * mysql-test/innodb_information_schema.test: + Fix Bug#47808 innodb_information_schema.test fails when run under + valgrind + +2009-10-15 The InnoDB Team + + * include/page0page.ic: + Fix Bug#47058 Failure to compile innodb_plugin on solaris 10u7 + spro + cc/CC 5.10 + +2009-10-13 The InnoDB Team + + * buf/buf0flu.c: + Call fsync() on datafiles after a batch of pages is written to disk + even when skip_innodb_doublewrite is set. + +2009-10-05 The InnoDB Team + + * buf/buf0buf.c: + Do not invalidate buffer pool while an LRU batch is active. Added code + to buf_pool_invalidate() to wait for the running batches to finish. + +2009-10-01 The InnoDB Team + + * handler/ha_innodb.cc: + Fix Bug#47763 typo in error message: Failed to open table %s after %lu + attemtps. + +2009-10-01 The InnoDB Team + + * fsp/fsp0fsp.c, row/row0merge.c: + Clean up after a crash during DROP INDEX. When InnoDB crashes + while dropping an index, ensure that the index will be completely + dropped during crash recovery. The MySQL .frm file may still + contain the dropped index, but there is little that we can do + about it. + +2009-09-28 The InnoDB Team + + * handler/ha_innodb.cc: + When a secondary index exists in the MySQL .frm file but not in + the InnoDB data dictionary, return an error instead of letting an + assertion fail in index_read. + +2009-09-28 The InnoDB Team + + * btr/btr0btr.c, buf/buf0buf.c, include/page0page.h, + include/page0zip.h, page/page0cur.c, page/page0page.c, + page/page0zip.c: + Do not write to PAGE_INDEX_ID when restoring an uncompressed page + after a compression failure. The field should only be written + when creating a B-tree page. This fix addresses a race condition + in a debug assertion. + +2009-09-28 The InnoDB Team + + * fil/fil0fil.c: + Try to prevent the reuse of tablespace identifiers after InnoDB + has crashed during table creation. Also, refuse to start if files + with duplicate tablespace identifiers are encountered. + +2009-09-25 The InnoDB Team + + * include/os0file.h, os/os0file.c: + Fix Bug#47055 unconditional exit(1) on ERROR_WORKING_SET_QUOTA + 1453 (0x5AD) for InnoDB backend + +2009-09-19 The InnoDB Team + + * handler/ha_innodb.cc, mysql-test/innodb-consistent-master.opt, + mysql-test/innodb-consistent.result, + mysql-test/innodb-consistent.test: + Fix Bug#37232 Innodb might get too many read locks for DML with + repeatable-read + +2009-09-19 The InnoDB Team + + * fsp/fsp0fsp.c: + Fix Bug#31183 Tablespace full problems not reported in error log, + error message unclear + +2009-09-17 The InnoDB Team + + * mysql-test/innodb-zip.result, mysql-test/innodb-zip.test: + Make the test pass with zlib 1.2.3.3. Apparently, the definition + of compressBound() has changed between zlib versions, and the + maximum record size of a table with 1K compressed page size has + been reduced by one byte. This is an arbitrary test. In practical + applications, for good write performance, the compressed page size + should be chosen to be bigger than the absolute minimum. + +2009-09-16 The InnoDB Team + + * handler/ha_innodb.cc: + Fix Bug#46256 drop table with unknown collation crashes innodb + +2009-09-16 The InnoDB Team + + * dict/dict0dict.c, handler/ha_innodb.cc, + mysql-test/innodb_bug44369.result, mysql-test/innodb_bug44369.test, + row/row0mysql.c: + Fix Bug#44369 InnoDB: Does not uniformly disallow disallowed column + names + +2009-09-16 The InnoDB Team + + * handler/ha_innodb.cc, include/db0err.h, + mysql-test/innodb_bug46000.result, mysql-test/innodb_bug46000.test: + Fix Bug#46000 using index called GEN_CLUST_INDEX crashes server + +2009-09-02 The InnoDB Team + + * include/lock0lock.h, include/row0mysql.h, lock/lock0lock.c, + row/row0mysql.c: + Fix a regression introduced by the fix for MySQL bug#26316. We check + whether a transaction holds any AUTOINC locks before we acquire + the kernel mutex and release those locks. + +2009-08-27 The InnoDB Team + + * dict/dict0dict.c, include/dict0dict.h, + mysql-test/innodb_bug44571.result, mysql-test/innodb_bug44571.test: + Fix Bug#44571 InnoDB Plugin crashes on ADD INDEX + +2009-08-27 The InnoDB Team + + * row/row0merge.c: + Fix a bug in the merge sort that can corrupt indexes in fast index + creation. Add some consistency checks. Check that the number of + records remains constant in every merge sort pass. + +2009-08-27 The InnoDB Team + + * buf/buf0buf.c, buf/buf0lru.c, buf/buf0rea.c, handler/ha_innodb.cc, + include/buf0buf.h, include/buf0buf.ic, include/buf0lru.h, + include/ut0ut.h, ut/ut0ut.c: + Make it possible to tune the buffer pool LRU eviction policy to be + more resistant against index scans. Introduce the settable global + variables innodb_old_blocks_pct and innodb_old_blocks_time for + controlling the buffer pool eviction policy. The parameter + innodb_old_blocks_pct (5..95) controls the desired amount of "old" + blocks in the LRU list. The default is 37, corresponding to the + old fixed ratio of 3/8. Each time a block is accessed, it will be + moved to the "new" blocks if its first access was at least + innodb_old_blocks_time milliseconds ago (default 0, meaning every + block). The idea is that in index scans, blocks will be accessed + a few times within innodb_old_blocks_time, and they will remain in + the "old" section of the LRU list. Thus, when innodb_old_blocks_time + is nonzero, blocks retrieved for one-time index scans will be more + likely candidates for eviction than blocks that are accessed in + random patterns. + +2009-08-26 The InnoDB Team + + * handler/ha_innodb.cc, os/os0file.c: + Fix Bug#42885 buf_read_ahead_random, buf_read_ahead_linear counters, + thread wakeups + +2009-08-20 The InnoDB Team + + * lock/lock0lock.c: + Fix Bug#46650 Innodb assertion autoinc_lock == lock in + lock_table_remove_low on INSERT SELECT + +2009-08-13 The InnoDB Team + + * handler/handler0alter.cc: + Fix Bug#46657 InnoDB plugin: invalid read in index_merge_innodb test + (Valgrind) + +2009-08-11 The InnoDB Team + + InnoDB Plugin 1.0.4 released + 2009-07-20 The InnoDB Team * buf/buf0rea.c, handler/ha_innodb.cc, include/srv0srv.h, === modified file 'storage/xtradb/Makefile.am' --- a/storage/xtradb/Makefile.am 2009-11-13 22:53:04 +0000 +++ b/storage/xtradb/Makefile.am 2010-01-15 15:58:25 +0000 @@ -22,7 +22,7 @@ MYSQLLIBdir= $(pkglibdir) pkgplugindir= $(pkglibdir)/plugin INCLUDES= -I$(top_srcdir)/include -I$(top_builddir)/include \ -I$(top_srcdir)/regex \ - -I$(top_srcdir)/storage/xtradb/include \ + -I$(srcdir)/include \ -I$(top_srcdir)/sql \ -I$(srcdir) @ZLIB_INCLUDES@ @@ -31,7 +31,6 @@ DEFS= @DEFS@ noinst_HEADERS= \ handler/ha_innodb.h \ - handler/handler0vars.h \ handler/i_s.h \ include/btr0btr.h \ include/btr0btr.ic \ === modified file 'storage/xtradb/btr/btr0btr.c' --- a/storage/xtradb/btr/btr0btr.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/btr/btr0btr.c 2010-01-06 12:00:14 +0000 @@ -790,14 +790,21 @@ btr_create( } else { /* It is a non-ibuf tree: create a file segment for leaf pages */ - fseg_create(space, page_no, - PAGE_HEADER + PAGE_BTR_SEG_LEAF, mtr); + if (!fseg_create(space, page_no, + PAGE_HEADER + PAGE_BTR_SEG_LEAF, mtr)) { + /* Not enough space for new segment, free root + segment before return. */ + btr_free_root(space, zip_size, page_no, mtr); + + return(FIL_NULL); + } + /* The fseg create acquires a second latch on the page, therefore we must declare it: */ buf_block_dbg_add_level(block, SYNC_TREE_NODE_NEW); } - /* Create a new index page on the the allocated segment page */ + /* Create a new index page on the allocated segment page */ page_zip = buf_block_get_page_zip(block); if (UNIV_LIKELY_NULL(page_zip)) { @@ -1011,7 +1018,26 @@ btr_page_reorganize_low( (!page_zip_compress(page_zip, page, index, NULL))) { /* Restore the old page and exit. */ - buf_frame_copy(page, temp_page); + +#if defined UNIV_DEBUG || defined UNIV_ZIP_DEBUG + /* Check that the bytes that we skip are identical. */ + ut_a(!memcmp(page, temp_page, PAGE_HEADER)); + ut_a(!memcmp(PAGE_HEADER + PAGE_N_RECS + page, + PAGE_HEADER + PAGE_N_RECS + temp_page, + PAGE_DATA - (PAGE_HEADER + PAGE_N_RECS))); + ut_a(!memcmp(UNIV_PAGE_SIZE - FIL_PAGE_DATA_END + page, + UNIV_PAGE_SIZE - FIL_PAGE_DATA_END + temp_page, + FIL_PAGE_DATA_END)); +#endif /* UNIV_DEBUG || UNIV_ZIP_DEBUG */ + + memcpy(PAGE_HEADER + page, PAGE_HEADER + temp_page, + PAGE_N_RECS - PAGE_N_DIR_SLOTS); + memcpy(PAGE_DATA + page, PAGE_DATA + temp_page, + UNIV_PAGE_SIZE - PAGE_DATA - FIL_PAGE_DATA_END); + +#if defined UNIV_DEBUG || defined UNIV_ZIP_DEBUG + ut_a(!memcmp(page, temp_page, UNIV_PAGE_SIZE)); +#endif /* UNIV_DEBUG || UNIV_ZIP_DEBUG */ goto func_exit; } @@ -1902,7 +1928,7 @@ func_start: n_uniq, &heap); /* If the new record is less than the existing record - the the split in the middle will copy the existing + the split in the middle will copy the existing record to the new node. */ if (cmp_dtuple_rec(tuple, first_rec, offsets) < 0) { split_rec = page_get_middle_rec(page); === modified file 'storage/xtradb/btr/btr0sea.c' --- a/storage/xtradb/btr/btr0sea.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/btr/btr0sea.c 2010-01-06 12:00:14 +0000 @@ -175,6 +175,21 @@ btr_search_sys_create( btr_search_sys->hash_index = ha_create(hash_size, 0, 0); } +/*****************************************************************//** +Frees the adaptive search system at a database shutdown. */ +UNIV_INTERN +void +btr_search_sys_free(void) +/*=====================*/ +{ + mem_free(btr_search_latch_temp); + btr_search_latch_temp = NULL; + mem_heap_free(btr_search_sys->hash_index->heap); + hash_table_free(btr_search_sys->hash_index); + mem_free(btr_search_sys); + btr_search_sys = NULL; +} + /********************************************************************//** Disable the adaptive hash search system and empty the index. */ UNIV_INTERN @@ -957,7 +972,7 @@ btr_search_guess_on_hash( /* Increment the page get statistics though we did not really fix the page: for user info only */ - buf_pool->n_page_gets++; + buf_pool->stat.n_page_gets++; return(TRUE); === modified file 'storage/xtradb/buf/buf0buddy.c' --- a/storage/xtradb/buf/buf0buddy.c 2009-11-13 21:26:08 +0000 +++ b/storage/xtradb/buf/buf0buddy.c 2010-01-15 15:58:25 +0000 @@ -531,11 +531,10 @@ buf_buddy_relocate( UNIV_MEM_ASSERT_W(src, size); mutex = buf_page_get_mutex_enter(bpage); - ut_a(mutex); mutex_enter(&zip_free_mutex); - if (buf_page_can_relocate(bpage)) { + if (mutex && buf_page_can_relocate(bpage)) { /* Relocate the compressed page. */ ut_a(bpage->zip.data == src); memcpy(dst, src, size); @@ -563,7 +562,9 @@ success: rw_lock_x_unlock(&page_hash_latch); } - mutex_exit(mutex); + if (mutex) { + mutex_exit(mutex); + } } else if (i == buf_buddy_get_slot(sizeof(buf_page_t))) { /* This must be a buf_page_t object. */ UNIV_MEM_ASSERT_RW(src, size); === modified file 'storage/xtradb/buf/buf0buf.c' --- a/storage/xtradb/buf/buf0buf.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/buf/buf0buf.c 2010-01-28 11:35:10 +0000 @@ -51,6 +51,40 @@ Created 11/5/1995 Heikki Tuuri #include "dict0dict.h" #include "log0recv.h" #include "page0zip.h" +#include "trx0trx.h" + +/* prototypes for new functions added to ha_innodb.cc */ +trx_t* innobase_get_trx(); + +inline void _increment_page_get_statistics(buf_block_t* block, trx_t* trx) +{ + ulint block_hash; + ulint block_hash_byte; + byte block_hash_offset; + + ut_ad(block); + + if (!innobase_get_slow_log() || !trx || !trx->take_stats) + return; + + if (!trx->distinct_page_access_hash) { + trx->distinct_page_access_hash = mem_alloc(DPAH_SIZE); + memset(trx->distinct_page_access_hash, 0, DPAH_SIZE); + } + + block_hash = ut_hash_ulint((block->page.space << 20) + block->page.space + + block->page.offset, DPAH_SIZE << 3); + block_hash_byte = block_hash >> 3; + block_hash_offset = (byte) block_hash & 0x07; + if (block_hash_byte >= DPAH_SIZE) + fprintf(stderr, "!!! block_hash_byte = %lu block_hash_offset = %lu !!!\n", (unsigned long) block_hash_byte, (unsigned long) block_hash_offset); + if (block_hash_offset > 7) + fprintf(stderr, "!!! block_hash_byte = %lu block_hash_offset = %lu !!!\n", (unsigned long) block_hash_byte, (unsigned long) block_hash_offset); + if ((trx->distinct_page_access_hash[block_hash_byte] & ((byte) 0x01 << block_hash_offset)) == 0) + trx->distinct_page_access++; + trx->distinct_page_access_hash[block_hash_byte] |= (byte) 0x01 << block_hash_offset; + return; +} /* IMPLEMENTATION OF THE BUFFER POOL @@ -845,16 +879,35 @@ buf_chunk_not_freed( block = chunk->blocks; for (i = chunk->size; i--; block++) { - mutex_enter(&block->mutex); - - if (buf_block_get_state(block) == BUF_BLOCK_FILE_PAGE - && !buf_flush_ready_for_replace(&block->page)) { + ibool ready; + switch (buf_block_get_state(block)) { + case BUF_BLOCK_ZIP_FREE: + case BUF_BLOCK_ZIP_PAGE: + case BUF_BLOCK_ZIP_DIRTY: + /* The uncompressed buffer pool should never + contain compressed block descriptors. */ + ut_error; + break; + case BUF_BLOCK_NOT_USED: + case BUF_BLOCK_READY_FOR_USE: + case BUF_BLOCK_MEMORY: + case BUF_BLOCK_REMOVE_HASH: + /* Skip blocks that are not being used for + file pages. */ + break; + case BUF_BLOCK_FILE_PAGE: + mutex_enter(&block->mutex); + ready = buf_flush_ready_for_replace(&block->page); mutex_exit(&block->mutex); - return(block); - } - mutex_exit(&block->mutex); + if (!ready) { + + return(block); + } + + break; + } } return(NULL); @@ -985,8 +1038,6 @@ buf_pool_init(void) buf_pool->no_flush[i] = os_event_create(NULL); } - buf_pool->ulint_clock = 1; - /* 3. Initialize LRU fields --------------------------- */ /* All fields are initialized by mem_zalloc(). */ @@ -1024,7 +1075,11 @@ buf_pool_free(void) os_mem_free_large(chunk->mem, chunk->mem_size); } - buf_pool->n_chunks = 0; + mem_free(buf_pool->chunks); + hash_table_free(buf_pool->page_hash); + hash_table_free(buf_pool->zip_hash); + mem_free(buf_pool); + buf_pool = NULL; } /********************************************************************//** @@ -1171,10 +1226,15 @@ buf_relocate( #ifdef UNIV_LRU_DEBUG /* buf_pool->LRU_old must be the first item in the LRU list whose "old" flag is set. */ + ut_a(buf_pool->LRU_old->old); ut_a(!UT_LIST_GET_PREV(LRU, buf_pool->LRU_old) || !UT_LIST_GET_PREV(LRU, buf_pool->LRU_old)->old); ut_a(!UT_LIST_GET_NEXT(LRU, buf_pool->LRU_old) || UT_LIST_GET_NEXT(LRU, buf_pool->LRU_old)->old); + } else { + /* Check that the "old" flag is consistent in + the block and its neighbours. */ + buf_page_set_old(dpage, buf_page_is_old(dpage)); #endif /* UNIV_LRU_DEBUG */ } @@ -1510,35 +1570,8 @@ buf_pool_resize(void) } /********************************************************************//** -Moves the block to the start of the LRU list if there is a danger -that the block would drift out of the buffer pool. */ -UNIV_INLINE -void -buf_block_make_young( -/*=================*/ - buf_page_t* bpage) /*!< in: block to make younger */ -{ - ut_ad(!buf_pool_mutex_own()); - - /* Note that we read freed_page_clock's without holding any mutex: - this is allowed since the result is used only in heuristics */ - - if (buf_page_peek_if_too_old(bpage)) { - - //buf_pool_mutex_enter(); - mutex_enter(&LRU_list_mutex); - /* There has been freeing activity in the LRU list: - best to move to the head of the LRU list */ - - buf_LRU_make_block_young(bpage); - //buf_pool_mutex_exit(); - mutex_exit(&LRU_list_mutex); - } -} - -/********************************************************************//** Moves a page to the start of the buffer pool LRU list. This high-level -function can be used to prevent an important page from from slipping out of +function can be used to prevent an important page from slipping out of the buffer pool. */ UNIV_INTERN void @@ -1558,6 +1591,42 @@ buf_page_make_young( } /********************************************************************//** +Sets the time of the first access of a page and moves a page to the +start of the buffer pool LRU list if it is too old. This high-level +function can be used to prevent an important page from slipping +out of the buffer pool. */ +static +void +buf_page_set_accessed_make_young( +/*=============================*/ + buf_page_t* bpage, /*!< in/out: buffer block of a + file page */ + unsigned access_time) /*!< in: bpage->access_time + read under mutex protection, + or 0 if unknown */ +{ + ut_ad(!buf_pool_mutex_own()); + ut_a(buf_page_in_file(bpage)); + + if (buf_page_peek_if_too_old(bpage)) { + //buf_pool_mutex_enter(); + mutex_enter(&LRU_list_mutex); + buf_LRU_make_block_young(bpage); + //buf_pool_mutex_exit(); + mutex_exit(&LRU_list_mutex); + } else if (!access_time) { + ulint time_ms = ut_time_ms(); + mutex_t* block_mutex = buf_page_get_mutex_enter(bpage); + //buf_pool_mutex_enter(); + if (block_mutex) { + buf_page_set_accessed(bpage, time_ms); + mutex_exit(block_mutex); + } + //buf_pool_mutex_exit(); + } +} + +/********************************************************************//** Resets the check_index_page_at_flush field of a page if found in the buffer pool. */ UNIV_INTERN @@ -1696,11 +1765,20 @@ buf_page_get_zip( buf_page_t* bpage; mutex_t* block_mutex; ibool must_read; + unsigned access_time; + trx_t* trx = NULL; + ulint sec; + ulint ms; + ib_uint64_t start_time; + ib_uint64_t finish_time; #ifndef UNIV_LOG_DEBUG ut_ad(!ibuf_inside()); #endif - buf_pool->n_page_gets++; + if (innobase_get_slow_log()) { + trx = innobase_get_trx(); + } + buf_pool->stat.n_page_gets++; for (;;) { //buf_pool_mutex_enter(); @@ -1716,7 +1794,7 @@ lookup: //buf_pool_mutex_exit(); rw_lock_s_unlock(&page_hash_latch); - buf_read_page(space, zip_size, offset); + buf_read_page(space, zip_size, offset, trx); #if defined UNIV_DEBUG || defined UNIV_BUF_DEBUG ut_a(++buf_dbg_counter % 37 || buf_validate()); @@ -1770,14 +1848,13 @@ err_exit: got_block: must_read = buf_page_get_io_fix(bpage) == BUF_IO_READ; + access_time = buf_page_is_accessed(bpage); //buf_pool_mutex_exit(); - buf_page_set_accessed(bpage, TRUE); - mutex_exit(block_mutex); - buf_block_make_young(bpage); + buf_page_set_accessed_make_young(bpage, access_time); #ifdef UNIV_DEBUG_FILE_ACCESSES ut_a(!bpage->file_page_was_freed); @@ -1793,6 +1870,13 @@ got_block: /* Let us wait until the read operation completes */ + if (innobase_get_slow_log() && trx && trx->take_stats) + { + ut_usectime(&sec, &ms); + start_time = (ib_uint64_t)sec * 1000000 + ms; + } else { + start_time = 0; + } for (;;) { enum buf_io_fix io_fix; @@ -1807,6 +1891,12 @@ got_block: break; } } + if (innobase_get_slow_log() && trx && trx->take_stats && start_time) + { + ut_usectime(&sec, &ms); + finish_time = (ib_uint64_t)sec * 1000000 + ms; + trx->io_reads_wait_timer += (ulint)(finish_time - start_time); + } } #ifdef UNIV_IBUF_COUNT_DEBUG @@ -1870,7 +1960,7 @@ buf_zip_decompress( switch (fil_page_get_type(frame)) { case FIL_PAGE_INDEX: if (page_zip_decompress(&block->page.zip, - block->frame)) { + block->frame, TRUE)) { return(TRUE); } @@ -2058,10 +2148,15 @@ buf_page_get_gen( mtr_t* mtr) /*!< in: mini-transaction */ { buf_block_t* block; - ibool accessed; + unsigned access_time; ulint fix_type; ibool must_read; mutex_t* block_mutex; + trx_t* trx = NULL; + ulint sec; + ulint ms; + ib_uint64_t start_time; + ib_uint64_t finish_time; ut_ad(mtr); ut_ad((rw_latch == RW_S_LATCH) @@ -2075,14 +2170,16 @@ buf_page_get_gen( #ifndef UNIV_LOG_DEBUG ut_ad(!ibuf_inside() || ibuf_page(space, zip_size, offset, NULL)); #endif - buf_pool->n_page_gets++; + if (innobase_get_slow_log()) { + trx = innobase_get_trx(); + } + buf_pool->stat.n_page_gets++; loop: block = guess; //buf_pool_mutex_enter(); if (block) { block_mutex = buf_page_get_mutex_enter((buf_page_t*)block); - ut_a(block_mutex); /* If the guess is a compressed page descriptor that has been allocated by buf_buddy_alloc(), it may have @@ -2092,7 +2189,9 @@ loop: the guess may be pointing to a buffer pool chunk that has been released when resizing the buffer pool. */ - if (!buf_block_is_uncompressed(block) + if (!block_mutex) { + block = guess = NULL; + } else if (!buf_block_is_uncompressed(block) || offset != block->page.offset || space != block->page.space || buf_block_get_state(block) != BUF_BLOCK_FILE_PAGE) { @@ -2127,7 +2226,7 @@ loop2: return(NULL); } - buf_read_page(space, zip_size, offset); + buf_read_page(space, zip_size, offset, trx); #if defined UNIV_DEBUG || defined UNIV_BUF_DEBUG ut_a(++buf_dbg_counter % 37 || buf_validate()); @@ -2351,17 +2450,17 @@ wait_until_unfixed: UNIV_MEM_ASSERT_RW(&block->page, sizeof block->page); buf_block_buf_fix_inc(block, file, line); - //buf_pool_mutex_exit(); - /* Check if this is the first access to the page */ + //mutex_exit(&block->mutex); - accessed = buf_page_is_accessed(&block->page); + /* Check if this is the first access to the page */ - buf_page_set_accessed(&block->page, TRUE); + access_time = buf_page_is_accessed(&block->page); + //buf_pool_mutex_exit(); mutex_exit(block_mutex); - buf_block_make_young(&block->page); + buf_page_set_accessed_make_young(&block->page, access_time); #ifdef UNIV_DEBUG_FILE_ACCESSES ut_a(!block->page.file_page_was_freed); @@ -2379,6 +2478,13 @@ wait_until_unfixed: /* Let us wait until the read operation completes */ + if (innobase_get_slow_log() && trx && trx->take_stats) + { + ut_usectime(&sec, &ms); + start_time = (ib_uint64_t)sec * 1000000 + ms; + } else { + start_time = 0; + } for (;;) { enum buf_io_fix io_fix; @@ -2393,6 +2499,12 @@ wait_until_unfixed: break; } } + if (innobase_get_slow_log() && trx && trx->take_stats && start_time) + { + ut_usectime(&sec, &ms); + finish_time = (ib_uint64_t)sec * 1000000 + ms; + trx->io_reads_wait_timer += (ulint)(finish_time - start_time); + } } fix_type = MTR_MEMO_BUF_FIX; @@ -2414,17 +2526,21 @@ wait_until_unfixed: mtr_memo_push(mtr, block, fix_type); - if (!accessed) { + if (!access_time) { /* In the case of a first access, try to apply linear read-ahead */ - buf_read_ahead_linear(space, zip_size, offset); + buf_read_ahead_linear(space, zip_size, offset, trx); } #ifdef UNIV_IBUF_COUNT_DEBUG ut_a(ibuf_count_get(buf_block_get_space(block), buf_block_get_page_no(block)) == 0); #endif + if (innobase_get_slow_log()) { + _increment_page_get_statistics(block, trx); + } + return(block); } @@ -2444,9 +2560,10 @@ buf_page_optimistic_get_func( ulint line, /*!< in: line where called */ mtr_t* mtr) /*!< in: mini-transaction */ { - ibool accessed; + unsigned access_time; ibool success; ulint fix_type; + trx_t* trx = NULL; ut_ad(mtr && block); ut_ad((rw_latch == RW_S_LATCH) || (rw_latch == RW_X_LATCH)); @@ -2461,14 +2578,16 @@ buf_page_optimistic_get_func( } buf_block_buf_fix_inc(block, file, line); - accessed = buf_page_is_accessed(&block->page); - buf_page_set_accessed(&block->page, TRUE); mutex_exit(&block->mutex); - buf_block_make_young(&block->page); + /* Check if this is the first access to the page. + We do a dirty read on purpose, to avoid mutex contention. + This field is only used for heuristic purposes; it does not + affect correctness. */ - /* Check if this is the first access to the page */ + access_time = buf_page_is_accessed(&block->page); + buf_page_set_accessed_make_young(&block->page, access_time); ut_ad(!ibuf_inside() || ibuf_page(buf_block_get_space(block), @@ -2520,21 +2639,28 @@ buf_page_optimistic_get_func( #ifdef UNIV_DEBUG_FILE_ACCESSES ut_a(block->page.file_page_was_freed == FALSE); #endif - if (UNIV_UNLIKELY(!accessed)) { + if (innobase_get_slow_log()) { + trx = innobase_get_trx(); + } + + if (UNIV_UNLIKELY(!access_time)) { /* In the case of a first access, try to apply linear read-ahead */ buf_read_ahead_linear(buf_block_get_space(block), buf_block_get_zip_size(block), - buf_block_get_page_no(block)); + buf_block_get_page_no(block), trx); } #ifdef UNIV_IBUF_COUNT_DEBUG ut_a(ibuf_count_get(buf_block_get_space(block), buf_block_get_page_no(block)) == 0); #endif - buf_pool->n_page_gets++; + buf_pool->stat.n_page_gets++; + if (innobase_get_slow_log()) { + _increment_page_get_statistics(block, trx); + } return(TRUE); } @@ -2556,6 +2682,7 @@ buf_page_get_known_nowait( { ibool success; ulint fix_type; + trx_t* trx = NULL; ut_ad(mtr); ut_ad((rw_latch == RW_S_LATCH) || (rw_latch == RW_X_LATCH)); @@ -2581,8 +2708,24 @@ buf_page_get_known_nowait( mutex_exit(&block->mutex); - if (mode == BUF_MAKE_YOUNG) { - buf_block_make_young(&block->page); + if (mode == BUF_MAKE_YOUNG && buf_page_peek_if_too_old(&block->page)) { + //buf_pool_mutex_enter(); + mutex_enter(&LRU_list_mutex); + buf_LRU_make_block_young(&block->page); + //buf_pool_mutex_exit(); + mutex_exit(&LRU_list_mutex); + } else if (!buf_page_is_accessed(&block->page)) { + /* Above, we do a dirty read on purpose, to avoid + mutex contention. The field buf_page_t::access_time + is only used for heuristic purposes. Writes to the + field must be protected by mutex, however. */ + ulint time_ms = ut_time_ms(); + + //buf_pool_mutex_enter(); + mutex_enter(&block->mutex); + buf_page_set_accessed(&block->page, time_ms); + //buf_pool_mutex_exit(); + mutex_exit(&block->mutex); } ut_ad(!ibuf_inside() || (mode == BUF_KEEP_OLD)); @@ -2621,7 +2764,12 @@ buf_page_get_known_nowait( || (ibuf_count_get(buf_block_get_space(block), buf_block_get_page_no(block)) == 0)); #endif - buf_pool->n_page_gets++; + buf_pool->stat.n_page_gets++; + + if (innobase_get_slow_log()) { + trx = innobase_get_trx(); + _increment_page_get_statistics(block, trx); + } return(TRUE); } @@ -2700,7 +2848,7 @@ buf_page_try_get_func( #endif /* UNIV_DEBUG_FILE_ACCESSES */ buf_block_dbg_add_level(block, SYNC_NO_ORDER_CHECK); - buf_pool->n_page_gets++; + buf_pool->stat.n_page_gets++; #ifdef UNIV_IBUF_COUNT_DEBUG ut_a(ibuf_count_get(buf_block_get_space(block), @@ -2719,10 +2867,10 @@ buf_page_init_low( buf_page_t* bpage) /*!< in: block to init */ { bpage->flush_type = BUF_FLUSH_LRU; - bpage->accessed = FALSE; bpage->io_fix = BUF_IO_NONE; bpage->buf_fix_count = 0; bpage->freed_page_clock = 0; + bpage->access_time = 0; bpage->newest_modification = 0; bpage->oldest_modification = 0; HASH_INVALIDATE(bpage, hash); @@ -3044,6 +3192,7 @@ buf_page_create( buf_frame_t* frame; buf_block_t* block; buf_block_t* free_block = NULL; + ulint time_ms = ut_time_ms(); ut_ad(mtr); ut_ad(space || !zip_size); @@ -3095,7 +3244,7 @@ buf_page_create( buf_LRU_add_block(&block->page, FALSE); buf_block_buf_fix_inc(block, __FILE__, __LINE__); - buf_pool->n_pages_created++; + buf_pool->stat.n_pages_created++; if (zip_size) { void* data; @@ -3132,13 +3281,13 @@ buf_page_create( rw_lock_x_unlock(&block->lock); } + buf_page_set_accessed(&block->page, time_ms); + //buf_pool_mutex_exit(); mutex_exit(&LRU_list_mutex); mtr_memo_push(mtr, block, MTR_MEMO_BUF_FIX); - buf_page_set_accessed(&block->page, TRUE); - mutex_exit(&block->mutex); /* Delete possible entries for the page from the insert buffer: @@ -3355,7 +3504,7 @@ corrupt: ut_ad(buf_pool->n_pend_reads > 0); buf_pool->n_pend_reads--; - buf_pool->n_pages_read++; + buf_pool->stat.n_pages_read++; if (uncompressed) { rw_lock_x_unlock_gen(&((buf_block_t*) bpage)->lock, @@ -3380,7 +3529,7 @@ corrupt: BUF_IO_WRITE); } - buf_pool->n_pages_written++; + buf_pool->stat.n_pages_written++; break; @@ -3411,7 +3560,32 @@ void buf_pool_invalidate(void) /*=====================*/ { - ibool freed; + ibool freed; + enum buf_flush i; + + buf_pool_mutex_enter(); + + for (i = BUF_FLUSH_LRU; i < BUF_FLUSH_N_TYPES; i++) { + + /* As this function is called during startup and + during redo application phase during recovery, InnoDB + is single threaded (apart from IO helper threads) at + this stage. No new write batch can be in intialization + stage at this point. */ + ut_ad(buf_pool->init_flush[i] == FALSE); + + /* However, it is possible that a write batch that has + been posted earlier is still not complete. For buffer + pool invalidation to proceed we must ensure there is NO + write activity happening. */ + if (buf_pool->n_flush[i] > 0) { + buf_pool_mutex_exit(); + buf_flush_wait_batch_end(i); + buf_pool_mutex_enter(); + } + } + + buf_pool_mutex_exit(); ut_ad(buf_all_freed()); @@ -3427,6 +3601,14 @@ buf_pool_invalidate(void) ut_ad(UT_LIST_GET_LEN(buf_pool->LRU) == 0); ut_ad(UT_LIST_GET_LEN(buf_pool->unzip_LRU) == 0); + buf_pool->freed_page_clock = 0; + buf_pool->LRU_old = NULL; + buf_pool->LRU_old_len = 0; + buf_pool->LRU_flush_ended = 0; + + memset(&buf_pool->stat, 0x00, sizeof(buf_pool->stat)); + buf_refresh_io_stats(); + //buf_pool_mutex_exit(); mutex_exit(&LRU_list_mutex); } @@ -3706,6 +3888,7 @@ buf_print(void) "n pending decompressions %lu\n" "n pending reads %lu\n" "n pending flush LRU %lu list %lu single page %lu\n" + "pages made young %lu, not young %lu\n" "pages read %lu, created %lu, written %lu\n", (ulong) size, (ulong) UT_LIST_GET_LEN(buf_pool->LRU), @@ -3716,8 +3899,11 @@ buf_print(void) (ulong) buf_pool->n_flush[BUF_FLUSH_LRU], (ulong) buf_pool->n_flush[BUF_FLUSH_LIST], (ulong) buf_pool->n_flush[BUF_FLUSH_SINGLE_PAGE], - (ulong) buf_pool->n_pages_read, buf_pool->n_pages_created, - (ulong) buf_pool->n_pages_written); + (ulong) buf_pool->stat.n_pages_made_young, + (ulong) buf_pool->stat.n_pages_not_made_young, + (ulong) buf_pool->stat.n_pages_read, + (ulong) buf_pool->stat.n_pages_created, + (ulong) buf_pool->stat.n_pages_written); /* Count the number of blocks belonging to each index in the buffer */ @@ -3927,10 +4113,9 @@ buf_print_io( { time_t current_time; double time_elapsed; - ulint size; + ulint n_gets_diff; ut_ad(buf_pool); - size = buf_pool->curr_size; //buf_pool_mutex_enter(); mutex_enter(&LRU_list_mutex); @@ -3943,13 +4128,15 @@ buf_print_io( "Buffer pool size, bytes %lu\n" "Free buffers %lu\n" "Database pages %lu\n" + "Old database pages %lu\n" "Modified db pages %lu\n" "Pending reads %lu\n" "Pending writes: LRU %lu, flush list %lu, single page %lu\n", - (ulong) size, - (ulong) size * UNIV_PAGE_SIZE, + (ulong) buf_pool->curr_size, + (ulong) buf_pool->curr_size * UNIV_PAGE_SIZE, (ulong) UT_LIST_GET_LEN(buf_pool->free), (ulong) UT_LIST_GET_LEN(buf_pool->LRU), + (ulong) buf_pool->LRU_old_len, (ulong) UT_LIST_GET_LEN(buf_pool->flush_list), (ulong) buf_pool->n_pend_reads, (ulong) buf_pool->n_flush[BUF_FLUSH_LRU] @@ -3961,37 +4148,66 @@ buf_print_io( current_time = time(NULL); time_elapsed = 0.001 + difftime(current_time, buf_pool->last_printout_time); - buf_pool->last_printout_time = current_time; fprintf(file, + "Pages made young %lu, not young %lu\n" + "%.2f youngs/s, %.2f non-youngs/s\n" "Pages read %lu, created %lu, written %lu\n" "%.2f reads/s, %.2f creates/s, %.2f writes/s\n", - (ulong) buf_pool->n_pages_read, - (ulong) buf_pool->n_pages_created, - (ulong) buf_pool->n_pages_written, - (buf_pool->n_pages_read - buf_pool->n_pages_read_old) + (ulong) buf_pool->stat.n_pages_made_young, + (ulong) buf_pool->stat.n_pages_not_made_young, + (buf_pool->stat.n_pages_made_young + - buf_pool->old_stat.n_pages_made_young) + / time_elapsed, + (buf_pool->stat.n_pages_not_made_young + - buf_pool->old_stat.n_pages_not_made_young) / time_elapsed, - (buf_pool->n_pages_created - buf_pool->n_pages_created_old) + (ulong) buf_pool->stat.n_pages_read, + (ulong) buf_pool->stat.n_pages_created, + (ulong) buf_pool->stat.n_pages_written, + (buf_pool->stat.n_pages_read + - buf_pool->old_stat.n_pages_read) / time_elapsed, - (buf_pool->n_pages_written - buf_pool->n_pages_written_old) + (buf_pool->stat.n_pages_created + - buf_pool->old_stat.n_pages_created) + / time_elapsed, + (buf_pool->stat.n_pages_written + - buf_pool->old_stat.n_pages_written) / time_elapsed); - if (buf_pool->n_page_gets > buf_pool->n_page_gets_old) { - fprintf(file, "Buffer pool hit rate %lu / 1000\n", + n_gets_diff = buf_pool->stat.n_page_gets - buf_pool->old_stat.n_page_gets; + + if (n_gets_diff) { + fprintf(file, + "Buffer pool hit rate %lu / 1000," + " young-making rate %lu / 1000 not %lu / 1000\n", (ulong) - (1000 - ((1000 * (buf_pool->n_pages_read - - buf_pool->n_pages_read_old)) - / (buf_pool->n_page_gets - - buf_pool->n_page_gets_old)))); + (1000 - ((1000 * (buf_pool->stat.n_pages_read + - buf_pool->old_stat.n_pages_read)) + / (buf_pool->stat.n_page_gets + - buf_pool->old_stat.n_page_gets))), + (ulong) + (1000 * (buf_pool->stat.n_pages_made_young + - buf_pool->old_stat.n_pages_made_young) + / n_gets_diff), + (ulong) + (1000 * (buf_pool->stat.n_pages_not_made_young + - buf_pool->old_stat.n_pages_not_made_young) + / n_gets_diff)); } else { fputs("No buffer pool page gets since the last printout\n", file); } - buf_pool->n_page_gets_old = buf_pool->n_page_gets; - buf_pool->n_pages_read_old = buf_pool->n_pages_read; - buf_pool->n_pages_created_old = buf_pool->n_pages_created; - buf_pool->n_pages_written_old = buf_pool->n_pages_written; + /* Statistics about read ahead algorithm */ + fprintf(file, "Pages read ahead %.2f/s," + " evicted without access %.2f/s\n", + (buf_pool->stat.n_ra_pages_read + - buf_pool->old_stat.n_ra_pages_read) + / time_elapsed, + (buf_pool->stat.n_ra_pages_evicted + - buf_pool->old_stat.n_ra_pages_evicted) + / time_elapsed); /* Print some values to help us with visualizing what is happening with LRU eviction. */ @@ -4003,6 +4219,7 @@ buf_print_io( buf_LRU_stat_sum.io, buf_LRU_stat_cur.io, buf_LRU_stat_sum.unzip, buf_LRU_stat_cur.unzip); + buf_refresh_io_stats(); //buf_pool_mutex_exit(); mutex_exit(&LRU_list_mutex); mutex_exit(&free_list_mutex); @@ -4018,10 +4235,7 @@ buf_refresh_io_stats(void) /*======================*/ { buf_pool->last_printout_time = time(NULL); - buf_pool->n_page_gets_old = buf_pool->n_page_gets; - buf_pool->n_pages_read_old = buf_pool->n_pages_read; - buf_pool->n_pages_created_old = buf_pool->n_pages_created; - buf_pool->n_pages_written_old = buf_pool->n_pages_written; + buf_pool->old_stat = buf_pool->stat; } /*********************************************************************//** === modified file 'storage/xtradb/buf/buf0flu.c' --- a/storage/xtradb/buf/buf0flu.c 2009-11-13 21:26:08 +0000 +++ b/storage/xtradb/buf/buf0flu.c 2010-01-15 15:58:25 +0000 @@ -331,6 +331,28 @@ buf_flush_write_complete( } /********************************************************************//** +Flush a batch of writes to the datafiles that have already been +written by the OS. */ +static +void +buf_flush_sync_datafiles(void) +/*==========================*/ +{ + /* Wake possible simulated aio thread to actually post the + writes to the operating system */ + os_aio_simulated_wake_handler_threads(); + + /* Wait that all async writes to tablespaces have been posted to + the OS */ + os_aio_wait_until_no_pending_writes(); + + /* Now we flush the data to disk (for example, with fsync) */ + fil_flush_file_spaces(FIL_TABLESPACE); + + return; +} + +/********************************************************************//** Flushes possible buffered writes from the doublewrite memory buffer to disk, and also wakes up the aio thread if simulated aio is used. It is very important to call this function after a batch of writes has been posted, @@ -347,8 +369,8 @@ buf_flush_buffered_writes(void) ulint i; if (!srv_use_doublewrite_buf || trx_doublewrite == NULL) { - os_aio_simulated_wake_handler_threads(); - + /* Sync the writes to the disk. */ + buf_flush_sync_datafiles(); return; } @@ -556,22 +578,10 @@ flush: buf_LRU_stat_inc_io(); } - /* Wake possible simulated aio thread to actually post the - writes to the operating system */ - - os_aio_simulated_wake_handler_threads(); - - /* Wait that all async writes to tablespaces have been posted to - the OS */ - - os_aio_wait_until_no_pending_writes(); - - /* Now we flush the data to disk (for example, with fsync) */ - - fil_flush_file_spaces(FIL_TABLESPACE); + /* Sync the writes to the disk. */ + buf_flush_sync_datafiles(); /* We can now reuse the doublewrite memory buffer: */ - trx_doublewrite->first_free = 0; mutex_exit(&(trx_doublewrite->mutex)); @@ -994,9 +1004,7 @@ buf_flush_try_neighbors( || buf_page_is_old(bpage)) { mutex_t* block_mutex = buf_page_get_mutex_enter(bpage); - ut_a(block_mutex); - - if (buf_flush_ready_for_flush(bpage, flush_type) + if (block_mutex && buf_flush_ready_for_flush(bpage, flush_type) && (i == offset || !bpage->buf_fix_count)) { /* We only try to flush those neighbors != offset where the buf fix count is @@ -1012,7 +1020,7 @@ buf_flush_try_neighbors( //buf_pool_mutex_enter(); rw_lock_s_lock(&page_hash_latch); - } else { + } else if (block_mutex) { mutex_exit(block_mutex); } } @@ -1050,6 +1058,7 @@ buf_flush_batch( min_n), otherwise ignored */ { buf_page_t* bpage; + buf_page_t* prev_bpage = NULL; ulint page_count = 0; ulint old_page_count; ulint space; @@ -1103,6 +1112,9 @@ flush_next: mutex_enter(&flush_list_mutex); remaining = UT_LIST_GET_LEN(buf_pool->flush_list); bpage = UT_LIST_GET_LAST(buf_pool->flush_list); + if (bpage) { + prev_bpage = UT_LIST_GET_PREV(flush_list, bpage); + } mutex_exit(&flush_list_mutex); if (!bpage || bpage->oldest_modification >= lsn_limit) { @@ -1123,11 +1135,14 @@ flush_next: mutex_t*block_mutex = buf_page_get_mutex_enter(bpage); ibool ready; - ut_a(buf_page_in_file(bpage)); + //ut_a(buf_page_in_file(bpage)); - ut_a(block_mutex); - ready = buf_flush_ready_for_flush(bpage, flush_type); - mutex_exit(block_mutex); + if (block_mutex) { + ready = buf_flush_ready_for_flush(bpage, flush_type); + mutex_exit(block_mutex); + } else { + ready = FALSE; + } if (ready) { space = buf_page_get_space(bpage); @@ -1162,6 +1177,13 @@ flush_next: mutex_enter(&flush_list_mutex); bpage = UT_LIST_GET_PREV(flush_list, bpage); //ut_ad(!bpage || bpage->in_flush_list); /* optimistic */ + if (bpage != prev_bpage) { + /* the search may warp.. retrying */ + bpage = NULL; + } + if (bpage) { + prev_bpage = UT_LIST_GET_PREV(flush_list, bpage); + } mutex_exit(&flush_list_mutex); remaining--; } @@ -1271,13 +1293,13 @@ buf_flush_LRU_recommendation(void) } block_mutex = buf_page_get_mutex_enter(bpage); - ut_a(block_mutex); - - if (buf_flush_ready_for_replace(bpage)) { + if (block_mutex && buf_flush_ready_for_replace(bpage)) { n_replaceable++; } - mutex_exit(block_mutex); + if (block_mutex) { + mutex_exit(block_mutex); + } distance++; === modified file 'storage/xtradb/buf/buf0lru.c' --- a/storage/xtradb/buf/buf0lru.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/buf/buf0lru.c 2010-01-06 12:00:14 +0000 @@ -49,18 +49,22 @@ Created 11/5/1995 Heikki Tuuri #include "log0recv.h" #include "srv0srv.h" -/** The number of blocks from the LRU_old pointer onward, including the block -pointed to, must be 3/8 of the whole LRU list length, except that the -tolerance defined below is allowed. Note that the tolerance must be small -enough such that for even the BUF_LRU_OLD_MIN_LEN long LRU list, the -LRU_old pointer is not allowed to point to either end of the LRU list. */ +/** The number of blocks from the LRU_old pointer onward, including +the block pointed to, must be buf_LRU_old_ratio/BUF_LRU_OLD_RATIO_DIV +of the whole LRU list length, except that the tolerance defined below +is allowed. Note that the tolerance must be small enough such that for +even the BUF_LRU_OLD_MIN_LEN long LRU list, the LRU_old pointer is not +allowed to point to either end of the LRU list. */ #define BUF_LRU_OLD_TOLERANCE 20 -/** The whole LRU list length is divided by this number to determine an -initial segment in buf_LRU_get_recent_limit */ - -#define BUF_LRU_INITIAL_RATIO 8 +/** The minimum amount of non-old blocks when the LRU_old list exists +(that is, when there are more than BUF_LRU_OLD_MIN_LEN blocks). +@see buf_LRU_old_adjust_len */ +#define BUF_LRU_NON_OLD_MIN_LEN 5 +#if BUF_LRU_NON_OLD_MIN_LEN >= BUF_LRU_OLD_MIN_LEN +# error "BUF_LRU_NON_OLD_MIN_LEN >= BUF_LRU_OLD_MIN_LEN" +#endif /** When dropping the search hash index entries before deleting an ibd file, we build a local array of pages belonging to that tablespace @@ -107,6 +111,15 @@ UNIV_INTERN buf_LRU_stat_t buf_LRU_stat_ /* @} */ +/** @name Heuristics for detecting index scan @{ */ +/** Reserve this much/BUF_LRU_OLD_RATIO_DIV of the buffer pool for +"old" blocks. Protected by buf_pool_mutex. */ +UNIV_INTERN uint buf_LRU_old_ratio; +/** Move blocks to "new" LRU list only if the first access was at +least this many milliseconds ago. Not protected by any mutex or latch. */ +UNIV_INTERN uint buf_LRU_old_threshold_ms; +/* @} */ + /******************************************************************//** Takes a block out of the LRU list and page hash table. If the block is compressed-only (BUF_BLOCK_ZIP_PAGE), @@ -255,9 +268,12 @@ scan_again: mutex_t* block_mutex = buf_page_get_mutex_enter(bpage); buf_page_t* prev_bpage; - ut_a(block_mutex); prev_bpage = UT_LIST_GET_PREV(LRU, bpage); + if (!block_mutex) { + goto next_page; + } + ut_a(buf_page_in_file(bpage)); if (buf_page_get_state(bpage) != BUF_BLOCK_FILE_PAGE @@ -360,9 +376,13 @@ scan_again: ut_a(buf_page_in_file(bpage)); - ut_a(block_mutex); prev_bpage = UT_LIST_GET_PREV(LRU, bpage); + if (!block_mutex) { + bpage = prev_bpage; + continue; + } + if (buf_page_get_space(bpage) == id) { if (bpage->buf_fix_count > 0 || buf_page_get_io_fix(bpage) != BUF_IO_NONE) { @@ -450,45 +470,6 @@ next_page: } } -/******************************************************************//** -Gets the minimum LRU_position field for the blocks in an initial segment -(determined by BUF_LRU_INITIAL_RATIO) of the LRU list. The limit is not -guaranteed to be precise, because the ulint_clock may wrap around. -@return the limit; zero if could not determine it */ -UNIV_INTERN -ulint -buf_LRU_get_recent_limit(void) -/*==========================*/ -{ - const buf_page_t* bpage; - ulint len; - ulint limit; - - //buf_pool_mutex_enter(); - mutex_enter(&LRU_list_mutex); - - len = UT_LIST_GET_LEN(buf_pool->LRU); - - if (len < BUF_LRU_OLD_MIN_LEN) { - /* The LRU list is too short to do read-ahead */ - - //buf_pool_mutex_exit(); - mutex_exit(&LRU_list_mutex); - - return(0); - } - - bpage = UT_LIST_GET_FIRST(buf_pool->LRU); - - limit = buf_page_get_LRU_position(bpage); - len /= BUF_LRU_INITIAL_RATIO; - - //buf_pool_mutex_exit(); - mutex_exit(&LRU_list_mutex); - - return(limit > len ? (limit - len) : 0); -} - /********************************************************************//** Insert a compressed block into buf_pool->zip_clean in the LRU order. */ UNIV_INTERN @@ -631,10 +612,13 @@ restart: bpage = UT_LIST_GET_PREV(LRU, bpage), distance--) { enum buf_lru_free_block_status freed; + unsigned accessed; mutex_t* block_mutex = buf_page_get_mutex_enter(bpage); - ut_a(block_mutex); + if (!block_mutex) { + goto restart; + } if (!bpage->in_LRU_list || !buf_page_in_file(bpage)) { @@ -645,11 +629,18 @@ restart: ut_ad(buf_page_in_file(bpage)); ut_ad(bpage->in_LRU_list); + accessed = buf_page_is_accessed(bpage); freed = buf_LRU_free_block(bpage, TRUE, NULL, have_LRU_mutex); mutex_exit(block_mutex); switch (freed) { case BUF_LRU_FREED: + /* Keep track of pages that are evicted without + ever being accessed. This gives us a measure of + the effectiveness of readahead */ + if (!accessed) { + ++buf_pool->stat.n_ra_pages_evicted; + } return(TRUE); case BUF_LRU_NOT_FREED: @@ -1027,8 +1018,10 @@ buf_LRU_old_adjust_len(void) ut_a(buf_pool->LRU_old); //ut_ad(buf_pool_mutex_own()); ut_ad(mutex_own(&LRU_list_mutex)); -#if 3 * (BUF_LRU_OLD_MIN_LEN / 8) <= BUF_LRU_OLD_TOLERANCE + 5 -# error "3 * (BUF_LRU_OLD_MIN_LEN / 8) <= BUF_LRU_OLD_TOLERANCE + 5" + ut_ad(buf_LRU_old_ratio >= BUF_LRU_OLD_RATIO_MIN); + ut_ad(buf_LRU_old_ratio <= BUF_LRU_OLD_RATIO_MAX); +#if BUF_LRU_OLD_RATIO_MIN * BUF_LRU_OLD_MIN_LEN <= BUF_LRU_OLD_RATIO_DIV * (BUF_LRU_OLD_TOLERANCE + 5) +# error "BUF_LRU_OLD_RATIO_MIN * BUF_LRU_OLD_MIN_LEN <= BUF_LRU_OLD_RATIO_DIV * (BUF_LRU_OLD_TOLERANCE + 5)" #endif #ifdef UNIV_LRU_DEBUG /* buf_pool->LRU_old must be the first item in the LRU list @@ -1040,34 +1033,39 @@ buf_LRU_old_adjust_len(void) || UT_LIST_GET_NEXT(LRU, buf_pool->LRU_old)->old); #endif /* UNIV_LRU_DEBUG */ + old_len = buf_pool->LRU_old_len; + new_len = ut_min(UT_LIST_GET_LEN(buf_pool->LRU) + * buf_LRU_old_ratio / BUF_LRU_OLD_RATIO_DIV, + UT_LIST_GET_LEN(buf_pool->LRU) + - (BUF_LRU_OLD_TOLERANCE + + BUF_LRU_NON_OLD_MIN_LEN)); + for (;;) { - old_len = buf_pool->LRU_old_len; - new_len = 3 * (UT_LIST_GET_LEN(buf_pool->LRU) / 8); + buf_page_t* LRU_old = buf_pool->LRU_old; - ut_ad(buf_pool->LRU_old->in_LRU_list); - ut_a(buf_pool->LRU_old); + ut_a(LRU_old); + ut_ad(LRU_old->in_LRU_list); #ifdef UNIV_LRU_DEBUG - ut_a(buf_pool->LRU_old->old); + ut_a(LRU_old->old); #endif /* UNIV_LRU_DEBUG */ /* Update the LRU_old pointer if necessary */ - if (old_len < new_len - BUF_LRU_OLD_TOLERANCE) { + if (old_len + BUF_LRU_OLD_TOLERANCE < new_len) { - buf_pool->LRU_old = UT_LIST_GET_PREV( - LRU, buf_pool->LRU_old); + buf_pool->LRU_old = LRU_old = UT_LIST_GET_PREV( + LRU, LRU_old); #ifdef UNIV_LRU_DEBUG - ut_a(!buf_pool->LRU_old->old); + ut_a(!LRU_old->old); #endif /* UNIV_LRU_DEBUG */ - buf_page_set_old(buf_pool->LRU_old, TRUE); - buf_pool->LRU_old_len++; + old_len = ++buf_pool->LRU_old_len; + buf_page_set_old(LRU_old, TRUE); } else if (old_len > new_len + BUF_LRU_OLD_TOLERANCE) { - buf_page_set_old(buf_pool->LRU_old, FALSE); - buf_pool->LRU_old = UT_LIST_GET_NEXT( - LRU, buf_pool->LRU_old); - buf_pool->LRU_old_len--; + buf_pool->LRU_old = UT_LIST_GET_NEXT(LRU, LRU_old); + old_len = --buf_pool->LRU_old_len; + buf_page_set_old(LRU_old, FALSE); } else { return; } @@ -1092,12 +1090,13 @@ buf_LRU_old_init(void) the adjust function to move the LRU_old pointer to the right position */ - bpage = UT_LIST_GET_FIRST(buf_pool->LRU); - - while (bpage != NULL) { + for (bpage = UT_LIST_GET_LAST(buf_pool->LRU); bpage != NULL; + bpage = UT_LIST_GET_PREV(LRU, bpage)) { ut_ad(bpage->in_LRU_list); - buf_page_set_old(bpage, TRUE); - bpage = UT_LIST_GET_NEXT(LRU, bpage); + ut_ad(buf_page_in_file(bpage)); + /* This loop temporarily violates the + assertions of buf_page_set_old(). */ + bpage->old = TRUE; } buf_pool->LRU_old = UT_LIST_GET_FIRST(buf_pool->LRU); @@ -1152,16 +1151,19 @@ buf_LRU_remove_block( if (UNIV_UNLIKELY(bpage == buf_pool->LRU_old)) { - /* Below: the previous block is guaranteed to exist, because - the LRU_old pointer is only allowed to differ by the - tolerance value from strict 3/8 of the LRU list length. */ + /* Below: the previous block is guaranteed to exist, + because the LRU_old pointer is only allowed to differ + by BUF_LRU_OLD_TOLERANCE from strict + buf_LRU_old_ratio/BUF_LRU_OLD_RATIO_DIV of the LRU + list length. */ + buf_page_t* prev_bpage = UT_LIST_GET_PREV(LRU, bpage); - buf_pool->LRU_old = UT_LIST_GET_PREV(LRU, bpage); - ut_a(buf_pool->LRU_old); + ut_a(prev_bpage); #ifdef UNIV_LRU_DEBUG - ut_a(!buf_pool->LRU_old->old); + ut_a(!prev_bpage->old); #endif /* UNIV_LRU_DEBUG */ - buf_page_set_old(buf_pool->LRU_old, TRUE); + buf_pool->LRU_old = prev_bpage; + buf_page_set_old(prev_bpage, TRUE); buf_pool->LRU_old_len++; } @@ -1172,10 +1174,19 @@ buf_LRU_remove_block( buf_unzip_LRU_remove_block_if_needed(bpage); - /* If the LRU list is so short that LRU_old not defined, return */ + /* If the LRU list is so short that LRU_old is not defined, + clear the "old" flags and return */ if (UT_LIST_GET_LEN(buf_pool->LRU) < BUF_LRU_OLD_MIN_LEN) { + for (bpage = UT_LIST_GET_FIRST(buf_pool->LRU); bpage != NULL; + bpage = UT_LIST_GET_NEXT(LRU, bpage)) { + /* This loop temporarily violates the + assertions of buf_page_set_old(). */ + bpage->old = FALSE; + } + buf_pool->LRU_old = NULL; + buf_pool->LRU_old_len = 0; return; } @@ -1227,8 +1238,6 @@ buf_LRU_add_block_to_end_low( /*=========================*/ buf_page_t* bpage) /*!< in: control block */ { - buf_page_t* last_bpage; - ut_ad(buf_pool); ut_ad(bpage); //ut_ad(buf_pool_mutex_own()); @@ -1236,31 +1245,18 @@ buf_LRU_add_block_to_end_low( ut_a(buf_page_in_file(bpage)); - last_bpage = UT_LIST_GET_LAST(buf_pool->LRU); - - if (last_bpage) { - bpage->LRU_position = last_bpage->LRU_position; - } else { - bpage->LRU_position = buf_pool_clock_tic(); - } - ut_ad(!bpage->in_LRU_list); UT_LIST_ADD_LAST(LRU, buf_pool->LRU, bpage); bpage->in_LRU_list = TRUE; - buf_page_set_old(bpage, TRUE); - - if (UT_LIST_GET_LEN(buf_pool->LRU) >= BUF_LRU_OLD_MIN_LEN) { - - buf_pool->LRU_old_len++; - } - if (UT_LIST_GET_LEN(buf_pool->LRU) > BUF_LRU_OLD_MIN_LEN) { ut_ad(buf_pool->LRU_old); /* Adjust the length of the old block list if necessary */ + buf_page_set_old(bpage, TRUE); + buf_pool->LRU_old_len++; buf_LRU_old_adjust_len(); } else if (UT_LIST_GET_LEN(buf_pool->LRU) == BUF_LRU_OLD_MIN_LEN) { @@ -1269,6 +1265,8 @@ buf_LRU_add_block_to_end_low( defined: init it */ buf_LRU_old_init(); + } else { + buf_page_set_old(bpage, buf_pool->LRU_old != NULL); } /* If this is a zipped block with decompressed frame as well @@ -1302,7 +1300,6 @@ buf_LRU_add_block_low( UT_LIST_ADD_FIRST(LRU, buf_pool->LRU, bpage); - bpage->LRU_position = buf_pool_clock_tic(); bpage->freed_page_clock = buf_pool->freed_page_clock; } else { #ifdef UNIV_LRU_DEBUG @@ -1317,23 +1314,17 @@ buf_LRU_add_block_low( UT_LIST_INSERT_AFTER(LRU, buf_pool->LRU, buf_pool->LRU_old, bpage); buf_pool->LRU_old_len++; - - /* We copy the LRU position field of the previous block - to the new block */ - - bpage->LRU_position = (buf_pool->LRU_old)->LRU_position; } bpage->in_LRU_list = TRUE; - buf_page_set_old(bpage, old); - if (UT_LIST_GET_LEN(buf_pool->LRU) > BUF_LRU_OLD_MIN_LEN) { ut_ad(buf_pool->LRU_old); /* Adjust the length of the old block list if necessary */ + buf_page_set_old(bpage, old); buf_LRU_old_adjust_len(); } else if (UT_LIST_GET_LEN(buf_pool->LRU) == BUF_LRU_OLD_MIN_LEN) { @@ -1342,6 +1333,8 @@ buf_LRU_add_block_low( defined: init it */ buf_LRU_old_init(); + } else { + buf_page_set_old(bpage, buf_pool->LRU_old != NULL); } /* If this is a zipped block with decompressed frame as well @@ -1375,6 +1368,13 @@ buf_LRU_make_block_young( /*=====================*/ buf_page_t* bpage) /*!< in: control block */ { + //ut_ad(buf_pool_mutex_own()); + ut_ad(mutex_own(&LRU_list_mutex)); + + if (bpage->old) { + buf_pool->stat.n_pages_made_young++; + } + buf_LRU_remove_block(bpage); buf_LRU_add_block_low(bpage, FALSE); } @@ -1571,15 +1571,6 @@ not_freed: buf_pool->LRU_old = b; } -#ifdef UNIV_LRU_DEBUG - ut_a(prev_b->old - || !UT_LIST_GET_NEXT(LRU, b) - || UT_LIST_GET_NEXT(LRU, b)->old); - } else { - ut_a(!prev_b->old - || !UT_LIST_GET_NEXT(LRU, b) - || !UT_LIST_GET_NEXT(LRU, b)->old); -#endif /* UNIV_LRU_DEBUG */ } lru_len = UT_LIST_GET_LEN(buf_pool->LRU); @@ -1595,6 +1586,11 @@ not_freed: defined: init it */ buf_LRU_old_init(); } +#ifdef UNIV_LRU_DEBUG + /* Check that the "old" flag is consistent + in the block and its neighbours. */ + buf_page_set_old(b, buf_page_is_old(b)); +#endif /* UNIV_LRU_DEBUG */ } else { b->in_LRU_list = FALSE; buf_LRU_add_block_low(b, buf_page_is_old(b)); @@ -1985,6 +1981,52 @@ buf_LRU_block_free_hashed_page( buf_LRU_block_free_non_file_page(block, have_page_hash_mutex); } +/**********************************************************************//** +Updates buf_LRU_old_ratio. +@return updated old_pct */ +UNIV_INTERN +uint +buf_LRU_old_ratio_update( +/*=====================*/ + uint old_pct,/*!< in: Reserve this percentage of + the buffer pool for "old" blocks. */ + ibool adjust) /*!< in: TRUE=adjust the LRU list; + FALSE=just assign buf_LRU_old_ratio + during the initialization of InnoDB */ +{ + uint ratio; + + ratio = old_pct * BUF_LRU_OLD_RATIO_DIV / 100; + if (ratio < BUF_LRU_OLD_RATIO_MIN) { + ratio = BUF_LRU_OLD_RATIO_MIN; + } else if (ratio > BUF_LRU_OLD_RATIO_MAX) { + ratio = BUF_LRU_OLD_RATIO_MAX; + } + + if (adjust) { + //buf_pool_mutex_enter(); + mutex_enter(&LRU_list_mutex); + + if (ratio != buf_LRU_old_ratio) { + buf_LRU_old_ratio = ratio; + + if (UT_LIST_GET_LEN(buf_pool->LRU) + >= BUF_LRU_OLD_MIN_LEN) { + buf_LRU_old_adjust_len(); + } + } + + //buf_pool_mutex_exit(); + mutex_exit(&LRU_list_mutex); + } else { + buf_LRU_old_ratio = ratio; + } + + /* the reverse of + ratio = old_pct * BUF_LRU_OLD_RATIO_DIV / 100 */ + return((uint) (ratio * 100 / (double) BUF_LRU_OLD_RATIO_DIV + 0.5)); +} + /********************************************************************//** Update the historical stats that we are collecting for LRU eviction policy at the end of each interval. */ @@ -2023,6 +2065,218 @@ func_exit: memset(&buf_LRU_stat_cur, 0, sizeof buf_LRU_stat_cur); } +/********************************************************************//** +Dump the LRU page list to the specific file. */ +#define LRU_DUMP_FILE "ib_lru_dump" + +UNIV_INTERN +ibool +buf_LRU_file_dump(void) +/*===================*/ +{ + os_file_t dump_file = -1; + ibool success; + byte* buffer_base = NULL; + byte* buffer = NULL; + buf_page_t* bpage; + ulint buffers; + ulint offset; + ibool ret = FALSE; + ulint i; + + for (i = 0; i < srv_n_data_files; i++) { + if (strstr(srv_data_file_names[i], LRU_DUMP_FILE) != NULL) { + fprintf(stderr, + " InnoDB: The name '%s' seems to be used for" + " innodb_data_file_path. Dumping LRU list is not" + " done for safeness.\n", LRU_DUMP_FILE); + goto end; + } + } + + buffer_base = ut_malloc(2 * UNIV_PAGE_SIZE); + buffer = ut_align(buffer_base, UNIV_PAGE_SIZE); + if (!buffer) { + fprintf(stderr, + " InnoDB: cannot allocate buffer.\n"); + goto end; + } + + dump_file = os_file_create(LRU_DUMP_FILE, OS_FILE_OVERWRITE, + OS_FILE_NORMAL, OS_DATA_FILE, &success); + if (!success) { + os_file_get_last_error(TRUE); + fprintf(stderr, + " InnoDB: cannot open %s\n", LRU_DUMP_FILE); + goto end; + } + + mutex_enter(&LRU_list_mutex); + bpage = UT_LIST_GET_LAST(buf_pool->LRU); + + buffers = offset = 0; + while (bpage != NULL) { + if (offset == 0) { + memset(buffer, 0, UNIV_PAGE_SIZE); + } + + mach_write_to_4(buffer + offset * 4, bpage->space); + offset++; + mach_write_to_4(buffer + offset * 4, bpage->offset); + offset++; + + if (offset == UNIV_PAGE_SIZE/4) { + success = os_file_write(LRU_DUMP_FILE, dump_file, buffer, + (buffers << UNIV_PAGE_SIZE_SHIFT) & 0xFFFFFFFFUL, + (buffers >> (32 - UNIV_PAGE_SIZE_SHIFT)), + UNIV_PAGE_SIZE); + if (!success) { + mutex_exit(&LRU_list_mutex); + fprintf(stderr, + " InnoDB: cannot write page %lu of %s\n", + buffers, LRU_DUMP_FILE); + goto end; + } + buffers++; + offset = 0; + } + + bpage = UT_LIST_GET_PREV(LRU, bpage); + } + mutex_exit(&LRU_list_mutex); + + if (offset == 0) { + memset(buffer, 0, UNIV_PAGE_SIZE); + } + + mach_write_to_4(buffer + offset * 4, 0xFFFFFFFFUL); + offset++; + mach_write_to_4(buffer + offset * 4, 0xFFFFFFFFUL); + offset++; + + success = os_file_write(LRU_DUMP_FILE, dump_file, buffer, + (buffers << UNIV_PAGE_SIZE_SHIFT) & 0xFFFFFFFFUL, + (buffers >> (32 - UNIV_PAGE_SIZE_SHIFT)), + UNIV_PAGE_SIZE); + if (!success) { + goto end; + } + + ret = TRUE; +end: + if (dump_file != -1) + os_file_close(dump_file); + if (buffer_base) + ut_free(buffer_base); + + return(ret); +} +/********************************************************************//** +Read the pages based on the specific file.*/ +UNIV_INTERN +ibool +buf_LRU_file_restore(void) +/*======================*/ +{ + os_file_t dump_file = -1; + ibool success; + byte* buffer_base = NULL; + byte* buffer = NULL; + ulint buffers; + ulint offset; + ulint reads = 0; + ulint req = 0; + ibool terminated = FALSE; + ibool ret = FALSE; + + buffer_base = ut_malloc(2 * UNIV_PAGE_SIZE); + buffer = ut_align(buffer_base, UNIV_PAGE_SIZE); + if (!buffer) { + fprintf(stderr, + " InnoDB: cannot allocate buffer.\n"); + goto end; + } + + dump_file = os_file_create_simple_no_error_handling( + LRU_DUMP_FILE, OS_FILE_OPEN, OS_FILE_READ_ONLY, &success); + if (!success) { + os_file_get_last_error(TRUE); + fprintf(stderr, + " InnoDB: cannot open %s\n", LRU_DUMP_FILE); + goto end; + } + + buffers = 0; + while (!terminated) { + success = os_file_read(dump_file, buffer, + (buffers << UNIV_PAGE_SIZE_SHIFT) & 0xFFFFFFFFUL, + (buffers >> (32 - UNIV_PAGE_SIZE_SHIFT)), + UNIV_PAGE_SIZE); + if (!success) { + fprintf(stderr, + " InnoDB: cannot read page %lu of %s," + " or meet unexpected terminal.", + buffers, LRU_DUMP_FILE); + goto end; + } + + for (offset = 0; offset < UNIV_PAGE_SIZE/4; offset += 2) { + ulint space_id, zip_size, page_no; + ulint err; + ib_int64_t tablespace_version; + + space_id = mach_read_from_4(buffer + offset * 4); + page_no = mach_read_from_4(buffer + (offset + 1) * 4); + if (space_id == 0xFFFFFFFFUL + || page_no == 0xFFFFFFFFUL) { + terminated = TRUE; + break; + } + + if (offset % 16 == 15) { + os_aio_simulated_wake_handler_threads(); + buf_flush_free_margin(FALSE); + } + + zip_size = fil_space_get_zip_size(space_id); + if (UNIV_UNLIKELY(zip_size == ULINT_UNDEFINED)) { + continue; + } + + if (fil_area_is_exist(space_id, zip_size, page_no, 0, + zip_size ? zip_size : UNIV_PAGE_SIZE)) { + + tablespace_version = fil_space_get_version(space_id); + + req++; + reads += buf_read_page_low(&err, FALSE, BUF_READ_ANY_PAGE + | OS_AIO_SIMULATED_WAKE_LATER, + space_id, zip_size, TRUE, + tablespace_version, page_no, NULL); + buf_LRU_stat_inc_io(); + } + } + + buffers++; + } + + os_aio_simulated_wake_handler_threads(); + buf_flush_free_margin(FALSE); + + ut_print_timestamp(stderr); + fprintf(stderr, + " InnoDB: reading pages based on the dumped LRU list was done." + " (requested: %lu, read: %lu)\n", req, reads); + ret = TRUE; +end: + if (dump_file != -1) + os_file_close(dump_file); + if (buffer_base) + ut_free(buffer_base); + + return(ret); +} + #if defined UNIV_DEBUG || defined UNIV_BUF_DEBUG /**********************************************************************//** Validates the LRU list. @@ -2036,7 +2290,6 @@ buf_LRU_validate(void) buf_block_t* block; ulint old_len; ulint new_len; - ulint LRU_pos; ut_ad(buf_pool); //buf_pool_mutex_enter(); @@ -2046,7 +2299,11 @@ buf_LRU_validate(void) ut_a(buf_pool->LRU_old); old_len = buf_pool->LRU_old_len; - new_len = 3 * (UT_LIST_GET_LEN(buf_pool->LRU) / 8); + new_len = ut_min(UT_LIST_GET_LEN(buf_pool->LRU) + * buf_LRU_old_ratio / BUF_LRU_OLD_RATIO_DIV, + UT_LIST_GET_LEN(buf_pool->LRU) + - (BUF_LRU_OLD_TOLERANCE + + BUF_LRU_NON_OLD_MIN_LEN)); ut_a(old_len >= new_len - BUF_LRU_OLD_TOLERANCE); ut_a(old_len <= new_len + BUF_LRU_OLD_TOLERANCE); } @@ -2077,28 +2334,24 @@ buf_LRU_validate(void) } if (buf_page_is_old(bpage)) { - old_len++; - } + const buf_page_t* prev + = UT_LIST_GET_PREV(LRU, bpage); + const buf_page_t* next + = UT_LIST_GET_NEXT(LRU, bpage); - if (buf_pool->LRU_old && (old_len == 1)) { - ut_a(buf_pool->LRU_old == bpage); - } + if (!old_len++) { + ut_a(buf_pool->LRU_old == bpage); + } else { + ut_a(!prev || buf_page_is_old(prev)); + } - LRU_pos = buf_page_get_LRU_position(bpage); + ut_a(!next || buf_page_is_old(next)); + } bpage = UT_LIST_GET_NEXT(LRU, bpage); - - if (bpage) { - /* If the following assert fails, it may - not be an error: just the buf_pool clock - has wrapped around */ - ut_a(LRU_pos >= buf_page_get_LRU_position(bpage)); - } } - if (buf_pool->LRU_old) { - ut_a(buf_pool->LRU_old_len == old_len); - } + ut_a(buf_pool->LRU_old_len == old_len); mutex_exit(&LRU_list_mutex); mutex_enter(&free_list_mutex); @@ -2149,9 +2402,6 @@ buf_LRU_print(void) //buf_pool_mutex_enter(); mutex_enter(&LRU_list_mutex); - fprintf(stderr, "Pool ulint clock %lu\n", - (ulong) buf_pool->ulint_clock); - bpage = UT_LIST_GET_FIRST(buf_pool->LRU); while (bpage != NULL) { @@ -2182,18 +2432,16 @@ buf_LRU_print(void) const byte* frame; case BUF_BLOCK_FILE_PAGE: frame = buf_block_get_frame((buf_block_t*) bpage); - fprintf(stderr, "\nLRU pos %lu type %lu" + fprintf(stderr, "\ntype %lu" " index id %lu\n", - (ulong) buf_page_get_LRU_position(bpage), (ulong) fil_page_get_type(frame), (ulong) ut_dulint_get_low( btr_page_get_index_id(frame))); break; case BUF_BLOCK_ZIP_PAGE: frame = bpage->zip.data; - fprintf(stderr, "\nLRU pos %lu type %lu size %lu" + fprintf(stderr, "\ntype %lu size %lu" " index id %lu\n", - (ulong) buf_page_get_LRU_position(bpage), (ulong) fil_page_get_type(frame), (ulong) buf_page_get_zip_size(bpage), (ulong) ut_dulint_get_low( @@ -2201,8 +2449,7 @@ buf_LRU_print(void) break; default: - fprintf(stderr, "\nLRU pos %lu !state %lu!\n", - (ulong) buf_page_get_LRU_position(bpage), + fprintf(stderr, "\n!state %lu!\n", (ulong) buf_page_get_state(bpage)); break; } === modified file 'storage/xtradb/buf/buf0rea.c' --- a/storage/xtradb/buf/buf0rea.c 2009-11-13 21:26:08 +0000 +++ b/storage/xtradb/buf/buf0rea.c 2010-01-15 15:58:25 +0000 @@ -38,14 +38,6 @@ Created 11/5/1995 Heikki Tuuri #include "srv0start.h" #include "srv0srv.h" -/** The size in blocks of the area where the random read-ahead algorithm counts -the accessed pages when deciding whether to read-ahead */ -#define BUF_READ_AHEAD_RANDOM_AREA BUF_READ_AHEAD_AREA - -/** There must be at least this many pages in buf_pool in the area to start -a random read-ahead */ -#define BUF_READ_AHEAD_RANDOM_THRESHOLD (1 + BUF_READ_AHEAD_RANDOM_AREA / 2) - /** The linear read-ahead area size */ #define BUF_READ_AHEAD_LINEAR_AREA BUF_READ_AHEAD_AREA @@ -62,8 +54,9 @@ flag is cleared and the x-lock released @return 1 if a read request was queued, 0 if the page already resided in buf_pool, or if the page is in the doublewrite buffer blocks in which case it is never read into the pool, or if the tablespace does -not exist or is being dropped */ -static +not exist or is being dropped +@return 1 if read request is issued. 0 if it is not */ +UNIV_INTERN ulint buf_read_page_low( /*==============*/ @@ -82,7 +75,8 @@ buf_read_page_low( treat the tablespace as dropped; this is a timestamp we use to stop dangling page reads from a tablespace which we have DISCARDed + IMPORTed back */ - ulint offset) /*!< in: page number */ + ulint offset, /*!< in: page number */ + trx_t* trx) { buf_page_t* bpage; ulint wake_later; @@ -183,15 +177,15 @@ not_to_recover: ut_ad(buf_page_in_file(bpage)); if (zip_size) { - *err = fil_io(OS_FILE_READ | wake_later, + *err = _fil_io(OS_FILE_READ | wake_later, sync, space, zip_size, offset, 0, zip_size, - bpage->zip.data, bpage); + bpage->zip.data, bpage, trx); } else { ut_a(buf_page_get_state(bpage) == BUF_BLOCK_FILE_PAGE); - *err = fil_io(OS_FILE_READ | wake_later, + *err = _fil_io(OS_FILE_READ | wake_later, sync, space, 0, offset, 0, UNIV_PAGE_SIZE, - ((buf_block_t*) bpage)->frame, bpage); + ((buf_block_t*) bpage)->frame, bpage, trx); } ut_a(*err == DB_SUCCESS); @@ -205,206 +199,33 @@ not_to_recover: } /********************************************************************//** -Applies a random read-ahead in buf_pool if there are at least a threshold -value of accessed pages from the random read-ahead area. Does not read any -page, not even the one at the position (space, offset), if the read-ahead -mechanism is not activated. NOTE 1: the calling thread may own latches on -pages: to avoid deadlocks this function must be written such that it cannot -end up waiting for these latches! NOTE 2: the calling thread must want -access to the page given: this rule is set to prevent unintended read-aheads -performed by ibuf routines, a situation which could result in a deadlock if -the OS does not support asynchronous i/o. -@return number of page read requests issued; NOTE that if we read ibuf -pages, it may happen that the page at the given page number does not -get read even if we return a positive value! */ -static -ulint -buf_read_ahead_random( -/*==================*/ - ulint space, /*!< in: space id */ - ulint zip_size,/*!< in: compressed page size in bytes, or 0 */ - ulint offset) /*!< in: page number of a page which the current thread - wants to access */ -{ - ib_int64_t tablespace_version; - ulint recent_blocks = 0; - ulint count; - ulint LRU_recent_limit; - ulint ibuf_mode; - ulint low, high; - ulint err; - ulint i; - ulint buf_read_ahead_random_area; - -// /* We have currently disabled random readahead */ -// return(0); - - if (!(srv_read_ahead & 1)) { - return(0); - } - - if (srv_startup_is_before_trx_rollback_phase) { - /* No read-ahead to avoid thread deadlocks */ - return(0); - } - - if (ibuf_bitmap_page(zip_size, offset) - || trx_sys_hdr_page(space, offset)) { - - /* If it is an ibuf bitmap page or trx sys hdr, we do - no read-ahead, as that could break the ibuf page access - order */ - - return(0); - } - - /* Remember the tablespace version before we ask te tablespace size - below: if DISCARD + IMPORT changes the actual .ibd file meanwhile, we - do not try to read outside the bounds of the tablespace! */ - - tablespace_version = fil_space_get_version(space); - - buf_read_ahead_random_area = BUF_READ_AHEAD_RANDOM_AREA; - - low = (offset / buf_read_ahead_random_area) - * buf_read_ahead_random_area; - high = (offset / buf_read_ahead_random_area + 1) - * buf_read_ahead_random_area; - if (high > fil_space_get_size(space)) { - - high = fil_space_get_size(space); - } - - /* Get the minimum LRU_position field value for an initial segment - of the LRU list, to determine which blocks have recently been added - to the start of the list. */ - - LRU_recent_limit = buf_LRU_get_recent_limit(); - - //buf_pool_mutex_enter(); - mutex_enter(&buf_pool_mutex); - - if (buf_pool->n_pend_reads - > buf_pool->curr_size / BUF_READ_AHEAD_PEND_LIMIT) { - //buf_pool_mutex_exit(); - mutex_exit(&buf_pool_mutex); - - return(0); - } - mutex_exit(&buf_pool_mutex); - - /* Count how many blocks in the area have been recently accessed, - that is, reside near the start of the LRU list. */ - - rw_lock_s_lock(&page_hash_latch); - for (i = low; i < high; i++) { - const buf_page_t* bpage = buf_page_hash_get(space, i); - - if (bpage - && buf_page_is_accessed(bpage) - && (buf_page_get_LRU_position(bpage) > LRU_recent_limit)) { - - recent_blocks++; - - if (recent_blocks >= BUF_READ_AHEAD_RANDOM_THRESHOLD) { - - //buf_pool_mutex_exit(); - rw_lock_s_unlock(&page_hash_latch); - goto read_ahead; - } - } - } - - //buf_pool_mutex_exit(); - rw_lock_s_unlock(&page_hash_latch); - /* Do nothing */ - return(0); - -read_ahead: - /* Read all the suitable blocks within the area */ - - if (ibuf_inside()) { - ibuf_mode = BUF_READ_IBUF_PAGES_ONLY; - } else { - ibuf_mode = BUF_READ_ANY_PAGE; - } - - count = 0; - - for (i = low; i < high; i++) { - /* It is only sensible to do read-ahead in the non-sync aio - mode: hence FALSE as the first parameter */ - - if (!ibuf_bitmap_page(zip_size, i)) { - count += buf_read_page_low( - &err, FALSE, - ibuf_mode | OS_AIO_SIMULATED_WAKE_LATER, - space, zip_size, FALSE, - tablespace_version, i); - if (err == DB_TABLESPACE_DELETED) { - ut_print_timestamp(stderr); - fprintf(stderr, - " InnoDB: Warning: in random" - " readahead trying to access\n" - "InnoDB: tablespace %lu page %lu,\n" - "InnoDB: but the tablespace does not" - " exist or is just being dropped.\n", - (ulong) space, (ulong) i); - } - } - } - - /* In simulated aio we wake the aio handler threads only after - queuing all aio requests, in native aio the following call does - nothing: */ - - os_aio_simulated_wake_handler_threads(); - -#ifdef UNIV_DEBUG - if (buf_debug_prints && (count > 0)) { - fprintf(stderr, - "Random read-ahead space %lu offset %lu pages %lu\n", - (ulong) space, (ulong) offset, - (ulong) count); - } -#endif /* UNIV_DEBUG */ - - ++srv_read_ahead_rnd; - return(count); -} - -/********************************************************************//** High-level function which reads a page asynchronously from a file to the buffer buf_pool if it is not already there. Sets the io_fix flag and sets an exclusive lock on the buffer frame. The flag is cleared and the x-lock -released by the i/o-handler thread. Does a random read-ahead if it seems -sensible. -@return number of page read requests issued: this can be greater than -1 if read-ahead occurred */ +released by the i/o-handler thread. +@return TRUE if page has been read in, FALSE in case of failure */ UNIV_INTERN -ulint +ibool buf_read_page( /*==========*/ ulint space, /*!< in: space id */ ulint zip_size,/*!< in: compressed page size in bytes, or 0 */ - ulint offset) /*!< in: page number */ + ulint offset, /*!< in: page number */ + trx_t* trx) { ib_int64_t tablespace_version; ulint count; - ulint count2; ulint err; tablespace_version = fil_space_get_version(space); - count = buf_read_ahead_random(space, zip_size, offset); - /* We do the i/o in the synchronous aio mode to save thread switches: hence TRUE */ - count2 = buf_read_page_low(&err, TRUE, BUF_READ_ANY_PAGE, space, - zip_size, FALSE, - tablespace_version, offset); - srv_buf_pool_reads+= count2; + count = buf_read_page_low(&err, TRUE, BUF_READ_ANY_PAGE, space, + zip_size, FALSE, + tablespace_version, offset, trx); + srv_buf_pool_reads += count; if (err == DB_TABLESPACE_DELETED) { ut_print_timestamp(stderr); fprintf(stderr, @@ -421,14 +242,14 @@ buf_read_page( /* Increment number of I/O operations used for LRU policy. */ buf_LRU_stat_inc_io(); - return(count + count2); + return(count > 0); } /********************************************************************//** Applies linear read-ahead if in the buf_pool the page is a border page of a linear read-ahead area and all the pages in the area have been accessed. Does not read any page if the read-ahead mechanism is not activated. Note -that the the algorithm looks at the 'natural' adjacent successor and +that the algorithm looks at the 'natural' adjacent successor and predecessor of the page, which on the leaf level of a B-tree are the next and previous page in the chain of leaves. To know these, the page specified in (space, offset) must already be present in the buf_pool. Thus, the @@ -454,8 +275,9 @@ buf_read_ahead_linear( /*==================*/ ulint space, /*!< in: space id */ ulint zip_size,/*!< in: compressed page size in bytes, or 0 */ - ulint offset) /*!< in: page number of a page; NOTE: the current thread + ulint offset, /*!< in: page number of a page; NOTE: the current thread must want access to this page (see NOTE 3 above) */ + trx_t* trx) { ib_int64_t tablespace_version; buf_page_t* bpage; @@ -557,9 +379,17 @@ buf_read_ahead_linear( fail_count++; } else if (pred_bpage) { - int res = (ut_ulint_cmp( - buf_page_get_LRU_position(bpage), - buf_page_get_LRU_position(pred_bpage))); + /* Note that buf_page_is_accessed() returns + the time of the first access. If some blocks + of the extent existed in the buffer pool at + the time of a linear access pattern, the first + access times may be nonmonotonic, even though + the latest access times were linear. The + threshold (srv_read_ahead_factor) should help + a little against this. */ + int res = ut_ulint_cmp( + buf_page_is_accessed(bpage), + buf_page_is_accessed(pred_bpage)); /* Accesses not in the right order */ if (res != 0 && res != asc_or_desc) { fail_count++; @@ -670,7 +500,7 @@ buf_read_ahead_linear( count += buf_read_page_low( &err, FALSE, ibuf_mode | OS_AIO_SIMULATED_WAKE_LATER, - space, zip_size, FALSE, tablespace_version, i); + space, zip_size, FALSE, tablespace_version, i, trx); if (err == DB_TABLESPACE_DELETED) { ut_print_timestamp(stderr); fprintf(stderr, @@ -705,7 +535,7 @@ buf_read_ahead_linear( LRU policy decision. */ buf_LRU_stat_inc_io(); - ++srv_read_ahead_seq; + buf_pool->stat.n_ra_pages_read += count; return(count); } @@ -760,7 +590,7 @@ buf_read_ibuf_merge_pages( buf_read_page_low(&err, sync && (i + 1 == n_stored), BUF_READ_ANY_PAGE, space_ids[i], zip_size, TRUE, space_versions[i], - page_nos[i]); + page_nos[i], NULL); if (UNIV_UNLIKELY(err == DB_TABLESPACE_DELETED)) { tablespace_deleted: @@ -857,12 +687,12 @@ buf_read_recv_pages( if ((i + 1 == n_stored) && sync) { buf_read_page_low(&err, TRUE, BUF_READ_ANY_PAGE, space, zip_size, TRUE, tablespace_version, - page_nos[i]); + page_nos[i], NULL); } else { buf_read_page_low(&err, FALSE, BUF_READ_ANY_PAGE | OS_AIO_SIMULATED_WAKE_LATER, space, zip_size, TRUE, - tablespace_version, page_nos[i]); + tablespace_version, page_nos[i], NULL); } } === modified file 'storage/xtradb/data/data0type.c' --- a/storage/xtradb/data/data0type.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/data/data0type.c 2010-01-06 12:00:14 +0000 @@ -237,6 +237,22 @@ dtype_print( fputs("DATA_SYS", stderr); break; + case DATA_FLOAT: + fputs("DATA_FLOAT", stderr); + break; + + case DATA_DOUBLE: + fputs("DATA_DOUBLE", stderr); + break; + + case DATA_DECIMAL: + fputs("DATA_DECIMAL", stderr); + break; + + case DATA_VARMYSQL: + fputs("DATA_VARMYSQL", stderr); + break; + default: fprintf(stderr, "type %lu", (ulong) mtype); break; === modified file 'storage/xtradb/dict/dict0crea.c' --- a/storage/xtradb/dict/dict0crea.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/dict/dict0crea.c 2010-01-06 12:00:14 +0000 @@ -1387,7 +1387,7 @@ dict_create_add_foreign_field_to_diction Add a single foreign key definition to the data dictionary tables in the database. We also generate names to constraints that were not named by the user. A generated constraint has a name of the format -databasename/tablename_ibfk_<number>, where the numbers start from 1, and +databasename/tablename_ibfk_NUMBER, where the numbers start from 1, and are given locally for this table, that is, the number is not global, as in the old format constraints < 4.0.18 it used to be. @return error code or DB_SUCCESS */ === modified file 'storage/xtradb/dict/dict0dict.c' --- a/storage/xtradb/dict/dict0dict.c 2009-12-03 11:34:11 +0000 +++ b/storage/xtradb/dict/dict0dict.c 2010-01-15 15:58:25 +0000 @@ -82,9 +82,10 @@ static char dict_ibfk[] = "_ibfk_"; /*******************************************************************//** Tries to find column names for the index and sets the col field of the -index. */ +index. +@return TRUE if the column names were found */ static -void +ibool dict_index_find_cols( /*=================*/ dict_table_t* table, /*!< in: table */ @@ -1261,7 +1262,7 @@ dict_index_too_big_for_undo( = TRX_UNDO_PAGE_HDR - TRX_UNDO_PAGE_HDR_SIZE + 2 /* next record pointer */ + 1 /* type_cmpl */ - + 11 /* trx->undo_no */ - 11 /* table->id */ + + 11 /* trx->undo_no */ + 11 /* table->id */ + 1 /* rec_get_info_bits() */ + 11 /* DB_TRX_ID */ + 11 /* DB_ROLL_PTR */ @@ -1440,7 +1441,11 @@ dict_index_too_big_for_tree( goto add_field_size; } + if (srv_relax_table_creation) { + field_max_size = dict_col_get_min_size(col); + } else { field_max_size = dict_col_get_max_size(col); + } field_ext_max_size = field_max_size < 256 ? 1 : 2; if (field->prefix_len) { @@ -1493,7 +1498,7 @@ add_field_size: /**********************************************************************//** Adds an index to the dictionary cache. -@return DB_SUCCESS or DB_TOO_BIG_RECORD */ +@return DB_SUCCESS, DB_TOO_BIG_RECORD, or DB_CORRUPTION */ UNIV_INTERN ulint dict_index_add_to_cache( @@ -1519,7 +1524,10 @@ dict_index_add_to_cache( ut_a(!dict_index_is_clust(index) || UT_LIST_GET_LEN(table->indexes) == 0); - dict_index_find_cols(table, index); + if (!dict_index_find_cols(table, index)) { + + return(DB_CORRUPTION); + } /* Build the cache internal representation of the index, containing also the added system fields */ @@ -1732,9 +1740,10 @@ dict_index_remove_from_cache( /*******************************************************************//** Tries to find column names for the index and sets the col field of the -index. */ +index. +@return TRUE if the column names were found */ static -void +ibool dict_index_find_cols( /*=================*/ dict_table_t* table, /*!< in: table */ @@ -1759,17 +1768,21 @@ dict_index_find_cols( } } +#ifdef UNIV_DEBUG /* It is an error not to find a matching column. */ fputs("InnoDB: Error: no matching column for ", stderr); ut_print_name(stderr, NULL, FALSE, field->name); fputs(" in ", stderr); dict_index_name_print(stderr, NULL, index); fputs("!\n", stderr); - ut_error; +#endif /* UNIV_DEBUG */ + return(FALSE); found: ; } + + return(TRUE); } #endif /* !UNIV_HOTBACKUP */ @@ -4711,6 +4724,26 @@ dict_ind_init(void) dict_ind_redundant->cached = dict_ind_compact->cached = TRUE; } +/**********************************************************************//** +Frees dict_ind_redundant and dict_ind_compact. */ +static +void +dict_ind_free(void) +/*===============*/ +{ + dict_table_t* table; + + table = dict_ind_compact->table; + dict_mem_index_free(dict_ind_compact); + dict_ind_compact = NULL; + dict_mem_table_free(table); + + table = dict_ind_redundant->table; + dict_mem_index_free(dict_ind_redundant); + dict_ind_redundant = NULL; + dict_mem_table_free(table); +} + #ifndef UNIV_HOTBACKUP /**********************************************************************//** Get index by name @@ -4836,4 +4869,55 @@ dict_table_check_for_dup_indexes( } } #endif /* UNIV_DEBUG */ + +/************************************************************************** +Closes the data dictionary module. */ +UNIV_INTERN +void +dict_close(void) +/*============*/ +{ + ulint i; + + /* Free the hash elements. We don't remove them from the table + because we are going to destroy the table anyway. */ + for (i = 0; i < hash_get_n_cells(dict_sys->table_hash); i++) { + dict_table_t* table; + + table = HASH_GET_FIRST(dict_sys->table_hash, i); + + while (table) { + dict_table_t* prev_table = table; + + table = HASH_GET_NEXT(name_hash, prev_table); +#ifdef UNIV_DEBUG + ut_a(prev_table->magic_n == DICT_TABLE_MAGIC_N); +#endif + /* Acquire only because it's a pre-condition. */ + mutex_enter(&dict_sys->mutex); + + dict_table_remove_from_cache(prev_table); + + mutex_exit(&dict_sys->mutex); + } + } + + hash_table_free(dict_sys->table_hash); + + /* The elements are the same instance as in dict_sys->table_hash, + therefore we don't delete the individual elements. */ + hash_table_free(dict_sys->table_id_hash); + + dict_ind_free(); + + mutex_free(&dict_sys->mutex); + + rw_lock_free(&dict_operation_lock); + memset(&dict_operation_lock, 0x0, sizeof(dict_operation_lock)); + + mutex_free(&dict_foreign_err_mutex); + + mem_free(dict_sys); + dict_sys = NULL; +} #endif /* !UNIV_HOTBACKUP */ === modified file 'storage/xtradb/fil/fil0fil.c' --- a/storage/xtradb/fil/fil0fil.c 2009-11-29 23:08:56 +0000 +++ b/storage/xtradb/fil/fil0fil.c 2010-01-15 15:58:25 +0000 @@ -327,6 +327,17 @@ fil_get_space_id_for_table( /*=======================*/ const char* name); /*!< in: table name in the standard 'databasename/tablename' format */ +/*******************************************************************//** +Frees a space object from the tablespace memory cache. Closes the files in +the chain but does not delete them. There must not be any pending i/o's or +flushes on the files. */ +static +ibool +fil_space_free( +/*===========*/ + /* out: TRUE if success */ + ulint id, /* in: space id */ + ibool own_mutex);/* in: TRUE if own system->mutex */ /********************************************************************//** Reads data from a space to a buffer. Remember that the possible incomplete blocks at the end of file are ignored: they are not taken into account when @@ -600,6 +611,11 @@ fil_node_create( UT_LIST_ADD_LAST(chain, space->chain, node); + if (id < SRV_LOG_SPACE_FIRST_ID && fil_system->max_assigned_id < id) { + + fil_system->max_assigned_id = id; + } + mutex_exit(&fil_system->mutex); } @@ -619,12 +635,10 @@ fil_node_open_file( ulint size_high; ibool ret; ibool success; -#ifndef UNIV_HOTBACKUP byte* buf2; byte* page; ulint space_id; ulint flags; -#endif /* !UNIV_HOTBACKUP */ ut_ad(mutex_own(&(system->mutex))); ut_a(node->n_pending == 0); @@ -660,9 +674,12 @@ fil_node_open_file( size_bytes = (((ib_int64_t)size_high) << 32) + (ib_int64_t)size_low; #ifdef UNIV_HOTBACKUP - node->size = (ulint) (size_bytes / UNIV_PAGE_SIZE); - /* TODO: adjust to zip_size, like below? */ -#else + if (space->id == 0) { + node->size = (ulint) (size_bytes / UNIV_PAGE_SIZE); + os_file_close(node->handle); + goto add_size; + } +#endif /* UNIV_HOTBACKUP */ ut_a(space->purpose != FIL_LOG); ut_a(space->id != 0); @@ -741,7 +758,10 @@ fil_node_open_file( (size_bytes / dict_table_flags_to_zip_size(flags)); } -#endif + +#ifdef UNIV_HOTBACKUP +add_size: +#endif /* UNIV_HOTBACKUP */ space->size += node->size; } @@ -961,7 +981,7 @@ close_more: " while the maximum\n" "InnoDB: allowed value would be %lu.\n" "InnoDB: You may need to raise the value of" - " innodb_max_files_open in\n" + " innodb_open_files in\n" "InnoDB: my.cnf.\n", (ulong) fil_system->n_open, (ulong) fil_system->max_n_open); @@ -1141,7 +1161,7 @@ try_again: mutex_exit(&fil_system->mutex); - fil_space_free(namesake_id); + fil_space_free(namesake_id, FALSE); goto try_again; } @@ -1266,17 +1286,21 @@ Frees a space object from the tablespace the chain but does not delete them. There must not be any pending i/o's or flushes on the files. @return TRUE if success */ -UNIV_INTERN +static ibool fil_space_free( /*===========*/ - ulint id) /*!< in: space id */ + /* out: TRUE if success */ + ulint id, /* in: space id */ + ibool own_mutex) /* in: TRUE if own system->mutex */ { fil_space_t* space; fil_space_t* namespace; fil_node_t* fil_node; - mutex_enter(&fil_system->mutex); + if (!own_mutex) { + mutex_enter(&fil_system->mutex); + } space = fil_space_get_by_id(id); @@ -1323,7 +1347,9 @@ fil_space_free( ut_a(0 == UT_LIST_GET_LEN(space->chain)); - mutex_exit(&fil_system->mutex); + if (!own_mutex) { + mutex_exit(&fil_system->mutex); + } rw_lock_free(&(space->latch)); @@ -1541,7 +1567,7 @@ fil_open_log_and_system_tablespace_files fprintf(stderr, "InnoDB: Warning: you must" " raise the value of" - " innodb_max_open_files in\n" + " innodb_open_files in\n" "InnoDB: my.cnf! Remember that" " InnoDB keeps all log files" " and all system\n" @@ -1583,6 +1609,8 @@ fil_close_all_files(void) space = UT_LIST_GET_FIRST(fil_system->space_list); while (space != NULL) { + fil_space_t* prev_space = space; + node = UT_LIST_GET_FIRST(space->chain); while (node != NULL) { @@ -1592,6 +1620,7 @@ fil_close_all_files(void) node = UT_LIST_GET_NEXT(chain, node); } space = UT_LIST_GET_NEXT(space_list, space); + fil_space_free(prev_space->id, TRUE); } mutex_exit(&fil_system->mutex); @@ -2223,7 +2252,7 @@ try_again: #endif /* printf("Deleting tablespace %s id %lu\n", space->name, id); */ - success = fil_space_free(id); + success = fil_space_free(id, FALSE); if (success) { success = os_file_delete(path); @@ -2929,7 +2958,6 @@ fil_open_single_table_tablespace( byte* page; ulint space_id; ulint space_flags; - ibool ret = TRUE; filepath = fil_make_ibd_name(name, FALSE); @@ -3330,7 +3358,7 @@ skip_write: (ulong) space_id, (ulong) space_flags, (ulong) id, (ulong) flags); - ret = FALSE; + success = FALSE; goto func_exit; } @@ -3350,7 +3378,7 @@ func_exit: os_file_close(file); mem_free(filepath); - return(ret); + return(success); } #endif /* !UNIV_HOTBACKUP */ @@ -3566,7 +3594,7 @@ fil_load_single_table_tablespace( fprintf(stderr, "InnoDB: Renaming tablespace %s of id %lu,\n" "InnoDB: to %s_ibbackup_old_vers_<timestamp>\n" - "InnoDB: because its size %lld is too small" + "InnoDB: because its size %" PRId64 " is too small" " (< 4 pages 16 kB each),\n" "InnoDB: or the space id in the file header" " is not sensible.\n" @@ -3628,7 +3656,17 @@ fil_load_single_table_tablespace( if (!success) { - goto func_exit; + if (srv_force_recovery > 0) { + fprintf(stderr, + "InnoDB: innodb_force_recovery" + " was set to %lu. Continuing crash recovery\n" + "InnoDB: even though the tablespace creation" + " of this table failed.\n", + srv_force_recovery); + goto func_exit; + } + + exit(1); } /* We do not use the size information we have about the file, because @@ -4163,7 +4201,7 @@ fil_extend_space_to_desired_size( node->name, node->handle, buf, offset_low, offset_high, page_size * n_pages, - NULL, NULL); + NULL, NULL, NULL); #endif if (success) { node->size += n_pages; @@ -4490,7 +4528,7 @@ Reads or writes data. This operation is i/o on a tablespace which does not exist */ UNIV_INTERN ulint -fil_io( +_fil_io( /*===*/ ulint type, /*!< in: OS_FILE_READ or OS_FILE_WRITE, ORed to OS_FILE_LOG, if a log i/o @@ -4515,8 +4553,9 @@ fil_io( void* buf, /*!< in/out: buffer where to store read data or from where to write; in aio this must be appropriately aligned */ - void* message) /*!< in: message for aio handler if non-sync + void* message, /*!< in: message for aio handler if non-sync aio used, else ignored */ + trx_t* trx) { ulint mode; fil_space_t* space; @@ -4686,7 +4725,7 @@ fil_io( #else /* Queue the aio request */ ret = os_aio(type, mode | wake_later, node->name, node->handle, buf, - offset_low, offset_high, len, node, message); + offset_low, offset_high, len, node, message, trx); #endif ut_a(ret); @@ -4706,6 +4745,78 @@ fil_io( return(DB_SUCCESS); } +/********************************************************************//** +Confirm whether the parameters are valid or not */ +UNIV_INTERN +ibool +fil_area_is_exist( +/*==============*/ + ulint space_id, /*!< in: space id */ + ulint zip_size, /*!< in: compressed page size in bytes; + 0 for uncompressed pages */ + ulint block_offset, /*!< in: offset in number of blocks */ + ulint byte_offset, /*!< in: remainder of offset in bytes; in + aio this must be divisible by the OS block + size */ + ulint len) /*!< in: how many bytes to read or write; this + must not cross a file boundary; in aio this + must be a block size multiple */ +{ + fil_space_t* space; + fil_node_t* node; + + /* Reserve the fil_system mutex and make sure that we can open at + least one file while holding it, if the file is not already open */ + + fil_mutex_enter_and_prepare_for_io(space_id); + + space = fil_space_get_by_id(space_id); + + if (!space) { + mutex_exit(&fil_system->mutex); + return(FALSE); + } + + node = UT_LIST_GET_FIRST(space->chain); + + for (;;) { + if (UNIV_UNLIKELY(node == NULL)) { + mutex_exit(&fil_system->mutex); + return(FALSE); + } + + if (space->id != 0 && node->size == 0) { + /* We do not know the size of a single-table tablespace + before we open the file */ + + break; + } + + if (node->size > block_offset) { + /* Found! */ + break; + } else { + block_offset -= node->size; + node = UT_LIST_GET_NEXT(chain, node); + } + } + + /* Open file if closed */ + fil_node_prepare_for_io(node, fil_system, space); + fil_node_complete_io(node, fil_system, OS_FILE_READ); + + /* Check that at least the start offset is within the bounds of a + single-table tablespace */ + if (UNIV_UNLIKELY(node->size <= block_offset) + && space->id != 0 && space->purpose == FIL_TABLESPACE) { + mutex_exit(&fil_system->mutex); + return(FALSE); + } + + mutex_exit(&fil_system->mutex); + return(TRUE); +} + #ifndef UNIV_HOTBACKUP /**********************************************************************//** Waits for an aio operation to complete. This function is used to write the @@ -5065,6 +5176,29 @@ fil_page_get_type( return(mach_read_from_2(page + FIL_PAGE_TYPE)); } +/******************************************************************** +Initializes the tablespace memory cache. */ +UNIV_INTERN +void +fil_close(void) +/*===========*/ +{ + /* The mutex should already have been freed. */ + ut_ad(fil_system->mutex.magic_n == 0); + + hash_table_free(fil_system->spaces); + + hash_table_free(fil_system->name_hash); + + ut_a(UT_LIST_GET_LEN(fil_system->LRU) == 0); + ut_a(UT_LIST_GET_LEN(fil_system->unflushed_spaces) == 0); + ut_a(UT_LIST_GET_LEN(fil_system->space_list) == 0); + + mem_free(fil_system); + + fil_system = NULL; +} + /************************************************************************* Return local hash table informations. */ === modified file 'storage/xtradb/fsp/fsp0fsp.c' --- a/storage/xtradb/fsp/fsp0fsp.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/fsp/fsp0fsp.c 2010-01-06 12:00:14 +0000 @@ -232,6 +232,9 @@ the extent are free and which contain ol #define XDES_ARR_OFFSET (FSP_HEADER_OFFSET + FSP_HEADER_SIZE) #ifndef UNIV_HOTBACKUP +/* Flag to indicate if we have printed the tablespace full error. */ +static ibool fsp_tbs_full_error_printed = FALSE; + /**********************************************************************//** Returns an extent to the free list of a space. */ static @@ -1099,7 +1102,7 @@ fsp_header_inc_size( /**********************************************************************//** Gets the current free limit of the system tablespace. The free limit -means the place of the first page which has never been put to the the +means the place of the first page which has never been put to the free list for allocation. The space above that address is initialized to zero. Sets also the global variable log_fsp_current_free_limit. @return free limit in megabytes */ @@ -1218,6 +1221,19 @@ fsp_try_extend_data_file( if (space == 0 && !srv_auto_extend_last_data_file) { + /* We print the error message only once to avoid + spamming the error log. Note that we don't need + to reset the flag to FALSE as dealing with this + error requires server restart. */ + if (fsp_tbs_full_error_printed == FALSE) { + fprintf(stderr, + "InnoDB: Error: Data file(s) ran" + " out of space.\n" + "Please add another data file or" + " use \'autoextend\' for the last" + " data file.\n"); + fsp_tbs_full_error_printed = TRUE; + } return(FALSE); } @@ -1832,6 +1848,8 @@ fsp_seg_inode_page_find_used( if (!ut_dulint_is_zero(mach_read_from_8(inode + FSEG_ID))) { /* This is used */ + ut_ad(mach_read_from_4(inode + FSEG_MAGIC_N) + == FSEG_MAGIC_N_VALUE); return(i); } } @@ -1863,6 +1881,9 @@ fsp_seg_inode_page_find_free( return(i); } + + ut_ad(mach_read_from_4(inode + FSEG_MAGIC_N) + == FSEG_MAGIC_N_VALUE); } return(ULINT_UNDEFINED); @@ -1981,6 +2002,8 @@ fsp_alloc_seg_inode( page + FSEG_INODE_PAGE_NODE, mtr); } + ut_ad(ut_dulint_is_zero(mach_read_from_8(inode + FSEG_ID)) + || mach_read_from_4(inode + FSEG_MAGIC_N) == FSEG_MAGIC_N_VALUE); return(inode); } @@ -2018,7 +2041,7 @@ fsp_free_seg_inode( } mlog_write_dulint(inode + FSEG_ID, ut_dulint_zero, mtr); - mlog_write_ulint(inode + FSEG_MAGIC_N, 0, MLOG_4BYTES, mtr); + mlog_write_ulint(inode + FSEG_MAGIC_N, 0xfa051ce3, MLOG_4BYTES, mtr); if (ULINT_UNDEFINED == fsp_seg_inode_page_find_used(page, zip_size, mtr)) { @@ -2034,11 +2057,11 @@ fsp_free_seg_inode( /**********************************************************************//** Returns the file segment inode, page x-latched. -@return segment inode, page x-latched */ +@return segment inode, page x-latched; NULL if the inode is free */ static fseg_inode_t* -fseg_inode_get( -/*===========*/ +fseg_inode_try_get( +/*===============*/ fseg_header_t* header, /*!< in: segment header */ ulint space, /*!< in: space id */ ulint zip_size,/*!< in: compressed page size in bytes @@ -2054,12 +2077,38 @@ fseg_inode_get( inode = fut_get_ptr(space, zip_size, inode_addr, RW_X_LATCH, mtr); - ut_ad(mach_read_from_4(inode + FSEG_MAGIC_N) == FSEG_MAGIC_N_VALUE); + if (UNIV_UNLIKELY + (ut_dulint_is_zero(mach_read_from_8(inode + FSEG_ID)))) { + + inode = NULL; + } else { + ut_ad(mach_read_from_4(inode + FSEG_MAGIC_N) + == FSEG_MAGIC_N_VALUE); + } return(inode); } /**********************************************************************//** +Returns the file segment inode, page x-latched. +@return segment inode, page x-latched */ +static +fseg_inode_t* +fseg_inode_get( +/*===========*/ + fseg_header_t* header, /*!< in: segment header */ + ulint space, /*!< in: space id */ + ulint zip_size,/*!< in: compressed page size in bytes + or 0 for uncompressed pages */ + mtr_t* mtr) /*!< in: mtr handle */ +{ + fseg_inode_t* inode + = fseg_inode_try_get(header, space, zip_size, mtr); + ut_a(inode); + return(inode); +} + +/**********************************************************************//** Gets the page number from the nth fragment page slot. @return page number, FIL_NULL if not in use */ UNIV_INLINE @@ -2073,6 +2122,7 @@ fseg_get_nth_frag_page_no( ut_ad(inode && mtr); ut_ad(n < FSEG_FRAG_ARR_N_SLOTS); ut_ad(mtr_memo_contains_page(mtr, inode, MTR_MEMO_PAGE_X_FIX)); + ut_ad(mach_read_from_4(inode + FSEG_MAGIC_N) == FSEG_MAGIC_N_VALUE); return(mach_read_from_4(inode + FSEG_FRAG_ARR + n * FSEG_FRAG_SLOT_SIZE)); } @@ -2091,6 +2141,7 @@ fseg_set_nth_frag_page_no( ut_ad(inode && mtr); ut_ad(n < FSEG_FRAG_ARR_N_SLOTS); ut_ad(mtr_memo_contains_page(mtr, inode, MTR_MEMO_PAGE_X_FIX)); + ut_ad(mach_read_from_4(inode + FSEG_MAGIC_N) == FSEG_MAGIC_N_VALUE); mlog_write_ulint(inode + FSEG_FRAG_ARR + n * FSEG_FRAG_SLOT_SIZE, page_no, MLOG_4BYTES, mtr); @@ -2451,6 +2502,8 @@ fseg_fill_free_list( xdes_set_state(descr, XDES_FSEG, mtr); seg_id = mtr_read_dulint(inode + FSEG_ID, mtr); + ut_ad(mach_read_from_4(inode + FSEG_MAGIC_N) + == FSEG_MAGIC_N_VALUE); mlog_write_dulint(descr + XDES_ID, seg_id, mtr); flst_add_last(inode + FSEG_FREE, descr + XDES_FLST_NODE, mtr); @@ -2479,6 +2532,7 @@ fseg_alloc_free_extent( fil_addr_t first; ut_ad(!((page_offset(inode) - FSEG_ARR_OFFSET) % FSEG_INODE_SIZE)); + ut_ad(mach_read_from_4(inode + FSEG_MAGIC_N) == FSEG_MAGIC_N_VALUE); if (flst_get_len(inode + FSEG_FREE, mtr) > 0) { /* Segment free list is not empty, allocate from it */ @@ -3136,6 +3190,8 @@ fseg_mark_page_used( ut_ad(seg_inode && mtr); ut_ad(!((page_offset(seg_inode) - FSEG_ARR_OFFSET) % FSEG_INODE_SIZE)); + ut_ad(mach_read_from_4(seg_inode + FSEG_MAGIC_N) + == FSEG_MAGIC_N_VALUE); descr = xdes_get_descriptor(space, zip_size, page, mtr); @@ -3373,6 +3429,8 @@ fseg_free_extent( ut_a(xdes_get_state(descr, mtr) == XDES_FSEG); ut_a(0 == ut_dulint_cmp(mtr_read_dulint(descr + XDES_ID, mtr), mtr_read_dulint(seg_inode + FSEG_ID, mtr))); + ut_ad(mach_read_from_4(seg_inode + FSEG_MAGIC_N) + == FSEG_MAGIC_N_VALUE); first_page_in_extent = page - (page % FSP_EXTENT_SIZE); @@ -3463,7 +3521,13 @@ fseg_free_step( ut_a(descr); ut_a(xdes_get_bit(descr, XDES_FREE_BIT, header_page % FSP_EXTENT_SIZE, mtr) == FALSE); - inode = fseg_inode_get(header, space, zip_size, mtr); + inode = fseg_inode_try_get(header, space, zip_size, mtr); + + if (UNIV_UNLIKELY(inode == NULL)) { + fprintf(stderr, "double free of inode from %u:%u\n", + (unsigned) space, (unsigned) header_page); + return(TRUE); + } descr = fseg_get_first_extent(inode, space, zip_size, mtr); @@ -3587,6 +3651,7 @@ fseg_get_first_extent( ut_ad(inode && mtr); ut_ad(space == page_get_space_id(page_align(inode))); + ut_ad(mach_read_from_4(inode + FSEG_MAGIC_N) == FSEG_MAGIC_N_VALUE); first = fil_addr_null; @@ -3801,6 +3866,7 @@ fseg_print_low( (ulong) reserved, (ulong) used, (ulong) n_full, (ulong) n_frag, (ulong) n_free, (ulong) n_not_full, (ulong) n_used); + ut_ad(mach_read_from_4(inode + FSEG_MAGIC_N) == FSEG_MAGIC_N_VALUE); } #ifdef UNIV_BTR_PRINT === modified file 'storage/xtradb/handler/ha_innodb.cc' --- a/storage/xtradb/handler/ha_innodb.cc 2009-12-03 11:34:11 +0000 +++ b/storage/xtradb/handler/ha_innodb.cc 2010-01-15 21:12:30 +0000 @@ -61,7 +61,9 @@ with this program; if not, write to the #pragma implementation // gcc: Class implementation #endif +#ifndef MYSQL_SERVER #define MYSQL_SERVER +#endif #include <mysql_priv.h> #ifdef MYSQL_SERVER @@ -77,6 +79,7 @@ with this program; if not, write to the /* Include necessary InnoDB headers */ extern "C" { #include "univ.i" +#include "buf0lru.h" #include "btr0sea.h" #include "os0file.h" #include "os0thread.h" @@ -109,7 +112,6 @@ extern "C" { #include "ha_innodb.h" #include "i_s.h" -#include "handler0vars.h" #ifdef MYSQL_SERVER // Defined in trx0sys.c @@ -120,9 +122,12 @@ extern ib_int64_t trx_sys_mysql_relay_lo #endif /* MYSQL_SERVER */ #ifndef MYSQL_SERVER -/* This is needed because of Bug #3596. Let us hope that pthread_mutex_t +# ifndef MYSQL_PLUGIN_IMPORT +# define MYSQL_PLUGIN_IMPORT /* nothing */ +# endif /* MYSQL_PLUGIN_IMPORT */ +/* This is needed because of Bug #3596. Let us hope that pthread_mutex_t is defined the same in both builds: the MySQL server and the InnoDB plugin. */ -extern pthread_mutex_t LOCK_thread_count; +extern MYSQL_PLUGIN_IMPORT pthread_mutex_t LOCK_thread_count; #if MYSQL_VERSION_ID < 50124 /* this is defined in mysql_priv.h inside #ifdef MYSQL_SERVER @@ -146,13 +151,9 @@ static bool innodb_inited = 0; /* In the Windows plugin, the return value of current_thd is undefined. Map it to NULL. */ -#if defined MYSQL_DYNAMIC_PLUGIN && defined __WIN__ -# undef current_thd -# define current_thd NULL -# define EQ_CURRENT_THD(thd) TRUE -#else /* MYSQL_DYNAMIC_PLUGIN && __WIN__ */ -# define EQ_CURRENT_THD(thd) ((thd) == current_thd) -#endif /* MYSQL_DYNAMIC_PLUGIN && __WIN__ */ + +#define EQ_CURRENT_THD(thd) ((thd) == current_thd) + static struct handlerton* innodb_hton_ptr; @@ -172,6 +173,10 @@ static ulong innobase_write_io_threads; static my_bool innobase_thread_concurrency_timer_based; static long long innobase_buffer_pool_size, innobase_log_file_size; +/** Percentage of the buffer pool to reserve for 'old' blocks. +Connected to buf_LRU_old_ratio. */ +static uint innobase_old_blocks_pct; + /* The default values for the following char* start-up parameters are determined in innobase_init below: */ @@ -186,9 +191,7 @@ file formats in the configuration file, of the supported file formats during runtime. */ static char* innobase_file_format_check = NULL; -/* The following has a misleading name: starting from 4.0.5, this also -affects Windows: */ -static char* innobase_unix_file_flush_method = NULL; +static char* innobase_file_flush_method = NULL; /* Below we have boolean-valued start-up parameters, and their default values */ @@ -202,7 +205,7 @@ static my_bool innobase_use_doublewrite static my_bool innobase_use_checksums = TRUE; static my_bool innobase_extra_undoslots = FALSE; static my_bool innobase_fast_recovery = FALSE; -static my_bool innobase_use_purge_thread = FALSE; +static my_bool innobase_recovery_stats = TRUE; static my_bool innobase_locks_unsafe_for_binlog = FALSE; static my_bool innobase_overwrite_relay_log_info = FALSE; static my_bool innobase_rollback_on_timeout = FALSE; @@ -238,10 +241,10 @@ static void free_share(INNOBASE_SHARE *s static int innobase_close_connection(handlerton *hton, THD* thd); static int innobase_commit(handlerton *hton, THD* thd, bool all); static int innobase_rollback(handlerton *hton, THD* thd, bool all); -static int innobase_rollback_to_savepoint(handlerton *hton, THD* thd, +static int innobase_rollback_to_savepoint(handlerton *hton, THD* thd, void *savepoint); static int innobase_savepoint(handlerton *hton, THD* thd, void *savepoint); -static int innobase_release_savepoint(handlerton *hton, THD* thd, +static int innobase_release_savepoint(handlerton *hton, THD* thd, void *savepoint); static handler *innobase_create_handler(handlerton *hton, TABLE_SHARE *table, @@ -285,10 +288,10 @@ innobase_file_format_check_on_off( /************************************************************//** Validate the file format check config parameters, as a side effect it sets the srv_check_file_format_at_startup variable. -@return true if valid config value */ +@return the format_id if valid config value, otherwise, return -1 */ static -bool -innobase_file_format_check_validate( +int +innobase_file_format_validate_and_set( /*================================*/ const char* format_check); /*!< in: parameter value */ /****************************************************************//** @@ -519,10 +522,10 @@ static SHOW_VAR innodb_status_variables[ (char*) &export_vars.innodb_buffer_pool_pages_misc, SHOW_LONG}, {"buffer_pool_pages_total", (char*) &export_vars.innodb_buffer_pool_pages_total, SHOW_LONG}, - {"buffer_pool_read_ahead_rnd", - (char*) &export_vars.innodb_buffer_pool_read_ahead_rnd, SHOW_LONG}, - {"buffer_pool_read_ahead_seq", - (char*) &export_vars.innodb_buffer_pool_read_ahead_seq, SHOW_LONG}, + {"buffer_pool_read_ahead", + (char*) &export_vars.innodb_buffer_pool_read_ahead, SHOW_LONG}, + {"buffer_pool_read_ahead_evicted", + (char*) &export_vars.innodb_buffer_pool_read_ahead_evicted, SHOW_LONG}, {"buffer_pool_read_requests", (char*) &export_vars.innodb_buffer_pool_read_requests, SHOW_LONG}, {"buffer_pool_reads", @@ -803,11 +806,20 @@ convert_error_code_to_mysql( case DB_SUCCESS: return(0); + case DB_INTERRUPTED: + my_error(ER_QUERY_INTERRUPTED, MYF(0)); + /* fall through */ case DB_ERROR: default: return(-1); /* unspecified error */ case DB_DUPLICATE_KEY: + /* Be cautious with returning this error, since + mysql could re-enter the storage layer to get + duplicated key info, the operation requires a + valid table handle and/or transaction information, + which might not always be available in the error + handling stage. */ return(HA_ERR_FOUND_DUPP_KEY); case DB_FOREIGN_DUPLICATE_KEY: @@ -894,17 +906,14 @@ convert_error_code_to_mysql( return(ER_PRIMARY_CANT_HAVE_NULL); case DB_TOO_MANY_CONCURRENT_TRXS: - /* Once MySQL add the appropriate code to errmsg.txt then - we can get rid of this #ifdef. NOTE: The code checked by - the #ifdef is the suggested name for the error condition - and the actual error code name could very well be different. - This will require some monitoring, ie. the status - of this request on our part.*/ -#ifdef ER_TOO_MANY_CONCURRENT_TRXS - return(ER_TOO_MANY_CONCURRENT_TRXS); -#else + /* New error code HA_ERR_TOO_MANY_CONCURRENT_TRXS is only + available in 5.1.38 and later, but the plugin should still + work with previous versions of MySQL. */ +#ifdef HA_ERR_TOO_MANY_CONCURRENT_TRXS + return(HA_ERR_TOO_MANY_CONCURRENT_TRXS); +#else /* HA_ERR_TOO_MANY_CONCURRENT_TRXS */ return(HA_ERR_RECORD_FILE_FULL); -#endif +#endif /* HA_ERR_TOO_MANY_CONCURRENT_TRXS */ case DB_UNSUPPORTED: return(HA_ERR_UNSUPPORTED); } @@ -978,7 +987,23 @@ innobase_get_cset_width( *mbminlen = cs->mbminlen; *mbmaxlen = cs->mbmaxlen; } else { - ut_a(cset == 0); + THD* thd = current_thd; + + if (thd && thd_sql_command(thd) == SQLCOM_DROP_TABLE) { + + /* Fix bug#46256: allow tables to be dropped if the + collation is not found, but issue a warning. */ + if ((global_system_variables.log_warnings) + && (cset != 0)){ + + sql_print_warning( + "Unknown collation #%lu.", cset); + } + } else { + + ut_a(cset == 0); + } + *mbminlen = *mbmaxlen = 0; } } @@ -1052,6 +1077,7 @@ innobase_get_charset( } #if defined (__WIN__) && defined (MYSQL_DYNAMIC_PLUGIN) +extern MYSQL_PLUGIN_IMPORT MY_TMPDIR mysql_tmpdir_list; /*******************************************************************//** Map an OS error to an errno value. The OS error number is stored in _doserrno and the mapped value is stored in errno) */ @@ -1339,6 +1365,16 @@ innobase_trx_init( trx->check_unique_secondary = !thd_test_options( thd, OPTION_RELAXED_UNIQUE_CHECKS); +#ifdef EXTENDED_SLOWLOG + if (thd_log_slow_verbosity(thd) & SLOG_V_INNODB) { + trx->take_stats = TRUE; + } else { + trx->take_stats = FALSE; + } +#else + trx->take_stats = FALSE; +#endif + DBUG_VOID_RETURN; } @@ -1395,6 +1431,32 @@ check_trx_exists( } +/************************************************************************* +Gets current trx. */ +extern "C" +trx_t* +innobase_get_trx() +{ + THD *thd=current_thd; + if (likely(thd != 0)) { + trx_t*& trx = thd_to_trx(thd); + return(trx); + } else { + return(NULL); + } +} + +extern "C" +ibool +innobase_get_slow_log() +{ +#ifdef EXTENDED_SLOWLOG + return((ibool) thd_opt_slow_log()); +#else + return(FALSE); +#endif +} + /*********************************************************************//** Construct ha_innobase handler. */ UNIV_INTERN @@ -1711,15 +1773,19 @@ innobase_convert_identifier( FALSE=id is an UTF-8 string */ { char nz[NAME_LEN + 1]; +#if MYSQL_VERSION_ID >= 50141 + char nz2[NAME_LEN + 1 + EXPLAIN_FILENAME_MAX_EXTRA_LENGTH]; +#else /* MYSQL_VERSION_ID >= 50141 */ char nz2[NAME_LEN + 1 + sizeof srv_mysql50_table_name_prefix]; +#endif /* MYSQL_VERSION_ID >= 50141 */ const char* s = id; int q; if (file_id) { - /* Decode the table name. The filename_to_tablename() - function expects a NUL-terminated string. The input and - output strings buffers must not be shared. */ + /* Decode the table name. The MySQL function expects + a NUL-terminated string. The input and output strings + buffers must not be shared. */ if (UNIV_UNLIKELY(idlen > (sizeof nz) - 1)) { idlen = (sizeof nz) - 1; @@ -1729,7 +1795,13 @@ innobase_convert_identifier( nz[idlen] = 0; s = nz2; +#if MYSQL_VERSION_ID >= 50141 + idlen = explain_filename((THD*) thd, nz, nz2, sizeof nz2, + EXPLAIN_PARTITIONS_AS_COMMENT); + goto no_quote; +#else /* MYSQL_VERSION_ID >= 50141 */ idlen = filename_to_tablename(nz, nz2, sizeof nz2); +#endif /* MYSQL_VERSION_ID >= 50141 */ } /* See if the identifier needs to be quoted. */ @@ -1740,6 +1812,9 @@ innobase_convert_identifier( } if (q == EOF) { +#if MYSQL_VERSION_ID >= 50141 +no_quote: +#endif /* MYSQL_VERSION_ID >= 50141 */ if (UNIV_UNLIKELY(idlen > buflen)) { idlen = buflen; } @@ -2222,8 +2297,8 @@ mem_free_and_error: /* Did the user specify a format name that we support ? As a side effect it will update the variable srv_check_file_format_at_startup */ - if (!innobase_file_format_check_validate( - innobase_file_format_check)) { + if (innobase_file_format_validate_and_set( + innobase_file_format_check) < 0) { sql_print_error("InnoDB: invalid " "innodb_file_format_check value: " @@ -2264,7 +2339,7 @@ innobase_change_buffering_inited_ok: /* --------------------------------------------------*/ - srv_file_flush_method_str = innobase_unix_file_flush_method; + srv_file_flush_method_str = innobase_file_flush_method; srv_n_log_groups = (ulint) innobase_mirrored_log_groups; srv_n_log_files = (ulint) innobase_log_files_in_group; @@ -2292,8 +2367,7 @@ innobase_change_buffering_inited_ok: srv_force_recovery = (ulint) innobase_force_recovery; srv_fast_recovery = (ibool) innobase_fast_recovery; - - srv_use_purge_thread = (ibool) innobase_use_purge_thread; + srv_recovery_stats = (ibool) innobase_recovery_stats; srv_use_doublewrite_buf = (ibool) innobase_use_doublewrite; srv_use_checksums = (ibool) innobase_use_checksums; @@ -2329,6 +2403,9 @@ innobase_change_buffering_inited_ok: ut_a(0 == strcmp(my_charset_latin1.name, "latin1_swedish_ci")); srv_latin1_ordering = my_charset_latin1.sort_order; + innobase_old_blocks_pct = buf_LRU_old_ratio_update( + innobase_old_blocks_pct, FALSE); + innobase_commit_concurrency_init_default(); /* Since we in this module access directly the fields of a trx @@ -2510,7 +2587,10 @@ innobase_alter_table_flags( { return(HA_ONLINE_ADD_INDEX_NO_WRITES | HA_ONLINE_DROP_INDEX_NO_WRITES - | HA_ONLINE_ADD_UNIQUE_INDEX_NO_WRITES + /* Current InnoDB doesn't sort unique indexes along mysqld's order + It is dangerous to use index. So it is disabled until + the bug http://bugs.mysql.com/47622 */ + /* | HA_ONLINE_ADD_UNIQUE_INDEX_NO_WRITES */ | HA_ONLINE_DROP_UNIQUE_INDEX_NO_WRITES | HA_ONLINE_ADD_PK_INDEX_NO_WRITES); } @@ -2676,6 +2756,19 @@ retry: } } + /* The following calls to read the MySQL binary log + file name and the position return consistent results: + 1) Other InnoDB transactions cannot intervene between + these calls as we are holding prepare_commit_mutex. + 2) Binary logging of other engines is not relevant + to InnoDB as all InnoDB requires is that committing + InnoDB transactions appear in the same order in the + MySQL binary log as they appear in InnoDB logs. + 3) A MySQL log file rotation cannot happen because + MySQL protects against this by having a counter of + transactions in prepared state and it only allows + a rotation when the counter drops to zero. See + LOCK_prep_xids and COND_prep_xids in log.cc. */ trx->mysql_log_file_name = mysql_bin_log_file_name(); trx->mysql_log_offset = (ib_int64_t) mysql_bin_log_file_pos(); @@ -2761,6 +2854,8 @@ innobase_rollback( innobase_release_stat_resources(trx); + trx->n_autoinc_rows = 0; /* Reset the number AUTO-INC rows required */ + /* If we had reserved the auto-inc lock for some table (if we come here to roll back the latest SQL statement) we release it now before a possibly lengthy rollback */ @@ -3322,7 +3417,7 @@ retry: if (is_part) { sql_print_error("Failed to open table %s after " - "%lu attemtps.\n", norm_name, + "%lu attempts.\n", norm_name, retries); } @@ -3926,7 +4021,6 @@ ha_innobase::store_key_val_for_row( as BLOB data in innodb. */ || mysql_type == MYSQL_TYPE_GEOMETRY) { - CHARSET_INFO* cs; ulint key_len; ulint true_len; @@ -4648,24 +4742,29 @@ no_commit: update the table upper limit. Note: last_value will be 0 if get_auto_increment() was not called.*/ - if (auto_inc <= col_max_value - && auto_inc >= prebuilt->autoinc_last_value) { + if (auto_inc >= prebuilt->autoinc_last_value) { set_max_autoinc: - ut_a(prebuilt->autoinc_increment > 0); - - ulonglong need; - ulonglong offset; - - offset = prebuilt->autoinc_offset; - need = prebuilt->autoinc_increment; - - auto_inc = innobase_next_autoinc( - auto_inc, need, offset, col_max_value); - - err = innobase_set_max_autoinc(auto_inc); - - if (err != DB_SUCCESS) { - error = err; + /* This should filter out the negative + values set explicitly by the user. */ + if (auto_inc <= col_max_value) { + ut_a(prebuilt->autoinc_increment > 0); + + ulonglong need; + ulonglong offset; + + offset = prebuilt->autoinc_offset; + need = prebuilt->autoinc_increment; + + auto_inc = innobase_next_autoinc( + auto_inc, + need, offset, col_max_value); + + err = innobase_set_max_autoinc( + auto_inc); + + if (err != DB_SUCCESS) { + error = err; + } } } break; @@ -5227,6 +5326,11 @@ ha_innobase::index_read( index = prebuilt->index; + if (UNIV_UNLIKELY(index == NULL)) { + prebuilt->index_usable = FALSE; + DBUG_RETURN(HA_ERR_CRASHED); + } + /* Note that if the index for which the search template is built is not necessarily prebuilt->index, but can also be the clustered index */ @@ -5386,6 +5490,7 @@ ha_innobase::change_active_index( if (UNIV_UNLIKELY(!prebuilt->index)) { sql_print_warning("InnoDB: change_active_index(%u) failed", keynr); + prebuilt->index_usable = FALSE; DBUG_RETURN(1); } @@ -5393,8 +5498,10 @@ ha_innobase::change_active_index( prebuilt->index); if (UNIV_UNLIKELY(!prebuilt->index_usable)) { - sql_print_warning("InnoDB: insufficient history for index %u", - keynr); + push_warning_printf(user_thd, MYSQL_ERROR::WARN_LEVEL_WARN, + HA_ERR_TABLE_DEF_CHANGED, + "InnoDB: insufficient history for index %u", + keynr); /* The caller seems to ignore this. Thus, we must check this again in row_search_for_mysql(). */ DBUG_RETURN(convert_error_code_to_mysql(DB_MISSING_HISTORY, @@ -5850,7 +5957,7 @@ create_table_def( number fits in one byte in prtype */ push_warning_printf( (THD*) trx->mysql_thd, - MYSQL_ERROR::WARN_LEVEL_ERROR, + MYSQL_ERROR::WARN_LEVEL_WARN, ER_CANT_CREATE_TABLE, "In InnoDB, charset-collation codes" " must be below 256." @@ -5882,17 +5989,8 @@ create_table_def( /* First check whether the column to be added has a system reserved name. */ if (dict_col_name_is_reserved(field->field_name)){ - push_warning_printf( - (THD*) trx->mysql_thd, - MYSQL_ERROR::WARN_LEVEL_WARN, - ER_CANT_CREATE_TABLE, - "Error creating table '%s' with " - "column name '%s'. '%s' is a " - "reserved name. Please try to " - "re-create the table with a " - "different column name.", - table->name, (char*) field->field_name, - (char*) field->field_name); + my_error(ER_WRONG_COLUMN_NAME, MYF(0), + field->field_name); dict_mem_table_free(table); trx_commit_for_mysql(trx); @@ -5914,6 +6012,14 @@ create_table_def( error = row_create_table_for_mysql(table, trx); + if (error == DB_DUPLICATE_KEY) { + char buf[100]; + innobase_convert_identifier(buf, sizeof buf, + table_name, strlen(table_name), + trx->mysql_thd, TRUE); + my_error(ER_TABLE_EXISTS_ERROR, MYF(0), buf); + } + error_ret: error = convert_error_code_to_mysql(error, flags, NULL); @@ -6064,7 +6170,6 @@ create_clustered_index_when_no_primary( /* We pass 0 as the space id, and determine at a lower level the space id where to store the table */ - index = dict_mem_index_create(table_name, innobase_index_reserve_name, 0, DICT_CLUSTERED, 0); @@ -6119,7 +6224,7 @@ create_options_are_valid( /* Valid value. */ break; default: - push_warning_printf(thd, MYSQL_ERROR::WARN_LEVEL_ERROR, + push_warning_printf(thd, MYSQL_ERROR::WARN_LEVEL_WARN, ER_ILLEGAL_HA_CREATE_OPTION, "InnoDB: invalid" " KEY_BLOCK_SIZE = %lu." @@ -6133,7 +6238,7 @@ create_options_are_valid( /* If KEY_BLOCK_SIZE was specified, check for its dependencies. */ if (kbs_specified && !srv_file_per_table) { - push_warning(thd, MYSQL_ERROR::WARN_LEVEL_ERROR, + push_warning(thd, MYSQL_ERROR::WARN_LEVEL_WARN, ER_ILLEGAL_HA_CREATE_OPTION, "InnoDB: KEY_BLOCK_SIZE" " requires innodb_file_per_table."); @@ -6141,7 +6246,7 @@ create_options_are_valid( } if (kbs_specified && srv_file_format < DICT_TF_FORMAT_ZIP) { - push_warning(thd, MYSQL_ERROR::WARN_LEVEL_ERROR, + push_warning(thd, MYSQL_ERROR::WARN_LEVEL_WARN, ER_ILLEGAL_HA_CREATE_OPTION, "InnoDB: KEY_BLOCK_SIZE" " requires innodb_file_format >" @@ -6165,7 +6270,7 @@ create_options_are_valid( if (!srv_file_per_table) { push_warning_printf( thd, - MYSQL_ERROR::WARN_LEVEL_ERROR, + MYSQL_ERROR::WARN_LEVEL_WARN, ER_ILLEGAL_HA_CREATE_OPTION, "InnoDB: ROW_FORMAT=%s" " requires innodb_file_per_table.", @@ -6177,7 +6282,7 @@ create_options_are_valid( if (srv_file_format < DICT_TF_FORMAT_ZIP) { push_warning_printf( thd, - MYSQL_ERROR::WARN_LEVEL_ERROR, + MYSQL_ERROR::WARN_LEVEL_WARN, ER_ILLEGAL_HA_CREATE_OPTION, "InnoDB: ROW_FORMAT=%s" " requires innodb_file_format >" @@ -6194,7 +6299,7 @@ create_options_are_valid( && form->s->row_type == ROW_TYPE_DYNAMIC) { push_warning_printf( thd, - MYSQL_ERROR::WARN_LEVEL_ERROR, + MYSQL_ERROR::WARN_LEVEL_WARN, ER_ILLEGAL_HA_CREATE_OPTION, "InnoDB: cannot specify" " ROW_FORMAT = DYNAMIC with" @@ -6218,7 +6323,7 @@ create_options_are_valid( if (kbs_specified) { push_warning_printf( thd, - MYSQL_ERROR::WARN_LEVEL_ERROR, + MYSQL_ERROR::WARN_LEVEL_WARN, ER_ILLEGAL_HA_CREATE_OPTION, "InnoDB: cannot specify" " ROW_FORMAT = %s with" @@ -6231,7 +6336,7 @@ create_options_are_valid( default: push_warning(thd, - MYSQL_ERROR::WARN_LEVEL_ERROR, + MYSQL_ERROR::WARN_LEVEL_WARN, ER_ILLEGAL_HA_CREATE_OPTION, "InnoDB: invalid ROW_FORMAT specifier."); ret = FALSE; @@ -6295,13 +6400,15 @@ ha_innobase::create( 1. <database_name>/<table_name>: for normal table creation 2. full path: for temp table creation, or sym link - When srv_file_per_table is on, check for full path pattern, i.e. + When srv_file_per_table is on and mysqld_embedded is off, + check for full path pattern, i.e. X:\dir\..., X is a driver letter, or \\dir1\dir2\..., UNC path returns error if it is in full path format, but not creating a temp. table. Currently InnoDB does not support symbolic link on Windows. */ if (srv_file_per_table + && !mysqld_embedded && (!create_info->options & HA_LEX_CREATE_TMP_TABLE)) { if ((name[1] == ':') @@ -6519,6 +6626,7 @@ ha_innobase::create( goto cleanup; } + /* Create the keys */ if (form->s->keys == 0 || primary_key_no == -1) { @@ -6967,6 +7075,24 @@ ha_innobase::rename_table( innobase_commit_low(trx); trx_free_for_mysql(trx); + /* Add a special case to handle the Duplicated Key error + and return DB_ERROR instead. + This is to avoid a possible SIGSEGV error from mysql error + handling code. Currently, mysql handles the Duplicated Key + error by re-entering the storage layer and getting dup key + info by calling get_dup_key(). This operation requires a valid + table handle ('row_prebuilt_t' structure) which could no + longer be available in the error handling stage. The suggested + solution is to report a 'table exists' error message (since + the dup key error here is due to an existing table whose name + is the one we are trying to rename to) and return the generic + error code. */ + if (error == (int) DB_DUPLICATE_KEY) { + my_error(ER_TABLE_EXISTS_ERROR, MYF(0), to); + + error = DB_ERROR; + } + error = convert_error_code_to_mysql(error, 0, NULL); DBUG_RETURN(error); @@ -7519,11 +7645,15 @@ ha_innobase::check( ret = row_check_table_for_mysql(prebuilt); - if (ret == DB_SUCCESS) { + switch (ret) { + case DB_SUCCESS: return(HA_ADMIN_OK); + case DB_INTERRUPTED: + my_error(ER_QUERY_INTERRUPTED, MYF(0)); + return(-1); + default: + return(HA_ADMIN_CORRUPT); } - - return(HA_ADMIN_CORRUPT); } /*************************************************************//** @@ -8069,8 +8199,11 @@ ha_innobase::external_lock( ulong const binlog_format= thd_binlog_format(thd); ulong const tx_isolation = thd_tx_isolation(ha_thd()); if (tx_isolation <= ISO_READ_COMMITTED - && binlog_format == BINLOG_FORMAT_STMT - && thd_binlog_filter_ok(thd)) + && binlog_format == BINLOG_FORMAT_STMT +#if MYSQL_VERSION_ID > 50140 + && thd_binlog_filter_ok(thd) +#endif /* MYSQL_VERSION_ID > 50140 */ + ) { char buf[256]; my_snprintf(buf, sizeof(buf), @@ -8183,6 +8316,23 @@ ha_innobase::external_lock( statement has ended */ if (trx->n_mysql_tables_in_use == 0) { +#ifdef EXTENDED_SLOWLOG + increment_thd_innodb_stats(thd, trx->io_reads, + trx->io_read, + trx->io_reads_wait_timer, + trx->lock_que_wait_timer, + trx->innodb_que_wait_timer, + trx->distinct_page_access); + + trx->io_reads = 0; + trx->io_read = 0; + trx->io_reads_wait_timer = 0; + trx->lock_que_wait_timer = 0; + trx->innodb_que_wait_timer = 0; + trx->distinct_page_access = 0; + if (trx->distinct_page_access_hash) + memset(trx->distinct_page_access_hash, 0, DPAH_SIZE); +#endif trx->mysql_n_tables_locked = 0; prebuilt->used_in_HANDLER = FALSE; @@ -8470,8 +8620,8 @@ innodb_mutex_show_status( rw_lock_wait_time += mutex->lspent_time; } #else /* UNIV_DEBUG */ - buf1len= (uint) my_snprintf(buf1, sizeof(buf1), "%s:%lu", - mutex->cfile_name, (ulong) mutex->cline); + buf1len= (uint) my_snprintf(buf1, sizeof(buf1), "%s", + mutex->cmutex_name); buf2len= (uint) my_snprintf(buf2, sizeof(buf2), "os_waits=%lu", mutex->count_os_wait); @@ -8496,8 +8646,8 @@ next_mutex: while (lock != NULL) { if (lock->count_os_wait && !buf_pool_is_block_lock(lock)) { - buf1len= my_snprintf(buf1, sizeof(buf1), "%s:%lu", - lock->cfile_name, (ulong) lock->cline); + buf1len= my_snprintf(buf1, sizeof(buf1), "%s", + lock->lock_name); buf2len= my_snprintf(buf2, sizeof(buf2), "os_waits=%lu", lock->count_os_wait); @@ -8718,6 +8868,7 @@ ha_innobase::store_lock( && isolation_level != TRX_ISO_SERIALIZABLE && (lock_type == TL_READ || lock_type == TL_READ_NO_INSERT) && (sql_command == SQLCOM_INSERT_SELECT + || sql_command == SQLCOM_REPLACE_SELECT || sql_command == SQLCOM_UPDATE || sql_command == SQLCOM_CREATE_TABLE)) { @@ -8725,10 +8876,11 @@ ha_innobase::store_lock( option set or this session is using READ COMMITTED isolation level and isolation level of the transaction is not set to serializable and MySQL is doing - INSERT INTO...SELECT or UPDATE ... = (SELECT ...) or - CREATE ... SELECT... without FOR UPDATE or - IN SHARE MODE in select, then we use consistent - read for select. */ + INSERT INTO...SELECT or REPLACE INTO...SELECT + or UPDATE ... = (SELECT ...) or CREATE ... + SELECT... without FOR UPDATE or IN SHARE + MODE in select, then we use consistent read + for select. */ prebuilt->select_lock_type = LOCK_NONE; prebuilt->stored_select_lock_type = LOCK_NONE; @@ -8957,8 +9109,7 @@ ha_innobase::get_auto_increment( col_max_value = innobase_get_int_col_max_value( table->next_number_field); - current = *first_value > col_max_value ? autoinc : *first_value; - + current = *first_value > col_max_value ? autoinc : *first_value; need = *nb_reserved_values * increment; /* Compute the last value in the interval */ @@ -9322,8 +9473,7 @@ innobase_xa_prepare( executing XA PREPARE and XA COMMIT commands. In this case we cannot know how many minutes or hours will be between XA PREPARE and XA COMMIT, and we don't want - to block for undefined period of time. - */ + to block for undefined period of time. */ pthread_mutex_lock(&prepare_commit_mutex); trx->active_trans = 2; } @@ -9575,6 +9725,20 @@ ha_innobase::check_if_incompatible_data( DBUG_RETURN(COMPATIBLE_DATA_NO); } + /* Renaming column asynchronizes dictionary between mysqld and InnoDB... + If not synchronized, treat as COMPATIBLE_DATA_NO + until the bug http://bugs.mysql.com/47621 is fixed officialily */ + { + uint i; + for (i = 0; i < table->s->fields; i++) { + if (table->field[i]->flags & FIELD_IN_ADD_INDEX + && innobase_strcasecmp(table->field[i]->field_name, + dict_table_get_col_name(prebuilt->table, i))) { + DBUG_RETURN(COMPATIBLE_DATA_NO); + } + } + } + /* Check if a column participating in a foreign key is being renamed. There is no mechanism for updating InnoDB foreign key definitions. */ if (foreign_key_column_is_being_renamed(prebuilt, table)) { @@ -9683,25 +9847,24 @@ innobase_file_format_check_on_off( /************************************************************//** Validate the file format check config parameters, as a side effect it sets the srv_check_file_format_at_startup variable. -@return true if valid config value */ +@return the format_id if valid config value, otherwise, return -1 */ static -bool -innobase_file_format_check_validate( +int +innobase_file_format_validate_and_set( /*================================*/ const char* format_check) /*!< in: parameter value */ { uint format_id; - bool ret = true; format_id = innobase_file_format_name_lookup(format_check); if (format_id < DICT_TF_FORMAT_MAX + 1) { srv_check_file_format_at_startup = format_id; + + return((int) format_id); } else { - ret = false; + return(-1); } - - return(ret); } /*************************************************************//** @@ -9720,7 +9883,6 @@ innodb_file_format_name_validate( struct st_mysql_value* value) /*!< in: incoming string */ { const char* file_format_input; - char* file_format_input_strdup; char buff[STRING_BUFFER_USUAL_SIZE]; int len = sizeof(buff); @@ -9737,18 +9899,12 @@ innodb_file_format_name_validate( if (format_id <= DICT_TF_FORMAT_MAX) { - /* Copy out from stack-allocated memory (which will not - survive return from this function). The memory will be - freed in innodb_file_format_check_update(). */ - file_format_input_strdup = thd_strmake(thd, file_format_input, len); + /* Save a pointer to the name in the + 'file_format_name_map' constant array. */ + *static_cast<const char**>(save) = + trx_sys_file_format_id_to_name(format_id); - *static_cast<char**>(save) = file_format_input_strdup; - - if (file_format_input_strdup == NULL) { - return(1); - } else { - return(0); - } + return(0); } } @@ -9808,9 +9964,9 @@ innodb_file_format_check_validate( struct st_mysql_value* value) /*!< in: incoming string */ { const char* file_format_input; - char* file_format_input_strdup; char buff[STRING_BUFFER_USUAL_SIZE]; int len = sizeof(buff); + int format_id; ut_a(save != NULL); ut_a(value != NULL); @@ -9823,33 +9979,35 @@ innodb_file_format_check_validate( message if they did so. */ if (innobase_file_format_check_on_off(file_format_input)) { - sql_print_warning( + push_warning_printf(thd, + MYSQL_ERROR::WARN_LEVEL_WARN, + ER_WRONG_ARGUMENTS, "InnoDB: invalid innodb_file_format_check " "value; on/off can only be set at startup or " "in the configuration file"); - } else if (innobase_file_format_check_validate( - file_format_input)) { + } else { + format_id = innobase_file_format_validate_and_set( + file_format_input); - /* Copy out from stack-allocated memory (which will not - survive return from this function). The memory will be - freed in innodb_file_format_check_update(). */ - file_format_input_strdup = thd_strmake(thd, file_format_input, len); + if (format_id >= 0) { + /* Save a pointer to the name in the + 'file_format_name_map' constant array. */ + *static_cast<const char**>(save) = + trx_sys_file_format_id_to_name( + (uint)format_id); - *static_cast<char**>(save) = file_format_input_strdup; + return(0); - if (file_format_input_strdup == NULL) { - return(1); } else { - return(0); + push_warning_printf(thd, + MYSQL_ERROR::WARN_LEVEL_WARN, + ER_WRONG_ARGUMENTS, + "InnoDB: invalid innodb_file_format_check " + "value; can be any format up to %s " + "or its equivalent numeric id", + trx_sys_file_format_id_to_name( + DICT_TF_FORMAT_MAX)); } - - } else { - sql_print_warning( - "InnoDB: invalid innodb_file_format_check " - "value; can be any format up to %s " - "or its equivalent numeric id", - trx_sys_file_format_id_to_name( - DICT_TF_FORMAT_MAX)); } } @@ -9880,6 +10038,7 @@ innodb_file_format_check_update( ut_a(var_ptr != NULL); format_name_in = *static_cast<const char*const*>(save); + if (!format_name_in) { return; @@ -9929,6 +10088,25 @@ innodb_adaptive_hash_index_update( } } +/****************************************************************//** +Update the system variable innodb_old_blocks_pct using the "saved" +value. This function is registered as a callback with MySQL. */ +static +void +innodb_old_blocks_pct_update( +/*=========================*/ + THD* thd, /*!< in: thread handle */ + struct st_mysql_sys_var* var, /*!< in: pointer to + system variable */ + void* var_ptr,/*!< out: where the + formal string goes */ + const void* save) /*!< in: immediate result + from check function */ +{ + innobase_old_blocks_pct = buf_LRU_old_ratio_update( + *static_cast<const uint*>(save), TRUE); +} + /*************************************************************//** Check if it is a valid value of innodb_change_buffering. This function is registered as a callback with MySQL. @@ -10075,13 +10253,18 @@ static MYSQL_SYSVAR_BOOL(extra_undoslots static MYSQL_SYSVAR_BOOL(fast_recovery, innobase_fast_recovery, PLUGIN_VAR_NOCMDARG | PLUGIN_VAR_READONLY, "Enable to use speed hack of recovery avoiding flush list sorting.", - NULL, NULL, FALSE); + NULL, NULL, TRUE); -static MYSQL_SYSVAR_BOOL(use_purge_thread, innobase_use_purge_thread, +static MYSQL_SYSVAR_BOOL(recovery_stats, innobase_recovery_stats, PLUGIN_VAR_NOCMDARG | PLUGIN_VAR_READONLY, - "Enable to use purge devoted thread.", + "Output statistics of recovery process after it.", NULL, NULL, FALSE); +static MYSQL_SYSVAR_ULONG(use_purge_thread, srv_use_purge_thread, + PLUGIN_VAR_RQCMDARG | PLUGIN_VAR_READONLY, + "Number of purge devoted threads. #### over 1 is EXPERIMENTAL ####", + NULL, NULL, 1, 0, 64, 0); + static MYSQL_SYSVAR_BOOL(overwrite_relay_log_info, innobase_overwrite_relay_log_info, PLUGIN_VAR_NOCMDARG | PLUGIN_VAR_READONLY, "During InnoDB crash recovery on slave overwrite relay-log.info " @@ -10122,12 +10305,15 @@ static MYSQL_SYSVAR_STR(file_format, inn innodb_file_format_name_validate, innodb_file_format_name_update, "Antelope"); +/* If a new file format is introduced, the file format +name needs to be updated accordingly. Please refer to +file_format_name_map[] defined in trx0sys.c for the next +file format name. */ static MYSQL_SYSVAR_STR(file_format_check, innobase_file_format_check, PLUGIN_VAR_OPCMDARG, "The highest file format in the tablespace.", innodb_file_format_check_validate, - innodb_file_format_check_update, - "on"); + innodb_file_format_check_update, "Barracuda"); static MYSQL_SYSVAR_ULONG(flush_log_at_trx_commit, srv_flush_log_at_trx_commit, PLUGIN_VAR_OPCMDARG, @@ -10136,7 +10322,7 @@ static MYSQL_SYSVAR_ULONG(flush_log_at_t " or 2 (write at commit, flush once per second).", NULL, NULL, 1, 0, 2, 0); -static MYSQL_SYSVAR_STR(flush_method, innobase_unix_file_flush_method, +static MYSQL_SYSVAR_STR(flush_method, innobase_file_flush_method, PLUGIN_VAR_RQCMDARG | PLUGIN_VAR_READONLY, "With which method to flush data.", NULL, NULL, NULL); @@ -10177,7 +10363,7 @@ static MYSQL_SYSVAR_ULONG(max_dirty_page static MYSQL_SYSVAR_BOOL(adaptive_flushing, srv_adaptive_flushing, PLUGIN_VAR_NOCMDARG, "Attempt flushing dirty pages to avoid IO bursts at checkpoints.", - NULL, NULL, TRUE); + NULL, NULL, FALSE); static MYSQL_SYSVAR_ULONG(max_purge_lag, srv_max_purge_lag, PLUGIN_VAR_RQCMDARG, @@ -10273,7 +10459,7 @@ static MYSQL_SYSVAR_ULONG(concurrency_ti NULL, NULL, 500L, 1L, ~0L, 0); static MYSQL_SYSVAR_LONG(file_io_threads, innobase_file_io_threads, - PLUGIN_VAR_RQCMDARG | PLUGIN_VAR_READONLY, + PLUGIN_VAR_RQCMDARG | PLUGIN_VAR_READONLY | PLUGIN_VAR_NOSYSVAR, "Number of file I/O threads in InnoDB.", NULL, NULL, 4, 4, 64, 0); @@ -10312,6 +10498,18 @@ static MYSQL_SYSVAR_LONG(mirrored_log_gr "Number of identical copies of log groups we keep for the database. Currently this should be set to 1.", NULL, NULL, 1, 1, 10, 0); +static MYSQL_SYSVAR_UINT(old_blocks_pct, innobase_old_blocks_pct, + PLUGIN_VAR_RQCMDARG, + "Percentage of the buffer pool to reserve for 'old' blocks.", + NULL, innodb_old_blocks_pct_update, 100 * 3 / 8, 5, 95, 0); + +static MYSQL_SYSVAR_UINT(old_blocks_time, buf_LRU_old_threshold_ms, + PLUGIN_VAR_RQCMDARG, + "Move blocks to the 'new' end of the buffer pool if the first access" + " was at least this many milliseconds ago." + " The timeout is disabled if 0 (the default).", + NULL, NULL, 0, 0, UINT_MAX32, 0); + static MYSQL_SYSVAR_LONG(open_files, innobase_open_files, PLUGIN_VAR_RQCMDARG | PLUGIN_VAR_READONLY, "How many files at the maximum InnoDB keeps open at the same time.", @@ -10390,13 +10588,18 @@ static MYSQL_SYSVAR_LONGLONG(ibuf_max_si static MYSQL_SYSVAR_ULONG(ibuf_active_contract, srv_ibuf_active_contract, PLUGIN_VAR_RQCMDARG, "Enable/Disable active_contract of insert buffer. 0:disable 1:enable", - NULL, NULL, 0, 0, 1, 0); + NULL, NULL, 1, 0, 1, 0); static MYSQL_SYSVAR_ULONG(ibuf_accel_rate, srv_ibuf_accel_rate, PLUGIN_VAR_RQCMDARG, "Tunes amount of insert buffer processing of background, in addition to innodb_io_capacity. (in percentage)", NULL, NULL, 100, 100, 999999999, 0); +static MYSQL_SYSVAR_ULONG(checkpoint_age_target, srv_checkpoint_age_target, + PLUGIN_VAR_RQCMDARG, + "Control soft limit of checkpoint age. (0 : not control)", + NULL, NULL, 0, 0, ~0UL, 0); + static MYSQL_SYSVAR_ULONG(flush_neighbor_pages, srv_flush_neighbor_pages, PLUGIN_VAR_RQCMDARG, "Enable/Disable flushing also neighbor pages. 0:disable 1:enable", @@ -10432,7 +10635,7 @@ TYPELIB read_ahead_typelib= }; static MYSQL_SYSVAR_ENUM(read_ahead, srv_read_ahead, PLUGIN_VAR_RQCMDARG, - "Control read ahead activity. (none, random, [linear], both)", + "Control read ahead activity (none, random, [linear], both). [from 1.0.5: random read ahead is ignored]", NULL, innodb_read_ahead_update, 2, &read_ahead_typelib); static @@ -10463,8 +10666,8 @@ TYPELIB adaptive_checkpoint_typelib= }; static MYSQL_SYSVAR_ENUM(adaptive_checkpoint, srv_adaptive_checkpoint, PLUGIN_VAR_RQCMDARG, - "Enable/Disable flushing along modified age. ([none], reflex, estimate)", - NULL, innodb_adaptive_checkpoint_update, 0, &adaptive_checkpoint_typelib); + "Enable/Disable flushing along modified age. (none, reflex, [estimate])", + NULL, innodb_adaptive_checkpoint_update, 2, &adaptive_checkpoint_typelib); static MYSQL_SYSVAR_ULONG(enable_unsafe_group_commit, srv_enable_unsafe_group_commit, PLUGIN_VAR_RQCMDARG, @@ -10486,6 +10689,11 @@ static MYSQL_SYSVAR_ULONG(dict_size_limi "Limit the allocated memory for dictionary cache. (0: unlimited)", NULL, NULL, 0, 0, LONG_MAX, 0); +static MYSQL_SYSVAR_ULONG(relax_table_creation, srv_relax_table_creation, + PLUGIN_VAR_RQCMDARG, + "Relax limitation of column size at table creation as builtin InnoDB.", + NULL, NULL, 0, 0, 1, 0); + static struct st_mysql_sys_var* innobase_system_variables[]= { MYSQL_SYSVAR(additional_mem_pool_size), MYSQL_SYSVAR(autoextend_increment), @@ -10498,6 +10706,7 @@ static struct st_mysql_sys_var* innobase MYSQL_SYSVAR(doublewrite), MYSQL_SYSVAR(extra_undoslots), MYSQL_SYSVAR(fast_recovery), + MYSQL_SYSVAR(recovery_stats), MYSQL_SYSVAR(fast_shutdown), MYSQL_SYSVAR(file_io_threads), MYSQL_SYSVAR(read_io_threads), @@ -10522,6 +10731,8 @@ static struct st_mysql_sys_var* innobase MYSQL_SYSVAR(adaptive_flushing), MYSQL_SYSVAR(max_purge_lag), MYSQL_SYSVAR(mirrored_log_groups), + MYSQL_SYSVAR(old_blocks_pct), + MYSQL_SYSVAR(old_blocks_time), MYSQL_SYSVAR(open_files), MYSQL_SYSVAR(overwrite_relay_log_info), MYSQL_SYSVAR(rollback_on_timeout), @@ -10548,6 +10759,7 @@ static struct st_mysql_sys_var* innobase MYSQL_SYSVAR(ibuf_max_size), MYSQL_SYSVAR(ibuf_active_contract), MYSQL_SYSVAR(ibuf_accel_rate), + MYSQL_SYSVAR(checkpoint_age_target), MYSQL_SYSVAR(flush_neighbor_pages), MYSQL_SYSVAR(read_ahead), MYSQL_SYSVAR(adaptive_checkpoint), @@ -10560,6 +10772,7 @@ static struct st_mysql_sys_var* innobase MYSQL_SYSVAR(read_ahead_threshold), MYSQL_SYSVAR(io_capacity), MYSQL_SYSVAR(use_purge_thread), + MYSQL_SYSVAR(relax_table_creation), NULL }; @@ -10591,6 +10804,7 @@ i_s_innodb_cmpmem, i_s_innodb_cmpmem_reset, i_s_innodb_table_stats, i_s_innodb_index_stats, +i_s_innodb_admin_command, i_s_innodb_patches mysql_declare_plugin_end; === modified file 'storage/xtradb/handler/ha_innodb.h' --- a/storage/xtradb/handler/ha_innodb.h 2009-12-03 11:34:11 +0000 +++ b/storage/xtradb/handler/ha_innodb.h 2010-01-15 15:58:25 +0000 @@ -258,12 +258,14 @@ int thd_binlog_format(const MYSQL_THD th */ void thd_mark_transaction_to_rollback(MYSQL_THD thd, bool all); +#if MYSQL_VERSION_ID > 50140 /** Check if binary logging is filtered for thread's current db. @param thd Thread handle @retval 1 the query is not filtered, 0 otherwise. */ bool thd_binlog_filter_ok(const MYSQL_THD thd); +#endif /* MYSQL_VERSION_ID > 50140 */ } typedef struct trx_struct trx_t; @@ -289,6 +291,8 @@ trx_t* innobase_trx_allocate( /*==================*/ MYSQL_THD thd); /*!< in: user thread handle */ + + /*********************************************************************//** This function checks each index name for a table against reserved system default primary index name 'GEN_CLUST_INDEX'. If a name === modified file 'storage/xtradb/handler/handler0alter.cc' --- a/storage/xtradb/handler/handler0alter.cc 2009-12-03 11:34:11 +0000 +++ b/storage/xtradb/handler/handler0alter.cc 2010-01-15 15:58:25 +0000 @@ -35,7 +35,6 @@ extern "C" { } #include "ha_innodb.h" -#include "handler0vars.h" /*************************************************************//** Copies an InnoDB column to a MySQL field. This function is @@ -629,7 +628,7 @@ ha_innobase::add_index( ulint num_created = 0; ibool dict_locked = FALSE; ulint new_primary; - ulint error; + int error; DBUG_ENTER("ha_innobase::add_index"); ut_a(table); @@ -668,7 +667,7 @@ ha_innobase::add_index( if (UNIV_UNLIKELY(error)) { err_exit: mem_heap_free(heap); - trx_general_rollback_for_mysql(trx, FALSE, NULL); + trx_general_rollback_for_mysql(trx, NULL); trx_free_for_mysql(trx); trx_commit_for_mysql(prebuilt->trx); DBUG_RETURN(error); @@ -766,10 +765,11 @@ err_exit: ut_ad(error == DB_SUCCESS); /* Commit the data dictionary transaction in order to release - the table locks on the system tables. Unfortunately, this - means that if MySQL crashes while creating a new primary key - inside row_merge_build_indexes(), indexed_table will not be - dropped on crash recovery. Thus, it will become orphaned. */ + the table locks on the system tables. This means that if + MySQL crashes while creating a new primary key inside + row_merge_build_indexes(), indexed_table will not be dropped + by trx_rollback_active(). It will have to be recovered or + dropped by the database administrator. */ trx_commit_for_mysql(trx); row_mysql_unlock_data_dictionary(trx); @@ -806,7 +806,7 @@ error_handling: alter table t drop index b, add index (b); The fix will have to parse the SQL and note that the index - being added has the same name as the the one being dropped and + being added has the same name as the one being dropped and ignore that in the dup index check.*/ //dict_table_check_for_dup_indexes(prebuilt->table); #endif @@ -868,6 +868,7 @@ error_handling: indexed_table->n_mysql_handles_opened++; error = row_merge_drop_table(trx, innodb_table); + innodb_table = indexed_table; goto convert_error; case DB_TOO_BIG_RECORD: @@ -882,7 +883,9 @@ error: /* fall through */ default: if (new_primary) { - row_merge_drop_table(trx, indexed_table); + if (indexed_table != innodb_table) { + row_merge_drop_table(trx, indexed_table); + } } else { if (!dict_locked) { row_mysql_lock_data_dictionary(trx); === removed file 'storage/xtradb/handler/handler0vars.h' --- a/storage/xtradb/handler/handler0vars.h 2009-11-13 21:26:08 +0000 +++ b/storage/xtradb/handler/handler0vars.h 1970-01-01 00:00:00 +0000 @@ -1,73 +0,0 @@ -/***************************************************************************** - -Copyright (c) 2008, 2009, Innobase Oy. All Rights Reserved. - -This program is free software; you can redistribute it and/or modify it under -the terms of the GNU General Public License as published by the Free Software -Foundation; version 2 of the License. - -This program is distributed in the hope that it will be useful, but WITHOUT -ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS -FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. - -You should have received a copy of the GNU General Public License along with -this program; if not, write to the Free Software Foundation, Inc., 59 Temple -Place, Suite 330, Boston, MA 02111-1307 USA - -*****************************************************************************/ - -/*******************************************************************//** -@file handler/handler0vars.h -This file contains accessor functions for dynamic plugin on Windows. -***********************************************************************/ - -#if defined __WIN__ && defined MYSQL_DYNAMIC_PLUGIN -/*******************************************************************//** -This is a list of externals that can not be resolved by delay loading. -They have to be resolved indirectly via their addresses in the .map file. -All of them are external variables. */ -extern MYSQL_PLUGIN_IMPORT CHARSET_INFO my_charset_bin; -extern MYSQL_PLUGIN_IMPORT CHARSET_INFO my_charset_latin1; -extern MYSQL_PLUGIN_IMPORT CHARSET_INFO my_charset_filename; -extern MYSQL_PLUGIN_IMPORT CHARSET_INFO* system_charset_info; -extern MYSQL_PLUGIN_IMPORT CHARSET_INFO* default_charset_info; -//extern MYSQL_PLUGIN_IMPORT CHARSET_INFO** all_charsets; -extern MYSQL_PLUGIN_IMPORT system_variables global_system_variables; -//extern MYSQL_PLUGIN_IMPORT char* mysql_real_data_home; -extern MYSQL_PLUGIN_IMPORT char* mysql_data_home; -//extern MYSQL_PLUGIN_IMPORT char** tx_isolation_names; -//extern MYSQL_PLUGIN_IMPORT char** binlog_format_names; -//extern MYSQL_PLUGIN_IMPORT char reg_ext; -extern MYSQL_PLUGIN_IMPORT pthread_mutex_t LOCK_thread_count; -extern MYSQL_PLUGIN_IMPORT key_map key_map_full; -extern MYSQL_PLUGIN_IMPORT MY_TMPDIR mysql_tmpdir_list; -extern MYSQL_PLUGIN_IMPORT bool mysqld_embedded; -extern MYSQL_PLUGIN_IMPORT uint lower_case_table_names; -extern MYSQL_PLUGIN_IMPORT ulong specialflag; -extern MYSQL_PLUGIN_IMPORT int my_umask; - -extern MYSQL_PLUGIN_IMPORT char *relay_log_info_file; - -/* -#define my_charset_bin (*wdl_my_charset_bin) -#define my_charset_latin1 (*wdl_my_charset_latin1) -#define my_charset_filename (*wdl_my_charset_filename) -#define system_charset_info (*wdl_system_charset_info) -#define default_charset_info (*wdl_default_charset_info) -#define all_charsets (wdl_all_charsets) -#define global_system_variables (*wdl_global_system_variables) -#define mysql_real_data_home (wdl_mysql_real_data_home) -#define mysql_data_home (*wdl_mysql_data_home) -#define tx_isolation_names (wdl_tx_isolation_names) -#define binlog_format_names (wdl_binlog_format_names) -#define reg_ext (wdl_reg_ext) -#define LOCK_thread_count (*wdl_LOCK_thread_count) -#define key_map_full (*wdl_key_map_full) -#define mysql_tmpdir_list (*wdl_mysql_tmpdir_list) -#define mysqld_embedded (*wdl_mysqld_embedded) -*/ -//#define lower_case_table_names (*wdl_lower_case_table_names) -//#define specialflag (*wdl_specialflag) -//#define my_umask (*wdl_my_umask) - -#endif === modified file 'storage/xtradb/handler/i_s.cc' --- a/storage/xtradb/handler/i_s.cc 2009-11-13 21:26:08 +0000 +++ b/storage/xtradb/handler/i_s.cc 2010-01-28 11:35:10 +0000 @@ -47,6 +47,7 @@ extern "C" { #include "trx0rseg.h" /* for trx_rseg_struct */ #include "trx0sys.h" /* for trx_sys */ #include "dict0dict.h" /* for dict_sys */ +#include "buf0lru.h" /* for XTRA_LRU_[DUMP/RESTORE] */ /* from buf0buf.c */ struct buf_chunk_struct{ ulint mem_size; /* allocated size of the chunk */ @@ -56,7 +57,6 @@ struct buf_chunk_struct{ buf_block_t* blocks; /* array of buffer control blocks */ }; } -#include "handler0vars.h" static const char plugin_author[] = "Innobase Oy"; @@ -84,14 +84,16 @@ do { \ #define STRUCT_FLD(name, value) value #endif -static const ST_FIELD_INFO END_OF_ST_FIELD_INFO = - {STRUCT_FLD(field_name, NULL), - STRUCT_FLD(field_length, 0), - STRUCT_FLD(field_type, MYSQL_TYPE_NULL), - STRUCT_FLD(value, 0), - STRUCT_FLD(field_flags, 0), - STRUCT_FLD(old_name, ""), - STRUCT_FLD(open_method, SKIP_OPEN_TABLE)}; +/* Don't use a static const variable here, as some C++ compilers (notably +HPUX aCC: HP ANSI C++ B3910B A.03.65) can't handle it. */ +#define END_OF_ST_FIELD_INFO \ + {STRUCT_FLD(field_name, NULL), \ + STRUCT_FLD(field_length, 0), \ + STRUCT_FLD(field_type, MYSQL_TYPE_NULL), \ + STRUCT_FLD(value, 0), \ + STRUCT_FLD(field_flags, 0), \ + STRUCT_FLD(old_name, ""), \ + STRUCT_FLD(open_method, SKIP_OPEN_TABLE)} /* Use the following types mapping: @@ -511,7 +513,7 @@ static ST_FIELD_INFO i_s_innodb_buffer_p STRUCT_FLD(old_name, ""), STRUCT_FLD(open_method, SKIP_OPEN_TABLE)}, - {STRUCT_FLD(field_name, "accessed"), + {STRUCT_FLD(field_name, "access_time"), STRUCT_FLD(field_length, MY_INT64_NUM_DECIMAL_DIGITS), STRUCT_FLD(field_type, MYSQL_TYPE_LONGLONG), STRUCT_FLD(value, 0), @@ -728,7 +730,7 @@ i_s_innodb_buffer_pool_pages_fill( field_store_string(table->field[0], page_type); table->field[1]->store(block->page.space); table->field[2]->store(block->page.offset); - table->field[3]->store(block->page.LRU_position); + table->field[3]->store(0); table->field[4]->store(block->page.buf_fix_count); table->field[5]->store(block->page.flush_type); @@ -817,11 +819,11 @@ i_s_innodb_buffer_pool_pages_index_fill( table->field[5]->store(page_get_n_recs(frame)); table->field[6]->store(page_get_data_size(frame)); table->field[7]->store(block->is_hashed); - table->field[8]->store(block->page.accessed); + table->field[8]->store(block->page.access_time); table->field[9]->store(block->page.newest_modification != 0); table->field[10]->store(block->page.oldest_modification != 0); table->field[11]->store(block->page.old); - table->field[12]->store(block->page.LRU_position); + table->field[12]->store(0); table->field[13]->store(block->page.buf_fix_count); table->field[14]->store(block->page.flush_type); @@ -915,7 +917,7 @@ i_s_innodb_buffer_pool_pages_blob_fill( table->field[4]->store(block->page.offset); } - table->field[5]->store(block->page.LRU_position); + table->field[5]->store(0); table->field[6]->store(block->page.buf_fix_count); table->field[7]->store(block->page.flush_type); @@ -2953,3 +2955,170 @@ UNIV_INTERN struct st_mysql_plugin i_s_i STRUCT_FLD(system_vars, NULL), STRUCT_FLD(__reserved1, NULL) }; + +/*********************************************************************** +*/ +static ST_FIELD_INFO i_s_innodb_admin_command_info[] = +{ + {STRUCT_FLD(field_name, "result_message"), + STRUCT_FLD(field_length, 1024), + STRUCT_FLD(field_type, MYSQL_TYPE_STRING), + STRUCT_FLD(value, 0), + STRUCT_FLD(field_flags, 0), + STRUCT_FLD(old_name, ""), + STRUCT_FLD(open_method, SKIP_OPEN_TABLE)}, + + END_OF_ST_FIELD_INFO +}; + +#ifndef INNODB_COMPATIBILITY_HOOKS +#error InnoDB needs MySQL to be built with #define INNODB_COMPATIBILITY_HOOKS +#endif + +extern "C" { +char **thd_query(MYSQL_THD thd); +} + +static +int +i_s_innodb_admin_command_fill( +/*==========================*/ + THD* thd, + TABLE_LIST* tables, + COND* cond) +{ + TABLE* i_s_table = (TABLE *) tables->table; + CHARSET_INFO *cs= system_charset_info; + char** query_str; + char* ptr; + char quote = '\0'; + const char* command_head = "XTRA_"; + + DBUG_ENTER("i_s_innodb_admin_command_fill"); + + /* deny access to non-superusers */ + if (check_global_access(thd, PROCESS_ACL)) { + DBUG_RETURN(0); + } + + if(thd_sql_command(thd) != SQLCOM_SELECT) { + field_store_string(i_s_table->field[0], + "SELECT command is only accepted."); + goto end_func; + } + + query_str = thd_query(thd); + ptr = *query_str; + + for (; *ptr; ptr++) { + if (*ptr == quote) { + quote = '\0'; + } else if (quote) { + } else if (*ptr == '`' || *ptr == '"') { + quote = *ptr; + } else { + long i; + for (i = 0; command_head[i]; i++) { + if (toupper((int)(unsigned char)(ptr[i])) + != toupper((int)(unsigned char) + (command_head[i]))) { + goto nomatch; + } + } + break; +nomatch: + ; + } + } + + if (!*ptr) { + field_store_string(i_s_table->field[0], + "No XTRA_* command in the SQL statement." + " Please add /*!XTRA_xxxx*/ to the SQL."); + goto end_func; + } + + if (!strncasecmp("XTRA_HELLO", ptr, 10)) { + /* This is example command XTRA_HELLO */ + + ut_print_timestamp(stderr); + fprintf(stderr, " InnoDB: administration command test for XtraDB" + " 'XTRA_HELLO' was detected.\n"); + + field_store_string(i_s_table->field[0], + "Hello!"); + goto end_func; + } + else if (!strncasecmp("XTRA_LRU_DUMP", ptr, 13)) { + ut_print_timestamp(stderr); + fprintf(stderr, " InnoDB: administration command 'XTRA_LRU_DUMP'" + " was detected.\n"); + + if (buf_LRU_file_dump()) { + field_store_string(i_s_table->field[0], + "XTRA_LRU_DUMP was succeeded."); + } else { + field_store_string(i_s_table->field[0], + "XTRA_LRU_DUMP was failed."); + } + + goto end_func; + } + else if (!strncasecmp("XTRA_LRU_RESTORE", ptr, 16)) { + ut_print_timestamp(stderr); + fprintf(stderr, " InnoDB: administration command 'XTRA_LRU_RESTORE'" + " was detected.\n"); + + if (buf_LRU_file_restore()) { + field_store_string(i_s_table->field[0], + "XTRA_LRU_RESTORE was succeeded."); + } else { + field_store_string(i_s_table->field[0], + "XTRA_LRU_RESTORE was failed."); + } + + goto end_func; + } + + field_store_string(i_s_table->field[0], + "Undefined XTRA_* command."); + goto end_func; + +end_func: + if (schema_table_store_record(thd, i_s_table)) { + DBUG_RETURN(1); + } else { + DBUG_RETURN(0); + } +} + +static +int +i_s_innodb_admin_command_init( +/*==========================*/ + void* p) +{ + DBUG_ENTER("i_s_innodb_admin_command_init"); + ST_SCHEMA_TABLE* schema = (ST_SCHEMA_TABLE*) p; + + schema->fields_info = i_s_innodb_admin_command_info; + schema->fill_table = i_s_innodb_admin_command_fill; + + DBUG_RETURN(0); +} + +UNIV_INTERN struct st_mysql_plugin i_s_innodb_admin_command = +{ + STRUCT_FLD(type, MYSQL_INFORMATION_SCHEMA_PLUGIN), + STRUCT_FLD(info, &i_s_info), + STRUCT_FLD(name, "XTRADB_ADMIN_COMMAND"), + STRUCT_FLD(author, plugin_author), + STRUCT_FLD(descr, "XtraDB specific command acceptor"), + STRUCT_FLD(license, PLUGIN_LICENSE_GPL), + STRUCT_FLD(init, i_s_innodb_admin_command_init), + STRUCT_FLD(deinit, i_s_common_deinit), + STRUCT_FLD(version, 0x0100 /* 1.0 */), + STRUCT_FLD(status_vars, NULL), + STRUCT_FLD(system_vars, NULL), + STRUCT_FLD(__reserved1, NULL) +}; === modified file 'storage/xtradb/handler/i_s.h' --- a/storage/xtradb/handler/i_s.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/handler/i_s.h 2009-11-04 20:11:12 +0000 @@ -40,5 +40,6 @@ extern struct st_mysql_plugin i_s_innodb extern struct st_mysql_plugin i_s_innodb_rseg; extern struct st_mysql_plugin i_s_innodb_table_stats; extern struct st_mysql_plugin i_s_innodb_index_stats; +extern struct st_mysql_plugin i_s_innodb_admin_command; #endif /* i_s_h */ === modified file 'storage/xtradb/handler/innodb_patch_info.h' --- a/storage/xtradb/handler/innodb_patch_info.h 2009-11-13 21:26:08 +0000 +++ b/storage/xtradb/handler/innodb_patch_info.h 2010-01-15 15:58:25 +0000 @@ -38,5 +38,10 @@ struct innodb_enhancement { {"innodb_stats","Additional features about InnoDB statistics/optimizer","","http://www.percona.com/docs/wiki/percona-xtradb"}, {"innodb_recovery_patches","Bugfixes and adjustments about recovery process","","http://www.percona.com/docs/wiki/percona-xtradb"}, {"innodb_purge_thread","Enable to use purge devoted thread","","http://www.percona.com/docs/wiki/percona-xtradb"}, +{"innodb_admin_command_base","XtraDB specific command interface through i_s","","http://www.percona.com/docs/wiki/percona-xtradb"}, +{"innodb_show_lock_name","Show mutex/lock name instead of crated file/line","","http://www.percona.com/docs/wiki/percona-xtradb"}, +{"innodb_extend_slow","Extended statistics in slow.log","It is InnoDB-part only. It needs to patch also to mysqld.","http://www.percona.com/docs/wiki/percona-xtradb"}, +{"innodb_relax_table_creation","Relax limitation of column size at table creation as builtin InnoDB.","","http://www.percona.com/docs/wiki/percona-xtradb"}, +{"innodb_lru_dump_restore","Dump and restore command for content of buffer pool","","http://www.percona.com/docs/wiki/percona-xtradb"}, {NULL, NULL, NULL, NULL} }; === removed file 'storage/xtradb/handler/win_delay_loader.cc' --- a/storage/xtradb/handler/win_delay_loader.cc 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/handler/win_delay_loader.cc 1970-01-01 00:00:00 +0000 @@ -1,1024 +0,0 @@ -/***************************************************************************** - -Copyright (c) 2008, 2009, Innobase Oy. All Rights Reserved. - -This program is free software; you can redistribute it and/or modify it under -the terms of the GNU General Public License as published by the Free Software -Foundation; version 2 of the License. - -This program is distributed in the hope that it will be useful, but WITHOUT -ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS -FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. - -You should have received a copy of the GNU General Public License along with -this program; if not, write to the Free Software Foundation, Inc., 59 Temple -Place, Suite 330, Boston, MA 02111-1307 USA - -*****************************************************************************/ - -/*******************************************************************//** -@file handler/win_delay_loader.cc -This file contains functions that implement the delay loader on Windows. - -This is a customized version of delay loader with limited functionalities. -It does not support: - -* (manual) unloading -* multiple delay loaded DLLs -* multiple loading of the same DLL - -This delay loader is used only by the InnoDB plugin. Other components (DLLs) -can still use the default delay loader, provided by MSVC. - -Several acronyms used by Microsoft: - * IAT: import address table - * INT: import name table - * RVA: Relative Virtual Address - -See http://msdn.microsoft.com/en-us/magazine/bb985992.aspx for details of -PE format. -***********************************************************************/ -#if defined (__WIN__) && defined (MYSQL_DYNAMIC_PLUGIN) -# define WIN32_LEAN_AND_MEAN -# include <windows.h> -# include <delayimp.h> -# include <mysql_priv.h> - -extern "C" { -# include "univ.i" -# include "hash0hash.h" -} - -/*******************************************************************//** -This following contains a list of externals that can not be resolved by -delay loading. They have to be resolved indirectly via their addresses -in the .map file. All of them are external variables. */ -CHARSET_INFO* wdl_my_charset_bin; -CHARSET_INFO* wdl_my_charset_latin1; -CHARSET_INFO* wdl_my_charset_filename; -CHARSET_INFO** wdl_system_charset_info; -CHARSET_INFO** wdl_default_charset_info; -CHARSET_INFO** wdl_all_charsets; -system_variables* wdl_global_system_variables; -char* wdl_mysql_real_data_home; -char** wdl_mysql_data_home; -char** wdl_tx_isolation_names; -char** wdl_binlog_format_names; -char* wdl_reg_ext; -pthread_mutex_t* wdl_LOCK_thread_count; -key_map* wdl_key_map_full; -MY_TMPDIR* wdl_mysql_tmpdir_list; -bool* wdl_mysqld_embedded; -uint* wdl_lower_case_table_names; -ulong* wdl_specialflag; -int* wdl_my_umask; - -/*******************************************************************//** -The preferred load-address defined in PE (portable executable format). */ -#if defined(_M_IA64) -#pragma section(".base", long, read) -extern "C" -__declspec(allocate(".base")) -const IMAGE_DOS_HEADER __ImageBase; -#else -extern "C" -const IMAGE_DOS_HEADER __ImageBase; -#endif - -/*******************************************************************//** -A template function for converting a relative address (RVA) to an -absolute address (VA). This is due to the pointers in the delay -descriptor (ImgDelayDescr in delayimp.h) have been changed from -VAs to RVAs to work on both 32- and 64-bit platforms. -@return absolute virtual address */ -template <class X> -X PFromRva( -/*=======*/ - RVA rva) /*!< in: relative virtual address */ -{ - return X(PBYTE(&__ImageBase) + rva); -} - -/*******************************************************************//** -Convert to the old format for convenience. The structure as well as its -element names follow the definition of ImgDelayDescr in delayimp.h. */ -struct InternalImgDelayDescr -{ - DWORD grAttrs; /*!< attributes */ - LPCSTR szName; /*!< pointer to dll name */ - HMODULE* phmod; /*!< address of module handle */ - PImgThunkData pIAT; /*!< address of the IAT */ - PCImgThunkData pINT; /*!< address of the INT */ - PCImgThunkData pBoundIAT; /*!< address of the optional bound IAT */ - PCImgThunkData pUnloadIAT; /*!< address of optional copy of - original IAT */ - DWORD dwTimeStamp; /*!< 0 if not bound, - otherwise date/time stamp of DLL - bound to (Old BIND) */ -}; - -typedef struct map_hash_chain_struct map_hash_chain_t; - -struct map_hash_chain_struct { - char* symbol; /*!< pointer to a symbol */ - ulint value; /*!< address of the symbol */ - map_hash_chain_t* next; /*!< pointer to the next cell - in the same folder. */ - map_hash_chain_t* chain; /*!< a linear chain used for - cleanup. */ -}; - -static HMODULE my_hmod = 0; -static struct hash_table_struct* m_htbl = NULL ; -static map_hash_chain_t* chain_header = NULL; -static ibool wdl_init = FALSE; -const ulint MAP_HASH_CELLS_NUM = 10000; - -#ifndef DBUG_OFF -/*******************************************************************//** -In the dynamic plugin, it is required to call the following dbug functions -in the server: - _db_pargs_ - _db_doprnt_ - _db_enter_ - _db_return_ - _db_dump_ - -The plugin will get those function pointers during the initialization. */ -typedef void (__cdecl* pfn_db_enter_)( - const char* _func_, - const char* _file_, - uint _line_, - const char** _sfunc_, - const char** _sfile_, - uint* _slevel_, - char***); - -typedef void (__cdecl* pfn_db_return_)( - uint _line_, - const char** _sfunc_, - const char** _sfile_, - uint* _slevel_); - -typedef void (__cdecl* pfn_db_pargs_)( - uint _line_, - const char* keyword); - -typedef void (__cdecl* pfn_db_doprnt_)( - const char* format, - ...); - -typedef void (__cdecl* pfn_db_dump_)( - uint _line_, - const char* keyword, - const unsigned char* memory, - size_t length); - -static pfn_db_enter_ wdl_db_enter_; -static pfn_db_return_ wdl_db_return_; -static pfn_db_pargs_ wdl_db_pargs_; -static pfn_db_doprnt_ wdl_db_doprnt_; -static pfn_db_dump_ wdl_db_dump_; -#endif /* !DBUG_OFF */ - -/*************************************************************//** -Creates a hash table with >= n array cells. The actual number of cells is -chosen to be a prime number slightly bigger than n. - -This is the same function as hash_create in hash0hash.c, except the -memory allocation. This function is invoked before the engine is -initialized, and buffer pools are not ready yet. -@return own: created hash table */ -static -hash_table_t* -wdl_hash_create( -/*============*/ - ulint n) /*!< in: number of array cells */ -{ - hash_cell_t* array; - ulint prime; - hash_table_t* table; - - prime = ut_find_prime(n); - - table = (hash_table_t*) malloc(sizeof(hash_table_t)); - if (table == NULL) { - return(NULL); - } - - array = (hash_cell_t*) malloc(sizeof(hash_cell_t) * prime); - if (array == NULL) { - free(table); - return(NULL); - } - - table->array = array; - table->n_cells = prime; - table->n_mutexes = 0; - table->mutexes = NULL; - table->heaps = NULL; - table->heap = NULL; - table->magic_n = HASH_TABLE_MAGIC_N; - - /* Initialize the cell array */ - hash_table_clear(table); - - return(table); -} - -/*************************************************************//** -Frees a hash table. */ -static -void -wdl_hash_table_free( -/*================*/ - hash_table_t* table) /*!< in, own: hash table */ -{ - ut_a(table != NULL); - ut_a(table->mutexes == NULL); - - free(table->array); - free(table); -} - -/*******************************************************************//** -Function for calculating the count of imports given the base of the IAT. -@return number of imports */ -static -ulint -wdl_import_count( -/*=============*/ - PCImgThunkData pitd_base) /*!< in: base of the IAT */ -{ - ulint ret = 0; - PCImgThunkData pitd = pitd_base; - - while (pitd->u1.Function) { - pitd++; - ret++; - } - - return(ret); -} - -/*******************************************************************//** -Read Mapfile to a hashtable for faster access -@return TRUE if the mapfile is loaded successfully. */ -static -ibool -wdl_load_mapfile( -/*=============*/ - const char* filename) /*!< in: name of the mapfile. */ -{ - FILE* fp; - const size_t nSize = 256; - char tmp_buf[nSize]; - char* func_name; - char* func_addr; - ulint load_addr = 0; - ibool valid_load_addr = FALSE; -#ifdef _WIN64 - const char* tmp_string = " Preferred load address is %16llx"; -#else - const char* tmp_string = " Preferred load address is %08x"; -#endif - - fp = fopen(filename, "r"); - if (fp == NULL) { - - return(FALSE); - } - - /* Check whether to create the hashtable */ - if (m_htbl == NULL) { - - m_htbl = wdl_hash_create(MAP_HASH_CELLS_NUM); - - if (m_htbl == NULL) { - - fclose(fp); - return(FALSE); - } - } - - /* Search start of symbol list and get the preferred load address */ - while (fgets(tmp_buf, sizeof(tmp_buf), fp)) { - - if (sscanf(tmp_buf, tmp_string, &load_addr) == 1) { - - valid_load_addr = TRUE; - } - - if (strstr(tmp_buf, "Rva+Base") != NULL) { - - break; - } - } - - if (valid_load_addr == FALSE) { - - /* No "Preferred load address", the map file is wrong. */ - fclose(fp); - return(FALSE); - } - - /* Read symbol list */ - while (fgets(tmp_buf, sizeof(tmp_buf), fp)) - { - map_hash_chain_t* map_cell; - ulint map_fold; - - if (*tmp_buf == 0) { - - continue; - } - - func_name = strtok(tmp_buf, " "); - func_name = strtok(NULL, " "); - func_addr = strtok(NULL, " "); - - if (func_name && func_addr) { - - ut_snprintf(tmp_buf, nSize, "0x%s", func_addr); - if (*func_name == '_') { - - func_name++; - } - - map_cell = (map_hash_chain_t*) - malloc(sizeof(map_hash_chain_t)); - if (map_cell == NULL) { - return(FALSE); - } - - /* Chain all cells together */ - map_cell->chain = chain_header; - chain_header = map_cell; - - map_cell->symbol = strdup(func_name); - map_cell->value = (ulint) _strtoui64(tmp_buf, NULL, 0) - - load_addr; - map_fold = ut_fold_string(map_cell->symbol); - - HASH_INSERT(map_hash_chain_t, - next, - m_htbl, - map_fold, - map_cell); - } - } - - fclose(fp); - - return(TRUE); -} - -/*************************************************************//** -Cleanup.during DLL unload */ -static -void -wdl_cleanup(void) -/*=============*/ -{ - while (chain_header != NULL) { - map_hash_chain_t* tmp; - - tmp = chain_header->chain; - free(chain_header->symbol); - free(chain_header); - chain_header = tmp; - } - - if (m_htbl != NULL) { - - wdl_hash_table_free(m_htbl); - } -} - -/*******************************************************************//** -Load the mapfile mysqld.map. -@return the module handle */ -static -HMODULE -wdl_get_mysqld_mapfile(void) -/*========================*/ -{ - char file_name[MAX_PATH]; - char* ext; - ulint err; - - if (my_hmod == 0) { - - size_t nSize = MAX_PATH - strlen(".map") -1; - - /* First find out the name of current executable */ - my_hmod = GetModuleHandle(NULL); - if (my_hmod == 0) { - - return(my_hmod); - } - - err = GetModuleFileName(my_hmod, file_name, nSize); - if (err == 0) { - - my_hmod = 0; - return(my_hmod); - } - - ext = strrchr(file_name, '.'); - if (ext != NULL) { - - *ext = 0; - strcat(file_name, ".map"); - - err = wdl_load_mapfile(file_name); - if (err == 0) { - - my_hmod = 0; - } - } else { - - my_hmod = 0; - } - } - - return(my_hmod); -} - -/*******************************************************************//** -Retrieves the address of an exported function. It follows the convention -of GetProcAddress(). -@return address of exported function. */ -static -FARPROC -wdl_get_procaddr_from_map( -/*======================*/ - HANDLE m_handle, /*!< in: module handle */ - const char* import_proc) /*!< in: procedure name */ -{ - map_hash_chain_t* hash_chain; - ulint map_fold; - - map_fold = ut_fold_string(import_proc); - HASH_SEARCH( - next, - m_htbl, - map_fold, - map_hash_chain_t*, - hash_chain, - , - (ut_strcmp(hash_chain->symbol, import_proc) == 0)); - - if (hash_chain == NULL) { - -#ifdef _WIN64 - /* On Win64, the leading '_' may not be taken out. In this - case, search again without the leading '_'. */ - if (*import_proc == '_') { - - import_proc++; - } - - map_fold = ut_fold_string(import_proc); - HASH_SEARCH( - next, - m_htbl, - map_fold, - map_hash_chain_t*, - hash_chain, - , - (ut_strcmp(hash_chain->symbol, import_proc) == 0)); - - if (hash_chain == NULL) { -#endif - if (wdl_init == TRUE) { - - sql_print_error( - "InnoDB: the procedure pointer of %s" - " is not found.", - import_proc); - } - - return(0); -#ifdef _WIN64 - } -#endif - } - - return((FARPROC) ((ulint) m_handle + hash_chain->value)); -} - -/*******************************************************************//** -Retrieves the address of an exported variable. -Note: It does not follow the Windows call convention FARPROC. -@return address of exported variable. */ -static -void* -wdl_get_varaddr_from_map( -/*=====================*/ - HANDLE m_handle, /*!< in: module handle */ - const char* import_variable) /*!< in: variable name */ -{ - map_hash_chain_t* hash_chain; - ulint map_fold; - - map_fold = ut_fold_string(import_variable); - HASH_SEARCH( - next, - m_htbl, - map_fold, - map_hash_chain_t*, - hash_chain, - , - (ut_strcmp(hash_chain->symbol, import_variable) == 0)); - - if (hash_chain == NULL) { - -#ifdef _WIN64 - /* On Win64, the leading '_' may not be taken out. In this - case, search again without the leading '_'. */ - if (*import_variable == '_') { - - import_variable++; - } - - map_fold = ut_fold_string(import_variable); - HASH_SEARCH( - next, - m_htbl, - map_fold, - map_hash_chain_t*, - hash_chain, - , - (ut_strcmp(hash_chain->symbol, import_variable) == 0)); - - if (hash_chain == NULL) { -#endif - if (wdl_init == TRUE) { - - sql_print_error( - "InnoDB: the variable address of %s" - " is not found.", - import_variable); - } - - return(0); -#ifdef _WIN64 - } -#endif - } - - return((void*) ((ulint) m_handle + hash_chain->value)); -} - -/*******************************************************************//** -Bind all unresolved external variables from the MySQL executable. -@return TRUE if successful */ -static -bool -wdl_get_external_variables(void) -/*============================*/ -{ - HMODULE hmod = wdl_get_mysqld_mapfile(); - - if (hmod == 0) { - - return(FALSE); - } - -#define GET_SYM(sym, var, type) \ - var = (type*) wdl_get_varaddr_from_map(hmod, sym); \ - if (var == NULL) return(FALSE) -#ifdef _WIN64 -#define GET_SYM2(sym1, sym2, var, type) \ - var = (type*) wdl_get_varaddr_from_map(hmod, sym1); \ - if (var == NULL) return(FALSE) -#else -#define GET_SYM2(sym1, sym2, var, type) \ - var = (type*) wdl_get_varaddr_from_map(hmod, sym2); \ - if (var == NULL) return(FALSE) -#endif // (_WIN64) -#define GET_C_SYM(sym, type) GET_SYM(#sym, wdl_##sym, type) -#define GET_PROC_ADDR(sym) \ - wdl##sym = (pfn##sym) wdl_get_procaddr_from_map(hmod, #sym) - - GET_C_SYM(my_charset_bin, CHARSET_INFO); - GET_C_SYM(my_charset_latin1, CHARSET_INFO); - GET_C_SYM(my_charset_filename, CHARSET_INFO); - GET_C_SYM(default_charset_info, CHARSET_INFO*); - GET_C_SYM(all_charsets, CHARSET_INFO*); - GET_C_SYM(my_umask, int); - - GET_SYM("?global_system_variables@@3Usystem_variables@@A", - wdl_global_system_variables, struct system_variables); - GET_SYM("?mysql_real_data_home@@3PADA", - wdl_mysql_real_data_home, char); - GET_SYM("?reg_ext@@3PADA", wdl_reg_ext, char); - GET_SYM("?LOCK_thread_count@@3U_RTL_CRITICAL_SECTION@@A", - wdl_LOCK_thread_count, pthread_mutex_t); - GET_SYM("?key_map_full@@3V?$Bitmap@$0EA@@@A", - wdl_key_map_full, key_map); - GET_SYM("?mysql_tmpdir_list@@3Ust_my_tmpdir@@A", - wdl_mysql_tmpdir_list, MY_TMPDIR); - GET_SYM("?mysqld_embedded@@3_NA", - wdl_mysqld_embedded, bool); - GET_SYM("?lower_case_table_names@@3IA", - wdl_lower_case_table_names, uint); - GET_SYM("?specialflag@@3KA", wdl_specialflag, ulong); - - GET_SYM2("?system_charset_info@@3PEAUcharset_info_st@@EA", - "?system_charset_info@@3PAUcharset_info_st@@A", - wdl_system_charset_info, CHARSET_INFO*); - GET_SYM2("?mysql_data_home@@3PEADEA", - "?mysql_data_home@@3PADA", - wdl_mysql_data_home, char*); - GET_SYM2("?tx_isolation_names@@3PAPEBDA", - "?tx_isolation_names@@3PAPBDA", - wdl_tx_isolation_names, char*); - GET_SYM2("?binlog_format_names@@3PAPEBDA", - "?binlog_format_names@@3PAPBDA", - wdl_binlog_format_names, char*); - -#ifndef DBUG_OFF - GET_PROC_ADDR(_db_enter_); - GET_PROC_ADDR(_db_return_); - GET_PROC_ADDR(_db_pargs_); - GET_PROC_ADDR(_db_doprnt_); - GET_PROC_ADDR(_db_dump_); - - /* If any of the dbug functions is not available, just make them - all invalid. This is the case when working with a non-debug - version of the server. */ - if (wdl_db_enter_ == NULL || wdl_db_return_ == NULL - || wdl_db_pargs_ == NULL || wdl_db_doprnt_ == NULL - || wdl_db_dump_ == NULL) { - - wdl_db_enter_ = NULL; - wdl_db_return_ = NULL; - wdl_db_pargs_ = NULL; - wdl_db_doprnt_ = NULL; - wdl_db_dump_ = NULL; - } -#endif /* !DBUG_OFF */ - - wdl_init = TRUE; - return(TRUE); - -#undef GET_SYM -#undef GET_SYM2 -#undef GET_C_SYM -#undef GET_PROC_ADDR -} - -/*******************************************************************//** -The DLL Delayed Loading Helper Function for resolving externals. - -The function may fail due to one of the three reasons: - -* Invalid parameter, which happens if the attributes in pidd aren't - specified correctly. -* Failed to load the map file mysqld.map. -* Failed to find an external name in the map file mysqld.map. - -Note: this function is called by run-time as well as __HrLoadAllImportsForDll. -So, it has to follow Windows call convention. -@return the address of the imported function */ -extern "C" -FARPROC WINAPI -__delayLoadHelper2( -/*===============*/ - PCImgDelayDescr pidd, /*!< in: a const pointer to a - ImgDelayDescr, see delayimp.h. */ - FARPROC* iat_entry) /*!< in/out: A pointer to the slot in - the delay load import address table - to be updated with the address of the - imported function. */ -{ - ulint iIAT, iINT; - HMODULE hmod; - PCImgThunkData pitd; - FARPROC fun = NULL; - - /* Set up data used for the hook procs */ - InternalImgDelayDescr idd = { - pidd->grAttrs, - PFromRva<LPCSTR>(pidd->rvaDLLName), - PFromRva<HMODULE*>(pidd->rvaHmod), - PFromRva<PImgThunkData>(pidd->rvaIAT), - PFromRva<PCImgThunkData>(pidd->rvaINT), - PFromRva<PCImgThunkData>(pidd->rvaBoundIAT), - PFromRva<PCImgThunkData>(pidd->rvaUnloadIAT), - pidd->dwTimeStamp - }; - - DelayLoadInfo dli = { - sizeof(DelayLoadInfo), - pidd, - iat_entry, - idd.szName, - {0}, - 0, - 0, - 0 - }; - - /* Check the Delay Load Attributes, log an error of invalid - parameter, which happens if the attributes in pidd are not - specified correctly. */ - if ((idd.grAttrs & dlattrRva) == 0) { - - sql_print_error("InnoDB: invalid parameter for delay loader."); - return(0); - } - - hmod = *idd.phmod; - - /* Calculate the index for the IAT entry in the import address table. - The INT entries are ordered the same as the IAT entries so the - calculation can be done on the IAT side. */ - iIAT = (PCImgThunkData) iat_entry - idd.pIAT; - iINT = iIAT; - - pitd = &(idd.pINT[iINT]); - - dli.dlp.fImportByName = !IMAGE_SNAP_BY_ORDINAL(pitd->u1.Ordinal); - - if (dli.dlp.fImportByName) { - - dli.dlp.szProcName = (LPCSTR) (PFromRva<PIMAGE_IMPORT_BY_NAME> - ((RVA) ((UINT_PTR) pitd->u1.AddressOfData))->Name); - } else { - - dli.dlp.dwOrdinal = (ulint) IMAGE_ORDINAL(pitd->u1.Ordinal); - } - - /* Now, load the mapfile, if it has not been done yet */ - if (hmod == 0) { - - hmod = wdl_get_mysqld_mapfile(); - } - - if (hmod == 0) { - /* LoadLibrary failed. */ - PDelayLoadInfo rgpdli[1] = {&dli}; - - dli.dwLastError = ::GetLastError(); - - sql_print_error( - "InnoDB: failed to load mysqld.map with error %d.", - dli.dwLastError); - - return(0); - } - - /* Store the library handle. */ - idd.phmod = &hmod; - - /* Go for the procedure now. */ - dli.hmodCur = hmod; - - if (pidd->rvaBoundIAT && pidd->dwTimeStamp) { - - /* Bound imports exist, check the timestamp from the target - image */ - PIMAGE_NT_HEADERS pinh; - - pinh = (PIMAGE_NT_HEADERS) ((byte*) hmod - + ((PIMAGE_DOS_HEADER) hmod)->e_lfanew); - - if (pinh->Signature == IMAGE_NT_SIGNATURE - && pinh->FileHeader.TimeDateStamp == idd.dwTimeStamp - && (DWORD) hmod == pinh->OptionalHeader.ImageBase) { - - /* We have a decent address in the bound IAT. */ - fun = (FARPROC) (UINT_PTR) - idd.pBoundIAT[iIAT].u1.Function; - - if (fun) { - - *iat_entry = fun; - return(fun); - } - } - } - - fun = wdl_get_procaddr_from_map(hmod, dli.dlp.szProcName); - - if (fun == 0) { - - return(0); - } - - *iat_entry = fun; - return(fun); -} - -/*******************************************************************//** -Unload a DLL that was delay loaded. This function is called by run-time. -@return TRUE is returned if the DLL is found and the IAT matches the -original one. */ -extern "C" -BOOL WINAPI -__FUnloadDelayLoadedDLL2( -/*=====================*/ - LPCSTR module_name) /*!< in: DLL name */ -{ - return(TRUE); -} - -/**************************************************************//** -Load all imports from a DLL that was specified with the /delayload linker -option. -Note: this function is called by run-time. So, it has to follow Windows call -convention. -@return S_OK if the DLL matches, otherwise ERROR_MOD_NOT_FOUND is returned. */ -extern "C" -HRESULT WINAPI -__HrLoadAllImportsForDll( -/*=====================*/ - LPCSTR module_name) /*!< in: DLL name */ -{ - PIMAGE_NT_HEADERS img; - PCImgDelayDescr pidd; - IMAGE_DATA_DIRECTORY* image_data; - LPCSTR current_module; - HRESULT ret = ERROR_MOD_NOT_FOUND; - HMODULE hmod = (HMODULE) &__ImageBase; - - img = (PIMAGE_NT_HEADERS) ((byte*) hmod - + ((PIMAGE_DOS_HEADER) hmod)->e_lfanew); - image_data = - &img->OptionalHeader.DataDirectory[IMAGE_DIRECTORY_ENTRY_DELAY_IMPORT]; - - /* Scan the delay load IAT/INT for the DLL */ - if (image_data->Size) { - - pidd = PFromRva<PCImgDelayDescr>(image_data->VirtualAddress); - - /* Check all of the listed DLLs we want to load. */ - while (pidd->rvaDLLName) { - - current_module = PFromRva<LPCSTR>(pidd->rvaDLLName); - - if (stricmp(module_name, current_module) == 0) { - - /* Found it, break out with pidd and - current_module set appropriately */ - break; - } - - /* To the next delay import descriptor */ - pidd++; - } - - if (pidd->rvaDLLName) { - - /* Found a matching DLL, now process it. */ - FARPROC* iat_entry; - size_t count; - - iat_entry = PFromRva<FARPROC*>(pidd->rvaIAT); - count = wdl_import_count((PCImgThunkData) iat_entry); - - /* now load all the imports from the DLL */ - while (count > 0) { - - /* No need to check the return value */ - __delayLoadHelper2(pidd, iat_entry); - iat_entry++; - count--; - } - - ret = S_OK; - } - } - - return ret; -} - -/**************************************************************//** -The main function of a DLL -@return TRUE if the call succeeds */ -BOOL -WINAPI -DllMain( -/*====*/ - HINSTANCE hinstDLL, /*!< in: handle to the DLL module */ - DWORD fdwReason, /*!< Reason code that indicates why the - DLL entry-point function is being - called.*/ - LPVOID lpvReserved) /*!< in: additional parameter based on - fdwReason */ -{ - BOOL success = TRUE; - - switch (fdwReason) { - - case DLL_PROCESS_ATTACH: - success = wdl_get_external_variables(); - break; - - case DLL_PROCESS_DETACH: - wdl_cleanup(); - break; - } - - return(success); -} - -#ifndef DBUG_OFF -/**************************************************************//** -Process entry point to user function. It makes the call to _db_enter_ -in mysqld.exe. The DBUG functions are defined in my_dbug.h. */ -extern "C" UNIV_INTERN -void -_db_enter_( - const char* _func_, /*!< in: current function name */ - const char* _file_, /*!< in: current file name */ - uint _line_, /*!< in: current source line number */ - const char** _sfunc_, /*!< out: previous _func_ */ - const char** _sfile_, /*!< out: previous _file_ */ - uint* _slevel_, /*!< out: previous nesting level */ - char*** _sframep_) /*!< out: previous frame pointer */ -{ - if (wdl_db_enter_ != NULL) { - - wdl_db_enter_(_func_, _file_, _line_, _sfunc_, _sfile_, - _slevel_, _sframep_); - } -} - -/**************************************************************//** -Process exit from user function. It makes the call to _db_return_() -in the server. */ -extern "C" UNIV_INTERN -void -_db_return_( - uint _line_, /*!< in: current source line number */ - const char** _sfunc_, /*!< out: previous _func_ */ - const char** _sfile_, /*!< out: previous _file_ */ - uint* _slevel_) /*!< out: previous level */ -{ - if (wdl_db_return_ != NULL) { - - wdl_db_return_(_line_, _sfunc_, _sfile_, _slevel_); - } -} - -/**************************************************************//** -Log arguments for subsequent use. It makes the call to _db_pargs_() -in the server. */ -extern "C" UNIV_INTERN -void -_db_pargs_( - uint _line_, /*!< in: current source line number */ - const char* keyword) /*!< in: keyword for current macro */ -{ - if (wdl_db_pargs_ != NULL) { - - wdl_db_pargs_(_line_, keyword); - } -} - -/**************************************************************//** -Handle print of debug lines. It saves the text into a buffer first, -then makes the call to _db_doprnt_() in the server. The text is -truncated to the size of buffer. */ -extern "C" UNIV_INTERN -void -_db_doprnt_( - const char* format, /*!< in: the format string */ - ...) /*!< in: list of arguments */ -{ - va_list argp; - char buffer[512]; - - if (wdl_db_doprnt_ != NULL) { - - va_start(argp, format); - /* it is ok to ignore the trunction. */ - _vsnprintf(buffer, sizeof(buffer), format, argp); - wdl_db_doprnt_(buffer); - va_end(argp); - } -} - -/**************************************************************//** -Dump a string in hex. It makes the call to _db_dump_() in the server. */ -extern "C" UNIV_INTERN -void -_db_dump_( - uint _line_, /*!< in: current source line - number */ - const char* keyword, /*!< in: keyword list */ - const unsigned char* memory, /*!< in: memory to dump */ - size_t length) /*!< in: bytes to dump */ -{ - if (wdl_db_dump_ != NULL) { - - wdl_db_dump_(_line_, keyword, memory, length); - } -} - -#endif /* !DBUG_OFF */ -#endif /* defined (__WIN__) && defined (MYSQL_DYNAMIC_PLUGIN) */ === modified file 'storage/xtradb/ibuf/ibuf0ibuf.c' --- a/storage/xtradb/ibuf/ibuf0ibuf.c 2009-11-13 21:26:08 +0000 +++ b/storage/xtradb/ibuf/ibuf0ibuf.c 2010-01-15 15:58:25 +0000 @@ -390,6 +390,27 @@ ibuf_count_set( #endif /******************************************************************//** +Closes insert buffer and frees the data structures. */ +UNIV_INTERN +void +ibuf_close(void) +/*============*/ +{ + mutex_free(&ibuf_pessimistic_insert_mutex); + memset(&ibuf_pessimistic_insert_mutex, + 0x0, sizeof(ibuf_pessimistic_insert_mutex)); + + mutex_free(&ibuf_mutex); + memset(&ibuf_mutex, 0x0, sizeof(ibuf_mutex)); + + mutex_free(&ibuf_bitmap_mutex); + memset(&ibuf_bitmap_mutex, 0x0, sizeof(ibuf_mutex)); + + mem_free(ibuf); + ibuf = NULL; +} + +/******************************************************************//** Updates the size information of the ibuf, assuming the segment size has not changed. */ static === modified file 'storage/xtradb/include/btr0cur.h' --- a/storage/xtradb/include/btr0cur.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/btr0cur.h 2010-01-06 12:00:14 +0000 @@ -618,7 +618,7 @@ enum btr_cur_method { hash_node, and might be necessary to update */ BTR_CUR_BINARY, /*!< success using the binary search */ - BTR_CUR_INSERT_TO_IBUF, /*!< performed the intended insert to + BTR_CUR_INSERT_TO_IBUF /*!< performed the intended insert to the insert buffer */ }; === modified file 'storage/xtradb/include/btr0sea.h' --- a/storage/xtradb/include/btr0sea.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/btr0sea.h 2010-01-06 12:00:14 +0000 @@ -41,6 +41,12 @@ void btr_search_sys_create( /*==================*/ ulint hash_size); /*!< in: hash index hash table size */ +/*****************************************************************//** +Frees the adaptive search system at a database shutdown. */ +UNIV_INTERN +void +btr_search_sys_free(void); +/*=====================*/ /********************************************************************//** Disable the adaptive hash search system and empty the index. */ === modified file 'storage/xtradb/include/buf0buf.h' --- a/storage/xtradb/include/buf0buf.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/buf0buf.h 2010-01-06 12:00:14 +0000 @@ -346,7 +346,7 @@ buf_page_release( mtr_t* mtr); /*!< in: mtr */ /********************************************************************//** Moves a page to the start of the buffer pool LRU list. This high-level -function can be used to prevent an important page from from slipping out of +function can be used to prevent an important page from slipping out of the buffer pool. */ UNIV_INTERN void @@ -707,15 +707,6 @@ buf_page_belongs_to_unzip_LRU( /*==========================*/ const buf_page_t* bpage) /*!< in: pointer to control block */ __attribute__((pure)); -/*********************************************************************//** -Determine the approximate LRU list position of a block. -@return LRU list position */ -UNIV_INLINE -ulint -buf_page_get_LRU_position( -/*======================*/ - const buf_page_t* bpage) /*!< in: control block */ - __attribute__((pure)); /*********************************************************************//** Gets the mutex of a block. @@ -825,14 +816,14 @@ buf_page_set_old( buf_page_t* bpage, /*!< in/out: control block */ ibool old); /*!< in: old */ /*********************************************************************//** -Determine if a block has been accessed in the buffer pool. -@return TRUE if accessed */ +Determine the time of first access of a block in the buffer pool. +@return ut_time_ms() at the time of first access, 0 if not accessed */ UNIV_INLINE -ibool +unsigned buf_page_is_accessed( /*=================*/ const buf_page_t* bpage) /*!< in: control block */ - __attribute__((pure)); + __attribute__((nonnull, pure)); /*********************************************************************//** Flag a block accessed. */ UNIV_INLINE @@ -840,7 +831,8 @@ void buf_page_set_accessed( /*==================*/ buf_page_t* bpage, /*!< in/out: control block */ - ibool accessed); /*!< in: accessed */ + ulint time_ms) /*!< in: ut_time_ms() */ + __attribute__((nonnull)); /*********************************************************************//** Gets the buf_block_t handle of a buffered file block if an uncompressed page frame exists, or NULL. @@ -1026,14 +1018,6 @@ buf_block_hash_get( /*===============*/ ulint space, /*!< in: space id */ ulint offset);/*!< in: offset of the page within space */ -/*******************************************************************//** -Increments the pool clock by one and returns its new value. Remember that -in the 32 bit version the clock wraps around at 4 billion! -@return new clock value */ -UNIV_INLINE -ulint -buf_pool_clock_tic(void); -/*====================*/ /*********************************************************************//** Gets the current length of the free list of buffer blocks. @return length of the free list */ @@ -1073,16 +1057,10 @@ struct buf_page_struct{ flushed to disk, this tells the flush_type. @see enum buf_flush */ - unsigned accessed:1; /*!< TRUE if the page has been accessed - while in the buffer pool: read-ahead - may read in pages which have not been - accessed yet; a thread is allowed to - read this for heuristic purposes - without holding any mutex or latch */ unsigned io_fix:2; /*!< type of pending I/O operation; also protected by buf_pool_mutex @see enum buf_io_fix */ - unsigned buf_fix_count:24;/*!< count of how manyfold this block + unsigned buf_fix_count:25;/*!< count of how manyfold this block is currently bufferfixed */ /* @} */ #endif /* !UNIV_HOTBACKUP */ @@ -1112,7 +1090,16 @@ struct buf_page_struct{ - BUF_BLOCK_FILE_PAGE: flush_list - BUF_BLOCK_ZIP_DIRTY: flush_list - BUF_BLOCK_ZIP_PAGE: zip_clean - - BUF_BLOCK_ZIP_FREE: zip_free[] */ + - BUF_BLOCK_ZIP_FREE: zip_free[] + + The contents of the list node + is undefined if !in_flush_list + && state == BUF_BLOCK_FILE_PAGE, + or if state is one of + BUF_BLOCK_MEMORY, + BUF_BLOCK_REMOVE_HASH or + BUF_BLOCK_READY_IN_USE. */ + /* resplit for optimistic use */ UT_LIST_NODE_T(buf_page_t) free; UT_LIST_NODE_T(buf_page_t) flush_list; @@ -1155,18 +1142,8 @@ struct buf_page_struct{ debugging */ //#endif /* UNIV_DEBUG */ unsigned old:1; /*!< TRUE if the block is in the old - blocks in the LRU list */ - unsigned LRU_position:31;/*!< value which monotonically - decreases (or may stay - constant if old==TRUE) toward - the end of the LRU list, if - buf_pool->ulint_clock has not - wrapped around: NOTE that this - value can only be used in - heuristic algorithms, because - of the possibility of a - wrap-around! */ - unsigned freed_page_clock:32;/*!< the value of + blocks in buf_pool->LRU_old */ + unsigned freed_page_clock:31;/*!< the value of buf_pool->freed_page_clock when this block was the last time put to the head of the @@ -1174,6 +1151,9 @@ struct buf_page_struct{ to read this for heuristic purposes without holding any mutex or latch */ + unsigned access_time:32; /*!< time of first access, or + 0 if the block was never accessed + in the buffer pool */ /* @} */ # ifdef UNIV_DEBUG_FILE_ACCESSES ibool file_page_was_freed; @@ -1318,6 +1298,31 @@ Compute the hash fold value for blocks i #define BUF_POOL_ZIP_FOLD_BPAGE(b) BUF_POOL_ZIP_FOLD((buf_block_t*) (b)) /* @} */ +/** @brief The buffer pool statistics structure. */ +struct buf_pool_stat_struct{ + ulint n_page_gets; /*!< number of page gets performed; + also successful searches through + the adaptive hash index are + counted as page gets; this field + is NOT protected by the buffer + pool mutex */ + ulint n_pages_read; /*!< number read operations */ + ulint n_pages_written;/*!< number write operations */ + ulint n_pages_created;/*!< number of pages created + in the pool with no read */ + ulint n_ra_pages_read;/*!< number of pages read in + as part of read ahead */ + ulint n_ra_pages_evicted;/*!< number of read ahead + pages that are evicted without + being accessed */ + ulint n_pages_made_young; /*!< number of pages made young, in + calls to buf_LRU_make_block_young() */ + ulint n_pages_not_made_young; /*!< number of pages not made + young because the first access + was not long enough ago, in + buf_page_peek_if_too_old() */ +}; + /** @brief The buffer pool structure. NOTE! The definition appears here only for other modules of this @@ -1342,28 +1347,16 @@ struct buf_pool_struct{ ulint n_pend_reads; /*!< number of pending read operations */ ulint n_pend_unzip; /*!< number of pending decompressions */ - time_t last_printout_time; /*!< when buf_print was last time + time_t last_printout_time; + /*!< when buf_print_io was last time called */ - ulint n_pages_read; /*!< number read operations */ - ulint n_pages_written;/*!< number write operations */ - ulint n_pages_created;/*!< number of pages created - in the pool with no read */ - ulint n_page_gets; /*!< number of page gets performed; - also successful searches through - the adaptive hash index are - counted as page gets; this field - is NOT protected by the buffer - pool mutex */ - ulint n_page_gets_old;/*!< n_page_gets when buf_print was - last time called: used to calculate - hit rate */ - ulint n_pages_read_old;/*!< n_pages_read when buf_print was - last time called */ - ulint n_pages_written_old;/*!< number write operations */ - ulint n_pages_created_old;/*!< number of pages created in - the pool with no read */ + buf_pool_stat_t stat; /*!< current statistics */ + buf_pool_stat_t old_stat; /*!< old statistics */ + /* @} */ + /** @name Page flushing algorithm fields */ + /* @{ */ UT_LIST_BASE_NODE_T(buf_page_t) flush_list; @@ -1379,10 +1372,6 @@ struct buf_pool_struct{ /*!< this is in the set state when there is no flush batch of the given type running */ - ulint ulint_clock; /*!< a sequence number used to count - time. NOTE! This counter wraps - around at 4 billion (if ulint == - 32 bits)! */ ulint freed_page_clock;/*!< a sequence number used to count the number of buffer blocks removed from the end of @@ -1406,17 +1395,18 @@ struct buf_pool_struct{ block list */ UT_LIST_BASE_NODE_T(buf_page_t) LRU; /*!< base node of the LRU list */ - buf_page_t* LRU_old; /*!< pointer to the about 3/8 oldest - blocks in the LRU list; NULL if LRU - length less than BUF_LRU_OLD_MIN_LEN; + buf_page_t* LRU_old; /*!< pointer to the about + buf_LRU_old_ratio/BUF_LRU_OLD_RATIO_DIV + oldest blocks in the LRU list; + NULL if LRU length less than + BUF_LRU_OLD_MIN_LEN; NOTE: when LRU_old != NULL, its length should always equal LRU_old_len */ ulint LRU_old_len; /*!< length of the LRU list from the block to which LRU_old points onward, including that block; see buf0lru.c for the restrictions - on this value; not defined if - LRU_old == NULL; + on this value; 0 if LRU_old == NULL; NOTE: LRU_old_len must be adjusted whenever LRU_old shrinks or grows! */ === modified file 'storage/xtradb/include/buf0buf.ic' --- a/storage/xtradb/include/buf0buf.ic 2009-11-13 21:26:08 +0000 +++ b/storage/xtradb/include/buf0buf.ic 2010-01-15 15:58:25 +0000 @@ -72,9 +72,30 @@ buf_page_peek_if_too_old( /*=====================*/ const buf_page_t* bpage) /*!< in: block to make younger */ { - return(buf_pool->freed_page_clock - >= buf_page_get_freed_page_clock(bpage) - + 1 + (buf_pool->curr_size / 4)); + if (UNIV_UNLIKELY(buf_pool->freed_page_clock == 0)) { + /* If eviction has not started yet, do not update the + statistics or move blocks in the LRU list. This is + either the warm-up phase or an in-memory workload. */ + return(FALSE); + } else if (buf_LRU_old_threshold_ms && bpage->old) { + unsigned access_time = buf_page_is_accessed(bpage); + + if (access_time > 0 + && (ut_time_ms() - access_time) + >= buf_LRU_old_threshold_ms) { + return(TRUE); + } + + buf_pool->stat.n_pages_not_made_young++; + return(FALSE); + } else { + /* FIXME: bpage->freed_page_clock is 31 bits */ + return((buf_pool->freed_page_clock & ((1UL << 31) - 1)) + > ((ulint) bpage->freed_page_clock + + (buf_pool->curr_size + * (BUF_LRU_OLD_RATIO_DIV - buf_LRU_old_ratio) + / (BUF_LRU_OLD_RATIO_DIV * 4)))); + } } /*********************************************************************//** @@ -125,23 +146,6 @@ try_again: return(lsn); } - -/*******************************************************************//** -Increments the buf_pool clock by one and returns its new value. Remember -that in the 32 bit version the clock wraps around at 4 billion! -@return new clock value */ -UNIV_INLINE -ulint -buf_pool_clock_tic(void) -/*====================*/ -{ - //ut_ad(buf_pool_mutex_own()); - ut_ad(mutex_own(&LRU_list_mutex)); - - buf_pool->ulint_clock++; - - return(buf_pool->ulint_clock); -} #endif /* !UNIV_HOTBACKUP */ /*********************************************************************//** @@ -288,21 +292,6 @@ buf_page_belongs_to_unzip_LRU( } /*********************************************************************//** -Determine the approximate LRU list position of a block. -@return LRU list position */ -UNIV_INLINE -ulint -buf_page_get_LRU_position( -/*======================*/ - const buf_page_t* bpage) /*!< in: control block */ -{ - ut_ad(buf_page_in_file(bpage)); - //ut_ad(buf_pool_mutex_own()); /* This is used in optimistic */ - - return(bpage->LRU_position); -} - -/*********************************************************************//** Gets the mutex of a block. @return pointer to mutex protecting bpage */ UNIV_INLINE @@ -508,10 +497,19 @@ buf_page_set_old( ut_ad(bpage->in_LRU_list); #ifdef UNIV_LRU_DEBUG - if (UT_LIST_GET_PREV(LRU, bpage) && UT_LIST_GET_NEXT(LRU, bpage) - && UT_LIST_GET_PREV(LRU, bpage)->old - == UT_LIST_GET_NEXT(LRU, bpage)->old) { - ut_a(UT_LIST_GET_PREV(LRU, bpage)->old == old); + ut_a((buf_pool->LRU_old_len == 0) == (buf_pool->LRU_old == NULL)); + /* If a block is flagged "old", the LRU_old list must exist. */ + ut_a(!old || buf_pool->LRU_old); + + if (UT_LIST_GET_PREV(LRU, bpage) && UT_LIST_GET_NEXT(LRU, bpage)) { + const buf_page_t* prev = UT_LIST_GET_PREV(LRU, bpage); + const buf_page_t* next = UT_LIST_GET_NEXT(LRU, bpage); + if (prev->old == next->old) { + ut_a(prev->old == old); + } else { + ut_a(!prev->old); + ut_a(buf_pool->LRU_old == (old ? bpage : next)); + } } #endif /* UNIV_LRU_DEBUG */ @@ -519,17 +517,17 @@ buf_page_set_old( } /*********************************************************************//** -Determine if a block has been accessed in the buffer pool. -@return TRUE if accessed */ +Determine the time of first access of a block in the buffer pool. +@return ut_time_ms() at the time of first access, 0 if not accessed */ UNIV_INLINE -ibool +unsigned buf_page_is_accessed( /*=================*/ const buf_page_t* bpage) /*!< in: control block */ { ut_ad(buf_page_in_file(bpage)); - return(bpage->accessed); + return(bpage->access_time); } /*********************************************************************//** @@ -539,12 +537,16 @@ void buf_page_set_accessed( /*==================*/ buf_page_t* bpage, /*!< in/out: control block */ - ibool accessed) /*!< in: accessed */ + ulint time_ms) /*!< in: ut_time_ms() */ { ut_a(buf_page_in_file(bpage)); + //ut_ad(buf_pool_mutex_own()); ut_ad(mutex_own(buf_page_get_mutex(bpage))); - bpage->accessed = accessed; + if (!bpage->access_time) { + /* Make this the time of the first access. */ + bpage->access_time = time_ms; + } } /*********************************************************************//** @@ -825,15 +827,15 @@ buf_page_get_newest_modification( ib_uint64_t lsn; mutex_t* block_mutex = buf_page_get_mutex_enter(bpage); - ut_a(block_mutex); - - if (buf_page_in_file(bpage)) { + if (block_mutex && buf_page_in_file(bpage)) { lsn = bpage->newest_modification; } else { lsn = 0; } - mutex_exit(block_mutex); + if (block_mutex) { + mutex_exit(block_mutex); + } return(lsn); } === modified file 'storage/xtradb/include/buf0lru.h' --- a/storage/xtradb/include/buf0lru.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/buf0lru.h 2010-01-06 12:00:14 +0000 @@ -69,7 +69,7 @@ These are low-level functions #########################################################################*/ /** Minimum LRU list length for which the LRU_old pointer is defined */ -#define BUF_LRU_OLD_MIN_LEN 80 +#define BUF_LRU_OLD_MIN_LEN 512 /* 8 megabytes of 16k pages */ /** Maximum LRU list search length in buf_flush_LRU_recommendation() */ #define BUF_LRU_FREE_SEARCH_LEN (5 + 2 * BUF_READ_AHEAD_AREA) @@ -84,15 +84,6 @@ void buf_LRU_invalidate_tablespace( /*==========================*/ ulint id); /*!< in: space id */ -/******************************************************************//** -Gets the minimum LRU_position field for the blocks in an initial segment -(determined by BUF_LRU_INITIAL_RATIO) of the LRU list. The limit is not -guaranteed to be precise, because the ulint_clock may wrap around. -@return the limit; zero if could not determine it */ -UNIV_INTERN -ulint -buf_LRU_get_recent_limit(void); -/*==========================*/ /********************************************************************//** Insert a compressed block into buf_pool->zip_clean in the LRU order. */ UNIV_INTERN @@ -203,6 +194,18 @@ void buf_LRU_make_block_old( /*===================*/ buf_page_t* bpage); /*!< in: control block */ +/**********************************************************************//** +Updates buf_LRU_old_ratio. +@return updated old_pct */ +UNIV_INTERN +uint +buf_LRU_old_ratio_update( +/*=====================*/ + uint old_pct,/*!< in: Reserve this percentage of + the buffer pool for "old" blocks. */ + ibool adjust);/*!< in: TRUE=adjust the LRU list; + FALSE=just assign buf_LRU_old_ratio + during the initialization of InnoDB */ /********************************************************************//** Update the historical stats that we are collecting for LRU eviction policy at the end of each interval. */ @@ -210,6 +213,18 @@ UNIV_INTERN void buf_LRU_stat_update(void); /*=====================*/ +/********************************************************************//** +Dump the LRU page list to the specific file. */ +UNIV_INTERN +ibool +buf_LRU_file_dump(void); +/*===================*/ +/********************************************************************//** +Read the pages based on the specific file.*/ +UNIV_INTERN +ibool +buf_LRU_file_restore(void); +/*======================*/ #if defined UNIV_DEBUG || defined UNIV_BUF_DEBUG /**********************************************************************//** @@ -229,6 +244,35 @@ buf_LRU_print(void); /*===============*/ #endif /* UNIV_DEBUG_PRINT || UNIV_DEBUG || UNIV_BUF_DEBUG */ +/** @name Heuristics for detecting index scan @{ */ +/** Reserve this much/BUF_LRU_OLD_RATIO_DIV of the buffer pool for +"old" blocks. Protected by buf_pool_mutex. */ +extern uint buf_LRU_old_ratio; +/** The denominator of buf_LRU_old_ratio. */ +#define BUF_LRU_OLD_RATIO_DIV 1024 +/** Maximum value of buf_LRU_old_ratio. +@see buf_LRU_old_adjust_len +@see buf_LRU_old_ratio_update */ +#define BUF_LRU_OLD_RATIO_MAX BUF_LRU_OLD_RATIO_DIV +/** Minimum value of buf_LRU_old_ratio. +@see buf_LRU_old_adjust_len +@see buf_LRU_old_ratio_update +The minimum must exceed +(BUF_LRU_OLD_TOLERANCE + 5) * BUF_LRU_OLD_RATIO_DIV / BUF_LRU_OLD_MIN_LEN. */ +#define BUF_LRU_OLD_RATIO_MIN 51 + +#if BUF_LRU_OLD_RATIO_MIN >= BUF_LRU_OLD_RATIO_MAX +# error "BUF_LRU_OLD_RATIO_MIN >= BUF_LRU_OLD_RATIO_MAX" +#endif +#if BUF_LRU_OLD_RATIO_MAX > BUF_LRU_OLD_RATIO_DIV +# error "BUF_LRU_OLD_RATIO_MAX > BUF_LRU_OLD_RATIO_DIV" +#endif + +/** Move blocks to "new" LRU list only if the first access was at +least this many milliseconds ago. Not protected by any mutex or latch. */ +extern uint buf_LRU_old_threshold_ms; +/* @} */ + /** @brief Statistics for selecting the LRU list for eviction. These statistics are not 'of' LRU but 'for' LRU. We keep count of I/O === modified file 'storage/xtradb/include/buf0rea.h' --- a/storage/xtradb/include/buf0rea.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/buf0rea.h 2010-01-06 12:00:14 +0000 @@ -27,28 +27,59 @@ Created 11/5/1995 Heikki Tuuri #define buf0rea_h #include "univ.i" +#include "trx0types.h" #include "buf0types.h" /********************************************************************//** +Low-level function which reads a page asynchronously from a file to the +buffer buf_pool if it is not already there, in which case does nothing. +Sets the io_fix flag and sets an exclusive lock on the buffer frame. The +flag is cleared and the x-lock released by an i/o-handler thread. +@return 1 if a read request was queued, 0 if the page already resided +in buf_pool, or if the page is in the doublewrite buffer blocks in +which case it is never read into the pool, or if the tablespace does +not exist or is being dropped +@return 1 if read request is issued. 0 if it is not */ +UNIV_INTERN +ulint +buf_read_page_low( +/*==============*/ + ulint* err, /*!< out: DB_SUCCESS or DB_TABLESPACE_DELETED if we are + trying to read from a non-existent tablespace, or a + tablespace which is just now being dropped */ + ibool sync, /*!< in: TRUE if synchronous aio is desired */ + ulint mode, /*!< in: BUF_READ_IBUF_PAGES_ONLY, ..., + ORed to OS_AIO_SIMULATED_WAKE_LATER (see below + at read-ahead functions) */ + ulint space, /*!< in: space id */ + ulint zip_size,/*!< in: compressed page size, or 0 */ + ibool unzip, /*!< in: TRUE=request uncompressed page */ + ib_int64_t tablespace_version, /*!< in: if the space memory object has + this timestamp different from what we are giving here, + treat the tablespace as dropped; this is a timestamp we + use to stop dangling page reads from a tablespace + which we have DISCARDed + IMPORTed back */ + ulint offset, /*!< in: page number */ + trx_t* trx); +/********************************************************************//** High-level function which reads a page asynchronously from a file to the buffer buf_pool if it is not already there. Sets the io_fix flag and sets an exclusive lock on the buffer frame. The flag is cleared and the x-lock -released by the i/o-handler thread. Does a random read-ahead if it seems -sensible. -@return number of page read requests issued: this can be greater than -1 if read-ahead occurred */ +released by the i/o-handler thread. +@return TRUE if page has been read in, FALSE in case of failure */ UNIV_INTERN -ulint +ibool buf_read_page( /*==========*/ ulint space, /*!< in: space id */ ulint zip_size,/*!< in: compressed page size in bytes, or 0 */ - ulint offset);/*!< in: page number */ + ulint offset, /*!< in: page number */ + trx_t* trx); /********************************************************************//** Applies linear read-ahead if in the buf_pool the page is a border page of a linear read-ahead area and all the pages in the area have been accessed. Does not read any page if the read-ahead mechanism is not activated. Note -that the the algorithm looks at the 'natural' adjacent successor and +that the algorithm looks at the 'natural' adjacent successor and predecessor of the page, which on the leaf level of a B-tree are the next and previous page in the chain of leaves. To know these, the page specified in (space, offset) must already be present in the buf_pool. Thus, the @@ -74,8 +105,9 @@ buf_read_ahead_linear( /*==================*/ ulint space, /*!< in: space id */ ulint zip_size,/*!< in: compressed page size in bytes, or 0 */ - ulint offset);/*!< in: page number of a page; NOTE: the current thread + ulint offset, /*!< in: page number of a page; NOTE: the current thread must want access to this page (see NOTE 3 above) */ + trx_t* trx); /********************************************************************//** Issues read requests for pages which the ibuf module wants to read in, in order to contract the insert buffer tree. Technically, this function is like === modified file 'storage/xtradb/include/buf0types.h' --- a/storage/xtradb/include/buf0types.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/buf0types.h 2010-01-06 12:00:14 +0000 @@ -34,6 +34,8 @@ typedef struct buf_block_struct buf_blo typedef struct buf_chunk_struct buf_chunk_t; /** Buffer pool comprising buf_chunk_t */ typedef struct buf_pool_struct buf_pool_t; +/** Buffer pool statistics struct */ +typedef struct buf_pool_stat_struct buf_pool_stat_t; /** A buffer frame. @see page_t */ typedef byte buf_frame_t; === modified file 'storage/xtradb/include/db0err.h' --- a/storage/xtradb/include/db0err.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/db0err.h 2010-01-06 12:00:14 +0000 @@ -32,6 +32,7 @@ enum db_err { /* The following are error codes */ DB_ERROR, + DB_INTERRUPTED, DB_OUT_OF_MEMORY, DB_OUT_OF_FILE_SPACE, DB_LOCK_WAIT, === modified file 'storage/xtradb/include/dict0crea.h' --- a/storage/xtradb/include/dict0crea.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/dict0crea.h 2010-01-06 12:00:14 +0000 @@ -110,7 +110,7 @@ dict_create_or_check_foreign_constraint_ Adds foreign key definitions to data dictionary tables in the database. We look at table->foreign_list, and also generate names to constraints that were not named by the user. A generated constraint has a name of the format -databasename/tablename_ibfk_<number>, where the numbers start from 1, and are +databasename/tablename_ibfk_NUMBER, where the numbers start from 1, and are given locally for this table, that is, the number is not global, as in the old format constraints < 4.0.18 it used to be. @return error code or DB_SUCCESS */ === modified file 'storage/xtradb/include/dict0dict.h' --- a/storage/xtradb/include/dict0dict.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/dict0dict.h 2010-01-06 12:00:14 +0000 @@ -712,7 +712,7 @@ dict_index_find_on_id_low( dulint id); /*!< in: index id */ /**********************************************************************//** Adds an index to the dictionary cache. -@return DB_SUCCESS or error code */ +@return DB_SUCCESS, DB_TOO_BIG_RECORD, or DB_CORRUPTION */ UNIV_INTERN ulint dict_index_add_to_cache( @@ -1157,6 +1157,13 @@ void dict_ind_init(void); /*===============*/ +/**********************************************************************//** +Closes the data dictionary module. */ +UNIV_INTERN +void +dict_close(void); +/*============*/ + #ifndef UNIV_NONINL #include "dict0dict.ic" #endif === modified file 'storage/xtradb/include/dict0mem.h' --- a/storage/xtradb/include/dict0mem.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/dict0mem.h 2010-01-06 12:00:14 +0000 @@ -317,7 +317,7 @@ struct dict_foreign_struct{ char* id; /*!< id of the constraint as a null-terminated string */ unsigned n_fields:10; /*!< number of indexes' first fields - for which the the foreign key + for which the foreign key constraint is defined: we allow the indexes to contain more fields than mentioned in the constraint, as long === modified file 'storage/xtradb/include/fil0fil.h' --- a/storage/xtradb/include/fil0fil.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/fil0fil.h 2010-01-06 12:00:14 +0000 @@ -224,15 +224,6 @@ fil_space_create( 0 for uncompressed tablespaces */ ulint purpose);/*!< in: FIL_TABLESPACE, or FIL_LOG if log */ /*******************************************************************//** -Frees a space object from a the tablespace memory cache. Closes the files in -the chain but does not delete them. -@return TRUE if success */ -UNIV_INTERN -ibool -fil_space_free( -/*===========*/ - ulint id); /*!< in: space id */ -/*******************************************************************//** Returns the size of the space in pages. The tablespace must be cached in the memory cache. @return space size, 0 if space not found */ @@ -278,6 +269,12 @@ fil_init( ulint hash_size, /*!< in: hash table size */ ulint max_n_open); /*!< in: max number of open files */ /*******************************************************************//** +Initializes the tablespace memory cache. */ +UNIV_INTERN +void +fil_close(void); +/*===========*/ +/*******************************************************************//** Opens all log files and system tablespace data files. They stay open until the database server shutdown. This should be called at a server startup after the space objects for the log and the system tablespace have been created. The @@ -614,9 +611,12 @@ fil_space_get_n_reserved_extents( Reads or writes data. This operation is asynchronous (aio). @return DB_SUCCESS, or DB_TABLESPACE_DELETED if we are trying to do i/o on a tablespace which does not exist */ +#define fil_io(type, sync, space_id, zip_size, block_offset, byte_offset, len, buf, message) \ + _fil_io(type, sync, space_id, zip_size, block_offset, byte_offset, len, buf, message, NULL) + UNIV_INTERN ulint -fil_io( +_fil_io( /*===*/ ulint type, /*!< in: OS_FILE_READ or OS_FILE_WRITE, ORed to OS_FILE_LOG, if a log i/o @@ -641,8 +641,25 @@ fil_io( void* buf, /*!< in/out: buffer where to store read data or from where to write; in aio this must be appropriately aligned */ - void* message); /*!< in: message for aio handler if non-sync + void* message, /*!< in: message for aio handler if non-sync aio used, else ignored */ + trx_t* trx); +/********************************************************************//** +Confirm whether the parameters are valid or not */ +UNIV_INTERN +ibool +fil_area_is_exist( +/*==============*/ + ulint space_id, /*!< in: space id */ + ulint zip_size, /*!< in: compressed page size in bytes; + 0 for uncompressed pages */ + ulint block_offset, /*!< in: offset in number of blocks */ + ulint byte_offset, /*!< in: remainder of offset in bytes; in + aio this must be divisible by the OS block + size */ + ulint len); /*!< in: how many bytes to read or write; this + must not cross a file boundary; in aio this + must be a block size multiple */ /**********************************************************************//** Waits for an aio operation to complete. This function is used to write the handler for completed requests. The aio array of pending requests is divided === modified file 'storage/xtradb/include/fsp0fsp.h' --- a/storage/xtradb/include/fsp0fsp.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/fsp0fsp.h 2010-01-06 12:00:14 +0000 @@ -42,7 +42,7 @@ fsp_init(void); /*==========*/ /**********************************************************************//** Gets the current free limit of the system tablespace. The free limit -means the place of the first page which has never been put to the the +means the place of the first page which has never been put to the free list for allocation. The space above that address is initialized to zero. Sets also the global variable log_fsp_current_free_limit. @return free limit in megabytes */ === modified file 'storage/xtradb/include/ibuf0ibuf.h' --- a/storage/xtradb/include/ibuf0ibuf.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/ibuf0ibuf.h 2010-01-06 12:00:14 +0000 @@ -356,6 +356,12 @@ void ibuf_print( /*=======*/ FILE* file); /*!< in: file where to print */ +/******************************************************************//** +Closes insert buffer and frees the data structures. */ +UNIV_INTERN +void +ibuf_close(void); +/*============*/ #define IBUF_HEADER_PAGE_NO FSP_IBUF_HEADER_PAGE_NO #define IBUF_TREE_ROOT_PAGE_NO FSP_IBUF_TREE_ROOT_PAGE_NO === modified file 'storage/xtradb/include/lock0lock.h' --- a/storage/xtradb/include/lock0lock.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/lock0lock.h 2010-01-06 12:00:14 +0000 @@ -59,6 +59,12 @@ lock_sys_create( /*============*/ ulint n_cells); /*!< in: number of slots in lock hash table */ /*********************************************************************//** +Closes the lock system at database shutdown. */ +UNIV_INTERN +void +lock_sys_close(void); +/*================*/ +/*********************************************************************//** Checks if some transaction has an implicit x-lock on a record in a clustered index. @return transaction which has the x-lock, or NULL */ @@ -630,6 +636,14 @@ lock_number_of_rows_locked( /*=======================*/ trx_t* trx); /*!< in: transaction */ /*******************************************************************//** +Check if a transaction holds any autoinc locks. +@return TRUE if the transaction holds any AUTOINC locks. */ +UNIV_INTERN +ibool +lock_trx_holds_autoinc_locks( +/*=========================*/ + const trx_t* trx); /*!< in: transaction */ +/*******************************************************************//** Release all the transaction's autoinc locks. */ UNIV_INTERN void === modified file 'storage/xtradb/include/log0log.h' --- a/storage/xtradb/include/log0log.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/log0log.h 2010-01-06 12:00:14 +0000 @@ -118,10 +118,9 @@ UNIV_INLINE ib_uint64_t log_reserve_and_write_fast( /*=======================*/ - byte* str, /*!< in: string */ + const void* str, /*!< in: string */ ulint len, /*!< in: string length */ - ib_uint64_t* start_lsn,/*!< out: start lsn of the log record */ - ibool* success);/*!< out: TRUE if success */ + ib_uint64_t* start_lsn);/*!< out: start lsn of the log record */ /***********************************************************************//** Releases the log mutex. */ UNIV_INLINE @@ -283,7 +282,7 @@ log_make_checkpoint_at( later lsn, if IB_ULONGLONG_MAX, makes a checkpoint at the latest lsn */ ibool write_always); /*!< in: the function normally checks if - the the new checkpoint would have a + the new checkpoint would have a greater lsn than the previous one: if not, then no physical write is done; by setting this parameter TRUE, a @@ -573,6 +572,18 @@ UNIV_INTERN void log_refresh_stats(void); /*===================*/ +/********************************************************** +Shutdown the log system but do not release all the memory. */ +UNIV_INTERN +void +log_shutdown(void); +/*==============*/ +/********************************************************** +Free the log system data structures. */ +UNIV_INTERN +void +log_mem_free(void); +/*==============*/ extern log_t* log_sys; @@ -585,7 +596,7 @@ extern log_t* log_sys; #define LOG_RECOVER 98887331 /* The counting of lsn's starts from this value: this must be non-zero */ -#define LOG_START_LSN ((ib_uint64_t) (16 * OS_FILE_LOG_BLOCK_SIZE)) +#define LOG_START_LSN ((ib_uint64_t) (16 * OS_FILE_LOG_BLOCK_SIZE)) #define LOG_BUFFER_SIZE (srv_log_buffer_size * UNIV_PAGE_SIZE) #define LOG_ARCHIVE_BUF_SIZE (srv_log_buffer_size * UNIV_PAGE_SIZE / 4) @@ -722,9 +733,12 @@ struct log_group_struct{ ulint lsn_offset; /*!< the offset of the above lsn */ ulint n_pending_writes;/*!< number of currently pending flush writes for this log group */ + byte** file_header_bufs_ptr;/*!< unaligned buffers */ byte** file_header_bufs;/*!< buffers for each file header in the group */ +#ifdef UNIV_LOG_ARCHIVE /*-----------------------------*/ + byte** archive_file_header_bufs_ptr;/*!< unaligned buffers */ byte** archive_file_header_bufs;/*!< buffers for each file header in the group */ ulint archive_space_id;/*!< file space which @@ -743,10 +757,12 @@ struct log_group_struct{ completion function then sets the new value to ..._file_no */ ulint next_archived_offset; /*!< like the preceding field */ +#endif /* UNIV_LOG_ARCHIVE */ /*-----------------------------*/ ib_uint64_t scanned_lsn; /*!< used only in recovery: recovery scan succeeded up to this lsn in this log group */ + byte* checkpoint_buf_ptr;/*!< unaligned checkpoint header */ byte* checkpoint_buf; /*!< checkpoint header is written from this buffer to the group */ UT_LIST_NODE_T(log_group_t) @@ -764,6 +780,7 @@ struct log_struct{ #ifndef UNIV_HOTBACKUP mutex_t mutex; /*!< mutex protecting the log */ #endif /* !UNIV_HOTBACKUP */ + byte* buf_ptr; /* unaligned log buffer */ byte* buf; /*!< log buffer */ ulint buf_size; /*!< log buffer size in bytes */ ulint max_buf_free; /*!< recommended maximum value of @@ -900,6 +917,7 @@ struct log_struct{ should wait for this without owning the log mutex */ #endif /* !UNIV_HOTBACKUP */ + byte* checkpoint_buf_ptr;/* unaligned checkpoint header */ byte* checkpoint_buf; /*!< checkpoint header is read to this buffer */ /* @} */ === modified file 'storage/xtradb/include/log0log.ic' --- a/storage/xtradb/include/log0log.ic 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/log0log.ic 2010-01-06 12:00:14 +0000 @@ -27,6 +27,7 @@ Created 12/9/1995 Heikki Tuuri #include "mach0data.h" #include "mtr0mtr.h" +#ifdef UNIV_LOG_DEBUG /******************************************************//** Checks by parsing that the catenated log segment for a single mtr is consistent. */ @@ -34,11 +35,12 @@ UNIV_INTERN ibool log_check_log_recs( /*===============*/ - byte* buf, /*!< in: pointer to the start of + const byte* buf, /*!< in: pointer to the start of the log segment in the log_sys->buf log buffer */ ulint len, /*!< in: segment length in bytes */ ib_uint64_t buf_start_lsn); /*!< in: buffer start lsn */ +#endif /* UNIV_LOG_DEBUG */ /************************************************************//** Gets a log block flush bit. @@ -305,55 +307,76 @@ UNIV_INLINE ib_uint64_t log_reserve_and_write_fast( /*=======================*/ - byte* str, /*!< in: string */ + const void* str, /*!< in: string */ ulint len, /*!< in: string length */ - ib_uint64_t* start_lsn,/*!< out: start lsn of the log record */ - ibool* success)/*!< out: TRUE if success */ + ib_uint64_t* start_lsn)/*!< out: start lsn of the log record */ { - log_t* log = log_sys; ulint data_len; - ib_uint64_t lsn; - - *success = TRUE; - - mutex_enter(&(log->mutex)); - - data_len = len + log->buf_free % OS_FILE_LOG_BLOCK_SIZE; +#ifdef UNIV_LOG_LSN_DEBUG + /* length of the LSN pseudo-record */ + ulint lsn_len = 1 + + mach_get_compressed_size(log_sys->lsn >> 32) + + mach_get_compressed_size(log_sys->lsn & 0xFFFFFFFFUL); +#endif /* UNIV_LOG_LSN_DEBUG */ + + mutex_enter(&log_sys->mutex); + + data_len = len +#ifdef UNIV_LOG_LSN_DEBUG + + lsn_len +#endif /* UNIV_LOG_LSN_DEBUG */ + + log_sys->buf_free % OS_FILE_LOG_BLOCK_SIZE; if (data_len >= OS_FILE_LOG_BLOCK_SIZE - LOG_BLOCK_TRL_SIZE) { /* The string does not fit within the current log block or the log block would become full */ - *success = FALSE; - - mutex_exit(&(log->mutex)); + mutex_exit(&log_sys->mutex); return(0); } - *start_lsn = log->lsn; + *start_lsn = log_sys->lsn; + +#ifdef UNIV_LOG_LSN_DEBUG + { + /* Write the LSN pseudo-record. */ + byte* b = &log_sys->buf[log_sys->buf_free]; + *b++ = MLOG_LSN | (MLOG_SINGLE_REC_FLAG & *(const byte*) str); + /* Write the LSN in two parts, + as a pseudo page number and space id. */ + b += mach_write_compressed(b, log_sys->lsn >> 32); + b += mach_write_compressed(b, log_sys->lsn & 0xFFFFFFFFUL); + ut_a(b - lsn_len == &log_sys->buf[log_sys->buf_free]); - ut_memcpy(log->buf + log->buf_free, str, len); + memcpy(b, str, len); + len += lsn_len; + } +#else /* UNIV_LOG_LSN_DEBUG */ + memcpy(log_sys->buf + log_sys->buf_free, str, len); +#endif /* UNIV_LOG_LSN_DEBUG */ - log_block_set_data_len((byte*) ut_align_down(log->buf + log->buf_free, + log_block_set_data_len((byte*) ut_align_down(log_sys->buf + + log_sys->buf_free, OS_FILE_LOG_BLOCK_SIZE), data_len); #ifdef UNIV_LOG_DEBUG - log->old_buf_free = log->buf_free; - log->old_lsn = log->lsn; + log_sys->old_buf_free = log_sys->buf_free; + log_sys->old_lsn = log_sys->lsn; #endif - log->buf_free += len; + log_sys->buf_free += len; - ut_ad(log->buf_free <= log->buf_size); + ut_ad(log_sys->buf_free <= log_sys->buf_size); - lsn = log->lsn += len; + log_sys->lsn += len; #ifdef UNIV_LOG_DEBUG - log_check_log_recs(log->buf + log->old_buf_free, - log->buf_free - log->old_buf_free, log->old_lsn); + log_check_log_recs(log_sys->buf + log_sys->old_buf_free, + log_sys->buf_free - log_sys->old_buf_free, + log_sys->old_lsn); #endif - return(lsn); + return(log_sys->lsn); } /***********************************************************************//** === modified file 'storage/xtradb/include/log0recv.h' --- a/storage/xtradb/include/log0recv.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/log0recv.h 2010-01-06 12:00:14 +0000 @@ -239,6 +239,18 @@ UNIV_INTERN void recv_sys_create(void); /*=================*/ +/**********************************************************//** +Release recovery system mutexes. */ +UNIV_INTERN +void +recv_sys_close(void); +/*================*/ +/********************************************************//** +Frees the recovery system memory. */ +UNIV_INTERN +void +recv_sys_mem_free(void); +/*===================*/ /********************************************************//** Inits the recovery system for a recovery operation. */ UNIV_INTERN @@ -246,6 +258,12 @@ void recv_sys_init( /*==========*/ ulint available_memory); /*!< in: available memory in bytes */ +/********************************************************//** +Reset the state of the recovery system variables. */ +UNIV_INTERN +void +recv_sys_var_init(void); +/*===================*/ /*******************************************************************//** Empties the hash table of stored log records, applying them to appropriate pages. */ @@ -412,6 +430,39 @@ struct recv_sys_struct{ hash_table_t* addr_hash;/*!< hash table of file addresses of pages */ ulint n_addrs;/*!< number of not processed hashed file addresses in the hash table */ + +/* If you modified the following defines at original file, + You should also modify them. */ +/* defined in os0file.c */ +#define OS_AIO_MERGE_N_CONSECUTIVE 64 +/* defined in log0recv.c */ +#define RECV_READ_AHEAD_AREA 32 + time_t stats_recv_start_time; + ulint stats_recv_turns; + + ulint stats_read_requested_pages; + ulint stats_read_in_area[RECV_READ_AHEAD_AREA]; + + ulint stats_read_io_pages; + ulint stats_read_io_consecutive[OS_AIO_MERGE_N_CONSECUTIVE]; + ulint stats_write_io_pages; + ulint stats_write_io_consecutive[OS_AIO_MERGE_N_CONSECUTIVE]; + + ulint stats_doublewrite_check_pages; + ulint stats_doublewrite_overwrite_pages; + + ulint stats_recover_pages_with_read; + ulint stats_recover_pages_without_read; + + ulint stats_log_recs; + ulint stats_log_len_sum; + + ulint stats_applied_log_recs; + ulint stats_applied_log_len_sum; + ulint stats_pages_already_new; + + ib_uint64_t stats_oldest_modified_lsn; + ib_uint64_t stats_newest_modified_lsn; }; /** The recovery system */ @@ -433,6 +484,11 @@ are allowed yet: the variable name is mi extern ibool recv_no_ibuf_operations; /** TRUE when recv_init_crash_recovery() has been called. */ extern ibool recv_needed_recovery; +#ifdef UNIV_DEBUG +/** TRUE if writing to the redo log (mtr_commit) is forbidden. +Protected by log_sys->mutex. */ +extern ibool recv_no_log_write; +#endif /* UNIV_DEBUG */ /** TRUE if buf_page_is_corrupted() should check if the log sequence number (FIL_PAGE_LSN) is in the future. Initially FALSE, and set by === modified file 'storage/xtradb/include/mem0mem.h' --- a/storage/xtradb/include/mem0mem.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/mem0mem.h 2010-01-06 12:00:14 +0000 @@ -82,6 +82,13 @@ void mem_init( /*=====*/ ulint size); /*!< in: common pool size in bytes */ +/******************************************************************//** +Closes the memory system. */ +UNIV_INTERN +void +mem_close(void); +/*===========*/ + /**************************************************************//** Use this macro instead of the corresponding function! Macro for memory heap creation. */ === modified file 'storage/xtradb/include/mem0pool.h' --- a/storage/xtradb/include/mem0pool.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/mem0pool.h 2010-01-06 12:00:14 +0000 @@ -62,6 +62,13 @@ mem_pool_create( /*============*/ ulint size); /*!< in: pool size in bytes */ /********************************************************************//** +Frees a memory pool. */ +UNIV_INTERN +void +mem_pool_free( +/*==========*/ + mem_pool_t* pool); /*!< in, own: memory pool */ +/********************************************************************//** Allocates memory from a pool. NOTE: This low-level function should only be used in mem0mem.*! @return own: allocated memory buffer */ === modified file 'storage/xtradb/include/mtr0mtr.h' --- a/storage/xtradb/include/mtr0mtr.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/mtr0mtr.h 2010-01-06 12:00:14 +0000 @@ -106,6 +106,9 @@ For 1 - 8 bytes, the flag value must giv #define MLOG_IBUF_BITMAP_INIT ((byte)27) /*!< initialize an ibuf bitmap page */ /*#define MLOG_FULL_PAGE ((byte)28) full contents of a page */ +#ifdef UNIV_LOG_LSN_DEBUG +# define MLOG_LSN ((byte)28) /* current LSN */ +#endif #define MLOG_INIT_FILE_PAGE ((byte)29) /*!< this means that a file page is taken into use and the prior @@ -118,7 +121,7 @@ For 1 - 8 bytes, the flag value must giv #define MLOG_WRITE_STRING ((byte)30) /*!< write a string to a page */ #define MLOG_MULTI_REC_END ((byte)31) /*!< if a single mtr writes - log records for several pages, + several log records, this log record ends the sequence of these records */ #define MLOG_DUMMY_RECORD ((byte)32) /*!< dummy log record used to === modified file 'storage/xtradb/include/os0file.h' --- a/storage/xtradb/include/os0file.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/os0file.h 2010-01-06 12:00:14 +0000 @@ -53,6 +53,7 @@ Created 10/21/1995 Heikki Tuuri #define os0file_h #include "univ.i" +#include "trx0types.h" #ifndef __WIN__ #include <dirent.h> @@ -157,6 +158,8 @@ log. */ to become available again */ #define OS_FILE_SHARING_VIOLATION 76 #define OS_FILE_ERROR_NOT_SPECIFIED 77 +#define OS_FILE_INSUFFICIENT_RESOURCE 78 +#define OS_FILE_OPERATION_ABORTED 79 /* @} */ /** Types for aio operations @{ */ @@ -497,9 +500,12 @@ os_file_get_last_error( /*******************************************************************//** Requests a synchronous read operation. @return TRUE if request was successful, FALSE if fail */ +#define os_file_read(file, buf, offset, offset_high, n) \ + _os_file_read(file, buf, offset, offset_high, n, NULL) + UNIV_INTERN ibool -os_file_read( +_os_file_read( /*=========*/ os_file_t file, /*!< in: handle to a file */ void* buf, /*!< in: buffer where to read */ @@ -507,7 +513,8 @@ os_file_read( offset where to read */ ulint offset_high,/*!< in: most significant 32 bits of offset */ - ulint n); /*!< in: number of bytes to read */ + ulint n, /*!< in: number of bytes to read */ + trx_t* trx); /*******************************************************************//** Rewind file to its start, read at most size - 1 bytes from it to str, and NUL-terminate str. All errors are silently ignored. This function is @@ -619,6 +626,13 @@ os_aio_init( ulint n_write_segs, /*<! in: number of writer threads */ ulint n_slots_sync); /*<! in: number of slots in the sync aio array */ +/*********************************************************************** +Frees the asynchronous io system. */ +UNIV_INTERN +void +os_aio_free(void); +/*=============*/ + /*******************************************************************//** Requests an asynchronous i/o operation. @return TRUE if request was queued successfully, FALSE if fail */ @@ -654,10 +668,11 @@ os_aio( (can be used to identify a completed aio operation); ignored if mode is OS_AIO_SYNC */ - void* message2);/*!< in: message for the aio handler + void* message2,/*!< in: message for the aio handler (can be used to identify a completed aio operation); ignored if mode is OS_AIO_SYNC */ + trx_t* trx); /************************************************************************//** Wakes up all async i/o threads so that they know to exit themselves in shutdown. */ === modified file 'storage/xtradb/include/os0sync.h' --- a/storage/xtradb/include/os0sync.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/os0sync.h 2010-01-06 12:00:14 +0000 @@ -285,44 +285,74 @@ os_fast_mutex_free( /**********************************************************//** Atomic compare-and-swap and increment for InnoDB. */ -#ifdef HAVE_GCC_ATOMIC_BUILTINS +#if defined(HAVE_IB_GCC_ATOMIC_BUILTINS) + +#define HAVE_ATOMIC_BUILTINS + /**********************************************************//** Returns true if swapped, ptr is pointer to target, old_val is value to compare to, new_val is the value to swap in. */ + # define os_compare_and_swap(ptr, old_val, new_val) \ __sync_bool_compare_and_swap(ptr, old_val, new_val) + # define os_compare_and_swap_ulint(ptr, old_val, new_val) \ os_compare_and_swap(ptr, old_val, new_val) + # define os_compare_and_swap_lint(ptr, old_val, new_val) \ os_compare_and_swap(ptr, old_val, new_val) -# define os_compare_and_swap_thread_id(ptr, old_val, new_val) \ + +# ifdef HAVE_IB_ATOMIC_PTHREAD_T_GCC +# define os_compare_and_swap_thread_id(ptr, old_val, new_val) \ os_compare_and_swap(ptr, old_val, new_val) +# define INNODB_RW_LOCKS_USE_ATOMICS +# define IB_ATOMICS_STARTUP_MSG \ + "Mutexes and rw_locks use GCC atomic builtins" +# else /* HAVE_IB_ATOMIC_PTHREAD_T_GCC */ +# define IB_ATOMICS_STARTUP_MSG \ + "Mutexes use GCC atomic builtins, rw_locks do not" +# endif /* HAVE_IB_ATOMIC_PTHREAD_T_GCC */ + /**********************************************************//** Returns the resulting value, ptr is pointer to target, amount is the amount of increment. */ + # define os_atomic_increment(ptr, amount) \ __sync_add_and_fetch(ptr, amount) + # define os_atomic_increment_lint(ptr, amount) \ os_atomic_increment(ptr, amount) + # define os_atomic_increment_ulint(ptr, amount) \ os_atomic_increment(ptr, amount) + /**********************************************************//** Returns the old value of *ptr, atomically sets *ptr to new_val */ + # define os_atomic_test_and_set_byte(ptr, new_val) \ __sync_lock_test_and_set(ptr, new_val) + +#elif defined(HAVE_IB_SOLARIS_ATOMICS) + +#define HAVE_ATOMIC_BUILTINS + /* If not compiling with GCC or GCC doesn't support the atomic intrinsics and running on Solaris >= 10 use Solaris atomics */ -#elif defined(HAVE_SOLARIS_ATOMICS) + #include <atomic.h> + /**********************************************************//** Returns true if swapped, ptr is pointer to target, old_val is value to compare to, new_val is the value to swap in. */ + # define os_compare_and_swap_ulint(ptr, old_val, new_val) \ (atomic_cas_ulong(ptr, old_val, new_val) == old_val) + # define os_compare_and_swap_lint(ptr, old_val, new_val) \ ((lint)atomic_cas_ulong((ulong_t*) ptr, old_val, new_val) == old_val) -# ifdef INNODB_RW_LOCKS_USE_ATOMICS -# if SIZEOF_PTHREAD_T == 4 + +# ifdef HAVE_IB_ATOMIC_PTHREAD_T_SOLARIS +# if SIZEOF_PTHREAD_T == 4 # define os_compare_and_swap_thread_id(ptr, old_val, new_val) \ ((pthread_t)atomic_cas_32(ptr, old_val, new_val) == old_val) # elif SIZEOF_PTHREAD_T == 8 @@ -331,21 +361,35 @@ compare to, new_val is the value to swap # else # error "SIZEOF_PTHREAD_T != 4 or 8" # endif /* SIZEOF_PTHREAD_T CHECK */ -# endif /* INNODB_RW_LOCKS_USE_ATOMICS */ +# define INNODB_RW_LOCKS_USE_ATOMICS +# define IB_ATOMICS_STARTUP_MSG \ + "Mutexes and rw_locks use Solaris atomic functions" +# else /* HAVE_IB_ATOMIC_PTHREAD_T_SOLARIS */ +# define IB_ATOMICS_STARTUP_MSG \ + "Mutexes use Solaris atomic functions, rw_locks do not" +# endif /* HAVE_IB_ATOMIC_PTHREAD_T_SOLARIS */ /**********************************************************//** Returns the resulting value, ptr is pointer to target, amount is the amount of increment. */ + # define os_atomic_increment_lint(ptr, amount) \ atomic_add_long_nv((ulong_t*) ptr, amount) + # define os_atomic_increment_ulint(ptr, amount) \ atomic_add_long_nv(ptr, amount) + /**********************************************************//** Returns the old value of *ptr, atomically sets *ptr to new_val */ + # define os_atomic_test_and_set_byte(ptr, new_val) \ atomic_swap_uchar(ptr, new_val) -/* On Windows, use Windows atomics / interlocked */ + #elif defined(HAVE_WINDOWS_ATOMICS) + +#define HAVE_ATOMIC_BUILTINS + +/* On Windows, use Windows atomics / interlocked */ # ifdef _WIN64 # define win_cmp_and_xchg InterlockedCompareExchange64 # define win_xchg_and_add InterlockedExchangeAdd64 @@ -353,31 +397,46 @@ Returns the old value of *ptr, atomicall # define win_cmp_and_xchg InterlockedCompareExchange # define win_xchg_and_add InterlockedExchangeAdd # endif + /**********************************************************//** Returns true if swapped, ptr is pointer to target, old_val is value to compare to, new_val is the value to swap in. */ + # define os_compare_and_swap_ulint(ptr, old_val, new_val) \ (win_cmp_and_xchg(ptr, new_val, old_val) == old_val) + # define os_compare_and_swap_lint(ptr, old_val, new_val) \ (win_cmp_and_xchg(ptr, new_val, old_val) == old_val) -# ifdef INNODB_RW_LOCKS_USE_ATOMICS -# define os_compare_and_swap_thread_id(ptr, old_val, new_val) \ + +/* windows thread objects can always be passed to windows atomic functions */ +# define os_compare_and_swap_thread_id(ptr, old_val, new_val) \ (InterlockedCompareExchange(ptr, new_val, old_val) == old_val) -# endif /* INNODB_RW_LOCKS_USE_ATOMICS */ +# define INNODB_RW_LOCKS_USE_ATOMICS +# define IB_ATOMICS_STARTUP_MSG \ + "Mutexes and rw_locks use Windows interlocked functions" + /**********************************************************//** Returns the resulting value, ptr is pointer to target, amount is the amount of increment. */ + # define os_atomic_increment_lint(ptr, amount) \ (win_xchg_and_add(ptr, amount) + amount) + # define os_atomic_increment_ulint(ptr, amount) \ ((ulint) (win_xchg_and_add(ptr, amount) + amount)) + /**********************************************************//** Returns the old value of *ptr, atomically sets *ptr to new_val. InterlockedExchange() operates on LONG, and the LONG will be clobbered */ + # define os_atomic_test_and_set_byte(ptr, new_val) \ ((byte) InterlockedExchange(ptr, new_val)) -#endif /* HAVE_GCC_ATOMIC_BUILTINS */ + +#else +# define IB_ATOMICS_STARTUP_MSG \ + "Mutexes and rw_locks use InnoDB's own implementation" +#endif #ifndef UNIV_NONINL #include "os0sync.ic" === modified file 'storage/xtradb/include/page0page.h' --- a/storage/xtradb/include/page0page.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/page0page.h 2010-01-06 12:00:14 +0000 @@ -76,8 +76,11 @@ typedef byte page_header_t; header which are set in a page create */ /*----*/ #define PAGE_LEVEL 26 /* level of the node in an index tree; the - leaf level is the level 0 */ -#define PAGE_INDEX_ID 28 /* index id where the page belongs */ + leaf level is the level 0. This field should + not be written to after page creation. */ +#define PAGE_INDEX_ID 28 /* index id where the page belongs. + This field should not be written to after + page creation. */ #define PAGE_BTR_SEG_LEAF 36 /* file segment header for the leaf pages in a B-tree: defined only on the root page of a B-tree, but not in the root of an ibuf tree */ === modified file 'storage/xtradb/include/page0page.ic' --- a/storage/xtradb/include/page0page.ic 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/page0page.ic 2010-01-06 12:00:14 +0000 @@ -907,7 +907,7 @@ page_get_data_size( /************************************************************//** Allocates a block of memory from the free list of an index page. */ -UNIV_INTERN +UNIV_INLINE void page_mem_alloc_free( /*================*/ === modified file 'storage/xtradb/include/page0zip.h' --- a/storage/xtradb/include/page0zip.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/page0zip.h 2010-01-06 12:00:14 +0000 @@ -127,8 +127,12 @@ page_zip_decompress( /*================*/ page_zip_des_t* page_zip,/*!< in: data, ssize; out: m_start, m_end, m_nonempty, n_blobs */ - page_t* page) /*!< out: uncompressed page, may be trashed */ - __attribute__((nonnull)); + page_t* page, /*!< out: uncompressed page, may be trashed */ + ibool all) /*!< in: TRUE=decompress the whole page; + FALSE=verify but do not copy some + page header fields that should not change + after page creation */ + __attribute__((nonnull(1,2))); #ifdef UNIV_DEBUG /**********************************************************************//** @@ -385,8 +389,8 @@ IMPORTANT: if page_zip_reorganize() is i non-clustered index, the caller must update the insert buffer free bits in the same mini-transaction in such a way that the modification will be redo-logged. -@return TRUE on success, FALSE on failure; page and page_zip will be -left intact on failure. */ +@return TRUE on success, FALSE on failure; page_zip will be left +intact on failure, but page will be overwritten. */ UNIV_INTERN ibool page_zip_reorganize( === modified file 'storage/xtradb/include/pars0pars.h' --- a/storage/xtradb/include/pars0pars.h 2009-11-13 21:26:08 +0000 +++ b/storage/xtradb/include/pars0pars.h 2010-01-15 15:58:25 +0000 @@ -583,6 +583,12 @@ pars_info_get_bound_id( pars_info_t* info, /*!< in: info struct */ const char* name); /*!< in: bound id name to find */ +/******************************************************************//** +Release any resources used by the lexer. */ +UNIV_INTERN +void +pars_lexer_close(void); +/*==================*/ /** Extra information supplied for pars_sql(). */ struct pars_info_struct { === modified file 'storage/xtradb/include/rem0cmp.h' --- a/storage/xtradb/include/rem0cmp.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/rem0cmp.h 2010-01-06 12:00:14 +0000 @@ -89,7 +89,7 @@ cmp_dfield_dfield( /*************************************************************//** This function is used to compare a data tuple to a physical record. Only dtuple->n_fields_cmp first fields are taken into account for -the the data tuple! If we denote by n = n_fields_cmp, then rec must +the data tuple! If we denote by n = n_fields_cmp, then rec must have either m >= n fields, or it must differ from dtuple in some of the m fields rec has. If rec has an externally stored field we do not compare it but return with value 0 if such a comparison should be === modified file 'storage/xtradb/include/rem0rec.ic' --- a/storage/xtradb/include/rem0rec.ic 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/rem0rec.ic 2010-01-06 12:00:14 +0000 @@ -65,7 +65,7 @@ most significant bytes and bits are writ - offset_of_this_record) mod 64Ki, where mod is the modulo as a non-negative number; - we can calculate the the offset of the next + we can calculate the offset of the next record with the formula: relative_offset + offset_of_this_record mod UNIV_PAGE_SIZE === modified file 'storage/xtradb/include/row0ins.h' --- a/storage/xtradb/include/row0ins.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/row0ins.h 2010-01-06 12:00:14 +0000 @@ -45,7 +45,7 @@ row_ins_check_foreign_constraint( /*=============================*/ ibool check_ref,/*!< in: TRUE If we want to check that the referenced table is ok, FALSE if we - want to to check the foreign key table */ + want to check the foreign key table */ dict_foreign_t* foreign,/*!< in: foreign constraint; NOTE that the tables mentioned in it must be in the dictionary cache if they exist at all */ === modified file 'storage/xtradb/include/row0mysql.h' --- a/storage/xtradb/include/row0mysql.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/row0mysql.h 2010-01-06 12:00:14 +0000 @@ -177,7 +177,9 @@ row_update_prebuilt_trx( in MySQL handle */ trx_t* trx); /*!< in: transaction handle */ /*********************************************************************//** -Unlocks AUTO_INC type locks that were possibly reserved by a trx. */ +Unlocks AUTO_INC type locks that were possibly reserved by a trx. This +function should be called at the the end of an SQL statement, by the +connection thread that owns the transaction (trx->mysql_thd). */ UNIV_INTERN void row_unlock_table_autoinc_for_mysql( @@ -754,8 +756,6 @@ struct row_prebuilt_struct { store it here so that we can return it to MySQL */ /*----------------------*/ - UT_LIST_NODE_T(row_prebuilt_t) prebuilts; - /*!< list node of table->prebuilts */ ulint magic_n2; /*!< this should be the same as magic_n */ }; === modified file 'storage/xtradb/include/srv0srv.h' --- a/storage/xtradb/include/srv0srv.h 2009-11-13 21:26:08 +0000 +++ b/storage/xtradb/include/srv0srv.h 2010-01-15 15:58:25 +0000 @@ -80,6 +80,9 @@ at a time */ #define SRV_AUTO_EXTEND_INCREMENT \ (srv_auto_extend_increment * ((1024 * 1024) / UNIV_PAGE_SIZE)) +/* prototypes for new functions added to ha_innodb.cc */ +ibool innobase_get_slow_log(); + /* This is set to TRUE if the MySQL user has set it in MySQL */ extern ibool srv_lower_case_table_names; @@ -133,8 +136,9 @@ extern ulint* srv_data_file_is_raw_parti extern ibool srv_extra_undoslots; extern ibool srv_fast_recovery; +extern ibool srv_recovery_stats; -extern ibool srv_use_purge_thread; +extern ulint srv_use_purge_thread; extern ibool srv_auto_extend_last_data_file; extern ulint srv_last_file_size_max; @@ -235,12 +239,14 @@ extern ulong srv_replication_delay; extern long long srv_ibuf_max_size; extern ulong srv_ibuf_active_contract; extern ulong srv_ibuf_accel_rate; +extern ulint srv_checkpoint_age_target; extern ulong srv_flush_neighbor_pages; extern ulong srv_enable_unsafe_group_commit; extern ulong srv_read_ahead; extern ulong srv_adaptive_checkpoint; extern ulong srv_expand_import; +extern ulint srv_relax_table_creation; extern ulong srv_extra_rsegments; extern ulong srv_dict_size_limit; @@ -345,10 +351,6 @@ extern ulint srv_buf_pool_flushed; /** Number of buffer pool reads that led to the reading of a disk page */ extern ulint srv_buf_pool_reads; -/** Number of sequential read-aheads */ -extern ulint srv_read_ahead_seq; -/** Number of random read-aheads */ -extern ulint srv_read_ahead_rnd; /** Status variables to be passed to MySQL */ typedef struct export_var_struct export_struc; @@ -428,6 +430,7 @@ enum srv_thread_type { SRV_INSERT, /**< thread flushing the insert buffer to disk */ #endif SRV_PURGE, /* thread purging undo records */ + SRV_PURGE_WORKER, /* thread purging undo records */ SRV_MASTER /**< the master thread, (whose type number must be biggest) */ }; @@ -446,7 +449,7 @@ void srv_init(void); /*==========*/ /*********************************************************************//** -Frees the OS fast mutex created in srv_boot(). */ +Frees the data structures created in srv_init(). */ UNIV_INTERN void srv_free(void); @@ -509,6 +512,13 @@ srv_purge_thread( /*=============*/ void* arg); /* in: a dummy parameter required by os_thread_create */ +/************************************************************************* +The undo purge thread. */ +UNIV_INTERN +os_thread_ret_t +srv_purge_worker_thread( +/*====================*/ + void* arg); /*******************************************************************//** Tells the Innobase server that there has been activity in the database and wakes up the master thread if it is suspended (not sleeping). Used @@ -645,13 +655,13 @@ struct export_var_struct{ #ifdef UNIV_DEBUG ulint innodb_buffer_pool_pages_latched; /*!< Latched pages */ #endif /* UNIV_DEBUG */ - ulint innodb_buffer_pool_read_requests; /*!< buf_pool->n_page_gets */ + ulint innodb_buffer_pool_read_requests; /*!< buf_pool->stat.n_page_gets */ ulint innodb_buffer_pool_reads; /*!< srv_buf_pool_reads */ ulint innodb_buffer_pool_wait_free; /*!< srv_buf_pool_wait_free */ ulint innodb_buffer_pool_pages_flushed; /*!< srv_buf_pool_flushed */ ulint innodb_buffer_pool_write_requests;/*!< srv_buf_pool_write_requests */ - ulint innodb_buffer_pool_read_ahead_seq;/*!< srv_read_ahead_seq */ - ulint innodb_buffer_pool_read_ahead_rnd;/*!< srv_read_ahead_rnd */ + ulint innodb_buffer_pool_read_ahead; /*!< srv_read_ahead */ + ulint innodb_buffer_pool_read_ahead_evicted;/*!< srv_read_ahead evicted*/ ulint innodb_dblwr_pages_written; /*!< srv_dblwr_pages_written */ ulint innodb_dblwr_writes; /*!< srv_dblwr_writes */ ibool innodb_have_atomic_builtins; /*!< HAVE_ATOMIC_BUILTINS */ @@ -663,9 +673,9 @@ struct export_var_struct{ ulint innodb_os_log_pending_writes; /*!< srv_os_log_pending_writes */ ulint innodb_os_log_pending_fsyncs; /*!< fil_n_pending_log_flushes */ ulint innodb_page_size; /*!< UNIV_PAGE_SIZE */ - ulint innodb_pages_created; /*!< buf_pool->n_pages_created */ - ulint innodb_pages_read; /*!< buf_pool->n_pages_read */ - ulint innodb_pages_written; /*!< buf_pool->n_pages_written */ + ulint innodb_pages_created; /*!< buf_pool->stat.n_pages_created */ + ulint innodb_pages_read; /*!< buf_pool->stat.n_pages_read */ + ulint innodb_pages_written; /*!< buf_pool->stat.n_pages_written */ ulint innodb_row_lock_waits; /*!< srv_n_lock_wait_count */ ulint innodb_row_lock_current_waits; /*!< srv_n_lock_wait_current_count */ ib_int64_t innodb_row_lock_time; /*!< srv_n_lock_wait_time === modified file 'storage/xtradb/include/sync0rw.h' --- a/storage/xtradb/include/sync0rw.h 2009-11-13 21:26:08 +0000 +++ b/storage/xtradb/include/sync0rw.h 2010-01-15 15:58:25 +0000 @@ -120,7 +120,7 @@ is necessary only if the memory block co # endif /* UNIV_SYNC_DEBUG */ #else /* UNIV_DEBUG */ # define rw_lock_create(L, level) \ - rw_lock_create_func((L), __FILE__, __LINE__) + rw_lock_create_func((L), #L, NULL, 0) #endif /* UNIV_DEBUG */ /******************************************************************//** @@ -137,8 +137,8 @@ rw_lock_create_func( # ifdef UNIV_SYNC_DEBUG ulint level, /*!< in: level */ # endif /* UNIV_SYNC_DEBUG */ - const char* cmutex_name, /*!< in: mutex name */ #endif /* UNIV_DEBUG */ + const char* cmutex_name, /*!< in: mutex name */ const char* cfile_name, /*!< in: file name where created */ ulint cline); /*!< in: file line where created */ /******************************************************************//** @@ -540,7 +540,8 @@ struct rw_lock_struct { ulint level; /*!< Level in the global latching order. */ #endif /* UNIV_SYNC_DEBUG */ ulint count_os_wait; /*!< Count of os_waits. May not be accurate */ - const char* cfile_name;/*!< File name where lock created */ + //const char* cfile_name;/*!< File name where lock created */ + const char* lock_name;/*!< lock name */ /* last s-lock file/line is not guaranteed to be correct */ const char* last_s_file_name;/*!< File name where last s-locked */ const char* last_x_file_name;/*!< File name where last x-locked */ @@ -551,7 +552,7 @@ struct rw_lock_struct { are at the start of this struct, thus we can peek this field without causing much memory bus traffic */ - unsigned cline:14; /*!< Line where created */ + //unsigned cline:14; /*!< Line where created */ unsigned last_s_line:14; /*!< Line number where last time s-locked */ unsigned last_x_line:14; /*!< Line number where last time x-locked */ ulint magic_n; /*!< RW_LOCK_MAGIC_N */ === modified file 'storage/xtradb/include/sync0sync.h' --- a/storage/xtradb/include/sync0sync.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/sync0sync.h 2010-01-06 12:00:14 +0000 @@ -80,7 +80,7 @@ necessary only if the memory block conta # endif #else # define mutex_create(M, level) \ - mutex_create_func((M), __FILE__, __LINE__) + mutex_create_func((M), #M, NULL, 0) #endif /******************************************************************//** @@ -93,8 +93,8 @@ void mutex_create_func( /*==============*/ mutex_t* mutex, /*!< in: pointer to memory */ -#ifdef UNIV_DEBUG const char* cmutex_name, /*!< in: mutex name */ +#ifdef UNIV_DEBUG # ifdef UNIV_SYNC_DEBUG ulint level, /*!< in: level */ # endif /* UNIV_SYNC_DEBUG */ @@ -513,7 +513,7 @@ struct mutex_struct { os_fast_mutex; /*!< We use this OS mutex in place of lock_word when atomic operations are not enabled */ #endif - ulint waiters; /*!< This ulint is set to 1 if there are (or + volatile ulint waiters; /*!< This ulint is set to 1 if there are (or may be) threads waiting in the global wait array for this mutex to be released. Otherwise, this is 0. */ @@ -524,9 +524,9 @@ struct mutex_struct { ulint line; /*!< Line where the mutex was locked */ ulint level; /*!< Level in the global latching order */ #endif /* UNIV_SYNC_DEBUG */ +#ifdef UNIV_DEBUG const char* cfile_name;/*!< File name where mutex created */ ulint cline; /*!< Line where created */ -#ifdef UNIV_DEBUG os_thread_id_t thread_id; /*!< The thread id of the thread which locked the mutex. */ ulint magic_n; /*!< MUTEX_MAGIC_N */ @@ -541,9 +541,9 @@ struct mutex_struct { ulong count_os_yield; /*!< count of os_wait */ ulonglong lspent_time; /*!< mutex os_wait timer msec */ ulonglong lmax_spent_time;/*!< mutex os_wait timer msec */ - const char* cmutex_name; /*!< mutex name */ ulint mutex_type; /*!< 0=usual mutex, 1=rw_lock mutex */ #endif /* UNIV_DEBUG */ + const char* cmutex_name; /*!< mutex name */ }; /** The global array of wait cells for implementation of the databases own === modified file 'storage/xtradb/include/thr0loc.h' --- a/storage/xtradb/include/thr0loc.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/thr0loc.h 2010-01-06 12:00:14 +0000 @@ -39,6 +39,12 @@ UNIV_INTERN void thr_local_init(void); /*================*/ + /****************************************************************//** +Close the thread local storage module. */ +UNIV_INTERN +void +thr_local_close(void); +/*=================*/ /*******************************************************************//** Creates a local storage struct for the calling new thread. */ UNIV_INTERN === modified file 'storage/xtradb/include/trx0i_s.h' --- a/storage/xtradb/include/trx0i_s.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/trx0i_s.h 2010-01-06 12:00:14 +0000 @@ -141,6 +141,13 @@ void trx_i_s_cache_init( /*===============*/ trx_i_s_cache_t* cache); /*!< out: cache to init */ +/*******************************************************************//** +Free the INFORMATION SCHEMA trx related cache. */ +UNIV_INTERN +void +trx_i_s_cache_free( +/*===============*/ + trx_i_s_cache_t* cache); /*!< in/out: cache to free */ /*******************************************************************//** Issue a shared/read lock on the tables cache. */ === modified file 'storage/xtradb/include/trx0purge.h' --- a/storage/xtradb/include/trx0purge.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/trx0purge.h 2010-01-06 12:00:14 +0000 @@ -71,6 +71,12 @@ void trx_purge_sys_create(void); /*======================*/ /********************************************************************//** +Frees the global purge system control structure. */ +UNIV_INTERN +void +trx_purge_sys_close(void); +/*======================*/ +/************************************************************************ Adds the update undo log as the first log in the history list. Removes the update undo log segment from the rseg slot if it is too big for reuse. */ UNIV_INTERN @@ -108,6 +114,25 @@ UNIV_INTERN ulint trx_purge(void); /*===========*/ +/********************************************************************** +This function runs a purge worker batch */ +UNIV_INTERN +void +trx_purge_worker( +/*=============*/ + ulint worker_id); +/********************************************************************** +This function waits the event for worker batch */ +UNIV_INTERN +void +trx_purge_worker_wait(void); +/*========================*/ +/********************************************************************** +This function wakes the waiting worker batch */ +UNIV_INTERN +void +trx_purge_worker_wake(void); +/*========================*/ /******************************************************************//** Prints information of the purge system to stderr. */ UNIV_INTERN @@ -125,6 +150,11 @@ struct trx_purge_struct{ of the trx system and it never ends */ que_t* query; /*!< The query graph which will do the parallelized purge operation */ + ulint n_worker; + os_event_t worker_event; + sess_t** sess_arr; + trx_t** trx_arr; + que_t** query_arr; rw_lock_t latch; /*!< The latch protecting the purge view. A purge operation must acquire an x-latch here for the instant at which === modified file 'storage/xtradb/include/trx0rec.h' --- a/storage/xtradb/include/trx0rec.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/trx0rec.h 2010-01-06 12:00:14 +0000 @@ -44,8 +44,8 @@ UNIV_INLINE trx_undo_rec_t* trx_undo_rec_copy( /*==============*/ - trx_undo_rec_t* undo_rec, /*!< in: undo log record */ - mem_heap_t* heap); /*!< in: heap where copied */ + const trx_undo_rec_t* undo_rec, /*!< in: undo log record */ + mem_heap_t* heap); /*!< in: heap where copied */ /**********************************************************************//** Reads the undo log record type. @return record type */ === modified file 'storage/xtradb/include/trx0rec.ic' --- a/storage/xtradb/include/trx0rec.ic 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/trx0rec.ic 2010-01-06 12:00:14 +0000 @@ -100,8 +100,8 @@ UNIV_INLINE trx_undo_rec_t* trx_undo_rec_copy( /*==============*/ - trx_undo_rec_t* undo_rec, /*!< in: undo log record */ - mem_heap_t* heap) /*!< in: heap where copied */ + const trx_undo_rec_t* undo_rec, /*!< in: undo log record */ + mem_heap_t* heap) /*!< in: heap where copied */ { ulint len; === modified file 'storage/xtradb/include/trx0roll.h' --- a/storage/xtradb/include/trx0roll.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/trx0roll.h 2010-01-06 12:00:14 +0000 @@ -133,6 +133,17 @@ trx_rollback( Rollback or clean up any incomplete transactions which were encountered in crash recovery. If the transaction already was committed, then we clean up a possible insert undo log. If the +transaction was not yet committed, then we roll it back. */ +UNIV_INTERN +void +trx_rollback_or_clean_recovered( +/*============================*/ + ibool all); /*!< in: FALSE=roll back dictionary transactions; + TRUE=roll back all non-PREPARED transactions */ +/*******************************************************************//** +Rollback or clean up any incomplete transactions which were +encountered in crash recovery. If the transaction already was +committed, then we clean up a possible insert undo log. If the transaction was not yet committed, then we roll it back. Note: this is done in a background thread. @return a dummy parameter */ @@ -208,9 +219,9 @@ int trx_general_rollback_for_mysql( /*===========================*/ trx_t* trx, /*!< in: transaction handle */ - ibool partial,/*!< in: TRUE if partial rollback requested */ trx_savept_t* savept);/*!< in: pointer to savepoint undo number, if - partial rollback requested */ + partial rollback requested, or NULL for + complete rollback */ /*******************************************************************//** Rolls back a transaction back to a named savepoint. Modifications after the savepoint are undone but InnoDB does NOT release the corresponding locks === modified file 'storage/xtradb/include/trx0rseg.h' --- a/storage/xtradb/include/trx0rseg.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/trx0rseg.h 2010-01-06 12:00:14 +0000 @@ -125,6 +125,13 @@ trx_rseg_create( ulint max_size, /*!< in: max size in pages */ ulint* id, /*!< out: rseg id */ mtr_t* mtr); /*!< in: mtr */ +/*************************************************************************** +Free's an instance of the rollback segment in memory. */ +UNIV_INTERN +void +trx_rseg_mem_free( +/*==============*/ + trx_rseg_t* rseg); /* in, own: instance to free */ /* Real max value may be 4076 in usual. But reserve 4 slot for safety or etc... */ === modified file 'storage/xtradb/include/trx0sys.h' --- a/storage/xtradb/include/trx0sys.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/trx0sys.h 2010-01-06 12:00:14 +0000 @@ -344,6 +344,12 @@ void trx_sys_file_format_tag_init(void); /*==============================*/ /*****************************************************************//** +Shutdown/Close the transaction system. */ +UNIV_INTERN +void +trx_sys_close(void); +/*===============*/ +/*****************************************************************//** Get the name representation of the file format from its id. @return pointer to the name */ UNIV_INTERN === modified file 'storage/xtradb/include/trx0sys.ic' --- a/storage/xtradb/include/trx0sys.ic 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/trx0sys.ic 2010-01-06 12:00:14 +0000 @@ -34,11 +34,11 @@ typedef byte trx_sysf_rseg_t; /* Rollback segment specification slot offsets */ /*-------------------------------------------------------------*/ -#define TRX_SYS_RSEG_SPACE 0 /* space where the the segment +#define TRX_SYS_RSEG_SPACE 0 /* space where the segment header is placed; starting with MySQL/InnoDB 5.1.7, this is UNIV_UNDEFINED if the slot is unused */ -#define TRX_SYS_RSEG_PAGE_NO 4 /* page number where the the segment +#define TRX_SYS_RSEG_PAGE_NO 4 /* page number where the segment header is placed; this is FIL_NULL if the slot is unused */ /*-------------------------------------------------------------*/ === modified file 'storage/xtradb/include/trx0trx.h' --- a/storage/xtradb/include/trx0trx.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/trx0trx.h 2010-01-06 12:00:14 +0000 @@ -179,7 +179,7 @@ trx_commit_off_kernel( /****************************************************************//** Cleans up a transaction at database startup. The cleanup is needed if the transaction already got to the middle of a commit when the database -crashed, andf we cannot roll it back. */ +crashed, and we cannot roll it back. */ UNIV_INTERN void trx_cleanup_at_db_startup( @@ -360,7 +360,7 @@ enum trx_dict_op { operation modes in crash recovery. */ TRX_DICT_OP_TABLE = 1, /** The transaction is creating or dropping an index in an - existing table. In crash recovery, the the data dictionary + existing table. In crash recovery, the data dictionary must be locked, but the table must not be dropped. */ TRX_DICT_OP_INDEX = 2 }; @@ -729,6 +729,17 @@ struct trx_struct{ /*------------------------------*/ char detailed_error[256]; /*!< detailed error message for last error, or empty. */ + /*------------------------------*/ + ulint io_reads; + ib_uint64_t io_read; + ulint io_reads_wait_timer; + ib_uint64_t lock_que_wait_ustarted; + ulint lock_que_wait_timer; + ulint innodb_que_wait_timer; + ulint distinct_page_access; +#define DPAH_SIZE 8192 + byte* distinct_page_access_hash; + ibool take_stats; }; #define TRX_MAX_N_THREADS 32 /* maximum number of === modified file 'storage/xtradb/include/trx0types.h' --- a/storage/xtradb/include/trx0types.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/trx0types.h 2010-01-06 12:00:14 +0000 @@ -70,7 +70,7 @@ typedef struct trx_named_savept_struct t enum trx_rb_ctx { RB_NONE = 0, /*!< no rollback */ RB_NORMAL, /*!< normal rollback */ - RB_RECOVERY, /*!< rolling back an incomplete transaction, + RB_RECOVERY /*!< rolling back an incomplete transaction, in crash recovery */ }; === modified file 'storage/xtradb/include/trx0undo.h' --- a/storage/xtradb/include/trx0undo.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/trx0undo.h 2010-01-06 12:00:14 +0000 @@ -333,6 +333,13 @@ trx_undo_parse_discard_latest( byte* end_ptr,/*!< in: buffer end */ page_t* page, /*!< in: page or NULL */ mtr_t* mtr); /*!< in: mtr or NULL */ +/************************************************************************ +Frees an undo log memory copy. */ +UNIV_INTERN +void +trx_undo_mem_free( +/*==============*/ + trx_undo_t* undo); /* in: the undo object to be freed */ /* Types of an undo log segment */ #define TRX_UNDO_INSERT 1 /* contains undo entries for inserts */ === modified file 'storage/xtradb/include/univ.i' --- a/storage/xtradb/include/univ.i 2009-11-13 21:26:08 +0000 +++ b/storage/xtradb/include/univ.i 2010-01-15 15:58:25 +0000 @@ -46,12 +46,12 @@ Created 1/20/1994 Heikki Tuuri #define INNODB_VERSION_MAJOR 1 #define INNODB_VERSION_MINOR 0 -#define INNODB_VERSION_BUGFIX 4 -#define PERCONA_INNODB_VERSION 8 +#define INNODB_VERSION_BUGFIX 6 +#define PERCONA_INNODB_VERSION 9 /* The following is the InnoDB version as shown in SELECT plugin_version FROM information_schema.plugins; -calculated in in make_version_string() in sql/sql_show.cc like this: +calculated in make_version_string() in sql/sql_show.cc like this: "version >> 8" . "version & 0xff" because the version is shown with only one dot, we skip the last component, i.e. we show M.N.P as M.N */ @@ -59,13 +59,14 @@ component, i.e. we show M.N.P as M.N */ (INNODB_VERSION_MAJOR << 8 | INNODB_VERSION_MINOR) /* auxiliary macros to help creating the version as string */ -#define __INNODB_VERSION(a, b, c, d) (#a "." #b "." #c "-" #d) -#define _INNODB_VERSION(a, b, c, d) __INNODB_VERSION(a, b, c, d) +#define __INNODB_VERSION(a, b, c, d) (#a "." #b "." #c "-" #d) +#define _INNODB_VERSION(a, b, c, d) __INNODB_VERSION(a, b, c, d) + #define INNODB_VERSION_STR \ _INNODB_VERSION(INNODB_VERSION_MAJOR, \ INNODB_VERSION_MINOR, \ - INNODB_VERSION_BUGFIX, \ + INNODB_VERSION_BUGFIX, \ PERCONA_INNODB_VERSION) #define REFMAN "http://dev.mysql.com/doc/refman/5.1/en/" @@ -80,17 +81,25 @@ the virtual method table (vtable) in GCC # define ha_innobase ha_innodb #endif /* MYSQL_DYNAMIC_PLUGIN */ +/* if any of the following macros is defined at this point this means +that the code from the "right" plug.in was executed and we do not +need to include ut0auxconf.h which would either define the same macros +or will be empty */ +#if !defined(HAVE_IB_GCC_ATOMIC_BUILTINS) \ + && !defined(HAVE_IB_ATOMIC_PTHREAD_T_GCC) \ + && !defined(HAVE_IB_SOLARIS_ATOMICS) \ + && !defined(HAVE_IB_ATOMIC_PTHREAD_T_SOLARIS) \ + && !defined(SIZEOF_PTHREAD_T) \ + && !defined(HAVE_IB_PAUSE_INSTRUCTION) +# include "ut0auxconf.h" +#endif + #if (defined(WIN32) || defined(_WIN32) || defined(WIN64) || defined(_WIN64)) && !defined(MYSQL_SERVER) && !defined(__WIN__) # undef __WIN__ # define __WIN__ # include <windows.h> -# if defined(HAVE_WINDOWS_ATOMICS) -/* If atomics are defined we use them in InnoDB mutex implementation */ -# define HAVE_ATOMIC_BUILTINS -# endif /* HAVE_WINDOWS_ATOMICS */ - # ifdef _NT_ # define __NT__ # endif @@ -113,45 +122,17 @@ if we are compiling on Windows. */ # include <sys/mman.h> /* mmap() for os0proc.c */ # endif -# undef PACKAGE -# undef VERSION - /* Include the header file generated by GNU autoconf */ # ifndef __WIN__ -#ifndef UNIV_HOTBACKUP -# include "config.h" -#endif /* UNIV_HOTBACKUP */ +# ifndef UNIV_HOTBACKUP +# include "config.h" +# endif /* UNIV_HOTBACKUP */ # endif # ifdef HAVE_SCHED_H # include <sched.h> # endif -# if defined(HAVE_GCC_ATOMIC_BUILTINS) || defined(HAVE_SOLARIS_ATOMICS) \ - || defined(HAVE_WINDOWS_ATOMICS) -/* If atomics are defined we use them in InnoDB mutex implementation */ -# define HAVE_ATOMIC_BUILTINS -# endif /* (HAVE_GCC_ATOMIC_BUILTINS) || (HAVE_SOLARIS_ATOMICS) - || (HAVE_WINDOWS_ATOMICS) */ - -/* For InnoDB rw_locks to work with atomics we need the thread_id -to be no more than machine word wide. The following enables using -atomics for InnoDB rw_locks where these conditions are met. */ -#ifdef HAVE_ATOMIC_BUILTINS -/* if HAVE_ATOMIC_PTHREAD_T is defined at this point that means that -the code from plug.in has defined it and we do not need to include -ut0auxconf.h which would either define HAVE_ATOMIC_PTHREAD_T or will -be empty */ -# ifndef HAVE_ATOMIC_PTHREAD_T -# include "ut0auxconf.h" -# endif /* HAVE_ATOMIC_PTHREAD_T */ -/* now HAVE_ATOMIC_PTHREAD_T is eventually defined either by plug.in or -from Makefile.in->ut0auxconf.h */ -# ifdef HAVE_ATOMIC_PTHREAD_T -# define INNODB_RW_LOCKS_USE_ATOMICS -# endif /* HAVE_ATOMIC_PTHREAD_T */ -#endif /* HAVE_ATOMIC_BUILTINS */ - /* We only try to do explicit inlining of functions with gcc and Sun Studio */ @@ -198,12 +179,18 @@ command. Not tested on Windows. */ debugging without UNIV_DEBUG */ #define UNIV_DEBUG /* Enable ut_ad() assertions and disable UNIV_INLINE */ +#define UNIV_DEBUG_LOCK_VALIDATE /* Enable + ut_ad(lock_rec_validate_page()) + assertions. */ #define UNIV_DEBUG_FILE_ACCESSES /* Debug .ibd file access (field file_page_was_freed in buf_page_t) */ #define UNIV_LRU_DEBUG /* debug the buffer pool LRU */ #define UNIV_HASH_DEBUG /* debug HASH_ macros */ #define UNIV_LIST_DEBUG /* debug UT_LIST_ macros */ +#define UNIV_LOG_LSN_DEBUG /* write LSN to the redo log; +this will break redo log file compatibility, but it may be useful when +debugging redo log application problems. */ #define UNIV_MEM_DEBUG /* detect memory leaks etc */ #define UNIV_IBUF_DEBUG /* debug the insert buffer */ #define UNIV_IBUF_COUNT_DEBUG /* debug the insert buffer; @@ -253,7 +240,7 @@ by one. */ /* Linkage specifier for non-static InnoDB symbols (variables and functions) that are only referenced from within InnoDB, not from MySQL */ -#if defined(__GNUC__) && (__GNUC__ >= 4) && !defined(UNIV_HOTBACKUP) +#if defined(__GNUC__) && (__GNUC__ >= 4) || defined(__INTEL_COMPILER) # define UNIV_INTERN __attribute__((visibility ("hidden"))) #else # define UNIV_INTERN @@ -410,7 +397,9 @@ it is read. */ /* Minimize cache-miss latency by moving data at addr into a cache before it is read or written. */ # define UNIV_PREFETCH_RW(addr) __builtin_prefetch(addr, 1, 3) -#elif defined(__SUNPRO_C) || defined(__SUNPRO_CC) +/* Sun Studio includes sun_prefetch.h as of version 5.9 */ +#elif (defined(__SUNPRO_C) && __SUNPRO_C >= 0x590) \ + || (defined(__SUNPRO_CC) && __SUNPRO_CC >= 0x590) # include <sun_prefetch.h> #if __SUNPRO_C >= 0x550 # undef UNIV_INTERN === modified file 'storage/xtradb/include/usr0sess.h' --- a/storage/xtradb/include/usr0sess.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/usr0sess.h 2010-01-06 12:00:14 +0000 @@ -44,14 +44,12 @@ sess_t* sess_open(void); /*============*/ /*********************************************************************//** -Closes a session, freeing the memory occupied by it, if it is in a state -where it should be closed. -@return TRUE if closed */ +Closes a session, freeing the memory occupied by it. */ UNIV_INTERN -ibool -sess_try_close( -/*===========*/ - sess_t* sess); /*!< in, own: session object */ +void +sess_close( +/*=======*/ + sess_t* sess); /* in, own: session object */ /* The session handle. All fields are protected by the kernel mutex */ struct sess_struct{ === modified file 'storage/xtradb/include/ut0auxconf.h' --- a/storage/xtradb/include/ut0auxconf.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/ut0auxconf.h 2010-01-06 12:00:14 +0000 @@ -1,14 +1,14 @@ /* Do not remove this file even though it is empty. This file is included in univ.i and will cause compilation failure if not present. -A custom check has been added in the generated +A custom checks have been added in the generated storage/innobase/Makefile.in that is shipped with the InnoDB Plugin -source archive. This check tries to compile a test program and if -successful then adds "#define HAVE_ATOMIC_PTHREAD_T" to this file. -This is a hack that has been developed in order to check for pthread_t -atomicity without the need to regenerate the ./configure script that is +source archive. These checks eventually define some macros and put +them in this file. +This is a hack that has been developed in order to deploy new compile +time checks without the need to regenerate the ./configure script that is distributed in the MySQL 5.1 official source archives. If by any chance Makefile.in and ./configure are regenerated and thus -the hack from Makefile.in wiped away then the "real" check from plug.in +the hack from Makefile.in wiped away then the "real" checks from plug.in will take over. */ === modified file 'storage/xtradb/include/ut0byte.h' --- a/storage/xtradb/include/ut0byte.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/ut0byte.h 2010-01-06 12:00:14 +0000 @@ -219,8 +219,8 @@ UNIV_INLINE void* ut_align( /*=====*/ - void* ptr, /*!< in: pointer */ - ulint align_no); /*!< in: align by this number */ + const void* ptr, /*!< in: pointer */ + ulint align_no); /*!< in: align by this number */ /*********************************************************//** The following function rounds down a pointer to the nearest aligned address. === modified file 'storage/xtradb/include/ut0byte.ic' --- a/storage/xtradb/include/ut0byte.ic 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/ut0byte.ic 2010-01-06 12:00:14 +0000 @@ -319,8 +319,8 @@ UNIV_INLINE void* ut_align( /*=====*/ - void* ptr, /*!< in: pointer */ - ulint align_no) /*!< in: align by this number */ + const void* ptr, /*!< in: pointer */ + ulint align_no) /*!< in: align by this number */ { ut_ad(align_no > 0); ut_ad(((align_no - 1) & align_no) == 0); === modified file 'storage/xtradb/include/ut0ut.h' --- a/storage/xtradb/include/ut0ut.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/ut0ut.h 2010-01-06 12:00:14 +0000 @@ -34,6 +34,11 @@ Created 1/20/1994 Heikki Tuuri #define ut0ut_h #include "univ.i" + +#ifndef UNIV_HOTBACKUP +# include "os0sync.h" /* for HAVE_ATOMIC_BUILTINS */ +#endif /* UNIV_HOTBACKUP */ + #include <time.h> #ifndef MYSQL_SERVER #include <ctype.h> @@ -47,7 +52,8 @@ Created 1/20/1994 Heikki Tuuri /** Time stamp */ typedef time_t ib_time_t; -#if defined(IB_HAVE_PAUSE_INSTRUCTION) +#ifndef UNIV_HOTBACKUP +#if defined(HAVE_IB_PAUSE_INSTRUCTION) # ifdef WIN32 /* In the Win32 API, the x86 PAUSE instruction is executed by calling the YieldProcessor macro defined in WinNT.h. It is a CPU architecture- @@ -84,6 +90,7 @@ do { \ os_thread_sleep(2000 /* 2 ms */); \ } \ } while (0) +#endif /* !UNIV_HOTBACKUP */ /********************************************************//** Gets the high 32 bits in a ulint. That is makes a shift >> 32, @@ -216,6 +223,7 @@ UNIV_INTERN ib_time_t ut_time(void); /*=========*/ +#ifndef UNIV_HOTBACKUP /**********************************************************//** Returns system time. Upon successful completion, the value 0 is returned; otherwise the @@ -239,6 +247,16 @@ ullint ut_time_us( /*=======*/ ullint* tloc); /*!< out: us since epoch, if non-NULL */ +/**********************************************************//** +Returns the number of milliseconds since some epoch. The +value may wrap around. It should only be used for heuristic +purposes. +@return ms since epoch */ +UNIV_INTERN +ulint +ut_time_ms(void); +/*============*/ +#endif /* !UNIV_HOTBACKUP */ /**********************************************************//** Returns the difference of two times in seconds. === modified file 'storage/xtradb/lock/lock0lock.c' --- a/storage/xtradb/lock/lock0lock.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/lock/lock0lock.c 2010-01-06 12:00:14 +0000 @@ -214,7 +214,7 @@ a waiting s-lock request on the next rec by a read cursor moving in the ascending order in the index, we cannot do the insert immediately, because when we finally commit our transaction, the read cursor should see also the new inserted record. So we should -move the read cursor backward from the the next record for it to pass over +move the read cursor backward from the next record for it to pass over the new inserted record. This move backward may be too cumbersome to implement. If we in this situation just enqueue a second x-lock request for our transaction on the next record, then the deadlock mechanism @@ -360,10 +360,9 @@ ibool lock_rec_validate_page( /*===================*/ ulint space, /*!< in: space id */ + ulint zip_size,/*!< in: compressed page size in bytes + or 0 for uncompressed pages */ ulint page_no);/*!< in: page number */ - -/* Define the following in order to enable lock_rec_validate_page() checks. */ -# undef UNIV_DEBUG_LOCK_VALIDATE #endif /* UNIV_DEBUG */ /* The lock system */ @@ -579,6 +578,23 @@ lock_sys_create( } /*********************************************************************//** +Closes the lock system at database shutdown. */ +UNIV_INTERN +void +lock_sys_close(void) +/*================*/ +{ + if (lock_latest_err_file != NULL) { + fclose(lock_latest_err_file); + lock_latest_err_file = NULL; + } + + hash_table_free(lock_sys->rec_hash); + mem_free(lock_sys); + lock_sys = NULL; +} + +/*********************************************************************//** Gets the size of a lock struct. @return size in bytes */ UNIV_INTERN @@ -1739,6 +1755,8 @@ lock_rec_enqueue_waiting( { lock_t* lock; trx_t* trx; + ulint sec; + ulint ms; ut_ad(mutex_own(&kernel_mutex)); @@ -1797,6 +1815,10 @@ lock_rec_enqueue_waiting( trx->que_state = TRX_QUE_LOCK_WAIT; trx->was_chosen_as_deadlock_victim = FALSE; trx->wait_started = time(NULL); + if (innobase_get_slow_log() && trx->take_stats) { + ut_usectime(&sec, &ms); + trx->lock_que_wait_ustarted = (ib_uint64_t)sec * 1000000 + ms; + } ut_a(que_thr_stop(thr)); @@ -2622,6 +2644,7 @@ lock_move_reorganize_page( #ifdef UNIV_DEBUG_LOCK_VALIDATE ut_ad(lock_rec_validate_page(buf_block_get_space(block), + buf_block_get_zip_size(block), buf_block_get_page_no(block))); #endif } @@ -2711,8 +2734,10 @@ lock_move_rec_list_end( #ifdef UNIV_DEBUG_LOCK_VALIDATE ut_ad(lock_rec_validate_page(buf_block_get_space(block), + buf_block_get_zip_size(block), buf_block_get_page_no(block))); ut_ad(lock_rec_validate_page(buf_block_get_space(new_block), + buf_block_get_zip_size(block), buf_block_get_page_no(new_block))); #endif } @@ -2822,6 +2847,7 @@ lock_move_rec_list_start( #ifdef UNIV_DEBUG_LOCK_VALIDATE ut_ad(lock_rec_validate_page(buf_block_get_space(block), + buf_block_get_zip_size(block), buf_block_get_page_no(block))); #endif } @@ -3574,7 +3600,8 @@ lock_table_remove_low( and lock_grant()). Therefore it can be empty and we need to check for that. */ - if (!ib_vector_is_empty(trx->autoinc_locks)) { + if (!lock_get_wait(lock) + && !ib_vector_is_empty(trx->autoinc_locks)) { lock_t* autoinc_lock; autoinc_lock = ib_vector_pop(trx->autoinc_locks); @@ -3607,6 +3634,8 @@ lock_table_enqueue_waiting( { lock_t* lock; trx_t* trx; + ulint sec; + ulint ms; ut_ad(mutex_own(&kernel_mutex)); @@ -3647,8 +3676,10 @@ lock_table_enqueue_waiting( if (lock_deadlock_occurs(lock, trx)) { - lock_reset_lock_and_trx_wait(lock); + /* The order here is important, we don't want to + lose the state of the lock before calling remove. */ lock_table_remove_low(lock); + lock_reset_lock_and_trx_wait(lock); return(DB_DEADLOCK); } @@ -3660,6 +3691,10 @@ lock_table_enqueue_waiting( return(DB_SUCCESS); } + if (innobase_get_slow_log() && trx->take_stats) { + ut_usectime(&sec, &ms); + trx->lock_que_wait_ustarted = (ib_uint64_t)sec * 1000000 + ms; + } trx->que_state = TRX_QUE_LOCK_WAIT; trx->was_chosen_as_deadlock_victim = FALSE; trx->wait_started = time(NULL); @@ -4627,6 +4662,10 @@ lock_rec_queue_validate( next function call: we have to release lock table mutex to obey the latching order */ + /* If this thread is holding the file space latch + (fil_space_t::latch), the following check WILL break + latching order and may cause a deadlock of threads. */ + impl_trx = lock_sec_rec_some_has_impl_off_kernel( rec, index, offsets); @@ -4684,6 +4723,8 @@ ibool lock_rec_validate_page( /*===================*/ ulint space, /*!< in: space id */ + ulint zip_size,/*!< in: compressed page size in bytes + or 0 for uncompressed pages */ ulint page_no)/*!< in: page number */ { dict_index_t* index; @@ -4694,7 +4735,6 @@ lock_rec_validate_page( ulint nth_lock = 0; ulint nth_bit = 0; ulint i; - ulint zip_size; mtr_t mtr; mem_heap_t* heap = NULL; ulint offsets_[REC_OFFS_NORMAL_SIZE]; @@ -4705,7 +4745,6 @@ lock_rec_validate_page( mtr_start(&mtr); - zip_size = fil_space_get_zip_size(space); ut_ad(zip_size != ULINT_UNDEFINED); block = buf_page_get(space, zip_size, page_no, RW_X_LATCH, &mtr); buf_block_dbg_add_level(block, SYNC_NO_ORDER_CHECK); @@ -4750,6 +4789,11 @@ loop: lock_mutex_exit_kernel(); + /* If this thread is holding the file space + latch (fil_space_t::latch), the following + check WILL break the latching order and may + cause a deadlock of threads. */ + lock_rec_queue_validate(block, rec, index, offsets); lock_mutex_enter_kernel(); @@ -4840,7 +4884,9 @@ lock_validate(void) lock_mutex_exit_kernel(); - lock_rec_validate_page(space, page_no); + lock_rec_validate_page(space, + fil_space_get_zip_size(space), + page_no); lock_mutex_enter_kernel(); @@ -5364,6 +5410,20 @@ lock_release_autoinc_last_lock( } /*******************************************************************//** +Check if a transaction holds any autoinc locks. +@return TRUE if the transaction holds any AUTOINC locks. */ +UNIV_INTERN +ibool +lock_trx_holds_autoinc_locks( +/*=========================*/ + const trx_t* trx) /*!< in: transaction */ +{ + ut_a(trx->autoinc_locks != NULL); + + return(!ib_vector_is_empty(trx->autoinc_locks)); +} + +/*******************************************************************//** Release all the transaction's autoinc locks. */ UNIV_INTERN void === modified file 'storage/xtradb/log/log0log.c' --- a/storage/xtradb/log/log0log.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/log/log0log.c 2010-01-06 12:00:14 +0000 @@ -241,6 +241,7 @@ log_reserve_and_open( ut_a(len < log->buf_size / 2); loop: mutex_enter(&(log->mutex)); + ut_ad(!recv_no_log_write); /* Calculate an upper limit for the space the string may take in the log buffer */ @@ -309,6 +310,7 @@ log_write_low( ut_ad(mutex_own(&(log->mutex))); part_loop: + ut_ad(!recv_no_log_write); /* Calculate a part length */ data_len = (log->buf_free % OS_FILE_LOG_BLOCK_SIZE) + str_len; @@ -362,6 +364,33 @@ part_loop: } /************************************************************//** +*/ +UNIV_INLINE +ulint +log_max_modified_age_async() +{ + if (srv_checkpoint_age_target) { + return(ut_min(log_sys->max_modified_age_async, + srv_checkpoint_age_target + - srv_checkpoint_age_target / 8)); + } else { + return(log_sys->max_modified_age_async); + } +} + +UNIV_INLINE +ulint +log_max_checkpoint_age_async() +{ + if (srv_checkpoint_age_target) { + return(ut_min(log_sys->max_checkpoint_age_async, + srv_checkpoint_age_target)); + } else { + return(log_sys->max_checkpoint_age_async); + } +} + +/************************************************************//** Closes the log. @return lsn */ UNIV_INTERN @@ -377,6 +406,7 @@ log_close(void) ib_uint64_t checkpoint_age; ut_ad(mutex_own(&(log->mutex))); + ut_ad(!recv_no_log_write); lsn = log->lsn; @@ -429,7 +459,7 @@ log_close(void) } } - if (checkpoint_age <= log->max_modified_age_async) { + if (checkpoint_age <= log_max_modified_age_async()) { goto function_exit; } @@ -437,8 +467,8 @@ log_close(void) oldest_lsn = buf_pool_get_oldest_modification(); if (!oldest_lsn - || lsn - oldest_lsn > log->max_modified_age_async - || checkpoint_age > log->max_checkpoint_age_async) { + || lsn - oldest_lsn > log_max_modified_age_async() + || checkpoint_age > log_max_checkpoint_age_async()) { log->check_flush_or_checkpoint = TRUE; } @@ -668,8 +698,6 @@ log_calc_max_ages(void) ulint archive_margin; ulint smallest_archive_margin; - ut_ad(!mutex_own(&(log_sys->mutex))); - mutex_enter(&(log_sys->mutex)); group = UT_LIST_GET_FIRST(log_sys->log_groups); @@ -770,8 +798,6 @@ void log_init(void) /*==========*/ { - byte* buf; - log_sys = mem_alloc(sizeof(log_t)); mutex_create(&log_sys->mutex, SYNC_LOG); @@ -786,8 +812,8 @@ log_init(void) ut_a(LOG_BUFFER_SIZE >= 16 * OS_FILE_LOG_BLOCK_SIZE); ut_a(LOG_BUFFER_SIZE >= 4 * UNIV_PAGE_SIZE); - buf = mem_alloc(LOG_BUFFER_SIZE + OS_FILE_LOG_BLOCK_SIZE); - log_sys->buf = ut_align(buf, OS_FILE_LOG_BLOCK_SIZE); + log_sys->buf_ptr = mem_alloc(LOG_BUFFER_SIZE + OS_FILE_LOG_BLOCK_SIZE); + log_sys->buf = ut_align(log_sys->buf_ptr, OS_FILE_LOG_BLOCK_SIZE); log_sys->buf_size = LOG_BUFFER_SIZE; @@ -832,9 +858,9 @@ log_init(void) rw_lock_create(&log_sys->checkpoint_lock, SYNC_NO_ORDER_CHECK); - log_sys->checkpoint_buf - = ut_align(mem_alloc(2 * OS_FILE_LOG_BLOCK_SIZE), - OS_FILE_LOG_BLOCK_SIZE); + log_sys->checkpoint_buf_ptr = mem_alloc(2 * OS_FILE_LOG_BLOCK_SIZE); + log_sys->checkpoint_buf = ut_align(log_sys->checkpoint_buf_ptr, + OS_FILE_LOG_BLOCK_SIZE); memset(log_sys->checkpoint_buf, '\0', OS_FILE_LOG_BLOCK_SIZE); /*----------------------------*/ @@ -917,23 +943,33 @@ log_group_init( group->lsn_offset = LOG_FILE_HDR_SIZE; group->n_pending_writes = 0; + group->file_header_bufs_ptr = mem_alloc(sizeof(byte*) * n_files); group->file_header_bufs = mem_alloc(sizeof(byte*) * n_files); #ifdef UNIV_LOG_ARCHIVE + group->archive_file_header_bufs_ptr = mem_alloc( + sizeof(byte*) * n_files); group->archive_file_header_bufs = mem_alloc(sizeof(byte*) * n_files); #endif /* UNIV_LOG_ARCHIVE */ for (i = 0; i < n_files; i++) { - *(group->file_header_bufs + i) = ut_align( - mem_alloc(LOG_FILE_HDR_SIZE + OS_FILE_LOG_BLOCK_SIZE), + group->file_header_bufs_ptr[i] = mem_alloc( + LOG_FILE_HDR_SIZE + OS_FILE_LOG_BLOCK_SIZE); + + group->file_header_bufs[i] = ut_align( + group->file_header_bufs_ptr[i], OS_FILE_LOG_BLOCK_SIZE); memset(*(group->file_header_bufs + i), '\0', LOG_FILE_HDR_SIZE); #ifdef UNIV_LOG_ARCHIVE - *(group->archive_file_header_bufs + i) = ut_align( - mem_alloc(LOG_FILE_HDR_SIZE + OS_FILE_LOG_BLOCK_SIZE), + group->archive_file_header_bufs_ptr[i] = mem_alloc( + LOG_FILE_HDR_SIZE + OS_FILE_LOG_BLOCK_SIZE); + + group->archive_file_header_bufs[i] = ut_align( + group->archive_file_header_bufs_ptr[i], OS_FILE_LOG_BLOCK_SIZE); + memset(*(group->archive_file_header_bufs + i), '\0', LOG_FILE_HDR_SIZE); #endif /* UNIV_LOG_ARCHIVE */ @@ -946,8 +982,9 @@ log_group_init( group->archived_offset = 0; #endif /* UNIV_LOG_ARCHIVE */ - group->checkpoint_buf = ut_align( - mem_alloc(2 * OS_FILE_LOG_BLOCK_SIZE), OS_FILE_LOG_BLOCK_SIZE); + group->checkpoint_buf_ptr = mem_alloc(2 * OS_FILE_LOG_BLOCK_SIZE); + group->checkpoint_buf = ut_align(group->checkpoint_buf_ptr, + OS_FILE_LOG_BLOCK_SIZE); memset(group->checkpoint_buf, '\0', OS_FILE_LOG_BLOCK_SIZE); @@ -1117,6 +1154,7 @@ log_io_complete( } mutex_enter(&(log_sys->mutex)); + ut_ad(!recv_no_log_write); ut_a(group->n_pending_writes > 0); ut_a(log_sys->n_pending_writes > 0); @@ -1148,6 +1186,7 @@ log_group_file_header_flush( ulint dest_offset; ut_ad(mutex_own(&(log_sys->mutex))); + ut_ad(!recv_no_log_write); ut_a(nth_file < group->n_files); buf = *(group->file_header_bufs + nth_file); @@ -1219,6 +1258,7 @@ log_group_write_buf( ulint i; ut_ad(mutex_own(&(log_sys->mutex))); + ut_ad(!recv_no_log_write); ut_a(len % OS_FILE_LOG_BLOCK_SIZE == 0); ut_a(((ulint) start_lsn) % OS_FILE_LOG_BLOCK_SIZE == 0); @@ -1361,6 +1401,7 @@ loop: #endif mutex_enter(&(log_sys->mutex)); + ut_ad(!recv_no_log_write); if (flush_to_disk && log_sys->flushed_to_disk_lsn >= lsn) { @@ -1974,6 +2015,7 @@ log_checkpoint( mutex_enter(&(log_sys->mutex)); + ut_ad(!recv_no_log_write); oldest_lsn = log_buf_pool_get_oldest_modification(); mutex_exit(&(log_sys->mutex)); @@ -2047,7 +2089,7 @@ log_make_checkpoint_at( later lsn, if IB_ULONGLONG_MAX, makes a checkpoint at the latest lsn */ ibool write_always) /*!< in: the function normally checks if - the the new checkpoint would have a + the new checkpoint would have a greater lsn than the previous one: if not, then no physical write is done; by setting this parameter TRUE, a @@ -2086,6 +2128,7 @@ loop: do_checkpoint = FALSE; mutex_enter(&(log->mutex)); + ut_ad(!recv_no_log_write); if (log->check_flush_or_checkpoint == FALSE) { mutex_exit(&(log->mutex)); @@ -2103,10 +2146,10 @@ loop: sync = TRUE; advance = 2 * (age - log->max_modified_age_sync); - } else if (age > log->max_modified_age_async) { + } else if (age > log_max_modified_age_async()) { /* A flush is not urgent: we do an asynchronous preflush */ - advance = age - log->max_modified_age_async; + advance = age - log_max_modified_age_async(); } else { advance = 0; } @@ -2120,7 +2163,7 @@ loop: do_checkpoint = TRUE; - } else if (checkpoint_age > log->max_checkpoint_age_async) { + } else if (checkpoint_age > log_max_checkpoint_age_async()) { /* A checkpoint is not urgent: do it asynchronously */ do_checkpoint = TRUE; @@ -3035,6 +3078,7 @@ loop: #endif /* UNIV_LOG_ARCHIVE */ mutex_enter(&(log_sys->mutex)); + ut_ad(!recv_no_log_write); if (log_sys->check_flush_or_checkpoint) { @@ -3120,6 +3164,16 @@ loop: goto loop; } + /* Check that the purge threads ended */ + if (srv_use_purge_thread + && (srv_n_threads_active[SRV_PURGE] != 0 + || srv_n_threads_active[SRV_PURGE_WORKER] != 0)) { + + mutex_exit(&kernel_mutex); + + goto loop; + } + mutex_exit(&kernel_mutex); mutex_enter(&(log_sys->mutex)); @@ -3234,6 +3288,7 @@ loop: ut_a(lsn == log_sys->lsn); } +#ifdef UNIV_LOG_DEBUG /******************************************************//** Checks by parsing that the catenated log segment for a single mtr is consistent. */ @@ -3241,7 +3296,7 @@ UNIV_INTERN ibool log_check_log_recs( /*===============*/ - byte* buf, /*!< in: pointer to the start of + const byte* buf, /*!< in: pointer to the start of the log segment in the log_sys->buf log buffer */ ulint len, /*!< in: segment length in bytes */ @@ -3249,8 +3304,8 @@ log_check_log_recs( { ib_uint64_t contiguous_lsn; ib_uint64_t scanned_lsn; - byte* start; - byte* end; + const byte* start; + const byte* end; byte* buf1; byte* scan_buf; @@ -3283,6 +3338,7 @@ log_check_log_recs( return(TRUE); } +#endif /* UNIV_LOG_DEBUG */ /******************************************************//** Peeks the current lsn. @@ -3326,10 +3382,12 @@ log_print( log_sys->last_checkpoint_lsn); fprintf(file, - "Max checkpoint age %lu\n" - "Modified age %lu\n" - "Checkpoint age %lu\n", + "Max checkpoint age %lu\n" + "Checkpoint age target %lu\n" + "Modified age %lu\n" + "Checkpoint age %lu\n", (ulong) log_sys->max_checkpoint_age, + (ulong) log_max_checkpoint_age_async(), (ulong) (log_sys->lsn - log_buf_pool_get_oldest_modification()), (ulong) (log_sys->lsn - log_sys->last_checkpoint_lsn)); @@ -3363,4 +3421,95 @@ log_refresh_stats(void) log_sys->n_log_ios_old = log_sys->n_log_ios; log_sys->last_printout_time = time(NULL); } + +/********************************************************************** +Closes a log group. */ +static +void +log_group_close( +/*===========*/ + log_group_t* group) /* in,own: log group to close */ +{ + ulint i; + + for (i = 0; i < group->n_files; i++) { + mem_free(group->file_header_bufs_ptr[i]); +#ifdef UNIV_LOG_ARCHIVE + mem_free(group->archive_file_header_bufs_ptr[i]); +#endif /* UNIV_LOG_ARCHIVE */ + } + + mem_free(group->file_header_bufs_ptr); + mem_free(group->file_header_bufs); + +#ifdef UNIV_LOG_ARCHIVE + mem_free(group->archive_file_header_bufs_ptr); + mem_free(group->archive_file_header_bufs); +#endif /* UNIV_LOG_ARCHIVE */ + + mem_free(group->checkpoint_buf_ptr); + + mem_free(group); +} + +/********************************************************** +Shutdown the log system but do not release all the memory. */ +UNIV_INTERN +void +log_shutdown(void) +/*==============*/ +{ + log_group_t* group; + + group = UT_LIST_GET_FIRST(log_sys->log_groups); + + while (UT_LIST_GET_LEN(log_sys->log_groups) > 0) { + log_group_t* prev_group = group; + + group = UT_LIST_GET_NEXT(log_groups, group); + UT_LIST_REMOVE(log_groups, log_sys->log_groups, prev_group); + + log_group_close(prev_group); + } + + mem_free(log_sys->buf_ptr); + log_sys->buf_ptr = NULL; + log_sys->buf = NULL; + mem_free(log_sys->checkpoint_buf_ptr); + log_sys->checkpoint_buf_ptr = NULL; + log_sys->checkpoint_buf = NULL; + + os_event_free(log_sys->no_flush_event); + os_event_free(log_sys->one_flushed_event); + + rw_lock_free(&log_sys->checkpoint_lock); + + mutex_free(&log_sys->mutex); + +#ifdef UNIV_LOG_ARCHIVE + rw_lock_free(&log_sys->archive_lock); + os_event_create(log_sys->archiving_on); +#endif /* UNIV_LOG_ARCHIVE */ + +#ifdef UNIV_LOG_DEBUG + recv_sys_debug_free(); +#endif + + recv_sys_close(); +} + +/********************************************************** +Free the log system data structures. */ +UNIV_INTERN +void +log_mem_free(void) +/*==============*/ +{ + if (log_sys != NULL) { + recv_sys_mem_free(); + mem_free(log_sys); + + log_sys = NULL; + } +} #endif /* !UNIV_HOTBACKUP */ === modified file 'storage/xtradb/log/log0recv.c' --- a/storage/xtradb/log/log0recv.c 2009-11-13 21:26:08 +0000 +++ b/storage/xtradb/log/log0recv.c 2010-01-15 15:58:25 +0000 @@ -69,20 +69,25 @@ UNIV_INTERN recv_sys_t* recv_sys = NULL; /** TRUE when applying redo log records during crash recovery; FALSE otherwise. Note that this is FALSE while a background thread is rolling back incomplete transactions. */ -UNIV_INTERN ibool recv_recovery_on = FALSE; +UNIV_INTERN ibool recv_recovery_on; #ifdef UNIV_LOG_ARCHIVE /** TRUE when applying redo log records from an archived log file */ -UNIV_INTERN ibool recv_recovery_from_backup_on = FALSE; +UNIV_INTERN ibool recv_recovery_from_backup_on; #endif /* UNIV_LOG_ARCHIVE */ #ifndef UNIV_HOTBACKUP /** TRUE when recv_init_crash_recovery() has been called. */ -UNIV_INTERN ibool recv_needed_recovery = FALSE; +UNIV_INTERN ibool recv_needed_recovery; +# ifdef UNIV_DEBUG +/** TRUE if writing to the redo log (mtr_commit) is forbidden. +Protected by log_sys->mutex. */ +UNIV_INTERN ibool recv_no_log_write = FALSE; +# endif /* UNIV_DEBUG */ /** TRUE if buf_page_is_corrupted() should check if the log sequence number (FIL_PAGE_LSN) is in the future. Initially FALSE, and set by recv_recovery_from_checkpoint_start_func(). */ -UNIV_INTERN ibool recv_lsn_checks_on = FALSE; +UNIV_INTERN ibool recv_lsn_checks_on; /** There are two conditions under which we scan the logs, the first is normal startup and the second is when we do a recovery from an @@ -92,7 +97,7 @@ startup. If we find log entries that wer we know that the server was not cleanly shutdown. We must then initialize the crash recovery environment before attempting to store these entries in the log hash table. */ -static ibool recv_log_scan_is_startup_type = FALSE; +static ibool recv_log_scan_is_startup_type; /** If the following is TRUE, the buffer pool file pages must be invalidated after recovery and no ibuf operations are allowed; this becomes TRUE if @@ -103,7 +108,7 @@ buffer pool before the pages have been r TRUE means that recovery is running and no operations on the log files are allowed yet: the variable name is misleading. */ -UNIV_INTERN ibool recv_no_ibuf_operations = FALSE; +UNIV_INTERN ibool recv_no_ibuf_operations; /** TRUE when the redo log is being backed up */ # define recv_is_making_a_backup FALSE /** TRUE when recovering from a backed up redo log file */ @@ -111,30 +116,30 @@ UNIV_INTERN ibool recv_no_ibuf_operation #else /* !UNIV_HOTBACKUP */ # define recv_needed_recovery FALSE /** TRUE when the redo log is being backed up */ -UNIV_INTERN ibool recv_is_making_a_backup = FALSE; +UNIV_INTERN ibool recv_is_making_a_backup = FALSE; /** TRUE when recovering from a backed up redo log file */ UNIV_INTERN ibool recv_is_from_backup = FALSE; # define buf_pool_get_curr_size() (5 * 1024 * 1024) #endif /* !UNIV_HOTBACKUP */ /** The following counter is used to decide when to print info on log scan */ -static ulint recv_scan_print_counter = 0; +static ulint recv_scan_print_counter; /** The type of the previous parsed redo log record */ -static ulint recv_previous_parsed_rec_type = 999999; +static ulint recv_previous_parsed_rec_type; /** The offset of the previous parsed redo log record */ -static ulint recv_previous_parsed_rec_offset = 0; +static ulint recv_previous_parsed_rec_offset; /** The 'multi' flag of the previous parsed redo log record */ -static ulint recv_previous_parsed_rec_is_multi = 0; +static ulint recv_previous_parsed_rec_is_multi; /** Maximum page number encountered in the redo log */ -UNIV_INTERN ulint recv_max_parsed_page_no = 0; +UNIV_INTERN ulint recv_max_parsed_page_no; /** This many frames must be left free in the buffer pool when we scan the log and store the scanned log records in the buffer pool: we will use these free frames to read in pages when we start applying the log records to the database. */ -UNIV_INTERN ulint recv_n_pool_free_frames = 1024; +UNIV_INTERN ulint recv_n_pool_free_frames; /** The maximum lsn we see for a page during the recovery process. If this is bigger than the lsn we are able to scan up to, that is an indication that @@ -165,15 +170,119 @@ recv_sys_create(void) return; } - recv_sys = mem_alloc(sizeof(recv_sys_t)); + recv_sys = mem_alloc(sizeof(*recv_sys)); + memset(recv_sys, 0x0, sizeof(*recv_sys)); mutex_create(&recv_sys->mutex, SYNC_RECV); recv_sys->heap = NULL; recv_sys->addr_hash = NULL; + + recv_sys->stats_recv_start_time = time(NULL); + recv_sys->stats_oldest_modified_lsn = IB_ULONGLONG_MAX; } /********************************************************//** +Release recovery system mutexes. */ +UNIV_INTERN +void +recv_sys_close(void) +/*================*/ +{ + if (recv_sys != NULL) { + if (recv_sys->addr_hash != NULL) { + hash_table_free(recv_sys->addr_hash); + } + + if (recv_sys->heap != NULL) { + mem_heap_free(recv_sys->heap); + } + + if (recv_sys->buf != NULL) { + ut_free(recv_sys->buf); + } + + if (recv_sys->last_block_buf_start != NULL) { + mem_free(recv_sys->last_block_buf_start); + } + + mutex_free(&recv_sys->mutex); + + mem_free(recv_sys); + recv_sys = NULL; + } +} + +/********************************************************//** +Frees the recovery system memory. */ +UNIV_INTERN +void +recv_sys_mem_free(void) +/*===================*/ +{ + if (recv_sys != NULL) { + if (recv_sys->addr_hash != NULL) { + hash_table_free(recv_sys->addr_hash); + } + + if (recv_sys->heap != NULL) { + mem_heap_free(recv_sys->heap); + } + + if (recv_sys->buf != NULL) { + ut_free(recv_sys->buf); + } + + if (recv_sys->last_block_buf_start != NULL) { + mem_free(recv_sys->last_block_buf_start); + } + + mem_free(recv_sys); + recv_sys = NULL; + } +} + +/************************************************************ +Reset the state of the recovery system variables. */ +UNIV_INTERN +void +recv_sys_var_init(void) +/*===================*/ +{ + recv_lsn_checks_on = FALSE; + + recv_n_pool_free_frames = 1024; + + recv_recovery_on = FALSE; + +#ifdef UNIV_LOG_ARCHIVE + recv_recovery_from_backup_on = FALSE; +#endif /* UNIV_LOG_ARCHIVE */ + + recv_needed_recovery = FALSE; + + recv_lsn_checks_on = FALSE; + + recv_log_scan_is_startup_type = FALSE; + + recv_no_ibuf_operations = FALSE; + + recv_scan_print_counter = 0; + + recv_previous_parsed_rec_type = 999999; + + recv_previous_parsed_rec_offset = 0; + + recv_previous_parsed_rec_is_multi = 0; + + recv_max_parsed_page_no = 0; + + recv_n_pool_free_frames = 1024; + + recv_max_page_lsn = 0; +} + +/************************************************************ Inits the recovery system for a recovery operation. */ UNIV_INTERN void @@ -248,8 +357,8 @@ recv_sys_empty_hash(void) Frees the recovery system. */ static void -recv_sys_free(void) -/*===============*/ +recv_sys_debug_free(void) +/*=====================*/ { mutex_enter(&(recv_sys->mutex)); @@ -258,8 +367,10 @@ recv_sys_free(void) ut_free(recv_sys->buf); mem_free(recv_sys->last_block_buf_start); - recv_sys->addr_hash = NULL; + recv_sys->buf = NULL; recv_sys->heap = NULL; + recv_sys->addr_hash = NULL; + recv_sys->last_block_buf_start = NULL; mutex_exit(&(recv_sys->mutex)); } @@ -853,6 +964,11 @@ recv_parse_or_apply_log_rec_body( } switch (type) { +#ifdef UNIV_LOG_LSN_DEBUG + case MLOG_LSN: + /* The LSN is checked in recv_parse_log_rec(). */ + break; +#endif /* UNIV_LOG_LSN_DEBUG */ case MLOG_1BYTE: case MLOG_2BYTES: case MLOG_4BYTES: case MLOG_8BYTES: #ifdef UNIV_DEBUG if (page && page_type == FIL_PAGE_TYPE_ALLOCATED @@ -1223,6 +1339,11 @@ recv_add_to_hash_table( len = rec_end - body; + if (srv_recovery_stats) { + recv_sys->stats_log_recs++; + recv_sys->stats_log_len_sum += len; + } + recv = mem_heap_alloc(recv_sys->heap, sizeof(recv_t)); recv->type = type; recv->len = rec_end - body; @@ -1269,7 +1390,7 @@ recv_add_to_hash_table( sizeof(recv_data_t) + len); *prev_field = recv_data; - ut_memcpy(((byte*)recv_data) + sizeof(recv_data_t), body, len); + memcpy(recv_data + 1, body, len); prev_field = &(recv_data->next); @@ -1327,12 +1448,14 @@ recv_recover_page_func( buf_block_t* block) /*!< in/out: buffer block */ { page_t* page; + page_zip_des_t* page_zip; recv_addr_t* recv_addr; recv_t* recv; byte* buf; ib_uint64_t start_lsn; ib_uint64_t end_lsn; ib_uint64_t page_lsn; + ib_uint64_t page_lsn_orig; ib_uint64_t page_newest_lsn; ibool modification_to_page; #ifndef UNIV_HOTBACKUP @@ -1372,12 +1495,21 @@ recv_recover_page_func( recv_addr->state = RECV_BEING_PROCESSED; + if (srv_recovery_stats) { + if (just_read_in) { + recv_sys->stats_recover_pages_with_read++; + } else { + recv_sys->stats_recover_pages_without_read++; + } + } + mutex_exit(&(recv_sys->mutex)); mtr_start(&mtr); mtr_set_log_mode(&mtr, MTR_LOG_NONE); page = block->frame; + page_zip = buf_block_get_page_zip(block); #ifndef UNIV_HOTBACKUP if (just_read_in) { @@ -1400,6 +1532,7 @@ recv_recover_page_func( /* Read the newest modification lsn from the page */ page_lsn = mach_read_ull(page + FIL_PAGE_LSN); + page_lsn_orig = page_lsn; #ifndef UNIV_HOTBACKUP /* It may be that the page has been modified in the buffer @@ -1419,6 +1552,21 @@ recv_recover_page_func( modification_to_page = FALSE; start_lsn = end_lsn = 0; + if (srv_recovery_stats) { + mutex_enter(&(recv_sys->mutex)); + if (page_lsn_orig && recv_sys->stats_oldest_modified_lsn > page_lsn_orig) { + recv_sys->stats_oldest_modified_lsn = page_lsn_orig; + } + if (page_lsn_orig && recv_sys->stats_newest_modified_lsn < page_lsn_orig) { + recv_sys->stats_newest_modified_lsn = page_lsn_orig; + } + if (UT_LIST_GET_LAST(recv_addr->rec_list)->start_lsn + < page_lsn_orig) { + recv_sys->stats_pages_already_new++; + } + mutex_exit(&(recv_sys->mutex)); + } + recv = UT_LIST_GET_FIRST(recv_addr->rec_list); while (recv) { @@ -1438,13 +1586,19 @@ recv_recover_page_func( if (recv->type == MLOG_INIT_FILE_PAGE) { page_lsn = page_newest_lsn; - mach_write_ull(page + UNIV_PAGE_SIZE - - FIL_PAGE_END_LSN_OLD_CHKSUM, 0); - mach_write_ull(page + FIL_PAGE_LSN, 0); + memset(FIL_PAGE_LSN + page, 0, 8); + memset(UNIV_PAGE_SIZE - FIL_PAGE_END_LSN_OLD_CHKSUM + + page, 0, 8); + + if (page_zip) { + memset(FIL_PAGE_LSN + page_zip->data, 0, 8); + } } if (recv->start_lsn >= page_lsn) { + ib_uint64_t end_lsn; + if (!modification_to_page) { modification_to_page = TRUE; @@ -1466,11 +1620,24 @@ recv_recover_page_func( recv_parse_or_apply_log_rec_body(recv->type, buf, buf + recv->len, block, &mtr); - mach_write_ull(page + UNIV_PAGE_SIZE - - FIL_PAGE_END_LSN_OLD_CHKSUM, - recv->start_lsn + recv->len); - mach_write_ull(page + FIL_PAGE_LSN, - recv->start_lsn + recv->len); + + if (srv_recovery_stats) { + mutex_enter(&(recv_sys->mutex)); + recv_sys->stats_applied_log_recs++; + recv_sys->stats_applied_log_len_sum += recv->len; + mutex_exit(&(recv_sys->mutex)); + } + + end_lsn = recv->start_lsn + recv->len; + mach_write_ull(FIL_PAGE_LSN + page, end_lsn); + mach_write_ull(UNIV_PAGE_SIZE + - FIL_PAGE_END_LSN_OLD_CHKSUM + + page, end_lsn); + + if (page_zip) { + mach_write_ull(FIL_PAGE_LSN + + page_zip->data, end_lsn); + } } if (recv->len > RECV_DATA_BLOCK_SIZE) { @@ -1561,6 +1728,13 @@ recv_read_in_area( } } + if (srv_recovery_stats && n) { + mutex_enter(&(recv_sys->mutex)); + recv_sys->stats_read_requested_pages += n; + recv_sys->stats_read_in_area[n - 1]++; + mutex_exit(&(recv_sys->mutex)); + } + buf_read_recv_pages(FALSE, space, zip_size, page_nos, n); /* fprintf(stderr, "Recv pages at %lu n %lu\n", page_nos[0], n); @@ -1688,6 +1862,7 @@ loop: /* Flush all the file pages to disk and invalidate them in the buffer pool */ + ut_d(recv_no_log_write = TRUE); mutex_exit(&(recv_sys->mutex)); mutex_exit(&(log_sys->mutex)); @@ -1701,6 +1876,7 @@ loop: mutex_enter(&(log_sys->mutex)); mutex_enter(&(recv_sys->mutex)); + ut_d(recv_no_log_write = FALSE); recv_no_ibuf_operations = FALSE; } @@ -1712,6 +1888,10 @@ loop: if (has_printed) { fprintf(stderr, "InnoDB: Apply batch completed\n"); + + if (srv_recovery_stats) { + recv_sys->stats_recv_turns++; + } } mutex_exit(&(recv_sys->mutex)); @@ -1912,6 +2092,17 @@ recv_parse_log_rec( return(0); } +#ifdef UNIV_LOG_LSN_DEBUG + if (*type == MLOG_LSN) { + ib_uint64_t lsn = (ib_uint64_t) *space << 32 | *page_no; +# ifdef UNIV_LOG_DEBUG + ut_a(lsn == log_sys->old_lsn); +# else /* UNIV_LOG_DEBUG */ + ut_a(lsn == recv_sys->recovered_lsn); +# endif /* UNIV_LOG_DEBUG */ + } +#endif /* UNIV_LOG_LSN_DEBUG */ + /* Check that page_no is sensible */ if (UNIV_UNLIKELY(*page_no > 0x8FFFFFFFUL)) { @@ -2169,6 +2360,12 @@ loop: #endif /* In normal mysqld crash recovery we do not try to replay file operations */ +#ifdef UNIV_LOG_LSN_DEBUG + } else if (type == MLOG_LSN) { + /* Do not add these records to the hash table. + The page number and space id fields are misused + for something else. */ +#endif /* UNIV_LOG_LSN_DEBUG */ } else { recv_add_to_hash_table(type, space, page_no, body, ptr + len, old_lsn, @@ -2200,11 +2397,11 @@ loop: = recv_sys->recovered_offset + total_len; recv_previous_parsed_rec_is_multi = 1; - if ((!store_to_hash) && (type != MLOG_MULTI_REC_END)) { #ifdef UNIV_LOG_DEBUG + if ((!store_to_hash) && (type != MLOG_MULTI_REC_END)) { recv_check_incomplete_log_recs(ptr, len); -#endif /* UNIV_LOG_DEBUG */ } +#endif /* UNIV_LOG_DEBUG */ #ifdef UNIV_DEBUG if (log_debug_writes) { @@ -2268,7 +2465,11 @@ loop: break; } - if (store_to_hash) { + if (store_to_hash +#ifdef UNIV_LOG_LSN_DEBUG + && type != MLOG_LSN +#endif /* UNIV_LOG_LSN_DEBUG */ + ) { recv_add_to_hash_table(type, space, page_no, body, ptr + len, old_lsn, @@ -2417,8 +2618,7 @@ recv_scan_log_recs( scanned_lsn = start_lsn; more_data = FALSE; - while (log_block < buf + len && !finished) { - + do { no = log_block_get_hdr_no(log_block); /* fprintf(stderr, "Log block header no %lu\n", no); @@ -2548,10 +2748,11 @@ recv_scan_log_recs( /* Log data for this group ends here */ finished = TRUE; + break; } else { log_block += OS_FILE_LOG_BLOCK_SIZE; } - } + } while (log_block < buf + len && !finished); *group_scanned_lsn = scanned_lsn; @@ -3078,6 +3279,84 @@ recv_recovery_from_checkpoint_finish(voi } #endif /* UNIV_DEBUG */ + if (recv_needed_recovery && srv_recovery_stats) { + FILE* file = stderr; + ulint i; + + fprintf(stderr, + "InnoDB: Applying log records was done. Its statistics are followings.\n"); + + fprintf(stderr, + "============================================================\n" + "-------------------\n" + "RECOVERY STATISTICS\n" + "-------------------\n"); + fprintf(stderr, + "Recovery time: %g sec. (%lu turns)\n", + difftime(time(NULL), recv_sys->stats_recv_start_time), + recv_sys->stats_recv_turns); + + fprintf(stderr, + "\n" + "Data page IO statistics\n" + " Requested pages: %lu\n" + " Read pages: %lu\n" + " Written pages: %lu\n" + " (Dirty blocks): %lu\n", + recv_sys->stats_read_requested_pages, + recv_sys->stats_read_io_pages, + recv_sys->stats_write_io_pages, + UT_LIST_GET_LEN(buf_pool->flush_list)); + + fprintf(stderr, + " Grouping IO [times]:\n" + "\tnumber of pages,\n" + "\t\tread request neighbors (in %d pages chunk),\n" + "\t\t\tcombined read IO,\n" + "\t\t\t\tcombined write IO\n", + RECV_READ_AHEAD_AREA); + for (i = 0; i < ut_max(RECV_READ_AHEAD_AREA, + OS_AIO_MERGE_N_CONSECUTIVE); i++) { + fprintf(stderr, + "\t%3lu,\t%lu,\t%lu,\t%lu\n", i + 1, + (i < RECV_READ_AHEAD_AREA) ? + recv_sys->stats_read_in_area[i] : 0, + (i < OS_AIO_MERGE_N_CONSECUTIVE) ? + recv_sys->stats_read_io_consecutive[i] : 0, + (i < OS_AIO_MERGE_N_CONSECUTIVE) ? + recv_sys->stats_write_io_consecutive[i] : 0); + } + + fprintf(stderr, + "\n" + "Recovery process statistics\n" + " Checked pages by doublewrite buffer: %lu\n" + " Overwritten pages from doublewrite: %lu\n" + " Recovered pages by io_thread: %lu\n" + " Recovered pages by main thread: %lu\n" + " Parsed log records to apply: %lu\n" + " Sum of the length: %lu\n" + " Applied log records: %lu\n" + " Sum of the length: %lu\n" + " Pages which are already new enough: %lu (It may not be accurate, if turns > 1)\n" + " Oldest page's LSN: %llu\n" + " Newest page's LSN: %llu\n", + recv_sys->stats_doublewrite_check_pages, + recv_sys->stats_doublewrite_overwrite_pages, + recv_sys->stats_recover_pages_with_read, + recv_sys->stats_recover_pages_without_read, + recv_sys->stats_log_recs, + recv_sys->stats_log_len_sum, + recv_sys->stats_applied_log_recs, + recv_sys->stats_applied_log_len_sum, + recv_sys->stats_pages_already_new, + recv_sys->stats_oldest_modified_lsn, + recv_sys->stats_newest_modified_lsn); + + fprintf(stderr, + "============================================================\n"); + } + if (recv_needed_recovery) { trx_sys_print_mysql_master_log_pos(); trx_sys_print_mysql_binlog_offset(); @@ -3104,8 +3383,13 @@ recv_recovery_from_checkpoint_finish(voi recv_recovery_on = FALSE; #ifndef UNIV_LOG_DEBUG - recv_sys_free(); + recv_sys_debug_free(); #endif + /* Roll back any recovered data dictionary transactions, so + that the data dictionary tables will be free of any locks. + The data dictionary latch should guarantee that there is at + most one data dictionary transaction active at a time. */ + trx_rollback_or_clean_recovered(FALSE); /* Drop partially created indexes. */ row_merge_drop_temp_indexes(); === modified file 'storage/xtradb/mem/mem0dbg.c' --- a/storage/xtradb/mem/mem0dbg.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/mem/mem0dbg.c 2010-01-06 12:00:14 +0000 @@ -170,6 +170,17 @@ mem_init( mem_comm_pool = mem_pool_create(size); } + +/******************************************************************//** +Closes the memory system. */ +UNIV_INTERN +void +mem_close(void) +/*===========*/ +{ + mem_pool_free(mem_comm_pool); + mem_comm_pool = NULL; +} #endif /* !UNIV_HOTBACKUP */ #ifdef UNIV_MEM_DEBUG === modified file 'storage/xtradb/mem/mem0mem.c' --- a/storage/xtradb/mem/mem0mem.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/mem/mem0mem.c 2010-01-06 12:00:14 +0000 @@ -475,16 +475,18 @@ mem_heap_block_free( len = block->len; block->magic_n = MEM_FREED_BLOCK_MAGIC_N; +#ifndef UNIV_HOTBACKUP + if (!srv_use_sys_malloc) { #ifdef UNIV_MEM_DEBUG - /* In the debug version we set the memory to a random combination - of hex 0xDE and 0xAD. */ + /* In the debug version we set the memory to a random + combination of hex 0xDE and 0xAD. */ - mem_erase_buf((byte*)block, len); + mem_erase_buf((byte*)block, len); #else /* UNIV_MEM_DEBUG */ - UNIV_MEM_ASSERT_AND_FREE(block, len); + UNIV_MEM_ASSERT_AND_FREE(block, len); #endif /* UNIV_MEM_DEBUG */ -#ifndef UNIV_HOTBACKUP + } if (type == MEM_HEAP_DYNAMIC || len < UNIV_PAGE_SIZE / 2) { ut_ad(!buf_block); @@ -495,6 +497,14 @@ mem_heap_block_free( buf_block_free(buf_block); } #else /* !UNIV_HOTBACKUP */ +#ifdef UNIV_MEM_DEBUG + /* In the debug version we set the memory to a random + combination of hex 0xDE and 0xAD. */ + + mem_erase_buf((byte*)block, len); +#else /* UNIV_MEM_DEBUG */ + UNIV_MEM_ASSERT_AND_FREE(block, len); +#endif /* UNIV_MEM_DEBUG */ ut_free(block); #endif /* !UNIV_HOTBACKUP */ } === modified file 'storage/xtradb/mem/mem0pool.c' --- a/storage/xtradb/mem/mem0pool.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/mem/mem0pool.c 2010-01-06 12:00:14 +0000 @@ -261,6 +261,18 @@ mem_pool_create( } /********************************************************************//** +Frees a memory pool. */ +UNIV_INTERN +void +mem_pool_free( +/*==========*/ + mem_pool_t* pool) /*!< in, own: memory pool */ +{ + ut_free(pool->buf); + ut_free(pool); +} + +/********************************************************************//** Fills the specified free list. @return TRUE if we were able to insert a block to the free list */ static === modified file 'storage/xtradb/mtr/mtr0mtr.c' --- a/storage/xtradb/mtr/mtr0mtr.c 2009-11-13 21:26:08 +0000 +++ b/storage/xtradb/mtr/mtr0mtr.c 2010-01-15 15:58:25 +0000 @@ -36,6 +36,7 @@ Created 11/26/1995 Heikki Tuuri #include "buf0flu.h" #ifndef UNIV_HOTBACKUP +# include "log0recv.h" /*****************************************************************//** Releases the item in the slot given. */ UNIV_INLINE @@ -148,7 +149,6 @@ mtr_log_reserve_and_write( dyn_array_t* mlog; dyn_block_t* block; ulint data_size; - ibool success; byte* first_data; ut_ad(mtr); @@ -167,8 +167,8 @@ mtr_log_reserve_and_write( if (mlog->heap == NULL) { mtr->end_lsn = log_reserve_and_write_fast( first_data, dyn_block_get_used(mlog), - &(mtr->start_lsn), &success); - if (success) { + &mtr->start_lsn); + if (mtr->end_lsn) { return; } @@ -215,6 +215,8 @@ mtr_commit( ut_d(mtr->state = MTR_COMMITTING); #ifndef UNIV_HOTBACKUP + /* This is a dirty read, for debugging. */ + ut_ad(!recv_no_log_write); write_log = mtr->modifications && mtr->n_log_recs; if (write_log) { === modified file 'storage/xtradb/os/os0file.c' --- a/storage/xtradb/os/os0file.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/os/os0file.c 2010-01-06 12:00:14 +0000 @@ -55,6 +55,9 @@ Created 10/21/1995 Heikki Tuuri #include "srv0start.h" #include "fil0fil.h" #include "buf0buf.h" +#include "trx0sys.h" +#include "trx0trx.h" +#include "log0recv.h" #ifndef UNIV_HOTBACKUP # include "os0sync.h" # include "os0thread.h" @@ -88,7 +91,9 @@ UNIV_INTERN ibool os_do_not_call_flush_a /* We do not call os_file_flush in every os_file_write. */ #endif /* UNIV_DO_FLUSH */ -#ifndef UNIV_HOTBACKUP +#ifdef UNIV_HOTBACKUP +# define os_aio_use_native_aio FALSE +#else /* UNIV_HOTBACKUP */ /* We use these mutexes to protect lseek + file i/o operation, if the OS does not provide an atomic pread or pwrite, or similar */ #define OS_FILE_N_SEEK_MUTEXES 16 @@ -235,7 +240,7 @@ static ulint os_aio_n_segments = ULINT_U /** If the following is TRUE, read i/o handler threads try to wait until a batch of new read requests have been posted */ static volatile ibool os_aio_recommend_sleep_for_read_threads = FALSE; -#endif /* !UNIV_HOTBACKUP */ +#endif /* UNIV_HOTBACKUP */ UNIV_INTERN ulint os_n_file_reads = 0; UNIV_INTERN ulint os_bytes_read_since_printout = 0; @@ -352,6 +357,19 @@ os_file_get_last_error( " software or another instance\n" "InnoDB: of MySQL." " Please close it to get rid of this error.\n"); + } else if (err == ERROR_WORKING_SET_QUOTA + || err == ERROR_NO_SYSTEM_RESOURCES) { + fprintf(stderr, + "InnoDB: The error means that there are no" + " sufficient system resources or quota to" + " complete the operation.\n"); + } else if (err == ERROR_OPERATION_ABORTED) { + fprintf(stderr, + "InnoDB: The error means that the I/O" + " operation has been aborted\n" + "InnoDB: because of either a thread exit" + " or an application request.\n" + "InnoDB: Retry attempt is made.\n"); } else { fprintf(stderr, "InnoDB: Some operating system error numbers" @@ -373,6 +391,11 @@ os_file_get_last_error( } else if (err == ERROR_SHARING_VIOLATION || err == ERROR_LOCK_VIOLATION) { return(OS_FILE_SHARING_VIOLATION); + } else if (err == ERROR_WORKING_SET_QUOTA + || err == ERROR_NO_SYSTEM_RESOURCES) { + return(OS_FILE_INSUFFICIENT_RESOURCE); + } else if (err == ERROR_OPERATION_ABORTED) { + return(OS_FILE_OPERATION_ABORTED); } else { return(100 + err); } @@ -491,6 +514,14 @@ os_file_handle_error_cond_exit( os_thread_sleep(10000000); /* 10 sec */ return(TRUE); + } else if (err == OS_FILE_INSUFFICIENT_RESOURCE) { + + os_thread_sleep(100000); /* 100 ms */ + return(TRUE); + } else if (err == OS_FILE_OPERATION_ABORTED) { + + os_thread_sleep(100000); /* 100 ms */ + return(TRUE); } else { if (name) { fprintf(stderr, "InnoDB: File name %s\n", name); @@ -854,6 +885,23 @@ next_file: ret = stat(full_path, &statinfo); if (ret) { + + if (errno == ENOENT) { + /* readdir() returned a file that does not exist, + it must have been deleted in the meantime. Do what + would have happened if the file was deleted before + readdir() - ignore and go to the next entry. + If this is the last entry then info->name will still + contain the name of the deleted file when this + function returns, but this is not an issue since the + caller shouldn't be looking at info when end of + directory is returned. */ + + ut_free(full_path); + + goto next_file; + } + os_file_handle_error_no_exit(full_path, "stat"); ut_free(full_path); @@ -1282,6 +1330,7 @@ try_again: } #endif #ifdef UNIV_NON_BUFFERED_IO +# ifndef UNIV_HOTBACKUP if (type == OS_LOG_FILE && srv_flush_log_at_trx_commit == 2) { /* Do not use unbuffered i/o to log files because value 2 denotes that we do not flush the log at every @@ -1290,10 +1339,14 @@ try_again: == SRV_WIN_IO_UNBUFFERED) { attributes = attributes | FILE_FLAG_NO_BUFFERING; } -#endif +# else /* !UNIV_HOTBACKUP */ + attributes = attributes | FILE_FLAG_NO_BUFFERING; +# endif /* !UNIV_HOTBACKUP */ +#endif /* UNIV_NON_BUFFERED_IO */ } else if (purpose == OS_FILE_NORMAL) { attributes = 0; #ifdef UNIV_NON_BUFFERED_IO +# ifndef UNIV_HOTBACKUP if (type == OS_LOG_FILE && srv_flush_log_at_trx_commit == 2) { /* Do not use unbuffered i/o to log files because value 2 denotes that we do not flush the log at every @@ -1302,7 +1355,10 @@ try_again: == SRV_WIN_IO_UNBUFFERED) { attributes = attributes | FILE_FLAG_NO_BUFFERING; } -#endif +# else /* !UNIV_HOTBACKUP */ + attributes = attributes | FILE_FLAG_NO_BUFFERING; +# endif /* !UNIV_HOTBACKUP */ +#endif /* UNIV_NON_BUFFERED_IO */ } else { attributes = 0; ut_error; @@ -2046,20 +2102,30 @@ os_file_flush( /*******************************************************************//** Does a synchronous read operation in Posix. @return number of bytes read, -1 if error */ +#define os_file_pread(file, buf, n, offset, offset_high) \ + _os_file_pread(file, buf, n, offset, offset_high, NULL); + static ssize_t -os_file_pread( +_os_file_pread( /*==========*/ os_file_t file, /*!< in: handle to a file */ void* buf, /*!< in: buffer where to read */ ulint n, /*!< in: number of bytes to read */ ulint offset, /*!< in: least significant 32 bits of file offset from where to read */ - ulint offset_high) /*!< in: most significant 32 bits of + ulint offset_high, /*!< in: most significant 32 bits of offset */ + trx_t* trx) { off_t offs; +#if defined(HAVE_PREAD) && !defined(HAVE_BROKEN_PREAD) ssize_t n_bytes; +#endif /* HAVE_PREAD && !HAVE_BROKEN_PREAD */ + ulint sec; + ulint ms; + ib_uint64_t start_time; + ib_uint64_t finish_time; ut_a((offset & 0xFFFFFFFFUL) == offset); @@ -2080,6 +2146,15 @@ os_file_pread( os_n_file_reads++; + if (innobase_get_slow_log() && trx && trx->take_stats) + { + trx->io_reads++; + trx->io_read += n; + ut_usectime(&sec, &ms); + start_time = (ib_uint64_t)sec * 1000000 + ms; + } else { + start_time = 0; + } #if defined(HAVE_PREAD) && !defined(HAVE_BROKEN_PREAD) os_mutex_enter(os_file_count_mutex); os_file_n_pending_preads++; @@ -2093,21 +2168,32 @@ os_file_pread( os_n_pending_reads--; os_mutex_exit(os_file_count_mutex); + if (innobase_get_slow_log() && trx && trx->take_stats && start_time) + { + ut_usectime(&sec, &ms); + finish_time = (ib_uint64_t)sec * 1000000 + ms; + trx->io_reads_wait_timer += (ulint)(finish_time - start_time); + } + return(n_bytes); #else { off_t ret_offset; ssize_t ret; +#ifndef UNIV_HOTBACKUP ulint i; +#endif /* !UNIV_HOTBACKUP */ os_mutex_enter(os_file_count_mutex); os_n_pending_reads++; os_mutex_exit(os_file_count_mutex); +#ifndef UNIV_HOTBACKUP /* Protect the seek / read operation with a mutex */ i = ((ulint) file) % OS_FILE_N_SEEK_MUTEXES; os_mutex_enter(os_file_seek_mutexes[i]); +#endif /* !UNIV_HOTBACKUP */ ret_offset = lseek(file, offs, SEEK_SET); @@ -2117,12 +2203,21 @@ os_file_pread( ret = read(file, buf, (ssize_t)n); } +#ifndef UNIV_HOTBACKUP os_mutex_exit(os_file_seek_mutexes[i]); +#endif /* !UNIV_HOTBACKUP */ os_mutex_enter(os_file_count_mutex); os_n_pending_reads--; os_mutex_exit(os_file_count_mutex); + if (innobase_get_slow_log() && trx && trx->take_stats && start_time) + { + ut_usectime(&sec, &ms); + finish_time = (ib_uint64_t)sec * 1000000 + ms; + trx->io_reads_wait_timer += (ulint)(finish_time - start_time); + } + return(ret); } #endif @@ -2195,16 +2290,20 @@ os_file_pwrite( #else { off_t ret_offset; +# ifndef UNIV_HOTBACKUP ulint i; +# endif /* !UNIV_HOTBACKUP */ os_mutex_enter(os_file_count_mutex); os_n_pending_writes++; os_mutex_exit(os_file_count_mutex); +# ifndef UNIV_HOTBACKUP /* Protect the seek / write operation with a mutex */ i = ((ulint) file) % OS_FILE_N_SEEK_MUTEXES; os_mutex_enter(os_file_seek_mutexes[i]); +# endif /* UNIV_HOTBACKUP */ ret_offset = lseek(file, offs, SEEK_SET); @@ -2230,7 +2329,9 @@ os_file_pwrite( # endif /* UNIV_DO_FLUSH */ func_exit: +# ifndef UNIV_HOTBACKUP os_mutex_exit(os_file_seek_mutexes[i]); +# endif /* !UNIV_HOTBACKUP */ os_mutex_enter(os_file_count_mutex); os_n_pending_writes--; @@ -2247,7 +2348,7 @@ Requests a synchronous positioned read o @return TRUE if request was successful, FALSE if fail */ UNIV_INTERN ibool -os_file_read( +_os_file_read( /*=========*/ os_file_t file, /*!< in: handle to a file */ void* buf, /*!< in: buffer where to read */ @@ -2255,7 +2356,8 @@ os_file_read( offset where to read */ ulint offset_high, /*!< in: most significant 32 bits of offset */ - ulint n) /*!< in: number of bytes to read */ + ulint n, /*!< in: number of bytes to read */ + trx_t* trx) { #ifdef __WIN__ BOOL ret; @@ -2264,7 +2366,9 @@ os_file_read( DWORD low; DWORD high; ibool retry; +#ifndef UNIV_HOTBACKUP ulint i; +#endif /* !UNIV_HOTBACKUP */ ut_a((offset & 0xFFFFFFFFUL) == offset); @@ -2283,16 +2387,20 @@ try_again: os_n_pending_reads++; os_mutex_exit(os_file_count_mutex); +#ifndef UNIV_HOTBACKUP /* Protect the seek / read operation with a mutex */ i = ((ulint) file) % OS_FILE_N_SEEK_MUTEXES; os_mutex_enter(os_file_seek_mutexes[i]); +#endif /* !UNIV_HOTBACKUP */ ret2 = SetFilePointer(file, low, &high, FILE_BEGIN); if (ret2 == 0xFFFFFFFF && GetLastError() != NO_ERROR) { +#ifndef UNIV_HOTBACKUP os_mutex_exit(os_file_seek_mutexes[i]); +#endif /* !UNIV_HOTBACKUP */ os_mutex_enter(os_file_count_mutex); os_n_pending_reads--; @@ -2303,7 +2411,9 @@ try_again: ret = ReadFile(file, buf, (DWORD) n, &len, NULL); +#ifndef UNIV_HOTBACKUP os_mutex_exit(os_file_seek_mutexes[i]); +#endif /* !UNIV_HOTBACKUP */ os_mutex_enter(os_file_count_mutex); os_n_pending_reads--; @@ -2312,14 +2422,14 @@ try_again: if (ret && len == n) { return(TRUE); } -#else +#else /* __WIN__ */ ibool retry; ssize_t ret; os_bytes_read_since_printout += n; try_again: - ret = os_file_pread(file, buf, n, offset, offset_high); + ret = _os_file_pread(file, buf, n, offset, offset_high, trx); if ((ulint)ret == n) { @@ -2331,7 +2441,7 @@ try_again: "InnoDB: Was only able to read %ld.\n", (ulong)n, (ulong)offset_high, (ulong)offset, (long)ret); -#endif +#endif /* __WIN__ */ #ifdef __WIN__ error_handling: #endif @@ -2380,7 +2490,9 @@ os_file_read_no_error_handling( DWORD low; DWORD high; ibool retry; +#ifndef UNIV_HOTBACKUP ulint i; +#endif /* !UNIV_HOTBACKUP */ ut_a((offset & 0xFFFFFFFFUL) == offset); @@ -2399,16 +2511,20 @@ try_again: os_n_pending_reads++; os_mutex_exit(os_file_count_mutex); +#ifndef UNIV_HOTBACKUP /* Protect the seek / read operation with a mutex */ i = ((ulint) file) % OS_FILE_N_SEEK_MUTEXES; os_mutex_enter(os_file_seek_mutexes[i]); +#endif /* !UNIV_HOTBACKUP */ ret2 = SetFilePointer(file, low, &high, FILE_BEGIN); if (ret2 == 0xFFFFFFFF && GetLastError() != NO_ERROR) { +#ifndef UNIV_HOTBACKUP os_mutex_exit(os_file_seek_mutexes[i]); +#endif /* !UNIV_HOTBACKUP */ os_mutex_enter(os_file_count_mutex); os_n_pending_reads--; @@ -2419,7 +2535,9 @@ try_again: ret = ReadFile(file, buf, (DWORD) n, &len, NULL); +#ifndef UNIV_HOTBACKUP os_mutex_exit(os_file_seek_mutexes[i]); +#endif /* !UNIV_HOTBACKUP */ os_mutex_enter(os_file_count_mutex); os_n_pending_reads--; @@ -2428,7 +2546,7 @@ try_again: if (ret && len == n) { return(TRUE); } -#else +#else /* __WIN__ */ ibool retry; ssize_t ret; @@ -2441,7 +2559,7 @@ try_again: return(TRUE); } -#endif +#endif /* __WIN__ */ #ifdef __WIN__ error_handling: #endif @@ -2500,9 +2618,11 @@ os_file_write( DWORD ret2; DWORD low; DWORD high; - ulint i; ulint n_retries = 0; ulint err; +#ifndef UNIV_HOTBACKUP + ulint i; +#endif /* !UNIV_HOTBACKUP */ ut_a((offset & 0xFFFFFFFF) == offset); @@ -2519,16 +2639,20 @@ retry: os_n_pending_writes++; os_mutex_exit(os_file_count_mutex); +#ifndef UNIV_HOTBACKUP /* Protect the seek / write operation with a mutex */ i = ((ulint) file) % OS_FILE_N_SEEK_MUTEXES; os_mutex_enter(os_file_seek_mutexes[i]); +#endif /* !UNIV_HOTBACKUP */ ret2 = SetFilePointer(file, low, &high, FILE_BEGIN); if (ret2 == 0xFFFFFFFF && GetLastError() != NO_ERROR) { +#ifndef UNIV_HOTBACKUP os_mutex_exit(os_file_seek_mutexes[i]); +#endif /* !UNIV_HOTBACKUP */ os_mutex_enter(os_file_count_mutex); os_n_pending_writes--; @@ -2562,7 +2686,9 @@ retry: } # endif /* UNIV_DO_FLUSH */ +#ifndef UNIV_HOTBACKUP os_mutex_exit(os_file_seek_mutexes[i]); +#endif /* !UNIV_HOTBACKUP */ os_mutex_enter(os_file_count_mutex); os_n_pending_writes--; @@ -2988,6 +3114,34 @@ os_aio_array_create( return(array); } +/************************************************************************//** +Frees an aio wait array. */ +static +void +os_aio_array_free( +/*==============*/ + os_aio_array_t* array) /*!< in, own: array to free */ +{ +#ifdef WIN_ASYNC_IO + ulint i; + + for (i = 0; i < array->n_slots; i++) { + os_aio_slot_t* slot = os_aio_array_get_nth_slot(array, i); + os_event_free(slot->event); + } +#endif /* WIN_ASYNC_IO */ + +#ifdef __WIN__ + ut_free(array->native_events); +#endif /* __WIN__ */ + os_mutex_free(array->mutex); + os_event_free(array->not_full); + os_event_free(array->is_empty); + + ut_free(array->slots); + ut_free(array); +} + /*********************************************************************** Initializes the asynchronous io system. Creates one array each for ibuf and log i/o. Also creates one array each for read and write where each @@ -3061,6 +3215,35 @@ os_aio_init( } +/*********************************************************************** +Frees the asynchronous io system. */ +UNIV_INTERN +void +os_aio_free(void) +/*=============*/ +{ + ulint i; + + os_aio_array_free(os_aio_ibuf_array); + os_aio_ibuf_array = NULL; + os_aio_array_free(os_aio_log_array); + os_aio_log_array = NULL; + os_aio_array_free(os_aio_read_array); + os_aio_read_array = NULL; + os_aio_array_free(os_aio_write_array); + os_aio_write_array = NULL; + os_aio_array_free(os_aio_sync_array); + os_aio_sync_array = NULL; + + for (i = 0; i < os_aio_n_segments; i++) { + os_event_free(os_aio_segment_wait_events[i]); + } + + ut_free(os_aio_segment_wait_events); + os_aio_segment_wait_events = 0; + os_aio_n_segments = 0; +} + #ifdef WIN_ASYNC_IO /************************************************************************//** Wakes up all async i/o threads in the array in Windows async i/o at @@ -3211,7 +3394,8 @@ os_aio_array_reserve_slot( offset */ ulint offset_high, /*!< in: most significant 32 bits of offset */ - ulint len) /*!< in: length of the block to read or write */ + ulint len, /*!< in: length of the block to read or write */ + trx_t* trx) { os_aio_slot_t* slot; #ifdef WIN_ASYNC_IO @@ -3432,9 +3616,21 @@ void os_aio_simulated_put_read_threads_to_sleep(void) /*============================================*/ { + +/* The idea of putting background IO threads to sleep is only for +Windows when using simulated AIO. Windows XP seems to schedule +background threads too eagerly to allow for coalescing during +readahead requests. */ +#ifdef __WIN__ os_aio_array_t* array; ulint g; + if (os_aio_use_native_aio) { + /* We do not use simulated aio: do nothing */ + + return; + } + os_aio_recommend_sleep_for_read_threads = TRUE; for (g = 0; g < os_aio_n_segments; g++) { @@ -3445,6 +3641,7 @@ os_aio_simulated_put_read_threads_to_sle os_event_reset(os_aio_segment_wait_events[g]); } } +#endif /* __WIN__ */ } /*******************************************************************//** @@ -3482,10 +3679,11 @@ os_aio( (can be used to identify a completed aio operation); ignored if mode is OS_AIO_SYNC */ - void* message2)/*!< in: message for the aio handler + void* message2,/*!< in: message for the aio handler (can be used to identify a completed aio operation); ignored if mode is OS_AIO_SYNC */ + trx_t* trx) { os_aio_array_t* array; os_aio_slot_t* slot; @@ -3524,8 +3722,8 @@ os_aio( wait in the Windows case. */ if (type == OS_FILE_READ) { - return(os_file_read(file, buf, offset, - offset_high, n)); + return(_os_file_read(file, buf, offset, + offset_high, n, trx)); } ut_a(type == OS_FILE_WRITE); @@ -3558,8 +3756,13 @@ try_again: ut_error; } + if (trx && type == OS_FILE_READ) + { + trx->io_reads++; + trx->io_read += n; + } slot = os_aio_array_reserve_slot(type, array, message1, message2, file, - name, buf, offset, offset_high, n); + name, buf, offset, offset_high, n, trx); if (type == OS_FILE_READ) { if (os_aio_use_native_aio) { #ifdef WIN_ASYNC_IO @@ -3679,6 +3882,7 @@ os_aio_windows_handle( ibool ret_val; BOOL ret; DWORD len; + BOOL retry = FALSE; if (segment == ULINT_UNDEFINED) { array = os_aio_sync_array; @@ -3732,14 +3936,52 @@ os_aio_windows_handle( ut_a(TRUE == os_file_flush(slot->file)); } #endif /* UNIV_DO_FLUSH */ + } else if (os_file_handle_error(slot->name, "Windows aio")) { + + retry = TRUE; } else { - os_file_handle_error(slot->name, "Windows aio"); ret_val = FALSE; } os_mutex_exit(array->mutex); + if (retry) { + /* retry failed read/write operation synchronously. + No need to hold array->mutex. */ + + switch (slot->type) { + case OS_FILE_WRITE: + ret = WriteFile(slot->file, slot->buf, + slot->len, &len, + &(slot->control)); + + break; + case OS_FILE_READ: + ret = ReadFile(slot->file, slot->buf, + slot->len, &len, + &(slot->control)); + + break; + default: + ut_error; + } + + if (!ret && GetLastError() == ERROR_IO_PENDING) { + /* aio was queued successfully! + We want a synchronous i/o operation on a + file where we also use async i/o: in Windows + we must use the same wait mechanism as for + async i/o */ + + ret = GetOverlappedResult(slot->file, + &(slot->control), + &len, TRUE); + } + + ret_val = ret && len == slot->len; + } + os_aio_array_free_slot(array, slot); return(ret_val); @@ -4018,6 +4260,18 @@ consecutive_loop: } } + if (srv_recovery_stats && recv_recovery_is_on() && n_consecutive) { + mutex_enter(&(recv_sys->mutex)); + if (slot->type == OS_FILE_READ) { + recv_sys->stats_read_io_pages += n_consecutive; + recv_sys->stats_read_io_consecutive[n_consecutive - 1]++; + } else if (slot->type == OS_FILE_WRITE) { + recv_sys->stats_write_io_pages += n_consecutive; + recv_sys->stats_write_io_consecutive[n_consecutive - 1]++; + } + mutex_exit(&(recv_sys->mutex)); + } + os_mutex_enter(array->mutex); /* Mark the i/os done in slots */ === modified file 'storage/xtradb/os/os0proc.c' --- a/storage/xtradb/os/os0proc.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/os/os0proc.c 2010-01-06 12:00:14 +0000 @@ -97,6 +97,7 @@ os_mem_alloc_large( fprintf(stderr, "InnoDB: HugeTLB: Warning: Failed to" " attach shared memory segment, errno %d\n", errno); + ptr = NULL; } /* Remove the shared memory segment so that it will be === modified file 'storage/xtradb/os/os0sync.c' --- a/storage/xtradb/os/os0sync.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/os/os0sync.c 2010-01-06 12:00:14 +0000 @@ -86,6 +86,9 @@ os_sync_init(void) UT_LIST_INIT(os_event_list); UT_LIST_INIT(os_mutex_list); + os_sync_mutex = NULL; + os_sync_mutex_inited = FALSE; + os_sync_mutex = os_mutex_create(NULL); os_sync_mutex_inited = TRUE; @@ -713,6 +716,7 @@ os_fast_mutex_free( os_mutex_enter(os_sync_mutex); } + ut_ad(os_fast_mutex_count > 0); os_fast_mutex_count--; if (UNIV_LIKELY(os_sync_mutex_inited)) { === modified file 'storage/xtradb/os/os0thread.c' --- a/storage/xtradb/os/os0thread.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/os/os0thread.c 2010-01-06 12:00:14 +0000 @@ -233,6 +233,7 @@ os_thread_exit( #ifdef __WIN__ ExitThread((DWORD)exit_value); #else + pthread_detach(pthread_self()); pthread_exit(exit_value); #endif } === modified file 'storage/xtradb/page/page0cur.c' --- a/storage/xtradb/page/page0cur.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/page/page0cur.c 2010-01-06 12:00:14 +0000 @@ -1195,7 +1195,7 @@ page_cur_insert_rec_zip_reorg( } /* Out of space: restore the page */ - if (!page_zip_decompress(page_zip, page)) { + if (!page_zip_decompress(page_zip, page, FALSE)) { ut_error; /* Memory corrupted? */ } ut_ad(page_validate(page, index)); === modified file 'storage/xtradb/page/page0page.c' --- a/storage/xtradb/page/page0page.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/page/page0page.c 2010-01-06 12:00:14 +0000 @@ -45,7 +45,7 @@ Created 2/2/1994 Heikki Tuuri ============== The index page consists of a page header which contains the page's -id and other information. On top of it are the the index records +id and other information. On top of it are the index records in a heap linked into a one way linear list according to alphabetic order. Just below page end is an array of pointers which we call page directory, @@ -679,7 +679,7 @@ page_copy_rec_list_end( if (UNIV_UNLIKELY (!page_zip_decompress(new_page_zip, - new_page))) { + new_page, FALSE))) { ut_error; } ut_ad(page_validate(new_page, index)); @@ -792,7 +792,7 @@ page_copy_rec_list_start( if (UNIV_UNLIKELY (!page_zip_decompress(new_page_zip, - new_page))) { + new_page, FALSE))) { ut_error; } ut_ad(page_validate(new_page, index)); === modified file 'storage/xtradb/page/page0zip.c' --- a/storage/xtradb/page/page0zip.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/page/page0zip.c 2010-01-06 12:00:14 +0000 @@ -47,8 +47,10 @@ Created June 2005 by Marko Makela # define buf_LRU_stat_inc_unzip() ((void) 0) #endif /* !UNIV_HOTBACKUP */ +#ifndef UNIV_HOTBACKUP /** Statistics on compression, indexed by page_zip_des_t::ssize - 1 */ UNIV_INTERN page_zip_stat_t page_zip_stat[PAGE_ZIP_NUM_SSIZE - 1]; +#endif /* !UNIV_HOTBACKUP */ /* Please refer to ../include/page0zip.ic for a description of the compressed page format. */ @@ -1144,7 +1146,9 @@ page_zip_compress( ulint* offsets = NULL; ulint n_blobs = 0; byte* storage;/* storage of uncompressed columns */ +#ifndef UNIV_HOTBACKUP ullint usec = ut_time_us(NULL); +#endif /* !UNIV_HOTBACKUP */ #ifdef PAGE_ZIP_COMPRESS_DBG FILE* logfile = NULL; #endif @@ -1208,7 +1212,9 @@ page_zip_compress( } } #endif /* PAGE_ZIP_COMPRESS_DBG */ +#ifndef UNIV_HOTBACKUP page_zip_stat[page_zip->ssize - 1].compressed++; +#endif /* !UNIV_HOTBACKUP */ if (UNIV_UNLIKELY(n_dense * PAGE_ZIP_DIR_SLOT_SIZE >= page_zip_get_size(page_zip))) { @@ -1345,8 +1351,10 @@ err_exit: fclose(logfile); } #endif /* PAGE_ZIP_COMPRESS_DBG */ +#ifndef UNIV_HOTBACKUP page_zip_stat[page_zip->ssize - 1].compressed_usec += ut_time_us(NULL) - usec; +#endif /* !UNIV_HOTBACKUP */ return(FALSE); } @@ -1404,12 +1412,14 @@ err_exit: fclose(logfile); } #endif /* PAGE_ZIP_COMPRESS_DBG */ +#ifndef UNIV_HOTBACKUP { page_zip_stat_t* zip_stat = &page_zip_stat[page_zip->ssize - 1]; zip_stat->compressed_ok++; zip_stat->compressed_usec += ut_time_us(NULL) - usec; } +#endif /* !UNIV_HOTBACKUP */ return(TRUE); } @@ -2811,7 +2821,11 @@ page_zip_decompress( /*================*/ page_zip_des_t* page_zip,/*!< in: data, ssize; out: m_start, m_end, m_nonempty, n_blobs */ - page_t* page) /*!< out: uncompressed page, may be trashed */ + page_t* page, /*!< out: uncompressed page, may be trashed */ + ibool all) /*!< in: TRUE=decompress the whole page; + FALSE=verify but do not copy some + page header fields that should not change + after page creation */ { z_stream d_stream; dict_index_t* index = NULL; @@ -2820,7 +2834,9 @@ page_zip_decompress( ulint trx_id_col = ULINT_UNDEFINED; mem_heap_t* heap; ulint* offsets; +#ifndef UNIV_HOTBACKUP ullint usec = ut_time_us(NULL); +#endif /* !UNIV_HOTBACKUP */ ut_ad(page_zip_simple_validate(page_zip)); UNIV_MEM_ASSERT_W(page, UNIV_PAGE_SIZE); @@ -2839,13 +2855,36 @@ page_zip_decompress( heap = mem_heap_create(n_dense * (3 * sizeof *recs) + UNIV_PAGE_SIZE); recs = mem_heap_alloc(heap, n_dense * (2 * sizeof *recs)); + if (all) { + /* Copy the page header. */ + memcpy(page, page_zip->data, PAGE_DATA); + } else { + /* Check that the bytes that we skip are identical. */ +#if defined UNIV_DEBUG || defined UNIV_ZIP_DEBUG + ut_a(!memcmp(FIL_PAGE_TYPE + page, + FIL_PAGE_TYPE + page_zip->data, + PAGE_HEADER - FIL_PAGE_TYPE)); + ut_a(!memcmp(PAGE_HEADER + PAGE_LEVEL + page, + PAGE_HEADER + PAGE_LEVEL + page_zip->data, + PAGE_DATA - (PAGE_HEADER + PAGE_LEVEL))); +#endif /* UNIV_DEBUG || UNIV_ZIP_DEBUG */ + + /* Copy the mutable parts of the page header. */ + memcpy(page, page_zip->data, FIL_PAGE_TYPE); + memcpy(PAGE_HEADER + page, PAGE_HEADER + page_zip->data, + PAGE_LEVEL - PAGE_N_DIR_SLOTS); + +#if defined UNIV_DEBUG || defined UNIV_ZIP_DEBUG + /* Check that the page headers match after copying. */ + ut_a(!memcmp(page, page_zip->data, PAGE_DATA)); +#endif /* UNIV_DEBUG || UNIV_ZIP_DEBUG */ + } + #ifdef UNIV_ZIP_DEBUG - /* Clear the page. */ - memset(page, 0x55, UNIV_PAGE_SIZE); + /* Clear the uncompressed page, except the header. */ + memset(PAGE_DATA + page, 0x55, UNIV_PAGE_SIZE - PAGE_DATA); #endif /* UNIV_ZIP_DEBUG */ - UNIV_MEM_INVALID(page, UNIV_PAGE_SIZE); - /* Copy the page header. */ - memcpy(page, page_zip->data, PAGE_DATA); + UNIV_MEM_INVALID(PAGE_DATA + page, UNIV_PAGE_SIZE - PAGE_DATA); /* Copy the page directory. */ if (UNIV_UNLIKELY(!page_zip_dir_decode(page_zip, page, recs, @@ -2976,12 +3015,14 @@ err_exit: page_zip_fields_free(index); mem_heap_free(heap); +#ifndef UNIV_HOTBACKUP { page_zip_stat_t* zip_stat = &page_zip_stat[page_zip->ssize - 1]; zip_stat->decompressed++; zip_stat->decompressed_usec += ut_time_us(NULL) - usec; } +#endif /* !UNIV_HOTBACKUP */ /* Update the stat counter for LRU policy. */ buf_LRU_stat_inc_unzip(); @@ -3084,7 +3125,7 @@ page_zip_validate_low( #endif /* UNIV_DEBUG_VALGRIND */ temp_page_zip = *page_zip; - valid = page_zip_decompress(&temp_page_zip, temp_page); + valid = page_zip_decompress(&temp_page_zip, temp_page, TRUE); if (!valid) { fputs("page_zip_validate(): failed to decompress\n", stderr); goto func_exit; @@ -4362,8 +4403,8 @@ IMPORTANT: if page_zip_reorganize() is i non-clustered index, the caller must update the insert buffer free bits in the same mini-transaction in such a way that the modification will be redo-logged. -@return TRUE on success, FALSE on failure; page and page_zip will be -left intact on failure. */ +@return TRUE on success, FALSE on failure; page_zip will be left +intact on failure, but page will be overwritten. */ UNIV_INTERN ibool page_zip_reorganize( @@ -4428,9 +4469,6 @@ page_zip_reorganize( if (UNIV_UNLIKELY(!page_zip_compress(page_zip, page, index, mtr))) { - /* Restore the old page and exit. */ - buf_frame_copy(page, temp_page); - #ifndef UNIV_HOTBACKUP buf_block_free(temp_block); #endif /* !UNIV_HOTBACKUP */ @@ -4591,7 +4629,8 @@ corrupt: memcpy(page_zip->data + page_zip_get_size(page_zip) - trailer_size, ptr + 8 + size, trailer_size); - if (UNIV_UNLIKELY(!page_zip_decompress(page_zip, page))) { + if (UNIV_UNLIKELY(!page_zip_decompress(page_zip, page, + TRUE))) { goto corrupt; } === modified file 'storage/xtradb/pars/lexyy.c' --- a/storage/xtradb/pars/lexyy.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/pars/lexyy.c 2010-01-17 11:41:32 +0000 @@ -2778,3 +2778,18 @@ static void yyfree (void * ptr ) + +/********************************************************************** +Release any resources used by the lexer. */ +UNIV_INTERN +void +pars_lexer_close(void) +/*==================*/ +{ + if (yy_buffer_stack) + yylex_destroy(); + if (stringbuf) + free(stringbuf); + stringbuf = NULL; + stringbuf_len_alloc = stringbuf_len = 0; +} === modified file 'storage/xtradb/pars/pars0lex.l' --- a/storage/xtradb/pars/pars0lex.l 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/pars/pars0lex.l 2010-01-17 11:41:32 +0000 @@ -661,3 +661,18 @@ In the state 'id', only two actions are } %% + +/********************************************************************** +Release any resources used by the lexer. */ +UNIV_INTERN +void +pars_lexer_close(void) +/*==================*/ +{ + if (yy_buffer_stack) + yylex_destroy(); + if (stringbuf) + free(stringbuf); + stringbuf = NULL; + stringbuf_len_alloc = stringbuf_len = 0; +} === modified file 'storage/xtradb/plug.in' --- a/storage/xtradb/plug.in 2009-11-13 21:26:08 +0000 +++ b/storage/xtradb/plug.in 2010-01-15 15:58:25 +0000 @@ -40,19 +40,11 @@ MYSQL_PLUGIN_ACTIONS(innobase, [ irix*|osf*|sysv5uw7*|openbsd*) CFLAGS="$CFLAGS -DUNIV_MUST_NOT_INLINE";; *solaris*|*SunOS*) - # Begin Solaris atomic function checks - AC_CHECK_FUNCS(atomic_cas_ulong atomic_cas_32 \ - atomic_cas_64 atomic_add_long, - AC_DEFINE( - [HAVE_SOLARIS_ATOMICS], - [1], - [Define to 1 if Solaris supports \ - atomic functions.])) - ### End Solaris atomic function checks - CFLAGS="$CFLAGS -DUNIV_SOLARIS";; esac + INNODB_DYNAMIC_CFLAGS="-DMYSQL_DYNAMIC_PLUGIN" + case "$target_cpu" in x86_64) # The AMD64 ABI forbids absolute addresses in shared libraries @@ -63,7 +55,60 @@ MYSQL_PLUGIN_ACTIONS(innobase, [ ;; esac AC_SUBST(INNODB_DYNAMIC_CFLAGS) + + AC_MSG_CHECKING(whether GCC atomic builtins are available) + # either define HAVE_IB_GCC_ATOMIC_BUILTINS or not + AC_TRY_RUN( + [ + int main() + { + long x; + long y; + long res; + char c; + + x = 10; + y = 123; + res = __sync_bool_compare_and_swap(&x, x, y); + if (!res || x != y) { + return(1); + } + + x = 10; + y = 123; + res = __sync_bool_compare_and_swap(&x, x + 1, y); + if (res || x != 10) { + return(1); + } + + x = 10; + y = 123; + res = __sync_add_and_fetch(&x, y); + if (res != 123 + 10 || x != 123 + 10) { + return(1); + } + + c = 10; + res = __sync_lock_test_and_set(&c, 123); + if (res != 10 || c != 123) { + return(1); + } + + return(0); + } + ], + [ + AC_DEFINE([HAVE_IB_GCC_ATOMIC_BUILTINS], [1], + [GCC atomic builtins are available]) + AC_MSG_RESULT(yes) + ], + [ + AC_MSG_RESULT(no) + ] + ) + AC_MSG_CHECKING(whether pthread_t can be used by GCC atomic builtins) + # either define HAVE_IB_ATOMIC_PTHREAD_T_GCC or not AC_TRY_RUN( [ #include <pthread.h> @@ -84,47 +129,73 @@ MYSQL_PLUGIN_ACTIONS(innobase, [ } ], [ - AC_DEFINE([HAVE_ATOMIC_PTHREAD_T], [1], + AC_DEFINE([HAVE_IB_ATOMIC_PTHREAD_T_GCC], [1], [pthread_t can be used by GCC atomic builtins]) AC_MSG_RESULT(yes) ], [ AC_MSG_RESULT(no) ] - ) + ) + + AC_MSG_CHECKING(whether Solaris libc atomic functions are available) + # either define HAVE_IB_SOLARIS_ATOMICS or not + AC_CHECK_FUNCS(atomic_add_long \ + atomic_cas_32 \ + atomic_cas_64 \ + atomic_cas_ulong, + + AC_DEFINE([HAVE_IB_SOLARIS_ATOMICS], [1], + [Define to 1 if Solaris libc atomic functions \ + are available]) + ) + + AC_MSG_CHECKING(whether pthread_t can be used by Solaris libc atomic functions) + # either define HAVE_IB_ATOMIC_PTHREAD_T_SOLARIS or not + AC_TRY_RUN( + [ + #include <pthread.h> + #include <string.h> - # Try using solaris atomics on SunOS if GCC atomics are not available - AC_CHECK_DECLS( - [HAVE_ATOMIC_PTHREAD_T], - [ - AC_MSG_NOTICE(no need to check pthread_t size) - ], - [ - AC_CHECK_DECLS( - [HAVE_SOLARIS_ATOMICS], - [ - AC_MSG_CHECKING(checking if pthread_t size is integral) - AC_TRY_RUN( - [ - #include <pthread.h> - int main() - { - pthread_t x = 0; - return(0); - } - ], - [ - AC_DEFINE([HAVE_ATOMIC_PTHREAD_T], [1], + int main(int argc, char** argv) { + pthread_t x1; + pthread_t x2; + pthread_t x3; + + memset(&x1, 0x0, sizeof(x1)); + memset(&x2, 0x0, sizeof(x2)); + memset(&x3, 0x0, sizeof(x3)); + + if (sizeof(pthread_t) == 4) { + + atomic_cas_32(&x1, x2, x3); + + } else if (sizeof(pthread_t) == 8) { + + atomic_cas_64(&x1, x2, x3); + + } else { + + return(1); + } + + return(0); + } + ], + [ + AC_DEFINE([HAVE_IB_ATOMIC_PTHREAD_T_SOLARIS], [1], [pthread_t can be used by solaris atomics]) - AC_MSG_RESULT(yes) - # size of pthread_t is needed for typed solaris atomics - AC_CHECK_SIZEOF([pthread_t], [], [#include <pthread.h>]) - ], - [ - AC_MSG_RESULT(no) - ]) - ]) - ]) + AC_MSG_RESULT(yes) + ], + [ + AC_MSG_RESULT(no) + ] + ) + + # this is needed to know which one of atomic_cas_32() or atomic_cas_64() + # to use in the source + AC_CHECK_SIZEOF([pthread_t], [], [#include <pthread.h>]) + # Check for x86 PAUSE instruction AC_MSG_CHECKING(for x86 PAUSE instruction) # We have to actually try running the test program, because of a bug @@ -141,7 +212,7 @@ MYSQL_PLUGIN_ACTIONS(innobase, [ } ], [ - AC_DEFINE([IB_HAVE_PAUSE_INSTRUCTION], [1], [Does x86 PAUSE instruction exist]) + AC_DEFINE([HAVE_IB_PAUSE_INSTRUCTION], [1], [Does x86 PAUSE instruction exist]) AC_MSG_RESULT(yes) ], [ === modified file 'storage/xtradb/que/que0que.c' --- a/storage/xtradb/que/que0que.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/que/que0que.c 2010-01-06 12:00:14 +0000 @@ -518,6 +518,7 @@ que_graph_free_recursive( upd_node_t* upd; tab_node_t* cre_tab; ind_node_t* cre_ind; + purge_node_t* purge; if (node == NULL) { @@ -579,6 +580,13 @@ que_graph_free_recursive( mem_heap_free(ins->entry_sys_heap); break; + case QUE_NODE_PURGE: + purge = node; + + mem_heap_free(purge->heap); + + break; + case QUE_NODE_UPDATE: upd = node; === modified file 'storage/xtradb/rem/rem0cmp.c' --- a/storage/xtradb/rem/rem0cmp.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/rem/rem0cmp.c 2010-01-06 12:00:14 +0000 @@ -36,7 +36,7 @@ Created 7/1/1994 Heikki Tuuri The records are put into alphabetical order in the following way: let F be the first field where two records disagree. -If there is a character in some position n where the the +If there is a character in some position n where the records disagree, the order is determined by comparison of the characters at position n, possibly after collating transformation. If there is no such character, @@ -76,7 +76,7 @@ cmp_debug_dtuple_rec_with_match( /*************************************************************//** This function is used to compare two data fields for which the data type is such that we must use MySQL code to compare them. The prototype here -must be a copy of the the one in ha_innobase.cc! +must be a copy of the one in ha_innobase.cc! @return 1, 0, -1, if a is greater, equal, less than b, respectively */ extern int @@ -399,7 +399,7 @@ next_byte: /*************************************************************//** This function is used to compare a data tuple to a physical record. Only dtuple->n_fields_cmp first fields are taken into account for -the the data tuple! If we denote by n = n_fields_cmp, then rec must +the data tuple! If we denote by n = n_fields_cmp, then rec must have either m >= n fields, or it must differ from dtuple in some of the m fields rec has. If rec has an externally stored field we do not compare it but return with value 0 if such a comparison should be === modified file 'storage/xtradb/row/row0ins.c' --- a/storage/xtradb/row/row0ins.c 2009-11-13 21:26:08 +0000 +++ b/storage/xtradb/row/row0ins.c 2010-01-15 15:58:25 +0000 @@ -141,7 +141,7 @@ row_ins_alloc_sys_fields( dfield = dtuple_get_nth_field(row, dict_col_get_no(col)); - ptr = mem_heap_alloc(heap, DATA_ROW_ID_LEN); + ptr = mem_heap_zalloc(heap, DATA_ROW_ID_LEN); dfield_set_data(dfield, ptr, DATA_ROW_ID_LEN); @@ -152,7 +152,7 @@ row_ins_alloc_sys_fields( col = dict_table_get_sys_col(table, DATA_TRX_ID); dfield = dtuple_get_nth_field(row, dict_col_get_no(col)); - ptr = mem_heap_alloc(heap, DATA_TRX_ID_LEN); + ptr = mem_heap_zalloc(heap, DATA_TRX_ID_LEN); dfield_set_data(dfield, ptr, DATA_TRX_ID_LEN); @@ -163,7 +163,7 @@ row_ins_alloc_sys_fields( col = dict_table_get_sys_col(table, DATA_ROLL_PTR); dfield = dtuple_get_nth_field(row, dict_col_get_no(col)); - ptr = mem_heap_alloc(heap, DATA_ROLL_PTR_LEN); + ptr = mem_heap_zalloc(heap, DATA_ROLL_PTR_LEN); dfield_set_data(dfield, ptr, DATA_ROLL_PTR_LEN); } @@ -1191,7 +1191,7 @@ row_ins_check_foreign_constraint( /*=============================*/ ibool check_ref,/*!< in: TRUE if we want to check that the referenced table is ok, FALSE if we - want to to check the foreign key table */ + want to check the foreign key table */ dict_foreign_t* foreign,/*!< in: foreign constraint; NOTE that the tables mentioned in it must be in the dictionary cache if they exist at all */ === modified file 'storage/xtradb/row/row0merge.c' --- a/storage/xtradb/row/row0merge.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/row/row0merge.c 2010-01-06 12:00:14 +0000 @@ -60,9 +60,19 @@ Completed by Sunny Bains and Marko Makel #ifdef UNIV_DEBUG /** Set these in order ot enable debug printout. */ /* @{ */ +/** Log the outcome of each row_merge_cmp() call, comparing records. */ static ibool row_merge_print_cmp; +/** Log each record read from temporary file. */ static ibool row_merge_print_read; +/** Log each record write to temporary file. */ static ibool row_merge_print_write; +/** Log each row_merge_blocks() call, merging two blocks of records to +a bigger one. */ +static ibool row_merge_print_block; +/** Log each block read from temporary file. */ +static ibool row_merge_print_block_read; +/** Log each block read from temporary file. */ +static ibool row_merge_print_block_write; /* @} */ #endif /* UNIV_DEBUG */ @@ -109,8 +119,9 @@ typedef struct row_merge_buf_struct row_ /** Information about temporary files used in merge sort */ struct merge_file_struct { - int fd; /*!< file descriptor */ - ulint offset; /*!< file offset */ + int fd; /*!< file descriptor */ + ulint offset; /*!< file offset (end of file) */ + ib_uint64_t n_rec; /*!< number of records in the file */ }; /** Information about temporary files used in merge sort */ @@ -682,6 +693,13 @@ row_merge_read( ib_uint64_t ofs = ((ib_uint64_t) offset) * sizeof *buf; ibool success; +#ifdef UNIV_DEBUG + if (row_merge_print_block_read) { + fprintf(stderr, "row_merge_read fd=%d ofs=%lu\n", + fd, (ulong) offset); + } +#endif /* UNIV_DEBUG */ + success = os_file_read_no_error_handling(OS_FILE_FROM_FD(fd), buf, (ulint) (ofs & 0xFFFFFFFF), (ulint) (ofs >> 32), @@ -709,6 +727,13 @@ row_merge_write( ib_uint64_t ofs = ((ib_uint64_t) offset) * sizeof(row_merge_block_t); +#ifdef UNIV_DEBUG + if (row_merge_print_block_write) { + fprintf(stderr, "row_merge_write fd=%d ofs=%lu\n", + fd, (ulong) offset); + } +#endif /* UNIV_DEBUG */ + return(UNIV_LIKELY(os_file_write("(merge)", OS_FILE_FROM_FD(fd), buf, (ulint) (ofs & 0xFFFFFFFF), (ulint) (ofs >> 32), @@ -718,7 +743,7 @@ row_merge_write( /********************************************************************//** Read a merge record. @return pointer to next record, or NULL on I/O error or end of list */ -static +static __attribute__((nonnull)) const byte* row_merge_read_rec( /*===============*/ @@ -1070,7 +1095,7 @@ row_merge_cmp( Reads clustered index of the table and create temporary files containing the index entries for the indexes to be built. @return DB_SUCCESS or error */ -static +static __attribute__((nonnull)) ulint row_merge_read_clustered_index( /*===========================*/ @@ -1175,6 +1200,12 @@ row_merge_read_clustered_index( in order to release the latch on the old page. */ if (btr_pcur_is_after_last_on_page(&pcur)) { + if (UNIV_UNLIKELY(trx_is_interrupted(trx))) { + i = 0; + err = DB_INTERRUPTED; + goto err_exit; + } + btr_pcur_store_position(&pcur, &mtr); mtr_commit(&mtr); mtr_start(&mtr); @@ -1233,6 +1264,7 @@ row_merge_read_clustered_index( if (UNIV_LIKELY (row && row_merge_buf_add(buf, row, ext))) { + file->n_rec++; continue; } @@ -1274,14 +1306,19 @@ err_exit: UNIV_MEM_INVALID(block[0], sizeof block[0]); merge_buf[i] = row_merge_buf_empty(buf); - /* Try writing the record again, now that - the buffer has been written out and emptied. */ + if (UNIV_LIKELY(row != NULL)) { + /* Try writing the record again, now + that the buffer has been written out + and emptied. */ + + if (UNIV_UNLIKELY + (!row_merge_buf_add(buf, row, ext))) { + /* An empty buffer should have enough + room for at least one record. */ + ut_error; + } - if (UNIV_UNLIKELY - (row && !row_merge_buf_add(buf, row, ext))) { - /* An empty buffer should have enough - room for at least one record. */ - ut_error; + file->n_rec++; } } @@ -1320,7 +1357,7 @@ func_exit: b2 = row_merge_write_rec(&block[2], &buf[2], b2, \ of->fd, &of->offset, \ mrec##N, offsets##N); \ - if (UNIV_UNLIKELY(!b2)) { \ + if (UNIV_UNLIKELY(!b2 || ++of->n_rec > file->n_rec)) { \ goto corrupt; \ } \ b##N = row_merge_read_rec(&block[N], &buf[N], \ @@ -1336,14 +1373,14 @@ func_exit: } while (0) /*************************************************************//** -Merge two blocks of linked lists on disk and write a bigger block. +Merge two blocks of records on disk and write a bigger block. @return DB_SUCCESS or error code */ static ulint row_merge_blocks( /*=============*/ const dict_index_t* index, /*!< in: index being created */ - merge_file_t* file, /*!< in/out: file containing + const merge_file_t* file, /*!< in: file containing index entries */ row_merge_block_t* block, /*!< in/out: 3 buffers */ ulint* foffs0, /*!< in/out: offset of first @@ -1366,6 +1403,17 @@ row_merge_blocks( ulint* offsets0;/* offsets of mrec0 */ ulint* offsets1;/* offsets of mrec1 */ +#ifdef UNIV_DEBUG + if (row_merge_print_block) { + fprintf(stderr, + "row_merge_blocks fd=%d ofs=%lu + fd=%d ofs=%lu" + " = fd=%d ofs=%lu\n", + file->fd, (ulong) *foffs0, + file->fd, (ulong) *foffs1, + of->fd, (ulong) of->offset); + } +#endif /* UNIV_DEBUG */ + heap = row_merge_heap_create(index, &offsets0, &offsets1); /* Write a record and read the next record. Split the output @@ -1438,16 +1486,88 @@ done1: } /*************************************************************//** +Copy a block of index entries. +@return TRUE on success, FALSE on failure */ +static __attribute__((nonnull)) +ibool +row_merge_blocks_copy( +/*==================*/ + const dict_index_t* index, /*!< in: index being created */ + const merge_file_t* file, /*!< in: input file */ + row_merge_block_t* block, /*!< in/out: 3 buffers */ + ulint* foffs0, /*!< in/out: input file offset */ + merge_file_t* of) /*!< in/out: output file */ +{ + mem_heap_t* heap; /*!< memory heap for offsets0, offsets1 */ + + mrec_buf_t buf[3]; /*!< buffer for handling + split mrec in block[] */ + const byte* b0; /*!< pointer to block[0] */ + byte* b2; /*!< pointer to block[2] */ + const mrec_t* mrec0; /*!< merge rec, points to block[0] */ + ulint* offsets0;/* offsets of mrec0 */ + ulint* offsets1;/* dummy offsets */ + +#ifdef UNIV_DEBUG + if (row_merge_print_block) { + fprintf(stderr, + "row_merge_blocks_copy fd=%d ofs=%lu" + " = fd=%d ofs=%lu\n", + file->fd, (ulong) foffs0, + of->fd, (ulong) of->offset); + } +#endif /* UNIV_DEBUG */ + + heap = row_merge_heap_create(index, &offsets0, &offsets1); + + /* Write a record and read the next record. Split the output + file in two halves, which can be merged on the following pass. */ + + if (!row_merge_read(file->fd, *foffs0, &block[0])) { +corrupt: + mem_heap_free(heap); + return(FALSE); + } + + b0 = block[0]; + b2 = block[2]; + + b0 = row_merge_read_rec(&block[0], &buf[0], b0, index, file->fd, + foffs0, &mrec0, offsets0); + if (UNIV_UNLIKELY(!b0 && mrec0)) { + + goto corrupt; + } + + if (mrec0) { + /* append all mrec0 to output */ + for (;;) { + ROW_MERGE_WRITE_GET_NEXT(0, goto done0); + } + } +done0: + + /* The file offset points to the beginning of the last page + that has been read. Update it to point to the next block. */ + (*foffs0)++; + + mem_heap_free(heap); + return(row_merge_write_eof(&block[2], b2, of->fd, &of->offset) + != NULL); +} + +/*************************************************************//** Merge disk files. @return DB_SUCCESS or error code */ -static +static __attribute__((nonnull)) ulint row_merge( /*======*/ + trx_t* trx, /*!< in: transaction */ const dict_index_t* index, /*!< in: index being created */ merge_file_t* file, /*!< in/out: file containing index entries */ - ulint half, /*!< in: half the file */ + ulint* half, /*!< in/out: half the file */ row_merge_block_t* block, /*!< in/out: 3 buffers */ int* tmpfd, /*!< in/out: temporary file handle */ TABLE* table) /*!< in/out: MySQL table, for @@ -1458,43 +1578,87 @@ row_merge( ulint foffs1; /*!< second input offset */ ulint error; /*!< error code */ merge_file_t of; /*!< output file */ + const ulint ihalf = *half; + /*!< half the input file */ + ulint ohalf; /*!< half the output file */ UNIV_MEM_ASSERT_W(block[0], 3 * sizeof block[0]); - ut_ad(half > 0); + ut_ad(ihalf < file->offset); of.fd = *tmpfd; of.offset = 0; + of.n_rec = 0; /* Merge blocks to the output file. */ + ohalf = 0; foffs0 = 0; - foffs1 = half; + foffs1 = ihalf; + + for (; foffs0 < ihalf && foffs1 < file->offset; foffs0++, foffs1++) { + ulint ahalf; /*!< arithmetic half the input file */ + + if (UNIV_UNLIKELY(trx_is_interrupted(trx))) { + return(DB_INTERRUPTED); + } - for (; foffs0 < half && foffs1 < file->offset; foffs0++, foffs1++) { error = row_merge_blocks(index, file, block, &foffs0, &foffs1, &of, table); if (error != DB_SUCCESS) { return(error); } + + /* Record the offset of the output file when + approximately half the output has been generated. In + this way, the next invocation of row_merge() will + spend most of the time in this loop. The initial + estimate is ohalf==0. */ + ahalf = file->offset / 2; + ut_ad(ohalf <= of.offset); + + /* Improve the estimate until reaching half the input + file size, or we can not get any closer to it. All + comparands should be non-negative when !(ohalf < ahalf) + because ohalf <= of.offset. */ + if (ohalf < ahalf || of.offset - ahalf < ohalf - ahalf) { + ohalf = of.offset; + } } - /* Copy the last block, if there is one. */ - while (foffs0 < half) { - if (!row_merge_read(file->fd, foffs0++, block) - || !row_merge_write(of.fd, of.offset++, block)) { + /* Copy the last blocks, if there are any. */ + + while (foffs0 < ihalf) { + if (UNIV_UNLIKELY(trx_is_interrupted(trx))) { + return(DB_INTERRUPTED); + } + + if (!row_merge_blocks_copy(index, file, block, &foffs0, &of)) { return(DB_CORRUPTION); } } + + ut_ad(foffs0 == ihalf); + while (foffs1 < file->offset) { - if (!row_merge_read(file->fd, foffs1++, block) - || !row_merge_write(of.fd, of.offset++, block)) { + if (UNIV_UNLIKELY(trx_is_interrupted(trx))) { + return(DB_INTERRUPTED); + } + + if (!row_merge_blocks_copy(index, file, block, &foffs1, &of)) { return(DB_CORRUPTION); } } + ut_ad(foffs1 == file->offset); + + if (UNIV_UNLIKELY(of.n_rec != file->n_rec)) { + return(DB_CORRUPTION); + } + /* Swap file descriptors for the next pass. */ *tmpfd = file->fd; *file = of; + *half = ohalf; UNIV_MEM_INVALID(block[0], 3 * sizeof block[0]); @@ -1508,6 +1672,7 @@ static ulint row_merge_sort( /*===========*/ + trx_t* trx, /*!< in: transaction */ const dict_index_t* index, /*!< in: index being created */ merge_file_t* file, /*!< in/out: file containing index entries */ @@ -1517,20 +1682,26 @@ row_merge_sort( reporting erroneous key value if applicable */ { - ulint blksz; /*!< block size */ + ulint half = file->offset / 2; + + /* The file should always contain at least one byte (the end + of file marker). Thus, it must be at least one block. */ + ut_ad(file->offset > 0); - for (blksz = 1; blksz < file->offset; blksz *= 2) { - ulint half; + do { ulint error; - ut_ad(ut_is_2pow(blksz)); - half = ut_2pow_round((file->offset + (blksz - 1)) / 2, blksz); - error = row_merge(index, file, half, block, tmpfd, table); + error = row_merge(trx, index, file, &half, + block, tmpfd, table); if (error != DB_SUCCESS) { return(error); } - } + + /* half > 0 should hold except when the file consists + of one block. No need to merge further then. */ + ut_ad(half > 0 || file->offset == 1); + } while (half < file->offset && half > 0); return(DB_SUCCESS); } @@ -1797,7 +1968,15 @@ row_merge_drop_index( static const char str1[] = "PROCEDURE DROP_INDEX_PROC () IS\n" "BEGIN\n" + /* Rename the index, so that it will be dropped by + row_merge_drop_temp_indexes() at crash recovery + if the server crashes before this trx is committed. */ + "UPDATE SYS_INDEXES SET NAME=CONCAT('" + TEMP_INDEX_PREFIX_STR "', NAME) WHERE ID = :indexid;\n" + "COMMIT WORK;\n" + /* Drop the field definitions of the index. */ "DELETE FROM SYS_FIELDS WHERE INDEX_ID = :indexid;\n" + /* Drop the index definition and the B-tree. */ "DELETE FROM SYS_INDEXES WHERE ID = :indexid\n" " AND TABLE_ID = :tableid;\n" "END;\n"; @@ -1909,6 +2088,7 @@ row_merge_file_create( { merge_file->fd = innobase_mysql_tmpfile(); merge_file->offset = 0; + merge_file->n_rec = 0; } /*********************************************************************//** @@ -2129,7 +2309,7 @@ row_merge_rename_tables( if (err != DB_SUCCESS) { err_exit: trx->error_state = DB_SUCCESS; - trx_general_rollback_for_mysql(trx, FALSE, NULL); + trx_general_rollback_for_mysql(trx, NULL); trx->error_state = DB_SUCCESS; } @@ -2331,7 +2511,7 @@ row_merge_build_indexes( sorting and inserting. */ for (i = 0; i < n_indexes; i++) { - error = row_merge_sort(indexes[i], &merge_files[i], + error = row_merge_sort(trx, indexes[i], &merge_files[i], block, &tmpfd, table); if (error == DB_SUCCESS) { === modified file 'storage/xtradb/row/row0mysql.c' --- a/storage/xtradb/row/row0mysql.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/row/row0mysql.c 2010-01-06 12:00:14 +0000 @@ -510,7 +510,7 @@ handle_new_error: switch (err) { case DB_LOCK_WAIT_TIMEOUT: if (row_rollback_on_timeout) { - trx_general_rollback_for_mysql(trx, FALSE, NULL); + trx_general_rollback_for_mysql(trx, NULL); break; } /* fall through */ @@ -526,7 +526,7 @@ handle_new_error: /* Roll back the latest, possibly incomplete insertion or update */ - trx_general_rollback_for_mysql(trx, TRUE, savept); + trx_general_rollback_for_mysql(trx, savept); } /* MySQL will roll back the latest SQL statement */ break; @@ -548,7 +548,7 @@ handle_new_error: /* Roll back the whole transaction; this resolution was added to version 3.23.43 */ - trx_general_rollback_for_mysql(trx, FALSE, NULL); + trx_general_rollback_for_mysql(trx, NULL); break; case DB_MUST_GET_MORE_FILE_SPACE: @@ -869,18 +869,22 @@ row_update_statistics_if_needed( } /*********************************************************************//** -Unlocks AUTO_INC type locks that were possibly reserved by a trx. */ +Unlocks AUTO_INC type locks that were possibly reserved by a trx. This +function should be called at the the end of an SQL statement, by the +connection thread that owns the transaction (trx->mysql_thd). */ UNIV_INTERN void row_unlock_table_autoinc_for_mysql( /*===============================*/ trx_t* trx) /*!< in/out: transaction */ { - mutex_enter(&kernel_mutex); + if (lock_trx_holds_autoinc_locks(trx)) { + mutex_enter(&kernel_mutex); - lock_release_autoinc_locks(trx); + lock_release_autoinc_locks(trx); - mutex_exit(&kernel_mutex); + mutex_exit(&kernel_mutex); + } } /*********************************************************************//** @@ -1770,7 +1774,6 @@ row_create_table_for_mysql( const char* table_name; ulint table_name_len; ulint err; - ulint i; ut_ad(trx->mysql_thread_id == os_thread_get_curr_id()); #ifdef UNIV_SYNC_DEBUG @@ -1805,15 +1808,6 @@ err_exit: goto err_exit; } - /* Check that no reserved column names are used. */ - for (i = 0; i < dict_table_get_n_user_cols(table); i++) { - if (dict_col_name_is_reserved( - dict_table_get_col_name(table, i))) { - - goto err_exit; - } - } - trx_start_if_not_started(trx); /* The table name is prefixed with the database name and a '/'. @@ -1888,7 +1882,9 @@ err_exit: if (UNIV_UNLIKELY(err != DB_SUCCESS)) { trx->error_state = DB_SUCCESS; - trx_general_rollback_for_mysql(trx, FALSE, NULL); + trx_general_rollback_for_mysql(trx, NULL); + /* TO DO: free table? The code below will dereference + table->name, though. */ } switch (err) { @@ -1907,31 +1903,6 @@ err_exit: break; case DB_DUPLICATE_KEY: - ut_print_timestamp(stderr); - fputs(" InnoDB: Error: table ", stderr); - ut_print_name(stderr, trx, TRUE, table->name); - fputs(" already exists in InnoDB internal\n" - "InnoDB: data dictionary. Have you deleted" - " the .frm file\n" - "InnoDB: and not used DROP TABLE?" - " Have you used DROP DATABASE\n" - "InnoDB: for InnoDB tables in" - " MySQL version <= 3.23.43?\n" - "InnoDB: See the Restrictions section" - " of the InnoDB manual.\n" - "InnoDB: You can drop the orphaned table" - " inside InnoDB by\n" - "InnoDB: creating an InnoDB table with" - " the same name in another\n" - "InnoDB: database and copying the .frm file" - " to the current database.\n" - "InnoDB: Then MySQL thinks the table exists," - " and DROP TABLE will\n" - "InnoDB: succeed.\n" - "InnoDB: You can look for further help from\n" - "InnoDB: " REFMAN "innodb-troubleshooting.html\n", - stderr); - /* We may also get err == DB_ERROR if the .ibd file for the table already exists */ @@ -2056,7 +2027,7 @@ error_handling: trx->error_state = DB_SUCCESS; - trx_general_rollback_for_mysql(trx, FALSE, NULL); + trx_general_rollback_for_mysql(trx, NULL); row_drop_table_for_mysql(table_name, trx, FALSE); @@ -2077,7 +2048,7 @@ Scans a table create SQL string and adds the foreign key constraints declared in the string. This function should be called after the indexes for a table have been created. Each foreign key constraint must be accompanied with indexes in -bot participating tables. The indexes are allowed to contain more +both participating tables. The indexes are allowed to contain more fields than mentioned in the constraint. Check also that foreign key constraints which reference this table are ok. @return error code or DB_SUCCESS */ @@ -2124,7 +2095,7 @@ row_table_add_foreign_constraints( trx->error_state = DB_SUCCESS; - trx_general_rollback_for_mysql(trx, FALSE, NULL); + trx_general_rollback_for_mysql(trx, NULL); row_drop_table_for_mysql(name, trx, FALSE); @@ -2491,7 +2462,7 @@ row_discard_tablespace_for_mysql( if (err != DB_SUCCESS) { trx->error_state = DB_SUCCESS; - trx_general_rollback_for_mysql(trx, FALSE, NULL); + trx_general_rollback_for_mysql(trx, NULL); trx->error_state = DB_SUCCESS; } else { dict_table_change_id_in_cache(table, new_id); @@ -2500,7 +2471,7 @@ row_discard_tablespace_for_mysql( if (!success) { trx->error_state = DB_SUCCESS; - trx_general_rollback_for_mysql(trx, FALSE, NULL); + trx_general_rollback_for_mysql(trx, NULL); trx->error_state = DB_SUCCESS; err = DB_ERROR; @@ -2952,7 +2923,7 @@ next_rec: if (err != DB_SUCCESS) { trx->error_state = DB_SUCCESS; - trx_general_rollback_for_mysql(trx, FALSE, NULL); + trx_general_rollback_for_mysql(trx, NULL); trx->error_state = DB_SUCCESS; ut_print_timestamp(stderr); fputs(" InnoDB: Unable to assign a new identifier to table ", @@ -3593,7 +3564,7 @@ row_delete_constraint( if ((err == DB_SUCCESS) && !strchr(id, '/')) { /* Old format < 4.0.18 constraints have constraint ids - <number>_<number>. We only try deleting them if the + NUMBER_NUMBER. We only try deleting them if the constraint name does not contain a '/' character, otherwise deleting a new format constraint named 'foo/bar' from database 'baz' would remove constraint 'bar' from database @@ -3857,7 +3828,7 @@ end: "InnoDB: succeed.\n", stderr); } trx->error_state = DB_SUCCESS; - trx_general_rollback_for_mysql(trx, FALSE, NULL); + trx_general_rollback_for_mysql(trx, NULL); trx->error_state = DB_SUCCESS; } else { /* The following call will also rename the .ibd data file if @@ -3866,7 +3837,7 @@ end: if (!dict_table_rename_in_cache(table, new_name, !new_is_tmp)) { trx->error_state = DB_SUCCESS; - trx_general_rollback_for_mysql(trx, FALSE, NULL); + trx_general_rollback_for_mysql(trx, NULL); trx->error_state = DB_SUCCESS; goto funct_exit; } @@ -3906,7 +3877,7 @@ end: ut_a(dict_table_rename_in_cache(table, old_name, FALSE)); trx->error_state = DB_SUCCESS; - trx_general_rollback_for_mysql(trx, FALSE, NULL); + trx_general_rollback_for_mysql(trx, NULL); trx->error_state = DB_SUCCESS; } } @@ -4166,6 +4137,7 @@ row_check_table_for_mysql( } if (trx_is_interrupted(prebuilt->trx)) { + ret = DB_INTERRUPTED; break; } === modified file 'storage/xtradb/row/row0sel.c' --- a/storage/xtradb/row/row0sel.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/row/row0sel.c 2010-01-15 21:12:30 +0000 @@ -4616,6 +4616,7 @@ row_search_autoinc_read_column( dict_index_t* index, /*!< in: index to read from */ const rec_t* rec, /*!< in: current rec */ ulint col_no, /*!< in: column number */ + ulint mtype, /*!< in: column main type */ ibool unsigned_type) /*!< in: signed or unsigned flag */ { ulint len; @@ -4632,10 +4633,27 @@ row_search_autoinc_read_column( data = rec_get_nth_field(rec, offsets, col_no, &len); ut_a(len != UNIV_SQL_NULL); - ut_a(len <= sizeof value); /* we assume AUTOINC value cannot be negative */ - value = mach_read_int_type(data, len, unsigned_type); + switch (mtype) { + case DATA_INT: + ut_a(len <= sizeof value); + value = mach_read_int_type(data, len, unsigned_type); + break; + + case DATA_FLOAT: + ut_a(len == sizeof(float)); + value = mach_float_read(data); + break; + + case DATA_DOUBLE: + ut_a(len == sizeof(double)); + value = mach_double_read(data); + break; + + default: + ut_error; + } if (UNIV_LIKELY_NULL(heap)) { mem_heap_free(heap); @@ -4721,7 +4739,8 @@ row_search_max_autoinc( dfield->col->prtype & DATA_UNSIGNED); *value = row_search_autoinc_read_column( - index, rec, i, unsigned_type); + index, rec, i, + dfield->col->mtype, unsigned_type); } } === modified file 'storage/xtradb/scripts/install_innodb_plugins.sql' --- a/storage/xtradb/scripts/install_innodb_plugins.sql 2009-06-25 01:43:25 +0000 +++ b/storage/xtradb/scripts/install_innodb_plugins.sql 2010-01-06 12:00:14 +0000 @@ -14,3 +14,4 @@ INSTALL PLUGIN INNODB_BUFFER_POOL_PAGES_ INSTALL PLUGIN innodb_rseg SONAME 'ha_innodb.so'; INSTALL PLUGIN innodb_table_stats SONAME 'ha_innodb.so'; INSTALL PLUGIN innodb_index_stats SONAME 'ha_innodb.so'; +INSTALL PLUGIN xtradb_admin_command SONAME 'ha_innodb.so'; === modified file 'storage/xtradb/scripts/install_innodb_plugins_win.sql' --- a/storage/xtradb/scripts/install_innodb_plugins_win.sql 2008-12-03 05:06:00 +0000 +++ b/storage/xtradb/scripts/install_innodb_plugins_win.sql 2010-01-06 12:00:14 +0000 @@ -7,3 +7,11 @@ INSTALL PLUGIN innodb_cmp SONAME 'ha_inn INSTALL PLUGIN innodb_cmp_reset SONAME 'ha_innodb.dll'; INSTALL PLUGIN innodb_cmpmem SONAME 'ha_innodb.dll'; INSTALL PLUGIN innodb_cmpmem_reset SONAME 'ha_innodb.dll'; +INSTALL PLUGIN XTRADB_ENHANCEMENTS SONAME 'ha_innodb.dll'; +INSTALL PLUGIN INNODB_BUFFER_POOL_PAGES SONAME 'ha_innodb.dll'; +INSTALL PLUGIN INNODB_BUFFER_POOL_PAGES_BLOB SONAME 'ha_innodb.dll'; +INSTALL PLUGIN INNODB_BUFFER_POOL_PAGES_INDEX SONAME 'ha_innodb.dll'; +INSTALL PLUGIN innodb_rseg SONAME 'ha_innodb.dll'; +INSTALL PLUGIN innodb_table_stats SONAME 'ha_innodb.dll'; +INSTALL PLUGIN innodb_index_stats SONAME 'ha_innodb.dll'; +INSTALL PLUGIN xtradb_admin_command SONAME 'ha_innodb.dll'; === modified file 'storage/xtradb/srv/srv0srv.c' --- a/storage/xtradb/srv/srv0srv.c 2010-01-06 21:27:53 +0000 +++ b/storage/xtradb/srv/srv0srv.c 2010-01-15 19:48:33 +0000 @@ -81,7 +81,6 @@ Created 10/8/1995 Heikki Tuuri #include "ut0mem.h" #include "ut0ut.h" #include "os0proc.h" -#include "os0sync.h" #include "mem0mem.h" #include "mem0pool.h" #include "sync0sync.h" @@ -103,6 +102,10 @@ Created 10/8/1995 Heikki Tuuri #include "row0mysql.h" #include "ha_prototypes.h" #include "trx0i_s.h" +#include "os0sync.h" /* for HAVE_ATOMIC_BUILTINS */ + +/* prototypes for new functions added to ha_innodb.cc */ +ibool innobase_get_slow_log(); /* This is set to TRUE if the MySQL user has set it in MySQL; currently affects only FOREIGN KEY definition parsing */ @@ -162,8 +165,9 @@ UNIV_INTERN ulint* srv_data_file_sizes = UNIV_INTERN ibool srv_extra_undoslots = FALSE; UNIV_INTERN ibool srv_fast_recovery = FALSE; +UNIV_INTERN ibool srv_recovery_stats = FALSE; -UNIV_INTERN ibool srv_use_purge_thread = FALSE; +UNIV_INTERN ulint srv_use_purge_thread = 0; /* if TRUE, then we auto-extend the last data file */ UNIV_INTERN ibool srv_auto_extend_last_data_file = FALSE; @@ -303,12 +307,6 @@ UNIV_INTERN ulint srv_buf_pool_flushed = reading of a disk page */ UNIV_INTERN ulint srv_buf_pool_reads = 0; -/** Number of sequential read-aheads */ -UNIV_INTERN ulint srv_read_ahead_seq = 0; - -/** Number of random read-aheads */ -UNIV_INTERN ulint srv_read_ahead_rnd = 0; - /* structure to pass status variables to MySQL */ UNIV_INTERN export_struc export_vars; @@ -399,6 +397,7 @@ UNIV_INTERN ulong srv_ibuf_active_contra UNIV_INTERN ulong srv_ibuf_accel_rate = 100; #define PCT_IBUF_IO(pct) ((ulint) (srv_io_capacity * srv_ibuf_accel_rate * ((double) pct / 10000.0))) +UNIV_INTERN ulint srv_checkpoint_age_target = 0; UNIV_INTERN ulong srv_flush_neighbor_pages = 1; /* 0:disable 1:enable */ UNIV_INTERN ulong srv_enable_unsafe_group_commit = 0; /* 0:disable 1:enable */ @@ -406,6 +405,7 @@ UNIV_INTERN ulong srv_read_ahead = 3; /* UNIV_INTERN ulong srv_adaptive_checkpoint = 0; /* 0: none 1: reflex 2: estimate */ UNIV_INTERN ulong srv_expand_import = 0; /* 0:disable 1:enable */ +UNIV_INTERN ulint srv_relax_table_creation = 0; /* 0:disable 1:enable */ UNIV_INTERN ulong srv_extra_rsegments = 0; /* extra rseg for users */ UNIV_INTERN ulong srv_dict_size_limit = 0; @@ -494,8 +494,6 @@ static ulint srv_main_background_loops static ulint srv_main_flush_loops = 0; /* Log writes involving flush. */ static ulint srv_log_writes_and_flush = 0; -/* Log writes not including flush. */ -static ulint srv_log_buffer_writes = 0; /* This is only ever touched by the master thread. It records the time when the last flush of log file has happened. The master @@ -644,7 +642,7 @@ future, but at the moment we plan to imp which could be called a global priority inheritance. If a thread has to wait for a long time, say 300 milliseconds, for a resource, we just guess that it may be waiting for a resource owned by a background -thread, and boost the the priority of all runnable background threads +thread, and boost the priority of all runnable background threads to the normal level. The background threads then themselves adjust their fixed priority back to background after releasing all resources they had (or, at some fixed points in their program code). @@ -744,9 +742,8 @@ srv_print_master_thread_info( srv_main_1_second_loops, srv_main_sleeps, srv_main_10_second_loops, srv_main_background_loops, srv_main_flush_loops); - fprintf(file, "srv_master_thread log flush and writes: %lu " - " log writes only: %lu\n", - srv_log_writes_and_flush, srv_log_buffer_writes); + fprintf(file, "srv_master_thread log flush and writes: %lu\n", + srv_log_writes_and_flush); } /*********************************************************************//** @@ -1044,13 +1041,26 @@ srv_init(void) } /*********************************************************************//** -Frees the OS fast mutex created in srv_init(). */ +Frees the data structures created in srv_init(). */ UNIV_INTERN void srv_free(void) /*==========*/ { os_fast_mutex_free(&srv_conc_mutex); + mem_free(srv_conc_slots); + srv_conc_slots = NULL; + + mem_free(srv_sys->threads); + mem_free(srv_sys); + srv_sys = NULL; + + mem_free(kernel_mutex_temp); + kernel_mutex_temp = NULL; + mem_free(srv_mysql_table); + srv_mysql_table = NULL; + + trx_i_s_cache_free(trx_i_s_cache); } /*********************************************************************//** @@ -1062,6 +1072,8 @@ srv_general_init(void) /*==================*/ { ut_mem_init(); + /* Reset the system variables in the recovery module. */ + recv_sys_var_init(); os_sync_init(); sync_init(); mem_init(srv_mem_pool_size); @@ -1077,7 +1089,7 @@ UNIV_INTERN ulong srv_max_purge_lag = 0 Puts an OS thread to wait if there are too many concurrent threads (>= srv_thread_concurrency) inside InnoDB. The threads wait in a FIFO queue. */ -#ifdef INNODB_RW_LOCKS_USE_ATOMICS +#ifdef HAVE_ATOMIC_BUILTINS static void enter_innodb_with_tickets(trx_t* trx) { @@ -1155,6 +1167,10 @@ srv_conc_enter_innodb( ibool has_slept = FALSE; srv_conc_slot_t* slot = NULL; ulint i; + ib_uint64_t start_time = 0L; + ib_uint64_t finish_time = 0L; + ulint sec; + ulint ms; if (trx->mysql_thd != NULL && thd_is_replication_slave_thread(trx->mysql_thd)) { @@ -1175,7 +1191,7 @@ srv_conc_enter_innodb( return; } -#ifdef INNODB_RW_LOCKS_USE_ATOMICS +#ifdef HAVE_ATOMIC_BUILTINS if (srv_thread_concurrency_timer_based) { srv_conc_enter_innodb_timer_based(trx); return; @@ -1231,6 +1247,7 @@ retry: switches. */ if (SRV_THREAD_SLEEP_DELAY > 0) { os_thread_sleep(SRV_THREAD_SLEEP_DELAY); + trx->innodb_que_wait_timer += SRV_THREAD_SLEEP_DELAY; } trx->op_info = ""; @@ -1286,12 +1303,25 @@ retry: /* Go to wait for the event; when a thread leaves InnoDB it will release this thread */ + if (innobase_get_slow_log() && trx->take_stats) { + ut_usectime(&sec, &ms); + start_time = (ib_uint64_t)sec * 1000000 + ms; + } else { + start_time = 0; + } + trx->op_info = "waiting in InnoDB queue"; os_event_wait(slot->event); trx->op_info = ""; + if (innobase_get_slow_log() && trx->take_stats && start_time) { + ut_usectime(&sec, &ms); + finish_time = (ib_uint64_t)sec * 1000000 + ms; + trx->innodb_que_wait_timer += (ulint)(finish_time - start_time); + } + os_fast_mutex_lock(&srv_conc_mutex); srv_conc_n_waiting_threads--; @@ -1325,7 +1355,7 @@ srv_conc_force_enter_innodb( } ut_ad(srv_conc_n_threads >= 0); -#ifdef INNODB_RW_LOCKS_USE_ATOMICS +#ifdef HAVE_ATOMIC_BUILTINS if (srv_thread_concurrency_timer_based) { os_atomic_increment_lint(&srv_conc_n_threads, 1); trx->declared_to_be_inside_innodb = TRUE; @@ -1366,7 +1396,7 @@ srv_conc_force_exit_innodb( return; } -#ifdef INNODB_RW_LOCKS_USE_ATOMICS +#ifdef HAVE_ATOMIC_BUILTINS if (srv_thread_concurrency_timer_based) { srv_conc_exit_innodb_timer_based(trx); return; @@ -2081,14 +2111,16 @@ srv_export_innodb_status(void) export_vars.innodb_data_writes = os_n_file_writes; export_vars.innodb_data_written = srv_data_written; export_vars.innodb_dict_tables= (dict_sys ? UT_LIST_GET_LEN(dict_sys->table_LRU) : 0); - export_vars.innodb_buffer_pool_read_requests = buf_pool->n_page_gets; + export_vars.innodb_buffer_pool_read_requests = buf_pool->stat.n_page_gets; export_vars.innodb_buffer_pool_write_requests = srv_buf_pool_write_requests; export_vars.innodb_buffer_pool_wait_free = srv_buf_pool_wait_free; export_vars.innodb_buffer_pool_pages_flushed = srv_buf_pool_flushed; export_vars.innodb_buffer_pool_reads = srv_buf_pool_reads; - export_vars.innodb_buffer_pool_read_ahead_rnd = srv_read_ahead_rnd; - export_vars.innodb_buffer_pool_read_ahead_seq = srv_read_ahead_seq; + export_vars.innodb_buffer_pool_read_ahead + = buf_pool->stat.n_ra_pages_read; + export_vars.innodb_buffer_pool_read_ahead_evicted + = buf_pool->stat.n_ra_pages_evicted; export_vars.innodb_buffer_pool_pages_data = UT_LIST_GET_LEN(buf_pool->LRU); export_vars.innodb_buffer_pool_pages_dirty @@ -2119,9 +2151,9 @@ srv_export_innodb_status(void) export_vars.innodb_log_writes = srv_log_writes; export_vars.innodb_dblwr_pages_written = srv_dblwr_pages_written; export_vars.innodb_dblwr_writes = srv_dblwr_writes; - export_vars.innodb_pages_created = buf_pool->n_pages_created; - export_vars.innodb_pages_read = buf_pool->n_pages_read; - export_vars.innodb_pages_written = buf_pool->n_pages_written; + export_vars.innodb_pages_created = buf_pool->stat.n_pages_created; + export_vars.innodb_pages_read = buf_pool->stat.n_pages_read; + export_vars.innodb_pages_written = buf_pool->stat.n_pages_written; export_vars.innodb_row_lock_waits = srv_n_lock_wait_count; export_vars.innodb_row_lock_current_waits = srv_n_lock_wait_current_count; @@ -2488,12 +2520,6 @@ srv_sync_log_buffer_in_background(void) log_buffer_sync_in_background(TRUE); srv_last_log_flush_time = current_time; srv_log_writes_and_flush++; - } else { - /* Actually we don't need to write logs here. - We are just being extra safe here by forcing - the log buffer to log file. */ - log_buffer_sync_in_background(FALSE); - srv_log_buffer_writes++; } } @@ -2533,10 +2559,9 @@ srv_master_thread( srv_main_thread_process_no = os_proc_get_number(); srv_main_thread_id = os_thread_pf(os_thread_get_curr_id()); - srv_table_reserve_slot(SRV_MASTER); - mutex_enter(&kernel_mutex); + srv_table_reserve_slot(SRV_MASTER); srv_n_threads_active[SRV_MASTER]++; mutex_exit(&kernel_mutex); @@ -2551,8 +2576,8 @@ loop: srv_main_thread_op_info = "reserving kernel mutex"; - n_ios_very_old = log_sys->n_log_ios + buf_pool->n_pages_read - + buf_pool->n_pages_written; + n_ios_very_old = log_sys->n_log_ios + buf_pool->stat.n_pages_read + + buf_pool->stat.n_pages_written; mutex_enter(&kernel_mutex); /* Store the user activity counter at the start of this loop */ @@ -2572,8 +2597,8 @@ loop: skip_sleep = FALSE; for (i = 0; i < 10; i++) { - n_ios_old = log_sys->n_log_ios + buf_pool->n_pages_read - + buf_pool->n_pages_written; + n_ios_old = log_sys->n_log_ios + buf_pool->stat.n_pages_read + + buf_pool->stat.n_pages_written; srv_main_thread_op_info = "sleeping"; srv_main_1_second_loops++; @@ -2625,8 +2650,8 @@ loop: n_pend_ios = buf_get_n_pending_ios() + log_sys->n_pending_writes; - n_ios = log_sys->n_log_ios + buf_pool->n_pages_read - + buf_pool->n_pages_written; + n_ios = log_sys->n_log_ios + buf_pool->stat.n_pages_read + + buf_pool->stat.n_pages_written; if (n_pend_ios < SRV_PEND_IO_THRESHOLD && (n_ios - n_ios_old < SRV_RECENT_IO_ACTIVITY)) { srv_main_thread_op_info = "doing insert buffer merge"; @@ -2642,6 +2667,8 @@ loop: /* Try to keep the number of modified pages in the buffer pool under the limit wished by the user */ + srv_main_thread_op_info = + "flushing buffer pool pages"; n_pages_flushed = buf_flush_batch(BUF_FLUSH_LIST, PCT_IO(100), IB_ULONGLONG_MAX); @@ -2664,6 +2691,8 @@ loop: ulint n_flush = buf_flush_get_desired_flush_rate(); if (n_flush) { + srv_main_thread_op_info = + "flushing buffer pool pages"; n_flush = ut_min(PCT_IO(100), n_flush); n_pages_flushed = buf_flush_batch( @@ -2835,8 +2864,8 @@ retry_flush_batch: are not required, and may be disabled. */ n_pend_ios = buf_get_n_pending_ios() + log_sys->n_pending_writes; - n_ios = log_sys->n_log_ios + buf_pool->n_pages_read - + buf_pool->n_pages_written; + n_ios = log_sys->n_log_ios + buf_pool->stat.n_pages_read + + buf_pool->stat.n_pages_written; srv_main_10_second_loops++; if (n_pend_ios < SRV_PEND_IO_THRESHOLD @@ -3130,20 +3159,34 @@ srv_purge_thread( ulint n_pages_purged_sum = 1; /* dummy */ ulint history_len; ulint sleep_ms= 10000; /* initial: 10 sec. */ + ibool can_be_last = FALSE; #ifdef UNIV_DEBUG_THREAD_CREATION fprintf(stderr, "Purge thread starts, id %lu\n", os_thread_pf(os_thread_get_curr_id())); #endif - srv_table_reserve_slot(SRV_PURGE); mutex_enter(&kernel_mutex); + srv_table_reserve_slot(SRV_PURGE); srv_n_threads_active[SRV_PURGE]++; mutex_exit(&kernel_mutex); loop: - if (srv_fast_shutdown && srv_shutdown_state > 0) { - goto exit_func; + if (srv_shutdown_state > 0) { + if (srv_fast_shutdown) { + /* someone other should wait the end of the workers */ + goto exit_func; + } + + mutex_enter(&kernel_mutex); + if (srv_n_threads_active[SRV_PURGE_WORKER]) { + can_be_last = FALSE; + } else { + can_be_last = TRUE; + } + mutex_exit(&kernel_mutex); + + sleep_ms = 10; } os_thread_sleep( sleep_ms * 1000 ); @@ -3164,6 +3207,15 @@ loop: n_pages_purged_sum += n_pages_purged; } while (n_pages_purged); + if (srv_shutdown_state > 0 && can_be_last) { + /* the last trx_purge() is executed without workers */ + goto exit_func; + } + + if (n_pages_purged_sum) { + srv_active_wake_master_thread(); + } + if (n_pages_purged_sum == 0) sleep_ms *= 10; if (sleep_ms > 10000) @@ -3172,9 +3224,62 @@ loop: goto loop; exit_func: - /* We count the number of threads in os_thread_exit(). A created - thread should always use that to exit and not use return() to exit. */ + trx_purge_worker_wake(); /* It may not make sense. for safety only */ + /* wake master thread to flush the pages */ + srv_wake_master_thread(); + + mutex_enter(&kernel_mutex); + srv_n_threads_active[SRV_PURGE]--; + mutex_exit(&kernel_mutex); + os_thread_exit(NULL); + + OS_THREAD_DUMMY_RETURN; +} + +/************************************************************************* +A thread which is devoted to purge, for take over the master thread's +purging */ +UNIV_INTERN +os_thread_ret_t +srv_purge_worker_thread( +/*====================*/ + void* arg) +{ + ulint worker_id; /* index for array */ + + worker_id = *((ulint*)arg); + +#ifdef UNIV_DEBUG_THREAD_CREATION + fprintf(stderr, "Purge worker thread starts, id %lu\n", + os_thread_pf(os_thread_get_curr_id())); +#endif + mutex_enter(&kernel_mutex); + srv_table_reserve_slot(SRV_PURGE_WORKER); + srv_n_threads_active[SRV_PURGE_WORKER]++; + mutex_exit(&kernel_mutex); + +loop: + /* purge worker threads only works when srv_shutdown_state==0 */ + /* for safety and exactness. */ + if (srv_shutdown_state > 0) { + goto exit_func; + } + + trx_purge_worker_wait(); + + if (srv_shutdown_state > 0) { + goto exit_func; + } + + trx_purge_worker(worker_id); + + goto loop; + +exit_func: + mutex_enter(&kernel_mutex); + srv_n_threads_active[SRV_PURGE_WORKER]--; + mutex_exit(&kernel_mutex); os_thread_exit(NULL); OS_THREAD_DUMMY_RETURN; === modified file 'storage/xtradb/srv/srv0start.c' --- a/storage/xtradb/srv/srv0start.c 2009-11-13 21:26:08 +0000 +++ b/storage/xtradb/srv/srv0start.c 2010-01-17 08:41:43 +0000 @@ -103,6 +103,8 @@ Created 2/16/1996 Heikki Tuuri # include "row0row.h" # include "row0mysql.h" # include "btr0pcur.h" +# include "thr0loc.h" +# include "os0sync.h" /* for INNODB_RW_LOCKS_USE_ATOMICS */ /** Log sequence number immediately after startup */ UNIV_INTERN ib_uint64_t srv_start_lsn; @@ -141,9 +143,9 @@ static mutex_t ios_mutex; static ulint ios; /** io_handler_thread parameters for thread identification */ -static ulint n[SRV_MAX_N_IO_THREADS + 5]; +static ulint n[SRV_MAX_N_IO_THREADS + 5 + 64]; /** io_handler_thread identifiers */ -static os_thread_id_t thread_ids[SRV_MAX_N_IO_THREADS + 5]; +static os_thread_id_t thread_ids[SRV_MAX_N_IO_THREADS + 5 + 64]; /** We use this mutex to test the return value of pthread_mutex_trylock on successful locking. HP-UX does NOT return 0, though Linux et al do. */ @@ -494,6 +496,8 @@ io_handler_thread( mutex_exit(&ios_mutex); } + thr_local_free(os_thread_get_curr_id()); + /* We count the number of threads in os_thread_exit(). A created thread should always use that to exit and not use return() to exit. The thread actually never comes here because it is exited in an @@ -530,32 +534,6 @@ srv_normalize_path_for_win( #endif } -/*********************************************************************//** -Adds a slash or a backslash to the end of a string if it is missing -and the string is not empty. -@return string which has the separator if the string is not empty */ -UNIV_INTERN -char* -srv_add_path_separator_if_needed( -/*=============================*/ - char* str) /*!< in: null-terminated character string */ -{ - char* out_str; - ulint len = ut_strlen(str); - - if (len == 0 || str[len - 1] == SRV_PATH_SEPARATOR) { - - return(str); - } - - out_str = ut_malloc(len + 2); - memcpy(out_str, str, len); - out_str[len] = SRV_PATH_SEPARATOR; - out_str[len + 1] = 0; - - return(out_str); -} - #ifndef UNIV_HOTBACKUP /*********************************************************************//** Calculates the low 32 bits when a file size which is given as a number @@ -604,19 +582,24 @@ open_or_create_log_file( ulint size; ulint size_high; char name[10000]; + ulint dirnamelen; UT_NOT_USED(create_new_db); *log_file_created = FALSE; srv_normalize_path_for_win(srv_log_group_home_dirs[k]); - srv_log_group_home_dirs[k] = srv_add_path_separator_if_needed( - srv_log_group_home_dirs[k]); - ut_a(strlen(srv_log_group_home_dirs[k]) - < (sizeof name) - 10 - sizeof "ib_logfile"); - sprintf(name, "%s%s%lu", srv_log_group_home_dirs[k], - "ib_logfile", (ulong) i); + dirnamelen = strlen(srv_log_group_home_dirs[k]); + ut_a(dirnamelen < (sizeof name) - 10 - sizeof "ib_logfile"); + memcpy(name, srv_log_group_home_dirs[k], dirnamelen); + + /* Add a path separator if needed. */ + if (dirnamelen && name[dirnamelen - 1] != SRV_PATH_SEPARATOR) { + name[dirnamelen++] = SRV_PATH_SEPARATOR; + } + + sprintf(name + dirnamelen, "%s%lu", "ib_logfile", (ulong) i); files[i] = os_file_create(name, OS_FILE_CREATE, OS_FILE_NORMAL, OS_LOG_FILE, &ret); @@ -779,14 +762,22 @@ open_or_create_data_files( *create_new_db = FALSE; srv_normalize_path_for_win(srv_data_home); - srv_data_home = srv_add_path_separator_if_needed(srv_data_home); for (i = 0; i < srv_n_data_files; i++) { + ulint dirnamelen; + srv_normalize_path_for_win(srv_data_file_names[i]); + dirnamelen = strlen(srv_data_home); - ut_a(strlen(srv_data_home) + strlen(srv_data_file_names[i]) + ut_a(dirnamelen + strlen(srv_data_file_names[i]) < (sizeof name) - 1); - sprintf(name, "%s%s", srv_data_home, srv_data_file_names[i]); + memcpy(name, srv_data_home, dirnamelen); + /* Add a path separator if needed. */ + if (dirnamelen && name[dirnamelen - 1] != SRV_PATH_SEPARATOR) { + name[dirnamelen++] = SRV_PATH_SEPARATOR; + } + + strcpy(name + dirnamelen, srv_data_file_names[i]); if (srv_data_file_is_raw_partition[i] == 0) { @@ -1008,7 +999,7 @@ skip_size_check: return(DB_SUCCESS); } -/****************************************************************//** +/******************************************************************** Starts InnoDB and creates a new database if database files are not found and the user wants. @return DB_SUCCESS or error code */ @@ -1096,6 +1087,10 @@ innobase_start_or_create_for_mysql(void) "InnoDB: !!!!!!!! UNIV_SEARCH_DEBUG switched on !!!!!!!!!\n"); #endif +#ifdef UNIV_LOG_LSN_DEBUG + fprintf(stderr, + "InnoDB: !!!!!!!! UNIV_LOG_LSN_DEBUG switched on !!!!!!!!!\n"); +#endif /* UNIV_LOG_LSN_DEBUG */ #ifdef UNIV_MEM_DEBUG fprintf(stderr, "InnoDB: !!!!!!!! UNIV_MEM_DEBUG switched on !!!!!!!!!\n"); @@ -1106,34 +1101,7 @@ innobase_start_or_create_for_mysql(void) "InnoDB: The InnoDB memory heap is disabled\n"); } -#ifdef HAVE_GCC_ATOMIC_BUILTINS -# ifdef INNODB_RW_LOCKS_USE_ATOMICS - fprintf(stderr, - "InnoDB: Mutexes and rw_locks use GCC atomic builtins.\n"); -# else /* INNODB_RW_LOCKS_USE_ATOMICS */ - fprintf(stderr, - "InnoDB: Mutexes use GCC atomic builtins, rw_locks do not.\n"); -# endif /* INNODB_RW_LOCKS_USE_ATOMICS */ -#elif defined(HAVE_SOLARIS_ATOMICS) -# ifdef INNODB_RW_LOCKS_USE_ATOMICS - fprintf(stderr, - "InnoDB: Mutexes and rw_locks use Solaris atomic functions.\n"); -# else - fprintf(stderr, - "InnoDB: Mutexes use Solaris atomic functions.\n"); -# endif /* INNODB_RW_LOCKS_USE_ATOMICS */ -#elif HAVE_WINDOWS_ATOMICS -# ifdef INNODB_RW_LOCKS_USE_ATOMICS - fprintf(stderr, - "InnoDB: Mutexes and rw_locks use Windows interlocked functions.\n"); -# else - fprintf(stderr, - "InnoDB: Mutexes use Windows interlocked functions.\n"); -# endif /* INNODB_RW_LOCKS_USE_ATOMICS */ -#else /* HAVE_GCC_ATOMIC_BUILTINS */ - fprintf(stderr, - "InnoDB: Neither mutexes nor rw_locks use GCC atomic builtins.\n"); -#endif /* HAVE_GCC_ATOMIC_BUILTINS */ + fprintf(stderr, "InnoDB: %s\n", IB_ATOMICS_STARTUP_MSG); /* Since InnoDB does not currently clean up all its internal data structures in MySQL Embedded Server Library server_end(), we @@ -1142,7 +1110,7 @@ innobase_start_or_create_for_mysql(void) if (srv_start_has_been_called) { fprintf(stderr, - "InnoDB: Error:startup called second time" + "InnoDB: Error: startup called second time" " during the process lifetime.\n" "InnoDB: In the MySQL Embedded Server Library" " you cannot call server_init()\n" @@ -1409,7 +1377,7 @@ innobase_start_or_create_for_mysql(void) sum_of_new_sizes += srv_data_file_sizes[i]; } - if (sum_of_new_sizes < 640) { + if (sum_of_new_sizes < 10485760 / UNIV_PAGE_SIZE) { fprintf(stderr, "InnoDB: Error: tablespace size must be" " at least 10 MB\n"); @@ -1739,8 +1707,17 @@ innobase_start_or_create_for_mysql(void) + (1 + SRV_MAX_N_IO_THREADS)); if (srv_use_purge_thread) { + ulint i; + os_thread_create(&srv_purge_thread, NULL, thread_ids + (4 + SRV_MAX_N_IO_THREADS)); + + for (i = 0; i < srv_use_purge_thread - 1; i++) { + n[5 + i + SRV_MAX_N_IO_THREADS] = i; /* using as index for arrays in purge_sys */ + os_thread_create(&srv_purge_worker_thread, + n + (5 + i + SRV_MAX_N_IO_THREADS), + thread_ids + (5 + i + SRV_MAX_N_IO_THREADS)); + } } #ifdef UNIV_DEBUG /* buf_debug_prints = TRUE; */ @@ -1853,7 +1830,7 @@ innobase_start_or_create_for_mysql(void) /* Actually, we did not change the undo log format between 4.0 and 4.1.1, and we would not need to run purge to completion. Note also that the purge algorithm in 4.1.1 - can process the the history list again even after a full + can process the history list again even after a full purge, because our algorithm does not cut the end of the history list in all cases so that it would become empty after a full purge. That mean that we may purge 4.0 type @@ -2005,8 +1982,10 @@ innobase_shutdown_for_mysql(void) /* All the threads have exited or are just exiting; NOTE that the threads may not have completed their exit yet. Should we use pthread_join() to make sure - they have exited? Now we just sleep 0.1 seconds and - hope that is enough! */ + they have exited? If we did, we would have to + remove the pthread_detach() from + os_thread_exit(). Now we just sleep 0.1 + seconds and hope that is enough! */ os_mutex_exit(os_sync_mutex); @@ -2045,36 +2024,40 @@ innobase_shutdown_for_mysql(void) srv_misc_tmpfile = 0; } + /* This must be disabled before closing the buffer pool + and closing the data dictionary. */ + btr_search_disable(); + + ibuf_close(); + log_shutdown(); + lock_sys_close(); + thr_local_close(); trx_sys_file_format_close(); + trx_sys_close(); mutex_free(&srv_monitor_file_mutex); mutex_free(&srv_dict_tmpfile_mutex); mutex_free(&srv_misc_tmpfile_mutex); + dict_close(); + btr_search_sys_free(); /* 3. Free all InnoDB's own mutexes and the os_fast_mutexes inside them */ + os_aio_free(); sync_close(); + srv_free(); + fil_close(); /* 4. Free the os_conc_mutex and all os_events and os_mutexes */ - srv_free(); os_sync_free(); - /* Check that all read views are closed except read view owned - by a purge. */ - - if (UT_LIST_GET_LEN(trx_sys->view_list) > 1) { - fprintf(stderr, - "InnoDB: Error: all read views were not closed" - " before shutdown:\n" - "InnoDB: %lu read views open \n", - UT_LIST_GET_LEN(trx_sys->view_list) - 1); - } - - /* 5. Free all allocated memory and the os_fast_mutex created in - ut0mem.c */ + /* 5. Free all allocated memory */ + pars_lexer_close(); + log_mem_free(); buf_pool_free(); + mem_close(); ut_free_all_mem(); if (os_thread_count != 0 @@ -2106,6 +2089,7 @@ innobase_shutdown_for_mysql(void) } srv_was_started = FALSE; + srv_start_has_been_called = FALSE; return((int) DB_SUCCESS); } === modified file 'storage/xtradb/sync/sync0arr.c' --- a/storage/xtradb/sync/sync0arr.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/sync/sync0arr.c 2010-01-06 12:00:14 +0000 @@ -227,24 +227,21 @@ sync_array_create( SYNC_ARRAY_MUTEX: determines the type of mutex protecting the data structure */ { + ulint sz; sync_array_t* arr; - sync_cell_t* cell_array; - sync_cell_t* cell; - ulint i; ut_a(n_cells > 0); /* Allocate memory for the data structures */ arr = ut_malloc(sizeof(sync_array_t)); + memset(arr, 0x0, sizeof(*arr)); - cell_array = ut_malloc(sizeof(sync_cell_t) * n_cells); + sz = sizeof(sync_cell_t) * n_cells; + arr->array = ut_malloc(sz); + memset(arr->array, 0x0, sz); arr->n_cells = n_cells; - arr->n_reserved = 0; - arr->array = cell_array; arr->protection = protection; - arr->sg_count = 0; - arr->res_count = 0; /* Then create the mutex to protect the wait array complex */ if (protection == SYNC_ARRAY_OS_MUTEX) { @@ -255,13 +252,6 @@ sync_array_create( ut_error; } - for (i = 0; i < n_cells; i++) { - cell = sync_array_get_nth_cell(arr, i); - cell->wait_object = NULL; - cell->waiting = FALSE; - cell->signal_count = 0; - } - return(arr); } @@ -492,12 +482,12 @@ sync_array_cell_print( mutex = cell->old_wait_mutex; fprintf(file, - "Mutex at %p created file %s line %lu, lock var %lu\n" + "Mutex at %p '%s', lock var %lu\n" #ifdef UNIV_SYNC_DEBUG "Last time reserved in file %s line %lu, " #endif /* UNIV_SYNC_DEBUG */ "waiters flag %lu\n", - (void*) mutex, mutex->cfile_name, (ulong) mutex->cline, + (void*) mutex, mutex->cmutex_name, (ulong) mutex->lock_word, #ifdef UNIV_SYNC_DEBUG mutex->file_name, (ulong) mutex->line, @@ -513,9 +503,8 @@ sync_array_cell_print( rwlock = cell->old_wait_rw_lock; fprintf(file, - " RW-latch at %p created in file %s line %lu\n", - (void*) rwlock, rwlock->cfile_name, - (ulong) rwlock->cline); + " RW-latch at %p '%s'\n", + (void*) rwlock, rwlock->lock_name); writer = rw_lock_get_writer(rwlock); if (writer != RW_LOCK_NOT_LOCKED) { fprintf(file, === modified file 'storage/xtradb/sync/sync0rw.c' --- a/storage/xtradb/sync/sync0rw.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/sync/sync0rw.c 2010-01-06 12:00:14 +0000 @@ -38,6 +38,7 @@ Created 9/11/1995 Heikki Tuuri #include "os0thread.h" #include "mem0mem.h" #include "srv0srv.h" +#include "os0sync.h" /* for INNODB_RW_LOCKS_USE_ATOMICS */ /* IMPLEMENTATION OF THE RW_LOCK @@ -230,8 +231,8 @@ rw_lock_create_func( # ifdef UNIV_SYNC_DEBUG ulint level, /*!< in: level */ # endif /* UNIV_SYNC_DEBUG */ - const char* cmutex_name, /*!< in: mutex name */ #endif /* UNIV_DEBUG */ + const char* cmutex_name, /*!< in: mutex name */ const char* cfile_name, /*!< in: file name where created */ ulint cline) /*!< in: file line where created */ { @@ -241,14 +242,15 @@ rw_lock_create_func( #ifndef INNODB_RW_LOCKS_USE_ATOMICS mutex_create(rw_lock_get_mutex(lock), SYNC_NO_ORDER_CHECK); - lock->mutex.cfile_name = cfile_name; - lock->mutex.cline = cline; + ut_d(lock->mutex.cfile_name = cfile_name); + ut_d(lock->mutex.cline = cline); - ut_d(lock->mutex.cmutex_name = cmutex_name); + lock->mutex.cmutex_name = cmutex_name; ut_d(lock->mutex.mutex_type = 1); #else /* INNODB_RW_LOCKS_USE_ATOMICS */ # ifdef UNIV_DEBUG - UT_NOT_USED(cmutex_name); + UT_NOT_USED(cfile_name); + UT_NOT_USED(cline); # endif #endif /* INNODB_RW_LOCKS_USE_ATOMICS */ @@ -268,8 +270,7 @@ rw_lock_create_func( lock->magic_n = RW_LOCK_MAGIC_N; - lock->cfile_name = cfile_name; - lock->cline = (unsigned int) cline; + lock->lock_name = cmutex_name; lock->count_os_wait = 0; lock->last_s_file_name = "not yet reserved"; @@ -304,8 +305,6 @@ rw_lock_free( ut_ad(rw_lock_validate(lock)); ut_a(lock->lock_word == X_LOCK_DECR); - lock->magic_n = 0; - #ifndef INNODB_RW_LOCKS_USE_ATOMICS mutex_free(rw_lock_get_mutex(lock)); #endif /* INNODB_RW_LOCKS_USE_ATOMICS */ @@ -325,6 +324,8 @@ rw_lock_free( UT_LIST_REMOVE(list, rw_lock_list, lock); mutex_exit(&rw_lock_list_mutex); + + lock->magic_n = 0; } #ifdef UNIV_DEBUG @@ -390,10 +391,10 @@ lock_loop: if (srv_print_latch_waits) { fprintf(stderr, "Thread %lu spin wait rw-s-lock at %p" - " cfile %s cline %lu rnds %lu\n", + " '%s' rnds %lu\n", (ulong) os_thread_pf(os_thread_get_curr_id()), (void*) lock, - lock->cfile_name, (ulong) lock->cline, (ulong) i); + lock->lock_name, (ulong) i); } /* We try once again to obtain the lock */ @@ -426,10 +427,9 @@ lock_loop: if (srv_print_latch_waits) { fprintf(stderr, "Thread %lu OS wait rw-s-lock at %p" - " cfile %s cline %lu\n", + " '%s'\n", os_thread_pf(os_thread_get_curr_id()), - (void*) lock, lock->cfile_name, - (ulong) lock->cline); + (void*) lock, lock->lock_name); } /* these stats may not be accurate */ @@ -648,9 +648,9 @@ lock_loop: if (srv_print_latch_waits) { fprintf(stderr, "Thread %lu spin wait rw-x-lock at %p" - " cfile %s cline %lu rnds %lu\n", + " '%s' rnds %lu\n", os_thread_pf(os_thread_get_curr_id()), (void*) lock, - lock->cfile_name, (ulong) lock->cline, (ulong) i); + lock->lock_name, (ulong) i); } sync_array_reserve_cell(sync_primary_wait_array, @@ -671,9 +671,9 @@ lock_loop: if (srv_print_latch_waits) { fprintf(stderr, "Thread %lu OS wait for rw-x-lock at %p" - " cfile %s cline %lu\n", + " '%s'\n", os_thread_pf(os_thread_get_curr_id()), (void*) lock, - lock->cfile_name, (ulong) lock->cline); + lock->lock_name); } /* these stats may not be accurate */ === modified file 'storage/xtradb/sync/sync0sync.c' --- a/storage/xtradb/sync/sync0sync.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/sync/sync0sync.c 2010-01-06 12:00:14 +0000 @@ -39,6 +39,7 @@ Created 9/5/1995 Heikki Tuuri #include "buf0buf.h" #include "srv0srv.h" #include "buf0types.h" +#include "os0sync.h" /* for HAVE_ATOMIC_BUILTINS */ /* REASONS FOR IMPLEMENTING THE SPIN LOCK MUTEX @@ -237,8 +238,8 @@ void mutex_create_func( /*==============*/ mutex_t* mutex, /*!< in: pointer to memory */ -#ifdef UNIV_DEBUG const char* cmutex_name, /*!< in: mutex name */ +#ifdef UNIV_DEBUG # ifdef UNIV_SYNC_DEBUG ulint level, /*!< in: level */ # endif /* UNIV_SYNC_DEBUG */ @@ -253,7 +254,7 @@ mutex_create_func( mutex->lock_word = 0; #endif mutex->event = os_event_create(NULL); - mutex_set_waiters(mutex, 0); + mutex->waiters = 0; #ifdef UNIV_DEBUG mutex->magic_n = MUTEX_MAGIC_N; #endif /* UNIV_DEBUG */ @@ -262,11 +263,13 @@ mutex_create_func( mutex->file_name = "not yet reserved"; mutex->level = level; #endif /* UNIV_SYNC_DEBUG */ +#ifdef UNIV_DEBUG mutex->cfile_name = cfile_name; mutex->cline = cline; +#endif /* UNIV_DEBUG */ mutex->count_os_wait = 0; -#ifdef UNIV_DEBUG mutex->cmutex_name= cmutex_name; +#ifdef UNIV_DEBUG mutex->count_using= 0; mutex->mutex_type= 0; mutex->lspent_time= 0; @@ -424,10 +427,18 @@ mutex_set_waiters( the value is stored to memory */ ut_ad(mutex); +#ifdef INNODB_RW_LOCKS_USE_ATOMICS + if (n) { + os_compare_and_swap_ulint(&mutex->waiters, 0, 1); + } else { + os_compare_and_swap_ulint(&mutex->waiters, 1, 0); + } +#else ptr = &(mutex->waiters); *ptr = n; /* Here we assume that the write of a single word in memory is atomic */ +#endif } /******************************************************************//** @@ -498,9 +509,9 @@ spin_loop: #ifdef UNIV_SRV_PRINT_LATCH_WAITS fprintf(stderr, "Thread %lu spin wait mutex at %p" - " cfile %s cline %lu rnds %lu\n", + " '%s' rnds %lu\n", (ulong) os_thread_pf(os_thread_get_curr_id()), (void*) mutex, - mutex->cfile_name, (ulong) mutex->cline, (ulong) i); + mutex->cmutex_name, (ulong) i); #endif mutex_spin_round_count += i; @@ -575,9 +586,9 @@ spin_loop: #ifdef UNIV_SRV_PRINT_LATCH_WAITS fprintf(stderr, - "Thread %lu OS wait mutex at %p cfile %s cline %lu rnds %lu\n", + "Thread %lu OS wait mutex at %p '%s' rnds %lu\n", (ulong) os_thread_pf(os_thread_get_curr_id()), (void*) mutex, - mutex->cfile_name, (ulong) mutex->cline, (ulong) i); + mutex->cmutex_name, (ulong) i); #endif mutex_os_wait_count++; @@ -849,7 +860,8 @@ sync_thread_levels_g( /*=================*/ sync_level_t* arr, /*!< in: pointer to level array for an OS thread */ - ulint limit) /*!< in: level limit */ + ulint limit, /*!< in: level limit */ + ulint warn) /*!< in: TRUE=display a diagnostic message */ { sync_level_t* slot; rw_lock_t* lock; @@ -863,6 +875,11 @@ sync_thread_levels_g( if (slot->latch != NULL) { if (slot->level <= limit) { + if (!warn) { + + return(FALSE); + } + lock = slot->latch; mutex = slot->latch; @@ -873,9 +890,8 @@ sync_thread_levels_g( if (mutex->magic_n == MUTEX_MAGIC_N) { fprintf(stderr, - "Mutex created at %s %lu\n", - mutex->cfile_name, - (ulong) mutex->cline); + "Mutex '%s'\n", + mutex->cmutex_name); if (mutex_get_lock_word(mutex) != 0) { const char* file_name; @@ -1106,7 +1122,7 @@ sync_thread_add_level( case SYNC_DICT_HEADER: case SYNC_TRX_I_S_RWLOCK: case SYNC_TRX_I_S_LAST_READ: - if (!sync_thread_levels_g(array, level)) { + if (!sync_thread_levels_g(array, level, TRUE)) { fprintf(stderr, "InnoDB: sync_thread_levels_g(array, %lu)" " does not hold!\n", level); @@ -1117,36 +1133,44 @@ sync_thread_add_level( /* Either the thread must own the buffer pool mutex (buf_pool_mutex), or it is allowed to latch only ONE buffer block (block->mutex or buf_pool_zip_mutex). */ - if (!sync_thread_levels_g(array, level)) { - ut_a(sync_thread_levels_g(array, level - 1)); + if (!sync_thread_levels_g(array, level, FALSE)) { + ut_a(sync_thread_levels_g(array, level - 1, TRUE)); ut_a(sync_thread_levels_contain(array, SYNC_BUF_LRU_LIST)); } break; case SYNC_REC_LOCK: - ut_a((sync_thread_levels_contain(array, SYNC_KERNEL) - && sync_thread_levels_g(array, SYNC_REC_LOCK - 1)) - || sync_thread_levels_g(array, SYNC_REC_LOCK)); + if (sync_thread_levels_contain(array, SYNC_KERNEL)) { + ut_a(sync_thread_levels_g(array, SYNC_REC_LOCK - 1, + TRUE)); + } else { + ut_a(sync_thread_levels_g(array, SYNC_REC_LOCK, TRUE)); + } break; case SYNC_IBUF_BITMAP: /* Either the thread must own the master mutex to all the bitmap pages, or it is allowed to latch only ONE bitmap page. */ - ut_a((sync_thread_levels_contain(array, SYNC_IBUF_BITMAP_MUTEX) - && sync_thread_levels_g(array, SYNC_IBUF_BITMAP - 1)) - || sync_thread_levels_g(array, SYNC_IBUF_BITMAP)); + if (sync_thread_levels_contain(array, + SYNC_IBUF_BITMAP_MUTEX)) { + ut_a(sync_thread_levels_g(array, SYNC_IBUF_BITMAP - 1, + TRUE)); + } else { + ut_a(sync_thread_levels_g(array, SYNC_IBUF_BITMAP, + TRUE)); + } break; case SYNC_FSP_PAGE: ut_a(sync_thread_levels_contain(array, SYNC_FSP)); break; case SYNC_FSP: ut_a(sync_thread_levels_contain(array, SYNC_FSP) - || sync_thread_levels_g(array, SYNC_FSP)); + || sync_thread_levels_g(array, SYNC_FSP, TRUE)); break; case SYNC_TRX_UNDO_PAGE: ut_a(sync_thread_levels_contain(array, SYNC_TRX_UNDO) || sync_thread_levels_contain(array, SYNC_RSEG) || sync_thread_levels_contain(array, SYNC_PURGE_SYS) - || sync_thread_levels_g(array, SYNC_TRX_UNDO_PAGE)); + || sync_thread_levels_g(array, SYNC_TRX_UNDO_PAGE, TRUE)); break; case SYNC_RSEG_HEADER: ut_a(sync_thread_levels_contain(array, SYNC_RSEG)); @@ -1158,37 +1182,41 @@ sync_thread_add_level( case SYNC_TREE_NODE: ut_a(sync_thread_levels_contain(array, SYNC_INDEX_TREE) || sync_thread_levels_contain(array, SYNC_DICT_OPERATION) - || sync_thread_levels_g(array, SYNC_TREE_NODE - 1)); + || sync_thread_levels_g(array, SYNC_TREE_NODE - 1, TRUE)); break; case SYNC_TREE_NODE_NEW: ut_a(sync_thread_levels_contain(array, SYNC_FSP_PAGE) || sync_thread_levels_contain(array, SYNC_IBUF_MUTEX)); break; case SYNC_INDEX_TREE: - ut_a((sync_thread_levels_contain(array, SYNC_IBUF_MUTEX) - && sync_thread_levels_contain(array, SYNC_FSP) - && sync_thread_levels_g(array, SYNC_FSP_PAGE - 1)) - || sync_thread_levels_g(array, SYNC_TREE_NODE - 1)); + if (sync_thread_levels_contain(array, SYNC_IBUF_MUTEX) + && sync_thread_levels_contain(array, SYNC_FSP)) { + ut_a(sync_thread_levels_g(array, SYNC_FSP_PAGE - 1, + TRUE)); + } else { + ut_a(sync_thread_levels_g(array, SYNC_TREE_NODE - 1, + TRUE)); + } break; case SYNC_IBUF_MUTEX: - ut_a(sync_thread_levels_g(array, SYNC_FSP_PAGE - 1)); + ut_a(sync_thread_levels_g(array, SYNC_FSP_PAGE - 1, TRUE)); break; case SYNC_IBUF_PESS_INSERT_MUTEX: - ut_a(sync_thread_levels_g(array, SYNC_FSP - 1) - && !sync_thread_levels_contain(array, SYNC_IBUF_MUTEX)); + ut_a(sync_thread_levels_g(array, SYNC_FSP - 1, TRUE)); + ut_a(!sync_thread_levels_contain(array, SYNC_IBUF_MUTEX)); break; case SYNC_IBUF_HEADER: - ut_a(sync_thread_levels_g(array, SYNC_FSP - 1) - && !sync_thread_levels_contain(array, SYNC_IBUF_MUTEX) - && !sync_thread_levels_contain( - array, SYNC_IBUF_PESS_INSERT_MUTEX)); + ut_a(sync_thread_levels_g(array, SYNC_FSP - 1, TRUE)); + ut_a(!sync_thread_levels_contain(array, SYNC_IBUF_MUTEX)); + ut_a(!sync_thread_levels_contain(array, + SYNC_IBUF_PESS_INSERT_MUTEX)); break; case SYNC_DICT: #ifdef UNIV_DEBUG ut_a(buf_debug_prints - || sync_thread_levels_g(array, SYNC_DICT)); + || sync_thread_levels_g(array, SYNC_DICT, TRUE)); #else /* UNIV_DEBUG */ - ut_a(sync_thread_levels_g(array, SYNC_DICT)); + ut_a(sync_thread_levels_g(array, SYNC_DICT, TRUE)); #endif /* UNIV_DEBUG */ break; default: @@ -1364,7 +1392,12 @@ sync_close(void) mutex_free(&mutex_list_mutex); #ifdef UNIV_SYNC_DEBUG mutex_free(&sync_thread_mutex); + + /* Switch latching order checks on in sync0sync.c */ + sync_order_checks_on = FALSE; #endif /* UNIV_SYNC_DEBUG */ + + sync_initialized = FALSE; } /*******************************************************************//** === modified file 'storage/xtradb/thr/thr0loc.c' --- a/storage/xtradb/thr/thr0loc.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/thr/thr0loc.c 2010-01-06 12:00:14 +0000 @@ -63,7 +63,7 @@ struct thr_local_struct{ os_thread_t handle; /*!< operating system handle to the thread */ ulint slot_no;/*!< the index of the slot in the thread table for this thread */ - ibool in_ibuf;/*!< TRUE if the the thread is doing an ibuf + ibool in_ibuf;/*!< TRUE if the thread is doing an ibuf operation */ hash_node_t hash; /*!< hash chain node */ ulint magic_n;/*!< magic number (THR_LOCAL_MAGIC_N) */ @@ -250,6 +250,37 @@ thr_local_init(void) mutex_create(&thr_local_mutex, SYNC_THR_LOCAL); } +/******************************************************************** +Close the thread local storage module. */ +UNIV_INTERN +void +thr_local_close(void) +/*=================*/ +{ + ulint i; + + ut_a(thr_local_hash != NULL); + + /* Free the hash elements. We don't remove them from the table + because we are going to destroy the table anyway. */ + for (i = 0; i < hash_get_n_cells(thr_local_hash); i++) { + thr_local_t* local; + + local = HASH_GET_FIRST(thr_local_hash, i); + + while (local) { + thr_local_t* prev_local = local; + + local = HASH_GET_NEXT(hash, prev_local); + ut_a(prev_local->magic_n == THR_LOCAL_MAGIC_N); + mem_free(prev_local); + } + } + + hash_table_free(thr_local_hash); + thr_local_hash = NULL; +} + /************************************************************************* Return local hash table informations. */ === modified file 'storage/xtradb/trx/trx0i_s.c' --- a/storage/xtradb/trx/trx0i_s.c 2009-11-29 23:08:56 +0000 +++ b/storage/xtradb/trx/trx0i_s.c 2010-01-15 15:58:25 +0000 @@ -60,7 +60,7 @@ Created July 17, 2007 Vasil Dimov /** @brief The maximum number of chunks to allocate for a table cache. The rows of a table cache are stored in a set of chunks. When a new -row is added a new chunk is allocated if necessary. Assuming that the +row is added a new chunk is allocated if necessary. Assuming that the first one is 1024 rows (TABLE_CACHE_INITIAL_ROWSNUM) and each subsequent is N/2 where N is the number of rows we have allocated till now, then 39th chunk would accommodate 1677416425 rows and all chunks @@ -238,6 +238,27 @@ table_cache_init( } /*******************************************************************//** +Frees a table cache. */ +static +void +table_cache_free( +/*=============*/ + i_s_table_cache_t* table_cache) /*!< in/out: table cache */ +{ + ulint i; + + for (i = 0; i < MEM_CHUNKS_IN_TABLE_CACHE; i++) { + + /* the memory is actually allocated in + table_cache_create_empty_row() */ + if (table_cache->chunks[i].base) { + mem_free(table_cache->chunks[i].base); + table_cache->chunks[i].base = NULL; + } + } +} + +/*******************************************************************//** Returns an empty row from a table cache. The row is allocated if no more empty rows are available. The number of used rows is incremented. If the memory limit is hit then NULL is returned and nothing is @@ -1252,6 +1273,22 @@ trx_i_s_cache_init( } /*******************************************************************//** +Free the INFORMATION SCHEMA trx related cache. */ +UNIV_INTERN +void +trx_i_s_cache_free( +/*===============*/ + trx_i_s_cache_t* cache) /*!< in, own: cache to free */ +{ + hash_table_free(cache->locks_hash); + ha_storage_free(cache->storage); + table_cache_free(&cache->innodb_trx); + table_cache_free(&cache->innodb_locks); + table_cache_free(&cache->innodb_lock_waits); + memset(cache, 0, sizeof *cache); +} + +/*******************************************************************//** Issue a shared/read lock on the tables cache. */ UNIV_INTERN void === modified file 'storage/xtradb/trx/trx0purge.c' --- a/storage/xtradb/trx/trx0purge.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/trx/trx0purge.c 2010-01-06 12:00:14 +0000 @@ -184,8 +184,9 @@ this query graph. @return own: the query graph */ static que_t* -trx_purge_graph_build(void) +trx_purge_graph_build( /*=======================*/ + trx_t* trx) { mem_heap_t* heap; que_fork_t* fork; @@ -194,7 +195,7 @@ trx_purge_graph_build(void) heap = mem_heap_create(512); fork = que_fork_create(NULL, NULL, QUE_FORK_PURGE, heap); - fork->trx = purge_sys->trx; + fork->trx = trx; thr = que_thr_create(fork, heap); @@ -243,10 +244,73 @@ trx_purge_sys_create(void) ut_a(trx_start_low(purge_sys->trx, ULINT_UNDEFINED)); - purge_sys->query = trx_purge_graph_build(); + purge_sys->query = trx_purge_graph_build(purge_sys->trx); purge_sys->view = read_view_oldest_copy_or_open_new(ut_dulint_zero, purge_sys->heap); + + purge_sys->n_worker = 0; + if (srv_use_purge_thread > 1) { + /* Use worker threads */ + ulint i; + + purge_sys->n_worker = srv_use_purge_thread - 1; + + purge_sys->sess_arr = mem_alloc(sizeof(sess_t*) * purge_sys->n_worker); + purge_sys->trx_arr = mem_alloc(sizeof(trx_t*) * purge_sys->n_worker); + purge_sys->query_arr = mem_alloc(sizeof(que_t*) * purge_sys->n_worker); + + purge_sys->worker_event = os_event_create(NULL); + os_event_reset(purge_sys->worker_event); + + for (i = 0; i < purge_sys->n_worker; i++) { + purge_sys->sess_arr[i] = sess_open(); + + purge_sys->trx_arr[i] = purge_sys->sess_arr[i]->trx; + purge_sys->trx_arr[i]->is_purge = 1; + ut_a(trx_start_low(purge_sys->trx_arr[i], ULINT_UNDEFINED)); + + purge_sys->query_arr[i] = trx_purge_graph_build(purge_sys->trx_arr[i]); + } + } +} + +/************************************************************************ +Frees the global purge system control structure. */ +UNIV_INTERN +void +trx_purge_sys_close(void) +/*======================*/ +{ + ut_ad(!mutex_own(&kernel_mutex)); + + que_graph_free(purge_sys->query); + + ut_a(purge_sys->sess->trx->is_purge); + purge_sys->sess->trx->conc_state = TRX_NOT_STARTED; + sess_close(purge_sys->sess); + purge_sys->sess = NULL; + + if (purge_sys->view != NULL) { + /* Because acquiring the kernel mutex is a pre-condition + of read_view_close(). We don't really need it here. */ + mutex_enter(&kernel_mutex); + + read_view_close(purge_sys->view); + purge_sys->view = NULL; + + mutex_exit(&kernel_mutex); + } + + trx_undo_arr_free(purge_sys->arr); + + rw_lock_free(&purge_sys->latch); + mutex_free(&purge_sys->mutex); + + mem_heap_free(purge_sys->heap); + mem_free(purge_sys); + + purge_sys = NULL; } /*================ UNDO LOG HISTORY LIST =============================*/ @@ -1110,7 +1174,7 @@ trx_purge(void) /* Handle at most 20 undo log pages in one purge batch */ - purge_sys->handle_limit = purge_sys->n_pages_handled + 20; + purge_sys->handle_limit = purge_sys->n_pages_handled + 20 * (srv_use_purge_thread + 1); old_pages_handled = purge_sys->n_pages_handled; @@ -1129,6 +1193,9 @@ trx_purge(void) mutex_exit(&kernel_mutex); + if (purge_sys->n_worker) + os_event_set(purge_sys->worker_event); + /* srv_que_task_enqueue(thr2); */ if (srv_print_thread_releases) { @@ -1138,6 +1205,9 @@ trx_purge(void) que_run_threads(thr); + if (purge_sys->n_worker) + os_event_reset(purge_sys->worker_event); + if (srv_print_thread_releases) { fprintf(stderr, @@ -1148,6 +1218,52 @@ trx_purge(void) return(purge_sys->n_pages_handled - old_pages_handled); } +/********************************************************************** +This function runs a purge worker batch */ +UNIV_INTERN +void +trx_purge_worker( +/*=============*/ + ulint worker_id) +{ + que_thr_t* thr; + + mutex_enter(&kernel_mutex); + + thr = que_fork_start_command(purge_sys->query_arr[worker_id]); + + ut_ad(thr); + + mutex_exit(&kernel_mutex); + + que_run_threads(thr); + + if (purge_sys->state == TRX_STOP_PURGE) { /* optimistic */ + os_event_reset(purge_sys->worker_event); + } +} + +/********************************************************************** +This function waits the event for worker batch */ +UNIV_INTERN +void +trx_purge_worker_wait(void) +/*=======================*/ +{ + os_event_wait(purge_sys->worker_event); +} + +/********************************************************************** +This function wakes the waiting worker batch */ +UNIV_INTERN +void +trx_purge_worker_wake(void) +/*=======================*/ +{ + if (purge_sys->n_worker) + os_event_set(purge_sys->worker_event); +} + /******************************************************************//** Prints information of the purge system to stderr. */ UNIV_INTERN === modified file 'storage/xtradb/trx/trx0rec.c' --- a/storage/xtradb/trx/trx0rec.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/trx/trx0rec.c 2010-01-06 12:00:14 +0000 @@ -1333,7 +1333,7 @@ trx_undo_get_undo_rec_low( ulint rseg_id; ulint page_no; ulint offset; - page_t* undo_page; + const page_t* undo_page; trx_rseg_t* rseg; ibool is_insert; mtr_t mtr; @@ -1572,7 +1572,7 @@ trx_undo_prev_version_build( /* We have to set the appropriate extern storage bits in the old version of the record: the extern bits in rec for those - fields that update does NOT update, as well as the the bits for + fields that update does NOT update, as well as the bits for those fields that update updates to become externally stored fields. Store the info: */ === modified file 'storage/xtradb/trx/trx0roll.c' --- a/storage/xtradb/trx/trx0roll.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/trx/trx0roll.c 2010-01-06 12:00:14 +0000 @@ -66,9 +66,9 @@ int trx_general_rollback_for_mysql( /*===========================*/ trx_t* trx, /*!< in: transaction handle */ - ibool partial,/*!< in: TRUE if partial rollback requested */ trx_savept_t* savept) /*!< in: pointer to savepoint undo number, if - partial rollback requested */ + partial rollback requested, or NULL for + complete rollback */ { mem_heap_t* heap; que_thr_t* thr; @@ -85,9 +85,8 @@ trx_general_rollback_for_mysql( roll_node = roll_node_create(heap); - roll_node->partial = partial; - - if (partial) { + if (savept) { + roll_node->partial = TRUE; roll_node->savept = *savept; } @@ -145,7 +144,7 @@ trx_rollback_for_mysql( the transaction object does not have an InnoDB session object, and we set a dummy session that we use for all MySQL transactions. */ - err = trx_general_rollback_for_mysql(trx, FALSE, NULL); + err = trx_general_rollback_for_mysql(trx, NULL); trx->op_info = ""; @@ -170,8 +169,7 @@ trx_rollback_last_sql_stat_for_mysql( trx->op_info = "rollback of SQL statement"; - err = trx_general_rollback_for_mysql(trx, TRUE, - &(trx->last_sql_stat_start)); + err = trx_general_rollback_for_mysql(trx, &trx->last_sql_stat_start); /* The following call should not be needed, but we play safe: */ trx_mark_sql_stat_end(trx); @@ -282,7 +280,7 @@ trx_rollback_to_savepoint_for_mysql( trx->op_info = "rollback to a savepoint"; - err = trx_general_rollback_for_mysql(trx, TRUE, &(savep->savept)); + err = trx_general_rollback_for_mysql(trx, &savep->savept); /* Store the current undo_no of the transaction so that we know where to roll back if we have to roll back the next SQL statement: */ @@ -534,28 +532,26 @@ trx_rollback_active( Rollback or clean up any incomplete transactions which were encountered in crash recovery. If the transaction already was committed, then we clean up a possible insert undo log. If the -transaction was not yet committed, then we roll it back. -Note: this is done in a background thread. -@return a dummy parameter */ +transaction was not yet committed, then we roll it back. */ UNIV_INTERN -os_thread_ret_t -trx_rollback_or_clean_all_recovered( -/*================================*/ - void* arg __attribute__((unused))) - /*!< in: a dummy parameter required by - os_thread_create */ +void +trx_rollback_or_clean_recovered( +/*============================*/ + ibool all) /*!< in: FALSE=roll back dictionary transactions; + TRUE=roll back all non-PREPARED transactions */ { trx_t* trx; mutex_enter(&kernel_mutex); - if (UT_LIST_GET_FIRST(trx_sys->trx_list)) { + if (!UT_LIST_GET_FIRST(trx_sys->trx_list)) { + goto leave_function; + } + if (all) { fprintf(stderr, "InnoDB: Starting in background the rollback" " of uncommitted transactions\n"); - } else { - goto leave_function; } mutex_exit(&kernel_mutex); @@ -584,18 +580,42 @@ loop: goto loop; case TRX_ACTIVE: - mutex_exit(&kernel_mutex); - trx_rollback_active(trx); - goto loop; + if (all || trx_get_dict_operation(trx) + != TRX_DICT_OP_NONE) { + mutex_exit(&kernel_mutex); + trx_rollback_active(trx); + goto loop; + } } } - ut_print_timestamp(stderr); - fprintf(stderr, - " InnoDB: Rollback of non-prepared transactions completed\n"); + if (all) { + ut_print_timestamp(stderr); + fprintf(stderr, + " InnoDB: Rollback of non-prepared" + " transactions completed\n"); + } leave_function: mutex_exit(&kernel_mutex); +} + +/*******************************************************************//** +Rollback or clean up any incomplete transactions which were +encountered in crash recovery. If the transaction already was +committed, then we clean up a possible insert undo log. If the +transaction was not yet committed, then we roll it back. +Note: this is done in a background thread. +@return a dummy parameter */ +UNIV_INTERN +os_thread_ret_t +trx_rollback_or_clean_all_recovered( +/*================================*/ + void* arg __attribute__((unused))) + /*!< in: a dummy parameter required by + os_thread_create */ +{ + trx_rollback_or_clean_recovered(TRUE); /* We count the number of threads in os_thread_exit(). A created thread should always use that to exit and not use return() to exit. */ === modified file 'storage/xtradb/trx/trx0rseg.c' --- a/storage/xtradb/trx/trx0rseg.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/trx/trx0rseg.c 2010-01-06 12:00:14 +0000 @@ -132,6 +132,49 @@ trx_rseg_header_create( } /***********************************************************************//** +Free's an instance of the rollback segment in memory. */ +UNIV_INTERN +void +trx_rseg_mem_free( +/*==============*/ + trx_rseg_t* rseg) /* in, own: instance to free */ +{ + trx_undo_t* undo; + + mutex_free(&rseg->mutex); + + /* There can't be any active transactions. */ + ut_a(UT_LIST_GET_LEN(rseg->update_undo_list) == 0); + ut_a(UT_LIST_GET_LEN(rseg->insert_undo_list) == 0); + + undo = UT_LIST_GET_FIRST(rseg->update_undo_cached); + + while (undo != NULL) { + trx_undo_t* prev_undo = undo; + + undo = UT_LIST_GET_NEXT(undo_list, undo); + UT_LIST_REMOVE(undo_list, rseg->update_undo_cached, prev_undo); + + trx_undo_mem_free(prev_undo); + } + + undo = UT_LIST_GET_FIRST(rseg->insert_undo_cached); + + while (undo != NULL) { + trx_undo_t* prev_undo = undo; + + undo = UT_LIST_GET_NEXT(undo_list, undo); + UT_LIST_REMOVE(undo_list, rseg->insert_undo_cached, prev_undo); + + trx_undo_mem_free(prev_undo); + } + + trx_sys_set_nth_rseg(trx_sys, rseg->id, NULL); + + mem_free(rseg); +} + +/*************************************************************************** Creates and initializes a rollback segment object. The values for the fields are read from the header. The object is inserted to the rseg list of the trx system object and a pointer is inserted in the rseg === modified file 'storage/xtradb/trx/trx0sys.c' --- a/storage/xtradb/trx/trx0sys.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/trx/trx0sys.c 2010-01-06 12:00:14 +0000 @@ -39,7 +39,9 @@ Created 3/26/1996 Heikki Tuuri #include "srv0srv.h" #include "trx0purge.h" #include "log0log.h" +#include "log0recv.h" #include "os0file.h" +#include "read0read.h" /** The file format tag structure with id and name. */ struct file_format_struct { @@ -552,6 +554,12 @@ trx_sys_doublewrite_init_or_restore_page zip_size ? zip_size : UNIV_PAGE_SIZE, read_buf, NULL); + if (srv_recovery_stats && recv_recovery_is_on()) { + mutex_enter(&(recv_sys->mutex)); + recv_sys->stats_doublewrite_check_pages++; + mutex_exit(&(recv_sys->mutex)); + } + /* Check if the page is corrupt */ if (UNIV_UNLIKELY @@ -599,6 +607,13 @@ trx_sys_doublewrite_init_or_restore_page zip_size, page_no, 0, zip_size ? zip_size : UNIV_PAGE_SIZE, page, NULL); + + if (srv_recovery_stats && recv_recovery_is_on()) { + mutex_enter(&(recv_sys->mutex)); + recv_sys->stats_doublewrite_overwrite_pages++; + mutex_exit(&(recv_sys->mutex)); + } + fprintf(stderr, "InnoDB: Recovered the page from" " the doublewrite buffer.\n"); @@ -1592,3 +1607,80 @@ trx_sys_file_format_id_to_name( } #endif /* !UNIV_HOTBACKUP */ + +/********************************************************************* +Shutdown/Close the transaction system. */ +UNIV_INTERN +void +trx_sys_close(void) +/*===============*/ +{ + trx_rseg_t* rseg; + read_view_t* view; + + ut_ad(trx_sys != NULL); + + /* Check that all read views are closed except read view owned + by a purge. */ + + if (UT_LIST_GET_LEN(trx_sys->view_list) > 1) { + fprintf(stderr, + "InnoDB: Error: all read views were not closed" + " before shutdown:\n" + "InnoDB: %lu read views open \n", + UT_LIST_GET_LEN(trx_sys->view_list) - 1); + } + + sess_close(trx_dummy_sess); + trx_dummy_sess = NULL; + + trx_purge_sys_close(); + + mutex_enter(&kernel_mutex); + + /* Free the double write data structures. */ + ut_a(trx_doublewrite != NULL); + ut_free(trx_doublewrite->write_buf_unaligned); + trx_doublewrite->write_buf_unaligned = NULL; + + mem_free(trx_doublewrite->buf_block_arr); + trx_doublewrite->buf_block_arr = NULL; + + mutex_free(&trx_doublewrite->mutex); + mem_free(trx_doublewrite); + trx_doublewrite = NULL; + + /* There can't be any active transactions. */ + rseg = UT_LIST_GET_FIRST(trx_sys->rseg_list); + + while (rseg != NULL) { + trx_rseg_t* prev_rseg = rseg; + + rseg = UT_LIST_GET_NEXT(rseg_list, prev_rseg); + UT_LIST_REMOVE(rseg_list, trx_sys->rseg_list, prev_rseg); + + trx_rseg_mem_free(prev_rseg); + } + + view = UT_LIST_GET_FIRST(trx_sys->view_list); + + while (view != NULL) { + read_view_t* prev_view = view; + + view = UT_LIST_GET_NEXT(view_list, prev_view); + + /* Views are allocated from the trx_sys->global_read_view_heap. + So, we simply remove the element here. */ + UT_LIST_REMOVE(view_list, trx_sys->view_list, prev_view); + } + + ut_a(UT_LIST_GET_LEN(trx_sys->trx_list) == 0); + ut_a(UT_LIST_GET_LEN(trx_sys->rseg_list) == 0); + ut_a(UT_LIST_GET_LEN(trx_sys->view_list) == 0); + ut_a(UT_LIST_GET_LEN(trx_sys->mysql_trx_list) == 0); + + mem_free(trx_sys); + + trx_sys = NULL; + mutex_exit(&kernel_mutex); +} === modified file 'storage/xtradb/trx/trx0trx.c' --- a/storage/xtradb/trx/trx0trx.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/trx/trx0trx.c 2010-01-06 12:00:14 +0000 @@ -178,6 +178,15 @@ trx_create( trx->global_read_view = NULL; trx->read_view = NULL; + trx->io_reads = 0; + trx->io_read = 0; + trx->io_reads_wait_timer = 0; + trx->lock_que_wait_timer = 0; + trx->innodb_que_wait_timer = 0; + trx->distinct_page_access = 0; + trx->distinct_page_access_hash = NULL; + trx->take_stats = FALSE; + /* Set X/Open XA transaction identification to NULL */ memset(&trx->xid, 0, sizeof(trx->xid)); trx->xid.formatID = -1; @@ -215,6 +224,11 @@ trx_allocate_for_mysql(void) trx->mysql_process_no = os_proc_get_number(); + if (innobase_get_slow_log() && trx->take_stats) { + trx->distinct_page_access_hash = mem_alloc(DPAH_SIZE); + memset(trx->distinct_page_access_hash, 0, DPAH_SIZE); + } + return(trx); } @@ -346,6 +360,12 @@ trx_free_for_mysql( /*===============*/ trx_t* trx) /*!< in, own: trx object */ { + if (trx->distinct_page_access_hash) + { + mem_free(trx->distinct_page_access_hash); + trx->distinct_page_access_hash= NULL; + } + mutex_enter(&kernel_mutex); UT_LIST_REMOVE(mysql_trx_list, trx_sys->mysql_trx_list, trx); @@ -367,6 +387,12 @@ trx_free_for_background( /*====================*/ trx_t* trx) /*!< in, own: trx object */ { + if (trx->distinct_page_access_hash) + { + mem_free(trx->distinct_page_access_hash); + trx->distinct_page_access_hash= NULL; + } + mutex_enter(&kernel_mutex); trx_free(trx); @@ -820,7 +846,7 @@ trx_commit_off_kernel( in exactly the same order as commit lsn's, if the transactions have different rollback segments. To get exactly the same order we should hold the kernel mutex up to this point, - adding to to the contention of the kernel mutex. However, if + adding to the contention of the kernel mutex. However, if a transaction T2 is able to see modifications made by a transaction T1, T2 will always get a bigger transaction number and a bigger commit lsn than T1. */ @@ -967,7 +993,7 @@ trx_commit_off_kernel( /****************************************************************//** Cleans up a transaction at database startup. The cleanup is needed if the transaction already got to the middle of a commit when the database -crashed, andf we cannot roll it back. */ +crashed, and we cannot roll it back. */ UNIV_INTERN void trx_cleanup_at_db_startup( @@ -1072,6 +1098,9 @@ trx_end_lock_wait( trx_t* trx) /*!< in: transaction */ { que_thr_t* thr; + ulint sec; + ulint ms; + ib_uint64_t now; ut_ad(mutex_own(&kernel_mutex)); ut_ad(trx->que_state == TRX_QUE_LOCK_WAIT); @@ -1086,6 +1115,11 @@ trx_end_lock_wait( thr = UT_LIST_GET_FIRST(trx->wait_thrs); } + if (innobase_get_slow_log() && trx->take_stats) { + ut_usectime(&sec, &ms); + now = (ib_uint64_t)sec * 1000000 + ms; + trx->lock_que_wait_timer += (ulint)(now - trx->lock_que_wait_ustarted); + } trx->que_state = TRX_QUE_RUNNING; } @@ -1099,6 +1133,9 @@ trx_lock_wait_to_suspended( trx_t* trx) /*!< in: transaction in the TRX_QUE_LOCK_WAIT state */ { que_thr_t* thr; + ulint sec; + ulint ms; + ib_uint64_t now; ut_ad(mutex_own(&kernel_mutex)); ut_ad(trx->que_state == TRX_QUE_LOCK_WAIT); @@ -1113,6 +1150,11 @@ trx_lock_wait_to_suspended( thr = UT_LIST_GET_FIRST(trx->wait_thrs); } + if (innobase_get_slow_log() && trx->take_stats) { + ut_usectime(&sec, &ms); + now = (ib_uint64_t)sec * 1000000 + ms; + trx->lock_que_wait_timer += (ulint)(now - trx->lock_que_wait_ustarted); + } trx->que_state = TRX_QUE_RUNNING; } === modified file 'storage/xtradb/trx/trx0undo.c' --- a/storage/xtradb/trx/trx0undo.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/trx/trx0undo.c 2010-01-06 12:00:14 +0000 @@ -1560,7 +1560,7 @@ trx_undo_mem_init_for_reuse( /********************************************************************//** Frees an undo log memory copy. */ -static +UNIV_INTERN void trx_undo_mem_free( /*==============*/ === modified file 'storage/xtradb/usr/usr0sess.c' --- a/storage/xtradb/usr/usr0sess.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/usr/usr0sess.c 2010-01-06 12:00:14 +0000 @@ -32,14 +32,6 @@ Created 6/25/1996 Heikki Tuuri #include "trx0trx.h" /*********************************************************************//** -Closes a session, freeing the memory occupied by it. */ -static -void -sess_close( -/*=======*/ - sess_t* sess); /*!< in, own: session object */ - -/*********************************************************************//** Opens a session. @return own: session object */ UNIV_INTERN @@ -64,35 +56,16 @@ sess_open(void) /*********************************************************************//** Closes a session, freeing the memory occupied by it. */ -static +UNIV_INTERN void sess_close( /*=======*/ sess_t* sess) /*!< in, own: session object */ { - ut_ad(mutex_own(&kernel_mutex)); - ut_ad(sess->trx == NULL); - - mem_free(sess); -} - -/*********************************************************************//** -Closes a session, freeing the memory occupied by it, if it is in a state -where it should be closed. -@return TRUE if closed */ -UNIV_INTERN -ibool -sess_try_close( -/*===========*/ - sess_t* sess) /*!< in, own: session object */ -{ - ut_ad(mutex_own(&kernel_mutex)); + ut_ad(!mutex_own(&kernel_mutex)); - if (UT_LIST_GET_LEN(sess->graphs) == 0) { - sess_close(sess); + ut_a(UT_LIST_GET_LEN(sess->graphs) == 0); - return(TRUE); - } - - return(FALSE); + trx_free_for_background(sess->trx); + mem_free(sess); } === modified file 'storage/xtradb/ut/ut0auxconf_atomic_pthread_t_solaris.c' --- a/storage/xtradb/ut/ut0auxconf_atomic_pthread_t_solaris.c 2009-09-23 00:06:02 +0000 +++ b/storage/xtradb/ut/ut0auxconf_atomic_pthread_t_solaris.c 2010-01-06 12:00:14 +0000 @@ -17,18 +17,38 @@ Place, Suite 330, Boston, MA 02111-1307 *****************************************************************************/ /***************************************************************************** -If this program compiles, then pthread_t objects can be used as arguments -to Solaris libc atomic functions. +If this program compiles and returns 0, then pthread_t objects can be used as +arguments to Solaris libc atomic functions. Created April 18, 2009 Vasil Dimov *****************************************************************************/ #include <pthread.h> +#include <string.h> int main(int argc, char** argv) { - pthread_t x = 0; + pthread_t x1; + pthread_t x2; + pthread_t x3; + + memset(&x1, 0x0, sizeof(x1)); + memset(&x2, 0x0, sizeof(x2)); + memset(&x3, 0x0, sizeof(x3)); + + if (sizeof(pthread_t) == 4) { + + atomic_cas_32(&x1, x2, x3); + + } else if (sizeof(pthread_t) == 8) { + + atomic_cas_64(&x1, x2, x3); + + } else { + + return(1); + } return(0); } === added file 'storage/xtradb/ut/ut0auxconf_have_gcc_atomics.c' --- a/storage/xtradb/ut/ut0auxconf_have_gcc_atomics.c 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/ut/ut0auxconf_have_gcc_atomics.c 2010-01-06 12:00:14 +0000 @@ -0,0 +1,61 @@ +/***************************************************************************** + +Copyright (c) 2009, Innobase Oy. All Rights Reserved. + +This program is free software; you can redistribute it and/or modify it under +the terms of the GNU General Public License as published by the Free Software +Foundation; version 2 of the License. + +This program is distributed in the hope that it will be useful, but WITHOUT +ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS +FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. + +You should have received a copy of the GNU General Public License along with +this program; if not, write to the Free Software Foundation, Inc., 59 Temple +Place, Suite 330, Boston, MA 02111-1307 USA + +*****************************************************************************/ + +/***************************************************************************** +If this program compiles and returns 0, then GCC atomic funcions are available. + +Created September 12, 2009 Vasil Dimov +*****************************************************************************/ + +int +main(int argc, char** argv) +{ + long x; + long y; + long res; + char c; + + x = 10; + y = 123; + res = __sync_bool_compare_and_swap(&x, x, y); + if (!res || x != y) { + return(1); + } + + x = 10; + y = 123; + res = __sync_bool_compare_and_swap(&x, x + 1, y); + if (res || x != 10) { + return(1); + } + + x = 10; + y = 123; + res = __sync_add_and_fetch(&x, y); + if (res != 123 + 10 || x != 123 + 10) { + return(1); + } + + c = 10; + res = __sync_lock_test_and_set(&c, 123); + if (res != 10 || c != 123) { + return(1); + } + + return(0); +} === modified file 'storage/xtradb/ut/ut0mem.c' --- a/storage/xtradb/ut/ut0mem.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/ut/ut0mem.c 2010-01-06 12:00:14 +0000 @@ -433,6 +433,8 @@ ut_free_all_mem(void) " total allocated memory is %lu\n", (ulong) ut_total_allocated_memory); } + + ut_mem_block_list_inited = FALSE; } #endif /* !UNIV_HOTBACKUP */ === modified file 'storage/xtradb/ut/ut0ut.c' --- a/storage/xtradb/ut/ut0ut.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/ut/ut0ut.c 2010-01-06 12:00:14 +0000 @@ -132,6 +132,7 @@ ut_time(void) return(time(NULL)); } +#ifndef UNIV_HOTBACKUP /**********************************************************//** Returns system time. Upon successful completion, the value 0 is returned; otherwise the @@ -200,6 +201,24 @@ ut_time_us( } /**********************************************************//** +Returns the number of milliseconds since some epoch. The +value may wrap around. It should only be used for heuristic +purposes. +@return ms since epoch */ +UNIV_INTERN +ulint +ut_time_ms(void) +/*============*/ +{ + struct timeval tv; + + ut_gettimeofday(&tv, NULL); + + return((ulint) tv.tv_sec * 1000 + tv.tv_usec / 1000); +} +#endif /* !UNIV_HOTBACKUP */ + +/**********************************************************//** Returns the difference of two times in seconds. @return time2 - time1 expressed in seconds */ UNIV_INTERN === removed directory 'storage/xtradb/win-plugin' === removed file 'storage/xtradb/win-plugin/README' --- a/storage/xtradb/win-plugin/README 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/win-plugin/README 1970-01-01 00:00:00 +0000 @@ -1,22 +0,0 @@ -This directory contains patches that need to be applied to the MySQL -source tree in order to build the dynamic plugin on Windows -- -HA_INNODB.DLL. Please note the followings when adding the patches: - -* The patch must be applied from the mysql top-level source directory. - patch -p0 < win-plugin.diff -* The patch filenames end in ".diff". -* All patches here are expected to apply cleanly to the latest MySQL 5.1 - tree when storage/innobase is replaced with this InnoDB branch. - -When applying the patch, the following files will be modified: - - * CMakeLists.txt - * sql/CMakeLists.txt - * win/configure.js - -Also, two new files will be added: - - * sql/mysqld.def - * sql/mysqld_x64.def - -You can get "patch" utility for Windows from http://unxutils.sourceforge.net/ === removed file 'storage/xtradb/win-plugin/win-plugin.diff' --- a/storage/xtradb/win-plugin/win-plugin.diff 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/win-plugin/win-plugin.diff 1970-01-01 00:00:00 +0000 @@ -1,279 +0,0 @@ -diff -Nur CMakeLists.txt.orig CMakeLists.txt ---- CMakeLists.txt.orig 2008-10-03 12:25:41 -05:00 -+++ CMakeLists.txt 2008-09-26 17:32:51 -05:00 -@@ -254,9 +254,9 @@ - IF(WITH_FEDERATED_STORAGE_ENGINE) - ADD_SUBDIRECTORY(storage/federated) - ENDIF(WITH_FEDERATED_STORAGE_ENGINE) --IF(WITH_INNOBASE_STORAGE_ENGINE) -+IF(WITH_INNOBASE_STORAGE_ENGINE OR INNODB_DYNAMIC_PLUGIN) - ADD_SUBDIRECTORY(storage/innobase) --ENDIF(WITH_INNOBASE_STORAGE_ENGINE) -+ENDIF(WITH_INNOBASE_STORAGE_ENGINE OR INNODB_DYNAMIC_PLUGIN) - ADD_SUBDIRECTORY(sql) - ADD_SUBDIRECTORY(server-tools/instance-manager) - ADD_SUBDIRECTORY(libmysql) - -diff -Nur sql/CMakeLists.txt.orig sql/CMakeLists.txt ---- sql/CMakeLists.txt.orig 2008-10-03 12:25:41 -05:00 -+++ sql/CMakeLists.txt 2008-09-24 03:58:19 -05:00 -@@ -98,6 +98,15 @@ - LINK_FLAGS "/PDB:${CMAKE_CFG_INTDIR}/mysqld${MYSQLD_EXE_SUFFIX}.pdb") - ENDIF(cmake_version EQUAL 20406) - -+# Checks for 64-bit version -+IF(CMAKE_SIZEOF_VOID_P MATCHES 8) -+SET_TARGET_PROPERTIES(mysqld PROPERTIES -+ LINK_FLAGS "/def:\"${PROJECT_SOURCE_DIR}/sql/mysqld_x64.def\"") -+ELSE(CMAKE_SIZEOF_VOID_P MATCHES 8) -+SET_TARGET_PROPERTIES(mysqld PROPERTIES -+ LINK_FLAGS "/def:\"${PROJECT_SOURCE_DIR}/sql/mysqld.def\"") -+ENDIF(CMAKE_SIZEOF_VOID_P MATCHES 8) -+ - IF(EMBED_MANIFESTS) - MYSQL_EMBED_MANIFEST("mysqld" "asInvoker") - ENDIF(EMBED_MANIFESTS) - -diff -Nur sql/mysqld.def.orig sql/mysqld.def ---- sql/mysqld.def.orig 1969-12-31 18:00:00 -06:00 -+++ sql/mysqld.def 2009-04-09 02:20:32 -05:00 -@@ -0,0 +1,111 @@ -+EXPORTS -+ ?use_hidden_primary_key@handler@@UAEXXZ -+ ?get_dynamic_partition_info@handler@@UAEXPAUPARTITION_INFO@@I@Z -+ ?read_first_row@handler@@UAEHPAEI@Z -+ ?read_range_next@handler@@UAEHXZ -+ ?read_range_first@handler@@UAEHPBUst_key_range@@0_N1@Z -+ ?read_multi_range_first@handler@@UAEHPAPAUst_key_multi_range@@PAU2@I_NPAUst_handler_buffer@@@Z -+ ?read_multi_range_next@handler@@UAEHPAPAUst_key_multi_range@@@Z -+ ?index_read_idx_map@handler@@UAEHPAEIPBEKW4ha_rkey_function@@@Z -+ ?print_error@handler@@UAEXHH@Z -+ ?clone@handler@@UAEPAV1@PAUst_mem_root@@@Z -+ ?get_auto_increment@handler@@UAEX_K00PA_K1@Z -+ ?index_next_same@handler@@UAEHPAEPBEI@Z -+ ?get_error_message@handler@@UAE_NHPAVString@@@Z -+ ?ha_thd@handler@@IBEPAVTHD@@XZ -+ ?update_auto_increment@handler@@QAEHXZ -+ ?ha_statistic_increment@handler@@IBEXPQsystem_status_var@@K@Z -+ ?trans_register_ha@@YAXPAVTHD@@_NPAUhandlerton@@@Z -+ ?cmp@Field_blob@@QAEHPBEI0I@Z -+ ?set_time@Field_timestamp@@QAEXXZ -+ ?sql_print_error@@YAXPBDZZ -+ ?sql_print_warning@@YAXPBDZZ -+ ?check_global_access@@YA_NPAVTHD@@K@Z -+ ?schema_table_store_record@@YA_NPAVTHD@@PAUst_table@@@Z -+ ?get_quote_char_for_identifier@@YAHPAVTHD@@PBDI@Z -+ ?copy@String@@QAE_NXZ -+ ?copy@String@@QAE_NABV1@@Z -+ ?copy@String@@QAE_NPBDIPAUcharset_info_st@@@Z -+ ?copy_and_convert@@YAIPADIPAUcharset_info_st@@PBDI1PAI@Z -+ ?filename_to_tablename@@YAIPBDPADI@Z -+ ?strconvert@@YAIPAUcharset_info_st@@PBD0PADIPAI@Z -+ ?calculate_key_len@@YAIPAUst_table@@IPBEK@Z -+ ?sql_alloc@@YAPAXI@Z -+ ?localtime_to_TIME@@YAXPAUst_mysql_time@@PAUtm@@@Z -+ ?push_warning@@YAPAVMYSQL_ERROR@@PAVTHD@@W4enum_warning_level@1@IPBD@Z -+ ?push_warning_printf@@YAXPAVTHD@@W4enum_warning_level@MYSQL_ERROR@@IPBDZZ -+ ?drop_table@handler@@EAEXPBD@Z -+ ?column_bitmaps_signal@handler@@UAEXXZ -+ ?delete_table@handler@@MAEHPBD@Z -+ ?rename_table@handler@@MAEHPBD0@Z -+ ?key_map_empty@@3V?$Bitmap@$0EA@@@B -+ ?THR_THD@@3PAVTHD@@A -+ ?end_of_list@@3Ulist_node@@A -+ ?mysql_tmpdir_list@@3Ust_my_tmpdir@@A -+ mysql_query_cache_invalidate4 -+ thd_query -+ thd_sql_command -+ thd_get_thread_id -+ thd_get_xid -+ thd_slave_thread -+ thd_non_transactional_update -+ thd_mark_transaction_to_rollback -+ thd_security_context -+ thd_charset -+ thd_test_options -+ thd_ha_data -+ thd_killed -+ thd_tx_isolation -+ thd_tablespace_op -+ thd_sql_command -+ thd_memdup -+ thd_make_lex_string -+ thd_in_lock_tables -+ thd_binlog_format -+ _my_hash_init -+ my_hash_free -+ my_tmpdir -+ check_if_legal_filename -+ my_filename -+ my_sync_dir_by_file -+ alloc_root -+ thr_lock_data_init -+ thr_lock_init -+ thr_lock_delete -+ my_multi_malloc -+ get_charset -+ unpack_filename -+ my_hash_insert -+ my_hash_search -+ my_hash_delete -+ mysql_bin_log_file_pos -+ mysql_bin_log_file_name -+ mysqld_embedded -+ my_thread_name -+ my_malloc -+ my_no_flags_free -+ _sanity -+ _mymalloc -+ _myfree -+ _my_strdup -+ _my_thread_var -+ my_error -+ pthread_cond_init -+ pthread_cond_signal -+ pthread_cond_wait -+ pthread_cond_destroy -+ localtime_r -+ my_strdup -+ deflate -+ deflateEnd -+ deflateReset -+ deflateInit2_ -+ inflateEnd -+ inflateInit_ -+ inflate -+ compressBound -+ inflateInit2_ -+ adler32 -+ longlong2str -+ strend -+ my_snprintf - -diff -Nur sql/mysqld_x64.def.orig sql/mysqld_x64.def ---- sql/mysqld_x64.def.orig 1969-12-31 18:00:00 -06:00 -+++ sql/mysqld_x64.def 2009-04-09 02:22:04 -05:00 -@@ -0,0 +1,111 @@ -+EXPORTS -+ ?use_hidden_primary_key@handler@@UEAAXXZ -+ ?get_dynamic_partition_info@handler@@UEAAXPEAUPARTITION_INFO@@I@Z -+ ?read_first_row@handler@@UEAAHPEAEI@Z -+ ?read_range_next@handler@@UEAAHXZ -+ ?read_range_first@handler@@UEAAHPEBUst_key_range@@0_N1@Z -+ ?read_multi_range_first@handler@@UEAAHPEAPEAUst_key_multi_range@@PEAU2@I_NPEAUst_handler_buffer@@@Z -+ ?read_multi_range_next@handler@@UEAAHPEAPEAUst_key_multi_range@@@Z -+ ?index_read_idx_map@handler@@UEAAHPEAEIPEBEKW4ha_rkey_function@@@Z -+ ?print_error@handler@@UEAAXHH@Z -+ ?clone@handler@@UEAAPEAV1@PEAUst_mem_root@@@Z -+ ?get_auto_increment@handler@@UEAAX_K00PEA_K1@Z -+ ?index_next_same@handler@@UEAAHPEAEPEBEI@Z -+ ?get_error_message@handler@@UEAA_NHPEAVString@@@Z -+ ?ha_thd@handler@@IEBAPEAVTHD@@XZ -+ ?update_auto_increment@handler@@QEAAHXZ -+ ?ha_statistic_increment@handler@@IEBAXPEQsystem_status_var@@K@Z -+ ?trans_register_ha@@YAXPEAVTHD@@_NPEAUhandlerton@@@Z -+ ?cmp@Field_blob@@QEAAHPEBEI0I@Z -+ ?set_time@Field_timestamp@@QEAAXXZ -+ ?sql_print_error@@YAXPEBDZZ -+ ?sql_print_warning@@YAXPEBDZZ -+ ?check_global_access@@YA_NPEAVTHD@@K@Z -+ ?schema_table_store_record@@YA_NPEAVTHD@@PEAUst_table@@@Z -+ ?get_quote_char_for_identifier@@YAHPEAVTHD@@PEBDI@Z -+ ?copy@String@@QEAA_NXZ -+ ?copy@String@@QEAA_NAEBV1@@Z -+ ?copy@String@@QEAA_NPEBDIPEAUcharset_info_st@@@Z -+ ?copy_and_convert@@YAIPEADIPEAUcharset_info_st@@PEBDI1PEAI@Z -+ ?filename_to_tablename@@YAIPEBDPEADI@Z -+ ?strconvert@@YAIPEAUcharset_info_st@@PEBD0PEADIPEAI@Z -+ ?calculate_key_len@@YAIPEAUst_table@@IPEBEK@Z -+ ?sql_alloc@@YAPEAX_K@Z -+ ?localtime_to_TIME@@YAXPEAUst_mysql_time@@PEAUtm@@@Z -+ ?push_warning@@YAPEAVMYSQL_ERROR@@PEAVTHD@@W4enum_warning_level@1@IPEBD@Z -+ ?push_warning_printf@@YAXPEAVTHD@@W4enum_warning_level@MYSQL_ERROR@@IPEBDZZ -+ ?drop_table@handler@@EEAAXPEBD@Z -+ ?column_bitmaps_signal@handler@@UEAAXXZ -+ ?delete_table@handler@@MEAAHPEBD@Z -+ ?rename_table@handler@@MEAAHPEBD0@Z -+ ?key_map_empty@@3V?$Bitmap@$0EA@@@B -+ ?THR_THD@@3PEAVTHD@@EA -+ ?end_of_list@@3Ulist_node@@A -+ ?mysql_tmpdir_list@@3Ust_my_tmpdir@@A -+ mysql_query_cache_invalidate4 -+ thd_query -+ thd_sql_command -+ thd_get_thread_id -+ thd_get_xid -+ thd_slave_thread -+ thd_non_transactional_update -+ thd_mark_transaction_to_rollback -+ thd_security_context -+ thd_charset -+ thd_test_options -+ thd_ha_data -+ thd_killed -+ thd_tx_isolation -+ thd_tablespace_op -+ thd_sql_command -+ thd_memdup -+ thd_make_lex_string -+ thd_in_lock_tables -+ thd_binlog_format -+ _my_hash_init -+ my_hash_free -+ my_tmpdir -+ check_if_legal_filename -+ my_filename -+ my_sync_dir_by_file -+ alloc_root -+ thr_lock_data_init -+ thr_lock_init -+ thr_lock_delete -+ my_multi_malloc -+ get_charset -+ unpack_filename -+ my_hash_insert -+ my_hash_search -+ my_hash_delete -+ mysql_bin_log_file_pos -+ mysql_bin_log_file_name -+ mysqld_embedded -+ my_thread_name -+ my_malloc -+ my_no_flags_free -+ _sanity -+ _mymalloc -+ _myfree -+ _my_strdup -+ _my_thread_var -+ my_error -+ pthread_cond_init -+ pthread_cond_signal -+ pthread_cond_wait -+ pthread_cond_destroy -+ localtime_r -+ my_strdup -+ deflate -+ deflateEnd -+ deflateReset -+ deflateInit2_ -+ inflateEnd -+ inflateInit_ -+ inflate -+ compressBound -+ inflateInit2_ -+ adler32 -+ longlong2str -+ strend -+ my_snprintf - -diff -Nur win/configure.js.orig win/configure.js ---- win/configure.js.orig 2008-09-26 21:18:37 -05:00 -+++ win/configure.js 2008-10-01 11:21:27 -05:00 -@@ -50,6 +50,7 @@ - case "EMBED_MANIFESTS": - case "EXTRA_DEBUG": - case "WITH_EMBEDDED_SERVER": -+ case "INNODB_DYNAMIC_PLUGIN": - configfile.WriteLine("SET (" + args.Item(i) + " TRUE)"); - break; - case "MYSQL_SERVER_SUFFIX": === modified file 'support-files/compiler_warnings.supp' --- a/support-files/compiler_warnings.supp 2010-01-06 21:27:53 +0000 +++ b/support-files/compiler_warnings.supp 2010-01-28 14:49:14 +0000 @@ -4,6 +4,7 @@ # integer.cpp: .*control reaches end of non-void function.*: 1288-1427 +integer.cpp: .*no return statement in function returning non-void.*: 1288-1427 DictTabInfo.cpp : .*invalid access to non-static.* DictTabInfo.cpp : .*macro was used incorrectly.* DbdihMain.cpp : .*unused variable.* : 6666-7013 @@ -27,7 +28,17 @@ _flex_tmp.c: .*not enough actual paramet pars0lex.l: .*conversion from 'ulint' to 'int', possible loss of data.* btr/btr0cur\.c: .*value computed is not used.*: 3175-3375 include/buf0buf\.ic: unused parameter ‘mtr’ +fil/fil0fil\.c: pointer targets in passing argument.*differ in signedness fil/fil0fil\.c: comparison between signed and unsigned : 3100-3199 +fil/fil0fil\.c: unused parameter +log/log0recv\.c: unused variable +os/os0file\.c: unused parameter +os/os0file\.c: pointer targets in assignment differ in signedness +handler/i_s\.cc: unused variable +sync/sync0rw\.c: unused parameter +sync/sync0sync\.c: unused parameter +sync/sync0sync\.c: unused variable +ut/ut0ut\.c: ignoring return value of # # bdb is not critical to keep up to date @@ -50,6 +61,11 @@ db_vrfy.c : .*comparison is always false .*/extra/libevent/select\.c : .*comparison between signed and unsigned.* : 270-280 # +# Ignore warnings from system libraries +# +/usr/share/aclocal/audiofile.m4 : .* + +# # Ignore all conversion warnings on windows 64 # (Is safe as we are not yet supporting strings >= 2G) # @@ -87,10 +103,15 @@ storage/maria/ma_pagecache.c: .*'info_ch # I think these are due to mix of C and C++. # storage/pbxt/ : typedef.*was ignored in this declaration +ha_pbxt\.cc : variable.*might be clobbered by.*longjmp # # Yassl include/runtime.hpp: .*pure_error.* +.*/extra/yassl/taocrypt/.*: comparison with string literal +.*/extra/yassl/taocrypt/src/blowfish\.cpp: array subscript is above array bounds +.*/extra/yassl/taocrypt/src/file\.cpp: ignoring return value +.*/extra/yassl/taocrypt/src/integer\.cpp: control reaches end of non-void function # # Groff warnings on OpenSUSE. @@ -102,6 +123,8 @@ include/runtime.hpp: .*pure_error.* # listener.cc : .*conversion from 'SOCKET' to 'int'.* net_serv.cc : .*conversion from 'SOCKET' to 'int'.* +set_var.cc: right-hand operand of comma has no effect : 1000-1400 + # allow a little moving space for the warning below mi_packrec.c : .*result of 32-bit shift implicitly converted to 64 bits.* : 560-600 === modified file 'tests/mysql_client_test.c' --- a/tests/mysql_client_test.c 2009-12-03 11:19:05 +0000 +++ b/tests/mysql_client_test.c 2010-01-11 13:15:28 +0000 @@ -2465,6 +2465,34 @@ static uint query_cache_hits(MYSQL *conn /* + Check that query cache is available in server. +*/ +static my_bool is_query_cache_available() +{ + int rc; + MYSQL_RES *result; + MYSQL_ROW row; + int res= -1; + + rc= mysql_query(mysql, "SHOW VARIABLES LIKE 'have_query_cache'"); + myquery(rc); + + result= mysql_store_result(mysql); + DIE_UNLESS(result); + + row= mysql_fetch_row(result); + DIE_UNLESS(row != NULL); + if (strcmp(row[1], "YES") == 0) + res= 1; + else if (strcmp(row[1], "NO") == 0) + res= 0; + mysql_free_result(result); + + DIE_UNLESS(res == 0 || res == 1); + return res; +} + +/* Test that prepared statements make use of the query cache just as normal statements (BUG#735). */ @@ -2508,6 +2536,12 @@ static void test_ps_query_cache() myheader("test_ps_query_cache"); + if (! is_query_cache_available()) + { + fprintf(stdout, "Skipping test_ps_query_cache: Query cache not available.\n"); + return; + } + rc= mysql_query(mysql, "SET SQL_MODE=''"); myquery(rc); @@ -17863,8 +17897,6 @@ static void test_bug43560(void) Bug#36326: nested transaction and select */ -#ifdef HAVE_QUERY_CACHE - static void test_bug36326() { int rc; @@ -17872,6 +17904,12 @@ static void test_bug36326() DBUG_ENTER("test_bug36326"); myheader("test_bug36326"); + if (! is_query_cache_available()) + { + fprintf(stdout, "Skipping test_bug36326: Query cache not available.\n"); + DBUG_VOID_RETURN; + } + rc= mysql_autocommit(mysql, TRUE); myquery(rc); rc= mysql_query(mysql, "DROP TABLE IF EXISTS t1"); @@ -17911,8 +17949,6 @@ static void test_bug36326() DBUG_VOID_RETURN; } -#endif - /** Bug#41078: With CURSOR_TYPE_READ_ONLY mysql_stmt_fetch() returns short string value. @@ -18373,9 +18409,7 @@ static struct my_tests_st my_tests[]= { { "test_bug38486", test_bug38486 }, { "test_bug40365", test_bug40365 }, { "test_bug43560", test_bug43560 }, -#ifdef HAVE_QUERY_CACHE { "test_bug36326", test_bug36326 }, -#endif { "test_bug41078", test_bug41078 }, { "test_bug44495", test_bug44495 }, { 0, 0 } === modified file 'unittest/mysys/waiting_threads-t.c' --- a/unittest/mysys/waiting_threads-t.c 2009-09-15 10:46:35 +0000 +++ b/unittest/mysys/waiting_threads-t.c 2010-01-28 14:49:14 +0000 @@ -258,7 +258,7 @@ void do_tests() #define test_kill_strategy(X) \ diag("kill strategy: " #X); \ DBUG_EXECUTE("reset_file", \ - { rewind(DBUG_FILE); ftruncate(fileno(DBUG_FILE), 0); }); \ + { rewind(DBUG_FILE); (void) ftruncate(fileno(DBUG_FILE), 0); }); \ DBUG_PRINT("info", ("kill strategy: " #X)); \ kill_strategy=X; \ do_one_test(); === modified file 'vio/vio.c' --- a/vio/vio.c 2009-11-02 22:19:58 +0000 +++ b/vio/vio.c 2009-11-20 12:09:50 +0000 @@ -62,10 +62,8 @@ static void vio_init(Vio* vio, enum enum vio->timeout=vio_win32_timeout; /* Set default timeout */ - vio->read_timeout_millis = INFINITE; - vio->write_timeout_millis = INFINITE; - - memset(&(vio->pipe_overlapped), 0, sizeof(OVERLAPPED)); + vio->read_timeout_ms= INFINITE; + vio->write_timeout_ms= INFINITE; vio->pipe_overlapped.hEvent= CreateEvent(NULL, TRUE, FALSE, NULL); DBUG_VOID_RETURN; } @@ -90,8 +88,8 @@ static void vio_init(Vio* vio, enum enum /* Currently, shared memory is on Windows only, hence the below is ok*/ vio->timeout= vio_win32_timeout; /* Set default timeout */ - vio->read_timeout_millis= INFINITE; - vio->write_timeout_millis= INFINITE; + vio->read_timeout_ms= INFINITE; + vio->write_timeout_ms= INFINITE; DBUG_VOID_RETURN; } #endif @@ -115,22 +113,20 @@ static void vio_init(Vio* vio, enum enum DBUG_VOID_RETURN; } #endif /* HAVE_OPENSSL */ - { - vio->viodelete =vio_delete; - vio->vioerrno =vio_errno; - vio->read= (flags & VIO_BUFFERED_READ) ? vio_read_buff : vio_read; - vio->write =vio_write; - vio->fastsend =vio_fastsend; - vio->viokeepalive =vio_keepalive; - vio->should_retry =vio_should_retry; - vio->was_interrupted=vio_was_interrupted; - vio->vioclose =vio_close; - vio->peer_addr =vio_peer_addr; - vio->in_addr =vio_in_addr; - vio->vioblocking =vio_blocking; - vio->is_blocking =vio_is_blocking; - vio->timeout =vio_timeout; - } + vio->viodelete =vio_delete; + vio->vioerrno =vio_errno; + vio->read= (flags & VIO_BUFFERED_READ) ? vio_read_buff : vio_read; + vio->write =vio_write; + vio->fastsend =vio_fastsend; + vio->viokeepalive =vio_keepalive; + vio->should_retry =vio_should_retry; + vio->was_interrupted=vio_was_interrupted; + vio->vioclose =vio_close; + vio->peer_addr =vio_peer_addr; + vio->in_addr =vio_in_addr; + vio->vioblocking =vio_blocking; + vio->is_blocking =vio_is_blocking; + vio->timeout =vio_timeout; DBUG_VOID_RETURN; } === modified file 'vio/viosocket.c' --- a/vio/viosocket.c 2009-12-03 11:19:05 +0000 +++ b/vio/viosocket.c 2010-01-15 15:27:55 +0000 @@ -428,14 +428,14 @@ void vio_timeout(Vio *vio, uint which, u /* Finish pending IO on pipe. Honor wait timeout */ -static int pipe_complete_io(Vio* vio, char* buf, size_t size, DWORD timeout_millis) +static size_t pipe_complete_io(Vio* vio, char* buf, size_t size, DWORD timeout_ms) { DWORD length; DWORD ret; DBUG_ENTER("pipe_complete_io"); - ret= WaitForSingleObject(vio->pipe_overlapped.hEvent, timeout_millis); + ret= WaitForSingleObject(vio->pipe_overlapped.hEvent, timeout_ms); /* WaitForSingleObjects will normally return WAIT_OBJECT_O (success, IO completed) or WAIT_TIMEOUT. @@ -444,14 +444,14 @@ static int pipe_complete_io(Vio* vio, ch { CancelIo(vio->hPipe); DBUG_PRINT("error",("WaitForSingleObject() returned %d", ret)); - DBUG_RETURN(-1); + DBUG_RETURN((size_t)-1); } if (!GetOverlappedResult(vio->hPipe,&(vio->pipe_overlapped),&length, FALSE)) { DBUG_PRINT("error",("GetOverlappedResult() returned last error %d", GetLastError())); - DBUG_RETURN(-1); + DBUG_RETURN((size_t)-1); } DBUG_RETURN(length); @@ -461,49 +461,58 @@ static int pipe_complete_io(Vio* vio, ch size_t vio_read_pipe(Vio * vio, uchar *buf, size_t size) { DWORD bytes_read; + size_t retval; DBUG_ENTER("vio_read_pipe"); DBUG_PRINT("enter", ("sd: %d buf: 0x%lx size: %u", vio->sd, (long) buf, (uint) size)); - if (!ReadFile(vio->hPipe, buf, (DWORD)size, &bytes_read, + if (ReadFile(vio->hPipe, buf, (DWORD)size, &bytes_read, &(vio->pipe_overlapped))) { + retval= bytes_read; + } + else + { if (GetLastError() != ERROR_IO_PENDING) { DBUG_PRINT("error",("ReadFile() returned last error %d", GetLastError())); DBUG_RETURN((size_t)-1); } - bytes_read= pipe_complete_io(vio, buf, size,vio->read_timeout_millis); + retval= pipe_complete_io(vio, buf, size,vio->read_timeout_ms); } - DBUG_PRINT("exit", ("%d", bytes_read)); - DBUG_RETURN(bytes_read); + DBUG_PRINT("exit", ("%lld", (longlong)retval)); + DBUG_RETURN(retval); } size_t vio_write_pipe(Vio * vio, const uchar* buf, size_t size) { DWORD bytes_written; + size_t retval; DBUG_ENTER("vio_write_pipe"); DBUG_PRINT("enter", ("sd: %d buf: 0x%lx size: %u", vio->sd, (long) buf, (uint) size)); - if (!WriteFile(vio->hPipe, buf, (DWORD)size, &bytes_written, + if (WriteFile(vio->hPipe, buf, (DWORD)size, &bytes_written, &(vio->pipe_overlapped))) { + retval= bytes_written; + } + else + { if (GetLastError() != ERROR_IO_PENDING) { DBUG_PRINT("vio_error",("WriteFile() returned last error %d", GetLastError())); DBUG_RETURN((size_t)-1); } - bytes_written = pipe_complete_io(vio, (char *)buf, size, - vio->write_timeout_millis); + retval= pipe_complete_io(vio, (char *)buf, size, vio->write_timeout_ms); } - DBUG_PRINT("exit", ("%d", bytes_written)); - DBUG_RETURN(bytes_written); + DBUG_PRINT("exit", ("%lld", (longlong)retval)); + DBUG_RETURN(retval); } @@ -528,21 +537,21 @@ int vio_close_pipe(Vio * vio) void vio_win32_timeout(Vio *vio, uint which , uint timeout_sec) { - DWORD timeout_millis; + DWORD timeout_ms; /* Windows is measuring timeouts in milliseconds. Check for possible int overflow. */ if (timeout_sec > UINT_MAX/1000) - timeout_millis= INFINITE; + timeout_ms= INFINITE; else - timeout_millis= timeout_sec * 1000; + timeout_ms= timeout_sec * 1000; /* which == 1 means "write", which == 0 means "read".*/ if(which) - vio->write_timeout_millis= timeout_millis; + vio->write_timeout_ms= timeout_ms; else - vio->read_timeout_millis= timeout_millis; + vio->read_timeout_ms= timeout_ms; } @@ -577,7 +586,7 @@ size_t vio_read_shared_memory(Vio * vio, WAIT_ABANDONED_0 and WAIT_TIMEOUT - fail. We can't read anything */ if (WaitForMultipleObjects(array_elements(events), events, FALSE, - vio->read_timeout_millis) != WAIT_OBJECT_0) + vio->read_timeout_ms) != WAIT_OBJECT_0) { DBUG_RETURN(-1); }; @@ -634,7 +643,7 @@ size_t vio_write_shared_memory(Vio * vio while (remain != 0) { if (WaitForMultipleObjects(array_elements(events), events, FALSE, - vio->write_timeout_millis) != WAIT_OBJECT_0) + vio->write_timeout_ms) != WAIT_OBJECT_0) { DBUG_RETURN((size_t) -1); } === modified file 'vio/viossl.c' --- a/vio/viossl.c 2008-02-07 08:55:49 +0000 +++ b/vio/viossl.c 2010-01-29 10:42:31 +0000 @@ -74,8 +74,11 @@ report_errors(SSL* ssl) } if (ssl) - DBUG_PRINT("error", ("error: %s", - ERR_error_string(SSL_get_error(ssl, l), buf))); + { + int error= SSL_get_error(ssl, l); + DBUG_PRINT("error", ("error: %s (%d)", + ERR_error_string(error, buf), error)); + } DBUG_PRINT("info", ("socket_errno: %d", socket_errno)); DBUG_VOID_RETURN; @@ -174,16 +177,17 @@ void vio_ssl_delete(Vio *vio) static int ssl_do(struct st_VioSSLFd *ptr, Vio *vio, long timeout, - int (*connect_accept_func)(SSL*)) + int (*connect_accept_func)(SSL*), char *error_string) { SSL *ssl; my_bool unused; my_bool was_blocking; - DBUG_ENTER("ssl_do"); DBUG_PRINT("enter", ("ptr: 0x%lx, sd: %d ctx: 0x%lx", (long) ptr, vio->sd, (long) ptr->ssl_context)); + error_string[0]= 0; + /* Set socket to blocking if not already set */ vio_blocking(vio, 1, &was_blocking); @@ -191,6 +195,7 @@ static int ssl_do(struct st_VioSSLFd *pt { DBUG_PRINT("error", ("SSL_new failure")); report_errors(ssl); + strmov(error_string, "SSL_new failed"); vio_blocking(vio, was_blocking, &unused); DBUG_RETURN(1); } @@ -203,6 +208,7 @@ static int ssl_do(struct st_VioSSLFd *pt { DBUG_PRINT("error", ("SSL_connect/accept failure")); report_errors(ssl); + ERR_error_string(SSL_get_error(ssl, 0), error_string); SSL_free(ssl); vio_blocking(vio, was_blocking, &unused); DBUG_RETURN(1); @@ -250,17 +256,19 @@ static int ssl_do(struct st_VioSSLFd *pt } -int sslaccept(struct st_VioSSLFd *ptr, Vio *vio, long timeout) +int sslaccept(struct st_VioSSLFd *ptr, Vio *vio, long timeout, + char *error_string) { DBUG_ENTER("sslaccept"); - DBUG_RETURN(ssl_do(ptr, vio, timeout, SSL_accept)); + DBUG_RETURN(ssl_do(ptr, vio, timeout, SSL_accept, error_string)); } -int sslconnect(struct st_VioSSLFd *ptr, Vio *vio, long timeout) +int sslconnect(struct st_VioSSLFd *ptr, Vio *vio, long timeout, + char *error_string) { DBUG_ENTER("sslconnect"); - DBUG_RETURN(ssl_do(ptr, vio, timeout, SSL_connect)); + DBUG_RETURN(ssl_do(ptr, vio, timeout, SSL_connect, error_string)); }

1 0

[Maria-developers] [Branch ~maria-captains/maria/5.1] Rev 2810: hide nm warnings in configure
by noreply＠launchpad.net 30 Jan '10

30 Jan '10

------------------------------------------------------------ revno: 2810 committer: Sergei Golubchik <sergii(a)pisem.net> branch nick: maria-5.1 timestamp: Fri 2010-01-29 21:37:22 +0100 message: hide nm warnings in configure modified: configure.in -- lp:maria https://code.launchpad.net/~maria-captains/maria/5.1 Your team Maria developers is subscribed to branch lp:maria. To unsubscribe from this branch go to https://code.launchpad.net/~maria-captains/maria/5.1/+edit-subscription.

1 0

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (monty:2808) Bug#31173
by Michael Widenius 29 Jan '10

29 Jan '10

#At lp:maria based on revid:monty@askmonty.org-20100129105202-5pyhpgwqcvxawxhh 2808 Michael Widenius 2010-01-29 Patch set contributed by Alex Budovski (MCA) Fix for Bug#31173: mysqlslap.exe crashes if called without any parameters modified: .bzrignore client/mysqlslap.c mysys/my_thr_init.c sql-common/client.c sql/CMakeLists.txt sql/mysqld.cc sql/sql_profile.cc sql/sql_profile.h sql/udf_example.def storage/maria/ma_close.c per-file messages: .bzrignore Fixed .bzrignore rules. Many were simply not ignoring what they were meant to. client/mysqlslap.c Fixed bug for Bug#31173: mysqlslap.exe crashes if called without any parameters The original patch could cause memory leaks and odd problems depending on how connection was made. This code ensures that all mysql_options() are set for each mysql_real_connect(). (This patch by Monty) mysys/my_thr_init.c Fixed multiply-initialized critical section on Windows, due to code incorrectly checking the wrong field in an attempt to prevent multiple-initialization. sql-common/client.c Don't use shared memory if it's not set (for example after failed mysql_real_connect). Ensure that mysql_close() resets all resources so that it's safe to call it twice. (Patch by monty, related to Bug#31173: mysqlslap.exe crashes if called without any parameters) sql/CMakeLists.txt Added page fault counters for SHOW PROFILE on Windows. sql/mysqld.cc Fixed attempt to set a NULL event. The code now only sets the event if appropriate (i.e. shared memory is being used) sql/sql_profile.cc Added page fault counters for SHOW PROFILE on Windows. sql/sql_profile.h Added page fault counters for SHOW PROFILE on Windows. sql/udf_example.def Some cleanup functions were not exported from udf_example.dll, causing them to never be executed, and as a result multiple-initialization of kernel objects occurred and resources were not being freed correctly. storage/maria/ma_close.c Condition variable share->key_del_cond was never being destroyed, while its containing heap block was being freed in maria_close(), leaking kernel resources. === modified file '.bzrignore' --- a/.bzrignore 2009-12-22 13:50:20 +0000 +++ b/.bzrignore 2010-01-29 18:42:22 +0000 @@ -40,15 +40,15 @@ *.dsp *.Po *.Plo -*/*.dir/* +*.dir/ */*_pure_*warnings */.deps */.libs/* */.pure -*/debug/* -*/minsizerel/* -*/release/* -*/relwithdebinfo/* +debug/ +MinSizeRel/ +Release/ +RelWithDebInfo/ *~ .*.swp ./CMakeCache.txt @@ -96,7 +96,7 @@ BitKeeper/tmp/gone BitKeeper/tmp BitKeeper/log BitKeeper/etc/SCCS -CMakeFiles/* +CMakeFiles/ COPYING COPYING.LIB Docs/#manual.texi# === modified file 'client/mysqlslap.c' --- a/client/mysqlslap.c 2009-12-03 11:34:11 +0000 +++ b/client/mysqlslap.c 2010-01-29 18:42:22 +0000 @@ -292,6 +292,25 @@ static int gettimeofday(struct timeval * } #endif +void set_mysql_connect_options(MYSQL *mysql) +{ + if (opt_compress) + mysql_options(mysql,MYSQL_OPT_COMPRESS,NullS); +#ifdef HAVE_OPENSSL + if (opt_use_ssl) + mysql_ssl_set(mysql, opt_ssl_key, opt_ssl_cert, opt_ssl_ca, + opt_ssl_capath, opt_ssl_cipher); +#endif + if (opt_protocol) + mysql_options(mysql,MYSQL_OPT_PROTOCOL,(char*)&opt_protocol); +#ifdef HAVE_SMEM + if (shared_memory_base_name) + mysql_options(mysql,MYSQL_SHARED_MEMORY_BASE_NAME,shared_memory_base_name); +#endif + mysql_options(mysql, MYSQL_SET_CHARSET_NAME, default_charset); +} + + int main(int argc, char **argv) { MYSQL mysql; @@ -323,20 +342,7 @@ int main(int argc, char **argv) exit(1); } mysql_init(&mysql); - if (opt_compress) - mysql_options(&mysql,MYSQL_OPT_COMPRESS,NullS); -#ifdef HAVE_OPENSSL - if (opt_use_ssl) - mysql_ssl_set(&mysql, opt_ssl_key, opt_ssl_cert, opt_ssl_ca, - opt_ssl_capath, opt_ssl_cipher); -#endif - if (opt_protocol) - mysql_options(&mysql,MYSQL_OPT_PROTOCOL,(char*)&opt_protocol); -#ifdef HAVE_SMEM - if (shared_memory_base_name) - mysql_options(&mysql,MYSQL_SHARED_MEMORY_BASE_NAME,shared_memory_base_name); -#endif - mysql_options(&mysql, MYSQL_SET_CHARSET_NAME, default_charset); + set_mysql_connect_options(&mysql); if (!opt_only_print) { @@ -1815,6 +1821,7 @@ pthread_handler_t run_task(void *p) my_progname, mysql_error(mysql)); exit(0); } + set_mysql_connect_options(mysql); if (mysql_thread_init()) { @@ -1855,7 +1862,6 @@ limit_not_met: my_progname, mysql_error(mysql)); exit(0); } - if (slap_connect(mysql)) goto end; } @@ -2223,6 +2229,7 @@ slap_connect(MYSQL *mysql) int x, connect_error= 1; for (x= 0; x < 10; x++) { + set_mysql_connect_options(mysql); if (mysql_real_connect(mysql, host, user, opt_password, create_schema_string, opt_mysql_port, === modified file 'mysys/my_thr_init.c' --- a/mysys/my_thr_init.c 2009-12-03 11:19:05 +0000 +++ b/mysys/my_thr_init.c 2010-01-29 18:42:22 +0000 @@ -317,7 +317,7 @@ my_bool my_thread_init(void) /* Skip initialization if the thread specific variable is already initialized */ - if (THR_KEY_mysys.id) + if (THR_KEY_mysys.init) goto end; tmp= &THR_KEY_mysys; #endif === modified file 'sql-common/client.c' --- a/sql-common/client.c 2010-01-29 10:42:31 +0000 +++ b/sql-common/client.c 2010-01-29 18:42:22 +0000 @@ -1940,7 +1940,8 @@ CLI_MYSQL_REAL_CONNECT(MYSQL *mysql,cons #if defined(HAVE_SMEM) if ((!mysql->options.protocol || mysql->options.protocol == MYSQL_PROTOCOL_MEMORY) && - (!host || !strcmp(host,LOCAL_HOST))) + (!host || !strcmp(host,LOCAL_HOST)) && + mysql->options.shared_memory_base_name) { if ((create_shared_memory(mysql,net, mysql->options.connect_timeout)) == INVALID_HANDLE_VALUE) @@ -1949,7 +1950,7 @@ CLI_MYSQL_REAL_CONNECT(MYSQL *mysql,cons ("host: '%s' socket: '%s' shared memory: %s have_tcpip: %d", host ? host : "<null>", unix_socket ? unix_socket : "<null>", - (int) mysql->options.shared_memory_base_name, + mysql->options.shared_memory_base_name, (int) have_tcpip)); if (mysql->options.protocol == MYSQL_PROTOCOL_MEMORY) goto error; @@ -2752,6 +2753,13 @@ void mysql_detach_stmt_list(LIST **stmt_ } +/* + Close a MySQL connection and free all resources attached to it. + + This function is coded in such that it can be called multiple times + (As some clients call this after mysql_real_connect() fails) +*/ + void STDCALL mysql_close(MYSQL *mysql) { DBUG_ENTER("mysql_close"); @@ -2785,10 +2793,16 @@ void STDCALL mysql_close(MYSQL *mysql) } #endif if (mysql != mysql->master) + { mysql_close(mysql->master); + mysql->master= 0; + } #ifndef MYSQL_SERVER if (mysql->thd) + { (*mysql->methods->free_embedded_thd)(mysql); + mysql->thd= 0; + } #endif if (mysql->free_me) my_free((uchar*) mysql,MYF(0)); === modified file 'sql/CMakeLists.txt' --- a/sql/CMakeLists.txt 2009-12-03 11:19:05 +0000 +++ b/sql/CMakeLists.txt 2010-01-29 18:42:22 +0000 @@ -97,7 +97,7 @@ SET_TARGET_PROPERTIES(mysqld PROPERTIES SET (MYSQLD_CORE_LIBS mysys zlib dbug strings yassl taocrypt vio regex sql libevent) TARGET_LINK_LIBRARIES(mysqld ${MYSQLD_CORE_LIBS} ${MYSQLD_STATIC_ENGINE_LIBS}) -TARGET_LINK_LIBRARIES(mysqld ws2_32.lib) +TARGET_LINK_LIBRARIES(mysqld ws2_32.lib psapi.lib) IF(MSVC AND NOT WITHOUT_DYNAMIC_PLUGINS) === modified file 'sql/mysqld.cc' --- a/sql/mysqld.cc 2010-01-28 11:35:10 +0000 +++ b/sql/mysqld.cc 2010-01-29 18:42:22 +0000 @@ -1173,11 +1173,14 @@ static void __cdecl kill_server(int sig_ /* Send event to smem_event_connect_request for aborting */ - if (!SetEvent(smem_event_connect_request)) + if (opt_enable_shared_memory) { - DBUG_PRINT("error", - ("Got error: %ld from SetEvent of smem_event_connect_request", - GetLastError())); + if (!SetEvent(smem_event_connect_request)) + { + DBUG_PRINT("error", + ("Got error: %ld from SetEvent of smem_event_connect_request", + GetLastError())); + } } #endif === modified file 'sql/sql_profile.cc' --- a/sql/sql_profile.cc 2009-10-15 21:38:29 +0000 +++ b/sql/sql_profile.cc 2010-01-29 18:42:22 +0000 @@ -131,6 +131,23 @@ int make_profile_table_for_show(THD *thd #define RUSAGE_USEC(tv) ((tv).tv_sec*1000*1000 + (tv).tv_usec) #define RUSAGE_DIFF_USEC(tv1, tv2) (RUSAGE_USEC((tv1))-RUSAGE_USEC((tv2))) +#ifdef __WIN__ +inline ULONGLONG FileTimeToQuadWord(FILETIME *ft) +{ + ULONGLONG nrv = 0; + nrv |= ft->dwHighDateTime; + nrv <<= 32; + nrv |= ft->dwLowDateTime; + return nrv; +} + + +// Get time difference between to FILETIME objects in seconds. +inline double GetTimeDiffInSeconds(FILETIME *a, FILETIME *b) +{ + return ((FileTimeToQuadWord(a) - FileTimeToQuadWord(b)) / 1e7); +} +#endif /* __WIN__ */ PROF_MEASUREMENT::PROF_MEASUREMENT(QUERY_PROFILE *profile_arg, const char *status_arg) @@ -221,6 +238,11 @@ void PROF_MEASUREMENT::collect() time_usecs= (double) my_getsystime() / 10.0; /* 1 sec was 1e7, now is 1e6 */ #ifdef HAVE_GETRUSAGE getrusage(RUSAGE_SELF, &rusage); +#elif defined(__WIN__) + FILETIME ftDummy; + GetProcessTimes(GetCurrentProcess(), &ftDummy, &ftDummy, &ftKernel, &ftUser); + GetProcessIoCounters(GetCurrentProcess(), &io_count); + GetProcessMemoryInfo(GetCurrentProcess(), &mem_count, sizeof(mem_count)); #endif } @@ -590,6 +612,23 @@ int PROFILING::fill_statistics_info(THD table->field[5]->store_decimal(&cpu_stime_decimal); table->field[4]->set_notnull(); table->field[5]->set_notnull(); +#elif defined(__WIN__) + my_decimal cpu_utime_decimal, cpu_stime_decimal; + + double2my_decimal(E_DEC_FATAL_ERROR, + GetTimeDiffInSeconds(&entry->ftUser, + &previous->ftUser), + &cpu_utime_decimal); + double2my_decimal(E_DEC_FATAL_ERROR, + GetTimeDiffInSeconds(&entry->ftKernel, + &previous->ftKernel), + &cpu_stime_decimal); + + // Store the result. + table->field[4]->store_decimal(&cpu_utime_decimal); + table->field[5]->store_decimal(&cpu_stime_decimal); + table->field[4]->set_notnull(); + table->field[5]->set_notnull(); #else /* TODO: Add CPU-usage info for non-BSD systems */ #endif @@ -612,6 +651,17 @@ int PROFILING::fill_statistics_info(THD table->field[9]->store((uint32)(entry->rusage.ru_oublock - previous->rusage.ru_oublock)); table->field[9]->set_notnull(); +#elif defined(__WIN__) + ULONGLONG reads_delta = entry->io_count.ReadOperationCount - + previous->io_count.ReadOperationCount; + ULONGLONG writes_delta = entry->io_count.WriteOperationCount - + previous->io_count.WriteOperationCount; + + table->field[8]->store((uint32)reads_delta); + table->field[8]->set_notnull(); + + table->field[9]->store((uint32)writes_delta); + table->field[9]->set_notnull(); #else /* TODO: Add block IO info for non-BSD systems */ #endif @@ -634,6 +684,13 @@ int PROFILING::fill_statistics_info(THD table->field[13]->store((uint32)(entry->rusage.ru_minflt - previous->rusage.ru_minflt), true); table->field[13]->set_notnull(); +#elif defined(__WIN__) + /* Windows APIs don't easily distinguish between hard and soft page + faults, so we just fill the 'major' column and leave the second NULL. + */ + table->field[12]->store((uint32)(entry->mem_count.PageFaultCount - + previous->mem_count.PageFaultCount), true); + table->field[12]->set_notnull(); #else /* TODO: Add page fault info for non-BSD systems */ #endif === modified file 'sql/sql_profile.h' --- a/sql/sql_profile.h 2009-09-07 20:50:10 +0000 +++ b/sql/sql_profile.h 2010-01-29 18:42:22 +0000 @@ -36,6 +36,10 @@ int make_profile_table_for_show(THD *thd #if defined(ENABLED_PROFILING) && defined(COMMUNITY_SERVER) #include "mysql_priv.h" +#ifdef __WIN__ +#include <psapi.h> +#endif + #ifdef HAVE_SYS_RESOURCE_H #include <sys/resource.h> #endif @@ -165,6 +169,10 @@ private: char *status; #ifdef HAVE_GETRUSAGE struct rusage rusage; +#elif defined(__WIN__) + FILETIME ftKernel, ftUser; + IO_COUNTERS io_count; + PROCESS_MEMORY_COUNTERS mem_count; #endif char *function; === modified file 'sql/udf_example.def' --- a/sql/udf_example.def 2007-10-29 14:01:40 +0000 +++ b/sql/udf_example.def 2010-01-29 18:42:22 +0000 @@ -3,8 +3,10 @@ VERSION 1.0 EXPORTS lookup lookup_init + lookup_deinit reverse_lookup reverse_lookup_init + reverse_lookup_deinit metaphon_init metaphon_deinit metaphon === modified file 'storage/maria/ma_close.c' --- a/storage/maria/ma_close.c 2009-02-05 22:38:30 +0000 +++ b/storage/maria/ma_close.c 2010-01-29 18:42:22 +0000 @@ -177,6 +177,7 @@ int maria_close(register MARIA_HA *info) { (void) pthread_mutex_destroy(&share->intern_lock); (void) pthread_mutex_destroy(&share->close_lock); + (void) pthread_cond_destroy(&share->key_del_cond); my_free((uchar *)share, MYF(0)); /* If share cannot be freed, it's because checkpoint has previously

1 0

[Maria-developers] [Branch ~maria-captains/maria/5.1] Rev 2808: Patch set contributed by Alex Budovski (MCA)
by noreply＠launchpad.net 29 Jan '10

29 Jan '10

------------------------------------------------------------ revno: 2808 committer: Michael Widenius <monty(a)askmonty.org> branch nick: maria-5.1 timestamp: Fri 2010-01-29 20:42:22 +0200 message: Patch set contributed by Alex Budovski (MCA) Fix for Bug#31173: mysqlslap.exe crashes if called without any parameters modified: .bzrignore client/mysqlslap.c mysys/my_thr_init.c sql-common/client.c sql/CMakeLists.txt sql/mysqld.cc sql/sql_profile.cc sql/sql_profile.h sql/udf_example.def storage/maria/ma_close.c -- lp:maria https://code.launchpad.net/~maria-captains/maria/5.1 Your team Maria developers is subscribed to branch lp:maria. To unsubscribe from this branch go to https://code.launchpad.net/~maria-captains/maria/5.1/+edit-subscription.

1 0

[Maria-developers] Fix for 2 failing tests on Windows
by Alex Budovski 29 Jan '10

29 Jan '10

Hello, Some tests that call mtr.add_suppression to ignore expected warnings hard-code the forward slash '/' character as the expected path delimiter. This breaks tests on Windows as the regex does not match as expected. E.g. Recovering table: './mysqltest/t_corrupted2' fails to match 100123 23:48:26 [Warning] Recovering table: '.\mysqltest\t_corrupted2' and hence the test fails spuriously. Attached is a simple patch that fixes this issue while maintaining compatibility with other platforms. Let me know what you think. - Alex PS. From the Windows buildbot test logs[1], only these two tests are observed to fail in this way. [1] http://askmonty.org/buildbot/builders/win32-tarball/builds/238/steps/test/l…

3 4

[Maria-developers] Updated (by Monty): Subquery optimization: Avoid recalculating subquery if external fields values found in subquery cache (66)
by worklog-noreply＠askmonty.org 29 Jan '10

29 Jan '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Subquery optimization: Avoid recalculating subquery if external fields values found in subquery cache CREATION DATE..: Wed, 25 Nov 2009, 22:25 SUPERVISOR.....: Monty IMPLEMENTOR....: Sanja COPIES TO......: CATEGORY.......: Server-BackLog TASK ID........: 66 (http://askmonty.org/worklog/?tid=66) VERSION........: Server-5.3 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Monty - Fri, 29 Jan 2010, 19:07)=-=- Version updated. --- /tmp/wklog.66.old.5893 2010-01-29 19:07:10.000000000 +0200 +++ /tmp/wklog.66.new.5893 2010-01-29 19:07:10.000000000 +0200 @@ -1 +1 @@ -Server-5.2 +Server-5.3 -=-=(Psergey - Wed, 20 Jan 2010, 14:50)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.26873 2010-01-20 14:50:41.000000000 +0200 +++ /tmp/wklog.66.new.26873 2010-01-20 14:50:41.000000000 +0200 @@ -4,7 +4,6 @@ To check/discuss: ----------------- -* Do we put subquery cache on all levels of subqueries or on highest level only * Will there be any means to measure subquery cache hit rate? * MySQL-6.0 has a one-element predicate result cache. It is called "left expression cache", grep for left_expr_cache in sql/item_subselect.* @@ -41,7 +40,12 @@ - subquery_item_result is 'bool' for subquery predicates, and is of some scalar or ROW(scalar1,...scalarN) type for scalar-context subquery. -We dont support cases when outer_expr or correlation_references are blobs. +We don't support cases when outer_expr or correlation_references are blobs. + +All subquery predicates are cached. That is, if one subquery predicate is +located within another, both of them will have caches (one option to reduce +cache memory usage was to use cache only for the upper-most select. we decided +against it). 2. Data structure used for the cache ------------------------------------ -=-=(Psergey - Wed, 20 Jan 2010, 13:07)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.17649 2010-01-20 13:07:07.000000000 +0200 +++ /tmp/wklog.66.new.17649 2010-01-20 13:07:07.000000000 +0200 @@ -3,7 +3,13 @@ To check/discuss: - To put subquery cache on all levels of subqueries or on highest level only. +----------------- +* Do we put subquery cache on all levels of subqueries or on highest level only +* Will there be any means to measure subquery cache hit rate? +* MySQL-6.0 has a one-element predicate result cache. It is called "left + expression cache", grep for left_expr_cache in sql/item_subselect.* + When this WL is merged with 6.0's optimizations, these two caches will + need to be unified somehow. <contents> -=-=(Psergey - Mon, 18 Jan 2010, 16:40)=-=- Low Level Design modified. --- /tmp/wklog.66.old.24899 2010-01-18 16:40:16.000000000 +0200 +++ /tmp/wklog.66.new.24899 2010-01-18 16:40:16.000000000 +0200 @@ -1,3 +1,5 @@ +* Target version: base on mysql-5.2 code + All items on which subquery depend could be collected in st_select_lex::mark_as_dependent (direct of indirect reference?) -=-=(Psergey - Mon, 18 Jan 2010, 16:37)=-=- Low Level Design modified. --- /tmp/wklog.66.old.24586 2010-01-18 16:37:07.000000000 +0200 +++ /tmp/wklog.66.new.24586 2010-01-18 16:37:07.000000000 +0200 @@ -4,6 +4,11 @@ Temporary table index should be created by all fields except result field (TMP_TABLE_PARAM::keyinfo). +How to fill the temptable +------------------------- +Can reuse approach from SJ-Materialization. Its code is in end_sj_materialize() +and is supposed to be quite trivial. + How to make lookups into temptable ---------------------------------- We'll reuse approach used by SJ-Materialization in 6.0. -=-=(Psergey - Mon, 18 Jan 2010, 16:34)=-=- Low Level Design modified. --- /tmp/wklog.66.old.24328 2010-01-18 16:34:19.000000000 +0200 +++ /tmp/wklog.66.new.24328 2010-01-18 16:34:19.000000000 +0200 @@ -32,8 +32,8 @@ Question: or perhaps that is not necessarry? </questionable> -Execution process -~~~~~~~~~~~~~~~~~ +Doing the lookup +~~~~~~~~~~~~~~~~ SJ-Materialization does lookup in sub_select_sjm(), with this code: /* Do index lookup in the materialized table */ @@ -42,4 +42,12 @@ if (res || !sjm->in_equality->val_int()) DBUG_RETURN(NESTED_LOOP_NO_MORE_ROWS); +The code in this WL will use the same approach +Extracting the value of the subquery predicate +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The goal of making the lookup is to get the value of subquery predicate. +This is done by creating an Item_field $I which refers to appropriate +temporary table's field and then subquery_predicate->val_int() will invoke +$I->val_int(), subquery_predicate->val_str() will invoke $I->val_str() and so +forth. -=-=(Psergey - Mon, 18 Jan 2010, 16:23)=-=- Low Level Design modified. --- /tmp/wklog.66.old.23203 2010-01-18 16:23:18.000000000 +0200 +++ /tmp/wklog.66.new.23203 2010-01-18 16:23:18.000000000 +0200 @@ -31,3 +31,15 @@ Question: or perhaps that is not necessarry? </questionable> + +Execution process +~~~~~~~~~~~~~~~~~ +SJ-Materialization does lookup in sub_select_sjm(), with this code: + + /* Do index lookup in the materialized table */ + if ((res= join_read_key2(join_tab, sjm->table, sjm->tab_ref)) == 1) + DBUG_RETURN(NESTED_LOOP_ERROR); /* purecov: inspected */ + if (res || !sjm->in_equality->val_int()) + DBUG_RETURN(NESTED_LOOP_NO_MORE_ROWS); + + -=-=(Psergey - Mon, 18 Jan 2010, 16:22)=-=- Low Level Design modified. --- /tmp/wklog.66.old.23076 2010-01-18 16:22:07.000000000 +0200 +++ /tmp/wklog.66.new.23076 2010-01-18 16:22:07.000000000 +0200 @@ -4,3 +4,30 @@ Temporary table index should be created by all fields except result field (TMP_TABLE_PARAM::keyinfo). +How to make lookups into temptable +---------------------------------- +We'll reuse approach used by SJ-Materialization in 6.0. + +Setup process +~~~~~~~~~~~~~ +Setup is performed in the same way as in setup_sj_materialization(), +see the code that starts these lines: + + /* + Create/initialize everything we will need to index lookups into the + temptable. + */ + +and ends at this line: + + Remove the injected semi-join IN-equalities from join_tab conds. This + +<questionable> +We'll also need to check equalities, i.e. do an equivalent of this: + + if (!(sjm->in_equality= create_subq_in_equalities(thd, sjm, + emb_sj_nest->sj_subq_pred))) + DBUG_RETURN(TRUE); /* purecov: inspected */ + +Question: or perhaps that is not necessarry? +</questionable> -=-=(Psergey - Tue, 12 Jan 2010, 18:39)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.31666 2010-01-12 18:39:43.000000000 +0200 +++ /tmp/wklog.66.new.31666 2010-01-12 18:39:43.000000000 +0200 @@ -4,3 +4,99 @@ To check/discuss: To put subquery cache on all levels of subqueries or on highest level only. + + +<contents> +1. Scope of the task +2. Data structure used for the cache +3. Cache size +4. Interplay with other subquery optimizations +5. User interface +</contents> + +1. Scope of the task +-------------------- +This WL should handle all subquery predicates, i.e. it should handle these +cases: + + outer_expr IN (SELECT correlated_select) + outer_expr $CMP$ ALL/ANY (SELECT correlated_select) + EXISTS (SELECT correlated_select) + scalar-context subquery: (SELECT correlated_select) + +The cache will maintain + + (outer_expr, correlation_references)-> subquery_item_result + +mapping, where +- correlation_references is a list of tablename.column_name that are referred + from the correlated_select but tablename is a table that is ouside the + subquery. +- subquery_item_result is 'bool' for subquery predicates, and is of +some scalar or ROW(scalar1,...scalarN) type for scalar-context subquery. + +We dont support cases when outer_expr or correlation_references are blobs. + +2. Data structure used for the cache +------------------------------------ +There are two data structures available in the codebase that will allow fast +equality lookups: + +1. HASH (mysys/hash.c) tables +2. Temporary tables (the ones that are used for e.g. GROUP BY) + +None of them has any support for element eviction on overflow (using LRU or +some other policy). + +Query cache and MyISAM/Maria's key/page cache ought to support some eviction +mechanism, but code-wise it is not readily reusable, one will need to factor +it out (or copy it). + +We choose to use #2, and not to have any eviction policy. See subsequent +sections for details and reasoning behind the decision. + +3. Cache size +------------- +Typically, a cache has some maximum size and a policy which is used to +select a cache entry for removal when the cache becomes full (e.g. find +and remove the least [recently] used entry) + +For this WL entry we will use a cache of infinite size. The reasoning behind +this is that: +- is is easy to do: we have temporary tables that can grow to arbitrarily + large size while still providing the same insert/lookup interface. +- it suits us: unless the subquery is resolved with one index lookup, + hitting the cache would be many times cheaper than re-running the + subquery, so cache is worth having. + +4. Interplay with other subquery optimizations +---------------------------------------------- +* This WL entry should not care about IN->EXISTS transformation: caching for + IN subquery and result of its conversion to EXISTS would work in the same + way. + +* This optimization is orthogonal to <=>ANY -> MIN/MAX rewrite (it will + work/be useful irrespectively of whether the rewrite has been performed or + not) + +* TODO: compare this with materialization for uncorrelated IN-subqueries. Is + this basically the same? + A: no, it is not: + - IN-Materialization has to perform full materialization before it can + do the first subquery evaluation. This WL's code has almost no startup + costs. + - This optimization has temp.table of (corr_reference, predicate_value), + while IN-materialization will have (corr_reference) only. + +5. User interface +----------------- +* There will be an @@optimizer_switch flag to turn this optimization on and + off (TODO: name of the flag?) + +* TODO: how do we show this in EXPLAIN [EXTENDED]? The most easiest is to + print something in the warning text of EXPLAIN EXTEDED that would indicate + use of cache. + +* temporary table sizing (max size for heap table, whether to use MyISAM or + Maria) will be controlled with common temp.table control variables. + -=-=(Psergey - Mon, 11 Jan 2010, 13:25)=-=- As of today, there is code that - collects outside references - creates a temporary table with index that would allow for fast lookups. there is no code to - fill the temporary table - make lookups into it Reported zero hours worked. Estimate unchanged. ------------------------------------------------------------ -=-=(View All Progress Notes, 13 total)=-=- http://askmonty.org/worklog/index.pl?tid=66&nolimit=1 DESCRIPTION: Collect all outer items/references (left part of the subquiery and outer references inside the subquery) in key string. Compare the string (which represents certain value set of the references) against values in hash table and return cached result of subquery if the reference values combination has already been used. For example in the following subquery: (L1, L2) IN (SELECT A, B FROM T WHERE T.F1>OTER_FIELD) set of references to look into the subquery cache is (L1, L2, OTER_FIELD). The subquery cache should be implemented as simple LRU connected to the subquery. Size of the subquery cache (in number of results (but maybe in used memory amount)) is limited by session variable (query parameter?). HIGH-LEVEL SPECIFICATION: Attach subquery cache to each Item_subquery. Interface should allow to use hash or temporary table inside. To check/discuss: ----------------- * Will there be any means to measure subquery cache hit rate? * MySQL-6.0 has a one-element predicate result cache. It is called "left expression cache", grep for left_expr_cache in sql/item_subselect.* When this WL is merged with 6.0's optimizations, these two caches will need to be unified somehow. <contents> 1. Scope of the task 2. Data structure used for the cache 3. Cache size 4. Interplay with other subquery optimizations 5. User interface </contents> 1. Scope of the task -------------------- This WL should handle all subquery predicates, i.e. it should handle these cases: outer_expr IN (SELECT correlated_select) outer_expr $CMP$ ALL/ANY (SELECT correlated_select) EXISTS (SELECT correlated_select) scalar-context subquery: (SELECT correlated_select) The cache will maintain (outer_expr, correlation_references)-> subquery_item_result mapping, where - correlation_references is a list of tablename.column_name that are referred from the correlated_select but tablename is a table that is ouside the subquery. - subquery_item_result is 'bool' for subquery predicates, and is of some scalar or ROW(scalar1,...scalarN) type for scalar-context subquery. We don't support cases when outer_expr or correlation_references are blobs. All subquery predicates are cached. That is, if one subquery predicate is located within another, both of them will have caches (one option to reduce cache memory usage was to use cache only for the upper-most select. we decided against it). 2. Data structure used for the cache ------------------------------------ There are two data structures available in the codebase that will allow fast equality lookups: 1. HASH (mysys/hash.c) tables 2. Temporary tables (the ones that are used for e.g. GROUP BY) None of them has any support for element eviction on overflow (using LRU or some other policy). Query cache and MyISAM/Maria's key/page cache ought to support some eviction mechanism, but code-wise it is not readily reusable, one will need to factor it out (or copy it). We choose to use #2, and not to have any eviction policy. See subsequent sections for details and reasoning behind the decision. 3. Cache size ------------- Typically, a cache has some maximum size and a policy which is used to select a cache entry for removal when the cache becomes full (e.g. find and remove the least [recently] used entry) For this WL entry we will use a cache of infinite size. The reasoning behind this is that: - is is easy to do: we have temporary tables that can grow to arbitrarily large size while still providing the same insert/lookup interface. - it suits us: unless the subquery is resolved with one index lookup, hitting the cache would be many times cheaper than re-running the subquery, so cache is worth having. 4. Interplay with other subquery optimizations ---------------------------------------------- * This WL entry should not care about IN->EXISTS transformation: caching for IN subquery and result of its conversion to EXISTS would work in the same way. * This optimization is orthogonal to <=>ANY -> MIN/MAX rewrite (it will work/be useful irrespectively of whether the rewrite has been performed or not) * TODO: compare this with materialization for uncorrelated IN-subqueries. Is this basically the same? A: no, it is not: - IN-Materialization has to perform full materialization before it can do the first subquery evaluation. This WL's code has almost no startup costs. - This optimization has temp.table of (corr_reference, predicate_value), while IN-materialization will have (corr_reference) only. 5. User interface ----------------- * There will be an @@optimizer_switch flag to turn this optimization on and off (TODO: name of the flag?) * TODO: how do we show this in EXPLAIN [EXTENDED]? The most easiest is to print something in the warning text of EXPLAIN EXTEDED that would indicate use of cache. * temporary table sizing (max size for heap table, whether to use MyISAM or Maria) will be controlled with common temp.table control variables. LOW-LEVEL DESIGN: * Target version: base on mysql-5.2 code All items on which subquery depend could be collected in st_select_lex::mark_as_dependent (direct of indirect reference?) Temporary table index should be created by all fields except result field (TMP_TABLE_PARAM::keyinfo). How to fill the temptable ------------------------- Can reuse approach from SJ-Materialization. Its code is in end_sj_materialize() and is supposed to be quite trivial. How to make lookups into temptable ---------------------------------- We'll reuse approach used by SJ-Materialization in 6.0. Setup process ~~~~~~~~~~~~~ Setup is performed in the same way as in setup_sj_materialization(), see the code that starts these lines: /* Create/initialize everything we will need to index lookups into the temptable. */ and ends at this line: Remove the injected semi-join IN-equalities from join_tab conds. This <questionable> We'll also need to check equalities, i.e. do an equivalent of this: if (!(sjm->in_equality= create_subq_in_equalities(thd, sjm, emb_sj_nest->sj_subq_pred))) DBUG_RETURN(TRUE); /* purecov: inspected */ Question: or perhaps that is not necessarry? </questionable> Doing the lookup ~~~~~~~~~~~~~~~~ SJ-Materialization does lookup in sub_select_sjm(), with this code: /* Do index lookup in the materialized table */ if ((res= join_read_key2(join_tab, sjm->table, sjm->tab_ref)) == 1) DBUG_RETURN(NESTED_LOOP_ERROR); /* purecov: inspected */ if (res || !sjm->in_equality->val_int()) DBUG_RETURN(NESTED_LOOP_NO_MORE_ROWS); The code in this WL will use the same approach Extracting the value of the subquery predicate ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The goal of making the lookup is to get the value of subquery predicate. This is done by creating an Item_field $I which refers to appropriate temporary table's field and then subquery_predicate->val_int() will invoke $I->val_int(), subquery_predicate->val_str() will invoke $I->val_str() and so forth. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Monty): Subquery optimization: Avoid recalculating subquery if external fields values found in subquery cache (66)
by worklog-noreply＠askmonty.org 29 Jan '10

29 Jan '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Subquery optimization: Avoid recalculating subquery if external fields values found in subquery cache CREATION DATE..: Wed, 25 Nov 2009, 22:25 SUPERVISOR.....: Monty IMPLEMENTOR....: Sanja COPIES TO......: CATEGORY.......: Server-BackLog TASK ID........: 66 (http://askmonty.org/worklog/?tid=66) VERSION........: Server-5.3 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Monty - Fri, 29 Jan 2010, 19:07)=-=- Version updated. --- /tmp/wklog.66.old.5893 2010-01-29 19:07:10.000000000 +0200 +++ /tmp/wklog.66.new.5893 2010-01-29 19:07:10.000000000 +0200 @@ -1 +1 @@ -Server-5.2 +Server-5.3 -=-=(Psergey - Wed, 20 Jan 2010, 14:50)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.26873 2010-01-20 14:50:41.000000000 +0200 +++ /tmp/wklog.66.new.26873 2010-01-20 14:50:41.000000000 +0200 @@ -4,7 +4,6 @@ To check/discuss: ----------------- -* Do we put subquery cache on all levels of subqueries or on highest level only * Will there be any means to measure subquery cache hit rate? * MySQL-6.0 has a one-element predicate result cache. It is called "left expression cache", grep for left_expr_cache in sql/item_subselect.* @@ -41,7 +40,12 @@ - subquery_item_result is 'bool' for subquery predicates, and is of some scalar or ROW(scalar1,...scalarN) type for scalar-context subquery. -We dont support cases when outer_expr or correlation_references are blobs. +We don't support cases when outer_expr or correlation_references are blobs. + +All subquery predicates are cached. That is, if one subquery predicate is +located within another, both of them will have caches (one option to reduce +cache memory usage was to use cache only for the upper-most select. we decided +against it). 2. Data structure used for the cache ------------------------------------ -=-=(Psergey - Wed, 20 Jan 2010, 13:07)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.17649 2010-01-20 13:07:07.000000000 +0200 +++ /tmp/wklog.66.new.17649 2010-01-20 13:07:07.000000000 +0200 @@ -3,7 +3,13 @@ To check/discuss: - To put subquery cache on all levels of subqueries or on highest level only. +----------------- +* Do we put subquery cache on all levels of subqueries or on highest level only +* Will there be any means to measure subquery cache hit rate? +* MySQL-6.0 has a one-element predicate result cache. It is called "left + expression cache", grep for left_expr_cache in sql/item_subselect.* + When this WL is merged with 6.0's optimizations, these two caches will + need to be unified somehow. <contents> -=-=(Psergey - Mon, 18 Jan 2010, 16:40)=-=- Low Level Design modified. --- /tmp/wklog.66.old.24899 2010-01-18 16:40:16.000000000 +0200 +++ /tmp/wklog.66.new.24899 2010-01-18 16:40:16.000000000 +0200 @@ -1,3 +1,5 @@ +* Target version: base on mysql-5.2 code + All items on which subquery depend could be collected in st_select_lex::mark_as_dependent (direct of indirect reference?) -=-=(Psergey - Mon, 18 Jan 2010, 16:37)=-=- Low Level Design modified. --- /tmp/wklog.66.old.24586 2010-01-18 16:37:07.000000000 +0200 +++ /tmp/wklog.66.new.24586 2010-01-18 16:37:07.000000000 +0200 @@ -4,6 +4,11 @@ Temporary table index should be created by all fields except result field (TMP_TABLE_PARAM::keyinfo). +How to fill the temptable +------------------------- +Can reuse approach from SJ-Materialization. Its code is in end_sj_materialize() +and is supposed to be quite trivial. + How to make lookups into temptable ---------------------------------- We'll reuse approach used by SJ-Materialization in 6.0. -=-=(Psergey - Mon, 18 Jan 2010, 16:34)=-=- Low Level Design modified. --- /tmp/wklog.66.old.24328 2010-01-18 16:34:19.000000000 +0200 +++ /tmp/wklog.66.new.24328 2010-01-18 16:34:19.000000000 +0200 @@ -32,8 +32,8 @@ Question: or perhaps that is not necessarry? </questionable> -Execution process -~~~~~~~~~~~~~~~~~ +Doing the lookup +~~~~~~~~~~~~~~~~ SJ-Materialization does lookup in sub_select_sjm(), with this code: /* Do index lookup in the materialized table */ @@ -42,4 +42,12 @@ if (res || !sjm->in_equality->val_int()) DBUG_RETURN(NESTED_LOOP_NO_MORE_ROWS); +The code in this WL will use the same approach +Extracting the value of the subquery predicate +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The goal of making the lookup is to get the value of subquery predicate. +This is done by creating an Item_field $I which refers to appropriate +temporary table's field and then subquery_predicate->val_int() will invoke +$I->val_int(), subquery_predicate->val_str() will invoke $I->val_str() and so +forth. -=-=(Psergey - Mon, 18 Jan 2010, 16:23)=-=- Low Level Design modified. --- /tmp/wklog.66.old.23203 2010-01-18 16:23:18.000000000 +0200 +++ /tmp/wklog.66.new.23203 2010-01-18 16:23:18.000000000 +0200 @@ -31,3 +31,15 @@ Question: or perhaps that is not necessarry? </questionable> + +Execution process +~~~~~~~~~~~~~~~~~ +SJ-Materialization does lookup in sub_select_sjm(), with this code: + + /* Do index lookup in the materialized table */ + if ((res= join_read_key2(join_tab, sjm->table, sjm->tab_ref)) == 1) + DBUG_RETURN(NESTED_LOOP_ERROR); /* purecov: inspected */ + if (res || !sjm->in_equality->val_int()) + DBUG_RETURN(NESTED_LOOP_NO_MORE_ROWS); + + -=-=(Psergey - Mon, 18 Jan 2010, 16:22)=-=- Low Level Design modified. --- /tmp/wklog.66.old.23076 2010-01-18 16:22:07.000000000 +0200 +++ /tmp/wklog.66.new.23076 2010-01-18 16:22:07.000000000 +0200 @@ -4,3 +4,30 @@ Temporary table index should be created by all fields except result field (TMP_TABLE_PARAM::keyinfo). +How to make lookups into temptable +---------------------------------- +We'll reuse approach used by SJ-Materialization in 6.0. + +Setup process +~~~~~~~~~~~~~ +Setup is performed in the same way as in setup_sj_materialization(), +see the code that starts these lines: + + /* + Create/initialize everything we will need to index lookups into the + temptable. + */ + +and ends at this line: + + Remove the injected semi-join IN-equalities from join_tab conds. This + +<questionable> +We'll also need to check equalities, i.e. do an equivalent of this: + + if (!(sjm->in_equality= create_subq_in_equalities(thd, sjm, + emb_sj_nest->sj_subq_pred))) + DBUG_RETURN(TRUE); /* purecov: inspected */ + +Question: or perhaps that is not necessarry? +</questionable> -=-=(Psergey - Tue, 12 Jan 2010, 18:39)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.31666 2010-01-12 18:39:43.000000000 +0200 +++ /tmp/wklog.66.new.31666 2010-01-12 18:39:43.000000000 +0200 @@ -4,3 +4,99 @@ To check/discuss: To put subquery cache on all levels of subqueries or on highest level only. + + +<contents> +1. Scope of the task +2. Data structure used for the cache +3. Cache size +4. Interplay with other subquery optimizations +5. User interface +</contents> + +1. Scope of the task +-------------------- +This WL should handle all subquery predicates, i.e. it should handle these +cases: + + outer_expr IN (SELECT correlated_select) + outer_expr $CMP$ ALL/ANY (SELECT correlated_select) + EXISTS (SELECT correlated_select) + scalar-context subquery: (SELECT correlated_select) + +The cache will maintain + + (outer_expr, correlation_references)-> subquery_item_result + +mapping, where +- correlation_references is a list of tablename.column_name that are referred + from the correlated_select but tablename is a table that is ouside the + subquery. +- subquery_item_result is 'bool' for subquery predicates, and is of +some scalar or ROW(scalar1,...scalarN) type for scalar-context subquery. + +We dont support cases when outer_expr or correlation_references are blobs. + +2. Data structure used for the cache +------------------------------------ +There are two data structures available in the codebase that will allow fast +equality lookups: + +1. HASH (mysys/hash.c) tables +2. Temporary tables (the ones that are used for e.g. GROUP BY) + +None of them has any support for element eviction on overflow (using LRU or +some other policy). + +Query cache and MyISAM/Maria's key/page cache ought to support some eviction +mechanism, but code-wise it is not readily reusable, one will need to factor +it out (or copy it). + +We choose to use #2, and not to have any eviction policy. See subsequent +sections for details and reasoning behind the decision. + +3. Cache size +------------- +Typically, a cache has some maximum size and a policy which is used to +select a cache entry for removal when the cache becomes full (e.g. find +and remove the least [recently] used entry) + +For this WL entry we will use a cache of infinite size. The reasoning behind +this is that: +- is is easy to do: we have temporary tables that can grow to arbitrarily + large size while still providing the same insert/lookup interface. +- it suits us: unless the subquery is resolved with one index lookup, + hitting the cache would be many times cheaper than re-running the + subquery, so cache is worth having. + +4. Interplay with other subquery optimizations +---------------------------------------------- +* This WL entry should not care about IN->EXISTS transformation: caching for + IN subquery and result of its conversion to EXISTS would work in the same + way. + +* This optimization is orthogonal to <=>ANY -> MIN/MAX rewrite (it will + work/be useful irrespectively of whether the rewrite has been performed or + not) + +* TODO: compare this with materialization for uncorrelated IN-subqueries. Is + this basically the same? + A: no, it is not: + - IN-Materialization has to perform full materialization before it can + do the first subquery evaluation. This WL's code has almost no startup + costs. + - This optimization has temp.table of (corr_reference, predicate_value), + while IN-materialization will have (corr_reference) only. + +5. User interface +----------------- +* There will be an @@optimizer_switch flag to turn this optimization on and + off (TODO: name of the flag?) + +* TODO: how do we show this in EXPLAIN [EXTENDED]? The most easiest is to + print something in the warning text of EXPLAIN EXTEDED that would indicate + use of cache. + +* temporary table sizing (max size for heap table, whether to use MyISAM or + Maria) will be controlled with common temp.table control variables. + -=-=(Psergey - Mon, 11 Jan 2010, 13:25)=-=- As of today, there is code that - collects outside references - creates a temporary table with index that would allow for fast lookups. there is no code to - fill the temporary table - make lookups into it Reported zero hours worked. Estimate unchanged. ------------------------------------------------------------ -=-=(View All Progress Notes, 13 total)=-=- http://askmonty.org/worklog/index.pl?tid=66&nolimit=1 DESCRIPTION: Collect all outer items/references (left part of the subquiery and outer references inside the subquery) in key string. Compare the string (which represents certain value set of the references) against values in hash table and return cached result of subquery if the reference values combination has already been used. For example in the following subquery: (L1, L2) IN (SELECT A, B FROM T WHERE T.F1>OTER_FIELD) set of references to look into the subquery cache is (L1, L2, OTER_FIELD). The subquery cache should be implemented as simple LRU connected to the subquery. Size of the subquery cache (in number of results (but maybe in used memory amount)) is limited by session variable (query parameter?). HIGH-LEVEL SPECIFICATION: Attach subquery cache to each Item_subquery. Interface should allow to use hash or temporary table inside. To check/discuss: ----------------- * Will there be any means to measure subquery cache hit rate? * MySQL-6.0 has a one-element predicate result cache. It is called "left expression cache", grep for left_expr_cache in sql/item_subselect.* When this WL is merged with 6.0's optimizations, these two caches will need to be unified somehow. <contents> 1. Scope of the task 2. Data structure used for the cache 3. Cache size 4. Interplay with other subquery optimizations 5. User interface </contents> 1. Scope of the task -------------------- This WL should handle all subquery predicates, i.e. it should handle these cases: outer_expr IN (SELECT correlated_select) outer_expr $CMP$ ALL/ANY (SELECT correlated_select) EXISTS (SELECT correlated_select) scalar-context subquery: (SELECT correlated_select) The cache will maintain (outer_expr, correlation_references)-> subquery_item_result mapping, where - correlation_references is a list of tablename.column_name that are referred from the correlated_select but tablename is a table that is ouside the subquery. - subquery_item_result is 'bool' for subquery predicates, and is of some scalar or ROW(scalar1,...scalarN) type for scalar-context subquery. We don't support cases when outer_expr or correlation_references are blobs. All subquery predicates are cached. That is, if one subquery predicate is located within another, both of them will have caches (one option to reduce cache memory usage was to use cache only for the upper-most select. we decided against it). 2. Data structure used for the cache ------------------------------------ There are two data structures available in the codebase that will allow fast equality lookups: 1. HASH (mysys/hash.c) tables 2. Temporary tables (the ones that are used for e.g. GROUP BY) None of them has any support for element eviction on overflow (using LRU or some other policy). Query cache and MyISAM/Maria's key/page cache ought to support some eviction mechanism, but code-wise it is not readily reusable, one will need to factor it out (or copy it). We choose to use #2, and not to have any eviction policy. See subsequent sections for details and reasoning behind the decision. 3. Cache size ------------- Typically, a cache has some maximum size and a policy which is used to select a cache entry for removal when the cache becomes full (e.g. find and remove the least [recently] used entry) For this WL entry we will use a cache of infinite size. The reasoning behind this is that: - is is easy to do: we have temporary tables that can grow to arbitrarily large size while still providing the same insert/lookup interface. - it suits us: unless the subquery is resolved with one index lookup, hitting the cache would be many times cheaper than re-running the subquery, so cache is worth having. 4. Interplay with other subquery optimizations ---------------------------------------------- * This WL entry should not care about IN->EXISTS transformation: caching for IN subquery and result of its conversion to EXISTS would work in the same way. * This optimization is orthogonal to <=>ANY -> MIN/MAX rewrite (it will work/be useful irrespectively of whether the rewrite has been performed or not) * TODO: compare this with materialization for uncorrelated IN-subqueries. Is this basically the same? A: no, it is not: - IN-Materialization has to perform full materialization before it can do the first subquery evaluation. This WL's code has almost no startup costs. - This optimization has temp.table of (corr_reference, predicate_value), while IN-materialization will have (corr_reference) only. 5. User interface ----------------- * There will be an @@optimizer_switch flag to turn this optimization on and off (TODO: name of the flag?) * TODO: how do we show this in EXPLAIN [EXTENDED]? The most easiest is to print something in the warning text of EXPLAIN EXTEDED that would indicate use of cache. * temporary table sizing (max size for heap table, whether to use MyISAM or Maria) will be controlled with common temp.table control variables. LOW-LEVEL DESIGN: * Target version: base on mysql-5.2 code All items on which subquery depend could be collected in st_select_lex::mark_as_dependent (direct of indirect reference?) Temporary table index should be created by all fields except result field (TMP_TABLE_PARAM::keyinfo). How to fill the temptable ------------------------- Can reuse approach from SJ-Materialization. Its code is in end_sj_materialize() and is supposed to be quite trivial. How to make lookups into temptable ---------------------------------- We'll reuse approach used by SJ-Materialization in 6.0. Setup process ~~~~~~~~~~~~~ Setup is performed in the same way as in setup_sj_materialization(), see the code that starts these lines: /* Create/initialize everything we will need to index lookups into the temptable. */ and ends at this line: Remove the injected semi-join IN-equalities from join_tab conds. This <questionable> We'll also need to check equalities, i.e. do an equivalent of this: if (!(sjm->in_equality= create_subq_in_equalities(thd, sjm, emb_sj_nest->sj_subq_pred))) DBUG_RETURN(TRUE); /* purecov: inspected */ Question: or perhaps that is not necessarry? </questionable> Doing the lookup ~~~~~~~~~~~~~~~~ SJ-Materialization does lookup in sub_select_sjm(), with this code: /* Do index lookup in the materialized table */ if ((res= join_read_key2(join_tab, sjm->table, sjm->tab_ref)) == 1) DBUG_RETURN(NESTED_LOOP_ERROR); /* purecov: inspected */ if (res || !sjm->in_equality->val_int()) DBUG_RETURN(NESTED_LOOP_NO_MORE_ROWS); The code in this WL will use the same approach Extracting the value of the subquery predicate ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The goal of making the lookup is to get the value of subquery predicate. This is done by creating an Item_field $I which refers to appropriate temporary table's field and then subquery_predicate->val_int() will invoke $I->val_int(), subquery_predicate->val_str() will invoke $I->val_str() and so forth. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Monty): Microseconds (10)
by worklog-noreply＠askmonty.org 29 Jan '10

29 Jan '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Microseconds CREATION DATE..: Thu, 26 Mar 2009, 00:29 SUPERVISOR.....: Monty IMPLEMENTOR....: Bothorsen COPIES TO......: CATEGORY.......: Server-Sprint TASK ID........: 10 (http://askmonty.org/worklog/?tid=10) VERSION........: Server-5.3 STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 80 (hours remain) ORIG. ESTIMATE.: 80 PROGRESS NOTES: -=-=(Monty - Fri, 29 Jan 2010, 19:05)=-=- Version updated. --- /tmp/wklog.10.old.5698 2010-01-29 19:05:42.000000000 +0200 +++ /tmp/wklog.10.new.5698 2010-01-29 19:05:42.000000000 +0200 @@ -1 +1 @@ -Server-5.2 +Server-5.3 DESCRIPTION: Add microsecond precision to NOW() Add new field types for time and datetime with microprecision ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Monty): Microseconds (10)
by worklog-noreply＠askmonty.org 29 Jan '10

29 Jan '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Microseconds CREATION DATE..: Thu, 26 Mar 2009, 00:29 SUPERVISOR.....: Monty IMPLEMENTOR....: Bothorsen COPIES TO......: CATEGORY.......: Server-Sprint TASK ID........: 10 (http://askmonty.org/worklog/?tid=10) VERSION........: Server-5.3 STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 80 (hours remain) ORIG. ESTIMATE.: 80 PROGRESS NOTES: -=-=(Monty - Fri, 29 Jan 2010, 19:05)=-=- Version updated. --- /tmp/wklog.10.old.5698 2010-01-29 19:05:42.000000000 +0200 +++ /tmp/wklog.10.new.5698 2010-01-29 19:05:42.000000000 +0200 @@ -1 +1 @@ -Server-5.2 +Server-5.3 DESCRIPTION: Add microsecond precision to NOW() Add new field types for time and datetime with microprecision ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (monty:2807)
by Michael Widenius 29 Jan '10

29 Jan '10

#At lp:maria based on revid:knielsen@knielsen-hq.org-20100128223422-t6apa3py61dgbjjw 2807 Michael Widenius 2010-01-29 [merge] Auto merge modified: client/mysqltest.cc configure.in extra/yassl/src/yassl_error.cpp include/violite.h mysql-test/mysql-test-run.pl mysql-test/r/grant.result mysql-test/r/openssl_1.result mysql-test/std_data/cacert.pem mysql-test/std_data/client-cert.pem mysql-test/std_data/client-key.pem mysql-test/std_data/server-cert.pem mysql-test/std_data/server-key.pem mysql-test/t/grant.test mysql-test/t/openssl_1.test mysql-test/t/partition_innodb_semi_consistent.test mysql-test/t/query_cache_debug.test sql-common/client.c sql/net_serv.cc sql/slave.cc sql/sql_cache.cc sql/sql_connect.cc sql/sql_parse.cc storage/myisam/ft_boolean_search.c storage/myisam/ft_parser.c storage/myisam/ft_stopwords.c support-files/compiler_warnings.supp unittest/mysys/waiting_threads-t.c vio/viossl.c === modified file 'client/mysqltest.cc' --- a/client/mysqltest.cc 2010-01-15 15:27:55 +0000 +++ b/client/mysqltest.cc 2010-01-28 14:49:14 +0000 @@ -657,7 +657,9 @@ public: } while ((bytes= fread(buf, 1, sizeof(buf), m_file)) > 0) - fwrite(buf, 1, bytes, stderr); + if (fwrite(buf, 1, bytes, stderr)) + die("Failed to write to '%s', errno: %d", + m_file_name, errno); if (!lines) { === modified file 'configure.in' --- a/configure.in 2010-01-17 17:22:46 +0000 +++ b/configure.in 2010-01-29 10:42:31 +0000 @@ -10,7 +10,7 @@ AC_CANONICAL_SYSTEM # # When changing major version number please also check switch statement # in mysqlbinlog::check_master_version(). -AM_INIT_AUTOMAKE(mysql, 5.1.42-MariaDB-rc) +AM_INIT_AUTOMAKE(mysql, 5.1.42-MariaDB) AM_CONFIG_HEADER([include/config.h:config.h.in]) # Request support for automake silent-rules if available. === modified file 'extra/yassl/src/yassl_error.cpp' --- a/extra/yassl/src/yassl_error.cpp 2008-11-18 16:45:44 +0000 +++ b/extra/yassl/src/yassl_error.cpp 2010-01-29 10:42:31 +0000 @@ -60,7 +60,7 @@ void SetErrorString(YasslError error, ch using namespace TaoCrypt; const int max = MAX_ERROR_SZ; // shorthand - switch (error) { + switch ((int) error) { // yaSSL proper errors case range_error : === modified file 'include/violite.h' --- a/include/violite.h 2010-01-15 15:27:55 +0000 +++ b/include/violite.h 2010-01-29 10:42:31 +0000 @@ -123,8 +123,8 @@ struct st_VioSSLFd SSL_CTX *ssl_context; }; -int sslaccept(struct st_VioSSLFd*, Vio *, long timeout); -int sslconnect(struct st_VioSSLFd*, Vio *, long timeout); +int sslaccept(struct st_VioSSLFd*, Vio *, long timeout, char *error_string); +int sslconnect(struct st_VioSSLFd*, Vio *, long timeout, char *error_string); struct st_VioSSLFd *new_VioSSLConnectorFd(const char *key_file, const char *cert_file, === modified file 'mysql-test/mysql-test-run.pl' --- a/mysql-test/mysql-test-run.pl 2010-01-27 17:41:05 +0000 +++ b/mysql-test/mysql-test-run.pl 2010-01-29 10:42:31 +0000 @@ -413,7 +413,6 @@ sub main { # Not all tests completed, failure mtr_report(); mtr_report("Only ", int(@$completed), " of $num_tests completed."); - mtr_error("Not all tests completed"); } mtr_print_line(); @@ -425,6 +424,10 @@ sub main { mtr_report_stats($fail, $completed, $extra_warnings); + if ( @$completed != $num_tests) + { + mtr_error("Not all tests completed"); + } exit(0); } === modified file 'mysql-test/r/grant.result' --- a/mysql-test/r/grant.result 2009-10-27 10:09:36 +0000 +++ b/mysql-test/r/grant.result 2010-01-29 10:42:31 +0000 @@ -29,15 +29,15 @@ show grants for mysqltest_1@localhost; Grants for mysqltest_1@localhost GRANT USAGE ON *.* TO 'mysqltest_1'@'localhost' GRANT SELECT ON `mysqltest`.* TO 'mysqltest_1'@'localhost' -grant USAGE on mysqltest.* to mysqltest_1@localhost require cipher "EDH-RSA-DES-CBC3-SHA" AND SUBJECT "testsubject" ISSUER "MySQL AB"; +grant USAGE on mysqltest.* to mysqltest_1@localhost require cipher "EDH-RSA-DES-CBC3-SHA" AND SUBJECT "testsubject" ISSUER "Monty Program Ab"; show grants for mysqltest_1@localhost; Grants for mysqltest_1@localhost -GRANT USAGE ON *.* TO 'mysqltest_1'@'localhost' REQUIRE ISSUER 'MySQL AB' SUBJECT 'testsubject' CIPHER 'EDH-RSA-DES-CBC3-SHA' +GRANT USAGE ON *.* TO 'mysqltest_1'@'localhost' REQUIRE ISSUER 'Monty Program Ab' SUBJECT 'testsubject' CIPHER 'EDH-RSA-DES-CBC3-SHA' GRANT SELECT ON `mysqltest`.* TO 'mysqltest_1'@'localhost' revoke all privileges on mysqltest.* from mysqltest_1@localhost; show grants for mysqltest_1@localhost; Grants for mysqltest_1@localhost -GRANT USAGE ON *.* TO 'mysqltest_1'@'localhost' REQUIRE ISSUER 'MySQL AB' SUBJECT 'testsubject' CIPHER 'EDH-RSA-DES-CBC3-SHA' +GRANT USAGE ON *.* TO 'mysqltest_1'@'localhost' REQUIRE ISSUER 'Monty Program Ab' SUBJECT 'testsubject' CIPHER 'EDH-RSA-DES-CBC3-SHA' delete from mysql.user where user='mysqltest_1'; flush privileges; delete from mysql.user where user='mysqltest_1'; === modified file 'mysql-test/r/openssl_1.result' --- a/mysql-test/r/openssl_1.result 2009-05-25 13:00:18 +0000 +++ b/mysql-test/r/openssl_1.result 2010-01-29 10:42:31 +0000 @@ -3,8 +3,10 @@ create table t1(f1 int); insert into t1 values (5); grant select on test.* to ssl_user1@localhost require SSL; grant select on test.* to ssl_user2@localhost require cipher "DHE-RSA-AES256-SHA"; -grant select on test.* to ssl_user3@localhost require cipher "DHE-RSA-AES256-SHA" AND SUBJECT "/C=SE/ST=Uppsala/O=MySQL AB/emailAddress=abstract.mysql.developer(a)mysql.com"; -grant select on test.* to ssl_user4@localhost require cipher "DHE-RSA-AES256-SHA" AND SUBJECT "/C=SE/ST=Uppsala/O=MySQL AB/emailAddress=abstract.mysql.developer(a)mysql.com" ISSUER "/C=SE/ST=Uppsala/L=Uppsala/O=MySQL AB"; +grant select on test.* to ssl_user3@localhost require cipher +"DHE-RSA-AES256-SHA" AND SUBJECT "/C=FI/ST=Tuusula/O=Monty Program Ab/emailAddress=abstract.developer(a)askmonty.org"; +grant select on test.* to ssl_user4@localhost require cipher +"DHE-RSA-AES256-SHA" AND SUBJECT "/C=FI/ST=Tuusula/O=Monty Program Ab/emailAddress=abstract.developer(a)askmonty.org" ISSUER "/C=FI/ST=Tuusula/O=Monty Program Ab/emailAddress=abstract.developer(a)askmonty.org"; grant select on test.* to ssl_user5@localhost require cipher "DHE-RSA-AES256-SHA" AND SUBJECT "xxx"; flush privileges; connect(localhost,ssl_user5,,test,MASTER_PORT,MASTER_SOCKET); @@ -44,9 +46,9 @@ ERROR 42000: DELETE command denied to us drop user ssl_user1@localhost, ssl_user2@localhost, ssl_user3@localhost, ssl_user4@localhost, ssl_user5@localhost; drop table t1; -mysqltest: Could not open connection 'default': 2026 SSL connection error -mysqltest: Could not open connection 'default': 2026 SSL connection error -mysqltest: Could not open connection 'default': 2026 SSL connection error +mysqltest: Could not open connection 'default': 2026 SSL error: ASN: bad other signature confirmation +mysqltest: Could not open connection 'default': 2026 SSL error: ASN: bad other signature confirmation +mysqltest: Could not open connection 'default': 2026 SSL error: ASN: bad other signature confirmation SSL error: Unable to get private key from '' mysqltest: Could not open connection 'default': 2026 SSL connection error SSL error: Unable to get certificate from '' === modified file 'mysql-test/std_data/cacert.pem' --- a/mysql-test/std_data/cacert.pem 2009-06-11 16:21:32 +0000 +++ b/mysql-test/std_data/cacert.pem 2010-01-29 10:42:31 +0000 @@ -1,17 +1,19 @@ -----BEGIN CERTIFICATE----- -MIICrTCCAhagAwIBAgIJAJXpePU0UOTVMA0GCSqGSIb3DQEBBQUAMEQxCzAJBgNV -BAYTAlNFMRAwDgYDVQQIEwdVcHBzYWxhMRAwDgYDVQQHEwdVcHBzYWxhMREwDwYD -VQQKEwhNeVNRTCBBQjAeFw0wOTAxMjgxMDQ5NDZaFw0xNDAxMjcxMDQ5NDZaMEQx -CzAJBgNVBAYTAlNFMRAwDgYDVQQIEwdVcHBzYWxhMRAwDgYDVQQHEwdVcHBzYWxh -MREwDwYDVQQKEwhNeVNRTCBBQjCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEA -4XQHAe5R1+TXC8noZtWf+d5E0v1C59FWpn9SWEUCBjE5UiIwuJvi4Y+7xWGOXLAI -/JzJx5gNXLBiTsE/zh0uX9fKlajLhxB0GN+QU0ZlpQ1BeYipEcNXeI/7cT499f6v -XWabnTflivdCgHSWUOQ20/Lzs6kP6/e6OoZd/DPSjPECAwEAAaOBpjCBozAdBgNV -HQ4EFgQU8uLqVWWkmuKsnZf1RWz294wRrd8wdAYDVR0jBG0wa4AU8uLqVWWkmuKs -nZf1RWz294wRrd+hSKRGMEQxCzAJBgNVBAYTAlNFMRAwDgYDVQQIEwdVcHBzYWxh -MRAwDgYDVQQHEwdVcHBzYWxhMREwDwYDVQQKEwhNeVNRTCBBQoIJAJXpePU0UOTV -MAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcNAQEFBQADgYEAMMTE5sDN+Z0ZlV7KvH3g -6+aKvql8dTpRT3hYukeQlWua0nq74WPGVw0c4e/M/vbiMwmJcCYpB9pd4+dHqzSw -aPyoenjY6UF8n7B4quWy3SIUk2LSHeJLW+kzJn2afN9gvipFhdVh/uU2TIyLGOur -Z/vmJX2W7hF1uqPnbfa8Lrw= +MIIDIjCCAougAwIBAgIJAJhuvLP+2mGwMA0GCSqGSIb3DQEBBQUAMGoxCzAJBgNV +BAYTAkZJMRAwDgYDVQQIEwdUdXVzdWxhMRkwFwYDVQQKExBNb250eSBQcm9ncmFt +IEFiMS4wLAYJKoZIhvcNAQkBFh9hYnN0cmFjdC5kZXZlbG9wZXJAYXNrbW9udHku +b3JnMB4XDTEwMDEyODIxNTcyNVoXDTEwMDIyNzIxNTcyNVowajELMAkGA1UEBhMC +RkkxEDAOBgNVBAgTB1R1dXN1bGExGTAXBgNVBAoTEE1vbnR5IFByb2dyYW0gQWIx +LjAsBgkqhkiG9w0BCQEWH2Fic3RyYWN0LmRldmVsb3BlckBhc2ttb250eS5vcmcw +gZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBAMIaY4pwbst50S32xJH3bSXhPGep +6gx1AWwZKsHTXL3VeMO6PHmC8zu5HM0zbOcrIJcXL3YVnpmE4b9OQxIiMSx1Yd+U +u8/sTkxgpsEKhCbIzECIwPhppyT/JP5aSXCadEvg+PSjikv8dOVkD68wVG4CcFIX +MFttsPebBVzEokZZAgMBAAGjgc8wgcwwHQYDVR0OBBYEFOCKaNHFFPrju8AwzWxS +f96IKfRwMIGcBgNVHSMEgZQwgZGAFOCKaNHFFPrju8AwzWxSf96IKfRwoW6kbDBq +MQswCQYDVQQGEwJGSTEQMA4GA1UECBMHVHV1c3VsYTEZMBcGA1UEChMQTW9udHkg +UHJvZ3JhbSBBYjEuMCwGCSqGSIb3DQEJARYfYWJzdHJhY3QuZGV2ZWxvcGVyQGFz +a21vbnR5Lm9yZ4IJAJhuvLP+2mGwMAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcNAQEF +BQADgYEAsmCX2/k9AInq2qhXtnkLip6cB0iOerLTNAzEijZc/aVf4wUjkL3cqhmC +kSTCwAHIOxp+ICwh6ky3xghXjoI9QnPFDVkRkzPT2tV0IoBaeQuI4e0CU2EY7L3P +XoDqp3oq1XtVcr9ZZdP68fBYUG/qcrWcXWk45ZFaBmBv3TotsGk= -----END CERTIFICATE----- === modified file 'mysql-test/std_data/client-cert.pem' --- a/mysql-test/std_data/client-cert.pem 2009-06-11 16:21:32 +0000 +++ b/mysql-test/std_data/client-cert.pem 2010-01-29 10:42:31 +0000 @@ -1,55 +1,60 @@ Certificate: Data: Version: 3 (0x2) - Serial Number: 3 (0x3) + Serial Number: 2 (0x2) Signature Algorithm: sha1WithRSAEncryption - Issuer: C=SE, ST=Uppsala, L=Uppsala, O=MySQL AB + Issuer: C=FI, ST=Tuusula, O=Monty Program Ab/emailAddress=abstract.developer(a)askmonty.org Validity - Not Before: Jan 28 11:04:39 2009 GMT - Not After : Jan 28 11:04:39 2010 GMT - Subject: C=SE, ST=Uppsala, O=MySQL AB/emailAddress=abstract.mysql.developer(a)mysql.com + Not Before: Jan 28 22:01:38 2010 GMT + Not After : Dec 7 22:01:38 2019 GMT + Subject: C=FI, ST=Tuusula, O=Monty Program Ab/emailAddress=abstract.developer(a)askmonty.org Subject Public Key Info: Public Key Algorithm: rsaEncryption - RSA Public Key: (512 bit) - Modulus (512 bit): - 00:e1:52:30:2c:d9:be:64:28:91:5d:7a:fd:d9:e9: - 14:35:7a:d2:94:4e:91:46:e0:db:9f:6b:79:f4:4c: - ac:6e:07:61:34:86:74:62:a7:a8:44:af:fa:87:87: - a8:7d:42:61:ff:ab:50:d4:7b:bf:75:fa:d5:d5:b3: - 74:fb:56:1e:37 + RSA Public Key: (1024 bit) + Modulus (1024 bit): + 00:bd:dc:3d:f8:3c:0b:d4:d2:c0:a3:9c:34:2d:e7: + 11:ff:4d:43:35:17:f6:0d:91:01:92:9e:4f:4d:c0: + 38:d5:62:03:55:33:db:66:a2:91:d3:f2:b6:23:34: + 95:53:50:3d:4f:e3:0c:d7:76:fd:f6:54:64:e6:f6: + dc:70:74:7c:6b:74:41:59:b0:19:5d:62:90:3c:a7: + c8:5e:21:8f:2b:22:6b:c7:43:9b:be:79:84:60:da: + 16:c9:ce:ee:fd:66:cb:54:81:e2:b5:1c:cf:f9:74: + de:38:2b:28:d4:31:33:55:d2:30:1c:a3:e4:c2:c7: + 31:46:43:d5:33:3d:8a:0b:47 Exponent: 65537 (0x10001) X509v3 extensions: X509v3 Basic Constraints: CA:FALSE + Netscape Comment: + OpenSSL Generated Certificate X509v3 Subject Key Identifier: - 58:30:B5:9B:2C:05:94:06:BA:3D:3C:F0:B2:CD:1D:67:65:E3:7F:85 + BE:E6:DB:19:8D:DB:72:9A:85:EE:B2:B8:5D:E7:FF:61:DF:09:08:AF X509v3 Authority Key Identifier: - keyid:F2:E2:EA:55:65:A4:9A:E2:AC:9D:97:F5:45:6C:F6:F7:8C:11:AD:DF - DirName:/C=SE/ST=Uppsala/L=Uppsala/O=MySQL AB - serial:95:E9:78:F5:34:50:E4:D5 + keyid:E0:8A:68:D1:C5:14:FA:E3:BB:C0:30:CD:6C:52:7F:DE:88:29:F4:70 Signature Algorithm: sha1WithRSAEncryption - 05:19:e3:13:14:fc:c5:28:bf:69:f8:00:b3:25:cb:bd:ca:9f: - 2f:4c:b3:a8:04:11:f0:74:27:bd:82:2c:b4:49:9b:a7:59:f0: - f7:87:d1:e0:ba:99:a2:fe:4b:1d:10:6f:e4:a2:b3:cd:7f:8b: - 68:31:46:ee:cd:9e:e2:47:e1:4c:fa:74:d1:e2:8b:cc:a0:4b: - a8:24:d1:a4:c3:6b:2a:c6:28:cd:41:e0:06:48:e6:cf:f2:3c: - ca:37:95:d7:29:64:6b:91:91:83:e7:ac:c8:0b:87:bc:da:a6: - aa:f1:44:43:c8:74:7b:15:26:91:2e:03:c4:71:50:6c:f8:68: - dc:8c + 41:95:6d:0a:a4:ee:af:68:cd:94:26:59:9a:18:b7:75:3c:c5: + 0f:22:d3:5c:31:9b:85:a0:93:b3:f0:50:29:ba:1e:d3:5a:43: + 0b:77:2d:98:87:a7:a7:39:0f:40:8d:03:d3:b3:67:43:77:bc: + 3c:51:c2:f9:9e:7a:2d:39:c4:5c:16:d7:70:d6:74:d1:6c:e1: + 6a:4d:fd:1f:10:af:64:3b:f4:64:e9:b2:b3:fb:c8:cd:c5:41: + cd:99:e0:ac:83:1d:81:2c:6b:99:ba:80:02:12:72:f7:3b:bb: + 93:72:00:da:ff:d3:87:75:d2:3a:a4:ca:4d:c1:8b:c1:21:50: + cb:57 -----BEGIN CERTIFICATE----- -MIICfzCCAeigAwIBAgIBAzANBgkqhkiG9w0BAQUFADBEMQswCQYDVQQGEwJTRTEQ -MA4GA1UECBMHVXBwc2FsYTEQMA4GA1UEBxMHVXBwc2FsYTERMA8GA1UEChMITXlT -UUwgQUIwHhcNMDkwMTI4MTEwNDM5WhcNMTAwMTI4MTEwNDM5WjBlMQswCQYDVQQG -EwJTRTEQMA4GA1UECBMHVXBwc2FsYTERMA8GA1UEChMITXlTUUwgQUIxMTAvBgkq -hkiG9w0BCQEWImFic3RyYWN0Lm15c3FsLmRldmVsb3BlckBteXNxbC5jb20wXDAN -BgkqhkiG9w0BAQEFAANLADBIAkEA4VIwLNm+ZCiRXXr92ekUNXrSlE6RRuDbn2t5 -9EysbgdhNIZ0YqeoRK/6h4eofUJh/6tQ1Hu/dfrV1bN0+1YeNwIDAQABo4GjMIGg -MAkGA1UdEwQCMAAwHQYDVR0OBBYEFFgwtZssBZQGuj088LLNHWdl43+FMHQGA1Ud -IwRtMGuAFPLi6lVlpJrirJ2X9UVs9veMEa3foUikRjBEMQswCQYDVQQGEwJTRTEQ -MA4GA1UECBMHVXBwc2FsYTEQMA4GA1UEBxMHVXBwc2FsYTERMA8GA1UEChMITXlT -UUwgQUKCCQCV6Xj1NFDk1TANBgkqhkiG9w0BAQUFAAOBgQAFGeMTFPzFKL9p+ACz -Jcu9yp8vTLOoBBHwdCe9giy0SZunWfD3h9Hgupmi/ksdEG/korPNf4toMUbuzZ7i -R+FM+nTR4ovMoEuoJNGkw2sqxijNQeAGSObP8jzKN5XXKWRrkZGD56zIC4e82qaq -8URDyHR7FSaRLgPEcVBs+GjcjA== +MIICxTCCAi6gAwIBAgIBAjANBgkqhkiG9w0BAQUFADBqMQswCQYDVQQGEwJGSTEQ +MA4GA1UECBMHVHV1c3VsYTEZMBcGA1UEChMQTW9udHkgUHJvZ3JhbSBBYjEuMCwG +CSqGSIb3DQEJARYfYWJzdHJhY3QuZGV2ZWxvcGVyQGFza21vbnR5Lm9yZzAeFw0x +MDAxMjgyMjAxMzhaFw0xOTEyMDcyMjAxMzhaMGoxCzAJBgNVBAYTAkZJMRAwDgYD +VQQIEwdUdXVzdWxhMRkwFwYDVQQKExBNb250eSBQcm9ncmFtIEFiMS4wLAYJKoZI +hvcNAQkBFh9hYnN0cmFjdC5kZXZlbG9wZXJAYXNrbW9udHkub3JnMIGfMA0GCSqG +SIb3DQEBAQUAA4GNADCBiQKBgQC93D34PAvU0sCjnDQt5xH/TUM1F/YNkQGSnk9N +wDjVYgNVM9tmopHT8rYjNJVTUD1P4wzXdv32VGTm9txwdHxrdEFZsBldYpA8p8he +IY8rImvHQ5u+eYRg2hbJzu79ZstUgeK1HM/5dN44KyjUMTNV0jAco+TCxzFGQ9Uz +PYoLRwIDAQABo3sweTAJBgNVHRMEAjAAMCwGCWCGSAGG+EIBDQQfFh1PcGVuU1NM +IEdlbmVyYXRlZCBDZXJ0aWZpY2F0ZTAdBgNVHQ4EFgQUvubbGY3bcpqF7rK4Xef/ +Yd8JCK8wHwYDVR0jBBgwFoAU4Ipo0cUU+uO7wDDNbFJ/3ogp9HAwDQYJKoZIhvcN +AQEFBQADgYEAQZVtCqTur2jNlCZZmhi3dTzFDyLTXDGbhaCTs/BQKboe01pDC3ct +mIenpzkPQI0D07NnQ3e8PFHC+Z56LTnEXBbXcNZ00Wzhak39HxCvZDv0ZOmys/vI +zcVBzZngrIMdgSxrmbqAAhJy9zu7k3IA2v/Th3XSOqTKTcGLwSFQy1c= -----END CERTIFICATE----- === modified file 'mysql-test/std_data/client-key.pem' --- a/mysql-test/std_data/client-key.pem 2009-06-11 16:21:32 +0000 +++ b/mysql-test/std_data/client-key.pem 2010-01-29 10:42:31 +0000 @@ -1,9 +1,15 @@ -----BEGIN RSA PRIVATE KEY----- -MIIBOQIBAAJBAOFSMCzZvmQokV16/dnpFDV60pROkUbg259refRMrG4HYTSGdGKn -qESv+oeHqH1CYf+rUNR7v3X61dWzdPtWHjcCAwEAAQJAXYooM8ZlcuEgj+VKU1ee -qyEFIMqJJxqcMk+E/nWCM96WxCP3zHNSrqNfSpI3ld7QzMwhdRz+gFLxT2gGNpIw -MQIhAPxzM/lDihe67X3ADYtDl9ZjA8Pm430x9sXlcxI17tCZAiEA5H1SyFl4mUee -9VnfSC2XGW7lwz72ZygfVX+b7tLWF08CIEh40gzW5MfXM+KLxdea+fXjyursV5ZT -R6KcMiKiNQLRAiAcmHqlzFzFgisotai2Fc6VRkXHG7gmzOSvBJt1VjmpDQIge6jf -2N7whTdvC4ferB+zUlgWQdyvx1c3T4gnt6PYdaY= +MIICXgIBAAKBgQC93D34PAvU0sCjnDQt5xH/TUM1F/YNkQGSnk9NwDjVYgNVM9tm +opHT8rYjNJVTUD1P4wzXdv32VGTm9txwdHxrdEFZsBldYpA8p8heIY8rImvHQ5u+ +eYRg2hbJzu79ZstUgeK1HM/5dN44KyjUMTNV0jAco+TCxzFGQ9UzPYoLRwIDAQAB +AoGBAJa2lprPT7UJ99Ho1aL6ota/RnKHKtNqII17DgjyZis9OtgP6kJ3GrvdF6iq +vT79my4nVrJTyxYXuGF/5U1/qqNjuPPBE1Xbu1ubQlFv8CT0kKYynQ7Z3ls8fAHC +B3VJXnUVlG+GHtUEFFG4FQVX1fn/Sga67ioJ6ivAiBlHKaPBAkEA5f2ToWlj4u9O +KgfRkN54wdIp4yu2c40pbhMfKGjGGsBAHk92+qSBpzEmxLcI6Ay+4/QysSR4jYmK +jCJuxiTu1QJBANNU3Hx8Il2SF/2BqGLcIh2SHxzKQIT5wAyD2jb+P2cHvbk6pKGR +VTmw5bibxXmYMS6J/L2zUF2xtFe+Svwz96sCQEnKYSqBqOWvyBFeLtPfPTlal8vm +Q4SxfuBtTCrn6t+8XRYcgt0KGPsunvSwkS/6nuh+eiExxWgMACLUDVyPjv0CQQC4 +sJJc7LOv6Oy0bWr2swHRrBEqvQsz63zOszCzHPHWHirNxZV5aiT8XT/2XZRwlvRs +gsVyGFLk/1fn0vN/g/8vAkEAxUdzUKvC1ZwjzGhgcz2bQU0tEZN4C9jBCiwOI2ud +BpAsPG0xAGGL2+hz0B0n88XiTHobiTZ1bg4Z41i4pXx2ZA== -----END RSA PRIVATE KEY----- === modified file 'mysql-test/std_data/server-cert.pem' --- a/mysql-test/std_data/server-cert.pem 2009-06-11 16:21:32 +0000 +++ b/mysql-test/std_data/server-cert.pem 2010-01-29 10:42:31 +0000 @@ -3,53 +3,59 @@ Certificate: Version: 3 (0x2) Serial Number: 1 (0x1) Signature Algorithm: sha1WithRSAEncryption - Issuer: C=SE, ST=Uppsala, L=Uppsala, O=MySQL AB + Issuer: C=FI, ST=Tuusula, O=Monty Program Ab/emailAddress=abstract.developer(a)askmonty.org Validity - Not Before: Jan 28 10:55:13 2009 GMT - Not After : Jan 28 10:55:13 2010 GMT - Subject: C=SE, ST=Uppsala, O=MySQL AB, CN=localhost/emailAddress=abstract.mysql.developer(a)mysql.com + Not Before: Jan 28 21:59:14 2010 GMT + Not After : Dec 7 21:59:14 2019 GMT + Subject: C=FI, ST=Tuusula, O=Monty Program Ab, CN=localhost/emailAddress=abstract.developer(a)askmonty.org Subject Public Key Info: Public Key Algorithm: rsaEncryption - RSA Public Key: (512 bit) - Modulus (512 bit): - 00:b6:8f:e5:b7:b4:86:83:13:8a:f9:bf:63:cb:64: - 2d:b9:51:d1:de:ab:7b:45:1f:aa:b5:66:73:13:f9: - a6:07:d5:ba:7c:fa:92:bd:37:e2:ad:87:db:3e:b6: - 6a:12:64:f8:ee:17:e3:15:06:2f:a8:82:68:bf:57: - 8d:c3:04:98:27 + RSA Public Key: (1024 bit) + Modulus (1024 bit): + 00:cc:79:74:2a:14:7e:77:06:b0:ec:1b:b6:da:70: + 4c:4f:0e:94:04:8a:e7:69:f4:4c:9c:67:22:41:56: + 3a:96:53:9e:95:9d:17:55:80:41:c0:13:d6:94:0f: + cd:2c:51:fe:a4:6e:f2:74:d8:5d:3b:3a:80:e3:85: + 5b:a5:bc:7d:5c:25:55:e5:40:77:fe:f3:cb:5b:cd: + da:a5:f4:36:64:30:a2:a0:7f:93:b3:c4:56:75:2a: + c0:f7:65:2a:d7:e6:ff:88:25:03:e0:b4:40:2e:74: + 4c:cb:46:95:73:cb:25:5c:87:0e:ec:0f:5a:19:c2: + b6:dc:9e:e8:f0:30:b1:9c:99 Exponent: 65537 (0x10001) X509v3 extensions: X509v3 Basic Constraints: CA:FALSE + Netscape Comment: + OpenSSL Generated Certificate X509v3 Subject Key Identifier: - D9:9A:B8:5F:22:EA:04:10:C8:25:7D:82:57:E6:2E:FD:19:29:E7:DA + 6D:13:3B:40:52:3C:AF:18:EA:33:D1:B7:56:21:1B:05:FE:0B:9E:38 X509v3 Authority Key Identifier: - keyid:F2:E2:EA:55:65:A4:9A:E2:AC:9D:97:F5:45:6C:F6:F7:8C:11:AD:DF - DirName:/C=SE/ST=Uppsala/L=Uppsala/O=MySQL AB - serial:95:E9:78:F5:34:50:E4:D5 + keyid:E0:8A:68:D1:C5:14:FA:E3:BB:C0:30:CD:6C:52:7F:DE:88:29:F4:70 Signature Algorithm: sha1WithRSAEncryption - 54:07:2d:21:0b:a5:af:3b:58:23:32:5e:56:7f:ab:58:63:48: - 91:aa:38:90:89:16:f9:cc:bf:a4:0e:78:2b:9f:c5:1b:58:a6: - e6:08:8f:2e:ae:97:03:21:9b:f1:cd:c0:26:8f:1d:d7:28:27: - a0:8e:81:09:1b:1c:0f:c9:a5:41:3a:2d:44:3f:9c:fa:87:ff: - c8:4c:2b:44:f7:1b:c1:3e:4f:01:7f:e9:26:cc:9f:1c:06:b5: - 0b:27:d1:10:90:be:93:0c:9c:e7:b0:d1:ea:27:99:4e:06:14: - 0c:7a:e9:c1:52:c5:33:68:bc:61:0d:db:81:3b:57:48:57:bf: - 42:9a + 97:db:65:23:7f:f1:15:3c:1e:83:ac:0e:0a:50:a0:0c:22:b8: + 45:d4:ca:21:05:47:3b:3d:03:b5:6c:4b:8d:bb:5f:57:c3:c7: + 4e:71:23:cf:33:a3:7f:a0:3d:bd:58:75:b8:37:22:16:2f:e9: + ed:ae:9b:94:29:81:6e:34:79:cf:41:bd:3d:8d:17:d7:22:1c: + 1b:58:c7:0f:79:13:56:1d:e8:d8:4e:e5:07:3f:79:1b:dd:c4: + 06:9b:c5:b6:02:34:43:c5:bf:e5:87:ad:f1:c1:8a:f2:be:c2: + 00:1d:d4:27:1f:87:c8:80:31:ec:6e:97:95:b4:84:40:d1:73: + 42:71 -----BEGIN CERTIFICATE----- -MIICkzCCAfygAwIBAgIBATANBgkqhkiG9w0BAQUFADBEMQswCQYDVQQGEwJTRTEQ -MA4GA1UECBMHVXBwc2FsYTEQMA4GA1UEBxMHVXBwc2FsYTERMA8GA1UEChMITXlT -UUwgQUIwHhcNMDkwMTI4MTA1NTEzWhcNMTAwMTI4MTA1NTEzWjB5MQswCQYDVQQG -EwJTRTEQMA4GA1UECBMHVXBwc2FsYTERMA8GA1UEChMITXlTUUwgQUIxEjAQBgNV -BAMTCWxvY2FsaG9zdDExMC8GCSqGSIb3DQEJARYiYWJzdHJhY3QubXlzcWwuZGV2 -ZWxvcGVyQG15c3FsLmNvbTBcMA0GCSqGSIb3DQEBAQUAA0sAMEgCQQC2j+W3tIaD -E4r5v2PLZC25UdHeq3tFH6q1ZnMT+aYH1bp8+pK9N+Kth9s+tmoSZPjuF+MVBi+o -gmi/V43DBJgnAgMBAAGjgaMwgaAwCQYDVR0TBAIwADAdBgNVHQ4EFgQU2Zq4XyLq -BBDIJX2CV+Yu/Rkp59owdAYDVR0jBG0wa4AU8uLqVWWkmuKsnZf1RWz294wRrd+h -SKRGMEQxCzAJBgNVBAYTAlNFMRAwDgYDVQQIEwdVcHBzYWxhMRAwDgYDVQQHEwdV -cHBzYWxhMREwDwYDVQQKEwhNeVNRTCBBQoIJAJXpePU0UOTVMA0GCSqGSIb3DQEB -BQUAA4GBAFQHLSELpa87WCMyXlZ/q1hjSJGqOJCJFvnMv6QOeCufxRtYpuYIjy6u -lwMhm/HNwCaPHdcoJ6COgQkbHA/JpUE6LUQ/nPqH/8hMK0T3G8E+TwF/6SbMnxwG -tQsn0RCQvpMMnOew0eonmU4GFAx66cFSxTNovGEN24E7V0hXv0Ka +MIIC2TCCAkKgAwIBAgIBATANBgkqhkiG9w0BAQUFADBqMQswCQYDVQQGEwJGSTEQ +MA4GA1UECBMHVHV1c3VsYTEZMBcGA1UEChMQTW9udHkgUHJvZ3JhbSBBYjEuMCwG +CSqGSIb3DQEJARYfYWJzdHJhY3QuZGV2ZWxvcGVyQGFza21vbnR5Lm9yZzAeFw0x +MDAxMjgyMTU5MTRaFw0xOTEyMDcyMTU5MTRaMH4xCzAJBgNVBAYTAkZJMRAwDgYD +VQQIEwdUdXVzdWxhMRkwFwYDVQQKExBNb250eSBQcm9ncmFtIEFiMRIwEAYDVQQD +Ewlsb2NhbGhvc3QxLjAsBgkqhkiG9w0BCQEWH2Fic3RyYWN0LmRldmVsb3BlckBh +c2ttb250eS5vcmcwgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBAMx5dCoUfncG +sOwbttpwTE8OlASK52n0TJxnIkFWOpZTnpWdF1WAQcAT1pQPzSxR/qRu8nTYXTs6 +gOOFW6W8fVwlVeVAd/7zy1vN2qX0NmQwoqB/k7PEVnUqwPdlKtfm/4glA+C0QC50 +TMtGlXPLJVyHDuwPWhnCttye6PAwsZyZAgMBAAGjezB5MAkGA1UdEwQCMAAwLAYJ +YIZIAYb4QgENBB8WHU9wZW5TU0wgR2VuZXJhdGVkIENlcnRpZmljYXRlMB0GA1Ud +DgQWBBRtEztAUjyvGOoz0bdWIRsF/gueODAfBgNVHSMEGDAWgBTgimjRxRT647vA +MM1sUn/eiCn0cDANBgkqhkiG9w0BAQUFAAOBgQCX22Ujf/EVPB6DrA4KUKAMIrhF +1MohBUc7PQO1bEuNu19Xw8dOcSPPM6N/oD29WHW4NyIWL+ntrpuUKYFuNHnPQb09 +jRfXIhwbWMcPeRNWHejYTuUHP3kb3cQGm8W2AjRDxb/lh63xwYryvsIAHdQnH4fI +gDHsbpeVtIRA0XNCcQ== -----END CERTIFICATE----- === modified file 'mysql-test/std_data/server-key.pem' --- a/mysql-test/std_data/server-key.pem 2009-06-11 16:21:32 +0000 +++ b/mysql-test/std_data/server-key.pem 2010-01-29 10:42:31 +0000 @@ -1,9 +1,15 @@ -----BEGIN RSA PRIVATE KEY----- -MIIBOQIBAAJBALaP5be0hoMTivm/Y8tkLblR0d6re0UfqrVmcxP5pgfVunz6kr03 -4q2H2z62ahJk+O4X4xUGL6iCaL9XjcMEmCcCAwEAAQJASA5VwgNb0CKHiPm0ntOk -hG+54SRX3DmafEy6gRjZIl/bZ/asSLhXUZ+CeohyrQh7BZgYWvykd8pRISL9eKsU -GQIhAOXkUrOtP/EtjyqNluEqZdG+RZi/7p61JS3Ce13Myu+LAiEAy0uMlV34AJpM -b40FPKqlHxw8DD/Dt1iKhNVAg8+LDVUCIFjv7fbJDbW2VG63/Cj8CAwOukoP5rbL -iaicVrHBKrllAiB9+MiaXeopZXNrxDS0jQFYr8Q9yt1aJVFgUkxx4Q9HKQIgZPs0 -KlF3NNNWw78INaAEkyf0IEssnLMsuoCWw0DIOak= +MIICXgIBAAKBgQDMeXQqFH53BrDsG7bacExPDpQEiudp9EycZyJBVjqWU56VnRdV +gEHAE9aUD80sUf6kbvJ02F07OoDjhVulvH1cJVXlQHf+88tbzdql9DZkMKKgf5Oz +xFZ1KsD3ZSrX5v+IJQPgtEAudEzLRpVzyyVchw7sD1oZwrbcnujwMLGcmQIDAQAB +AoGBAMdMYkNZsmJFbVDVOobzCg3Mgc1jrmeBrOKNS8AvUe+QFXRyp3m5B102eOHb +/PmD+hU/5qao9UZzoYkiRM/oRq45jrqJEYwWrX007bKK0F9hnErtC1ImM1nBFVhx +6+6cr+ShUkvtj8+wJ2d5bIccUzGCUfFR5tb5BnePTXK8IVoBAkEA7WGNxHAVKgjS +AzlpHr5fvpivA07hNVJizTwZdWGGYeETilZhkkuMRwREceeohF6ILMf0FTZdFSa/ +8EeLa3icIQJBANyDKFjynKwWy5pyRSz75mVwrEi+4eTQPsCPNWLkbpbEPwqPLYWJ +2VSFkISXF7b7Od48JkQWgiB8/kXqMDEdsXkCQQCzZvj3ryWvoP7nhOoXXBWMPGR4 +gZLe86bMKVGsTsp7CtnzwRj4sbQQr/7yfvvzHmaYQX4M0gtDQwfolomd7YdBAkEA +y24ETuqjNu9grf81aiaJipPDnOjcJOcovSRgr/blPxmUvv0Pld5yLNN7W5a4PgrO +fAMpmi7ZpXcqbP17sBQgoQJAWTDFKAmfHPVdDGZuCw4yceP5d+Tv7ABglZUvpPKx +kAvGN1WBASUuCQJDOIgzl6gvYX07S5p147i9mv7UBWOpvw== -----END RSA PRIVATE KEY----- === modified file 'mysql-test/t/grant.test' --- a/mysql-test/t/grant.test 2009-06-15 15:53:45 +0000 +++ b/mysql-test/t/grant.test 2010-01-29 10:42:31 +0000 @@ -35,7 +35,7 @@ revoke delete on mysqltest.* from mysqlt show grants for mysqltest_1@localhost; grant select on mysqltest.* to mysqltest_1@localhost require NONE; show grants for mysqltest_1@localhost; -grant USAGE on mysqltest.* to mysqltest_1@localhost require cipher "EDH-RSA-DES-CBC3-SHA" AND SUBJECT "testsubject" ISSUER "MySQL AB"; +grant USAGE on mysqltest.* to mysqltest_1@localhost require cipher "EDH-RSA-DES-CBC3-SHA" AND SUBJECT "testsubject" ISSUER "Monty Program Ab"; show grants for mysqltest_1@localhost; revoke all privileges on mysqltest.* from mysqltest_1@localhost; show grants for mysqltest_1@localhost; === modified file 'mysql-test/t/openssl_1.test' --- a/mysql-test/t/openssl_1.test 2009-05-25 13:00:18 +0000 +++ b/mysql-test/t/openssl_1.test 2010-01-29 10:42:31 +0000 @@ -15,8 +15,10 @@ insert into t1 values (5); grant select on test.* to ssl_user1@localhost require SSL; grant select on test.* to ssl_user2@localhost require cipher "DHE-RSA-AES256-SHA"; -grant select on test.* to ssl_user3@localhost require cipher "DHE-RSA-AES256-SHA" AND SUBJECT "/C=SE/ST=Uppsala/O=MySQL AB/emailAddress=abstract.mysql.developer(a)mysql.com"; -grant select on test.* to ssl_user4@localhost require cipher "DHE-RSA-AES256-SHA" AND SUBJECT "/C=SE/ST=Uppsala/O=MySQL AB/emailAddress=abstract.mysql.developer(a)mysql.com" ISSUER "/C=SE/ST=Uppsala/L=Uppsala/O=MySQL AB"; +grant select on test.* to ssl_user3@localhost require cipher +"DHE-RSA-AES256-SHA" AND SUBJECT "/C=FI/ST=Tuusula/O=Monty Program Ab/emailAddress=abstract.developer(a)askmonty.org"; +grant select on test.* to ssl_user4@localhost require cipher +"DHE-RSA-AES256-SHA" AND SUBJECT "/C=FI/ST=Tuusula/O=Monty Program Ab/emailAddress=abstract.developer(a)askmonty.org" ISSUER "/C=FI/ST=Tuusula/O=Monty Program Ab/emailAddress=abstract.developer(a)askmonty.org"; grant select on test.* to ssl_user5@localhost require cipher "DHE-RSA-AES256-SHA" AND SUBJECT "xxx"; flush privileges; === modified file 'mysql-test/t/partition_innodb_semi_consistent.test' --- a/mysql-test/t/partition_innodb_semi_consistent.test 2009-01-13 22:12:16 +0000 +++ b/mysql-test/t/partition_innodb_semi_consistent.test 2010-01-28 14:49:14 +0000 @@ -187,6 +187,7 @@ SELECT * FROM t1; --echo # Switch to connection con2 connection con2; +--reap SELECT * FROM t1; connection default; === modified file 'mysql-test/t/query_cache_debug.test' --- a/mysql-test/t/query_cache_debug.test 2009-09-07 20:50:10 +0000 +++ b/mysql-test/t/query_cache_debug.test 2010-01-29 10:42:31 +0000 @@ -1,5 +1,4 @@ --source include/not_embedded.inc ---source include/big_test.inc --source include/have_query_cache.inc --source include/have_debug.inc === modified file 'sql-common/client.c' --- a/sql-common/client.c 2009-12-03 15:26:54 +0000 +++ b/sql-common/client.c 2010-01-29 10:42:31 +0000 @@ -1863,6 +1863,7 @@ CLI_MYSQL_REAL_CONNECT(MYSQL *mysql,cons uint port, const char *unix_socket,ulong client_flag) { char buff[NAME_LEN+USERNAME_LENGTH+100]; + char error_string[1024]; char *end,*host_info= NULL; my_socket sock; in_addr_t ip_addr; @@ -2331,9 +2332,14 @@ CLI_MYSQL_REAL_CONNECT(MYSQL *mysql,cons /* Connect to the server */ DBUG_PRINT("info", ("IO layer change in progress...")); if (sslconnect(ssl_fd, mysql->net.vio, - (long) (mysql->options.connect_timeout))) + (long) (mysql->options.connect_timeout), + error_string)) { - set_mysql_error(mysql, CR_SSL_CONNECTION_ERROR, unknown_sqlstate); + set_mysql_extended_error(mysql, CR_SSL_CONNECTION_ERROR, + unknown_sqlstate, + "SSL error: %s", + error_string[0] ? error_string : + ER(CR_SSL_CONNECTION_ERROR)); goto error; } DBUG_PRINT("info", ("IO layer change done!")); === modified file 'sql/net_serv.cc' --- a/sql/net_serv.cc 2009-09-07 20:50:10 +0000 +++ b/sql/net_serv.cc 2010-01-29 10:42:31 +0000 @@ -219,7 +219,7 @@ my_bool net_realloc(NET *net, size_t len -1 Don't know if data is ready or not */ -#if !defined(EMBEDDED_LIBRARY) +#if !defined(EMBEDDED_LIBRARY) && defined(DBUG_OFF) static int net_data_is_ready(my_socket sd) { @@ -280,13 +280,18 @@ static int net_data_is_ready(my_socket s void net_clear(NET *net, my_bool clear_buffer) { -#if !defined(EMBEDDED_LIBRARY) +#if !defined(EMBEDDED_LIBRARY) && defined(DBUG_OFF) size_t count; int ready; #endif DBUG_ENTER("net_clear"); -#if !defined(EMBEDDED_LIBRARY) +/* + We don't do a clear in case of DBUG_OFF to catch bugs + in the protocol handling +*/ + +#if !defined(EMBEDDED_LIBRARY) && defined(DBUG_OFF) if (clear_buffer) { while ((ready= net_data_is_ready(net->vio->sd)) > 0) === modified file 'sql/slave.cc' --- a/sql/slave.cc 2009-12-06 17:51:48 +0000 +++ b/sql/slave.cc 2010-01-29 10:42:31 +0000 @@ -3803,10 +3803,11 @@ static int connect_to_master(THD* thd, M suppress_warnings= 0; mi->report(ERROR_LEVEL, last_errno, "error %s to master '%s@%s:%d'" - " - retry-time: %d retries: %lu", + " - retry-time: %d retries: %lu message: %s", (reconnect ? "reconnecting" : "connecting"), mi->user, mi->host, mi->port, - mi->connect_retry, master_retry_count); + mi->connect_retry, master_retry_count, + mysql_error(mysql)); } /* By default we try forever. The reason is that failure will trigger === modified file 'sql/sql_cache.cc' --- a/sql/sql_cache.cc 2010-01-15 15:27:55 +0000 +++ b/sql/sql_cache.cc 2010-01-29 10:42:31 +0000 @@ -377,9 +377,11 @@ TODO list: static void debug_wait_for_kill(const char *info) { - DBUG_ENTER("debug_wait_for_kill"); const char *prev_info; THD *thd; + char buff[1024]; + DBUG_ENTER("debug_wait_for_kill"); + thd= current_thd; prev_info= thd->proc_info; thd->proc_info= info; @@ -387,8 +389,16 @@ static void debug_wait_for_kill(const ch while(!thd->killed) my_sleep(1000); thd->killed= THD::NOT_KILLED; + /* + Remove the set debug variable, to ensure we don't get stuck on it again + This is needed as for MyISAM, invalidate_table() may be called twice + (Once from mysql_delete() and once from mi_update_status()) + */ + sprintf(buff, "-d,%s", info); + DBUG_SET(buff); sql_print_information("Exit debug_wait_for_kill"); thd->proc_info= prev_info; + DBUG_VOID_RETURN; } @@ -914,15 +924,18 @@ void query_cache_insert(NET *net, const void query_cache_abort(NET *net) { + THD *thd; DBUG_ENTER("query_cache_abort"); - THD *thd= current_thd; /* See the comment on double-check locking usage above. */ if (net->query_cache_query == 0) DBUG_VOID_RETURN; if (query_cache.try_lock()) + { + net->query_cache_query = 0; DBUG_VOID_RETURN; + } /* While we were waiting another thread might have changed the status @@ -932,6 +945,7 @@ void query_cache_abort(NET *net) net->query_cache_query); if (query_block) { + thd= current_thd; thd_proc_info(thd, "storing result in query cache"); DUMP(&query_cache); BLOCK_LOCK_WR(query_block); @@ -941,6 +955,7 @@ void query_cache_abort(NET *net) DBUG_EXECUTE("check_querycache",query_cache.check_integrity(1);); } + DBUG_ASSERT(!net->query_cache_query); query_cache.unlock(); DBUG_VOID_RETURN; } @@ -970,8 +985,12 @@ void query_cache_end_of_result(THD *thd) #endif if (query_cache.try_lock()) + { + thd->net.query_cache_query= 0; DBUG_VOID_RETURN; + } + /* thd->net.query_cache_query may have changed during resize */ query_block= ((Query_cache_block*) thd->net.query_cache_query); if (query_block) { @@ -997,8 +1016,8 @@ void query_cache_end_of_result(THD *thd) to this function. In the release version that query should be ignored and removed from QC. */ - DBUG_ASSERT(0); query_cache.free_query(query_block); + thd->net.query_cache_query= 0; query_cache.unlock(); DBUG_VOID_RETURN; } === modified file 'sql/sql_connect.cc' --- a/sql/sql_connect.cc 2009-12-03 11:19:05 +0000 +++ b/sql/sql_connect.cc 2010-01-29 10:42:31 +0000 @@ -798,6 +798,7 @@ static int check_connection(THD *thd) DBUG_PRINT("info", ("client capabilities: %lu", thd->client_capabilities)); if (thd->client_capabilities & CLIENT_SSL) { + char error_string[1024]; /* Do the SSL layering. */ if (!ssl_acceptor_fd) { @@ -806,7 +807,7 @@ static int check_connection(THD *thd) return 1; } DBUG_PRINT("info", ("IO layer change in progress...")); - if (sslaccept(ssl_acceptor_fd, net->vio, net->read_timeout)) + if (sslaccept(ssl_acceptor_fd, net->vio, net->read_timeout, error_string)) { DBUG_PRINT("error", ("Failed to accept new SSL connection")); inc_host_errors(&thd->remote.sin_addr); === modified file 'sql/sql_parse.cc' --- a/sql/sql_parse.cc 2010-01-15 15:27:55 +0000 +++ b/sql/sql_parse.cc 2010-01-29 10:42:31 +0000 @@ -1252,12 +1252,14 @@ bool dispatch_command(enum enum_server_c ha_maria::implicit_commit(thd, FALSE); #endif - net_end_statement(thd); - query_cache_end_of_result(thd); /* Multiple queries exits, execute them individually */ close_thread_tables(thd); + + net_end_statement(thd); + query_cache_end_of_result(thd); + ulong length= (ulong)(packet_end - beginning_of_next_stmt); log_slow_statement(thd); @@ -1645,13 +1647,22 @@ bool dispatch_command(enum enum_server_c ha_maria::implicit_commit(thd, FALSE); #endif - net_end_statement(thd); - query_cache_end_of_result(thd); - + if (!(sql_command_flags[thd->lex->sql_command] & CF_CHANGES_DATA)) + { + /* No changes in data; We can send ok at once to the client */ + net_end_statement(thd); + query_cache_end_of_result(thd); + } thd->proc_info= "closing tables"; /* Free tables */ close_thread_tables(thd); + if (sql_command_flags[thd->lex->sql_command] & CF_CHANGES_DATA) + { + net_end_statement(thd); + query_cache_end_of_result(thd); + } + log_slow_statement(thd); thd_proc_info(thd, "cleaning up"); === modified file 'storage/myisam/ft_boolean_search.c' --- a/storage/myisam/ft_boolean_search.c 2010-01-27 21:53:08 +0000 +++ b/storage/myisam/ft_boolean_search.c 2010-01-28 14:49:14 +0000 @@ -321,7 +321,7 @@ static int _ftb_parse_query(FTB *ftb, uc param->mysql_add_word= ftb_query_add_word; param->mysql_ftparam= (void *)&ftb_param; param->cs= ftb->charset; - param->doc= (char*) query; + param->doc= query; param->length= len; param->flags= 0; param->mode= MYSQL_FTPARSER_FULL_BOOLEAN_INFO; @@ -702,7 +702,7 @@ static int _ftb_check_phrase(FTB *ftb, c param->mysql_add_word= ftb_phrase_add_word; param->mysql_ftparam= (void *)&ftb_param; param->cs= ftb->charset; - param->doc= (char *) document; + param->doc= document; param->length= len; param->flags= 0; param->mode= MYSQL_FTPARSER_WITH_STOPWORDS; @@ -1000,7 +1000,7 @@ float ft_boolean_find_relevance(FT_INFO { if (!ftsi.pos) continue; - param->doc= (char *)ftsi.pos; + param->doc= ftsi.pos; param->length= ftsi.len; if (unlikely(parser->parse(param))) return 0; === modified file 'storage/myisam/ft_parser.c' --- a/storage/myisam/ft_parser.c 2010-01-27 21:53:08 +0000 +++ b/storage/myisam/ft_parser.c 2010-01-28 14:49:14 +0000 @@ -175,7 +175,7 @@ uchar ft_get_word(CHARSET_INFO *cs, cons if ((param->trunc=(doc<end && *doc == FTB_TRUNC))) doc++; - if (((length >= ft_min_word_len && !is_stopword((char*) word->pos, + if (((length >= ft_min_word_len && !is_stopword(word->pos, word->len)) || param->trunc) && length < ft_max_word_len) { @@ -298,7 +298,7 @@ static int ft_parse_internal(MYSQL_FTPAR DBUG_ENTER("ft_parse_internal"); while (ft_simple_get_word(wtree->custom_arg, &doc, end, &w, TRUE)) - if (param->mysql_add_word(param, (char*) w.pos, w.len, 0)) + if (param->mysql_add_word(param, w.pos, w.len, 0)) DBUG_RETURN(1); DBUG_RETURN(0); } @@ -319,7 +319,7 @@ int ft_parse(TREE *wtree, const uchar *d param->mysql_add_word= ft_add_word; param->mysql_ftparam= &my_param; param->cs= wtree->custom_arg; - param->doc= (char*) doc; + param->doc= doc; param->length= doclen; param->mode= MYSQL_FTPARSER_SIMPLE_MODE; DBUG_RETURN(parser->parse(param)); === modified file 'storage/myisam/ft_stopwords.c' --- a/storage/myisam/ft_stopwords.c 2010-01-27 21:53:08 +0000 +++ b/storage/myisam/ft_stopwords.c 2010-01-28 14:49:14 +0000 @@ -20,7 +20,7 @@ typedef struct st_ft_stopwords { - const char * pos; + const uchar* pos; size_t len; } FT_STOPWORD; @@ -30,15 +30,15 @@ static int FT_STOPWORD_cmp(void* cmp_arg FT_STOPWORD *w1, FT_STOPWORD *w2) { return ha_compare_text(default_charset_info, - (uchar *)w1->pos,w1->len, - (uchar *)w2->pos,w2->len,0,0); + w1->pos, w1->len, + w2->pos, w2->len, 0, 0); } static void FT_STOPWORD_free(FT_STOPWORD *w, TREE_FREE action, void *arg __attribute__((unused))) { if (action == free_free) - my_free((uchar*) w->pos, MYF(0)); + my_free((void*) w->pos, MYF(0)); } static int ft_add_stopword(const char *w) === modified file 'support-files/compiler_warnings.supp' --- a/support-files/compiler_warnings.supp 2010-01-28 11:35:10 +0000 +++ b/support-files/compiler_warnings.supp 2010-01-28 14:49:14 +0000 @@ -4,6 +4,7 @@ # integer.cpp: .*control reaches end of non-void function.*: 1288-1427 +integer.cpp: .*no return statement in function returning non-void.*: 1288-1427 DictTabInfo.cpp : .*invalid access to non-static.* DictTabInfo.cpp : .*macro was used incorrectly.* DbdihMain.cpp : .*unused variable.* : 6666-7013 @@ -27,13 +28,17 @@ _flex_tmp.c: .*not enough actual paramet pars0lex.l: .*conversion from 'ulint' to 'int', possible loss of data.* btr/btr0cur\.c: .*value computed is not used.*: 3175-3375 include/buf0buf\.ic: unused parameter ��mtr�� +fil/fil0fil\.c: pointer targets in passing argument.*differ in signedness fil/fil0fil\.c: comparison between signed and unsigned : 3100-3199 fil/fil0fil\.c: unused parameter log/log0recv\.c: unused variable os/os0file\.c: unused parameter +os/os0file\.c: pointer targets in assignment differ in signedness handler/i_s\.cc: unused variable sync/sync0rw\.c: unused parameter sync/sync0sync\.c: unused parameter +sync/sync0sync\.c: unused variable +ut/ut0ut\.c: ignoring return value of # # bdb is not critical to keep up to date @@ -98,10 +103,15 @@ storage/maria/ma_pagecache.c: .*'info_ch # I think these are due to mix of C and C++. # storage/pbxt/ : typedef.*was ignored in this declaration +ha_pbxt\.cc : variable.*might be clobbered by.*longjmp # # Yassl include/runtime.hpp: .*pure_error.* +.*/extra/yassl/taocrypt/.*: comparison with string literal +.*/extra/yassl/taocrypt/src/blowfish\.cpp: array subscript is above array bounds +.*/extra/yassl/taocrypt/src/file\.cpp: ignoring return value +.*/extra/yassl/taocrypt/src/integer\.cpp: control reaches end of non-void function # # Groff warnings on OpenSUSE. === modified file 'unittest/mysys/waiting_threads-t.c' --- a/unittest/mysys/waiting_threads-t.c 2009-09-15 10:46:35 +0000 +++ b/unittest/mysys/waiting_threads-t.c 2010-01-28 14:49:14 +0000 @@ -258,7 +258,7 @@ void do_tests() #define test_kill_strategy(X) \ diag("kill strategy: " #X); \ DBUG_EXECUTE("reset_file", \ - { rewind(DBUG_FILE); ftruncate(fileno(DBUG_FILE), 0); }); \ + { rewind(DBUG_FILE); (void) ftruncate(fileno(DBUG_FILE), 0); }); \ DBUG_PRINT("info", ("kill strategy: " #X)); \ kill_strategy=X; \ do_one_test(); === modified file 'vio/viossl.c' --- a/vio/viossl.c 2008-02-07 08:55:49 +0000 +++ b/vio/viossl.c 2010-01-29 10:42:31 +0000 @@ -74,8 +74,11 @@ report_errors(SSL* ssl) } if (ssl) - DBUG_PRINT("error", ("error: %s", - ERR_error_string(SSL_get_error(ssl, l), buf))); + { + int error= SSL_get_error(ssl, l); + DBUG_PRINT("error", ("error: %s (%d)", + ERR_error_string(error, buf), error)); + } DBUG_PRINT("info", ("socket_errno: %d", socket_errno)); DBUG_VOID_RETURN; @@ -174,16 +177,17 @@ void vio_ssl_delete(Vio *vio) static int ssl_do(struct st_VioSSLFd *ptr, Vio *vio, long timeout, - int (*connect_accept_func)(SSL*)) + int (*connect_accept_func)(SSL*), char *error_string) { SSL *ssl; my_bool unused; my_bool was_blocking; - DBUG_ENTER("ssl_do"); DBUG_PRINT("enter", ("ptr: 0x%lx, sd: %d ctx: 0x%lx", (long) ptr, vio->sd, (long) ptr->ssl_context)); + error_string[0]= 0; + /* Set socket to blocking if not already set */ vio_blocking(vio, 1, &was_blocking); @@ -191,6 +195,7 @@ static int ssl_do(struct st_VioSSLFd *pt { DBUG_PRINT("error", ("SSL_new failure")); report_errors(ssl); + strmov(error_string, "SSL_new failed"); vio_blocking(vio, was_blocking, &unused); DBUG_RETURN(1); } @@ -203,6 +208,7 @@ static int ssl_do(struct st_VioSSLFd *pt { DBUG_PRINT("error", ("SSL_connect/accept failure")); report_errors(ssl); + ERR_error_string(SSL_get_error(ssl, 0), error_string); SSL_free(ssl); vio_blocking(vio, was_blocking, &unused); DBUG_RETURN(1); @@ -250,17 +256,19 @@ static int ssl_do(struct st_VioSSLFd *pt } -int sslaccept(struct st_VioSSLFd *ptr, Vio *vio, long timeout) +int sslaccept(struct st_VioSSLFd *ptr, Vio *vio, long timeout, + char *error_string) { DBUG_ENTER("sslaccept"); - DBUG_RETURN(ssl_do(ptr, vio, timeout, SSL_accept)); + DBUG_RETURN(ssl_do(ptr, vio, timeout, SSL_accept, error_string)); } -int sslconnect(struct st_VioSSLFd *ptr, Vio *vio, long timeout) +int sslconnect(struct st_VioSSLFd *ptr, Vio *vio, long timeout, + char *error_string) { DBUG_ENTER("sslconnect"); - DBUG_RETURN(ssl_do(ptr, vio, timeout, SSL_connect)); + DBUG_RETURN(ssl_do(ptr, vio, timeout, SSL_connect, error_string)); }

1 0

[Maria-developers] [Branch ~maria-captains/maria/5.1] Rev 2807: Auto merge
by noreply＠launchpad.net 29 Jan '10

29 Jan '10

Merge authors: Matthew Montgomery <mmontgomery(a)mysql.com> Michael Widenius (monty) ------------------------------------------------------------ revno: 2807 [merge] committer: Michael Widenius <monty(a)askmonty.org> branch nick: maria-5.1 timestamp: Fri 2010-01-29 12:52:02 +0200 message: Auto merge modified: client/mysqltest.cc configure.in extra/yassl/src/yassl_error.cpp include/violite.h mysql-test/mysql-test-run.pl mysql-test/r/grant.result mysql-test/r/openssl_1.result mysql-test/std_data/cacert.pem mysql-test/std_data/client-cert.pem mysql-test/std_data/client-key.pem mysql-test/std_data/server-cert.pem mysql-test/std_data/server-key.pem mysql-test/t/grant.test mysql-test/t/openssl_1.test mysql-test/t/partition_innodb_semi_consistent.test mysql-test/t/query_cache_debug.test sql-common/client.c sql/net_serv.cc sql/slave.cc sql/sql_cache.cc sql/sql_connect.cc sql/sql_parse.cc storage/myisam/ft_boolean_search.c storage/myisam/ft_parser.c storage/myisam/ft_stopwords.c support-files/compiler_warnings.supp unittest/mysys/waiting_threads-t.c vio/viossl.c -- lp:maria https://code.launchpad.net/~maria-captains/maria/5.1 Your team Maria developers is subscribed to branch lp:maria. To unsubscribe from this branch go to https://code.launchpad.net/~maria-captains/maria/5.1/+edit-subscription.

1 0

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (monty:2807)
by Michael Widenius 29 Jan '10

29 Jan '10

#At lp:maria based on revid:monty@askmonty.org-20100128144914-lqwgn4dphka023yz 2807 Michael Widenius 2010-01-29 [merge] Changed version number from RC to stable Fixed bug in Yassle to get correct error messages in case of errors Provide better error messages in case of ssl connect failure Updated out-of-date ssl certificates to fix failing mysql-test-system (certificates now active for 10 years) Fixed bug in query_cache that could cause asserts and hangs in DEBUG builds. Fixed bug where one connection did not see changes done by another connection. modified: configure.in extra/yassl/src/yassl_error.cpp include/violite.h mysql-test/mysql-test-run.pl mysql-test/r/grant.result mysql-test/r/openssl_1.result mysql-test/std_data/cacert.pem mysql-test/std_data/client-cert.pem mysql-test/std_data/client-key.pem mysql-test/std_data/server-cert.pem mysql-test/std_data/server-key.pem mysql-test/t/grant.test mysql-test/t/openssl_1.test mysql-test/t/query_cache_debug.test sql-common/client.c sql/net_serv.cc sql/slave.cc sql/sql_cache.cc sql/sql_connect.cc sql/sql_parse.cc vio/viossl.c per-file messages: configure.in Changed version number from RC to stable extra/yassl/src/yassl_error.cpp Fixed bug in Yassle to get correct error messages in case of errors - 'error' is an enum that hold more error numbers than the enum was defined for include/violite.h Added error output string for sslaccept() and sslconnect() to get reason for connect failure mysql-test/mysql-test-run.pl Write failed test cases if mysql-test-run fails because of too many errors mysql-test/r/grant.result Update results to reflect new certificates mysql-test/r/openssl_1.result Update results to reflect new certificates mysql-test/std_data/cacert.pem Update ssl certificate mysql-test/std_data/client-cert.pem Update ssl certificate mysql-test/std_data/client-key.pem Update ssl certificate mysql-test/std_data/server-cert.pem Update ssl certificate mysql-test/std_data/server-key.pem Update ssl certificate mysql-test/t/grant.test Update test to reflect new certificates mysql-test/t/openssl_1.test Update test to reflect new certificates mysql-test/t/query_cache_debug.test Remove 'big_test' as test is now fast sql-common/client.c Give a better error message if ssl connect fails sql/net_serv.cc Fixed compiler warnings sql/slave.cc Give a better error message in logs if ssl connect fails sql/sql_cache.cc debug_wait_for_kill() now removes the set watch variable after kill signal This is needed as invalidate_table() may be called twice for one query. Ensure that net->query_cache_query is reset after query. This fixes assert in query_cache_end_of_result() if query_cache_query holds results from previous query. Removed DBUG_ASSERT(0), as this code can be run by query_cache_debug.test sql/sql_connect.cc Give a better error message if ssl connect fails sql/sql_parse.cc Fixed bug where one connection did not see changes done by another connection. For statements that changes tables, close_thread_tables() MUST be called before sending OK as a table handler may not make the changes available for other connections before unlock_tables(). vio/viossl.c Give a better error message if ssl connect fails === modified file 'configure.in' --- a/configure.in 2010-01-17 17:22:46 +0000 +++ b/configure.in 2010-01-29 10:42:31 +0000 @@ -10,7 +10,7 @@ AC_CANONICAL_SYSTEM # # When changing major version number please also check switch statement # in mysqlbinlog::check_master_version(). -AM_INIT_AUTOMAKE(mysql, 5.1.42-MariaDB-rc) +AM_INIT_AUTOMAKE(mysql, 5.1.42-MariaDB) AM_CONFIG_HEADER([include/config.h:config.h.in]) # Request support for automake silent-rules if available. === modified file 'extra/yassl/src/yassl_error.cpp' --- a/extra/yassl/src/yassl_error.cpp 2008-11-18 16:45:44 +0000 +++ b/extra/yassl/src/yassl_error.cpp 2010-01-29 10:42:31 +0000 @@ -60,7 +60,7 @@ void SetErrorString(YasslError error, ch using namespace TaoCrypt; const int max = MAX_ERROR_SZ; // shorthand - switch (error) { + switch ((int) error) { // yaSSL proper errors case range_error : === modified file 'include/violite.h' --- a/include/violite.h 2010-01-15 15:27:55 +0000 +++ b/include/violite.h 2010-01-29 10:42:31 +0000 @@ -123,8 +123,8 @@ struct st_VioSSLFd SSL_CTX *ssl_context; }; -int sslaccept(struct st_VioSSLFd*, Vio *, long timeout); -int sslconnect(struct st_VioSSLFd*, Vio *, long timeout); +int sslaccept(struct st_VioSSLFd*, Vio *, long timeout, char *error_string); +int sslconnect(struct st_VioSSLFd*, Vio *, long timeout, char *error_string); struct st_VioSSLFd *new_VioSSLConnectorFd(const char *key_file, const char *cert_file, === modified file 'mysql-test/mysql-test-run.pl' --- a/mysql-test/mysql-test-run.pl 2010-01-27 17:41:05 +0000 +++ b/mysql-test/mysql-test-run.pl 2010-01-29 10:42:31 +0000 @@ -413,7 +413,6 @@ sub main { # Not all tests completed, failure mtr_report(); mtr_report("Only ", int(@$completed), " of $num_tests completed."); - mtr_error("Not all tests completed"); } mtr_print_line(); @@ -425,6 +424,10 @@ sub main { mtr_report_stats($fail, $completed, $extra_warnings); + if ( @$completed != $num_tests) + { + mtr_error("Not all tests completed"); + } exit(0); } === modified file 'mysql-test/r/grant.result' --- a/mysql-test/r/grant.result 2009-10-27 10:09:36 +0000 +++ b/mysql-test/r/grant.result 2010-01-29 10:42:31 +0000 @@ -29,15 +29,15 @@ show grants for mysqltest_1@localhost; Grants for mysqltest_1@localhost GRANT USAGE ON *.* TO 'mysqltest_1'@'localhost' GRANT SELECT ON `mysqltest`.* TO 'mysqltest_1'@'localhost' -grant USAGE on mysqltest.* to mysqltest_1@localhost require cipher "EDH-RSA-DES-CBC3-SHA" AND SUBJECT "testsubject" ISSUER "MySQL AB"; +grant USAGE on mysqltest.* to mysqltest_1@localhost require cipher "EDH-RSA-DES-CBC3-SHA" AND SUBJECT "testsubject" ISSUER "Monty Program Ab"; show grants for mysqltest_1@localhost; Grants for mysqltest_1@localhost -GRANT USAGE ON *.* TO 'mysqltest_1'@'localhost' REQUIRE ISSUER 'MySQL AB' SUBJECT 'testsubject' CIPHER 'EDH-RSA-DES-CBC3-SHA' +GRANT USAGE ON *.* TO 'mysqltest_1'@'localhost' REQUIRE ISSUER 'Monty Program Ab' SUBJECT 'testsubject' CIPHER 'EDH-RSA-DES-CBC3-SHA' GRANT SELECT ON `mysqltest`.* TO 'mysqltest_1'@'localhost' revoke all privileges on mysqltest.* from mysqltest_1@localhost; show grants for mysqltest_1@localhost; Grants for mysqltest_1@localhost -GRANT USAGE ON *.* TO 'mysqltest_1'@'localhost' REQUIRE ISSUER 'MySQL AB' SUBJECT 'testsubject' CIPHER 'EDH-RSA-DES-CBC3-SHA' +GRANT USAGE ON *.* TO 'mysqltest_1'@'localhost' REQUIRE ISSUER 'Monty Program Ab' SUBJECT 'testsubject' CIPHER 'EDH-RSA-DES-CBC3-SHA' delete from mysql.user where user='mysqltest_1'; flush privileges; delete from mysql.user where user='mysqltest_1'; === modified file 'mysql-test/r/openssl_1.result' --- a/mysql-test/r/openssl_1.result 2009-05-25 13:00:18 +0000 +++ b/mysql-test/r/openssl_1.result 2010-01-29 10:42:31 +0000 @@ -3,8 +3,10 @@ create table t1(f1 int); insert into t1 values (5); grant select on test.* to ssl_user1@localhost require SSL; grant select on test.* to ssl_user2@localhost require cipher "DHE-RSA-AES256-SHA"; -grant select on test.* to ssl_user3@localhost require cipher "DHE-RSA-AES256-SHA" AND SUBJECT "/C=SE/ST=Uppsala/O=MySQL AB/emailAddress=abstract.mysql.developer(a)mysql.com"; -grant select on test.* to ssl_user4@localhost require cipher "DHE-RSA-AES256-SHA" AND SUBJECT "/C=SE/ST=Uppsala/O=MySQL AB/emailAddress=abstract.mysql.developer(a)mysql.com" ISSUER "/C=SE/ST=Uppsala/L=Uppsala/O=MySQL AB"; +grant select on test.* to ssl_user3@localhost require cipher +"DHE-RSA-AES256-SHA" AND SUBJECT "/C=FI/ST=Tuusula/O=Monty Program Ab/emailAddress=abstract.developer(a)askmonty.org"; +grant select on test.* to ssl_user4@localhost require cipher +"DHE-RSA-AES256-SHA" AND SUBJECT "/C=FI/ST=Tuusula/O=Monty Program Ab/emailAddress=abstract.developer(a)askmonty.org" ISSUER "/C=FI/ST=Tuusula/O=Monty Program Ab/emailAddress=abstract.developer(a)askmonty.org"; grant select on test.* to ssl_user5@localhost require cipher "DHE-RSA-AES256-SHA" AND SUBJECT "xxx"; flush privileges; connect(localhost,ssl_user5,,test,MASTER_PORT,MASTER_SOCKET); @@ -44,9 +46,9 @@ ERROR 42000: DELETE command denied to us drop user ssl_user1@localhost, ssl_user2@localhost, ssl_user3@localhost, ssl_user4@localhost, ssl_user5@localhost; drop table t1; -mysqltest: Could not open connection 'default': 2026 SSL connection error -mysqltest: Could not open connection 'default': 2026 SSL connection error -mysqltest: Could not open connection 'default': 2026 SSL connection error +mysqltest: Could not open connection 'default': 2026 SSL error: ASN: bad other signature confirmation +mysqltest: Could not open connection 'default': 2026 SSL error: ASN: bad other signature confirmation +mysqltest: Could not open connection 'default': 2026 SSL error: ASN: bad other signature confirmation SSL error: Unable to get private key from '' mysqltest: Could not open connection 'default': 2026 SSL connection error SSL error: Unable to get certificate from '' === modified file 'mysql-test/std_data/cacert.pem' --- a/mysql-test/std_data/cacert.pem 2009-06-11 16:21:32 +0000 +++ b/mysql-test/std_data/cacert.pem 2010-01-29 10:42:31 +0000 @@ -1,17 +1,19 @@ -----BEGIN CERTIFICATE----- -MIICrTCCAhagAwIBAgIJAJXpePU0UOTVMA0GCSqGSIb3DQEBBQUAMEQxCzAJBgNV -BAYTAlNFMRAwDgYDVQQIEwdVcHBzYWxhMRAwDgYDVQQHEwdVcHBzYWxhMREwDwYD -VQQKEwhNeVNRTCBBQjAeFw0wOTAxMjgxMDQ5NDZaFw0xNDAxMjcxMDQ5NDZaMEQx -CzAJBgNVBAYTAlNFMRAwDgYDVQQIEwdVcHBzYWxhMRAwDgYDVQQHEwdVcHBzYWxh -MREwDwYDVQQKEwhNeVNRTCBBQjCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEA -4XQHAe5R1+TXC8noZtWf+d5E0v1C59FWpn9SWEUCBjE5UiIwuJvi4Y+7xWGOXLAI -/JzJx5gNXLBiTsE/zh0uX9fKlajLhxB0GN+QU0ZlpQ1BeYipEcNXeI/7cT499f6v -XWabnTflivdCgHSWUOQ20/Lzs6kP6/e6OoZd/DPSjPECAwEAAaOBpjCBozAdBgNV -HQ4EFgQU8uLqVWWkmuKsnZf1RWz294wRrd8wdAYDVR0jBG0wa4AU8uLqVWWkmuKs -nZf1RWz294wRrd+hSKRGMEQxCzAJBgNVBAYTAlNFMRAwDgYDVQQIEwdVcHBzYWxh -MRAwDgYDVQQHEwdVcHBzYWxhMREwDwYDVQQKEwhNeVNRTCBBQoIJAJXpePU0UOTV -MAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcNAQEFBQADgYEAMMTE5sDN+Z0ZlV7KvH3g -6+aKvql8dTpRT3hYukeQlWua0nq74WPGVw0c4e/M/vbiMwmJcCYpB9pd4+dHqzSw -aPyoenjY6UF8n7B4quWy3SIUk2LSHeJLW+kzJn2afN9gvipFhdVh/uU2TIyLGOur -Z/vmJX2W7hF1uqPnbfa8Lrw= +MIIDIjCCAougAwIBAgIJAJhuvLP+2mGwMA0GCSqGSIb3DQEBBQUAMGoxCzAJBgNV +BAYTAkZJMRAwDgYDVQQIEwdUdXVzdWxhMRkwFwYDVQQKExBNb250eSBQcm9ncmFt +IEFiMS4wLAYJKoZIhvcNAQkBFh9hYnN0cmFjdC5kZXZlbG9wZXJAYXNrbW9udHku +b3JnMB4XDTEwMDEyODIxNTcyNVoXDTEwMDIyNzIxNTcyNVowajELMAkGA1UEBhMC +RkkxEDAOBgNVBAgTB1R1dXN1bGExGTAXBgNVBAoTEE1vbnR5IFByb2dyYW0gQWIx +LjAsBgkqhkiG9w0BCQEWH2Fic3RyYWN0LmRldmVsb3BlckBhc2ttb250eS5vcmcw +gZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBAMIaY4pwbst50S32xJH3bSXhPGep +6gx1AWwZKsHTXL3VeMO6PHmC8zu5HM0zbOcrIJcXL3YVnpmE4b9OQxIiMSx1Yd+U +u8/sTkxgpsEKhCbIzECIwPhppyT/JP5aSXCadEvg+PSjikv8dOVkD68wVG4CcFIX +MFttsPebBVzEokZZAgMBAAGjgc8wgcwwHQYDVR0OBBYEFOCKaNHFFPrju8AwzWxS +f96IKfRwMIGcBgNVHSMEgZQwgZGAFOCKaNHFFPrju8AwzWxSf96IKfRwoW6kbDBq +MQswCQYDVQQGEwJGSTEQMA4GA1UECBMHVHV1c3VsYTEZMBcGA1UEChMQTW9udHkg +UHJvZ3JhbSBBYjEuMCwGCSqGSIb3DQEJARYfYWJzdHJhY3QuZGV2ZWxvcGVyQGFz +a21vbnR5Lm9yZ4IJAJhuvLP+2mGwMAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcNAQEF +BQADgYEAsmCX2/k9AInq2qhXtnkLip6cB0iOerLTNAzEijZc/aVf4wUjkL3cqhmC +kSTCwAHIOxp+ICwh6ky3xghXjoI9QnPFDVkRkzPT2tV0IoBaeQuI4e0CU2EY7L3P +XoDqp3oq1XtVcr9ZZdP68fBYUG/qcrWcXWk45ZFaBmBv3TotsGk= -----END CERTIFICATE----- === modified file 'mysql-test/std_data/client-cert.pem' --- a/mysql-test/std_data/client-cert.pem 2009-06-11 16:21:32 +0000 +++ b/mysql-test/std_data/client-cert.pem 2010-01-29 10:42:31 +0000 @@ -1,55 +1,60 @@ Certificate: Data: Version: 3 (0x2) - Serial Number: 3 (0x3) + Serial Number: 2 (0x2) Signature Algorithm: sha1WithRSAEncryption - Issuer: C=SE, ST=Uppsala, L=Uppsala, O=MySQL AB + Issuer: C=FI, ST=Tuusula, O=Monty Program Ab/emailAddress=abstract.developer(a)askmonty.org Validity - Not Before: Jan 28 11:04:39 2009 GMT - Not After : Jan 28 11:04:39 2010 GMT - Subject: C=SE, ST=Uppsala, O=MySQL AB/emailAddress=abstract.mysql.developer(a)mysql.com + Not Before: Jan 28 22:01:38 2010 GMT + Not After : Dec 7 22:01:38 2019 GMT + Subject: C=FI, ST=Tuusula, O=Monty Program Ab/emailAddress=abstract.developer(a)askmonty.org Subject Public Key Info: Public Key Algorithm: rsaEncryption - RSA Public Key: (512 bit) - Modulus (512 bit): - 00:e1:52:30:2c:d9:be:64:28:91:5d:7a:fd:d9:e9: - 14:35:7a:d2:94:4e:91:46:e0:db:9f:6b:79:f4:4c: - ac:6e:07:61:34:86:74:62:a7:a8:44:af:fa:87:87: - a8:7d:42:61:ff:ab:50:d4:7b:bf:75:fa:d5:d5:b3: - 74:fb:56:1e:37 + RSA Public Key: (1024 bit) + Modulus (1024 bit): + 00:bd:dc:3d:f8:3c:0b:d4:d2:c0:a3:9c:34:2d:e7: + 11:ff:4d:43:35:17:f6:0d:91:01:92:9e:4f:4d:c0: + 38:d5:62:03:55:33:db:66:a2:91:d3:f2:b6:23:34: + 95:53:50:3d:4f:e3:0c:d7:76:fd:f6:54:64:e6:f6: + dc:70:74:7c:6b:74:41:59:b0:19:5d:62:90:3c:a7: + c8:5e:21:8f:2b:22:6b:c7:43:9b:be:79:84:60:da: + 16:c9:ce:ee:fd:66:cb:54:81:e2:b5:1c:cf:f9:74: + de:38:2b:28:d4:31:33:55:d2:30:1c:a3:e4:c2:c7: + 31:46:43:d5:33:3d:8a:0b:47 Exponent: 65537 (0x10001) X509v3 extensions: X509v3 Basic Constraints: CA:FALSE + Netscape Comment: + OpenSSL Generated Certificate X509v3 Subject Key Identifier: - 58:30:B5:9B:2C:05:94:06:BA:3D:3C:F0:B2:CD:1D:67:65:E3:7F:85 + BE:E6:DB:19:8D:DB:72:9A:85:EE:B2:B8:5D:E7:FF:61:DF:09:08:AF X509v3 Authority Key Identifier: - keyid:F2:E2:EA:55:65:A4:9A:E2:AC:9D:97:F5:45:6C:F6:F7:8C:11:AD:DF - DirName:/C=SE/ST=Uppsala/L=Uppsala/O=MySQL AB - serial:95:E9:78:F5:34:50:E4:D5 + keyid:E0:8A:68:D1:C5:14:FA:E3:BB:C0:30:CD:6C:52:7F:DE:88:29:F4:70 Signature Algorithm: sha1WithRSAEncryption - 05:19:e3:13:14:fc:c5:28:bf:69:f8:00:b3:25:cb:bd:ca:9f: - 2f:4c:b3:a8:04:11:f0:74:27:bd:82:2c:b4:49:9b:a7:59:f0: - f7:87:d1:e0:ba:99:a2:fe:4b:1d:10:6f:e4:a2:b3:cd:7f:8b: - 68:31:46:ee:cd:9e:e2:47:e1:4c:fa:74:d1:e2:8b:cc:a0:4b: - a8:24:d1:a4:c3:6b:2a:c6:28:cd:41:e0:06:48:e6:cf:f2:3c: - ca:37:95:d7:29:64:6b:91:91:83:e7:ac:c8:0b:87:bc:da:a6: - aa:f1:44:43:c8:74:7b:15:26:91:2e:03:c4:71:50:6c:f8:68: - dc:8c + 41:95:6d:0a:a4:ee:af:68:cd:94:26:59:9a:18:b7:75:3c:c5: + 0f:22:d3:5c:31:9b:85:a0:93:b3:f0:50:29:ba:1e:d3:5a:43: + 0b:77:2d:98:87:a7:a7:39:0f:40:8d:03:d3:b3:67:43:77:bc: + 3c:51:c2:f9:9e:7a:2d:39:c4:5c:16:d7:70:d6:74:d1:6c:e1: + 6a:4d:fd:1f:10:af:64:3b:f4:64:e9:b2:b3:fb:c8:cd:c5:41: + cd:99:e0:ac:83:1d:81:2c:6b:99:ba:80:02:12:72:f7:3b:bb: + 93:72:00:da:ff:d3:87:75:d2:3a:a4:ca:4d:c1:8b:c1:21:50: + cb:57 -----BEGIN CERTIFICATE----- -MIICfzCCAeigAwIBAgIBAzANBgkqhkiG9w0BAQUFADBEMQswCQYDVQQGEwJTRTEQ -MA4GA1UECBMHVXBwc2FsYTEQMA4GA1UEBxMHVXBwc2FsYTERMA8GA1UEChMITXlT -UUwgQUIwHhcNMDkwMTI4MTEwNDM5WhcNMTAwMTI4MTEwNDM5WjBlMQswCQYDVQQG -EwJTRTEQMA4GA1UECBMHVXBwc2FsYTERMA8GA1UEChMITXlTUUwgQUIxMTAvBgkq -hkiG9w0BCQEWImFic3RyYWN0Lm15c3FsLmRldmVsb3BlckBteXNxbC5jb20wXDAN -BgkqhkiG9w0BAQEFAANLADBIAkEA4VIwLNm+ZCiRXXr92ekUNXrSlE6RRuDbn2t5 -9EysbgdhNIZ0YqeoRK/6h4eofUJh/6tQ1Hu/dfrV1bN0+1YeNwIDAQABo4GjMIGg -MAkGA1UdEwQCMAAwHQYDVR0OBBYEFFgwtZssBZQGuj088LLNHWdl43+FMHQGA1Ud -IwRtMGuAFPLi6lVlpJrirJ2X9UVs9veMEa3foUikRjBEMQswCQYDVQQGEwJTRTEQ -MA4GA1UECBMHVXBwc2FsYTEQMA4GA1UEBxMHVXBwc2FsYTERMA8GA1UEChMITXlT -UUwgQUKCCQCV6Xj1NFDk1TANBgkqhkiG9w0BAQUFAAOBgQAFGeMTFPzFKL9p+ACz -Jcu9yp8vTLOoBBHwdCe9giy0SZunWfD3h9Hgupmi/ksdEG/korPNf4toMUbuzZ7i -R+FM+nTR4ovMoEuoJNGkw2sqxijNQeAGSObP8jzKN5XXKWRrkZGD56zIC4e82qaq -8URDyHR7FSaRLgPEcVBs+GjcjA== +MIICxTCCAi6gAwIBAgIBAjANBgkqhkiG9w0BAQUFADBqMQswCQYDVQQGEwJGSTEQ +MA4GA1UECBMHVHV1c3VsYTEZMBcGA1UEChMQTW9udHkgUHJvZ3JhbSBBYjEuMCwG +CSqGSIb3DQEJARYfYWJzdHJhY3QuZGV2ZWxvcGVyQGFza21vbnR5Lm9yZzAeFw0x +MDAxMjgyMjAxMzhaFw0xOTEyMDcyMjAxMzhaMGoxCzAJBgNVBAYTAkZJMRAwDgYD +VQQIEwdUdXVzdWxhMRkwFwYDVQQKExBNb250eSBQcm9ncmFtIEFiMS4wLAYJKoZI +hvcNAQkBFh9hYnN0cmFjdC5kZXZlbG9wZXJAYXNrbW9udHkub3JnMIGfMA0GCSqG +SIb3DQEBAQUAA4GNADCBiQKBgQC93D34PAvU0sCjnDQt5xH/TUM1F/YNkQGSnk9N +wDjVYgNVM9tmopHT8rYjNJVTUD1P4wzXdv32VGTm9txwdHxrdEFZsBldYpA8p8he +IY8rImvHQ5u+eYRg2hbJzu79ZstUgeK1HM/5dN44KyjUMTNV0jAco+TCxzFGQ9Uz +PYoLRwIDAQABo3sweTAJBgNVHRMEAjAAMCwGCWCGSAGG+EIBDQQfFh1PcGVuU1NM +IEdlbmVyYXRlZCBDZXJ0aWZpY2F0ZTAdBgNVHQ4EFgQUvubbGY3bcpqF7rK4Xef/ +Yd8JCK8wHwYDVR0jBBgwFoAU4Ipo0cUU+uO7wDDNbFJ/3ogp9HAwDQYJKoZIhvcN +AQEFBQADgYEAQZVtCqTur2jNlCZZmhi3dTzFDyLTXDGbhaCTs/BQKboe01pDC3ct +mIenpzkPQI0D07NnQ3e8PFHC+Z56LTnEXBbXcNZ00Wzhak39HxCvZDv0ZOmys/vI +zcVBzZngrIMdgSxrmbqAAhJy9zu7k3IA2v/Th3XSOqTKTcGLwSFQy1c= -----END CERTIFICATE----- === modified file 'mysql-test/std_data/client-key.pem' --- a/mysql-test/std_data/client-key.pem 2009-06-11 16:21:32 +0000 +++ b/mysql-test/std_data/client-key.pem 2010-01-29 10:42:31 +0000 @@ -1,9 +1,15 @@ -----BEGIN RSA PRIVATE KEY----- -MIIBOQIBAAJBAOFSMCzZvmQokV16/dnpFDV60pROkUbg259refRMrG4HYTSGdGKn -qESv+oeHqH1CYf+rUNR7v3X61dWzdPtWHjcCAwEAAQJAXYooM8ZlcuEgj+VKU1ee -qyEFIMqJJxqcMk+E/nWCM96WxCP3zHNSrqNfSpI3ld7QzMwhdRz+gFLxT2gGNpIw -MQIhAPxzM/lDihe67X3ADYtDl9ZjA8Pm430x9sXlcxI17tCZAiEA5H1SyFl4mUee -9VnfSC2XGW7lwz72ZygfVX+b7tLWF08CIEh40gzW5MfXM+KLxdea+fXjyursV5ZT -R6KcMiKiNQLRAiAcmHqlzFzFgisotai2Fc6VRkXHG7gmzOSvBJt1VjmpDQIge6jf -2N7whTdvC4ferB+zUlgWQdyvx1c3T4gnt6PYdaY= +MIICXgIBAAKBgQC93D34PAvU0sCjnDQt5xH/TUM1F/YNkQGSnk9NwDjVYgNVM9tm +opHT8rYjNJVTUD1P4wzXdv32VGTm9txwdHxrdEFZsBldYpA8p8heIY8rImvHQ5u+ +eYRg2hbJzu79ZstUgeK1HM/5dN44KyjUMTNV0jAco+TCxzFGQ9UzPYoLRwIDAQAB +AoGBAJa2lprPT7UJ99Ho1aL6ota/RnKHKtNqII17DgjyZis9OtgP6kJ3GrvdF6iq +vT79my4nVrJTyxYXuGF/5U1/qqNjuPPBE1Xbu1ubQlFv8CT0kKYynQ7Z3ls8fAHC +B3VJXnUVlG+GHtUEFFG4FQVX1fn/Sga67ioJ6ivAiBlHKaPBAkEA5f2ToWlj4u9O +KgfRkN54wdIp4yu2c40pbhMfKGjGGsBAHk92+qSBpzEmxLcI6Ay+4/QysSR4jYmK +jCJuxiTu1QJBANNU3Hx8Il2SF/2BqGLcIh2SHxzKQIT5wAyD2jb+P2cHvbk6pKGR +VTmw5bibxXmYMS6J/L2zUF2xtFe+Svwz96sCQEnKYSqBqOWvyBFeLtPfPTlal8vm +Q4SxfuBtTCrn6t+8XRYcgt0KGPsunvSwkS/6nuh+eiExxWgMACLUDVyPjv0CQQC4 +sJJc7LOv6Oy0bWr2swHRrBEqvQsz63zOszCzHPHWHirNxZV5aiT8XT/2XZRwlvRs +gsVyGFLk/1fn0vN/g/8vAkEAxUdzUKvC1ZwjzGhgcz2bQU0tEZN4C9jBCiwOI2ud +BpAsPG0xAGGL2+hz0B0n88XiTHobiTZ1bg4Z41i4pXx2ZA== -----END RSA PRIVATE KEY----- === modified file 'mysql-test/std_data/server-cert.pem' --- a/mysql-test/std_data/server-cert.pem 2009-06-11 16:21:32 +0000 +++ b/mysql-test/std_data/server-cert.pem 2010-01-29 10:42:31 +0000 @@ -3,53 +3,59 @@ Certificate: Version: 3 (0x2) Serial Number: 1 (0x1) Signature Algorithm: sha1WithRSAEncryption - Issuer: C=SE, ST=Uppsala, L=Uppsala, O=MySQL AB + Issuer: C=FI, ST=Tuusula, O=Monty Program Ab/emailAddress=abstract.developer(a)askmonty.org Validity - Not Before: Jan 28 10:55:13 2009 GMT - Not After : Jan 28 10:55:13 2010 GMT - Subject: C=SE, ST=Uppsala, O=MySQL AB, CN=localhost/emailAddress=abstract.mysql.developer(a)mysql.com + Not Before: Jan 28 21:59:14 2010 GMT + Not After : Dec 7 21:59:14 2019 GMT + Subject: C=FI, ST=Tuusula, O=Monty Program Ab, CN=localhost/emailAddress=abstract.developer(a)askmonty.org Subject Public Key Info: Public Key Algorithm: rsaEncryption - RSA Public Key: (512 bit) - Modulus (512 bit): - 00:b6:8f:e5:b7:b4:86:83:13:8a:f9:bf:63:cb:64: - 2d:b9:51:d1:de:ab:7b:45:1f:aa:b5:66:73:13:f9: - a6:07:d5:ba:7c:fa:92:bd:37:e2:ad:87:db:3e:b6: - 6a:12:64:f8:ee:17:e3:15:06:2f:a8:82:68:bf:57: - 8d:c3:04:98:27 + RSA Public Key: (1024 bit) + Modulus (1024 bit): + 00:cc:79:74:2a:14:7e:77:06:b0:ec:1b:b6:da:70: + 4c:4f:0e:94:04:8a:e7:69:f4:4c:9c:67:22:41:56: + 3a:96:53:9e:95:9d:17:55:80:41:c0:13:d6:94:0f: + cd:2c:51:fe:a4:6e:f2:74:d8:5d:3b:3a:80:e3:85: + 5b:a5:bc:7d:5c:25:55:e5:40:77:fe:f3:cb:5b:cd: + da:a5:f4:36:64:30:a2:a0:7f:93:b3:c4:56:75:2a: + c0:f7:65:2a:d7:e6:ff:88:25:03:e0:b4:40:2e:74: + 4c:cb:46:95:73:cb:25:5c:87:0e:ec:0f:5a:19:c2: + b6:dc:9e:e8:f0:30:b1:9c:99 Exponent: 65537 (0x10001) X509v3 extensions: X509v3 Basic Constraints: CA:FALSE + Netscape Comment: + OpenSSL Generated Certificate X509v3 Subject Key Identifier: - D9:9A:B8:5F:22:EA:04:10:C8:25:7D:82:57:E6:2E:FD:19:29:E7:DA + 6D:13:3B:40:52:3C:AF:18:EA:33:D1:B7:56:21:1B:05:FE:0B:9E:38 X509v3 Authority Key Identifier: - keyid:F2:E2:EA:55:65:A4:9A:E2:AC:9D:97:F5:45:6C:F6:F7:8C:11:AD:DF - DirName:/C=SE/ST=Uppsala/L=Uppsala/O=MySQL AB - serial:95:E9:78:F5:34:50:E4:D5 + keyid:E0:8A:68:D1:C5:14:FA:E3:BB:C0:30:CD:6C:52:7F:DE:88:29:F4:70 Signature Algorithm: sha1WithRSAEncryption - 54:07:2d:21:0b:a5:af:3b:58:23:32:5e:56:7f:ab:58:63:48: - 91:aa:38:90:89:16:f9:cc:bf:a4:0e:78:2b:9f:c5:1b:58:a6: - e6:08:8f:2e:ae:97:03:21:9b:f1:cd:c0:26:8f:1d:d7:28:27: - a0:8e:81:09:1b:1c:0f:c9:a5:41:3a:2d:44:3f:9c:fa:87:ff: - c8:4c:2b:44:f7:1b:c1:3e:4f:01:7f:e9:26:cc:9f:1c:06:b5: - 0b:27:d1:10:90:be:93:0c:9c:e7:b0:d1:ea:27:99:4e:06:14: - 0c:7a:e9:c1:52:c5:33:68:bc:61:0d:db:81:3b:57:48:57:bf: - 42:9a + 97:db:65:23:7f:f1:15:3c:1e:83:ac:0e:0a:50:a0:0c:22:b8: + 45:d4:ca:21:05:47:3b:3d:03:b5:6c:4b:8d:bb:5f:57:c3:c7: + 4e:71:23:cf:33:a3:7f:a0:3d:bd:58:75:b8:37:22:16:2f:e9: + ed:ae:9b:94:29:81:6e:34:79:cf:41:bd:3d:8d:17:d7:22:1c: + 1b:58:c7:0f:79:13:56:1d:e8:d8:4e:e5:07:3f:79:1b:dd:c4: + 06:9b:c5:b6:02:34:43:c5:bf:e5:87:ad:f1:c1:8a:f2:be:c2: + 00:1d:d4:27:1f:87:c8:80:31:ec:6e:97:95:b4:84:40:d1:73: + 42:71 -----BEGIN CERTIFICATE----- -MIICkzCCAfygAwIBAgIBATANBgkqhkiG9w0BAQUFADBEMQswCQYDVQQGEwJTRTEQ -MA4GA1UECBMHVXBwc2FsYTEQMA4GA1UEBxMHVXBwc2FsYTERMA8GA1UEChMITXlT -UUwgQUIwHhcNMDkwMTI4MTA1NTEzWhcNMTAwMTI4MTA1NTEzWjB5MQswCQYDVQQG -EwJTRTEQMA4GA1UECBMHVXBwc2FsYTERMA8GA1UEChMITXlTUUwgQUIxEjAQBgNV -BAMTCWxvY2FsaG9zdDExMC8GCSqGSIb3DQEJARYiYWJzdHJhY3QubXlzcWwuZGV2 -ZWxvcGVyQG15c3FsLmNvbTBcMA0GCSqGSIb3DQEBAQUAA0sAMEgCQQC2j+W3tIaD -E4r5v2PLZC25UdHeq3tFH6q1ZnMT+aYH1bp8+pK9N+Kth9s+tmoSZPjuF+MVBi+o -gmi/V43DBJgnAgMBAAGjgaMwgaAwCQYDVR0TBAIwADAdBgNVHQ4EFgQU2Zq4XyLq -BBDIJX2CV+Yu/Rkp59owdAYDVR0jBG0wa4AU8uLqVWWkmuKsnZf1RWz294wRrd+h -SKRGMEQxCzAJBgNVBAYTAlNFMRAwDgYDVQQIEwdVcHBzYWxhMRAwDgYDVQQHEwdV -cHBzYWxhMREwDwYDVQQKEwhNeVNRTCBBQoIJAJXpePU0UOTVMA0GCSqGSIb3DQEB -BQUAA4GBAFQHLSELpa87WCMyXlZ/q1hjSJGqOJCJFvnMv6QOeCufxRtYpuYIjy6u -lwMhm/HNwCaPHdcoJ6COgQkbHA/JpUE6LUQ/nPqH/8hMK0T3G8E+TwF/6SbMnxwG -tQsn0RCQvpMMnOew0eonmU4GFAx66cFSxTNovGEN24E7V0hXv0Ka +MIIC2TCCAkKgAwIBAgIBATANBgkqhkiG9w0BAQUFADBqMQswCQYDVQQGEwJGSTEQ +MA4GA1UECBMHVHV1c3VsYTEZMBcGA1UEChMQTW9udHkgUHJvZ3JhbSBBYjEuMCwG +CSqGSIb3DQEJARYfYWJzdHJhY3QuZGV2ZWxvcGVyQGFza21vbnR5Lm9yZzAeFw0x +MDAxMjgyMTU5MTRaFw0xOTEyMDcyMTU5MTRaMH4xCzAJBgNVBAYTAkZJMRAwDgYD +VQQIEwdUdXVzdWxhMRkwFwYDVQQKExBNb250eSBQcm9ncmFtIEFiMRIwEAYDVQQD +Ewlsb2NhbGhvc3QxLjAsBgkqhkiG9w0BCQEWH2Fic3RyYWN0LmRldmVsb3BlckBh +c2ttb250eS5vcmcwgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBAMx5dCoUfncG +sOwbttpwTE8OlASK52n0TJxnIkFWOpZTnpWdF1WAQcAT1pQPzSxR/qRu8nTYXTs6 +gOOFW6W8fVwlVeVAd/7zy1vN2qX0NmQwoqB/k7PEVnUqwPdlKtfm/4glA+C0QC50 +TMtGlXPLJVyHDuwPWhnCttye6PAwsZyZAgMBAAGjezB5MAkGA1UdEwQCMAAwLAYJ +YIZIAYb4QgENBB8WHU9wZW5TU0wgR2VuZXJhdGVkIENlcnRpZmljYXRlMB0GA1Ud +DgQWBBRtEztAUjyvGOoz0bdWIRsF/gueODAfBgNVHSMEGDAWgBTgimjRxRT647vA +MM1sUn/eiCn0cDANBgkqhkiG9w0BAQUFAAOBgQCX22Ujf/EVPB6DrA4KUKAMIrhF +1MohBUc7PQO1bEuNu19Xw8dOcSPPM6N/oD29WHW4NyIWL+ntrpuUKYFuNHnPQb09 +jRfXIhwbWMcPeRNWHejYTuUHP3kb3cQGm8W2AjRDxb/lh63xwYryvsIAHdQnH4fI +gDHsbpeVtIRA0XNCcQ== -----END CERTIFICATE----- === modified file 'mysql-test/std_data/server-key.pem' --- a/mysql-test/std_data/server-key.pem 2009-06-11 16:21:32 +0000 +++ b/mysql-test/std_data/server-key.pem 2010-01-29 10:42:31 +0000 @@ -1,9 +1,15 @@ -----BEGIN RSA PRIVATE KEY----- -MIIBOQIBAAJBALaP5be0hoMTivm/Y8tkLblR0d6re0UfqrVmcxP5pgfVunz6kr03 -4q2H2z62ahJk+O4X4xUGL6iCaL9XjcMEmCcCAwEAAQJASA5VwgNb0CKHiPm0ntOk -hG+54SRX3DmafEy6gRjZIl/bZ/asSLhXUZ+CeohyrQh7BZgYWvykd8pRISL9eKsU -GQIhAOXkUrOtP/EtjyqNluEqZdG+RZi/7p61JS3Ce13Myu+LAiEAy0uMlV34AJpM -b40FPKqlHxw8DD/Dt1iKhNVAg8+LDVUCIFjv7fbJDbW2VG63/Cj8CAwOukoP5rbL -iaicVrHBKrllAiB9+MiaXeopZXNrxDS0jQFYr8Q9yt1aJVFgUkxx4Q9HKQIgZPs0 -KlF3NNNWw78INaAEkyf0IEssnLMsuoCWw0DIOak= +MIICXgIBAAKBgQDMeXQqFH53BrDsG7bacExPDpQEiudp9EycZyJBVjqWU56VnRdV +gEHAE9aUD80sUf6kbvJ02F07OoDjhVulvH1cJVXlQHf+88tbzdql9DZkMKKgf5Oz +xFZ1KsD3ZSrX5v+IJQPgtEAudEzLRpVzyyVchw7sD1oZwrbcnujwMLGcmQIDAQAB +AoGBAMdMYkNZsmJFbVDVOobzCg3Mgc1jrmeBrOKNS8AvUe+QFXRyp3m5B102eOHb +/PmD+hU/5qao9UZzoYkiRM/oRq45jrqJEYwWrX007bKK0F9hnErtC1ImM1nBFVhx +6+6cr+ShUkvtj8+wJ2d5bIccUzGCUfFR5tb5BnePTXK8IVoBAkEA7WGNxHAVKgjS +AzlpHr5fvpivA07hNVJizTwZdWGGYeETilZhkkuMRwREceeohF6ILMf0FTZdFSa/ +8EeLa3icIQJBANyDKFjynKwWy5pyRSz75mVwrEi+4eTQPsCPNWLkbpbEPwqPLYWJ +2VSFkISXF7b7Od48JkQWgiB8/kXqMDEdsXkCQQCzZvj3ryWvoP7nhOoXXBWMPGR4 +gZLe86bMKVGsTsp7CtnzwRj4sbQQr/7yfvvzHmaYQX4M0gtDQwfolomd7YdBAkEA +y24ETuqjNu9grf81aiaJipPDnOjcJOcovSRgr/blPxmUvv0Pld5yLNN7W5a4PgrO +fAMpmi7ZpXcqbP17sBQgoQJAWTDFKAmfHPVdDGZuCw4yceP5d+Tv7ABglZUvpPKx +kAvGN1WBASUuCQJDOIgzl6gvYX07S5p147i9mv7UBWOpvw== -----END RSA PRIVATE KEY----- === modified file 'mysql-test/t/grant.test' --- a/mysql-test/t/grant.test 2009-06-15 15:53:45 +0000 +++ b/mysql-test/t/grant.test 2010-01-29 10:42:31 +0000 @@ -35,7 +35,7 @@ revoke delete on mysqltest.* from mysqlt show grants for mysqltest_1@localhost; grant select on mysqltest.* to mysqltest_1@localhost require NONE; show grants for mysqltest_1@localhost; -grant USAGE on mysqltest.* to mysqltest_1@localhost require cipher "EDH-RSA-DES-CBC3-SHA" AND SUBJECT "testsubject" ISSUER "MySQL AB"; +grant USAGE on mysqltest.* to mysqltest_1@localhost require cipher "EDH-RSA-DES-CBC3-SHA" AND SUBJECT "testsubject" ISSUER "Monty Program Ab"; show grants for mysqltest_1@localhost; revoke all privileges on mysqltest.* from mysqltest_1@localhost; show grants for mysqltest_1@localhost; === modified file 'mysql-test/t/openssl_1.test' --- a/mysql-test/t/openssl_1.test 2009-05-25 13:00:18 +0000 +++ b/mysql-test/t/openssl_1.test 2010-01-29 10:42:31 +0000 @@ -15,8 +15,10 @@ insert into t1 values (5); grant select on test.* to ssl_user1@localhost require SSL; grant select on test.* to ssl_user2@localhost require cipher "DHE-RSA-AES256-SHA"; -grant select on test.* to ssl_user3@localhost require cipher "DHE-RSA-AES256-SHA" AND SUBJECT "/C=SE/ST=Uppsala/O=MySQL AB/emailAddress=abstract.mysql.developer(a)mysql.com"; -grant select on test.* to ssl_user4@localhost require cipher "DHE-RSA-AES256-SHA" AND SUBJECT "/C=SE/ST=Uppsala/O=MySQL AB/emailAddress=abstract.mysql.developer(a)mysql.com" ISSUER "/C=SE/ST=Uppsala/L=Uppsala/O=MySQL AB"; +grant select on test.* to ssl_user3@localhost require cipher +"DHE-RSA-AES256-SHA" AND SUBJECT "/C=FI/ST=Tuusula/O=Monty Program Ab/emailAddress=abstract.developer(a)askmonty.org"; +grant select on test.* to ssl_user4@localhost require cipher +"DHE-RSA-AES256-SHA" AND SUBJECT "/C=FI/ST=Tuusula/O=Monty Program Ab/emailAddress=abstract.developer(a)askmonty.org" ISSUER "/C=FI/ST=Tuusula/O=Monty Program Ab/emailAddress=abstract.developer(a)askmonty.org"; grant select on test.* to ssl_user5@localhost require cipher "DHE-RSA-AES256-SHA" AND SUBJECT "xxx"; flush privileges; === modified file 'mysql-test/t/query_cache_debug.test' --- a/mysql-test/t/query_cache_debug.test 2009-09-07 20:50:10 +0000 +++ b/mysql-test/t/query_cache_debug.test 2010-01-29 10:42:31 +0000 @@ -1,5 +1,4 @@ --source include/not_embedded.inc ---source include/big_test.inc --source include/have_query_cache.inc --source include/have_debug.inc === modified file 'sql-common/client.c' --- a/sql-common/client.c 2009-12-03 15:26:54 +0000 +++ b/sql-common/client.c 2010-01-29 10:42:31 +0000 @@ -1863,6 +1863,7 @@ CLI_MYSQL_REAL_CONNECT(MYSQL *mysql,cons uint port, const char *unix_socket,ulong client_flag) { char buff[NAME_LEN+USERNAME_LENGTH+100]; + char error_string[1024]; char *end,*host_info= NULL; my_socket sock; in_addr_t ip_addr; @@ -2331,9 +2332,14 @@ CLI_MYSQL_REAL_CONNECT(MYSQL *mysql,cons /* Connect to the server */ DBUG_PRINT("info", ("IO layer change in progress...")); if (sslconnect(ssl_fd, mysql->net.vio, - (long) (mysql->options.connect_timeout))) + (long) (mysql->options.connect_timeout), + error_string)) { - set_mysql_error(mysql, CR_SSL_CONNECTION_ERROR, unknown_sqlstate); + set_mysql_extended_error(mysql, CR_SSL_CONNECTION_ERROR, + unknown_sqlstate, + "SSL error: %s", + error_string[0] ? error_string : + ER(CR_SSL_CONNECTION_ERROR)); goto error; } DBUG_PRINT("info", ("IO layer change done!")); === modified file 'sql/net_serv.cc' --- a/sql/net_serv.cc 2010-01-28 14:49:14 +0000 +++ b/sql/net_serv.cc 2010-01-29 10:42:31 +0000 @@ -219,7 +219,7 @@ my_bool net_realloc(NET *net, size_t len -1 Don't know if data is ready or not */ -#if !defined(EMBEDDED_LIBRARY) +#if !defined(EMBEDDED_LIBRARY) && defined(DBUG_OFF) static int net_data_is_ready(my_socket sd) { @@ -280,7 +280,7 @@ static int net_data_is_ready(my_socket s void net_clear(NET *net, my_bool clear_buffer) { -#if !defined(EMBEDDED_LIBRARY) +#if !defined(EMBEDDED_LIBRARY) && defined(DBUG_OFF) size_t count; int ready; #endif === modified file 'sql/slave.cc' --- a/sql/slave.cc 2009-12-06 17:51:48 +0000 +++ b/sql/slave.cc 2010-01-29 10:42:31 +0000 @@ -3803,10 +3803,11 @@ static int connect_to_master(THD* thd, M suppress_warnings= 0; mi->report(ERROR_LEVEL, last_errno, "error %s to master '%s@%s:%d'" - " - retry-time: %d retries: %lu", + " - retry-time: %d retries: %lu message: %s", (reconnect ? "reconnecting" : "connecting"), mi->user, mi->host, mi->port, - mi->connect_retry, master_retry_count); + mi->connect_retry, master_retry_count, + mysql_error(mysql)); } /* By default we try forever. The reason is that failure will trigger === modified file 'sql/sql_cache.cc' --- a/sql/sql_cache.cc 2010-01-15 15:27:55 +0000 +++ b/sql/sql_cache.cc 2010-01-29 10:42:31 +0000 @@ -377,9 +377,11 @@ TODO list: static void debug_wait_for_kill(const char *info) { - DBUG_ENTER("debug_wait_for_kill"); const char *prev_info; THD *thd; + char buff[1024]; + DBUG_ENTER("debug_wait_for_kill"); + thd= current_thd; prev_info= thd->proc_info; thd->proc_info= info; @@ -387,8 +389,16 @@ static void debug_wait_for_kill(const ch while(!thd->killed) my_sleep(1000); thd->killed= THD::NOT_KILLED; + /* + Remove the set debug variable, to ensure we don't get stuck on it again + This is needed as for MyISAM, invalidate_table() may be called twice + (Once from mysql_delete() and once from mi_update_status()) + */ + sprintf(buff, "-d,%s", info); + DBUG_SET(buff); sql_print_information("Exit debug_wait_for_kill"); thd->proc_info= prev_info; + DBUG_VOID_RETURN; } @@ -914,15 +924,18 @@ void query_cache_insert(NET *net, const void query_cache_abort(NET *net) { + THD *thd; DBUG_ENTER("query_cache_abort"); - THD *thd= current_thd; /* See the comment on double-check locking usage above. */ if (net->query_cache_query == 0) DBUG_VOID_RETURN; if (query_cache.try_lock()) + { + net->query_cache_query = 0; DBUG_VOID_RETURN; + } /* While we were waiting another thread might have changed the status @@ -932,6 +945,7 @@ void query_cache_abort(NET *net) net->query_cache_query); if (query_block) { + thd= current_thd; thd_proc_info(thd, "storing result in query cache"); DUMP(&query_cache); BLOCK_LOCK_WR(query_block); @@ -941,6 +955,7 @@ void query_cache_abort(NET *net) DBUG_EXECUTE("check_querycache",query_cache.check_integrity(1);); } + DBUG_ASSERT(!net->query_cache_query); query_cache.unlock(); DBUG_VOID_RETURN; } @@ -970,8 +985,12 @@ void query_cache_end_of_result(THD *thd) #endif if (query_cache.try_lock()) + { + thd->net.query_cache_query= 0; DBUG_VOID_RETURN; + } + /* thd->net.query_cache_query may have changed during resize */ query_block= ((Query_cache_block*) thd->net.query_cache_query); if (query_block) { @@ -997,8 +1016,8 @@ void query_cache_end_of_result(THD *thd) to this function. In the release version that query should be ignored and removed from QC. */ - DBUG_ASSERT(0); query_cache.free_query(query_block); + thd->net.query_cache_query= 0; query_cache.unlock(); DBUG_VOID_RETURN; } === modified file 'sql/sql_connect.cc' --- a/sql/sql_connect.cc 2009-12-03 11:19:05 +0000 +++ b/sql/sql_connect.cc 2010-01-29 10:42:31 +0000 @@ -798,6 +798,7 @@ static int check_connection(THD *thd) DBUG_PRINT("info", ("client capabilities: %lu", thd->client_capabilities)); if (thd->client_capabilities & CLIENT_SSL) { + char error_string[1024]; /* Do the SSL layering. */ if (!ssl_acceptor_fd) { @@ -806,7 +807,7 @@ static int check_connection(THD *thd) return 1; } DBUG_PRINT("info", ("IO layer change in progress...")); - if (sslaccept(ssl_acceptor_fd, net->vio, net->read_timeout)) + if (sslaccept(ssl_acceptor_fd, net->vio, net->read_timeout, error_string)) { DBUG_PRINT("error", ("Failed to accept new SSL connection")); inc_host_errors(&thd->remote.sin_addr); === modified file 'sql/sql_parse.cc' --- a/sql/sql_parse.cc 2010-01-15 15:27:55 +0000 +++ b/sql/sql_parse.cc 2010-01-29 10:42:31 +0000 @@ -1252,12 +1252,14 @@ bool dispatch_command(enum enum_server_c ha_maria::implicit_commit(thd, FALSE); #endif - net_end_statement(thd); - query_cache_end_of_result(thd); /* Multiple queries exits, execute them individually */ close_thread_tables(thd); + + net_end_statement(thd); + query_cache_end_of_result(thd); + ulong length= (ulong)(packet_end - beginning_of_next_stmt); log_slow_statement(thd); @@ -1645,13 +1647,22 @@ bool dispatch_command(enum enum_server_c ha_maria::implicit_commit(thd, FALSE); #endif - net_end_statement(thd); - query_cache_end_of_result(thd); - + if (!(sql_command_flags[thd->lex->sql_command] & CF_CHANGES_DATA)) + { + /* No changes in data; We can send ok at once to the client */ + net_end_statement(thd); + query_cache_end_of_result(thd); + } thd->proc_info= "closing tables"; /* Free tables */ close_thread_tables(thd); + if (sql_command_flags[thd->lex->sql_command] & CF_CHANGES_DATA) + { + net_end_statement(thd); + query_cache_end_of_result(thd); + } + log_slow_statement(thd); thd_proc_info(thd, "cleaning up"); === modified file 'vio/viossl.c' --- a/vio/viossl.c 2008-02-07 08:55:49 +0000 +++ b/vio/viossl.c 2010-01-29 10:42:31 +0000 @@ -74,8 +74,11 @@ report_errors(SSL* ssl) } if (ssl) - DBUG_PRINT("error", ("error: %s", - ERR_error_string(SSL_get_error(ssl, l), buf))); + { + int error= SSL_get_error(ssl, l); + DBUG_PRINT("error", ("error: %s (%d)", + ERR_error_string(error, buf), error)); + } DBUG_PRINT("info", ("socket_errno: %d", socket_errno)); DBUG_VOID_RETURN; @@ -174,16 +177,17 @@ void vio_ssl_delete(Vio *vio) static int ssl_do(struct st_VioSSLFd *ptr, Vio *vio, long timeout, - int (*connect_accept_func)(SSL*)) + int (*connect_accept_func)(SSL*), char *error_string) { SSL *ssl; my_bool unused; my_bool was_blocking; - DBUG_ENTER("ssl_do"); DBUG_PRINT("enter", ("ptr: 0x%lx, sd: %d ctx: 0x%lx", (long) ptr, vio->sd, (long) ptr->ssl_context)); + error_string[0]= 0; + /* Set socket to blocking if not already set */ vio_blocking(vio, 1, &was_blocking); @@ -191,6 +195,7 @@ static int ssl_do(struct st_VioSSLFd *pt { DBUG_PRINT("error", ("SSL_new failure")); report_errors(ssl); + strmov(error_string, "SSL_new failed"); vio_blocking(vio, was_blocking, &unused); DBUG_RETURN(1); } @@ -203,6 +208,7 @@ static int ssl_do(struct st_VioSSLFd *pt { DBUG_PRINT("error", ("SSL_connect/accept failure")); report_errors(ssl); + ERR_error_string(SSL_get_error(ssl, 0), error_string); SSL_free(ssl); vio_blocking(vio, was_blocking, &unused); DBUG_RETURN(1); @@ -250,17 +256,19 @@ static int ssl_do(struct st_VioSSLFd *pt } -int sslaccept(struct st_VioSSLFd *ptr, Vio *vio, long timeout) +int sslaccept(struct st_VioSSLFd *ptr, Vio *vio, long timeout, + char *error_string) { DBUG_ENTER("sslaccept"); - DBUG_RETURN(ssl_do(ptr, vio, timeout, SSL_accept)); + DBUG_RETURN(ssl_do(ptr, vio, timeout, SSL_accept, error_string)); } -int sslconnect(struct st_VioSSLFd *ptr, Vio *vio, long timeout) +int sslconnect(struct st_VioSSLFd *ptr, Vio *vio, long timeout, + char *error_string) { DBUG_ENTER("sslconnect"); - DBUG_RETURN(ssl_do(ptr, vio, timeout, SSL_connect)); + DBUG_RETURN(ssl_do(ptr, vio, timeout, SSL_connect, error_string)); }

1 0

[Maria-developers] Strange compiler error from pbxt
by Michael Widenius 29 Jan '10

29 Jan '10

Hi! On one of the buildbot systems We get the following compiler warning: ha_pbxt.cc:1563: warning: variable 'selfâ' might be clobbered by 'longjmp' I have now disabled this warning in buildbot, but you may want to take a look at this. Regards, Monty

2 1

[Maria-developers] [Branch ~maria-captains/maria/5.1] Rev 2806: Add support in bintar build script for OpenSolaris.
by noreply＠launchpad.net 28 Jan '10

28 Jan '10

------------------------------------------------------------ revno: 2806 committer: knielsen(a)knielsen-hq.org branch nick: work-5.1 timestamp: Thu 2010-01-28 23:34:22 +0100 message: Add support in bintar build script for OpenSolaris. modified: BUILD/util.sh -- lp:maria https://code.launchpad.net/~maria-captains/maria/5.1 Your team Maria developers is subscribed to branch lp:maria. To unsubscribe from this branch go to https://code.launchpad.net/~maria-captains/maria/5.1/+edit-subscription.

1 0

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (knielsen:2806)
by knielsen＠knielsen-hq.org 28 Jan '10

28 Jan '10

#At lp:maria 2806 knielsen(a)knielsen-hq.org 2010-01-28 Add support in bintar build script for OpenSolaris. modified: BUILD/util.sh === modified file 'BUILD/util.sh' --- a/BUILD/util.sh 2010-01-07 12:02:18 +0000 +++ b/BUILD/util.sh 2010-01-28 22:34:22 +0000 @@ -16,6 +16,8 @@ # Setting cpu options. get_cpuopt () { + case "$(uname -o)" in + *Linux*) case "$(gcc -dumpmachine)" in x86_64-*) # gcc barfs on -march=... on x64 @@ -26,7 +28,13 @@ get_cpuopt () { CPUOPT="-m32 -march=i586 -mtune=generic" ;; esac - return 0 + ;; + *Solaris*) + # ToDo: handle 32-bit build? For now default to 64-bit. + CPUOPT="-D__sun -m64 -mtune=athlon64" + ;; + esac + return 0 } # Default to a parallel build, but only if AM_MAKEFLAGS is not set.

1 0

[Maria-developers] Rev 2750: BUG#31480: Incorrect result for nested subquery when executed via semi join in file:///home/psergey/dev/maria-5.3-subqueries-r3/
by Sergey Petrunya 28 Jan '10

28 Jan '10

At file:///home/psergey/dev/maria-5.3-subqueries-r3/ ------------------------------------------------------------ revno: 2750 revision-id: psergey(a)askmonty.org-20100128152533-q0dicbcuu9fvol5p parent: psergey(a)askmonty.org-20100128134833-9000udjp5wa3tsff committer: Sergey Petrunya <psergey(a)askmonty.org> branch nick: maria-5.3-subqueries-r3 timestamp: Thu 2010-01-28 18:25:33 +0300 message: BUG#31480: Incorrect result for nested subquery when executed via semi join - Make Item_subselect store its "depth", distance to furthest_correlated_ancestor, and bitmaps of tables that it depends on in ancestor tables. This allows to have Item_subselect to update its attributes when it has been pulled out from a select to its parent. === modified file 'mysql-test/r/subselect_sj.result' --- a/mysql-test/r/subselect_sj.result 2010-01-17 14:51:10 +0000 +++ b/mysql-test/r/subselect_sj.result 2010-01-28 15:25:33 +0000 @@ -779,3 +779,48 @@ 1 PRIMARY it2 ALL NULL NULL NULL NULL 20 Using where; End temporary DROP TABLE ot1, it1, it2; # End of BUG#38075 +# +# BUG#31480: Incorrect result for nested subquery when executed via semi join +# +create table t1 (a int not null, b int not null); +create table t2 (c int not null, d int not null); +create table t3 (e int not null); +insert into t1 values (1,10); +insert into t1 values (2,10); +insert into t1 values (1,20); +insert into t1 values (2,20); +insert into t1 values (3,20); +insert into t1 values (2,30); +insert into t1 values (4,40); +insert into t2 values (2,10); +insert into t2 values (2,20); +insert into t2 values (4,10); +insert into t2 values (5,10); +insert into t2 values (3,20); +insert into t2 values (2,40); +insert into t3 values (10); +insert into t3 values (30); +insert into t3 values (10); +insert into t3 values (20); +explain extended +select a from t1 +where a in (select c from t2 where d >= some(select e from t3 where b=e)); +id select_type table type possible_keys key key_len ref rows filtered Extra +1 PRIMARY t2 ALL NULL NULL NULL NULL 6 100.00 Start temporary +1 PRIMARY t1 ALL NULL NULL NULL NULL 7 100.00 Using where; End temporary; Using join buffer +3 DEPENDENT SUBQUERY t3 ALL NULL NULL NULL NULL 4 100.00 Using where +Warnings: +Note 1276 Field or reference 'test.t1.b' of SELECT #3 was resolved in SELECT #1 +Note 1003 select `test`.`t1`.`a` AS `a` from `test`.`t1` semi join (`test`.`t2`) where ((`test`.`t1`.`a` = `test`.`t2`.`c`) and <nop>(<in_optimizer>(`test`.`t2`.`d`,<exists>(select 1 AS `Not_used` from `test`.`t3` where ((`test`.`t1`.`b` = `test`.`t3`.`e`) and (<cache>(`test`.`t2`.`d`) >= `test`.`t3`.`e`)))))) +show warnings; +Level Code Message +Note 1276 Field or reference 'test.t1.b' of SELECT #3 was resolved in SELECT #1 +Note 1003 select `test`.`t1`.`a` AS `a` from `test`.`t1` semi join (`test`.`t2`) where ((`test`.`t1`.`a` = `test`.`t2`.`c`) and <nop>(<in_optimizer>(`test`.`t2`.`d`,<exists>(select 1 AS `Not_used` from `test`.`t3` where ((`test`.`t1`.`b` = `test`.`t3`.`e`) and (<cache>(`test`.`t2`.`d`) >= `test`.`t3`.`e`)))))) +select a from t1 +where a in (select c from t2 where d >= some(select e from t3 where b=e)); +a +2 +2 +3 +2 +drop table t1, t2, t3; === modified file 'mysql-test/r/subselect_sj_jcl6.result' --- a/mysql-test/r/subselect_sj_jcl6.result 2010-01-17 14:51:10 +0000 +++ b/mysql-test/r/subselect_sj_jcl6.result 2010-01-28 15:25:33 +0000 @@ -783,6 +783,51 @@ 1 PRIMARY it2 ALL NULL NULL NULL NULL 20 Using where; End temporary; Using join buffer DROP TABLE ot1, it1, it2; # End of BUG#38075 +# +# BUG#31480: Incorrect result for nested subquery when executed via semi join +# +create table t1 (a int not null, b int not null); +create table t2 (c int not null, d int not null); +create table t3 (e int not null); +insert into t1 values (1,10); +insert into t1 values (2,10); +insert into t1 values (1,20); +insert into t1 values (2,20); +insert into t1 values (3,20); +insert into t1 values (2,30); +insert into t1 values (4,40); +insert into t2 values (2,10); +insert into t2 values (2,20); +insert into t2 values (4,10); +insert into t2 values (5,10); +insert into t2 values (3,20); +insert into t2 values (2,40); +insert into t3 values (10); +insert into t3 values (30); +insert into t3 values (10); +insert into t3 values (20); +explain extended +select a from t1 +where a in (select c from t2 where d >= some(select e from t3 where b=e)); +id select_type table type possible_keys key key_len ref rows filtered Extra +1 PRIMARY t2 ALL NULL NULL NULL NULL 6 100.00 Start temporary +1 PRIMARY t1 ALL NULL NULL NULL NULL 7 100.00 Using where; End temporary; Using join buffer +3 DEPENDENT SUBQUERY t3 ALL NULL NULL NULL NULL 4 100.00 Using where +Warnings: +Note 1276 Field or reference 'test.t1.b' of SELECT #3 was resolved in SELECT #1 +Note 1003 select `test`.`t1`.`a` AS `a` from `test`.`t1` semi join (`test`.`t2`) where ((`test`.`t1`.`a` = `test`.`t2`.`c`) and <nop>(<in_optimizer>(`test`.`t2`.`d`,<exists>(select 1 AS `Not_used` from `test`.`t3` where ((`test`.`t1`.`b` = `test`.`t3`.`e`) and (<cache>(`test`.`t2`.`d`) >= `test`.`t3`.`e`)))))) +show warnings; +Level Code Message +Note 1276 Field or reference 'test.t1.b' of SELECT #3 was resolved in SELECT #1 +Note 1003 select `test`.`t1`.`a` AS `a` from `test`.`t1` semi join (`test`.`t2`) where ((`test`.`t1`.`a` = `test`.`t2`.`c`) and <nop>(<in_optimizer>(`test`.`t2`.`d`,<exists>(select 1 AS `Not_used` from `test`.`t3` where ((`test`.`t1`.`b` = `test`.`t3`.`e`) and (<cache>(`test`.`t2`.`d`) >= `test`.`t3`.`e`)))))) +select a from t1 +where a in (select c from t2 where d >= some(select e from t3 where b=e)); +a +2 +2 +3 +2 +drop table t1, t2, t3; set join_cache_level=default; show variables like 'join_cache_level'; Variable_name Value === modified file 'mysql-test/t/subselect_sj.test' --- a/mysql-test/t/subselect_sj.test 2010-01-17 14:51:10 +0000 +++ b/mysql-test/t/subselect_sj.test 2010-01-28 15:25:33 +0000 @@ -681,3 +681,41 @@ DROP TABLE ot1, it1, it2; --echo # End of BUG#38075 + +--echo # +--echo # BUG#31480: Incorrect result for nested subquery when executed via semi join +--echo # +create table t1 (a int not null, b int not null); +create table t2 (c int not null, d int not null); +create table t3 (e int not null); + +insert into t1 values (1,10); +insert into t1 values (2,10); +insert into t1 values (1,20); +insert into t1 values (2,20); +insert into t1 values (3,20); +insert into t1 values (2,30); +insert into t1 values (4,40); + +insert into t2 values (2,10); +insert into t2 values (2,20); +insert into t2 values (4,10); +insert into t2 values (5,10); +insert into t2 values (3,20); +insert into t2 values (2,40); + +insert into t3 values (10); +insert into t3 values (30); +insert into t3 values (10); +insert into t3 values (20); + +explain extended +select a from t1 +where a in (select c from t2 where d >= some(select e from t3 where b=e)); +show warnings; + +select a from t1 +where a in (select c from t2 where d >= some(select e from t3 where b=e)); + +drop table t1, t2, t3; + === modified file 'sql/item.cc' --- a/sql/item.cc 2010-01-17 14:55:08 +0000 +++ b/sql/item.cc 2010-01-28 15:25:33 +0000 @@ -2211,7 +2211,8 @@ } -void Item_field::fix_after_pullout(st_select_lex *new_parent, Item **ref) +void Item_field::fix_after_pullout(st_select_lex *new_parent, + uint parent_tables, Item **ref) { if (new_parent == depended_from) depended_from= NULL; @@ -3648,7 +3649,7 @@ static void mark_as_dependent(THD *thd, SELECT_LEX *last, SELECT_LEX *current, Item_ident *resolved_item, - Item_ident *mark_item) + Item_ident *mark_item, table_map dep_map) { const char *db_name= (resolved_item->db_name ? resolved_item->db_name : ""); @@ -3657,7 +3658,7 @@ /* store pointer on SELECT_LEX from which item is dependent */ if (mark_item) mark_item->depended_from= last; - current->mark_as_dependent(last, resolved_item); + current->mark_as_dependent(last, resolved_item, dep_map); if (thd->lex->describe & DESCRIBE_EXTENDED) { push_warning_printf(thd, MYSQL_ERROR::WARN_LEVEL_NOTE, @@ -3715,21 +3716,26 @@ Item_subselect *prev_subselect_item= previous_select->master_unit()->item; Item_ident *dependent= resolved_item; + table_map found_used_tables; if (found_field == view_ref_found) { Item::Type type= found_item->type(); + found_used_tables= found_item->used_tables(); prev_subselect_item->used_tables_cache|= - found_item->used_tables(); + found_used_tables; dependent= ((type == Item::REF_ITEM || type == Item::FIELD_ITEM) ? (Item_ident*) found_item : 0); } else + { + found_used_tables= found_field->table->map; prev_subselect_item->used_tables_cache|= found_field->table->map; + } prev_subselect_item->const_item_cache= 0; mark_as_dependent(thd, last_select, current_sel, resolved_item, - dependent); + dependent, found_used_tables); } } @@ -4010,6 +4016,7 @@ SELECT_LEX *current_sel= (SELECT_LEX *) thd->lex->current_select; Name_resolution_context *outer_context= 0; SELECT_LEX *select= 0; + uint n_levels= 0; /* Currently derived tables cannot be correlated */ if (current_sel->master_unit()->first_select()->linkage != DERIVED_TABLE_TYPE) @@ -4102,7 +4109,8 @@ context->select_lex, this, ((ref_type == REF_ITEM || ref_type == FIELD_ITEM) ? - (Item_ident*) (*reference) : 0)); + (Item_ident*) (*reference) : 0), + (*from_field)->table->map); return 0; } } @@ -4117,7 +4125,8 @@ context->select_lex, this, ((ref_type == REF_ITEM || ref_type == FIELD_ITEM) ? (Item_ident*) (*reference) : - 0)); + 0), + (*reference)->used_tables()); /* A reference to a view field had been found and we substituted it instead of this Item (find_field_in_tables @@ -4151,6 +4160,7 @@ */ prev_subselect_item->used_tables_cache|= OUTER_REF_TABLE_BIT; prev_subselect_item->const_item_cache= 0; + n_levels++; } DBUG_ASSERT(ref != 0); @@ -4218,14 +4228,15 @@ mark_as_dependent(thd, last_checked_context->select_lex, context->select_lex, this, - rf); + rf, rf->used_tables()); return 0; } else { mark_as_dependent(thd, last_checked_context->select_lex, context->select_lex, - this, (Item_ident*)*reference); + this, (Item_ident*)*reference, + (*reference)->used_tables()); if (last_checked_context->select_lex->having_fix_field) { Item_ref *rf; @@ -5947,7 +5958,8 @@ ((refer_type == REF_ITEM || refer_type == FIELD_ITEM) ? (Item_ident*) (*reference) : - 0)); + 0), + (*reference)->used_tables()); /* view reference found, we substituted it instead of this Item, so can quit @@ -5997,7 +6009,8 @@ goto error; thd->change_item_tree(reference, fld); mark_as_dependent(thd, last_checked_context->select_lex, - thd->lex->current_select, this, fld); + thd->lex->current_select, this, fld, ++ from_field->table->map); /* A reference is resolved to a nest level that's outer or the same as the nest level of the enclosing set function : adjust the value of @@ -6020,7 +6033,8 @@ /* Should be checked in resolve_ref_in_select_and_group(). */ DBUG_ASSERT(*ref && (*ref)->fixed); mark_as_dependent(thd, last_checked_context->select_lex, - context->select_lex, this, this); + context->select_lex, this, this, + (*ref)->used_tables()); /* A reference is resolved to a nest level that's outer or the same as the nest level of the enclosing set function : adjust the value of @@ -6432,20 +6446,22 @@ } -void Item_outer_ref::fix_after_pullout(st_select_lex *new_parent, Item **ref) +void Item_outer_ref::fix_after_pullout(st_select_lex *new_parent, + uint parent_tables, Item **ref) { if (depended_from == new_parent) { *ref= outer_ref; - outer_ref->fix_after_pullout(new_parent, ref); + (*ref)->fix_after_pullout(new_parent, parent_tables, ref); } } -void Item_ref::fix_after_pullout(st_select_lex *new_parent, Item **refptr) +void Item_ref::fix_after_pullout(st_select_lex *new_parent, + uint parent_tables, Item **refptr) { if (depended_from == new_parent) { - (*ref)->fix_after_pullout(new_parent, ref); + (*ref)->fix_after_pullout(new_parent, parent_tables, ref); depended_from= NULL; } } === modified file 'sql/item.h' --- a/sql/item.h 2010-01-28 13:48:33 +0000 +++ b/sql/item.h 2010-01-28 15:25:33 +0000 @@ -569,7 +569,8 @@ Fix after some tables has been pulled out. Basically re-calculate all attributes that are dependent on the tables. */ - virtual void fix_after_pullout(st_select_lex *new_parent, Item **ref) {}; + virtual void fix_after_pullout(st_select_lex *new_parent, uint parent_tables, + Item **ref) {}; /* should be used in case where we are sure that we do not need @@ -1574,7 +1575,8 @@ bool send(Protocol *protocol, String *str_arg); void reset_field(Field *f); bool fix_fields(THD *, Item **); - void fix_after_pullout(st_select_lex *new_parent, Item **ref); + void fix_after_pullout(st_select_lex *new_parent, uint parent_tables, + Item **ref); void make_field(Send_field *tmp_field); int save_in_field(Field *field,bool no_conversions); void save_org_in_field(Field *field); @@ -2343,7 +2345,8 @@ bool send(Protocol *prot, String *tmp); void make_field(Send_field *field); bool fix_fields(THD *, Item **); - void fix_after_pullout(st_select_lex *new_parent, Item **ref); + void fix_after_pullout(st_select_lex *new_parent, uint parent_tables, + Item **ref); int save_in_field(Field *field, bool no_conversions); void save_org_in_field(Field *field); enum Item_result result_type () const { return (*ref)->result_type(); } @@ -2520,7 +2523,8 @@ outer_ref->save_org_in_field(result_field); } bool fix_fields(THD *, Item **); - void fix_after_pullout(st_select_lex *new_parent, Item **ref); + void fix_after_pullout(st_select_lex *new_parent, uint parent_tables, + Item **ref); table_map used_tables() const { return (*ref)->const_item() ? 0 : OUTER_REF_TABLE_BIT; === modified file 'sql/item_cmpfunc.cc' --- a/sql/item_cmpfunc.cc 2010-01-17 14:55:08 +0000 +++ b/sql/item_cmpfunc.cc 2010-01-28 15:25:33 +0000 @@ -3959,7 +3959,8 @@ } -void Item_cond::fix_after_pullout(st_select_lex *new_parent, Item **ref) +void Item_cond::fix_after_pullout(st_select_lex *new_parent, + uint parent_tables, Item **ref) { List_iterator<Item> li(list); Item *item; @@ -3973,7 +3974,7 @@ while ((item=li++)) { table_map tmp_table_map; - item->fix_after_pullout(new_parent, li.ref()); + item->fix_after_pullout(new_parent, parent_tables, li.ref()); item= *li.ref(); used_tables_cache|= item->used_tables(); const_item_cache&= item->const_item(); === modified file 'sql/item_cmpfunc.h' --- a/sql/item_cmpfunc.h 2010-01-17 14:55:08 +0000 +++ b/sql/item_cmpfunc.h 2010-01-28 15:25:33 +0000 @@ -1472,7 +1472,8 @@ bool add_at_head(Item *item) { return list.push_front(item); } void add_at_head(List<Item> *nlist) { list.prepand(nlist); } bool fix_fields(THD *, Item **ref); - void fix_after_pullout(st_select_lex *new_parent, Item **ref); + void fix_after_pullout(st_select_lex *new_parent, uint parent_tables, + Item **ref); enum Type type() const { return COND_ITEM; } List<Item>* argument_list() { return &list; } === modified file 'sql/item_func.cc' --- a/sql/item_func.cc 2010-01-28 13:48:33 +0000 +++ b/sql/item_func.cc 2010-01-28 15:25:33 +0000 @@ -206,7 +206,8 @@ } -void Item_func::fix_after_pullout(st_select_lex *new_parent, Item **ref) +void Item_func::fix_after_pullout(st_select_lex *new_parent, + uint parent_tables, Item **ref) { Item **arg,**arg_end; @@ -217,7 +218,7 @@ { for (arg=args, arg_end=args+arg_count; arg != arg_end ; arg++) { - (*arg)->fix_after_pullout(new_parent, arg); + (*arg)->fix_after_pullout(new_parent, parent_tables, arg); Item *item= *arg; used_tables_cache|= item->used_tables(); === modified file 'sql/item_func.h' --- a/sql/item_func.h 2010-01-28 13:48:33 +0000 +++ b/sql/item_func.h 2010-01-28 15:25:33 +0000 @@ -117,7 +117,8 @@ // Constructor used for Item_cond_and/or (see Item comment) Item_func(THD *thd, Item_func *item); bool fix_fields(THD *, Item **ref); - void fix_after_pullout(st_select_lex *new_parent, Item **ref); + void fix_after_pullout(st_select_lex *new_parent, uint parent_tables, + Item **ref); table_map used_tables() const; table_map not_null_tables() const; void update_used_tables(); === modified file 'sql/item_row.cc' --- a/sql/item_row.cc 2010-01-17 14:55:08 +0000 +++ b/sql/item_row.cc 2010-01-28 15:25:33 +0000 @@ -125,13 +125,14 @@ } -void Item_row::fix_after_pullout(st_select_lex *new_parent, Item **ref) +void Item_row::fix_after_pullout(st_select_lex *new_parent, + uint parent_tables, Item **ref) { used_tables_cache= 0; const_item_cache= 1; for (uint i= 0; i < arg_count; i++) { - items[i]->fix_after_pullout(new_parent, &items[i]); + items[i]->fix_after_pullout(new_parent, parent_tables, &items[i]); used_tables_cache|= items[i]->used_tables(); const_item_cache&= items[i]->const_item(); } === modified file 'sql/item_row.h' --- a/sql/item_row.h 2010-01-17 14:55:08 +0000 +++ b/sql/item_row.h 2010-01-28 15:25:33 +0000 @@ -59,7 +59,8 @@ return 0; }; bool fix_fields(THD *thd, Item **ref); - void fix_after_pullout(st_select_lex *new_parent, Item **ref); + void fix_after_pullout(st_select_lex *new_parent, uint parent_tables, + Item **ref); void cleanup(); void split_sum_func(THD *thd, Item **ref_pointer_array, List<Item> &fields); table_map used_tables() const { return used_tables_cache; }; === modified file 'sql/item_subselect.cc' --- a/sql/item_subselect.cc 2010-01-28 13:48:33 +0000 +++ b/sql/item_subselect.cc 2010-01-28 15:25:33 +0000 @@ -39,8 +39,8 @@ Item_subselect::Item_subselect(): Item_result_field(), value_assigned(0), thd(0), substitution(0), engine(0), old_engine(0), used_tables_cache(0), have_to_be_excluded(0), - const_item_cache(1), in_fix_fields(0), engine_changed(0), changed(0), - is_correlated(FALSE) + const_item_cache(1), inside_first_fix_fields(0), in_fix_fields(0), + ancestor_used_tables(0), engine_changed(0), changed(0), is_correlated(FALSE) { with_subselect= 1; reset(); @@ -159,6 +159,15 @@ } +void Item_subselect::set_depth() +{ + uint n= 0; + for (SELECT_LEX *s= unit->first_select(); s; s= s->outer_select()) + n++; + this->depth= n - 1; +} + + bool Item_subselect::fix_fields(THD *thd_param, Item **ref) { char const *save_where= thd_param->where; @@ -169,6 +178,18 @@ engine->set_thd((thd= thd_param)); if (!in_fix_fields) refers_to.empty(); + if (!ancestor_used_tables) + { + set_depth(); + size_t size= (1+depth) * sizeof(table_map); + if (!(ancestor_used_tables= (table_map*) + alloc_root(thd->stmt_arena->mem_root, size))) + return TRUE; + bzero(ancestor_used_tables, size); + furthest_correlated_ancestor= 0; + inside_first_fix_fields= TRUE; + } + eliminated= FALSE; if (check_stack_overrun(thd, STACK_MIN_SIZE, (uchar*)&res)) @@ -179,6 +200,7 @@ // all transformation is done (used by prepared statements) changed= 1; + inside_first_fix_fields= FALSE; if (!res) { @@ -258,6 +280,65 @@ return FALSE; } +void Item_subselect::mark_as_dependent(uint n_levels, table_map dep_map) +{ + if (inside_first_fix_fields) + { + is_correlated= TRUE; + furthest_correlated_ancestor= max(furthest_correlated_ancestor, n_levels); + if (n_levels > 1) + ancestor_used_tables[n_levels - 2] |= dep_map; + } +} + + +/* + Adjust attributes after our parent select has been merged into grandparent + + DESCRIPTION + Subquery is a composite object which may be correlated, that is, it may + have + 1. references to tables of the parent select (i.e. one that has the clause + with the subquery predicate) + 2. references to tables of the grandparent select + 3. references to tables of further ancestors. + + Before the pullout, this item indicates: + - #1 with table bits in used_tables() + - #2 and #3 with OUTER_REF_TABLE_BIT. + + After parent has been merged with grandparent: + - references to parent and grandparent tables should be indicated with + table bits. + - references to greatgrandparent and further ancestors - with + OUTER_REF_TABLE_BIT. + + This is exactly what this function does, based on pre-collected info in + ancestor_used_tables and furthest_correlated_ancestor. +*/ + +void Item_subselect::fix_after_pullout(st_select_lex *new_parent, + uint parent_tables, Item **ref) +{ + used_tables_cache= (used_tables_cache << parent_tables) | + ancestor_used_tables[0]; + for (uint i=0; i < depth; i++) + ancestor_used_tables[i]= ancestor_used_tables[i+1]; + depth--; + + if (furthest_correlated_ancestor) + furthest_correlated_ancestor--; + used_tables_cache &= ~OUTER_REF_TABLE_BIT; + if (furthest_correlated_ancestor > 1) + used_tables_cache |= OUTER_REF_TABLE_BIT; + /* + Don't update const_tables_cache yet as we don't yet know which of the + parent's tables are constant. Parent will call update_used_tables() anyway, + and that will be our chance to update. + */ +} + + bool Item_subselect::walk(Item_processor processor, bool walk_subquery, uchar *argument) { === modified file 'sql/item_subselect.h' --- a/sql/item_subselect.h 2010-01-28 13:48:33 +0000 +++ b/sql/item_subselect.h 2010-01-28 15:25:33 +0000 @@ -68,6 +68,37 @@ /* cache of constant state */ bool const_item_cache; + int inside_first_fix_fields; +public: + /* + Depth of the subquery predicate. + If the subquery predicate is attatched to some clause of the top-level + select, depth will be 1 + If it is attached to a clause in a subquery of the top-level select, depth + will be 2 and so forth. + */ + uint depth; + + /* + Maximum correlation level of the select + - select that has no references to outside will have 0, + - select that references tables in the select it is located will have 1, + - select that has references to tables of its parent select will have 2, + - select that has references to tables of grandparent will have 3 + and so forth. + */ + uint furthest_correlated_ancestor; + /* + This is used_tables() for non-direct ancestors. That is, + - used_tables() shows which tables of the parent select are referred to + from within the subquery, + - ancestor_used_tables[0] shows which tables of the grandparent select are + referred to from within the subquery, + - ancestor_used_tables[1] shows which tables of the great grand parent + select... and so forth. + */ + table_map *ancestor_used_tables; + public: /* References from inside the subquery to the select that this predicate is @@ -92,6 +123,7 @@ Item_subselect(); virtual subs_type substype() { return UNKNOWN_SUBS; } + void set_depth(); /* We need this method, because some compilers do not allow 'this' @@ -117,6 +149,9 @@ return null_value; } bool fix_fields(THD *thd, Item **ref); + void mark_as_dependent(uint n_levels, table_map dep_map); + void fix_after_pullout(st_select_lex *new_parent, uint parent_tables, + Item **ref); virtual bool exec(); virtual void fix_length_and_dec(); table_map used_tables() const; === modified file 'sql/item_sum.cc' --- a/sql/item_sum.cc 2009-10-15 21:38:29 +0000 +++ b/sql/item_sum.cc 2010-01-28 15:25:33 +0000 @@ -350,7 +350,7 @@ sl= sl->master_unit()->outer_select() ) sl->master_unit()->item->with_sum_func= 1; } - thd->lex->current_select->mark_as_dependent(aggr_sel, NULL); + thd->lex->current_select->mark_as_dependent(aggr_sel, NULL, 0); return FALSE; } === modified file 'sql/sql_lex.cc' --- a/sql/sql_lex.cc 2010-01-28 13:48:33 +0000 +++ b/sql/sql_lex.cc 2010-01-28 15:25:33 +0000 @@ -1841,8 +1841,10 @@ 'last' should be reachable from this st_select_lex_node */ -void st_select_lex::mark_as_dependent(st_select_lex *last, Item *dependency) +void st_select_lex::mark_as_dependent(st_select_lex *last, Item *dependency, + table_map dep_map) { + uint n_levels= master_unit()->item->depth; SELECT_LEX *next_to_last; /* Mark all selects from resolved to 1 before select where was @@ -1868,6 +1870,11 @@ } } next_to_last= s; + Item_subselect *subquery_predicate= s->master_unit()->item; + if (subquery_predicate) + subquery_predicate->mark_as_dependent(n_levels, dep_map); + n_levels--; + //psergey-merge-todo: next_to_last should be removed?? } is_correlated= TRUE; this->master_unit()->item->is_correlated= TRUE; === modified file 'sql/sql_lex.h' --- a/sql/sql_lex.h 2010-01-28 13:48:33 +0000 +++ b/sql/sql_lex.h 2010-01-28 15:25:33 +0000 @@ -747,7 +747,8 @@ return master_unit()->return_after_parsing(); } - void mark_as_dependent(st_select_lex *last, Item *dependency); + void mark_as_dependent(st_select_lex *last, Item *dependency, + table_map dep_map); bool set_braces(bool value); bool inc_in_sum_expr(); === modified file 'sql/sql_select.cc' --- a/sql/sql_select.cc 2010-01-28 13:48:33 +0000 +++ b/sql/sql_select.cc 2010-01-28 15:25:33 +0000 @@ -3144,16 +3144,23 @@ } -void fix_list_after_tbl_changes(SELECT_LEX *new_parent, List<TABLE_LIST> *tlist) +void fix_list_after_tbl_changes(SELECT_LEX *new_parent, uint parent_tables, + List<TABLE_LIST> *tlist) { List_iterator<TABLE_LIST> it(*tlist); TABLE_LIST *table; while ((table= it++)) { if (table->on_expr) - table->on_expr->fix_after_pullout(new_parent, &table->on_expr); + { + table->on_expr->fix_after_pullout(new_parent, parent_tables, + &table->on_expr); + } if (table->nested_join) - fix_list_after_tbl_changes(new_parent, &table->nested_join->join_list); + { + fix_list_after_tbl_changes(new_parent, parent_tables, + &table->nested_join->join_list); + } } } @@ -3357,6 +3364,7 @@ /*TODO: also reset the 'with_subselect' there. */ /* n. Adjust the parent_join->tables counter */ + uint parent_tables= parent_join->tables; uint table_no= parent_join->tables; /* n. Walk through child's tables and adjust table->map */ for (tl= subq_lex->leaf_tables; tl; tl= tl->next_leaf, table_no++) @@ -3434,8 +3442,10 @@ Walk through sj nest's WHERE and ON expressions and call item->fix_table_changes() for all items. */ - sj_nest->sj_on_expr->fix_after_pullout(parent_lex, &sj_nest->sj_on_expr); - fix_list_after_tbl_changes(parent_lex, &sj_nest->nested_join->join_list); + sj_nest->sj_on_expr->fix_after_pullout(parent_lex, parent_join->tables, + &sj_nest->sj_on_expr); + fix_list_after_tbl_changes(parent_lex, parent_join->tables, + &sj_nest->nested_join->join_list); /* Unlink the child select_lex so it doesn't show up in EXPLAIN: */

1 1

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (monty:2806)
by Michael Widenius 28 Jan '10

28 Jan '10

#At lp:maria based on revid:monty@askmonty.org-20100128113510-bmn8nf14b76l3mof 2806 Michael Widenius 2010-01-28 Fixed compiler warnings & failing test cases When compiling with debug, don't clear buffer in 'net_clear()' - This allows us to easier find bugs in the protocol and also get repeatable test failures in test cases where someone forgot to do --reap modified: client/mysqltest.cc mysql-test/t/partition_innodb_semi_consistent.test sql/net_serv.cc storage/myisam/ft_boolean_search.c storage/myisam/ft_parser.c storage/myisam/ft_stopwords.c support-files/compiler_warnings.supp unittest/mysys/waiting_threads-t.c per-file messages: client/mysqltest.cc Fixed compiler warning mysql-test/t/partition_innodb_semi_consistent.test Added missing --reap (fixes random failure) sql/net_serv.cc When compiling with debug, don't clear buffer in 'net_clear()' - This allows us to easier find bugs in the protocol and also get repeatable test failures in test cases where someone forgot to do --reap storage/myisam/ft_boolean_search.c Fixed compiler warnings storage/myisam/ft_parser.c Fixed compiler warnings storage/myisam/ft_stopwords.c Fixed compiler warnings support-files/compiler_warnings.supp Added a lot of new suppression of not relevant warnings and warnings in systems we are not in charge of unittest/mysys/waiting_threads-t.c Fixed compiler warnings === modified file 'client/mysqltest.cc' --- a/client/mysqltest.cc 2010-01-15 15:27:55 +0000 +++ b/client/mysqltest.cc 2010-01-28 14:49:14 +0000 @@ -657,7 +657,9 @@ public: } while ((bytes= fread(buf, 1, sizeof(buf), m_file)) > 0) - fwrite(buf, 1, bytes, stderr); + if (fwrite(buf, 1, bytes, stderr)) + die("Failed to write to '%s', errno: %d", + m_file_name, errno); if (!lines) { === modified file 'mysql-test/t/partition_innodb_semi_consistent.test' --- a/mysql-test/t/partition_innodb_semi_consistent.test 2009-01-13 22:12:16 +0000 +++ b/mysql-test/t/partition_innodb_semi_consistent.test 2010-01-28 14:49:14 +0000 @@ -187,6 +187,7 @@ SELECT * FROM t1; --echo # Switch to connection con2 connection con2; +--reap SELECT * FROM t1; connection default; === modified file 'sql/net_serv.cc' --- a/sql/net_serv.cc 2009-09-07 20:50:10 +0000 +++ b/sql/net_serv.cc 2010-01-28 14:49:14 +0000 @@ -286,7 +286,12 @@ void net_clear(NET *net, my_bool clear_b #endif DBUG_ENTER("net_clear"); -#if !defined(EMBEDDED_LIBRARY) +/* + We don't do a clear in case of DBUG_OFF to catch bugs + in the protocol handling +*/ + +#if !defined(EMBEDDED_LIBRARY) && defined(DBUG_OFF) if (clear_buffer) { while ((ready= net_data_is_ready(net->vio->sd)) > 0) === modified file 'storage/myisam/ft_boolean_search.c' --- a/storage/myisam/ft_boolean_search.c 2010-01-27 21:53:08 +0000 +++ b/storage/myisam/ft_boolean_search.c 2010-01-28 14:49:14 +0000 @@ -321,7 +321,7 @@ static int _ftb_parse_query(FTB *ftb, uc param->mysql_add_word= ftb_query_add_word; param->mysql_ftparam= (void *)&ftb_param; param->cs= ftb->charset; - param->doc= (char*) query; + param->doc= query; param->length= len; param->flags= 0; param->mode= MYSQL_FTPARSER_FULL_BOOLEAN_INFO; @@ -702,7 +702,7 @@ static int _ftb_check_phrase(FTB *ftb, c param->mysql_add_word= ftb_phrase_add_word; param->mysql_ftparam= (void *)&ftb_param; param->cs= ftb->charset; - param->doc= (char *) document; + param->doc= document; param->length= len; param->flags= 0; param->mode= MYSQL_FTPARSER_WITH_STOPWORDS; @@ -1000,7 +1000,7 @@ float ft_boolean_find_relevance(FT_INFO { if (!ftsi.pos) continue; - param->doc= (char *)ftsi.pos; + param->doc= ftsi.pos; param->length= ftsi.len; if (unlikely(parser->parse(param))) return 0; === modified file 'storage/myisam/ft_parser.c' --- a/storage/myisam/ft_parser.c 2010-01-27 21:53:08 +0000 +++ b/storage/myisam/ft_parser.c 2010-01-28 14:49:14 +0000 @@ -175,7 +175,7 @@ uchar ft_get_word(CHARSET_INFO *cs, cons if ((param->trunc=(doc<end && *doc == FTB_TRUNC))) doc++; - if (((length >= ft_min_word_len && !is_stopword((char*) word->pos, + if (((length >= ft_min_word_len && !is_stopword(word->pos, word->len)) || param->trunc) && length < ft_max_word_len) { @@ -298,7 +298,7 @@ static int ft_parse_internal(MYSQL_FTPAR DBUG_ENTER("ft_parse_internal"); while (ft_simple_get_word(wtree->custom_arg, &doc, end, &w, TRUE)) - if (param->mysql_add_word(param, (char*) w.pos, w.len, 0)) + if (param->mysql_add_word(param, w.pos, w.len, 0)) DBUG_RETURN(1); DBUG_RETURN(0); } @@ -319,7 +319,7 @@ int ft_parse(TREE *wtree, const uchar *d param->mysql_add_word= ft_add_word; param->mysql_ftparam= &my_param; param->cs= wtree->custom_arg; - param->doc= (char*) doc; + param->doc= doc; param->length= doclen; param->mode= MYSQL_FTPARSER_SIMPLE_MODE; DBUG_RETURN(parser->parse(param)); === modified file 'storage/myisam/ft_stopwords.c' --- a/storage/myisam/ft_stopwords.c 2010-01-27 21:53:08 +0000 +++ b/storage/myisam/ft_stopwords.c 2010-01-28 14:49:14 +0000 @@ -20,7 +20,7 @@ typedef struct st_ft_stopwords { - const char * pos; + const uchar* pos; size_t len; } FT_STOPWORD; @@ -30,15 +30,15 @@ static int FT_STOPWORD_cmp(void* cmp_arg FT_STOPWORD *w1, FT_STOPWORD *w2) { return ha_compare_text(default_charset_info, - (uchar *)w1->pos,w1->len, - (uchar *)w2->pos,w2->len,0,0); + w1->pos, w1->len, + w2->pos, w2->len, 0, 0); } static void FT_STOPWORD_free(FT_STOPWORD *w, TREE_FREE action, void *arg __attribute__((unused))) { if (action == free_free) - my_free((uchar*) w->pos, MYF(0)); + my_free((void*) w->pos, MYF(0)); } static int ft_add_stopword(const char *w) === modified file 'support-files/compiler_warnings.supp' --- a/support-files/compiler_warnings.supp 2010-01-28 11:35:10 +0000 +++ b/support-files/compiler_warnings.supp 2010-01-28 14:49:14 +0000 @@ -4,6 +4,7 @@ # integer.cpp: .*control reaches end of non-void function.*: 1288-1427 +integer.cpp: .*no return statement in function returning non-void.*: 1288-1427 DictTabInfo.cpp : .*invalid access to non-static.* DictTabInfo.cpp : .*macro was used incorrectly.* DbdihMain.cpp : .*unused variable.* : 6666-7013 @@ -27,13 +28,17 @@ _flex_tmp.c: .*not enough actual paramet pars0lex.l: .*conversion from 'ulint' to 'int', possible loss of data.* btr/btr0cur\.c: .*value computed is not used.*: 3175-3375 include/buf0buf\.ic: unused parameter ��mtr�� +fil/fil0fil\.c: pointer targets in passing argument.*differ in signedness fil/fil0fil\.c: comparison between signed and unsigned : 3100-3199 fil/fil0fil\.c: unused parameter log/log0recv\.c: unused variable os/os0file\.c: unused parameter +os/os0file\.c: pointer targets in assignment differ in signedness handler/i_s\.cc: unused variable sync/sync0rw\.c: unused parameter sync/sync0sync\.c: unused parameter +sync/sync0sync\.c: unused variable +ut/ut0ut\.c: ignoring return value of # # bdb is not critical to keep up to date @@ -98,10 +103,15 @@ storage/maria/ma_pagecache.c: .*'info_ch # I think these are due to mix of C and C++. # storage/pbxt/ : typedef.*was ignored in this declaration +ha_pbxt\.cc : variable.*might be clobbered by.*longjmp # # Yassl include/runtime.hpp: .*pure_error.* +.*/extra/yassl/taocrypt/.*: comparison with string literal +.*/extra/yassl/taocrypt/src/blowfish\.cpp: array subscript is above array bounds +.*/extra/yassl/taocrypt/src/file\.cpp: ignoring return value +.*/extra/yassl/taocrypt/src/integer\.cpp: control reaches end of non-void function # # Groff warnings on OpenSUSE. === modified file 'unittest/mysys/waiting_threads-t.c' --- a/unittest/mysys/waiting_threads-t.c 2009-09-15 10:46:35 +0000 +++ b/unittest/mysys/waiting_threads-t.c 2010-01-28 14:49:14 +0000 @@ -258,7 +258,7 @@ void do_tests() #define test_kill_strategy(X) \ diag("kill strategy: " #X); \ DBUG_EXECUTE("reset_file", \ - { rewind(DBUG_FILE); ftruncate(fileno(DBUG_FILE), 0); }); \ + { rewind(DBUG_FILE); (void) ftruncate(fileno(DBUG_FILE), 0); }); \ DBUG_PRINT("info", ("kill strategy: " #X)); \ kill_strategy=X; \ do_one_test();

1 0

[Maria-developers] Rev 2749: Subquery optimizations: non-semijoin materialization in file:///home/psergey/dev/maria-5.3-subqueries-r3/
by Sergey Petrunya 28 Jan '10

28 Jan '10

At file:///home/psergey/dev/maria-5.3-subqueries-r3/ ------------------------------------------------------------ revno: 2749 revision-id: psergey(a)askmonty.org-20100128134833-9000udjp5wa3tsff parent: psergey(a)askmonty.org-20100118223223-0sxbi4py9i0guf1l committer: Sergey Petrunya <psergey(a)askmonty.org> branch nick: maria-5.3-subqueries-r3 timestamp: Thu 2010-01-28 16:48:33 +0300 message: Subquery optimizations: non-semijoin materialization - Backport into Maria DB 5.3, part 1 Diff too large for email (2209 lines, the limit is 1000).

1 0

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (monty:2805)
by Michael Widenius 28 Jan '10

28 Jan '10

#At lp:maria based on revid:knielsen@knielsen-hq.org-20100128072046-x28qsbhoudenjj0f 2805 Michael Widenius 2010-01-28 [merge] Merge with fixes for compiler warnings and 2 fixed test cases Fixed some additional compiler warnings from OpenSolaris build. modified: extra/libevent/devpoll.c include/ft_global.h include/maria.h mysql-test/r/show_check.result mysql-test/suite/rpl/t/rpl_rotate_logs.test mysql-test/t/show_check-master.opt mysys/my_file.c sql/item.cc sql/mysqld.cc sql/rpl_record.cc storage/maria/ft_maria.c storage/maria/ma_blockrec.c storage/myisam/ft_boolean_search.c storage/myisam/ft_myisam.c storage/myisam/ft_nlq_search.c storage/myisam/ft_parser.c storage/myisam/ft_stopwords.c storage/myisam/ftdefs.h storage/xtradb/buf/buf0buf.c storage/xtradb/handler/i_s.cc support-files/compiler_warnings.supp per-file messages: extra/libevent/devpoll.c Fixed compiler warning mysys/my_file.c Fixed compiler warning sql/mysqld.cc Fixed compiler warning sql/rpl_record.cc Removed not used variable storage/maria/ma_blockrec.c Fixed compiler warning storage/xtradb/buf/buf0buf.c Fixed compiler warning storage/xtradb/handler/i_s.cc Fixed compiler warning support-files/compiler_warnings.supp Added suppression of compiler warnings in InnoDB/XtraDB Added suppression of compiler warnings that can safely be ignored. === modified file 'extra/libevent/devpoll.c' --- a/extra/libevent/devpoll.c 2010-01-07 13:00:06 +0000 +++ b/extra/libevent/devpoll.c 2010-01-28 11:35:10 +0000 @@ -140,7 +140,7 @@ devpoll_init(struct event_base *base) return (NULL); if (getrlimit(RLIMIT_NOFILE, &rl) == 0 && - rl.rlim_cur != RLIM_INFINITY) + (unsigned long long) rl.rlim_cur != (unsigned long long) RLIM_INFINITY) nfiles = rl.rlim_cur - 1; /* Initialize the kernel queue */ === modified file 'include/ft_global.h' --- a/include/ft_global.h 2009-11-30 13:36:06 +0000 +++ b/include/ft_global.h 2010-01-27 21:53:08 +0000 @@ -62,7 +62,8 @@ void ft_free_stopwords(void); #define FT_SORTED 2 #define FT_EXPAND 4 /* query expansion */ -FT_INFO *ft_init_search(uint,void *, uint, uchar *, uint,CHARSET_INFO *, uchar *); +FT_INFO *ft_init_search(uint,void *, uint, uchar *, size_t, + CHARSET_INFO *, uchar *); my_bool ft_boolean_check_syntax_string(const uchar *); /* Internal symbols for fulltext between maria and MyISAM */ === modified file 'include/maria.h' --- a/include/maria.h 2009-02-19 09:01:25 +0000 +++ b/include/maria.h 2010-01-27 21:53:08 +0000 @@ -462,7 +462,7 @@ void maria_versioning(MARIA_HA *info, my void maria_ignore_trids(MARIA_HA *info); /* fulltext functions */ -FT_INFO *maria_ft_init_search(uint,void *, uint, uchar *, uint, +FT_INFO *maria_ft_init_search(uint,void *, uint, uchar *, size_t, CHARSET_INFO *, uchar *); /* 'Almost-internal' Maria functions */ === modified file 'mysql-test/r/show_check.result' --- a/mysql-test/r/show_check.result 2009-12-15 09:03:24 +0000 +++ b/mysql-test/r/show_check.result 2010-01-27 21:53:08 +0000 @@ -252,13 +252,11 @@ drop table t1; flush tables; show open tables; Database Table In_use Name_locked -mysql general_log 0 0 create table t1(n int); insert into t1 values (1); show open tables; Database Table In_use Name_locked test t1 0 0 -mysql general_log 0 0 drop table t1; create table t1 (a int not null, b VARCHAR(10), INDEX (b) ) AVG_ROW_LENGTH=10 CHECKSUM=1 COMMENT="test" ENGINE=MYISAM MIN_ROWS=10 MAX_ROWS=100 PACK_KEYS=1 DELAY_KEY_WRITE=1 ROW_FORMAT=fixed; show create table t1; === modified file 'mysql-test/suite/rpl/t/rpl_rotate_logs.test' --- a/mysql-test/suite/rpl/t/rpl_rotate_logs.test 2009-03-12 09:48:41 +0000 +++ b/mysql-test/suite/rpl/t/rpl_rotate_logs.test 2010-01-27 21:53:08 +0000 @@ -153,7 +153,7 @@ disable_query_log; while ($1) { #eval means expand $ expressions - eval insert into t3 values($1 + 4); + eval insert HIGH_PRIORITY into t3 values($1 + 4); dec $1; } enable_query_log; === modified file 'mysql-test/t/show_check-master.opt' --- a/mysql-test/t/show_check-master.opt 2009-12-03 11:19:05 +0000 +++ b/mysql-test/t/show_check-master.opt 2010-01-27 21:53:08 +0000 @@ -1 +1 @@ ---log-output=table --slow-query-log --log-long-format --log-queries-not-using-indexes --myisam-recover="" --general-log --general-log-file="" --slow-query-log-file="" +--log-output=file --slow-query-log --log-long-format --log-queries-not-using-indexes --myisam-recover="" --general-log --general-log-file="foo" --slow-query-log-file="" === modified file 'mysys/my_file.c' --- a/mysys/my_file.c 2006-12-23 19:20:40 +0000 +++ b/mysys/my_file.c 2010-01-28 11:35:10 +0000 @@ -51,7 +51,7 @@ static uint set_max_open_files(uint max_ DBUG_PRINT("info", ("rlim_cur: %u rlim_max: %u", (uint) rlimit.rlim_cur, (uint) rlimit.rlim_max)); - if (rlimit.rlim_cur == RLIM_INFINITY) + if ((ulonglong) rlimit.rlim_cur == (ulonglong) RLIM_INFINITY) rlimit.rlim_cur = max_file_limit; if (rlimit.rlim_cur >= max_file_limit) DBUG_RETURN(rlimit.rlim_cur); /* purecov: inspected */ === modified file 'sql/item.cc' --- a/sql/item.cc 2010-01-15 15:27:55 +0000 +++ b/sql/item.cc 2010-01-27 21:53:08 +0000 @@ -7119,7 +7119,7 @@ double Item_cache_decimal::val_real() DBUG_ASSERT(fixed); double res; if (!value_cached && !cache_value()) - return NULL; + return 0.0; my_decimal2double(E_DEC_FATAL_ERROR, &decimal_value, &res); return res; } === modified file 'sql/mysqld.cc' --- a/sql/mysqld.cc 2010-01-15 15:27:55 +0000 +++ b/sql/mysqld.cc 2010-01-28 11:35:10 +0000 @@ -2704,7 +2704,7 @@ static void init_signals(void) { /* Change limits so that we will get a core file */ STRUCT_RLIMIT rl; - rl.rlim_cur = rl.rlim_max = RLIM_INFINITY; + rl.rlim_cur = rl.rlim_max = (rlim_t) RLIM_INFINITY; if (setrlimit(RLIMIT_CORE, &rl) && global_system_variables.log_warnings) sql_print_warning("setrlimit could not change the size of core files to 'infinity'; We may not be able to generate a core file on signals"); } === modified file 'sql/rpl_record.cc' --- a/sql/rpl_record.cc 2009-10-22 00:15:45 +0000 +++ b/sql/rpl_record.cc 2010-01-28 11:35:10 +0000 @@ -366,7 +366,6 @@ int prepare_record(TABLE *const table, */ for (Field **field_ptr= table->field+skip; *field_ptr; ++field_ptr) { - uint32 const mask= NOT_NULL_FLAG | NO_DEFAULT_VALUE_FLAG; Field *const f= *field_ptr; if ((f->flags & NO_DEFAULT_VALUE_FLAG) && (f->real_type() != MYSQL_TYPE_ENUM)) === modified file 'storage/maria/ft_maria.c' --- a/storage/maria/ft_maria.c 2007-07-02 17:45:15 +0000 +++ b/storage/maria/ft_maria.c 2010-01-27 21:53:08 +0000 @@ -22,8 +22,8 @@ #include "ma_ftdefs.h" FT_INFO *maria_ft_init_search(uint flags, void *info, uint keynr, - uchar *query, uint query_len, CHARSET_INFO *cs, - uchar *record) + uchar *query, size_t query_len, + CHARSET_INFO *cs, uchar *record) { FT_INFO *res; if (flags & FT_BOOL) === modified file 'storage/maria/ma_blockrec.c' --- a/storage/maria/ma_blockrec.c 2010-01-06 21:27:53 +0000 +++ b/storage/maria/ma_blockrec.c 2010-01-28 11:35:10 +0000 @@ -1688,7 +1688,8 @@ static my_bool get_head_or_tail_page(MAR if (!page_link.changed) goto crashed; - DBUG_ASSERT((res->buff[PAGE_TYPE_OFFSET] & PAGE_TYPE_MASK) == page_type); + DBUG_ASSERT((uint) (res->buff[PAGE_TYPE_OFFSET] & PAGE_TYPE_MASK) == + page_type); if (!(dir= find_free_position(page_type == HEAD_PAGE ? info : 0, res->buff, block_size, &res->rownr, &res->length, &res->empty_space))) === modified file 'storage/myisam/ft_boolean_search.c' --- a/storage/myisam/ft_boolean_search.c 2010-01-15 15:27:55 +0000 +++ b/storage/myisam/ft_boolean_search.c 2010-01-27 21:53:08 +0000 @@ -299,7 +299,7 @@ static int ftb_parse_query_internal(MYSQ } -static int _ftb_parse_query(FTB *ftb, uchar *query, uint len, +static int _ftb_parse_query(FTB *ftb, uchar *query, mysql_ft_size_t len, struct st_mysql_ftparser *parser) { MYSQL_FTPARSER_PARAM *param; @@ -540,7 +540,7 @@ static void _ftb_init_index_search(FT_IN FT_INFO * ft_init_boolean_search(MI_INFO *info, uint keynr, uchar *query, - uint query_len, CHARSET_INFO *cs) + mysql_ft_size_t query_len, CHARSET_INFO *cs) { FTB *ftb; FTB_EXPR *ftbe; @@ -679,8 +679,9 @@ static int ftb_check_phrase_internal(MYS -1 is returned if error occurs. */ -static int _ftb_check_phrase(FTB *ftb, const uchar *document, uint len, - FTB_EXPR *ftbe, struct st_mysql_ftparser *parser) +static int _ftb_check_phrase(FTB *ftb, const uchar *document, + mysql_ft_size_t len, + FTB_EXPR *ftbe, struct st_mysql_ftparser *parser) { MY_FTB_PHRASE_PARAM ftb_param; MYSQL_FTPARSER_PARAM *param; === modified file 'storage/myisam/ft_myisam.c' --- a/storage/myisam/ft_myisam.c 2007-07-02 17:45:15 +0000 +++ b/storage/myisam/ft_myisam.c 2010-01-27 21:53:08 +0000 @@ -23,8 +23,8 @@ #include "ftdefs.h" FT_INFO *ft_init_search(uint flags, void *info, uint keynr, - uchar *query, uint query_len, CHARSET_INFO *cs, - uchar *record) + uchar *query, size_t query_len, + CHARSET_INFO *cs, uchar *record) { FT_INFO *res; if (flags & FT_BOOL) === modified file 'storage/myisam/ft_nlq_search.c' --- a/storage/myisam/ft_nlq_search.c 2009-11-30 13:36:06 +0000 +++ b/storage/myisam/ft_nlq_search.c 2010-01-27 21:53:08 +0000 @@ -204,7 +204,8 @@ static int FT_DOC_cmp(void *unused __att FT_INFO *ft_init_nlq_search(MI_INFO *info, uint keynr, uchar *query, - uint query_len, uint flags, uchar *record) + mysql_ft_size_t query_len, uint flags, + uchar *record) { TREE wtree; ALL_IN_ONE aio; === modified file 'storage/myisam/ft_parser.c' --- a/storage/myisam/ft_parser.c 2009-11-30 13:36:06 +0000 +++ b/storage/myisam/ft_parser.c 2010-01-27 21:53:08 +0000 @@ -304,7 +304,7 @@ static int ft_parse_internal(MYSQL_FTPAR } -int ft_parse(TREE *wtree, const uchar *doc, int doclen, +int ft_parse(TREE *wtree, const uchar *doc, mysql_ft_size_t doclen, struct st_mysql_ftparser *parser, MYSQL_FTPARSER_PARAM *param, MEM_ROOT *mem_root) { === modified file 'storage/myisam/ft_stopwords.c' --- a/storage/myisam/ft_stopwords.c 2009-11-29 23:08:56 +0000 +++ b/storage/myisam/ft_stopwords.c 2010-01-27 21:53:08 +0000 @@ -21,7 +21,7 @@ typedef struct st_ft_stopwords { const char * pos; - uint len; + size_t len; } FT_STOPWORD; static TREE *stopwords3=NULL; === modified file 'storage/myisam/ftdefs.h' --- a/storage/myisam/ftdefs.h 2009-11-29 23:08:56 +0000 +++ b/storage/myisam/ftdefs.h 2010-01-27 21:53:08 +0000 @@ -97,8 +97,8 @@ typedef struct st_ft_word { const uchar *pos; - uint len; double weight; + size_t len; } FT_WORD; int is_stopword(const uchar *word, size_t len); @@ -111,7 +111,8 @@ uchar ft_simple_get_word(CHARSET_INFO *, FT_WORD *, my_bool); typedef struct _st_ft_seg_iterator { - uint num, len; + uint num; + mysql_ft_size_t len; HA_KEYSEG *seg; const uchar *rec, *pos; } FT_SEG_ITERATOR; @@ -128,8 +129,9 @@ FT_WORD * _mi_ft_parserecord(MI_INFO *, uint _mi_ft_parse(TREE *, MI_INFO *, uint, const uchar *, MYSQL_FTPARSER_PARAM *, MEM_ROOT *); -FT_INFO *ft_init_nlq_search(MI_INFO *, uint, uchar *, uint, uint, uchar *); -FT_INFO *ft_init_boolean_search(MI_INFO *, uint, uchar *, uint, CHARSET_INFO *); +FT_INFO *ft_init_nlq_search(MI_INFO *, uint, uchar *, mysql_ft_size_t, uint, + uchar *); +FT_INFO *ft_init_boolean_search(MI_INFO *, uint, uchar *, mysql_ft_size_t, CHARSET_INFO *); extern const struct _ft_vft _ft_vft_nlq; int ft_nlq_read_next(FT_INFO *, char *); === modified file 'storage/xtradb/buf/buf0buf.c' --- a/storage/xtradb/buf/buf0buf.c 2010-01-06 12:00:14 +0000 +++ b/storage/xtradb/buf/buf0buf.c 2010-01-28 11:35:10 +0000 @@ -76,10 +76,10 @@ inline void _increment_page_get_statisti block->page.offset, DPAH_SIZE << 3); block_hash_byte = block_hash >> 3; block_hash_offset = (byte) block_hash & 0x07; - if (block_hash_byte < 0 || block_hash_byte >= DPAH_SIZE) - fprintf(stderr, "!!! block_hash_byte = %lu block_hash_offset = %lu !!!\n", block_hash_byte, block_hash_offset); - if (block_hash_offset < 0 || block_hash_offset > 7) - fprintf(stderr, "!!! block_hash_byte = %lu block_hash_offset = %lu !!!\n", block_hash_byte, block_hash_offset); + if (block_hash_byte >= DPAH_SIZE) + fprintf(stderr, "!!! block_hash_byte = %lu block_hash_offset = %lu !!!\n", (unsigned long) block_hash_byte, (unsigned long) block_hash_offset); + if (block_hash_offset > 7) + fprintf(stderr, "!!! block_hash_byte = %lu block_hash_offset = %lu !!!\n", (unsigned long) block_hash_byte, (unsigned long) block_hash_offset); if ((trx->distinct_page_access_hash[block_hash_byte] & ((byte) 0x01 << block_hash_offset)) == 0) trx->distinct_page_access++; trx->distinct_page_access_hash[block_hash_byte] |= (byte) 0x01 << block_hash_offset; === modified file 'storage/xtradb/handler/i_s.cc' --- a/storage/xtradb/handler/i_s.cc 2010-01-15 15:58:25 +0000 +++ b/storage/xtradb/handler/i_s.cc 2010-01-28 11:35:10 +0000 @@ -2992,7 +2992,7 @@ i_s_innodb_admin_command_fill( char** query_str; char* ptr; char quote = '\0'; - char* command_head = "XTRA_"; + const char* command_head = "XTRA_"; DBUG_ENTER("i_s_innodb_admin_command_fill"); === modified file 'support-files/compiler_warnings.supp' --- a/support-files/compiler_warnings.supp 2010-01-06 21:27:53 +0000 +++ b/support-files/compiler_warnings.supp 2010-01-28 11:35:10 +0000 @@ -28,6 +28,12 @@ pars0lex.l: .*conversion from 'ulint' to btr/btr0cur\.c: .*value computed is not used.*: 3175-3375 include/buf0buf\.ic: unused parameter ��mtr�� fil/fil0fil\.c: comparison between signed and unsigned : 3100-3199 +fil/fil0fil\.c: unused parameter +log/log0recv\.c: unused variable +os/os0file\.c: unused parameter +handler/i_s\.cc: unused variable +sync/sync0rw\.c: unused parameter +sync/sync0sync\.c: unused parameter # # bdb is not critical to keep up to date @@ -50,6 +56,11 @@ db_vrfy.c : .*comparison is always false .*/extra/libevent/select\.c : .*comparison between signed and unsigned.* : 270-280 # +# Ignore warnings from system libraries +# +/usr/share/aclocal/audiofile.m4 : .* + +# # Ignore all conversion warnings on windows 64 # (Is safe as we are not yet supporting strings >= 2G) # @@ -102,6 +113,8 @@ include/runtime.hpp: .*pure_error.* # listener.cc : .*conversion from 'SOCKET' to 'int'.* net_serv.cc : .*conversion from 'SOCKET' to 'int'.* +set_var.cc: right-hand operand of comma has no effect : 1000-1400 + # allow a little moving space for the warning below mi_packrec.c : .*result of 32-bit shift implicitly converted to 64 bits.* : 560-600

1 0

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (knielsen:2804)
by knielsen＠knielsen-hq.org 28 Jan '10

28 Jan '10

#At lp:maria 2804 knielsen(a)knielsen-hq.org 2010-01-28 [merge] automerge windows fixes. added: mysql-test/t/mysqlbinlog2-master.opt mysql-test/t/mysqlbinlog_row-master.opt mysql-test/t/mysqlbinlog_row_innodb-master.opt mysql-test/t/mysqlbinlog_row_myisam-master.opt mysql-test/t/mysqlbinlog_row_trans-master.opt modified: mysql-test/include/ps_conv.inc mysql-test/mysql-test-run.pl mysql-test/suite/binlog/r/binlog_unsafe.result mysql-test/suite/binlog/t/binlog_unsafe.test mysql-test/suite/maria/r/maria-recover.result mysql-test/suite/maria/t/maria-recover.test mysql-test/suite/parts/inc/partition_timestamp.inc mysql-test/suite/parts/r/partition_recover_myisam.result mysql-test/suite/parts/t/partition_recover_myisam.test mysql-test/suite/rpl/r/rpl_mixed_ddl_dml.result mysql-test/suite/rpl/t/rpl_mixed_ddl_dml.test === modified file 'mysql-test/include/ps_conv.inc' --- a/mysql-test/include/ps_conv.inc 2007-07-04 09:20:53 +0000 +++ b/mysql-test/include/ps_conv.inc 2010-01-19 10:36:52 +0000 @@ -30,6 +30,9 @@ --disable_warnings drop table if exists t5 ; --enable_warnings +--disable_query_log +SET TIME_ZONE= '+03:00'; +--enable_query_log set @arg01= 8; set @arg02= 8.0; set @arg03= 80.00000000000e-1; === modified file 'mysql-test/mysql-test-run.pl' --- a/mysql-test/mysql-test-run.pl 2010-01-06 21:27:53 +0000 +++ b/mysql-test/mysql-test-run.pl 2010-01-27 17:41:05 +0000 @@ -3425,7 +3425,7 @@ sub restart_forced_by_test # Return timezone value of tinfo or default value sub timezone { my ($tinfo)= @_; - return $tinfo->{timezone} || "GMT-3"; + return $tinfo->{timezone} || "DEFAULT"; } @@ -3455,7 +3455,11 @@ sub run_testcase ($$) { # Init variables that can change between each test case # ------------------------------------------------------- my $timezone= timezone($tinfo); - $ENV{'TZ'}= $timezone; + if ($timezone ne 'DEFAULT') { + $ENV{'TZ'}= $timezone; + } else { + delete($ENV{'TZ'}); + } mtr_verbose("Setting timezone: $timezone"); if ( ! using_extern() ) === modified file 'mysql-test/suite/binlog/r/binlog_unsafe.result' --- a/mysql-test/suite/binlog/r/binlog_unsafe.result 2010-01-15 15:27:55 +0000 +++ b/mysql-test/suite/binlog/r/binlog_unsafe.result 2010-01-19 10:36:52 +0000 @@ -380,6 +380,7 @@ INSERT INTO t1 VALUES (VERSION()); Warnings: Note 1592 Statement may not be safe to log in statement format. DELETE FROM t1; +SET TIME_ZONE= '+03:00'; SET TIMESTAMP=1000000; INSERT INTO t1 VALUES (CURDATE()), === modified file 'mysql-test/suite/binlog/t/binlog_unsafe.test' --- a/mysql-test/suite/binlog/t/binlog_unsafe.test 2010-01-15 15:27:55 +0000 +++ b/mysql-test/suite/binlog/t/binlog_unsafe.test 2010-01-19 10:36:52 +0000 @@ -422,6 +422,7 @@ DELETE FROM t1; # following following functions depend on the TIMESTAMP variable and # don't generate a warning. +SET TIME_ZONE= '+03:00'; SET TIMESTAMP=1000000; INSERT INTO t1 VALUES (CURDATE()), === modified file 'mysql-test/suite/maria/r/maria-recover.result' --- a/mysql-test/suite/maria/r/maria-recover.result 2009-02-19 09:01:25 +0000 +++ b/mysql-test/suite/maria/r/maria-recover.result 2010-01-27 17:41:05 +0000 @@ -1,6 +1,3 @@ -call mtr.add_suppression("Checking table: '.\/mysqltest\/t_corrupted2'"); -call mtr.add_suppression("Recovering table: '.\/mysqltest\/t_corrupted2'"); -call mtr.add_suppression("Table '.\/mysqltest\/t_corrupted2' is marked as crashed and should be repaired"); select @@global.maria_recover; @@global.maria_recover BACKUP === modified file 'mysql-test/suite/maria/t/maria-recover.test' --- a/mysql-test/suite/maria/t/maria-recover.test 2009-10-09 08:09:24 +0000 +++ b/mysql-test/suite/maria/t/maria-recover.test 2010-01-27 17:41:05 +0000 @@ -2,9 +2,13 @@ --source include/have_maria.inc -call mtr.add_suppression("Checking table: '.\/mysqltest\/t_corrupted2'"); -call mtr.add_suppression("Recovering table: '.\/mysqltest\/t_corrupted2'"); -call mtr.add_suppression("Table '.\/mysqltest\/t_corrupted2' is marked as crashed and should be repaired"); +--disable_query_log +# Note: \\. matches a single period. We use '.' as directory separator to +# account for Unix and Windows variation. +call mtr.add_suppression("Checking table: '\\..mysqltest.t_corrupted2'"); +call mtr.add_suppression("Recovering table: '\\..mysqltest.t_corrupted2'"); +call mtr.add_suppression("Table '\\..mysqltest.t_corrupted2' is marked as crashed and should be repaired"); +--enable_query_log # Note: we're setting an environment variable (not prefixing it by $), # so that the perl code below can access it. === modified file 'mysql-test/suite/parts/inc/partition_timestamp.inc' --- a/mysql-test/suite/parts/inc/partition_timestamp.inc 2009-10-28 07:52:34 +0000 +++ b/mysql-test/suite/parts/inc/partition_timestamp.inc 2010-01-27 17:41:05 +0000 @@ -48,6 +48,7 @@ show create table t3; let $count=12; --echo $count inserts; --disable_query_log +SET TIME_ZONE= '+03:00'; begin; while ($count) { === modified file 'mysql-test/suite/parts/r/partition_recover_myisam.result' --- a/mysql-test/suite/parts/r/partition_recover_myisam.result 2009-08-29 21:29:47 +0000 +++ b/mysql-test/suite/parts/r/partition_recover_myisam.result 2010-01-27 17:41:05 +0000 @@ -1,5 +1,3 @@ -call mtr.add_suppression("./test/t1_will_crash"); -call mtr.add_suppression("Got an error from unknown thread, ha_myisam.cc"); CREATE TABLE t1_will_crash (a INT, KEY (a)) ENGINE=MyISAM; INSERT INTO t1_will_crash VALUES (1), (2), (3), (4), (5), (6), (7), (8), (9), (10), (11); FLUSH TABLES; === modified file 'mysql-test/suite/parts/t/partition_recover_myisam.test' --- a/mysql-test/suite/parts/t/partition_recover_myisam.test 2009-12-03 11:19:05 +0000 +++ b/mysql-test/suite/parts/t/partition_recover_myisam.test 2010-01-27 17:41:05 +0000 @@ -1,7 +1,9 @@ # test the auto-recover (--myisam-recover) of partitioned myisam tables -call mtr.add_suppression("./test/t1_will_crash"); +--disable_query_log +call mtr.add_suppression("..test.t1_will_crash"); call mtr.add_suppression("Got an error from unknown thread, ha_myisam.cc"); +--enable_query_log --source include/have_partition.inc --disable_warnings === modified file 'mysql-test/suite/rpl/r/rpl_mixed_ddl_dml.result' --- a/mysql-test/suite/rpl/r/rpl_mixed_ddl_dml.result 2008-11-04 17:07:14 +0000 +++ b/mysql-test/suite/rpl/r/rpl_mixed_ddl_dml.result 2010-01-19 10:36:52 +0000 @@ -18,6 +18,7 @@ Server_id Host Port Rpl_recovery_rank Ma drop table t1; stop slave; create table t2(id int auto_increment primary key, created datetime); +SET TIME_ZONE= '+03:00'; set timestamp=12345; insert into t2 set created=now(); select * from t2; === modified file 'mysql-test/suite/rpl/t/rpl_mixed_ddl_dml.test' --- a/mysql-test/suite/rpl/t/rpl_mixed_ddl_dml.test 2007-06-27 12:28:02 +0000 +++ b/mysql-test/suite/rpl/t/rpl_mixed_ddl_dml.test 2010-01-19 10:36:52 +0000 @@ -23,6 +23,7 @@ connection master; # Test replication of timestamp create table t2(id int auto_increment primary key, created datetime); +SET TIME_ZONE= '+03:00'; set timestamp=12345; insert into t2 set created=now(); select * from t2; === added file 'mysql-test/t/mysqlbinlog2-master.opt' --- a/mysql-test/t/mysqlbinlog2-master.opt 1970-01-01 00:00:00 +0000 +++ b/mysql-test/t/mysqlbinlog2-master.opt 2010-01-19 10:36:52 +0000 @@ -0,0 +1 @@ +--timezone=GMT-3 === added file 'mysql-test/t/mysqlbinlog_row-master.opt' --- a/mysql-test/t/mysqlbinlog_row-master.opt 1970-01-01 00:00:00 +0000 +++ b/mysql-test/t/mysqlbinlog_row-master.opt 2010-01-27 17:41:05 +0000 @@ -0,0 +1 @@ +--timezone=GMT-3 === added file 'mysql-test/t/mysqlbinlog_row_innodb-master.opt' --- a/mysql-test/t/mysqlbinlog_row_innodb-master.opt 1970-01-01 00:00:00 +0000 +++ b/mysql-test/t/mysqlbinlog_row_innodb-master.opt 2010-01-27 17:41:05 +0000 @@ -0,0 +1 @@ +--timezone=GMT-3 === added file 'mysql-test/t/mysqlbinlog_row_myisam-master.opt' --- a/mysql-test/t/mysqlbinlog_row_myisam-master.opt 1970-01-01 00:00:00 +0000 +++ b/mysql-test/t/mysqlbinlog_row_myisam-master.opt 2010-01-27 17:41:05 +0000 @@ -0,0 +1 @@ +--timezone=GMT-3 === added file 'mysql-test/t/mysqlbinlog_row_trans-master.opt' --- a/mysql-test/t/mysqlbinlog_row_trans-master.opt 1970-01-01 00:00:00 +0000 +++ b/mysql-test/t/mysqlbinlog_row_trans-master.opt 2010-01-27 17:41:05 +0000 @@ -0,0 +1 @@ +--timezone=GMT-3

1 0

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (monty:2803)
by Michael Widenius 27 Jan '10

27 Jan '10

#At lp:maria based on revid:hakan@askmonty.org-20100122211921-e177xzarwy7l8j5y 2803 Michael Widenius 2010-01-27 Fixed some compiler warnings and errors in test suite found by buildbot modified: include/ft_global.h include/maria.h mysql-test/r/show_check.result mysql-test/suite/rpl/t/rpl_rotate_logs.test mysql-test/t/show_check-master.opt sql/item.cc storage/maria/ft_maria.c storage/myisam/ft_boolean_search.c storage/myisam/ft_myisam.c storage/myisam/ft_nlq_search.c storage/myisam/ft_parser.c storage/myisam/ft_stopwords.c storage/myisam/ftdefs.h per-file messages: include/ft_global.h Changed string size to 'size_t' (to avoid compiler warnings later) include/maria.h Changed string size to 'size_t' (to avoid compiler warnings later) mysql-test/r/show_check.result Don't use logging to tables, as there is no guarantee in which order tables are showed in 'show open tables' mysql-test/suite/rpl/t/rpl_rotate_logs.test Use HIGH_PRIORITY inserts to avoid problem with concurrent inserts on the slave. mysql-test/t/show_check-master.opt Don't use logging to tables, as there is no guarantee in which order tables are showed in 'show open tables' sql/item.cc Fixed compiler warning storage/maria/ft_maria.c Changed string size to 'size_t' (to avoid compiler warnings later) storage/myisam/ft_boolean_search.c Fixed compiler warnings storage/myisam/ft_myisam.c Changed string size to 'size_t' (to avoid compiler warnings later) storage/myisam/ft_nlq_search.c Changed string size to 'size_t' (to avoid compiler warnings later) storage/myisam/ft_parser.c Changed string size to 'size_t' (to avoid compiler warnings) storage/myisam/ft_stopwords.c Changed string size to 'size_t' (to avoid compiler warnings) storage/myisam/ftdefs.h Changed string size to 'size_t' (to avoid compiler warnings) === modified file 'include/ft_global.h' --- a/include/ft_global.h 2009-11-30 13:36:06 +0000 +++ b/include/ft_global.h 2010-01-27 21:53:08 +0000 @@ -62,7 +62,8 @@ void ft_free_stopwords(void); #define FT_SORTED 2 #define FT_EXPAND 4 /* query expansion */ -FT_INFO *ft_init_search(uint,void *, uint, uchar *, uint,CHARSET_INFO *, uchar *); +FT_INFO *ft_init_search(uint,void *, uint, uchar *, size_t, + CHARSET_INFO *, uchar *); my_bool ft_boolean_check_syntax_string(const uchar *); /* Internal symbols for fulltext between maria and MyISAM */ === modified file 'include/maria.h' --- a/include/maria.h 2009-02-19 09:01:25 +0000 +++ b/include/maria.h 2010-01-27 21:53:08 +0000 @@ -462,7 +462,7 @@ void maria_versioning(MARIA_HA *info, my void maria_ignore_trids(MARIA_HA *info); /* fulltext functions */ -FT_INFO *maria_ft_init_search(uint,void *, uint, uchar *, uint, +FT_INFO *maria_ft_init_search(uint,void *, uint, uchar *, size_t, CHARSET_INFO *, uchar *); /* 'Almost-internal' Maria functions */ === modified file 'mysql-test/r/show_check.result' --- a/mysql-test/r/show_check.result 2009-12-15 09:03:24 +0000 +++ b/mysql-test/r/show_check.result 2010-01-27 21:53:08 +0000 @@ -252,13 +252,11 @@ drop table t1; flush tables; show open tables; Database Table In_use Name_locked -mysql general_log 0 0 create table t1(n int); insert into t1 values (1); show open tables; Database Table In_use Name_locked test t1 0 0 -mysql general_log 0 0 drop table t1; create table t1 (a int not null, b VARCHAR(10), INDEX (b) ) AVG_ROW_LENGTH=10 CHECKSUM=1 COMMENT="test" ENGINE=MYISAM MIN_ROWS=10 MAX_ROWS=100 PACK_KEYS=1 DELAY_KEY_WRITE=1 ROW_FORMAT=fixed; show create table t1; === modified file 'mysql-test/suite/rpl/t/rpl_rotate_logs.test' --- a/mysql-test/suite/rpl/t/rpl_rotate_logs.test 2009-03-12 09:48:41 +0000 +++ b/mysql-test/suite/rpl/t/rpl_rotate_logs.test 2010-01-27 21:53:08 +0000 @@ -153,7 +153,7 @@ disable_query_log; while ($1) { #eval means expand $ expressions - eval insert into t3 values($1 + 4); + eval insert HIGH_PRIORITY into t3 values($1 + 4); dec $1; } enable_query_log; === modified file 'mysql-test/t/show_check-master.opt' --- a/mysql-test/t/show_check-master.opt 2009-12-03 11:19:05 +0000 +++ b/mysql-test/t/show_check-master.opt 2010-01-27 21:53:08 +0000 @@ -1 +1 @@ ---log-output=table --slow-query-log --log-long-format --log-queries-not-using-indexes --myisam-recover="" --general-log --general-log-file="" --slow-query-log-file="" +--log-output=file --slow-query-log --log-long-format --log-queries-not-using-indexes --myisam-recover="" --general-log --general-log-file="foo" --slow-query-log-file="" === modified file 'sql/item.cc' --- a/sql/item.cc 2010-01-15 15:27:55 +0000 +++ b/sql/item.cc 2010-01-27 21:53:08 +0000 @@ -7119,7 +7119,7 @@ double Item_cache_decimal::val_real() DBUG_ASSERT(fixed); double res; if (!value_cached && !cache_value()) - return NULL; + return 0.0; my_decimal2double(E_DEC_FATAL_ERROR, &decimal_value, &res); return res; } === modified file 'storage/maria/ft_maria.c' --- a/storage/maria/ft_maria.c 2007-07-02 17:45:15 +0000 +++ b/storage/maria/ft_maria.c 2010-01-27 21:53:08 +0000 @@ -22,8 +22,8 @@ #include "ma_ftdefs.h" FT_INFO *maria_ft_init_search(uint flags, void *info, uint keynr, - uchar *query, uint query_len, CHARSET_INFO *cs, - uchar *record) + uchar *query, size_t query_len, + CHARSET_INFO *cs, uchar *record) { FT_INFO *res; if (flags & FT_BOOL) === modified file 'storage/myisam/ft_boolean_search.c' --- a/storage/myisam/ft_boolean_search.c 2010-01-15 15:27:55 +0000 +++ b/storage/myisam/ft_boolean_search.c 2010-01-27 21:53:08 +0000 @@ -299,7 +299,7 @@ static int ftb_parse_query_internal(MYSQ } -static int _ftb_parse_query(FTB *ftb, uchar *query, uint len, +static int _ftb_parse_query(FTB *ftb, uchar *query, mysql_ft_size_t len, struct st_mysql_ftparser *parser) { MYSQL_FTPARSER_PARAM *param; @@ -540,7 +540,7 @@ static void _ftb_init_index_search(FT_IN FT_INFO * ft_init_boolean_search(MI_INFO *info, uint keynr, uchar *query, - uint query_len, CHARSET_INFO *cs) + mysql_ft_size_t query_len, CHARSET_INFO *cs) { FTB *ftb; FTB_EXPR *ftbe; @@ -679,8 +679,9 @@ static int ftb_check_phrase_internal(MYS -1 is returned if error occurs. */ -static int _ftb_check_phrase(FTB *ftb, const uchar *document, uint len, - FTB_EXPR *ftbe, struct st_mysql_ftparser *parser) +static int _ftb_check_phrase(FTB *ftb, const uchar *document, + mysql_ft_size_t len, + FTB_EXPR *ftbe, struct st_mysql_ftparser *parser) { MY_FTB_PHRASE_PARAM ftb_param; MYSQL_FTPARSER_PARAM *param; === modified file 'storage/myisam/ft_myisam.c' --- a/storage/myisam/ft_myisam.c 2007-07-02 17:45:15 +0000 +++ b/storage/myisam/ft_myisam.c 2010-01-27 21:53:08 +0000 @@ -23,8 +23,8 @@ #include "ftdefs.h" FT_INFO *ft_init_search(uint flags, void *info, uint keynr, - uchar *query, uint query_len, CHARSET_INFO *cs, - uchar *record) + uchar *query, size_t query_len, + CHARSET_INFO *cs, uchar *record) { FT_INFO *res; if (flags & FT_BOOL) === modified file 'storage/myisam/ft_nlq_search.c' --- a/storage/myisam/ft_nlq_search.c 2009-11-30 13:36:06 +0000 +++ b/storage/myisam/ft_nlq_search.c 2010-01-27 21:53:08 +0000 @@ -204,7 +204,8 @@ static int FT_DOC_cmp(void *unused __att FT_INFO *ft_init_nlq_search(MI_INFO *info, uint keynr, uchar *query, - uint query_len, uint flags, uchar *record) + mysql_ft_size_t query_len, uint flags, + uchar *record) { TREE wtree; ALL_IN_ONE aio; === modified file 'storage/myisam/ft_parser.c' --- a/storage/myisam/ft_parser.c 2009-11-30 13:36:06 +0000 +++ b/storage/myisam/ft_parser.c 2010-01-27 21:53:08 +0000 @@ -304,7 +304,7 @@ static int ft_parse_internal(MYSQL_FTPAR } -int ft_parse(TREE *wtree, const uchar *doc, int doclen, +int ft_parse(TREE *wtree, const uchar *doc, mysql_ft_size_t doclen, struct st_mysql_ftparser *parser, MYSQL_FTPARSER_PARAM *param, MEM_ROOT *mem_root) { === modified file 'storage/myisam/ft_stopwords.c' --- a/storage/myisam/ft_stopwords.c 2009-11-29 23:08:56 +0000 +++ b/storage/myisam/ft_stopwords.c 2010-01-27 21:53:08 +0000 @@ -21,7 +21,7 @@ typedef struct st_ft_stopwords { const char * pos; - uint len; + size_t len; } FT_STOPWORD; static TREE *stopwords3=NULL; === modified file 'storage/myisam/ftdefs.h' --- a/storage/myisam/ftdefs.h 2009-11-29 23:08:56 +0000 +++ b/storage/myisam/ftdefs.h 2010-01-27 21:53:08 +0000 @@ -97,8 +97,8 @@ typedef struct st_ft_word { const uchar *pos; - uint len; double weight; + size_t len; } FT_WORD; int is_stopword(const uchar *word, size_t len); @@ -111,7 +111,8 @@ uchar ft_simple_get_word(CHARSET_INFO *, FT_WORD *, my_bool); typedef struct _st_ft_seg_iterator { - uint num, len; + uint num; + mysql_ft_size_t len; HA_KEYSEG *seg; const uchar *rec, *pos; } FT_SEG_ITERATOR; @@ -128,8 +129,9 @@ FT_WORD * _mi_ft_parserecord(MI_INFO *, uint _mi_ft_parse(TREE *, MI_INFO *, uint, const uchar *, MYSQL_FTPARSER_PARAM *, MEM_ROOT *); -FT_INFO *ft_init_nlq_search(MI_INFO *, uint, uchar *, uint, uint, uchar *); -FT_INFO *ft_init_boolean_search(MI_INFO *, uint, uchar *, uint, CHARSET_INFO *); +FT_INFO *ft_init_nlq_search(MI_INFO *, uint, uchar *, mysql_ft_size_t, uint, + uchar *); +FT_INFO *ft_init_boolean_search(MI_INFO *, uint, uchar *, mysql_ft_size_t, CHARSET_INFO *); extern const struct _ft_vft _ft_vft_nlq; int ft_nlq_read_next(FT_INFO *, char *);

1 0

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (knielsen:2794)
by knielsen＠knielsen-hq.org 27 Jan '10

27 Jan '10

#At lp:maria 2794 knielsen(a)knielsen-hq.org 2010-01-27 [merge] automerge for buildbot test added: mysql-test/t/mysqlbinlog_row-master.opt mysql-test/t/mysqlbinlog_row_innodb-master.opt mysql-test/t/mysqlbinlog_row_myisam-master.opt mysql-test/t/mysqlbinlog_row_trans-master.opt modified: mysql-test/mysql-test-run.pl mysql-test/suite/maria/r/maria-recover.result mysql-test/suite/maria/t/maria-recover.test mysql-test/suite/parts/inc/partition_timestamp.inc mysql-test/suite/parts/r/partition_recover_myisam.result mysql-test/suite/parts/t/partition_recover_myisam.test === modified file 'mysql-test/mysql-test-run.pl' --- a/mysql-test/mysql-test-run.pl 2010-01-19 10:36:52 +0000 +++ b/mysql-test/mysql-test-run.pl 2010-01-27 17:41:05 +0000 @@ -3457,6 +3457,8 @@ sub run_testcase ($$) { my $timezone= timezone($tinfo); if ($timezone ne 'DEFAULT') { $ENV{'TZ'}= $timezone; + } else { + delete($ENV{'TZ'}); } mtr_verbose("Setting timezone: $timezone"); === modified file 'mysql-test/suite/maria/r/maria-recover.result' --- a/mysql-test/suite/maria/r/maria-recover.result 2009-02-19 09:01:25 +0000 +++ b/mysql-test/suite/maria/r/maria-recover.result 2010-01-27 17:41:05 +0000 @@ -1,6 +1,3 @@ -call mtr.add_suppression("Checking table: '.\/mysqltest\/t_corrupted2'"); -call mtr.add_suppression("Recovering table: '.\/mysqltest\/t_corrupted2'"); -call mtr.add_suppression("Table '.\/mysqltest\/t_corrupted2' is marked as crashed and should be repaired"); select @@global.maria_recover; @@global.maria_recover BACKUP === modified file 'mysql-test/suite/maria/t/maria-recover.test' --- a/mysql-test/suite/maria/t/maria-recover.test 2009-10-09 08:09:24 +0000 +++ b/mysql-test/suite/maria/t/maria-recover.test 2010-01-27 17:41:05 +0000 @@ -2,9 +2,13 @@ --source include/have_maria.inc -call mtr.add_suppression("Checking table: '.\/mysqltest\/t_corrupted2'"); -call mtr.add_suppression("Recovering table: '.\/mysqltest\/t_corrupted2'"); -call mtr.add_suppression("Table '.\/mysqltest\/t_corrupted2' is marked as crashed and should be repaired"); +--disable_query_log +# Note: \\. matches a single period. We use '.' as directory separator to +# account for Unix and Windows variation. +call mtr.add_suppression("Checking table: '\\..mysqltest.t_corrupted2'"); +call mtr.add_suppression("Recovering table: '\\..mysqltest.t_corrupted2'"); +call mtr.add_suppression("Table '\\..mysqltest.t_corrupted2' is marked as crashed and should be repaired"); +--enable_query_log # Note: we're setting an environment variable (not prefixing it by $), # so that the perl code below can access it. === modified file 'mysql-test/suite/parts/inc/partition_timestamp.inc' --- a/mysql-test/suite/parts/inc/partition_timestamp.inc 2009-10-28 07:52:34 +0000 +++ b/mysql-test/suite/parts/inc/partition_timestamp.inc 2010-01-27 17:41:05 +0000 @@ -48,6 +48,7 @@ show create table t3; let $count=12; --echo $count inserts; --disable_query_log +SET TIME_ZONE= '+03:00'; begin; while ($count) { === modified file 'mysql-test/suite/parts/r/partition_recover_myisam.result' --- a/mysql-test/suite/parts/r/partition_recover_myisam.result 2009-08-29 21:29:47 +0000 +++ b/mysql-test/suite/parts/r/partition_recover_myisam.result 2010-01-27 17:41:05 +0000 @@ -1,5 +1,3 @@ -call mtr.add_suppression("./test/t1_will_crash"); -call mtr.add_suppression("Got an error from unknown thread, ha_myisam.cc"); CREATE TABLE t1_will_crash (a INT, KEY (a)) ENGINE=MyISAM; INSERT INTO t1_will_crash VALUES (1), (2), (3), (4), (5), (6), (7), (8), (9), (10), (11); FLUSH TABLES; === modified file 'mysql-test/suite/parts/t/partition_recover_myisam.test' --- a/mysql-test/suite/parts/t/partition_recover_myisam.test 2009-12-03 11:19:05 +0000 +++ b/mysql-test/suite/parts/t/partition_recover_myisam.test 2010-01-27 17:41:05 +0000 @@ -1,7 +1,9 @@ # test the auto-recover (--myisam-recover) of partitioned myisam tables -call mtr.add_suppression("./test/t1_will_crash"); +--disable_query_log +call mtr.add_suppression("..test.t1_will_crash"); call mtr.add_suppression("Got an error from unknown thread, ha_myisam.cc"); +--enable_query_log --source include/have_partition.inc --disable_warnings === added file 'mysql-test/t/mysqlbinlog_row-master.opt' --- a/mysql-test/t/mysqlbinlog_row-master.opt 1970-01-01 00:00:00 +0000 +++ b/mysql-test/t/mysqlbinlog_row-master.opt 2010-01-27 17:41:05 +0000 @@ -0,0 +1 @@ +--timezone=GMT-3 === added file 'mysql-test/t/mysqlbinlog_row_innodb-master.opt' --- a/mysql-test/t/mysqlbinlog_row_innodb-master.opt 1970-01-01 00:00:00 +0000 +++ b/mysql-test/t/mysqlbinlog_row_innodb-master.opt 2010-01-27 17:41:05 +0000 @@ -0,0 +1 @@ +--timezone=GMT-3 === added file 'mysql-test/t/mysqlbinlog_row_myisam-master.opt' --- a/mysql-test/t/mysqlbinlog_row_myisam-master.opt 1970-01-01 00:00:00 +0000 +++ b/mysql-test/t/mysqlbinlog_row_myisam-master.opt 2010-01-27 17:41:05 +0000 @@ -0,0 +1 @@ +--timezone=GMT-3 === added file 'mysql-test/t/mysqlbinlog_row_trans-master.opt' --- a/mysql-test/t/mysqlbinlog_row_trans-master.opt 1970-01-01 00:00:00 +0000 +++ b/mysql-test/t/mysqlbinlog_row_trans-master.opt 2010-01-27 17:41:05 +0000 @@ -0,0 +1 @@ +--timezone=GMT-3

1 0

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (knielsen:2803)
by knielsen＠knielsen-hq.org 27 Jan '10

27 Jan '10

#At lp:maria 2803 knielsen(a)knielsen-hq.org 2010-01-27 Fix test failures due to previous change of not setting TZ by default. Fix by explicitly setting timezone for a few more tests that need it. (We avoid setting TZ everywhere by default as this breaks some tests on windows). Also add fix of two other windows failures due to non-portable suppressions, thanks to Alex Budovski. added: mysql-test/t/mysqlbinlog_row-master.opt mysql-test/t/mysqlbinlog_row_innodb-master.opt mysql-test/t/mysqlbinlog_row_myisam-master.opt mysql-test/t/mysqlbinlog_row_trans-master.opt modified: mysql-test/mysql-test-run.pl mysql-test/suite/maria/r/maria-recover.result mysql-test/suite/maria/t/maria-recover.test mysql-test/suite/parts/inc/partition_timestamp.inc mysql-test/suite/parts/r/partition_recover_myisam.result mysql-test/suite/parts/t/partition_recover_myisam.test per-file messages: mysql-test/mysql-test-run.pl Don't let --timezone option from one test stray into the next test (mostly to make results more deterministic rather than depend on whatever test happens to run just before). mysql-test/suite/maria/r/maria-recover.result Fix suppression pattern to also work with Windows \ path separator. mysql-test/suite/maria/t/maria-recover.test Fix suppression pattern to also work with Windows \ path separator. mysql-test/suite/parts/inc/partition_timestamp.inc Set timezone explicitly for test that needs it. mysql-test/suite/parts/r/partition_recover_myisam.result Fix suppression pattern to also work with Windows \ path separator. mysql-test/suite/parts/t/partition_recover_myisam.test Fix suppression pattern to also work with Windows \ path separator. mysql-test/t/mysqlbinlog_row-master.opt Set timezone explicitly for test that needs it. mysql-test/t/mysqlbinlog_row_innodb-master.opt Set timezone explicitly for test that needs it. mysql-test/t/mysqlbinlog_row_myisam-master.opt Set timezone explicitly for test that needs it. mysql-test/t/mysqlbinlog_row_trans-master.opt Set timezone explicitly for test that needs it. === modified file 'mysql-test/mysql-test-run.pl' --- a/mysql-test/mysql-test-run.pl 2010-01-19 10:36:52 +0000 +++ b/mysql-test/mysql-test-run.pl 2010-01-27 17:41:05 +0000 @@ -3457,6 +3457,8 @@ sub run_testcase ($$) { my $timezone= timezone($tinfo); if ($timezone ne 'DEFAULT') { $ENV{'TZ'}= $timezone; + } else { + delete($ENV{'TZ'}); } mtr_verbose("Setting timezone: $timezone"); === modified file 'mysql-test/suite/maria/r/maria-recover.result' --- a/mysql-test/suite/maria/r/maria-recover.result 2009-02-19 09:01:25 +0000 +++ b/mysql-test/suite/maria/r/maria-recover.result 2010-01-27 17:41:05 +0000 @@ -1,6 +1,3 @@ -call mtr.add_suppression("Checking table: '.\/mysqltest\/t_corrupted2'"); -call mtr.add_suppression("Recovering table: '.\/mysqltest\/t_corrupted2'"); -call mtr.add_suppression("Table '.\/mysqltest\/t_corrupted2' is marked as crashed and should be repaired"); select @@global.maria_recover; @@global.maria_recover BACKUP === modified file 'mysql-test/suite/maria/t/maria-recover.test' --- a/mysql-test/suite/maria/t/maria-recover.test 2009-10-09 08:09:24 +0000 +++ b/mysql-test/suite/maria/t/maria-recover.test 2010-01-27 17:41:05 +0000 @@ -2,9 +2,13 @@ --source include/have_maria.inc -call mtr.add_suppression("Checking table: '.\/mysqltest\/t_corrupted2'"); -call mtr.add_suppression("Recovering table: '.\/mysqltest\/t_corrupted2'"); -call mtr.add_suppression("Table '.\/mysqltest\/t_corrupted2' is marked as crashed and should be repaired"); +--disable_query_log +# Note: \\. matches a single period. We use '.' as directory separator to +# account for Unix and Windows variation. +call mtr.add_suppression("Checking table: '\\..mysqltest.t_corrupted2'"); +call mtr.add_suppression("Recovering table: '\\..mysqltest.t_corrupted2'"); +call mtr.add_suppression("Table '\\..mysqltest.t_corrupted2' is marked as crashed and should be repaired"); +--enable_query_log # Note: we're setting an environment variable (not prefixing it by $), # so that the perl code below can access it. === modified file 'mysql-test/suite/parts/inc/partition_timestamp.inc' --- a/mysql-test/suite/parts/inc/partition_timestamp.inc 2009-10-28 07:52:34 +0000 +++ b/mysql-test/suite/parts/inc/partition_timestamp.inc 2010-01-27 17:41:05 +0000 @@ -48,6 +48,7 @@ show create table t3; let $count=12; --echo $count inserts; --disable_query_log +SET TIME_ZONE= '+03:00'; begin; while ($count) { === modified file 'mysql-test/suite/parts/r/partition_recover_myisam.result' --- a/mysql-test/suite/parts/r/partition_recover_myisam.result 2009-08-29 21:29:47 +0000 +++ b/mysql-test/suite/parts/r/partition_recover_myisam.result 2010-01-27 17:41:05 +0000 @@ -1,5 +1,3 @@ -call mtr.add_suppression("./test/t1_will_crash"); -call mtr.add_suppression("Got an error from unknown thread, ha_myisam.cc"); CREATE TABLE t1_will_crash (a INT, KEY (a)) ENGINE=MyISAM; INSERT INTO t1_will_crash VALUES (1), (2), (3), (4), (5), (6), (7), (8), (9), (10), (11); FLUSH TABLES; === modified file 'mysql-test/suite/parts/t/partition_recover_myisam.test' --- a/mysql-test/suite/parts/t/partition_recover_myisam.test 2009-12-03 11:19:05 +0000 +++ b/mysql-test/suite/parts/t/partition_recover_myisam.test 2010-01-27 17:41:05 +0000 @@ -1,7 +1,9 @@ # test the auto-recover (--myisam-recover) of partitioned myisam tables -call mtr.add_suppression("./test/t1_will_crash"); +--disable_query_log +call mtr.add_suppression("..test.t1_will_crash"); call mtr.add_suppression("Got an error from unknown thread, ha_myisam.cc"); +--enable_query_log --source include/have_partition.inc --disable_warnings === added file 'mysql-test/t/mysqlbinlog_row-master.opt' --- a/mysql-test/t/mysqlbinlog_row-master.opt 1970-01-01 00:00:00 +0000 +++ b/mysql-test/t/mysqlbinlog_row-master.opt 2010-01-27 17:41:05 +0000 @@ -0,0 +1 @@ +--timezone=GMT-3 === added file 'mysql-test/t/mysqlbinlog_row_innodb-master.opt' --- a/mysql-test/t/mysqlbinlog_row_innodb-master.opt 1970-01-01 00:00:00 +0000 +++ b/mysql-test/t/mysqlbinlog_row_innodb-master.opt 2010-01-27 17:41:05 +0000 @@ -0,0 +1 @@ +--timezone=GMT-3 === added file 'mysql-test/t/mysqlbinlog_row_myisam-master.opt' --- a/mysql-test/t/mysqlbinlog_row_myisam-master.opt 1970-01-01 00:00:00 +0000 +++ b/mysql-test/t/mysqlbinlog_row_myisam-master.opt 2010-01-27 17:41:05 +0000 @@ -0,0 +1 @@ +--timezone=GMT-3 === added file 'mysql-test/t/mysqlbinlog_row_trans-master.opt' --- a/mysql-test/t/mysqlbinlog_row_trans-master.opt 1970-01-01 00:00:00 +0000 +++ b/mysql-test/t/mysqlbinlog_row_trans-master.opt 2010-01-27 17:41:05 +0000 @@ -0,0 +1 @@ +--timezone=GMT-3

1 0

[Maria-developers] [Branch ~maria-captains/maria/5.1] Rev 2803: YaSSL cert info buffer overflow fix
by noreply＠launchpad.net 27 Jan '10

27 Jan '10

------------------------------------------------------------ revno: 2803 committer: Sergei Golubchik <sergii(a)pisem.net> branch nick: maria-5.1 timestamp: Wed 2010-01-27 11:38:29 +0100 message: YaSSL cert info buffer overflow fix Apply a diff from [Yassl-commit] yassl/taocrypt/src asn.cpp,1.13,1.14 [Yassl-commit] yassl/taocrypt/include asn.hpp,1.9,1.10 Original patch http://lists.mysql.com/commits/96697 modified: extra/yassl/taocrypt/include/asn.hpp extra/yassl/taocrypt/src/asn.cpp -- lp:maria https://code.launchpad.net/~maria-captains/maria/5.1 Your team Maria developers is subscribed to branch lp:maria. To unsubscribe from this branch go to https://code.launchpad.net/~maria-captains/maria/5.1/+edit-subscription.

1 0

[Maria-developers] [Branch ~maria-captains/maria/5.1] Rev 2802: Fix for https://bugs.launchpad.net/maria/+bug/509795
by noreply＠launchpad.net 26 Jan '10

26 Jan '10

------------------------------------------------------------ revno: 2802 committer: Hakan Kuecuekyilmaz <hakan(a)askmonty.org> branch nick: maria timestamp: Fri 2010-01-22 22:19:21 +0100 message: Fix for https://bugs.launchpad.net/maria/+bug/509795 Result of reverse_lookup("localhost") is system depended. Therefore we disable the result of it. modified: mysql-test/r/udf.result mysql-test/t/udf.test -- lp:maria https://code.launchpad.net/~maria-captains/maria/5.1 Your team Maria developers is subscribed to branch lp:maria. To unsubscribe from this branch go to https://code.launchpad.net/~maria-captains/maria/5.1/+edit-subscription.

1 0

[Maria-developers] BuildBot Status Checker
by Adam M. Dutko 26 Jan '10

26 Jan '10

I realized my buildbots went offline earlier today but wasn't notified when they did. I've worked to rectify that situation by writing a small PHP script. The lastest commit is at http://git.thefuturegrid.org/?p=mariadb-scripts;a=commit;h=3d26dd7d842e2e5a… . It currently does the following: $ php checkStatus.php -h Welcome to the MariaDB BuildBot Checker: usage: php checkStatus.php ${URL} ${BUILDBOT} (show a bots info) php checkStatus.php ${URL} 1 (show all bots) note: A local file can be used in place of a URL. You can check individual buildbot info: $ php checkStatus.php http://askmonty.org/buildbot/buildslaves adutko-centos5-amd64 adutko-centos5-amd64: -------------- Used by Builders: centos5-amd64-minimal Slave is currently connected Admin: Kristian Nielsen Last heard from: about 34 minutes ago (2010-Jan-12 05:22:32) or get a list of all buildbots: $ php checkStatus.php http://askmonty.org/buildbot/buildslaves 1 All buildbots ... FOUND BOT NAME --> adutko-centos5-amd64 FOUND BOT NAME --> adutko-ultrasparc3 FOUND BOT NAME --> archivist-amd64 FOUND BOT NAME --> archivist-cnc FOUND BOT NAME --> bo-win-1 FOUND BOT NAME --> centos5 FOUND BOT NAME --> hakan-mac-g5 FOUND BOT NAME --> knielsen-kvm-x86 FOUND BOT NAME --> mariadb-brs FOUND BOT NAME --> opensolaris-x86 FOUND BOT NAME --> pcrews-win FOUND BOT NAME --> psergey-foxhole-x86 FOUND BOT NAME --> psergey-pslp2-x86 FOUND BOT NAME --> psergey-pylon-amd64 FOUND BOT NAME --> psergey-pylon-x86 FOUND BOT NAME --> psergey-win32box FOUND BOT NAME --> sanja-gentoo-x86_64 FOUND BOT NAME --> sanja-mac-x86 FOUND BOT NAME --> toby-sol10 FOUND BOT NAME --> toby-sol10-sparc FOUND BOT NAME --> valgrind FOUND BOT NAME --> vm-win-1 FOUND BOT NAME --> work-opensuse-amd64 You can also use a static file of the page you want to crawl as opposed to an online URL: $ php checkStatus.php file://buildslaves.htm adutko-centos5-amd64 adutko-centos5-amd64: -------------- Used by Builders: centos5-amd64-minimal Slave is currently connected Admin: Kristian Nielsen Last heard from: 0 seconds ago (2010-Jan-12 03:27:32) I am going to try to add an e-mail notification piece so I can be notified when a buildbot goes offline so "ideally" one could minimize downtime. I hope someone else finds this useful. -Adam

2 2

[Maria-developers] Patch for Windows problem with setting TZ
by Kristian Nielsen 26 Jan '10

26 Jan '10

Hi, Alex Budovski has done some work to debug why some tests (binlog tests in particular) are failing on Windows, and found that this is because mysql-test-run.pl set the TZ variable. This interacts poorly with the Windows Operating System. It's actually only a few test cases that need to set TZ. Here is a quick patch that avoids the need to set TZ except for tests that specifically requests it (with the already existing --timezone=XXX mechanism in -master.opt). Opinions? - Kristian.

3 6

[Maria-developers] Rev 2802: Fix for https://bugs.launchpad.net/maria/+bug/509795 in file:///Users/hakan/work/monty_program/maria/
by Hakan Kuecuekyilmaz 22 Jan '10

22 Jan '10

At file:///Users/hakan/work/monty_program/maria/ ------------------------------------------------------------ revno: 2802 revision-id: hakan(a)askmonty.org-20100122211921-e177xzarwy7l8j5y parent: knielsen(a)knielsen-hq.org-20100118125610-s55p0dsjofl9eclf committer: Hakan Kuecuekyilmaz <hakan(a)askmonty.org> branch nick: maria timestamp: Fri 2010-01-22 22:19:21 +0100 message: Fix for https://bugs.launchpad.net/maria/+bug/509795 Result of reverse_lookup("localhost") is system depended. Therefore we disable the result of it. === modified file 'mysql-test/r/udf.result' --- a/mysql-test/r/udf.result 2010-01-11 13:15:28 +0000 +++ b/mysql-test/r/udf.result 2010-01-22 21:19:21 +0000 @@ -38,8 +38,6 @@ select reverse_lookup("127.0.0.1"); select reverse_lookup(127,0,0,1); select reverse_lookup("localhost"); -reverse_lookup("localhost") -NULL select avgcost(); ERROR HY000: Can't initialize function 'avgcost'; wrong number of arguments: AVGCOST() requires two arguments select avgcost(100,23.76); === modified file 'mysql-test/t/udf.test' --- a/mysql-test/t/udf.test 2010-01-11 13:15:28 +0000 +++ b/mysql-test/t/udf.test 2010-01-22 21:19:21 +0000 @@ -56,9 +56,9 @@ --disable_result_log select reverse_lookup("127.0.0.1"); select reverse_lookup(127,0,0,1); +select reverse_lookup("localhost"); --enable_result_log -select reverse_lookup("localhost"); --error ER_CANT_INITIALIZE_UDF select avgcost(); --error ER_CANT_INITIALIZE_UDF

1 0

[Maria-developers] Updated (by Knielsen): Add support for google protocol buffers (34)
by worklog-noreply＠askmonty.org 22 Jan '10

22 Jan '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Add support for google protocol buffers CREATION DATE..: Tue, 21 Jul 2009, 21:11 SUPERVISOR.....: Monty IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Server-Sprint TASK ID........: 34 (http://askmonty.org/worklog/?tid=34) VERSION........: WorkLog-3.4 STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Knielsen - Fri, 22 Jan 2010, 11:38)=-=- Low Level Design modified. --- /tmp/wklog.34.old.29965 2010-01-22 11:38:57.000000000 +0200 +++ /tmp/wklog.34.new.29965 2010-01-22 11:38:57.000000000 +0200 @@ -2,3 +2,12 @@ and a parser for text form of .proto file which then exposes the parsed file via standard GPB message navigation API. +* We should have both server-side support and client-side support (client side + means functions in libmysqlclient so that user can select the full BLOB and + extract fields in the application). + +* Add some kind of header to the GPB blob to support versioning and future + extensibility. + +* Add complete syntax description (update, add, drop, exists, ...). + -=-=(Psergey - Tue, 21 Jul 2009, 21:13)=-=- Low Level Design modified. --- /tmp/wklog.34.old.6462 2009-07-21 21:13:13.000000000 +0300 +++ /tmp/wklog.34.new.6462 2009-07-21 21:13:13.000000000 +0300 @@ -1 +1,4 @@ +* GPB tarball contains a protocol definition for .proto file structure itself + and a parser for text form of .proto file which then exposes the parsed + file via standard GPB message navigation API. -=-=(Psergey - Tue, 21 Jul 2009, 21:12)=-=- High-Level Specification modified. --- /tmp/wklog.34.old.6399 2009-07-21 21:12:23.000000000 +0300 +++ /tmp/wklog.34.new.6399 2009-07-21 21:12:23.000000000 +0300 @@ -1 +1,78 @@ +<contents> +1. GPB Encoding overview +2. GPB in an SQL database +2.1 Informing server about GPB field names and types +2.2 Addressing GPB fields +2.2.1 Option1: SQL Function +2.2.2 Option2: SQL columns +</contents> + + +1. GPB Encoding overview +======================== + +GBB is a compact encoding for structured and typed data. A unit of GPB data +(it is called message) is only partially self-describing: it's possible to +iterate over its parts, but, quoting the spec + +http://code.google.com/apis/protocolbuffers/docs/encoding.html: + " the name and declared type for each field can only be determined on the + decoding end by referencing the message type's definition (i.e. the .proto + file). " + +2. GPB in an SQL database +========================= + +It is possible to store GPB data in MariaDB today - one can declare a binary +blob column and use it to store GPB messages. Storing and retrieving entire +messages will be the only available operations, though, as the server has no +idea about the GPB format. +It is apparent that ability to peek inside GPB data from SQL layer would be of +great advantage: one would be able to +- select only certain fields or parts of GPB messages +- filter records based on the values of GPB fields +- etc +performing such operations at SQL layer will allow to reduce client<->server +traffic right away, and will open path to getting the best possible +performance. + +2.1 Informing server about GPB field names and types +---------------------------------------------------- +User-friendly/meaningful access to GPB fields requires knowledge of GPB field +names and types, which are not available from GPB message itself (see "GPB +encoding overview" section). + +So the first issue to be addressed is to get the server to know the definition +of stored messages. We intend to assume that all records have GPB messages +that conform to a certain single definition, which gives one definition per +GPB field. + +DecisionToMake: How to pass the server the GPB definition? +First idea: add a CREATE TABLE parameter which will specify either the +definition itself or path to .proto file with the definition. + +2.2 Addressing GPB fields +------------------------- +We'll need to provide a way to access GPB fields. This can be complicated as +structures that are encoded in GPB message can be nested and recursive. + +2.2.1 Option1: SQL Function +~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Introduce an SQL function GPB_FIELD(path) which will return contents of the +field. +- Return type of the function will be determined from GPB message definition. +- For path, we can use XPath selector (a subset of XPath) syntax. + +(TODO ^ the above needs to be specified in more detail. is the selector as +simple as filesystem path or we allow quantifiers (with predicates?)?) + +2.2.2 Option2: SQL columns +~~~~~~~~~~~~~~~~~~~~~~~~~~ +Make GPB columns to be accessible as SQL columns. +This approach has problems: +- It might be hard to implement code-wise + - (TODO will Virtual columns patch help??) +- It is not clear how to access fields from nested structures. Should we allow + quoted names like `foo/bar[2]/baz' ? + DESCRIPTION: Add support for Google Protocol Buffers (further GPB). It should be possible to have columns that store GPB-encoded data, as well as use SQL constructs to extract parts of GPB data for use in select list, for filtering, and so forth. Any support for indexing GPB data is outside of scope of this WL entry. HIGH-LEVEL SPECIFICATION: <contents> 1. GPB Encoding overview 2. GPB in an SQL database 2.1 Informing server about GPB field names and types 2.2 Addressing GPB fields 2.2.1 Option1: SQL Function 2.2.2 Option2: SQL columns </contents> 1. GPB Encoding overview ======================== GBB is a compact encoding for structured and typed data. A unit of GPB data (it is called message) is only partially self-describing: it's possible to iterate over its parts, but, quoting the spec http://code.google.com/apis/protocolbuffers/docs/encoding.html: " the name and declared type for each field can only be determined on the decoding end by referencing the message type's definition (i.e. the .proto file). " 2. GPB in an SQL database ========================= It is possible to store GPB data in MariaDB today - one can declare a binary blob column and use it to store GPB messages. Storing and retrieving entire messages will be the only available operations, though, as the server has no idea about the GPB format. It is apparent that ability to peek inside GPB data from SQL layer would be of great advantage: one would be able to - select only certain fields or parts of GPB messages - filter records based on the values of GPB fields - etc performing such operations at SQL layer will allow to reduce client<->server traffic right away, and will open path to getting the best possible performance. 2.1 Informing server about GPB field names and types ---------------------------------------------------- User-friendly/meaningful access to GPB fields requires knowledge of GPB field names and types, which are not available from GPB message itself (see "GPB encoding overview" section). So the first issue to be addressed is to get the server to know the definition of stored messages. We intend to assume that all records have GPB messages that conform to a certain single definition, which gives one definition per GPB field. DecisionToMake: How to pass the server the GPB definition? First idea: add a CREATE TABLE parameter which will specify either the definition itself or path to .proto file with the definition. 2.2 Addressing GPB fields ------------------------- We'll need to provide a way to access GPB fields. This can be complicated as structures that are encoded in GPB message can be nested and recursive. 2.2.1 Option1: SQL Function ~~~~~~~~~~~~~~~~~~~~~~~~~~~ Introduce an SQL function GPB_FIELD(path) which will return contents of the field. - Return type of the function will be determined from GPB message definition. - For path, we can use XPath selector (a subset of XPath) syntax. (TODO ^ the above needs to be specified in more detail. is the selector as simple as filesystem path or we allow quantifiers (with predicates?)?) 2.2.2 Option2: SQL columns ~~~~~~~~~~~~~~~~~~~~~~~~~~ Make GPB columns to be accessible as SQL columns. This approach has problems: - It might be hard to implement code-wise - (TODO will Virtual columns patch help??) - It is not clear how to access fields from nested structures. Should we allow quoted names like `foo/bar[2]/baz' ? LOW-LEVEL DESIGN: * GPB tarball contains a protocol definition for .proto file structure itself and a parser for text form of .proto file which then exposes the parsed file via standard GPB message navigation API. * We should have both server-side support and client-side support (client side means functions in libmysqlclient so that user can select the full BLOB and extract fields in the application). * Add some kind of header to the GPB blob to support versioning and future extensibility. * Add complete syntax description (update, add, drop, exists, ...). ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Knielsen): Add support for google protocol buffers (34)
by worklog-noreply＠askmonty.org 22 Jan '10

22 Jan '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Add support for google protocol buffers CREATION DATE..: Tue, 21 Jul 2009, 21:11 SUPERVISOR.....: Monty IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Server-Sprint TASK ID........: 34 (http://askmonty.org/worklog/?tid=34) VERSION........: WorkLog-3.4 STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Knielsen - Fri, 22 Jan 2010, 11:38)=-=- Low Level Design modified. --- /tmp/wklog.34.old.29965 2010-01-22 11:38:57.000000000 +0200 +++ /tmp/wklog.34.new.29965 2010-01-22 11:38:57.000000000 +0200 @@ -2,3 +2,12 @@ and a parser for text form of .proto file which then exposes the parsed file via standard GPB message navigation API. +* We should have both server-side support and client-side support (client side + means functions in libmysqlclient so that user can select the full BLOB and + extract fields in the application). + +* Add some kind of header to the GPB blob to support versioning and future + extensibility. + +* Add complete syntax description (update, add, drop, exists, ...). + -=-=(Psergey - Tue, 21 Jul 2009, 21:13)=-=- Low Level Design modified. --- /tmp/wklog.34.old.6462 2009-07-21 21:13:13.000000000 +0300 +++ /tmp/wklog.34.new.6462 2009-07-21 21:13:13.000000000 +0300 @@ -1 +1,4 @@ +* GPB tarball contains a protocol definition for .proto file structure itself + and a parser for text form of .proto file which then exposes the parsed + file via standard GPB message navigation API. -=-=(Psergey - Tue, 21 Jul 2009, 21:12)=-=- High-Level Specification modified. --- /tmp/wklog.34.old.6399 2009-07-21 21:12:23.000000000 +0300 +++ /tmp/wklog.34.new.6399 2009-07-21 21:12:23.000000000 +0300 @@ -1 +1,78 @@ +<contents> +1. GPB Encoding overview +2. GPB in an SQL database +2.1 Informing server about GPB field names and types +2.2 Addressing GPB fields +2.2.1 Option1: SQL Function +2.2.2 Option2: SQL columns +</contents> + + +1. GPB Encoding overview +======================== + +GBB is a compact encoding for structured and typed data. A unit of GPB data +(it is called message) is only partially self-describing: it's possible to +iterate over its parts, but, quoting the spec + +http://code.google.com/apis/protocolbuffers/docs/encoding.html: + " the name and declared type for each field can only be determined on the + decoding end by referencing the message type's definition (i.e. the .proto + file). " + +2. GPB in an SQL database +========================= + +It is possible to store GPB data in MariaDB today - one can declare a binary +blob column and use it to store GPB messages. Storing and retrieving entire +messages will be the only available operations, though, as the server has no +idea about the GPB format. +It is apparent that ability to peek inside GPB data from SQL layer would be of +great advantage: one would be able to +- select only certain fields or parts of GPB messages +- filter records based on the values of GPB fields +- etc +performing such operations at SQL layer will allow to reduce client<->server +traffic right away, and will open path to getting the best possible +performance. + +2.1 Informing server about GPB field names and types +---------------------------------------------------- +User-friendly/meaningful access to GPB fields requires knowledge of GPB field +names and types, which are not available from GPB message itself (see "GPB +encoding overview" section). + +So the first issue to be addressed is to get the server to know the definition +of stored messages. We intend to assume that all records have GPB messages +that conform to a certain single definition, which gives one definition per +GPB field. + +DecisionToMake: How to pass the server the GPB definition? +First idea: add a CREATE TABLE parameter which will specify either the +definition itself or path to .proto file with the definition. + +2.2 Addressing GPB fields +------------------------- +We'll need to provide a way to access GPB fields. This can be complicated as +structures that are encoded in GPB message can be nested and recursive. + +2.2.1 Option1: SQL Function +~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Introduce an SQL function GPB_FIELD(path) which will return contents of the +field. +- Return type of the function will be determined from GPB message definition. +- For path, we can use XPath selector (a subset of XPath) syntax. + +(TODO ^ the above needs to be specified in more detail. is the selector as +simple as filesystem path or we allow quantifiers (with predicates?)?) + +2.2.2 Option2: SQL columns +~~~~~~~~~~~~~~~~~~~~~~~~~~ +Make GPB columns to be accessible as SQL columns. +This approach has problems: +- It might be hard to implement code-wise + - (TODO will Virtual columns patch help??) +- It is not clear how to access fields from nested structures. Should we allow + quoted names like `foo/bar[2]/baz' ? + DESCRIPTION: Add support for Google Protocol Buffers (further GPB). It should be possible to have columns that store GPB-encoded data, as well as use SQL constructs to extract parts of GPB data for use in select list, for filtering, and so forth. Any support for indexing GPB data is outside of scope of this WL entry. HIGH-LEVEL SPECIFICATION: <contents> 1. GPB Encoding overview 2. GPB in an SQL database 2.1 Informing server about GPB field names and types 2.2 Addressing GPB fields 2.2.1 Option1: SQL Function 2.2.2 Option2: SQL columns </contents> 1. GPB Encoding overview ======================== GBB is a compact encoding for structured and typed data. A unit of GPB data (it is called message) is only partially self-describing: it's possible to iterate over its parts, but, quoting the spec http://code.google.com/apis/protocolbuffers/docs/encoding.html: " the name and declared type for each field can only be determined on the decoding end by referencing the message type's definition (i.e. the .proto file). " 2. GPB in an SQL database ========================= It is possible to store GPB data in MariaDB today - one can declare a binary blob column and use it to store GPB messages. Storing and retrieving entire messages will be the only available operations, though, as the server has no idea about the GPB format. It is apparent that ability to peek inside GPB data from SQL layer would be of great advantage: one would be able to - select only certain fields or parts of GPB messages - filter records based on the values of GPB fields - etc performing such operations at SQL layer will allow to reduce client<->server traffic right away, and will open path to getting the best possible performance. 2.1 Informing server about GPB field names and types ---------------------------------------------------- User-friendly/meaningful access to GPB fields requires knowledge of GPB field names and types, which are not available from GPB message itself (see "GPB encoding overview" section). So the first issue to be addressed is to get the server to know the definition of stored messages. We intend to assume that all records have GPB messages that conform to a certain single definition, which gives one definition per GPB field. DecisionToMake: How to pass the server the GPB definition? First idea: add a CREATE TABLE parameter which will specify either the definition itself or path to .proto file with the definition. 2.2 Addressing GPB fields ------------------------- We'll need to provide a way to access GPB fields. This can be complicated as structures that are encoded in GPB message can be nested and recursive. 2.2.1 Option1: SQL Function ~~~~~~~~~~~~~~~~~~~~~~~~~~~ Introduce an SQL function GPB_FIELD(path) which will return contents of the field. - Return type of the function will be determined from GPB message definition. - For path, we can use XPath selector (a subset of XPath) syntax. (TODO ^ the above needs to be specified in more detail. is the selector as simple as filesystem path or we allow quantifiers (with predicates?)?) 2.2.2 Option2: SQL columns ~~~~~~~~~~~~~~~~~~~~~~~~~~ Make GPB columns to be accessible as SQL columns. This approach has problems: - It might be hard to implement code-wise - (TODO will Virtual columns patch help??) - It is not clear how to access fields from nested structures. Should we allow quoted names like `foo/bar[2]/baz' ? LOW-LEVEL DESIGN: * GPB tarball contains a protocol definition for .proto file structure itself and a parser for text form of .proto file which then exposes the parsed file via standard GPB message navigation API. * We should have both server-side support and client-side support (client side means functions in libmysqlclient so that user can select the full BLOB and extract fields in the application). * Add some kind of header to the GPB blob to support versioning and future extensibility. * Add complete syntax description (update, add, drop, exists, ...). ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] New (by Knielsen): Re-merge PBXT into MariaDB with proper history using bzr merge-into (76)
by worklog-noreply＠askmonty.org 22 Jan '10

22 Jan '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Re-merge PBXT into MariaDB with proper history using bzr merge-into CREATION DATE..: Fri, 22 Jan 2010, 10:03 SUPERVISOR.....: Knielsen IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Server-BackLog TASK ID........: 76 (http://askmonty.org/worklog/?tid=76) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: DESCRIPTION: PBXT is maintained in its own repository separate from MariaDB (as it is also used for MySQL and Drizzle). Merging new PBXT features into MariaDB is currently a manual process where a copy inside the MariaDB tree is manually patched to match the latest version in the real PBXT repository. This is a pity, as it requires extra work, and the PBXT revision history is not available in the MariaDB tree. Bzr actually has a feature to properly merge one repository into another with preservation of full history and bzr-assisted merging. This is used for merging XtraDB into MariaDB. Some information on this here: http://askmonty.org/wiki/index.php/Merging_into_MariaDB#Merging_from_XtraDB… We want to do this for PBXT also. However, this is currently blocked on a bug in bzr: bug: https://bugs.launchpad.net/bzr/+bug/375898 Once this bug is resolved, we would want to re-do the inclusion of PBXT into MariaDB, using `bzr merge-into` and following the procedure in the bug report. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] New (by Knielsen): Extend build to create a shared libmysqld.so library (75)
by worklog-noreply＠askmonty.org 22 Jan '10

22 Jan '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Extend build to create a shared libmysqld.so library CREATION DATE..: Fri, 22 Jan 2010, 09:39 SUPERVISOR.....: Knielsen IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Server-RawIdeaBin TASK ID........: 75 (http://askmonty.org/worklog/?tid=75) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: DESCRIPTION: Currently, the embedded library libmysqld is only built as a static library, libmysqld.a. However, there is also a need for a shared embedded library, libmysqld.so. A good example is Amarok, which is now using MySQL (and by default an embedded libmysqld) for storing the user's music collection. Amarok is designed to load a number of modules as .so plugins, and the code using libmysqld is one such plugin. Any code that is to be loaded as a shared object on Linux must be built with -fPIC (position independent code). Any library used must thus also be built with -fPIC and preferably itself be linked statically. Amarok is a widely used application (end-user desktop). It could be a good leverage for making libmysqld more popular eg. in distros. However, currently the distros need to resort to various hacks to make things work with libmysqld and Amarok. Some links: Fedora patches .spec to add -fPIC in the build, extract all objects from libmysqld.a, and re-link them as libmysqld.so. http://cvs.fedoraproject.org/viewvc/devel/mysql/mysql.spec?r1=1.108&r2=1.109 Gentoo apparently have a patch for MySQL to build libmysqld.so: http://bugs.gentoo.org/attachment.cgi?id=188057 Debian seems to suggest building both libmysqld.a and libmysqld_pic.a, the latter to be used for shared objects linking with embedded server: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=508406#52 It would be nice if the main build system was able to build properly libmysqld.so, so different distros would not need to resort to different hacks to get things working. The normal way to build both .a and .so as far as I know is to use libtool; it will then build each object twice (with and without -fPIC) if needed. This one might want to make optional to reduce build times. It would also need to be done for each storage engine. An alternative would be to build everything with -fPIC. This should probably be optional also, as on some architectures (ELF x86 32-bit), there is some speed penalty for -fPIC. The libmysqld.so would then be made only in this case. Distro packages could do a separate build with -fPIC to make libmysqld.so (to not get -fPIC into the main server code). There is a MySQL bug for this: http://bugs.mysql.com/bug.php?id=39288 ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Psergey): Subquery optimization: Avoid recalculating subquery if external fields values found in subquery cache (66)
by worklog-noreply＠askmonty.org 20 Jan '10

20 Jan '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Subquery optimization: Avoid recalculating subquery if external fields values found in subquery cache CREATION DATE..: Wed, 25 Nov 2009, 22:25 SUPERVISOR.....: Monty IMPLEMENTOR....: Sanja COPIES TO......: CATEGORY.......: Server-BackLog TASK ID........: 66 (http://askmonty.org/worklog/?tid=66) VERSION........: Server-5.2 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Psergey - Wed, 20 Jan 2010, 14:50)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.26873 2010-01-20 14:50:41.000000000 +0200 +++ /tmp/wklog.66.new.26873 2010-01-20 14:50:41.000000000 +0200 @@ -4,7 +4,6 @@ To check/discuss: ----------------- -* Do we put subquery cache on all levels of subqueries or on highest level only * Will there be any means to measure subquery cache hit rate? * MySQL-6.0 has a one-element predicate result cache. It is called "left expression cache", grep for left_expr_cache in sql/item_subselect.* @@ -41,7 +40,12 @@ - subquery_item_result is 'bool' for subquery predicates, and is of some scalar or ROW(scalar1,...scalarN) type for scalar-context subquery. -We dont support cases when outer_expr or correlation_references are blobs. +We don't support cases when outer_expr or correlation_references are blobs. + +All subquery predicates are cached. That is, if one subquery predicate is +located within another, both of them will have caches (one option to reduce +cache memory usage was to use cache only for the upper-most select. we decided +against it). 2. Data structure used for the cache ------------------------------------ -=-=(Psergey - Wed, 20 Jan 2010, 13:07)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.17649 2010-01-20 13:07:07.000000000 +0200 +++ /tmp/wklog.66.new.17649 2010-01-20 13:07:07.000000000 +0200 @@ -3,7 +3,13 @@ To check/discuss: - To put subquery cache on all levels of subqueries or on highest level only. +----------------- +* Do we put subquery cache on all levels of subqueries or on highest level only +* Will there be any means to measure subquery cache hit rate? +* MySQL-6.0 has a one-element predicate result cache. It is called "left + expression cache", grep for left_expr_cache in sql/item_subselect.* + When this WL is merged with 6.0's optimizations, these two caches will + need to be unified somehow. <contents> -=-=(Psergey - Mon, 18 Jan 2010, 16:40)=-=- Low Level Design modified. --- /tmp/wklog.66.old.24899 2010-01-18 16:40:16.000000000 +0200 +++ /tmp/wklog.66.new.24899 2010-01-18 16:40:16.000000000 +0200 @@ -1,3 +1,5 @@ +* Target version: base on mysql-5.2 code + All items on which subquery depend could be collected in st_select_lex::mark_as_dependent (direct of indirect reference?) -=-=(Psergey - Mon, 18 Jan 2010, 16:37)=-=- Low Level Design modified. --- /tmp/wklog.66.old.24586 2010-01-18 16:37:07.000000000 +0200 +++ /tmp/wklog.66.new.24586 2010-01-18 16:37:07.000000000 +0200 @@ -4,6 +4,11 @@ Temporary table index should be created by all fields except result field (TMP_TABLE_PARAM::keyinfo). +How to fill the temptable +------------------------- +Can reuse approach from SJ-Materialization. Its code is in end_sj_materialize() +and is supposed to be quite trivial. + How to make lookups into temptable ---------------------------------- We'll reuse approach used by SJ-Materialization in 6.0. -=-=(Psergey - Mon, 18 Jan 2010, 16:34)=-=- Low Level Design modified. --- /tmp/wklog.66.old.24328 2010-01-18 16:34:19.000000000 +0200 +++ /tmp/wklog.66.new.24328 2010-01-18 16:34:19.000000000 +0200 @@ -32,8 +32,8 @@ Question: or perhaps that is not necessarry? </questionable> -Execution process -~~~~~~~~~~~~~~~~~ +Doing the lookup +~~~~~~~~~~~~~~~~ SJ-Materialization does lookup in sub_select_sjm(), with this code: /* Do index lookup in the materialized table */ @@ -42,4 +42,12 @@ if (res || !sjm->in_equality->val_int()) DBUG_RETURN(NESTED_LOOP_NO_MORE_ROWS); +The code in this WL will use the same approach +Extracting the value of the subquery predicate +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The goal of making the lookup is to get the value of subquery predicate. +This is done by creating an Item_field $I which refers to appropriate +temporary table's field and then subquery_predicate->val_int() will invoke +$I->val_int(), subquery_predicate->val_str() will invoke $I->val_str() and so +forth. -=-=(Psergey - Mon, 18 Jan 2010, 16:23)=-=- Low Level Design modified. --- /tmp/wklog.66.old.23203 2010-01-18 16:23:18.000000000 +0200 +++ /tmp/wklog.66.new.23203 2010-01-18 16:23:18.000000000 +0200 @@ -31,3 +31,15 @@ Question: or perhaps that is not necessarry? </questionable> + +Execution process +~~~~~~~~~~~~~~~~~ +SJ-Materialization does lookup in sub_select_sjm(), with this code: + + /* Do index lookup in the materialized table */ + if ((res= join_read_key2(join_tab, sjm->table, sjm->tab_ref)) == 1) + DBUG_RETURN(NESTED_LOOP_ERROR); /* purecov: inspected */ + if (res || !sjm->in_equality->val_int()) + DBUG_RETURN(NESTED_LOOP_NO_MORE_ROWS); + + -=-=(Psergey - Mon, 18 Jan 2010, 16:22)=-=- Low Level Design modified. --- /tmp/wklog.66.old.23076 2010-01-18 16:22:07.000000000 +0200 +++ /tmp/wklog.66.new.23076 2010-01-18 16:22:07.000000000 +0200 @@ -4,3 +4,30 @@ Temporary table index should be created by all fields except result field (TMP_TABLE_PARAM::keyinfo). +How to make lookups into temptable +---------------------------------- +We'll reuse approach used by SJ-Materialization in 6.0. + +Setup process +~~~~~~~~~~~~~ +Setup is performed in the same way as in setup_sj_materialization(), +see the code that starts these lines: + + /* + Create/initialize everything we will need to index lookups into the + temptable. + */ + +and ends at this line: + + Remove the injected semi-join IN-equalities from join_tab conds. This + +<questionable> +We'll also need to check equalities, i.e. do an equivalent of this: + + if (!(sjm->in_equality= create_subq_in_equalities(thd, sjm, + emb_sj_nest->sj_subq_pred))) + DBUG_RETURN(TRUE); /* purecov: inspected */ + +Question: or perhaps that is not necessarry? +</questionable> -=-=(Psergey - Tue, 12 Jan 2010, 18:39)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.31666 2010-01-12 18:39:43.000000000 +0200 +++ /tmp/wklog.66.new.31666 2010-01-12 18:39:43.000000000 +0200 @@ -4,3 +4,99 @@ To check/discuss: To put subquery cache on all levels of subqueries or on highest level only. + + +<contents> +1. Scope of the task +2. Data structure used for the cache +3. Cache size +4. Interplay with other subquery optimizations +5. User interface +</contents> + +1. Scope of the task +-------------------- +This WL should handle all subquery predicates, i.e. it should handle these +cases: + + outer_expr IN (SELECT correlated_select) + outer_expr $CMP$ ALL/ANY (SELECT correlated_select) + EXISTS (SELECT correlated_select) + scalar-context subquery: (SELECT correlated_select) + +The cache will maintain + + (outer_expr, correlation_references)-> subquery_item_result + +mapping, where +- correlation_references is a list of tablename.column_name that are referred + from the correlated_select but tablename is a table that is ouside the + subquery. +- subquery_item_result is 'bool' for subquery predicates, and is of +some scalar or ROW(scalar1,...scalarN) type for scalar-context subquery. + +We dont support cases when outer_expr or correlation_references are blobs. + +2. Data structure used for the cache +------------------------------------ +There are two data structures available in the codebase that will allow fast +equality lookups: + +1. HASH (mysys/hash.c) tables +2. Temporary tables (the ones that are used for e.g. GROUP BY) + +None of them has any support for element eviction on overflow (using LRU or +some other policy). + +Query cache and MyISAM/Maria's key/page cache ought to support some eviction +mechanism, but code-wise it is not readily reusable, one will need to factor +it out (or copy it). + +We choose to use #2, and not to have any eviction policy. See subsequent +sections for details and reasoning behind the decision. + +3. Cache size +------------- +Typically, a cache has some maximum size and a policy which is used to +select a cache entry for removal when the cache becomes full (e.g. find +and remove the least [recently] used entry) + +For this WL entry we will use a cache of infinite size. The reasoning behind +this is that: +- is is easy to do: we have temporary tables that can grow to arbitrarily + large size while still providing the same insert/lookup interface. +- it suits us: unless the subquery is resolved with one index lookup, + hitting the cache would be many times cheaper than re-running the + subquery, so cache is worth having. + +4. Interplay with other subquery optimizations +---------------------------------------------- +* This WL entry should not care about IN->EXISTS transformation: caching for + IN subquery and result of its conversion to EXISTS would work in the same + way. + +* This optimization is orthogonal to <=>ANY -> MIN/MAX rewrite (it will + work/be useful irrespectively of whether the rewrite has been performed or + not) + +* TODO: compare this with materialization for uncorrelated IN-subqueries. Is + this basically the same? + A: no, it is not: + - IN-Materialization has to perform full materialization before it can + do the first subquery evaluation. This WL's code has almost no startup + costs. + - This optimization has temp.table of (corr_reference, predicate_value), + while IN-materialization will have (corr_reference) only. + +5. User interface +----------------- +* There will be an @@optimizer_switch flag to turn this optimization on and + off (TODO: name of the flag?) + +* TODO: how do we show this in EXPLAIN [EXTENDED]? The most easiest is to + print something in the warning text of EXPLAIN EXTEDED that would indicate + use of cache. + +* temporary table sizing (max size for heap table, whether to use MyISAM or + Maria) will be controlled with common temp.table control variables. + -=-=(Psergey - Mon, 11 Jan 2010, 13:25)=-=- As of today, there is code that - collects outside references - creates a temporary table with index that would allow for fast lookups. there is no code to - fill the temporary table - make lookups into it Reported zero hours worked. Estimate unchanged. -=-=(Sanja - Fri, 11 Dec 2009, 15:09)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.28164 2009-12-11 15:09:15.000000000 +0200 +++ /tmp/wklog.66.new.28164 2009-12-11 15:09:15.000000000 +0200 @@ -3,4 +3,4 @@ To check/discuss: -Are there sens to put subquery cache on all levels of subqueries of on highest. + To put subquery cache on all levels of subqueries or on highest level only. ------------------------------------------------------------ -=-=(View All Progress Notes, 12 total)=-=- http://askmonty.org/worklog/index.pl?tid=66&nolimit=1 DESCRIPTION: Collect all outer items/references (left part of the subquiery and outer references inside the subquery) in key string. Compare the string (which represents certain value set of the references) against values in hash table and return cached result of subquery if the reference values combination has already been used. For example in the following subquery: (L1, L2) IN (SELECT A, B FROM T WHERE T.F1>OTER_FIELD) set of references to look into the subquery cache is (L1, L2, OTER_FIELD). The subquery cache should be implemented as simple LRU connected to the subquery. Size of the subquery cache (in number of results (but maybe in used memory amount)) is limited by session variable (query parameter?). HIGH-LEVEL SPECIFICATION: Attach subquery cache to each Item_subquery. Interface should allow to use hash or temporary table inside. To check/discuss: ----------------- * Will there be any means to measure subquery cache hit rate? * MySQL-6.0 has a one-element predicate result cache. It is called "left expression cache", grep for left_expr_cache in sql/item_subselect.* When this WL is merged with 6.0's optimizations, these two caches will need to be unified somehow. <contents> 1. Scope of the task 2. Data structure used for the cache 3. Cache size 4. Interplay with other subquery optimizations 5. User interface </contents> 1. Scope of the task -------------------- This WL should handle all subquery predicates, i.e. it should handle these cases: outer_expr IN (SELECT correlated_select) outer_expr $CMP$ ALL/ANY (SELECT correlated_select) EXISTS (SELECT correlated_select) scalar-context subquery: (SELECT correlated_select) The cache will maintain (outer_expr, correlation_references)-> subquery_item_result mapping, where - correlation_references is a list of tablename.column_name that are referred from the correlated_select but tablename is a table that is ouside the subquery. - subquery_item_result is 'bool' for subquery predicates, and is of some scalar or ROW(scalar1,...scalarN) type for scalar-context subquery. We don't support cases when outer_expr or correlation_references are blobs. All subquery predicates are cached. That is, if one subquery predicate is located within another, both of them will have caches (one option to reduce cache memory usage was to use cache only for the upper-most select. we decided against it). 2. Data structure used for the cache ------------------------------------ There are two data structures available in the codebase that will allow fast equality lookups: 1. HASH (mysys/hash.c) tables 2. Temporary tables (the ones that are used for e.g. GROUP BY) None of them has any support for element eviction on overflow (using LRU or some other policy). Query cache and MyISAM/Maria's key/page cache ought to support some eviction mechanism, but code-wise it is not readily reusable, one will need to factor it out (or copy it). We choose to use #2, and not to have any eviction policy. See subsequent sections for details and reasoning behind the decision. 3. Cache size ------------- Typically, a cache has some maximum size and a policy which is used to select a cache entry for removal when the cache becomes full (e.g. find and remove the least [recently] used entry) For this WL entry we will use a cache of infinite size. The reasoning behind this is that: - is is easy to do: we have temporary tables that can grow to arbitrarily large size while still providing the same insert/lookup interface. - it suits us: unless the subquery is resolved with one index lookup, hitting the cache would be many times cheaper than re-running the subquery, so cache is worth having. 4. Interplay with other subquery optimizations ---------------------------------------------- * This WL entry should not care about IN->EXISTS transformation: caching for IN subquery and result of its conversion to EXISTS would work in the same way. * This optimization is orthogonal to <=>ANY -> MIN/MAX rewrite (it will work/be useful irrespectively of whether the rewrite has been performed or not) * TODO: compare this with materialization for uncorrelated IN-subqueries. Is this basically the same? A: no, it is not: - IN-Materialization has to perform full materialization before it can do the first subquery evaluation. This WL's code has almost no startup costs. - This optimization has temp.table of (corr_reference, predicate_value), while IN-materialization will have (corr_reference) only. 5. User interface ----------------- * There will be an @@optimizer_switch flag to turn this optimization on and off (TODO: name of the flag?) * TODO: how do we show this in EXPLAIN [EXTENDED]? The most easiest is to print something in the warning text of EXPLAIN EXTEDED that would indicate use of cache. * temporary table sizing (max size for heap table, whether to use MyISAM or Maria) will be controlled with common temp.table control variables. LOW-LEVEL DESIGN: * Target version: base on mysql-5.2 code All items on which subquery depend could be collected in st_select_lex::mark_as_dependent (direct of indirect reference?) Temporary table index should be created by all fields except result field (TMP_TABLE_PARAM::keyinfo). How to fill the temptable ------------------------- Can reuse approach from SJ-Materialization. Its code is in end_sj_materialize() and is supposed to be quite trivial. How to make lookups into temptable ---------------------------------- We'll reuse approach used by SJ-Materialization in 6.0. Setup process ~~~~~~~~~~~~~ Setup is performed in the same way as in setup_sj_materialization(), see the code that starts these lines: /* Create/initialize everything we will need to index lookups into the temptable. */ and ends at this line: Remove the injected semi-join IN-equalities from join_tab conds. This <questionable> We'll also need to check equalities, i.e. do an equivalent of this: if (!(sjm->in_equality= create_subq_in_equalities(thd, sjm, emb_sj_nest->sj_subq_pred))) DBUG_RETURN(TRUE); /* purecov: inspected */ Question: or perhaps that is not necessarry? </questionable> Doing the lookup ~~~~~~~~~~~~~~~~ SJ-Materialization does lookup in sub_select_sjm(), with this code: /* Do index lookup in the materialized table */ if ((res= join_read_key2(join_tab, sjm->table, sjm->tab_ref)) == 1) DBUG_RETURN(NESTED_LOOP_ERROR); /* purecov: inspected */ if (res || !sjm->in_equality->val_int()) DBUG_RETURN(NESTED_LOOP_NO_MORE_ROWS); The code in this WL will use the same approach Extracting the value of the subquery predicate ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The goal of making the lookup is to get the value of subquery predicate. This is done by creating an Item_field $I which refers to appropriate temporary table's field and then subquery_predicate->val_int() will invoke $I->val_int(), subquery_predicate->val_str() will invoke $I->val_str() and so forth. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Psergey): Subquery optimization: Avoid recalculating subquery if external fields values found in subquery cache (66)
by worklog-noreply＠askmonty.org 20 Jan '10

20 Jan '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Subquery optimization: Avoid recalculating subquery if external fields values found in subquery cache CREATION DATE..: Wed, 25 Nov 2009, 22:25 SUPERVISOR.....: Monty IMPLEMENTOR....: Sanja COPIES TO......: CATEGORY.......: Server-BackLog TASK ID........: 66 (http://askmonty.org/worklog/?tid=66) VERSION........: Server-5.2 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Psergey - Wed, 20 Jan 2010, 14:50)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.26873 2010-01-20 14:50:41.000000000 +0200 +++ /tmp/wklog.66.new.26873 2010-01-20 14:50:41.000000000 +0200 @@ -4,7 +4,6 @@ To check/discuss: ----------------- -* Do we put subquery cache on all levels of subqueries or on highest level only * Will there be any means to measure subquery cache hit rate? * MySQL-6.0 has a one-element predicate result cache. It is called "left expression cache", grep for left_expr_cache in sql/item_subselect.* @@ -41,7 +40,12 @@ - subquery_item_result is 'bool' for subquery predicates, and is of some scalar or ROW(scalar1,...scalarN) type for scalar-context subquery. -We dont support cases when outer_expr or correlation_references are blobs. +We don't support cases when outer_expr or correlation_references are blobs. + +All subquery predicates are cached. That is, if one subquery predicate is +located within another, both of them will have caches (one option to reduce +cache memory usage was to use cache only for the upper-most select. we decided +against it). 2. Data structure used for the cache ------------------------------------ -=-=(Psergey - Wed, 20 Jan 2010, 13:07)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.17649 2010-01-20 13:07:07.000000000 +0200 +++ /tmp/wklog.66.new.17649 2010-01-20 13:07:07.000000000 +0200 @@ -3,7 +3,13 @@ To check/discuss: - To put subquery cache on all levels of subqueries or on highest level only. +----------------- +* Do we put subquery cache on all levels of subqueries or on highest level only +* Will there be any means to measure subquery cache hit rate? +* MySQL-6.0 has a one-element predicate result cache. It is called "left + expression cache", grep for left_expr_cache in sql/item_subselect.* + When this WL is merged with 6.0's optimizations, these two caches will + need to be unified somehow. <contents> -=-=(Psergey - Mon, 18 Jan 2010, 16:40)=-=- Low Level Design modified. --- /tmp/wklog.66.old.24899 2010-01-18 16:40:16.000000000 +0200 +++ /tmp/wklog.66.new.24899 2010-01-18 16:40:16.000000000 +0200 @@ -1,3 +1,5 @@ +* Target version: base on mysql-5.2 code + All items on which subquery depend could be collected in st_select_lex::mark_as_dependent (direct of indirect reference?) -=-=(Psergey - Mon, 18 Jan 2010, 16:37)=-=- Low Level Design modified. --- /tmp/wklog.66.old.24586 2010-01-18 16:37:07.000000000 +0200 +++ /tmp/wklog.66.new.24586 2010-01-18 16:37:07.000000000 +0200 @@ -4,6 +4,11 @@ Temporary table index should be created by all fields except result field (TMP_TABLE_PARAM::keyinfo). +How to fill the temptable +------------------------- +Can reuse approach from SJ-Materialization. Its code is in end_sj_materialize() +and is supposed to be quite trivial. + How to make lookups into temptable ---------------------------------- We'll reuse approach used by SJ-Materialization in 6.0. -=-=(Psergey - Mon, 18 Jan 2010, 16:34)=-=- Low Level Design modified. --- /tmp/wklog.66.old.24328 2010-01-18 16:34:19.000000000 +0200 +++ /tmp/wklog.66.new.24328 2010-01-18 16:34:19.000000000 +0200 @@ -32,8 +32,8 @@ Question: or perhaps that is not necessarry? </questionable> -Execution process -~~~~~~~~~~~~~~~~~ +Doing the lookup +~~~~~~~~~~~~~~~~ SJ-Materialization does lookup in sub_select_sjm(), with this code: /* Do index lookup in the materialized table */ @@ -42,4 +42,12 @@ if (res || !sjm->in_equality->val_int()) DBUG_RETURN(NESTED_LOOP_NO_MORE_ROWS); +The code in this WL will use the same approach +Extracting the value of the subquery predicate +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The goal of making the lookup is to get the value of subquery predicate. +This is done by creating an Item_field $I which refers to appropriate +temporary table's field and then subquery_predicate->val_int() will invoke +$I->val_int(), subquery_predicate->val_str() will invoke $I->val_str() and so +forth. -=-=(Psergey - Mon, 18 Jan 2010, 16:23)=-=- Low Level Design modified. --- /tmp/wklog.66.old.23203 2010-01-18 16:23:18.000000000 +0200 +++ /tmp/wklog.66.new.23203 2010-01-18 16:23:18.000000000 +0200 @@ -31,3 +31,15 @@ Question: or perhaps that is not necessarry? </questionable> + +Execution process +~~~~~~~~~~~~~~~~~ +SJ-Materialization does lookup in sub_select_sjm(), with this code: + + /* Do index lookup in the materialized table */ + if ((res= join_read_key2(join_tab, sjm->table, sjm->tab_ref)) == 1) + DBUG_RETURN(NESTED_LOOP_ERROR); /* purecov: inspected */ + if (res || !sjm->in_equality->val_int()) + DBUG_RETURN(NESTED_LOOP_NO_MORE_ROWS); + + -=-=(Psergey - Mon, 18 Jan 2010, 16:22)=-=- Low Level Design modified. --- /tmp/wklog.66.old.23076 2010-01-18 16:22:07.000000000 +0200 +++ /tmp/wklog.66.new.23076 2010-01-18 16:22:07.000000000 +0200 @@ -4,3 +4,30 @@ Temporary table index should be created by all fields except result field (TMP_TABLE_PARAM::keyinfo). +How to make lookups into temptable +---------------------------------- +We'll reuse approach used by SJ-Materialization in 6.0. + +Setup process +~~~~~~~~~~~~~ +Setup is performed in the same way as in setup_sj_materialization(), +see the code that starts these lines: + + /* + Create/initialize everything we will need to index lookups into the + temptable. + */ + +and ends at this line: + + Remove the injected semi-join IN-equalities from join_tab conds. This + +<questionable> +We'll also need to check equalities, i.e. do an equivalent of this: + + if (!(sjm->in_equality= create_subq_in_equalities(thd, sjm, + emb_sj_nest->sj_subq_pred))) + DBUG_RETURN(TRUE); /* purecov: inspected */ + +Question: or perhaps that is not necessarry? +</questionable> -=-=(Psergey - Tue, 12 Jan 2010, 18:39)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.31666 2010-01-12 18:39:43.000000000 +0200 +++ /tmp/wklog.66.new.31666 2010-01-12 18:39:43.000000000 +0200 @@ -4,3 +4,99 @@ To check/discuss: To put subquery cache on all levels of subqueries or on highest level only. + + +<contents> +1. Scope of the task +2. Data structure used for the cache +3. Cache size +4. Interplay with other subquery optimizations +5. User interface +</contents> + +1. Scope of the task +-------------------- +This WL should handle all subquery predicates, i.e. it should handle these +cases: + + outer_expr IN (SELECT correlated_select) + outer_expr $CMP$ ALL/ANY (SELECT correlated_select) + EXISTS (SELECT correlated_select) + scalar-context subquery: (SELECT correlated_select) + +The cache will maintain + + (outer_expr, correlation_references)-> subquery_item_result + +mapping, where +- correlation_references is a list of tablename.column_name that are referred + from the correlated_select but tablename is a table that is ouside the + subquery. +- subquery_item_result is 'bool' for subquery predicates, and is of +some scalar or ROW(scalar1,...scalarN) type for scalar-context subquery. + +We dont support cases when outer_expr or correlation_references are blobs. + +2. Data structure used for the cache +------------------------------------ +There are two data structures available in the codebase that will allow fast +equality lookups: + +1. HASH (mysys/hash.c) tables +2. Temporary tables (the ones that are used for e.g. GROUP BY) + +None of them has any support for element eviction on overflow (using LRU or +some other policy). + +Query cache and MyISAM/Maria's key/page cache ought to support some eviction +mechanism, but code-wise it is not readily reusable, one will need to factor +it out (or copy it). + +We choose to use #2, and not to have any eviction policy. See subsequent +sections for details and reasoning behind the decision. + +3. Cache size +------------- +Typically, a cache has some maximum size and a policy which is used to +select a cache entry for removal when the cache becomes full (e.g. find +and remove the least [recently] used entry) + +For this WL entry we will use a cache of infinite size. The reasoning behind +this is that: +- is is easy to do: we have temporary tables that can grow to arbitrarily + large size while still providing the same insert/lookup interface. +- it suits us: unless the subquery is resolved with one index lookup, + hitting the cache would be many times cheaper than re-running the + subquery, so cache is worth having. + +4. Interplay with other subquery optimizations +---------------------------------------------- +* This WL entry should not care about IN->EXISTS transformation: caching for + IN subquery and result of its conversion to EXISTS would work in the same + way. + +* This optimization is orthogonal to <=>ANY -> MIN/MAX rewrite (it will + work/be useful irrespectively of whether the rewrite has been performed or + not) + +* TODO: compare this with materialization for uncorrelated IN-subqueries. Is + this basically the same? + A: no, it is not: + - IN-Materialization has to perform full materialization before it can + do the first subquery evaluation. This WL's code has almost no startup + costs. + - This optimization has temp.table of (corr_reference, predicate_value), + while IN-materialization will have (corr_reference) only. + +5. User interface +----------------- +* There will be an @@optimizer_switch flag to turn this optimization on and + off (TODO: name of the flag?) + +* TODO: how do we show this in EXPLAIN [EXTENDED]? The most easiest is to + print something in the warning text of EXPLAIN EXTEDED that would indicate + use of cache. + +* temporary table sizing (max size for heap table, whether to use MyISAM or + Maria) will be controlled with common temp.table control variables. + -=-=(Psergey - Mon, 11 Jan 2010, 13:25)=-=- As of today, there is code that - collects outside references - creates a temporary table with index that would allow for fast lookups. there is no code to - fill the temporary table - make lookups into it Reported zero hours worked. Estimate unchanged. -=-=(Sanja - Fri, 11 Dec 2009, 15:09)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.28164 2009-12-11 15:09:15.000000000 +0200 +++ /tmp/wklog.66.new.28164 2009-12-11 15:09:15.000000000 +0200 @@ -3,4 +3,4 @@ To check/discuss: -Are there sens to put subquery cache on all levels of subqueries of on highest. + To put subquery cache on all levels of subqueries or on highest level only. ------------------------------------------------------------ -=-=(View All Progress Notes, 12 total)=-=- http://askmonty.org/worklog/index.pl?tid=66&nolimit=1 DESCRIPTION: Collect all outer items/references (left part of the subquiery and outer references inside the subquery) in key string. Compare the string (which represents certain value set of the references) against values in hash table and return cached result of subquery if the reference values combination has already been used. For example in the following subquery: (L1, L2) IN (SELECT A, B FROM T WHERE T.F1>OTER_FIELD) set of references to look into the subquery cache is (L1, L2, OTER_FIELD). The subquery cache should be implemented as simple LRU connected to the subquery. Size of the subquery cache (in number of results (but maybe in used memory amount)) is limited by session variable (query parameter?). HIGH-LEVEL SPECIFICATION: Attach subquery cache to each Item_subquery. Interface should allow to use hash or temporary table inside. To check/discuss: ----------------- * Will there be any means to measure subquery cache hit rate? * MySQL-6.0 has a one-element predicate result cache. It is called "left expression cache", grep for left_expr_cache in sql/item_subselect.* When this WL is merged with 6.0's optimizations, these two caches will need to be unified somehow. <contents> 1. Scope of the task 2. Data structure used for the cache 3. Cache size 4. Interplay with other subquery optimizations 5. User interface </contents> 1. Scope of the task -------------------- This WL should handle all subquery predicates, i.e. it should handle these cases: outer_expr IN (SELECT correlated_select) outer_expr $CMP$ ALL/ANY (SELECT correlated_select) EXISTS (SELECT correlated_select) scalar-context subquery: (SELECT correlated_select) The cache will maintain (outer_expr, correlation_references)-> subquery_item_result mapping, where - correlation_references is a list of tablename.column_name that are referred from the correlated_select but tablename is a table that is ouside the subquery. - subquery_item_result is 'bool' for subquery predicates, and is of some scalar or ROW(scalar1,...scalarN) type for scalar-context subquery. We don't support cases when outer_expr or correlation_references are blobs. All subquery predicates are cached. That is, if one subquery predicate is located within another, both of them will have caches (one option to reduce cache memory usage was to use cache only for the upper-most select. we decided against it). 2. Data structure used for the cache ------------------------------------ There are two data structures available in the codebase that will allow fast equality lookups: 1. HASH (mysys/hash.c) tables 2. Temporary tables (the ones that are used for e.g. GROUP BY) None of them has any support for element eviction on overflow (using LRU or some other policy). Query cache and MyISAM/Maria's key/page cache ought to support some eviction mechanism, but code-wise it is not readily reusable, one will need to factor it out (or copy it). We choose to use #2, and not to have any eviction policy. See subsequent sections for details and reasoning behind the decision. 3. Cache size ------------- Typically, a cache has some maximum size and a policy which is used to select a cache entry for removal when the cache becomes full (e.g. find and remove the least [recently] used entry) For this WL entry we will use a cache of infinite size. The reasoning behind this is that: - is is easy to do: we have temporary tables that can grow to arbitrarily large size while still providing the same insert/lookup interface. - it suits us: unless the subquery is resolved with one index lookup, hitting the cache would be many times cheaper than re-running the subquery, so cache is worth having. 4. Interplay with other subquery optimizations ---------------------------------------------- * This WL entry should not care about IN->EXISTS transformation: caching for IN subquery and result of its conversion to EXISTS would work in the same way. * This optimization is orthogonal to <=>ANY -> MIN/MAX rewrite (it will work/be useful irrespectively of whether the rewrite has been performed or not) * TODO: compare this with materialization for uncorrelated IN-subqueries. Is this basically the same? A: no, it is not: - IN-Materialization has to perform full materialization before it can do the first subquery evaluation. This WL's code has almost no startup costs. - This optimization has temp.table of (corr_reference, predicate_value), while IN-materialization will have (corr_reference) only. 5. User interface ----------------- * There will be an @@optimizer_switch flag to turn this optimization on and off (TODO: name of the flag?) * TODO: how do we show this in EXPLAIN [EXTENDED]? The most easiest is to print something in the warning text of EXPLAIN EXTEDED that would indicate use of cache. * temporary table sizing (max size for heap table, whether to use MyISAM or Maria) will be controlled with common temp.table control variables. LOW-LEVEL DESIGN: * Target version: base on mysql-5.2 code All items on which subquery depend could be collected in st_select_lex::mark_as_dependent (direct of indirect reference?) Temporary table index should be created by all fields except result field (TMP_TABLE_PARAM::keyinfo). How to fill the temptable ------------------------- Can reuse approach from SJ-Materialization. Its code is in end_sj_materialize() and is supposed to be quite trivial. How to make lookups into temptable ---------------------------------- We'll reuse approach used by SJ-Materialization in 6.0. Setup process ~~~~~~~~~~~~~ Setup is performed in the same way as in setup_sj_materialization(), see the code that starts these lines: /* Create/initialize everything we will need to index lookups into the temptable. */ and ends at this line: Remove the injected semi-join IN-equalities from join_tab conds. This <questionable> We'll also need to check equalities, i.e. do an equivalent of this: if (!(sjm->in_equality= create_subq_in_equalities(thd, sjm, emb_sj_nest->sj_subq_pred))) DBUG_RETURN(TRUE); /* purecov: inspected */ Question: or perhaps that is not necessarry? </questionable> Doing the lookup ~~~~~~~~~~~~~~~~ SJ-Materialization does lookup in sub_select_sjm(), with this code: /* Do index lookup in the materialized table */ if ((res= join_read_key2(join_tab, sjm->table, sjm->tab_ref)) == 1) DBUG_RETURN(NESTED_LOOP_ERROR); /* purecov: inspected */ if (res || !sjm->in_equality->val_int()) DBUG_RETURN(NESTED_LOOP_NO_MORE_ROWS); The code in this WL will use the same approach Extracting the value of the subquery predicate ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The goal of making the lookup is to get the value of subquery predicate. This is done by creating an Item_field $I which refers to appropriate temporary table's field and then subquery_predicate->val_int() will invoke $I->val_int(), subquery_predicate->val_str() will invoke $I->val_str() and so forth. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Psergey): Subquery optimization: Avoid recalculating subquery if external fields values found in subquery cache (66)
by worklog-noreply＠askmonty.org 20 Jan '10

20 Jan '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Subquery optimization: Avoid recalculating subquery if external fields values found in subquery cache CREATION DATE..: Wed, 25 Nov 2009, 22:25 SUPERVISOR.....: Monty IMPLEMENTOR....: Sanja COPIES TO......: CATEGORY.......: Server-BackLog TASK ID........: 66 (http://askmonty.org/worklog/?tid=66) VERSION........: Server-5.2 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Psergey - Wed, 20 Jan 2010, 13:07)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.17649 2010-01-20 13:07:07.000000000 +0200 +++ /tmp/wklog.66.new.17649 2010-01-20 13:07:07.000000000 +0200 @@ -3,7 +3,13 @@ To check/discuss: - To put subquery cache on all levels of subqueries or on highest level only. +----------------- +* Do we put subquery cache on all levels of subqueries or on highest level only +* Will there be any means to measure subquery cache hit rate? +* MySQL-6.0 has a one-element predicate result cache. It is called "left + expression cache", grep for left_expr_cache in sql/item_subselect.* + When this WL is merged with 6.0's optimizations, these two caches will + need to be unified somehow. <contents> -=-=(Psergey - Mon, 18 Jan 2010, 16:40)=-=- Low Level Design modified. --- /tmp/wklog.66.old.24899 2010-01-18 16:40:16.000000000 +0200 +++ /tmp/wklog.66.new.24899 2010-01-18 16:40:16.000000000 +0200 @@ -1,3 +1,5 @@ +* Target version: base on mysql-5.2 code + All items on which subquery depend could be collected in st_select_lex::mark_as_dependent (direct of indirect reference?) -=-=(Psergey - Mon, 18 Jan 2010, 16:37)=-=- Low Level Design modified. --- /tmp/wklog.66.old.24586 2010-01-18 16:37:07.000000000 +0200 +++ /tmp/wklog.66.new.24586 2010-01-18 16:37:07.000000000 +0200 @@ -4,6 +4,11 @@ Temporary table index should be created by all fields except result field (TMP_TABLE_PARAM::keyinfo). +How to fill the temptable +------------------------- +Can reuse approach from SJ-Materialization. Its code is in end_sj_materialize() +and is supposed to be quite trivial. + How to make lookups into temptable ---------------------------------- We'll reuse approach used by SJ-Materialization in 6.0. -=-=(Psergey - Mon, 18 Jan 2010, 16:34)=-=- Low Level Design modified. --- /tmp/wklog.66.old.24328 2010-01-18 16:34:19.000000000 +0200 +++ /tmp/wklog.66.new.24328 2010-01-18 16:34:19.000000000 +0200 @@ -32,8 +32,8 @@ Question: or perhaps that is not necessarry? </questionable> -Execution process -~~~~~~~~~~~~~~~~~ +Doing the lookup +~~~~~~~~~~~~~~~~ SJ-Materialization does lookup in sub_select_sjm(), with this code: /* Do index lookup in the materialized table */ @@ -42,4 +42,12 @@ if (res || !sjm->in_equality->val_int()) DBUG_RETURN(NESTED_LOOP_NO_MORE_ROWS); +The code in this WL will use the same approach +Extracting the value of the subquery predicate +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The goal of making the lookup is to get the value of subquery predicate. +This is done by creating an Item_field $I which refers to appropriate +temporary table's field and then subquery_predicate->val_int() will invoke +$I->val_int(), subquery_predicate->val_str() will invoke $I->val_str() and so +forth. -=-=(Psergey - Mon, 18 Jan 2010, 16:23)=-=- Low Level Design modified. --- /tmp/wklog.66.old.23203 2010-01-18 16:23:18.000000000 +0200 +++ /tmp/wklog.66.new.23203 2010-01-18 16:23:18.000000000 +0200 @@ -31,3 +31,15 @@ Question: or perhaps that is not necessarry? </questionable> + +Execution process +~~~~~~~~~~~~~~~~~ +SJ-Materialization does lookup in sub_select_sjm(), with this code: + + /* Do index lookup in the materialized table */ + if ((res= join_read_key2(join_tab, sjm->table, sjm->tab_ref)) == 1) + DBUG_RETURN(NESTED_LOOP_ERROR); /* purecov: inspected */ + if (res || !sjm->in_equality->val_int()) + DBUG_RETURN(NESTED_LOOP_NO_MORE_ROWS); + + -=-=(Psergey - Mon, 18 Jan 2010, 16:22)=-=- Low Level Design modified. --- /tmp/wklog.66.old.23076 2010-01-18 16:22:07.000000000 +0200 +++ /tmp/wklog.66.new.23076 2010-01-18 16:22:07.000000000 +0200 @@ -4,3 +4,30 @@ Temporary table index should be created by all fields except result field (TMP_TABLE_PARAM::keyinfo). +How to make lookups into temptable +---------------------------------- +We'll reuse approach used by SJ-Materialization in 6.0. + +Setup process +~~~~~~~~~~~~~ +Setup is performed in the same way as in setup_sj_materialization(), +see the code that starts these lines: + + /* + Create/initialize everything we will need to index lookups into the + temptable. + */ + +and ends at this line: + + Remove the injected semi-join IN-equalities from join_tab conds. This + +<questionable> +We'll also need to check equalities, i.e. do an equivalent of this: + + if (!(sjm->in_equality= create_subq_in_equalities(thd, sjm, + emb_sj_nest->sj_subq_pred))) + DBUG_RETURN(TRUE); /* purecov: inspected */ + +Question: or perhaps that is not necessarry? +</questionable> -=-=(Psergey - Tue, 12 Jan 2010, 18:39)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.31666 2010-01-12 18:39:43.000000000 +0200 +++ /tmp/wklog.66.new.31666 2010-01-12 18:39:43.000000000 +0200 @@ -4,3 +4,99 @@ To check/discuss: To put subquery cache on all levels of subqueries or on highest level only. + + +<contents> +1. Scope of the task +2. Data structure used for the cache +3. Cache size +4. Interplay with other subquery optimizations +5. User interface +</contents> + +1. Scope of the task +-------------------- +This WL should handle all subquery predicates, i.e. it should handle these +cases: + + outer_expr IN (SELECT correlated_select) + outer_expr $CMP$ ALL/ANY (SELECT correlated_select) + EXISTS (SELECT correlated_select) + scalar-context subquery: (SELECT correlated_select) + +The cache will maintain + + (outer_expr, correlation_references)-> subquery_item_result + +mapping, where +- correlation_references is a list of tablename.column_name that are referred + from the correlated_select but tablename is a table that is ouside the + subquery. +- subquery_item_result is 'bool' for subquery predicates, and is of +some scalar or ROW(scalar1,...scalarN) type for scalar-context subquery. + +We dont support cases when outer_expr or correlation_references are blobs. + +2. Data structure used for the cache +------------------------------------ +There are two data structures available in the codebase that will allow fast +equality lookups: + +1. HASH (mysys/hash.c) tables +2. Temporary tables (the ones that are used for e.g. GROUP BY) + +None of them has any support for element eviction on overflow (using LRU or +some other policy). + +Query cache and MyISAM/Maria's key/page cache ought to support some eviction +mechanism, but code-wise it is not readily reusable, one will need to factor +it out (or copy it). + +We choose to use #2, and not to have any eviction policy. See subsequent +sections for details and reasoning behind the decision. + +3. Cache size +------------- +Typically, a cache has some maximum size and a policy which is used to +select a cache entry for removal when the cache becomes full (e.g. find +and remove the least [recently] used entry) + +For this WL entry we will use a cache of infinite size. The reasoning behind +this is that: +- is is easy to do: we have temporary tables that can grow to arbitrarily + large size while still providing the same insert/lookup interface. +- it suits us: unless the subquery is resolved with one index lookup, + hitting the cache would be many times cheaper than re-running the + subquery, so cache is worth having. + +4. Interplay with other subquery optimizations +---------------------------------------------- +* This WL entry should not care about IN->EXISTS transformation: caching for + IN subquery and result of its conversion to EXISTS would work in the same + way. + +* This optimization is orthogonal to <=>ANY -> MIN/MAX rewrite (it will + work/be useful irrespectively of whether the rewrite has been performed or + not) + +* TODO: compare this with materialization for uncorrelated IN-subqueries. Is + this basically the same? + A: no, it is not: + - IN-Materialization has to perform full materialization before it can + do the first subquery evaluation. This WL's code has almost no startup + costs. + - This optimization has temp.table of (corr_reference, predicate_value), + while IN-materialization will have (corr_reference) only. + +5. User interface +----------------- +* There will be an @@optimizer_switch flag to turn this optimization on and + off (TODO: name of the flag?) + +* TODO: how do we show this in EXPLAIN [EXTENDED]? The most easiest is to + print something in the warning text of EXPLAIN EXTEDED that would indicate + use of cache. + +* temporary table sizing (max size for heap table, whether to use MyISAM or + Maria) will be controlled with common temp.table control variables. + -=-=(Psergey - Mon, 11 Jan 2010, 13:25)=-=- As of today, there is code that - collects outside references - creates a temporary table with index that would allow for fast lookups. there is no code to - fill the temporary table - make lookups into it Reported zero hours worked. Estimate unchanged. -=-=(Sanja - Fri, 11 Dec 2009, 15:09)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.28164 2009-12-11 15:09:15.000000000 +0200 +++ /tmp/wklog.66.new.28164 2009-12-11 15:09:15.000000000 +0200 @@ -3,4 +3,4 @@ To check/discuss: -Are there sens to put subquery cache on all levels of subqueries of on highest. + To put subquery cache on all levels of subqueries or on highest level only. -=-=(Sanja - Fri, 11 Dec 2009, 15:08)=-=- Low Level Design modified. --- /tmp/wklog.66.old.28072 2009-12-11 15:08:16.000000000 +0200 +++ /tmp/wklog.66.new.28072 2009-12-11 15:08:16.000000000 +0200 @@ -1 +1,6 @@ +All items on which subquery depend could be collected in +st_select_lex::mark_as_dependent (direct of indirect reference?) + +Temporary table index should be created by all fields except result field +(TMP_TABLE_PARAM::keyinfo). ------------------------------------------------------------ -=-=(View All Progress Notes, 11 total)=-=- http://askmonty.org/worklog/index.pl?tid=66&nolimit=1 DESCRIPTION: Collect all outer items/references (left part of the subquiery and outer references inside the subquery) in key string. Compare the string (which represents certain value set of the references) against values in hash table and return cached result of subquery if the reference values combination has already been used. For example in the following subquery: (L1, L2) IN (SELECT A, B FROM T WHERE T.F1>OTER_FIELD) set of references to look into the subquery cache is (L1, L2, OTER_FIELD). The subquery cache should be implemented as simple LRU connected to the subquery. Size of the subquery cache (in number of results (but maybe in used memory amount)) is limited by session variable (query parameter?). HIGH-LEVEL SPECIFICATION: Attach subquery cache to each Item_subquery. Interface should allow to use hash or temporary table inside. To check/discuss: ----------------- * Do we put subquery cache on all levels of subqueries or on highest level only * Will there be any means to measure subquery cache hit rate? * MySQL-6.0 has a one-element predicate result cache. It is called "left expression cache", grep for left_expr_cache in sql/item_subselect.* When this WL is merged with 6.0's optimizations, these two caches will need to be unified somehow. <contents> 1. Scope of the task 2. Data structure used for the cache 3. Cache size 4. Interplay with other subquery optimizations 5. User interface </contents> 1. Scope of the task -------------------- This WL should handle all subquery predicates, i.e. it should handle these cases: outer_expr IN (SELECT correlated_select) outer_expr $CMP$ ALL/ANY (SELECT correlated_select) EXISTS (SELECT correlated_select) scalar-context subquery: (SELECT correlated_select) The cache will maintain (outer_expr, correlation_references)-> subquery_item_result mapping, where - correlation_references is a list of tablename.column_name that are referred from the correlated_select but tablename is a table that is ouside the subquery. - subquery_item_result is 'bool' for subquery predicates, and is of some scalar or ROW(scalar1,...scalarN) type for scalar-context subquery. We dont support cases when outer_expr or correlation_references are blobs. 2. Data structure used for the cache ------------------------------------ There are two data structures available in the codebase that will allow fast equality lookups: 1. HASH (mysys/hash.c) tables 2. Temporary tables (the ones that are used for e.g. GROUP BY) None of them has any support for element eviction on overflow (using LRU or some other policy). Query cache and MyISAM/Maria's key/page cache ought to support some eviction mechanism, but code-wise it is not readily reusable, one will need to factor it out (or copy it). We choose to use #2, and not to have any eviction policy. See subsequent sections for details and reasoning behind the decision. 3. Cache size ------------- Typically, a cache has some maximum size and a policy which is used to select a cache entry for removal when the cache becomes full (e.g. find and remove the least [recently] used entry) For this WL entry we will use a cache of infinite size. The reasoning behind this is that: - is is easy to do: we have temporary tables that can grow to arbitrarily large size while still providing the same insert/lookup interface. - it suits us: unless the subquery is resolved with one index lookup, hitting the cache would be many times cheaper than re-running the subquery, so cache is worth having. 4. Interplay with other subquery optimizations ---------------------------------------------- * This WL entry should not care about IN->EXISTS transformation: caching for IN subquery and result of its conversion to EXISTS would work in the same way. * This optimization is orthogonal to <=>ANY -> MIN/MAX rewrite (it will work/be useful irrespectively of whether the rewrite has been performed or not) * TODO: compare this with materialization for uncorrelated IN-subqueries. Is this basically the same? A: no, it is not: - IN-Materialization has to perform full materialization before it can do the first subquery evaluation. This WL's code has almost no startup costs. - This optimization has temp.table of (corr_reference, predicate_value), while IN-materialization will have (corr_reference) only. 5. User interface ----------------- * There will be an @@optimizer_switch flag to turn this optimization on and off (TODO: name of the flag?) * TODO: how do we show this in EXPLAIN [EXTENDED]? The most easiest is to print something in the warning text of EXPLAIN EXTEDED that would indicate use of cache. * temporary table sizing (max size for heap table, whether to use MyISAM or Maria) will be controlled with common temp.table control variables. LOW-LEVEL DESIGN: * Target version: base on mysql-5.2 code All items on which subquery depend could be collected in st_select_lex::mark_as_dependent (direct of indirect reference?) Temporary table index should be created by all fields except result field (TMP_TABLE_PARAM::keyinfo). How to fill the temptable ------------------------- Can reuse approach from SJ-Materialization. Its code is in end_sj_materialize() and is supposed to be quite trivial. How to make lookups into temptable ---------------------------------- We'll reuse approach used by SJ-Materialization in 6.0. Setup process ~~~~~~~~~~~~~ Setup is performed in the same way as in setup_sj_materialization(), see the code that starts these lines: /* Create/initialize everything we will need to index lookups into the temptable. */ and ends at this line: Remove the injected semi-join IN-equalities from join_tab conds. This <questionable> We'll also need to check equalities, i.e. do an equivalent of this: if (!(sjm->in_equality= create_subq_in_equalities(thd, sjm, emb_sj_nest->sj_subq_pred))) DBUG_RETURN(TRUE); /* purecov: inspected */ Question: or perhaps that is not necessarry? </questionable> Doing the lookup ~~~~~~~~~~~~~~~~ SJ-Materialization does lookup in sub_select_sjm(), with this code: /* Do index lookup in the materialized table */ if ((res= join_read_key2(join_tab, sjm->table, sjm->tab_ref)) == 1) DBUG_RETURN(NESTED_LOOP_ERROR); /* purecov: inspected */ if (res || !sjm->in_equality->val_int()) DBUG_RETURN(NESTED_LOOP_NO_MORE_ROWS); The code in this WL will use the same approach Extracting the value of the subquery predicate ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The goal of making the lookup is to get the value of subquery predicate. This is done by creating an Item_field $I which refers to appropriate temporary table's field and then subquery_predicate->val_int() will invoke $I->val_int(), subquery_predicate->val_str() will invoke $I->val_str() and so forth. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Psergey): Subquery optimization: Avoid recalculating subquery if external fields values found in subquery cache (66)
by worklog-noreply＠askmonty.org 20 Jan '10

20 Jan '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Subquery optimization: Avoid recalculating subquery if external fields values found in subquery cache CREATION DATE..: Wed, 25 Nov 2009, 22:25 SUPERVISOR.....: Monty IMPLEMENTOR....: Sanja COPIES TO......: CATEGORY.......: Server-BackLog TASK ID........: 66 (http://askmonty.org/worklog/?tid=66) VERSION........: Server-5.2 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Psergey - Wed, 20 Jan 2010, 13:07)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.17649 2010-01-20 13:07:07.000000000 +0200 +++ /tmp/wklog.66.new.17649 2010-01-20 13:07:07.000000000 +0200 @@ -3,7 +3,13 @@ To check/discuss: - To put subquery cache on all levels of subqueries or on highest level only. +----------------- +* Do we put subquery cache on all levels of subqueries or on highest level only +* Will there be any means to measure subquery cache hit rate? +* MySQL-6.0 has a one-element predicate result cache. It is called "left + expression cache", grep for left_expr_cache in sql/item_subselect.* + When this WL is merged with 6.0's optimizations, these two caches will + need to be unified somehow. <contents> -=-=(Psergey - Mon, 18 Jan 2010, 16:40)=-=- Low Level Design modified. --- /tmp/wklog.66.old.24899 2010-01-18 16:40:16.000000000 +0200 +++ /tmp/wklog.66.new.24899 2010-01-18 16:40:16.000000000 +0200 @@ -1,3 +1,5 @@ +* Target version: base on mysql-5.2 code + All items on which subquery depend could be collected in st_select_lex::mark_as_dependent (direct of indirect reference?) -=-=(Psergey - Mon, 18 Jan 2010, 16:37)=-=- Low Level Design modified. --- /tmp/wklog.66.old.24586 2010-01-18 16:37:07.000000000 +0200 +++ /tmp/wklog.66.new.24586 2010-01-18 16:37:07.000000000 +0200 @@ -4,6 +4,11 @@ Temporary table index should be created by all fields except result field (TMP_TABLE_PARAM::keyinfo). +How to fill the temptable +------------------------- +Can reuse approach from SJ-Materialization. Its code is in end_sj_materialize() +and is supposed to be quite trivial. + How to make lookups into temptable ---------------------------------- We'll reuse approach used by SJ-Materialization in 6.0. -=-=(Psergey - Mon, 18 Jan 2010, 16:34)=-=- Low Level Design modified. --- /tmp/wklog.66.old.24328 2010-01-18 16:34:19.000000000 +0200 +++ /tmp/wklog.66.new.24328 2010-01-18 16:34:19.000000000 +0200 @@ -32,8 +32,8 @@ Question: or perhaps that is not necessarry? </questionable> -Execution process -~~~~~~~~~~~~~~~~~ +Doing the lookup +~~~~~~~~~~~~~~~~ SJ-Materialization does lookup in sub_select_sjm(), with this code: /* Do index lookup in the materialized table */ @@ -42,4 +42,12 @@ if (res || !sjm->in_equality->val_int()) DBUG_RETURN(NESTED_LOOP_NO_MORE_ROWS); +The code in this WL will use the same approach +Extracting the value of the subquery predicate +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The goal of making the lookup is to get the value of subquery predicate. +This is done by creating an Item_field $I which refers to appropriate +temporary table's field and then subquery_predicate->val_int() will invoke +$I->val_int(), subquery_predicate->val_str() will invoke $I->val_str() and so +forth. -=-=(Psergey - Mon, 18 Jan 2010, 16:23)=-=- Low Level Design modified. --- /tmp/wklog.66.old.23203 2010-01-18 16:23:18.000000000 +0200 +++ /tmp/wklog.66.new.23203 2010-01-18 16:23:18.000000000 +0200 @@ -31,3 +31,15 @@ Question: or perhaps that is not necessarry? </questionable> + +Execution process +~~~~~~~~~~~~~~~~~ +SJ-Materialization does lookup in sub_select_sjm(), with this code: + + /* Do index lookup in the materialized table */ + if ((res= join_read_key2(join_tab, sjm->table, sjm->tab_ref)) == 1) + DBUG_RETURN(NESTED_LOOP_ERROR); /* purecov: inspected */ + if (res || !sjm->in_equality->val_int()) + DBUG_RETURN(NESTED_LOOP_NO_MORE_ROWS); + + -=-=(Psergey - Mon, 18 Jan 2010, 16:22)=-=- Low Level Design modified. --- /tmp/wklog.66.old.23076 2010-01-18 16:22:07.000000000 +0200 +++ /tmp/wklog.66.new.23076 2010-01-18 16:22:07.000000000 +0200 @@ -4,3 +4,30 @@ Temporary table index should be created by all fields except result field (TMP_TABLE_PARAM::keyinfo). +How to make lookups into temptable +---------------------------------- +We'll reuse approach used by SJ-Materialization in 6.0. + +Setup process +~~~~~~~~~~~~~ +Setup is performed in the same way as in setup_sj_materialization(), +see the code that starts these lines: + + /* + Create/initialize everything we will need to index lookups into the + temptable. + */ + +and ends at this line: + + Remove the injected semi-join IN-equalities from join_tab conds. This + +<questionable> +We'll also need to check equalities, i.e. do an equivalent of this: + + if (!(sjm->in_equality= create_subq_in_equalities(thd, sjm, + emb_sj_nest->sj_subq_pred))) + DBUG_RETURN(TRUE); /* purecov: inspected */ + +Question: or perhaps that is not necessarry? +</questionable> -=-=(Psergey - Tue, 12 Jan 2010, 18:39)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.31666 2010-01-12 18:39:43.000000000 +0200 +++ /tmp/wklog.66.new.31666 2010-01-12 18:39:43.000000000 +0200 @@ -4,3 +4,99 @@ To check/discuss: To put subquery cache on all levels of subqueries or on highest level only. + + +<contents> +1. Scope of the task +2. Data structure used for the cache +3. Cache size +4. Interplay with other subquery optimizations +5. User interface +</contents> + +1. Scope of the task +-------------------- +This WL should handle all subquery predicates, i.e. it should handle these +cases: + + outer_expr IN (SELECT correlated_select) + outer_expr $CMP$ ALL/ANY (SELECT correlated_select) + EXISTS (SELECT correlated_select) + scalar-context subquery: (SELECT correlated_select) + +The cache will maintain + + (outer_expr, correlation_references)-> subquery_item_result + +mapping, where +- correlation_references is a list of tablename.column_name that are referred + from the correlated_select but tablename is a table that is ouside the + subquery. +- subquery_item_result is 'bool' for subquery predicates, and is of +some scalar or ROW(scalar1,...scalarN) type for scalar-context subquery. + +We dont support cases when outer_expr or correlation_references are blobs. + +2. Data structure used for the cache +------------------------------------ +There are two data structures available in the codebase that will allow fast +equality lookups: + +1. HASH (mysys/hash.c) tables +2. Temporary tables (the ones that are used for e.g. GROUP BY) + +None of them has any support for element eviction on overflow (using LRU or +some other policy). + +Query cache and MyISAM/Maria's key/page cache ought to support some eviction +mechanism, but code-wise it is not readily reusable, one will need to factor +it out (or copy it). + +We choose to use #2, and not to have any eviction policy. See subsequent +sections for details and reasoning behind the decision. + +3. Cache size +------------- +Typically, a cache has some maximum size and a policy which is used to +select a cache entry for removal when the cache becomes full (e.g. find +and remove the least [recently] used entry) + +For this WL entry we will use a cache of infinite size. The reasoning behind +this is that: +- is is easy to do: we have temporary tables that can grow to arbitrarily + large size while still providing the same insert/lookup interface. +- it suits us: unless the subquery is resolved with one index lookup, + hitting the cache would be many times cheaper than re-running the + subquery, so cache is worth having. + +4. Interplay with other subquery optimizations +---------------------------------------------- +* This WL entry should not care about IN->EXISTS transformation: caching for + IN subquery and result of its conversion to EXISTS would work in the same + way. + +* This optimization is orthogonal to <=>ANY -> MIN/MAX rewrite (it will + work/be useful irrespectively of whether the rewrite has been performed or + not) + +* TODO: compare this with materialization for uncorrelated IN-subqueries. Is + this basically the same? + A: no, it is not: + - IN-Materialization has to perform full materialization before it can + do the first subquery evaluation. This WL's code has almost no startup + costs. + - This optimization has temp.table of (corr_reference, predicate_value), + while IN-materialization will have (corr_reference) only. + +5. User interface +----------------- +* There will be an @@optimizer_switch flag to turn this optimization on and + off (TODO: name of the flag?) + +* TODO: how do we show this in EXPLAIN [EXTENDED]? The most easiest is to + print something in the warning text of EXPLAIN EXTEDED that would indicate + use of cache. + +* temporary table sizing (max size for heap table, whether to use MyISAM or + Maria) will be controlled with common temp.table control variables. + -=-=(Psergey - Mon, 11 Jan 2010, 13:25)=-=- As of today, there is code that - collects outside references - creates a temporary table with index that would allow for fast lookups. there is no code to - fill the temporary table - make lookups into it Reported zero hours worked. Estimate unchanged. -=-=(Sanja - Fri, 11 Dec 2009, 15:09)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.28164 2009-12-11 15:09:15.000000000 +0200 +++ /tmp/wklog.66.new.28164 2009-12-11 15:09:15.000000000 +0200 @@ -3,4 +3,4 @@ To check/discuss: -Are there sens to put subquery cache on all levels of subqueries of on highest. + To put subquery cache on all levels of subqueries or on highest level only. -=-=(Sanja - Fri, 11 Dec 2009, 15:08)=-=- Low Level Design modified. --- /tmp/wklog.66.old.28072 2009-12-11 15:08:16.000000000 +0200 +++ /tmp/wklog.66.new.28072 2009-12-11 15:08:16.000000000 +0200 @@ -1 +1,6 @@ +All items on which subquery depend could be collected in +st_select_lex::mark_as_dependent (direct of indirect reference?) + +Temporary table index should be created by all fields except result field +(TMP_TABLE_PARAM::keyinfo). ------------------------------------------------------------ -=-=(View All Progress Notes, 11 total)=-=- http://askmonty.org/worklog/index.pl?tid=66&nolimit=1 DESCRIPTION: Collect all outer items/references (left part of the subquiery and outer references inside the subquery) in key string. Compare the string (which represents certain value set of the references) against values in hash table and return cached result of subquery if the reference values combination has already been used. For example in the following subquery: (L1, L2) IN (SELECT A, B FROM T WHERE T.F1>OTER_FIELD) set of references to look into the subquery cache is (L1, L2, OTER_FIELD). The subquery cache should be implemented as simple LRU connected to the subquery. Size of the subquery cache (in number of results (but maybe in used memory amount)) is limited by session variable (query parameter?). HIGH-LEVEL SPECIFICATION: Attach subquery cache to each Item_subquery. Interface should allow to use hash or temporary table inside. To check/discuss: ----------------- * Do we put subquery cache on all levels of subqueries or on highest level only * Will there be any means to measure subquery cache hit rate? * MySQL-6.0 has a one-element predicate result cache. It is called "left expression cache", grep for left_expr_cache in sql/item_subselect.* When this WL is merged with 6.0's optimizations, these two caches will need to be unified somehow. <contents> 1. Scope of the task 2. Data structure used for the cache 3. Cache size 4. Interplay with other subquery optimizations 5. User interface </contents> 1. Scope of the task -------------------- This WL should handle all subquery predicates, i.e. it should handle these cases: outer_expr IN (SELECT correlated_select) outer_expr $CMP$ ALL/ANY (SELECT correlated_select) EXISTS (SELECT correlated_select) scalar-context subquery: (SELECT correlated_select) The cache will maintain (outer_expr, correlation_references)-> subquery_item_result mapping, where - correlation_references is a list of tablename.column_name that are referred from the correlated_select but tablename is a table that is ouside the subquery. - subquery_item_result is 'bool' for subquery predicates, and is of some scalar or ROW(scalar1,...scalarN) type for scalar-context subquery. We dont support cases when outer_expr or correlation_references are blobs. 2. Data structure used for the cache ------------------------------------ There are two data structures available in the codebase that will allow fast equality lookups: 1. HASH (mysys/hash.c) tables 2. Temporary tables (the ones that are used for e.g. GROUP BY) None of them has any support for element eviction on overflow (using LRU or some other policy). Query cache and MyISAM/Maria's key/page cache ought to support some eviction mechanism, but code-wise it is not readily reusable, one will need to factor it out (or copy it). We choose to use #2, and not to have any eviction policy. See subsequent sections for details and reasoning behind the decision. 3. Cache size ------------- Typically, a cache has some maximum size and a policy which is used to select a cache entry for removal when the cache becomes full (e.g. find and remove the least [recently] used entry) For this WL entry we will use a cache of infinite size. The reasoning behind this is that: - is is easy to do: we have temporary tables that can grow to arbitrarily large size while still providing the same insert/lookup interface. - it suits us: unless the subquery is resolved with one index lookup, hitting the cache would be many times cheaper than re-running the subquery, so cache is worth having. 4. Interplay with other subquery optimizations ---------------------------------------------- * This WL entry should not care about IN->EXISTS transformation: caching for IN subquery and result of its conversion to EXISTS would work in the same way. * This optimization is orthogonal to <=>ANY -> MIN/MAX rewrite (it will work/be useful irrespectively of whether the rewrite has been performed or not) * TODO: compare this with materialization for uncorrelated IN-subqueries. Is this basically the same? A: no, it is not: - IN-Materialization has to perform full materialization before it can do the first subquery evaluation. This WL's code has almost no startup costs. - This optimization has temp.table of (corr_reference, predicate_value), while IN-materialization will have (corr_reference) only. 5. User interface ----------------- * There will be an @@optimizer_switch flag to turn this optimization on and off (TODO: name of the flag?) * TODO: how do we show this in EXPLAIN [EXTENDED]? The most easiest is to print something in the warning text of EXPLAIN EXTEDED that would indicate use of cache. * temporary table sizing (max size for heap table, whether to use MyISAM or Maria) will be controlled with common temp.table control variables. LOW-LEVEL DESIGN: * Target version: base on mysql-5.2 code All items on which subquery depend could be collected in st_select_lex::mark_as_dependent (direct of indirect reference?) Temporary table index should be created by all fields except result field (TMP_TABLE_PARAM::keyinfo). How to fill the temptable ------------------------- Can reuse approach from SJ-Materialization. Its code is in end_sj_materialize() and is supposed to be quite trivial. How to make lookups into temptable ---------------------------------- We'll reuse approach used by SJ-Materialization in 6.0. Setup process ~~~~~~~~~~~~~ Setup is performed in the same way as in setup_sj_materialization(), see the code that starts these lines: /* Create/initialize everything we will need to index lookups into the temptable. */ and ends at this line: Remove the injected semi-join IN-equalities from join_tab conds. This <questionable> We'll also need to check equalities, i.e. do an equivalent of this: if (!(sjm->in_equality= create_subq_in_equalities(thd, sjm, emb_sj_nest->sj_subq_pred))) DBUG_RETURN(TRUE); /* purecov: inspected */ Question: or perhaps that is not necessarry? </questionable> Doing the lookup ~~~~~~~~~~~~~~~~ SJ-Materialization does lookup in sub_select_sjm(), with this code: /* Do index lookup in the materialized table */ if ((res= join_read_key2(join_tab, sjm->table, sjm->tab_ref)) == 1) DBUG_RETURN(NESTED_LOOP_ERROR); /* purecov: inspected */ if (res || !sjm->in_equality->val_int()) DBUG_RETURN(NESTED_LOOP_NO_MORE_ROWS); The code in this WL will use the same approach Extracting the value of the subquery predicate ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The goal of making the lookup is to get the value of subquery predicate. This is done by creating an Item_field $I which refers to appropriate temporary table's field and then subquery_predicate->val_int() will invoke $I->val_int(), subquery_predicate->val_str() will invoke $I->val_str() and so forth. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Rev 2769: Added create options for table^ fields and keys (MWL#43). in file:///Users/bell/maria/bzr/work-maria-5.1-create-options/
by sanja＠askmonty.org 19 Jan '10

19 Jan '10

At file:///Users/bell/maria/bzr/work-maria-5.1-create-options/ ------------------------------------------------------------ revno: 2769 revision-id: sanja(a)askmonty.org-20091201120747-cvp0uni2rd4iy5xi parent: bo.thorsen(a)canonical.com-20091126153249-0xfszhcynbym2lvr committer: sanja(a)askmonty.org branch nick: work-maria-5.1-create-options timestamp: Tue 2009-12-01 14:07:47 +0200 message: Added create options for table^ fields and keys (MWL#43). Diff too large for email (1985 lines, the limit is 1000).

3 6

[Maria-developers] Updated (by Timour): Subquery optimization: Efficient NOT IN execution with NULLs (68)
by worklog-noreply＠askmonty.org 19 Jan '10

19 Jan '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Subquery optimization: Efficient NOT IN execution with NULLs CREATION DATE..: Fri, 27 Nov 2009, 13:22 SUPERVISOR.....: Monty IMPLEMENTOR....: Timour COPIES TO......: CATEGORY.......: Server-Sprint TASK ID........: 68 (http://askmonty.org/worklog/?tid=68) VERSION........: Server-9.x STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Timour - Tue, 19 Jan 2010, 18:44)=-=- High-Level Specification modified. --- /tmp/wklog.68.old.22569 2010-01-19 18:44:01.000000000 +0200 +++ /tmp/wklog.68.new.22569 2010-01-19 18:44:01.000000000 +0200 @@ -132,11 +132,10 @@ if (nonull_key && ! nonull_key->lookup(outer_ref)) return FALSE - if (nonull_key) - pq.insert(nonull_key) for (i = 1; i <= n; i++) { + if (vkey[i] != nonull_key) vkey[i].lookup(outer_ref) if (! vkey[i].is_eof()) pq.insert(i) @@ -167,7 +166,7 @@ /* There cannot be a complete match, as we already checked for one. */ assert(matching_keys.elements < n) } - else if (cur_min_key == nonull_key) + else if (vkey[cur_min_key] == nonull_key) { /* The non-NULL key has no corresponding NULL index, so we know for @@ -183,8 +182,10 @@ /* Check if all null_keys contain a NULL at row 'min_row'. The procedure internally checks all keys in a special precomputed order. A prior - procedure determines an optimal order and a mapping - idx_no -> idx_order (encoded as an array). + procedure determines an optimal order and a mapping idx_no -> idx_order + (encoded as an array). + + This procedure makes sure not to match the non-NULL column. */ if (test_null_row(null_keys, min_row)) return TRUE @@ -198,6 +199,14 @@ vkey[cur_min_key].next() if (! vkey[cur_min_key].is_eof()) pq.insert(cur_min_key) + else if (vkey[cur_min_key] == nonull_key) + { + /* + If there can't be more matches for the nonull_key, we know for sure + there is no match, since there is no possible NULL match. + */ + return FALSE + } if (pq.is_empty()) { @@ -216,7 +225,6 @@ } - 3. Directions for improvement ======================================================================== -=-=(Timour - Tue, 19 Jan 2010, 18:29)=-=- High-Level Specification modified. --- /tmp/wklog.68.old.21045 2010-01-19 18:29:12.000000000 +0200 +++ /tmp/wklog.68.new.21045 2010-01-19 18:29:12.000000000 +0200 @@ -132,6 +132,8 @@ if (nonull_key && ! nonull_key->lookup(outer_ref)) return FALSE + if (nonull_key) + pq.insert(nonull_key) for (i = 1; i <= n; i++) { -=-=(Guest - Tue, 19 Jan 2010, 18:15)=-=- High-Level Specification modified. --- /tmp/wklog.68.old.19825 2010-01-19 18:15:30.000000000 +0200 +++ /tmp/wklog.68.new.19825 2010-01-19 18:15:30.000000000 +0200 @@ -1,8 +1,16 @@ -This a copy of the initial algorithm proposed by Igor: -====================================================== +Contents +======================================================================== -For each left side tuple (v_1,...,v_n) we have to find the following set -of rowids for the temp table containing N rows as the result of +1. Initial idea as proposed by Igor +2. Algorithm for IN execution with partial matching +3. Directions for improvement + + +1. Initial idea as proposed by Igor +======================================================================== + +For each left side tuple (v_1,...,v_n) we have to find the following +set of rowids for the temp table containing N rows as the result of materialization of the subquery: R= INTERSECT (rowid{a_i=v_i} UNION rowid{a_i is null} where i runs @@ -18,38 +26,198 @@ - it requires minimum memory: not more than N*n bits in total - search of an element in a set is extremely cheap -Taken all above into account I could suggest the following algorithm to -build R: +Taken all above into account I could suggest the following algorithm +to build R: - Using indexes (read about them below) for each column participating in the - intersection, - merge ordered sets rowid{a_i=v_i} in the following manner. + Using indexes (read about them below) for each column participating + in the intersection, merge ordered sets rowid{a_i=v_i} in the + following manner. If a rowid r has been encountered maximum in k sets -rowid{a_i1=v_i1},...,rowid(a_ik=v_ik), + rowid{a_i1=v_i1},...,rowid(a_ik=v_ik), then it has to be checked against all rowid{a_i=v_i} such that i is -not in {i1,...,ik}. + not in {i1,...,ik}. As soon as we fail to find r in one of these sets we discard it. If r has been found in all of them then r belongs to the set R. -Here we use the property (1): any r from rowid{a_i=v_i} UNION rowid{a_i -is null} is either +Here we use the property (1): +any r from rowid{a_i=v_i} UNION rowid{a_i is null} is either belongs to rowid{a_i=v_i} or to rowid{a_i is null}. From this we can -infer that for any r from R -indexes a_i can be uniquely divided into two groups: one contains -indexes a_i where r belongs to -the sets rowid{a_i=v_i}, the other contains indexes a_j such that r -belongs to rowid{a_j is null}. - -Now let's talk how to get elements from rowid{a_i=v_i} in a sorted order -needed for the merge procedure. We could use BTREE indexes for temp -table. But they are rather expensive and -take a lot of memory as the are implemented with RB trees. +infer that for any r from R indexes a_i can be uniquely divided into +two groups: +- one contains indexes a_i where r belongs to the sets rowid{a_i=v_i}, +- the other contains indexes a_j such that r belongs to + rowid{a_j is null}. + +Now let's talk how to get elements from rowid{a_i=v_i} in a sorted +order needed for the merge procedure. We could use BTREE indexes for +temp table. But they are rather expensive and take a lot of memory as +the are implemented with RB trees. I would suggest creating for each column from the temporary table just an array of rowids sorted by the value from column a. Index lookup in such an array is cheap. It's also rather cheap to check that the next rowid refers to a row with a different value in column a. The array can be created on demand. +2. Algorithm for IN execution with partial matching +======================================================================== + +2.1 Below is shown the top-level algorithm to execute an IN predicate +with partial matching. This algorithm is essentially the implementation +of Item_subselect:exec(). + +int lookup_with_null_semantics(outer_ref[], mat_subquery) +{ + if (index_lookup(outer_ref, mat_subquery) + return TRUE + else + { + /* + Check if there is a partial match (UNKNOWN) or no match (NULL). + */ + if (this is the first partial match) + { + vkey[] = build array of value keys for each NULL-able column + of mat_subquery. + nkey[] = build a bitmap NULL index for each column of mat_subquery + that contains NULLs + nonull_key = build a key over all non-NULL columns of mat_subquery + } + if (partial_match(outer_ref, vkey[], nkey[], nonull_key) + return UNKNOWN + else + return FALSE + } +} + +2.2 The implementation of partial matching is as follows + +/* + Assumptions: + - It has already been checked if there is a complete match by a + regular index lookup, and the test failed. + - It has already been checked if there is a complete NULL row, + and if there was we wouldn't call this function. Thus we assume + that there is no complete NULL row. + - Not all vidx_i are empty, but some can be empty. If all were empty, + then the only possibility for a match is a complete NULL row, which + we already checked. + + @param outer_ref - the uter (left) IN argument. + @param vidx[] - array of value keys + Ordered sequences of rowids of the corresponding columns a_i, such + that all rowids in idx_i are the ones where column a_i contains some + value or NULL. Each idx_i is derived dynamically, for each different + left argument of an IN predicate. + @param nidx[] - array of NULL keys + Bitmpas, one per each column, where a bit is set if the corresponding + row has a NULL value for the corresponding column. + @nonull_key - the only key over all columns of the materialized subquery + that do not contain NULLs + + @returns + @retval FALSE if there is no match + @retval TRUE if there is a partial match +*/ + +Boolean partial_match(outer_ref, vkey[], nkey[], nonull_key) +{ + /* Set of the keys (columns) that form a partial match. */ + Set matching_keys = {} + /* A subset of all keys that need to be checked for NULL matches. */ + Set null_keys = {} + Int min_key /* Key that contains the current minimum position. */ + Int min_row /* Current row number of min_key. */ + Int cur_min_key, cur_min_row + PriorityQueue pq + + if (nonull_key && ! nonull_key->lookup(outer_ref)) + return FALSE + + for (i = 1; i <= n; i++) + { + vkey[i].lookup(outer_ref) + if (! vkey[i].is_eof()) + pq.insert(i) + } + /* + Not all value keys are empty, thus we don't have only NULL + keys. If we had, the only possible match is a NULL row, and + we cheked there is no such row, therefore the result is known + to be FALSE. + In fact this algorithm makes sense for at least two non-NULL + columns. + */ + assert(pq.elements > 1) + + (min_key, min_row) = pq.pop() + matching_keys.add(min_key) + vkey[min_key].next() + if (! vkey[min_key].is_eof()) + pq.insert(min_key) + + while (TRUE) + { + (cur_min_key, cur_min_row) = pq.pop() + + if (cur_min_row == min_row) + { + matching_keys.add(cur_min_key) + /* There cannot be a complete match, as we already checked for one. */ + assert(matching_keys.elements < n) + } + else if (cur_min_key == nonull_key) + { + /* + The non-NULL key has no corresponding NULL index, so we know for + sure that the row 'min_row' is not a match. + */ + (min_key, min_row) = (cur_min_key, cur_min_row) + matching_keys = {min_key} + } + else + { + assert(cur_min_row > min_row) /* Follows from the use of PQ. */ + null_keys = set_difference(all keys vkey[], matching_keys) + /* + Check if all null_keys contain a NULL at row 'min_row'. The procedure + internally checks all keys in a special precomputed order. A prior + procedure determines an optimal order and a mapping + idx_no -> idx_order (encoded as an array). + */ + if (test_null_row(null_keys, min_row)) + return TRUE + else + { + (min_key, min_row) = (cur_min_key, cur_min_row) + matching_keys = {min_key} + } + } + + vkey[cur_min_key].next() + if (! vkey[cur_min_key].is_eof()) + pq.insert(cur_min_key) + + if (pq.is_empty()) + { + /* Check the last row of the last column in PQ for NULL matches. */ + null_keys = set_difference(all keys vkey[], matching_keys) + if (test_null_row(null_keys, min_row)) + return TRUE + else + return FALSE + } + } + + /* We should never get here. */ + assert(FALSE) + return FALSE +} + + + +3. Directions for improvement +======================================================================== + Other consideration that may be taken into account: 1. If columns a_j1,...,a_jm do not contain null values in the temporary -=-=(Timour - Sun, 06 Dec 2009, 14:36)=-=- High-Level Specification modified. --- /tmp/wklog.68.old.12919 2009-12-06 14:36:18.000000000 +0200 +++ /tmp/wklog.68.new.12919 2009-12-06 14:36:18.000000000 +0200 @@ -87,3 +87,8 @@ 7. If you get a row with nulls in all columns stop filling the temporary table and return UNKNOWN for any tuple <v1,...,vn>. +8. [timour] + Consider that due to materialization, we already have a unique index +on all columns <a_1,..., a_n>. We can use the first key part of this index +over column a_1, instead of the index rowid{a_i=v_i}. Thus we can avoid +creating the index rowid{a_i=v_i}. -=-=(Timour - Fri, 04 Dec 2009, 14:04)=-=- High-Level Specification modified. --- /tmp/wklog.68.old.16724 2009-12-04 14:04:28.000000000 +0200 +++ /tmp/wklog.68.new.16724 2009-12-04 14:04:28.000000000 +0200 @@ -10,7 +10,8 @@ Bear in mind the following specifics of this intersection: (1) For each i: rowid{a_i=v_i} and rowid{a_i is null} are disjoint - (2) For each i: rowid{a_i is null} is the same for each tuple + (2) For each i: rowid{a_i is null} is the same for each tuple, + that is, this set is independent of the left-side tuples. Due to (2) it makes sense to build rowid{a_i is null} only once. A good representation for such sets would be bitmaps: -=-=(Timour - Fri, 04 Dec 2009, 11:27)=-=- Version updated. --- /tmp/wklog.68.old.5257 2009-12-04 11:27:11.000000000 +0200 +++ /tmp/wklog.68.new.5257 2009-12-04 11:27:11.000000000 +0200 @@ -1 +1 @@ -Benchmarks-3.0 +Server-9.x -=-=(Timour - Fri, 04 Dec 2009, 11:27)=-=- Category updated. --- /tmp/wklog.68.old.5242 2009-12-04 11:27:02.000000000 +0200 +++ /tmp/wklog.68.new.5242 2009-12-04 11:27:02.000000000 +0200 @@ -1 +1 @@ -Server-RawIdeaBin +Server-Sprint -=-=(Timour - Fri, 04 Dec 2009, 11:27)=-=- Status updated. --- /tmp/wklog.68.old.5242 2009-12-04 11:27:02.000000000 +0200 +++ /tmp/wklog.68.new.5242 2009-12-04 11:27:02.000000000 +0200 @@ -1 +1 @@ -Un-Assigned +Assigned -=-=(Timour - Fri, 04 Dec 2009, 11:26)=-=- High-Level Specification modified. --- /tmp/wklog.68.old.5182 2009-12-04 11:26:25.000000000 +0200 +++ /tmp/wklog.68.new.5182 2009-12-04 11:26:25.000000000 +0200 @@ -50,23 +50,39 @@ The array can be created on demand. Other consideration that may be taken into account: + 1. If columns a_j1,...,a_jm do not contain null values in the temporary -table at all, create for them only one index array (and of course do not -create any bitmaps for them). -2. Consider the ratio d(a_i)=N'/ V(a_i), where N' is the number of rows -where a_i is not null and V(a_i) is the number of distinct values for -a_i excluding nulls. - If d(a_i) is close to 1 then do not create any index array: check +table at all and v_j1,...,v_jm cannot be null, create for these columns +only one index array (and of course do not create any bitmaps for them). + +2. Consider the ratio d(a_i)=N'(a_i)/V(a_i), where N'(a_i) is the number +of rows, where a_i is not null and V(a_i) is the number of distinct +values for a_i excluding nulls. +If d(a_i) is close to N'(a_i) then do not create any index array: check whether there is a match running through the records that have been -filtered in. Anyway if d(a_i) is close to 1 then a intersection with -rowid{a_i=v_i} would not reduce the number of remaining rowids +filtered in. Anyway if d(a_i) is close to N'(a_i) then the intersection + with rowid{a_i=v_i} will not reduce the number of remaining rowids significantly. - If additionally N-N' is small do not create a bitmap for this column -either. -3. If for a column a_i d(a_i) is not close to 1, but N-N' is small a -sorted array of rowids from the set rowid{a_i is null} can be used -instead of a bitmap. +In other words is V(a_i) exceeds some threshold there is no sense to +create an index for a_i. +If additionally N-N'(a_i) is small do not create a bitmap for this +column either. + +3. If for a column a_i d(a_i) is not close to N'(a_i), but N-N'(a_i) is +small a sorted array of rowids from the set rowid{a_i is null} can be +used instead of a bitmap. + 4. We always have a match if R0= INTERSECT rowid{a_i is null} is not empty. Here i runs through all indexes from [1..n] such that v_i is not null. For a given subset of columns this fact has to be checked only once. It can be easily done with bitmap intersection. + +5. If v1,...,vn never can be a null, then indexes (sorted arrays) can be +created only for rows with nulls. + +6. If v1,...,vn never can be a null and number of rows with nulls is +small do not create indexes and do not create bitmaps. + +7. If you get a row with nulls in all columns stop filling the temporary +table and return UNKNOWN for any tuple <v1,...,vn>. + -=-=(Timour - Fri, 27 Nov 2009, 13:23)=-=- High-Level Specification modified. --- /tmp/wklog.68.old.17140 2009-11-27 11:23:17.000000000 +0000 +++ /tmp/wklog.68.new.17140 2009-11-27 11:23:17.000000000 +0000 @@ -1 +1,72 @@ +This a copy of the initial algorithm proposed by Igor: +====================================================== +For each left side tuple (v_1,...,v_n) we have to find the following set +of rowids for the temp table containing N rows as the result of +materialization of the subquery: + + R= INTERSECT (rowid{a_i=v_i} UNION rowid{a_i is null} where i runs +trough all indexes from [1..n] such that v_i is not null. + +Bear in mind the following specifics of this intersection: + (1) For each i: rowid{a_i=v_i} and rowid{a_i is null} are disjoint + (2) For each i: rowid{a_i is null} is the same for each tuple + +Due to (2) it makes sense to build rowid{a_i is null} only once. +A good representation for such sets would be bitmaps: +- it requires minimum memory: not more than N*n bits in total +- search of an element in a set is extremely cheap + +Taken all above into account I could suggest the following algorithm to +build R: + + Using indexes (read about them below) for each column participating in the + intersection, + merge ordered sets rowid{a_i=v_i} in the following manner. + If a rowid r has been encountered maximum in k sets +rowid{a_i1=v_i1},...,rowid(a_ik=v_ik), + then it has to be checked against all rowid{a_i=v_i} such that i is +not in {i1,...,ik}. + As soon as we fail to find r in one of these sets we discard it. + If r has been found in all of them then r belongs to the set R. + +Here we use the property (1): any r from rowid{a_i=v_i} UNION rowid{a_i +is null} is either +belongs to rowid{a_i=v_i} or to rowid{a_i is null}. From this we can +infer that for any r from R +indexes a_i can be uniquely divided into two groups: one contains +indexes a_i where r belongs to +the sets rowid{a_i=v_i}, the other contains indexes a_j such that r +belongs to rowid{a_j is null}. + +Now let's talk how to get elements from rowid{a_i=v_i} in a sorted order +needed for the merge procedure. We could use BTREE indexes for temp +table. But they are rather expensive and +take a lot of memory as the are implemented with RB trees. +I would suggest creating for each column from the temporary table just +an array of rowids sorted by the value from column a. +Index lookup in such an array is cheap. It's also rather cheap to check +that the next rowid refers to a row with a different value in column a. +The array can be created on demand. + +Other consideration that may be taken into account: +1. If columns a_j1,...,a_jm do not contain null values in the temporary +table at all, create for them only one index array (and of course do not +create any bitmaps for them). +2. Consider the ratio d(a_i)=N'/ V(a_i), where N' is the number of rows +where a_i is not null and V(a_i) is the number of distinct values for +a_i excluding nulls. + If d(a_i) is close to 1 then do not create any index array: check +whether there is a match running through the records that have been +filtered in. Anyway if d(a_i) is close to 1 then a intersection with +rowid{a_i=v_i} would not reduce the number of remaining rowids +significantly. + If additionally N-N' is small do not create a bitmap for this column +either. +3. If for a column a_i d(a_i) is not close to 1, but N-N' is small a +sorted array of rowids from the set rowid{a_i is null} can be used +instead of a bitmap. +4. We always have a match if R0= INTERSECT rowid{a_i is null} is not +empty. Here i runs through all indexes from [1..n] such that v_i is not +null. For a given subset of columns this fact has to be checked only +once. It can be easily done with bitmap intersection. DESCRIPTION: The goal of this task is to implement efficient execution of NOT IN subquery predicates of the form: <oe_1,...,oe_n> NOT IN <non_correlated subquery> when either some oe_i, or some subqury result column contains NULLs. The problem with such predicates is that it is possible to use index lookups only when neither argument of the predicate contains NULLs. If some argument contains a NULL, then due to NULL semantics, it plays the role of a wildcard. If we were to use regular index lookups, then we would get 'no match' for some outer tuple (thus the predicate evaluates to FALSE), while the SQL semantics means 'partial match', and the predicate should evaluate to NULL. This task implements an efficient algorithm to compute such 'parial matches', where a NULL matches any value. HIGH-LEVEL SPECIFICATION: Contents ======================================================================== 1. Initial idea as proposed by Igor 2. Algorithm for IN execution with partial matching 3. Directions for improvement 1. Initial idea as proposed by Igor ======================================================================== For each left side tuple (v_1,...,v_n) we have to find the following set of rowids for the temp table containing N rows as the result of materialization of the subquery: R= INTERSECT (rowid{a_i=v_i} UNION rowid{a_i is null} where i runs trough all indexes from [1..n] such that v_i is not null. Bear in mind the following specifics of this intersection: (1) For each i: rowid{a_i=v_i} and rowid{a_i is null} are disjoint (2) For each i: rowid{a_i is null} is the same for each tuple, that is, this set is independent of the left-side tuples. Due to (2) it makes sense to build rowid{a_i is null} only once. A good representation for such sets would be bitmaps: - it requires minimum memory: not more than N*n bits in total - search of an element in a set is extremely cheap Taken all above into account I could suggest the following algorithm to build R: Using indexes (read about them below) for each column participating in the intersection, merge ordered sets rowid{a_i=v_i} in the following manner. If a rowid r has been encountered maximum in k sets rowid{a_i1=v_i1},...,rowid(a_ik=v_ik), then it has to be checked against all rowid{a_i=v_i} such that i is not in {i1,...,ik}. As soon as we fail to find r in one of these sets we discard it. If r has been found in all of them then r belongs to the set R. Here we use the property (1): any r from rowid{a_i=v_i} UNION rowid{a_i is null} is either belongs to rowid{a_i=v_i} or to rowid{a_i is null}. From this we can infer that for any r from R indexes a_i can be uniquely divided into two groups: - one contains indexes a_i where r belongs to the sets rowid{a_i=v_i}, - the other contains indexes a_j such that r belongs to rowid{a_j is null}. Now let's talk how to get elements from rowid{a_i=v_i} in a sorted order needed for the merge procedure. We could use BTREE indexes for temp table. But they are rather expensive and take a lot of memory as the are implemented with RB trees. I would suggest creating for each column from the temporary table just an array of rowids sorted by the value from column a. Index lookup in such an array is cheap. It's also rather cheap to check that the next rowid refers to a row with a different value in column a. The array can be created on demand. 2. Algorithm for IN execution with partial matching ======================================================================== 2.1 Below is shown the top-level algorithm to execute an IN predicate with partial matching. This algorithm is essentially the implementation of Item_subselect:exec(). int lookup_with_null_semantics(outer_ref[], mat_subquery) { if (index_lookup(outer_ref, mat_subquery) return TRUE else { /* Check if there is a partial match (UNKNOWN) or no match (NULL). */ if (this is the first partial match) { vkey[] = build array of value keys for each NULL-able column of mat_subquery. nkey[] = build a bitmap NULL index for each column of mat_subquery that contains NULLs nonull_key = build a key over all non-NULL columns of mat_subquery } if (partial_match(outer_ref, vkey[], nkey[], nonull_key) return UNKNOWN else return FALSE } } 2.2 The implementation of partial matching is as follows /* Assumptions: - It has already been checked if there is a complete match by a regular index lookup, and the test failed. - It has already been checked if there is a complete NULL row, and if there was we wouldn't call this function. Thus we assume that there is no complete NULL row. - Not all vidx_i are empty, but some can be empty. If all were empty, then the only possibility for a match is a complete NULL row, which we already checked. @param outer_ref - the uter (left) IN argument. @param vidx[] - array of value keys Ordered sequences of rowids of the corresponding columns a_i, such that all rowids in idx_i are the ones where column a_i contains some value or NULL. Each idx_i is derived dynamically, for each different left argument of an IN predicate. @param nidx[] - array of NULL keys Bitmpas, one per each column, where a bit is set if the corresponding row has a NULL value for the corresponding column. @nonull_key - the only key over all columns of the materialized subquery that do not contain NULLs @returns @retval FALSE if there is no match @retval TRUE if there is a partial match */ Boolean partial_match(outer_ref, vkey[], nkey[], nonull_key) { /* Set of the keys (columns) that form a partial match. */ Set matching_keys = {} /* A subset of all keys that need to be checked for NULL matches. */ Set null_keys = {} Int min_key /* Key that contains the current minimum position. */ Int min_row /* Current row number of min_key. */ Int cur_min_key, cur_min_row PriorityQueue pq if (nonull_key && ! nonull_key->lookup(outer_ref)) return FALSE for (i = 1; i <= n; i++) { if (vkey[i] != nonull_key) vkey[i].lookup(outer_ref) if (! vkey[i].is_eof()) pq.insert(i) } /* Not all value keys are empty, thus we don't have only NULL keys. If we had, the only possible match is a NULL row, and we cheked there is no such row, therefore the result is known to be FALSE. In fact this algorithm makes sense for at least two non-NULL columns. */ assert(pq.elements > 1) (min_key, min_row) = pq.pop() matching_keys.add(min_key) vkey[min_key].next() if (! vkey[min_key].is_eof()) pq.insert(min_key) while (TRUE) { (cur_min_key, cur_min_row) = pq.pop() if (cur_min_row == min_row) { matching_keys.add(cur_min_key) /* There cannot be a complete match, as we already checked for one. */ assert(matching_keys.elements < n) } else if (vkey[cur_min_key] == nonull_key) { /* The non-NULL key has no corresponding NULL index, so we know for sure that the row 'min_row' is not a match. */ (min_key, min_row) = (cur_min_key, cur_min_row) matching_keys = {min_key} } else { assert(cur_min_row > min_row) /* Follows from the use of PQ. */ null_keys = set_difference(all keys vkey[], matching_keys) /* Check if all null_keys contain a NULL at row 'min_row'. The procedure internally checks all keys in a special precomputed order. A prior procedure determines an optimal order and a mapping idx_no -> idx_order (encoded as an array). This procedure makes sure not to match the non-NULL column. */ if (test_null_row(null_keys, min_row)) return TRUE else { (min_key, min_row) = (cur_min_key, cur_min_row) matching_keys = {min_key} } } vkey[cur_min_key].next() if (! vkey[cur_min_key].is_eof()) pq.insert(cur_min_key) else if (vkey[cur_min_key] == nonull_key) { /* If there can't be more matches for the nonull_key, we know for sure there is no match, since there is no possible NULL match. */ return FALSE } if (pq.is_empty()) { /* Check the last row of the last column in PQ for NULL matches. */ null_keys = set_difference(all keys vkey[], matching_keys) if (test_null_row(null_keys, min_row)) return TRUE else return FALSE } } /* We should never get here. */ assert(FALSE) return FALSE } 3. Directions for improvement ======================================================================== Other consideration that may be taken into account: 1. If columns a_j1,...,a_jm do not contain null values in the temporary table at all and v_j1,...,v_jm cannot be null, create for these columns only one index array (and of course do not create any bitmaps for them). 2. Consider the ratio d(a_i)=N'(a_i)/V(a_i), where N'(a_i) is the number of rows, where a_i is not null and V(a_i) is the number of distinct values for a_i excluding nulls. If d(a_i) is close to N'(a_i) then do not create any index array: check whether there is a match running through the records that have been filtered in. Anyway if d(a_i) is close to N'(a_i) then the intersection with rowid{a_i=v_i} will not reduce the number of remaining rowids significantly. In other words is V(a_i) exceeds some threshold there is no sense to create an index for a_i. If additionally N-N'(a_i) is small do not create a bitmap for this column either. 3. If for a column a_i d(a_i) is not close to N'(a_i), but N-N'(a_i) is small a sorted array of rowids from the set rowid{a_i is null} can be used instead of a bitmap. 4. We always have a match if R0= INTERSECT rowid{a_i is null} is not empty. Here i runs through all indexes from [1..n] such that v_i is not null. For a given subset of columns this fact has to be checked only once. It can be easily done with bitmap intersection. 5. If v1,...,vn never can be a null, then indexes (sorted arrays) can be created only for rows with nulls. 6. If v1,...,vn never can be a null and number of rows with nulls is small do not create indexes and do not create bitmaps. 7. If you get a row with nulls in all columns stop filling the temporary table and return UNKNOWN for any tuple <v1,...,vn>. 8. [timour] Consider that due to materialization, we already have a unique index on all columns <a_1,..., a_n>. We can use the first key part of this index over column a_1, instead of the index rowid{a_i=v_i}. Thus we can avoid creating the index rowid{a_i=v_i}. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Timour): Subquery optimization: Efficient NOT IN execution with NULLs (68)
by worklog-noreply＠askmonty.org 19 Jan '10

19 Jan '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Subquery optimization: Efficient NOT IN execution with NULLs CREATION DATE..: Fri, 27 Nov 2009, 13:22 SUPERVISOR.....: Monty IMPLEMENTOR....: Timour COPIES TO......: CATEGORY.......: Server-Sprint TASK ID........: 68 (http://askmonty.org/worklog/?tid=68) VERSION........: Server-9.x STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Timour - Tue, 19 Jan 2010, 18:44)=-=- High-Level Specification modified. --- /tmp/wklog.68.old.22569 2010-01-19 18:44:01.000000000 +0200 +++ /tmp/wklog.68.new.22569 2010-01-19 18:44:01.000000000 +0200 @@ -132,11 +132,10 @@ if (nonull_key && ! nonull_key->lookup(outer_ref)) return FALSE - if (nonull_key) - pq.insert(nonull_key) for (i = 1; i <= n; i++) { + if (vkey[i] != nonull_key) vkey[i].lookup(outer_ref) if (! vkey[i].is_eof()) pq.insert(i) @@ -167,7 +166,7 @@ /* There cannot be a complete match, as we already checked for one. */ assert(matching_keys.elements < n) } - else if (cur_min_key == nonull_key) + else if (vkey[cur_min_key] == nonull_key) { /* The non-NULL key has no corresponding NULL index, so we know for @@ -183,8 +182,10 @@ /* Check if all null_keys contain a NULL at row 'min_row'. The procedure internally checks all keys in a special precomputed order. A prior - procedure determines an optimal order and a mapping - idx_no -> idx_order (encoded as an array). + procedure determines an optimal order and a mapping idx_no -> idx_order + (encoded as an array). + + This procedure makes sure not to match the non-NULL column. */ if (test_null_row(null_keys, min_row)) return TRUE @@ -198,6 +199,14 @@ vkey[cur_min_key].next() if (! vkey[cur_min_key].is_eof()) pq.insert(cur_min_key) + else if (vkey[cur_min_key] == nonull_key) + { + /* + If there can't be more matches for the nonull_key, we know for sure + there is no match, since there is no possible NULL match. + */ + return FALSE + } if (pq.is_empty()) { @@ -216,7 +225,6 @@ } - 3. Directions for improvement ======================================================================== -=-=(Timour - Tue, 19 Jan 2010, 18:29)=-=- High-Level Specification modified. --- /tmp/wklog.68.old.21045 2010-01-19 18:29:12.000000000 +0200 +++ /tmp/wklog.68.new.21045 2010-01-19 18:29:12.000000000 +0200 @@ -132,6 +132,8 @@ if (nonull_key && ! nonull_key->lookup(outer_ref)) return FALSE + if (nonull_key) + pq.insert(nonull_key) for (i = 1; i <= n; i++) { -=-=(Guest - Tue, 19 Jan 2010, 18:15)=-=- High-Level Specification modified. --- /tmp/wklog.68.old.19825 2010-01-19 18:15:30.000000000 +0200 +++ /tmp/wklog.68.new.19825 2010-01-19 18:15:30.000000000 +0200 @@ -1,8 +1,16 @@ -This a copy of the initial algorithm proposed by Igor: -====================================================== +Contents +======================================================================== -For each left side tuple (v_1,...,v_n) we have to find the following set -of rowids for the temp table containing N rows as the result of +1. Initial idea as proposed by Igor +2. Algorithm for IN execution with partial matching +3. Directions for improvement + + +1. Initial idea as proposed by Igor +======================================================================== + +For each left side tuple (v_1,...,v_n) we have to find the following +set of rowids for the temp table containing N rows as the result of materialization of the subquery: R= INTERSECT (rowid{a_i=v_i} UNION rowid{a_i is null} where i runs @@ -18,38 +26,198 @@ - it requires minimum memory: not more than N*n bits in total - search of an element in a set is extremely cheap -Taken all above into account I could suggest the following algorithm to -build R: +Taken all above into account I could suggest the following algorithm +to build R: - Using indexes (read about them below) for each column participating in the - intersection, - merge ordered sets rowid{a_i=v_i} in the following manner. + Using indexes (read about them below) for each column participating + in the intersection, merge ordered sets rowid{a_i=v_i} in the + following manner. If a rowid r has been encountered maximum in k sets -rowid{a_i1=v_i1},...,rowid(a_ik=v_ik), + rowid{a_i1=v_i1},...,rowid(a_ik=v_ik), then it has to be checked against all rowid{a_i=v_i} such that i is -not in {i1,...,ik}. + not in {i1,...,ik}. As soon as we fail to find r in one of these sets we discard it. If r has been found in all of them then r belongs to the set R. -Here we use the property (1): any r from rowid{a_i=v_i} UNION rowid{a_i -is null} is either +Here we use the property (1): +any r from rowid{a_i=v_i} UNION rowid{a_i is null} is either belongs to rowid{a_i=v_i} or to rowid{a_i is null}. From this we can -infer that for any r from R -indexes a_i can be uniquely divided into two groups: one contains -indexes a_i where r belongs to -the sets rowid{a_i=v_i}, the other contains indexes a_j such that r -belongs to rowid{a_j is null}. - -Now let's talk how to get elements from rowid{a_i=v_i} in a sorted order -needed for the merge procedure. We could use BTREE indexes for temp -table. But they are rather expensive and -take a lot of memory as the are implemented with RB trees. +infer that for any r from R indexes a_i can be uniquely divided into +two groups: +- one contains indexes a_i where r belongs to the sets rowid{a_i=v_i}, +- the other contains indexes a_j such that r belongs to + rowid{a_j is null}. + +Now let's talk how to get elements from rowid{a_i=v_i} in a sorted +order needed for the merge procedure. We could use BTREE indexes for +temp table. But they are rather expensive and take a lot of memory as +the are implemented with RB trees. I would suggest creating for each column from the temporary table just an array of rowids sorted by the value from column a. Index lookup in such an array is cheap. It's also rather cheap to check that the next rowid refers to a row with a different value in column a. The array can be created on demand. +2. Algorithm for IN execution with partial matching +======================================================================== + +2.1 Below is shown the top-level algorithm to execute an IN predicate +with partial matching. This algorithm is essentially the implementation +of Item_subselect:exec(). + +int lookup_with_null_semantics(outer_ref[], mat_subquery) +{ + if (index_lookup(outer_ref, mat_subquery) + return TRUE + else + { + /* + Check if there is a partial match (UNKNOWN) or no match (NULL). + */ + if (this is the first partial match) + { + vkey[] = build array of value keys for each NULL-able column + of mat_subquery. + nkey[] = build a bitmap NULL index for each column of mat_subquery + that contains NULLs + nonull_key = build a key over all non-NULL columns of mat_subquery + } + if (partial_match(outer_ref, vkey[], nkey[], nonull_key) + return UNKNOWN + else + return FALSE + } +} + +2.2 The implementation of partial matching is as follows + +/* + Assumptions: + - It has already been checked if there is a complete match by a + regular index lookup, and the test failed. + - It has already been checked if there is a complete NULL row, + and if there was we wouldn't call this function. Thus we assume + that there is no complete NULL row. + - Not all vidx_i are empty, but some can be empty. If all were empty, + then the only possibility for a match is a complete NULL row, which + we already checked. + + @param outer_ref - the uter (left) IN argument. + @param vidx[] - array of value keys + Ordered sequences of rowids of the corresponding columns a_i, such + that all rowids in idx_i are the ones where column a_i contains some + value or NULL. Each idx_i is derived dynamically, for each different + left argument of an IN predicate. + @param nidx[] - array of NULL keys + Bitmpas, one per each column, where a bit is set if the corresponding + row has a NULL value for the corresponding column. + @nonull_key - the only key over all columns of the materialized subquery + that do not contain NULLs + + @returns + @retval FALSE if there is no match + @retval TRUE if there is a partial match +*/ + +Boolean partial_match(outer_ref, vkey[], nkey[], nonull_key) +{ + /* Set of the keys (columns) that form a partial match. */ + Set matching_keys = {} + /* A subset of all keys that need to be checked for NULL matches. */ + Set null_keys = {} + Int min_key /* Key that contains the current minimum position. */ + Int min_row /* Current row number of min_key. */ + Int cur_min_key, cur_min_row + PriorityQueue pq + + if (nonull_key && ! nonull_key->lookup(outer_ref)) + return FALSE + + for (i = 1; i <= n; i++) + { + vkey[i].lookup(outer_ref) + if (! vkey[i].is_eof()) + pq.insert(i) + } + /* + Not all value keys are empty, thus we don't have only NULL + keys. If we had, the only possible match is a NULL row, and + we cheked there is no such row, therefore the result is known + to be FALSE. + In fact this algorithm makes sense for at least two non-NULL + columns. + */ + assert(pq.elements > 1) + + (min_key, min_row) = pq.pop() + matching_keys.add(min_key) + vkey[min_key].next() + if (! vkey[min_key].is_eof()) + pq.insert(min_key) + + while (TRUE) + { + (cur_min_key, cur_min_row) = pq.pop() + + if (cur_min_row == min_row) + { + matching_keys.add(cur_min_key) + /* There cannot be a complete match, as we already checked for one. */ + assert(matching_keys.elements < n) + } + else if (cur_min_key == nonull_key) + { + /* + The non-NULL key has no corresponding NULL index, so we know for + sure that the row 'min_row' is not a match. + */ + (min_key, min_row) = (cur_min_key, cur_min_row) + matching_keys = {min_key} + } + else + { + assert(cur_min_row > min_row) /* Follows from the use of PQ. */ + null_keys = set_difference(all keys vkey[], matching_keys) + /* + Check if all null_keys contain a NULL at row 'min_row'. The procedure + internally checks all keys in a special precomputed order. A prior + procedure determines an optimal order and a mapping + idx_no -> idx_order (encoded as an array). + */ + if (test_null_row(null_keys, min_row)) + return TRUE + else + { + (min_key, min_row) = (cur_min_key, cur_min_row) + matching_keys = {min_key} + } + } + + vkey[cur_min_key].next() + if (! vkey[cur_min_key].is_eof()) + pq.insert(cur_min_key) + + if (pq.is_empty()) + { + /* Check the last row of the last column in PQ for NULL matches. */ + null_keys = set_difference(all keys vkey[], matching_keys) + if (test_null_row(null_keys, min_row)) + return TRUE + else + return FALSE + } + } + + /* We should never get here. */ + assert(FALSE) + return FALSE +} + + + +3. Directions for improvement +======================================================================== + Other consideration that may be taken into account: 1. If columns a_j1,...,a_jm do not contain null values in the temporary -=-=(Timour - Sun, 06 Dec 2009, 14:36)=-=- High-Level Specification modified. --- /tmp/wklog.68.old.12919 2009-12-06 14:36:18.000000000 +0200 +++ /tmp/wklog.68.new.12919 2009-12-06 14:36:18.000000000 +0200 @@ -87,3 +87,8 @@ 7. If you get a row with nulls in all columns stop filling the temporary table and return UNKNOWN for any tuple <v1,...,vn>. +8. [timour] + Consider that due to materialization, we already have a unique index +on all columns <a_1,..., a_n>. We can use the first key part of this index +over column a_1, instead of the index rowid{a_i=v_i}. Thus we can avoid +creating the index rowid{a_i=v_i}. -=-=(Timour - Fri, 04 Dec 2009, 14:04)=-=- High-Level Specification modified. --- /tmp/wklog.68.old.16724 2009-12-04 14:04:28.000000000 +0200 +++ /tmp/wklog.68.new.16724 2009-12-04 14:04:28.000000000 +0200 @@ -10,7 +10,8 @@ Bear in mind the following specifics of this intersection: (1) For each i: rowid{a_i=v_i} and rowid{a_i is null} are disjoint - (2) For each i: rowid{a_i is null} is the same for each tuple + (2) For each i: rowid{a_i is null} is the same for each tuple, + that is, this set is independent of the left-side tuples. Due to (2) it makes sense to build rowid{a_i is null} only once. A good representation for such sets would be bitmaps: -=-=(Timour - Fri, 04 Dec 2009, 11:27)=-=- Version updated. --- /tmp/wklog.68.old.5257 2009-12-04 11:27:11.000000000 +0200 +++ /tmp/wklog.68.new.5257 2009-12-04 11:27:11.000000000 +0200 @@ -1 +1 @@ -Benchmarks-3.0 +Server-9.x -=-=(Timour - Fri, 04 Dec 2009, 11:27)=-=- Category updated. --- /tmp/wklog.68.old.5242 2009-12-04 11:27:02.000000000 +0200 +++ /tmp/wklog.68.new.5242 2009-12-04 11:27:02.000000000 +0200 @@ -1 +1 @@ -Server-RawIdeaBin +Server-Sprint -=-=(Timour - Fri, 04 Dec 2009, 11:27)=-=- Status updated. --- /tmp/wklog.68.old.5242 2009-12-04 11:27:02.000000000 +0200 +++ /tmp/wklog.68.new.5242 2009-12-04 11:27:02.000000000 +0200 @@ -1 +1 @@ -Un-Assigned +Assigned -=-=(Timour - Fri, 04 Dec 2009, 11:26)=-=- High-Level Specification modified. --- /tmp/wklog.68.old.5182 2009-12-04 11:26:25.000000000 +0200 +++ /tmp/wklog.68.new.5182 2009-12-04 11:26:25.000000000 +0200 @@ -50,23 +50,39 @@ The array can be created on demand. Other consideration that may be taken into account: + 1. If columns a_j1,...,a_jm do not contain null values in the temporary -table at all, create for them only one index array (and of course do not -create any bitmaps for them). -2. Consider the ratio d(a_i)=N'/ V(a_i), where N' is the number of rows -where a_i is not null and V(a_i) is the number of distinct values for -a_i excluding nulls. - If d(a_i) is close to 1 then do not create any index array: check +table at all and v_j1,...,v_jm cannot be null, create for these columns +only one index array (and of course do not create any bitmaps for them). + +2. Consider the ratio d(a_i)=N'(a_i)/V(a_i), where N'(a_i) is the number +of rows, where a_i is not null and V(a_i) is the number of distinct +values for a_i excluding nulls. +If d(a_i) is close to N'(a_i) then do not create any index array: check whether there is a match running through the records that have been -filtered in. Anyway if d(a_i) is close to 1 then a intersection with -rowid{a_i=v_i} would not reduce the number of remaining rowids +filtered in. Anyway if d(a_i) is close to N'(a_i) then the intersection + with rowid{a_i=v_i} will not reduce the number of remaining rowids significantly. - If additionally N-N' is small do not create a bitmap for this column -either. -3. If for a column a_i d(a_i) is not close to 1, but N-N' is small a -sorted array of rowids from the set rowid{a_i is null} can be used -instead of a bitmap. +In other words is V(a_i) exceeds some threshold there is no sense to +create an index for a_i. +If additionally N-N'(a_i) is small do not create a bitmap for this +column either. + +3. If for a column a_i d(a_i) is not close to N'(a_i), but N-N'(a_i) is +small a sorted array of rowids from the set rowid{a_i is null} can be +used instead of a bitmap. + 4. We always have a match if R0= INTERSECT rowid{a_i is null} is not empty. Here i runs through all indexes from [1..n] such that v_i is not null. For a given subset of columns this fact has to be checked only once. It can be easily done with bitmap intersection. + +5. If v1,...,vn never can be a null, then indexes (sorted arrays) can be +created only for rows with nulls. + +6. If v1,...,vn never can be a null and number of rows with nulls is +small do not create indexes and do not create bitmaps. + +7. If you get a row with nulls in all columns stop filling the temporary +table and return UNKNOWN for any tuple <v1,...,vn>. + -=-=(Timour - Fri, 27 Nov 2009, 13:23)=-=- High-Level Specification modified. --- /tmp/wklog.68.old.17140 2009-11-27 11:23:17.000000000 +0000 +++ /tmp/wklog.68.new.17140 2009-11-27 11:23:17.000000000 +0000 @@ -1 +1,72 @@ +This a copy of the initial algorithm proposed by Igor: +====================================================== +For each left side tuple (v_1,...,v_n) we have to find the following set +of rowids for the temp table containing N rows as the result of +materialization of the subquery: + + R= INTERSECT (rowid{a_i=v_i} UNION rowid{a_i is null} where i runs +trough all indexes from [1..n] such that v_i is not null. + +Bear in mind the following specifics of this intersection: + (1) For each i: rowid{a_i=v_i} and rowid{a_i is null} are disjoint + (2) For each i: rowid{a_i is null} is the same for each tuple + +Due to (2) it makes sense to build rowid{a_i is null} only once. +A good representation for such sets would be bitmaps: +- it requires minimum memory: not more than N*n bits in total +- search of an element in a set is extremely cheap + +Taken all above into account I could suggest the following algorithm to +build R: + + Using indexes (read about them below) for each column participating in the + intersection, + merge ordered sets rowid{a_i=v_i} in the following manner. + If a rowid r has been encountered maximum in k sets +rowid{a_i1=v_i1},...,rowid(a_ik=v_ik), + then it has to be checked against all rowid{a_i=v_i} such that i is +not in {i1,...,ik}. + As soon as we fail to find r in one of these sets we discard it. + If r has been found in all of them then r belongs to the set R. + +Here we use the property (1): any r from rowid{a_i=v_i} UNION rowid{a_i +is null} is either +belongs to rowid{a_i=v_i} or to rowid{a_i is null}. From this we can +infer that for any r from R +indexes a_i can be uniquely divided into two groups: one contains +indexes a_i where r belongs to +the sets rowid{a_i=v_i}, the other contains indexes a_j such that r +belongs to rowid{a_j is null}. + +Now let's talk how to get elements from rowid{a_i=v_i} in a sorted order +needed for the merge procedure. We could use BTREE indexes for temp +table. But they are rather expensive and +take a lot of memory as the are implemented with RB trees. +I would suggest creating for each column from the temporary table just +an array of rowids sorted by the value from column a. +Index lookup in such an array is cheap. It's also rather cheap to check +that the next rowid refers to a row with a different value in column a. +The array can be created on demand. + +Other consideration that may be taken into account: +1. If columns a_j1,...,a_jm do not contain null values in the temporary +table at all, create for them only one index array (and of course do not +create any bitmaps for them). +2. Consider the ratio d(a_i)=N'/ V(a_i), where N' is the number of rows +where a_i is not null and V(a_i) is the number of distinct values for +a_i excluding nulls. + If d(a_i) is close to 1 then do not create any index array: check +whether there is a match running through the records that have been +filtered in. Anyway if d(a_i) is close to 1 then a intersection with +rowid{a_i=v_i} would not reduce the number of remaining rowids +significantly. + If additionally N-N' is small do not create a bitmap for this column +either. +3. If for a column a_i d(a_i) is not close to 1, but N-N' is small a +sorted array of rowids from the set rowid{a_i is null} can be used +instead of a bitmap. +4. We always have a match if R0= INTERSECT rowid{a_i is null} is not +empty. Here i runs through all indexes from [1..n] such that v_i is not +null. For a given subset of columns this fact has to be checked only +once. It can be easily done with bitmap intersection. DESCRIPTION: The goal of this task is to implement efficient execution of NOT IN subquery predicates of the form: <oe_1,...,oe_n> NOT IN <non_correlated subquery> when either some oe_i, or some subqury result column contains NULLs. The problem with such predicates is that it is possible to use index lookups only when neither argument of the predicate contains NULLs. If some argument contains a NULL, then due to NULL semantics, it plays the role of a wildcard. If we were to use regular index lookups, then we would get 'no match' for some outer tuple (thus the predicate evaluates to FALSE), while the SQL semantics means 'partial match', and the predicate should evaluate to NULL. This task implements an efficient algorithm to compute such 'parial matches', where a NULL matches any value. HIGH-LEVEL SPECIFICATION: Contents ======================================================================== 1. Initial idea as proposed by Igor 2. Algorithm for IN execution with partial matching 3. Directions for improvement 1. Initial idea as proposed by Igor ======================================================================== For each left side tuple (v_1,...,v_n) we have to find the following set of rowids for the temp table containing N rows as the result of materialization of the subquery: R= INTERSECT (rowid{a_i=v_i} UNION rowid{a_i is null} where i runs trough all indexes from [1..n] such that v_i is not null. Bear in mind the following specifics of this intersection: (1) For each i: rowid{a_i=v_i} and rowid{a_i is null} are disjoint (2) For each i: rowid{a_i is null} is the same for each tuple, that is, this set is independent of the left-side tuples. Due to (2) it makes sense to build rowid{a_i is null} only once. A good representation for such sets would be bitmaps: - it requires minimum memory: not more than N*n bits in total - search of an element in a set is extremely cheap Taken all above into account I could suggest the following algorithm to build R: Using indexes (read about them below) for each column participating in the intersection, merge ordered sets rowid{a_i=v_i} in the following manner. If a rowid r has been encountered maximum in k sets rowid{a_i1=v_i1},...,rowid(a_ik=v_ik), then it has to be checked against all rowid{a_i=v_i} such that i is not in {i1,...,ik}. As soon as we fail to find r in one of these sets we discard it. If r has been found in all of them then r belongs to the set R. Here we use the property (1): any r from rowid{a_i=v_i} UNION rowid{a_i is null} is either belongs to rowid{a_i=v_i} or to rowid{a_i is null}. From this we can infer that for any r from R indexes a_i can be uniquely divided into two groups: - one contains indexes a_i where r belongs to the sets rowid{a_i=v_i}, - the other contains indexes a_j such that r belongs to rowid{a_j is null}. Now let's talk how to get elements from rowid{a_i=v_i} in a sorted order needed for the merge procedure. We could use BTREE indexes for temp table. But they are rather expensive and take a lot of memory as the are implemented with RB trees. I would suggest creating for each column from the temporary table just an array of rowids sorted by the value from column a. Index lookup in such an array is cheap. It's also rather cheap to check that the next rowid refers to a row with a different value in column a. The array can be created on demand. 2. Algorithm for IN execution with partial matching ======================================================================== 2.1 Below is shown the top-level algorithm to execute an IN predicate with partial matching. This algorithm is essentially the implementation of Item_subselect:exec(). int lookup_with_null_semantics(outer_ref[], mat_subquery) { if (index_lookup(outer_ref, mat_subquery) return TRUE else { /* Check if there is a partial match (UNKNOWN) or no match (NULL). */ if (this is the first partial match) { vkey[] = build array of value keys for each NULL-able column of mat_subquery. nkey[] = build a bitmap NULL index for each column of mat_subquery that contains NULLs nonull_key = build a key over all non-NULL columns of mat_subquery } if (partial_match(outer_ref, vkey[], nkey[], nonull_key) return UNKNOWN else return FALSE } } 2.2 The implementation of partial matching is as follows /* Assumptions: - It has already been checked if there is a complete match by a regular index lookup, and the test failed. - It has already been checked if there is a complete NULL row, and if there was we wouldn't call this function. Thus we assume that there is no complete NULL row. - Not all vidx_i are empty, but some can be empty. If all were empty, then the only possibility for a match is a complete NULL row, which we already checked. @param outer_ref - the uter (left) IN argument. @param vidx[] - array of value keys Ordered sequences of rowids of the corresponding columns a_i, such that all rowids in idx_i are the ones where column a_i contains some value or NULL. Each idx_i is derived dynamically, for each different left argument of an IN predicate. @param nidx[] - array of NULL keys Bitmpas, one per each column, where a bit is set if the corresponding row has a NULL value for the corresponding column. @nonull_key - the only key over all columns of the materialized subquery that do not contain NULLs @returns @retval FALSE if there is no match @retval TRUE if there is a partial match */ Boolean partial_match(outer_ref, vkey[], nkey[], nonull_key) { /* Set of the keys (columns) that form a partial match. */ Set matching_keys = {} /* A subset of all keys that need to be checked for NULL matches. */ Set null_keys = {} Int min_key /* Key that contains the current minimum position. */ Int min_row /* Current row number of min_key. */ Int cur_min_key, cur_min_row PriorityQueue pq if (nonull_key && ! nonull_key->lookup(outer_ref)) return FALSE for (i = 1; i <= n; i++) { if (vkey[i] != nonull_key) vkey[i].lookup(outer_ref) if (! vkey[i].is_eof()) pq.insert(i) } /* Not all value keys are empty, thus we don't have only NULL keys. If we had, the only possible match is a NULL row, and we cheked there is no such row, therefore the result is known to be FALSE. In fact this algorithm makes sense for at least two non-NULL columns. */ assert(pq.elements > 1) (min_key, min_row) = pq.pop() matching_keys.add(min_key) vkey[min_key].next() if (! vkey[min_key].is_eof()) pq.insert(min_key) while (TRUE) { (cur_min_key, cur_min_row) = pq.pop() if (cur_min_row == min_row) { matching_keys.add(cur_min_key) /* There cannot be a complete match, as we already checked for one. */ assert(matching_keys.elements < n) } else if (vkey[cur_min_key] == nonull_key) { /* The non-NULL key has no corresponding NULL index, so we know for sure that the row 'min_row' is not a match. */ (min_key, min_row) = (cur_min_key, cur_min_row) matching_keys = {min_key} } else { assert(cur_min_row > min_row) /* Follows from the use of PQ. */ null_keys = set_difference(all keys vkey[], matching_keys) /* Check if all null_keys contain a NULL at row 'min_row'. The procedure internally checks all keys in a special precomputed order. A prior procedure determines an optimal order and a mapping idx_no -> idx_order (encoded as an array). This procedure makes sure not to match the non-NULL column. */ if (test_null_row(null_keys, min_row)) return TRUE else { (min_key, min_row) = (cur_min_key, cur_min_row) matching_keys = {min_key} } } vkey[cur_min_key].next() if (! vkey[cur_min_key].is_eof()) pq.insert(cur_min_key) else if (vkey[cur_min_key] == nonull_key) { /* If there can't be more matches for the nonull_key, we know for sure there is no match, since there is no possible NULL match. */ return FALSE } if (pq.is_empty()) { /* Check the last row of the last column in PQ for NULL matches. */ null_keys = set_difference(all keys vkey[], matching_keys) if (test_null_row(null_keys, min_row)) return TRUE else return FALSE } } /* We should never get here. */ assert(FALSE) return FALSE } 3. Directions for improvement ======================================================================== Other consideration that may be taken into account: 1. If columns a_j1,...,a_jm do not contain null values in the temporary table at all and v_j1,...,v_jm cannot be null, create for these columns only one index array (and of course do not create any bitmaps for them). 2. Consider the ratio d(a_i)=N'(a_i)/V(a_i), where N'(a_i) is the number of rows, where a_i is not null and V(a_i) is the number of distinct values for a_i excluding nulls. If d(a_i) is close to N'(a_i) then do not create any index array: check whether there is a match running through the records that have been filtered in. Anyway if d(a_i) is close to N'(a_i) then the intersection with rowid{a_i=v_i} will not reduce the number of remaining rowids significantly. In other words is V(a_i) exceeds some threshold there is no sense to create an index for a_i. If additionally N-N'(a_i) is small do not create a bitmap for this column either. 3. If for a column a_i d(a_i) is not close to N'(a_i), but N-N'(a_i) is small a sorted array of rowids from the set rowid{a_i is null} can be used instead of a bitmap. 4. We always have a match if R0= INTERSECT rowid{a_i is null} is not empty. Here i runs through all indexes from [1..n] such that v_i is not null. For a given subset of columns this fact has to be checked only once. It can be easily done with bitmap intersection. 5. If v1,...,vn never can be a null, then indexes (sorted arrays) can be created only for rows with nulls. 6. If v1,...,vn never can be a null and number of rows with nulls is small do not create indexes and do not create bitmaps. 7. If you get a row with nulls in all columns stop filling the temporary table and return UNKNOWN for any tuple <v1,...,vn>. 8. [timour] Consider that due to materialization, we already have a unique index on all columns <a_1,..., a_n>. We can use the first key part of this index over column a_1, instead of the index rowid{a_i=v_i}. Thus we can avoid creating the index rowid{a_i=v_i}. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Timour): Subquery optimization: Efficient NOT IN execution with NULLs (68)
by worklog-noreply＠askmonty.org 19 Jan '10

19 Jan '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Subquery optimization: Efficient NOT IN execution with NULLs CREATION DATE..: Fri, 27 Nov 2009, 13:22 SUPERVISOR.....: Monty IMPLEMENTOR....: Timour COPIES TO......: CATEGORY.......: Server-Sprint TASK ID........: 68 (http://askmonty.org/worklog/?tid=68) VERSION........: Server-9.x STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Timour - Tue, 19 Jan 2010, 18:29)=-=- High-Level Specification modified. --- /tmp/wklog.68.old.21045 2010-01-19 18:29:12.000000000 +0200 +++ /tmp/wklog.68.new.21045 2010-01-19 18:29:12.000000000 +0200 @@ -132,6 +132,8 @@ if (nonull_key && ! nonull_key->lookup(outer_ref)) return FALSE + if (nonull_key) + pq.insert(nonull_key) for (i = 1; i <= n; i++) { -=-=(Guest - Tue, 19 Jan 2010, 18:15)=-=- High-Level Specification modified. --- /tmp/wklog.68.old.19825 2010-01-19 18:15:30.000000000 +0200 +++ /tmp/wklog.68.new.19825 2010-01-19 18:15:30.000000000 +0200 @@ -1,8 +1,16 @@ -This a copy of the initial algorithm proposed by Igor: -====================================================== +Contents +======================================================================== -For each left side tuple (v_1,...,v_n) we have to find the following set -of rowids for the temp table containing N rows as the result of +1. Initial idea as proposed by Igor +2. Algorithm for IN execution with partial matching +3. Directions for improvement + + +1. Initial idea as proposed by Igor +======================================================================== + +For each left side tuple (v_1,...,v_n) we have to find the following +set of rowids for the temp table containing N rows as the result of materialization of the subquery: R= INTERSECT (rowid{a_i=v_i} UNION rowid{a_i is null} where i runs @@ -18,38 +26,198 @@ - it requires minimum memory: not more than N*n bits in total - search of an element in a set is extremely cheap -Taken all above into account I could suggest the following algorithm to -build R: +Taken all above into account I could suggest the following algorithm +to build R: - Using indexes (read about them below) for each column participating in the - intersection, - merge ordered sets rowid{a_i=v_i} in the following manner. + Using indexes (read about them below) for each column participating + in the intersection, merge ordered sets rowid{a_i=v_i} in the + following manner. If a rowid r has been encountered maximum in k sets -rowid{a_i1=v_i1},...,rowid(a_ik=v_ik), + rowid{a_i1=v_i1},...,rowid(a_ik=v_ik), then it has to be checked against all rowid{a_i=v_i} such that i is -not in {i1,...,ik}. + not in {i1,...,ik}. As soon as we fail to find r in one of these sets we discard it. If r has been found in all of them then r belongs to the set R. -Here we use the property (1): any r from rowid{a_i=v_i} UNION rowid{a_i -is null} is either +Here we use the property (1): +any r from rowid{a_i=v_i} UNION rowid{a_i is null} is either belongs to rowid{a_i=v_i} or to rowid{a_i is null}. From this we can -infer that for any r from R -indexes a_i can be uniquely divided into two groups: one contains -indexes a_i where r belongs to -the sets rowid{a_i=v_i}, the other contains indexes a_j such that r -belongs to rowid{a_j is null}. - -Now let's talk how to get elements from rowid{a_i=v_i} in a sorted order -needed for the merge procedure. We could use BTREE indexes for temp -table. But they are rather expensive and -take a lot of memory as the are implemented with RB trees. +infer that for any r from R indexes a_i can be uniquely divided into +two groups: +- one contains indexes a_i where r belongs to the sets rowid{a_i=v_i}, +- the other contains indexes a_j such that r belongs to + rowid{a_j is null}. + +Now let's talk how to get elements from rowid{a_i=v_i} in a sorted +order needed for the merge procedure. We could use BTREE indexes for +temp table. But they are rather expensive and take a lot of memory as +the are implemented with RB trees. I would suggest creating for each column from the temporary table just an array of rowids sorted by the value from column a. Index lookup in such an array is cheap. It's also rather cheap to check that the next rowid refers to a row with a different value in column a. The array can be created on demand. +2. Algorithm for IN execution with partial matching +======================================================================== + +2.1 Below is shown the top-level algorithm to execute an IN predicate +with partial matching. This algorithm is essentially the implementation +of Item_subselect:exec(). + +int lookup_with_null_semantics(outer_ref[], mat_subquery) +{ + if (index_lookup(outer_ref, mat_subquery) + return TRUE + else + { + /* + Check if there is a partial match (UNKNOWN) or no match (NULL). + */ + if (this is the first partial match) + { + vkey[] = build array of value keys for each NULL-able column + of mat_subquery. + nkey[] = build a bitmap NULL index for each column of mat_subquery + that contains NULLs + nonull_key = build a key over all non-NULL columns of mat_subquery + } + if (partial_match(outer_ref, vkey[], nkey[], nonull_key) + return UNKNOWN + else + return FALSE + } +} + +2.2 The implementation of partial matching is as follows + +/* + Assumptions: + - It has already been checked if there is a complete match by a + regular index lookup, and the test failed. + - It has already been checked if there is a complete NULL row, + and if there was we wouldn't call this function. Thus we assume + that there is no complete NULL row. + - Not all vidx_i are empty, but some can be empty. If all were empty, + then the only possibility for a match is a complete NULL row, which + we already checked. + + @param outer_ref - the uter (left) IN argument. + @param vidx[] - array of value keys + Ordered sequences of rowids of the corresponding columns a_i, such + that all rowids in idx_i are the ones where column a_i contains some + value or NULL. Each idx_i is derived dynamically, for each different + left argument of an IN predicate. + @param nidx[] - array of NULL keys + Bitmpas, one per each column, where a bit is set if the corresponding + row has a NULL value for the corresponding column. + @nonull_key - the only key over all columns of the materialized subquery + that do not contain NULLs + + @returns + @retval FALSE if there is no match + @retval TRUE if there is a partial match +*/ + +Boolean partial_match(outer_ref, vkey[], nkey[], nonull_key) +{ + /* Set of the keys (columns) that form a partial match. */ + Set matching_keys = {} + /* A subset of all keys that need to be checked for NULL matches. */ + Set null_keys = {} + Int min_key /* Key that contains the current minimum position. */ + Int min_row /* Current row number of min_key. */ + Int cur_min_key, cur_min_row + PriorityQueue pq + + if (nonull_key && ! nonull_key->lookup(outer_ref)) + return FALSE + + for (i = 1; i <= n; i++) + { + vkey[i].lookup(outer_ref) + if (! vkey[i].is_eof()) + pq.insert(i) + } + /* + Not all value keys are empty, thus we don't have only NULL + keys. If we had, the only possible match is a NULL row, and + we cheked there is no such row, therefore the result is known + to be FALSE. + In fact this algorithm makes sense for at least two non-NULL + columns. + */ + assert(pq.elements > 1) + + (min_key, min_row) = pq.pop() + matching_keys.add(min_key) + vkey[min_key].next() + if (! vkey[min_key].is_eof()) + pq.insert(min_key) + + while (TRUE) + { + (cur_min_key, cur_min_row) = pq.pop() + + if (cur_min_row == min_row) + { + matching_keys.add(cur_min_key) + /* There cannot be a complete match, as we already checked for one. */ + assert(matching_keys.elements < n) + } + else if (cur_min_key == nonull_key) + { + /* + The non-NULL key has no corresponding NULL index, so we know for + sure that the row 'min_row' is not a match. + */ + (min_key, min_row) = (cur_min_key, cur_min_row) + matching_keys = {min_key} + } + else + { + assert(cur_min_row > min_row) /* Follows from the use of PQ. */ + null_keys = set_difference(all keys vkey[], matching_keys) + /* + Check if all null_keys contain a NULL at row 'min_row'. The procedure + internally checks all keys in a special precomputed order. A prior + procedure determines an optimal order and a mapping + idx_no -> idx_order (encoded as an array). + */ + if (test_null_row(null_keys, min_row)) + return TRUE + else + { + (min_key, min_row) = (cur_min_key, cur_min_row) + matching_keys = {min_key} + } + } + + vkey[cur_min_key].next() + if (! vkey[cur_min_key].is_eof()) + pq.insert(cur_min_key) + + if (pq.is_empty()) + { + /* Check the last row of the last column in PQ for NULL matches. */ + null_keys = set_difference(all keys vkey[], matching_keys) + if (test_null_row(null_keys, min_row)) + return TRUE + else + return FALSE + } + } + + /* We should never get here. */ + assert(FALSE) + return FALSE +} + + + +3. Directions for improvement +======================================================================== + Other consideration that may be taken into account: 1. If columns a_j1,...,a_jm do not contain null values in the temporary -=-=(Timour - Sun, 06 Dec 2009, 14:36)=-=- High-Level Specification modified. --- /tmp/wklog.68.old.12919 2009-12-06 14:36:18.000000000 +0200 +++ /tmp/wklog.68.new.12919 2009-12-06 14:36:18.000000000 +0200 @@ -87,3 +87,8 @@ 7. If you get a row with nulls in all columns stop filling the temporary table and return UNKNOWN for any tuple <v1,...,vn>. +8. [timour] + Consider that due to materialization, we already have a unique index +on all columns <a_1,..., a_n>. We can use the first key part of this index +over column a_1, instead of the index rowid{a_i=v_i}. Thus we can avoid +creating the index rowid{a_i=v_i}. -=-=(Timour - Fri, 04 Dec 2009, 14:04)=-=- High-Level Specification modified. --- /tmp/wklog.68.old.16724 2009-12-04 14:04:28.000000000 +0200 +++ /tmp/wklog.68.new.16724 2009-12-04 14:04:28.000000000 +0200 @@ -10,7 +10,8 @@ Bear in mind the following specifics of this intersection: (1) For each i: rowid{a_i=v_i} and rowid{a_i is null} are disjoint - (2) For each i: rowid{a_i is null} is the same for each tuple + (2) For each i: rowid{a_i is null} is the same for each tuple, + that is, this set is independent of the left-side tuples. Due to (2) it makes sense to build rowid{a_i is null} only once. A good representation for such sets would be bitmaps: -=-=(Timour - Fri, 04 Dec 2009, 11:27)=-=- Version updated. --- /tmp/wklog.68.old.5257 2009-12-04 11:27:11.000000000 +0200 +++ /tmp/wklog.68.new.5257 2009-12-04 11:27:11.000000000 +0200 @@ -1 +1 @@ -Benchmarks-3.0 +Server-9.x -=-=(Timour - Fri, 04 Dec 2009, 11:27)=-=- Category updated. --- /tmp/wklog.68.old.5242 2009-12-04 11:27:02.000000000 +0200 +++ /tmp/wklog.68.new.5242 2009-12-04 11:27:02.000000000 +0200 @@ -1 +1 @@ -Server-RawIdeaBin +Server-Sprint -=-=(Timour - Fri, 04 Dec 2009, 11:27)=-=- Status updated. --- /tmp/wklog.68.old.5242 2009-12-04 11:27:02.000000000 +0200 +++ /tmp/wklog.68.new.5242 2009-12-04 11:27:02.000000000 +0200 @@ -1 +1 @@ -Un-Assigned +Assigned -=-=(Timour - Fri, 04 Dec 2009, 11:26)=-=- High-Level Specification modified. --- /tmp/wklog.68.old.5182 2009-12-04 11:26:25.000000000 +0200 +++ /tmp/wklog.68.new.5182 2009-12-04 11:26:25.000000000 +0200 @@ -50,23 +50,39 @@ The array can be created on demand. Other consideration that may be taken into account: + 1. If columns a_j1,...,a_jm do not contain null values in the temporary -table at all, create for them only one index array (and of course do not -create any bitmaps for them). -2. Consider the ratio d(a_i)=N'/ V(a_i), where N' is the number of rows -where a_i is not null and V(a_i) is the number of distinct values for -a_i excluding nulls. - If d(a_i) is close to 1 then do not create any index array: check +table at all and v_j1,...,v_jm cannot be null, create for these columns +only one index array (and of course do not create any bitmaps for them). + +2. Consider the ratio d(a_i)=N'(a_i)/V(a_i), where N'(a_i) is the number +of rows, where a_i is not null and V(a_i) is the number of distinct +values for a_i excluding nulls. +If d(a_i) is close to N'(a_i) then do not create any index array: check whether there is a match running through the records that have been -filtered in. Anyway if d(a_i) is close to 1 then a intersection with -rowid{a_i=v_i} would not reduce the number of remaining rowids +filtered in. Anyway if d(a_i) is close to N'(a_i) then the intersection + with rowid{a_i=v_i} will not reduce the number of remaining rowids significantly. - If additionally N-N' is small do not create a bitmap for this column -either. -3. If for a column a_i d(a_i) is not close to 1, but N-N' is small a -sorted array of rowids from the set rowid{a_i is null} can be used -instead of a bitmap. +In other words is V(a_i) exceeds some threshold there is no sense to +create an index for a_i. +If additionally N-N'(a_i) is small do not create a bitmap for this +column either. + +3. If for a column a_i d(a_i) is not close to N'(a_i), but N-N'(a_i) is +small a sorted array of rowids from the set rowid{a_i is null} can be +used instead of a bitmap. + 4. We always have a match if R0= INTERSECT rowid{a_i is null} is not empty. Here i runs through all indexes from [1..n] such that v_i is not null. For a given subset of columns this fact has to be checked only once. It can be easily done with bitmap intersection. + +5. If v1,...,vn never can be a null, then indexes (sorted arrays) can be +created only for rows with nulls. + +6. If v1,...,vn never can be a null and number of rows with nulls is +small do not create indexes and do not create bitmaps. + +7. If you get a row with nulls in all columns stop filling the temporary +table and return UNKNOWN for any tuple <v1,...,vn>. + -=-=(Timour - Fri, 27 Nov 2009, 13:23)=-=- High-Level Specification modified. --- /tmp/wklog.68.old.17140 2009-11-27 11:23:17.000000000 +0000 +++ /tmp/wklog.68.new.17140 2009-11-27 11:23:17.000000000 +0000 @@ -1 +1,72 @@ +This a copy of the initial algorithm proposed by Igor: +====================================================== +For each left side tuple (v_1,...,v_n) we have to find the following set +of rowids for the temp table containing N rows as the result of +materialization of the subquery: + + R= INTERSECT (rowid{a_i=v_i} UNION rowid{a_i is null} where i runs +trough all indexes from [1..n] such that v_i is not null. + +Bear in mind the following specifics of this intersection: + (1) For each i: rowid{a_i=v_i} and rowid{a_i is null} are disjoint + (2) For each i: rowid{a_i is null} is the same for each tuple + +Due to (2) it makes sense to build rowid{a_i is null} only once. +A good representation for such sets would be bitmaps: +- it requires minimum memory: not more than N*n bits in total +- search of an element in a set is extremely cheap + +Taken all above into account I could suggest the following algorithm to +build R: + + Using indexes (read about them below) for each column participating in the + intersection, + merge ordered sets rowid{a_i=v_i} in the following manner. + If a rowid r has been encountered maximum in k sets +rowid{a_i1=v_i1},...,rowid(a_ik=v_ik), + then it has to be checked against all rowid{a_i=v_i} such that i is +not in {i1,...,ik}. + As soon as we fail to find r in one of these sets we discard it. + If r has been found in all of them then r belongs to the set R. + +Here we use the property (1): any r from rowid{a_i=v_i} UNION rowid{a_i +is null} is either +belongs to rowid{a_i=v_i} or to rowid{a_i is null}. From this we can +infer that for any r from R +indexes a_i can be uniquely divided into two groups: one contains +indexes a_i where r belongs to +the sets rowid{a_i=v_i}, the other contains indexes a_j such that r +belongs to rowid{a_j is null}. + +Now let's talk how to get elements from rowid{a_i=v_i} in a sorted order +needed for the merge procedure. We could use BTREE indexes for temp +table. But they are rather expensive and +take a lot of memory as the are implemented with RB trees. +I would suggest creating for each column from the temporary table just +an array of rowids sorted by the value from column a. +Index lookup in such an array is cheap. It's also rather cheap to check +that the next rowid refers to a row with a different value in column a. +The array can be created on demand. + +Other consideration that may be taken into account: +1. If columns a_j1,...,a_jm do not contain null values in the temporary +table at all, create for them only one index array (and of course do not +create any bitmaps for them). +2. Consider the ratio d(a_i)=N'/ V(a_i), where N' is the number of rows +where a_i is not null and V(a_i) is the number of distinct values for +a_i excluding nulls. + If d(a_i) is close to 1 then do not create any index array: check +whether there is a match running through the records that have been +filtered in. Anyway if d(a_i) is close to 1 then a intersection with +rowid{a_i=v_i} would not reduce the number of remaining rowids +significantly. + If additionally N-N' is small do not create a bitmap for this column +either. +3. If for a column a_i d(a_i) is not close to 1, but N-N' is small a +sorted array of rowids from the set rowid{a_i is null} can be used +instead of a bitmap. +4. We always have a match if R0= INTERSECT rowid{a_i is null} is not +empty. Here i runs through all indexes from [1..n] such that v_i is not +null. For a given subset of columns this fact has to be checked only +once. It can be easily done with bitmap intersection. DESCRIPTION: The goal of this task is to implement efficient execution of NOT IN subquery predicates of the form: <oe_1,...,oe_n> NOT IN <non_correlated subquery> when either some oe_i, or some subqury result column contains NULLs. The problem with such predicates is that it is possible to use index lookups only when neither argument of the predicate contains NULLs. If some argument contains a NULL, then due to NULL semantics, it plays the role of a wildcard. If we were to use regular index lookups, then we would get 'no match' for some outer tuple (thus the predicate evaluates to FALSE), while the SQL semantics means 'partial match', and the predicate should evaluate to NULL. This task implements an efficient algorithm to compute such 'parial matches', where a NULL matches any value. HIGH-LEVEL SPECIFICATION: Contents ======================================================================== 1. Initial idea as proposed by Igor 2. Algorithm for IN execution with partial matching 3. Directions for improvement 1. Initial idea as proposed by Igor ======================================================================== For each left side tuple (v_1,...,v_n) we have to find the following set of rowids for the temp table containing N rows as the result of materialization of the subquery: R= INTERSECT (rowid{a_i=v_i} UNION rowid{a_i is null} where i runs trough all indexes from [1..n] such that v_i is not null. Bear in mind the following specifics of this intersection: (1) For each i: rowid{a_i=v_i} and rowid{a_i is null} are disjoint (2) For each i: rowid{a_i is null} is the same for each tuple, that is, this set is independent of the left-side tuples. Due to (2) it makes sense to build rowid{a_i is null} only once. A good representation for such sets would be bitmaps: - it requires minimum memory: not more than N*n bits in total - search of an element in a set is extremely cheap Taken all above into account I could suggest the following algorithm to build R: Using indexes (read about them below) for each column participating in the intersection, merge ordered sets rowid{a_i=v_i} in the following manner. If a rowid r has been encountered maximum in k sets rowid{a_i1=v_i1},...,rowid(a_ik=v_ik), then it has to be checked against all rowid{a_i=v_i} such that i is not in {i1,...,ik}. As soon as we fail to find r in one of these sets we discard it. If r has been found in all of them then r belongs to the set R. Here we use the property (1): any r from rowid{a_i=v_i} UNION rowid{a_i is null} is either belongs to rowid{a_i=v_i} or to rowid{a_i is null}. From this we can infer that for any r from R indexes a_i can be uniquely divided into two groups: - one contains indexes a_i where r belongs to the sets rowid{a_i=v_i}, - the other contains indexes a_j such that r belongs to rowid{a_j is null}. Now let's talk how to get elements from rowid{a_i=v_i} in a sorted order needed for the merge procedure. We could use BTREE indexes for temp table. But they are rather expensive and take a lot of memory as the are implemented with RB trees. I would suggest creating for each column from the temporary table just an array of rowids sorted by the value from column a. Index lookup in such an array is cheap. It's also rather cheap to check that the next rowid refers to a row with a different value in column a. The array can be created on demand. 2. Algorithm for IN execution with partial matching ======================================================================== 2.1 Below is shown the top-level algorithm to execute an IN predicate with partial matching. This algorithm is essentially the implementation of Item_subselect:exec(). int lookup_with_null_semantics(outer_ref[], mat_subquery) { if (index_lookup(outer_ref, mat_subquery) return TRUE else { /* Check if there is a partial match (UNKNOWN) or no match (NULL). */ if (this is the first partial match) { vkey[] = build array of value keys for each NULL-able column of mat_subquery. nkey[] = build a bitmap NULL index for each column of mat_subquery that contains NULLs nonull_key = build a key over all non-NULL columns of mat_subquery } if (partial_match(outer_ref, vkey[], nkey[], nonull_key) return UNKNOWN else return FALSE } } 2.2 The implementation of partial matching is as follows /* Assumptions: - It has already been checked if there is a complete match by a regular index lookup, and the test failed. - It has already been checked if there is a complete NULL row, and if there was we wouldn't call this function. Thus we assume that there is no complete NULL row. - Not all vidx_i are empty, but some can be empty. If all were empty, then the only possibility for a match is a complete NULL row, which we already checked. @param outer_ref - the uter (left) IN argument. @param vidx[] - array of value keys Ordered sequences of rowids of the corresponding columns a_i, such that all rowids in idx_i are the ones where column a_i contains some value or NULL. Each idx_i is derived dynamically, for each different left argument of an IN predicate. @param nidx[] - array of NULL keys Bitmpas, one per each column, where a bit is set if the corresponding row has a NULL value for the corresponding column. @nonull_key - the only key over all columns of the materialized subquery that do not contain NULLs @returns @retval FALSE if there is no match @retval TRUE if there is a partial match */ Boolean partial_match(outer_ref, vkey[], nkey[], nonull_key) { /* Set of the keys (columns) that form a partial match. */ Set matching_keys = {} /* A subset of all keys that need to be checked for NULL matches. */ Set null_keys = {} Int min_key /* Key that contains the current minimum position. */ Int min_row /* Current row number of min_key. */ Int cur_min_key, cur_min_row PriorityQueue pq if (nonull_key && ! nonull_key->lookup(outer_ref)) return FALSE if (nonull_key) pq.insert(nonull_key) for (i = 1; i <= n; i++) { vkey[i].lookup(outer_ref) if (! vkey[i].is_eof()) pq.insert(i) } /* Not all value keys are empty, thus we don't have only NULL keys. If we had, the only possible match is a NULL row, and we cheked there is no such row, therefore the result is known to be FALSE. In fact this algorithm makes sense for at least two non-NULL columns. */ assert(pq.elements > 1) (min_key, min_row) = pq.pop() matching_keys.add(min_key) vkey[min_key].next() if (! vkey[min_key].is_eof()) pq.insert(min_key) while (TRUE) { (cur_min_key, cur_min_row) = pq.pop() if (cur_min_row == min_row) { matching_keys.add(cur_min_key) /* There cannot be a complete match, as we already checked for one. */ assert(matching_keys.elements < n) } else if (cur_min_key == nonull_key) { /* The non-NULL key has no corresponding NULL index, so we know for sure that the row 'min_row' is not a match. */ (min_key, min_row) = (cur_min_key, cur_min_row) matching_keys = {min_key} } else { assert(cur_min_row > min_row) /* Follows from the use of PQ. */ null_keys = set_difference(all keys vkey[], matching_keys) /* Check if all null_keys contain a NULL at row 'min_row'. The procedure internally checks all keys in a special precomputed order. A prior procedure determines an optimal order and a mapping idx_no -> idx_order (encoded as an array). */ if (test_null_row(null_keys, min_row)) return TRUE else { (min_key, min_row) = (cur_min_key, cur_min_row) matching_keys = {min_key} } } vkey[cur_min_key].next() if (! vkey[cur_min_key].is_eof()) pq.insert(cur_min_key) if (pq.is_empty()) { /* Check the last row of the last column in PQ for NULL matches. */ null_keys = set_difference(all keys vkey[], matching_keys) if (test_null_row(null_keys, min_row)) return TRUE else return FALSE } } /* We should never get here. */ assert(FALSE) return FALSE } 3. Directions for improvement ======================================================================== Other consideration that may be taken into account: 1. If columns a_j1,...,a_jm do not contain null values in the temporary table at all and v_j1,...,v_jm cannot be null, create for these columns only one index array (and of course do not create any bitmaps for them). 2. Consider the ratio d(a_i)=N'(a_i)/V(a_i), where N'(a_i) is the number of rows, where a_i is not null and V(a_i) is the number of distinct values for a_i excluding nulls. If d(a_i) is close to N'(a_i) then do not create any index array: check whether there is a match running through the records that have been filtered in. Anyway if d(a_i) is close to N'(a_i) then the intersection with rowid{a_i=v_i} will not reduce the number of remaining rowids significantly. In other words is V(a_i) exceeds some threshold there is no sense to create an index for a_i. If additionally N-N'(a_i) is small do not create a bitmap for this column either. 3. If for a column a_i d(a_i) is not close to N'(a_i), but N-N'(a_i) is small a sorted array of rowids from the set rowid{a_i is null} can be used instead of a bitmap. 4. We always have a match if R0= INTERSECT rowid{a_i is null} is not empty. Here i runs through all indexes from [1..n] such that v_i is not null. For a given subset of columns this fact has to be checked only once. It can be easily done with bitmap intersection. 5. If v1,...,vn never can be a null, then indexes (sorted arrays) can be created only for rows with nulls. 6. If v1,...,vn never can be a null and number of rows with nulls is small do not create indexes and do not create bitmaps. 7. If you get a row with nulls in all columns stop filling the temporary table and return UNKNOWN for any tuple <v1,...,vn>. 8. [timour] Consider that due to materialization, we already have a unique index on all columns <a_1,..., a_n>. We can use the first key part of this index over column a_1, instead of the index rowid{a_i=v_i}. Thus we can avoid creating the index rowid{a_i=v_i}. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Timour): Subquery optimization: Efficient NOT IN execution with NULLs (68)
by worklog-noreply＠askmonty.org 19 Jan '10

19 Jan '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Subquery optimization: Efficient NOT IN execution with NULLs CREATION DATE..: Fri, 27 Nov 2009, 13:22 SUPERVISOR.....: Monty IMPLEMENTOR....: Timour COPIES TO......: CATEGORY.......: Server-Sprint TASK ID........: 68 (http://askmonty.org/worklog/?tid=68) VERSION........: Server-9.x STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Timour - Tue, 19 Jan 2010, 18:29)=-=- High-Level Specification modified. --- /tmp/wklog.68.old.21045 2010-01-19 18:29:12.000000000 +0200 +++ /tmp/wklog.68.new.21045 2010-01-19 18:29:12.000000000 +0200 @@ -132,6 +132,8 @@ if (nonull_key && ! nonull_key->lookup(outer_ref)) return FALSE + if (nonull_key) + pq.insert(nonull_key) for (i = 1; i <= n; i++) { -=-=(Guest - Tue, 19 Jan 2010, 18:15)=-=- High-Level Specification modified. --- /tmp/wklog.68.old.19825 2010-01-19 18:15:30.000000000 +0200 +++ /tmp/wklog.68.new.19825 2010-01-19 18:15:30.000000000 +0200 @@ -1,8 +1,16 @@ -This a copy of the initial algorithm proposed by Igor: -====================================================== +Contents +======================================================================== -For each left side tuple (v_1,...,v_n) we have to find the following set -of rowids for the temp table containing N rows as the result of +1. Initial idea as proposed by Igor +2. Algorithm for IN execution with partial matching +3. Directions for improvement + + +1. Initial idea as proposed by Igor +======================================================================== + +For each left side tuple (v_1,...,v_n) we have to find the following +set of rowids for the temp table containing N rows as the result of materialization of the subquery: R= INTERSECT (rowid{a_i=v_i} UNION rowid{a_i is null} where i runs @@ -18,38 +26,198 @@ - it requires minimum memory: not more than N*n bits in total - search of an element in a set is extremely cheap -Taken all above into account I could suggest the following algorithm to -build R: +Taken all above into account I could suggest the following algorithm +to build R: - Using indexes (read about them below) for each column participating in the - intersection, - merge ordered sets rowid{a_i=v_i} in the following manner. + Using indexes (read about them below) for each column participating + in the intersection, merge ordered sets rowid{a_i=v_i} in the + following manner. If a rowid r has been encountered maximum in k sets -rowid{a_i1=v_i1},...,rowid(a_ik=v_ik), + rowid{a_i1=v_i1},...,rowid(a_ik=v_ik), then it has to be checked against all rowid{a_i=v_i} such that i is -not in {i1,...,ik}. + not in {i1,...,ik}. As soon as we fail to find r in one of these sets we discard it. If r has been found in all of them then r belongs to the set R. -Here we use the property (1): any r from rowid{a_i=v_i} UNION rowid{a_i -is null} is either +Here we use the property (1): +any r from rowid{a_i=v_i} UNION rowid{a_i is null} is either belongs to rowid{a_i=v_i} or to rowid{a_i is null}. From this we can -infer that for any r from R -indexes a_i can be uniquely divided into two groups: one contains -indexes a_i where r belongs to -the sets rowid{a_i=v_i}, the other contains indexes a_j such that r -belongs to rowid{a_j is null}. - -Now let's talk how to get elements from rowid{a_i=v_i} in a sorted order -needed for the merge procedure. We could use BTREE indexes for temp -table. But they are rather expensive and -take a lot of memory as the are implemented with RB trees. +infer that for any r from R indexes a_i can be uniquely divided into +two groups: +- one contains indexes a_i where r belongs to the sets rowid{a_i=v_i}, +- the other contains indexes a_j such that r belongs to + rowid{a_j is null}. + +Now let's talk how to get elements from rowid{a_i=v_i} in a sorted +order needed for the merge procedure. We could use BTREE indexes for +temp table. But they are rather expensive and take a lot of memory as +the are implemented with RB trees. I would suggest creating for each column from the temporary table just an array of rowids sorted by the value from column a. Index lookup in such an array is cheap. It's also rather cheap to check that the next rowid refers to a row with a different value in column a. The array can be created on demand. +2. Algorithm for IN execution with partial matching +======================================================================== + +2.1 Below is shown the top-level algorithm to execute an IN predicate +with partial matching. This algorithm is essentially the implementation +of Item_subselect:exec(). + +int lookup_with_null_semantics(outer_ref[], mat_subquery) +{ + if (index_lookup(outer_ref, mat_subquery) + return TRUE + else + { + /* + Check if there is a partial match (UNKNOWN) or no match (NULL). + */ + if (this is the first partial match) + { + vkey[] = build array of value keys for each NULL-able column + of mat_subquery. + nkey[] = build a bitmap NULL index for each column of mat_subquery + that contains NULLs + nonull_key = build a key over all non-NULL columns of mat_subquery + } + if (partial_match(outer_ref, vkey[], nkey[], nonull_key) + return UNKNOWN + else + return FALSE + } +} + +2.2 The implementation of partial matching is as follows + +/* + Assumptions: + - It has already been checked if there is a complete match by a + regular index lookup, and the test failed. + - It has already been checked if there is a complete NULL row, + and if there was we wouldn't call this function. Thus we assume + that there is no complete NULL row. + - Not all vidx_i are empty, but some can be empty. If all were empty, + then the only possibility for a match is a complete NULL row, which + we already checked. + + @param outer_ref - the uter (left) IN argument. + @param vidx[] - array of value keys + Ordered sequences of rowids of the corresponding columns a_i, such + that all rowids in idx_i are the ones where column a_i contains some + value or NULL. Each idx_i is derived dynamically, for each different + left argument of an IN predicate. + @param nidx[] - array of NULL keys + Bitmpas, one per each column, where a bit is set if the corresponding + row has a NULL value for the corresponding column. + @nonull_key - the only key over all columns of the materialized subquery + that do not contain NULLs + + @returns + @retval FALSE if there is no match + @retval TRUE if there is a partial match +*/ + +Boolean partial_match(outer_ref, vkey[], nkey[], nonull_key) +{ + /* Set of the keys (columns) that form a partial match. */ + Set matching_keys = {} + /* A subset of all keys that need to be checked for NULL matches. */ + Set null_keys = {} + Int min_key /* Key that contains the current minimum position. */ + Int min_row /* Current row number of min_key. */ + Int cur_min_key, cur_min_row + PriorityQueue pq + + if (nonull_key && ! nonull_key->lookup(outer_ref)) + return FALSE + + for (i = 1; i <= n; i++) + { + vkey[i].lookup(outer_ref) + if (! vkey[i].is_eof()) + pq.insert(i) + } + /* + Not all value keys are empty, thus we don't have only NULL + keys. If we had, the only possible match is a NULL row, and + we cheked there is no such row, therefore the result is known + to be FALSE. + In fact this algorithm makes sense for at least two non-NULL + columns. + */ + assert(pq.elements > 1) + + (min_key, min_row) = pq.pop() + matching_keys.add(min_key) + vkey[min_key].next() + if (! vkey[min_key].is_eof()) + pq.insert(min_key) + + while (TRUE) + { + (cur_min_key, cur_min_row) = pq.pop() + + if (cur_min_row == min_row) + { + matching_keys.add(cur_min_key) + /* There cannot be a complete match, as we already checked for one. */ + assert(matching_keys.elements < n) + } + else if (cur_min_key == nonull_key) + { + /* + The non-NULL key has no corresponding NULL index, so we know for + sure that the row 'min_row' is not a match. + */ + (min_key, min_row) = (cur_min_key, cur_min_row) + matching_keys = {min_key} + } + else + { + assert(cur_min_row > min_row) /* Follows from the use of PQ. */ + null_keys = set_difference(all keys vkey[], matching_keys) + /* + Check if all null_keys contain a NULL at row 'min_row'. The procedure + internally checks all keys in a special precomputed order. A prior + procedure determines an optimal order and a mapping + idx_no -> idx_order (encoded as an array). + */ + if (test_null_row(null_keys, min_row)) + return TRUE + else + { + (min_key, min_row) = (cur_min_key, cur_min_row) + matching_keys = {min_key} + } + } + + vkey[cur_min_key].next() + if (! vkey[cur_min_key].is_eof()) + pq.insert(cur_min_key) + + if (pq.is_empty()) + { + /* Check the last row of the last column in PQ for NULL matches. */ + null_keys = set_difference(all keys vkey[], matching_keys) + if (test_null_row(null_keys, min_row)) + return TRUE + else + return FALSE + } + } + + /* We should never get here. */ + assert(FALSE) + return FALSE +} + + + +3. Directions for improvement +======================================================================== + Other consideration that may be taken into account: 1. If columns a_j1,...,a_jm do not contain null values in the temporary -=-=(Timour - Sun, 06 Dec 2009, 14:36)=-=- High-Level Specification modified. --- /tmp/wklog.68.old.12919 2009-12-06 14:36:18.000000000 +0200 +++ /tmp/wklog.68.new.12919 2009-12-06 14:36:18.000000000 +0200 @@ -87,3 +87,8 @@ 7. If you get a row with nulls in all columns stop filling the temporary table and return UNKNOWN for any tuple <v1,...,vn>. +8. [timour] + Consider that due to materialization, we already have a unique index +on all columns <a_1,..., a_n>. We can use the first key part of this index +over column a_1, instead of the index rowid{a_i=v_i}. Thus we can avoid +creating the index rowid{a_i=v_i}. -=-=(Timour - Fri, 04 Dec 2009, 14:04)=-=- High-Level Specification modified. --- /tmp/wklog.68.old.16724 2009-12-04 14:04:28.000000000 +0200 +++ /tmp/wklog.68.new.16724 2009-12-04 14:04:28.000000000 +0200 @@ -10,7 +10,8 @@ Bear in mind the following specifics of this intersection: (1) For each i: rowid{a_i=v_i} and rowid{a_i is null} are disjoint - (2) For each i: rowid{a_i is null} is the same for each tuple + (2) For each i: rowid{a_i is null} is the same for each tuple, + that is, this set is independent of the left-side tuples. Due to (2) it makes sense to build rowid{a_i is null} only once. A good representation for such sets would be bitmaps: -=-=(Timour - Fri, 04 Dec 2009, 11:27)=-=- Version updated. --- /tmp/wklog.68.old.5257 2009-12-04 11:27:11.000000000 +0200 +++ /tmp/wklog.68.new.5257 2009-12-04 11:27:11.000000000 +0200 @@ -1 +1 @@ -Benchmarks-3.0 +Server-9.x -=-=(Timour - Fri, 04 Dec 2009, 11:27)=-=- Category updated. --- /tmp/wklog.68.old.5242 2009-12-04 11:27:02.000000000 +0200 +++ /tmp/wklog.68.new.5242 2009-12-04 11:27:02.000000000 +0200 @@ -1 +1 @@ -Server-RawIdeaBin +Server-Sprint -=-=(Timour - Fri, 04 Dec 2009, 11:27)=-=- Status updated. --- /tmp/wklog.68.old.5242 2009-12-04 11:27:02.000000000 +0200 +++ /tmp/wklog.68.new.5242 2009-12-04 11:27:02.000000000 +0200 @@ -1 +1 @@ -Un-Assigned +Assigned -=-=(Timour - Fri, 04 Dec 2009, 11:26)=-=- High-Level Specification modified. --- /tmp/wklog.68.old.5182 2009-12-04 11:26:25.000000000 +0200 +++ /tmp/wklog.68.new.5182 2009-12-04 11:26:25.000000000 +0200 @@ -50,23 +50,39 @@ The array can be created on demand. Other consideration that may be taken into account: + 1. If columns a_j1,...,a_jm do not contain null values in the temporary -table at all, create for them only one index array (and of course do not -create any bitmaps for them). -2. Consider the ratio d(a_i)=N'/ V(a_i), where N' is the number of rows -where a_i is not null and V(a_i) is the number of distinct values for -a_i excluding nulls. - If d(a_i) is close to 1 then do not create any index array: check +table at all and v_j1,...,v_jm cannot be null, create for these columns +only one index array (and of course do not create any bitmaps for them). + +2. Consider the ratio d(a_i)=N'(a_i)/V(a_i), where N'(a_i) is the number +of rows, where a_i is not null and V(a_i) is the number of distinct +values for a_i excluding nulls. +If d(a_i) is close to N'(a_i) then do not create any index array: check whether there is a match running through the records that have been -filtered in. Anyway if d(a_i) is close to 1 then a intersection with -rowid{a_i=v_i} would not reduce the number of remaining rowids +filtered in. Anyway if d(a_i) is close to N'(a_i) then the intersection + with rowid{a_i=v_i} will not reduce the number of remaining rowids significantly. - If additionally N-N' is small do not create a bitmap for this column -either. -3. If for a column a_i d(a_i) is not close to 1, but N-N' is small a -sorted array of rowids from the set rowid{a_i is null} can be used -instead of a bitmap. +In other words is V(a_i) exceeds some threshold there is no sense to +create an index for a_i. +If additionally N-N'(a_i) is small do not create a bitmap for this +column either. + +3. If for a column a_i d(a_i) is not close to N'(a_i), but N-N'(a_i) is +small a sorted array of rowids from the set rowid{a_i is null} can be +used instead of a bitmap. + 4. We always have a match if R0= INTERSECT rowid{a_i is null} is not empty. Here i runs through all indexes from [1..n] such that v_i is not null. For a given subset of columns this fact has to be checked only once. It can be easily done with bitmap intersection. + +5. If v1,...,vn never can be a null, then indexes (sorted arrays) can be +created only for rows with nulls. + +6. If v1,...,vn never can be a null and number of rows with nulls is +small do not create indexes and do not create bitmaps. + +7. If you get a row with nulls in all columns stop filling the temporary +table and return UNKNOWN for any tuple <v1,...,vn>. + -=-=(Timour - Fri, 27 Nov 2009, 13:23)=-=- High-Level Specification modified. --- /tmp/wklog.68.old.17140 2009-11-27 11:23:17.000000000 +0000 +++ /tmp/wklog.68.new.17140 2009-11-27 11:23:17.000000000 +0000 @@ -1 +1,72 @@ +This a copy of the initial algorithm proposed by Igor: +====================================================== +For each left side tuple (v_1,...,v_n) we have to find the following set +of rowids for the temp table containing N rows as the result of +materialization of the subquery: + + R= INTERSECT (rowid{a_i=v_i} UNION rowid{a_i is null} where i runs +trough all indexes from [1..n] such that v_i is not null. + +Bear in mind the following specifics of this intersection: + (1) For each i: rowid{a_i=v_i} and rowid{a_i is null} are disjoint + (2) For each i: rowid{a_i is null} is the same for each tuple + +Due to (2) it makes sense to build rowid{a_i is null} only once. +A good representation for such sets would be bitmaps: +- it requires minimum memory: not more than N*n bits in total +- search of an element in a set is extremely cheap + +Taken all above into account I could suggest the following algorithm to +build R: + + Using indexes (read about them below) for each column participating in the + intersection, + merge ordered sets rowid{a_i=v_i} in the following manner. + If a rowid r has been encountered maximum in k sets +rowid{a_i1=v_i1},...,rowid(a_ik=v_ik), + then it has to be checked against all rowid{a_i=v_i} such that i is +not in {i1,...,ik}. + As soon as we fail to find r in one of these sets we discard it. + If r has been found in all of them then r belongs to the set R. + +Here we use the property (1): any r from rowid{a_i=v_i} UNION rowid{a_i +is null} is either +belongs to rowid{a_i=v_i} or to rowid{a_i is null}. From this we can +infer that for any r from R +indexes a_i can be uniquely divided into two groups: one contains +indexes a_i where r belongs to +the sets rowid{a_i=v_i}, the other contains indexes a_j such that r +belongs to rowid{a_j is null}. + +Now let's talk how to get elements from rowid{a_i=v_i} in a sorted order +needed for the merge procedure. We could use BTREE indexes for temp +table. But they are rather expensive and +take a lot of memory as the are implemented with RB trees. +I would suggest creating for each column from the temporary table just +an array of rowids sorted by the value from column a. +Index lookup in such an array is cheap. It's also rather cheap to check +that the next rowid refers to a row with a different value in column a. +The array can be created on demand. + +Other consideration that may be taken into account: +1. If columns a_j1,...,a_jm do not contain null values in the temporary +table at all, create for them only one index array (and of course do not +create any bitmaps for them). +2. Consider the ratio d(a_i)=N'/ V(a_i), where N' is the number of rows +where a_i is not null and V(a_i) is the number of distinct values for +a_i excluding nulls. + If d(a_i) is close to 1 then do not create any index array: check +whether there is a match running through the records that have been +filtered in. Anyway if d(a_i) is close to 1 then a intersection with +rowid{a_i=v_i} would not reduce the number of remaining rowids +significantly. + If additionally N-N' is small do not create a bitmap for this column +either. +3. If for a column a_i d(a_i) is not close to 1, but N-N' is small a +sorted array of rowids from the set rowid{a_i is null} can be used +instead of a bitmap. +4. We always have a match if R0= INTERSECT rowid{a_i is null} is not +empty. Here i runs through all indexes from [1..n] such that v_i is not +null. For a given subset of columns this fact has to be checked only +once. It can be easily done with bitmap intersection. DESCRIPTION: The goal of this task is to implement efficient execution of NOT IN subquery predicates of the form: <oe_1,...,oe_n> NOT IN <non_correlated subquery> when either some oe_i, or some subqury result column contains NULLs. The problem with such predicates is that it is possible to use index lookups only when neither argument of the predicate contains NULLs. If some argument contains a NULL, then due to NULL semantics, it plays the role of a wildcard. If we were to use regular index lookups, then we would get 'no match' for some outer tuple (thus the predicate evaluates to FALSE), while the SQL semantics means 'partial match', and the predicate should evaluate to NULL. This task implements an efficient algorithm to compute such 'parial matches', where a NULL matches any value. HIGH-LEVEL SPECIFICATION: Contents ======================================================================== 1. Initial idea as proposed by Igor 2. Algorithm for IN execution with partial matching 3. Directions for improvement 1. Initial idea as proposed by Igor ======================================================================== For each left side tuple (v_1,...,v_n) we have to find the following set of rowids for the temp table containing N rows as the result of materialization of the subquery: R= INTERSECT (rowid{a_i=v_i} UNION rowid{a_i is null} where i runs trough all indexes from [1..n] such that v_i is not null. Bear in mind the following specifics of this intersection: (1) For each i: rowid{a_i=v_i} and rowid{a_i is null} are disjoint (2) For each i: rowid{a_i is null} is the same for each tuple, that is, this set is independent of the left-side tuples. Due to (2) it makes sense to build rowid{a_i is null} only once. A good representation for such sets would be bitmaps: - it requires minimum memory: not more than N*n bits in total - search of an element in a set is extremely cheap Taken all above into account I could suggest the following algorithm to build R: Using indexes (read about them below) for each column participating in the intersection, merge ordered sets rowid{a_i=v_i} in the following manner. If a rowid r has been encountered maximum in k sets rowid{a_i1=v_i1},...,rowid(a_ik=v_ik), then it has to be checked against all rowid{a_i=v_i} such that i is not in {i1,...,ik}. As soon as we fail to find r in one of these sets we discard it. If r has been found in all of them then r belongs to the set R. Here we use the property (1): any r from rowid{a_i=v_i} UNION rowid{a_i is null} is either belongs to rowid{a_i=v_i} or to rowid{a_i is null}. From this we can infer that for any r from R indexes a_i can be uniquely divided into two groups: - one contains indexes a_i where r belongs to the sets rowid{a_i=v_i}, - the other contains indexes a_j such that r belongs to rowid{a_j is null}. Now let's talk how to get elements from rowid{a_i=v_i} in a sorted order needed for the merge procedure. We could use BTREE indexes for temp table. But they are rather expensive and take a lot of memory as the are implemented with RB trees. I would suggest creating for each column from the temporary table just an array of rowids sorted by the value from column a. Index lookup in such an array is cheap. It's also rather cheap to check that the next rowid refers to a row with a different value in column a. The array can be created on demand. 2. Algorithm for IN execution with partial matching ======================================================================== 2.1 Below is shown the top-level algorithm to execute an IN predicate with partial matching. This algorithm is essentially the implementation of Item_subselect:exec(). int lookup_with_null_semantics(outer_ref[], mat_subquery) { if (index_lookup(outer_ref, mat_subquery) return TRUE else { /* Check if there is a partial match (UNKNOWN) or no match (NULL). */ if (this is the first partial match) { vkey[] = build array of value keys for each NULL-able column of mat_subquery. nkey[] = build a bitmap NULL index for each column of mat_subquery that contains NULLs nonull_key = build a key over all non-NULL columns of mat_subquery } if (partial_match(outer_ref, vkey[], nkey[], nonull_key) return UNKNOWN else return FALSE } } 2.2 The implementation of partial matching is as follows /* Assumptions: - It has already been checked if there is a complete match by a regular index lookup, and the test failed. - It has already been checked if there is a complete NULL row, and if there was we wouldn't call this function. Thus we assume that there is no complete NULL row. - Not all vidx_i are empty, but some can be empty. If all were empty, then the only possibility for a match is a complete NULL row, which we already checked. @param outer_ref - the uter (left) IN argument. @param vidx[] - array of value keys Ordered sequences of rowids of the corresponding columns a_i, such that all rowids in idx_i are the ones where column a_i contains some value or NULL. Each idx_i is derived dynamically, for each different left argument of an IN predicate. @param nidx[] - array of NULL keys Bitmpas, one per each column, where a bit is set if the corresponding row has a NULL value for the corresponding column. @nonull_key - the only key over all columns of the materialized subquery that do not contain NULLs @returns @retval FALSE if there is no match @retval TRUE if there is a partial match */ Boolean partial_match(outer_ref, vkey[], nkey[], nonull_key) { /* Set of the keys (columns) that form a partial match. */ Set matching_keys = {} /* A subset of all keys that need to be checked for NULL matches. */ Set null_keys = {} Int min_key /* Key that contains the current minimum position. */ Int min_row /* Current row number of min_key. */ Int cur_min_key, cur_min_row PriorityQueue pq if (nonull_key && ! nonull_key->lookup(outer_ref)) return FALSE if (nonull_key) pq.insert(nonull_key) for (i = 1; i <= n; i++) { vkey[i].lookup(outer_ref) if (! vkey[i].is_eof()) pq.insert(i) } /* Not all value keys are empty, thus we don't have only NULL keys. If we had, the only possible match is a NULL row, and we cheked there is no such row, therefore the result is known to be FALSE. In fact this algorithm makes sense for at least two non-NULL columns. */ assert(pq.elements > 1) (min_key, min_row) = pq.pop() matching_keys.add(min_key) vkey[min_key].next() if (! vkey[min_key].is_eof()) pq.insert(min_key) while (TRUE) { (cur_min_key, cur_min_row) = pq.pop() if (cur_min_row == min_row) { matching_keys.add(cur_min_key) /* There cannot be a complete match, as we already checked for one. */ assert(matching_keys.elements < n) } else if (cur_min_key == nonull_key) { /* The non-NULL key has no corresponding NULL index, so we know for sure that the row 'min_row' is not a match. */ (min_key, min_row) = (cur_min_key, cur_min_row) matching_keys = {min_key} } else { assert(cur_min_row > min_row) /* Follows from the use of PQ. */ null_keys = set_difference(all keys vkey[], matching_keys) /* Check if all null_keys contain a NULL at row 'min_row'. The procedure internally checks all keys in a special precomputed order. A prior procedure determines an optimal order and a mapping idx_no -> idx_order (encoded as an array). */ if (test_null_row(null_keys, min_row)) return TRUE else { (min_key, min_row) = (cur_min_key, cur_min_row) matching_keys = {min_key} } } vkey[cur_min_key].next() if (! vkey[cur_min_key].is_eof()) pq.insert(cur_min_key) if (pq.is_empty()) { /* Check the last row of the last column in PQ for NULL matches. */ null_keys = set_difference(all keys vkey[], matching_keys) if (test_null_row(null_keys, min_row)) return TRUE else return FALSE } } /* We should never get here. */ assert(FALSE) return FALSE } 3. Directions for improvement ======================================================================== Other consideration that may be taken into account: 1. If columns a_j1,...,a_jm do not contain null values in the temporary table at all and v_j1,...,v_jm cannot be null, create for these columns only one index array (and of course do not create any bitmaps for them). 2. Consider the ratio d(a_i)=N'(a_i)/V(a_i), where N'(a_i) is the number of rows, where a_i is not null and V(a_i) is the number of distinct values for a_i excluding nulls. If d(a_i) is close to N'(a_i) then do not create any index array: check whether there is a match running through the records that have been filtered in. Anyway if d(a_i) is close to N'(a_i) then the intersection with rowid{a_i=v_i} will not reduce the number of remaining rowids significantly. In other words is V(a_i) exceeds some threshold there is no sense to create an index for a_i. If additionally N-N'(a_i) is small do not create a bitmap for this column either. 3. If for a column a_i d(a_i) is not close to N'(a_i), but N-N'(a_i) is small a sorted array of rowids from the set rowid{a_i is null} can be used instead of a bitmap. 4. We always have a match if R0= INTERSECT rowid{a_i is null} is not empty. Here i runs through all indexes from [1..n] such that v_i is not null. For a given subset of columns this fact has to be checked only once. It can be easily done with bitmap intersection. 5. If v1,...,vn never can be a null, then indexes (sorted arrays) can be created only for rows with nulls. 6. If v1,...,vn never can be a null and number of rows with nulls is small do not create indexes and do not create bitmaps. 7. If you get a row with nulls in all columns stop filling the temporary table and return UNKNOWN for any tuple <v1,...,vn>. 8. [timour] Consider that due to materialization, we already have a unique index on all columns <a_1,..., a_n>. We can use the first key part of this index over column a_1, instead of the index rowid{a_i=v_i}. Thus we can avoid creating the index rowid{a_i=v_i}. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Guest): Subquery optimization: Efficient NOT IN execution with NULLs (68)
by worklog-noreply＠askmonty.org 19 Jan '10

19 Jan '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Subquery optimization: Efficient NOT IN execution with NULLs CREATION DATE..: Fri, 27 Nov 2009, 13:22 SUPERVISOR.....: Monty IMPLEMENTOR....: Timour COPIES TO......: CATEGORY.......: Server-Sprint TASK ID........: 68 (http://askmonty.org/worklog/?tid=68) VERSION........: Server-9.x STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Guest - Tue, 19 Jan 2010, 18:15)=-=- High-Level Specification modified. --- /tmp/wklog.68.old.19825 2010-01-19 18:15:30.000000000 +0200 +++ /tmp/wklog.68.new.19825 2010-01-19 18:15:30.000000000 +0200 @@ -1,8 +1,16 @@ -This a copy of the initial algorithm proposed by Igor: -====================================================== +Contents +======================================================================== -For each left side tuple (v_1,...,v_n) we have to find the following set -of rowids for the temp table containing N rows as the result of +1. Initial idea as proposed by Igor +2. Algorithm for IN execution with partial matching +3. Directions for improvement + + +1. Initial idea as proposed by Igor +======================================================================== + +For each left side tuple (v_1,...,v_n) we have to find the following +set of rowids for the temp table containing N rows as the result of materialization of the subquery: R= INTERSECT (rowid{a_i=v_i} UNION rowid{a_i is null} where i runs @@ -18,38 +26,198 @@ - it requires minimum memory: not more than N*n bits in total - search of an element in a set is extremely cheap -Taken all above into account I could suggest the following algorithm to -build R: +Taken all above into account I could suggest the following algorithm +to build R: - Using indexes (read about them below) for each column participating in the - intersection, - merge ordered sets rowid{a_i=v_i} in the following manner. + Using indexes (read about them below) for each column participating + in the intersection, merge ordered sets rowid{a_i=v_i} in the + following manner. If a rowid r has been encountered maximum in k sets -rowid{a_i1=v_i1},...,rowid(a_ik=v_ik), + rowid{a_i1=v_i1},...,rowid(a_ik=v_ik), then it has to be checked against all rowid{a_i=v_i} such that i is -not in {i1,...,ik}. + not in {i1,...,ik}. As soon as we fail to find r in one of these sets we discard it. If r has been found in all of them then r belongs to the set R. -Here we use the property (1): any r from rowid{a_i=v_i} UNION rowid{a_i -is null} is either +Here we use the property (1): +any r from rowid{a_i=v_i} UNION rowid{a_i is null} is either belongs to rowid{a_i=v_i} or to rowid{a_i is null}. From this we can -infer that for any r from R -indexes a_i can be uniquely divided into two groups: one contains -indexes a_i where r belongs to -the sets rowid{a_i=v_i}, the other contains indexes a_j such that r -belongs to rowid{a_j is null}. - -Now let's talk how to get elements from rowid{a_i=v_i} in a sorted order -needed for the merge procedure. We could use BTREE indexes for temp -table. But they are rather expensive and -take a lot of memory as the are implemented with RB trees. +infer that for any r from R indexes a_i can be uniquely divided into +two groups: +- one contains indexes a_i where r belongs to the sets rowid{a_i=v_i}, +- the other contains indexes a_j such that r belongs to + rowid{a_j is null}. + +Now let's talk how to get elements from rowid{a_i=v_i} in a sorted +order needed for the merge procedure. We could use BTREE indexes for +temp table. But they are rather expensive and take a lot of memory as +the are implemented with RB trees. I would suggest creating for each column from the temporary table just an array of rowids sorted by the value from column a. Index lookup in such an array is cheap. It's also rather cheap to check that the next rowid refers to a row with a different value in column a. The array can be created on demand. +2. Algorithm for IN execution with partial matching +======================================================================== + +2.1 Below is shown the top-level algorithm to execute an IN predicate +with partial matching. This algorithm is essentially the implementation +of Item_subselect:exec(). + +int lookup_with_null_semantics(outer_ref[], mat_subquery) +{ + if (index_lookup(outer_ref, mat_subquery) + return TRUE + else + { + /* + Check if there is a partial match (UNKNOWN) or no match (NULL). + */ + if (this is the first partial match) + { + vkey[] = build array of value keys for each NULL-able column + of mat_subquery. + nkey[] = build a bitmap NULL index for each column of mat_subquery + that contains NULLs + nonull_key = build a key over all non-NULL columns of mat_subquery + } + if (partial_match(outer_ref, vkey[], nkey[], nonull_key) + return UNKNOWN + else + return FALSE + } +} + +2.2 The implementation of partial matching is as follows + +/* + Assumptions: + - It has already been checked if there is a complete match by a + regular index lookup, and the test failed. + - It has already been checked if there is a complete NULL row, + and if there was we wouldn't call this function. Thus we assume + that there is no complete NULL row. + - Not all vidx_i are empty, but some can be empty. If all were empty, + then the only possibility for a match is a complete NULL row, which + we already checked. + + @param outer_ref - the uter (left) IN argument. + @param vidx[] - array of value keys + Ordered sequences of rowids of the corresponding columns a_i, such + that all rowids in idx_i are the ones where column a_i contains some + value or NULL. Each idx_i is derived dynamically, for each different + left argument of an IN predicate. + @param nidx[] - array of NULL keys + Bitmpas, one per each column, where a bit is set if the corresponding + row has a NULL value for the corresponding column. + @nonull_key - the only key over all columns of the materialized subquery + that do not contain NULLs + + @returns + @retval FALSE if there is no match + @retval TRUE if there is a partial match +*/ + +Boolean partial_match(outer_ref, vkey[], nkey[], nonull_key) +{ + /* Set of the keys (columns) that form a partial match. */ + Set matching_keys = {} + /* A subset of all keys that need to be checked for NULL matches. */ + Set null_keys = {} + Int min_key /* Key that contains the current minimum position. */ + Int min_row /* Current row number of min_key. */ + Int cur_min_key, cur_min_row + PriorityQueue pq + + if (nonull_key && ! nonull_key->lookup(outer_ref)) + return FALSE + + for (i = 1; i <= n; i++) + { + vkey[i].lookup(outer_ref) + if (! vkey[i].is_eof()) + pq.insert(i) + } + /* + Not all value keys are empty, thus we don't have only NULL + keys. If we had, the only possible match is a NULL row, and + we cheked there is no such row, therefore the result is known + to be FALSE. + In fact this algorithm makes sense for at least two non-NULL + columns. + */ + assert(pq.elements > 1) + + (min_key, min_row) = pq.pop() + matching_keys.add(min_key) + vkey[min_key].next() + if (! vkey[min_key].is_eof()) + pq.insert(min_key) + + while (TRUE) + { + (cur_min_key, cur_min_row) = pq.pop() + + if (cur_min_row == min_row) + { + matching_keys.add(cur_min_key) + /* There cannot be a complete match, as we already checked for one. */ + assert(matching_keys.elements < n) + } + else if (cur_min_key == nonull_key) + { + /* + The non-NULL key has no corresponding NULL index, so we know for + sure that the row 'min_row' is not a match. + */ + (min_key, min_row) = (cur_min_key, cur_min_row) + matching_keys = {min_key} + } + else + { + assert(cur_min_row > min_row) /* Follows from the use of PQ. */ + null_keys = set_difference(all keys vkey[], matching_keys) + /* + Check if all null_keys contain a NULL at row 'min_row'. The procedure + internally checks all keys in a special precomputed order. A prior + procedure determines an optimal order and a mapping + idx_no -> idx_order (encoded as an array). + */ + if (test_null_row(null_keys, min_row)) + return TRUE + else + { + (min_key, min_row) = (cur_min_key, cur_min_row) + matching_keys = {min_key} + } + } + + vkey[cur_min_key].next() + if (! vkey[cur_min_key].is_eof()) + pq.insert(cur_min_key) + + if (pq.is_empty()) + { + /* Check the last row of the last column in PQ for NULL matches. */ + null_keys = set_difference(all keys vkey[], matching_keys) + if (test_null_row(null_keys, min_row)) + return TRUE + else + return FALSE + } + } + + /* We should never get here. */ + assert(FALSE) + return FALSE +} + + + +3. Directions for improvement +======================================================================== + Other consideration that may be taken into account: 1. If columns a_j1,...,a_jm do not contain null values in the temporary -=-=(Timour - Sun, 06 Dec 2009, 14:36)=-=- High-Level Specification modified. --- /tmp/wklog.68.old.12919 2009-12-06 14:36:18.000000000 +0200 +++ /tmp/wklog.68.new.12919 2009-12-06 14:36:18.000000000 +0200 @@ -87,3 +87,8 @@ 7. If you get a row with nulls in all columns stop filling the temporary table and return UNKNOWN for any tuple <v1,...,vn>. +8. [timour] + Consider that due to materialization, we already have a unique index +on all columns <a_1,..., a_n>. We can use the first key part of this index +over column a_1, instead of the index rowid{a_i=v_i}. Thus we can avoid +creating the index rowid{a_i=v_i}. -=-=(Timour - Fri, 04 Dec 2009, 14:04)=-=- High-Level Specification modified. --- /tmp/wklog.68.old.16724 2009-12-04 14:04:28.000000000 +0200 +++ /tmp/wklog.68.new.16724 2009-12-04 14:04:28.000000000 +0200 @@ -10,7 +10,8 @@ Bear in mind the following specifics of this intersection: (1) For each i: rowid{a_i=v_i} and rowid{a_i is null} are disjoint - (2) For each i: rowid{a_i is null} is the same for each tuple + (2) For each i: rowid{a_i is null} is the same for each tuple, + that is, this set is independent of the left-side tuples. Due to (2) it makes sense to build rowid{a_i is null} only once. A good representation for such sets would be bitmaps: -=-=(Timour - Fri, 04 Dec 2009, 11:27)=-=- Version updated. --- /tmp/wklog.68.old.5257 2009-12-04 11:27:11.000000000 +0200 +++ /tmp/wklog.68.new.5257 2009-12-04 11:27:11.000000000 +0200 @@ -1 +1 @@ -Benchmarks-3.0 +Server-9.x -=-=(Timour - Fri, 04 Dec 2009, 11:27)=-=- Category updated. --- /tmp/wklog.68.old.5242 2009-12-04 11:27:02.000000000 +0200 +++ /tmp/wklog.68.new.5242 2009-12-04 11:27:02.000000000 +0200 @@ -1 +1 @@ -Server-RawIdeaBin +Server-Sprint -=-=(Timour - Fri, 04 Dec 2009, 11:27)=-=- Status updated. --- /tmp/wklog.68.old.5242 2009-12-04 11:27:02.000000000 +0200 +++ /tmp/wklog.68.new.5242 2009-12-04 11:27:02.000000000 +0200 @@ -1 +1 @@ -Un-Assigned +Assigned -=-=(Timour - Fri, 04 Dec 2009, 11:26)=-=- High-Level Specification modified. --- /tmp/wklog.68.old.5182 2009-12-04 11:26:25.000000000 +0200 +++ /tmp/wklog.68.new.5182 2009-12-04 11:26:25.000000000 +0200 @@ -50,23 +50,39 @@ The array can be created on demand. Other consideration that may be taken into account: + 1. If columns a_j1,...,a_jm do not contain null values in the temporary -table at all, create for them only one index array (and of course do not -create any bitmaps for them). -2. Consider the ratio d(a_i)=N'/ V(a_i), where N' is the number of rows -where a_i is not null and V(a_i) is the number of distinct values for -a_i excluding nulls. - If d(a_i) is close to 1 then do not create any index array: check +table at all and v_j1,...,v_jm cannot be null, create for these columns +only one index array (and of course do not create any bitmaps for them). + +2. Consider the ratio d(a_i)=N'(a_i)/V(a_i), where N'(a_i) is the number +of rows, where a_i is not null and V(a_i) is the number of distinct +values for a_i excluding nulls. +If d(a_i) is close to N'(a_i) then do not create any index array: check whether there is a match running through the records that have been -filtered in. Anyway if d(a_i) is close to 1 then a intersection with -rowid{a_i=v_i} would not reduce the number of remaining rowids +filtered in. Anyway if d(a_i) is close to N'(a_i) then the intersection + with rowid{a_i=v_i} will not reduce the number of remaining rowids significantly. - If additionally N-N' is small do not create a bitmap for this column -either. -3. If for a column a_i d(a_i) is not close to 1, but N-N' is small a -sorted array of rowids from the set rowid{a_i is null} can be used -instead of a bitmap. +In other words is V(a_i) exceeds some threshold there is no sense to +create an index for a_i. +If additionally N-N'(a_i) is small do not create a bitmap for this +column either. + +3. If for a column a_i d(a_i) is not close to N'(a_i), but N-N'(a_i) is +small a sorted array of rowids from the set rowid{a_i is null} can be +used instead of a bitmap. + 4. We always have a match if R0= INTERSECT rowid{a_i is null} is not empty. Here i runs through all indexes from [1..n] such that v_i is not null. For a given subset of columns this fact has to be checked only once. It can be easily done with bitmap intersection. + +5. If v1,...,vn never can be a null, then indexes (sorted arrays) can be +created only for rows with nulls. + +6. If v1,...,vn never can be a null and number of rows with nulls is +small do not create indexes and do not create bitmaps. + +7. If you get a row with nulls in all columns stop filling the temporary +table and return UNKNOWN for any tuple <v1,...,vn>. + -=-=(Timour - Fri, 27 Nov 2009, 13:23)=-=- High-Level Specification modified. --- /tmp/wklog.68.old.17140 2009-11-27 11:23:17.000000000 +0000 +++ /tmp/wklog.68.new.17140 2009-11-27 11:23:17.000000000 +0000 @@ -1 +1,72 @@ +This a copy of the initial algorithm proposed by Igor: +====================================================== +For each left side tuple (v_1,...,v_n) we have to find the following set +of rowids for the temp table containing N rows as the result of +materialization of the subquery: + + R= INTERSECT (rowid{a_i=v_i} UNION rowid{a_i is null} where i runs +trough all indexes from [1..n] such that v_i is not null. + +Bear in mind the following specifics of this intersection: + (1) For each i: rowid{a_i=v_i} and rowid{a_i is null} are disjoint + (2) For each i: rowid{a_i is null} is the same for each tuple + +Due to (2) it makes sense to build rowid{a_i is null} only once. +A good representation for such sets would be bitmaps: +- it requires minimum memory: not more than N*n bits in total +- search of an element in a set is extremely cheap + +Taken all above into account I could suggest the following algorithm to +build R: + + Using indexes (read about them below) for each column participating in the + intersection, + merge ordered sets rowid{a_i=v_i} in the following manner. + If a rowid r has been encountered maximum in k sets +rowid{a_i1=v_i1},...,rowid(a_ik=v_ik), + then it has to be checked against all rowid{a_i=v_i} such that i is +not in {i1,...,ik}. + As soon as we fail to find r in one of these sets we discard it. + If r has been found in all of them then r belongs to the set R. + +Here we use the property (1): any r from rowid{a_i=v_i} UNION rowid{a_i +is null} is either +belongs to rowid{a_i=v_i} or to rowid{a_i is null}. From this we can +infer that for any r from R +indexes a_i can be uniquely divided into two groups: one contains +indexes a_i where r belongs to +the sets rowid{a_i=v_i}, the other contains indexes a_j such that r +belongs to rowid{a_j is null}. + +Now let's talk how to get elements from rowid{a_i=v_i} in a sorted order +needed for the merge procedure. We could use BTREE indexes for temp +table. But they are rather expensive and +take a lot of memory as the are implemented with RB trees. +I would suggest creating for each column from the temporary table just +an array of rowids sorted by the value from column a. +Index lookup in such an array is cheap. It's also rather cheap to check +that the next rowid refers to a row with a different value in column a. +The array can be created on demand. + +Other consideration that may be taken into account: +1. If columns a_j1,...,a_jm do not contain null values in the temporary +table at all, create for them only one index array (and of course do not +create any bitmaps for them). +2. Consider the ratio d(a_i)=N'/ V(a_i), where N' is the number of rows +where a_i is not null and V(a_i) is the number of distinct values for +a_i excluding nulls. + If d(a_i) is close to 1 then do not create any index array: check +whether there is a match running through the records that have been +filtered in. Anyway if d(a_i) is close to 1 then a intersection with +rowid{a_i=v_i} would not reduce the number of remaining rowids +significantly. + If additionally N-N' is small do not create a bitmap for this column +either. +3. If for a column a_i d(a_i) is not close to 1, but N-N' is small a +sorted array of rowids from the set rowid{a_i is null} can be used +instead of a bitmap. +4. We always have a match if R0= INTERSECT rowid{a_i is null} is not +empty. Here i runs through all indexes from [1..n] such that v_i is not +null. For a given subset of columns this fact has to be checked only +once. It can be easily done with bitmap intersection. DESCRIPTION: The goal of this task is to implement efficient execution of NOT IN subquery predicates of the form: <oe_1,...,oe_n> NOT IN <non_correlated subquery> when either some oe_i, or some subqury result column contains NULLs. The problem with such predicates is that it is possible to use index lookups only when neither argument of the predicate contains NULLs. If some argument contains a NULL, then due to NULL semantics, it plays the role of a wildcard. If we were to use regular index lookups, then we would get 'no match' for some outer tuple (thus the predicate evaluates to FALSE), while the SQL semantics means 'partial match', and the predicate should evaluate to NULL. This task implements an efficient algorithm to compute such 'parial matches', where a NULL matches any value. HIGH-LEVEL SPECIFICATION: Contents ======================================================================== 1. Initial idea as proposed by Igor 2. Algorithm for IN execution with partial matching 3. Directions for improvement 1. Initial idea as proposed by Igor ======================================================================== For each left side tuple (v_1,...,v_n) we have to find the following set of rowids for the temp table containing N rows as the result of materialization of the subquery: R= INTERSECT (rowid{a_i=v_i} UNION rowid{a_i is null} where i runs trough all indexes from [1..n] such that v_i is not null. Bear in mind the following specifics of this intersection: (1) For each i: rowid{a_i=v_i} and rowid{a_i is null} are disjoint (2) For each i: rowid{a_i is null} is the same for each tuple, that is, this set is independent of the left-side tuples. Due to (2) it makes sense to build rowid{a_i is null} only once. A good representation for such sets would be bitmaps: - it requires minimum memory: not more than N*n bits in total - search of an element in a set is extremely cheap Taken all above into account I could suggest the following algorithm to build R: Using indexes (read about them below) for each column participating in the intersection, merge ordered sets rowid{a_i=v_i} in the following manner. If a rowid r has been encountered maximum in k sets rowid{a_i1=v_i1},...,rowid(a_ik=v_ik), then it has to be checked against all rowid{a_i=v_i} such that i is not in {i1,...,ik}. As soon as we fail to find r in one of these sets we discard it. If r has been found in all of them then r belongs to the set R. Here we use the property (1): any r from rowid{a_i=v_i} UNION rowid{a_i is null} is either belongs to rowid{a_i=v_i} or to rowid{a_i is null}. From this we can infer that for any r from R indexes a_i can be uniquely divided into two groups: - one contains indexes a_i where r belongs to the sets rowid{a_i=v_i}, - the other contains indexes a_j such that r belongs to rowid{a_j is null}. Now let's talk how to get elements from rowid{a_i=v_i} in a sorted order needed for the merge procedure. We could use BTREE indexes for temp table. But they are rather expensive and take a lot of memory as the are implemented with RB trees. I would suggest creating for each column from the temporary table just an array of rowids sorted by the value from column a. Index lookup in such an array is cheap. It's also rather cheap to check that the next rowid refers to a row with a different value in column a. The array can be created on demand. 2. Algorithm for IN execution with partial matching ======================================================================== 2.1 Below is shown the top-level algorithm to execute an IN predicate with partial matching. This algorithm is essentially the implementation of Item_subselect:exec(). int lookup_with_null_semantics(outer_ref[], mat_subquery) { if (index_lookup(outer_ref, mat_subquery) return TRUE else { /* Check if there is a partial match (UNKNOWN) or no match (NULL). */ if (this is the first partial match) { vkey[] = build array of value keys for each NULL-able column of mat_subquery. nkey[] = build a bitmap NULL index for each column of mat_subquery that contains NULLs nonull_key = build a key over all non-NULL columns of mat_subquery } if (partial_match(outer_ref, vkey[], nkey[], nonull_key) return UNKNOWN else return FALSE } } 2.2 The implementation of partial matching is as follows /* Assumptions: - It has already been checked if there is a complete match by a regular index lookup, and the test failed. - It has already been checked if there is a complete NULL row, and if there was we wouldn't call this function. Thus we assume that there is no complete NULL row. - Not all vidx_i are empty, but some can be empty. If all were empty, then the only possibility for a match is a complete NULL row, which we already checked. @param outer_ref - the uter (left) IN argument. @param vidx[] - array of value keys Ordered sequences of rowids of the corresponding columns a_i, such that all rowids in idx_i are the ones where column a_i contains some value or NULL. Each idx_i is derived dynamically, for each different left argument of an IN predicate. @param nidx[] - array of NULL keys Bitmpas, one per each column, where a bit is set if the corresponding row has a NULL value for the corresponding column. @nonull_key - the only key over all columns of the materialized subquery that do not contain NULLs @returns @retval FALSE if there is no match @retval TRUE if there is a partial match */ Boolean partial_match(outer_ref, vkey[], nkey[], nonull_key) { /* Set of the keys (columns) that form a partial match. */ Set matching_keys = {} /* A subset of all keys that need to be checked for NULL matches. */ Set null_keys = {} Int min_key /* Key that contains the current minimum position. */ Int min_row /* Current row number of min_key. */ Int cur_min_key, cur_min_row PriorityQueue pq if (nonull_key && ! nonull_key->lookup(outer_ref)) return FALSE for (i = 1; i <= n; i++) { vkey[i].lookup(outer_ref) if (! vkey[i].is_eof()) pq.insert(i) } /* Not all value keys are empty, thus we don't have only NULL keys. If we had, the only possible match is a NULL row, and we cheked there is no such row, therefore the result is known to be FALSE. In fact this algorithm makes sense for at least two non-NULL columns. */ assert(pq.elements > 1) (min_key, min_row) = pq.pop() matching_keys.add(min_key) vkey[min_key].next() if (! vkey[min_key].is_eof()) pq.insert(min_key) while (TRUE) { (cur_min_key, cur_min_row) = pq.pop() if (cur_min_row == min_row) { matching_keys.add(cur_min_key) /* There cannot be a complete match, as we already checked for one. */ assert(matching_keys.elements < n) } else if (cur_min_key == nonull_key) { /* The non-NULL key has no corresponding NULL index, so we know for sure that the row 'min_row' is not a match. */ (min_key, min_row) = (cur_min_key, cur_min_row) matching_keys = {min_key} } else { assert(cur_min_row > min_row) /* Follows from the use of PQ. */ null_keys = set_difference(all keys vkey[], matching_keys) /* Check if all null_keys contain a NULL at row 'min_row'. The procedure internally checks all keys in a special precomputed order. A prior procedure determines an optimal order and a mapping idx_no -> idx_order (encoded as an array). */ if (test_null_row(null_keys, min_row)) return TRUE else { (min_key, min_row) = (cur_min_key, cur_min_row) matching_keys = {min_key} } } vkey[cur_min_key].next() if (! vkey[cur_min_key].is_eof()) pq.insert(cur_min_key) if (pq.is_empty()) { /* Check the last row of the last column in PQ for NULL matches. */ null_keys = set_difference(all keys vkey[], matching_keys) if (test_null_row(null_keys, min_row)) return TRUE else return FALSE } } /* We should never get here. */ assert(FALSE) return FALSE } 3. Directions for improvement ======================================================================== Other consideration that may be taken into account: 1. If columns a_j1,...,a_jm do not contain null values in the temporary table at all and v_j1,...,v_jm cannot be null, create for these columns only one index array (and of course do not create any bitmaps for them). 2. Consider the ratio d(a_i)=N'(a_i)/V(a_i), where N'(a_i) is the number of rows, where a_i is not null and V(a_i) is the number of distinct values for a_i excluding nulls. If d(a_i) is close to N'(a_i) then do not create any index array: check whether there is a match running through the records that have been filtered in. Anyway if d(a_i) is close to N'(a_i) then the intersection with rowid{a_i=v_i} will not reduce the number of remaining rowids significantly. In other words is V(a_i) exceeds some threshold there is no sense to create an index for a_i. If additionally N-N'(a_i) is small do not create a bitmap for this column either. 3. If for a column a_i d(a_i) is not close to N'(a_i), but N-N'(a_i) is small a sorted array of rowids from the set rowid{a_i is null} can be used instead of a bitmap. 4. We always have a match if R0= INTERSECT rowid{a_i is null} is not empty. Here i runs through all indexes from [1..n] such that v_i is not null. For a given subset of columns this fact has to be checked only once. It can be easily done with bitmap intersection. 5. If v1,...,vn never can be a null, then indexes (sorted arrays) can be created only for rows with nulls. 6. If v1,...,vn never can be a null and number of rows with nulls is small do not create indexes and do not create bitmaps. 7. If you get a row with nulls in all columns stop filling the temporary table and return UNKNOWN for any tuple <v1,...,vn>. 8. [timour] Consider that due to materialization, we already have a unique index on all columns <a_1,..., a_n>. We can use the first key part of this index over column a_1, instead of the index rowid{a_i=v_i}. Thus we can avoid creating the index rowid{a_i=v_i}. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Guest): Subquery optimization: Efficient NOT IN execution with NULLs (68)
by worklog-noreply＠askmonty.org 19 Jan '10

19 Jan '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Subquery optimization: Efficient NOT IN execution with NULLs CREATION DATE..: Fri, 27 Nov 2009, 13:22 SUPERVISOR.....: Monty IMPLEMENTOR....: Timour COPIES TO......: CATEGORY.......: Server-Sprint TASK ID........: 68 (http://askmonty.org/worklog/?tid=68) VERSION........: Server-9.x STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Guest - Tue, 19 Jan 2010, 18:15)=-=- High-Level Specification modified. --- /tmp/wklog.68.old.19825 2010-01-19 18:15:30.000000000 +0200 +++ /tmp/wklog.68.new.19825 2010-01-19 18:15:30.000000000 +0200 @@ -1,8 +1,16 @@ -This a copy of the initial algorithm proposed by Igor: -====================================================== +Contents +======================================================================== -For each left side tuple (v_1,...,v_n) we have to find the following set -of rowids for the temp table containing N rows as the result of +1. Initial idea as proposed by Igor +2. Algorithm for IN execution with partial matching +3. Directions for improvement + + +1. Initial idea as proposed by Igor +======================================================================== + +For each left side tuple (v_1,...,v_n) we have to find the following +set of rowids for the temp table containing N rows as the result of materialization of the subquery: R= INTERSECT (rowid{a_i=v_i} UNION rowid{a_i is null} where i runs @@ -18,38 +26,198 @@ - it requires minimum memory: not more than N*n bits in total - search of an element in a set is extremely cheap -Taken all above into account I could suggest the following algorithm to -build R: +Taken all above into account I could suggest the following algorithm +to build R: - Using indexes (read about them below) for each column participating in the - intersection, - merge ordered sets rowid{a_i=v_i} in the following manner. + Using indexes (read about them below) for each column participating + in the intersection, merge ordered sets rowid{a_i=v_i} in the + following manner. If a rowid r has been encountered maximum in k sets -rowid{a_i1=v_i1},...,rowid(a_ik=v_ik), + rowid{a_i1=v_i1},...,rowid(a_ik=v_ik), then it has to be checked against all rowid{a_i=v_i} such that i is -not in {i1,...,ik}. + not in {i1,...,ik}. As soon as we fail to find r in one of these sets we discard it. If r has been found in all of them then r belongs to the set R. -Here we use the property (1): any r from rowid{a_i=v_i} UNION rowid{a_i -is null} is either +Here we use the property (1): +any r from rowid{a_i=v_i} UNION rowid{a_i is null} is either belongs to rowid{a_i=v_i} or to rowid{a_i is null}. From this we can -infer that for any r from R -indexes a_i can be uniquely divided into two groups: one contains -indexes a_i where r belongs to -the sets rowid{a_i=v_i}, the other contains indexes a_j such that r -belongs to rowid{a_j is null}. - -Now let's talk how to get elements from rowid{a_i=v_i} in a sorted order -needed for the merge procedure. We could use BTREE indexes for temp -table. But they are rather expensive and -take a lot of memory as the are implemented with RB trees. +infer that for any r from R indexes a_i can be uniquely divided into +two groups: +- one contains indexes a_i where r belongs to the sets rowid{a_i=v_i}, +- the other contains indexes a_j such that r belongs to + rowid{a_j is null}. + +Now let's talk how to get elements from rowid{a_i=v_i} in a sorted +order needed for the merge procedure. We could use BTREE indexes for +temp table. But they are rather expensive and take a lot of memory as +the are implemented with RB trees. I would suggest creating for each column from the temporary table just an array of rowids sorted by the value from column a. Index lookup in such an array is cheap. It's also rather cheap to check that the next rowid refers to a row with a different value in column a. The array can be created on demand. +2. Algorithm for IN execution with partial matching +======================================================================== + +2.1 Below is shown the top-level algorithm to execute an IN predicate +with partial matching. This algorithm is essentially the implementation +of Item_subselect:exec(). + +int lookup_with_null_semantics(outer_ref[], mat_subquery) +{ + if (index_lookup(outer_ref, mat_subquery) + return TRUE + else + { + /* + Check if there is a partial match (UNKNOWN) or no match (NULL). + */ + if (this is the first partial match) + { + vkey[] = build array of value keys for each NULL-able column + of mat_subquery. + nkey[] = build a bitmap NULL index for each column of mat_subquery + that contains NULLs + nonull_key = build a key over all non-NULL columns of mat_subquery + } + if (partial_match(outer_ref, vkey[], nkey[], nonull_key) + return UNKNOWN + else + return FALSE + } +} + +2.2 The implementation of partial matching is as follows + +/* + Assumptions: + - It has already been checked if there is a complete match by a + regular index lookup, and the test failed. + - It has already been checked if there is a complete NULL row, + and if there was we wouldn't call this function. Thus we assume + that there is no complete NULL row. + - Not all vidx_i are empty, but some can be empty. If all were empty, + then the only possibility for a match is a complete NULL row, which + we already checked. + + @param outer_ref - the uter (left) IN argument. + @param vidx[] - array of value keys + Ordered sequences of rowids of the corresponding columns a_i, such + that all rowids in idx_i are the ones where column a_i contains some + value or NULL. Each idx_i is derived dynamically, for each different + left argument of an IN predicate. + @param nidx[] - array of NULL keys + Bitmpas, one per each column, where a bit is set if the corresponding + row has a NULL value for the corresponding column. + @nonull_key - the only key over all columns of the materialized subquery + that do not contain NULLs + + @returns + @retval FALSE if there is no match + @retval TRUE if there is a partial match +*/ + +Boolean partial_match(outer_ref, vkey[], nkey[], nonull_key) +{ + /* Set of the keys (columns) that form a partial match. */ + Set matching_keys = {} + /* A subset of all keys that need to be checked for NULL matches. */ + Set null_keys = {} + Int min_key /* Key that contains the current minimum position. */ + Int min_row /* Current row number of min_key. */ + Int cur_min_key, cur_min_row + PriorityQueue pq + + if (nonull_key && ! nonull_key->lookup(outer_ref)) + return FALSE + + for (i = 1; i <= n; i++) + { + vkey[i].lookup(outer_ref) + if (! vkey[i].is_eof()) + pq.insert(i) + } + /* + Not all value keys are empty, thus we don't have only NULL + keys. If we had, the only possible match is a NULL row, and + we cheked there is no such row, therefore the result is known + to be FALSE. + In fact this algorithm makes sense for at least two non-NULL + columns. + */ + assert(pq.elements > 1) + + (min_key, min_row) = pq.pop() + matching_keys.add(min_key) + vkey[min_key].next() + if (! vkey[min_key].is_eof()) + pq.insert(min_key) + + while (TRUE) + { + (cur_min_key, cur_min_row) = pq.pop() + + if (cur_min_row == min_row) + { + matching_keys.add(cur_min_key) + /* There cannot be a complete match, as we already checked for one. */ + assert(matching_keys.elements < n) + } + else if (cur_min_key == nonull_key) + { + /* + The non-NULL key has no corresponding NULL index, so we know for + sure that the row 'min_row' is not a match. + */ + (min_key, min_row) = (cur_min_key, cur_min_row) + matching_keys = {min_key} + } + else + { + assert(cur_min_row > min_row) /* Follows from the use of PQ. */ + null_keys = set_difference(all keys vkey[], matching_keys) + /* + Check if all null_keys contain a NULL at row 'min_row'. The procedure + internally checks all keys in a special precomputed order. A prior + procedure determines an optimal order and a mapping + idx_no -> idx_order (encoded as an array). + */ + if (test_null_row(null_keys, min_row)) + return TRUE + else + { + (min_key, min_row) = (cur_min_key, cur_min_row) + matching_keys = {min_key} + } + } + + vkey[cur_min_key].next() + if (! vkey[cur_min_key].is_eof()) + pq.insert(cur_min_key) + + if (pq.is_empty()) + { + /* Check the last row of the last column in PQ for NULL matches. */ + null_keys = set_difference(all keys vkey[], matching_keys) + if (test_null_row(null_keys, min_row)) + return TRUE + else + return FALSE + } + } + + /* We should never get here. */ + assert(FALSE) + return FALSE +} + + + +3. Directions for improvement +======================================================================== + Other consideration that may be taken into account: 1. If columns a_j1,...,a_jm do not contain null values in the temporary -=-=(Timour - Sun, 06 Dec 2009, 14:36)=-=- High-Level Specification modified. --- /tmp/wklog.68.old.12919 2009-12-06 14:36:18.000000000 +0200 +++ /tmp/wklog.68.new.12919 2009-12-06 14:36:18.000000000 +0200 @@ -87,3 +87,8 @@ 7. If you get a row with nulls in all columns stop filling the temporary table and return UNKNOWN for any tuple <v1,...,vn>. +8. [timour] + Consider that due to materialization, we already have a unique index +on all columns <a_1,..., a_n>. We can use the first key part of this index +over column a_1, instead of the index rowid{a_i=v_i}. Thus we can avoid +creating the index rowid{a_i=v_i}. -=-=(Timour - Fri, 04 Dec 2009, 14:04)=-=- High-Level Specification modified. --- /tmp/wklog.68.old.16724 2009-12-04 14:04:28.000000000 +0200 +++ /tmp/wklog.68.new.16724 2009-12-04 14:04:28.000000000 +0200 @@ -10,7 +10,8 @@ Bear in mind the following specifics of this intersection: (1) For each i: rowid{a_i=v_i} and rowid{a_i is null} are disjoint - (2) For each i: rowid{a_i is null} is the same for each tuple + (2) For each i: rowid{a_i is null} is the same for each tuple, + that is, this set is independent of the left-side tuples. Due to (2) it makes sense to build rowid{a_i is null} only once. A good representation for such sets would be bitmaps: -=-=(Timour - Fri, 04 Dec 2009, 11:27)=-=- Version updated. --- /tmp/wklog.68.old.5257 2009-12-04 11:27:11.000000000 +0200 +++ /tmp/wklog.68.new.5257 2009-12-04 11:27:11.000000000 +0200 @@ -1 +1 @@ -Benchmarks-3.0 +Server-9.x -=-=(Timour - Fri, 04 Dec 2009, 11:27)=-=- Category updated. --- /tmp/wklog.68.old.5242 2009-12-04 11:27:02.000000000 +0200 +++ /tmp/wklog.68.new.5242 2009-12-04 11:27:02.000000000 +0200 @@ -1 +1 @@ -Server-RawIdeaBin +Server-Sprint -=-=(Timour - Fri, 04 Dec 2009, 11:27)=-=- Status updated. --- /tmp/wklog.68.old.5242 2009-12-04 11:27:02.000000000 +0200 +++ /tmp/wklog.68.new.5242 2009-12-04 11:27:02.000000000 +0200 @@ -1 +1 @@ -Un-Assigned +Assigned -=-=(Timour - Fri, 04 Dec 2009, 11:26)=-=- High-Level Specification modified. --- /tmp/wklog.68.old.5182 2009-12-04 11:26:25.000000000 +0200 +++ /tmp/wklog.68.new.5182 2009-12-04 11:26:25.000000000 +0200 @@ -50,23 +50,39 @@ The array can be created on demand. Other consideration that may be taken into account: + 1. If columns a_j1,...,a_jm do not contain null values in the temporary -table at all, create for them only one index array (and of course do not -create any bitmaps for them). -2. Consider the ratio d(a_i)=N'/ V(a_i), where N' is the number of rows -where a_i is not null and V(a_i) is the number of distinct values for -a_i excluding nulls. - If d(a_i) is close to 1 then do not create any index array: check +table at all and v_j1,...,v_jm cannot be null, create for these columns +only one index array (and of course do not create any bitmaps for them). + +2. Consider the ratio d(a_i)=N'(a_i)/V(a_i), where N'(a_i) is the number +of rows, where a_i is not null and V(a_i) is the number of distinct +values for a_i excluding nulls. +If d(a_i) is close to N'(a_i) then do not create any index array: check whether there is a match running through the records that have been -filtered in. Anyway if d(a_i) is close to 1 then a intersection with -rowid{a_i=v_i} would not reduce the number of remaining rowids +filtered in. Anyway if d(a_i) is close to N'(a_i) then the intersection + with rowid{a_i=v_i} will not reduce the number of remaining rowids significantly. - If additionally N-N' is small do not create a bitmap for this column -either. -3. If for a column a_i d(a_i) is not close to 1, but N-N' is small a -sorted array of rowids from the set rowid{a_i is null} can be used -instead of a bitmap. +In other words is V(a_i) exceeds some threshold there is no sense to +create an index for a_i. +If additionally N-N'(a_i) is small do not create a bitmap for this +column either. + +3. If for a column a_i d(a_i) is not close to N'(a_i), but N-N'(a_i) is +small a sorted array of rowids from the set rowid{a_i is null} can be +used instead of a bitmap. + 4. We always have a match if R0= INTERSECT rowid{a_i is null} is not empty. Here i runs through all indexes from [1..n] such that v_i is not null. For a given subset of columns this fact has to be checked only once. It can be easily done with bitmap intersection. + +5. If v1,...,vn never can be a null, then indexes (sorted arrays) can be +created only for rows with nulls. + +6. If v1,...,vn never can be a null and number of rows with nulls is +small do not create indexes and do not create bitmaps. + +7. If you get a row with nulls in all columns stop filling the temporary +table and return UNKNOWN for any tuple <v1,...,vn>. + -=-=(Timour - Fri, 27 Nov 2009, 13:23)=-=- High-Level Specification modified. --- /tmp/wklog.68.old.17140 2009-11-27 11:23:17.000000000 +0000 +++ /tmp/wklog.68.new.17140 2009-11-27 11:23:17.000000000 +0000 @@ -1 +1,72 @@ +This a copy of the initial algorithm proposed by Igor: +====================================================== +For each left side tuple (v_1,...,v_n) we have to find the following set +of rowids for the temp table containing N rows as the result of +materialization of the subquery: + + R= INTERSECT (rowid{a_i=v_i} UNION rowid{a_i is null} where i runs +trough all indexes from [1..n] such that v_i is not null. + +Bear in mind the following specifics of this intersection: + (1) For each i: rowid{a_i=v_i} and rowid{a_i is null} are disjoint + (2) For each i: rowid{a_i is null} is the same for each tuple + +Due to (2) it makes sense to build rowid{a_i is null} only once. +A good representation for such sets would be bitmaps: +- it requires minimum memory: not more than N*n bits in total +- search of an element in a set is extremely cheap + +Taken all above into account I could suggest the following algorithm to +build R: + + Using indexes (read about them below) for each column participating in the + intersection, + merge ordered sets rowid{a_i=v_i} in the following manner. + If a rowid r has been encountered maximum in k sets +rowid{a_i1=v_i1},...,rowid(a_ik=v_ik), + then it has to be checked against all rowid{a_i=v_i} such that i is +not in {i1,...,ik}. + As soon as we fail to find r in one of these sets we discard it. + If r has been found in all of them then r belongs to the set R. + +Here we use the property (1): any r from rowid{a_i=v_i} UNION rowid{a_i +is null} is either +belongs to rowid{a_i=v_i} or to rowid{a_i is null}. From this we can +infer that for any r from R +indexes a_i can be uniquely divided into two groups: one contains +indexes a_i where r belongs to +the sets rowid{a_i=v_i}, the other contains indexes a_j such that r +belongs to rowid{a_j is null}. + +Now let's talk how to get elements from rowid{a_i=v_i} in a sorted order +needed for the merge procedure. We could use BTREE indexes for temp +table. But they are rather expensive and +take a lot of memory as the are implemented with RB trees. +I would suggest creating for each column from the temporary table just +an array of rowids sorted by the value from column a. +Index lookup in such an array is cheap. It's also rather cheap to check +that the next rowid refers to a row with a different value in column a. +The array can be created on demand. + +Other consideration that may be taken into account: +1. If columns a_j1,...,a_jm do not contain null values in the temporary +table at all, create for them only one index array (and of course do not +create any bitmaps for them). +2. Consider the ratio d(a_i)=N'/ V(a_i), where N' is the number of rows +where a_i is not null and V(a_i) is the number of distinct values for +a_i excluding nulls. + If d(a_i) is close to 1 then do not create any index array: check +whether there is a match running through the records that have been +filtered in. Anyway if d(a_i) is close to 1 then a intersection with +rowid{a_i=v_i} would not reduce the number of remaining rowids +significantly. + If additionally N-N' is small do not create a bitmap for this column +either. +3. If for a column a_i d(a_i) is not close to 1, but N-N' is small a +sorted array of rowids from the set rowid{a_i is null} can be used +instead of a bitmap. +4. We always have a match if R0= INTERSECT rowid{a_i is null} is not +empty. Here i runs through all indexes from [1..n] such that v_i is not +null. For a given subset of columns this fact has to be checked only +once. It can be easily done with bitmap intersection. DESCRIPTION: The goal of this task is to implement efficient execution of NOT IN subquery predicates of the form: <oe_1,...,oe_n> NOT IN <non_correlated subquery> when either some oe_i, or some subqury result column contains NULLs. The problem with such predicates is that it is possible to use index lookups only when neither argument of the predicate contains NULLs. If some argument contains a NULL, then due to NULL semantics, it plays the role of a wildcard. If we were to use regular index lookups, then we would get 'no match' for some outer tuple (thus the predicate evaluates to FALSE), while the SQL semantics means 'partial match', and the predicate should evaluate to NULL. This task implements an efficient algorithm to compute such 'parial matches', where a NULL matches any value. HIGH-LEVEL SPECIFICATION: Contents ======================================================================== 1. Initial idea as proposed by Igor 2. Algorithm for IN execution with partial matching 3. Directions for improvement 1. Initial idea as proposed by Igor ======================================================================== For each left side tuple (v_1,...,v_n) we have to find the following set of rowids for the temp table containing N rows as the result of materialization of the subquery: R= INTERSECT (rowid{a_i=v_i} UNION rowid{a_i is null} where i runs trough all indexes from [1..n] such that v_i is not null. Bear in mind the following specifics of this intersection: (1) For each i: rowid{a_i=v_i} and rowid{a_i is null} are disjoint (2) For each i: rowid{a_i is null} is the same for each tuple, that is, this set is independent of the left-side tuples. Due to (2) it makes sense to build rowid{a_i is null} only once. A good representation for such sets would be bitmaps: - it requires minimum memory: not more than N*n bits in total - search of an element in a set is extremely cheap Taken all above into account I could suggest the following algorithm to build R: Using indexes (read about them below) for each column participating in the intersection, merge ordered sets rowid{a_i=v_i} in the following manner. If a rowid r has been encountered maximum in k sets rowid{a_i1=v_i1},...,rowid(a_ik=v_ik), then it has to be checked against all rowid{a_i=v_i} such that i is not in {i1,...,ik}. As soon as we fail to find r in one of these sets we discard it. If r has been found in all of them then r belongs to the set R. Here we use the property (1): any r from rowid{a_i=v_i} UNION rowid{a_i is null} is either belongs to rowid{a_i=v_i} or to rowid{a_i is null}. From this we can infer that for any r from R indexes a_i can be uniquely divided into two groups: - one contains indexes a_i where r belongs to the sets rowid{a_i=v_i}, - the other contains indexes a_j such that r belongs to rowid{a_j is null}. Now let's talk how to get elements from rowid{a_i=v_i} in a sorted order needed for the merge procedure. We could use BTREE indexes for temp table. But they are rather expensive and take a lot of memory as the are implemented with RB trees. I would suggest creating for each column from the temporary table just an array of rowids sorted by the value from column a. Index lookup in such an array is cheap. It's also rather cheap to check that the next rowid refers to a row with a different value in column a. The array can be created on demand. 2. Algorithm for IN execution with partial matching ======================================================================== 2.1 Below is shown the top-level algorithm to execute an IN predicate with partial matching. This algorithm is essentially the implementation of Item_subselect:exec(). int lookup_with_null_semantics(outer_ref[], mat_subquery) { if (index_lookup(outer_ref, mat_subquery) return TRUE else { /* Check if there is a partial match (UNKNOWN) or no match (NULL). */ if (this is the first partial match) { vkey[] = build array of value keys for each NULL-able column of mat_subquery. nkey[] = build a bitmap NULL index for each column of mat_subquery that contains NULLs nonull_key = build a key over all non-NULL columns of mat_subquery } if (partial_match(outer_ref, vkey[], nkey[], nonull_key) return UNKNOWN else return FALSE } } 2.2 The implementation of partial matching is as follows /* Assumptions: - It has already been checked if there is a complete match by a regular index lookup, and the test failed. - It has already been checked if there is a complete NULL row, and if there was we wouldn't call this function. Thus we assume that there is no complete NULL row. - Not all vidx_i are empty, but some can be empty. If all were empty, then the only possibility for a match is a complete NULL row, which we already checked. @param outer_ref - the uter (left) IN argument. @param vidx[] - array of value keys Ordered sequences of rowids of the corresponding columns a_i, such that all rowids in idx_i are the ones where column a_i contains some value or NULL. Each idx_i is derived dynamically, for each different left argument of an IN predicate. @param nidx[] - array of NULL keys Bitmpas, one per each column, where a bit is set if the corresponding row has a NULL value for the corresponding column. @nonull_key - the only key over all columns of the materialized subquery that do not contain NULLs @returns @retval FALSE if there is no match @retval TRUE if there is a partial match */ Boolean partial_match(outer_ref, vkey[], nkey[], nonull_key) { /* Set of the keys (columns) that form a partial match. */ Set matching_keys = {} /* A subset of all keys that need to be checked for NULL matches. */ Set null_keys = {} Int min_key /* Key that contains the current minimum position. */ Int min_row /* Current row number of min_key. */ Int cur_min_key, cur_min_row PriorityQueue pq if (nonull_key && ! nonull_key->lookup(outer_ref)) return FALSE for (i = 1; i <= n; i++) { vkey[i].lookup(outer_ref) if (! vkey[i].is_eof()) pq.insert(i) } /* Not all value keys are empty, thus we don't have only NULL keys. If we had, the only possible match is a NULL row, and we cheked there is no such row, therefore the result is known to be FALSE. In fact this algorithm makes sense for at least two non-NULL columns. */ assert(pq.elements > 1) (min_key, min_row) = pq.pop() matching_keys.add(min_key) vkey[min_key].next() if (! vkey[min_key].is_eof()) pq.insert(min_key) while (TRUE) { (cur_min_key, cur_min_row) = pq.pop() if (cur_min_row == min_row) { matching_keys.add(cur_min_key) /* There cannot be a complete match, as we already checked for one. */ assert(matching_keys.elements < n) } else if (cur_min_key == nonull_key) { /* The non-NULL key has no corresponding NULL index, so we know for sure that the row 'min_row' is not a match. */ (min_key, min_row) = (cur_min_key, cur_min_row) matching_keys = {min_key} } else { assert(cur_min_row > min_row) /* Follows from the use of PQ. */ null_keys = set_difference(all keys vkey[], matching_keys) /* Check if all null_keys contain a NULL at row 'min_row'. The procedure internally checks all keys in a special precomputed order. A prior procedure determines an optimal order and a mapping idx_no -> idx_order (encoded as an array). */ if (test_null_row(null_keys, min_row)) return TRUE else { (min_key, min_row) = (cur_min_key, cur_min_row) matching_keys = {min_key} } } vkey[cur_min_key].next() if (! vkey[cur_min_key].is_eof()) pq.insert(cur_min_key) if (pq.is_empty()) { /* Check the last row of the last column in PQ for NULL matches. */ null_keys = set_difference(all keys vkey[], matching_keys) if (test_null_row(null_keys, min_row)) return TRUE else return FALSE } } /* We should never get here. */ assert(FALSE) return FALSE } 3. Directions for improvement ======================================================================== Other consideration that may be taken into account: 1. If columns a_j1,...,a_jm do not contain null values in the temporary table at all and v_j1,...,v_jm cannot be null, create for these columns only one index array (and of course do not create any bitmaps for them). 2. Consider the ratio d(a_i)=N'(a_i)/V(a_i), where N'(a_i) is the number of rows, where a_i is not null and V(a_i) is the number of distinct values for a_i excluding nulls. If d(a_i) is close to N'(a_i) then do not create any index array: check whether there is a match running through the records that have been filtered in. Anyway if d(a_i) is close to N'(a_i) then the intersection with rowid{a_i=v_i} will not reduce the number of remaining rowids significantly. In other words is V(a_i) exceeds some threshold there is no sense to create an index for a_i. If additionally N-N'(a_i) is small do not create a bitmap for this column either. 3. If for a column a_i d(a_i) is not close to N'(a_i), but N-N'(a_i) is small a sorted array of rowids from the set rowid{a_i is null} can be used instead of a bitmap. 4. We always have a match if R0= INTERSECT rowid{a_i is null} is not empty. Here i runs through all indexes from [1..n] such that v_i is not null. For a given subset of columns this fact has to be checked only once. It can be easily done with bitmap intersection. 5. If v1,...,vn never can be a null, then indexes (sorted arrays) can be created only for rows with nulls. 6. If v1,...,vn never can be a null and number of rows with nulls is small do not create indexes and do not create bitmaps. 7. If you get a row with nulls in all columns stop filling the temporary table and return UNKNOWN for any tuple <v1,...,vn>. 8. [timour] Consider that due to materialization, we already have a unique index on all columns <a_1,..., a_n>. We can use the first key part of this index over column a_1, instead of the index rowid{a_i=v_i}. Thus we can avoid creating the index rowid{a_i=v_i}. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (knielsen:2793)
by knielsen＠knielsen-hq.org 19 Jan '10

19 Jan '10

#At lp:maria 2793 knielsen(a)knielsen-hq.org 2010-01-19 [merge] Merge TZ fix for Buildbot test removed: mysql-test/suite/maria/t/maria2-master.opt storage/xtradb/handler/handler0vars.h storage/xtradb/handler/win_delay_loader.cc storage/xtradb/win-plugin/ storage/xtradb/win-plugin/README storage/xtradb/win-plugin/win-plugin.diff added: mysql-test/extra/rpl_tests/rpl_not_null.test mysql-test/r/bug47671.result mysql-test/r/innodb-consistent.result mysql-test/r/innodb_bug44571.result mysql-test/r/innodb_bug46676.result mysql-test/r/innodb_bug47167.result mysql-test/std_data/bug47012.ARM mysql-test/std_data/bug47012.ARZ mysql-test/std_data/bug47012.frm mysql-test/suite/innodb/r/innodb_bug46676.result mysql-test/suite/innodb/r/innodb_bug47167.result mysql-test/suite/innodb/t/innodb_bug46676.test mysql-test/suite/innodb/t/innodb_bug47167.test mysql-test/suite/rpl/r/rpl_loaddata_symlink.result mysql-test/suite/rpl/r/rpl_nondeterministic_functions.result mysql-test/suite/rpl/r/rpl_not_null_innodb.result mysql-test/suite/rpl/r/rpl_not_null_myisam.result mysql-test/suite/rpl/r/rpl_row_trunc_temp.result mysql-test/suite/rpl/t/rpl_loaddata_symlink-master.opt mysql-test/suite/rpl/t/rpl_loaddata_symlink-master.sh mysql-test/suite/rpl/t/rpl_loaddata_symlink-slave.opt mysql-test/suite/rpl/t/rpl_loaddata_symlink-slave.sh mysql-test/suite/rpl/t/rpl_loaddata_symlink.test mysql-test/suite/rpl/t/rpl_nondeterministic_functions.test mysql-test/suite/rpl/t/rpl_not_null_innodb.test mysql-test/suite/rpl/t/rpl_not_null_myisam.test mysql-test/suite/rpl/t/rpl_row_trunc_temp.test mysql-test/t/bug47671-master.opt mysql-test/t/bug47671.test mysql-test/t/innodb-consistent-master.opt mysql-test/t/innodb-consistent.test mysql-test/t/innodb_bug44571.test mysql-test/t/innodb_bug46676.test mysql-test/t/innodb_bug47167.test mysql-test/t/mysqlbinlog2-master.opt storage/xtradb/ut/ut0auxconf_have_gcc_atomics.c modified: Makefile.am client/mysql.cc client/mysqlbinlog.cc configure.in extra/comp_err.c include/my_pthread.h include/mysql.h include/mysql.h.pp include/violite.h libmysql/libmysql.c libmysql/libmysql.def libmysqld/libmysqld.def mysql-test/collections/default.experimental mysql-test/extra/rpl_tests/rpl_extraSlave_Col.test mysql-test/extra/rpl_tests/rpl_row_tabledefs.test mysql-test/extra/rpl_tests/rpl_stm_000001.test mysql-test/include/mtr_warnings.sql mysql-test/include/ps_conv.inc mysql-test/lib/mtr_cases.pm mysql-test/mysql-test-run.pl mysql-test/r/archive.result mysql-test/r/delayed.result mysql-test/r/delete.result mysql-test/r/fulltext.result mysql-test/r/func_group.result mysql-test/r/grant2.result mysql-test/r/group_min_max.result mysql-test/r/information_schema.result mysql-test/r/information_schema_all_engines.result mysql-test/r/innodb-autoinc.result mysql-test/r/innodb-index.result mysql-test/r/innodb-timeout.result mysql-test/r/innodb-zip.result mysql-test/r/innodb.result mysql-test/r/innodb_bug36169.result mysql-test/r/innodb_bug44369.result mysql-test/r/innodb_file_format.result mysql-test/r/innodb_lock_wait_timeout_1.result mysql-test/r/innodb_mysql.result mysql-test/r/innodb_xtradb_bug317074.result mysql-test/r/mysql.result mysql-test/r/olap.result mysql-test/r/order_by.result mysql-test/r/partition.result mysql-test/r/range.result mysql-test/r/select.result mysql-test/r/show_check.result mysql-test/r/sp-destruct.result mysql-test/r/sp-security.result mysql-test/r/sp.result mysql-test/r/trigger.result mysql-test/r/trigger_notembedded.result mysql-test/r/type_newdecimal.result mysql-test/r/type_year.result mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result mysql-test/suite/binlog/r/binlog_stm_row.result mysql-test/suite/binlog/r/binlog_unsafe.result mysql-test/suite/binlog/t/binlog_killed.test mysql-test/suite/binlog/t/binlog_stm_mix_innodb_myisam.test mysql-test/suite/binlog/t/binlog_stm_row.test mysql-test/suite/binlog/t/binlog_unsafe.test mysql-test/suite/funcs_1/r/is_columns_is.result mysql-test/suite/funcs_1/r/is_tables_is.result mysql-test/suite/innodb/r/innodb-index.result mysql-test/suite/innodb/t/innodb-consistent-master.opt mysql-test/suite/innodb/t/innodb-index.test mysql-test/suite/parts/t/partition_alter1_2_innodb.test mysql-test/suite/parts/t/partition_alter2_1_innodb.test mysql-test/suite/parts/t/partition_alter2_2_innodb.test mysql-test/suite/parts/t/partition_alter4_innodb.test mysql-test/suite/pbxt/r/func_group.result mysql-test/suite/pbxt/r/mysqlshow.result mysql-test/suite/rpl/r/rpl_err_ignoredtable.result mysql-test/suite/rpl/r/rpl_extraCol_innodb.result mysql-test/suite/rpl/r/rpl_extraCol_myisam.result mysql-test/suite/rpl/r/rpl_get_lock.result mysql-test/suite/rpl/r/rpl_mixed_ddl_dml.result mysql-test/suite/rpl/r/rpl_relayspace.result mysql-test/suite/rpl/r/rpl_row_create_table.result mysql-test/suite/rpl/r/rpl_row_tabledefs_2myisam.result mysql-test/suite/rpl/r/rpl_row_tabledefs_3innodb.result mysql-test/suite/rpl/r/rpl_stm_000001.result mysql-test/suite/rpl/r/rpl_trigger.result mysql-test/suite/rpl/t/disabled.def mysql-test/suite/rpl/t/rpl_err_ignoredtable.test mysql-test/suite/rpl/t/rpl_get_lock.test mysql-test/suite/rpl/t/rpl_mixed_ddl_dml.test mysql-test/suite/rpl/t/rpl_relayspace.test mysql-test/suite/rpl/t/rpl_row_create_table.test mysql-test/suite/rpl/t/rpl_trigger.test mysql-test/suite/rpl_ndb/r/rpl_ndb_extraCol.result mysql-test/t/archive.test mysql-test/t/delayed.test mysql-test/t/delete.test mysql-test/t/disabled.def mysql-test/t/fulltext.test mysql-test/t/func_group.test mysql-test/t/grant2.test mysql-test/t/group_min_max.test mysql-test/t/innodb-analyze.test mysql-test/t/innodb-autoinc.test mysql-test/t/innodb-index.test mysql-test/t/innodb-master.opt mysql-test/t/innodb-semi-consistent-master.opt mysql-test/t/innodb-timeout.test mysql-test/t/innodb-use-sys-malloc-master.opt mysql-test/t/innodb-zip.test mysql-test/t/innodb.test mysql-test/t/innodb_bug34300.test mysql-test/t/innodb_bug36169.test mysql-test/t/innodb_bug36172.test mysql-test/t/innodb_bug42101-nonzero-master.opt mysql-test/t/innodb_bug44369.test mysql-test/t/innodb_file_format.test mysql-test/t/innodb_information_schema.test mysql-test/t/innodb_lock_wait_timeout_1.test mysql-test/t/innodb_mysql.test mysql-test/t/innodb_xtradb_bug317074.test mysql-test/t/mysql.test mysql-test/t/olap.test mysql-test/t/order_by.test mysql-test/t/partition.test mysql-test/t/range.test mysql-test/t/select.test mysql-test/t/show_check.test mysql-test/t/sp-destruct.test mysql-test/t/sp-security.test mysql-test/t/sp.test mysql-test/t/trigger.test mysql-test/t/trigger_notembedded.test mysql-test/t/type_newdecimal.test mysql-test/t/type_year.test mysql-test/valgrind.supp mysys/my_getopt.c mysys/my_sync.c scripts/make_win_bin_dist scripts/mysql_secure_installation.pl.in scripts/mysql_secure_installation.sh sql/event_db_repository.cc sql/field.cc sql/field.h sql/handler.h sql/item.cc sql/item.h sql/item_cmpfunc.cc sql/item_cmpfunc.h sql/item_create.cc sql/item_func.cc sql/item_func.h sql/item_geofunc.cc sql/item_strfunc.cc sql/item_subselect.cc sql/item_subselect.h sql/item_sum.cc sql/item_sum.h sql/item_timefunc.cc sql/item_xmlfunc.cc sql/log.cc sql/log_event.cc sql/log_event.h sql/mysqld.cc sql/opt_range.cc sql/repl_failsafe.cc sql/rpl_record.cc sql/rpl_record.h sql/rpl_rli.cc sql/rpl_tblmap.cc sql/set_var.cc sql/sp.cc sql/sp.h sql/sp_cache.cc sql/sp_head.cc sql/sp_head.h sql/sp_rcontext.cc sql/sql_acl.cc sql/sql_acl.h sql/sql_base.cc sql/sql_cache.cc sql/sql_cache.h sql/sql_class.cc sql/sql_delete.cc sql/sql_insert.cc sql/sql_load.cc sql/sql_parse.cc sql/sql_partition.cc sql/sql_select.cc sql/sql_show.cc sql/sql_table.cc sql/sql_yacc.yy sql/table.cc sql/table.h storage/archive/CMakeLists.txt* storage/archive/azio.c storage/archive/ha_archive.cc storage/federated/CMakeLists.txt* storage/innobase/btr/btr0btr.c storage/innobase/data/data0type.c storage/innobase/handler/ha_innodb.cc storage/innobase/include/ha_prototypes.h storage/innobase/include/mach0data.h storage/innobase/include/mach0data.ic storage/innobase/include/os0file.h storage/innobase/include/trx0trx.h storage/innobase/lock/lock0lock.c storage/innobase/os/os0file.c storage/innobase/row/row0sel.c storage/innobase/trx/trx0trx.c storage/innobase/ut/ut0ut.c storage/innodb_plugin/CMakeLists.txt storage/innodb_plugin/ChangeLog storage/innodb_plugin/btr/btr0btr.c storage/innodb_plugin/btr/btr0sea.c storage/innodb_plugin/buf/buf0buf.c storage/innodb_plugin/data/data0type.c storage/innodb_plugin/dict/dict0dict.c storage/innodb_plugin/fil/fil0fil.c storage/innodb_plugin/handler/ha_innodb.cc storage/innodb_plugin/handler/ha_innodb.h storage/innodb_plugin/handler/handler0alter.cc storage/innodb_plugin/ibuf/ibuf0ibuf.c storage/innodb_plugin/include/btr0sea.h storage/innodb_plugin/include/db0err.h storage/innodb_plugin/include/dict0dict.h storage/innodb_plugin/include/fil0fil.h storage/innodb_plugin/include/ha_prototypes.h storage/innodb_plugin/include/ibuf0ibuf.h storage/innodb_plugin/include/lock0lock.h storage/innodb_plugin/include/log0log.h storage/innodb_plugin/include/log0recv.h storage/innodb_plugin/include/mem0mem.h storage/innodb_plugin/include/mem0pool.h storage/innodb_plugin/include/os0file.h storage/innodb_plugin/include/pars0pars.h storage/innodb_plugin/include/srv0srv.h storage/innodb_plugin/include/thr0loc.h storage/innodb_plugin/include/trx0i_s.h storage/innodb_plugin/include/trx0purge.h storage/innodb_plugin/include/trx0rseg.h storage/innodb_plugin/include/trx0sys.h storage/innodb_plugin/include/trx0trx.h storage/innodb_plugin/include/trx0undo.h storage/innodb_plugin/include/univ.i storage/innodb_plugin/include/usr0sess.h storage/innodb_plugin/lock/lock0lock.c storage/innodb_plugin/log/log0log.c storage/innodb_plugin/log/log0recv.c storage/innodb_plugin/mem/mem0dbg.c storage/innodb_plugin/mem/mem0pool.c storage/innodb_plugin/os/os0file.c storage/innodb_plugin/os/os0sync.c storage/innodb_plugin/os/os0thread.c storage/innodb_plugin/pars/lexyy.c storage/innodb_plugin/pars/pars0lex.l storage/innodb_plugin/que/que0que.c storage/innodb_plugin/row/row0merge.c storage/innodb_plugin/row/row0mysql.c storage/innodb_plugin/srv/srv0srv.c storage/innodb_plugin/srv/srv0start.c storage/innodb_plugin/sync/sync0arr.c storage/innodb_plugin/sync/sync0sync.c storage/innodb_plugin/thr/thr0loc.c storage/innodb_plugin/trx/trx0i_s.c storage/innodb_plugin/trx/trx0purge.c storage/innodb_plugin/trx/trx0rseg.c storage/innodb_plugin/trx/trx0sys.c storage/innodb_plugin/trx/trx0trx.c storage/innodb_plugin/trx/trx0undo.c storage/innodb_plugin/usr/usr0sess.c storage/innodb_plugin/ut/ut0mem.c storage/maria/ma_check.c storage/maria/ma_create.c storage/myisam/ft_boolean_search.c storage/myisam/mi_check.c storage/myisam/mi_create.c storage/xtradb/CMakeLists.txt storage/xtradb/ChangeLog storage/xtradb/Makefile.am storage/xtradb/btr/btr0btr.c storage/xtradb/btr/btr0sea.c storage/xtradb/buf/buf0buddy.c storage/xtradb/buf/buf0buf.c storage/xtradb/buf/buf0flu.c storage/xtradb/buf/buf0lru.c storage/xtradb/buf/buf0rea.c storage/xtradb/data/data0type.c storage/xtradb/dict/dict0crea.c storage/xtradb/dict/dict0dict.c storage/xtradb/fil/fil0fil.c storage/xtradb/fsp/fsp0fsp.c storage/xtradb/handler/ha_innodb.cc storage/xtradb/handler/ha_innodb.h storage/xtradb/handler/handler0alter.cc storage/xtradb/handler/i_s.cc storage/xtradb/handler/i_s.h storage/xtradb/handler/innodb_patch_info.h storage/xtradb/ibuf/ibuf0ibuf.c storage/xtradb/include/btr0cur.h storage/xtradb/include/btr0sea.h storage/xtradb/include/buf0buf.h storage/xtradb/include/buf0buf.ic storage/xtradb/include/buf0lru.h storage/xtradb/include/buf0rea.h storage/xtradb/include/buf0types.h storage/xtradb/include/db0err.h storage/xtradb/include/dict0crea.h storage/xtradb/include/dict0dict.h storage/xtradb/include/dict0mem.h storage/xtradb/include/fil0fil.h storage/xtradb/include/fsp0fsp.h storage/xtradb/include/ibuf0ibuf.h storage/xtradb/include/lock0lock.h storage/xtradb/include/log0log.h storage/xtradb/include/log0log.ic storage/xtradb/include/log0recv.h storage/xtradb/include/mem0mem.h storage/xtradb/include/mem0pool.h storage/xtradb/include/mtr0mtr.h storage/xtradb/include/os0file.h storage/xtradb/include/os0sync.h storage/xtradb/include/page0page.h storage/xtradb/include/page0page.ic storage/xtradb/include/page0zip.h storage/xtradb/include/pars0pars.h storage/xtradb/include/rem0cmp.h storage/xtradb/include/rem0rec.ic storage/xtradb/include/row0ins.h storage/xtradb/include/row0mysql.h storage/xtradb/include/srv0srv.h storage/xtradb/include/sync0rw.h storage/xtradb/include/sync0sync.h storage/xtradb/include/thr0loc.h storage/xtradb/include/trx0i_s.h storage/xtradb/include/trx0purge.h storage/xtradb/include/trx0rec.h storage/xtradb/include/trx0rec.ic storage/xtradb/include/trx0roll.h storage/xtradb/include/trx0rseg.h storage/xtradb/include/trx0sys.h storage/xtradb/include/trx0sys.ic storage/xtradb/include/trx0trx.h storage/xtradb/include/trx0types.h storage/xtradb/include/trx0undo.h storage/xtradb/include/univ.i storage/xtradb/include/usr0sess.h storage/xtradb/include/ut0auxconf.h storage/xtradb/include/ut0byte.h storage/xtradb/include/ut0byte.ic storage/xtradb/include/ut0ut.h storage/xtradb/lock/lock0lock.c storage/xtradb/log/log0log.c storage/xtradb/log/log0recv.c storage/xtradb/mem/mem0dbg.c storage/xtradb/mem/mem0mem.c storage/xtradb/mem/mem0pool.c storage/xtradb/mtr/mtr0mtr.c storage/xtradb/os/os0file.c storage/xtradb/os/os0proc.c storage/xtradb/os/os0sync.c storage/xtradb/os/os0thread.c storage/xtradb/page/page0cur.c storage/xtradb/page/page0page.c storage/xtradb/page/page0zip.c storage/xtradb/pars/lexyy.c storage/xtradb/pars/pars0lex.l storage/xtradb/plug.in storage/xtradb/que/que0que.c storage/xtradb/rem/rem0cmp.c storage/xtradb/row/row0ins.c storage/xtradb/row/row0merge.c storage/xtradb/row/row0mysql.c storage/xtradb/row/row0sel.c storage/xtradb/scripts/install_innodb_plugins.sql storage/xtradb/scripts/install_innodb_plugins_win.sql storage/xtradb/srv/srv0srv.c storage/xtradb/srv/srv0start.c storage/xtradb/sync/sync0arr.c storage/xtradb/sync/sync0rw.c storage/xtradb/sync/sync0sync.c storage/xtradb/thr/thr0loc.c storage/xtradb/trx/trx0i_s.c storage/xtradb/trx/trx0purge.c storage/xtradb/trx/trx0rec.c storage/xtradb/trx/trx0roll.c storage/xtradb/trx/trx0rseg.c storage/xtradb/trx/trx0sys.c storage/xtradb/trx/trx0trx.c storage/xtradb/trx/trx0undo.c storage/xtradb/usr/usr0sess.c storage/xtradb/ut/ut0auxconf_atomic_pthread_t_solaris.c storage/xtradb/ut/ut0mem.c storage/xtradb/ut/ut0ut.c vio/vio.c vio/viosocket.c === modified file 'Makefile.am' --- a/Makefile.am 2009-12-03 11:19:05 +0000 +++ b/Makefile.am 2010-01-15 15:27:55 +0000 @@ -208,10 +208,6 @@ test-bt-fast: -cd mysql-test ; MTR_BUILD_THREAD=auto \ @PERL@ ./mysql-test-run.pl $(MTR_EXTRA_OPTIONS) --force --comment=stress --suite=stress -test-bt-fast2: - -cd mysql-test ; MTR_BUILD_THREAD=auto \ - @PERL@ ./mysql-test-run.pl $(MTR_EXTRA_OPTIONS) --force --comment=ps --ps-protocol --report-features - test-bt-debug: -cd mysql-test ; MTR_BUILD_THREAD=auto \ @PERL@ ./mysql-test-run.pl $(MTR_EXTRA_OPTIONS) --comment=debug --force --timer \ === modified file 'client/mysql.cc' --- a/client/mysql.cc 2009-12-03 11:34:11 +0000 +++ b/client/mysql.cc 2010-01-15 15:27:55 +0000 @@ -4356,7 +4356,7 @@ com_status(String *buffer __attribute__( Don't remove "limit 1", it is protection againts SQL_SELECT_LIMIT=0 */ - if (mysql_store_result_for_lazy(&result)) + if (!mysql_store_result_for_lazy(&result)) { MYSQL_ROW cur=mysql_fetch_row(result); if (cur) @@ -4401,7 +4401,7 @@ com_status(String *buffer __attribute__( if (mysql_errno(&mysql) == CR_SERVER_GONE_ERROR) return 0; } - if (mysql_store_result_for_lazy(&result)) + if (!mysql_store_result_for_lazy(&result)) { MYSQL_ROW cur=mysql_fetch_row(result); if (cur) @@ -4496,9 +4496,7 @@ server_version_string(MYSQL *con) */ if (server_version == NULL) - { - server_version= strdup(mysql_get_server_info(con)); - } + server_version= my_strdup(mysql_get_server_info(con), MYF(MY_WME)); } return server_version ? server_version : ""; === modified file 'client/mysqlbinlog.cc' --- a/client/mysqlbinlog.cc 2009-12-03 11:19:05 +0000 +++ b/client/mysqlbinlog.cc 2010-01-09 09:04:51 +0000 @@ -1378,6 +1378,10 @@ static int parse_args(int *argc, char*** */ static Exit_status safe_connect() { + /* Close and old connections to MySQL */ + if (mysql) + mysql_close(mysql); + mysql= mysql_init(NULL); if (!mysql) === modified file 'configure.in' --- a/configure.in 2010-01-09 10:45:27 +0000 +++ b/configure.in 2010-01-17 17:22:46 +0000 @@ -9,15 +9,16 @@ AC_CANONICAL_SYSTEM # remember to also update version.c in ndb # # When changing major version number please also check switch statement -# in mysqlbinlog.cc / check_master_version(). -# -# When merging new MySQL releases, update the version number to match the -# MySQL version number. -# -# Note: the following line must be parseable by win/configure.js:GetVersion() -AM_INIT_AUTOMAKE(mysql, 5.1.41m1-MariaDB-rc) +# in mysqlbinlog::check_master_version(). +AM_INIT_AUTOMAKE(mysql, 5.1.42-MariaDB-rc) AM_CONFIG_HEADER([include/config.h:config.h.in]) +# Request support for automake silent-rules if available. +# Default to verbose output. One can use the configure-time +# option --enable-silent-rules or make V=0 to activate +# silent rules. +m4_ifdef([AM_SILENT_RULES], [AM_SILENT_RULES([no])]) + PROTOCOL_VERSION=10 DOT_FRM_VERSION=6 # See the libtool docs for information on how to do shared lib versions. @@ -224,14 +225,6 @@ then GXX="no" fi -if test "$ac_cv_prog_gcc" = "yes" -then - AS="$CC -c" - AC_SUBST(AS) -else - AC_PATH_PROG(AS, as, as) -fi - # Still need ranlib for readline; local static use only so no libtool. AC_PROG_RANLIB # We use libtool @@ -688,7 +681,7 @@ AC_ARG_ENABLE(assembler, AC_MSG_CHECKING(if we should use assembler functions) # For now we only support assembler on i386 and sparc systems -AM_CONDITIONAL(ASSEMBLER_x86, test "$ENABLE_ASSEMBLER" = "yes" -a "$BASE_MACHINE_TYPE" = "i386" && $AS strings/strings-x86.s -o checkassembler >/dev/null 2>&1 && test -f checkassembler && (rm -f checkassembler; exit 0;)) +AM_CONDITIONAL(ASSEMBLER_x86, test "$ENABLE_ASSEMBLER" = "yes" -a "$BASE_MACHINE_TYPE" = "i386" && $CCAS $CCASFLAGS -c strings/strings-x86.s -o checkassembler >/dev/null 2>&1 && test -f checkassembler && (rm -f checkassembler; exit 0;)) AM_CONDITIONAL(ASSEMBLER_sparc32, test "$ENABLE_ASSEMBLER" = "yes" -a "$BASE_MACHINE_TYPE" = "sparc") AM_CONDITIONAL(ASSEMBLER_sparc64, test "$ENABLE_ASSEMBLER" = "yes" -a "$BASE_MACHINE_TYPE" = "sparcv9") AM_CONDITIONAL(ASSEMBLER, test "$ASSEMBLER_x86_TRUE" = "" -o "$ASSEMBLER_sparc32_TRUE" = "") === modified file 'extra/comp_err.c' --- a/extra/comp_err.c 2009-02-13 16:41:47 +0000 +++ b/extra/comp_err.c 2009-11-20 10:11:31 +0000 @@ -660,7 +660,7 @@ static ha_checksum checksum_format_speci case 'u': case 'x': case 's': - chksum= my_checksum(chksum, start, (uint) (p - start)); + chksum= my_checksum(chksum, start, (uint) (p + 1 - start)); start= 0; /* Not in format specifier anymore */ break; @@ -1030,8 +1030,10 @@ static char *parse_text_line(char *pos) { int i, nr; char *row= pos; + size_t len; DBUG_ENTER("parse_text_line"); + len= strlen (pos); while (*pos) { if (*pos == '\\') @@ -1039,11 +1041,11 @@ static char *parse_text_line(char *pos) switch (*++pos) { case '\\': case '"': - VOID(strmov(pos - 1, pos)); + VOID(memmove (pos - 1, pos, len - (row - pos))); break; case 'n': pos[-1]= '\n'; - VOID(strmov(pos, pos + 1)); + VOID(memmove (pos, pos + 1, len - (row - pos))); break; default: if (*pos >= '0' && *pos < '8') @@ -1053,10 +1055,10 @@ static char *parse_text_line(char *pos) nr= nr * 8 + (*(pos++) - '0'); pos -= i; pos[-1]= nr; - VOID(strmov(pos, pos + i)); + VOID(memmove (pos, pos + i, len - (row - pos))); } else if (*pos) - VOID(strmov(pos - 1, pos)); /* Remove '\' */ + VOID(memmove (pos - 1, pos, len - (row - pos))); /* Remove '\' */ } } else === modified file 'include/my_pthread.h' --- a/include/my_pthread.h 2009-06-30 12:01:29 +0000 +++ b/include/my_pthread.h 2010-01-14 16:51:00 +0000 @@ -543,9 +543,9 @@ void safe_mutex_free_deadlock_data(safe_ #else #define my_pthread_mutex_init(A,B,C,D) pthread_mutex_init((A),(B)) #define my_pthread_mutex_lock(A,B) pthread_mutex_lock(A) -#define safe_mutex_assert_owner(mp) -#define safe_mutex_assert_not_owner(mp) -#define safe_mutex_free_deadlock_data(mp) +#define safe_mutex_assert_owner(mp) do {} while(0) +#define safe_mutex_assert_not_owner(mp) do {} while(0) +#define safe_mutex_free_deadlock_data(mp) do {} while(0) #endif /* SAFE_MUTEX */ #if defined(MY_PTHREAD_FASTMUTEX) && !defined(SAFE_MUTEX) === modified file 'include/mysql.h' --- a/include/mysql.h 2009-12-03 11:19:05 +0000 +++ b/include/mysql.h 2010-01-15 15:27:55 +0000 @@ -558,16 +558,6 @@ unsigned long STDCALL mysql_real_escape_ char *to,const char *from, unsigned long length); void STDCALL mysql_debug(const char *debug); -char * STDCALL mysql_odbc_escape_string(MYSQL *mysql, - char *to, - unsigned long to_length, - const char *from, - unsigned long from_length, - void *param, - char * - (*extend_buffer) - (void *, char *to, - unsigned long *length)); void STDCALL myodbc_remove_escape(MYSQL *mysql,char *name); unsigned int STDCALL mysql_thread_safe(void); my_bool STDCALL mysql_embedded(void); === modified file 'include/mysql.h.pp' --- a/include/mysql.h.pp 2009-12-03 11:19:05 +0000 +++ b/include/mysql.h.pp 2010-01-15 15:27:55 +0000 @@ -518,16 +518,6 @@ unsigned long mysql_real_escape_string(M char *to,const char *from, unsigned long length); void mysql_debug(const char *debug); -char * mysql_odbc_escape_string(MYSQL *mysql, - char *to, - unsigned long to_length, - const char *from, - unsigned long from_length, - void *param, - char * - (*extend_buffer) - (void *, char *to, - unsigned long *length)); void myodbc_remove_escape(MYSQL *mysql,char *name); unsigned int mysql_thread_safe(void); my_bool mysql_embedded(void); === modified file 'include/violite.h' --- a/include/violite.h 2009-12-03 11:19:05 +0000 +++ b/include/violite.h 2010-01-15 15:27:55 +0000 @@ -225,8 +225,8 @@ struct st_vio #endif /* HAVE_SMEM */ #ifdef _WIN32 OVERLAPPED pipe_overlapped; - DWORD read_timeout_millis; - DWORD write_timeout_millis; + DWORD read_timeout_ms; + DWORD write_timeout_ms; #endif }; #endif /* vio_violite_h_ */ === modified file 'libmysql/libmysql.c' --- a/libmysql/libmysql.c 2009-12-03 11:34:11 +0000 +++ b/libmysql/libmysql.c 2010-01-15 15:27:55 +0000 @@ -1642,20 +1642,6 @@ mysql_real_escape_string(MYSQL *mysql, c return (uint) escape_string_for_mysql(mysql->charset, to, 0, from, length); } - -char * STDCALL -mysql_odbc_escape_string(MYSQL *mysql __attribute__((unused)), - char *to __attribute__((unused)), - ulong to_length __attribute__((unused)), - const char *from __attribute__((unused)), - ulong from_length __attribute__((unused)), - void *param __attribute__((unused)), - char * (*extend_buffer)(void *, char *, ulong *) - __attribute__((unused))) -{ - return NULL; -} - void STDCALL myodbc_remove_escape(MYSQL *mysql,char *name) { === modified file 'libmysql/libmysql.def' --- a/libmysql/libmysql.def 2009-12-03 11:19:05 +0000 +++ b/libmysql/libmysql.def 2010-01-15 15:27:55 +0000 @@ -78,7 +78,6 @@ EXPORTS mysql_next_result mysql_num_fields mysql_num_rows - mysql_odbc_escape_string mysql_options mysql_stmt_param_count mysql_stmt_param_metadata === modified file 'libmysqld/libmysqld.def' --- a/libmysqld/libmysqld.def 2009-12-03 11:19:05 +0000 +++ b/libmysqld/libmysqld.def 2010-01-15 15:27:55 +0000 @@ -50,7 +50,6 @@ EXPORTS mysql_next_result mysql_num_fields mysql_num_rows - mysql_odbc_escape_string mysql_options mysql_ping mysql_query === modified file 'mysql-test/collections/default.experimental' --- a/mysql-test/collections/default.experimental 2009-10-26 12:33:03 +0000 +++ b/mysql-test/collections/default.experimental 2009-12-02 09:47:49 +0000 @@ -13,15 +13,13 @@ funcs_1.ndb* funcs_2.ndb_charset # joro : NDB tests marked as experimental as agreed with bochklin main.ctype_gbk_binlog @solaris # Bug#46010: main.ctype_gbk_binlog fails sporadically : Table 't2' already exists -main.innodb-autoinc* # Bug#47809 2009-10-04 joro innodb-autoinc.test fails with valgrind errors with the innodb plugin main.plugin_load @solaris # Bug#42144 ndb.* # joro : NDB tests marked as experimental as agreed with bochklin -rpl.rpl_cross_version* # Bug #43913 2009-10-26 joro rpl_cross_version can't pass on conflicts complainig clash with --slave-load-tm -rpl.rpl_get_master_version_and_clock* # Bug#46931 2009-08-26 alik rpl.rpl_get_master_version_and_clock fails on hpux11.31 +rpl.rpl_cross_version* # Bug#48340 2009-12-01 Daogang rpl_cross_version: Found warnings/errors in server log file! +rpl.rpl_get_master_version_and_clock* # Bug #49191 2009-12-01 Daogang rpl_get_master_version_and_clock failed on PB2: COM_REGISTER_SLAVE failed rpl.rpl_innodb_bug28430* @solaris # Bug#46029 -rpl.rpl_row_create_table* # Bug#45576: rpl_row_create_table fails on PB2 rpl.rpl_trigger* # Bug#47810 2009-10-04 joro rpl.rpl_trigger.test fails with valgrind errors with the innodb plugin rpl_ndb.* # joro : NDB tests marked as experimental as agreed with bochklin === modified file 'mysql-test/extra/rpl_tests/rpl_extraSlave_Col.test' --- a/mysql-test/extra/rpl_tests/rpl_extraSlave_Col.test 2009-08-28 14:13:27 +0000 +++ b/mysql-test/extra/rpl_tests/rpl_extraSlave_Col.test 2009-10-22 00:21:50 +0000 @@ -407,37 +407,57 @@ sync_slave_with_master; ########################################### # Bug#22234, Bug#23907 Extra Slave Col is not # erroring on extra col with no default values. -######################################################## +############################################################### +# Error reaction is up to sql_mode of the slave sql (bug#38173) #--echo *** Create t9 on slave *** -STOP SLAVE; -RESET SLAVE; -eval CREATE TABLE t9 (a INT KEY, b BLOB, c CHAR(5), - d TIMESTAMP, - e INT NOT NULL) ENGINE=$engine_type; - ---echo *** Create t9 on Master *** -connection master; -eval CREATE TABLE t9 (a INT PRIMARY KEY, b BLOB, c CHAR(5) +# Please, check BUG#47741 to see why you are not testing NDB. +if (`SELECT $engine_type != 'NDB'`) +{ + STOP SLAVE; + RESET SLAVE; + eval CREATE TABLE t9 (a INT KEY, b BLOB, c CHAR(5), + d TIMESTAMP, + e INT NOT NULL, + f text not null, + g text, + h blob not null, + i blob) ENGINE=$engine_type; + + --echo *** Create t9 on Master *** + connection master; + eval CREATE TABLE t9 (a INT PRIMARY KEY, b BLOB, c CHAR(5) ) ENGINE=$engine_type; -RESET MASTER; + RESET MASTER; ---echo *** Start Slave *** -connection slave; -START SLAVE; - ---echo *** Master Data Insert *** -connection master; -set @b1 = 'b1b1b1b1'; -set @b1 = concat(@b1,@b1); -INSERT INTO t9 () VALUES(1,@b1,'Kyle'),(2,@b1,'JOE'),(3,@b1,'QA'); - -connection slave; ---source include/wait_for_slave_sql_to_stop.inc ---replace_result $MASTER_MYPORT MASTER_PORT ---replace_column 1 # 4 # 7 # 8 # 9 # 16 # 22 # 23 # 33 # 35 # 36 # ---query_vertical SHOW SLAVE STATUS -SET GLOBAL SQL_SLAVE_SKIP_COUNTER=2; -START SLAVE; + --echo *** Start Slave *** + connection slave; + START SLAVE; + + --echo *** Master Data Insert *** + connection master; + set @b1 = 'b1b1b1b1'; + + set @b1 = concat(@b1,@b1); + INSERT INTO t9 () VALUES(1,@b1,'Kyle'),(2,@b1,'JOE'),(3,@b1,'QA'); + + # the test would stop slave if @@sql_mode for the sql thread + # was set to strict. Otherwise, as with this tests setup, + # the implicit defaults will be inserted into fields even though + # they are declared without DEFAULT clause. + + sync_slave_with_master; + select * from t9; + + # todo: fix Bug #43992 slave sql thread can't tune own sql_mode ... + # and add/restore waiting for stop test + + #--source include/wait_for_slave_sql_to_stop.inc + #--replace_result $MASTER_MYPORT MASTER_PORT + #--replace_column 1 # 4 # 7 # 8 # 9 # 16 # 22 # 23 # 33 # 35 # 36 # + #--query_vertical SHOW SLAVE STATUS + #SET GLOBAL SQL_SLAVE_SKIP_COUNTER=2; + #START SLAVE; +} #--echo *** Drop t9 *** #connection master; === added file 'mysql-test/extra/rpl_tests/rpl_not_null.test' --- a/mysql-test/extra/rpl_tests/rpl_not_null.test 1970-01-01 00:00:00 +0000 +++ b/mysql-test/extra/rpl_tests/rpl_not_null.test 2009-10-22 00:19:52 +0000 @@ -0,0 +1,364 @@ +################################################################################# +# This test checks if the replication between "null" fields to either "null" +# fields or "not null" fields works properly. In the first case, the execution +# should work fine. In the second case, it may fail according to the sql_mode +# being used. +# +# The test is devided in three main parts: +# +# 1 - NULL --> NULL (no failures) +# 2 - NULL --> NOT NULL ( sql-mode = STRICT and failures) +# 3 - NULL --> NOT NULL ( sql-mode != STRICT and no failures) +# +################################################################################# +connection master; + +SET SQL_LOG_BIN= 0; +eval CREATE TABLE t1(`a` INT, `b` DATE DEFAULT NULL, +`c` INT DEFAULT NULL, +PRIMARY KEY(`a`)) ENGINE=$engine DEFAULT CHARSET=LATIN1; + +eval CREATE TABLE t2(`a` INT, `b` DATE DEFAULT NULL, +PRIMARY KEY(`a`)) ENGINE=$engine DEFAULT CHARSET=LATIN1; + +eval CREATE TABLE t3(`a` INT, `b` DATE DEFAULT NULL, +PRIMARY KEY(`a`)) ENGINE=$engine DEFAULT CHARSET=LATIN1; + +eval CREATE TABLE t4(`a` INT, `b` DATE DEFAULT NULL, +`c` INT DEFAULT NULL, +PRIMARY KEY(`a`)) ENGINE=$engine DEFAULT CHARSET=LATIN1; +SET SQL_LOG_BIN= 1; + +connection slave; + +eval CREATE TABLE t1(`a` INT, `b` DATE DEFAULT NULL, +`c` INT DEFAULT NULL, +PRIMARY KEY(`a`)) ENGINE=$engine DEFAULT CHARSET=LATIN1; + +eval CREATE TABLE t2(`a` INT, `b` DATE DEFAULT NULL, +PRIMARY KEY(`a`)) ENGINE=$engine DEFAULT CHARSET=LATIN1; + +eval CREATE TABLE t3(`a` INT, `b` DATE DEFAULT '0000-00-00', +`c` INT DEFAULT 500, +PRIMARY KEY(`a`)) ENGINE=$engine DEFAULT CHARSET=LATIN1; + +eval CREATE TABLE t4(`a` INT, `b` DATE DEFAULT '0000-00-00', +PRIMARY KEY(`a`)) ENGINE=$engine DEFAULT CHARSET=LATIN1; + +--echo ************* EXECUTION WITH INSERTS ************* +connection master; +INSERT INTO t1(a,b,c) VALUES (1, null, 1); +INSERT INTO t1(a,b,c) VALUES (2,'1111-11-11', 2); +INSERT INTO t1(a,b) VALUES (3, null); +INSERT INTO t1(a,c) VALUES (4, 4); +INSERT INTO t1(a) VALUES (5); + +INSERT INTO t2(a,b) VALUES (1, null); +INSERT INTO t2(a,b) VALUES (2,'1111-11-11'); +INSERT INTO t2(a) VALUES (3); + +INSERT INTO t3(a,b) VALUES (1, null); +INSERT INTO t3(a,b) VALUES (2,'1111-11-11'); +INSERT INTO t3(a) VALUES (3); + +INSERT INTO t4(a,b,c) VALUES (1, null, 1); +INSERT INTO t4(a,b,c) VALUES (2,'1111-11-11', 2); +INSERT INTO t4(a,b) VALUES (3, null); +INSERT INTO t4(a,c) VALUES (4, 4); +INSERT INTO t4(a) VALUES (5); + +--echo ************* SHOWING THE RESULT SETS WITH INSERTS ************* +sync_slave_with_master; + +--echo TABLES t1 and t2 must be equal otherwise an error will be thrown. +let $diff_table_1=master:test.t1; +let $diff_table_2=slave:test.t1; +source include/diff_tables.inc; + +let $diff_table_1=master:test.t2; +let $diff_table_2=slave:test.t2; +source include/diff_tables.inc; + +--echo TABLES t2 and t3 must be different. +connection master; +SELECT * FROM t3 ORDER BY a; +connection slave; +SELECT * FROM t3 ORDER BY a; +connection master; +SELECT * FROM t4 ORDER BY a; +connection slave; +SELECT * FROM t4 ORDER BY a; + +--echo ************* EXECUTION WITH UPDATES and REPLACES ************* +connection master; +DELETE FROM t1; +INSERT INTO t1(a,b,c) VALUES (1,'1111-11-11', 1); +REPLACE INTO t1(a,b,c) VALUES (2,'1111-11-11', 2); +UPDATE t1 set b= NULL, c= 300 where a= 1; +REPLACE INTO t1(a,b,c) VALUES (2, NULL, 300); + +--echo ************* SHOWING THE RESULT SETS WITH UPDATES and REPLACES ************* +sync_slave_with_master; + +--echo TABLES t1 and t2 must be equal otherwise an error will be thrown. +let $diff_table_1=master:test.t1; +let $diff_table_2=slave:test.t1; +source include/diff_tables.inc; + +--echo ************* CLEANING ************* +connection master; + +DROP TABLE t1; +DROP TABLE t2; +DROP TABLE t3; +DROP TABLE t4; + +sync_slave_with_master; + +connection master; + +SET SQL_LOG_BIN= 0; +eval CREATE TABLE t1 (`a` INT, `b` BIT DEFAULT NULL, `c` BIT DEFAULT NULL, +PRIMARY KEY (`a`)) ENGINE= $engine; +SET SQL_LOG_BIN= 1; + +connection slave; + +eval CREATE TABLE t1 (`a` INT, `b` BIT DEFAULT b'01', `c` BIT DEFAULT NULL, +PRIMARY KEY (`a`)) ENGINE= $engine; + +--echo ************* EXECUTION WITH INSERTS ************* +connection master; +INSERT INTO t1(a,b,c) VALUES (1, null, b'01'); +INSERT INTO t1(a,b,c) VALUES (2,b'00', b'01'); +INSERT INTO t1(a,b) VALUES (3, null); +INSERT INTO t1(a,c) VALUES (4, b'01'); +INSERT INTO t1(a) VALUES (5); + +--echo ************* SHOWING THE RESULT SETS WITH INSERTS ************* +--echo TABLES t1 and t2 must be different. +sync_slave_with_master; +connection master; +SELECT a,b+0,c+0 FROM t1 ORDER BY a; +connection slave; +SELECT a,b+0,c+0 FROM t1 ORDER BY a; + +--echo ************* EXECUTION WITH UPDATES and REPLACES ************* +connection master; +DELETE FROM t1; +INSERT INTO t1(a,b,c) VALUES (1,b'00', b'01'); +REPLACE INTO t1(a,b,c) VALUES (2,b'00',b'01'); +UPDATE t1 set b= NULL, c= b'00' where a= 1; +REPLACE INTO t1(a,b,c) VALUES (2, NULL, b'00'); + +--echo ************* SHOWING THE RESULT SETS WITH UPDATES and REPLACES ************* +--echo TABLES t1 and t2 must be equal otherwise an error will be thrown. +sync_slave_with_master; +let $diff_table_1=master:test.t1; +let $diff_table_2=slave:test.t1; +source include/diff_tables.inc; + +connection master; + +DROP TABLE t1; + +sync_slave_with_master; + +--echo ################################################################################ +--echo # NULL ---> NOT NULL (STRICT MODE) +--echo # UNCOMMENT THIS AFTER FIXING BUG#43992 +--echo ################################################################################ +#connection slave; +#SET GLOBAL sql_mode="TRADITIONAL"; +# +#STOP SLAVE; +#--source include/wait_for_slave_to_stop.inc +#START SLAVE; +#--source include/wait_for_slave_to_start.inc +# +#let $y=0; +#while (`select $y < 6`) +#{ +# connection master; +# +# SET SQL_LOG_BIN= 0; +# eval CREATE TABLE t1(`a` INT NOT NULL, `b` INT, +# PRIMARY KEY(`a`)) ENGINE=$engine DEFAULT CHARSET=LATIN1; +# eval CREATE TABLE t2(`a` INT NOT NULL, `b` INT, +# PRIMARY KEY(`a`)) ENGINE=$engine DEFAULT CHARSET=LATIN1; +# eval CREATE TABLE t3(`a` INT NOT NULL, `b` INT, +# PRIMARY KEY(`a`)) ENGINE=$engine DEFAULT CHARSET=LATIN1; +# SET SQL_LOG_BIN= 1; +# +# connection slave; +# +# eval CREATE TABLE t1(`a` INT NOT NULL, `b` INT NOT NULL, +# `c` INT NOT NULL, +# PRIMARY KEY(`a`)) ENGINE=$engine DEFAULT CHARSET=LATIN1; +# eval CREATE TABLE t2(`a` INT NOT NULL, `b` INT NOT NULL, +# `c` INT, +# PRIMARY KEY(`a`)) ENGINE=$engine DEFAULT CHARSET=LATIN1; +# eval CREATE TABLE t3(`a` INT NOT NULL, `b` INT NOT NULL, +# `c` INT DEFAULT 500, +# PRIMARY KEY(`a`)) ENGINE=$engine DEFAULT CHARSET=LATIN1; +# +# if (`select $y=0`) +# { +# --echo ************* EXECUTION WITH INSERTS ************* +# connection master; +# INSERT INTO t1(a) VALUES (1); +# } +# +# if (`select $y=1`) +# { +# --echo ************* EXECUTION WITH INSERTS ************* +# connection master; +# INSERT INTO t1(a, b) VALUES (1, NULL); +# } +# +# if (`select $y=2`) +# { +# --echo ************* EXECUTION WITH UPDATES ************* +# connection master; +# INSERT INTO t3(a, b) VALUES (1, 1); +# INSERT INTO t3(a, b) VALUES (2, 1); +# UPDATE t3 SET b = NULL where a= 1; +# } +# +# if (`select $y=3`) +# { +# --echo ************* EXECUTION WITH INSERTS/REPLACES ************* +# connection master; +# REPLACE INTO t3(a, b) VALUES (1, null); +# } +# +# if (`select $y=4`) +# { +# --echo ************* EXECUTION WITH UPDATES/REPLACES ************* +# connection master; +# INSERT INTO t3(a, b) VALUES (1, 1); +# REPLACE INTO t3(a, b) VALUES (1, null); +# } +# +# if (`select $y=5`) +# { +# --echo ************* EXECUTION WITH MULTI-ROW INSERTS ************* +# connection master; +# +# SET SQL_LOG_BIN= 0; +# INSERT INTO t2(a, b) VALUES (1, 1); +# INSERT INTO t2(a, b) VALUES (2, 1); +# INSERT INTO t2(a, b) VALUES (3, null); +# INSERT INTO t2(a, b) VALUES (4, 1); +# INSERT INTO t2(a, b) VALUES (5, 1); +# SET SQL_LOG_BIN= 1; +# +# INSERT INTO t2 SELECT a + 10, b from t2; +# --echo The statement below is just executed to stop processing +# INSERT INTO t1(a) VALUES (1); +# } +# +# --echo ************* SHOWING THE RESULT SETS ************* +# connection slave; +# --source include/wait_for_slave_sql_to_stop.inc +# connection master; +# SELECT * FROM t1 ORDER BY a; +# connection slave; +# SELECT * FROM t1 ORDER BY a; +# connection master; +# SELECT * FROM t2 ORDER BY a; +# connection slave; +# SELECT * FROM t2 ORDER BY a; +# connection master; +# SELECT * FROM t3 ORDER BY a; +# connection slave; +# SELECT * FROM t3 ORDER BY a; +# --source include/reset_master_and_slave.inc +# +# connection master; +# +# DROP TABLE t1; +# DROP TABLE t2; +# DROP TABLE t3; +# +# sync_slave_with_master; +# +# inc $y; +#} +#connection slave; +#SET GLOBAL sql_mode=""; +# +#STOP SLAVE; +#source include/wait_for_slave_to_stop.inc; +#START SLAVE; +#--source include/wait_for_slave_to_start.inc + +--echo ################################################################################ +--echo # NULL ---> NOT NULL (NON-STRICT MODE) +--echo ################################################################################ +connection master; + +SET SQL_LOG_BIN= 0; +eval CREATE TABLE t1(`a` INT NOT NULL, `b` INT, +PRIMARY KEY(`a`)) ENGINE=$engine DEFAULT CHARSET=LATIN1; +eval CREATE TABLE t2(`a` INT NOT NULL, `b` INT, +PRIMARY KEY(`a`)) ENGINE=$engine DEFAULT CHARSET=LATIN1; +eval CREATE TABLE t3(`a` INT NOT NULL, `b` INT, +PRIMARY KEY(`a`)) ENGINE=$engine DEFAULT CHARSET=LATIN1; +SET SQL_LOG_BIN= 1; + +connection slave; + +eval CREATE TABLE t1(`a` INT NOT NULL, `b` INT NOT NULL, +`c` INT NOT NULL, +PRIMARY KEY(`a`)) ENGINE=$engine DEFAULT CHARSET=LATIN1; +eval CREATE TABLE t2(`a` INT NOT NULL, `b` INT NOT NULL, +`c` INT, +PRIMARY KEY(`a`)) ENGINE=$engine DEFAULT CHARSET=LATIN1; +eval CREATE TABLE t3(`a` INT NOT NULL, `b` INT NOT NULL, +`c` INT DEFAULT 500, +PRIMARY KEY(`a`)) ENGINE=$engine DEFAULT CHARSET=LATIN1; + +--echo ************* EXECUTION WITH INSERTS ************* +connection master; +INSERT INTO t1(a) VALUES (1); +INSERT INTO t1(a, b) VALUES (2, NULL); +INSERT INTO t1(a, b) VALUES (3, 1); + +INSERT INTO t2(a) VALUES (1); +INSERT INTO t2(a, b) VALUES (2, NULL); +INSERT INTO t2(a, b) VALUES (3, 1); + +INSERT INTO t3(a) VALUES (1); +INSERT INTO t3(a, b) VALUES (2, NULL); +INSERT INTO t3(a, b) VALUES (3, 1); +INSERT INTO t3(a, b) VALUES (4, 1); +REPLACE INTO t3(a, b) VALUES (5, null); + +REPLACE INTO t3(a, b) VALUES (3, null); +UPDATE t3 SET b = NULL where a = 4; + +--echo ************* SHOWING THE RESULT SETS ************* +connection master; +sync_slave_with_master; + +connection master; +SELECT * FROM t1 ORDER BY a; +connection slave; +SELECT * FROM t1 ORDER BY a; +connection master; +SELECT * FROM t2 ORDER BY a; +connection slave; +SELECT * FROM t2 ORDER BY a; +connection master; +SELECT * FROM t3 ORDER BY a; +connection slave; +SELECT * FROM t3 ORDER BY a; + +connection master; + +DROP TABLE t1; +DROP TABLE t2; +DROP TABLE t3; + +sync_slave_with_master; === modified file 'mysql-test/extra/rpl_tests/rpl_row_tabledefs.test' --- a/mysql-test/extra/rpl_tests/rpl_row_tabledefs.test 2008-03-14 20:02:52 +0000 +++ b/mysql-test/extra/rpl_tests/rpl_row_tabledefs.test 2009-10-22 00:10:42 +0000 @@ -111,21 +111,18 @@ SELECT a,b,x FROM t1_int ORDER BY a; SELECT a,b,HEX(x),HEX(y),HEX(z) FROM t1_bit ORDER BY a; SELECT a,b,x FROM t1_char ORDER BY a; -# Each of these inserts should generate an error and stop the slave - connection master; INSERT INTO t9 VALUES (2); sync_slave_with_master; # Now slave is guaranteed to be running connection master; INSERT INTO t1_nodef VALUES (1,2); -connection slave; ---source include/wait_for_slave_sql_to_stop.inc ---replace_result $MASTER_MYPORT MASTER_PORT ---replace_column 1 # 4 # 7 # 8 # 9 # 20 <Last_Error> 22 # 23 # 33 # 35 <Last_IO_Errno> 36 <Last_IO_Error> 38 <Last_SQL_Error> ---query_vertical SHOW SLAVE STATUS -SET GLOBAL SQL_SLAVE_SKIP_COUNTER=2; -START SLAVE; + +# Last insert on wider slave table succeeds while slave sql sql_mode permits. +# The previous version of the above test expected slave sql to stop. +# bug#38173 relaxed conditions to stop only with the strict mode. +sync_slave_with_master; +select count(*) from t1_nodef; # # Replicating to tables with fewer columns at the end works as of WL#3228 === modified file 'mysql-test/extra/rpl_tests/rpl_stm_000001.test' --- a/mysql-test/extra/rpl_tests/rpl_stm_000001.test 2009-10-20 18:00:07 +0000 +++ b/mysql-test/extra/rpl_tests/rpl_stm_000001.test 2009-11-18 14:50:31 +0000 @@ -1,6 +1,11 @@ --- source include/have_binlog_format_mixed_or_statement.inc +# Requires binlog_format=statement format since query involving +# get_lock() is logged in row format if binlog_format=mixed or row. +-- source include/have_binlog_format_statement.inc -- source include/master-slave.inc +CALL mtr.add_suppression("Statement may not be safe to log in statement format."); + +# Load some data into t1 create table t1 (word char(20) not null); load data infile '../../std_data/words.dat' into table t1; --replace_result $MYSQL_TEST_DIR MYSQL_TEST_DIR @@ -10,9 +15,7 @@ select * from t1 limit 10; # # Test slave with wrong password # -save_master_pos; -connection slave; -sync_with_master; +sync_slave_with_master; stop slave; connection master; set password for root@"localhost" = password('foo'); @@ -29,16 +32,12 @@ sleep 2; create table t3(n int); insert into t3 values(1),(2); -save_master_pos; -connection slave; -sync_with_master; +sync_slave_with_master; select * from t3; select sum(length(word)) from t1; connection master; drop table t1,t3; -save_master_pos; -connection slave; -sync_with_master; +sync_slave_with_master; # Test if the slave SQL thread can be more than 16K behind the slave # I/O thread (> IO_SIZE) @@ -77,12 +76,13 @@ unlock tables; connection master; create table t2(id int); insert into t2 values(connection_id()); -save_master_pos; connection master1; # Avoid generating result create temporary table t3(n int); +--disable_warnings insert into t3 select get_lock('crash_lock%20C', 1) from t2; +--enable_warnings connection master; send update t1 set n = n + get_lock('crash_lock%20C', 2); @@ -93,8 +93,11 @@ kill @id; # We don't drop t3 as this is a temporary table drop table t2; connection master; +# The get_lock function causes warning for unsafe statement. +--disable_warnings --error 1317,2013 reap; +--enable_warnings connection slave; # The SQL slave thread should now have stopped because the query was killed on # the master (so it has a non-zero error code in the binlog). @@ -117,16 +120,12 @@ insert into mysql.user (Host, User, Pass select select_priv,user from mysql.user where user = _binary'blafasel2'; update mysql.user set Select_priv = "Y" where User= _binary"blafasel2"; select select_priv,user from mysql.user where user = _binary'blafasel2'; -save_master_pos; -connection slave; -sync_with_master; +sync_slave_with_master; select n from t1; select select_priv,user from mysql.user where user = _binary'blafasel2'; connection master1; drop table t1; delete from mysql.user where user="blafasel2"; -save_master_pos; -connection slave; -sync_with_master; +sync_slave_with_master; # End of 4.1 tests === modified file 'mysql-test/include/mtr_warnings.sql' --- a/mysql-test/include/mtr_warnings.sql 2009-12-03 11:19:05 +0000 +++ b/mysql-test/include/mtr_warnings.sql 2010-01-15 15:27:55 +0000 @@ -175,6 +175,8 @@ INSERT INTO global_suppressions VALUES ("Can't find file: '.\\\\test\\\\\\?{8}.frm'"), ("Slave: Unknown table 't1' Error_code: 1051"), + /* Maria storage engine dependent tests */ + /* maria-recovery.test has warning about missing log file */ ("File '.*maria_log.000.*' not found \$Errcode: 2\$"), /* and about marked-corrupted table */ @@ -184,6 +186,14 @@ INSERT INTO global_suppressions VALUES ("Table '..mysqltest.t_corrupted2' is marked as crashed and should be"), ("Incorrect key file for table '..mysqltest.t_corrupted2.MAI'"), + /* + Transient network failures that cause warnings on reconnect. + BUG#47743 and BUG#47983. + */ + ("Slave I/O: Get master SERVER_ID failed with error:.*"), + ("Slave I/O: Get master clock failed with error:.*"), + ("Slave I/O: Get master COLLATION_SERVER failed with error:.*"), + ("Slave I/O: Get master TIME_ZONE failed with error:.*"), ("THE_LAST_SUPPRESSION")|| === modified file 'mysql-test/include/ps_conv.inc' --- a/mysql-test/include/ps_conv.inc 2007-07-04 09:20:53 +0000 +++ b/mysql-test/include/ps_conv.inc 2010-01-19 10:36:52 +0000 @@ -30,6 +30,9 @@ --disable_warnings drop table if exists t5 ; --enable_warnings +--disable_query_log +SET TIME_ZONE= '+03:00'; +--enable_query_log set @arg01= 8; set @arg02= 8.0; set @arg03= 80.00000000000e-1; === modified file 'mysql-test/lib/mtr_cases.pm' --- a/mysql-test/lib/mtr_cases.pm 2009-12-06 17:34:54 +0000 +++ b/mysql-test/lib/mtr_cases.pm 2010-01-15 15:27:55 +0000 @@ -524,6 +524,10 @@ sub collect_one_suite next if ($test->{'name'} eq 'sys_vars.innodb_thread_concurrency_basic'); # Can't work with InnoPlug. Test framework needs to be re-designed. next if ($test->{'name'} eq 'main.innodb_bug46000'); + # Fails with innodb plugin + next if ($test->{'name'} eq 'main.innodb-autoinc'); + # Fails with innodb plugin: r6185 Testcases changes not included + next if ($test->{'name'} eq 'main.innodb_bug44369'); # Copy test options my $new_test= My::Test->new(); while (my ($key, $value) = each(%$test)) === modified file 'mysql-test/mysql-test-run.pl' --- a/mysql-test/mysql-test-run.pl 2010-01-06 21:27:53 +0000 +++ b/mysql-test/mysql-test-run.pl 2010-01-19 10:36:52 +0000 @@ -3425,7 +3425,7 @@ sub restart_forced_by_test # Return timezone value of tinfo or default value sub timezone { my ($tinfo)= @_; - return $tinfo->{timezone} || "GMT-3"; + return $tinfo->{timezone} || "DEFAULT"; } @@ -3455,7 +3455,9 @@ sub run_testcase ($$) { # Init variables that can change between each test case # ------------------------------------------------------- my $timezone= timezone($tinfo); - $ENV{'TZ'}= $timezone; + if ($timezone ne 'DEFAULT') { + $ENV{'TZ'}= $timezone; + } mtr_verbose("Setting timezone: $timezone"); if ( ! using_extern() ) === modified file 'mysql-test/r/archive.result' --- a/mysql-test/r/archive.result 2009-09-10 06:58:13 +0000 +++ b/mysql-test/r/archive.result 2009-11-11 08:03:29 +0000 @@ -12717,3 +12717,14 @@ COUNT(t1.a) 729 DROP TABLE t1; SET @@join_buffer_size= @save_join_buffer_size; +SHOW CREATE TABLE t1; +ERROR HY000: Table upgrade required. Please do "REPAIR TABLE `t1`" or dump/reload to fix it! +SELECT * FROM t1; +ERROR HY000: Table upgrade required. Please do "REPAIR TABLE `t1`" or dump/reload to fix it! +INSERT INTO t1 (col1, col2) VALUES (1, "value"); +ERROR HY000: Table upgrade required. Please do "REPAIR TABLE `t1`" or dump/reload to fix it! +REPAIR TABLE t1; +Table Op Msg_type Msg_text +test.t1 repair Error Table upgrade required. Please do "REPAIR TABLE `t1`" or dump/reload to fix it! +test.t1 repair error Corrupt +DROP TABLE t1; === added file 'mysql-test/r/bug47671.result' --- a/mysql-test/r/bug47671.result 1970-01-01 00:00:00 +0000 +++ b/mysql-test/r/bug47671.result 2010-01-15 15:27:55 +0000 @@ -0,0 +1,14 @@ +# +# Bug#47671 - wrong character-set after upgrade from 5.1.34 to 5.1.39 +# +# Extract only charset information from 'status' command output using regex +-------------- + +Server: MariaDB +Server characterset: utf8 +Db characterset: utf8 +Client characterset: utf8 +Conn. characterset: utf8 + +-------------- + === modified file 'mysql-test/r/delayed.result' --- a/mysql-test/r/delayed.result 2009-03-11 15:32:42 +0000 +++ b/mysql-test/r/delayed.result 2010-01-15 15:27:55 +0000 @@ -314,4 +314,16 @@ a b 2 2 drop table t1; set global low_priority_updates = @old_delayed_updates; +# +# Bug #47682 strange behaviour of INSERT DELAYED +# +DROP TABLE IF EXISTS t1, t2; +CREATE TABLE t1 (f1 integer); +CREATE TABLE t2 (f1 integer); +FLUSH TABLES WITH READ LOCK; +LOCK TABLES t1 READ; +INSERT DELAYED INTO t2 VALUES (1); +Got one of the listed errors +UNLOCK TABLES; +DROP TABLE t1, t2; End of 5.1 tests === modified file 'mysql-test/r/delete.result' --- a/mysql-test/r/delete.result 2009-09-28 10:48:52 +0000 +++ b/mysql-test/r/delete.result 2009-11-18 09:32:03 +0000 @@ -324,3 +324,16 @@ a 1 2 DROP TABLE t1, t2, t3; +# +# Bug #46425 crash in Diagnostics_area::set_ok_status, +# empty statement, DELETE IGNORE +# +CREATE table t1 (i INTEGER); +INSERT INTO t1 VALUES (1); +CREATE TRIGGER tr1 AFTER DELETE ON t1 FOR EACH ROW +BEGIN +INSERT INTO t1 SELECT * FROM t1 AS A; +END | +DELETE IGNORE FROM t1; +ERROR HY000: Can't update table 't1' in stored function/trigger because it is already used by statement which invoked this stored function/trigger. +DROP TABLE t1; === modified file 'mysql-test/r/fulltext.result' --- a/mysql-test/r/fulltext.result 2009-09-07 20:50:10 +0000 +++ b/mysql-test/r/fulltext.result 2010-01-15 15:27:55 +0000 @@ -559,3 +559,42 @@ EXECUTE s; MATCH (col) AGAINST('findme') DEALLOCATE PREPARE s; DROP TABLE t1; +# +# Bug #47930: MATCH IN BOOLEAN MODE returns too many results +# inside subquery +# +CREATE TABLE t1 (a int); +INSERT INTO t1 VALUES (1), (2); +CREATE TABLE t2 (a int, b2 char(10), FULLTEXT KEY b2 (b2)); +INSERT INTO t2 VALUES (1,'Scargill'); +CREATE TABLE t3 (a int, b int); +INSERT INTO t3 VALUES (1,1), (2,1); +# t2 should use full text index +EXPLAIN +SELECT count(*) FROM t1 WHERE +not exists( +SELECT 1 FROM t2, t3 +WHERE t3.a=t1.a AND MATCH(b2) AGAINST('scargill' IN BOOLEAN MODE) +); +id select_type table type possible_keys key key_len ref rows Extra +1 PRIMARY t1 ALL NULL NULL NULL NULL 2 Using where +2 DEPENDENT SUBQUERY t2 fulltext b2 b2 0 1 Using where +2 DEPENDENT SUBQUERY t3 ALL NULL NULL NULL NULL 2 Using where +# should return 0 +SELECT count(*) FROM t1 WHERE +not exists( +SELECT 1 FROM t2, t3 +WHERE t3.a=t1.a AND MATCH(b2) AGAINST('scargill' IN BOOLEAN MODE) +); +count(*) +0 +# should return 0 +SELECT count(*) FROM t1 WHERE +not exists( +SELECT 1 FROM t2 IGNORE INDEX (b2), t3 +WHERE t3.a=t1.a AND MATCH(b2) AGAINST('scargill' IN BOOLEAN MODE) +); +count(*) +0 +DROP TABLE t1,t2,t3; +End of 5.1 tests === modified file 'mysql-test/r/func_group.result' --- a/mysql-test/r/func_group.result 2009-10-14 08:46:50 +0000 +++ b/mysql-test/r/func_group.result 2009-11-24 15:26:13 +0000 @@ -885,7 +885,7 @@ cast(sum(distinct df) as signed) 3 select cast(min(df) as signed) from t1; cast(min(df) as signed) -0 +1 select 1e8 * sum(distinct df) from t1; 1e8 * sum(distinct df) 330000000 @@ -1520,4 +1520,197 @@ max i # Cleanup # DROP TABLE t1; +# +# Bug#43668: Wrong comparison and MIN/MAX for YEAR(2) +# +create table t1 (f1 year(2), f2 year(4), f3 date, f4 datetime); +insert into t1 values +(98,1998,19980101,"1998-01-01 00:00:00"), +(00,2000,20000101,"2000-01-01 00:00:01"), +(02,2002,20020101,"2002-01-01 23:59:59"), +(60,2060,20600101,"2060-01-01 11:11:11"), +(70,1970,19700101,"1970-11-11 22:22:22"), +(NULL,NULL,NULL,NULL); +select min(f1),max(f1) from t1; +min(f1) max(f1) +70 60 +select min(f2),max(f2) from t1; +min(f2) max(f2) +1970 2060 +select min(f3),max(f3) from t1; +min(f3) max(f3) +1970-01-01 2060-01-01 +select min(f4),max(f4) from t1; +min(f4) max(f4) +1970-11-11 22:22:22 2060-01-01 11:11:11 +select a.f1 as a, b.f1 as b, a.f1 > b.f1 as gt, +a.f1 < b.f1 as lt, a.f1<=>b.f1 as eq +from t1 a, t1 b; +a b gt lt eq +98 98 0 0 1 +00 98 1 0 0 +02 98 1 0 0 +60 98 1 0 0 +70 98 0 1 0 +NULL 98 NULL NULL 0 +98 00 0 1 0 +00 00 0 0 1 +02 00 1 0 0 +60 00 1 0 0 +70 00 0 1 0 +NULL 00 NULL NULL 0 +98 02 0 1 0 +00 02 0 1 0 +02 02 0 0 1 +60 02 1 0 0 +70 02 0 1 0 +NULL 02 NULL NULL 0 +98 60 0 1 0 +00 60 0 1 0 +02 60 0 1 0 +60 60 0 0 1 +70 60 0 1 0 +NULL 60 NULL NULL 0 +98 70 1 0 0 +00 70 1 0 0 +02 70 1 0 0 +60 70 1 0 0 +70 70 0 0 1 +NULL 70 NULL NULL 0 +98 NULL NULL NULL 0 +00 NULL NULL NULL 0 +02 NULL NULL NULL 0 +60 NULL NULL NULL 0 +70 NULL NULL NULL 0 +NULL NULL NULL NULL 1 +select a.f1 as a, b.f2 as b, a.f1 > b.f2 as gt, +a.f1 < b.f2 as lt, a.f1<=>b.f2 as eq +from t1 a, t1 b; +a b gt lt eq +98 1998 0 0 1 +00 1998 1 0 0 +02 1998 1 0 0 +60 1998 1 0 0 +70 1998 0 1 0 +NULL 1998 NULL NULL 0 +98 2000 0 1 0 +00 2000 0 0 1 +02 2000 1 0 0 +60 2000 1 0 0 +70 2000 0 1 0 +NULL 2000 NULL NULL 0 +98 2002 0 1 0 +00 2002 0 1 0 +02 2002 0 0 1 +60 2002 1 0 0 +70 2002 0 1 0 +NULL 2002 NULL NULL 0 +98 2060 0 1 0 +00 2060 0 1 0 +02 2060 0 1 0 +60 2060 0 0 1 +70 2060 0 1 0 +NULL 2060 NULL NULL 0 +98 1970 1 0 0 +00 1970 1 0 0 +02 1970 1 0 0 +60 1970 1 0 0 +70 1970 0 0 1 +NULL 1970 NULL NULL 0 +98 NULL NULL NULL 0 +00 NULL NULL NULL 0 +02 NULL NULL NULL 0 +60 NULL NULL NULL 0 +70 NULL NULL NULL 0 +NULL NULL NULL NULL 1 +select a.f1 as a, b.f3 as b, a.f1 > b.f3 as gt, +a.f1 < b.f3 as lt, a.f1<=>b.f3 as eq +from t1 a, t1 b; +a b gt lt eq +98 1998-01-01 0 1 0 +00 1998-01-01 1 0 0 +02 1998-01-01 1 0 0 +60 1998-01-01 1 0 0 +70 1998-01-01 0 1 0 +NULL 1998-01-01 NULL NULL 0 +98 2000-01-01 0 1 0 +00 2000-01-01 0 1 0 +02 2000-01-01 1 0 0 +60 2000-01-01 1 0 0 +70 2000-01-01 0 1 0 +NULL 2000-01-01 NULL NULL 0 +98 2002-01-01 0 1 0 +00 2002-01-01 0 1 0 +02 2002-01-01 0 1 0 +60 2002-01-01 1 0 0 +70 2002-01-01 0 1 0 +NULL 2002-01-01 NULL NULL 0 +98 2060-01-01 0 1 0 +00 2060-01-01 0 1 0 +02 2060-01-01 0 1 0 +60 2060-01-01 0 1 0 +70 2060-01-01 0 1 0 +NULL 2060-01-01 NULL NULL 0 +98 1970-01-01 1 0 0 +00 1970-01-01 1 0 0 +02 1970-01-01 1 0 0 +60 1970-01-01 1 0 0 +70 1970-01-01 0 1 0 +NULL 1970-01-01 NULL NULL 0 +98 NULL NULL NULL 0 +00 NULL NULL NULL 0 +02 NULL NULL NULL 0 +60 NULL NULL NULL 0 +70 NULL NULL NULL 0 +NULL NULL NULL NULL 1 +select a.f1 as a, b.f4 as b, a.f1 > b.f4 as gt, +a.f1 < b.f4 as lt, a.f1<=>b.f4 as eq +from t1 a, t1 b; +a b gt lt eq +98 1998-01-01 00:00:00 0 1 0 +00 1998-01-01 00:00:00 1 0 0 +02 1998-01-01 00:00:00 1 0 0 +60 1998-01-01 00:00:00 1 0 0 +70 1998-01-01 00:00:00 0 1 0 +NULL 1998-01-01 00:00:00 NULL NULL 0 +98 2000-01-01 00:00:01 0 1 0 +00 2000-01-01 00:00:01 0 1 0 +02 2000-01-01 00:00:01 1 0 0 +60 2000-01-01 00:00:01 1 0 0 +70 2000-01-01 00:00:01 0 1 0 +NULL 2000-01-01 00:00:01 NULL NULL 0 +98 2002-01-01 23:59:59 0 1 0 +00 2002-01-01 23:59:59 0 1 0 +02 2002-01-01 23:59:59 0 1 0 +60 2002-01-01 23:59:59 1 0 0 +70 2002-01-01 23:59:59 0 1 0 +NULL 2002-01-01 23:59:59 NULL NULL 0 +98 2060-01-01 11:11:11 0 1 0 +00 2060-01-01 11:11:11 0 1 0 +02 2060-01-01 11:11:11 0 1 0 +60 2060-01-01 11:11:11 0 1 0 +70 2060-01-01 11:11:11 0 1 0 +NULL 2060-01-01 11:11:11 NULL NULL 0 +98 1970-11-11 22:22:22 1 0 0 +00 1970-11-11 22:22:22 1 0 0 +02 1970-11-11 22:22:22 1 0 0 +60 1970-11-11 22:22:22 1 0 0 +70 1970-11-11 22:22:22 0 1 0 +NULL 1970-11-11 22:22:22 NULL NULL 0 +98 NULL NULL NULL 0 +00 NULL NULL NULL 0 +02 NULL NULL NULL 0 +60 NULL NULL NULL 0 +70 NULL NULL NULL 0 +NULL NULL NULL NULL 1 +select *, f1 = f2 from t1; +f1 f2 f3 f4 f1 = f2 +98 1998 1998-01-01 1998-01-01 00:00:00 1 +00 2000 2000-01-01 2000-01-01 00:00:01 1 +02 2002 2002-01-01 2002-01-01 23:59:59 1 +60 2060 2060-01-01 2060-01-01 11:11:11 1 +70 1970 1970-01-01 1970-11-11 22:22:22 1 +NULL NULL NULL NULL NULL +drop table t1; +# End of 5.1 tests === modified file 'mysql-test/r/grant2.result' --- a/mysql-test/r/grant2.result 2009-02-27 08:03:47 +0000 +++ b/mysql-test/r/grant2.result 2009-10-30 05:06:10 +0000 @@ -443,3 +443,30 @@ DROP TABLE db1.t1, db1.t2; DROP USER mysqltest1@localhost; DROP DATABASE db1; End of 5.0 tests +USE mysql; +SELECT LEFT(CURRENT_USER(),INSTR(CURRENT_USER(),'@')-1) INTO @u; +SELECT MID(CURRENT_USER(),INSTR(CURRENT_USER(),'@')+1) INTO @h; +SELECT password FROM user WHERE user=@u AND host=@h INTO @pwd; +SELECT user,host,password,insert_priv FROM user WHERE user=@u AND host=@h; +user host password insert_priv +root localhost Y +UPDATE user SET insert_priv='N' WHERE user=@u AND host=@h; +SELECT user,host,password,insert_priv FROM user WHERE user=@u AND host=@h; +user host password insert_priv +root localhost N +GRANT INSERT ON *.* TO CURRENT_USER(); +SELECT user,host,password,insert_priv FROM user WHERE user=@u AND host=@h; +user host password insert_priv +root localhost Y +UPDATE user SET insert_priv='N' WHERE user=@u AND host=@h; +GRANT INSERT ON *.* TO CURRENT_USER() IDENTIFIED BY 'keksdose'; +SELECT user,host,password,insert_priv FROM user WHERE user=@u AND host=@h; +user host password insert_priv +root localhost *0BB7188CF0DE9B403BA66E9DD810D82652D002EB Y +UPDATE user SET password=@pwd WHERE user=@u AND host=@h; +SELECT user,host,password,insert_priv FROM user WHERE user=@u AND host=@h; +user host password insert_priv +root localhost Y +FLUSH PRIVILEGES; +USE test; +End of 5.1 tests === modified file 'mysql-test/r/group_min_max.result' --- a/mysql-test/r/group_min_max.result 2009-10-09 09:30:40 +0000 +++ b/mysql-test/r/group_min_max.result 2009-11-23 10:04:17 +0000 @@ -2501,6 +2501,17 @@ SELECT a, MAX(b) FROM t WHERE b > 0 AND a MAX(b) 2 1 DROP TABLE t; +# +# Bug #48472: Loose index scan inappropriately chosen for some WHERE +# conditions +# +CREATE TABLE t (a INT, b INT, INDEX (a,b)); +INSERT INTO t VALUES (2,0), (2,0), (2,1), (2,1); +INSERT INTO t SELECT * FROM t; +SELECT a, MAX(b) FROM t WHERE 0=b+0 GROUP BY a; +a MAX(b) +2 0 +DROP TABLE t; End of 5.0 tests # # Bug #46607: Assertion failed: (cond_type == Item::FUNC_ITEM) results in === modified file 'mysql-test/r/information_schema.result' --- a/mysql-test/r/information_schema.result 2009-09-29 20:19:43 +0000 +++ b/mysql-test/r/information_schema.result 2010-01-15 15:58:25 +0000 @@ -85,6 +85,7 @@ TABLE_PRIVILEGES TRIGGERS USER_PRIVILEGES VIEWS +XTRADB_ADMIN_COMMAND XTRADB_ENHANCEMENTS columns_priv db @@ -865,8 +866,8 @@ TABLE_CONSTRAINTS TABLE_NAME select TABLE_PRIVILEGES TABLE_NAME select VIEWS TABLE_NAME select INNODB_BUFFER_POOL_PAGES_INDEX table_name select -INNODB_INDEX_STATS table_name select INNODB_TABLE_STATS table_name select +INNODB_INDEX_STATS table_name select delete from mysql.user where user='mysqltest_4'; delete from mysql.db where user='mysqltest_4'; flush privileges; === modified file 'mysql-test/r/information_schema_all_engines.result' --- a/mysql-test/r/information_schema_all_engines.result 2009-08-03 20:09:53 +0000 +++ b/mysql-test/r/information_schema_all_engines.result 2010-01-15 15:58:25 +0000 @@ -35,7 +35,7 @@ INNODB_CMP INNODB_RSEG XTRADB_ENHANCEMENTS INNODB_BUFFER_POOL_PAGES_INDEX -INNODB_INDEX_STATS +XTRADB_ADMIN_COMMAND INNODB_TRX INNODB_CMP_RESET INNODB_LOCK_WAITS @@ -44,6 +44,7 @@ INNODB_LOCKS INNODB_CMPMEM INNODB_TABLE_STATS INNODB_BUFFER_POOL_PAGES_BLOB +INNODB_INDEX_STATS SELECT t.table_name, c1.column_name FROM information_schema.tables t INNER JOIN @@ -93,7 +94,7 @@ INNODB_CMP page_size INNODB_RSEG rseg_id XTRADB_ENHANCEMENTS name INNODB_BUFFER_POOL_PAGES_INDEX schema_name -INNODB_INDEX_STATS table_name +XTRADB_ADMIN_COMMAND result_message INNODB_TRX trx_id INNODB_CMP_RESET page_size INNODB_LOCK_WAITS requesting_trx_id @@ -102,6 +103,7 @@ INNODB_LOCKS lock_id INNODB_CMPMEM page_size INNODB_TABLE_STATS table_name INNODB_BUFFER_POOL_PAGES_BLOB space_id +INNODB_INDEX_STATS table_name SELECT t.table_name, c1.column_name FROM information_schema.tables t INNER JOIN @@ -151,7 +153,7 @@ INNODB_CMP page_size INNODB_RSEG rseg_id XTRADB_ENHANCEMENTS name INNODB_BUFFER_POOL_PAGES_INDEX schema_name -INNODB_INDEX_STATS table_name +XTRADB_ADMIN_COMMAND result_message INNODB_TRX trx_id INNODB_CMP_RESET page_size INNODB_LOCK_WAITS requesting_trx_id @@ -160,6 +162,7 @@ INNODB_LOCKS lock_id INNODB_CMPMEM page_size INNODB_TABLE_STATS table_name INNODB_BUFFER_POOL_PAGES_BLOB space_id +INNODB_INDEX_STATS table_name select 1 as f1 from information_schema.tables where "CHARACTER_SETS"= (select cast(table_name as char) from information_schema.tables order by table_name limit 1) limit 1; @@ -262,7 +265,7 @@ Database: information_schema | INNODB_RSEG | | XTRADB_ENHANCEMENTS | | INNODB_BUFFER_POOL_PAGES_INDEX | -| INNODB_INDEX_STATS | +| XTRADB_ADMIN_COMMAND | | INNODB_TRX | | INNODB_CMP_RESET | | INNODB_LOCK_WAITS | @@ -271,6 +274,7 @@ Database: information_schema | INNODB_CMPMEM | | INNODB_TABLE_STATS | | INNODB_BUFFER_POOL_PAGES_BLOB | +| INNODB_INDEX_STATS | +---------------------------------------+ Database: INFORMATION_SCHEMA +---------------------------------------+ @@ -310,7 +314,7 @@ Database: INFORMATION_SCHEMA | INNODB_RSEG | | XTRADB_ENHANCEMENTS | | INNODB_BUFFER_POOL_PAGES_INDEX | -| INNODB_INDEX_STATS | +| XTRADB_ADMIN_COMMAND | | INNODB_TRX | | INNODB_CMP_RESET | | INNODB_LOCK_WAITS | @@ -319,6 +323,7 @@ Database: INFORMATION_SCHEMA | INNODB_CMPMEM | | INNODB_TABLE_STATS | | INNODB_BUFFER_POOL_PAGES_BLOB | +| INNODB_INDEX_STATS | +---------------------------------------+ Wildcard: inf_rmation_schema +--------------------+ @@ -328,5 +333,5 @@ Wildcard: inf_rmation_schema +--------------------+ SELECT table_schema, count(*) FROM information_schema.TABLES WHERE table_schema IN ('mysql', 'INFORMATION_SCHEMA', 'test', 'mysqltest') AND table_name<>'ndb_binlog_index' AND table_name<>'ndb_apply_status' GROUP BY TABLE_SCHEMA; table_schema count(*) -information_schema 43 +information_schema 44 mysql 22 === modified file 'mysql-test/r/innodb-autoinc.result' --- a/mysql-test/r/innodb-autoinc.result 2009-12-03 11:34:11 +0000 +++ b/mysql-test/r/innodb-autoinc.result 2010-01-15 17:02:57 +0000 @@ -197,7 +197,7 @@ c1 c2 5 9 DROP TABLE t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=100, @@SESSION.AUTO_INCREMENT_OFFSET=10; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 100 auto_increment_offset 10 @@ -230,7 +230,7 @@ c1 DROP TABLE t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 1 auto_increment_offset 1 @@ -269,7 +269,7 @@ c1 DROP TABLE t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 1 auto_increment_offset 1 @@ -282,7 +282,7 @@ SELECT * FROM t1; c1 -1 SET @@SESSION.AUTO_INCREMENT_INCREMENT=100, @@SESSION.AUTO_INCREMENT_OFFSET=10; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 100 auto_increment_offset 10 @@ -315,7 +315,7 @@ c1 DROP TABLE t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 1 auto_increment_offset 1 @@ -330,7 +330,7 @@ SELECT * FROM t1; c1 1 SET @@SESSION.AUTO_INCREMENT_INCREMENT=100, @@SESSION.AUTO_INCREMENT_OFFSET=10; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 100 auto_increment_offset 10 @@ -370,7 +370,7 @@ c1 DROP TABLE t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 1 auto_increment_offset 1 @@ -385,7 +385,7 @@ SELECT * FROM t1; c1 1 SET @@SESSION.AUTO_INCREMENT_INCREMENT=100, @@SESSION.AUTO_INCREMENT_OFFSET=10; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 100 auto_increment_offset 10 @@ -419,7 +419,7 @@ c1 DROP TABLE t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 1 auto_increment_offset 1 @@ -434,7 +434,7 @@ c1 1 9223372036854775794 SET @@SESSION.AUTO_INCREMENT_INCREMENT=2, @@SESSION.AUTO_INCREMENT_OFFSET=10; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 2 auto_increment_offset 10 @@ -452,7 +452,7 @@ c1 DROP TABLE t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 1 auto_increment_offset 1 @@ -467,7 +467,7 @@ c1 1 18446744073709551603 SET @@SESSION.AUTO_INCREMENT_INCREMENT=2, @@SESSION.AUTO_INCREMENT_OFFSET=10; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 2 auto_increment_offset 10 @@ -485,7 +485,7 @@ c1 DROP TABLE t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 1 auto_increment_offset 1 @@ -500,7 +500,7 @@ c1 1 18446744073709551603 SET @@SESSION.AUTO_INCREMENT_INCREMENT=5, @@SESSION.AUTO_INCREMENT_OFFSET=7; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 5 auto_increment_offset 7 @@ -514,7 +514,7 @@ c1 DROP TABLE t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 1 auto_increment_offset 1 @@ -533,7 +533,7 @@ c1 -9223372036854775806 1 SET @@SESSION.AUTO_INCREMENT_INCREMENT=3, @@SESSION.AUTO_INCREMENT_OFFSET=3; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 3 auto_increment_offset 3 @@ -550,7 +550,7 @@ c1 DROP TABLE t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 1 auto_increment_offset 1 @@ -568,7 +568,7 @@ SET @@SESSION.AUTO_INCREMENT_INCREMENT=1 Warnings: Warning 1292 Truncated incorrect auto_increment_increment value: '1152921504606846976' Warning 1292 Truncated incorrect auto_increment_offset value: '1152921504606846976' -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 65535 auto_increment_offset 65535 @@ -581,7 +581,7 @@ c1 DROP TABLE t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 1 auto_increment_offset 1 @@ -875,11 +875,11 @@ ALTER TABLE t1 CHANGE c1 d1 INT NOT NULL SELECT * FROM t1; d1 1 -3 +2 SELECT * FROM t1; d1 1 -3 +2 INSERT INTO t1 VALUES(null); Got one of the listed errors ALTER TABLE t1 AUTO_INCREMENT = 3; @@ -888,16 +888,16 @@ Table Create Table t1 CREATE TABLE `t1` ( `d1` int(11) NOT NULL AUTO_INCREMENT, PRIMARY KEY (`d1`) -) ENGINE=InnoDB AUTO_INCREMENT=4 DEFAULT CHARSET=latin1 +) ENGINE=InnoDB AUTO_INCREMENT=3 DEFAULT CHARSET=latin1 INSERT INTO t1 VALUES(null); SELECT * FROM t1; d1 1 +2 3 -4 DROP TABLE t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; Variable_name Value auto_increment_increment 1 auto_increment_offset 1 @@ -1126,3 +1126,61 @@ SELECT * FROM T1; c1 c2 10 0 DROP TABLE T1; +CREATE TABLE T1(C1 DOUBLE AUTO_INCREMENT KEY, C2 CHAR(10)) ENGINE=InnoDB; +INSERT INTO T1(C1, C2) VALUES (1, 'innodb'), (3, 'innodb'); +INSERT INTO T1(C2) VALUES ('innodb'); +SHOW CREATE TABLE T1; +Table Create Table +T1 CREATE TABLE `T1` ( + `C1` double NOT NULL AUTO_INCREMENT, + `C2` char(10) DEFAULT NULL, + PRIMARY KEY (`C1`) +) ENGINE=InnoDB AUTO_INCREMENT=5 DEFAULT CHARSET=latin1 +DROP TABLE T1; +CREATE TABLE T1(C1 FLOAT AUTO_INCREMENT KEY, C2 CHAR(10)) ENGINE=InnoDB; +INSERT INTO T1(C1, C2) VALUES (1, 'innodb'), (3, 'innodb'); +INSERT INTO T1(C2) VALUES ('innodb'); +SHOW CREATE TABLE T1; +Table Create Table +T1 CREATE TABLE `T1` ( + `C1` float NOT NULL AUTO_INCREMENT, + `C2` char(10) DEFAULT NULL, + PRIMARY KEY (`C1`) +) ENGINE=InnoDB AUTO_INCREMENT=5 DEFAULT CHARSET=latin1 +DROP TABLE T1; +CREATE TABLE t1 (c1 INT AUTO_INCREMENT PRIMARY KEY) ENGINE=InnoDB; +INSERT INTO t1 SET c1 = 1; +SHOW CREATE TABLE t1; +Table Create Table +t1 CREATE TABLE `t1` ( + `c1` int(11) NOT NULL AUTO_INCREMENT, + PRIMARY KEY (`c1`) +) ENGINE=InnoDB AUTO_INCREMENT=2 DEFAULT CHARSET=latin1 +INSERT INTO t1 SET c1 = 2; +INSERT INTO t1 SET c1 = -1; +SELECT * FROM t1; +c1 +-1 +1 +2 +INSERT INTO t1 SET c1 = -1; +Got one of the listed errors +SHOW CREATE TABLE t1; +Table Create Table +t1 CREATE TABLE `t1` ( + `c1` int(11) NOT NULL AUTO_INCREMENT, + PRIMARY KEY (`c1`) +) ENGINE=InnoDB AUTO_INCREMENT=3 DEFAULT CHARSET=latin1 +REPLACE INTO t1 VALUES (-1); +SELECT * FROM t1; +c1 +-1 +1 +2 +SHOW CREATE TABLE t1; +Table Create Table +t1 CREATE TABLE `t1` ( + `c1` int(11) NOT NULL AUTO_INCREMENT, + PRIMARY KEY (`c1`) +) ENGINE=InnoDB AUTO_INCREMENT=3 DEFAULT CHARSET=latin1 +DROP TABLE t1; === added file 'mysql-test/r/innodb-consistent.result' --- a/mysql-test/r/innodb-consistent.result 1970-01-01 00:00:00 +0000 +++ b/mysql-test/r/innodb-consistent.result 2010-01-15 15:58:25 +0000 @@ -0,0 +1,35 @@ +drop table if exists t1; +set session transaction isolation level read committed; +create table t1(a int not null) engine=innodb DEFAULT CHARSET=latin1; +create table t2 like t1; +insert into t2 values (1),(2),(3),(4),(5),(6),(7); +set autocommit=0; +begin; +replace into t1 select * from t2; +set session transaction isolation level read committed; +set autocommit=0; +delete from t2 where a=5; +commit; +delete from t2; +commit; +commit; +begin; +insert into t1 select * from t2; +set session transaction isolation level read committed; +set autocommit=0; +delete from t2 where a=5; +commit; +delete from t2; +commit; +commit; +select * from t1; +a +1 +2 +3 +4 +5 +6 +7 +drop table t1; +drop table t2; === modified file 'mysql-test/r/innodb-index.result' --- a/mysql-test/r/innodb-index.result 2009-11-30 21:37:27 +0000 +++ b/mysql-test/r/innodb-index.result 2010-01-15 15:58:25 +0000 @@ -1,4 +1,3 @@ -SET @save_innodb_file_format_check=@@global.innodb_file_format_check; create table t1(a int not null, b int, c char(10) not null, d varchar(20)) engine = innodb; insert into t1 values (5,5,'oo','oo'),(4,4,'tr','tr'),(3,4,'ad','ad'),(2,3,'ak','ak'); commit; @@ -629,7 +628,7 @@ drop table t1; create table t1(a int not null, b int, c char(10), d varchar(20), primary key (a)) engine = innodb; insert into t1 values (1,1,'ab','ab'),(2,2,'ac','ac'),(3,3,'ac','ac'),(4,4,'afe','afe'),(5,4,'affe','affe'); alter table t1 add unique index (b), add unique index (c), add unique index (d); -ERROR 23000: Duplicate entry '4' for key 'b' +ERROR 23000: Duplicate entry 'ac' for key 'c' alter table t1 add unique index (c), add unique index (b), add index (d); ERROR 23000: Duplicate entry 'ac' for key 'c' show create table t1; @@ -970,6 +969,7 @@ create index t1u on t1 (u(1)); drop table t1; set global innodb_file_per_table=0; set global innodb_file_format=Antelope; +set global innodb_file_format_check=Antelope; SET @OLD_UNIQUE_CHECKS=@@UNIQUE_CHECKS, UNIQUE_CHECKS=0; SET @OLD_FOREIGN_KEY_CHECKS=@@FOREIGN_KEY_CHECKS, FOREIGN_KEY_CHECKS=0; CREATE TABLE t1( @@ -1171,4 +1171,3 @@ a b 3 a 3 b DROP TABLE t1; -SET GLOBAL innodb_file_format_check=@save_innodb_file_format_check; === modified file 'mysql-test/r/innodb-timeout.result' --- a/mysql-test/r/innodb-timeout.result 2009-06-09 13:19:13 +0000 +++ b/mysql-test/r/innodb-timeout.result 2010-01-18 12:56:10 +0000 @@ -23,6 +23,7 @@ select @@innodb_lock_wait_timeout; create table t1(a int primary key)engine=innodb; begin; insert into t1 values(1),(2),(3); +set innodb_lock_wait_timeout=5; select * from t1 for update; commit; a @@ -31,8 +32,15 @@ a 3 begin; insert into t1 values(4); +set innodb_lock_wait_timeout=2; +set @a= current_timestamp(); select * from t1 for update; -commit; ERROR HY000: Lock wait timeout exceeded; try restarting transaction +set @b= current_timestamp(); +set @c= timestampdiff(SECOND, @a, @b); +select if(@c >= 1 and @c <= 10, 'OK', concat("NOT OK, time passed=", @c)); +if(@c >= 1 and @c <= 10, 'OK', concat("NOT OK, time passed=", @c)) +OK +commit; drop table t1; set global innodb_lock_wait_timeout=50; === modified file 'mysql-test/r/innodb-zip.result' --- a/mysql-test/r/innodb-zip.result 2009-06-09 15:08:46 +0000 +++ b/mysql-test/r/innodb-zip.result 2010-01-15 15:58:25 +0000 @@ -196,15 +196,15 @@ drop table t1; set innodb_strict_mode = on; create table t1 (id int primary key) engine = innodb key_block_size = 0; ERROR HY000: Can't create table 'test.t1' (errno: 1478) -show errors; +show warnings; Level Code Message -Error 1478 InnoDB: invalid KEY_BLOCK_SIZE = 0. Valid values are [1, 2, 4, 8, 16] +Warning 1478 InnoDB: invalid KEY_BLOCK_SIZE = 0. Valid values are [1, 2, 4, 8, 16] Error 1005 Can't create table 'test.t1' (errno: 1478) create table t2 (id int primary key) engine = innodb key_block_size = 9; ERROR HY000: Can't create table 'test.t2' (errno: 1478) -show errors; +show warnings; Level Code Message -Error 1478 InnoDB: invalid KEY_BLOCK_SIZE = 9. Valid values are [1, 2, 4, 8, 16] +Warning 1478 InnoDB: invalid KEY_BLOCK_SIZE = 9. Valid values are [1, 2, 4, 8, 16] Error 1005 Can't create table 'test.t2' (errno: 1478) create table t3 (id int primary key) engine = innodb key_block_size = 1; create table t4 (id int primary key) engine = innodb key_block_size = 2; @@ -233,30 +233,30 @@ key_block_size = 8 row_format = compress create table t2 (id int primary key) engine = innodb key_block_size = 8 row_format = redundant; ERROR HY000: Can't create table 'test.t2' (errno: 1478) -show errors; +show warnings; Level Code Message -Error 1478 InnoDB: cannot specify ROW_FORMAT = REDUNDANT with KEY_BLOCK_SIZE. +Warning 1478 InnoDB: cannot specify ROW_FORMAT = REDUNDANT with KEY_BLOCK_SIZE. Error 1005 Can't create table 'test.t2' (errno: 1478) create table t3 (id int primary key) engine = innodb key_block_size = 8 row_format = compact; ERROR HY000: Can't create table 'test.t3' (errno: 1478) -show errors; +show warnings; Level Code Message -Error 1478 InnoDB: cannot specify ROW_FORMAT = COMPACT with KEY_BLOCK_SIZE. +Warning 1478 InnoDB: cannot specify ROW_FORMAT = COMPACT with KEY_BLOCK_SIZE. Error 1005 Can't create table 'test.t3' (errno: 1478) create table t4 (id int primary key) engine = innodb key_block_size = 8 row_format = dynamic; ERROR HY000: Can't create table 'test.t4' (errno: 1478) -show errors; +show warnings; Level Code Message -Error 1478 InnoDB: cannot specify ROW_FORMAT = DYNAMIC with KEY_BLOCK_SIZE. +Warning 1478 InnoDB: cannot specify ROW_FORMAT = DYNAMIC with KEY_BLOCK_SIZE. Error 1005 Can't create table 'test.t4' (errno: 1478) create table t5 (id int primary key) engine = innodb key_block_size = 8 row_format = default; ERROR HY000: Can't create table 'test.t5' (errno: 1478) -show errors; +show warnings; Level Code Message -Error 1478 InnoDB: cannot specify ROW_FORMAT = COMPACT with KEY_BLOCK_SIZE. +Warning 1478 InnoDB: cannot specify ROW_FORMAT = COMPACT with KEY_BLOCK_SIZE. Error 1005 Can't create table 'test.t5' (errno: 1478) SELECT table_schema, table_name, row_format FROM information_schema.tables WHERE engine='innodb'; @@ -266,26 +266,26 @@ drop table t1; create table t1 (id int primary key) engine = innodb key_block_size = 9 row_format = redundant; ERROR HY000: Can't create table 'test.t1' (errno: 1478) -show errors; +show warnings; Level Code Message -Error 1478 InnoDB: invalid KEY_BLOCK_SIZE = 9. Valid values are [1, 2, 4, 8, 16] -Error 1478 InnoDB: cannot specify ROW_FORMAT = REDUNDANT with KEY_BLOCK_SIZE. +Warning 1478 InnoDB: invalid KEY_BLOCK_SIZE = 9. Valid values are [1, 2, 4, 8, 16] +Warning 1478 InnoDB: cannot specify ROW_FORMAT = REDUNDANT with KEY_BLOCK_SIZE. Error 1005 Can't create table 'test.t1' (errno: 1478) create table t2 (id int primary key) engine = innodb key_block_size = 9 row_format = compact; ERROR HY000: Can't create table 'test.t2' (errno: 1478) -show errors; +show warnings; Level Code Message -Error 1478 InnoDB: invalid KEY_BLOCK_SIZE = 9. Valid values are [1, 2, 4, 8, 16] -Error 1478 InnoDB: cannot specify ROW_FORMAT = COMPACT with KEY_BLOCK_SIZE. +Warning 1478 InnoDB: invalid KEY_BLOCK_SIZE = 9. Valid values are [1, 2, 4, 8, 16] +Warning 1478 InnoDB: cannot specify ROW_FORMAT = COMPACT with KEY_BLOCK_SIZE. Error 1005 Can't create table 'test.t2' (errno: 1478) create table t2 (id int primary key) engine = innodb key_block_size = 9 row_format = dynamic; ERROR HY000: Can't create table 'test.t2' (errno: 1478) -show errors; +show warnings; Level Code Message -Error 1478 InnoDB: invalid KEY_BLOCK_SIZE = 9. Valid values are [1, 2, 4, 8, 16] -Error 1478 InnoDB: cannot specify ROW_FORMAT = DYNAMIC with KEY_BLOCK_SIZE. +Warning 1478 InnoDB: invalid KEY_BLOCK_SIZE = 9. Valid values are [1, 2, 4, 8, 16] +Warning 1478 InnoDB: cannot specify ROW_FORMAT = DYNAMIC with KEY_BLOCK_SIZE. Error 1005 Can't create table 'test.t2' (errno: 1478) SELECT table_schema, table_name, row_format FROM information_schema.tables WHERE engine='innodb'; @@ -293,45 +293,45 @@ table_schema table_name row_format set global innodb_file_per_table = off; create table t1 (id int primary key) engine = innodb key_block_size = 1; ERROR HY000: Can't create table 'test.t1' (errno: 1478) -show errors; +show warnings; Level Code Message -Error 1478 InnoDB: KEY_BLOCK_SIZE requires innodb_file_per_table. +Warning 1478 InnoDB: KEY_BLOCK_SIZE requires innodb_file_per_table. Error 1005 Can't create table 'test.t1' (errno: 1478) create table t2 (id int primary key) engine = innodb key_block_size = 2; ERROR HY000: Can't create table 'test.t2' (errno: 1478) -show errors; +show warnings; Level Code Message -Error 1478 InnoDB: KEY_BLOCK_SIZE requires innodb_file_per_table. +Warning 1478 InnoDB: KEY_BLOCK_SIZE requires innodb_file_per_table. Error 1005 Can't create table 'test.t2' (errno: 1478) create table t3 (id int primary key) engine = innodb key_block_size = 4; ERROR HY000: Can't create table 'test.t3' (errno: 1478) -show errors; +show warnings; Level Code Message -Error 1478 InnoDB: KEY_BLOCK_SIZE requires innodb_file_per_table. +Warning 1478 InnoDB: KEY_BLOCK_SIZE requires innodb_file_per_table. Error 1005 Can't create table 'test.t3' (errno: 1478) create table t4 (id int primary key) engine = innodb key_block_size = 8; ERROR HY000: Can't create table 'test.t4' (errno: 1478) -show errors; +show warnings; Level Code Message -Error 1478 InnoDB: KEY_BLOCK_SIZE requires innodb_file_per_table. +Warning 1478 InnoDB: KEY_BLOCK_SIZE requires innodb_file_per_table. Error 1005 Can't create table 'test.t4' (errno: 1478) create table t5 (id int primary key) engine = innodb key_block_size = 16; ERROR HY000: Can't create table 'test.t5' (errno: 1478) -show errors; +show warnings; Level Code Message -Error 1478 InnoDB: KEY_BLOCK_SIZE requires innodb_file_per_table. +Warning 1478 InnoDB: KEY_BLOCK_SIZE requires innodb_file_per_table. Error 1005 Can't create table 'test.t5' (errno: 1478) create table t6 (id int primary key) engine = innodb row_format = compressed; ERROR HY000: Can't create table 'test.t6' (errno: 1478) -show errors; +show warnings; Level Code Message -Error 1478 InnoDB: ROW_FORMAT=COMPRESSED requires innodb_file_per_table. +Warning 1478 InnoDB: ROW_FORMAT=COMPRESSED requires innodb_file_per_table. Error 1005 Can't create table 'test.t6' (errno: 1478) create table t7 (id int primary key) engine = innodb row_format = dynamic; ERROR HY000: Can't create table 'test.t7' (errno: 1478) -show errors; +show warnings; Level Code Message -Error 1478 InnoDB: ROW_FORMAT=DYNAMIC requires innodb_file_per_table. +Warning 1478 InnoDB: ROW_FORMAT=DYNAMIC requires innodb_file_per_table. Error 1005 Can't create table 'test.t7' (errno: 1478) create table t8 (id int primary key) engine = innodb row_format = compact; create table t9 (id int primary key) engine = innodb row_format = redundant; @@ -345,45 +345,45 @@ set global innodb_file_per_table = on; set global innodb_file_format = `0`; create table t1 (id int primary key) engine = innodb key_block_size = 1; ERROR HY000: Can't create table 'test.t1' (errno: 1478) -show errors; +show warnings; Level Code Message -Error 1478 InnoDB: KEY_BLOCK_SIZE requires innodb_file_format > Antelope. +Warning 1478 InnoDB: KEY_BLOCK_SIZE requires innodb_file_format > Antelope. Error 1005 Can't create table 'test.t1' (errno: 1478) create table t2 (id int primary key) engine = innodb key_block_size = 2; ERROR HY000: Can't create table 'test.t2' (errno: 1478) -show errors; +show warnings; Level Code Message -Error 1478 InnoDB: KEY_BLOCK_SIZE requires innodb_file_format > Antelope. +Warning 1478 InnoDB: KEY_BLOCK_SIZE requires innodb_file_format > Antelope. Error 1005 Can't create table 'test.t2' (errno: 1478) create table t3 (id int primary key) engine = innodb key_block_size = 4; ERROR HY000: Can't create table 'test.t3' (errno: 1478) -show errors; +show warnings; Level Code Message -Error 1478 InnoDB: KEY_BLOCK_SIZE requires innodb_file_format > Antelope. +Warning 1478 InnoDB: KEY_BLOCK_SIZE requires innodb_file_format > Antelope. Error 1005 Can't create table 'test.t3' (errno: 1478) create table t4 (id int primary key) engine = innodb key_block_size = 8; ERROR HY000: Can't create table 'test.t4' (errno: 1478) -show errors; +show warnings; Level Code Message -Error 1478 InnoDB: KEY_BLOCK_SIZE requires innodb_file_format > Antelope. +Warning 1478 InnoDB: KEY_BLOCK_SIZE requires innodb_file_format > Antelope. Error 1005 Can't create table 'test.t4' (errno: 1478) create table t5 (id int primary key) engine = innodb key_block_size = 16; ERROR HY000: Can't create table 'test.t5' (errno: 1478) -show errors; +show warnings; Level Code Message -Error 1478 InnoDB: KEY_BLOCK_SIZE requires innodb_file_format > Antelope. +Warning 1478 InnoDB: KEY_BLOCK_SIZE requires innodb_file_format > Antelope. Error 1005 Can't create table 'test.t5' (errno: 1478) create table t6 (id int primary key) engine = innodb row_format = compressed; ERROR HY000: Can't create table 'test.t6' (errno: 1478) -show errors; +show warnings; Level Code Message -Error 1478 InnoDB: ROW_FORMAT=COMPRESSED requires innodb_file_format > Antelope. +Warning 1478 InnoDB: ROW_FORMAT=COMPRESSED requires innodb_file_format > Antelope. Error 1005 Can't create table 'test.t6' (errno: 1478) create table t7 (id int primary key) engine = innodb row_format = dynamic; ERROR HY000: Can't create table 'test.t7' (errno: 1478) -show errors; +show warnings; Level Code Message -Error 1478 InnoDB: ROW_FORMAT=DYNAMIC requires innodb_file_format > Antelope. +Warning 1478 InnoDB: ROW_FORMAT=DYNAMIC requires innodb_file_format > Antelope. Error 1005 Can't create table 'test.t7' (errno: 1478) create table t8 (id int primary key) engine = innodb row_format = compact; create table t9 (id int primary key) engine = innodb row_format = redundant; === modified file 'mysql-test/r/innodb.result' --- a/mysql-test/r/innodb.result 2009-12-27 13:54:41 +0000 +++ b/mysql-test/r/innodb.result 2010-01-15 15:58:25 +0000 @@ -3090,7 +3090,7 @@ ERROR HY000: Lock wait timeout exceeded; commit; drop table t1, t2, t3, t5, t6, t8, t9; CREATE TABLE t1 (DB_ROW_ID int) engine=innodb; -ERROR HY000: Can't create table 'test.t1' (errno: -1) +ERROR 42000: Incorrect column name 'DB_ROW_ID' CREATE TABLE t1 ( a BIGINT(20) NOT NULL, PRIMARY KEY (a) === modified file 'mysql-test/r/innodb_bug36169.result' --- a/mysql-test/r/innodb_bug36169.result 2009-11-13 21:26:08 +0000 +++ b/mysql-test/r/innodb_bug36169.result 2010-01-15 15:58:25 +0000 @@ -1,4 +1,2 @@ -set @old_innodb_file_format=@@innodb_file_format; -set @old_innodb_file_per_table=@@innodb_file_per_table; SET GLOBAL innodb_file_format='Barracuda'; SET GLOBAL innodb_file_per_table=ON; === modified file 'mysql-test/r/innodb_bug44369.result' --- a/mysql-test/r/innodb_bug44369.result 2009-11-02 14:59:44 +0000 +++ b/mysql-test/r/innodb_bug44369.result 2010-01-15 15:58:25 +0000 @@ -1,14 +1,6 @@ create table bug44369 (DB_ROW_ID int) engine=innodb; -ERROR HY000: Can't create table 'test.bug44369' (errno: -1) +ERROR 42000: Incorrect column name 'DB_ROW_ID' create table bug44369 (db_row_id int) engine=innodb; -ERROR HY000: Can't create table 'test.bug44369' (errno: -1) -show warnings; -Level Code Message -Warning 1005 Error creating table 'test/bug44369' with column name 'db_row_id'. 'db_row_id' is a reserved name. Please try to re-create the table with a different column name. -Error 1005 Can't create table 'test.bug44369' (errno: -1) +ERROR 42000: Incorrect column name 'db_row_id' create table bug44369 (db_TRX_Id int) engine=innodb; -ERROR HY000: Can't create table 'test.bug44369' (errno: -1) -show warnings; -Level Code Message -Warning 1005 Error creating table 'test/bug44369' with column name 'db_TRX_Id'. 'db_TRX_Id' is a reserved name. Please try to re-create the table with a different column name. -Error 1005 Can't create table 'test.bug44369' (errno: -1) +ERROR 42000: Incorrect column name 'db_TRX_Id' === added file 'mysql-test/r/innodb_bug44571.result' --- a/mysql-test/r/innodb_bug44571.result 1970-01-01 00:00:00 +0000 +++ b/mysql-test/r/innodb_bug44571.result 2010-01-15 15:58:25 +0000 @@ -0,0 +1,7 @@ +CREATE TABLE bug44571 (foo INT) ENGINE=InnoDB; +ALTER TABLE bug44571 CHANGE foo bar INT; +ALTER TABLE bug44571 ADD INDEX bug44571b (foo); +ERROR 42000: Key column 'foo' doesn't exist in table +ALTER TABLE bug44571 ADD INDEX bug44571b (bar); +CREATE INDEX bug44571c ON bug44571 (bar); +DROP TABLE bug44571; === added file 'mysql-test/r/innodb_bug46676.result' --- a/mysql-test/r/innodb_bug46676.result 1970-01-01 00:00:00 +0000 +++ b/mysql-test/r/innodb_bug46676.result 2010-01-15 15:58:25 +0000 @@ -0,0 +1,9 @@ +SET foreign_key_checks=0; +CREATE TABLE t1 (id int, foreign key (id) references t2(id)) ENGINE=INNODB; +CREATE TABLE t2 (id int, foreign key (id) references t1(id)) ENGINE=INNODB; +SET foreign_key_checks=1; +SELECT COUNT(*) FROM information_schema.key_column_usage WHERE REFERENCED_TABLE_NAME in ('t1', 't2'); +COUNT(*) +2 +SET foreign_key_checks=0; +DROP TABLE t1, t2; === added file 'mysql-test/r/innodb_bug47167.result' --- a/mysql-test/r/innodb_bug47167.result 1970-01-01 00:00:00 +0000 +++ b/mysql-test/r/innodb_bug47167.result 2010-01-15 15:58:25 +0000 @@ -0,0 +1,24 @@ +set @old_innodb_file_format_check=@@innodb_file_format_check; +select @old_innodb_file_format_check; +@old_innodb_file_format_check +Antelope +set global innodb_file_format_check = Barracuda; +select @@innodb_file_format_check; +@@innodb_file_format_check +Barracuda +set global innodb_file_format_check = DEFAULT; +select @@innodb_file_format_check; +@@innodb_file_format_check +Barracuda +set global innodb_file_format_check = @old_innodb_file_format_check; +select @@innodb_file_format_check; +@@innodb_file_format_check +Antelope +set global innodb_file_format_check = cheetah; +ERROR HY000: Incorrect arguments to SET +set global innodb_file_format_check = Bear; +ERROR HY000: Incorrect arguments to SET +set global innodb_file_format_check = on; +ERROR HY000: Incorrect arguments to SET +set global innodb_file_format_check = off; +ERROR HY000: Incorrect arguments to SET === modified file 'mysql-test/r/innodb_file_format.result' --- a/mysql-test/r/innodb_file_format.result 2009-11-30 21:37:27 +0000 +++ b/mysql-test/r/innodb_file_format.result 2010-01-15 15:58:25 +0000 @@ -1,4 +1,3 @@ -set @old_innodb_file_format=@@innodb_file_format; call mtr.add_suppression("InnoDB: invalid innodb_file_format_check value"); select @@innodb_file_format; @@innodb_file_format @@ -32,8 +31,6 @@ select @@innodb_file_format_check; @@innodb_file_format_check Barracuda set global innodb_file_format_check=default; -Warnings: -Warning 1210 Ignoring SET innodb_file_format=on select @@innodb_file_format_check; @@innodb_file_format_check Barracuda @@ -44,5 +41,4 @@ ERROR HY000: Incorrect arguments to SET select @@innodb_file_format_check; @@innodb_file_format_check Barracuda -set global innodb_file_format=@old_innodb_file_format; -set global innodb_file_format_check=Antelope; +set global innodb_file_format_check=antelope; === modified file 'mysql-test/r/innodb_lock_wait_timeout_1.result' --- a/mysql-test/r/innodb_lock_wait_timeout_1.result 2009-11-03 17:45:52 +0000 +++ b/mysql-test/r/innodb_lock_wait_timeout_1.result 2009-11-12 11:43:33 +0000 @@ -48,6 +48,24 @@ commit; set autocommit=default; drop table t1; # +# Bug #37183 insert ignore into .. select ... hangs +# after deadlock was encountered +# +create table t1(id int primary key,v int)engine=innodb; +insert into t1 values (1,1),(2,2),(3,3),(4,4),(5,5),(6,6),(7,7); +create table t2 like t1; +begin; +update t1 set v=id*2 where id=1; +begin; +update t1 set v=id*2 where id=2; +update t1 set v=id*2 where id=2; +ERROR HY000: Lock wait timeout exceeded; try restarting transaction +insert ignore into t2 select * from t1 where id=1; +ERROR HY000: Lock wait timeout exceeded; try restarting transaction +rollback; +rollback; +drop table t1, t2; +# # Bug#41756 Strange error messages about locks from InnoDB # drop table if exists t1; === modified file 'mysql-test/r/innodb_mysql.result' --- a/mysql-test/r/innodb_mysql.result 2009-12-03 11:19:05 +0000 +++ b/mysql-test/r/innodb_mysql.result 2010-01-15 15:27:55 +0000 @@ -2251,4 +2251,26 @@ c >= '2009-10-09 00:00:00.001' AND c <= id select_type table type possible_keys key key_len ref rows Extra 1 SIMPLE NULL NULL NULL NULL NULL NULL NULL Impossible WHERE noticed after reading const tables DROP TABLE t1; +# +# Bug #46175: NULL read_view and consistent read assertion +# +CREATE TABLE t1(a CHAR(13),KEY(a)) ENGINE=innodb; +CREATE TABLE t2(b DATETIME,KEY(b)) ENGINE=innodb; +INSERT INTO t1 VALUES (),(); +INSERT INTO t2 VALUES (),(); +CREATE OR REPLACE VIEW v1 AS SELECT 1 FROM t2 +WHERE b =(SELECT a FROM t1 LIMIT 1); +CREATE PROCEDURE p1(num INT) +BEGIN +DECLARE i INT DEFAULT 0; +REPEAT +SHOW CREATE VIEW v1; +SET i:=i+1; +UNTIL i>num END REPEAT; +END| +# Should not crash +# Should not crash +DROP PROCEDURE p1; +DROP VIEW v1; +DROP TABLE t1,t2; End of 5.1 tests === modified file 'mysql-test/r/innodb_xtradb_bug317074.result' --- a/mysql-test/r/innodb_xtradb_bug317074.result 2009-10-28 07:52:34 +0000 +++ b/mysql-test/r/innodb_xtradb_bug317074.result 2010-01-15 15:58:25 +0000 @@ -1,6 +1,5 @@ SET @old_innodb_file_format=@@innodb_file_format; SET @old_innodb_file_per_table=@@innodb_file_per_table; -SET @old_innodb_file_format_check=@@innodb_file_format_check; SET GLOBAL innodb_file_format='Barracuda'; SET GLOBAL innodb_file_per_table=ON; DROP TABLE IF EXISTS `test1`; @@ -29,4 +28,4 @@ ALTER TABLE test1 ENGINE=MyISAM; DROP TABLE test1; SET GLOBAL innodb_file_format=@old_innodb_file_format; SET GLOBAL innodb_file_per_table=@old_innodb_file_per_table; -SET GLOBAL innodb_file_format_check=@old_innodb_file_format_check; +set global innodb_file_format_check=Antelope; === modified file 'mysql-test/r/mysql.result' --- a/mysql-test/r/mysql.result 2009-07-31 23:43:46 +0000 +++ b/mysql-test/r/mysql.result 2009-11-27 14:41:45 +0000 @@ -229,5 +229,4 @@ a: b </row> </resultset> drop table t1; - -End of tests +End of 5.0 tests === modified file 'mysql-test/r/olap.result' --- a/mysql-test/r/olap.result 2009-10-30 15:59:06 +0000 +++ b/mysql-test/r/olap.result 2009-12-08 09:26:11 +0000 @@ -753,4 +753,16 @@ b 100 NULL DROP TABLE t1, t2; +# +# Bug #48475: DISTINCT is ignored with GROUP BY WITH ROLLUP +# and only const tables +CREATE TABLE t1 (a INT); +CREATE TABLE t2 (b INT); +INSERT INTO t1 VALUES (1); +INSERT INTO t2 VALUES (1); +SELECT DISTINCT b FROM t1, t2 GROUP BY a, b WITH ROLLUP; +b +1 +NULL +DROP TABLE t1, t2; End of 5.0 tests === modified file 'mysql-test/r/order_by.result' --- a/mysql-test/r/order_by.result 2009-10-14 14:30:39 +0000 +++ b/mysql-test/r/order_by.result 2009-11-10 08:58:43 +0000 @@ -1444,6 +1444,27 @@ FROM t3; 2 NULL DROP TABLE t1, t2, t3; +# +# Bug #42760: Select doesn't return desired results when we have null +# values +# +CREATE TABLE t1 ( +a INT, +c INT, +UNIQUE KEY a_c (a,c), +KEY (a)); +INSERT INTO t1 VALUES (1, 10), (2, NULL); +# Must use ref-or-null on the a_c index +EXPLAIN +SELECT 1 AS col FROM t1 WHERE a=2 AND (c=10 OR c IS NULL) ORDER BY c; +id select_type table type possible_keys key key_len ref rows Extra +1 SIMPLE t1 ref_or_null a_c,a a_c 10 const,const 1 Using where +# Must return 1 row +SELECT 1 AS col FROM t1 WHERE a=2 AND (c=10 OR c IS NULL) ORDER BY c; +col +1 +DROP TABLE t1; +End of 5.0 tests CREATE TABLE t2 (a varchar(32), b int(11), c float, d double, UNIQUE KEY a (a,b,c), KEY b (b), KEY c (c)); CREATE TABLE t1 (a varchar(32), b char(3), UNIQUE KEY a (a,b), KEY b (b)); === modified file 'mysql-test/r/partition.result' --- a/mysql-test/r/partition.result 2009-12-03 11:19:05 +0000 +++ b/mysql-test/r/partition.result 2010-01-15 15:27:55 +0000 @@ -1,4 +1,10 @@ drop table if exists t1, t2; +CREATE TABLE t1 (a INT, b INT) +PARTITION BY LIST (a) +SUBPARTITION BY HASH (b) +(PARTITION p1 VALUES IN (1)); +ALTER TABLE t1 ADD COLUMN c INT; +DROP TABLE t1; CREATE TABLE t1 ( a int NOT NULL, b int NOT NULL); @@ -50,6 +56,13 @@ t1 CREATE TABLE `t1` ( PARTITION p3 VALUES LESS THAN (733969) ENGINE = MyISAM, PARTITION pmax VALUES LESS THAN MAXVALUE ENGINE = MyISAM) */ DROP TABLE t1; +create table t1 (a int NOT NULL, b varchar(5) NOT NULL) +default charset=utf8 +partition by list (a) +subpartition by key (b) +(partition p0 values in (1), +partition p1 values in (2)); +drop table t1; create table t1 (a int, b int, key(a)) partition by list (a) ( partition p0 values in (1), @@ -2045,10 +2058,15 @@ DROP TABLE t1; # # Bug #45807: crash accessing partitioned table and sql_mode # contains ONLY_FULL_GROUP_BY +# Bug#46923: select count(*) from partitioned table fails with +# ONLY_FULL_GROUP_BY # SET SESSION SQL_MODE='ONLY_FULL_GROUP_BY'; CREATE TABLE t1(id INT,KEY(id)) ENGINE=MYISAM PARTITION BY HASH(id) PARTITIONS 2; +SELECT COUNT(*) FROM t1; +COUNT(*) +0 DROP TABLE t1; SET SESSION SQL_MODE=DEFAULT; # === modified file 'mysql-test/r/range.result' --- a/mysql-test/r/range.result 2009-11-02 12:24:07 +0000 +++ b/mysql-test/r/range.result 2009-12-08 09:26:11 +0000 @@ -1603,4 +1603,54 @@ SELECT str_to_date('', '%Y-%m-%d'); str_to_date('', '%Y-%m-%d') 0000-00-00 DROP TABLE t1, t2; +# +# Bug#48459: valgrind errors with query using 'Range checked for each +# record' +# +CREATE TABLE t1 ( +a INT, +b CHAR(2), +c INT, +d INT, +KEY ( c ), +KEY ( d, a, b ( 2 ) ), +KEY ( b ( 1 ) ) +); +INSERT INTO t1 VALUES ( NULL, 'a', 1, 2 ), ( NULL, 'a', 1, 2 ), +( 1, 'a', 1, 2 ), ( 1, 'a', 1, 2 ); +CREATE TABLE t2 ( +a INT, +c INT, +e INT, +KEY ( e ) +); +INSERT INTO t2 VALUES ( 1, 1, NULL ), ( 1, 1, NULL ); +# Should not give Valgrind warnings +SELECT 1 +FROM t1, t2 +WHERE t1.d <> '1' AND t1.b > '1' +AND t1.a = t2.a AND t1.c = t2.c; +1 +1 +1 +1 +1 +DROP TABLE t1, t2; +# +# Bug #48665: sql-bench's insert test fails due to wrong result +# +CREATE TABLE t1 (a INT, b INT, PRIMARY KEY (a)); +INSERT INTO t1 VALUES (0,0), (1,1); +EXPLAIN +SELECT * FROM t1 FORCE INDEX (PRIMARY) +WHERE (a>=1 AND a<=2) OR (a>=4 AND a<=5) OR (a>=0 AND a <=10); +id select_type table type possible_keys key key_len ref rows Extra +@ @ @ range @ @ @ @ @ @ +# Should return 2 rows +SELECT * FROM t1 FORCE INDEX (PRIMARY) +WHERE (a>=1 AND a<=2) OR (a>=4 AND a<=5) OR (a>=0 AND a <=10); +a b +0 0 +1 1 +DROP TABLE t1; End of 5.1 tests === modified file 'mysql-test/r/select.result' --- a/mysql-test/r/select.result 2009-12-03 11:19:05 +0000 +++ b/mysql-test/r/select.result 2010-01-15 15:27:55 +0000 @@ -4426,6 +4426,20 @@ ROW(a,a) <=> ROW((SELECT 1 FROM t1 WHERE INTO @var0; ERROR 21000: Subquery returns more than 1 row DROP TABLE t1; +# +# Bug #48458: simple query tries to allocate enormous amount of +# memory +# +CREATE TABLE t1(a INT NOT NULL, b YEAR); +INSERT INTO t1 VALUES (); +Warnings: +Warning 1364 Field 'a' doesn't have a default value +CREATE TABLE t2(c INT); +# Should not err out because of out-of-memory +SELECT 1 FROM t2 JOIN t1 ON 1=1 +WHERE a != '1' AND NOT a >= b OR NOT ROW(b,a )<> ROW(a,a); +1 +DROP TABLE t1,t2; End of 5.0 tests create table t1(a INT, KEY (a)); INSERT INTO t1 VALUES (1),(2),(3),(4),(5); @@ -4576,4 +4590,47 @@ field2 15:13:38 drop table A,AA,B,BB; #end of test for bug#45266 +# +# BUG#48052: Valgrind warning - uninitialized value in init_read_record() +# +CREATE TABLE t1 ( +pk int(11) NOT NULL, +i int(11) DEFAULT NULL, +v varchar(1) DEFAULT NULL, +PRIMARY KEY (pk) +); +INSERT INTO t1 VALUES (2,7,'m'); +INSERT INTO t1 VALUES (3,9,'m'); +SELECT v +FROM t1 +WHERE NOT pk > 0 +HAVING v <= 't' +ORDER BY pk; +v +DROP TABLE t1; +# +# Bug#49489 Uninitialized cache led to a wrong result. +# +CREATE TABLE t1(c1 DOUBLE(5,4)); +INSERT INTO t1 VALUES (9.1234); +SELECT * FROM t1 WHERE c1 < 9.12345; +c1 +9.1234 +DROP TABLE t1; +# End of test for bug#49489. +# +# Bug #49517: Inconsistent behavior while using +# NULLable BIGINT and INT columns in comparison +# +CREATE TABLE t1(a BIGINT UNSIGNED NOT NULL, b BIGINT NULL, c INT NULL); +INSERT INTO t1 VALUES(105, NULL, NULL); +SELECT * FROM t1 WHERE b < 102; +a b c +SELECT * FROM t1 WHERE c < 102; +a b c +SELECT * FROM t1 WHERE 102 < b; +a b c +SELECT * FROM t1 WHERE 102 < c; +a b c +DROP TABLE t1; End of 5.1 tests === modified file 'mysql-test/r/show_check.result' --- a/mysql-test/r/show_check.result 2009-03-06 14:56:17 +0000 +++ b/mysql-test/r/show_check.result 2009-12-15 09:03:24 +0000 @@ -1454,4 +1454,10 @@ GRANT PROCESS ON *.* TO test_u@localhost SHOW ENGINE MYISAM MUTEX; SHOW ENGINE MYISAM STATUS; DROP USER test_u@localhost; +# +# Bug #48985: show create table crashes if previous access to the table +# was killed +# +SHOW CREATE TABLE non_existent; +ERROR 70100: Query execution was interrupted End of 5.1 tests === modified file 'mysql-test/r/sp-destruct.result' --- a/mysql-test/r/sp-destruct.result 2008-04-08 14:51:26 +0000 +++ b/mysql-test/r/sp-destruct.result 2009-11-21 11:18:21 +0000 @@ -1,3 +1,4 @@ +call mtr.add_suppression("Column count of mysql.proc is wrong. Expected 20, found 19. The table is probably corrupted"); use test; drop procedure if exists bug14233; drop function if exists bug14233; @@ -11,11 +12,13 @@ create table t1 (id int); create trigger t1_ai after insert on t1 for each row call bug14233(); alter table mysql.proc drop type; call bug14233(); -ERROR HY000: Failed to load routine test.bug14233. The table mysql.proc is missing, corrupt, or contains bad data (internal code -5) +ERROR HY000: Column count of mysql.proc is wrong. Expected 20, found 19. The table is probably corrupted create view v1 as select bug14233_f(); -ERROR HY000: Failed to load routine test.bug14233_f. The table mysql.proc is missing, corrupt, or contains bad data (internal code -5) +ERROR HY000: Column count of mysql.proc is wrong. Expected 20, found 19. The table is probably corrupted insert into t1 values (0); -ERROR HY000: Failed to load routine test.bug14233. The table mysql.proc is missing, corrupt, or contains bad data (internal code -5) +ERROR HY000: Column count of mysql.proc is wrong. Expected 20, found 19. The table is probably corrupted +show procedure status; +ERROR HY000: Column count of mysql.proc is wrong. Expected 20, found 19. The table is probably corrupted flush table mysql.proc; call bug14233(); ERROR HY000: Incorrect information in file: './mysql/proc.frm' @@ -88,3 +91,28 @@ show procedure status where db=DATABASE( Db Name Type Definer Modified Created Security_type Comment character_set_client collation_connection Database Collation show function status where db=DATABASE(); Db Name Type Definer Modified Created Security_type Comment character_set_client collation_connection Database Collation +DROP TABLE IF EXISTS proc_backup; +DROP PROCEDURE IF EXISTS p1; +# Backup the proc table +RENAME TABLE mysql.proc TO proc_backup; +CREATE TABLE mysql.proc LIKE proc_backup; +FLUSH TABLE mysql.proc; +# Test with a valid table. +CREATE PROCEDURE p1() +SET @foo = 10; +CALL p1(); +SHOW PROCEDURE STATUS; +Db Name Type Definer Modified Created Security_type Comment character_set_client collation_connection Database Collation +test p1 PROCEDURE root@localhost 0000-00-00 00:00:00 0000-00-00 00:00:00 DEFINER latin1 latin1_swedish_ci latin1_swedish_ci +# Modify a field of the table. +ALTER TABLE mysql.proc MODIFY comment CHAR (32); +CREATE PROCEDURE p2() +SET @foo = 10; +ERROR HY000: Cannot load from mysql.proc. The table is probably corrupted +# Procedure loaded from the cache +CALL p1(); +SHOW PROCEDURE STATUS; +ERROR HY000: Cannot load from mysql.proc. The table is probably corrupted +DROP TABLE mysql.proc; +RENAME TABLE proc_backup TO mysql.proc; +FLUSH TABLE mysql.proc; === modified file 'mysql-test/r/sp-security.result' --- a/mysql-test/r/sp-security.result 2009-03-06 14:56:17 +0000 +++ b/mysql-test/r/sp-security.result 2009-11-27 16:10:28 +0000 @@ -510,4 +510,60 @@ DROP USER mysqltest_u1@localhost; DROP PROCEDURE p_suid; DROP FUNCTION f_suid; DROP TABLE t1; +# +# Bug #48872 : Privileges for stored functions ignored if function name +# is mixed case +# +CREATE DATABASE B48872; +USE B48872; +CREATE TABLE `TestTab` (id INT); +INSERT INTO `TestTab` VALUES (1),(2); +CREATE FUNCTION `f_Test`() RETURNS INT RETURN 123; +CREATE FUNCTION `f_Test_denied`() RETURNS INT RETURN 123; +CREATE USER 'tester'; +CREATE USER 'Tester'; +GRANT SELECT ON TABLE `TestTab` TO 'tester'; +GRANT EXECUTE ON FUNCTION `f_Test` TO 'tester'; +GRANT EXECUTE ON FUNCTION `f_Test_denied` TO 'Tester'; +SELECT f_Test(); +f_Test() +123 +SELECT * FROM TestTab; +id +1 +2 +SELECT * FROM TestTab; +id +1 +2 +SELECT `f_Test`(); +`f_Test`() +123 +SELECT `F_TEST`(); +`F_TEST`() +123 +SELECT f_Test(); +f_Test() +123 +SELECT F_TEST(); +F_TEST() +123 +SELECT * FROM TestTab; +SELECT `f_Test`(); +SELECT `F_TEST`(); +SELECT f_Test(); +SELECT F_TEST(); +SELECT `f_Test_denied`(); +`f_Test_denied`() +123 +SELECT `F_TEST_DENIED`(); +`F_TEST_DENIED`() +123 +DROP TABLE `TestTab`; +DROP FUNCTION `f_Test`; +DROP FUNCTION `f_Test_denied`; +USE test; +DROP USER 'tester'; +DROP USER 'Tester'; +DROP DATABASE B48872; End of 5.0 tests. === modified file 'mysql-test/r/sp.result' --- a/mysql-test/r/sp.result 2009-10-23 13:54:58 +0000 +++ b/mysql-test/r/sp.result 2009-11-13 01:03:26 +0000 @@ -6979,6 +6979,64 @@ CALL p1; ERROR 42S22: Unknown column 'A.b' in 'IN/ALL/ANY subquery' DROP PROCEDURE p1; DROP TABLE t1, t2; +# +# Bug#47627: SET @@{global.session}.local_variable in stored routine causes crash +# Bug#48626: Crash or lost connection using SET for declared variables with @@ +# +DROP PROCEDURE IF EXISTS p1; +DROP PROCEDURE IF EXISTS p2; +DROP PROCEDURE IF EXISTS p3; +CREATE PROCEDURE p1() +BEGIN +DECLARE v INT DEFAULT 0; +SET @@SESSION.v= 10; +END// +ERROR HY000: Unknown system variable 'v' +CREATE PROCEDURE p2() +BEGIN +DECLARE v INT DEFAULT 0; +SET v= 10; +END// +call p2()// +CREATE PROCEDURE p3() +BEGIN +DECLARE v INT DEFAULT 0; +SELECT @@SESSION.v; +END// +ERROR HY000: Unknown system variable 'v' +CREATE PROCEDURE p4() +BEGIN +DECLARE v INT DEFAULT 0; +SET @@GLOBAL.v= 10; +END// +ERROR HY000: Unknown system variable 'v' +CREATE PROCEDURE p5() +BEGIN +DECLARE init_connect INT DEFAULT 0; +SET init_connect= 10; +SET @@GLOBAL.init_connect= 'SELECT 1'; +SET @@SESSION.IDENTITY= 1; +SELECT @@SESSION.IDENTITY; +SELECT @@GLOBAL.init_connect; +SELECT init_connect; +END// +CREATE PROCEDURE p6() +BEGIN +DECLARE v INT DEFAULT 0; +SET @@v= 0; +END// +ERROR HY000: Unknown system variable 'v' +SET @old_init_connect= @@GLOBAL.init_connect; +CALL p5(); +@@SESSION.IDENTITY +1 +@@GLOBAL.init_connect +SELECT 1 +init_connect +10 +SET @@GLOBAL.init_connect= @old_init_connect; +DROP PROCEDURE p2; +DROP PROCEDURE p5; # ------------------------------------------------------------------ # -- End of 5.1 tests # ------------------------------------------------------------------ === modified file 'mysql-test/r/trigger.result' --- a/mysql-test/r/trigger.result 2009-06-22 12:51:33 +0000 +++ b/mysql-test/r/trigger.result 2010-01-12 08:19:48 +0000 @@ -1448,33 +1448,6 @@ isave 1 2 drop table t1, t2, t3; -CREATE TABLE t1 (id INTEGER); -CREATE TABLE t2 (id INTEGER); -INSERT INTO t2 VALUES (1),(2); -CREATE TRIGGER t1_test AFTER INSERT ON t1 FOR EACH ROW -INSERT INTO t2 VALUES (new.id); -SELECT GET_LOCK('B26162',120); -GET_LOCK('B26162',120) -1 -SELECT 'rl_acquirer', GET_LOCK('B26162',120), id FROM t2 WHERE id = 1; -SET SESSION LOW_PRIORITY_UPDATES=1; -SET GLOBAL LOW_PRIORITY_UPDATES=1; -INSERT INTO t1 VALUES (5); -SELECT 'rl_contender', id FROM t2 WHERE id > 1; -SELECT RELEASE_LOCK('B26162'); -RELEASE_LOCK('B26162') -1 -rl_acquirer GET_LOCK('B26162',120) id -rl_acquirer 1 1 -SELECT RELEASE_LOCK('B26162'); -RELEASE_LOCK('B26162') -1 -rl_contender id -rl_contender 2 -DROP TRIGGER t1_test; -DROP TABLE t1,t2; -SET SESSION LOW_PRIORITY_UPDATES=DEFAULT; -SET GLOBAL LOW_PRIORITY_UPDATES=DEFAULT; Bug#28502 Triggers that update another innodb table will block on X lock unnecessarily === modified file 'mysql-test/r/trigger_notembedded.result' --- a/mysql-test/r/trigger_notembedded.result 2009-09-17 11:33:23 +0000 +++ b/mysql-test/r/trigger_notembedded.result 2010-01-12 08:19:48 +0000 @@ -445,6 +445,33 @@ DROP TABLE t2; DROP TABLE t1; DROP DATABASE mysqltest_db1; USE test; +CREATE TABLE t1 (id INTEGER); +CREATE TABLE t2 (id INTEGER); +INSERT INTO t2 VALUES (1),(2); +CREATE TRIGGER t1_test AFTER INSERT ON t1 FOR EACH ROW +INSERT INTO t2 VALUES (new.id); +SELECT GET_LOCK('B26162',120); +GET_LOCK('B26162',120) +1 +SELECT 'rl_acquirer', GET_LOCK('B26162',120), id FROM t2 WHERE id = 1; +SET SESSION LOW_PRIORITY_UPDATES=1; +SET GLOBAL LOW_PRIORITY_UPDATES=1; +INSERT INTO t1 VALUES (5); +SELECT 'rl_contender', id FROM t2 WHERE id > 1; +SELECT RELEASE_LOCK('B26162'); +RELEASE_LOCK('B26162') +1 +rl_acquirer GET_LOCK('B26162',120) id +rl_acquirer 1 1 +SELECT RELEASE_LOCK('B26162'); +RELEASE_LOCK('B26162') +1 +rl_contender id +rl_contender 2 +DROP TRIGGER t1_test; +DROP TABLE t1,t2; +SET SESSION LOW_PRIORITY_UPDATES=DEFAULT; +SET GLOBAL LOW_PRIORITY_UPDATES=DEFAULT; End of 5.0 tests. drop table if exists t1; create table t1 (i int); === modified file 'mysql-test/r/type_newdecimal.result' --- a/mysql-test/r/type_newdecimal.result 2009-11-02 11:21:39 +0000 +++ b/mysql-test/r/type_newdecimal.result 2009-12-08 09:26:11 +0000 @@ -1630,3 +1630,287 @@ SELECT my_col FROM t1; my_col 0.012345687012345687012345687012 DROP TABLE t1; +# +# Bug#45261: Crash, stored procedure + decimal +# +DROP TABLE IF EXISTS t1; +CREATE TABLE t1 SELECT +/* 81 */ 100000000000000000000000000000000000000000000000000000000000000000000000000000001 +AS c1; +Warnings: +Warning 1264 Out of range value for column 'c1' at row 1 +DESC t1; +Field Type Null Key Default Extra +c1 decimal(65,0) NO 0 +SELECT * FROM t1; +c1 +99999999999999999999999999999999999999999999999999999999999999999 +DROP TABLE t1; +CREATE TABLE t1 SELECT +/* 81 */ 100000000000000000000000000000000000000000000000000000000000000000000000000000001. +AS c1; +Warnings: +Warning 1264 Out of range value for column 'c1' at row 1 +DESC t1; +Field Type Null Key Default Extra +c1 decimal(65,0) NO 0 +SELECT * FROM t1; +c1 +99999999999999999999999999999999999999999999999999999999999999999 +DROP TABLE t1; +CREATE TABLE t1 SELECT +/* 81 */ 100000000000000000000000000000000000000000000000000000000000000000000000000000001.1 /* 1 */ +AS c1; +Warnings: +Warning 1264 Out of range value for column 'c1' at row 1 +DESC t1; +Field Type Null Key Default Extra +c1 decimal(65,0) NO 0 +SELECT * FROM t1; +c1 +99999999999999999999999999999999999999999999999999999999999999999 +DROP TABLE t1; +CREATE TABLE t1 SELECT +/* 82 */ 1000000000000000000000000000000000000000000000000000000000000000000000000000000001 +AS c1; +Warnings: +Error 1292 Truncated incorrect DECIMAL value: '' +DESC t1; +Field Type Null Key Default Extra +c1 decimal(65,0) NO 0 +SELECT * FROM t1; +c1 +99999999999999999999999999999999999999999999999999999999999999999 +DROP TABLE t1; +CREATE TABLE t1 SELECT +/* 40 */ 1000000000000000000000000000000000000001.1000000000000000000000000000000000000001 /* 40 */ +AS c1; +Warnings: +Warning 1264 Out of range value for column 'c1' at row 1 +DESC t1; +Field Type Null Key Default Extra +c1 decimal(65,30) NO 0.000000000000000000000000000000 +SELECT * FROM t1; +c1 +99999999999999999999999999999999999.999999999999999999999999999999 +DROP TABLE t1; +CREATE TABLE t1 SELECT +/* 1 */ 1.10000000000000000000000000000000000000000000000000000000000000000000000000000001 /* 80 */ +AS c1; +DESC t1; +Field Type Null Key Default Extra +c1 decimal(31,30) NO 0.000000000000000000000000000000 +SELECT * FROM t1; +c1 +1.100000000000000000000000000000 +DROP TABLE t1; +CREATE TABLE t1 SELECT +/* 1 */ 1.100000000000000000000000000000000000000000000000000000000000000000000000000000001 /* 81 */ +AS c1; +DESC t1; +Field Type Null Key Default Extra +c1 decimal(31,30) NO 0.000000000000000000000000000000 +SELECT * FROM t1; +c1 +1.100000000000000000000000000000 +DROP TABLE t1; +CREATE TABLE t1 SELECT +.100000000000000000000000000000000000000000000000000000000000000000000000000000001 /* 81 */ +AS c1; +Warnings: +Note 1265 Data truncated for column 'c1' at row 1 +DESC t1; +Field Type Null Key Default Extra +c1 decimal(30,30) NO 0.000000000000000000000000000000 +SELECT * FROM t1; +c1 +0.100000000000000000000000000000 +DROP TABLE t1; +CREATE TABLE t1 SELECT +/* 45 */ 123456789012345678901234567890123456789012345.123456789012345678901234567890123456789012345 /* 45 */ +AS c1; +Warnings: +Warning 1264 Out of range value for column 'c1' at row 1 +DESC t1; +Field Type Null Key Default Extra +c1 decimal(65,30) NO 0.000000000000000000000000000000 +SELECT * FROM t1; +c1 +99999999999999999999999999999999999.999999999999999999999999999999 +DROP TABLE t1; +CREATE TABLE t1 SELECT +/* 65 */ 12345678901234567890123456789012345678901234567890123456789012345.1 /* 1 */ +AS c1; +Warnings: +Warning 1264 Out of range value for column 'c1' at row 1 +DESC t1; +Field Type Null Key Default Extra +c1 decimal(65,1) NO 0.0 +SELECT * FROM t1; +c1 +9999999999999999999999999999999999999999999999999999999999999999.9 +DROP TABLE t1; +CREATE TABLE t1 SELECT +/* 66 */ 123456789012345678901234567890123456789012345678901234567890123456.1 /* 1 */ +AS c1; +Warnings: +Warning 1264 Out of range value for column 'c1' at row 1 +DESC t1; +Field Type Null Key Default Extra +c1 decimal(65,1) NO 0.0 +SELECT * FROM t1; +c1 +9999999999999999999999999999999999999999999999999999999999999999.9 +DROP TABLE t1; +CREATE TABLE t1 SELECT +.123456789012345678901234567890123456789012345678901234567890123456 /* 66 */ +AS c1; +Warnings: +Note 1265 Data truncated for column 'c1' at row 1 +DESC t1; +Field Type Null Key Default Extra +c1 decimal(30,30) NO 0.000000000000000000000000000000 +SELECT * FROM t1; +c1 +0.123456789012345678901234567890 +DROP TABLE t1; +CREATE TABLE t1 AS SELECT 123.1234567890123456789012345678901 /* 31 */ AS c1; +Warnings: +Note 1265 Data truncated for column 'c1' at row 1 +DESC t1; +Field Type Null Key Default Extra +c1 decimal(33,30) NO 0.000000000000000000000000000000 +SELECT * FROM t1; +c1 +123.123456789012345678901234567890 +DROP TABLE t1; +CREATE TABLE t1 SELECT 1.1 + CAST(1 AS DECIMAL(65,30)) AS c1; +DESC t1; +Field Type Null Key Default Extra +c1 decimal(65,30) NO 0.000000000000000000000000000000 +SELECT * FROM t1; +c1 +2.100000000000000000000000000000 +DROP TABLE t1; +# +# Test that the integer and decimal parts are properly calculated. +# +CREATE TABLE t1 (a DECIMAL(30,30)); +INSERT INTO t1 VALUES (0.1),(0.2),(0.3); +CREATE TABLE t2 SELECT MIN(a + 0.0000000000000000000000000000001) AS c1 FROM t1; +Warnings: +Note 1265 Data truncated for column 'c1' at row 3 +DESC t2; +Field Type Null Key Default Extra +c1 decimal(32,30) YES NULL +DROP TABLE t1,t2; +CREATE TABLE t1 (a DECIMAL(30,30)); +INSERT INTO t1 VALUES (0.1),(0.2),(0.3); +CREATE TABLE t2 SELECT IFNULL(a + 0.0000000000000000000000000000001, NULL) AS c1 FROM t1; +Warnings: +Note 1265 Data truncated for column 'c1' at row 1 +Note 1265 Data truncated for column 'c1' at row 2 +Note 1265 Data truncated for column 'c1' at row 3 +DESC t2; +Field Type Null Key Default Extra +c1 decimal(34,0) YES NULL +DROP TABLE t1,t2; +CREATE TABLE t1 (a DECIMAL(30,30)); +INSERT INTO t1 VALUES (0.1),(0.2),(0.3); +CREATE TABLE t2 SELECT CASE a WHEN 0.1 THEN 0.0000000000000000000000000000000000000000000000000000000000000000001 END AS c1 FROM t1; +Warnings: +Note 1265 Data truncated for column 'c1' at row 1 +DESC t2; +Field Type Null Key Default Extra +c1 decimal(65,30) YES NULL +DROP TABLE t1,t2; +# +# Test that variables get maximum precision. +# +SET @decimal= 1.1; +CREATE TABLE t1 SELECT @decimal AS c1; +DESC t1; +Field Type Null Key Default Extra +c1 decimal(65,30) YES NULL +SELECT * FROM t1; +c1 +1.100000000000000000000000000000 +DROP TABLE t1; +# +# Bug #45261 : Crash, stored procedure + decimal +# Original test by the reporter. +# +# should not crash +CREATE TABLE t1 +SELECT .123456789012345678901234567890123456789012345678901234567890123456 AS a; +Warnings: +Note 1265 Data truncated for column 'a' at row 1 +DROP TABLE t1; +CREATE PROCEDURE test_proc() +BEGIN +# The las non critical CUSER definition is: +# DECLARE mycursor CURSOR FOR SELECT 1 % +# .12345678912345678912345678912345678912345678912345678912345678912 AS my_col; +DECLARE mycursor CURSOR FOR +SELECT 1 % +.123456789123456789123456789123456789123456789123456789123456789123456789123456789 +AS my_col; +OPEN mycursor; +CLOSE mycursor; +END| +# should not crash +CALL test_proc(); +DROP PROCEDURE test_proc; +# +# Bug #48370 Absolutely wrong calculations with GROUP BY and +# decimal fields when using IF +# +CREATE TABLE currencies (id int, rate decimal(16,4), +PRIMARY KEY (id), KEY (rate)); +INSERT INTO currencies VALUES (11,0.7028); +INSERT INTO currencies VALUES (1,1); +CREATE TABLE payments ( +id int, +supplier_id int, +status int, +currency_id int, +vat decimal(7,4), +PRIMARY KEY (id), +KEY currency_id (currency_id), +KEY supplier_id (supplier_id) +); +INSERT INTO payments (id,status,vat,supplier_id,currency_id) VALUES +(3001,2,0.0000,344,11), (1,2,0.0000,1,1); +CREATE TABLE sub_tasks ( +id int, +currency_id int, +price decimal(16,4), +discount decimal(10,4), +payment_id int, +PRIMARY KEY (id), +KEY currency_id (currency_id), +KEY payment_id (payment_id) +) ; +INSERT INTO sub_tasks (id, price, discount, payment_id, currency_id) VALUES +(52, 12.60, 0, 3001, 11), (56, 14.58, 0, 3001, 11); +# should return 1 and the same values in col 2 and 3 +select STRAIGHT_JOIN +(1 + PAY.vat) AS mult, +SUM(ROUND((SUB.price - ROUND(ROUND(SUB.price, 2) * SUB.discount, 2)) * +CUR.rate / CUR.rate, 2) +) v_net_with_discount, +SUM(ROUND((SUB.price - ROUND(ROUND(SUB.price, 2) * SUB.discount, 1)) * +CUR.rate / CUR.rate , 2) +* (1 + PAY.vat) +) v_total +from +currencies CUR, payments PAY, sub_tasks SUB +where +SUB.payment_id = PAY.id and +PAY.currency_id = CUR.id and +PAY.id > 2 +group by PAY.id + 1; +mult v_net_with_discount v_total +1.0000 27.18 27.180000 +DROP TABLE currencies, payments, sub_tasks; +End of 5.1 tests === modified file 'mysql-test/r/type_year.result' --- a/mysql-test/r/type_year.result 2007-03-29 04:08:30 +0000 +++ b/mysql-test/r/type_year.result 2009-12-15 08:37:10 +0000 @@ -46,3 +46,267 @@ a 2001 drop table t1; End of 5.0 tests +# +# Bug #49480: WHERE using YEAR columns returns unexpected results +# +CREATE TABLE t2(yy YEAR(2), c2 CHAR(4)); +CREATE TABLE t4(yyyy YEAR(4), c4 CHAR(4)); +INSERT INTO t2 (c2) VALUES (NULL),(1970),(1999),(2000),(2001),(2069); +INSERT INTO t4 (c4) SELECT c2 FROM t2; +UPDATE t2 SET yy = c2; +UPDATE t4 SET yyyy = c4; +SELECT * FROM t2; +yy c2 +NULL NULL +70 1970 +99 1999 +00 2000 +01 2001 +69 2069 +SELECT * FROM t4; +yyyy c4 +NULL NULL +1970 1970 +1999 1999 +2000 2000 +2001 2001 +2069 2069 +# Comparison of YEAR(2) with YEAR(4) +SELECT * FROM t2, t4 WHERE yy = yyyy; +yy c2 yyyy c4 +70 1970 1970 1970 +99 1999 1999 1999 +00 2000 2000 2000 +01 2001 2001 2001 +69 2069 2069 2069 +SELECT * FROM t2, t4 WHERE yy <=> yyyy; +yy c2 yyyy c4 +NULL NULL NULL NULL +70 1970 1970 1970 +99 1999 1999 1999 +00 2000 2000 2000 +01 2001 2001 2001 +69 2069 2069 2069 +SELECT * FROM t2, t4 WHERE yy < yyyy; +yy c2 yyyy c4 +70 1970 1999 1999 +70 1970 2000 2000 +99 1999 2000 2000 +70 1970 2001 2001 +99 1999 2001 2001 +00 2000 2001 2001 +70 1970 2069 2069 +99 1999 2069 2069 +00 2000 2069 2069 +01 2001 2069 2069 +SELECT * FROM t2, t4 WHERE yy > yyyy; +yy c2 yyyy c4 +99 1999 1970 1970 +00 2000 1970 1970 +01 2001 1970 1970 +69 2069 1970 1970 +00 2000 1999 1999 +01 2001 1999 1999 +69 2069 1999 1999 +01 2001 2000 2000 +69 2069 2000 2000 +69 2069 2001 2001 +# Comparison of YEAR(2) with YEAR(2) +SELECT * FROM t2 a, t2 b WHERE a.yy = b.yy; +yy c2 yy c2 +70 1970 70 1970 +99 1999 99 1999 +00 2000 00 2000 +01 2001 01 2001 +69 2069 69 2069 +SELECT * FROM t2 a, t2 b WHERE a.yy <=> b.yy; +yy c2 yy c2 +NULL NULL NULL NULL +70 1970 70 1970 +99 1999 99 1999 +00 2000 00 2000 +01 2001 01 2001 +69 2069 69 2069 +SELECT * FROM t2 a, t2 b WHERE a.yy < b.yy; +yy c2 yy c2 +70 1970 99 1999 +70 1970 00 2000 +99 1999 00 2000 +70 1970 01 2001 +99 1999 01 2001 +00 2000 01 2001 +70 1970 69 2069 +99 1999 69 2069 +00 2000 69 2069 +01 2001 69 2069 +# Comparison of YEAR(4) with YEAR(4) +SELECT * FROM t4 a, t4 b WHERE a.yyyy = b.yyyy; +yyyy c4 yyyy c4 +1970 1970 1970 1970 +1999 1999 1999 1999 +2000 2000 2000 2000 +2001 2001 2001 2001 +2069 2069 2069 2069 +SELECT * FROM t4 a, t4 b WHERE a.yyyy <=> b.yyyy; +yyyy c4 yyyy c4 +NULL NULL NULL NULL +1970 1970 1970 1970 +1999 1999 1999 1999 +2000 2000 2000 2000 +2001 2001 2001 2001 +2069 2069 2069 2069 +SELECT * FROM t4 a, t4 b WHERE a.yyyy < b.yyyy; +yyyy c4 yyyy c4 +1970 1970 1999 1999 +1970 1970 2000 2000 +1999 1999 2000 2000 +1970 1970 2001 2001 +1999 1999 2001 2001 +2000 2000 2001 2001 +1970 1970 2069 2069 +1999 1999 2069 2069 +2000 2000 2069 2069 +2001 2001 2069 2069 +# Comparison with constants: +SELECT * FROM t2 WHERE yy = NULL; +yy c2 +SELECT * FROM t4 WHERE yyyy = NULL; +yyyy c4 +SELECT * FROM t2 WHERE yy <=> NULL; +yy c2 +NULL NULL +SELECT * FROM t4 WHERE yyyy <=> NULL; +yyyy c4 +NULL NULL +SELECT * FROM t2 WHERE yy < NULL; +yy c2 +SELECT * FROM t2 WHERE yy > NULL; +yy c2 +SELECT * FROM t2 WHERE yy = NOW(); +yy c2 +SELECT * FROM t4 WHERE yyyy = NOW(); +yyyy c4 +SELECT * FROM t2 WHERE yy = 99; +yy c2 +99 1999 +SELECT * FROM t2 WHERE 99 = yy; +yy c2 +99 1999 +SELECT * FROM t4 WHERE yyyy = 99; +yyyy c4 +1999 1999 +SELECT * FROM t2 WHERE yy = 'test'; +yy c2 +00 2000 +Warnings: +Warning 1292 Truncated incorrect DOUBLE value: 'test' +SELECT * FROM t4 WHERE yyyy = 'test'; +yyyy c4 +Warnings: +Warning 1292 Truncated incorrect DOUBLE value: 'test' +SELECT * FROM t2 WHERE yy = '1999'; +yy c2 +99 1999 +SELECT * FROM t4 WHERE yyyy = '1999'; +yyyy c4 +1999 1999 +SELECT * FROM t2 WHERE yy = 1999; +yy c2 +99 1999 +SELECT * FROM t4 WHERE yyyy = 1999; +yyyy c4 +1999 1999 +SELECT * FROM t2 WHERE yy = 1999.1; +yy c2 +99 1999 +SELECT * FROM t4 WHERE yyyy = 1999.1; +yyyy c4 +1999 1999 +SELECT * FROM t2 WHERE yy = 1998.9; +yy c2 +99 1999 +SELECT * FROM t4 WHERE yyyy = 1998.9; +yyyy c4 +1999 1999 +# Coverage tests for YEAR with zero/2000 constants: +SELECT * FROM t2 WHERE yy = 0; +yy c2 +00 2000 +SELECT * FROM t2 WHERE yy = '0'; +yy c2 +00 2000 +SELECT * FROM t2 WHERE yy = '0000'; +yy c2 +00 2000 +SELECT * FROM t2 WHERE yy = '2000'; +yy c2 +00 2000 +SELECT * FROM t2 WHERE yy = 2000; +yy c2 +00 2000 +SELECT * FROM t4 WHERE yyyy = 0; +yyyy c4 +SELECT * FROM t4 WHERE yyyy = '0'; +yyyy c4 +2000 2000 +SELECT * FROM t4 WHERE yyyy = '0000'; +yyyy c4 +SELECT * FROM t4 WHERE yyyy = '2000'; +yyyy c4 +2000 2000 +SELECT * FROM t4 WHERE yyyy = 2000; +yyyy c4 +2000 2000 +# Comparison with constants those are out of YEAR range +# (coverage test for backward compatibility) +SELECT COUNT(yy) FROM t2; +COUNT(yy) +5 +SELECT COUNT(yyyy) FROM t4; +COUNT(yyyy) +5 +SELECT COUNT(*) FROM t2 WHERE yy = -1; +COUNT(*) +0 +SELECT COUNT(*) FROM t4 WHERE yyyy > -1; +COUNT(*) +5 +SELECT COUNT(*) FROM t2 WHERE yy > -1000000000000000000; +COUNT(*) +5 +SELECT COUNT(*) FROM t4 WHERE yyyy > -1000000000000000000; +COUNT(*) +5 +SELECT COUNT(*) FROM t2 WHERE yy < 2156; +COUNT(*) +5 +SELECT COUNT(*) FROM t4 WHERE yyyy < 2156; +COUNT(*) +5 +SELECT COUNT(*) FROM t2 WHERE yy < 1000000000000000000; +COUNT(*) +5 +SELECT COUNT(*) FROM t4 WHERE yyyy < 1000000000000000000; +COUNT(*) +5 +SELECT * FROM t2 WHERE yy < 123; +yy c2 +70 1970 +99 1999 +00 2000 +01 2001 +69 2069 +SELECT * FROM t2 WHERE yy > 123; +yy c2 +SELECT * FROM t4 WHERE yyyy < 123; +yyyy c4 +SELECT * FROM t4 WHERE yyyy > 123; +yyyy c4 +1970 1970 +1999 1999 +2000 2000 +2001 2001 +2069 2069 +DROP TABLE t2, t4; +# +End of 5.1 tests === added file 'mysql-test/std_data/bug47012.ARM' Binary files a/mysql-test/std_data/bug47012.ARM 1970-01-01 00:00:00 +0000 and b/mysql-test/std_data/bug47012.ARM 2009-11-11 08:03:29 +0000 differ === added file 'mysql-test/std_data/bug47012.ARZ' Binary files a/mysql-test/std_data/bug47012.ARZ 1970-01-01 00:00:00 +0000 and b/mysql-test/std_data/bug47012.ARZ 2009-11-11 08:03:29 +0000 differ === added file 'mysql-test/std_data/bug47012.frm' Binary files a/mysql-test/std_data/bug47012.frm 1970-01-01 00:00:00 +0000 and b/mysql-test/std_data/bug47012.frm 2009-11-11 08:03:29 +0000 differ === modified file 'mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result' --- a/mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result 2009-09-28 12:41:10 +0000 +++ b/mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result 2009-11-18 14:50:31 +0000 @@ -1,3 +1,4 @@ +CALL mtr.add_suppression("Statement may not be safe to log in statement format."); drop table if exists t1, t2; create table t1 (a int) engine=innodb; create table t2 (a int) engine=myisam; @@ -224,6 +225,8 @@ create table t0 (n int); insert t0 select * from t1; set autocommit=1; insert into t0 select GET_LOCK("lock1",null); +Warnings: +Note 1592 Statement may not be safe to log in statement format. set autocommit=0; create table t2 (n int) engine=innodb; insert into t2 values (3); === modified file 'mysql-test/suite/binlog/r/binlog_stm_row.result' --- a/mysql-test/suite/binlog/r/binlog_stm_row.result 2009-09-07 20:50:10 +0000 +++ b/mysql-test/suite/binlog/r/binlog_stm_row.result 2010-01-15 15:27:55 +0000 @@ -1,3 +1,4 @@ +CALL mtr.add_suppression("Statement may not be safe to log in statement format."); DROP TABLE IF EXISTS t1; DROP TABLE IF EXISTS t2; set @saved_global_binlog_format = @@global.binlog_format; @@ -29,6 +30,8 @@ SELECT RELEASE_LOCK('Bug#34306'); RELEASE_LOCK('Bug#34306') 1 # con2 +Warnings: +Note 1592 Statement may not be safe to log in statement format. SELECT RELEASE_LOCK('Bug#34306'); RELEASE_LOCK('Bug#34306') 1 === modified file 'mysql-test/suite/binlog/r/binlog_unsafe.result' --- a/mysql-test/suite/binlog/r/binlog_unsafe.result 2009-09-07 20:50:10 +0000 +++ b/mysql-test/suite/binlog/r/binlog_unsafe.result 2010-01-19 10:36:52 +0000 @@ -327,4 +327,87 @@ Warnings: Note 1592 Statement may not be safe to log in statement format. DROP TABLE t1, t2; SET @@SESSION.SQL_MODE = @save_sql_mode; +CREATE TABLE t1 (a VARCHAR(1000)); +INSERT INTO t1 VALUES (CURRENT_USER()); +Warnings: +Note 1592 Statement may not be safe to log in statement format. +INSERT INTO t1 VALUES (FOUND_ROWS()); +Warnings: +Note 1592 Statement may not be safe to log in statement format. +INSERT INTO t1 VALUES (GET_LOCK('tmp', 1)); +Warnings: +Note 1592 Statement may not be safe to log in statement format. +INSERT INTO t1 VALUES (IS_FREE_LOCK('tmp')); +Warnings: +Note 1592 Statement may not be safe to log in statement format. +INSERT INTO t1 VALUES (IS_USED_LOCK('tmp')); +Warnings: +Note 1592 Statement may not be safe to log in statement format. +INSERT INTO t1 VALUES (LOAD_FILE('../../std_data/words2.dat')); +Warnings: +Note 1592 Statement may not be safe to log in statement format. +INSERT INTO t1 VALUES (MASTER_POS_WAIT('dummy arg', 4711, 1)); +Warnings: +Note 1592 Statement may not be safe to log in statement format. +INSERT INTO t1 VALUES (RELEASE_LOCK('tmp')); +Warnings: +Note 1592 Statement may not be safe to log in statement format. +INSERT INTO t1 VALUES (ROW_COUNT()); +Warnings: +Note 1592 Statement may not be safe to log in statement format. +INSERT INTO t1 VALUES (SESSION_USER()); +Warnings: +Note 1592 Statement may not be safe to log in statement format. +INSERT INTO t1 VALUES (SLEEP(1)); +Warnings: +Note 1592 Statement may not be safe to log in statement format. +INSERT INTO t1 VALUES (SYSDATE()); +Warnings: +Note 1592 Statement may not be safe to log in statement format. +INSERT INTO t1 VALUES (SYSTEM_USER()); +Warnings: +Note 1592 Statement may not be safe to log in statement format. +INSERT INTO t1 VALUES (USER()); +Warnings: +Note 1592 Statement may not be safe to log in statement format. +INSERT INTO t1 VALUES (UUID()); +Warnings: +Note 1592 Statement may not be safe to log in statement format. +INSERT INTO t1 VALUES (UUID_SHORT()); +Warnings: +Note 1592 Statement may not be safe to log in statement format. +INSERT INTO t1 VALUES (VERSION()); +Warnings: +Note 1592 Statement may not be safe to log in statement format. +DELETE FROM t1; +SET TIME_ZONE= '+03:00'; +SET TIMESTAMP=1000000; +INSERT INTO t1 VALUES +(CURDATE()), +(CURRENT_DATE()), +(CURRENT_TIME()), +(CURRENT_TIMESTAMP()), +(CURTIME()), +(LOCALTIME()), +(LOCALTIMESTAMP()), +(NOW()), +(UNIX_TIMESTAMP()), +(UTC_DATE()), +(UTC_TIME()), +(UTC_TIMESTAMP()); +SELECT * FROM t1; +a +1970-01-12 +1970-01-12 +16:46:40 +1970-01-12 16:46:40 +16:46:40 +1970-01-12 16:46:40 +1970-01-12 16:46:40 +1970-01-12 16:46:40 +1000000 +1970-01-12 +13:46:40 +1970-01-12 13:46:40 +DROP TABLE t1; "End of tests" === modified file 'mysql-test/suite/binlog/t/binlog_killed.test' --- a/mysql-test/suite/binlog/t/binlog_killed.test 2008-10-23 19:27:09 +0000 +++ b/mysql-test/suite/binlog/t/binlog_killed.test 2009-11-18 14:50:31 +0000 @@ -1,5 +1,5 @@ -- source include/have_innodb.inc --- source include/have_binlog_format_mixed_or_statement.inc +-- source include/have_binlog_format_statement.inc # You cannot use `KILL' with the Embedded MySQL Server library, # because the embedded server merely runs inside the threads of the host === modified file 'mysql-test/suite/binlog/t/binlog_stm_mix_innodb_myisam.test' --- a/mysql-test/suite/binlog/t/binlog_stm_mix_innodb_myisam.test 2008-02-28 11:21:44 +0000 +++ b/mysql-test/suite/binlog/t/binlog_stm_mix_innodb_myisam.test 2009-11-18 14:50:31 +0000 @@ -2,6 +2,9 @@ # For both statement and row based bin logs 9/19/2005 [jbm] -- source include/have_binlog_format_statement.inc + +CALL mtr.add_suppression("Statement may not be safe to log in statement format."); + -- source extra/binlog_tests/mix_innodb_myisam_binlog.test set @@session.binlog_format=statement; === modified file 'mysql-test/suite/binlog/t/binlog_stm_row.test' --- a/mysql-test/suite/binlog/t/binlog_stm_row.test 2009-02-19 09:01:25 +0000 +++ b/mysql-test/suite/binlog/t/binlog_stm_row.test 2010-01-15 15:27:55 +0000 @@ -1,5 +1,8 @@ --source include/have_log_bin.inc ---source include/have_binlog_format_row_or_statement.inc +# Test sets its own binlog_format, so we restrict it to run only once +--source include/have_binlog_format_row.inc + +CALL mtr.add_suppression("Statement may not be safe to log in statement format."); # Get rid of previous tests binlog --disable_query_log === modified file 'mysql-test/suite/binlog/t/binlog_unsafe.test' --- a/mysql-test/suite/binlog/t/binlog_unsafe.test 2009-09-07 20:50:10 +0000 +++ b/mysql-test/suite/binlog/t/binlog_unsafe.test 2010-01-19 10:36:52 +0000 @@ -388,4 +388,57 @@ DELETE FROM t1 LIMIT 1; DROP TABLE t1, t2; SET @@SESSION.SQL_MODE = @save_sql_mode; + +# +# BUG#47995: Mark user functions as unsafe +# +# Test that the system functions that are supposed to be marked unsafe +# generate a warning. Each INSERT statement below should generate a +# warning. +# + +CREATE TABLE t1 (a VARCHAR(1000)); +INSERT INTO t1 VALUES (CURRENT_USER()); #marked unsafe before BUG#47995 +INSERT INTO t1 VALUES (FOUND_ROWS()); #marked unsafe before BUG#47995 +INSERT INTO t1 VALUES (GET_LOCK('tmp', 1)); +INSERT INTO t1 VALUES (IS_FREE_LOCK('tmp')); +INSERT INTO t1 VALUES (IS_USED_LOCK('tmp')); +INSERT INTO t1 VALUES (LOAD_FILE('../../std_data/words2.dat')); #marked unsafe before BUG#47995 +INSERT INTO t1 VALUES (MASTER_POS_WAIT('dummy arg', 4711, 1)); +INSERT INTO t1 VALUES (RELEASE_LOCK('tmp')); +INSERT INTO t1 VALUES (ROW_COUNT()); #marked unsafe before BUG#47995 +INSERT INTO t1 VALUES (SESSION_USER()); #marked unsafe before BUG#47995 +INSERT INTO t1 VALUES (SLEEP(1)); +INSERT INTO t1 VALUES (SYSDATE()); +INSERT INTO t1 VALUES (SYSTEM_USER()); #marked unsafe before BUG#47995 +INSERT INTO t1 VALUES (USER()); #marked unsafe before BUG#47995 +INSERT INTO t1 VALUES (UUID()); #marked unsafe before BUG#47995 +INSERT INTO t1 VALUES (UUID_SHORT()); #marked unsafe before BUG#47995 +INSERT INTO t1 VALUES (VERSION()); +DELETE FROM t1; + +# Since we replicate the TIMESTAMP variable, functions affected by the +# TIMESTAMP variable are safe to replicate. So we check that the +# following following functions depend on the TIMESTAMP variable and +# don't generate a warning. + +SET TIME_ZONE= '+03:00'; +SET TIMESTAMP=1000000; +INSERT INTO t1 VALUES + (CURDATE()), + (CURRENT_DATE()), + (CURRENT_TIME()), + (CURRENT_TIMESTAMP()), + (CURTIME()), + (LOCALTIME()), + (LOCALTIMESTAMP()), + (NOW()), + (UNIX_TIMESTAMP()), + (UTC_DATE()), + (UTC_TIME()), + (UTC_TIMESTAMP()); +SELECT * FROM t1; + +DROP TABLE t1; + --echo "End of tests" === modified file 'mysql-test/suite/funcs_1/r/is_columns_is.result' --- a/mysql-test/suite/funcs_1/r/is_columns_is.result 2009-10-10 09:59:06 +0000 +++ b/mysql-test/suite/funcs_1/r/is_columns_is.result 2010-01-16 05:12:57 +0000 @@ -127,7 +127,7 @@ NULL information_schema INNODB_BUFFER_PO NULL information_schema INNODB_BUFFER_POOL_PAGES_BLOB page_no 2 0 NO bigint NULL NULL 19 0 NULL NULL bigint(21) unsigned select NULL information_schema INNODB_BUFFER_POOL_PAGES_BLOB part_len 4 0 NO bigint NULL NULL 19 0 NULL NULL bigint(21) unsigned select NULL information_schema INNODB_BUFFER_POOL_PAGES_BLOB space_id 1 0 NO bigint NULL NULL 19 0 NULL NULL bigint(21) unsigned select -NULL information_schema INNODB_BUFFER_POOL_PAGES_INDEX accessed 9 0 NO bigint NULL NULL 19 0 NULL NULL bigint(21) unsigned select +NULL information_schema INNODB_BUFFER_POOL_PAGES_INDEX access_time 9 0 NO bigint NULL NULL 19 0 NULL NULL bigint(21) unsigned select NULL information_schema INNODB_BUFFER_POOL_PAGES_INDEX data_size 7 0 NO bigint NULL NULL 19 0 NULL NULL bigint(21) unsigned select NULL information_schema INNODB_BUFFER_POOL_PAGES_INDEX dirty 11 0 NO bigint NULL NULL 19 0 NULL NULL bigint(21) unsigned select NULL information_schema INNODB_BUFFER_POOL_PAGES_INDEX fix_count 14 0 NO bigint NULL NULL 19 0 NULL NULL bigint(21) unsigned select @@ -394,6 +394,7 @@ NULL information_schema VIEWS TABLE_CATA NULL information_schema VIEWS TABLE_NAME 3 NO varchar 64 192 NULL NULL utf8 utf8_general_ci varchar(64) select NULL information_schema VIEWS TABLE_SCHEMA 2 NO varchar 64 192 NULL NULL utf8 utf8_general_ci varchar(64) select NULL information_schema VIEWS VIEW_DEFINITION 4 NULL NO longtext 4294967295 4294967295 NULL NULL utf8 utf8_general_ci longtext select +NULL information_schema XTRADB_ADMIN_COMMAND result_message 1 NO varchar 1024 3072 NULL NULL utf8 utf8_general_ci varchar(1024) select NULL information_schema XTRADB_ENHANCEMENTS comment 3 NO varchar 100 300 NULL NULL utf8 utf8_general_ci varchar(100) select NULL information_schema XTRADB_ENHANCEMENTS description 2 NO varchar 255 765 NULL NULL utf8 utf8_general_ci varchar(255) select NULL information_schema XTRADB_ENHANCEMENTS link 4 NO varchar 255 765 NULL NULL utf8 utf8_general_ci varchar(255) select @@ -589,7 +590,7 @@ NULL information_schema INNODB_BUFFER_PO NULL information_schema INNODB_BUFFER_POOL_PAGES_INDEX n_recs bigint NULL NULL NULL NULL bigint(21) unsigned NULL information_schema INNODB_BUFFER_POOL_PAGES_INDEX data_size bigint NULL NULL NULL NULL bigint(21) unsigned NULL information_schema INNODB_BUFFER_POOL_PAGES_INDEX hashed bigint NULL NULL NULL NULL bigint(21) unsigned -NULL information_schema INNODB_BUFFER_POOL_PAGES_INDEX accessed bigint NULL NULL NULL NULL bigint(21) unsigned +NULL information_schema INNODB_BUFFER_POOL_PAGES_INDEX access_time bigint NULL NULL NULL NULL bigint(21) unsigned NULL information_schema INNODB_BUFFER_POOL_PAGES_INDEX modified bigint NULL NULL NULL NULL bigint(21) unsigned NULL information_schema INNODB_BUFFER_POOL_PAGES_INDEX dirty bigint NULL NULL NULL NULL bigint(21) unsigned NULL information_schema INNODB_BUFFER_POOL_PAGES_INDEX old bigint NULL NULL NULL NULL bigint(21) unsigned @@ -848,6 +849,7 @@ NULL information_schema TRIGGERS CREATED 3.0000 information_schema VIEWS SECURITY_TYPE varchar 7 21 utf8 utf8_general_ci varchar(7) 3.0000 information_schema VIEWS CHARACTER_SET_CLIENT varchar 32 96 utf8 utf8_general_ci varchar(32) 3.0000 information_schema VIEWS COLLATION_CONNECTION varchar 32 96 utf8 utf8_general_ci varchar(32) +3.0000 information_schema XTRADB_ADMIN_COMMAND result_message varchar 1024 3072 utf8 utf8_general_ci varchar(1024) 3.0000 information_schema XTRADB_ENHANCEMENTS name varchar 255 765 utf8 utf8_general_ci varchar(255) 3.0000 information_schema XTRADB_ENHANCEMENTS description varchar 255 765 utf8 utf8_general_ci varchar(255) 3.0000 information_schema XTRADB_ENHANCEMENTS comment varchar 100 300 utf8 utf8_general_ci varchar(100) === modified file 'mysql-test/suite/funcs_1/r/is_tables_is.result' --- a/mysql-test/suite/funcs_1/r/is_tables_is.result 2009-10-10 09:59:06 +0000 +++ b/mysql-test/suite/funcs_1/r/is_tables_is.result 2010-01-16 05:12:57 +0000 @@ -958,6 +958,29 @@ user_comment Separator ----------------------------------------------------- TABLE_CATALOG NULL TABLE_SCHEMA information_schema +TABLE_NAME XTRADB_ADMIN_COMMAND +TABLE_TYPE SYSTEM VIEW +ENGINE MEMORY +VERSION 10 +ROW_FORMAT Fixed +TABLE_ROWS #TBLR# +AVG_ROW_LENGTH #ARL# +DATA_LENGTH #DL# +MAX_DATA_LENGTH #MDL# +INDEX_LENGTH #IL# +DATA_FREE #DF# +AUTO_INCREMENT NULL +CREATE_TIME #CRT# +UPDATE_TIME #UT# +CHECK_TIME #CT# +TABLE_COLLATION utf8_general_ci +CHECKSUM NULL +CREATE_OPTIONS #CO# +TABLE_COMMENT #TC# +user_comment +Separator ----------------------------------------------------- +TABLE_CATALOG NULL +TABLE_SCHEMA information_schema TABLE_NAME XTRADB_ENHANCEMENTS TABLE_TYPE SYSTEM VIEW ENGINE MEMORY @@ -1941,6 +1964,29 @@ user_comment Separator ----------------------------------------------------- TABLE_CATALOG NULL TABLE_SCHEMA information_schema +TABLE_NAME XTRADB_ADMIN_COMMAND +TABLE_TYPE SYSTEM VIEW +ENGINE MEMORY +VERSION 10 +ROW_FORMAT Fixed +TABLE_ROWS #TBLR# +AVG_ROW_LENGTH #ARL# +DATA_LENGTH #DL# +MAX_DATA_LENGTH #MDL# +INDEX_LENGTH #IL# +DATA_FREE #DF# +AUTO_INCREMENT NULL +CREATE_TIME #CRT# +UPDATE_TIME #UT# +CHECK_TIME #CT# +TABLE_COLLATION utf8_general_ci +CHECKSUM NULL +CREATE_OPTIONS #CO# +TABLE_COMMENT #TC# +user_comment +Separator ----------------------------------------------------- +TABLE_CATALOG NULL +TABLE_SCHEMA information_schema TABLE_NAME XTRADB_ENHANCEMENTS TABLE_TYPE SYSTEM VIEW ENGINE MEMORY === modified file 'mysql-test/suite/innodb/r/innodb-index.result' --- a/mysql-test/suite/innodb/r/innodb-index.result 2009-06-10 13:51:20 +0000 +++ b/mysql-test/suite/innodb/r/innodb-index.result 2009-11-30 12:49:13 +0000 @@ -968,6 +968,7 @@ create index t1u on t1 (u(1)); drop table t1; set global innodb_file_per_table=0; set global innodb_file_format=Antelope; +set global innodb_file_format_check=Antelope; SET @OLD_UNIQUE_CHECKS=@@UNIQUE_CHECKS, UNIQUE_CHECKS=0; SET @OLD_FOREIGN_KEY_CHECKS=@@FOREIGN_KEY_CHECKS, FOREIGN_KEY_CHECKS=0; CREATE TABLE t1( === added file 'mysql-test/suite/innodb/r/innodb_bug46676.result' --- a/mysql-test/suite/innodb/r/innodb_bug46676.result 1970-01-01 00:00:00 +0000 +++ b/mysql-test/suite/innodb/r/innodb_bug46676.result 2009-11-30 12:24:54 +0000 @@ -0,0 +1,9 @@ +SET foreign_key_checks=0; +CREATE TABLE t1 (id int, foreign key (id) references t2(id)) ENGINE=INNODB; +CREATE TABLE t2 (id int, foreign key (id) references t1(id)) ENGINE=INNODB; +SET foreign_key_checks=1; +SELECT COUNT(*) FROM information_schema.key_column_usage WHERE REFERENCED_TABLE_NAME in ('t1', 't2'); +COUNT(*) +2 +SET foreign_key_checks=0; +DROP TABLE t1, t2; === added file 'mysql-test/suite/innodb/r/innodb_bug47167.result' --- a/mysql-test/suite/innodb/r/innodb_bug47167.result 1970-01-01 00:00:00 +0000 +++ b/mysql-test/suite/innodb/r/innodb_bug47167.result 2009-11-30 11:56:21 +0000 @@ -0,0 +1,24 @@ +set @old_innodb_file_format_check=@@innodb_file_format_check; +select @old_innodb_file_format_check; +@old_innodb_file_format_check +Antelope +set global innodb_file_format_check = Barracuda; +select @@innodb_file_format_check; +@@innodb_file_format_check +Barracuda +set global innodb_file_format_check = DEFAULT; +select @@innodb_file_format_check; +@@innodb_file_format_check +Barracuda +set global innodb_file_format_check = @old_innodb_file_format_check; +select @@innodb_file_format_check; +@@innodb_file_format_check +Antelope +set global innodb_file_format_check = cheetah; +ERROR HY000: Incorrect arguments to SET +set global innodb_file_format_check = Bear; +ERROR HY000: Incorrect arguments to SET +set global innodb_file_format_check = on; +ERROR HY000: Incorrect arguments to SET +set global innodb_file_format_check = off; +ERROR HY000: Incorrect arguments to SET === modified file 'mysql-test/suite/innodb/t/innodb-consistent-master.opt' --- a/mysql-test/suite/innodb/t/innodb-consistent-master.opt 2009-10-09 13:37:47 +0000 +++ b/mysql-test/suite/innodb/t/innodb-consistent-master.opt 2009-11-30 12:49:13 +0000 @@ -1 +1 @@ ---innodb_lock_wait_timeout=2 +--loose-innodb_lock_wait_timeout=2 === modified file 'mysql-test/suite/innodb/t/innodb-index.test' --- a/mysql-test/suite/innodb/t/innodb-index.test 2009-06-11 12:57:44 +0000 +++ b/mysql-test/suite/innodb/t/innodb-index.test 2009-11-30 12:49:13 +0000 @@ -1,5 +1,7 @@ -- source include/have_innodb.inc +let $innodb_file_format_check_orig=`select @@innodb_file_format_check`; + create table t1(a int not null, b int, c char(10) not null, d varchar(20)) engine = innodb; insert into t1 values (5,5,'oo','oo'),(4,4,'tr','tr'),(3,4,'ad','ad'),(2,3,'ak','ak'); commit; @@ -398,6 +400,7 @@ create index t1u on t1 (u(1)); drop table t1; eval set global innodb_file_per_table=$per_table; eval set global innodb_file_format=$format; +eval set global innodb_file_format_check=$format; # # Test to check whether CREATE INDEX handles implicit foreign key @@ -532,3 +535,10 @@ disconnect a; disconnect b; DROP TABLE t1; + +# +# restore environment to the state it was before this test execution +# + +-- disable_query_log +eval SET GLOBAL innodb_file_format_check=$innodb_file_format_check_orig; === added file 'mysql-test/suite/innodb/t/innodb_bug46676.test' --- a/mysql-test/suite/innodb/t/innodb_bug46676.test 1970-01-01 00:00:00 +0000 +++ b/mysql-test/suite/innodb/t/innodb_bug46676.test 2009-11-30 12:24:54 +0000 @@ -0,0 +1,16 @@ +# This is the test for bug 46676: mysqld got exception 0xc0000005 +# It is reproducible with InnoDB plugin 1.0.4 + MySQL 5.1.37. +# But no longer reproducible after MySQL 5.1.38 (with plugin 1.0.5). + +--source include/have_innodb.inc + +SET foreign_key_checks=0; +CREATE TABLE t1 (id int, foreign key (id) references t2(id)) ENGINE=INNODB; +CREATE TABLE t2 (id int, foreign key (id) references t1(id)) ENGINE=INNODB; +SET foreign_key_checks=1; + +# Server crashes +SELECT COUNT(*) FROM information_schema.key_column_usage WHERE REFERENCED_TABLE_NAME in ('t1', 't2'); + +SET foreign_key_checks=0; +DROP TABLE t1, t2; === added file 'mysql-test/suite/innodb/t/innodb_bug47167.test' --- a/mysql-test/suite/innodb/t/innodb_bug47167.test 1970-01-01 00:00:00 +0000 +++ b/mysql-test/suite/innodb/t/innodb_bug47167.test 2009-11-30 11:56:21 +0000 @@ -0,0 +1,46 @@ +# This is the unit test for bug *47167. +# It tests setting the global variable +# "innodb_file_format_check" with a +# user-Defined Variable. + +--source include/have_innodb.inc +-- source suite/innodb/include/have_innodb_plugin.inc + +# Save the value (Antelope) in 'innodb_file_format_check' to +# 'old_innodb_file_format_check' +set @old_innodb_file_format_check=@@innodb_file_format_check; + +# @old_innodb_file_format_check shall have the value of 'Antelope' +select @old_innodb_file_format_check; + +# Reset the value in 'innodb_file_format_check' to 'Barracuda' +set global innodb_file_format_check = Barracuda; + +select @@innodb_file_format_check; + +# Set 'innodb_file_format_check' to its default value, which +# is the latest file format supported in the current release. +set global innodb_file_format_check = DEFAULT; + +select @@innodb_file_format_check; + +# Put the saved value back to 'innodb_file_format_check' +set global innodb_file_format_check = @old_innodb_file_format_check; + +# Check whether 'innodb_file_format_check' get its original value. +select @@innodb_file_format_check; + +# Following are negative tests, all should fail. +--disable_warnings +--error ER_WRONG_ARGUMENTS +set global innodb_file_format_check = cheetah; + +--error ER_WRONG_ARGUMENTS +set global innodb_file_format_check = Bear; + +--error ER_WRONG_ARGUMENTS +set global innodb_file_format_check = on; + +--error ER_WRONG_ARGUMENTS +set global innodb_file_format_check = off; +--enable_warnings === removed file 'mysql-test/suite/maria/t/maria2-master.opt' --- a/mysql-test/suite/maria/t/maria2-master.opt 2009-02-15 10:58:34 +0000 +++ b/mysql-test/suite/maria/t/maria2-master.opt 1970-01-01 00:00:00 +0000 @@ -1,2 +0,0 @@ ---secure-file-priv="" - === modified file 'mysql-test/suite/parts/t/partition_alter1_2_innodb.test' --- a/mysql-test/suite/parts/t/partition_alter1_2_innodb.test 2009-10-09 13:08:09 +0000 +++ b/mysql-test/suite/parts/t/partition_alter1_2_innodb.test 2010-01-15 15:27:55 +0000 @@ -28,6 +28,8 @@ #------------------------------------------------------------------------------# # General not engine specific settings and requirements +--source include/big_test.inc + ##### Options, for debugging support ##### let $debug= 0; let $with_partitioning= 1; === modified file 'mysql-test/suite/parts/t/partition_alter2_1_innodb.test' --- a/mysql-test/suite/parts/t/partition_alter2_1_innodb.test 2009-10-09 13:08:09 +0000 +++ b/mysql-test/suite/parts/t/partition_alter2_1_innodb.test 2010-01-15 15:27:55 +0000 @@ -22,6 +22,8 @@ # any of the variables. # +--source include/big_test.inc + #------------------------------------------------------------------------------# # General not engine specific settings and requirements === modified file 'mysql-test/suite/parts/t/partition_alter2_2_innodb.test' --- a/mysql-test/suite/parts/t/partition_alter2_2_innodb.test 2009-10-09 13:08:09 +0000 +++ b/mysql-test/suite/parts/t/partition_alter2_2_innodb.test 2010-01-15 15:27:55 +0000 @@ -22,6 +22,8 @@ # any of the variables. # +--source include/big_test.inc + #------------------------------------------------------------------------------# # General not engine specific settings and requirements === modified file 'mysql-test/suite/parts/t/partition_alter4_innodb.test' --- a/mysql-test/suite/parts/t/partition_alter4_innodb.test 2009-10-09 13:08:09 +0000 +++ b/mysql-test/suite/parts/t/partition_alter4_innodb.test 2010-01-15 15:27:55 +0000 @@ -22,6 +22,8 @@ # any of the variables. # +--source include/big_test.inc + #------------------------------------------------------------------------------# # General not engine specific settings and requirements === modified file 'mysql-test/suite/pbxt/r/func_group.result' --- a/mysql-test/suite/pbxt/r/func_group.result 2009-11-24 10:19:08 +0000 +++ b/mysql-test/suite/pbxt/r/func_group.result 2010-01-16 05:12:57 +0000 @@ -885,7 +885,7 @@ cast(sum(distinct df) as signed) 3 select cast(min(df) as signed) from t1; cast(min(df) as signed) -0 +1 select 1e8 * sum(distinct df) from t1; 1e8 * sum(distinct df) 330000000 === modified file 'mysql-test/suite/pbxt/r/mysqlshow.result' --- a/mysql-test/suite/pbxt/r/mysqlshow.result 2009-08-17 15:57:58 +0000 +++ b/mysql-test/suite/pbxt/r/mysqlshow.result 2010-01-16 05:12:57 +0000 @@ -113,7 +113,7 @@ Database: information_schema | INNODB_RSEG | | XTRADB_ENHANCEMENTS | | INNODB_BUFFER_POOL_PAGES_INDEX | -| INNODB_INDEX_STATS | +| XTRADB_ADMIN_COMMAND | | INNODB_TRX | | INNODB_CMP_RESET | | INNODB_LOCK_WAITS | @@ -122,6 +122,7 @@ Database: information_schema | INNODB_CMPMEM | | INNODB_TABLE_STATS | | INNODB_BUFFER_POOL_PAGES_BLOB | +| INNODB_INDEX_STATS | +---------------------------------------+ Database: INFORMATION_SCHEMA +---------------------------------------+ @@ -161,7 +162,7 @@ Database: INFORMATION_SCHEMA | INNODB_RSEG | | XTRADB_ENHANCEMENTS | | INNODB_BUFFER_POOL_PAGES_INDEX | -| INNODB_INDEX_STATS | +| XTRADB_ADMIN_COMMAND | | INNODB_TRX | | INNODB_CMP_RESET | | INNODB_LOCK_WAITS | @@ -170,6 +171,7 @@ Database: INFORMATION_SCHEMA | INNODB_CMPMEM | | INNODB_TABLE_STATS | | INNODB_BUFFER_POOL_PAGES_BLOB | +| INNODB_INDEX_STATS | +---------------------------------------+ Wildcard: inf_rmation_schema +--------------------+ === modified file 'mysql-test/suite/rpl/r/rpl_err_ignoredtable.result' --- a/mysql-test/suite/rpl/r/rpl_err_ignoredtable.result 2007-06-27 12:28:02 +0000 +++ b/mysql-test/suite/rpl/r/rpl_err_ignoredtable.result 2009-11-18 14:50:31 +0000 @@ -4,6 +4,7 @@ reset master; reset slave; drop table if exists t1,t2,t3,t4,t5,t6,t7,t8,t9; start slave; +CALL mtr.add_suppression("Statement may not be safe to log in statement format."); create table t1 (a int primary key); create table t4 (a int primary key); insert into t1 values (1),(1); === modified file 'mysql-test/suite/rpl/r/rpl_extraCol_innodb.result' --- a/mysql-test/suite/rpl/r/rpl_extraCol_innodb.result 2009-08-28 14:13:27 +0000 +++ b/mysql-test/suite/rpl/r/rpl_extraCol_innodb.result 2009-10-22 00:10:42 +0000 @@ -404,7 +404,11 @@ STOP SLAVE; RESET SLAVE; CREATE TABLE t9 (a INT KEY, b BLOB, c CHAR(5), d TIMESTAMP, -e INT NOT NULL) ENGINE='InnoDB'; +e INT NOT NULL, +f text not null, +g text, +h blob not null, +i blob) ENGINE='InnoDB'; *** Create t9 on Master *** CREATE TABLE t9 (a INT PRIMARY KEY, b BLOB, c CHAR(5) ) ENGINE='InnoDB'; @@ -415,47 +419,11 @@ START SLAVE; set @b1 = 'b1b1b1b1'; set @b1 = concat(@b1,@b1); INSERT INTO t9 () VALUES(1,@b1,'Kyle'),(2,@b1,'JOE'),(3,@b1,'QA'); -SHOW SLAVE STATUS; -Slave_IO_State # -Master_Host 127.0.0.1 -Master_User root -Master_Port # -Connect_Retry 1 -Master_Log_File master-bin.000001 -Read_Master_Log_Pos # -Relay_Log_File # -Relay_Log_Pos # -Relay_Master_Log_File master-bin.000001 -Slave_IO_Running Yes -Slave_SQL_Running No -Replicate_Do_DB -Replicate_Ignore_DB -Replicate_Do_Table -Replicate_Ignore_Table # -Replicate_Wild_Do_Table -Replicate_Wild_Ignore_Table -Last_Errno 1364 -Last_Error Could not execute Write_rows event on table test.t9; Field 'e' doesn't have a default value, Error_code: 1364; handler error HA_ERR_ROWS_EVENT_APPLY; the event's master log master-bin.000001, end_log_pos 330 -Skip_Counter 0 -Exec_Master_Log_Pos # -Relay_Log_Space # -Until_Condition None -Until_Log_File -Until_Log_Pos 0 -Master_SSL_Allowed No -Master_SSL_CA_File -Master_SSL_CA_Path -Master_SSL_Cert -Master_SSL_Cipher -Master_SSL_Key -Seconds_Behind_Master # -Master_SSL_Verify_Server_Cert No -Last_IO_Errno # -Last_IO_Error # -Last_SQL_Errno 1364 -Last_SQL_Error Could not execute Write_rows event on table test.t9; Field 'e' doesn't have a default value, Error_code: 1364; handler error HA_ERR_ROWS_EVENT_APPLY; the event's master log master-bin.000001, end_log_pos 330 -SET GLOBAL SQL_SLAVE_SKIP_COUNTER=2; -START SLAVE; +select * from t9; +a b c d e f g h i +1 b1b1b1b1b1b1b1b1 Kyle 0000-00-00 00:00:00 0 NULL NULL +2 b1b1b1b1b1b1b1b1 JOE 0000-00-00 00:00:00 0 NULL NULL +3 b1b1b1b1b1b1b1b1 QA 0000-00-00 00:00:00 0 NULL NULL *** Create t10 on slave *** STOP SLAVE; RESET SLAVE; === modified file 'mysql-test/suite/rpl/r/rpl_extraCol_myisam.result' --- a/mysql-test/suite/rpl/r/rpl_extraCol_myisam.result 2009-08-28 14:13:27 +0000 +++ b/mysql-test/suite/rpl/r/rpl_extraCol_myisam.result 2009-10-22 00:10:42 +0000 @@ -404,7 +404,11 @@ STOP SLAVE; RESET SLAVE; CREATE TABLE t9 (a INT KEY, b BLOB, c CHAR(5), d TIMESTAMP, -e INT NOT NULL) ENGINE='MyISAM'; +e INT NOT NULL, +f text not null, +g text, +h blob not null, +i blob) ENGINE='MyISAM'; *** Create t9 on Master *** CREATE TABLE t9 (a INT PRIMARY KEY, b BLOB, c CHAR(5) ) ENGINE='MyISAM'; @@ -415,47 +419,11 @@ START SLAVE; set @b1 = 'b1b1b1b1'; set @b1 = concat(@b1,@b1); INSERT INTO t9 () VALUES(1,@b1,'Kyle'),(2,@b1,'JOE'),(3,@b1,'QA'); -SHOW SLAVE STATUS; -Slave_IO_State # -Master_Host 127.0.0.1 -Master_User root -Master_Port # -Connect_Retry 1 -Master_Log_File master-bin.000001 -Read_Master_Log_Pos # -Relay_Log_File # -Relay_Log_Pos # -Relay_Master_Log_File master-bin.000001 -Slave_IO_Running Yes -Slave_SQL_Running No -Replicate_Do_DB -Replicate_Ignore_DB -Replicate_Do_Table -Replicate_Ignore_Table # -Replicate_Wild_Do_Table -Replicate_Wild_Ignore_Table -Last_Errno 1364 -Last_Error Could not execute Write_rows event on table test.t9; Field 'e' doesn't have a default value, Error_code: 1364; handler error HA_ERR_ROWS_EVENT_APPLY; the event's master log master-bin.000001, end_log_pos 330 -Skip_Counter 0 -Exec_Master_Log_Pos # -Relay_Log_Space # -Until_Condition None -Until_Log_File -Until_Log_Pos 0 -Master_SSL_Allowed No -Master_SSL_CA_File -Master_SSL_CA_Path -Master_SSL_Cert -Master_SSL_Cipher -Master_SSL_Key -Seconds_Behind_Master # -Master_SSL_Verify_Server_Cert No -Last_IO_Errno # -Last_IO_Error # -Last_SQL_Errno 1364 -Last_SQL_Error Could not execute Write_rows event on table test.t9; Field 'e' doesn't have a default value, Error_code: 1364; handler error HA_ERR_ROWS_EVENT_APPLY; the event's master log master-bin.000001, end_log_pos 330 -SET GLOBAL SQL_SLAVE_SKIP_COUNTER=2; -START SLAVE; +select * from t9; +a b c d e f g h i +1 b1b1b1b1b1b1b1b1 Kyle 0000-00-00 00:00:00 0 NULL NULL +2 b1b1b1b1b1b1b1b1 JOE 0000-00-00 00:00:00 0 NULL NULL +3 b1b1b1b1b1b1b1b1 QA 0000-00-00 00:00:00 0 NULL NULL *** Create t10 on slave *** STOP SLAVE; RESET SLAVE; === modified file 'mysql-test/suite/rpl/r/rpl_get_lock.result' --- a/mysql-test/suite/rpl/r/rpl_get_lock.result 2008-02-12 19:09:16 +0000 +++ b/mysql-test/suite/rpl/r/rpl_get_lock.result 2009-11-18 14:50:31 +0000 @@ -4,6 +4,7 @@ reset master; reset slave; drop table if exists t1,t2,t3,t4,t5,t6,t7,t8,t9; start slave; +CALL mtr.add_suppression("Statement may not be safe to log in statement format."); create table t1(n int); insert into t1 values(get_lock("lock",2)); select get_lock("lock",2); === added file 'mysql-test/suite/rpl/r/rpl_loaddata_symlink.result' --- a/mysql-test/suite/rpl/r/rpl_loaddata_symlink.result 1970-01-01 00:00:00 +0000 +++ b/mysql-test/suite/rpl/r/rpl_loaddata_symlink.result 2009-11-28 04:43:16 +0000 @@ -0,0 +1,17 @@ +stop slave; +drop table if exists t1,t2,t3,t4,t5,t6,t7,t8,t9; +reset master; +reset slave; +drop table if exists t1,t2,t3,t4,t5,t6,t7,t8,t9; +start slave; +create table t1(a int not null auto_increment, b int, primary key(a) ); +load data infile '../../std_data/rpl_loaddata.dat' into table t1; +select * from t1; +a b +1 10 +2 15 +select * from t1; +a b +1 10 +2 15 +drop table t1; === modified file 'mysql-test/suite/rpl/r/rpl_mixed_ddl_dml.result' --- a/mysql-test/suite/rpl/r/rpl_mixed_ddl_dml.result 2008-11-04 17:07:14 +0000 +++ b/mysql-test/suite/rpl/r/rpl_mixed_ddl_dml.result 2010-01-19 10:36:52 +0000 @@ -18,6 +18,7 @@ Server_id Host Port Rpl_recovery_rank Ma drop table t1; stop slave; create table t2(id int auto_increment primary key, created datetime); +SET TIME_ZONE= '+03:00'; set timestamp=12345; insert into t2 set created=now(); select * from t2; === added file 'mysql-test/suite/rpl/r/rpl_nondeterministic_functions.result' --- a/mysql-test/suite/rpl/r/rpl_nondeterministic_functions.result 1970-01-01 00:00:00 +0000 +++ b/mysql-test/suite/rpl/r/rpl_nondeterministic_functions.result 2009-11-18 14:50:31 +0000 @@ -0,0 +1,26 @@ +stop slave; +drop table if exists t1,t2,t3,t4,t5,t6,t7,t8,t9; +reset master; +reset slave; +drop table if exists t1,t2,t3,t4,t5,t6,t7,t8,t9; +start slave; +CREATE TABLE t1 (a VARCHAR(1000)); +INSERT INTO t1 VALUES (CONNECTION_ID()); +INSERT INTO t1 VALUES (CONNECTION_ID()); +INSERT INTO t1 VALUES +(CURDATE()), +(CURRENT_DATE()), +(CURRENT_TIME()), +(CURRENT_TIMESTAMP()), +(CURTIME()), +(LOCALTIME()), +(LOCALTIMESTAMP()), +(NOW()), +(UNIX_TIMESTAMP()), +(UTC_DATE()), +(UTC_TIME()), +(UTC_TIMESTAMP()); +INSERT INTO t1 VALUES (RAND()); +INSERT INTO t1 VALUES (LAST_INSERT_ID()); +Comparing tables master:test.t1 and slave:test.t1 +DROP TABLE t1; === added file 'mysql-test/suite/rpl/r/rpl_not_null_innodb.result' --- a/mysql-test/suite/rpl/r/rpl_not_null_innodb.result 1970-01-01 00:00:00 +0000 +++ b/mysql-test/suite/rpl/r/rpl_not_null_innodb.result 2009-10-22 00:19:52 +0000 @@ -0,0 +1,202 @@ +stop slave; +drop table if exists t1,t2,t3,t4,t5,t6,t7,t8,t9; +reset master; +reset slave; +drop table if exists t1,t2,t3,t4,t5,t6,t7,t8,t9; +start slave; +SET SQL_LOG_BIN= 0; +CREATE TABLE t1(`a` INT, `b` DATE DEFAULT NULL, +`c` INT DEFAULT NULL, +PRIMARY KEY(`a`)) ENGINE=Innodb DEFAULT CHARSET=LATIN1; +CREATE TABLE t2(`a` INT, `b` DATE DEFAULT NULL, +PRIMARY KEY(`a`)) ENGINE=Innodb DEFAULT CHARSET=LATIN1; +CREATE TABLE t3(`a` INT, `b` DATE DEFAULT NULL, +PRIMARY KEY(`a`)) ENGINE=Innodb DEFAULT CHARSET=LATIN1; +CREATE TABLE t4(`a` INT, `b` DATE DEFAULT NULL, +`c` INT DEFAULT NULL, +PRIMARY KEY(`a`)) ENGINE=Innodb DEFAULT CHARSET=LATIN1; +SET SQL_LOG_BIN= 1; +CREATE TABLE t1(`a` INT, `b` DATE DEFAULT NULL, +`c` INT DEFAULT NULL, +PRIMARY KEY(`a`)) ENGINE=Innodb DEFAULT CHARSET=LATIN1; +CREATE TABLE t2(`a` INT, `b` DATE DEFAULT NULL, +PRIMARY KEY(`a`)) ENGINE=Innodb DEFAULT CHARSET=LATIN1; +CREATE TABLE t3(`a` INT, `b` DATE DEFAULT '0000-00-00', +`c` INT DEFAULT 500, +PRIMARY KEY(`a`)) ENGINE=Innodb DEFAULT CHARSET=LATIN1; +CREATE TABLE t4(`a` INT, `b` DATE DEFAULT '0000-00-00', +PRIMARY KEY(`a`)) ENGINE=Innodb DEFAULT CHARSET=LATIN1; +************* EXECUTION WITH INSERTS ************* +INSERT INTO t1(a,b,c) VALUES (1, null, 1); +INSERT INTO t1(a,b,c) VALUES (2,'1111-11-11', 2); +INSERT INTO t1(a,b) VALUES (3, null); +INSERT INTO t1(a,c) VALUES (4, 4); +INSERT INTO t1(a) VALUES (5); +INSERT INTO t2(a,b) VALUES (1, null); +INSERT INTO t2(a,b) VALUES (2,'1111-11-11'); +INSERT INTO t2(a) VALUES (3); +INSERT INTO t3(a,b) VALUES (1, null); +INSERT INTO t3(a,b) VALUES (2,'1111-11-11'); +INSERT INTO t3(a) VALUES (3); +INSERT INTO t4(a,b,c) VALUES (1, null, 1); +INSERT INTO t4(a,b,c) VALUES (2,'1111-11-11', 2); +INSERT INTO t4(a,b) VALUES (3, null); +INSERT INTO t4(a,c) VALUES (4, 4); +INSERT INTO t4(a) VALUES (5); +************* SHOWING THE RESULT SETS WITH INSERTS ************* +TABLES t1 and t2 must be equal otherwise an error will be thrown. +Comparing tables master:test.t1 and slave:test.t1 +Comparing tables master:test.t2 and slave:test.t2 +TABLES t2 and t3 must be different. +SELECT * FROM t3 ORDER BY a; +a b +1 NULL +2 1111-11-11 +3 NULL +SELECT * FROM t3 ORDER BY a; +a b c +1 NULL 500 +2 1111-11-11 500 +3 NULL 500 +SELECT * FROM t4 ORDER BY a; +a b c +1 NULL 1 +2 1111-11-11 2 +3 NULL NULL +4 NULL 4 +5 NULL NULL +SELECT * FROM t4 ORDER BY a; +a b +1 NULL +2 1111-11-11 +3 NULL +4 NULL +5 NULL +************* EXECUTION WITH UPDATES and REPLACES ************* +DELETE FROM t1; +INSERT INTO t1(a,b,c) VALUES (1,'1111-11-11', 1); +REPLACE INTO t1(a,b,c) VALUES (2,'1111-11-11', 2); +UPDATE t1 set b= NULL, c= 300 where a= 1; +REPLACE INTO t1(a,b,c) VALUES (2, NULL, 300); +************* SHOWING THE RESULT SETS WITH UPDATES and REPLACES ************* +TABLES t1 and t2 must be equal otherwise an error will be thrown. +Comparing tables master:test.t1 and slave:test.t1 +************* CLEANING ************* +DROP TABLE t1; +DROP TABLE t2; +DROP TABLE t3; +DROP TABLE t4; +SET SQL_LOG_BIN= 0; +CREATE TABLE t1 (`a` INT, `b` BIT DEFAULT NULL, `c` BIT DEFAULT NULL, +PRIMARY KEY (`a`)) ENGINE= Innodb; +SET SQL_LOG_BIN= 1; +CREATE TABLE t1 (`a` INT, `b` BIT DEFAULT b'01', `c` BIT DEFAULT NULL, +PRIMARY KEY (`a`)) ENGINE= Innodb; +************* EXECUTION WITH INSERTS ************* +INSERT INTO t1(a,b,c) VALUES (1, null, b'01'); +INSERT INTO t1(a,b,c) VALUES (2,b'00', b'01'); +INSERT INTO t1(a,b) VALUES (3, null); +INSERT INTO t1(a,c) VALUES (4, b'01'); +INSERT INTO t1(a) VALUES (5); +************* SHOWING THE RESULT SETS WITH INSERTS ************* +TABLES t1 and t2 must be different. +SELECT a,b+0,c+0 FROM t1 ORDER BY a; +a b+0 c+0 +1 NULL 1 +2 0 1 +3 NULL NULL +4 NULL 1 +5 NULL NULL +SELECT a,b+0,c+0 FROM t1 ORDER BY a; +a b+0 c+0 +1 NULL 1 +2 0 1 +3 NULL NULL +4 NULL 1 +5 NULL NULL +************* EXECUTION WITH UPDATES and REPLACES ************* +DELETE FROM t1; +INSERT INTO t1(a,b,c) VALUES (1,b'00', b'01'); +REPLACE INTO t1(a,b,c) VALUES (2,b'00',b'01'); +UPDATE t1 set b= NULL, c= b'00' where a= 1; +REPLACE INTO t1(a,b,c) VALUES (2, NULL, b'00'); +************* SHOWING THE RESULT SETS WITH UPDATES and REPLACES ************* +TABLES t1 and t2 must be equal otherwise an error will be thrown. +Comparing tables master:test.t1 and slave:test.t1 +DROP TABLE t1; +################################################################################ +# NULL ---> NOT NULL (STRICT MODE) +# UNCOMMENT THIS AFTER FIXING BUG#43992 +################################################################################ +################################################################################ +# NULL ---> NOT NULL (NON-STRICT MODE) +################################################################################ +SET SQL_LOG_BIN= 0; +CREATE TABLE t1(`a` INT NOT NULL, `b` INT, +PRIMARY KEY(`a`)) ENGINE=Innodb DEFAULT CHARSET=LATIN1; +CREATE TABLE t2(`a` INT NOT NULL, `b` INT, +PRIMARY KEY(`a`)) ENGINE=Innodb DEFAULT CHARSET=LATIN1; +CREATE TABLE t3(`a` INT NOT NULL, `b` INT, +PRIMARY KEY(`a`)) ENGINE=Innodb DEFAULT CHARSET=LATIN1; +SET SQL_LOG_BIN= 1; +CREATE TABLE t1(`a` INT NOT NULL, `b` INT NOT NULL, +`c` INT NOT NULL, +PRIMARY KEY(`a`)) ENGINE=Innodb DEFAULT CHARSET=LATIN1; +CREATE TABLE t2(`a` INT NOT NULL, `b` INT NOT NULL, +`c` INT, +PRIMARY KEY(`a`)) ENGINE=Innodb DEFAULT CHARSET=LATIN1; +CREATE TABLE t3(`a` INT NOT NULL, `b` INT NOT NULL, +`c` INT DEFAULT 500, +PRIMARY KEY(`a`)) ENGINE=Innodb DEFAULT CHARSET=LATIN1; +************* EXECUTION WITH INSERTS ************* +INSERT INTO t1(a) VALUES (1); +INSERT INTO t1(a, b) VALUES (2, NULL); +INSERT INTO t1(a, b) VALUES (3, 1); +INSERT INTO t2(a) VALUES (1); +INSERT INTO t2(a, b) VALUES (2, NULL); +INSERT INTO t2(a, b) VALUES (3, 1); +INSERT INTO t3(a) VALUES (1); +INSERT INTO t3(a, b) VALUES (2, NULL); +INSERT INTO t3(a, b) VALUES (3, 1); +INSERT INTO t3(a, b) VALUES (4, 1); +REPLACE INTO t3(a, b) VALUES (5, null); +REPLACE INTO t3(a, b) VALUES (3, null); +UPDATE t3 SET b = NULL where a = 4; +************* SHOWING THE RESULT SETS ************* +SELECT * FROM t1 ORDER BY a; +a b +1 NULL +2 NULL +3 1 +SELECT * FROM t1 ORDER BY a; +a b c +1 0 0 +2 0 0 +3 1 0 +SELECT * FROM t2 ORDER BY a; +a b +1 NULL +2 NULL +3 1 +SELECT * FROM t2 ORDER BY a; +a b c +1 0 NULL +2 0 NULL +3 1 NULL +SELECT * FROM t3 ORDER BY a; +a b +1 NULL +2 NULL +3 NULL +4 NULL +5 NULL +SELECT * FROM t3 ORDER BY a; +a b c +1 0 500 +2 0 500 +3 0 500 +4 0 500 +5 0 500 +DROP TABLE t1; +DROP TABLE t2; +DROP TABLE t3; === added file 'mysql-test/suite/rpl/r/rpl_not_null_myisam.result' --- a/mysql-test/suite/rpl/r/rpl_not_null_myisam.result 1970-01-01 00:00:00 +0000 +++ b/mysql-test/suite/rpl/r/rpl_not_null_myisam.result 2009-10-22 00:19:52 +0000 @@ -0,0 +1,202 @@ +stop slave; +drop table if exists t1,t2,t3,t4,t5,t6,t7,t8,t9; +reset master; +reset slave; +drop table if exists t1,t2,t3,t4,t5,t6,t7,t8,t9; +start slave; +SET SQL_LOG_BIN= 0; +CREATE TABLE t1(`a` INT, `b` DATE DEFAULT NULL, +`c` INT DEFAULT NULL, +PRIMARY KEY(`a`)) ENGINE=MyISAM DEFAULT CHARSET=LATIN1; +CREATE TABLE t2(`a` INT, `b` DATE DEFAULT NULL, +PRIMARY KEY(`a`)) ENGINE=MyISAM DEFAULT CHARSET=LATIN1; +CREATE TABLE t3(`a` INT, `b` DATE DEFAULT NULL, +PRIMARY KEY(`a`)) ENGINE=MyISAM DEFAULT CHARSET=LATIN1; +CREATE TABLE t4(`a` INT, `b` DATE DEFAULT NULL, +`c` INT DEFAULT NULL, +PRIMARY KEY(`a`)) ENGINE=MyISAM DEFAULT CHARSET=LATIN1; +SET SQL_LOG_BIN= 1; +CREATE TABLE t1(`a` INT, `b` DATE DEFAULT NULL, +`c` INT DEFAULT NULL, +PRIMARY KEY(`a`)) ENGINE=MyISAM DEFAULT CHARSET=LATIN1; +CREATE TABLE t2(`a` INT, `b` DATE DEFAULT NULL, +PRIMARY KEY(`a`)) ENGINE=MyISAM DEFAULT CHARSET=LATIN1; +CREATE TABLE t3(`a` INT, `b` DATE DEFAULT '0000-00-00', +`c` INT DEFAULT 500, +PRIMARY KEY(`a`)) ENGINE=MyISAM DEFAULT CHARSET=LATIN1; +CREATE TABLE t4(`a` INT, `b` DATE DEFAULT '0000-00-00', +PRIMARY KEY(`a`)) ENGINE=MyISAM DEFAULT CHARSET=LATIN1; +************* EXECUTION WITH INSERTS ************* +INSERT INTO t1(a,b,c) VALUES (1, null, 1); +INSERT INTO t1(a,b,c) VALUES (2,'1111-11-11', 2); +INSERT INTO t1(a,b) VALUES (3, null); +INSERT INTO t1(a,c) VALUES (4, 4); +INSERT INTO t1(a) VALUES (5); +INSERT INTO t2(a,b) VALUES (1, null); +INSERT INTO t2(a,b) VALUES (2,'1111-11-11'); +INSERT INTO t2(a) VALUES (3); +INSERT INTO t3(a,b) VALUES (1, null); +INSERT INTO t3(a,b) VALUES (2,'1111-11-11'); +INSERT INTO t3(a) VALUES (3); +INSERT INTO t4(a,b,c) VALUES (1, null, 1); +INSERT INTO t4(a,b,c) VALUES (2,'1111-11-11', 2); +INSERT INTO t4(a,b) VALUES (3, null); +INSERT INTO t4(a,c) VALUES (4, 4); +INSERT INTO t4(a) VALUES (5); +************* SHOWING THE RESULT SETS WITH INSERTS ************* +TABLES t1 and t2 must be equal otherwise an error will be thrown. +Comparing tables master:test.t1 and slave:test.t1 +Comparing tables master:test.t2 and slave:test.t2 +TABLES t2 and t3 must be different. +SELECT * FROM t3 ORDER BY a; +a b +1 NULL +2 1111-11-11 +3 NULL +SELECT * FROM t3 ORDER BY a; +a b c +1 NULL 500 +2 1111-11-11 500 +3 NULL 500 +SELECT * FROM t4 ORDER BY a; +a b c +1 NULL 1 +2 1111-11-11 2 +3 NULL NULL +4 NULL 4 +5 NULL NULL +SELECT * FROM t4 ORDER BY a; +a b +1 NULL +2 1111-11-11 +3 NULL +4 NULL +5 NULL +************* EXECUTION WITH UPDATES and REPLACES ************* +DELETE FROM t1; +INSERT INTO t1(a,b,c) VALUES (1,'1111-11-11', 1); +REPLACE INTO t1(a,b,c) VALUES (2,'1111-11-11', 2); +UPDATE t1 set b= NULL, c= 300 where a= 1; +REPLACE INTO t1(a,b,c) VALUES (2, NULL, 300); +************* SHOWING THE RESULT SETS WITH UPDATES and REPLACES ************* +TABLES t1 and t2 must be equal otherwise an error will be thrown. +Comparing tables master:test.t1 and slave:test.t1 +************* CLEANING ************* +DROP TABLE t1; +DROP TABLE t2; +DROP TABLE t3; +DROP TABLE t4; +SET SQL_LOG_BIN= 0; +CREATE TABLE t1 (`a` INT, `b` BIT DEFAULT NULL, `c` BIT DEFAULT NULL, +PRIMARY KEY (`a`)) ENGINE= MyISAM; +SET SQL_LOG_BIN= 1; +CREATE TABLE t1 (`a` INT, `b` BIT DEFAULT b'01', `c` BIT DEFAULT NULL, +PRIMARY KEY (`a`)) ENGINE= MyISAM; +************* EXECUTION WITH INSERTS ************* +INSERT INTO t1(a,b,c) VALUES (1, null, b'01'); +INSERT INTO t1(a,b,c) VALUES (2,b'00', b'01'); +INSERT INTO t1(a,b) VALUES (3, null); +INSERT INTO t1(a,c) VALUES (4, b'01'); +INSERT INTO t1(a) VALUES (5); +************* SHOWING THE RESULT SETS WITH INSERTS ************* +TABLES t1 and t2 must be different. +SELECT a,b+0,c+0 FROM t1 ORDER BY a; +a b+0 c+0 +1 NULL 1 +2 0 1 +3 NULL NULL +4 NULL 1 +5 NULL NULL +SELECT a,b+0,c+0 FROM t1 ORDER BY a; +a b+0 c+0 +1 NULL 1 +2 0 1 +3 NULL NULL +4 NULL 1 +5 NULL NULL +************* EXECUTION WITH UPDATES and REPLACES ************* +DELETE FROM t1; +INSERT INTO t1(a,b,c) VALUES (1,b'00', b'01'); +REPLACE INTO t1(a,b,c) VALUES (2,b'00',b'01'); +UPDATE t1 set b= NULL, c= b'00' where a= 1; +REPLACE INTO t1(a,b,c) VALUES (2, NULL, b'00'); +************* SHOWING THE RESULT SETS WITH UPDATES and REPLACES ************* +TABLES t1 and t2 must be equal otherwise an error will be thrown. +Comparing tables master:test.t1 and slave:test.t1 +DROP TABLE t1; +################################################################################ +# NULL ---> NOT NULL (STRICT MODE) +# UNCOMMENT THIS AFTER FIXING BUG#43992 +################################################################################ +################################################################################ +# NULL ---> NOT NULL (NON-STRICT MODE) +################################################################################ +SET SQL_LOG_BIN= 0; +CREATE TABLE t1(`a` INT NOT NULL, `b` INT, +PRIMARY KEY(`a`)) ENGINE=MyISAM DEFAULT CHARSET=LATIN1; +CREATE TABLE t2(`a` INT NOT NULL, `b` INT, +PRIMARY KEY(`a`)) ENGINE=MyISAM DEFAULT CHARSET=LATIN1; +CREATE TABLE t3(`a` INT NOT NULL, `b` INT, +PRIMARY KEY(`a`)) ENGINE=MyISAM DEFAULT CHARSET=LATIN1; +SET SQL_LOG_BIN= 1; +CREATE TABLE t1(`a` INT NOT NULL, `b` INT NOT NULL, +`c` INT NOT NULL, +PRIMARY KEY(`a`)) ENGINE=MyISAM DEFAULT CHARSET=LATIN1; +CREATE TABLE t2(`a` INT NOT NULL, `b` INT NOT NULL, +`c` INT, +PRIMARY KEY(`a`)) ENGINE=MyISAM DEFAULT CHARSET=LATIN1; +CREATE TABLE t3(`a` INT NOT NULL, `b` INT NOT NULL, +`c` INT DEFAULT 500, +PRIMARY KEY(`a`)) ENGINE=MyISAM DEFAULT CHARSET=LATIN1; +************* EXECUTION WITH INSERTS ************* +INSERT INTO t1(a) VALUES (1); +INSERT INTO t1(a, b) VALUES (2, NULL); +INSERT INTO t1(a, b) VALUES (3, 1); +INSERT INTO t2(a) VALUES (1); +INSERT INTO t2(a, b) VALUES (2, NULL); +INSERT INTO t2(a, b) VALUES (3, 1); +INSERT INTO t3(a) VALUES (1); +INSERT INTO t3(a, b) VALUES (2, NULL); +INSERT INTO t3(a, b) VALUES (3, 1); +INSERT INTO t3(a, b) VALUES (4, 1); +REPLACE INTO t3(a, b) VALUES (5, null); +REPLACE INTO t3(a, b) VALUES (3, null); +UPDATE t3 SET b = NULL where a = 4; +************* SHOWING THE RESULT SETS ************* +SELECT * FROM t1 ORDER BY a; +a b +1 NULL +2 NULL +3 1 +SELECT * FROM t1 ORDER BY a; +a b c +1 0 0 +2 0 0 +3 1 0 +SELECT * FROM t2 ORDER BY a; +a b +1 NULL +2 NULL +3 1 +SELECT * FROM t2 ORDER BY a; +a b c +1 0 NULL +2 0 NULL +3 1 NULL +SELECT * FROM t3 ORDER BY a; +a b +1 NULL +2 NULL +3 NULL +4 NULL +5 NULL +SELECT * FROM t3 ORDER BY a; +a b c +1 0 500 +2 0 500 +3 0 500 +4 0 500 +5 0 500 +DROP TABLE t1; +DROP TABLE t2; +DROP TABLE t3; === modified file 'mysql-test/suite/rpl/r/rpl_relayspace.result' --- a/mysql-test/suite/rpl/r/rpl_relayspace.result 2007-06-27 12:28:02 +0000 +++ b/mysql-test/suite/rpl/r/rpl_relayspace.result 2010-01-18 12:56:10 +0000 @@ -14,6 +14,6 @@ start slave io_thread; stop slave io_thread; reset slave; start slave; -select master_pos_wait('master-bin.001',200,6)=-1; -master_pos_wait('master-bin.001',200,6)=-1 +select master_pos_wait('MASTER_LOG_FILE',200,30)=-1; +master_pos_wait('MASTER_LOG_FILE',200,30)=-1 0 === modified file 'mysql-test/suite/rpl/r/rpl_row_create_table.result' --- a/mysql-test/suite/rpl/r/rpl_row_create_table.result 2009-10-06 00:54:00 +0000 +++ b/mysql-test/suite/rpl/r/rpl_row_create_table.result 2009-11-27 13:34:39 +0000 @@ -476,4 +476,30 @@ master-bin.000001 # Table_map # # table_ master-bin.000001 # Write_rows # # table_id: # flags: STMT_END_F master-bin.000001 # Query # # COMMIT DROP DATABASE mysqltest1; +stop slave; +drop table if exists t1,t2,t3,t4,t5,t6,t7,t8,t9; +reset master; +reset slave; +drop table if exists t1,t2,t3,t4,t5,t6,t7,t8,t9; +start slave; +CREATE TEMPORARY TABLE t7(c1 INT); +CREATE TABLE t5(c1 INT); +CREATE TABLE t4(c1 INT); +CREATE VIEW bug48506_t1 AS SELECT 1; +CREATE VIEW bug48506_t2 AS SELECT * FROM t4; +CREATE VIEW bug48506_t3 AS SELECT t5.c1 AS A, t4.c1 AS B FROM t5, t4; +CREATE TABLE bug48506_t4(c1 INT); +DROP VIEW bug48506_t1, bug48506_t2, bug48506_t3; +DROP TABLE bug48506_t4; +CREATE TABLE IF NOT EXISTS bug48506_t1 LIKE t7; +CREATE TABLE IF NOT EXISTS bug48506_t2 LIKE t7; +CREATE TABLE IF NOT EXISTS bug48506_t3 LIKE t7; +CREATE TABLE IF NOT EXISTS bug48506_t4 LIKE t7; +SHOW TABLES LIKE 'bug48506%'; +Tables_in_test (bug48506%) +bug48506_t4 +DROP VIEW IF EXISTS bug48506_t1, bug48506_t2, bug48506_t3; +DROP TEMPORARY TABLES t7; +DROP TABLES t4, t5; +DROP TABLES IF EXISTS bug48506_t4; end of the tests === modified file 'mysql-test/suite/rpl/r/rpl_row_tabledefs_2myisam.result' --- a/mysql-test/suite/rpl/r/rpl_row_tabledefs_2myisam.result 2008-03-14 20:02:52 +0000 +++ b/mysql-test/suite/rpl/r/rpl_row_tabledefs_2myisam.result 2009-10-22 00:10:42 +0000 @@ -105,47 +105,9 @@ a b x 2 10 Foo is a bar INSERT INTO t9 VALUES (2); INSERT INTO t1_nodef VALUES (1,2); -SHOW SLAVE STATUS; -Slave_IO_State # -Master_Host 127.0.0.1 -Master_User root -Master_Port # -Connect_Retry 1 -Master_Log_File master-bin.000001 -Read_Master_Log_Pos # -Relay_Log_File # -Relay_Log_Pos # -Relay_Master_Log_File master-bin.000001 -Slave_IO_Running Yes -Slave_SQL_Running No -Replicate_Do_DB -Replicate_Ignore_DB -Replicate_Do_Table -Replicate_Ignore_Table -Replicate_Wild_Do_Table -Replicate_Wild_Ignore_Table -Last_Errno 1364 -Last_Error <Last_Error> -Skip_Counter 0 -Exec_Master_Log_Pos # -Relay_Log_Space # -Until_Condition None -Until_Log_File -Until_Log_Pos 0 -Master_SSL_Allowed No -Master_SSL_CA_File -Master_SSL_CA_Path -Master_SSL_Cert -Master_SSL_Cipher -Master_SSL_Key -Seconds_Behind_Master # -Master_SSL_Verify_Server_Cert No -Last_IO_Errno <Last_IO_Errno> -Last_IO_Error <Last_IO_Error> -Last_SQL_Errno 1364 -Last_SQL_Error <Last_SQL_Error> -SET GLOBAL SQL_SLAVE_SKIP_COUNTER=2; -START SLAVE; +select count(*) from t1_nodef; +count(*) +1 INSERT INTO t9 VALUES (2); **** On Master **** INSERT INTO t2 VALUES (2,4); === modified file 'mysql-test/suite/rpl/r/rpl_row_tabledefs_3innodb.result' --- a/mysql-test/suite/rpl/r/rpl_row_tabledefs_3innodb.result 2008-03-14 20:02:52 +0000 +++ b/mysql-test/suite/rpl/r/rpl_row_tabledefs_3innodb.result 2009-10-22 00:10:42 +0000 @@ -105,47 +105,9 @@ a b x 2 10 Foo is a bar INSERT INTO t9 VALUES (2); INSERT INTO t1_nodef VALUES (1,2); -SHOW SLAVE STATUS; -Slave_IO_State # -Master_Host 127.0.0.1 -Master_User root -Master_Port # -Connect_Retry 1 -Master_Log_File master-bin.000001 -Read_Master_Log_Pos # -Relay_Log_File # -Relay_Log_Pos # -Relay_Master_Log_File master-bin.000001 -Slave_IO_Running Yes -Slave_SQL_Running No -Replicate_Do_DB -Replicate_Ignore_DB -Replicate_Do_Table -Replicate_Ignore_Table -Replicate_Wild_Do_Table -Replicate_Wild_Ignore_Table -Last_Errno 1364 -Last_Error <Last_Error> -Skip_Counter 0 -Exec_Master_Log_Pos # -Relay_Log_Space # -Until_Condition None -Until_Log_File -Until_Log_Pos 0 -Master_SSL_Allowed No -Master_SSL_CA_File -Master_SSL_CA_Path -Master_SSL_Cert -Master_SSL_Cipher -Master_SSL_Key -Seconds_Behind_Master # -Master_SSL_Verify_Server_Cert No -Last_IO_Errno <Last_IO_Errno> -Last_IO_Error <Last_IO_Error> -Last_SQL_Errno 1364 -Last_SQL_Error <Last_SQL_Error> -SET GLOBAL SQL_SLAVE_SKIP_COUNTER=2; -START SLAVE; +select count(*) from t1_nodef; +count(*) +1 INSERT INTO t9 VALUES (2); **** On Master **** INSERT INTO t2 VALUES (2,4); === added file 'mysql-test/suite/rpl/r/rpl_row_trunc_temp.result' --- a/mysql-test/suite/rpl/r/rpl_row_trunc_temp.result 1970-01-01 00:00:00 +0000 +++ b/mysql-test/suite/rpl/r/rpl_row_trunc_temp.result 2009-11-22 05:10:33 +0000 @@ -0,0 +1,29 @@ +stop slave; +drop table if exists t1,t2,t3,t4,t5,t6,t7,t8,t9; +reset master; +reset slave; +drop table if exists t1,t2,t3,t4,t5,t6,t7,t8,t9; +start slave; +CREATE TEMPORARY TABLE t1(c1 INTEGER); +CREATE TABLE t2(c1 INTEGER); +CREATE TABLE t1(c1 INTEGER); +INSERT INTO t1 VALUES(1), (2); +INSERT INTO t2 VALUES(1), (2); +SELECT * FROM t1; +c1 +1 +2 +SELECT * FROM t2; +c1 +1 +2 +TRUNCATE t1; +TRUNCATE t2; +SELECT * FROM t1; +c1 +1 +2 +SELECT * FROM t2; +c1 +DROP TABLE t1; +DROP TABLE t2; === modified file 'mysql-test/suite/rpl/r/rpl_stm_000001.result' --- a/mysql-test/suite/rpl/r/rpl_stm_000001.result 2007-12-12 17:19:24 +0000 +++ b/mysql-test/suite/rpl/r/rpl_stm_000001.result 2009-11-18 14:50:31 +0000 @@ -4,6 +4,7 @@ reset master; reset slave; drop table if exists t1,t2,t3,t4,t5,t6,t7,t8,t9; start slave; +CALL mtr.add_suppression("Statement may not be safe to log in statement format."); create table t1 (word char(20) not null); load data infile '../../std_data/words.dat' into table t1; load data local infile 'MYSQL_TEST_DIR/std_data/words.dat' into table t1; === modified file 'mysql-test/suite/rpl/r/rpl_trigger.result' --- a/mysql-test/suite/rpl/r/rpl_trigger.result 2009-08-03 09:47:45 +0000 +++ b/mysql-test/suite/rpl/r/rpl_trigger.result 2009-11-18 14:50:31 +0000 @@ -4,6 +4,7 @@ reset master; reset slave; drop table if exists t1,t2,t3,t4,t5,t6,t7,t8,t9; start slave; +CALL mtr.add_suppression("Statement may not be safe to log in statement format."); DROP TABLE IF EXISTS t1; DROP TABLE IF EXISTS t2; DROP TABLE IF EXISTS t3; === modified file 'mysql-test/suite/rpl/t/disabled.def' --- a/mysql-test/suite/rpl/t/disabled.def 2009-09-27 10:12:58 +0000 +++ b/mysql-test/suite/rpl/t/disabled.def 2009-12-01 09:21:15 +0000 @@ -10,3 +10,4 @@ # ############################################################################## +rpl_row_create_table : Bug#45576 2009-12-01 joro rpl_row_create_table fails on PB2 === modified file 'mysql-test/suite/rpl/t/rpl_err_ignoredtable.test' --- a/mysql-test/suite/rpl/t/rpl_err_ignoredtable.test 2009-10-20 18:00:07 +0000 +++ b/mysql-test/suite/rpl/t/rpl_err_ignoredtable.test 2009-11-18 14:50:31 +0000 @@ -7,6 +7,8 @@ -- source include/master-slave.inc +CALL mtr.add_suppression("Statement may not be safe to log in statement format."); + connection master; create table t1 (a int primary key); create table t4 (a int primary key); @@ -14,19 +16,15 @@ create table t4 (a int primary key); --error 1022, ER_DUP_ENTRY insert into t1 values (1),(1); insert into t4 values (1),(2); -save_master_pos; -connection slave; # as the t1 table is ignored on the slave, the slave should be able to sync -sync_with_master; +sync_slave_with_master; # check that the table has been ignored, because otherwise the test is nonsense show tables like 't1'; show tables like 't4'; SELECT * FROM test.t4 ORDER BY a; connection master; drop table t1; -save_master_pos; -connection slave; -sync_with_master; +sync_slave_with_master; # Now test that even critical errors (connection killed) # are ignored if rules allow it. @@ -50,18 +48,17 @@ kill @id; drop table t2,t3; insert into t4 values (3),(4); connection master; +# The get_lock function causes warning for unsafe statement. +--disable_warnings --error 0,1317,2013 reap; +--enable_warnings connection master1; -save_master_pos; -connection slave; -sync_with_master; +sync_slave_with_master; SELECT * FROM test.t4 ORDER BY a; connection master1; DROP TABLE test.t4; -save_master_pos; -connection slave; -sync_with_master; +sync_slave_with_master; # End of 4.1 tests # Adding comment for force manual merge 5.0 -> wl1012. delete me if needed === modified file 'mysql-test/suite/rpl/t/rpl_get_lock.test' --- a/mysql-test/suite/rpl/t/rpl_get_lock.test 2007-06-27 12:28:02 +0000 +++ b/mysql-test/suite/rpl/t/rpl_get_lock.test 2009-11-18 14:50:31 +0000 @@ -1,7 +1,12 @@ source include/master-slave.inc; +CALL mtr.add_suppression("Statement may not be safe to log in statement format."); + create table t1(n int); +# Use of get_lock gives a warning for unsafeness if binlog_format=statement +--disable_warnings insert into t1 values(get_lock("lock",2)); +--enable_warnings dirty_close master; connection master1; select get_lock("lock",2); === added file 'mysql-test/suite/rpl/t/rpl_loaddata_symlink-master.opt' --- a/mysql-test/suite/rpl/t/rpl_loaddata_symlink-master.opt 1970-01-01 00:00:00 +0000 +++ b/mysql-test/suite/rpl/t/rpl_loaddata_symlink-master.opt 2009-11-28 04:43:16 +0000 @@ -0,0 +1 @@ +--secure-file-priv=$MYSQLTEST_VARDIR/std_data_master_link === added file 'mysql-test/suite/rpl/t/rpl_loaddata_symlink-master.sh' --- a/mysql-test/suite/rpl/t/rpl_loaddata_symlink-master.sh 1970-01-01 00:00:00 +0000 +++ b/mysql-test/suite/rpl/t/rpl_loaddata_symlink-master.sh 2009-11-28 04:43:16 +0000 @@ -0,0 +1 @@ +ln -s $MYSQLTEST_VARDIR/std_data $MYSQLTEST_VARDIR/std_data_master_link === added file 'mysql-test/suite/rpl/t/rpl_loaddata_symlink-slave.opt' --- a/mysql-test/suite/rpl/t/rpl_loaddata_symlink-slave.opt 1970-01-01 00:00:00 +0000 +++ b/mysql-test/suite/rpl/t/rpl_loaddata_symlink-slave.opt 2009-11-28 04:43:16 +0000 @@ -0,0 +1 @@ +--slave-load-tmpdir=$MYSQLTEST_VARDIR/std_data_slave_link === added file 'mysql-test/suite/rpl/t/rpl_loaddata_symlink-slave.sh' --- a/mysql-test/suite/rpl/t/rpl_loaddata_symlink-slave.sh 1970-01-01 00:00:00 +0000 +++ b/mysql-test/suite/rpl/t/rpl_loaddata_symlink-slave.sh 2009-11-28 04:43:16 +0000 @@ -0,0 +1 @@ +ln -s $MYSQLTEST_VARDIR/std_data $MYSQLTEST_VARDIR/std_data_slave_link === added file 'mysql-test/suite/rpl/t/rpl_loaddata_symlink.test' --- a/mysql-test/suite/rpl/t/rpl_loaddata_symlink.test 1970-01-01 00:00:00 +0000 +++ b/mysql-test/suite/rpl/t/rpl_loaddata_symlink.test 2009-11-28 04:43:16 +0000 @@ -0,0 +1,20 @@ +# +# BUG#43913 +# This test verifies if loading data infile will work fine +# if the path of the load data file is a symbolic link. +# +--source include/master-slave.inc +--source include/have_binlog_format_statement.inc + +create table t1(a int not null auto_increment, b int, primary key(a) ); +load data infile '../../std_data/rpl_loaddata.dat' into table t1; +select * from t1; + +sync_slave_with_master; +connection slave; +select * from t1; + +connection master; +drop table t1; +sync_slave_with_master; + === modified file 'mysql-test/suite/rpl/t/rpl_mixed_ddl_dml.test' --- a/mysql-test/suite/rpl/t/rpl_mixed_ddl_dml.test 2007-06-27 12:28:02 +0000 +++ b/mysql-test/suite/rpl/t/rpl_mixed_ddl_dml.test 2010-01-19 10:36:52 +0000 @@ -23,6 +23,7 @@ connection master; # Test replication of timestamp create table t2(id int auto_increment primary key, created datetime); +SET TIME_ZONE= '+03:00'; set timestamp=12345; insert into t2 set created=now(); select * from t2; === added file 'mysql-test/suite/rpl/t/rpl_nondeterministic_functions.test' --- a/mysql-test/suite/rpl/t/rpl_nondeterministic_functions.test 1970-01-01 00:00:00 +0000 +++ b/mysql-test/suite/rpl/t/rpl_nondeterministic_functions.test 2009-11-18 14:50:31 +0000 @@ -0,0 +1,53 @@ +# ==== Purpose ==== +# +# Test that nondeterministic system functions are correctly replicated. +# +# (Some functions are only correctly replicated if binlog_format=MIXED +# or ROW. See binlog_unsafe.test for a test that those variables are +# indeed unsafe.) +# +# ==== Implementation ==== +# +# We insert the values of each unsafe function into a table. Then we +# replicate and check that the table is identical on slave. +# +# ==== Related bugs ==== +# +# BUG#47995 + +--source include/master-slave.inc + +CREATE TABLE t1 (a VARCHAR(1000)); + +# We replicate the connection_id in the query_log_event +INSERT INTO t1 VALUES (CONNECTION_ID()); +--connection master1 +INSERT INTO t1 VALUES (CONNECTION_ID()); + +# We replicate the TIMESTAMP variable, so the following functions that +# are affected by the TIMESTAMP variable should be safe to replicate. +INSERT INTO t1 VALUES + (CURDATE()), + (CURRENT_DATE()), + (CURRENT_TIME()), + (CURRENT_TIMESTAMP()), + (CURTIME()), + (LOCALTIME()), + (LOCALTIMESTAMP()), + (NOW()), + (UNIX_TIMESTAMP()), + (UTC_DATE()), + (UTC_TIME()), + (UTC_TIMESTAMP()); + +# We replicate the random seed in a rand_log_event +INSERT INTO t1 VALUES (RAND()); +# We replicate the last_insert_id in an intvar_log_event +INSERT INTO t1 VALUES (LAST_INSERT_ID()); + +--sync_slave_with_master +--let $diff_table_1= master:test.t1 +--let $diff_table_2= slave:test.t1 +--source include/diff_tables.inc + +DROP TABLE t1; === added file 'mysql-test/suite/rpl/t/rpl_not_null_innodb.test' --- a/mysql-test/suite/rpl/t/rpl_not_null_innodb.test 1970-01-01 00:00:00 +0000 +++ b/mysql-test/suite/rpl/t/rpl_not_null_innodb.test 2009-10-22 00:15:45 +0000 @@ -0,0 +1,19 @@ +################################################################################# +# This test checks if the replication between "null" fields to either "null" +# fields or "not null" fields works properly. In the first case, the execution +# should work fine. In the second case, it may fail according to the sql_mode +# being used. +# +# The test is devided in three main parts: +# +# 1 - NULL --> NULL (no failures) +# 2 - NULL --> NOT NULL ( sql-mode = STRICT and failures) +# 3 - NULL --> NOT NULL ( sql-mode != STRICT and no failures) +# +################################################################################# +--source include/master-slave.inc +--source include/have_innodb.inc +--source include/have_binlog_format_row.inc + +let $engine=Innodb; +--source extra/rpl_tests/rpl_not_null.test === added file 'mysql-test/suite/rpl/t/rpl_not_null_myisam.test' --- a/mysql-test/suite/rpl/t/rpl_not_null_myisam.test 1970-01-01 00:00:00 +0000 +++ b/mysql-test/suite/rpl/t/rpl_not_null_myisam.test 2009-10-22 00:15:45 +0000 @@ -0,0 +1,18 @@ +################################################################################# +# This test checks if the replication between "null" fields to either "null" +# fields or "not null" fields works properly. In the first case, the execution +# should work fine. In the second case, it may fail according to the sql_mode +# being used. +# +# The test is devided in three main parts: +# +# 1 - NULL --> NULL (no failures) +# 2 - NULL --> NOT NULL ( sql-mode = STRICT and failures) +# 3 - NULL --> NOT NULL ( sql-mode != STRICT and no failures) +# +################################################################################# +--source include/master-slave.inc +--source include/have_binlog_format_row.inc + +let $engine=MyISAM; +--source extra/rpl_tests/rpl_not_null.test === modified file 'mysql-test/suite/rpl/t/rpl_relayspace.test' --- a/mysql-test/suite/rpl/t/rpl_relayspace.test 2008-02-28 11:36:14 +0000 +++ b/mysql-test/suite/rpl/t/rpl_relayspace.test 2010-01-18 12:56:10 +0000 @@ -2,8 +2,10 @@ # to force the deadlock after one event. source include/master-slave.inc; +let $master_log_file= query_get_value(SHOW MASTER STATUS, File, 1); connection slave; stop slave; +source include/wait_for_slave_to_stop.inc; connection master; # This will generate a master's binlog > 10 bytes create table t1 (a int); @@ -20,6 +22,7 @@ source include/wait_for_slave_param.inc; # A bug caused the I/O thread to refuse stopping. stop slave io_thread; +source include/wait_for_slave_io_to_stop.inc; reset slave; start slave; # The I/O thread stops filling the relay log when @@ -29,9 +32,11 @@ start slave; # So we should have a deadlock. # if it is not resolved automatically we'll detect # it with master_pos_wait that waits for farther than 1Ob; -# it will timeout after 10 seconds; +# it will timeout; # also the slave will probably not cooperate to shutdown # (as 2 threads are locked) -select master_pos_wait('master-bin.001',200,6)=-1; +--replace_result $master_log_file MASTER_LOG_FILE +eval select master_pos_wait('$master_log_file',200,30)=-1; + # End of 4.1 tests === modified file 'mysql-test/suite/rpl/t/rpl_row_create_table.test' --- a/mysql-test/suite/rpl/t/rpl_row_create_table.test 2009-01-23 12:22:05 +0000 +++ b/mysql-test/suite/rpl/t/rpl_row_create_table.test 2009-11-27 13:34:39 +0000 @@ -292,4 +292,40 @@ connection master; DROP DATABASE mysqltest1; sync_slave_with_master; +# +# BUG#48506: crash in CREATE TABLE <existing_view> IF NOT EXISTS LIKE +# <tmp_tbl> with RBL +# + +source include/master-slave-reset.inc; + +connection master; +CREATE TEMPORARY TABLE t7(c1 INT); +CREATE TABLE t5(c1 INT); +CREATE TABLE t4(c1 INT); +CREATE VIEW bug48506_t1 AS SELECT 1; +CREATE VIEW bug48506_t2 AS SELECT * FROM t4; +CREATE VIEW bug48506_t3 AS SELECT t5.c1 AS A, t4.c1 AS B FROM t5, t4; +CREATE TABLE bug48506_t4(c1 INT); +--disable_warnings +sync_slave_with_master; +DROP VIEW bug48506_t1, bug48506_t2, bug48506_t3; +DROP TABLE bug48506_t4; + +connection master; +CREATE TABLE IF NOT EXISTS bug48506_t1 LIKE t7; +CREATE TABLE IF NOT EXISTS bug48506_t2 LIKE t7; +CREATE TABLE IF NOT EXISTS bug48506_t3 LIKE t7; +CREATE TABLE IF NOT EXISTS bug48506_t4 LIKE t7; +--enable_warnings +sync_slave_with_master; + +SHOW TABLES LIKE 'bug48506%'; + +connection master; +DROP VIEW IF EXISTS bug48506_t1, bug48506_t2, bug48506_t3; +DROP TEMPORARY TABLES t7; +DROP TABLES t4, t5; +DROP TABLES IF EXISTS bug48506_t4; +source include/master-slave-end.inc; --echo end of the tests === added file 'mysql-test/suite/rpl/t/rpl_row_trunc_temp.test' --- a/mysql-test/suite/rpl/t/rpl_row_trunc_temp.test 1970-01-01 00:00:00 +0000 +++ b/mysql-test/suite/rpl/t/rpl_row_trunc_temp.test 2009-11-22 05:10:33 +0000 @@ -0,0 +1,35 @@ +# +# Bug#48350 truncate temporary table crashes replication +# +# All statements operating on temporary tables should not be binlogged in RBR. +# However, before fix of bug#48350, 'TRUNCATE ...' statement on a temporary +# table was binlogged in RBR. +# + +--source include/master-slave.inc +--source include/have_binlog_format_row.inc + +#This statement is not binlogged in RBR. +CREATE TEMPORARY TABLE t1(c1 INTEGER); +CREATE TABLE t2(c1 INTEGER); +sync_slave_with_master; + +CREATE TABLE t1(c1 INTEGER); +INSERT INTO t1 VALUES(1), (2); +INSERT INTO t2 VALUES(1), (2); +SELECT * FROM t1; +SELECT * FROM t2; + +connection master; +TRUNCATE t1; +TRUNCATE t2; +sync_slave_with_master; +# t1 will have nothing, if 'TRUNCATE t1' has been replicate from master to +# slave. +SELECT * FROM t1; +SELECT * FROM t2; + +DROP TABLE t1; +connection master; +DROP TABLE t2; +--source include/master-slave-end.inc === modified file 'mysql-test/suite/rpl/t/rpl_trigger.test' --- a/mysql-test/suite/rpl/t/rpl_trigger.test 2009-08-03 15:01:06 +0000 +++ b/mysql-test/suite/rpl/t/rpl_trigger.test 2009-11-18 14:50:31 +0000 @@ -5,6 +5,8 @@ --source include/have_binlog_format_mixed_or_statement.inc --source include/master-slave.inc +CALL mtr.add_suppression("Statement may not be safe to log in statement format."); + --disable_warnings DROP TABLE IF EXISTS t1; DROP TABLE IF EXISTS t2; @@ -89,7 +91,11 @@ end | delimiter ;| +# The trigger causes a warning for unsafe statement when +# binlog_format=statement since it uses get_lock. +--disable_warnings insert into t1 set a = now(); +--enable_warnings select a=b && a=c from t1; let $time=`select a from t1`; @@ -135,7 +141,11 @@ disconnect con2; truncate table t1; drop trigger t1_first; +# The trigger causes a warning for unsafe statement when +# binlog_format=statement since it uses get_lock. +--disable_warnings insert into t1 values ("2003-03-03","2003-03-03","2003-03-03"),(bug12480(),bug12480(),bug12480()),(now(),now(),now()); +--enable_warnings select a=b && a=c from t1; drop function bug12480; === modified file 'mysql-test/suite/rpl_ndb/r/rpl_ndb_extraCol.result' --- a/mysql-test/suite/rpl_ndb/r/rpl_ndb_extraCol.result 2009-08-29 08:30:59 +0000 +++ b/mysql-test/suite/rpl_ndb/r/rpl_ndb_extraCol.result 2009-10-22 00:21:50 +0000 @@ -400,62 +400,6 @@ set @b1 = concat(@b1,@b1); INSERT INTO t8 () VALUES(1,@b1,'Kyle'),(2,@b1,'JOE'),(3,@b1,'QA'); *** Drop t8 *** DROP TABLE t8; -STOP SLAVE; -RESET SLAVE; -CREATE TABLE t9 (a INT KEY, b BLOB, c CHAR(5), -d TIMESTAMP, -e INT NOT NULL) ENGINE='NDB'; -*** Create t9 on Master *** -CREATE TABLE t9 (a INT PRIMARY KEY, b BLOB, c CHAR(5) -) ENGINE='NDB'; -RESET MASTER; -*** Start Slave *** -START SLAVE; -*** Master Data Insert *** -set @b1 = 'b1b1b1b1'; -set @b1 = concat(@b1,@b1); -INSERT INTO t9 () VALUES(1,@b1,'Kyle'),(2,@b1,'JOE'),(3,@b1,'QA'); -SHOW SLAVE STATUS; -Slave_IO_State # -Master_Host 127.0.0.1 -Master_User root -Master_Port # -Connect_Retry 1 -Master_Log_File master-bin.000001 -Read_Master_Log_Pos # -Relay_Log_File # -Relay_Log_Pos # -Relay_Master_Log_File master-bin.000001 -Slave_IO_Running Yes -Slave_SQL_Running No -Replicate_Do_DB -Replicate_Ignore_DB -Replicate_Do_Table -Replicate_Ignore_Table # -Replicate_Wild_Do_Table -Replicate_Wild_Ignore_Table -Last_Errno 1364 -Last_Error Could not execute Write_rows event on table test.t9; Field 'e' doesn't have a default value, Error_code: 1364; handler error HA_ERR_ROWS_EVENT_APPLY; the event's master log master-bin.000001, end_log_pos 447 -Skip_Counter 0 -Exec_Master_Log_Pos # -Relay_Log_Space # -Until_Condition None -Until_Log_File -Until_Log_Pos 0 -Master_SSL_Allowed No -Master_SSL_CA_File -Master_SSL_CA_Path -Master_SSL_Cert -Master_SSL_Cipher -Master_SSL_Key -Seconds_Behind_Master # -Master_SSL_Verify_Server_Cert No -Last_IO_Errno # -Last_IO_Error # -Last_SQL_Errno 1364 -Last_SQL_Error Could not execute Write_rows event on table test.t9; Field 'e' doesn't have a default value, Error_code: 1364; handler error HA_ERR_ROWS_EVENT_APPLY; the event's master log master-bin.000001, end_log_pos 447 -SET GLOBAL SQL_SLAVE_SKIP_COUNTER=2; -START SLAVE; *** Create t10 on slave *** STOP SLAVE; RESET SLAVE; === modified file 'mysql-test/t/archive.test' --- a/mysql-test/t/archive.test 2009-12-03 11:19:05 +0000 +++ b/mysql-test/t/archive.test 2010-01-15 15:27:55 +0000 @@ -1625,3 +1625,24 @@ INSERT INTO t1 VALUES('aaaaaaaaaaaaaaaaa SELECT COUNT(t1.a) FROM t1, t1 a, t1 b, t1 c, t1 d, t1 e; DROP TABLE t1; SET @@join_buffer_size= @save_join_buffer_size; + +# +# BUG#47012 archive tables are not upgradeable, and server crashes on any access +# +let $MYSQLD_DATADIR= `SELECT @@datadir`; +copy_file std_data/bug47012.frm $MYSQLD_DATADIR/test/t1.frm; +copy_file std_data/bug47012.ARZ $MYSQLD_DATADIR/test/t1.ARZ; +copy_file std_data/bug47012.ARM $MYSQLD_DATADIR/test/t1.ARM; + +--error ER_TABLE_NEEDS_UPGRADE +SHOW CREATE TABLE t1; + +--error ER_TABLE_NEEDS_UPGRADE +SELECT * FROM t1; + +--error ER_TABLE_NEEDS_UPGRADE +INSERT INTO t1 (col1, col2) VALUES (1, "value"); + +REPAIR TABLE t1; +DROP TABLE t1; +remove_file $MYSQLD_DATADIR/test/t1.ARM; === added file 'mysql-test/t/bug47671-master.opt' --- a/mysql-test/t/bug47671-master.opt 1970-01-01 00:00:00 +0000 +++ b/mysql-test/t/bug47671-master.opt 2009-11-25 06:55:49 +0000 @@ -0,0 +1 @@ +--default-character-set=utf8 --skip-character-set-client-handshake === added file 'mysql-test/t/bug47671.test' --- a/mysql-test/t/bug47671.test 1970-01-01 00:00:00 +0000 +++ b/mysql-test/t/bug47671.test 2009-11-30 05:24:26 +0000 @@ -0,0 +1,9 @@ +# Embedded server doesn't support external clients +--source include/not_embedded.inc + +--echo # +--echo # Bug#47671 - wrong character-set after upgrade from 5.1.34 to 5.1.39 +--echo # +--echo # Extract only charset information from 'status' command output using regex +--replace_regex /.*mysql.*// /Connection.*// /Current.*// /SSL.*// /Using.*// /Server version.*// /Protocol.*// /UNIX.*// /Uptime.*// /Threads.*// /TCP.*// +--exec $MYSQL -e "status"; === modified file 'mysql-test/t/delayed.test' --- a/mysql-test/t/delayed.test 2009-03-11 15:32:42 +0000 +++ b/mysql-test/t/delayed.test 2010-01-15 15:27:55 +0000 @@ -341,4 +341,28 @@ drop table t1; set global low_priority_updates = @old_delayed_updates; + +--echo # +--echo # Bug #47682 strange behaviour of INSERT DELAYED +--echo # + +--disable_warnings +DROP TABLE IF EXISTS t1, t2; +--enable_warnings + +CREATE TABLE t1 (f1 integer); +CREATE TABLE t2 (f1 integer); + +FLUSH TABLES WITH READ LOCK; +LOCK TABLES t1 READ; + +# ER_CANT_UPDATE_WITH_READLOCK with normal execution +# ER_TABLE_NOT_LOCKED when executed as prepared statement +--error ER_CANT_UPDATE_WITH_READLOCK, ER_TABLE_NOT_LOCKED +INSERT DELAYED INTO t2 VALUES (1); + +UNLOCK TABLES; +DROP TABLE t1, t2; + + --echo End of 5.1 tests === modified file 'mysql-test/t/delete.test' --- a/mysql-test/t/delete.test 2009-09-28 10:48:52 +0000 +++ b/mysql-test/t/delete.test 2009-11-18 09:32:03 +0000 @@ -336,3 +336,25 @@ SELECT * FROM t2; SELECT * FROM t3; DROP TABLE t1, t2, t3; + +--echo # +--echo # Bug #46425 crash in Diagnostics_area::set_ok_status, +--echo # empty statement, DELETE IGNORE +--echo # + +CREATE table t1 (i INTEGER); + +INSERT INTO t1 VALUES (1); + +--delimiter | + +CREATE TRIGGER tr1 AFTER DELETE ON t1 FOR EACH ROW +BEGIN + INSERT INTO t1 SELECT * FROM t1 AS A; +END | + +--delimiter ; +--error ER_CANT_UPDATE_USED_TABLE_IN_SF_OR_TRG +DELETE IGNORE FROM t1; + +DROP TABLE t1; \ No newline at end of file === modified file 'mysql-test/t/disabled.def' --- a/mysql-test/t/disabled.def 2009-12-03 11:19:05 +0000 +++ b/mysql-test/t/disabled.def 2010-01-15 17:02:57 +0000 @@ -11,7 +11,4 @@ ############################################################################## kill : Bug#37780 2008-12-03 HHunger need some changes to be robust enough for pushbuild. query_cache_28249 : Bug#43861 2009-03-25 main.query_cache_28249 fails sporadically -partition_innodb_builtin : Bug#32430 2009-09-25 mattiasj Waiting for push of Innodb changes -partition_innodb_plugin : Bug#32430 2009-09-25 mattiasj Waiting for push of Innodb changes -innodb-autoinc : Bug#48482 2009-11-02 svoj innodb-autoinc.test fails with results difference rpl_killed_ddl : Bug#45520: rpl_killed_ddl fails sporadically in pb2 === modified file 'mysql-test/t/fulltext.test' --- a/mysql-test/t/fulltext.test 2009-12-27 13:54:41 +0000 +++ b/mysql-test/t/fulltext.test 2010-01-15 15:27:55 +0000 @@ -496,3 +496,44 @@ PREPARE s FROM EXECUTE s; DEALLOCATE PREPARE s; DROP TABLE t1; + +--echo # +--echo # Bug #47930: MATCH IN BOOLEAN MODE returns too many results +--echo # inside subquery +--echo # + +CREATE TABLE t1 (a int); +INSERT INTO t1 VALUES (1), (2); + +CREATE TABLE t2 (a int, b2 char(10), FULLTEXT KEY b2 (b2)); +INSERT INTO t2 VALUES (1,'Scargill'); + +CREATE TABLE t3 (a int, b int); +INSERT INTO t3 VALUES (1,1), (2,1); + +--echo # t2 should use full text index +EXPLAIN +SELECT count(*) FROM t1 WHERE + not exists( + SELECT 1 FROM t2, t3 + WHERE t3.a=t1.a AND MATCH(b2) AGAINST('scargill' IN BOOLEAN MODE) + ); + +--echo # should return 0 +SELECT count(*) FROM t1 WHERE + not exists( + SELECT 1 FROM t2, t3 + WHERE t3.a=t1.a AND MATCH(b2) AGAINST('scargill' IN BOOLEAN MODE) + ); + +--echo # should return 0 +SELECT count(*) FROM t1 WHERE + not exists( + SELECT 1 FROM t2 IGNORE INDEX (b2), t3 + WHERE t3.a=t1.a AND MATCH(b2) AGAINST('scargill' IN BOOLEAN MODE) + ); + +DROP TABLE t1,t2,t3; + + +--echo End of 5.1 tests === modified file 'mysql-test/t/func_group.test' --- a/mysql-test/t/func_group.test 2009-10-14 08:46:50 +0000 +++ b/mysql-test/t/func_group.test 2009-11-24 15:26:13 +0000 @@ -1053,4 +1053,35 @@ ORDER BY max; --echo # DROP TABLE t1; +--echo # +--echo # Bug#43668: Wrong comparison and MIN/MAX for YEAR(2) +--echo # +create table t1 (f1 year(2), f2 year(4), f3 date, f4 datetime); +insert into t1 values + (98,1998,19980101,"1998-01-01 00:00:00"), + (00,2000,20000101,"2000-01-01 00:00:01"), + (02,2002,20020101,"2002-01-01 23:59:59"), + (60,2060,20600101,"2060-01-01 11:11:11"), + (70,1970,19700101,"1970-11-11 22:22:22"), + (NULL,NULL,NULL,NULL); +select min(f1),max(f1) from t1; +select min(f2),max(f2) from t1; +select min(f3),max(f3) from t1; +select min(f4),max(f4) from t1; +select a.f1 as a, b.f1 as b, a.f1 > b.f1 as gt, + a.f1 < b.f1 as lt, a.f1<=>b.f1 as eq +from t1 a, t1 b; +select a.f1 as a, b.f2 as b, a.f1 > b.f2 as gt, + a.f1 < b.f2 as lt, a.f1<=>b.f2 as eq +from t1 a, t1 b; +select a.f1 as a, b.f3 as b, a.f1 > b.f3 as gt, + a.f1 < b.f3 as lt, a.f1<=>b.f3 as eq +from t1 a, t1 b; +select a.f1 as a, b.f4 as b, a.f1 > b.f4 as gt, + a.f1 < b.f4 as lt, a.f1<=>b.f4 as eq +from t1 a, t1 b; +select *, f1 = f2 from t1; +drop table t1; +--echo # --echo End of 5.1 tests + === modified file 'mysql-test/t/grant2.test' --- a/mysql-test/t/grant2.test 2009-02-27 08:03:47 +0000 +++ b/mysql-test/t/grant2.test 2009-10-30 05:06:10 +0000 @@ -632,5 +632,40 @@ DROP DATABASE db1; --echo End of 5.0 tests +# +# Bug #48319: Server crashes on "GRANT/REVOKE ... TO CURRENT_USER" +# + +# work out who we are. +USE mysql; +SELECT LEFT(CURRENT_USER(),INSTR(CURRENT_USER(),'@')-1) INTO @u; +SELECT MID(CURRENT_USER(),INSTR(CURRENT_USER(),'@')+1) INTO @h; +SELECT password FROM user WHERE user=@u AND host=@h INTO @pwd; + +# show current privs. +SELECT user,host,password,insert_priv FROM user WHERE user=@u AND host=@h; + +# toggle INSERT +UPDATE user SET insert_priv='N' WHERE user=@u AND host=@h; +SELECT user,host,password,insert_priv FROM user WHERE user=@u AND host=@h; + +# show that GRANT ... TO CURRENT_USER() no longer crashes +GRANT INSERT ON *.* TO CURRENT_USER(); +SELECT user,host,password,insert_priv FROM user WHERE user=@u AND host=@h; +UPDATE user SET insert_priv='N' WHERE user=@u AND host=@h; + +# show that GRANT ... TO CURRENT_USER() IDENTIFIED BY ... works now +GRANT INSERT ON *.* TO CURRENT_USER() IDENTIFIED BY 'keksdose'; +SELECT user,host,password,insert_priv FROM user WHERE user=@u AND host=@h; + +UPDATE user SET password=@pwd WHERE user=@u AND host=@h; +SELECT user,host,password,insert_priv FROM user WHERE user=@u AND host=@h; + +FLUSH PRIVILEGES; + +USE test; + +--echo End of 5.1 tests + # Wait till we reached the initial number of concurrent sessions --source include/wait_until_count_sessions.inc === modified file 'mysql-test/t/group_min_max.test' --- a/mysql-test/t/group_min_max.test 2009-08-30 07:03:37 +0000 +++ b/mysql-test/t/group_min_max.test 2009-11-23 10:04:17 +0000 @@ -1016,6 +1016,18 @@ SELECT a, MAX(b) FROM t WHERE b > 0 AND DROP TABLE t; +--echo # +--echo # Bug #48472: Loose index scan inappropriately chosen for some WHERE +--echo # conditions +--echo # + +CREATE TABLE t (a INT, b INT, INDEX (a,b)); +INSERT INTO t VALUES (2,0), (2,0), (2,1), (2,1); +INSERT INTO t SELECT * FROM t; + +SELECT a, MAX(b) FROM t WHERE 0=b+0 GROUP BY a; + +DROP TABLE t; --echo End of 5.0 tests === modified file 'mysql-test/t/innodb-analyze.test' --- a/mysql-test/t/innodb-analyze.test 2009-11-13 21:26:08 +0000 +++ b/mysql-test/t/innodb-analyze.test 2010-01-15 15:58:25 +0000 @@ -11,7 +11,7 @@ -- disable_result_log -- enable_warnings -SET @old_innodb_stats_sample_pages=@@innodb_stats_sample_pages; +let $sample_pages=`select @@innodb_stats_sample_pages`; SET GLOBAL innodb_stats_sample_pages=0; # check that the value has been adjusted to 1 @@ -62,4 +62,4 @@ SET GLOBAL innodb_stats_sample_pages=16; ANALYZE TABLE innodb_analyze; DROP TABLE innodb_analyze; -SET GLOBAL innodb_stats_sample_pages=@old_innodb_stats_sample_pages; +EVAL SET GLOBAL innodb_stats_sample_pages=$sample_pages; === modified file 'mysql-test/t/innodb-autoinc.test' --- a/mysql-test/t/innodb-autoinc.test 2009-12-03 11:34:11 +0000 +++ b/mysql-test/t/innodb-autoinc.test 2010-01-15 21:12:30 +0000 @@ -2,6 +2,8 @@ # embedded server ignores 'delayed', so skip this -- source include/not_embedded.inc +let $file_format_check=`select @@innodb_file_format_check`; + --disable_warnings drop table if exists t1; --enable_warnings @@ -156,7 +158,7 @@ DROP TABLE t1; # # Test changes to AUTOINC next value calculation SET @@SESSION.AUTO_INCREMENT_INCREMENT=100, @@SESSION.AUTO_INCREMENT_OFFSET=10; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; DROP TABLE IF EXISTS t1; CREATE TABLE t1 (c1 INT AUTO_INCREMENT, PRIMARY KEY(c1)) ENGINE=InnoDB; INSERT INTO t1 VALUES (NULL),(5),(NULL); @@ -173,7 +175,7 @@ DROP TABLE t1; # Reset the AUTOINC session variables SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; DROP TABLE IF EXISTS t1; CREATE TABLE t1 (c1 INT AUTO_INCREMENT, PRIMARY KEY(c1)) ENGINE=InnoDB; INSERT INTO t1 VALUES(0); @@ -193,13 +195,13 @@ DROP TABLE t1; # Reset the AUTOINC session variables SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; DROP TABLE IF EXISTS t1; CREATE TABLE t1 (c1 INT AUTO_INCREMENT, PRIMARY KEY(c1)) ENGINE=InnoDB; INSERT INTO t1 VALUES(-1); SELECT * FROM t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=100, @@SESSION.AUTO_INCREMENT_OFFSET=10; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; INSERT INTO t1 VALUES (-2), (NULL),(2),(NULL); INSERT INTO t1 VALUES (250),(NULL); SELECT * FROM t1; @@ -214,13 +216,13 @@ DROP TABLE t1; # Reset the AUTOINC session variables SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; DROP TABLE IF EXISTS t1; CREATE TABLE t1 (c1 INT UNSIGNED AUTO_INCREMENT, PRIMARY KEY(c1)) ENGINE=InnoDB; INSERT INTO t1 VALUES(-1); SELECT * FROM t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=100, @@SESSION.AUTO_INCREMENT_OFFSET=10; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; INSERT INTO t1 VALUES (-2); INSERT INTO t1 VALUES (NULL); INSERT INTO t1 VALUES (2); @@ -240,13 +242,13 @@ DROP TABLE t1; # Reset the AUTOINC session variables SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; DROP TABLE IF EXISTS t1; CREATE TABLE t1 (c1 INT UNSIGNED AUTO_INCREMENT, PRIMARY KEY(c1)) ENGINE=InnoDB; INSERT INTO t1 VALUES(-1); SELECT * FROM t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=100, @@SESSION.AUTO_INCREMENT_OFFSET=10; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; INSERT INTO t1 VALUES (-2),(NULL),(2),(NULL); INSERT INTO t1 VALUES (250),(NULL); SELECT * FROM t1; @@ -262,7 +264,7 @@ DROP TABLE t1; # Check for overflow handling when increment is > 1 SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; DROP TABLE IF EXISTS t1; CREATE TABLE t1 (c1 BIGINT AUTO_INCREMENT, PRIMARY KEY(c1)) ENGINE=InnoDB; # TODO: Fix the autoinc init code @@ -271,7 +273,7 @@ INSERT INTO t1 VALUES(NULL); INSERT INTO t1 VALUES (9223372036854775794); #-- 2^63 - 14 SELECT * FROM t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=2, @@SESSION.AUTO_INCREMENT_OFFSET=10; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; # This should just fit INSERT INTO t1 VALUES (NULL),(NULL),(NULL),(NULL),(NULL),(NULL); SELECT * FROM t1; @@ -281,7 +283,7 @@ DROP TABLE t1; # Check for overflow handling when increment and offser are > 1 SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; DROP TABLE IF EXISTS t1; CREATE TABLE t1 (c1 BIGINT UNSIGNED AUTO_INCREMENT, PRIMARY KEY(c1)) ENGINE=InnoDB; # TODO: Fix the autoinc init code @@ -290,7 +292,7 @@ INSERT INTO t1 VALUES(NULL); INSERT INTO t1 VALUES (18446744073709551603); #-- 2^64 - 13 SELECT * FROM t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=2, @@SESSION.AUTO_INCREMENT_OFFSET=10; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; # This should fail because of overflow but it doesn't, it seems to be # a MySQL server bug. It wraps around to 0 for the last value. # See MySQL Bug# 39828 @@ -313,7 +315,7 @@ DROP TABLE t1; # Check for overflow handling when increment and offset are odd numbers SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; DROP TABLE IF EXISTS t1; CREATE TABLE t1 (c1 BIGINT UNSIGNED AUTO_INCREMENT, PRIMARY KEY(c1)) ENGINE=InnoDB; # TODO: Fix the autoinc init code @@ -322,7 +324,7 @@ INSERT INTO t1 VALUES(NULL); INSERT INTO t1 VALUES (18446744073709551603); #-- 2^64 - 13 SELECT * FROM t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=5, @@SESSION.AUTO_INCREMENT_OFFSET=7; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; # This should fail because of overflow but it doesn't. It fails with # a duplicate entry message because of a MySQL server bug, it wraps # around. See MySQL Bug# 39828, once MySQL fix the bug we can replace @@ -344,7 +346,7 @@ DROP TABLE t1; # and check for large -ve numbers SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; DROP TABLE IF EXISTS t1; CREATE TABLE t1 (c1 BIGINT AUTO_INCREMENT, PRIMARY KEY(c1)) ENGINE=InnoDB; # TODO: Fix the autoinc init code @@ -355,7 +357,7 @@ INSERT INTO t1 VALUES(-92233720368547758 INSERT INTO t1 VALUES(-9223372036854775808); #-- -2^63 SELECT * FROM t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=3, @@SESSION.AUTO_INCREMENT_OFFSET=3; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; INSERT INTO t1 VALUES (NULL),(NULL), (NULL); SELECT * FROM t1; DROP TABLE t1; @@ -364,7 +366,7 @@ DROP TABLE t1; # large numbers 2^60 SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; DROP TABLE IF EXISTS t1; CREATE TABLE t1 (c1 BIGINT UNSIGNED AUTO_INCREMENT, PRIMARY KEY(c1)) ENGINE=InnoDB; # TODO: Fix the autoinc init code @@ -373,7 +375,7 @@ INSERT INTO t1 VALUES(NULL); INSERT INTO t1 VALUES (18446744073709551610); #-- 2^64 - 2 SELECT * FROM t1; SET @@SESSION.AUTO_INCREMENT_INCREMENT=1152921504606846976, @@SESSION.AUTO_INCREMENT_OFFSET=1152921504606846976; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; # This should fail because of overflow but it doesn't. It wraps around # and the autoinc values look bogus too. # See MySQL Bug# 39828, once MySQL fix the bug we can enable the error @@ -396,7 +398,7 @@ DROP TABLE t1; # SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; SET @@INSERT_ID=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; CREATE TABLE t1 (c1 DOUBLE NOT NULL AUTO_INCREMENT, c2 INT, PRIMARY KEY (c1)) ENGINE=InnoDB; INSERT INTO t1 VALUES(NULL, 1); INSERT INTO t1 VALUES(NULL, 2); @@ -508,7 +510,7 @@ DROP TABLE t1; # If the user has specified negative values for an AUTOINC column then # InnoDB should ignore those values when setting the table's max value. SET @@SESSION.AUTO_INCREMENT_INCREMENT=1, @@SESSION.AUTO_INCREMENT_OFFSET=1; -SHOW VARIABLES LIKE "%auto_inc%"; +SHOW VARIABLES LIKE "auto_inc%"; # TINYINT CREATE TABLE t1 (c1 TINYINT PRIMARY KEY AUTO_INCREMENT, c2 VARCHAR(10)) ENGINE=InnoDB; INSERT INTO t1 VALUES (1, NULL); @@ -620,3 +622,42 @@ SHOW CREATE TABLE T1; INSERT INTO T1 (c2) values (0); SELECT * FROM T1; DROP TABLE T1; + +## +# 49032: Use the correct function to read the AUTOINC column value +# +CREATE TABLE T1(C1 DOUBLE AUTO_INCREMENT KEY, C2 CHAR(10)) ENGINE=InnoDB; +INSERT INTO T1(C1, C2) VALUES (1, 'innodb'), (3, 'innodb'); +# Restart the server +-- source include/restart_mysqld.inc +INSERT INTO T1(C2) VALUES ('innodb'); +SHOW CREATE TABLE T1; +DROP TABLE T1; +CREATE TABLE T1(C1 FLOAT AUTO_INCREMENT KEY, C2 CHAR(10)) ENGINE=InnoDB; +INSERT INTO T1(C1, C2) VALUES (1, 'innodb'), (3, 'innodb'); +# Restart the server +-- source include/restart_mysqld.inc +INSERT INTO T1(C2) VALUES ('innodb'); +SHOW CREATE TABLE T1; +DROP TABLE T1; + +## +# 47720: REPLACE INTO Autoincrement column with negative values +# +CREATE TABLE t1 (c1 INT AUTO_INCREMENT PRIMARY KEY) ENGINE=InnoDB; +INSERT INTO t1 SET c1 = 1; +SHOW CREATE TABLE t1; +INSERT INTO t1 SET c1 = 2; +INSERT INTO t1 SET c1 = -1; +SELECT * FROM t1; +-- error ER_DUP_ENTRY,1062 +INSERT INTO t1 SET c1 = -1; +SHOW CREATE TABLE t1; +REPLACE INTO t1 VALUES (-1); +SELECT * FROM t1; +SHOW CREATE TABLE t1; +DROP TABLE t1; + +--disable_query_log +EVAL SET GLOBAL innodb_file_format_check=$file_format_check; +--enable_query_log === added file 'mysql-test/t/innodb-consistent-master.opt' --- a/mysql-test/t/innodb-consistent-master.opt 1970-01-01 00:00:00 +0000 +++ b/mysql-test/t/innodb-consistent-master.opt 2010-01-15 15:58:25 +0000 @@ -0,0 +1 @@ +--loose-innodb_lock_wait_timeout=2 === added file 'mysql-test/t/innodb-consistent.test' --- a/mysql-test/t/innodb-consistent.test 1970-01-01 00:00:00 +0000 +++ b/mysql-test/t/innodb-consistent.test 2010-01-15 15:58:25 +0000 @@ -0,0 +1,58 @@ +-- source include/not_embedded.inc +-- source include/have_innodb.inc + +--disable_warnings +drop table if exists t1; +--enable_warnings + +# REPLACE INTO ... SELECT and INSERT INTO ... SELECT should do +# a consistent read of the source table. + +connect (a,localhost,root,,); +connect (b,localhost,root,,); +connection a; +set session transaction isolation level read committed; +create table t1(a int not null) engine=innodb DEFAULT CHARSET=latin1; +create table t2 like t1; +insert into t2 values (1),(2),(3),(4),(5),(6),(7); +set autocommit=0; + +# REPLACE INTO ... SELECT case +begin; +# this should not result in any locks on t2. +replace into t1 select * from t2; + +connection b; +set session transaction isolation level read committed; +set autocommit=0; +# should not cuase a lock wait. +delete from t2 where a=5; +commit; +delete from t2; +commit; +connection a; +commit; + +# INSERT INTO ... SELECT case +begin; +# this should not result in any locks on t2. +insert into t1 select * from t2; + +connection b; +set session transaction isolation level read committed; +set autocommit=0; +# should not cuase a lock wait. +delete from t2 where a=5; +commit; +delete from t2; +commit; +connection a; +commit; + +select * from t1; +drop table t1; +drop table t2; + +connection default; +disconnect a; +disconnect b; === modified file 'mysql-test/t/innodb-index.test' --- a/mysql-test/t/innodb-index.test 2009-11-30 21:37:27 +0000 +++ b/mysql-test/t/innodb-index.test 2010-01-15 15:58:25 +0000 @@ -1,6 +1,6 @@ -- source include/have_innodb.inc -SET @save_innodb_file_format_check=@@global.innodb_file_format_check; +let $innodb_file_format_check_orig=`select @@innodb_file_format_check`; create table t1(a int not null, b int, c char(10) not null, d varchar(20)) engine = innodb; insert into t1 values (5,5,'oo','oo'),(4,4,'tr','tr'),(3,4,'ad','ad'),(2,3,'ak','ak'); @@ -404,6 +404,7 @@ create index t1u on t1 (u(1)); drop table t1; eval set global innodb_file_per_table=$per_table; eval set global innodb_file_format=$format; +eval set global innodb_file_format_check=$format; # # Test to check whether CREATE INDEX handles implicit foreign key @@ -541,4 +542,9 @@ disconnect b; DROP TABLE t1; -SET GLOBAL innodb_file_format_check=@save_innodb_file_format_check; +# +# restore environment to the state it was before this test execution +# + +-- disable_query_log +eval SET GLOBAL innodb_file_format_check=$innodb_file_format_check_orig; === modified file 'mysql-test/t/innodb-master.opt' --- a/mysql-test/t/innodb-master.opt 2009-06-09 13:19:13 +0000 +++ b/mysql-test/t/innodb-master.opt 2010-01-15 15:58:25 +0000 @@ -1 +1 @@ ---binlog_cache_size=32768 --innodb_lock_wait_timeout=1 +--binlog_cache_size=32768 --loose_innodb_lock_wait_timeout=1 === modified file 'mysql-test/t/innodb-semi-consistent-master.opt' --- a/mysql-test/t/innodb-semi-consistent-master.opt 2009-06-09 13:19:13 +0000 +++ b/mysql-test/t/innodb-semi-consistent-master.opt 2010-01-15 15:58:25 +0000 @@ -1 +1 @@ ---innodb_lock_wait_timeout=2 +--loose-innodb_lock_wait_timeout=2 === modified file 'mysql-test/t/innodb-timeout.test' --- a/mysql-test/t/innodb-timeout.test 2009-06-09 13:19:13 +0000 +++ b/mysql-test/t/innodb-timeout.test 2010-01-18 12:56:10 +0000 @@ -30,6 +30,7 @@ begin; insert into t1 values(1),(2),(3); connection b; +set innodb_lock_wait_timeout=5; --send select * from t1 for update; @@ -44,16 +45,20 @@ begin; insert into t1 values(4); connection b; ---send +# Test that we get a lock timeout. +# We cannot reliably test that the timeout is exactly 1 seconds due to +# process scheduling differences on the host running the test suite. But we +# can test that we are within reasonable range. +set innodb_lock_wait_timeout=2; +set @a= current_timestamp(); +--error ER_LOCK_WAIT_TIMEOUT select * from t1 for update; +set @b= current_timestamp(); +set @c= timestampdiff(SECOND, @a, @b); +select if(@c >= 1 and @c <= 10, 'OK', concat("NOT OK, time passed=", @c)); connection a; -sleep 2; commit; - -connection b; ---error ER_LOCK_WAIT_TIMEOUT -reap; drop table t1; connection default; === modified file 'mysql-test/t/innodb-use-sys-malloc-master.opt' --- a/mysql-test/t/innodb-use-sys-malloc-master.opt 2009-06-09 13:19:13 +0000 +++ b/mysql-test/t/innodb-use-sys-malloc-master.opt 2010-01-15 15:58:25 +0000 @@ -1,2 +1 @@ ---innodb-use-sys-malloc=true ---innodb-use-sys-malloc=true +--loose-innodb-use-sys-malloc=true === modified file 'mysql-test/t/innodb-zip.test' --- a/mysql-test/t/innodb-zip.test 2009-06-09 15:08:46 +0000 +++ b/mysql-test/t/innodb-zip.test 2010-01-15 15:58:25 +0000 @@ -178,11 +178,11 @@ set innodb_strict_mode = on; --error ER_CANT_CREATE_TABLE create table t1 (id int primary key) engine = innodb key_block_size = 0; -show errors; +show warnings; --error ER_CANT_CREATE_TABLE create table t2 (id int primary key) engine = innodb key_block_size = 9; -show errors; +show warnings; create table t3 (id int primary key) engine = innodb key_block_size = 1; @@ -208,22 +208,22 @@ key_block_size = 8 row_format = compress --error ER_CANT_CREATE_TABLE create table t2 (id int primary key) engine = innodb key_block_size = 8 row_format = redundant; -show errors; +show warnings; --error ER_CANT_CREATE_TABLE create table t3 (id int primary key) engine = innodb key_block_size = 8 row_format = compact; -show errors; +show warnings; --error ER_CANT_CREATE_TABLE create table t4 (id int primary key) engine = innodb key_block_size = 8 row_format = dynamic; -show errors; +show warnings; --error ER_CANT_CREATE_TABLE create table t5 (id int primary key) engine = innodb key_block_size = 8 row_format = default; -show errors; +show warnings; SELECT table_schema, table_name, row_format FROM information_schema.tables WHERE engine='innodb'; @@ -233,17 +233,17 @@ drop table t1; --error ER_CANT_CREATE_TABLE create table t1 (id int primary key) engine = innodb key_block_size = 9 row_format = redundant; -show errors; +show warnings; --error ER_CANT_CREATE_TABLE create table t2 (id int primary key) engine = innodb key_block_size = 9 row_format = compact; -show errors; +show warnings; --error ER_CANT_CREATE_TABLE create table t2 (id int primary key) engine = innodb key_block_size = 9 row_format = dynamic; -show errors; +show warnings; SELECT table_schema, table_name, row_format FROM information_schema.tables WHERE engine='innodb'; @@ -253,25 +253,25 @@ set global innodb_file_per_table = off; --error ER_CANT_CREATE_TABLE create table t1 (id int primary key) engine = innodb key_block_size = 1; -show errors; +show warnings; --error ER_CANT_CREATE_TABLE create table t2 (id int primary key) engine = innodb key_block_size = 2; -show errors; +show warnings; --error ER_CANT_CREATE_TABLE create table t3 (id int primary key) engine = innodb key_block_size = 4; -show errors; +show warnings; --error ER_CANT_CREATE_TABLE create table t4 (id int primary key) engine = innodb key_block_size = 8; -show errors; +show warnings; --error ER_CANT_CREATE_TABLE create table t5 (id int primary key) engine = innodb key_block_size = 16; -show errors; +show warnings; --error ER_CANT_CREATE_TABLE create table t6 (id int primary key) engine = innodb row_format = compressed; -show errors; +show warnings; --error ER_CANT_CREATE_TABLE create table t7 (id int primary key) engine = innodb row_format = dynamic; -show errors; +show warnings; create table t8 (id int primary key) engine = innodb row_format = compact; create table t9 (id int primary key) engine = innodb row_format = redundant; @@ -285,25 +285,25 @@ set global innodb_file_format = `0`; --error ER_CANT_CREATE_TABLE create table t1 (id int primary key) engine = innodb key_block_size = 1; -show errors; +show warnings; --error ER_CANT_CREATE_TABLE create table t2 (id int primary key) engine = innodb key_block_size = 2; -show errors; +show warnings; --error ER_CANT_CREATE_TABLE create table t3 (id int primary key) engine = innodb key_block_size = 4; -show errors; +show warnings; --error ER_CANT_CREATE_TABLE create table t4 (id int primary key) engine = innodb key_block_size = 8; -show errors; +show warnings; --error ER_CANT_CREATE_TABLE create table t5 (id int primary key) engine = innodb key_block_size = 16; -show errors; +show warnings; --error ER_CANT_CREATE_TABLE create table t6 (id int primary key) engine = innodb row_format = compressed; -show errors; +show warnings; --error ER_CANT_CREATE_TABLE create table t7 (id int primary key) engine = innodb row_format = dynamic; -show errors; +show warnings; create table t8 (id int primary key) engine = innodb row_format = compact; create table t9 (id int primary key) engine = innodb row_format = redundant; === modified file 'mysql-test/t/innodb.test' --- a/mysql-test/t/innodb.test 2009-12-27 13:54:41 +0000 +++ b/mysql-test/t/innodb.test 2010-01-15 15:58:25 +0000 @@ -2270,7 +2270,7 @@ disconnect j; drop table t1, t2, t3, t5, t6, t8, t9; # bug 18934, "InnoDB crashes when table uses column names like DB_ROW_ID" ---error 1005 +--error ER_WRONG_COLUMN_NAME CREATE TABLE t1 (DB_ROW_ID int) engine=innodb; # === modified file 'mysql-test/t/innodb_bug34300.test' --- a/mysql-test/t/innodb_bug34300.test 2009-11-13 21:26:08 +0000 +++ b/mysql-test/t/innodb_bug34300.test 2010-01-15 15:58:25 +0000 @@ -9,7 +9,7 @@ -- disable_result_log # set packet size and reconnect -SET @save_max_allowed_packet=@@global.max_allowed_packet; +let $max_packet=`select @@global.max_allowed_packet`; SET @@global.max_allowed_packet=16777216; --connect (newconn, localhost, root,,) @@ -33,4 +33,4 @@ SELECT f4, f8 FROM bug34300; DROP TABLE bug34300; disconnect newconn; connection default; -SET @@global.max_allowed_packet=@save_max_allowed_packet; +EVAL SET @@global.max_allowed_packet=$max_packet; === modified file 'mysql-test/t/innodb_bug36169.test' --- a/mysql-test/t/innodb_bug36169.test 2009-11-13 21:26:08 +0000 +++ b/mysql-test/t/innodb_bug36169.test 2010-01-15 15:58:25 +0000 @@ -4,9 +4,9 @@ # -- source include/have_innodb.inc -set @old_innodb_file_format=@@innodb_file_format; -set @old_innodb_file_per_table=@@innodb_file_per_table; +let $file_format=`select @@innodb_file_format`; +let $file_per_table=`select @@innodb_file_per_table`; SET GLOBAL innodb_file_format='Barracuda'; SET GLOBAL innodb_file_per_table=ON; @@ -1155,5 +1155,5 @@ DROP TABLE IF EXISTS table4; DROP TABLE IF EXISTS table5; DROP TABLE IF EXISTS table6; -set global innodb_file_format=@old_innodb_file_format; -set global innodb_file_per_table=@old_innodb_file_per_table; +EVAL SET GLOBAL innodb_file_format=$file_format; +EVAL SET GLOBAL innodb_file_per_table=$file_per_table; === modified file 'mysql-test/t/innodb_bug36172.test' --- a/mysql-test/t/innodb_bug36172.test 2009-11-13 21:26:08 +0000 +++ b/mysql-test/t/innodb_bug36172.test 2010-01-15 15:58:25 +0000 @@ -13,9 +13,10 @@ SET storage_engine=InnoDB; -- disable_query_log -- disable_result_log -set @old_innodb_file_per_table=@@innodb_file_per_table; -set @old_innodb_file_format=@@innodb_file_format; +let $file_format=`select @@innodb_file_format`; +let $file_format_check=`select @@innodb_file_format_check`; +let $file_per_table=`select @@innodb_file_per_table`; SET GLOBAL innodb_file_format='Barracuda'; SET GLOBAL innodb_file_per_table=on; @@ -27,6 +28,6 @@ INSERT IGNORE INTO `table0` SET `col19` CHECK TABLE table0 EXTENDED; DROP TABLE table0; -set global innodb_file_per_table=@old_innodb_file_per_table; -set global innodb_file_format=@old_innodb_file_format; -set global innodb_file_format_check=Antelope; +EVAL SET GLOBAL innodb_file_format=$file_format; +EVAL SET GLOBAL innodb_file_format_check=$file_format_check; +EVAL SET GLOBAL innodb_file_per_table=$file_per_table; === modified file 'mysql-test/t/innodb_bug42101-nonzero-master.opt' --- a/mysql-test/t/innodb_bug42101-nonzero-master.opt 2009-05-19 08:20:28 +0000 +++ b/mysql-test/t/innodb_bug42101-nonzero-master.opt 2010-01-15 15:58:25 +0000 @@ -1 +1 @@ ---innodb_commit_concurrency=1 +--loose_innodb_commit_concurrency=1 === modified file 'mysql-test/t/innodb_bug44369.test' --- a/mysql-test/t/innodb_bug44369.test 2009-11-02 14:59:44 +0000 +++ b/mysql-test/t/innodb_bug44369.test 2010-01-15 15:58:25 +0000 @@ -6,16 +6,12 @@ --source include/have_innodb.inc # This create table operation should fail. ---error ER_CANT_CREATE_TABLE +--error ER_WRONG_COLUMN_NAME create table bug44369 (DB_ROW_ID int) engine=innodb; # This create should fail as well ---error ER_CANT_CREATE_TABLE +--error ER_WRONG_COLUMN_NAME create table bug44369 (db_row_id int) engine=innodb; -show warnings; - ---error ER_CANT_CREATE_TABLE +--error ER_WRONG_COLUMN_NAME create table bug44369 (db_TRX_Id int) engine=innodb; - -show warnings; === added file 'mysql-test/t/innodb_bug44571.test' --- a/mysql-test/t/innodb_bug44571.test 1970-01-01 00:00:00 +0000 +++ b/mysql-test/t/innodb_bug44571.test 2010-01-15 15:58:25 +0000 @@ -0,0 +1,13 @@ +# +# Bug#44571 InnoDB Plugin crashes on ADD INDEX +# http://bugs.mysql.com/44571 +# +-- source include/have_innodb.inc + +CREATE TABLE bug44571 (foo INT) ENGINE=InnoDB; +ALTER TABLE bug44571 CHANGE foo bar INT; +-- error ER_KEY_COLUMN_DOES_NOT_EXITS +ALTER TABLE bug44571 ADD INDEX bug44571b (foo); +ALTER TABLE bug44571 ADD INDEX bug44571b (bar); +CREATE INDEX bug44571c ON bug44571 (bar); +DROP TABLE bug44571; === added file 'mysql-test/t/innodb_bug46676.test' --- a/mysql-test/t/innodb_bug46676.test 1970-01-01 00:00:00 +0000 +++ b/mysql-test/t/innodb_bug46676.test 2010-01-15 15:58:25 +0000 @@ -0,0 +1,16 @@ +# This is the test for bug 46676: mysqld got exception 0xc0000005 +# It is reproducible with InnoDB plugin 1.0.4 + MySQL 5.1.37. +# But no longer reproducible after MySQL 5.1.38 (with plugin 1.0.5). + +--source include/have_innodb.inc + +SET foreign_key_checks=0; +CREATE TABLE t1 (id int, foreign key (id) references t2(id)) ENGINE=INNODB; +CREATE TABLE t2 (id int, foreign key (id) references t1(id)) ENGINE=INNODB; +SET foreign_key_checks=1; + +# Server crashes +SELECT COUNT(*) FROM information_schema.key_column_usage WHERE REFERENCED_TABLE_NAME in ('t1', 't2'); + +SET foreign_key_checks=0; +DROP TABLE t1, t2; === added file 'mysql-test/t/innodb_bug47167.test' --- a/mysql-test/t/innodb_bug47167.test 1970-01-01 00:00:00 +0000 +++ b/mysql-test/t/innodb_bug47167.test 2010-01-15 15:58:25 +0000 @@ -0,0 +1,45 @@ +# This is the unit test for bug *47167. +# It tests setting the global variable +# "innodb_file_format_check" with a +# user-Defined Variable. + +--source include/have_innodb.inc + +# Save the value (Antelope) in 'innodb_file_format_check' to +# 'old_innodb_file_format_check' +set @old_innodb_file_format_check=@@innodb_file_format_check; + +# @old_innodb_file_format_check shall have the value of 'Antelope' +select @old_innodb_file_format_check; + +# Reset the value in 'innodb_file_format_check' to 'Barracuda' +set global innodb_file_format_check = Barracuda; + +select @@innodb_file_format_check; + +# Set 'innodb_file_format_check' to its default value, which +# is the latest file format supported in the current release. +set global innodb_file_format_check = DEFAULT; + +select @@innodb_file_format_check; + +# Put the saved value back to 'innodb_file_format_check' +set global innodb_file_format_check = @old_innodb_file_format_check; + +# Check whether 'innodb_file_format_check' get its original value. +select @@innodb_file_format_check; + +# Following are negative tests, all should fail. +--disable_warnings +--error ER_WRONG_ARGUMENTS +set global innodb_file_format_check = cheetah; + +--error ER_WRONG_ARGUMENTS +set global innodb_file_format_check = Bear; + +--error ER_WRONG_ARGUMENTS +set global innodb_file_format_check = on; + +--error ER_WRONG_ARGUMENTS +set global innodb_file_format_check = off; +--enable_warnings === modified file 'mysql-test/t/innodb_file_format.test' --- a/mysql-test/t/innodb_file_format.test 2009-11-30 21:37:27 +0000 +++ b/mysql-test/t/innodb_file_format.test 2010-01-15 15:58:25 +0000 @@ -1,5 +1,4 @@ -- source include/have_innodb.inc -set @old_innodb_file_format=@@innodb_file_format; call mtr.add_suppression("InnoDB: invalid innodb_file_format_check value"); @@ -29,6 +28,4 @@ set global innodb_file_format=on; --error ER_WRONG_ARGUMENTS set global innodb_file_format=off; select @@innodb_file_format_check; - -set global innodb_file_format=@old_innodb_file_format; -set global innodb_file_format_check=Antelope; +set global innodb_file_format_check=antelope; === modified file 'mysql-test/t/innodb_information_schema.test' --- a/mysql-test/t/innodb_information_schema.test 2009-06-23 12:00:24 +0000 +++ b/mysql-test/t/innodb_information_schema.test 2010-01-15 15:58:25 +0000 @@ -109,19 +109,18 @@ SELECT * FROM ```t'\"_str` WHERE c1 = '3 -- send SELECT * FROM ```t'\"_str` WHERE c1 = '4' FOR UPDATE; +-- enable_result_log -- connection con_verify_innodb_locks - -# Loop, giving time for the above 2 queries to execute before continuing. -# Without this, it sometimes happens that the SELECT FROM innodb_locks +# Wait for the above queries to execute before continuing. +# Without this, it sometimes happens that the SELECT from innodb_locks # executes before some of them, resulting in less than expected number -# of rows being selected from innodb_locks. -SET @counter := 0; -while (`SELECT (@counter := @counter + 1) <= 50 AND COUNT(*) != 14 FROM INFORMATION_SCHEMA.INNODB_LOCKS`) -{ - sleep 0.1; -} - --- enable_result_log +# of rows being selected from innodb_locks. If there is a bug and there +# are no 14 rows in innodb_locks then this test will fail with timeout. +let $count = 14; +let $table = INFORMATION_SCHEMA.INNODB_LOCKS; +-- source include/wait_until_rows_count.inc +# the above enables the query log, re-disable it +-- disable_query_log SELECT lock_mode, lock_type, lock_table, lock_index, lock_rec, lock_data FROM INFORMATION_SCHEMA.INNODB_LOCKS ORDER BY lock_data; === modified file 'mysql-test/t/innodb_lock_wait_timeout_1.test' --- a/mysql-test/t/innodb_lock_wait_timeout_1.test 2009-11-03 17:45:52 +0000 +++ b/mysql-test/t/innodb_lock_wait_timeout_1.test 2009-11-12 11:43:33 +0000 @@ -71,6 +71,40 @@ set autocommit=default; drop table t1; --echo # +--echo # Bug #37183 insert ignore into .. select ... hangs +--echo # after deadlock was encountered +--echo # +connect (con1,localhost,root,,); +create table t1(id int primary key,v int)engine=innodb; +insert into t1 values (1,1),(2,2),(3,3),(4,4),(5,5),(6,6),(7,7); +create table t2 like t1; + +--connection con1 +begin; +update t1 set v=id*2 where id=1; + +--connection default +begin; +update t1 set v=id*2 where id=2; + +--connection con1 +--error 1205 +update t1 set v=id*2 where id=2; + +--connection default +--error 1205 +insert ignore into t2 select * from t1 where id=1; +rollback; + +--connection con1 +rollback; + +--connection default +disconnect con1; +drop table t1, t2; + + +--echo # --echo # Bug#41756 Strange error messages about locks from InnoDB --echo # --disable_warnings === modified file 'mysql-test/t/innodb_mysql.test' --- a/mysql-test/t/innodb_mysql.test 2009-12-03 11:19:05 +0000 +++ b/mysql-test/t/innodb_mysql.test 2010-01-15 15:27:55 +0000 @@ -491,5 +491,51 @@ EXPLAIN SELECT * FROM t1 WHERE a = 'TEST c >= '2009-10-09 00:00:00.001' AND c <= '2009-10-09 00:00:00.00'; DROP TABLE t1; +--echo # +--echo # Bug #46175: NULL read_view and consistent read assertion +--echo # + +CREATE TABLE t1(a CHAR(13),KEY(a)) ENGINE=innodb; +CREATE TABLE t2(b DATETIME,KEY(b)) ENGINE=innodb; +INSERT INTO t1 VALUES (),(); +INSERT INTO t2 VALUES (),(); +CREATE OR REPLACE VIEW v1 AS SELECT 1 FROM t2 + WHERE b =(SELECT a FROM t1 LIMIT 1); + +--disable_query_log +--disable_result_log +CONNECT (con1, localhost, root,,); +--enable_query_log +--enable_result_log +CONNECTION default; + +DELIMITER |; +CREATE PROCEDURE p1(num INT) +BEGIN + DECLARE i INT DEFAULT 0; + REPEAT + SHOW CREATE VIEW v1; + SET i:=i+1; + UNTIL i>num END REPEAT; +END| +DELIMITER ;| + +--echo # Should not crash +--disable_query_log +--disable_result_log +--send CALL p1(1000) +CONNECTION con1; +--echo # Should not crash +CALL p1(1000); + +CONNECTION default; +--reap +--enable_query_log +--enable_result_log + +DISCONNECT con1; +DROP PROCEDURE p1; +DROP VIEW v1; +DROP TABLE t1,t2; --echo End of 5.1 tests === modified file 'mysql-test/t/innodb_xtradb_bug317074.test' --- a/mysql-test/t/innodb_xtradb_bug317074.test 2009-10-28 07:52:34 +0000 +++ b/mysql-test/t/innodb_xtradb_bug317074.test 2010-01-15 15:58:25 +0000 @@ -2,7 +2,7 @@ SET @old_innodb_file_format=@@innodb_file_format; SET @old_innodb_file_per_table=@@innodb_file_per_table; -SET @old_innodb_file_format_check=@@innodb_file_format_check; +let $innodb_file_format_check_orig=`select @@innodb_file_format_check`; SET GLOBAL innodb_file_format='Barracuda'; SET GLOBAL innodb_file_per_table=ON; @@ -45,4 +45,4 @@ ALTER TABLE test1 ENGINE=MyISAM; DROP TABLE test1; SET GLOBAL innodb_file_format=@old_innodb_file_format; SET GLOBAL innodb_file_per_table=@old_innodb_file_per_table; -SET GLOBAL innodb_file_format_check=@old_innodb_file_format_check; +eval set global innodb_file_format_check=$innodb_file_format_check_orig; === modified file 'mysql-test/t/mysql.test' --- a/mysql-test/t/mysql.test 2009-10-05 13:22:23 +0000 +++ b/mysql-test/t/mysql.test 2010-01-15 15:27:55 +0000 @@ -386,10 +386,16 @@ drop tables t1, t2; # # Bug #27884: mysql --html does not quote HTML special characters in output # ---exec $MYSQL --html test -e "select '< & >' as '<'" +--write_file $MYSQLTEST_VARDIR/tmp/bug27884.sql +SELECT '< & >' AS `<`; +EOF +--exec $MYSQL --html test < $MYSQLTEST_VARDIR/tmp/bug27884.sql + +remove_file $MYSQLTEST_VARDIR/tmp/bug27884.sql; + # -# Bug #27884: mysql client + null byte +# Bug #28203: mysql client + null byte # create table t1 (a char(5)); insert into t1 values ('\0b\0'); @@ -402,5 +408,5 @@ insert into t1 values ('\0b\0'); --exec $MYSQL --xml test -e "select a from t1" drop table t1; ---echo ---echo End of tests + +--echo End of 5.0 tests === added file 'mysql-test/t/mysqlbinlog2-master.opt' --- a/mysql-test/t/mysqlbinlog2-master.opt 1970-01-01 00:00:00 +0000 +++ b/mysql-test/t/mysqlbinlog2-master.opt 2010-01-19 10:36:52 +0000 @@ -0,0 +1 @@ +--timezone=GMT-3 === modified file 'mysql-test/t/olap.test' --- a/mysql-test/t/olap.test 2009-10-30 15:54:53 +0000 +++ b/mysql-test/t/olap.test 2009-12-08 09:26:11 +0000 @@ -390,4 +390,17 @@ SELECT DISTINCT b FROM t1, t2 GROUP BY a DROP TABLE t1, t2; +--echo # +--echo # Bug #48475: DISTINCT is ignored with GROUP BY WITH ROLLUP +--echo # and only const tables + +CREATE TABLE t1 (a INT); +CREATE TABLE t2 (b INT); +INSERT INTO t1 VALUES (1); +INSERT INTO t2 VALUES (1); + +SELECT DISTINCT b FROM t1, t2 GROUP BY a, b WITH ROLLUP; + +DROP TABLE t1, t2; + --echo End of 5.0 tests === modified file 'mysql-test/t/order_by.test' --- a/mysql-test/t/order_by.test 2009-12-03 11:19:05 +0000 +++ b/mysql-test/t/order_by.test 2010-01-15 15:27:55 +0000 @@ -869,6 +869,31 @@ SELECT DROP TABLE t1, t2, t3; +--echo # +--echo # Bug #42760: Select doesn't return desired results when we have null +--echo # values +--echo # + +CREATE TABLE t1 ( + a INT, + c INT, + UNIQUE KEY a_c (a,c), + KEY (a)); + +INSERT INTO t1 VALUES (1, 10), (2, NULL); + +--echo # Must use ref-or-null on the a_c index +EXPLAIN +SELECT 1 AS col FROM t1 WHERE a=2 AND (c=10 OR c IS NULL) ORDER BY c; +--echo # Must return 1 row +SELECT 1 AS col FROM t1 WHERE a=2 AND (c=10 OR c IS NULL) ORDER BY c; + +DROP TABLE t1; + + +--echo End of 5.0 tests + + # # Bug #35206: select query result different if the key is indexed or not # === modified file 'mysql-test/t/partition.test' --- a/mysql-test/t/partition.test 2009-12-03 11:19:05 +0000 +++ b/mysql-test/t/partition.test 2010-01-15 15:27:55 +0000 @@ -15,6 +15,15 @@ drop table if exists t1, t2; --enable_warnings # +# Bug#48276: can't add column if subpartition exists +CREATE TABLE t1 (a INT, b INT) +PARTITION BY LIST (a) +SUBPARTITION BY HASH (b) +(PARTITION p1 VALUES IN (1)); +ALTER TABLE t1 ADD COLUMN c INT; +DROP TABLE t1; + +# # Bug#46639: 1030 (HY000): Got error 124 from storage engine on # INSERT ... SELECT ... CREATE TABLE t1 ( @@ -62,6 +71,17 @@ SHOW CREATE TABLE t1; DROP TABLE t1; # +# Bug#45904: Error when CHARSET=utf8 and subpartitioning +# +create table t1 (a int NOT NULL, b varchar(5) NOT NULL) +default charset=utf8 +partition by list (a) +subpartition by key (b) +(partition p0 values in (1), + partition p1 values in (2)); +drop table t1; + +# # Bug#44059: rec_per_key on empty partition gives weird optimiser results # create table t1 (a int, b int, key(a)) @@ -2035,11 +2055,14 @@ DROP TABLE t1; --echo # --echo # Bug #45807: crash accessing partitioned table and sql_mode --echo # contains ONLY_FULL_GROUP_BY +--echo # Bug#46923: select count(*) from partitioned table fails with +--echo # ONLY_FULL_GROUP_BY --echo # SET SESSION SQL_MODE='ONLY_FULL_GROUP_BY'; CREATE TABLE t1(id INT,KEY(id)) ENGINE=MYISAM PARTITION BY HASH(id) PARTITIONS 2; +SELECT COUNT(*) FROM t1; DROP TABLE t1; SET SESSION SQL_MODE=DEFAULT; === modified file 'mysql-test/t/range.test' --- a/mysql-test/t/range.test 2009-11-02 12:24:07 +0000 +++ b/mysql-test/t/range.test 2009-12-08 09:26:11 +0000 @@ -1260,4 +1260,57 @@ SELECT str_to_date('', '%Y-%m-%d'); DROP TABLE t1, t2; +--echo # +--echo # Bug#48459: valgrind errors with query using 'Range checked for each +--echo # record' +--echo # +CREATE TABLE t1 ( + a INT, + b CHAR(2), + c INT, + d INT, + KEY ( c ), + KEY ( d, a, b ( 2 ) ), + KEY ( b ( 1 ) ) +); + +INSERT INTO t1 VALUES ( NULL, 'a', 1, 2 ), ( NULL, 'a', 1, 2 ), + ( 1, 'a', 1, 2 ), ( 1, 'a', 1, 2 ); + +CREATE TABLE t2 ( + a INT, + c INT, + e INT, + KEY ( e ) +); + +INSERT INTO t2 VALUES ( 1, 1, NULL ), ( 1, 1, NULL ); + +--echo # Should not give Valgrind warnings +SELECT 1 +FROM t1, t2 +WHERE t1.d <> '1' AND t1.b > '1' +AND t1.a = t2.a AND t1.c = t2.c; + +DROP TABLE t1, t2; + +--echo # +--echo # Bug #48665: sql-bench's insert test fails due to wrong result +--echo # + +CREATE TABLE t1 (a INT, b INT, PRIMARY KEY (a)); + +INSERT INTO t1 VALUES (0,0), (1,1); + +--replace_column 1 @ 2 @ 3 @ 5 @ 6 @ 7 @ 8 @ 9 @ 10 @ +EXPLAIN +SELECT * FROM t1 FORCE INDEX (PRIMARY) + WHERE (a>=1 AND a<=2) OR (a>=4 AND a<=5) OR (a>=0 AND a <=10); + +--echo # Should return 2 rows +SELECT * FROM t1 FORCE INDEX (PRIMARY) + WHERE (a>=1 AND a<=2) OR (a>=4 AND a<=5) OR (a>=0 AND a <=10); + +DROP TABLE t1; + --echo End of 5.1 tests === modified file 'mysql-test/t/select.test' --- a/mysql-test/t/select.test 2009-10-30 14:13:13 +0000 +++ b/mysql-test/t/select.test 2009-12-15 17:08:21 +0000 @@ -3772,6 +3772,19 @@ INTO @var0; DROP TABLE t1; +--echo # +--echo # Bug #48458: simple query tries to allocate enormous amount of +--echo # memory +--echo # + +CREATE TABLE t1(a INT NOT NULL, b YEAR); +INSERT INTO t1 VALUES (); +CREATE TABLE t2(c INT); +--echo # Should not err out because of out-of-memory +SELECT 1 FROM t2 JOIN t1 ON 1=1 + WHERE a != '1' AND NOT a >= b OR NOT ROW(b,a )<> ROW(a,a); +DROP TABLE t1,t2; + --echo End of 5.0 tests @@ -3918,4 +3931,60 @@ SELECT table1 .`time_key` field2 FROM B drop table A,AA,B,BB; --echo #end of test for bug#45266 + +--echo # +--echo # BUG#48052: Valgrind warning - uninitialized value in init_read_record() +--echo # + +# Needed in 6.0 codebase +#--echo # Disable Index condition pushdown +#--replace_column 1 # +#SELECT @old_icp:=@@engine_condition_pushdown; +#SET SESSION engine_condition_pushdown = 'OFF'; + +CREATE TABLE t1 ( + pk int(11) NOT NULL, + i int(11) DEFAULT NULL, + v varchar(1) DEFAULT NULL, + PRIMARY KEY (pk) +); + +INSERT INTO t1 VALUES (2,7,'m'); +INSERT INTO t1 VALUES (3,9,'m'); + +SELECT v +FROM t1 +WHERE NOT pk > 0 +HAVING v <= 't' +ORDER BY pk; + +# Needed in 6.0 codebase +#--echo # Restore old value for Index condition pushdown +#SET SESSION engine_condition_pushdown=@old_icp; + +DROP TABLE t1; + +--echo # +--echo # Bug#49489 Uninitialized cache led to a wrong result. +--echo # +CREATE TABLE t1(c1 DOUBLE(5,4)); +INSERT INTO t1 VALUES (9.1234); +SELECT * FROM t1 WHERE c1 < 9.12345; +DROP TABLE t1; +--echo # End of test for bug#49489. + + +--echo # +--echo # Bug #49517: Inconsistent behavior while using +--echo # NULLable BIGINT and INT columns in comparison +--echo # +CREATE TABLE t1(a BIGINT UNSIGNED NOT NULL, b BIGINT NULL, c INT NULL); +INSERT INTO t1 VALUES(105, NULL, NULL); +SELECT * FROM t1 WHERE b < 102; +SELECT * FROM t1 WHERE c < 102; +SELECT * FROM t1 WHERE 102 < b; +SELECT * FROM t1 WHERE 102 < c; +DROP TABLE t1; + + --echo End of 5.1 tests === modified file 'mysql-test/t/show_check.test' --- a/mysql-test/t/show_check.test 2009-03-06 14:56:17 +0000 +++ b/mysql-test/t/show_check.test 2009-12-15 09:03:24 +0000 @@ -1207,6 +1207,28 @@ connection default; DROP USER test_u@localhost; +--echo # +--echo # Bug #48985: show create table crashes if previous access to the table +--echo # was killed +--echo # + +connect(con1,localhost,root,,); +CONNECTION con1; +LET $ID= `SELECT connection_id()`; + +CONNECTION default; +--disable_query_log +eval KILL QUERY $ID; +--enable_query_log + +CONNECTION con1; +--error ER_QUERY_INTERRUPTED +SHOW CREATE TABLE non_existent; + +CONNECTION default; +DISCONNECT con1; + + --echo End of 5.1 tests # Wait till all disconnects are completed === modified file 'mysql-test/t/sp-destruct.test' --- a/mysql-test/t/sp-destruct.test 2008-04-08 14:51:26 +0000 +++ b/mysql-test/t/sp-destruct.test 2009-11-21 11:18:21 +0000 @@ -12,6 +12,9 @@ # mysqltest should be fixed to allow REPLACE_RESULT in error message -- source include/not_embedded.inc +# Supress warnings written to the log file +call mtr.add_suppression("Column count of mysql.proc is wrong. Expected 20, found 19. The table is probably corrupted"); + # Backup proc table let $MYSQLD_DATADIR= `select @@datadir`; --copy_file $MYSQLD_DATADIR/mysql/proc.frm $MYSQLTEST_VARDIR/tmp/proc.frm @@ -38,15 +41,14 @@ create trigger t1_ai after insert on t1 # Unsupported tampering with the mysql.proc definition alter table mysql.proc drop type; ---replace_result $MYSQL_TEST_DIR . ---error ER_SP_PROC_TABLE_CORRUPT +--error ER_COL_COUNT_DOESNT_MATCH_CORRUPTED call bug14233(); ---replace_result $MYSQL_TEST_DIR . ---error ER_SP_PROC_TABLE_CORRUPT +--error ER_COL_COUNT_DOESNT_MATCH_CORRUPTED create view v1 as select bug14233_f(); ---replace_result $MYSQL_TEST_DIR . ---error ER_SP_PROC_TABLE_CORRUPT +--error ER_COL_COUNT_DOESNT_MATCH_CORRUPTED insert into t1 values (0); +--error ER_COL_COUNT_DOESNT_MATCH_CORRUPTED +show procedure status; flush table mysql.proc; @@ -155,3 +157,43 @@ drop procedure bug14233_3; # Assert: These should show nothing. show procedure status where db=DATABASE(); show function status where db=DATABASE(); + +# +# Bug#41726 upgrade from 5.0 to 5.1.30 crashes if you didn't run mysql_upgrade +# + + +--disable_warnings +DROP TABLE IF EXISTS proc_backup; +DROP PROCEDURE IF EXISTS p1; +--enable_warnings + +--echo # Backup the proc table + +RENAME TABLE mysql.proc TO proc_backup; +CREATE TABLE mysql.proc LIKE proc_backup; +FLUSH TABLE mysql.proc; + +--echo # Test with a valid table. + +CREATE PROCEDURE p1() + SET @foo = 10; +CALL p1(); +--replace_column 5 '0000-00-00 00:00:00' 6 '0000-00-00 00:00:00' +SHOW PROCEDURE STATUS; + +--echo # Modify a field of the table. + +ALTER TABLE mysql.proc MODIFY comment CHAR (32); + +--error ER_CANNOT_LOAD_FROM_TABLE +CREATE PROCEDURE p2() + SET @foo = 10; +--echo # Procedure loaded from the cache +CALL p1(); +--error ER_CANNOT_LOAD_FROM_TABLE +SHOW PROCEDURE STATUS; + +DROP TABLE mysql.proc; +RENAME TABLE proc_backup TO mysql.proc; +FLUSH TABLE mysql.proc; === modified file 'mysql-test/t/sp-security.test' --- a/mysql-test/t/sp-security.test 2009-03-06 14:56:17 +0000 +++ b/mysql-test/t/sp-security.test 2009-11-27 16:10:28 +0000 @@ -865,6 +865,65 @@ DROP PROCEDURE p_suid; DROP FUNCTION f_suid; DROP TABLE t1; +--echo # +--echo # Bug #48872 : Privileges for stored functions ignored if function name +--echo # is mixed case +--echo # + +CREATE DATABASE B48872; +USE B48872; +CREATE TABLE `TestTab` (id INT); +INSERT INTO `TestTab` VALUES (1),(2); +CREATE FUNCTION `f_Test`() RETURNS INT RETURN 123; +CREATE FUNCTION `f_Test_denied`() RETURNS INT RETURN 123; +CREATE USER 'tester'; +CREATE USER 'Tester'; +GRANT SELECT ON TABLE `TestTab` TO 'tester'; +GRANT EXECUTE ON FUNCTION `f_Test` TO 'tester'; +GRANT EXECUTE ON FUNCTION `f_Test_denied` TO 'Tester'; + +SELECT f_Test(); +SELECT * FROM TestTab; + +CONNECT (con_tester,localhost,tester,,B48872); +CONNECT (con_tester_denied,localhost,Tester,,B48872); +CONNECTION con_tester; + +SELECT * FROM TestTab; +SELECT `f_Test`(); +SELECT `F_TEST`(); +SELECT f_Test(); +SELECT F_TEST(); + +CONNECTION con_tester_denied; + +--disable_result_log +--error ER_TABLEACCESS_DENIED_ERROR +SELECT * FROM TestTab; +--error ER_PROCACCESS_DENIED_ERROR +SELECT `f_Test`(); +--error ER_PROCACCESS_DENIED_ERROR +SELECT `F_TEST`(); +--error ER_PROCACCESS_DENIED_ERROR +SELECT f_Test(); +--error ER_PROCACCESS_DENIED_ERROR +SELECT F_TEST(); +--enable_result_log +SELECT `f_Test_denied`(); +SELECT `F_TEST_DENIED`(); + +CONNECTION default; +DISCONNECT con_tester; +DISCONNECT con_tester_denied; +DROP TABLE `TestTab`; +DROP FUNCTION `f_Test`; +DROP FUNCTION `f_Test_denied`; + +USE test; +DROP USER 'tester'; +DROP USER 'Tester'; +DROP DATABASE B48872; + --echo End of 5.0 tests. # Wait till all disconnects are completed === modified file 'mysql-test/t/sp.test' --- a/mysql-test/t/sp.test 2009-10-23 13:54:58 +0000 +++ b/mysql-test/t/sp.test 2009-11-13 01:03:26 +0000 @@ -8263,6 +8263,73 @@ CALL p1; DROP PROCEDURE p1; DROP TABLE t1, t2; +--echo # +--echo # Bug#47627: SET @@{global.session}.local_variable in stored routine causes crash +--echo # Bug#48626: Crash or lost connection using SET for declared variables with @@ +--echo # + +--disable_warnings +DROP PROCEDURE IF EXISTS p1; +DROP PROCEDURE IF EXISTS p2; +DROP PROCEDURE IF EXISTS p3; +--enable_warnings + +delimiter //; + +--error ER_UNKNOWN_SYSTEM_VARIABLE +CREATE PROCEDURE p1() +BEGIN + DECLARE v INT DEFAULT 0; + SET @@SESSION.v= 10; +END// + +CREATE PROCEDURE p2() +BEGIN + DECLARE v INT DEFAULT 0; + SET v= 10; +END// +call p2()// + +--error ER_UNKNOWN_SYSTEM_VARIABLE +CREATE PROCEDURE p3() +BEGIN + DECLARE v INT DEFAULT 0; + SELECT @@SESSION.v; +END// + +--error ER_UNKNOWN_SYSTEM_VARIABLE +CREATE PROCEDURE p4() +BEGIN + DECLARE v INT DEFAULT 0; + SET @@GLOBAL.v= 10; +END// + +CREATE PROCEDURE p5() +BEGIN + DECLARE init_connect INT DEFAULT 0; + SET init_connect= 10; + SET @@GLOBAL.init_connect= 'SELECT 1'; + SET @@SESSION.IDENTITY= 1; + SELECT @@SESSION.IDENTITY; + SELECT @@GLOBAL.init_connect; + SELECT init_connect; +END// + +--error ER_UNKNOWN_SYSTEM_VARIABLE +CREATE PROCEDURE p6() +BEGIN + DECLARE v INT DEFAULT 0; + SET @@v= 0; +END// + +delimiter ;// + +SET @old_init_connect= @@GLOBAL.init_connect; +CALL p5(); +SET @@GLOBAL.init_connect= @old_init_connect; + +DROP PROCEDURE p2; +DROP PROCEDURE p5; --echo # ------------------------------------------------------------------ --echo # -- End of 5.1 tests === modified file 'mysql-test/t/trigger.test' --- a/mysql-test/t/trigger.test 2009-06-22 12:51:33 +0000 +++ b/mysql-test/t/trigger.test 2010-01-12 08:19:48 +0000 @@ -1767,68 +1767,6 @@ drop table t1, t2, t3; disconnect addconroot1; disconnect addconroot2; disconnect addconwithoutdb; -# -# Bug #26162: Trigger DML ignores low_priority_updates setting -# -CREATE TABLE t1 (id INTEGER); -CREATE TABLE t2 (id INTEGER); - -INSERT INTO t2 VALUES (1),(2); - -# trigger that produces the high priority insert, but should be low, adding -# LOW_PRIORITY fixes this -CREATE TRIGGER t1_test AFTER INSERT ON t1 FOR EACH ROW - INSERT INTO t2 VALUES (new.id); - -CONNECT (rl_holder, localhost, root,,); -CONNECT (rl_acquirer, localhost, root,,); -CONNECT (wl_acquirer, localhost, root,,); -CONNECT (rl_contender, localhost, root,,); - -CONNECTION rl_holder; -SELECT GET_LOCK('B26162',120); - -CONNECTION rl_acquirer; ---send -SELECT 'rl_acquirer', GET_LOCK('B26162',120), id FROM t2 WHERE id = 1; - -CONNECTION wl_acquirer; -SET SESSION LOW_PRIORITY_UPDATES=1; -SET GLOBAL LOW_PRIORITY_UPDATES=1; -#need to wait for rl_acquirer to lock on the B26162 lock -sleep 2; ---send -INSERT INTO t1 VALUES (5); - -CONNECTION rl_contender; -# must not "see" the row inserted by the INSERT (as it must run before the -# INSERT) ---send -SELECT 'rl_contender', id FROM t2 WHERE id > 1; - -CONNECTION rl_holder; -#need to wait for wl_acquirer and rl_contender to lock on t2 -sleep 2; -SELECT RELEASE_LOCK('B26162'); - -CONNECTION rl_acquirer; ---reap -SELECT RELEASE_LOCK('B26162'); -CONNECTION wl_acquirer; ---reap -CONNECTION rl_contender; ---reap - -CONNECTION default; -DISCONNECT rl_acquirer; -DISCONNECT wl_acquirer; -DISCONNECT rl_contender; -DISCONNECT rl_holder; - -DROP TRIGGER t1_test; -DROP TABLE t1,t2; -SET SESSION LOW_PRIORITY_UPDATES=DEFAULT; -SET GLOBAL LOW_PRIORITY_UPDATES=DEFAULT; --echo --echo Bug#28502 Triggers that update another innodb table will block --echo on X lock unnecessarily === modified file 'mysql-test/t/trigger_notembedded.test' --- a/mysql-test/t/trigger_notembedded.test 2009-06-25 10:52:50 +0000 +++ b/mysql-test/t/trigger_notembedded.test 2010-01-12 08:19:48 +0000 @@ -875,6 +875,79 @@ DROP TABLE t1; DROP DATABASE mysqltest_db1; USE test; +# +# Bug #26162: Trigger DML ignores low_priority_updates setting +# +CREATE TABLE t1 (id INTEGER); +CREATE TABLE t2 (id INTEGER); + +INSERT INTO t2 VALUES (1),(2); + +# trigger that produces the high priority insert, but should be low, adding +# LOW_PRIORITY fixes this +CREATE TRIGGER t1_test AFTER INSERT ON t1 FOR EACH ROW + INSERT INTO t2 VALUES (new.id); + +CONNECT (rl_holder, localhost, root,,); +CONNECT (rl_acquirer, localhost, root,,); +CONNECT (wl_acquirer, localhost, root,,); +CONNECT (rl_contender, localhost, root,,); + +CONNECTION rl_holder; +SELECT GET_LOCK('B26162',120); + +CONNECTION rl_acquirer; +let $rl_acquirer_thread_id = `SELECT @@pseudo_thread_id`; +--send +SELECT 'rl_acquirer', GET_LOCK('B26162',120), id FROM t2 WHERE id = 1; + +CONNECTION wl_acquirer; +let $wl_acquirer_thread_id = `SELECT @@pseudo_thread_id`; +SET SESSION LOW_PRIORITY_UPDATES=1; +SET GLOBAL LOW_PRIORITY_UPDATES=1; +#need to wait for rl_acquirer to lock on the B26162 lock +let $wait_condition= + SELECT STATE = 'User lock' FROM INFORMATION_SCHEMA.PROCESSLIST + WHERE ID = $rl_acquirer_thread_id; +--source include/wait_condition.inc +--send +INSERT INTO t1 VALUES (5); + +CONNECTION rl_contender; +# Wait until wl_acquirer is waiting for the read lock on t2 to be released. +let $wait_condition= + SELECT STATE = 'Locked' FROM INFORMATION_SCHEMA.PROCESSLIST + WHERE ID = $wl_acquirer_thread_id; +--source include/wait_condition.inc +# must not "see" the row inserted by the INSERT (as it must run before the +# INSERT) +--send +SELECT 'rl_contender', id FROM t2 WHERE id > 1; + +CONNECTION rl_holder; +#need to wait for wl_acquirer and rl_contender to lock on t2 +sleep 2; +SELECT RELEASE_LOCK('B26162'); + +CONNECTION rl_acquirer; +--reap +SELECT RELEASE_LOCK('B26162'); +CONNECTION wl_acquirer; +--reap +CONNECTION rl_contender; +--reap + +CONNECTION default; +DISCONNECT rl_acquirer; +DISCONNECT wl_acquirer; +DISCONNECT rl_contender; +DISCONNECT rl_holder; + +DROP TRIGGER t1_test; +DROP TABLE t1,t2; +SET SESSION LOW_PRIORITY_UPDATES=DEFAULT; +SET GLOBAL LOW_PRIORITY_UPDATES=DEFAULT; + --echo End of 5.0 tests. # === modified file 'mysql-test/t/type_newdecimal.test' --- a/mysql-test/t/type_newdecimal.test 2009-11-02 11:21:39 +0000 +++ b/mysql-test/t/type_newdecimal.test 2009-12-08 09:26:11 +0000 @@ -1286,3 +1286,229 @@ CREATE TABLE t1 SELECT 1 % .123456789123 DESCRIBE t1; SELECT my_col FROM t1; DROP TABLE t1; + +--echo # +--echo # Bug#45261: Crash, stored procedure + decimal +--echo # + +--disable_warnings +DROP TABLE IF EXISTS t1; +--enable_warnings + +CREATE TABLE t1 SELECT + /* 81 */ 100000000000000000000000000000000000000000000000000000000000000000000000000000001 + AS c1; +DESC t1; +SELECT * FROM t1; +DROP TABLE t1; + +CREATE TABLE t1 SELECT + /* 81 */ 100000000000000000000000000000000000000000000000000000000000000000000000000000001. + AS c1; +DESC t1; +SELECT * FROM t1; +DROP TABLE t1; + +CREATE TABLE t1 SELECT + /* 81 */ 100000000000000000000000000000000000000000000000000000000000000000000000000000001.1 /* 1 */ + AS c1; +DESC t1; +SELECT * FROM t1; +DROP TABLE t1; + +CREATE TABLE t1 SELECT + /* 82 */ 1000000000000000000000000000000000000000000000000000000000000000000000000000000001 + AS c1; +DESC t1; +SELECT * FROM t1; +DROP TABLE t1; + +CREATE TABLE t1 SELECT + /* 40 */ 1000000000000000000000000000000000000001.1000000000000000000000000000000000000001 /* 40 */ + AS c1; +DESC t1; +SELECT * FROM t1; +DROP TABLE t1; + +CREATE TABLE t1 SELECT + /* 1 */ 1.10000000000000000000000000000000000000000000000000000000000000000000000000000001 /* 80 */ + AS c1; +DESC t1; +SELECT * FROM t1; +DROP TABLE t1; + +CREATE TABLE t1 SELECT + /* 1 */ 1.100000000000000000000000000000000000000000000000000000000000000000000000000000001 /* 81 */ + AS c1; +DESC t1; +SELECT * FROM t1; +DROP TABLE t1; + +CREATE TABLE t1 SELECT + .100000000000000000000000000000000000000000000000000000000000000000000000000000001 /* 81 */ + AS c1; +DESC t1; +SELECT * FROM t1; +DROP TABLE t1; + +CREATE TABLE t1 SELECT + /* 45 */ 123456789012345678901234567890123456789012345.123456789012345678901234567890123456789012345 /* 45 */ + AS c1; +DESC t1; +SELECT * FROM t1; +DROP TABLE t1; + +CREATE TABLE t1 SELECT + /* 65 */ 12345678901234567890123456789012345678901234567890123456789012345.1 /* 1 */ + AS c1; +DESC t1; +SELECT * FROM t1; +DROP TABLE t1; + +CREATE TABLE t1 SELECT + /* 66 */ 123456789012345678901234567890123456789012345678901234567890123456.1 /* 1 */ + AS c1; +DESC t1; +SELECT * FROM t1; +DROP TABLE t1; + +CREATE TABLE t1 SELECT + .123456789012345678901234567890123456789012345678901234567890123456 /* 66 */ + AS c1; +DESC t1; +SELECT * FROM t1; +DROP TABLE t1; + +CREATE TABLE t1 AS SELECT 123.1234567890123456789012345678901 /* 31 */ AS c1; +DESC t1; +SELECT * FROM t1; +DROP TABLE t1; + +CREATE TABLE t1 SELECT 1.1 + CAST(1 AS DECIMAL(65,30)) AS c1; +DESC t1; +SELECT * FROM t1; +DROP TABLE t1; + +--echo # +--echo # Test that the integer and decimal parts are properly calculated. +--echo # + +CREATE TABLE t1 (a DECIMAL(30,30)); +INSERT INTO t1 VALUES (0.1),(0.2),(0.3); +CREATE TABLE t2 SELECT MIN(a + 0.0000000000000000000000000000001) AS c1 FROM t1; +DESC t2; +DROP TABLE t1,t2; + +CREATE TABLE t1 (a DECIMAL(30,30)); +INSERT INTO t1 VALUES (0.1),(0.2),(0.3); +CREATE TABLE t2 SELECT IFNULL(a + 0.0000000000000000000000000000001, NULL) AS c1 FROM t1; +DESC t2; +DROP TABLE t1,t2; + +CREATE TABLE t1 (a DECIMAL(30,30)); +INSERT INTO t1 VALUES (0.1),(0.2),(0.3); +CREATE TABLE t2 SELECT CASE a WHEN 0.1 THEN 0.0000000000000000000000000000000000000000000000000000000000000000001 END AS c1 FROM t1; +DESC t2; +DROP TABLE t1,t2; + +--echo # +--echo # Test that variables get maximum precision. +--echo # + +SET @decimal= 1.1; +CREATE TABLE t1 SELECT @decimal AS c1; +DESC t1; +SELECT * FROM t1; +DROP TABLE t1; + +--echo # +--echo # Bug #45261 : Crash, stored procedure + decimal +--echo # Original test by the reporter. +--echo # + +--echo # should not crash +CREATE TABLE t1 +SELECT .123456789012345678901234567890123456789012345678901234567890123456 AS a; +DROP TABLE t1; + +delimiter |; +CREATE PROCEDURE test_proc() +BEGIN + # The las non critical CUSER definition is: + # DECLARE mycursor CURSOR FOR SELECT 1 % + # .12345678912345678912345678912345678912345678912345678912345678912 AS my_col; + DECLARE mycursor CURSOR FOR +SELECT 1 % +.123456789123456789123456789123456789123456789123456789123456789123456789123456789 + AS my_col; + + OPEN mycursor; + CLOSE mycursor; +END| +delimiter ;| +--echo # should not crash +CALL test_proc(); +DROP PROCEDURE test_proc; + +--echo # +--echo # Bug #48370 Absolutely wrong calculations with GROUP BY and +--echo # decimal fields when using IF +--echo # + +CREATE TABLE currencies (id int, rate decimal(16,4), + PRIMARY KEY (id), KEY (rate)); + +INSERT INTO currencies VALUES (11,0.7028); +INSERT INTO currencies VALUES (1,1); + +CREATE TABLE payments ( + id int, + supplier_id int, + status int, + currency_id int, + vat decimal(7,4), + PRIMARY KEY (id), + KEY currency_id (currency_id), + KEY supplier_id (supplier_id) +); + +INSERT INTO payments (id,status,vat,supplier_id,currency_id) VALUES +(3001,2,0.0000,344,11), (1,2,0.0000,1,1); + +CREATE TABLE sub_tasks ( + id int, + currency_id int, + price decimal(16,4), + discount decimal(10,4), + payment_id int, + PRIMARY KEY (id), + KEY currency_id (currency_id), + KEY payment_id (payment_id) +) ; + +INSERT INTO sub_tasks (id, price, discount, payment_id, currency_id) VALUES +(52, 12.60, 0, 3001, 11), (56, 14.58, 0, 3001, 11); + +--echo # should return 1 and the same values in col 2 and 3 +select STRAIGHT_JOIN + (1 + PAY.vat) AS mult, + SUM(ROUND((SUB.price - ROUND(ROUND(SUB.price, 2) * SUB.discount, 2)) * + CUR.rate / CUR.rate, 2) + ) v_net_with_discount, + + SUM(ROUND((SUB.price - ROUND(ROUND(SUB.price, 2) * SUB.discount, 1)) * + CUR.rate / CUR.rate , 2) + * (1 + PAY.vat) + ) v_total +from + currencies CUR, payments PAY, sub_tasks SUB +where + SUB.payment_id = PAY.id and + PAY.currency_id = CUR.id and + PAY.id > 2 +group by PAY.id + 1; + +DROP TABLE currencies, payments, sub_tasks; + + +--echo End of 5.1 tests === modified file 'mysql-test/t/type_year.test' --- a/mysql-test/t/type_year.test 2007-03-29 04:08:30 +0000 +++ b/mysql-test/t/type_year.test 2009-12-15 08:37:10 +0000 @@ -30,3 +30,109 @@ select * from t1; drop table t1; --echo End of 5.0 tests + +--echo # +--echo # Bug #49480: WHERE using YEAR columns returns unexpected results +--echo # + +CREATE TABLE t2(yy YEAR(2), c2 CHAR(4)); +CREATE TABLE t4(yyyy YEAR(4), c4 CHAR(4)); + +INSERT INTO t2 (c2) VALUES (NULL),(1970),(1999),(2000),(2001),(2069); +INSERT INTO t4 (c4) SELECT c2 FROM t2; +UPDATE t2 SET yy = c2; +UPDATE t4 SET yyyy = c4; + +SELECT * FROM t2; +SELECT * FROM t4; + +--echo # Comparison of YEAR(2) with YEAR(4) + +SELECT * FROM t2, t4 WHERE yy = yyyy; +SELECT * FROM t2, t4 WHERE yy <=> yyyy; +SELECT * FROM t2, t4 WHERE yy < yyyy; +SELECT * FROM t2, t4 WHERE yy > yyyy; + +--echo # Comparison of YEAR(2) with YEAR(2) + +SELECT * FROM t2 a, t2 b WHERE a.yy = b.yy; +SELECT * FROM t2 a, t2 b WHERE a.yy <=> b.yy; +SELECT * FROM t2 a, t2 b WHERE a.yy < b.yy; + +--echo # Comparison of YEAR(4) with YEAR(4) + +SELECT * FROM t4 a, t4 b WHERE a.yyyy = b.yyyy; +SELECT * FROM t4 a, t4 b WHERE a.yyyy <=> b.yyyy; +SELECT * FROM t4 a, t4 b WHERE a.yyyy < b.yyyy; + +--echo # Comparison with constants: + +SELECT * FROM t2 WHERE yy = NULL; +SELECT * FROM t4 WHERE yyyy = NULL; +SELECT * FROM t2 WHERE yy <=> NULL; +SELECT * FROM t4 WHERE yyyy <=> NULL; +SELECT * FROM t2 WHERE yy < NULL; +SELECT * FROM t2 WHERE yy > NULL; + +SELECT * FROM t2 WHERE yy = NOW(); +SELECT * FROM t4 WHERE yyyy = NOW(); + +SELECT * FROM t2 WHERE yy = 99; +SELECT * FROM t2 WHERE 99 = yy; +SELECT * FROM t4 WHERE yyyy = 99; + +SELECT * FROM t2 WHERE yy = 'test'; +SELECT * FROM t4 WHERE yyyy = 'test'; + +SELECT * FROM t2 WHERE yy = '1999'; +SELECT * FROM t4 WHERE yyyy = '1999'; + +SELECT * FROM t2 WHERE yy = 1999; +SELECT * FROM t4 WHERE yyyy = 1999; + +SELECT * FROM t2 WHERE yy = 1999.1; +SELECT * FROM t4 WHERE yyyy = 1999.1; + +SELECT * FROM t2 WHERE yy = 1998.9; +SELECT * FROM t4 WHERE yyyy = 1998.9; + +--echo # Coverage tests for YEAR with zero/2000 constants: + +SELECT * FROM t2 WHERE yy = 0; +SELECT * FROM t2 WHERE yy = '0'; +SELECT * FROM t2 WHERE yy = '0000'; +SELECT * FROM t2 WHERE yy = '2000'; +SELECT * FROM t2 WHERE yy = 2000; + +SELECT * FROM t4 WHERE yyyy = 0; +SELECT * FROM t4 WHERE yyyy = '0'; +SELECT * FROM t4 WHERE yyyy = '0000'; +SELECT * FROM t4 WHERE yyyy = '2000'; +SELECT * FROM t4 WHERE yyyy = 2000; + +--echo # Comparison with constants those are out of YEAR range +--echo # (coverage test for backward compatibility) + +SELECT COUNT(yy) FROM t2; +SELECT COUNT(yyyy) FROM t4; + +SELECT COUNT(*) FROM t2 WHERE yy = -1; +SELECT COUNT(*) FROM t4 WHERE yyyy > -1; +SELECT COUNT(*) FROM t2 WHERE yy > -1000000000000000000; +SELECT COUNT(*) FROM t4 WHERE yyyy > -1000000000000000000; + +SELECT COUNT(*) FROM t2 WHERE yy < 2156; +SELECT COUNT(*) FROM t4 WHERE yyyy < 2156; +SELECT COUNT(*) FROM t2 WHERE yy < 1000000000000000000; +SELECT COUNT(*) FROM t4 WHERE yyyy < 1000000000000000000; + +SELECT * FROM t2 WHERE yy < 123; +SELECT * FROM t2 WHERE yy > 123; +SELECT * FROM t4 WHERE yyyy < 123; +SELECT * FROM t4 WHERE yyyy > 123; + +DROP TABLE t2, t4; + +--echo # + +--echo End of 5.1 tests === modified file 'mysql-test/valgrind.supp' --- a/mysql-test/valgrind.supp 2009-12-03 11:19:05 +0000 +++ b/mysql-test/valgrind.supp 2010-01-18 12:56:10 +0000 @@ -991,3 +991,15 @@ fun:_dl_allocate_tls fun:pthread_create* } + +# +# Bug in Glibc 2.9: http://sourceware.org/bugzilla/show_bug.cgi?id=10391 +# Fixed in latest Glibc, but suppressed here for running tests on hosts +# with older Glibc version. +# +{ + Glibc bug in __libc_res_nsend + Memcheck:Cond + fun:__libc_res_nsend + fun:__libc_res_nquery +} === modified file 'mysys/my_getopt.c' --- a/mysys/my_getopt.c 2009-12-03 11:19:05 +0000 +++ b/mysys/my_getopt.c 2010-01-15 15:27:55 +0000 @@ -414,17 +414,11 @@ invalid value '%s'", (optp->var_type & GET_TYPE_MASK) == GET_ENUM)) { if (optend == disabled_my_option) - if ((optp->var_type & GET_TYPE_MASK) == GET_BOOL) - *((my_bool*) value)= (my_bool) 0; - else - *((ulong*) value)= (ulong) 0; + init_one_value(optp, value, 0); else { if (!optend) /* No argument -> enable option */ - if ((optp->var_type & GET_TYPE_MASK) == GET_BOOL) - *((my_bool*) value)= (my_bool) 1; - else - *((ulong*) value)= (ulong) 1; + init_one_value(optp, value, 1); else argument= optend; } === modified file 'mysys/my_sync.c' --- a/mysys/my_sync.c 2010-01-06 21:27:53 +0000 +++ b/mysys/my_sync.c 2010-01-15 15:27:55 +0000 @@ -104,11 +104,11 @@ int my_sync_dir(const char *dir_name __a myf my_flags __attribute__((unused))) { #ifdef NEED_EXPLICIT_SYNC_DIR - DBUG_ENTER("my_sync_dir"); - DBUG_PRINT("my",("Dir: '%s' my_flags: %d", dir_name, my_flags)); File dir_fd; int res= 0; const char *correct_dir_name; + DBUG_ENTER("my_sync_dir"); + DBUG_PRINT("my",("Dir: '%s' my_flags: %d", dir_name, my_flags)); /* Sometimes the path does not contain an explicit directory */ correct_dir_name= (dir_name[0] == 0) ? cur_dir_name : dir_name; /* === modified file 'scripts/make_win_bin_dist' --- a/scripts/make_win_bin_dist 2009-12-03 11:19:05 +0000 +++ b/scripts/make_win_bin_dist 2010-01-15 15:27:55 +0000 @@ -352,7 +352,7 @@ mkdir $DESTDIR/mysql-test cp mysql-test/mysql-test-run.pl $DESTDIR/mysql-test/ cp mysql-test/mysql-stress-test.pl $DESTDIR/mysql-test/ cp mysql-test/README $DESTDIR/mysql-test/ -cp -R mysql-test/{t,r,include,suite,std_data,lib} $DESTDIR/mysql-test/ +cp -R mysql-test/{t,r,include,suite,std_data,lib,collections} $DESTDIR/mysql-test/ rm -rf $DESTDIR/mysql-test/lib/My/SafeProcess/my_safe_kill.{dir,vcproj} rm -rf $DESTDIR/mysql-test/lib/My/SafeProcess/my_safe_process.{dir,vcproj} === modified file 'scripts/mysql_secure_installation.pl.in' --- a/scripts/mysql_secure_installation.pl.in 2007-12-28 21:58:54 +0000 +++ b/scripts/mysql_secure_installation.pl.in 2009-11-03 21:34:01 +0000 @@ -17,16 +17,41 @@ # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA use Fcntl; +use File::Spec; +use if $^O eq 'MSWin32', 'Term::ReadKey' => qw/ReadMode/; use strict; my $config = ".my.cnf.$$"; my $command = ".mysql.$$"; my $hadpass = 0; +my $mysql; # How to call the mysql client +my $rootpass = ""; -# FIXME -# trap "interrupt" 2 -my $rootpass = ""; +$SIG{QUIT} = $SIG{INT} = sub { + print "\nAborting!\n\n"; + echo_on(); + cleanup(); + exit 1; +}; + + +END { + # Remove temporary files, even if exiting via die(), etc. + cleanup(); +} + + +sub read_without_echo { + my ($prompt) = @_; + print $prompt; + echo_off(); + my $answer = <STDIN>; + echo_on(); + print "\n"; + chomp($answer); + return $answer; +} sub echo_on { if ($^O eq 'MSWin32') { @@ -55,6 +80,25 @@ sub write_file { } sub prepare { + # Locate the mysql client; look in current directory first, then + # in path + our $SAVEERR; # Suppress Perl warning message + open SAVEERR, ">& STDERR"; + close STDERR; + for my $m (File::Spec->catfile('bin', 'mysql'), 'mysql') { + # mysql --version should always work + qx($m --no-defaults --version); + next unless $? == 0; + + $mysql = $m; + last; + } + open STDERR, ">& SAVEERR"; + + die "Can't find a 'mysql' client in PATH or ./bin\n" + unless $mysql; + + # Create safe files to avoid leaking info to other users foreach my $file ( $config, $command ) { next if -f $file; # Already exists local *FILE; @@ -64,30 +108,50 @@ sub prepare { } } +# Simple escape mechanism (\-escape any ' and \), suitable for two contexts: +# - single-quoted SQL strings +# - single-quoted option values on the right hand side of = in my.cnf +# +# These two contexts don't handle escapes identically. SQL strings allow +# quoting any character (\C => C, for any C), but my.cnf parsing allows +# quoting only \, ' or ". For example, password='a\b' quotes a 3-character +# string in my.cnf, but a 2-character string in SQL. +# +# This simple escape works correctly in both places. +sub basic_single_escape { + my ($str) = @_; + # Inside a character class, \ is not special; this escapes both \ and ' + $str =~ s/([\'])/\\$1/g; + return $str; +} + sub do_query { my $query = shift; write_file($command, $query); - system("mysql --defaults-file=$config < $command"); - return $?; + my $rv = system("$mysql --defaults-file=$config < $command"); + # system() returns -1 if exec fails (e.g., command not found, etc.); die + # in this case because nothing is going to work + die "Failed to execute mysql client '$mysql'\n" if $rv == -1; + # Return true if query executed OK, or false if there was some problem + # (for example, SQL error or wrong password) + return ($rv == 0 ? 1 : undef); } sub make_config { my $password = shift; + my $esc_pass = basic_single_escape($rootpass); write_file($config, "# mysql_secure_installation config file", "[mysql]", "user=root", - "password=$rootpass"); + "password='$esc_pass'"); } sub get_root_password { - my $status = 1; - while ( $status == 1 ) { - echo_off(); - print "Enter current password for root (enter for none): "; - my $password = <STDIN>; - echo_on(); + my $attempts = 3; + for (;;) { + my $password = read_without_echo("Enter current password for root (enter for none): "); if ( $password ) { $hadpass = 1; } else { @@ -95,64 +159,56 @@ sub get_root_password { } $rootpass = $password; make_config($rootpass); - do_query(""); - $status = $?; + last if do_query(""); + + die "Unable to connect to the server as root user, giving up.\n" + if --$attempts == 0; } print "OK, successfully used password, moving on...\n\n"; } sub set_root_password { - echo_off(); - print "New password: "; - my $password1 = <STDIN>; - print "\nRe-enter new password: "; - my $password2 = <STDIN>; - print "\n"; - echo_on(); - - if ( $password1 eq $password2 ) { - print "Sorry, passwords do not match.\n\n"; - return 1; - } + my $password1; + for (;;) { + $password1 = read_without_echo("New password: "); + + if ( !$password1 ) { + print "Sorry, you can't use an empty password here.\n\n"; + next; + } - if ( !$password1 ) { - print "Sorry, you can't use an empty password here.\n\n"; - return 1; - } + my $password2 = read_without_echo("Re-enter new password: "); - do_query("UPDATE mysql.user SET Password=PASSWORD('$password1') WHERE User='root';"); - if ( $? == 0 ) { - print "Password updated successfully!\n"; - print "Reloading privilege tables..\n"; - if ( !reload_privilege_tables() ) { - exit 1; + if ( $password1 ne $password2 ) { + print "Sorry, passwords do not match.\n\n"; + next; } - print "\n"; - $rootpass = $password1; - make_config($rootpass); - } else { - print "Password update failed!\n"; - exit 1; + + last; } - return 0; + my $esc_pass = basic_single_escape($password1); + do_query("UPDATE mysql.user SET Password=PASSWORD('$esc_pass') WHERE User='root';") + or die "Password update failed!\n"; + + print "Password updated successfully!\n"; + print "Reloading privilege tables..\n"; + reload_privilege_tables() + or die "Can not continue.\n"; + + print "\n"; + $rootpass = $password1; + make_config($rootpass); } sub remove_anonymous_users { - do_query("DELETE FROM mysql.user WHERE User='';"); - if ( $? == 0 ) { - print " ... Success!\n"; - } else { - print " ... Failed!\n"; - exit 1; - } - - return 0; + do_query("DELETE FROM mysql.user WHERE User='';") + or die print " ... Failed!\n"; + print " ... Success!\n"; } sub remove_remote_root { - do_query("DELETE FROM mysql.user WHERE User='root' AND Host!='localhost';"); - if ( $? == 0 ) { + if (do_query("DELETE FROM mysql.user WHERE User='root' AND Host!='localhost';")) { print " ... Success!\n"; } else { print " ... Failed!\n"; @@ -161,44 +217,31 @@ sub remove_remote_root { sub remove_test_database { print " - Dropping test database...\n"; - do_query("DROP DATABASE test;"); - if ( $? == 0 ) { + if (do_query("DROP DATABASE test;")) { print " ... Success!\n"; } else { print " ... Failed! Not critical, keep moving...\n"; } print " - Removing privileges on test database...\n"; - do_query("DELETE FROM mysql.db WHERE Db='test' OR Db='test\\_%'"); - if ( $? == 0 ) { + if (do_query("DELETE FROM mysql.db WHERE Db='test' OR Db='test\\_%'")) { print " ... Success!\n"; } else { print " ... Failed! Not critical, keep moving...\n"; } - - return 0; } sub reload_privilege_tables { - do_query("FLUSH PRIVILEGES;"); - if ( $? == 0 ) { + if (do_query("FLUSH PRIVILEGES;")) { print " ... Success!\n"; - return 0; + return 1; } else { print " ... Failed!\n"; - return 1; + return undef; } } -sub interrupt { - print "\nAborting!\n\n"; - cleanup(); - echo_on(); - exit 1; -} - sub cleanup { - print "Cleaning up...\n"; unlink($config,$command); } @@ -242,11 +285,7 @@ my $reply = <STDIN>; if ( $reply =~ /n/i ) { print " ... skipping.\n"; } else { - my $status = 1; - while ( $status == 1 ) { - set_root_password(); - $status = $?; - } + set_root_password(); } print "\n"; @@ -334,8 +373,6 @@ if ( $reply =~ /n/i ) { } print "\n"; -cleanup(); - print <<HERE; === modified file 'scripts/mysql_secure_installation.sh' --- a/scripts/mysql_secure_installation.sh 2009-10-23 16:48:54 +0000 +++ b/scripts/mysql_secure_installation.sh 2010-01-15 15:27:55 +0000 @@ -189,16 +189,39 @@ prepare() { } do_query() { - echo $1 >$command + echo "$1" >$command + #sed 's,^,> ,' < $command # Debugging $bindir/mysql --defaults-file=$config <$command return $? } +# Simple escape mechanism (\-escape any ' and \), suitable for two contexts: +# - single-quoted SQL strings +# - single-quoted option values on the right hand side of = in my.cnf +# +# These two contexts don't handle escapes identically. SQL strings allow +# quoting any character (\C => C, for any C), but my.cnf parsing allows +# quoting only \, ' or ". For example, password='a\b' quotes a 3-character +# string in my.cnf, but a 2-character string in SQL. +# +# This simple escape works correctly in both places. +basic_single_escape () { + # The quoting on this sed command is a bit complex. Single-quoted strings + # don't allow *any* escape mechanism, so they cannot contain a single + # quote. The string sed gets (as argv[1]) is: s/$['\]$/\\\1/g + # + # Inside a character class, \ and ' are not special, so the ['\] character + # class is balanced and contains two characters. + echo "$1" | sed 's/$['"'"'\]$/\\\1/g' +} + make_config() { echo "# mysql_secure_installation config file" >$config echo "[mysql]" >>$config echo "user=root" >>$config - echo "password=$rootpass" >>$config + esc_pass=`basic_single_escape "$rootpass"` + echo "password='$esc_pass'" >>$config + #sed 's,^,> ,' < $config # Debugging } get_root_password() { @@ -245,13 +268,12 @@ set_root_password() { return 1 fi - do_query "UPDATE mysql.user SET Password=PASSWORD('$password1') WHERE User='root';" + esc_pass=`basic_single_escape "$password1"` + do_query "UPDATE mysql.user SET Password=PASSWORD('$esc_pass') WHERE User='root';" if [ $? -eq 0 ]; then echo "Password updated successfully!" echo "Reloading privilege tables.." - if ! reload_privilege_tables; then - exit 1 - fi + reload_privilege_tables || exit 1 echo rootpass=$password1 make_config === modified file 'sql/event_db_repository.cc' --- a/sql/event_db_repository.cc 2009-12-03 11:19:05 +0000 +++ b/sql/event_db_repository.cc 2010-01-15 15:27:55 +0000 @@ -26,7 +26,7 @@ */ static -const TABLE_FIELD_W_TYPE event_table_fields[ET_FIELD_COUNT] = +const TABLE_FIELD_TYPE event_table_fields[ET_FIELD_COUNT] = { { { C_STRING_WITH_LEN("db") }, @@ -151,6 +151,24 @@ const TABLE_FIELD_W_TYPE event_table_fie } }; +static const TABLE_FIELD_DEF + event_table_def= {ET_FIELD_COUNT, event_table_fields}; + +class Event_db_intact : public Table_check_intact +{ +protected: + void report_error(uint, const char *fmt, ...) + { + va_list args; + va_start(args, fmt); + error_log_print(ERROR_LEVEL, fmt, args); + va_end(args); + } +}; + +/** In case of an error, a message is printed to the error log. */ +static Event_db_intact table_intact; + /** Puts some data common to CREATE and ALTER EVENT into a row. @@ -1117,10 +1135,8 @@ Event_db_repository::check_system_tables } else { - if (table_check_intact(tables.table, MYSQL_DB_FIELD_COUNT, - mysql_db_table_fields)) + if (table_intact.check(tables.table, &mysql_db_table_def)) ret= 1; - /* in case of an error, the message is printed inside table_check_intact */ close_thread_tables(thd); } @@ -1154,9 +1170,8 @@ Event_db_repository::check_system_tables } else { - if (table_check_intact(tables.table, ET_FIELD_COUNT, event_table_fields)) + if (table_intact.check(tables.table, &event_table_def)) ret= 1; - /* in case of an error, the message is printed inside table_check_intact */ close_thread_tables(thd); } === modified file 'sql/field.cc' --- a/sql/field.cc 2009-12-03 11:19:05 +0000 +++ b/sql/field.cc 2010-01-15 15:27:55 +0000 @@ -2487,6 +2487,50 @@ Field_new_decimal::Field_new_decimal(uin } +Field *Field_new_decimal::create_from_item (Item *item) +{ + uint8 dec= item->decimals; + uint8 intg= item->decimal_precision() - dec; + uint32 len= item->max_length; + + DBUG_ASSERT (item->result_type() == DECIMAL_RESULT); + + /* + Trying to put too many digits overall in a DECIMAL(prec,dec) + will always throw a warning. We must limit dec to + DECIMAL_MAX_SCALE however to prevent an assert() later. + */ + + if (dec > 0) + { + signed int overflow; + + dec= min(dec, DECIMAL_MAX_SCALE); + + /* + If the value still overflows the field with the corrected dec, + we'll throw out decimals rather than integers. This is still + bad and of course throws a truncation warning. + +1: for decimal point + */ + + const int required_length= + my_decimal_precision_to_length(intg + dec, dec, + item->unsigned_flag); + + overflow= required_length - len; + + if (overflow > 0) + dec= max(0, dec - overflow); // too long, discard fract + else + /* Corrected value fits. */ + len= required_length; + } + return new Field_new_decimal(len, item->maybe_null, item->name, + dec, item->unsigned_flag); +} + + int Field_new_decimal::reset(void) { store_value(&decimal_zero); === modified file 'sql/field.h' --- a/sql/field.h 2009-12-03 11:19:05 +0000 +++ b/sql/field.h 2010-01-15 15:27:55 +0000 @@ -807,6 +807,7 @@ public: uint is_equal(Create_field *new_field); virtual const uchar *unpack(uchar* to, const uchar *from, uint param_data, bool low_byte_first); + static Field *create_from_item (Item *); }; === modified file 'sql/handler.h' --- a/sql/handler.h 2010-01-04 13:12:53 +0000 +++ b/sql/handler.h 2010-01-14 16:51:00 +0000 @@ -891,9 +891,9 @@ typedef struct { ulonglong delete_length; ha_rows records; ulong mean_rec_length; - ulong create_time; - ulong check_time; - ulong update_time; + time_t create_time; + time_t check_time; + time_t update_time; ulonglong check_sum; } PARTITION_INFO; @@ -1060,9 +1060,9 @@ public: ha_rows records; ha_rows deleted; /* Deleted records */ ulong mean_rec_length; /* physical reclength */ - ulong create_time; /* When table was created */ - ulong check_time; - ulong update_time; + time_t create_time; /* When table was created */ + time_t check_time; + time_t update_time; uint block_size; /* index block size */ ha_statistics(): === modified file 'sql/item.cc' --- a/sql/item.cc 2009-12-03 11:19:05 +0000 +++ b/sql/item.cc 2010-01-15 15:27:55 +0000 @@ -4908,9 +4908,7 @@ Field *Item::tmp_table_field_from_field_ switch (field_type()) { case MYSQL_TYPE_DECIMAL: case MYSQL_TYPE_NEWDECIMAL: - field= new Field_new_decimal((uchar*) 0, max_length, null_ptr, 0, - Field::NONE, name, decimals, 0, - unsigned_flag); + field= Field_new_decimal::create_from_item(this); break; case MYSQL_TYPE_TINY: field= new Field_tiny((uchar*) 0, max_length, null_ptr, 0, Field::NONE, @@ -6949,9 +6947,24 @@ int stored_field_cmp_to_item(THD *thd, F Item_cache* Item_cache::get_cache(const Item *item) { - switch (item->result_type()) { + return get_cache(item, item->result_type()); +} + + +/** + Get a cache item of given type. + + @param item value to be cached + @param type required type of cache + + @return cache item +*/ + +Item_cache* Item_cache::get_cache(const Item *item, const Item_result type) +{ + switch (type) { case INT_RESULT: - return new Item_cache_int(); + return new Item_cache_int(item->field_type()); case REAL_RESULT: return new Item_cache_real(); case DECIMAL_RESULT: @@ -6967,6 +6980,13 @@ Item_cache* Item_cache::get_cache(const } } +void Item_cache::store(Item *item) +{ + example= item; + if (!item) + null_value= TRUE; + value_cached= FALSE; +} void Item_cache::print(String *str, enum_query_type query_type) { @@ -6978,17 +6998,22 @@ void Item_cache::print(String *str, enum str->append(')'); } - -void Item_cache_int::store(Item *item) +bool Item_cache_int::cache_value() { - value= item->val_int_result(); - null_value= item->null_value; - unsigned_flag= item->unsigned_flag; + if (!example) + return FALSE; + value_cached= TRUE; + value= example->val_int_result(); + null_value= example->null_value; + unsigned_flag= example->unsigned_flag; + return TRUE; } void Item_cache_int::store(Item *item, longlong val_arg) { + /* An explicit values is given, save it. */ + value_cached= TRUE; value= val_arg; null_value= item->null_value; unsigned_flag= item->unsigned_flag; @@ -6998,6 +7023,8 @@ void Item_cache_int::store(Item *item, l String *Item_cache_int::val_str(String *str) { DBUG_ASSERT(fixed == 1); + if (!value_cached && !cache_value()) + return NULL; str->set(value, default_charset()); return str; } @@ -7006,21 +7033,52 @@ String *Item_cache_int::val_str(String * my_decimal *Item_cache_int::val_decimal(my_decimal *decimal_val) { DBUG_ASSERT(fixed == 1); + if (!value_cached && !cache_value()) + return NULL; int2my_decimal(E_DEC_FATAL_ERROR, value, unsigned_flag, decimal_val); return decimal_val; } +double Item_cache_int::val_real() +{ + DBUG_ASSERT(fixed == 1); + if (!value_cached && !cache_value()) + return 0.0; + return (double) value; +} -void Item_cache_real::store(Item *item) +longlong Item_cache_int::val_int() { - value= item->val_result(); - null_value= item->null_value; + DBUG_ASSERT(fixed == 1); + if (!value_cached && !cache_value()) + return 0; + return value; } +bool Item_cache_real::cache_value() +{ + if (!example) + return FALSE; + value_cached= TRUE; + value= example->val_result(); + null_value= example->null_value; + return TRUE; +} + + +double Item_cache_real::val_real() +{ + DBUG_ASSERT(fixed == 1); + if (!value_cached && !cache_value()) + return 0.0; + return value; +} longlong Item_cache_real::val_int() { DBUG_ASSERT(fixed == 1); + if (!value_cached && !cache_value()) + return 0; return (longlong) rint(value); } @@ -7028,6 +7086,8 @@ longlong Item_cache_real::val_int() String* Item_cache_real::val_str(String *str) { DBUG_ASSERT(fixed == 1); + if (!value_cached && !cache_value()) + return NULL; str->set_real(value, decimals, default_charset()); return str; } @@ -7036,22 +7096,30 @@ String* Item_cache_real::val_str(String my_decimal *Item_cache_real::val_decimal(my_decimal *decimal_val) { DBUG_ASSERT(fixed == 1); + if (!value_cached && !cache_value()) + return NULL; double2my_decimal(E_DEC_FATAL_ERROR, value, decimal_val); return decimal_val; } -void Item_cache_decimal::store(Item *item) +bool Item_cache_decimal::cache_value() { - my_decimal *val= item->val_decimal_result(&decimal_value); - if (!(null_value= item->null_value) && val != &decimal_value) + if (!example) + return FALSE; + value_cached= TRUE; + my_decimal *val= example->val_decimal_result(&decimal_value); + if (!(null_value= example->null_value) && val != &decimal_value) my_decimal2decimal(val, &decimal_value); + return TRUE; } double Item_cache_decimal::val_real() { DBUG_ASSERT(fixed); double res; + if (!value_cached && !cache_value()) + return NULL; my_decimal2double(E_DEC_FATAL_ERROR, &decimal_value, &res); return res; } @@ -7060,6 +7128,8 @@ longlong Item_cache_decimal::val_int() { DBUG_ASSERT(fixed); longlong res; + if (!value_cached && !cache_value()) + return 0; my_decimal2int(E_DEC_FATAL_ERROR, &decimal_value, unsigned_flag, &res); return res; } @@ -7067,6 +7137,8 @@ longlong Item_cache_decimal::val_int() String* Item_cache_decimal::val_str(String *str) { DBUG_ASSERT(fixed); + if (!value_cached && !cache_value()) + return NULL; my_decimal_round(E_DEC_FATAL_ERROR, &decimal_value, decimals, FALSE, &decimal_value); my_decimal2string(E_DEC_FATAL_ERROR, &decimal_value, 0, 0, 0, str); @@ -7076,15 +7148,20 @@ String* Item_cache_decimal::val_str(Stri my_decimal *Item_cache_decimal::val_decimal(my_decimal *val) { DBUG_ASSERT(fixed); + if (!value_cached && !cache_value()) + return NULL; return &decimal_value; } -void Item_cache_str::store(Item *item) +bool Item_cache_str::cache_value() { - value_buff.set(buffer, sizeof(buffer), item->collation.collation); - value= item->str_result(&value_buff); - if ((null_value= item->null_value)) + if (!example) + return FALSE; + value_cached= TRUE; + value_buff.set(buffer, sizeof(buffer), example->collation.collation); + value= example->str_result(&value_buff); + if ((null_value= example->null_value)) value= 0; else if (value != &value_buff) { @@ -7099,6 +7176,7 @@ void Item_cache_str::store(Item *item) value_buff.copy(*value); value= &value_buff; } + return TRUE; } double Item_cache_str::val_real() @@ -7106,6 +7184,8 @@ double Item_cache_str::val_real() DBUG_ASSERT(fixed == 1); int err_not_used; char *end_not_used; + if (!value_cached && !cache_value()) + return 0.0; if (value) return my_strntod(value->charset(), (char*) value->ptr(), value->length(), &end_not_used, &err_not_used); @@ -7117,6 +7197,8 @@ longlong Item_cache_str::val_int() { DBUG_ASSERT(fixed == 1); int err; + if (!value_cached && !cache_value()) + return 0; if (value) return my_strntoll(value->charset(), value->ptr(), value->length(), 10, (char**) 0, &err); @@ -7124,9 +7206,21 @@ longlong Item_cache_str::val_int() return (longlong)0; } + +String* Item_cache_str::val_str(String *str) +{ + DBUG_ASSERT(fixed == 1); + if (!value_cached && !cache_value()) + return 0; + return value; +} + + my_decimal *Item_cache_str::val_decimal(my_decimal *decimal_val) { DBUG_ASSERT(fixed == 1); + if (!value_cached && !cache_value()) + return NULL; if (value) string2my_decimal(E_DEC_FATAL_ERROR, value, decimal_val); else @@ -7137,6 +7231,8 @@ my_decimal *Item_cache_str::val_decimal( int Item_cache_str::save_in_field(Field *field, bool no_conversions) { + if (!value_cached && !cache_value()) + return 0; int res= Item_cache::save_in_field(field, no_conversions); return (is_varbinary && field->type() == MYSQL_TYPE_STRING && value->length() < field->field_length) ? 1 : res; @@ -7171,13 +7267,30 @@ bool Item_cache_row::setup(Item * item) void Item_cache_row::store(Item * item) { + example= item; + if (!item) + { + null_value= TRUE; + return; + } + for (uint i= 0; i < item_count; i++) + values[i]->store(item->element_index(i)); +} + + +bool Item_cache_row::cache_value() +{ + if (!example) + return FALSE; + value_cached= TRUE; null_value= 0; - item->bring_value(); + example->bring_value(); for (uint i= 0; i < item_count; i++) { - values[i]->store(item->element_index(i)); + values[i]->cache_value(); null_value|= values[i]->null_value; } + return TRUE; } === modified file 'sql/item.h' --- a/sql/item.h 2009-12-03 11:19:05 +0000 +++ b/sql/item.h 2010-01-15 15:27:55 +0000 @@ -1053,7 +1053,11 @@ class sp_head; class Item_basic_constant :public Item { + table_map used_table_map; public: + Item_basic_constant(): Item(), used_table_map(0) {}; + void set_used_tables(table_map map) { used_table_map= map; } + table_map used_tables() const { return used_table_map; } /* to prevent drop fixed flag (no need parent cleanup call) */ void cleanup() { @@ -1065,7 +1069,6 @@ public: if (orig_name) name= orig_name; } - Item_basic_constant() {} /* Remove gcc warning */ }; @@ -2165,6 +2168,23 @@ public: save_in_field(result_field, no_conversions); } void cleanup(); + /* + This method is used for debug purposes to print the name of an + item to the debug log. The second use of this method is as + a helper function of print() and error messages, where it is + applicable. To suit both goals it should return a meaningful, + distinguishable and sintactically correct string. This method + should not be used for runtime type identification, use enum + {Sum}Functype and Item_func::functype()/Item_sum::sum_func() + instead. + Added here, to the parent class of both Item_func and Item_sum_func. + + NOTE: for Items inherited from Item_sum, func_name() return part of + function name till first argument (including '(') to make difference in + names for functions with 'distinct' clause and without 'distinct' and + also to make printing of items inherited from Item_sum uniform. + */ + virtual const char *func_name() const= 0; }; @@ -2924,15 +2944,25 @@ protected: */ Field *cached_field; enum enum_field_types cached_field_type; -public: - Item_cache(): - example(0), used_table_map(0), cached_field(0), cached_field_type(MYSQL_TYPE_STRING) + /* + TRUE <=> cache holds value of the last stored item (i.e actual value). + store() stores item to be cached and sets this flag to FALSE. + On the first call of val_xxx function if this flag is set to FALSE the + cache_value() will be called to actually cache value of saved item. + cache_value() will set this flag to TRUE. + */ + bool value_cached; +public: + Item_cache(): + example(0), used_table_map(0), cached_field(0), cached_field_type(MYSQL_TYPE_STRING), + value_cached(0) { fixed= 1; null_value= 1; } Item_cache(enum_field_types field_type_arg): - example(0), used_table_map(0), cached_field(0), cached_field_type(field_type_arg) + example(0), used_table_map(0), cached_field(0), cached_field_type(field_type_arg), + value_cached(0) { fixed= 1; null_value= 1; @@ -2952,10 +2982,10 @@ public: cached_field= ((Item_field *)item)->field; return 0; }; - virtual void store(Item *)= 0; enum Type type() const { return CACHE_ITEM; } enum_field_types field_type() const { return cached_field_type; } static Item_cache* get_cache(const Item *item); + static Item_cache* get_cache(const Item* item, const Item_result type); table_map used_tables() const { return used_table_map; } virtual void keep_array() {} virtual void print(String *str, enum_query_type query_type); @@ -2967,6 +2997,8 @@ public: { return this == item; } + virtual void store(Item *item); + virtual bool cache_value()= 0; }; @@ -2975,18 +3007,19 @@ class Item_cache_int: public Item_cache protected: longlong value; public: - Item_cache_int(): Item_cache(), value(0) {} + Item_cache_int(): Item_cache(), + value(0) {} Item_cache_int(enum_field_types field_type_arg): Item_cache(field_type_arg), value(0) {} - void store(Item *item); void store(Item *item, longlong val_arg); - double val_real() { DBUG_ASSERT(fixed == 1); return (double) value; } - longlong val_int() { DBUG_ASSERT(fixed == 1); return value; } + double val_real(); + longlong val_int(); String* val_str(String *str); my_decimal *val_decimal(my_decimal *); enum Item_result result_type() const { return INT_RESULT; } bool result_as_longlong() { return TRUE; } + bool cache_value(); }; @@ -2994,14 +3027,15 @@ class Item_cache_real: public Item_cache { double value; public: - Item_cache_real(): Item_cache(), value(0) {} + Item_cache_real(): Item_cache(), + value(0) {} - void store(Item *item); - double val_real() { DBUG_ASSERT(fixed == 1); return value; } + double val_real(); longlong val_int(); String* val_str(String *str); my_decimal *val_decimal(my_decimal *); enum Item_result result_type() const { return REAL_RESULT; } + bool cache_value(); }; @@ -3012,12 +3046,12 @@ protected: public: Item_cache_decimal(): Item_cache() {} - void store(Item *item); double val_real(); longlong val_int(); String* val_str(String *str); my_decimal *val_decimal(my_decimal *); enum Item_result result_type() const { return DECIMAL_RESULT; } + bool cache_value(); }; @@ -3035,14 +3069,14 @@ public: MYSQL_TYPE_VARCHAR && !((const Item_field *) item)->field->has_charset()) {} - void store(Item *item); double val_real(); longlong val_int(); - String* val_str(String *) { DBUG_ASSERT(fixed == 1); return value; } + String* val_str(String *); my_decimal *val_decimal(my_decimal *); enum Item_result result_type() const { return STRING_RESULT; } CHARSET_INFO *charset() const { return value->charset(); }; int save_in_field(Field *field, bool no_conversions); + bool cache_value(); }; class Item_cache_row: public Item_cache @@ -3052,7 +3086,8 @@ class Item_cache_row: public Item_cache bool save_array; public: Item_cache_row() - :Item_cache(), values(0), item_count(2), save_array(0) {} + :Item_cache(), values(0), item_count(2), + save_array(0) {} /* 'allocate' used only in row transformer, to preallocate space for row @@ -3110,6 +3145,7 @@ public: values= 0; DBUG_VOID_RETURN; } + bool cache_value(); }; === modified file 'sql/item_cmpfunc.cc' --- a/sql/item_cmpfunc.cc 2009-12-03 11:19:05 +0000 +++ b/sql/item_cmpfunc.cc 2010-01-15 15:27:55 +0000 @@ -30,6 +30,9 @@ #include "sql_select.h" static bool convert_constant_item(THD *, Item_field *, Item **); +static longlong +get_year_value(THD *thd, Item ***item_arg, Item **cache_arg, + Item *warn_item, bool *is_null); static Item_result item_store_type(Item_result a, Item *item, my_bool unsigned_flag) @@ -533,11 +536,12 @@ void Item_bool_func2::fix_length_and_dec } -int Arg_comparator::set_compare_func(Item_bool_func2 *item, Item_result type) +int Arg_comparator::set_compare_func(Item_result_field *item, Item_result type) { owner= item; func= comparator_matrix[type] - [test(owner->functype() == Item_func::EQUAL_FUNC)]; + [is_owner_equal_func()]; + switch (type) { case ROW_RESULT: { @@ -557,7 +561,8 @@ int Arg_comparator::set_compare_func(Ite my_error(ER_OPERAND_COLUMNS, MYF(0), (*a)->element_index(i)->cols()); return 1; } - if (comparators[i].set_cmp_func(owner, (*a)->addr(i), (*b)->addr(i))) + if (comparators[i].set_cmp_func(owner, (*a)->addr(i), (*b)->addr(i), + set_null)) return 1; } break; @@ -571,7 +576,8 @@ int Arg_comparator::set_compare_func(Ite if (cmp_collation.set((*a)->collation, (*b)->collation) || cmp_collation.derivation == DERIVATION_NONE) { - my_coll_agg_error((*a)->collation, (*b)->collation, owner->func_name()); + my_coll_agg_error((*a)->collation, (*b)->collation, + owner->func_name()); return 1; } if (cmp_collation.collation == &my_charset_bin) @@ -785,15 +791,21 @@ Arg_comparator::can_compare_as_dates(Ite if (cmp_type != CMP_DATE_DFLT) { + THD *thd= current_thd; /* Do not cache GET_USER_VAR() function as its const_item() may return TRUE for the current thread but it still may change during the execution. + Don't use cache while in the context analysis mode only (i.e. for + EXPLAIN/CREATE VIEW and similar queries). Cache is useless in such + cases and can cause problems. For example evaluating subqueries can + confuse storage engines since in context analysis mode tables + aren't locked. */ - if (cmp_type != CMP_DATE_WITH_DATE && str_arg->const_item() && + if (!thd->is_context_analysis_only() && + cmp_type != CMP_DATE_WITH_DATE && str_arg->const_item() && (str_arg->type() != Item::FUNC_ITEM || ((Item_func*)str_arg)->functype() != Item_func::GUSERVAR_FUNC)) { - THD *thd= current_thd; ulonglong value; bool error; String tmp, *str_val= 0; @@ -875,18 +887,20 @@ get_time_value(THD *thd, Item ***item_ar } -int Arg_comparator::set_cmp_func(Item_bool_func2 *owner_arg, +int Arg_comparator::set_cmp_func(Item_result_field *owner_arg, Item **a1, Item **a2, Item_result type) { ulonglong const_value= (ulonglong)-1; + thd= current_thd; + owner= owner_arg; + set_null= set_null && owner_arg; a= a1; b= a2; + thd= current_thd; if (can_compare_as_dates(*a, *b, &const_value)) { - thd= current_thd; - owner= owner_arg; a_type= (*a)->field_type(); b_type= (*b)->field_type(); a_cache= 0; @@ -894,6 +908,10 @@ int Arg_comparator::set_cmp_func(Item_bo if (const_value != (ulonglong)-1) { + /* + cache_converted_constant can't be used here because it can't + correctly convert a DATETIME value from string to int representation. + */ Item_cache_int *cache= new Item_cache_int(); /* Mark the cache as non-const to prevent re-caching. */ cache->set_used_tables(1); @@ -910,22 +928,22 @@ int Arg_comparator::set_cmp_func(Item_bo b= (Item **)&b_cache; } } - is_nulls_eq= test(owner && owner->functype() == Item_func::EQUAL_FUNC); + is_nulls_eq= is_owner_equal_func(); func= &Arg_comparator::compare_datetime; - get_value_func= &get_datetime_value; + get_value_a_func= &get_datetime_value; + get_value_b_func= &get_datetime_value; return 0; } else if (type == STRING_RESULT && (*a)->field_type() == MYSQL_TYPE_TIME && (*b)->field_type() == MYSQL_TYPE_TIME) { /* Compare TIME values as integers. */ - thd= current_thd; - owner= owner_arg; a_cache= 0; b_cache= 0; - is_nulls_eq= test(owner && owner->functype() == Item_func::EQUAL_FUNC); + is_nulls_eq= is_owner_equal_func(); func= &Arg_comparator::compare_datetime; - get_value_func= &get_time_value; + get_value_a_func= &get_time_value; + get_value_b_func= &get_time_value; return 0; } else if (type == STRING_RESULT && @@ -934,20 +952,97 @@ int Arg_comparator::set_cmp_func(Item_bo { DTCollation coll; coll.set((*a)->collation.collation); - if (agg_item_set_converter(coll, owner_arg->func_name(), + if (agg_item_set_converter(coll, owner->func_name(), b, 1, MY_COLL_CMP_CONV, 1)) return 1; } + else if (try_year_cmp_func(type)) + return 0; + a= cache_converted_constant(thd, a, &a_cache, type); + b= cache_converted_constant(thd, b, &b_cache, type); return set_compare_func(owner_arg, type); } -void Arg_comparator::set_datetime_cmp_func(Item **a1, Item **b1) +/* + Helper function to call from Arg_comparator::set_cmp_func() +*/ + +bool Arg_comparator::try_year_cmp_func(Item_result type) +{ + if (type == ROW_RESULT) + return FALSE; + + bool a_is_year= (*a)->field_type() == MYSQL_TYPE_YEAR; + bool b_is_year= (*b)->field_type() == MYSQL_TYPE_YEAR; + + if (!a_is_year && !b_is_year) + return FALSE; + + if (a_is_year && b_is_year) + { + get_value_a_func= &get_year_value; + get_value_b_func= &get_year_value; + } + else if (a_is_year && (*b)->is_datetime()) + { + get_value_a_func= &get_year_value; + get_value_b_func= &get_datetime_value; + } + else if (b_is_year && (*a)->is_datetime()) + { + get_value_b_func= &get_year_value; + get_value_a_func= &get_datetime_value; + } + else + return FALSE; + + is_nulls_eq= is_owner_equal_func(); + func= &Arg_comparator::compare_datetime; + + return TRUE; +} + +/** + Convert and cache a constant. + + @param value [in] An item to cache + @param cache_item [out] Placeholder for the cache item + @param type [in] Comparison type + + @details + When given item is a constant and its type differs from comparison type + then cache its value to avoid type conversion of this constant on each + evaluation. In this case the value is cached and the reference to the cache + is returned. + Original value is returned otherwise. + + @return cache item or original value. +*/ + +Item** Arg_comparator::cache_converted_constant(THD *thd, Item **value, + Item **cache_item, + Item_result type) +{ + /* Don't need cache if doing context analysis only. */ + if (!thd->is_context_analysis_only() && + (*value)->const_item() && type != (*value)->result_type()) + { + Item_cache *cache= Item_cache::get_cache(*value, type); + cache->setup(*value); + *cache_item= cache; + return cache_item; + } + return value; +} + + +void Arg_comparator::set_datetime_cmp_func(Item_result_field *owner_arg, + Item **a1, Item **b1) { thd= current_thd; - /* A caller will handle null values by itself. */ - owner= NULL; + owner= owner_arg; a= a1; b= b1; a_type= (*a)->field_type(); @@ -956,7 +1051,8 @@ void Arg_comparator::set_datetime_cmp_fu b_cache= 0; is_nulls_eq= FALSE; func= &Arg_comparator::compare_datetime; - get_value_func= &get_datetime_value; + get_value_a_func= &get_datetime_value; + get_value_b_func= &get_datetime_value; } @@ -1056,6 +1152,56 @@ get_datetime_value(THD *thd, Item ***ite return value; } + +/* + Retrieves YEAR value of 19XX-00-00 00:00:00 form from given item. + + SYNOPSIS + get_year_value() + thd thread handle + item_arg [in/out] item to retrieve YEAR value from + cache_arg [in/out] pointer to place to store the caching item to + warn_item [in] item for issuing the conversion warning + is_null [out] TRUE <=> the item_arg is null + + DESCRIPTION + Retrieves the YEAR value of 19XX form from given item for comparison by the + compare_datetime() function. + Converts year to DATETIME of form YYYY-00-00 00:00:00 for the compatibility + with the get_datetime_value function result. + + RETURN + obtained value +*/ + +static longlong +get_year_value(THD *thd, Item ***item_arg, Item **cache_arg, + Item *warn_item, bool *is_null) +{ + longlong value= 0; + Item *item= **item_arg; + + value= item->val_int(); + *is_null= item->null_value; + if (*is_null) + return ~(ulonglong) 0; + + /* + Coerce value to the 19XX form in order to correctly compare + YEAR(2) & YEAR(4) types. + */ + if (value < 70) + value+= 100; + if (value <= 1900) + value+= 1900; + + /* Convert year to DATETIME of form YYYY-00-00 00:00:00 (YYYY0000000000). */ + value*= 10000000000LL; + + return value; +} + + /* Compare items values as dates. @@ -1088,25 +1234,25 @@ int Arg_comparator::compare_datetime() longlong a_value, b_value; /* Get DATE/DATETIME/TIME value of the 'a' item. */ - a_value= (*get_value_func)(thd, &a, &a_cache, *b, &a_is_null); + a_value= (*get_value_a_func)(thd, &a, &a_cache, *b, &a_is_null); if (!is_nulls_eq && a_is_null) { - if (owner) + if (set_null) owner->null_value= 1; return -1; } /* Get DATE/DATETIME/TIME value of the 'b' item. */ - b_value= (*get_value_func)(thd, &b, &b_cache, *a, &b_is_null); + b_value= (*get_value_b_func)(thd, &b, &b_cache, *a, &b_is_null); if (a_is_null || b_is_null) { - if (owner) + if (set_null) owner->null_value= is_nulls_eq ? 0 : 1; return is_nulls_eq ? (a_is_null == b_is_null) : -1; } /* Here we have two not-NULL values. */ - if (owner) + if (set_null) owner->null_value= 0; /* Compare values. */ @@ -1119,15 +1265,17 @@ int Arg_comparator::compare_datetime() int Arg_comparator::compare_string() { String *res1,*res2; - if ((res1= (*a)->val_str(&owner->tmp_value1))) + if ((res1= (*a)->val_str(&value1))) { - if ((res2= (*b)->val_str(&owner->tmp_value2))) + if ((res2= (*b)->val_str(&value2))) { - owner->null_value= 0; + if (set_null) + owner->null_value= 0; return sortcmp(res1,res2,cmp_collation.collation); } } - owner->null_value= 1; + if (set_null) + owner->null_value= 1; return -1; } @@ -1146,18 +1294,20 @@ int Arg_comparator::compare_string() int Arg_comparator::compare_binary_string() { String *res1,*res2; - if ((res1= (*a)->val_str(&owner->tmp_value1))) + if ((res1= (*a)->val_str(&value1))) { - if ((res2= (*b)->val_str(&owner->tmp_value2))) + if ((res2= (*b)->val_str(&value2))) { - owner->null_value= 0; + if (set_null) + owner->null_value= 0; uint res1_length= res1->length(); uint res2_length= res2->length(); int cmp= memcmp(res1->ptr(), res2->ptr(), min(res1_length,res2_length)); return cmp ? cmp : (int) (res1_length - res2_length); } } - owner->null_value= 1; + if (set_null) + owner->null_value= 1; return -1; } @@ -1170,8 +1320,8 @@ int Arg_comparator::compare_binary_strin int Arg_comparator::compare_e_string() { String *res1,*res2; - res1= (*a)->val_str(&owner->tmp_value1); - res2= (*b)->val_str(&owner->tmp_value2); + res1= (*a)->val_str(&value1); + res2= (*b)->val_str(&value2); if (!res1 || !res2) return test(res1 == res2); return test(sortcmp(res1, res2, cmp_collation.collation) == 0); @@ -1181,8 +1331,8 @@ int Arg_comparator::compare_e_string() int Arg_comparator::compare_e_binary_string() { String *res1,*res2; - res1= (*a)->val_str(&owner->tmp_value1); - res2= (*b)->val_str(&owner->tmp_value2); + res1= (*a)->val_str(&value1); + res2= (*b)->val_str(&value2); if (!res1 || !res2) return test(res1 == res2); return test(stringcmp(res1, res2) == 0); @@ -1203,13 +1353,15 @@ int Arg_comparator::compare_real() val2= (*b)->val_real(); if (!(*b)->null_value) { - owner->null_value= 0; + if (set_null) + owner->null_value= 0; if (val1 < val2) return -1; if (val1 == val2) return 0; return 1; } } - owner->null_value= 1; + if (set_null) + owner->null_value= 1; return -1; } @@ -1223,11 +1375,13 @@ int Arg_comparator::compare_decimal() my_decimal *val2= (*b)->val_decimal(&value2); if (!(*b)->null_value) { - owner->null_value= 0; + if (set_null) + owner->null_value= 0; return my_decimal_cmp(val1, val2); } } - owner->null_value= 1; + if (set_null) + owner->null_value= 1; return -1; } @@ -1265,7 +1419,8 @@ int Arg_comparator::compare_real_fixed() val2= (*b)->val_real(); if (!(*b)->null_value) { - owner->null_value= 0; + if (set_null) + owner->null_value= 0; if (val1 == val2 || fabs(val1 - val2) < precision) return 0; if (val1 < val2) @@ -1273,7 +1428,8 @@ int Arg_comparator::compare_real_fixed() return 1; } } - owner->null_value= 1; + if (set_null) + owner->null_value= 1; return -1; } @@ -1296,13 +1452,15 @@ int Arg_comparator::compare_int_signed() longlong val2= (*b)->val_int(); if (!(*b)->null_value) { - owner->null_value= 0; + if (set_null) + owner->null_value= 0; if (val1 < val2) return -1; if (val1 == val2) return 0; return 1; } } - owner->null_value= 1; + if (set_null) + owner->null_value= 1; return -1; } @@ -1319,13 +1477,15 @@ int Arg_comparator::compare_int_unsigned ulonglong val2= (*b)->val_int(); if (!(*b)->null_value) { - owner->null_value= 0; + if (set_null) + owner->null_value= 0; if (val1 < val2) return -1; if (val1 == val2) return 0; return 1; } } - owner->null_value= 1; + if (set_null) + owner->null_value= 1; return -1; } @@ -1342,7 +1502,8 @@ int Arg_comparator::compare_int_signed_u ulonglong uval2= (ulonglong)(*b)->val_int(); if (!(*b)->null_value) { - owner->null_value= 0; + if (set_null) + owner->null_value= 0; if (sval1 < 0 || (ulonglong)sval1 < uval2) return -1; if ((ulonglong)sval1 == uval2) @@ -1350,7 +1511,8 @@ int Arg_comparator::compare_int_signed_u return 1; } } - owner->null_value= 1; + if (set_null) + owner->null_value= 1; return -1; } @@ -1367,7 +1529,8 @@ int Arg_comparator::compare_int_unsigned longlong sval2= (*b)->val_int(); if (!(*b)->null_value) { - owner->null_value= 0; + if (set_null) + owner->null_value= 0; if (sval2 < 0) return 1; if (uval1 < (ulonglong)sval2) @@ -1377,7 +1540,8 @@ int Arg_comparator::compare_int_unsigned return 1; } } - owner->null_value= 1; + if (set_null) + owner->null_value= 1; return -1; } @@ -1413,10 +1577,11 @@ int Arg_comparator::compare_row() for (uint i= 0; i<n; i++) { res= comparators[i].compare(); - if (owner->null_value) + /* Aggregate functions don't need special null handling. */ + if (owner->null_value && owner->type() == Item::FUNC_ITEM) { // NULL was compared - switch (owner->functype()) { + switch (((Item_func*)owner)->functype()) { case Item_func::NE_FUNC: break; // NE never aborts on NULL even if abort_on_null is set case Item_func::LT_FUNC: @@ -1425,7 +1590,7 @@ int Arg_comparator::compare_row() case Item_func::GE_FUNC: return -1; // <, <=, > and >= always fail on NULL default: // EQ_FUNC - if (owner->abort_on_null) + if (((Item_bool_func2*)owner)->abort_on_null) return -1; // We do not need correct NULL returning } was_null= 1; @@ -1581,6 +1746,7 @@ longlong Item_in_optimizer::val_int() bool tmp; DBUG_ASSERT(fixed == 1); cache->store(args[0]); + cache->cache_value(); if (cache->null_value) { @@ -1748,8 +1914,8 @@ longlong Item_func_lt::val_int() longlong Item_func_strcmp::val_int() { DBUG_ASSERT(fixed == 1); - String *a=args[0]->val_str(&tmp_value1); - String *b=args[1]->val_str(&tmp_value2); + String *a=args[0]->val_str(&cmp.value1); + String *b=args[1]->val_str(&cmp.value2); if (!a || !b) { null_value=1; @@ -2032,8 +2198,8 @@ void Item_func_between::fix_length_and_d if (compare_as_dates) { - ge_cmp.set_datetime_cmp_func(args, args + 1); - le_cmp.set_datetime_cmp_func(args, args + 2); + ge_cmp.set_datetime_cmp_func(this, args, args + 1); + le_cmp.set_datetime_cmp_func(this, args, args + 2); } else if (time_items_found == 3) { @@ -4370,13 +4536,13 @@ void Item_func_isnotnull::print(String * longlong Item_func_like::val_int() { DBUG_ASSERT(fixed == 1); - String* res = args[0]->val_str(&tmp_value1); + String* res = args[0]->val_str(&cmp.value1); if (args[0]->null_value) { null_value=1; return 0; } - String* res2 = args[1]->val_str(&tmp_value2); + String* res2 = args[1]->val_str(&cmp.value2); if (args[1]->null_value) { null_value=1; @@ -4400,7 +4566,7 @@ Item_func::optimize_type Item_func_like: { if (args[1]->const_item()) { - String* res2= args[1]->val_str((String *)&tmp_value2); + String* res2= args[1]->val_str((String *)&cmp.value2); if (!res2) return OPTIMIZE_NONE; @@ -4431,7 +4597,7 @@ bool Item_func_like::fix_fields(THD *thd if (escape_item->const_item()) { /* If we are on execution stage */ - String *escape_str= escape_item->val_str(&tmp_value1); + String *escape_str= escape_item->val_str(&cmp.value1); if (escape_str) { if (escape_used_in_parsing && ( @@ -4486,7 +4652,7 @@ bool Item_func_like::fix_fields(THD *thd if (args[1]->const_item() && !use_strnxfrm(collation.collation) && !(specialflag & SPECIAL_NO_NEW_FUNC)) { - String* res2 = args[1]->val_str(&tmp_value2); + String* res2 = args[1]->val_str(&cmp.value2); if (!res2) return FALSE; // Null argument === modified file 'sql/item_cmpfunc.h' --- a/sql/item_cmpfunc.h 2009-12-03 11:19:05 +0000 +++ b/sql/item_cmpfunc.h 2010-01-15 15:27:55 +0000 @@ -32,7 +32,7 @@ class Arg_comparator: public Sql_alloc { Item **a, **b; arg_cmp_func func; - Item_bool_func2 *owner; + Item_result_field *owner; Arg_comparator *comparators; // used only for compare_row() double precision; /* Fields used in DATE/DATETIME comparison. */ @@ -40,30 +40,40 @@ class Arg_comparator: public Sql_alloc enum_field_types a_type, b_type; // Types of a and b items Item *a_cache, *b_cache; // Cached values of a and b items bool is_nulls_eq; // TRUE <=> compare for the EQUAL_FUNC + bool set_null; // TRUE <=> set owner->null_value + // when one of arguments is NULL. enum enum_date_cmp_type { CMP_DATE_DFLT= 0, CMP_DATE_WITH_DATE, CMP_DATE_WITH_STR, CMP_STR_WITH_DATE }; - longlong (*get_value_func)(THD *thd, Item ***item_arg, Item **cache_arg, - Item *warn_item, bool *is_null); + longlong (*get_value_a_func)(THD *thd, Item ***item_arg, Item **cache_arg, + Item *warn_item, bool *is_null); + longlong (*get_value_b_func)(THD *thd, Item ***item_arg, Item **cache_arg, + Item *warn_item, bool *is_null); + bool try_year_cmp_func(Item_result type); public: DTCollation cmp_collation; + /* Allow owner function to use string buffers. */ + String value1, value2; - Arg_comparator(): thd(0), a_cache(0), b_cache(0) {}; + Arg_comparator(): thd(0), a_cache(0), b_cache(0), set_null(TRUE), + get_value_a_func(0), get_value_b_func(0) {}; Arg_comparator(Item **a1, Item **a2): a(a1), b(a2), thd(0), - a_cache(0), b_cache(0) {}; + a_cache(0), b_cache(0), set_null(TRUE), + get_value_a_func(0), get_value_b_func(0) {}; - int set_compare_func(Item_bool_func2 *owner, Item_result type); - inline int set_compare_func(Item_bool_func2 *owner_arg) + int set_compare_func(Item_result_field *owner, Item_result type); + inline int set_compare_func(Item_result_field *owner_arg) { return set_compare_func(owner_arg, item_cmp_type((*a)->result_type(), (*b)->result_type())); } - int set_cmp_func(Item_bool_func2 *owner_arg, + int set_cmp_func(Item_result_field *owner_arg, Item **a1, Item **a2, Item_result type); - inline int set_cmp_func(Item_bool_func2 *owner_arg, - Item **a1, Item **a2) + inline int set_cmp_func(Item_result_field *owner_arg, + Item **a1, Item **a2, bool set_null_arg) { + set_null= set_null_arg; return set_cmp_func(owner_arg, a1, a2, item_cmp_type((*a1)->result_type(), (*a2)->result_type())); @@ -93,8 +103,15 @@ public: static enum enum_date_cmp_type can_compare_as_dates(Item *a, Item *b, ulonglong *const_val_arg); - void set_datetime_cmp_func(Item **a1, Item **b1); + Item** cache_converted_constant(THD *thd, Item **value, Item **cache, + Item_result type); + void set_datetime_cmp_func(Item_result_field *owner_arg, Item **a1, Item **b1); static arg_cmp_func comparator_matrix [5][2]; + inline bool is_owner_equal_func() + { + return (owner->type() == Item::FUNC_ITEM && + ((Item_func*)owner)->functype() == Item_func::EQUAL_FUNC); + } friend class Item_func; }; @@ -324,7 +341,6 @@ class Item_bool_func2 :public Item_int_f { /* Bool with 2 string args */ protected: Arg_comparator cmp; - String tmp_value1,tmp_value2; bool abort_on_null; public: @@ -333,7 +349,7 @@ public: void fix_length_and_dec(); void set_cmp_func() { - cmp.set_cmp_func(this, tmp_arg, tmp_arg+1); + cmp.set_cmp_func(this, tmp_arg, tmp_arg+1, TRUE); } optimize_type select_optimize() const { return OPTIMIZE_OP; } virtual enum Functype rev_functype() const { return UNKNOWN_FUNC; } === modified file 'sql/item_create.cc' --- a/sql/item_create.cc 2009-10-15 21:38:29 +0000 +++ b/sql/item_create.cc 2010-01-15 15:27:55 +0000 @@ -3524,6 +3524,7 @@ Create_func_get_lock Create_func_get_loc Item* Create_func_get_lock::create(THD *thd, Item *arg1, Item *arg2) { + thd->lex->set_stmt_unsafe(); thd->lex->uncacheable(UNCACHEABLE_SIDEEFFECT); return new (thd->mem_root) Item_func_get_lock(arg1, arg2); } @@ -3635,6 +3636,7 @@ Create_func_is_free_lock Create_func_is_ Item* Create_func_is_free_lock::create(THD *thd, Item *arg1) { + thd->lex->set_stmt_unsafe(); thd->lex->uncacheable(UNCACHEABLE_SIDEEFFECT); return new (thd->mem_root) Item_func_is_free_lock(arg1); } @@ -3645,6 +3647,7 @@ Create_func_is_used_lock Create_func_is_ Item* Create_func_is_used_lock::create(THD *thd, Item *arg1) { + thd->lex->set_stmt_unsafe(); thd->lex->uncacheable(UNCACHEABLE_SIDEEFFECT); return new (thd->mem_root) Item_func_is_used_lock(arg1); } @@ -3961,6 +3964,8 @@ Create_func_master_pos_wait::create_nati Item *func= NULL; int arg_count= 0; + thd->lex->set_stmt_unsafe(); + if (item_list != NULL) arg_count= item_list->elements; @@ -4203,6 +4208,7 @@ Create_func_release_lock Create_func_rel Item* Create_func_release_lock::create(THD *thd, Item *arg1) { + thd->lex->set_stmt_unsafe(); thd->lex->uncacheable(UNCACHEABLE_SIDEEFFECT); return new (thd->mem_root) Item_func_release_lock(arg1); } @@ -4325,6 +4331,7 @@ Create_func_sleep Create_func_sleep::s_s Item* Create_func_sleep::create(THD *thd, Item *arg1) { + thd->lex->set_stmt_unsafe(); thd->lex->uncacheable(UNCACHEABLE_SIDEEFFECT); return new (thd->mem_root) Item_func_sleep(arg1); } @@ -4591,6 +4598,7 @@ Create_func_version Create_func_version: Item* Create_func_version::create(THD *thd) { + thd->lex->set_stmt_unsafe(); return new (thd->mem_root) Item_static_string_func("version()", server_version, (uint) strlen(server_version), === modified file 'sql/item_func.cc' --- a/sql/item_func.cc 2009-12-03 11:19:05 +0000 +++ b/sql/item_func.cc 2010-01-15 15:27:55 +0000 @@ -450,45 +450,8 @@ Field *Item_func::tmp_table_field(TABLE case STRING_RESULT: return make_string_field(table); case DECIMAL_RESULT: - { - uint8 dec= decimals; - uint8 intg= decimal_precision() - dec; - uint32 len= max_length; - - /* - Trying to put too many digits overall in a DECIMAL(prec,dec) - will always throw a warning. We must limit dec to - DECIMAL_MAX_SCALE however to prevent an assert() later. - */ - - if (dec > 0) - { - int overflow; - - dec= min(dec, DECIMAL_MAX_SCALE); - - /* - If the value still overflows the field with the corrected dec, - we'll throw out decimals rather than integers. This is still - bad and of course throws a truncation warning. - */ - - const int required_length= - my_decimal_precision_to_length(intg + dec, dec, - unsigned_flag); - - overflow= required_length - len; - - if (overflow > 0) - dec= max(0, dec - overflow); // too long, discard fract - else - /* Corrected value fits. */ - len= required_length; - } - - field= new Field_new_decimal(len, maybe_null, name, dec, unsigned_flag); + field= Field_new_decimal::create_from_item(this); break; - } case ROW_RESULT: default: // This case should never be chosen === modified file 'sql/item_func.h' --- a/sql/item_func.h 2009-12-03 11:19:05 +0000 +++ b/sql/item_func.h 2010-01-15 15:27:55 +0000 @@ -124,17 +124,6 @@ public: virtual optimize_type select_optimize() const { return OPTIMIZE_NONE; } virtual bool have_rev_func() const { return 0; } virtual Item *key_item() const { return args[0]; } - /* - This method is used for debug purposes to print the name of an - item to the debug log. The second use of this method is as - a helper function of print(), where it is applicable. - To suit both goals it should return a meaningful, - distinguishable and sintactically correct string. This method - should not be used for runtime type identification, use enum - {Sum}Functype and Item_func::functype()/Item_sum::sum_func() - instead. - */ - virtual const char *func_name() const= 0; virtual bool const_item() const { return const_item_cache; } inline Item **arguments() const { return args; } void set_arguments(List<Item> &list); === modified file 'sql/item_geofunc.cc' --- a/sql/item_geofunc.cc 2009-10-24 06:57:31 +0000 +++ b/sql/item_geofunc.cc 2009-12-08 09:26:11 +0000 @@ -511,8 +511,8 @@ err: longlong Item_func_spatial_rel::val_int() { DBUG_ASSERT(fixed == 1); - String *res1= args[0]->val_str(&tmp_value1); - String *res2= args[1]->val_str(&tmp_value2); + String *res1= args[0]->val_str(&cmp.value1); + String *res2= args[1]->val_str(&cmp.value2); Geometry_buffer buffer1, buffer2; Geometry *g1, *g2; MBR mbr1, mbr2; === modified file 'sql/item_strfunc.cc' --- a/sql/item_strfunc.cc 2009-12-03 11:19:05 +0000 +++ b/sql/item_strfunc.cc 2010-01-15 15:27:55 +0000 @@ -1828,8 +1828,9 @@ String *Item_func_database::val_str(Stri /** - @todo - make USER() replicate properly (currently it is replicated to "") + @note USER() is replicated correctly if binlog_format=ROW or (as of + BUG#28086) binlog_format=MIXED, but is incorrectly replicated to '' + if binlog_format=STATEMENT. */ bool Item_func_user::init(const char *user, const char *host) { === modified file 'sql/item_subselect.cc' --- a/sql/item_subselect.cc 2009-12-03 11:19:05 +0000 +++ b/sql/item_subselect.cc 2010-01-15 15:27:55 +0000 @@ -503,6 +503,7 @@ Item_singlerow_subselect::select_transfo void Item_singlerow_subselect::store(uint i, Item *item) { row[i]->store(item); + row[i]->cache_value(); } enum Item_result Item_singlerow_subselect::result_type() const @@ -1854,6 +1855,7 @@ void subselect_engine::set_row(List<Item if (!(row[i]= Item_cache::get_cache(sel_item))) return; row[i]->setup(sel_item); + row[i]->store(sel_item); } if (item_list.elements > 1) res_type= ROW_RESULT; === modified file 'sql/item_subselect.h' --- a/sql/item_subselect.h 2009-08-31 20:02:09 +0000 +++ b/sql/item_subselect.h 2010-01-15 15:27:55 +0000 @@ -142,6 +142,7 @@ public: @return the SELECT_LEX structure associated with this Item */ st_select_lex* get_select_lex(); + const char *func_name() const { DBUG_ASSERT(0); return "subselect"; } friend class select_subselect; friend class Item_in_optimizer; === modified file 'sql/item_sum.cc' --- a/sql/item_sum.cc 2009-12-03 11:19:05 +0000 +++ b/sql/item_sum.cc 2010-01-15 15:27:55 +0000 @@ -517,8 +517,7 @@ Field *Item_sum::create_tmp_field(bool g name, table->s, collation.collation); break; case DECIMAL_RESULT: - field= new Field_new_decimal(max_length, maybe_null, name, - decimals, unsigned_flag); + field= Field_new_decimal::create_from_item(this); break; case ROW_RESULT: default: @@ -610,35 +609,6 @@ Item_sum_num::fix_fields(THD *thd, Item } -Item_sum_hybrid::Item_sum_hybrid(THD *thd, Item_sum_hybrid *item) - :Item_sum(thd, item), value(item->value), hybrid_type(item->hybrid_type), - hybrid_field_type(item->hybrid_field_type), cmp_sign(item->cmp_sign), - was_values(item->was_values) -{ - /* copy results from old value */ - switch (hybrid_type) { - case INT_RESULT: - sum_int= item->sum_int; - break; - case DECIMAL_RESULT: - my_decimal2decimal(&item->sum_dec, &sum_dec); - break; - case REAL_RESULT: - sum= item->sum; - break; - case STRING_RESULT: - /* - This can happen with ROLLUP. Note that the value is already - copied at function call. - */ - break; - case ROW_RESULT: - default: - DBUG_ASSERT(0); - } - collation.set(item->collation); -} - bool Item_sum_hybrid::fix_fields(THD *thd, Item **ref) { @@ -658,15 +628,12 @@ Item_sum_hybrid::fix_fields(THD *thd, It switch (hybrid_type= item->result_type()) { case INT_RESULT: max_length= 20; - sum_int= 0; break; case DECIMAL_RESULT: max_length= item->max_length; - my_decimal_set_zero(&sum_dec); break; case REAL_RESULT: max_length= float_length(decimals); - sum= 0.0; break; case STRING_RESULT: max_length= item->max_length; @@ -675,10 +642,10 @@ Item_sum_hybrid::fix_fields(THD *thd, It default: DBUG_ASSERT(0); }; + setup(args[0], NULL); /* MIN/MAX can return NULL for empty set indepedent of the used column */ maybe_null= 1; unsigned_flag=item->unsigned_flag; - collation.set(item->collation); result_field=0; null_value=1; fix_length_and_dec(); @@ -696,6 +663,30 @@ Item_sum_hybrid::fix_fields(THD *thd, It return FALSE; } + +/** + MIN/MAX function setup. + + @param item argument of MIN/MAX function + @param value_arg calculated value of MIN/MAX function + + @details + Setup cache/comparator of MIN/MAX functions. When called by the + copy_or_same function value_arg parameter contains calculated value + of the original MIN/MAX object and it is saved in this object's cache. +*/ + +void Item_sum_hybrid::setup(Item *item, Item *value_arg) +{ + value= Item_cache::get_cache(item); + value->setup(item); + value->store(value_arg); + cmp= new Arg_comparator(); + cmp->set_cmp_func(this, args, (Item**)&value, FALSE); + collation.set(item->collation); +} + + Field *Item_sum_hybrid::create_tmp_field(bool group, TABLE *table, uint convert_blob_length) { @@ -1265,8 +1256,7 @@ Field *Item_sum_avg::create_tmp_field(bo 0, name, &my_charset_bin); } else if (hybrid_type == DECIMAL_RESULT) - field= new Field_new_decimal(max_length, maybe_null, name, - decimals, unsigned_flag); + field= Field_new_decimal::create_from_item(this); else field= new Field_double(max_length, maybe_null, name, decimals, TRUE); if (field) @@ -1587,19 +1577,7 @@ void Item_sum_variance::update_field() void Item_sum_hybrid::clear() { - switch (hybrid_type) { - case INT_RESULT: - sum_int= 0; - break; - case DECIMAL_RESULT: - my_decimal_set_zero(&sum_dec); - break; - case REAL_RESULT: - sum= 0.0; - break; - default: - value.length(0); - } + value->null_value= 1; null_value= 1; } @@ -1608,30 +1586,7 @@ double Item_sum_hybrid::val_real() DBUG_ASSERT(fixed == 1); if (null_value) return 0.0; - switch (hybrid_type) { - case STRING_RESULT: - { - char *end_not_used; - int err_not_used; - String *res; res=val_str(&str_value); - return (res ? my_strntod(res->charset(), (char*) res->ptr(), res->length(), - &end_not_used, &err_not_used) : 0.0); - } - case INT_RESULT: - if (unsigned_flag) - return ulonglong2double(sum_int); - return (double) sum_int; - case DECIMAL_RESULT: - my_decimal2double(E_DEC_FATAL_ERROR, &sum_dec, &sum); - return sum; - case REAL_RESULT: - return sum; - case ROW_RESULT: - default: - // This case should never be choosen - DBUG_ASSERT(0); - return 0; - } + return value->val_real(); } longlong Item_sum_hybrid::val_int() @@ -1639,18 +1594,7 @@ longlong Item_sum_hybrid::val_int() DBUG_ASSERT(fixed == 1); if (null_value) return 0; - switch (hybrid_type) { - case INT_RESULT: - return sum_int; - case DECIMAL_RESULT: - { - longlong result; - my_decimal2int(E_DEC_FATAL_ERROR, &sum_dec, unsigned_flag, &result); - return sum_int; - } - default: - return (longlong) rint(Item_sum_hybrid::val_real()); - } + return value->val_int(); } @@ -1659,26 +1603,7 @@ my_decimal *Item_sum_hybrid::val_decimal DBUG_ASSERT(fixed == 1); if (null_value) return 0; - switch (hybrid_type) { - case STRING_RESULT: - string2my_decimal(E_DEC_FATAL_ERROR, &value, val); - break; - case REAL_RESULT: - double2my_decimal(E_DEC_FATAL_ERROR, sum, val); - break; - case DECIMAL_RESULT: - val= &sum_dec; - break; - case INT_RESULT: - int2my_decimal(E_DEC_FATAL_ERROR, sum_int, unsigned_flag, val); - break; - case ROW_RESULT: - default: - // This case should never be choosen - DBUG_ASSERT(0); - break; - } - return val; // Keep compiler happy + return value->val_decimal(val); } @@ -1688,25 +1613,7 @@ Item_sum_hybrid::val_str(String *str) DBUG_ASSERT(fixed == 1); if (null_value) return 0; - switch (hybrid_type) { - case STRING_RESULT: - return &value; - case REAL_RESULT: - str->set_real(sum,decimals, &my_charset_bin); - break; - case DECIMAL_RESULT: - my_decimal2string(E_DEC_FATAL_ERROR, &sum_dec, 0, 0, 0, str); - return str; - case INT_RESULT: - str->set_int(sum_int, unsigned_flag, &my_charset_bin); - break; - case ROW_RESULT: - default: - // This case should never be choosen - DBUG_ASSERT(0); - break; - } - return str; // Keep compiler happy + return value->val_str(str); } @@ -1715,7 +1622,9 @@ void Item_sum_hybrid::cleanup() DBUG_ENTER("Item_sum_hybrid::cleanup"); Item_sum::cleanup(); forced_const= FALSE; - + if (cmp) + delete cmp; + cmp= 0; /* by default it is TRUE to avoid TRUE reporting by Item_func_not_all/Item_func_nop_all if this item was never called. @@ -1736,63 +1645,22 @@ void Item_sum_hybrid::no_rows_in_result( Item *Item_sum_min::copy_or_same(THD* thd) { - return new (thd->mem_root) Item_sum_min(thd, this); + Item_sum_min *item= new (thd->mem_root) Item_sum_min(thd, this); + item->setup(args[0], value); + return item; } bool Item_sum_min::add() { - switch (hybrid_type) { - case STRING_RESULT: + /* args[0] < value */ + int res= cmp->compare(); + if (!args[0]->null_value && + (null_value || res < 0)) { - String *result=args[0]->val_str(&tmp_value); - if (!args[0]->null_value && - (null_value || sortcmp(&value,result,collation.collation) > 0)) - { - value.copy(*result); - null_value=0; - } - } - break; - case INT_RESULT: - { - longlong nr=args[0]->val_int(); - if (!args[0]->null_value && (null_value || - (unsigned_flag && - (ulonglong) nr < (ulonglong) sum_int) || - (!unsigned_flag && nr < sum_int))) - { - sum_int=nr; - null_value=0; - } - } - break; - case DECIMAL_RESULT: - { - my_decimal value_buff, *val= args[0]->val_decimal(&value_buff); - if (!args[0]->null_value && - (null_value || (my_decimal_cmp(&sum_dec, val) > 0))) - { - my_decimal2decimal(val, &sum_dec); - null_value= 0; - } - } - break; - case REAL_RESULT: - { - double nr= args[0]->val_real(); - if (!args[0]->null_value && (null_value || nr < sum)) - { - sum=nr; - null_value=0; - } - } - break; - case ROW_RESULT: - default: - // This case should never be choosen - DBUG_ASSERT(0); - break; + value->store(args[0]); + value->cache_value(); + null_value= 0; } return 0; } @@ -1800,63 +1668,22 @@ bool Item_sum_min::add() Item *Item_sum_max::copy_or_same(THD* thd) { - return new (thd->mem_root) Item_sum_max(thd, this); + Item_sum_max *item= new (thd->mem_root) Item_sum_max(thd, this); + item->setup(args[0], value); + return item; } bool Item_sum_max::add() { - switch (hybrid_type) { - case STRING_RESULT: + /* args[0] > value */ + int res= cmp->compare(); + if (!args[0]->null_value && + (null_value || res > 0)) { - String *result=args[0]->val_str(&tmp_value); - if (!args[0]->null_value && - (null_value || sortcmp(&value,result,collation.collation) < 0)) - { - value.copy(*result); - null_value=0; - } - } - break; - case INT_RESULT: - { - longlong nr=args[0]->val_int(); - if (!args[0]->null_value && (null_value || - (unsigned_flag && - (ulonglong) nr > (ulonglong) sum_int) || - (!unsigned_flag && nr > sum_int))) - { - sum_int=nr; - null_value=0; - } - } - break; - case DECIMAL_RESULT: - { - my_decimal value_buff, *val= args[0]->val_decimal(&value_buff); - if (!args[0]->null_value && - (null_value || (my_decimal_cmp(val, &sum_dec) > 0))) - { - my_decimal2decimal(val, &sum_dec); - null_value= 0; - } - } - break; - case REAL_RESULT: - { - double nr= args[0]->val_real(); - if (!args[0]->null_value && (null_value || nr > sum)) - { - sum=nr; - null_value=0; - } - } - break; - case ROW_RESULT: - default: - // This case should never be choosen - DBUG_ASSERT(0); - break; + value->store(args[0]); + value->cache_value(); + null_value= 0; } return 0; } @@ -2221,14 +2048,15 @@ void Item_sum_hybrid::update_field() void Item_sum_hybrid::min_max_update_str_field() { - String *res_str=args[0]->val_str(&value); + DBUG_ASSERT(cmp); + String *res_str=args[0]->val_str(&cmp->value1); if (!args[0]->null_value) { - result_field->val_str(&tmp_value); + result_field->val_str(&cmp->value2); if (result_field->is_null() || - (cmp_sign * sortcmp(res_str,&tmp_value,collation.collation)) < 0) + (cmp_sign * sortcmp(res_str,&cmp->value2,collation.collation)) < 0) result_field->store(res_str->ptr(),res_str->length(),res_str->charset()); result_field->set_notnull(); } === modified file 'sql/item_sum.h' --- a/sql/item_sum.h 2009-09-15 10:46:35 +0000 +++ b/sql/item_sum.h 2010-01-15 15:27:55 +0000 @@ -329,22 +329,6 @@ public: virtual void update_field()=0; virtual bool keep_field_type(void) const { return 0; } virtual void fix_length_and_dec() { maybe_null=1; null_value=1; } - /* - This method is used for debug purposes to print the name of an - item to the debug log. The second use of this method is as - a helper function of print(), where it is applicable. - To suit both goals it should return a meaningful, - distinguishable and sintactically correct string. This method - should not be used for runtime type identification, use enum - {Sum}Functype and Item_func::functype()/Item_sum::sum_func() - instead. - - NOTE: for Items inherited from Item_sum, func_name() return part of - function name till first argument (including '(') to make difference in - names for functions with 'distinct' clause and without 'distinct' and - also to make printing of items inherited from Item_sum uniform. - */ - virtual const char *func_name() const= 0; virtual Item *result_item(Field *field) { return new Item_field(field); } /* @@ -679,6 +663,7 @@ public: } void fix_length_and_dec() {} enum Item_result result_type () const { return hybrid_type; } + const char *func_name() const { DBUG_ASSERT(0); return "avg_field"; } }; @@ -747,6 +732,7 @@ public: } void fix_length_and_dec() {} enum Item_result result_type () const { return hybrid_type; } + const char *func_name() const { DBUG_ASSERT(0); return "variance_field"; } }; @@ -822,6 +808,7 @@ public: my_decimal *val_decimal(my_decimal *); enum Item_result result_type () const { return REAL_RESULT; } enum_field_types field_type() const { return MYSQL_TYPE_DOUBLE;} + const char *func_name() const { DBUG_ASSERT(0); return "std_field"; } }; /* @@ -847,14 +834,13 @@ class Item_sum_std :public Item_sum_vari }; // This class is a string or number function depending on num_func - +class Arg_comparator; +class Item_cache; class Item_sum_hybrid :public Item_sum { protected: - String value,tmp_value; - double sum; - longlong sum_int; - my_decimal sum_dec; + Item_cache *value; + Arg_comparator *cmp; Item_result hybrid_type; enum_field_types hybrid_field_type; int cmp_sign; @@ -862,12 +848,17 @@ protected: public: Item_sum_hybrid(Item *item_par,int sign) - :Item_sum(item_par), sum(0.0), sum_int(0), + :Item_sum(item_par), value(0), cmp(0), hybrid_type(INT_RESULT), hybrid_field_type(MYSQL_TYPE_LONGLONG), cmp_sign(sign), was_values(TRUE) { collation.set(&my_charset_bin); } - Item_sum_hybrid(THD *thd, Item_sum_hybrid *item); + Item_sum_hybrid(THD *thd, Item_sum_hybrid *item) + :Item_sum(thd, item), value(item->value), hybrid_type(item->hybrid_type), + hybrid_field_type(item->hybrid_field_type), cmp_sign(item->cmp_sign), + was_values(item->was_values) + { } bool fix_fields(THD *, Item **); + void setup(Item *item, Item *value_arg); void clear(); double val_real(); longlong val_int(); === modified file 'sql/item_timefunc.cc' --- a/sql/item_timefunc.cc 2009-12-03 11:19:05 +0000 +++ b/sql/item_timefunc.cc 2010-01-15 15:27:55 +0000 @@ -391,7 +391,7 @@ static bool extract_date_time(DATE_TIME_ if (tmp - val > 6) tmp= (char*) val + 6; l_time->second_part= (int) my_strtoll10(val, &tmp, &error); - frac_part= 6 - (uint) (tmp - val); + frac_part= 6 - (int) (tmp - val); if (frac_part > 0) l_time->second_part*= (ulong) log_10_int[frac_part]; val= tmp; @@ -882,9 +882,9 @@ static bool get_interval_info(const char value= value*LL(10) + (longlong) (*str - '0'); if (transform_msec && i == count - 1) // microseconds always last { - long msec_length= 6 - (uint) (str - start); + int msec_length= 6 - (int)(str - start); if (msec_length > 0) - value*= (long) log_10_int[msec_length]; + value*= (long)log_10_int[msec_length]; } values[i]= value; while (str != end && !my_isdigit(cs,*str)) === modified file 'sql/item_xmlfunc.cc' --- a/sql/item_xmlfunc.cc 2009-12-03 11:19:05 +0000 +++ b/sql/item_xmlfunc.cc 2010-01-15 15:27:55 +0000 @@ -941,14 +941,16 @@ static Item *create_comparator(MY_XPATH in a loop through all of the nodes in the node set. */ - Item *fake= new Item_string("", 0, xpath->cs); + Item_string *fake= new Item_string("", 0, xpath->cs); + /* Don't cache fake because its value will be changed during comparison.*/ + fake->set_used_tables(RAND_TABLE_BIT); Item_nodeset_func *nodeset; Item *scalar, *comp; if (a->type() == Item::XPATH_NODESET) { nodeset= (Item_nodeset_func*) a; scalar= b; - comp= eq_func(oper, fake, scalar); + comp= eq_func(oper, (Item*)fake, scalar); } else { === modified file 'sql/log.cc' --- a/sql/log.cc 2010-01-04 18:25:29 +0000 +++ b/sql/log.cc 2010-01-15 15:27:55 +0000 @@ -5691,9 +5691,8 @@ int TC_LOG_BINLOG::recover(IO_CACHE *log Xid_log_event *xev=(Xid_log_event *)ev; uchar *x= (uchar *) memdup_root(&mem_root, (uchar*) &xev->xid, sizeof(xev->xid)); - if (! x) + if (!x || my_hash_insert(&xids, x)) goto err2; - my_hash_insert(&xids, x); } delete ev; } === modified file 'sql/log_event.cc' --- a/sql/log_event.cc 2009-12-03 11:34:11 +0000 +++ b/sql/log_event.cc 2010-01-15 15:27:55 +0000 @@ -8453,13 +8453,17 @@ Rows_log_event::write_row(const Relay_lo auto_afree_ptr<char> key(NULL); /* fill table->record[0] with default values */ - + bool abort_on_warnings= (rli->sql_thd->variables.sql_mode & + (MODE_STRICT_TRANS_TABLES | MODE_STRICT_ALL_TABLES)); if ((error= prepare_record(table, m_width, - TRUE /* check if columns have def. values */))) + table->file->ht->db_type != DB_TYPE_NDBCLUSTER, + abort_on_warnings, m_curr_row == m_rows_buf))) DBUG_RETURN(error); /* unpack row into table->record[0] */ - error= unpack_current_row(rli); // TODO: how to handle errors? + if ((error= unpack_current_row(rli, abort_on_warnings))) + DBUG_RETURN(error); + if (m_curr_row == m_rows_buf) { /* this is the first row to be inserted, we estimate the rows with @@ -9256,8 +9260,12 @@ Update_rows_log_event::do_exec_row(const store_record(m_table,record[1]); + bool abort_on_warnings= (rli->sql_thd->variables.sql_mode & + (MODE_STRICT_TRANS_TABLES | MODE_STRICT_ALL_TABLES)); m_curr_row= m_curr_row_end; - error= unpack_current_row(rli); // this also updates m_curr_row_end + /* this also updates m_curr_row_end */ + if ((error= unpack_current_row(rli, abort_on_warnings))) + return error; /* Now we have the right row to update. The old row (the one we're === modified file 'sql/log_event.h' --- a/sql/log_event.h 2009-12-03 11:19:05 +0000 +++ b/sql/log_event.h 2010-01-15 15:27:55 +0000 @@ -3541,12 +3541,16 @@ protected: int write_row(const Relay_log_info *const, const bool); // Unpack the current row into m_table->record[0] - int unpack_current_row(const Relay_log_info *const rli) + int unpack_current_row(const Relay_log_info *const rli, + const bool abort_on_warning= TRUE) { DBUG_ASSERT(m_table); + + bool first_row= (m_curr_row == m_rows_buf); ASSERT_OR_RETURN_ERROR(m_curr_row < m_rows_end, HA_ERR_CORRUPT_EVENT); int const result= ::unpack_row(rli, m_table, m_width, m_curr_row, &m_cols, - &m_curr_row_end, &m_master_reclength); + &m_curr_row_end, &m_master_reclength, + abort_on_warning, first_row); if (m_curr_row_end > m_rows_end) my_error(ER_SLAVE_CORRUPT_EVENT, MYF(0)); ASSERT_OR_RETURN_ERROR(m_curr_row_end <= m_rows_end, HA_ERR_CORRUPT_EVENT); === modified file 'sql/mysqld.cc' --- a/sql/mysqld.cc 2009-12-04 15:12:22 +0000 +++ b/sql/mysqld.cc 2010-01-15 15:27:55 +0000 @@ -4005,6 +4005,27 @@ server."); if (opt_bin_log) { + /* Reports an error and aborts, if the --log-bin's path + is a directory.*/ + if (opt_bin_logname && + opt_bin_logname[strlen(opt_bin_logname) - 1] == FN_LIBCHAR) + { + sql_print_error("Path '%s' is a directory name, please specify \ +a file name for --log-bin option", opt_bin_logname); + unireg_abort(1); + } + + /* Reports an error and aborts, if the --log-bin-index's path + is a directory.*/ + if (opt_binlog_index_name && + opt_binlog_index_name[strlen(opt_binlog_index_name) - 1] + == FN_LIBCHAR) + { + sql_print_error("Path '%s' is a directory name, please specify \ +a file name for --log-bin-index option", opt_binlog_index_name); + unireg_abort(1); + } + char buf[FN_REFLEN]; const char *ln; ln= mysql_bin_log.generate_name(opt_bin_logname, "-bin", 1, buf); @@ -5355,12 +5376,16 @@ pthread_handler_t handle_connections_soc pthread_handler_t handle_connections_namedpipes(void *arg) { HANDLE hConnectedPipe; - OVERLAPPED connectOverlapped = {0}; + OVERLAPPED connectOverlapped= {0}; THD *thd; my_thread_init(); DBUG_ENTER("handle_connections_namedpipes"); - connectOverlapped.hEvent = CreateEvent(NULL, TRUE, FALSE, NULL); - + connectOverlapped.hEvent= CreateEvent(NULL, TRUE, FALSE, NULL); + if (!connectOverlapped.hEvent) + { + sql_print_error("Can't create event, last error=%u", GetLastError()); + unireg_abort(1); + } DBUG_PRINT("general",("Waiting for named pipe connections.")); while (!abort_loop) { @@ -5383,7 +5408,8 @@ pthread_handler_t handle_connections_nam { CloseHandle(hPipe); if ((hPipe= CreateNamedPipe(pipe_name, - PIPE_ACCESS_DUPLEX|FILE_FLAG_OVERLAPPED, + PIPE_ACCESS_DUPLEX | + FILE_FLAG_OVERLAPPED, PIPE_TYPE_BYTE | PIPE_READMODE_BYTE | PIPE_WAIT, @@ -5403,7 +5429,8 @@ pthread_handler_t handle_connections_nam hConnectedPipe = hPipe; /* create new pipe for new connection */ if ((hPipe = CreateNamedPipe(pipe_name, - PIPE_ACCESS_DUPLEX|FILE_FLAG_OVERLAPPED, + PIPE_ACCESS_DUPLEX | + FILE_FLAG_OVERLAPPED, PIPE_TYPE_BYTE | PIPE_READMODE_BYTE | PIPE_WAIT, @@ -8974,14 +9001,8 @@ static int fix_paths(void) pos[0]= FN_LIBCHAR; pos[1]= 0; } - convert_dirname(mysql_real_data_home,mysql_real_data_home,NullS); - my_realpath(mysql_unpacked_real_data_home, mysql_real_data_home, MYF(0)); - mysql_unpacked_real_data_home_len= strlen(mysql_unpacked_real_data_home); - if (mysql_unpacked_real_data_home[mysql_unpacked_real_data_home_len-1] == FN_LIBCHAR) - --mysql_unpacked_real_data_home_len; - - convert_dirname(language,language,NullS); + convert_dirname(mysql_real_data_home,mysql_real_data_home,NullS); (void) my_load_path(mysql_home,mysql_home,""); // Resolve current dir (void) my_load_path(mysql_real_data_home,mysql_real_data_home,mysql_home); (void) my_load_path(pidfile_name,pidfile_name,mysql_real_data_home); @@ -8989,6 +9010,12 @@ static int fix_paths(void) get_relative_path(PLUGINDIR), mysql_home); opt_plugin_dir_ptr= opt_plugin_dir; + my_realpath(mysql_unpacked_real_data_home, mysql_real_data_home, MYF(0)); + mysql_unpacked_real_data_home_len= + (int) strlen(mysql_unpacked_real_data_home); + if (mysql_unpacked_real_data_home[mysql_unpacked_real_data_home_len-1] == FN_LIBCHAR) + --mysql_unpacked_real_data_home_len; + char *sharedir=get_relative_path(SHAREDIR); if (test_if_hard_path(sharedir)) strmake(buff,sharedir,sizeof(buff)-1); /* purecov: tested */ @@ -9019,8 +9046,8 @@ static int fix_paths(void) /* Convert the secure-file-priv option to system format, allowing a quick strcmp to check if read or write is in an allowed dir - */ - if (opt_secure_file_priv) + */ + if (opt_secure_file_priv && opt_secure_file_priv[0]) { convert_dirname(buff, opt_secure_file_priv, NullS); my_free(opt_secure_file_priv, MYF(0)); === modified file 'sql/opt_range.cc' --- a/sql/opt_range.cc 2009-12-03 11:19:05 +0000 +++ b/sql/opt_range.cc 2010-01-15 15:27:55 +0000 @@ -446,9 +446,9 @@ public: range_key, *range_key_flag); *range_key_flag|= key_tree->min_flag; if (key_tree->next_key_part && + key_tree->next_key_part->type == SEL_ARG::KEY_RANGE && key_tree->next_key_part->part == key_tree->part+1 && - !(*range_key_flag & (NO_MIN_RANGE | NEAR_MIN)) && - key_tree->next_key_part->type == SEL_ARG::KEY_RANGE) + !(*range_key_flag & (NO_MIN_RANGE | NEAR_MIN))) res+= key_tree->next_key_part->store_min_key(key, range_key, range_key_flag); return res; @@ -462,9 +462,9 @@ public: range_key, *range_key_flag); (*range_key_flag)|= key_tree->max_flag; if (key_tree->next_key_part && + key_tree->next_key_part->type == SEL_ARG::KEY_RANGE && key_tree->next_key_part->part == key_tree->part+1 && - !(*range_key_flag & (NO_MAX_RANGE | NEAR_MAX)) && - key_tree->next_key_part->type == SEL_ARG::KEY_RANGE) + !(*range_key_flag & (NO_MAX_RANGE | NEAR_MAX))) res+= key_tree->next_key_part->store_max_key(key, range_key, range_key_flag); return res; @@ -1700,6 +1700,7 @@ SEL_ARG *SEL_ARG::clone(RANGE_OPT_PARAM tmp->prev= *next_arg; // Link into next/prev chain (*next_arg)->next=tmp; (*next_arg)= tmp; + tmp->part= this->part; } else { @@ -6672,6 +6673,7 @@ key_or(RANGE_OPT_PARAM *param, SEL_ARG * else if ((cmp=tmp->cmp_max_to_min(key2)) < 0) { // Found tmp.max < key2.min SEL_ARG *next=tmp->next; + /* key1 on the left of key2 non-overlapping */ if (cmp == -2 && eq_tree(tmp->next_key_part,key2->next_key_part)) { // Join near ranges like tmp.max < 0 and key2.min >= 0 @@ -6700,6 +6702,7 @@ key_or(RANGE_OPT_PARAM *param, SEL_ARG * int tmp_cmp; if ((tmp_cmp=tmp->cmp_min_to_max(key2)) > 0) // if tmp.min > key2.max { + /* tmp is on the right of key2 non-overlapping */ if (tmp_cmp == 2 && eq_tree(tmp->next_key_part,key2->next_key_part)) { // ranges are connected tmp->copy_min_to_min(key2); @@ -6734,25 +6737,52 @@ key_or(RANGE_OPT_PARAM *param, SEL_ARG * } } - // tmp.max >= key2.min && tmp.min <= key.max (overlapping ranges) + /* + tmp.min >= key2.min && tmp.min <= key.max (overlapping ranges) + key2.min <= tmp.min <= key2.max + */ if (eq_tree(tmp->next_key_part,key2->next_key_part)) { if (tmp->is_same(key2)) { + /* + Found exact match of key2 inside key1. + Use the relevant range in key1. + */ tmp->merge_flags(key2); // Copy maybe flags key2->increment_use_count(-1); // Free not used tree } else { SEL_ARG *last=tmp; + SEL_ARG *first=tmp; + /* + Find the last range in tmp that overlaps key2 and has the same + condition on the rest of the keyparts. + */ while (last->next && last->next->cmp_min_to_max(key2) <= 0 && eq_tree(last->next->next_key_part,key2->next_key_part)) { + /* + We've found the last overlapping key1 range in last. + This means that the ranges between (and including) the + first overlapping range (tmp) and the last overlapping range + (last) are fully nested into the current range of key2 + and can safely be discarded. We just need the minimum endpoint + of the first overlapping range (tmp) so we can compare it with + the minimum endpoint of the enclosing key2 range. + */ SEL_ARG *save=last; last=last->next; key1=key1->tree_delete(save); } - last->copy_min(tmp); + /* + The tmp range (the first overlapping range) could have been discarded + by the previous loop. We should re-direct tmp to the new united range + that's taking its place. + */ + tmp= last; + last->copy_min(first); bool full_range= last->copy_min(key2); if (!full_range) { @@ -7262,27 +7292,25 @@ int test_rb_tree(SEL_ARG *element,SEL_AR } -/* - Count how many times SEL_ARG graph "root" refers to its part "key" +/** + Count how many times SEL_ARG graph "root" refers to its part "key" via + transitive closure. - SYNOPSIS - count_key_part_usage() - root An RB-Root node in a SEL_ARG graph. - key Another RB-Root node in that SEL_ARG graph. + @param root An RB-Root node in a SEL_ARG graph. + @param key Another RB-Root node in that SEL_ARG graph. - DESCRIPTION - The passed "root" node may refer to "key" node via root->next_key_part, - root->next->n + The passed "root" node may refer to "key" node via root->next_key_part, + root->next->n - This function counts how many times the node "key" is referred (via - SEL_ARG::next_key_part) by - - intervals of RB-tree pointed by "root", - - intervals of RB-trees that are pointed by SEL_ARG::next_key_part from - intervals of RB-tree pointed by "root", - - and so on. + This function counts how many times the node "key" is referred (via + SEL_ARG::next_key_part) by + - intervals of RB-tree pointed by "root", + - intervals of RB-trees that are pointed by SEL_ARG::next_key_part from + intervals of RB-tree pointed by "root", + - and so on. - Here is an example (horizontal links represent next_key_part pointers, - vertical links - next/prev prev pointers): + Here is an example (horizontal links represent next_key_part pointers, + vertical links - next/prev prev pointers): +----+ $ |root|-----------------+ @@ -7302,8 +7330,8 @@ int test_rb_tree(SEL_ARG *element,SEL_AR ... +---+ $ | | |------------+ +---+ $ - RETURN - Number of links to "key" from nodes reachable from "root". + @return + Number of links to "key" from nodes reachable from "root". */ static ulong count_key_part_usage(SEL_ARG *root, SEL_ARG *key) @@ -7558,8 +7586,8 @@ check_quick_keys(PARAM *param, uint idx, param->first_null_comp= key_tree->part+1; if (key_tree->next_key_part && - key_tree->next_key_part->part == key_tree->part+1 && - key_tree->next_key_part->type == SEL_ARG::KEY_RANGE) + key_tree->next_key_part->type == SEL_ARG::KEY_RANGE && + key_tree->next_key_part->part == key_tree->part+1) { // const key as prefix if (min_key_length == max_key_length && !memcmp(min_key, max_key, (uint) (tmp_max_key - max_key)) && @@ -7840,8 +7868,8 @@ get_quick_keys(PARAM *param,QUICK_RANGE_ &tmp_max_key,max_key_flag); if (key_tree->next_key_part && - key_tree->next_key_part->part == key_tree->part+1 && - key_tree->next_key_part->type == SEL_ARG::KEY_RANGE) + key_tree->next_key_part->type == SEL_ARG::KEY_RANGE && + key_tree->next_key_part->part == key_tree->part+1) { // const key as prefix if ((tmp_min_key - min_key) == (tmp_max_key - max_key) && memcmp(min_key, max_key, (uint)(tmp_max_key - max_key))==0 && @@ -9823,7 +9851,11 @@ check_group_min_max_predicates(COND *con } else if (cur_arg->const_item()) { - DBUG_RETURN(TRUE); + /* + For predicates of the form "const OP expr" we also have to check 'expr' + to make a decision. + */ + continue; } else DBUG_RETURN(FALSE); === modified file 'sql/repl_failsafe.cc' --- a/sql/repl_failsafe.cc 2009-09-23 13:10:23 +0000 +++ b/sql/repl_failsafe.cc 2009-11-20 15:18:01 +0000 @@ -559,7 +559,12 @@ HOSTS"; goto err; } si->server_id = log_server_id; - my_hash_insert(&slave_list, (uchar*)si); + if (my_hash_insert(&slave_list, (uchar*)si)) + { + error= "the slave is out of memory"; + pthread_mutex_unlock(&LOCK_slave_list); + goto err; + } } strmake(si->host, row[1], sizeof(si->host)-1); si->port = atoi(row[port_ind]); === modified file 'sql/rpl_record.cc' --- a/sql/rpl_record.cc 2009-03-05 19:54:53 +0000 +++ b/sql/rpl_record.cc 2009-10-22 00:15:45 +0000 @@ -180,7 +180,8 @@ int unpack_row(Relay_log_info const *rli, TABLE *table, uint const colcnt, uchar const *const row_data, MY_BITMAP const *cols, - uchar const **const row_end, ulong *const master_reclength) + uchar const **const row_end, ulong *const master_reclength, + const bool abort_on_warning, const bool first_row) { DBUG_ENTER("unpack_row"); DBUG_ASSERT(row_data); @@ -224,8 +225,35 @@ unpack_row(Relay_log_info const *rli, /* Field...::unpack() cannot return 0 */ DBUG_ASSERT(pack_ptr != NULL); - if ((null_bits & null_mask) && f->maybe_null()) - f->set_null(); + if (null_bits & null_mask) + { + if (f->maybe_null()) + { + DBUG_PRINT("debug", ("Was NULL; null mask: 0x%x; null bits: 0x%x", + null_mask, null_bits)); + f->set_null(); + } + else + { + MYSQL_ERROR::enum_warning_level error_type= + MYSQL_ERROR::WARN_LEVEL_NOTE; + if (abort_on_warning && (table->file->has_transactions() || + first_row)) + { + error = HA_ERR_ROWS_EVENT_APPLY; + error_type= MYSQL_ERROR::WARN_LEVEL_ERROR; + } + else + { + f->set_default(); + error_type= MYSQL_ERROR::WARN_LEVEL_WARN; + } + push_warning_printf(current_thd, error_type, + ER_BAD_NULL_ERROR, + ER(ER_BAD_NULL_ERROR), + f->field_name); + } + } else { f->set_notnull(); @@ -305,13 +333,17 @@ unpack_row(Relay_log_info const *rli, @param table Table whose record[0] buffer is prepared. @param skip Number of columns for which default/nullable check should be skipped. - @param check Indicates if errors should be raised when checking - default/nullable field properties. + @param check Specifies if lack of default error needs checking. + @param abort_on_warning + Controls how to react on lack of a field's default. + The parameter mimics the master side one for + @c check_that_all_fields_are_given_values. @returns 0 on success or a handler level error code */ int prepare_record(TABLE *const table, - const uint skip, const bool check) + const uint skip, const bool check, + const bool abort_on_warning, const bool first_row) { DBUG_ENTER("prepare_record"); @@ -326,17 +358,37 @@ int prepare_record(TABLE *const table, if (skip >= table->s->fields || !check) DBUG_RETURN(0); - /* Checking if exists default/nullable fields in the default values. */ - - for (Field **field_ptr= table->field+skip ; *field_ptr ; ++field_ptr) + /* + For fields the extra fields on the slave, we check if they have a default. + The check follows the same rules as the INSERT query without specifying an + explicit value for a field not having the explicit default + (@c check_that_all_fields_are_given_values()). + */ + for (Field **field_ptr= table->field+skip; *field_ptr; ++field_ptr) { uint32 const mask= NOT_NULL_FLAG | NO_DEFAULT_VALUE_FLAG; Field *const f= *field_ptr; - - if (((f->flags & mask) == mask)) + if ((f->flags & NO_DEFAULT_VALUE_FLAG) && + (f->real_type() != MYSQL_TYPE_ENUM)) { - my_error(ER_NO_DEFAULT_FOR_FIELD, MYF(0), f->field_name); - error = HA_ERR_ROWS_EVENT_APPLY; + + MYSQL_ERROR::enum_warning_level error_type= + MYSQL_ERROR::WARN_LEVEL_NOTE; + if (abort_on_warning && (table->file->has_transactions() || + first_row)) + { + error= HA_ERR_ROWS_EVENT_APPLY; + error_type= MYSQL_ERROR::WARN_LEVEL_ERROR; + } + else + { + f->set_default(); + error_type= MYSQL_ERROR::WARN_LEVEL_WARN; + } + push_warning_printf(current_thd, error_type, + ER_NO_DEFAULT_FOR_FIELD, + ER(ER_NO_DEFAULT_FOR_FIELD), + f->field_name); } } === modified file 'sql/rpl_record.h' --- a/sql/rpl_record.h 2008-01-31 12:54:03 +0000 +++ b/sql/rpl_record.h 2009-10-22 00:15:45 +0000 @@ -27,10 +27,13 @@ size_t pack_row(TABLE* table, MY_BITMAP int unpack_row(Relay_log_info const *rli, TABLE *table, uint const colcnt, uchar const *const row_data, MY_BITMAP const *cols, - uchar const **const row_end, ulong *const master_reclength); + uchar const **const row_end, ulong *const master_reclength, + const bool abort_on_warning= TRUE, const bool first_row= TRUE); // Fill table's record[0] with default values. -int prepare_record(TABLE *const, const uint =0, const bool =FALSE); +int prepare_record(TABLE *const table, const uint skip, const bool check, + const bool abort_on_warning= TRUE, + const bool first_row= TRUE); #endif #endif === modified file 'sql/rpl_rli.cc' --- a/sql/rpl_rli.cc 2009-12-03 11:19:05 +0000 +++ b/sql/rpl_rli.cc 2010-01-15 15:27:55 +0000 @@ -100,7 +100,8 @@ int init_relay_log_info(Relay_log_info* rli->tables_to_lock_count= 0; char pattern[FN_REFLEN]; - if (fn_format(pattern, PREFIX_SQL_LOAD, slave_load_tmpdir, "", + (void) my_realpath(pattern, slave_load_tmpdir, 0); + if (fn_format(pattern, PREFIX_SQL_LOAD, pattern, "", MY_SAFE_PATH | MY_RETURN_REAL_PATH) == NullS) { pthread_mutex_unlock(&rli->data_lock); @@ -127,6 +128,29 @@ int init_relay_log_info(Relay_log_info* rli->relay_log.max_size (and mysql_bin_log.max_size). */ { + /* Reports an error and returns, if the --relay-log's path + is a directory.*/ + if (opt_relay_logname && + opt_relay_logname[strlen(opt_relay_logname) - 1] == FN_LIBCHAR) + { + pthread_mutex_unlock(&rli->data_lock); + sql_print_error("Path '%s' is a directory name, please specify \ +a file name for --relay-log option", opt_relay_logname); + DBUG_RETURN(1); + } + + /* Reports an error and returns, if the --relay-log-index's path + is a directory.*/ + if (opt_relaylog_index_name && + opt_relaylog_index_name[strlen(opt_relaylog_index_name) - 1] + == FN_LIBCHAR) + { + pthread_mutex_unlock(&rli->data_lock); + sql_print_error("Path '%s' is a directory name, please specify \ +a file name for --relay-log-index option", opt_relaylog_index_name); + DBUG_RETURN(1); + } + char buf[FN_REFLEN]; const char *ln; static bool name_warning_sent= 0; === modified file 'sql/rpl_tblmap.cc' --- a/sql/rpl_tblmap.cc 2008-08-20 14:06:31 +0000 +++ b/sql/rpl_tblmap.cc 2009-11-20 15:18:01 +0000 @@ -119,7 +119,13 @@ int table_mapping::set_table(ulong table } e->table_id= table_id; e->table= table; - my_hash_insert(&m_table_ids,(uchar *)e); + if (my_hash_insert(&m_table_ids,(uchar *)e)) + { + /* we add this entry to the chain of free (free for use) entries */ + e->next= m_free; + m_free= e; + DBUG_RETURN(ERR_MEMORY_ALLOCATION); + } DBUG_PRINT("info", ("tid %lu -> table 0x%lx (%s)", table_id, (long) e->table, === modified file 'sql/set_var.cc' --- a/sql/set_var.cc 2009-12-03 11:19:05 +0000 +++ b/sql/set_var.cc 2010-01-14 16:32:41 +0000 @@ -58,6 +58,9 @@ #include <my_getopt.h> #include <thr_alarm.h> #include <myisam.h> +#ifdef WITH_MARIA_STORAGE_ENGINE +#include <maria.h> +#endif #include <my_dir.h> #include <waiting_threads.h> #include "events.h" === modified file 'sql/sp.cc' --- a/sql/sp.cc 2009-10-16 10:29:42 +0000 +++ b/sql/sp.cc 2009-11-21 11:18:21 +0000 @@ -70,6 +70,122 @@ enum MYSQL_PROC_FIELD_COUNT }; +static const +TABLE_FIELD_TYPE proc_table_fields[MYSQL_PROC_FIELD_COUNT] = +{ + { + { C_STRING_WITH_LEN("db") }, + { C_STRING_WITH_LEN("char(64)") }, + { C_STRING_WITH_LEN("utf8") } + }, + { + { C_STRING_WITH_LEN("name") }, + { C_STRING_WITH_LEN("char(64)") }, + { C_STRING_WITH_LEN("utf8") } + }, + { + { C_STRING_WITH_LEN("type") }, + { C_STRING_WITH_LEN("enum('FUNCTION','PROCEDURE')") }, + { NULL, 0 } + }, + { + { C_STRING_WITH_LEN("specific_name") }, + { C_STRING_WITH_LEN("char(64)") }, + { C_STRING_WITH_LEN("utf8") } + }, + { + { C_STRING_WITH_LEN("language") }, + { C_STRING_WITH_LEN("enum('SQL')") }, + { NULL, 0 } + }, + { + { C_STRING_WITH_LEN("sql_data_access") }, + { C_STRING_WITH_LEN("enum('CONTAINS_SQL','NO_SQL','READS_SQL_DATA','MODIFIES_SQL_DATA')") }, + { NULL, 0 } + }, + { + { C_STRING_WITH_LEN("is_deterministic") }, + { C_STRING_WITH_LEN("enum('YES','NO')") }, + { NULL, 0 } + }, + { + { C_STRING_WITH_LEN("security_type") }, + { C_STRING_WITH_LEN("enum('INVOKER','DEFINER')") }, + { NULL, 0 } + }, + { + { C_STRING_WITH_LEN("param_list") }, + { C_STRING_WITH_LEN("blob") }, + { NULL, 0 } + }, + + { + { C_STRING_WITH_LEN("returns") }, + { C_STRING_WITH_LEN("longblob") }, + { NULL, 0 } + }, + { + { C_STRING_WITH_LEN("body") }, + { C_STRING_WITH_LEN("longblob") }, + { NULL, 0 } + }, + { + { C_STRING_WITH_LEN("definer") }, + { C_STRING_WITH_LEN("char(77)") }, + { C_STRING_WITH_LEN("utf8") } + }, + { + { C_STRING_WITH_LEN("created") }, + { C_STRING_WITH_LEN("timestamp") }, + { NULL, 0 } + }, + { + { C_STRING_WITH_LEN("modified") }, + { C_STRING_WITH_LEN("timestamp") }, + { NULL, 0 } + }, + { + { C_STRING_WITH_LEN("sql_mode") }, + { C_STRING_WITH_LEN("set('REAL_AS_FLOAT','PIPES_AS_CONCAT','ANSI_QUOTES'," + "'IGNORE_SPACE','NOT_USED','ONLY_FULL_GROUP_BY','NO_UNSIGNED_SUBTRACTION'," + "'NO_DIR_IN_CREATE','POSTGRESQL','ORACLE','MSSQL','DB2','MAXDB'," + "'NO_KEY_OPTIONS','NO_TABLE_OPTIONS','NO_FIELD_OPTIONS','MYSQL323','MYSQL40'," + "'ANSI','NO_AUTO_VALUE_ON_ZERO','NO_BACKSLASH_ESCAPES','STRICT_TRANS_TABLES'," + "'STRICT_ALL_TABLES','NO_ZERO_IN_DATE','NO_ZERO_DATE','INVALID_DATES'," + "'ERROR_FOR_DIVISION_BY_ZERO','TRADITIONAL','NO_AUTO_CREATE_USER'," + "'HIGH_NOT_PRECEDENCE','NO_ENGINE_SUBSTITUTION','PAD_CHAR_TO_FULL_LENGTH')") }, + { NULL, 0 } + }, + { + { C_STRING_WITH_LEN("comment") }, + { C_STRING_WITH_LEN("char(64)") }, + { C_STRING_WITH_LEN("utf8") } + }, + { + { C_STRING_WITH_LEN("character_set_client") }, + { C_STRING_WITH_LEN("char(32)") }, + { C_STRING_WITH_LEN("utf8") } + }, + { + { C_STRING_WITH_LEN("collation_connection") }, + { C_STRING_WITH_LEN("char(32)") }, + { C_STRING_WITH_LEN("utf8") } + }, + { + { C_STRING_WITH_LEN("db_collation") }, + { C_STRING_WITH_LEN("char(32)") }, + { C_STRING_WITH_LEN("utf8") } + }, + { + { C_STRING_WITH_LEN("body_utf8") }, + { C_STRING_WITH_LEN("longblob") }, + { NULL, 0 } + } +}; + +static const TABLE_FIELD_DEF + proc_table_def= {MYSQL_PROC_FIELD_COUNT, proc_table_fields}; + /*************************************************************************/ /** @@ -247,6 +363,50 @@ Stored_routine_creation_ctx::load_from_d /*************************************************************************/ +class Proc_table_intact : public Table_check_intact +{ +private: + bool m_print_once; + +public: + Proc_table_intact() : m_print_once(TRUE) {} + +protected: + void report_error(uint code, const char *fmt, ...); +}; + + +/** + Report failure to validate the mysql.proc table definition. + Print a message to the error log only once. +*/ + +void Proc_table_intact::report_error(uint code, const char *fmt, ...) +{ + va_list args; + char buf[512]; + + va_start(args, fmt); + my_vsnprintf(buf, sizeof(buf), fmt, args); + va_end(args); + + if (code) + my_message(code, buf, MYF(0)); + else + my_error(ER_CANNOT_LOAD_FROM_TABLE, MYF(0), "proc"); + + if (m_print_once) + { + m_print_once= FALSE; + sql_print_error("%s", buf); + } +}; + + +/** Single instance used to control printing to the error log. */ +static Proc_table_intact proc_table_intact; + + /** Open the mysql.proc table for read. @@ -266,15 +426,17 @@ TABLE *open_proc_table_for_read(THD *thd DBUG_ENTER("open_proc_table_for_read"); TABLE_LIST table; - bzero((char*) &table, sizeof(table)); - table.db= (char*) "mysql"; - table.table_name= table.alias= (char*)"proc"; - table.lock_type= TL_READ; + table.init_one_table("mysql", "proc", TL_READ); + + if (open_system_tables_for_read(thd, &table, backup)) + DBUG_RETURN(NULL); - if (!open_system_tables_for_read(thd, &table, backup)) + if (!proc_table_intact.check(table.table, &proc_table_def)) DBUG_RETURN(table.table); - else - DBUG_RETURN(0); + + close_system_tables(thd, backup); + + DBUG_RETURN(NULL); } @@ -296,13 +458,19 @@ static TABLE *open_proc_table_for_update { DBUG_ENTER("open_proc_table_for_update"); - TABLE_LIST table; - bzero((char*) &table, sizeof(table)); - table.db= (char*) "mysql"; - table.table_name= table.alias= (char*)"proc"; - table.lock_type= TL_WRITE; + TABLE *table; + TABLE_LIST table_list; + table_list.init_one_table("mysql", "proc", TL_WRITE); + + if (!(table= open_system_table_for_update(thd, &table_list))) + DBUG_RETURN(NULL); + + if (!proc_table_intact.check(table, &proc_table_def)) + DBUG_RETURN(table); + + close_thread_tables(thd); - DBUG_RETURN(open_system_table_for_update(thd, &table)); + DBUG_RETURN(NULL); } @@ -1506,7 +1674,8 @@ static bool add_used_routine(LEX *lex, Q rn->key.length= key->length; rn->key.str= (char *)rn + sizeof(Sroutine_hash_entry); memcpy(rn->key.str, key->str, key->length + 1); - my_hash_insert(&lex->sroutines, (uchar *)rn); + if (my_hash_insert(&lex->sroutines, (uchar *)rn)) + return FALSE; lex->sroutines_list.link_in_list((uchar *)rn, (uchar **)&rn->next); rn->belong_to_view= belong_to_view; return TRUE; @@ -1584,16 +1753,24 @@ void sp_remove_not_own_routines(LEX *lex dependant on time of life of elements from source hash. It also won't touch lists linking elements in source and destination hashes. + + @returns + @return TRUE Failure + @return FALSE Success */ -void sp_update_sp_used_routines(HASH *dst, HASH *src) +bool sp_update_sp_used_routines(HASH *dst, HASH *src) { for (uint i=0 ; i < src->records ; i++) { Sroutine_hash_entry *rt= (Sroutine_hash_entry *)hash_element(src, i); if (!hash_search(dst, (uchar *)rt->key.str, rt->key.length)) - my_hash_insert(dst, (uchar *)rt); + { + if (my_hash_insert(dst, (uchar *)rt)) + return TRUE; + } } + return FALSE; } === modified file 'sql/sp.h' --- a/sql/sp.h 2009-07-28 17:44:38 +0000 +++ b/sql/sp.h 2009-11-20 15:18:01 +0000 @@ -69,7 +69,7 @@ void sp_get_prelocking_info(THD *thd, bo void sp_add_used_routine(LEX *lex, Query_arena *arena, sp_name *rt, char rt_type); void sp_remove_not_own_routines(LEX *lex); -void sp_update_sp_used_routines(HASH *dst, HASH *src); +bool sp_update_sp_used_routines(HASH *dst, HASH *src); int sp_cache_routines_and_add_tables(THD *thd, LEX *lex, bool first_no_prelock); int sp_cache_routines_and_add_tables_for_view(THD *thd, LEX *lex, === modified file 'sql/sp_cache.cc' --- a/sql/sp_cache.cc 2008-12-02 22:02:52 +0000 +++ b/sql/sp_cache.cc 2010-01-15 15:27:55 +0000 @@ -36,10 +36,16 @@ public: sp_cache(); ~sp_cache(); - inline void insert(sp_head *sp) + /** + Inserts a sp_head object into a hash table. + + @returns Success status + @return TRUE Failure + @return FALSE Success + */ + inline bool insert(sp_head *sp) { - /* TODO: why don't we check return value? */ - my_hash_insert(&m_hashtable, (const uchar *)sp); + return my_hash_insert(&m_hashtable, (const uchar *)sp); } inline sp_head *lookup(char *name, uint namelen) === modified file 'sql/sp_head.cc' --- a/sql/sp_head.cc 2009-12-03 11:34:11 +0000 +++ b/sql/sp_head.cc 2010-01-15 15:27:55 +0000 @@ -2090,8 +2090,18 @@ sp_head::reset_lex(THD *thd) DBUG_RETURN(FALSE); } -/// Restore lex during parsing, after we have parsed a sub statement. -void + +/** + Restore lex during parsing, after we have parsed a sub statement. + + @param thd Thread handle + + @return + @retval TRUE failure + @retval FALSE success +*/ + +bool sp_head::restore_lex(THD *thd) { DBUG_ENTER("sp_head::restore_lex"); @@ -2102,7 +2112,7 @@ sp_head::restore_lex(THD *thd) oldlex= (LEX *)m_lex.pop(); if (! oldlex) - return; // Nothing to restore + DBUG_RETURN(FALSE); // Nothing to restore oldlex->trg_table_fields.push_back(&sublex->trg_table_fields); @@ -2118,7 +2128,8 @@ sp_head::restore_lex(THD *thd) Add routines which are used by statement to respective set for this routine. */ - sp_update_sp_used_routines(&m_sroutines, &sublex->sroutines); + if (sp_update_sp_used_routines(&m_sroutines, &sublex->sroutines)) + DBUG_RETURN(TRUE); /* Merge tables used by this statement (but not by its functions or procedures) to multiset of tables used by this routine. @@ -2130,7 +2141,7 @@ sp_head::restore_lex(THD *thd) delete sublex; } thd->lex= oldlex; - DBUG_VOID_RETURN; + DBUG_RETURN(FALSE); } /** @@ -3867,7 +3878,8 @@ sp_head::merge_table_list(THD *thd, TABL tab->lock_type= table->lock_type; tab->lock_count= tab->query_lock_count= 1; tab->trg_event_map= table->trg_event_map; - my_hash_insert(&m_sptabs, (uchar *)tab); + if (my_hash_insert(&m_sptabs, (uchar *)tab)) + return FALSE; } } return TRUE; === modified file 'sql/sp_head.h' --- a/sql/sp_head.h 2009-04-29 02:59:10 +0000 +++ b/sql/sp_head.h 2009-11-20 15:18:01 +0000 @@ -340,7 +340,7 @@ public: @todo Conflicting comment in sp_head.cc */ - void + bool restore_lex(THD *thd); /// Put the instruction on the backpatch list, associated with the label. === modified file 'sql/sp_rcontext.cc' --- a/sql/sp_rcontext.cc 2008-01-23 22:36:57 +0000 +++ b/sql/sp_rcontext.cc 2009-11-06 19:34:25 +0000 @@ -617,7 +617,7 @@ sp_rcontext::set_case_expr(THD *thd, int } m_case_expr_holders[case_expr_id]->store(case_expr_item); - + m_case_expr_holders[case_expr_id]->cache_value(); return FALSE; } === modified file 'sql/sql_acl.cc' --- a/sql/sql_acl.cc 2009-12-03 11:19:05 +0000 +++ b/sql/sql_acl.cc 2010-01-15 15:27:55 +0000 @@ -31,9 +31,8 @@ #include "sp_head.h" #include "sp.h" -time_t mysql_db_table_last_check= 0L; - -TABLE_FIELD_W_TYPE mysql_db_table_fields[MYSQL_DB_FIELD_COUNT] = { +static const +TABLE_FIELD_TYPE mysql_db_table_fields[MYSQL_DB_FIELD_COUNT] = { { { C_STRING_WITH_LEN("Host") }, { C_STRING_WITH_LEN("char(60)") }, @@ -146,6 +145,8 @@ TABLE_FIELD_W_TYPE mysql_db_table_fields } }; +const TABLE_FIELD_DEF + mysql_db_table_def= {MYSQL_DB_FIELD_COUNT, mysql_db_table_fields}; #ifndef NO_EMBEDDED_ACCESS_CHECKS @@ -2405,7 +2406,12 @@ GRANT_TABLE::GRANT_TABLE(TABLE *form, TA privs = cols = 0; /* purecov: deadcode */ return; /* purecov: deadcode */ } - my_hash_insert(&hash_columns, (uchar *) mem_check); + if (my_hash_insert(&hash_columns, (uchar *) mem_check)) + { + /* Invalidate this entry */ + privs= cols= 0; + return; + } } while (!col_privs->file->index_next(col_privs->record[0]) && !key_cmp_if_same(col_privs,key,0,key_prefix_len)); col_privs->file->ha_index_end(); @@ -2439,14 +2445,17 @@ static GRANT_NAME *name_hash_search(HASH const char *host,const char* ip, const char *db, const char *user, const char *tname, - bool exact) + bool exact, bool name_tolower) { - char helping [NAME_LEN*2+USERNAME_LENGTH+3]; + char helping [NAME_LEN*2+USERNAME_LENGTH+3], *name_ptr; uint len; GRANT_NAME *grant_name,*found=0; HASH_SEARCH_STATE state; - len = (uint) (strmov(strmov(strmov(helping,user)+1,db)+1,tname)-helping)+ 1; + name_ptr= strmov(strmov(helping, user) + 1, db) + 1; + len = (uint) (strmov(name_ptr, tname) - helping) + 1; + if (name_tolower) + my_casedn_str(files_charset_info, name_ptr); for (grant_name= (GRANT_NAME*) hash_first(name_hash, (uchar*) helping, len, &state); grant_name ; @@ -2479,7 +2488,7 @@ routine_hash_search(const char *host, co { return (GRANT_TABLE*) name_hash_search(proc ? &proc_priv_hash : &func_priv_hash, - host, ip, db, user, tname, exact); + host, ip, db, user, tname, exact, TRUE); } @@ -2488,7 +2497,7 @@ table_hash_search(const char *host, cons const char *user, const char *tname, bool exact) { return (GRANT_TABLE*) name_hash_search(&column_priv_hash, host, ip, db, - user, tname, exact); + user, tname, exact, FALSE); } @@ -2610,7 +2619,11 @@ static int replace_column_table(GRANT_TA goto end; /* purecov: inspected */ } grant_column= new GRANT_COLUMN(column->column,privileges); - my_hash_insert(&g_t->hash_columns,(uchar*) grant_column); + if (my_hash_insert(&g_t->hash_columns,(uchar*) grant_column)) + { + result= -1; + goto end; + } } } @@ -3135,12 +3148,12 @@ int mysql_table_grant(THD *thd, TABLE_LI Str->user.str, table_name, rights, column_priv); - if (!grant_table) // end of memory + if (!grant_table || + my_hash_insert(&column_priv_hash,(uchar*) grant_table)) { result= TRUE; /* purecov: deadcode */ continue; /* purecov: deadcode */ } - my_hash_insert(&column_priv_hash,(uchar*) grant_table); } /* If revoke_grant, calculate the new column privilege for tables_priv */ @@ -3344,12 +3357,13 @@ bool mysql_routine_grant(THD *thd, TABLE grant_name= new GRANT_NAME(Str->host.str, db_name, Str->user.str, table_name, rights, TRUE); - if (!grant_name) + if (!grant_name || + my_hash_insert(is_proc ? + &proc_priv_hash : &func_priv_hash,(uchar*) grant_name)) { result= TRUE; continue; } - my_hash_insert(is_proc ? &proc_priv_hash : &func_priv_hash,(uchar*) grant_name); } if (replace_routine_table(thd, grant_name, tables[1].table, *Str, @@ -3452,6 +3466,13 @@ bool mysql_grant(THD *thd, const char *d result= TRUE; continue; } + /* + No User, but a password? + They did GRANT ... TO CURRENT_USER() IDENTIFIED BY ... ! + Get the current user, and shallow-copy the new password to them! + */ + if (!tmp_Str->user.str && tmp_Str->password.str) + Str->password= tmp_Str->password; if (replace_user_table(thd, tables[0].table, *Str, (!db ? rights : 0), revoke_grant, create_new_users, test(thd->variables.sql_mode & === modified file 'sql/sql_acl.h' --- a/sql/sql_acl.h 2009-05-29 13:37:54 +0000 +++ b/sql/sql_acl.h 2009-11-21 11:18:21 +0000 @@ -159,8 +159,7 @@ enum mysql_db_table_field MYSQL_DB_FIELD_COUNT }; -extern TABLE_FIELD_W_TYPE mysql_db_table_fields[]; -extern time_t mysql_db_table_last_check; +extern const TABLE_FIELD_DEF mysql_db_table_def; /* Classes */ === modified file 'sql/sql_base.cc' --- a/sql/sql_base.cc 2009-12-04 15:12:22 +0000 +++ b/sql/sql_base.cc 2010-01-15 15:27:55 +0000 @@ -2938,7 +2938,12 @@ TABLE *open_table(THD *thd, TABLE_LIST * DBUG_PRINT("info", ("inserting table '%s'.'%s' 0x%lx into the cache", table->s->db.str, table->s->table_name.str, (long) table)); - VOID(my_hash_insert(&open_cache,(uchar*) table)); + if (my_hash_insert(&open_cache,(uchar*) table)) + { + my_free(table, MYF(0)); + VOID(pthread_mutex_unlock(&LOCK_open)); + DBUG_RETURN(NULL); + } } check_unused(); // Debugging call === modified file 'sql/sql_cache.cc' --- a/sql/sql_cache.cc 2009-12-03 11:19:05 +0000 +++ b/sql/sql_cache.cc 2010-01-15 15:27:55 +0000 @@ -421,12 +421,16 @@ TYPELIB query_cache_type_typelib= effect by another thread. This enables a quick path in execution to skip waits when the outcome is known. + @param use_timeout TRUE if the lock can abort because of a timeout. + + @note use_timeout is optional and default value is FALSE. + @return @retval FALSE An exclusive lock was taken @retval TRUE The locking attempt failed */ -bool Query_cache::try_lock(void) +bool Query_cache::try_lock(bool use_timeout) { bool interrupt= FALSE; DBUG_ENTER("Query_cache::try_lock"); @@ -456,7 +460,26 @@ bool Query_cache::try_lock(void) else { DBUG_ASSERT(m_cache_lock_status == Query_cache::LOCKED); - pthread_cond_wait(&COND_cache_status_changed, &structure_guard_mutex); + /* + To prevent send_result_to_client() and query_cache_insert() from + blocking execution for too long a timeout is put on the lock. + */ + if (use_timeout) + { + struct timespec waittime; + set_timespec_nsec(waittime,(ulong)(50000000L)); /* Wait for 50 msec */ + int res= pthread_cond_timedwait(&COND_cache_status_changed, + &structure_guard_mutex,&waittime); + if (res == ETIMEDOUT) + { + interrupt= TRUE; + break; + } + } + else + { + pthread_cond_wait(&COND_cache_status_changed, &structure_guard_mutex); + } } } pthread_mutex_unlock(&structure_guard_mutex); @@ -1190,8 +1213,14 @@ def_week_frmt: %lu, in_trans: %d, autoco A table- or a full flush operation can potentially take a long time to finish. We choose not to wait for them and skip caching statements instead. + + In case the wait time can't be determined there is an upper limit which + causes try_lock() to abort with a time out. + + The 'TRUE' parameter indicate that the lock is allowed to timeout + */ - if (try_lock()) + if (try_lock(TRUE)) DBUG_VOID_RETURN; if (query_cache_size == 0) { @@ -1306,8 +1335,8 @@ end: to the user. RESULTS - 1 Query was not cached. - 0 The query was cached and user was sent the result. + 0 Query was not cached. + 1 The query was cached and user was sent the result. -1 The query was cached but we didn't have rights to use it. No error is sent to the client yet. @@ -1388,8 +1417,10 @@ Query_cache::send_result_to_client(THD * Try to obtain an exclusive lock on the query cache. If the cache is disabled or if a full cache flush is in progress, the attempt to get the lock is aborted. + + The 'TRUE' parameter indicate that the lock is allowed to timeout */ - if (try_lock()) + if (try_lock(TRUE)) goto err; if (query_cache_size == 0) === modified file 'sql/sql_cache.h' --- a/sql/sql_cache.h 2009-06-16 08:34:47 +0000 +++ b/sql/sql_cache.h 2009-11-20 12:49:06 +0000 @@ -485,7 +485,7 @@ protected: const char *name); my_bool in_blocks(Query_cache_block * point); - bool try_lock(void); + bool try_lock(bool use_timeout= FALSE); void lock(void); void lock_and_suspend(void); void unlock(void); === modified file 'sql/sql_class.cc' --- a/sql/sql_class.cc 2009-12-04 15:12:22 +0000 +++ b/sql/sql_class.cc 2010-01-15 15:27:55 +0000 @@ -379,6 +379,8 @@ char *thd_security_context(THD *thd, cha str.append(proc_info); } + pthread_mutex_lock(&thd->LOCK_thd_data); + if (thd->query()) { if (max_query_len < 1) @@ -388,6 +390,9 @@ char *thd_security_context(THD *thd, cha str.append('\n'); str.append(thd->query(), len); } + + pthread_mutex_unlock(&thd->LOCK_thd_data); + if (str.c_ptr_safe() == buffer) return buffer; === modified file 'sql/sql_delete.cc' --- a/sql/sql_delete.cc 2009-12-03 11:34:11 +0000 +++ b/sql/sql_delete.cc 2010-01-15 15:27:55 +0000 @@ -426,7 +426,8 @@ cleanup: } DBUG_ASSERT(transactional_table || !deleted || thd->transaction.stmt.modified_non_trans_table); free_underlaid_joins(thd, select_lex); - if (error < 0 || (thd->lex->ignore && !thd->is_fatal_error)) + if (error < 0 || + (thd->lex->ignore && !thd->is_error() && !thd->is_fatal_error)) { /* If a TRUNCATE TABLE was issued, the number of rows should be reported as @@ -1089,6 +1090,7 @@ bool mysql_truncate(THD *thd, TABLE_LIST TABLE *table; bool error; uint path_length; + bool is_temporary_table= false; DBUG_ENTER("mysql_truncate"); bzero((char*) &create_info,sizeof(create_info)); @@ -1101,6 +1103,8 @@ bool mysql_truncate(THD *thd, TABLE_LIST { TABLE_SHARE *share= table->s; handlerton *table_type= share->db_type(); + is_temporary_table= true; + if (!ha_check_storage_engine_flag(table_type, HTON_CAN_RECREATE)) goto trunc_by_del; @@ -1166,11 +1170,9 @@ end: { if (!error) { - /* - TRUNCATE must always be statement-based binlogged (not row-based) so - we don't test current_stmt_binlog_row_based. - */ - write_bin_log(thd, TRUE, thd->query(), thd->query_length()); + /* In RBR, the statement is not binlogged if the table is temporary. */ + if (!is_temporary_table || !thd->current_stmt_binlog_row_based) + write_bin_log(thd, TRUE, thd->query(), thd->query_length()); my_ok(thd); // This should return record count } VOID(pthread_mutex_lock(&LOCK_open)); === modified file 'sql/sql_insert.cc' --- a/sql/sql_insert.cc 2009-12-04 15:12:22 +0000 +++ b/sql/sql_insert.cc 2010-01-15 15:27:55 +0000 @@ -521,6 +521,22 @@ bool open_and_lock_for_insert_delayed(TH DBUG_ENTER("open_and_lock_for_insert_delayed"); #ifndef EMBEDDED_LIBRARY + if (thd->locked_tables && thd->global_read_lock) + { + /* + If this connection has the global read lock, the handler thread + will not be able to lock the table. It will wait for the global + read lock to go away, but this will never happen since the + connection thread will be stuck waiting for the handler thread + to open and lock the table. + If we are not in locked tables mode, INSERT will seek protection + against the global read lock (and fail), thus we will only get + to this point in locked tables mode. + */ + my_error(ER_CANT_UPDATE_WITH_READLOCK, MYF(0)); + DBUG_RETURN(TRUE); + } + if (delayed_get_table(thd, table_list)) DBUG_RETURN(TRUE); === modified file 'sql/sql_load.cc' --- a/sql/sql_load.cc 2009-12-03 11:19:05 +0000 +++ b/sql/sql_load.cc 2010-01-15 15:27:55 +0000 @@ -304,7 +304,8 @@ int mysql_load(THD *thd,sql_exchange *ex else { (void) fn_format(name, ex->file_name, mysql_real_data_home, "", - MY_RELATIVE_PATH | MY_UNPACK_FILENAME); + MY_RELATIVE_PATH | MY_UNPACK_FILENAME | + MY_RETURN_REAL_PATH); #if !defined(__WIN__) && ! defined(__NETWARE__) MY_STAT stat_info; if (!my_stat(name,&stat_info,MYF(MY_WME))) @@ -347,12 +348,16 @@ int mysql_load(THD *thd,sql_exchange *ex DBUG_ASSERT(FALSE); #endif } - else if (opt_secure_file_priv && - strncmp(opt_secure_file_priv, name, strlen(opt_secure_file_priv))) + else if (opt_secure_file_priv) { - /* Read only allowed from within dir specified by secure_file_priv */ - my_error(ER_OPTION_PREVENTS_STATEMENT, MYF(0), "--secure-file-priv"); - DBUG_RETURN(TRUE); + char secure_file_real_path[FN_REFLEN]; + (void) my_realpath(secure_file_real_path, opt_secure_file_priv, 0); + if (strncmp(secure_file_real_path, name, strlen(secure_file_real_path))) + { + /* Read only allowed from within dir specified by secure_file_priv */ + my_error(ER_OPTION_PREVENTS_STATEMENT, MYF(0), "--secure-file-priv"); + DBUG_RETURN(TRUE); + } } } === modified file 'sql/sql_parse.cc' --- a/sql/sql_parse.cc 2009-12-03 11:34:11 +0000 +++ b/sql/sql_parse.cc 2010-01-15 15:27:55 +0000 @@ -7632,6 +7632,9 @@ void get_default_definer(THD *thd, LEX_U definer->host.str= (char *) sctx->priv_host; definer->host.length= strlen(definer->host.str); + + definer->password.str= NULL; + definer->password.length= 0; } @@ -7683,6 +7686,8 @@ LEX_USER *create_definer(THD *thd, LEX_S definer->user= *user_name; definer->host= *host_name; + definer->password.str= NULL; + definer->password.length= 0; return definer; } === modified file 'sql/sql_partition.cc' --- a/sql/sql_partition.cc 2009-12-03 11:19:05 +0000 +++ b/sql/sql_partition.cc 2010-01-15 15:27:55 +0000 @@ -196,26 +196,27 @@ bool partition_default_handling(TABLE *t { DBUG_ENTER("partition_default_handling"); - if (part_info->use_default_no_partitions) + if (!is_create_table_ind) { - if (!is_create_table_ind && - table->file->get_no_parts(normalized_path, &part_info->no_parts)) + if (part_info->use_default_no_partitions) { - DBUG_RETURN(TRUE); + if (table->file->get_no_parts(normalized_path, &part_info->no_parts)) + { + DBUG_RETURN(TRUE); + } } - } - else if (part_info->is_sub_partitioned() && - part_info->use_default_no_subpartitions) - { - uint no_parts; - if (!is_create_table_ind && - (table->file->get_no_parts(normalized_path, &no_parts))) + else if (part_info->is_sub_partitioned() && + part_info->use_default_no_subpartitions) { - DBUG_RETURN(TRUE); + uint no_parts; + if (table->file->get_no_parts(normalized_path, &no_parts)) + { + DBUG_RETURN(TRUE); + } + DBUG_ASSERT(part_info->no_parts > 0); + DBUG_ASSERT((no_parts % part_info->no_parts) == 0); + part_info->no_subparts= no_parts / part_info->no_parts; } - DBUG_ASSERT(part_info->no_parts > 0); - part_info->no_subparts= no_parts / part_info->no_parts; - DBUG_ASSERT((no_parts % part_info->no_parts) == 0); } part_info->set_up_defaults_for_partitioning(table->file, (ulonglong)0, (uint)0); @@ -905,6 +906,8 @@ bool fix_fields_part_func(THD *thd, Item char* db_name; char db_name_string[FN_REFLEN]; bool save_use_only_table_context; + uint8 saved_full_group_by_flag; + nesting_map saved_allow_sum_func; DBUG_ENTER("fix_fields_part_func"); if (part_info->fixed) @@ -974,9 +977,19 @@ bool fix_fields_part_func(THD *thd, Item save_use_only_table_context= thd->lex->use_only_table_context; thd->lex->use_only_table_context= TRUE; thd->lex->current_select->cur_pos_in_select_list= UNDEF_POS; + saved_full_group_by_flag= thd->lex->current_select->full_group_by_flag; + saved_allow_sum_func= thd->lex->allow_sum_func; + thd->lex->allow_sum_func= 0; error= func_expr->fix_fields(thd, (Item**)&func_expr); + /* + Restore full_group_by_flag and allow_sum_func, + fix_fields should not affect mysql_select later, see Bug#46923. + */ + thd->lex->current_select->full_group_by_flag= saved_full_group_by_flag; + thd->lex->allow_sum_func= saved_allow_sum_func; + thd->lex->use_only_table_context= save_use_only_table_context; context->table_list= save_table_list; @@ -1679,7 +1692,7 @@ bool fix_partition_func(THD *thd, TABLE if (((part_info->part_type != HASH_PARTITION || part_info->list_of_part_fields == FALSE) && check_part_func_fields(part_info->part_field_array, TRUE)) || - (part_info->list_of_part_fields == FALSE && + (part_info->list_of_subpart_fields == FALSE && part_info->is_sub_partitioned() && check_part_func_fields(part_info->subpart_field_array, TRUE))) { === modified file 'sql/sql_select.cc' --- a/sql/sql_select.cc 2009-12-03 11:34:11 +0000 +++ b/sql/sql_select.cc 2010-01-15 15:27:55 +0000 @@ -992,14 +992,20 @@ JOIN::optimize() DBUG_RETURN(1); } - if (select_lex->olap == ROLLUP_TYPE && rollup_process_const_fields()) + if (rollup.state != ROLLUP::STATE_NONE) { - DBUG_PRINT("error", ("Error: rollup_process_fields() failed")); - DBUG_RETURN(1); + if (rollup_process_const_fields()) + { + DBUG_PRINT("error", ("Error: rollup_process_fields() failed")); + DBUG_RETURN(1); + } + } + else + { + /* Remove distinct if only const tables */ + select_distinct= select_distinct && (const_tables != tables); } - /* Remove distinct if only const tables */ - select_distinct= select_distinct && (const_tables != tables); thd_proc_info(thd, "preparing"); if (result->initialize_tables(this)) { @@ -1298,11 +1304,14 @@ JOIN::optimize() - We are using an ORDER BY or GROUP BY on fields not in the first table - We are using different ORDER BY and GROUP BY orders - The user wants us to buffer the result. + When the WITH ROLLUP modifier is present, we cannot skip temporary table + creation for the DISTINCT clause just because there are only const tables. */ - need_tmp= (const_tables != tables && + need_tmp= ((const_tables != tables && ((select_distinct || !simple_order || !simple_group) || (group_list && order) || - test(select_options & OPTION_BUFFER_RESULT))); + test(select_options & OPTION_BUFFER_RESULT))) || + (rollup.state != ROLLUP::STATE_NONE && select_distinct)); // No cache for MATCH make_join_readinfo(this, @@ -2144,17 +2153,13 @@ JOIN::exec() DBUG_VOID_RETURN; if (!curr_table->select->cond) curr_table->select->cond= sort_table_cond; - else // This should never happen + else { if (!(curr_table->select->cond= new Item_cond_and(curr_table->select->cond, sort_table_cond))) DBUG_VOID_RETURN; - /* - Item_cond_and do not need fix_fields for execution, its parameters - are fixed or do not need fix_fields, too - */ - curr_table->select->cond->quick_fix_field(); + curr_table->select->cond->fix_fields(thd, 0); } curr_table->select_cond= curr_table->select->cond; curr_table->select_cond->top_level_item(); @@ -6565,6 +6570,56 @@ void rr_unlock_row(st_join_table *tab) +/** + Pick the appropriate access method functions + + Sets the functions for the selected table access method + + @param tab Table reference to put access method +*/ + +static void +pick_table_access_method(JOIN_TAB *tab) +{ + switch (tab->type) + { + case JT_REF: + tab->read_first_record= join_read_always_key; + tab->read_record.read_record= join_read_next_same; + break; + + case JT_REF_OR_NULL: + tab->read_first_record= join_read_always_key_or_null; + tab->read_record.read_record= join_read_next_same_or_null; + break; + + case JT_CONST: + tab->read_first_record= join_read_const; + tab->read_record.read_record= join_no_more_records; + break; + + case JT_EQ_REF: + tab->read_first_record= join_read_key; + tab->read_record.read_record= join_no_more_records; + break; + + case JT_FT: + tab->read_first_record= join_ft_read_first; + tab->read_record.read_record= join_ft_read_next; + break; + + case JT_SYSTEM: + tab->read_first_record= join_read_system; + tab->read_record.read_record= join_no_more_records; + break; + + /* keep gcc happy */ + default: + break; + } +} + + static void make_join_readinfo(JOIN *join, ulonglong options) { @@ -6599,45 +6654,15 @@ make_join_readinfo(JOIN *join, ulonglong tab->sorted= sorted; sorted= 0; // only first must be sorted + table->status=STATUS_NO_RECORD; + pick_table_access_method (tab); + switch (tab->type) { - case JT_SYSTEM: // Only happens with left join - table->status=STATUS_NO_RECORD; - tab->read_first_record= join_read_system; - tab->read_record.read_record= join_no_more_records; - break; - case JT_CONST: // Only happens with left join - table->status=STATUS_NO_RECORD; - tab->read_first_record= join_read_const; - tab->read_record.read_record= join_no_more_records; - if (table->covering_keys.is_set(tab->ref.key) && - !table->no_keyread) - { - table->key_read=1; - table->file->extra(HA_EXTRA_KEYREAD); - } - break; case JT_EQ_REF: - table->status=STATUS_NO_RECORD; - if (tab->select) - { - delete tab->select->quick; - tab->select->quick=0; - } - delete tab->quick; - tab->quick=0; - tab->read_first_record= join_read_key; tab->read_record.unlock_row= join_read_key_unlock_row; - tab->read_record.read_record= join_no_more_records; - if (table->covering_keys.is_set(tab->ref.key) && - !table->no_keyread) - { - table->key_read=1; - table->file->extra(HA_EXTRA_KEYREAD); - } - break; + /* fall through */ case JT_REF_OR_NULL: case JT_REF: - table->status=STATUS_NO_RECORD; if (tab->select) { delete tab->select->quick; @@ -6645,34 +6670,20 @@ make_join_readinfo(JOIN *join, ulonglong } delete tab->quick; tab->quick=0; + /* fall through */ + case JT_CONST: // Only happens with left join if (table->covering_keys.is_set(tab->ref.key) && !table->no_keyread) { table->key_read=1; table->file->extra(HA_EXTRA_KEYREAD); } - if (tab->type == JT_REF) - { - tab->read_first_record= join_read_always_key; - tab->read_record.read_record= join_read_next_same; - } - else - { - tab->read_first_record= join_read_always_key_or_null; - tab->read_record.read_record= join_read_next_same_or_null; - } - break; - case JT_FT: - table->status=STATUS_NO_RECORD; - tab->read_first_record= join_ft_read_first; - tab->read_record.read_record= join_ft_read_next; break; case JT_ALL: /* If previous table use cache If the incoming data set is already sorted don't use cache. */ - table->status=STATUS_NO_RECORD; if (i != join->const_tables && !(options & SELECT_NO_JOIN_CACHE) && tab->use_quick != 2 && !tab->first_inner && !ordered_set) { @@ -6758,6 +6769,9 @@ make_join_readinfo(JOIN *join, ulonglong } } break; + case JT_FT: + case JT_SYSTEM: + break; default: DBUG_PRINT("error",("Table type %d found",tab->type)); /* purecov: deadcode */ break; /* purecov: deadcode */ @@ -7909,12 +7923,12 @@ static COND *build_equal_items_for_cond( { item_equal->fix_length_and_dec(); item_equal->update_used_tables(); + set_if_bigger(thd->lex->current_select->max_equal_elems, + item_equal->members()); + return item_equal; } - else - item_equal= (Item_equal *) eq_list.pop(); - set_if_bigger(thd->lex->current_select->max_equal_elems, - item_equal->members()); - return item_equal; + + return eq_list.pop(); } else { @@ -9552,47 +9566,8 @@ static Field *create_tmp_field_from_item new_field->set_derivation(item->collation.derivation); break; case DECIMAL_RESULT: - { - uint8 dec= item->decimals; - uint8 intg= ((Item_decimal *) item)->decimal_precision() - dec; - uint32 len= item->max_length; - - /* - Trying to put too many digits overall in a DECIMAL(prec,dec) - will always throw a warning. We must limit dec to - DECIMAL_MAX_SCALE however to prevent an assert() later. - */ - - if (dec > 0) - { - signed int overflow; - - dec= min(dec, DECIMAL_MAX_SCALE); - - /* - If the value still overflows the field with the corrected dec, - we'll throw out decimals rather than integers. This is still - bad and of course throws a truncation warning. - +1: for decimal point - */ - - const int required_length= - my_decimal_precision_to_length(intg + dec, dec, - item->unsigned_flag); - - overflow= required_length - len; - - if (overflow > 0) - dec= max(0, dec - overflow); // too long, discard fract - else - /* Corrected value fits. */ - len= required_length; - } - - new_field= new Field_new_decimal(len, maybe_null, item->name, - dec, item->unsigned_flag); + new_field= Field_new_decimal::create_from_item(item); break; - } case ROW_RESULT: default: // This case should never be choosen @@ -13496,6 +13471,8 @@ test_if_skip_sort_order(JOIN_TAB *tab,OR if (create_ref_for_key(tab->join, tab, keyuse, tab->join->const_table_map)) DBUG_RETURN(0); + + pick_table_access_method(tab); } else { @@ -14305,7 +14282,10 @@ static int remove_dup_with_hash_index(TH goto err; } else - (void) my_hash_insert(&hash, org_key_pos); + { + if (my_hash_insert(&hash, org_key_pos)) + goto err; + } key_pos+=extra_length; } my_free((char*) key_buffer,MYF(0)); === modified file 'sql/sql_show.cc' --- a/sql/sql_show.cc 2009-12-03 11:34:11 +0000 +++ b/sql/sql_show.cc 2010-01-15 15:27:55 +0000 @@ -721,7 +721,7 @@ mysqld_show_create(THD *thd, TABLE_LIST thd->push_internal_handler(&view_error_suppressor); bool error= open_normal_and_derived_tables(thd, table_list, 0); thd->pop_internal_handler(); - if (error && thd->main_da.is_error()) + if (error && (thd->killed || thd->main_da.is_error())) DBUG_RETURN(TRUE); } === modified file 'sql/sql_table.cc' --- a/sql/sql_table.cc 2009-12-03 11:19:05 +0000 +++ b/sql/sql_table.cc 2010-01-15 15:27:55 +0000 @@ -5428,12 +5428,20 @@ binlog: } VOID(pthread_mutex_unlock(&LOCK_open)); - IF_DBUG(int result=) - store_create_info(thd, table, &query, - create_info, FALSE /* show_database */); + /* + The condition avoids a crash as described in BUG#48506. Other + binlogging problems related to CREATE TABLE IF NOT EXISTS LIKE + when the existing object is a view will be solved by BUG 47442. + */ + if (!table->view) + { + IF_DBUG(int result=) + store_create_info(thd, table, &query, + create_info, FALSE /* show_database */); - DBUG_ASSERT(result == 0); // store_create_info() always return 0 - write_bin_log(thd, TRUE, query.ptr(), query.length()); + DBUG_ASSERT(result == 0); // store_create_info() always return 0 + write_bin_log(thd, TRUE, query.ptr(), query.length()); + } } else // Case 1 write_bin_log(thd, TRUE, thd->query(), thd->query_length()); === modified file 'sql/sql_yacc.yy' --- a/sql/sql_yacc.yy 2009-12-03 11:19:05 +0000 +++ b/sql/sql_yacc.yy 2010-01-17 17:22:46 +0000 @@ -389,6 +389,138 @@ void case_stmt_action_end_case(LEX *lex, lex->sphead->do_cont_backpatch(); } + +static bool +find_sys_var_null_base(THD *thd, struct sys_var_with_base *tmp) +{ + tmp->var= find_sys_var(thd, tmp->base_name.str, tmp->base_name.length); + + if (tmp->var == NULL) + my_error(ER_UNKNOWN_SYSTEM_VARIABLE, MYF(0), tmp->base_name.str); + else + tmp->base_name= null_lex_str; + + return thd->is_error(); +} + + +/** + Helper action for a SET statement. + Used to push a system variable into the assignment list. + + @param thd the current thread + @param tmp the system variable with base name + @param var_type the scope of the variable + @param val the value being assigned to the variable + + @return TRUE if error, FALSE otherwise. +*/ + +static bool +set_system_variable(THD *thd, struct sys_var_with_base *tmp, + enum enum_var_type var_type, Item *val) +{ + set_var *var; + LEX *lex= thd->lex; + + /* No AUTOCOMMIT from a stored function or trigger. */ + if (lex->spcont && tmp->var == &sys_autocommit) + lex->sphead->m_flags|= sp_head::HAS_SET_AUTOCOMMIT_STMT; + + if (! (var= new set_var(var_type, tmp->var, &tmp->base_name, val))) + return TRUE; + + return lex->var_list.push_back(var); +} + + +/** + Helper action for a SET statement. + Used to push a SP local variable into the assignment list. + + @param thd the current thread + @param var_type the SP local variable + @param val the value being assigned to the variable + + @return TRUE if error, FALSE otherwise. +*/ + +static bool +set_local_variable(THD *thd, sp_variable_t *spv, Item *val) +{ + Item *it; + LEX *lex= thd->lex; + sp_instr_set *sp_set; + + if (val) + it= val; + else if (spv->dflt) + it= spv->dflt; + else + { + it= new (thd->mem_root) Item_null(); + if (it == NULL) + return TRUE; + } + + sp_set= new sp_instr_set(lex->sphead->instructions(), lex->spcont, + spv->offset, it, spv->type, lex, TRUE); + + return (sp_set == NULL || lex->sphead->add_instr(sp_set)); +} + + +/** + Helper action for a SET statement. + Used to SET a field of NEW row. + + @param thd the current thread + @param name the field name + @param val the value being assigned to the row + + @return TRUE if error, FALSE otherwise. +*/ + +static bool +set_trigger_new_row(THD *thd, LEX_STRING *name, Item *val) +{ + LEX *lex= thd->lex; + Item_trigger_field *trg_fld; + sp_instr_set_trigger_field *sp_fld; + + /* QQ: Shouldn't this be field's default value ? */ + if (! val) + val= new Item_null(); + + DBUG_ASSERT(lex->trg_chistics.action_time == TRG_ACTION_BEFORE && + (lex->trg_chistics.event == TRG_EVENT_INSERT || + lex->trg_chistics.event == TRG_EVENT_UPDATE)); + + trg_fld= new (thd->mem_root) + Item_trigger_field(lex->current_context(), + Item_trigger_field::NEW_ROW, + name->str, UPDATE_ACL, FALSE); + + if (trg_fld == NULL) + return TRUE; + + sp_fld= new sp_instr_set_trigger_field(lex->sphead->instructions(), + lex->spcont, trg_fld, val, lex); + + if (sp_fld == NULL) + return TRUE; + + /* + Let us add this item to list of all Item_trigger_field + objects in trigger. + */ + lex->trg_table_fields.link_in_list((uchar *) trg_fld, + (uchar **) &trg_fld->next_trg_field); + + return lex->sphead->add_instr(sp_fld); +} + + /** Helper to resolve the SQL:2003 Syntax exception 1) in <in predicate>. See SQL:2003, Part 2, section 8.4 <in predicate>, Note 184, page 383. @@ -2335,8 +2467,8 @@ sp_decl: } pctx->declare_var_boundary(0); - lex->sphead->restore_lex(YYTHD); - + if (lex->sphead->restore_lex(YYTHD)) + MYSQL_YYABORT; $$.vars= $2; $$.conds= $$.hndlrs= $$.curs= 0; } @@ -2446,7 +2578,8 @@ sp_cursor_stmt: } lex->sp_lex_in_use= TRUE; $$= lex; - lex->sphead->restore_lex(YYTHD); + if (lex->sphead->restore_lex(YYTHD)) + MYSQL_YYABORT; } ; @@ -2665,7 +2798,8 @@ sp_proc_stmt_statement: sp->add_instr(i)) MYSQL_YYABORT; } - sp->restore_lex(thd); + if (sp->restore_lex(thd)) + MYSQL_YYABORT; } ; @@ -2693,7 +2827,8 @@ sp_proc_stmt_return: MYSQL_YYABORT; sp->m_flags|= sp_head::HAS_RETURN; } - sp->restore_lex(YYTHD); + if (sp->restore_lex(YYTHD)) + MYSQL_YYABORT; } ; @@ -2933,7 +3068,8 @@ sp_if: sp->add_cont_backpatch(i) || sp->add_instr(i)) MYSQL_YYABORT; - sp->restore_lex(YYTHD); + if (sp->restore_lex(YYTHD)) + MYSQL_YYABORT; } sp_proc_stmts1 { @@ -2979,7 +3115,9 @@ simple_case_stmt: if (case_stmt_action_expr(lex, $3)) MYSQL_YYABORT; - lex->sphead->restore_lex(YYTHD); /* For expr $3 */ + /* For expr $3 */ + if (lex->sphead->restore_lex(YYTHD)) + MYSQL_YYABORT; } simple_when_clause_list else_clause_opt @@ -3029,7 +3167,9 @@ simple_when_clause: LEX *lex= Lex; if (case_stmt_action_when(lex, $3, true)) MYSQL_YYABORT; - lex->sphead->restore_lex(YYTHD); /* For expr $3 */ + /* For expr $3 */ + if (lex->sphead->restore_lex(YYTHD)) + MYSQL_YYABORT; } THEN_SYM sp_proc_stmts1 @@ -3050,7 +3190,9 @@ searched_when_clause: LEX *lex= Lex; if (case_stmt_action_when(lex, $3, false)) MYSQL_YYABORT; - lex->sphead->restore_lex(YYTHD); /* For expr $3 */ + /* For expr $3 */ + if (lex->sphead->restore_lex(YYTHD)) + MYSQL_YYABORT; } THEN_SYM sp_proc_stmts1 @@ -3227,7 +3369,8 @@ sp_unlabeled_control: sp->new_cont_backpatch(i) || sp->add_instr(i)) MYSQL_YYABORT; - sp->restore_lex(YYTHD); + if (sp->restore_lex(YYTHD)) + MYSQL_YYABORT; } sp_proc_stmts1 END WHILE_SYM { @@ -3253,7 +3396,8 @@ sp_unlabeled_control: if (i == NULL || lex->sphead->add_instr(i)) MYSQL_YYABORT; - lex->sphead->restore_lex(YYTHD); + if (lex->sphead->restore_lex(YYTHD)) + MYSQL_YYABORT; /* We can shortcut the cont_backpatch here */ i->m_cont_dest= ip+1; } @@ -7539,6 +7683,14 @@ function_call_nonkeyword: } | SYSDATE optional_braces { + /* + Unlike other time-related functions, SYSDATE() is + replication-unsafe because it is not affected by the + TIMESTAMP variable. It is unsafe even if + sysdate_is_now=1, because the slave may have + sysdate_is_now=0. + */ + Lex->set_stmt_unsafe(); if (global_system_variables.sysdate_is_now == 0) $$= new (YYTHD->mem_root) Item_func_sysdate_local(); else @@ -9483,16 +9635,12 @@ replace: insert_lock_option: /* empty */ { -#ifdef HAVE_QUERY_CACHE /* - If it is SP we do not allow insert optimisation whan result of + If it is SP we do not allow insert optimisation when result of insert visible only after the table unlocking but everyone can read table. */ $$= (Lex->sphead ? TL_WRITE_DEFAULT : TL_WRITE_CONCURRENT_INSERT); -#else - $$= TL_WRITE_CONCURRENT_INSERT; -#endif } | LOW_PRIORITY { $$= TL_WRITE_LOW_PRIORITY; } | DELAYED_SYM { $$= TL_WRITE_DELAYED; } @@ -10515,15 +10663,11 @@ load_data_lock: /* empty */ { $$= TL_WRITE_DEFAULT; } | CONCURRENT { -#ifdef HAVE_QUERY_CACHE /* - Ignore this option in SP to avoid problem with query cache + Ignore this option in SP to avoid problem with query cache and + triggers with non default priority locks */ - if (Lex->sphead != 0) - $$= TL_WRITE_DEFAULT; - else -#endif - $$= TL_WRITE_CONCURRENT_INSERT; + $$= (Lex->sphead ? TL_WRITE_DEFAULT : TL_WRITE_CONCURRENT_INSERT); } | LOW_PRIORITY { $$= TL_WRITE_LOW_PRIORITY; } ; @@ -11791,7 +11935,8 @@ option_type_value: if (sp->add_instr(i)) MYSQL_YYABORT; } - lex->sphead->restore_lex(thd); + if (lex->sphead->restore_lex(thd)) + MYSQL_YYABORT; } } ; @@ -11831,98 +11976,42 @@ sys_option_value: option_type internal_variable_name equal set_expr_or_default { THD *thd= YYTHD; - LEX *lex=Lex; + LEX *lex= Lex; + LEX_STRING *name= &$2.base_name; if ($2.var == trg_new_row_fake_var) { /* We are in trigger and assigning value to field of new row */ - Item *it; - Item_trigger_field *trg_fld; - sp_instr_set_trigger_field *sp_fld; - LINT_INIT(sp_fld); if ($1) { my_parse_error(ER(ER_SYNTAX_ERROR)); MYSQL_YYABORT; } - if ($4) - it= $4; - else - { - /* QQ: Shouldn't this be field's default value ? */ - it= new Item_null(); - } - - DBUG_ASSERT(lex->trg_chistics.action_time == TRG_ACTION_BEFORE && - (lex->trg_chistics.event == TRG_EVENT_INSERT || - lex->trg_chistics.event == TRG_EVENT_UPDATE)); - - trg_fld= new (thd->mem_root) - Item_trigger_field(Lex->current_context(), - Item_trigger_field::NEW_ROW, - $2.base_name.str, - UPDATE_ACL, FALSE); - if (trg_fld == NULL) - MYSQL_YYABORT; - - sp_fld= new sp_instr_set_trigger_field(lex->sphead-> - instructions(), - lex->spcont, - trg_fld, - it, lex); - if (sp_fld == NULL) - MYSQL_YYABORT; - - /* - Let us add this item to list of all Item_trigger_field - objects in trigger. - */ - lex->trg_table_fields.link_in_list((uchar *)trg_fld, - (uchar **) &trg_fld-> - next_trg_field); - - if (lex->sphead->add_instr(sp_fld)) + if (set_trigger_new_row(YYTHD, name, $4)) MYSQL_YYABORT; } else if ($2.var) - { /* System variable */ + { if ($1) lex->option_type= $1; - set_var *var= new set_var(lex->option_type, $2.var, - &$2.base_name, $4); - if (var == NULL) + + /* It is a system variable. */ + if (set_system_variable(thd, &$2, lex->option_type, $4)) MYSQL_YYABORT; - lex->var_list.push_back(var); } else { - /* An SP local variable */ - sp_pcontext *ctx= lex->spcont; - sp_variable_t *spv; - sp_instr_set *sp_set; - Item *it; + sp_pcontext *spc= lex->spcont; + sp_variable_t *spv= spc->find_variable(name); + if ($1) { my_parse_error(ER(ER_SYNTAX_ERROR)); MYSQL_YYABORT; } - spv= ctx->find_variable(&$2.base_name); - - if ($4) - it= $4; - else if (spv->dflt) - it= spv->dflt; - else - { - it= new (thd->mem_root) Item_null(); - if (it == NULL) - MYSQL_YYABORT; - } - sp_set= new sp_instr_set(lex->sphead->instructions(), ctx, - spv->offset, it, spv->type, lex, TRUE); - if (sp_set == NULL || - lex->sphead->add_instr(sp_set)) + /* It is a local variable. */ + if (set_local_variable(thd, spv, $4)) MYSQL_YYABORT; } } @@ -11958,11 +12047,16 @@ option_value: } | '@' '@' opt_var_ident_type internal_variable_name equal set_expr_or_default { - LEX *lex=Lex; - set_var *var= new set_var($3, $4.var, &$4.base_name, $6); - if (var == NULL) + THD *thd= YYTHD; + struct sys_var_with_base tmp= $4; + /* Lookup if necessary: must be a system variable. */ + if (tmp.var == NULL) + { + if (find_sys_var_null_base(thd, &tmp)) + MYSQL_YYABORT; + } + if (set_system_variable(thd, &tmp, $3, $6)) MYSQL_YYABORT; - lex->var_list.push_back(var); } | charset old_or_new_charset_name_or_default { @@ -12055,31 +12149,26 @@ internal_variable_name: ident { THD *thd= YYTHD; - LEX *lex= thd->lex; - sp_pcontext *spc= lex->spcont; + sp_pcontext *spc= thd->lex->spcont; sp_variable_t *spv; - /* We have to lookup here since local vars can shadow sysvars */ + /* Best effort lookup for system variable. */ if (!spc || !(spv = spc->find_variable(&$1))) { + struct sys_var_with_base tmp= {NULL, $1}; + /* Not an SP local variable */ - sys_var *tmp=find_sys_var(thd, $1.str, $1.length); - if (!tmp) + if (find_sys_var_null_base(thd, &tmp)) MYSQL_YYABORT; - $$.var= tmp; - $$.base_name= null_lex_str; - if (spc && tmp == &sys_autocommit) - { - /* - We don't allow setting AUTOCOMMIT from a stored function - or trigger. - */ - lex->sphead->m_flags|= sp_head::HAS_SET_AUTOCOMMIT_STMT; - } + + $$= tmp; } else { - /* An SP local variable */ + /* + Possibly an SP local variable (or a shadowed sysvar). + Will depend on the context of the SET statement. + */ $$.var= NULL; $$.base_name= $1; } @@ -12237,12 +12326,7 @@ lock_option: | WRITE_SYM { $$= TL_WRITE_DEFAULT; } | WRITE_SYM CONCURRENT { -#ifdef HAVE_QUERY_CACHE - if (Lex->sphead != 0) - $$= TL_WRITE_DEFAULT; - else -#endif - $$= TL_WRITE_CONCURRENT_INSERT; + $$= (Lex->sphead ? TL_WRITE_DEFAULT : TL_WRITE_CONCURRENT_INSERT); } | LOW_PRIORITY WRITE_SYM { $$= TL_WRITE_LOW_PRIORITY; } === modified file 'sql/table.cc' --- a/sql/table.cc 2009-12-03 11:34:11 +0000 +++ b/sql/table.cc 2010-01-15 15:27:55 +0000 @@ -1316,8 +1316,16 @@ static int open_binary_frm(THD *thd, TAB share->timestamp_field_offset= i; if (use_hash) - (void) my_hash_insert(&share->name_hash, - (uchar*) field_ptr); // never fail + if (my_hash_insert(&share->name_hash, (uchar*) field_ptr) ) + { + /* + Set return code 8 here to indicate that an error has + occurred but that the error message already has been + sent (OOM). + */ + error= 8; + goto err; + } } *field_ptr=0; // End marker @@ -2804,34 +2812,38 @@ bool check_column_name(const char *name) and such errors never reach the user. */ -my_bool -table_check_intact(TABLE *table, const uint table_f_count, - const TABLE_FIELD_W_TYPE *table_def) +bool +Table_check_intact::check(TABLE *table, const TABLE_FIELD_DEF *table_def) { uint i; my_bool error= FALSE; - my_bool fields_diff_count; + const TABLE_FIELD_TYPE *field_def= table_def->field; DBUG_ENTER("table_check_intact"); DBUG_PRINT("info",("table: %s expected_count: %d", - table->alias, table_f_count)); + table->alias, table_def->count)); - fields_diff_count= (table->s->fields != table_f_count); - if (fields_diff_count) + /* Whether the table definition has already been validated. */ + if (table->s->table_field_def_cache == table_def) + DBUG_RETURN(FALSE); + + if (table->s->fields != table_def->count) { DBUG_PRINT("info", ("Column count has changed, checking the definition")); /* previous MySQL version */ if (MYSQL_VERSION_ID > table->s->mysql_version) { - sql_print_error(ER(ER_COL_COUNT_DOESNT_MATCH_PLEASE_UPDATE), - table->alias, table_f_count, table->s->fields, - table->s->mysql_version, MYSQL_VERSION_ID); + report_error(ER_COL_COUNT_DOESNT_MATCH_PLEASE_UPDATE, + ER(ER_COL_COUNT_DOESNT_MATCH_PLEASE_UPDATE), + table->alias, table_def->count, table->s->fields, + table->s->mysql_version, MYSQL_VERSION_ID); DBUG_RETURN(TRUE); } else if (MYSQL_VERSION_ID == table->s->mysql_version) { - sql_print_error(ER(ER_COL_COUNT_DOESNT_MATCH_CORRUPTED), table->alias, - table_f_count, table->s->fields); + report_error(ER_COL_COUNT_DOESNT_MATCH_CORRUPTED, + ER(ER_COL_COUNT_DOESNT_MATCH_CORRUPTED), table->alias, + table_def->count, table->s->fields); DBUG_RETURN(TRUE); } /* @@ -2843,7 +2855,7 @@ table_check_intact(TABLE *table, const u */ } char buffer[STRING_BUFFER_USUAL_SIZE]; - for (i=0 ; i < table_f_count; i++, table_def++) + for (i=0 ; i < table_def->count; i++, field_def++) { String sql_type(buffer, sizeof(buffer), system_charset_info); sql_type.length(0); @@ -2851,18 +2863,18 @@ table_check_intact(TABLE *table, const u { Field *field= table->field[i]; - if (strncmp(field->field_name, table_def->name.str, - table_def->name.length)) + if (strncmp(field->field_name, field_def->name.str, + field_def->name.length)) { /* Name changes are not fatal, we use ordinal numbers to access columns. Still this can be a sign of a tampered table, output an error to the error log. */ - sql_print_error("Incorrect definition of table %s.%s: " - "expected column '%s' at position %d, found '%s'.", - table->s->db.str, table->alias, table_def->name.str, i, - field->field_name); + report_error(0, "Incorrect definition of table %s.%s: " + "expected column '%s' at position %d, found '%s'.", + table->s->db.str, table->alias, field_def->name.str, i, + field->field_name); } field->sql_type(sql_type); /* @@ -2882,47 +2894,51 @@ table_check_intact(TABLE *table, const u the new table definition is backward compatible with the original one. */ - if (strncmp(sql_type.c_ptr_safe(), table_def->type.str, - table_def->type.length - 1)) + if (strncmp(sql_type.c_ptr_safe(), field_def->type.str, + field_def->type.length - 1)) { - sql_print_error("Incorrect definition of table %s.%s: " - "expected column '%s' at position %d to have type " - "%s, found type %s.", table->s->db.str, table->alias, - table_def->name.str, i, table_def->type.str, - sql_type.c_ptr_safe()); + report_error(0, "Incorrect definition of table %s.%s: " + "expected column '%s' at position %d to have type " + "%s, found type %s.", table->s->db.str, table->alias, + field_def->name.str, i, field_def->type.str, + sql_type.c_ptr_safe()); error= TRUE; } - else if (table_def->cset.str && !field->has_charset()) + else if (field_def->cset.str && !field->has_charset()) { - sql_print_error("Incorrect definition of table %s.%s: " - "expected the type of column '%s' at position %d " - "to have character set '%s' but the type has no " - "character set.", table->s->db.str, table->alias, - table_def->name.str, i, table_def->cset.str); + report_error(0, "Incorrect definition of table %s.%s: " + "expected the type of column '%s' at position %d " + "to have character set '%s' but the type has no " + "character set.", table->s->db.str, table->alias, + field_def->name.str, i, field_def->cset.str); error= TRUE; } - else if (table_def->cset.str && - strcmp(field->charset()->csname, table_def->cset.str)) + else if (field_def->cset.str && + strcmp(field->charset()->csname, field_def->cset.str)) { - sql_print_error("Incorrect definition of table %s.%s: " - "expected the type of column '%s' at position %d " - "to have character set '%s' but found " - "character set '%s'.", table->s->db.str, table->alias, - table_def->name.str, i, table_def->cset.str, - field->charset()->csname); + report_error(0, "Incorrect definition of table %s.%s: " + "expected the type of column '%s' at position %d " + "to have character set '%s' but found " + "character set '%s'.", table->s->db.str, table->alias, + field_def->name.str, i, field_def->cset.str, + field->charset()->csname); error= TRUE; } } else { - sql_print_error("Incorrect definition of table %s.%s: " - "expected column '%s' at position %d to have type %s " - " but the column is not found.", - table->s->db.str, table->alias, - table_def->name.str, i, table_def->type.str); + report_error(0, "Incorrect definition of table %s.%s: " + "expected column '%s' at position %d to have type %s " + " but the column is not found.", + table->s->db.str, table->alias, + field_def->name.str, i, field_def->type.str); error= TRUE; } } + + if (! error) + table->s->table_field_def_cache= table_def; + DBUG_RETURN(error); } === modified file 'sql/table.h' --- a/sql/table.h 2009-12-03 11:19:05 +0000 +++ b/sql/table.h 2010-01-15 15:27:55 +0000 @@ -285,6 +285,36 @@ typedef enum enum_table_category TABLE_C TABLE_CATEGORY get_table_category(const LEX_STRING *db, const LEX_STRING *name); + +typedef struct st_table_field_type +{ + LEX_STRING name; + LEX_STRING type; + LEX_STRING cset; +} TABLE_FIELD_TYPE; + + +typedef struct st_table_field_def +{ + uint count; + const TABLE_FIELD_TYPE *field; +} TABLE_FIELD_DEF; + + +class Table_check_intact +{ +protected: + virtual void report_error(uint code, const char *fmt, ...)= 0; + +public: + Table_check_intact() {} + virtual ~Table_check_intact() {} + + /** Checks whether a table is intact. */ + bool check(TABLE *table, const TABLE_FIELD_DEF *table_def); +}; + + /* This structure is shared between different table objects. There is one instance of table share per one table in the database. @@ -423,6 +453,18 @@ typedef struct st_table_share handlerton *default_part_db_type; #endif + /** + Cache the checked structure of this table. + + The pointer data is used to describe the structure that + a instance of the table must have. Each element of the + array specifies a field that must exist on the table. + + The pointer is cached in order to perform the check only + once -- when the table is loaded from the disk. + */ + const TABLE_FIELD_DEF *table_field_def_cache; + /** place to store storage engine specific data */ void *ha_data; @@ -1674,17 +1716,6 @@ typedef struct st_open_table_list{ uint32 in_use,locked; } OPEN_TABLE_LIST; -typedef struct st_table_field_w_type -{ - LEX_STRING name; - LEX_STRING type; - LEX_STRING cset; -} TABLE_FIELD_W_TYPE; - - -my_bool -table_check_intact(TABLE *table, const uint table_f_count, - const TABLE_FIELD_W_TYPE *table_def); static inline my_bitmap_map *tmp_use_all_columns(TABLE *table, MY_BITMAP *bitmap) === modified file 'storage/archive/CMakeLists.txt' (properties changed: -x to +x) --- a/storage/archive/CMakeLists.txt 2009-06-10 08:59:49 +0000 +++ b/storage/archive/CMakeLists.txt 2009-11-10 19:41:43 +0000 @@ -13,6 +13,9 @@ # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +SET(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} -DSAFEMALLOC -DSAFE_MUTEX") +SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} -DSAFEMALLOC -DSAFE_MUTEX") + INCLUDE("${PROJECT_SOURCE_DIR}/storage/mysql_storage_engine.cmake") SET(ARCHIVE_SOURCES azio.c ha_archive.cc ha_archive.h) MYSQL_STORAGE_ENGINE(ARCHIVE) === modified file 'storage/archive/azio.c' --- a/storage/archive/azio.c 2009-05-22 12:38:50 +0000 +++ b/storage/archive/azio.c 2010-01-15 15:27:55 +0000 @@ -71,7 +71,8 @@ int az_open (azio_stream *s, const char s->transparent = 0; s->mode = 'r'; s->version = (unsigned char)az_magic[1]; /* this needs to be a define to version */ - s->version = (unsigned char)az_magic[2]; /* minor version */ + s->minor_version= (unsigned char) az_magic[2]; /* minor version */ + s->dirty= AZ_STATE_CLEAN; /* We do our own version of append by nature. @@ -354,10 +355,19 @@ void read_header(azio_stream *s, unsigne s->comment_length= (unsigned int)uint4korr(buffer + AZ_COMMENT_LENGTH_POS); s->dirty= (unsigned int)buffer[AZ_DIRTY_POS]; } - else + else if (buffer[0] == gz_magic[0] && buffer[1] == gz_magic[1]) { - DBUG_ASSERT(buffer[0] == az_magic[0] && buffer[1] == az_magic[1]); - return; + /* + Set version number to previous version (2). + */ + s->version= (unsigned char) 2; + } else { + /* + Unknown version. + Most probably due to a corrupt archive. + */ + s->dirty= AZ_STATE_DIRTY; + s->z_err= Z_VERSION_ERROR; } } === modified file 'storage/archive/ha_archive.cc' --- a/storage/archive/ha_archive.cc 2009-12-03 11:19:05 +0000 +++ b/storage/archive/ha_archive.cc 2010-01-15 15:27:55 +0000 @@ -360,6 +360,12 @@ ARCHIVE_SHARE *ha_archive::get_share(con stats.auto_increment_value= archive_tmp.auto_increment + 1; share->rows_recorded= (ha_rows)archive_tmp.rows; share->crashed= archive_tmp.dirty; + /* + If archive version is less than 3, It should be upgraded before + use. + */ + if (archive_tmp.version < ARCHIVE_VERSION) + *rc= HA_ERR_TABLE_NEEDS_UPGRADE; azclose(&archive_tmp); VOID(my_hash_insert(&archive_open_tables, (uchar*) share)); @@ -491,7 +497,15 @@ int ha_archive::open(const char *name, i (open_options & HA_OPEN_FOR_REPAIR) ? "yes" : "no")); share= get_share(name, &rc); - if (rc == HA_ERR_CRASHED_ON_USAGE && !(open_options & HA_OPEN_FOR_REPAIR)) + /* + Allow open on crashed table in repair mode only. + Block open on 5.0 ARCHIVE table. Though we have almost all + routines to access these tables, they were not well tested. + For now we have to refuse to open such table to avoid + potential data loss. + */ + if ((rc == HA_ERR_CRASHED_ON_USAGE && !(open_options & HA_OPEN_FOR_REPAIR)) + || rc == HA_ERR_TABLE_NEEDS_UPGRADE) { /* purecov: begin inspected */ free_share(); === modified file 'storage/federated/CMakeLists.txt' (properties changed: -x to +x) --- a/storage/federated/CMakeLists.txt 2009-06-10 08:59:49 +0000 +++ b/storage/federated/CMakeLists.txt 2009-11-10 19:41:43 +0000 @@ -1,18 +1,21 @@ # Copyright (C) 2006 MySQL AB -# +# # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; version 2 of the License. -# +# # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. -# +# # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +SET(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} -DSAFEMALLOC -DSAFE_MUTEX") +SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} -DSAFEMALLOC -DSAFE_MUTEX") + INCLUDE("${PROJECT_SOURCE_DIR}/storage/mysql_storage_engine.cmake") SET(FEDERATED_SOURCES ha_federated.cc) MYSQL_STORAGE_ENGINE(FEDERATED) === modified file 'storage/innobase/btr/btr0btr.c' --- a/storage/innobase/btr/btr0btr.c 2007-07-10 14:34:21 +0000 +++ b/storage/innobase/btr/btr0btr.c 2009-11-30 08:50:08 +0000 @@ -709,8 +709,15 @@ btr_create( } else { /* It is a non-ibuf tree: create a file segment for leaf pages */ - fseg_create(space, page_no, PAGE_HEADER + PAGE_BTR_SEG_LEAF, - mtr); + if (!fseg_create(space, page_no, + PAGE_HEADER + PAGE_BTR_SEG_LEAF, mtr)) { + /* Not enough space for new segment, free root + segment before return. */ + btr_free_root(space, page_no, mtr); + + return(FIL_NULL); + } + /* The fseg create acquires a second latch on the page, therefore we must declare it: */ #ifdef UNIV_SYNC_DEBUG === modified file 'storage/innobase/data/data0type.c' --- a/storage/innobase/data/data0type.c 2007-07-10 11:37:43 +0000 +++ b/storage/innobase/data/data0type.c 2009-11-30 08:53:52 +0000 @@ -252,6 +252,22 @@ dtype_print( fputs("DATA_SYS", stderr); break; + case DATA_FLOAT: + fputs("DATA_FLOAT", stderr); + break; + + case DATA_DOUBLE: + fputs("DATA_DOUBLE", stderr); + break; + + case DATA_DECIMAL: + fputs("DATA_DECIMAL", stderr); + break; + + case DATA_VARMYSQL: + fputs("DATA_VARMYSQL", stderr); + break; + default: fprintf(stderr, "type %lu", (ulong) mtype); break; === modified file 'storage/innobase/handler/ha_innodb.cc' --- a/storage/innobase/handler/ha_innodb.cc 2009-12-03 11:19:05 +0000 +++ b/storage/innobase/handler/ha_innodb.cc 2010-01-15 15:27:55 +0000 @@ -662,6 +662,12 @@ convert_error_code_to_mysql( } else if (error == (int) DB_DUPLICATE_KEY) { + /* Be cautious with returning this error, since + mysql could re-enter the storage layer to get + duplicated key info, the operation requires a + valid table handle and/or transaction information, + which might not always be available in the error + handling stage. */ return(HA_ERR_FOUND_DUPP_KEY); } else if (error == (int) DB_FOREIGN_DUPLICATE_KEY) { @@ -765,35 +771,6 @@ convert_error_code_to_mysql( } /***************************************************************** -If you want to print a thd that is not associated with the current thread, -you must call this function before reserving the InnoDB kernel_mutex, to -protect MySQL from setting thd->query NULL. If you print a thd of the current -thread, we know that MySQL cannot modify thd->query, and it is not necessary -to call this. Call innobase_mysql_end_print_arbitrary_thd() after you release -the kernel_mutex. -NOTE that /mysql/innobase/lock/lock0lock.c must contain the prototype for this -function! */ -extern "C" -void -innobase_mysql_prepare_print_arbitrary_thd(void) -/*============================================*/ -{ - VOID(pthread_mutex_lock(&LOCK_thread_count)); -} - -/***************************************************************** -Releases the mutex reserved by innobase_mysql_prepare_print_arbitrary_thd(). -NOTE that /mysql/innobase/lock/lock0lock.c must contain the prototype for this -function! */ -extern "C" -void -innobase_mysql_end_print_arbitrary_thd(void) -/*========================================*/ -{ - VOID(pthread_mutex_unlock(&LOCK_thread_count)); -} - -/***************************************************************** Prints info of a THD object (== user session thread) to the given file. NOTE that /mysql/innobase/trx/trx0trx.c must contain the prototype for this function! */ @@ -1499,70 +1476,148 @@ innobase_invalidate_query_cache( #endif } -/********************************************************************* -Display an SQL identifier. */ -extern "C" -void -innobase_print_identifier( -/*======================*/ - FILE* f, /* in: output stream */ - trx_t* trx, /* in: transaction */ - ibool table_id,/* in: TRUE=print a table name, - FALSE=print other identifier */ - const char* name, /* in: name to print */ - ulint namelen)/* in: length of name */ -{ - const char* s = name; - char* qname = NULL; +/*****************************************************************//** +Convert an SQL identifier to the MySQL system_charset_info (UTF-8) +and quote it if needed. +@return pointer to the end of buf */ +static +char* +innobase_convert_identifier( +/*========================*/ + char* buf, /*!< out: buffer for converted identifier */ + ulint buflen, /*!< in: length of buf, in bytes */ + const char* id, /*!< in: identifier to convert */ + ulint idlen, /*!< in: length of id, in bytes */ + void* thd, /*!< in: MySQL connection thread, or NULL */ + ibool file_id)/*!< in: TRUE=id is a table or database name; + FALSE=id is an UTF-8 string */ +{ + char nz[NAME_LEN + 1]; +#if MYSQL_VERSION_ID >= 50141 + char nz2[NAME_LEN + 1 + EXPLAIN_FILENAME_MAX_EXTRA_LENGTH]; +#else /* MYSQL_VERSION_ID >= 50141 */ + char nz2[NAME_LEN + 1 + sizeof srv_mysql50_table_name_prefix]; +#endif /* MYSQL_VERSION_ID >= 50141 */ + + const char* s = id; int q; - if (table_id) { - /* Decode the table name. The filename_to_tablename() - function expects a NUL-terminated string. The input and - output strings buffers must not be shared. The function - only produces more output when the name contains other - characters than [0-9A-Z_a-z]. */ - char* temp_name = (char*) my_malloc((uint) namelen + 1, MYF(MY_WME)); - uint qnamelen = (uint) (namelen - + (1 + sizeof srv_mysql50_table_name_prefix)); - - if (temp_name) { - qname = (char*) my_malloc(qnamelen, MYF(MY_WME)); - if (qname) { - memcpy(temp_name, name, namelen); - temp_name[namelen] = 0; - s = qname; - namelen = filename_to_tablename(temp_name, - qname, qnamelen); - } - my_free(temp_name, MYF(0)); - } + if (file_id) { + /* Decode the table name. The MySQL function expects + a NUL-terminated string. The input and output strings + buffers must not be shared. */ + + if (UNIV_UNLIKELY(idlen > (sizeof nz) - 1)) { + idlen = (sizeof nz) - 1; + } + + memcpy(nz, id, idlen); + nz[idlen] = 0; + + s = nz2; +#if MYSQL_VERSION_ID >= 50141 + idlen = explain_filename((THD*) thd, nz, nz2, sizeof nz2, + EXPLAIN_PARTITIONS_AS_COMMENT); + goto no_quote; +#else /* MYSQL_VERSION_ID >= 50141 */ + idlen = filename_to_tablename(nz, nz2, sizeof nz2); +#endif /* MYSQL_VERSION_ID >= 50141 */ } - if (!trx || !trx->mysql_thd) { - + /* See if the identifier needs to be quoted. */ + if (UNIV_UNLIKELY(!thd)) { q = '"'; } else { - q = get_quote_char_for_identifier((THD*) trx->mysql_thd, - s, (int) namelen); + q = get_quote_char_for_identifier((THD*) thd, s, (int) idlen); } if (q == EOF) { - fwrite(s, 1, namelen, f); - } else { - const char* e = s + namelen; - putc(q, f); - while (s < e) { - int c = *s++; - if (c == q) { - putc(c, f); +#if MYSQL_VERSION_ID >= 50141 +no_quote: +#endif /* MYSQL_VERSION_ID >= 50141 */ + if (UNIV_UNLIKELY(idlen > buflen)) { + idlen = buflen; + } + memcpy(buf, s, idlen); + return(buf + idlen); + } + + /* Quote the identifier. */ + if (buflen < 2) { + return(buf); + } + + *buf++ = q; + buflen--; + + for (; idlen; idlen--) { + int c = *s++; + if (UNIV_UNLIKELY(c == q)) { + if (UNIV_UNLIKELY(buflen < 3)) { + break; + } + + *buf++ = c; + *buf++ = c; + buflen -= 2; + } else { + if (UNIV_UNLIKELY(buflen < 2)) { + break; } - putc(c, f); + + *buf++ = c; + buflen--; + } + } + + *buf++ = q; + return(buf); +} + +/*****************************************************************//** +Convert a table or index name to the MySQL system_charset_info (UTF-8) +and quote it if needed. +@return pointer to the end of buf */ +extern "C" +char* +innobase_convert_name( +/*==================*/ + char* buf, /*!< out: buffer for converted identifier */ + ulint buflen, /*!< in: length of buf, in bytes */ + const char* id, /*!< in: identifier to convert */ + ulint idlen, /*!< in: length of id, in bytes */ + void* thd, /*!< in: MySQL connection thread, or NULL */ + ibool table_id)/*!< in: TRUE=id is a table or database name; + FALSE=id is an index name */ +{ + char* s = buf; + const char* bufend = buf + buflen; + + if (table_id) { + const char* slash = (const char*) memchr(id, '/', idlen); + if (!slash) { + + goto no_db_name; } - putc(q, f); + + /* Print the database name and table name separately. */ + s = innobase_convert_identifier(s, bufend - s, id, slash - id, + thd, TRUE); + if (UNIV_LIKELY(s < bufend)) { + *s++ = '.'; + s = innobase_convert_identifier(s, bufend - s, + slash + 1, idlen + - (slash - id) - 1, + thd, TRUE); + } + } else { +no_db_name: + s = innobase_convert_identifier(buf, buflen, id, idlen, + thd, table_id); } - my_free(qname, MYF(MY_ALLOW_ZERO_PTR)); + return(s); + } /************************************************************************** @@ -3986,24 +4041,29 @@ no_commit: update the table upper limit. Note: last_value will be 0 if get_auto_increment() was not called.*/ - if (auto_inc <= col_max_value - && auto_inc >= prebuilt->autoinc_last_value) { + if (auto_inc >= prebuilt->autoinc_last_value) { set_max_autoinc: - ut_a(prebuilt->autoinc_increment > 0); - - ulonglong need; - ulonglong offset; - - offset = prebuilt->autoinc_offset; - need = prebuilt->autoinc_increment; - - auto_inc = innobase_next_autoinc( - auto_inc, need, offset, col_max_value); - - err = innobase_set_max_autoinc(auto_inc); - - if (err != DB_SUCCESS) { - error = err; + /* This should filter out the negative + values set explicitly by the user. */ + if (auto_inc <= col_max_value) { + ut_a(prebuilt->autoinc_increment > 0); + + ulonglong need; + ulonglong offset; + + offset = prebuilt->autoinc_offset; + need = prebuilt->autoinc_increment; + + auto_inc = innobase_next_autoinc( + auto_inc, + need, offset, col_max_value); + + err = innobase_set_max_autoinc( + auto_inc); + + if (err != DB_SUCCESS) { + error = err; + } } } break; @@ -5970,6 +6030,24 @@ ha_innobase::rename_table( innobase_commit_low(trx); trx_free_for_mysql(trx); + /* Add a special case to handle the Duplicated Key error + and return DB_ERROR instead. + This is to avoid a possible SIGSEGV error from mysql error + handling code. Currently, mysql handles the Duplicated Key + error by re-entering the storage layer and getting dup key + info by calling get_dup_key(). This operation requires a valid + table handle ('row_prebuilt_t' structure) which could no + longer be available in the error handling stage. The suggested + solution is to report a 'table exists' error message (since + the dup key error here is due to an existing table whose name + is the one we are trying to rename to) and return the generic + error code. */ + if (error == (int) DB_DUPLICATE_KEY) { + my_error(ER_TABLE_EXISTS_ERROR, MYF(0), to); + + error = DB_ERROR; + } + error = convert_error_code_to_mysql(error, NULL); DBUG_RETURN(error); @@ -8204,8 +8282,7 @@ innobase_xa_prepare( executing XA PREPARE and XA COMMIT commands. In this case we cannot know how many minutes or hours will be between XA PREPARE and XA COMMIT, and we don't want - to block for undefined period of time. - */ + to block for undefined period of time. */ pthread_mutex_lock(&prepare_commit_mutex); trx->active_trans = 2; } === modified file 'storage/innobase/include/ha_prototypes.h' --- a/storage/innobase/include/ha_prototypes.h 2008-12-14 19:28:19 +0000 +++ b/storage/innobase/include/ha_prototypes.h 2009-11-30 08:26:45 +0000 @@ -24,18 +24,21 @@ innobase_convert_string( CHARSET_INFO* from_cs, uint* errors); -/********************************************************************* -Display an SQL identifier. */ - -void -innobase_print_identifier( -/*======================*/ - FILE* f, /* in: output stream */ - trx_t* trx, /* in: transaction */ - ibool table_id,/* in: TRUE=print a table name, - FALSE=print other identifier */ - const char* name, /* in: name to print */ - ulint namelen);/* in: length of name */ +/*****************************************************************//** +Convert a table or index name to the MySQL system_charset_info (UTF-8) +and quote it if needed. +@return pointer to the end of buf */ + +char* +innobase_convert_name( +/*==================*/ + char* buf, /*!< out: buffer for converted identifier */ + ulint buflen, /*!< in: length of buf, in bytes */ + const char* id, /*!< in: identifier to convert */ + ulint idlen, /*!< in: length of id, in bytes */ + void* thd, /*!< in: MySQL connection thread, or NULL */ + ibool table_id);/*!< in: TRUE=id is a table or database name; + FALSE=id is an index name */ /********************************************************************** Returns true if the thread is the replication thread on the slave === modified file 'storage/innobase/include/mach0data.h' --- a/storage/innobase/include/mach0data.h 2008-08-20 00:37:41 +0000 +++ b/storage/innobase/include/mach0data.h 2009-11-30 09:41:38 +0000 @@ -266,8 +266,8 @@ UNIV_INLINE double mach_double_read( /*=============*/ - /* out: double read */ - byte* b); /* in: pointer to memory from where to read */ + /* out: double read */ + const byte* b); /* in: pointer to memory from where to read */ /************************************************************* Writes a double. It is stored in a little-endian format. */ UNIV_INLINE @@ -282,8 +282,8 @@ UNIV_INLINE float mach_float_read( /*============*/ - /* out: float read */ - byte* b); /* in: pointer to memory from where to read */ + /* out: float read */ + const byte* b); /* in: pointer to memory from where to read */ /************************************************************* Writes a float. It is stored in a little-endian format. */ UNIV_INLINE === modified file 'storage/innobase/include/mach0data.ic' --- a/storage/innobase/include/mach0data.ic 2008-08-20 00:37:41 +0000 +++ b/storage/innobase/include/mach0data.ic 2009-11-30 09:41:38 +0000 @@ -504,8 +504,8 @@ UNIV_INLINE double mach_double_read( /*=============*/ - /* out: double read */ - byte* b) /* in: pointer to memory from where to read */ + /* out: double read */ + const byte* b) /* in: pointer to memory from where to read */ { double d; ulint i; @@ -553,8 +553,8 @@ UNIV_INLINE float mach_float_read( /*============*/ - /* out: float read */ - byte* b) /* in: pointer to memory from where to read */ + /* out: float read */ + const byte* b) /* in: pointer to memory from where to read */ { float d; ulint i; === modified file 'storage/innobase/include/os0file.h' --- a/storage/innobase/include/os0file.h 2007-07-10 14:34:21 +0000 +++ b/storage/innobase/include/os0file.h 2009-11-30 08:40:31 +0000 @@ -96,6 +96,8 @@ log. */ to become available again */ #define OS_FILE_SHARING_VIOLATION 76 #define OS_FILE_ERROR_NOT_SPECIFIED 77 + /* 78 is used in the plugin */ +#define OS_FILE_OPERATION_ABORTED 79 /* Types for aio operations */ #define OS_FILE_READ 10 === modified file 'storage/innobase/include/trx0trx.h' --- a/storage/innobase/include/trx0trx.h 2009-07-10 23:12:13 +0000 +++ b/storage/innobase/include/trx0trx.h 2009-12-01 10:38:40 +0000 @@ -318,9 +318,7 @@ trx_commit_step( /************************************************************************** Prints info about a transaction to the given file. The caller must own the -kernel mutex and must have called -innobase_mysql_prepare_print_arbitrary_thd(), unless he knows that MySQL -or InnoDB cannot meanwhile change the info printed here. */ +kernel mutex. */ void trx_print( === modified file 'storage/innobase/lock/lock0lock.c' --- a/storage/innobase/lock/lock0lock.c 2009-07-10 23:12:13 +0000 +++ b/storage/innobase/lock/lock0lock.c 2009-12-01 10:38:40 +0000 @@ -22,31 +22,6 @@ Created 5/7/1996 Heikki Tuuri #include "trx0sys.h" -/* 2 function prototypes copied from ha_innodb.cc: */ - -/***************************************************************** -If you want to print a thd that is not associated with the current thread, -you must call this function before reserving the InnoDB kernel_mutex, to -protect MySQL from setting thd->query NULL. If you print a thd of the current -thread, we know that MySQL cannot modify thd->query, and it is not necessary -to call this. Call innobase_mysql_end_print_arbitrary_thd() after you release -the kernel_mutex. -NOTE that /mysql/innobase/lock/lock0lock.c must contain the prototype for this -function! */ - -void -innobase_mysql_prepare_print_arbitrary_thd(void); -/*============================================*/ - -/***************************************************************** -Relases the mutex reserved by innobase_mysql_prepare_print_arbitrary_thd(). -NOTE that /mysql/innobase/lock/lock0lock.c must contain the prototype for this -function! */ - -void -innobase_mysql_end_print_arbitrary_thd(void); -/*========================================*/ - /* Restricts the length of search we will do in the waits-for graph of transactions */ #define LOCK_MAX_N_STEPS_IN_DEADLOCK_CHECK 1000000 @@ -4222,11 +4197,6 @@ lock_print_info_summary( /*====================*/ FILE* file) /* in: file where to print */ { - /* We must protect the MySQL thd->query field with a MySQL mutex, and - because the MySQL mutex must be reserved before the kernel_mutex of - InnoDB, we call innobase_mysql_prepare_print_arbitrary_thd() here. */ - - innobase_mysql_prepare_print_arbitrary_thd(); lock_mutex_enter_kernel(); if (lock_deadlock_found) { @@ -4314,7 +4284,6 @@ loop: if (trx == NULL) { lock_mutex_exit_kernel(); - innobase_mysql_end_print_arbitrary_thd(); ut_ad(lock_validate()); @@ -4386,7 +4355,6 @@ loop: if (load_page_first) { lock_mutex_exit_kernel(); - innobase_mysql_end_print_arbitrary_thd(); mtr_start(&mtr); @@ -4397,7 +4365,6 @@ loop: load_page_first = FALSE; - innobase_mysql_prepare_print_arbitrary_thd(); lock_mutex_enter_kernel(); goto loop; === modified file 'storage/innobase/os/os0file.c' --- a/storage/innobase/os/os0file.c 2008-12-14 19:15:12 +0000 +++ b/storage/innobase/os/os0file.c 2009-11-30 08:40:31 +0000 @@ -257,6 +257,13 @@ os_file_get_last_error( " software or another instance\n" "InnoDB: of MySQL." " Please close it to get rid of this error.\n"); + } else if (err == ERROR_OPERATION_ABORTED) { + fprintf(stderr, + "InnoDB: The error means that the I/O" + " operation has been aborted\n" + "InnoDB: because of either a thread exit" + " or an application request.\n" + "InnoDB: Retry attempt is made.\n"); } else { fprintf(stderr, "InnoDB: Some operating system error numbers" @@ -278,6 +285,8 @@ os_file_get_last_error( } else if (err == ERROR_SHARING_VIOLATION || err == ERROR_LOCK_VIOLATION) { return(OS_FILE_SHARING_VIOLATION); + } else if (err == ERROR_OPERATION_ABORTED) { + return(OS_FILE_OPERATION_ABORTED); } else { return(100 + err); } @@ -402,6 +411,10 @@ os_file_handle_error_cond_exit( os_thread_sleep(10000000); /* 10 sec */ return(TRUE); + } else if (err == OS_FILE_OPERATION_ABORTED) { + + os_thread_sleep(100000); /* 100 ms */ + return(TRUE); } else { if (name) { fprintf(stderr, "InnoDB: File name %s\n", name); @@ -3692,6 +3705,7 @@ os_aio_windows_handle( ibool ret_val; BOOL ret; DWORD len; + BOOL retry = FALSE; if (segment == ULINT_UNDEFINED) { array = os_aio_sync_array; @@ -3745,14 +3759,52 @@ os_aio_windows_handle( ut_a(TRUE == os_file_flush(slot->file)); } # endif /* UNIV_DO_FLUSH */ + } else if (os_file_handle_error(slot->name, "Windows aio")) { + + retry = TRUE; } else { - os_file_handle_error(slot->name, "Windows aio"); ret_val = FALSE; } os_mutex_exit(array->mutex); + if (retry) { + /* retry failed read/write operation synchronously. + No need to hold array->mutex. */ + + switch (slot->type) { + case OS_FILE_WRITE: + ret = WriteFile(slot->file, slot->buf, + slot->len, &len, + &(slot->control)); + + break; + case OS_FILE_READ: + ret = ReadFile(slot->file, slot->buf, + slot->len, &len, + &(slot->control)); + + break; + default: + ut_error; + } + + if (!ret && GetLastError() == ERROR_IO_PENDING) { + /* aio was queued successfully! + We want a synchronous i/o operation on a + file where we also use async i/o: in Windows + we must use the same wait mechanism as for + async i/o */ + + ret = GetOverlappedResult(slot->file, + &(slot->control), + &len, TRUE); + } + + ret_val = ret && len == slot->len; + } + os_aio_array_free_slot(array, slot); return(ret_val); === modified file 'storage/innobase/row/row0sel.c' --- a/storage/innobase/row/row0sel.c 2009-07-10 23:12:13 +0000 +++ b/storage/innobase/row/row0sel.c 2009-11-30 09:41:38 +0000 @@ -4514,6 +4514,7 @@ row_search_autoinc_read_column( dict_index_t* index, /* in: index to read from */ const rec_t* rec, /* in: current rec */ ulint col_no, /* in: column number */ + ulint mtype, /*!< in: column main type */ ibool unsigned_type) /* in: signed or unsigned flag */ { ulint len; @@ -4535,9 +4536,26 @@ row_search_autoinc_read_column( data = rec_get_nth_field((rec_t*)rec, offsets, col_no, &len); ut_a(len != UNIV_SQL_NULL); - ut_a(len <= sizeof value); - value = mach_read_int_type(data, len, unsigned_type); + switch (mtype) { + case DATA_INT: + ut_a(len <= sizeof value); + value = mach_read_int_type(data, len, unsigned_type); + break; + + case DATA_FLOAT: + ut_a(len == sizeof(float)); + value = mach_float_read(data); + break; + + case DATA_DOUBLE: + ut_a(len == sizeof(double)); + value = mach_double_read(data); + break; + + default: + ut_error; + } if (UNIV_LIKELY_NULL(heap)) { mem_heap_free(heap); @@ -4625,7 +4643,8 @@ row_search_max_autoinc( dfield->col->prtype & DATA_UNSIGNED); *value = row_search_autoinc_read_column( - index, rec, i, unsigned_type); + index, rec, i, + dfield->col->mtype, unsigned_type); } } === modified file 'storage/innobase/trx/trx0trx.c' --- a/storage/innobase/trx/trx0trx.c 2009-07-10 23:12:13 +0000 +++ b/storage/innobase/trx/trx0trx.c 2009-12-01 10:38:40 +0000 @@ -1652,9 +1652,7 @@ trx_mark_sql_stat_end( /************************************************************************** Prints info about a transaction to the given file. The caller must own the -kernel mutex and must have called -innobase_mysql_prepare_print_arbitrary_thd(), unless he knows that MySQL -or InnoDB cannot meanwhile change the info printed here. */ +kernel mutex. */ void trx_print( === modified file 'storage/innobase/ut/ut0ut.c' --- a/storage/innobase/ut/ut0ut.c 2008-12-14 19:18:59 +0000 +++ b/storage/innobase/ut/ut0ut.c 2009-11-30 08:26:45 +0000 @@ -19,6 +19,7 @@ Created 5/11/1994 Heikki Tuuri #include "ut0sort.h" #include "trx0trx.h" #include "ha_prototypes.h" +#include "mysql_com.h" /* NAME_LEN */ ibool ut_always_false = FALSE; @@ -484,26 +485,17 @@ ut_print_namel( const char* name, /* in: name to print */ ulint namelen)/* in: length of name */ { -#ifdef UNIV_HOTBACKUP - fwrite(name, 1, namelen, f); -#else - if (table_id) { - char* slash = memchr(name, '/', namelen); - if (!slash) { - - goto no_db_name; - } + /* 2 * NAME_LEN for database and table name, + and some slack for the #mysql50# prefix and quotes */ + char buf[3 * NAME_LEN]; + const char* bufend; + + bufend = innobase_convert_name(buf, sizeof buf, + name, namelen, + trx ? trx->mysql_thd : NULL, + table_id); - /* Print the database name and table name separately. */ - innobase_print_identifier(f, trx, TRUE, name, slash - name); - putc('.', f); - innobase_print_identifier(f, trx, TRUE, slash + 1, - namelen - (slash - name) - 1); - } else { -no_db_name: - innobase_print_identifier(f, trx, table_id, name, namelen); - } -#endif + fwrite(buf, 1, bufend - buf, f); } /************************************************************************** === modified file 'storage/innodb_plugin/CMakeLists.txt' --- a/storage/innodb_plugin/CMakeLists.txt 2009-12-03 11:19:05 +0000 +++ b/storage/innodb_plugin/CMakeLists.txt 2010-01-15 15:27:55 +0000 @@ -83,4 +83,4 @@ SET(INNODB_PLUGIN_SOURCES btr/btr0btr.c ADD_DEFINITIONS(-DHAVE_WINDOWS_ATOMICS -DIB_HAVE_PAUSE_INSTRUCTION) #Disable storage engine, as we are using XtraDB -#MYSQL_STORAGE_ENGINE(INNODB_PLUGIN) +#MYSQL_STORAGE_ENGINE(INNOBASE) === modified file 'storage/innodb_plugin/ChangeLog' --- a/storage/innodb_plugin/ChangeLog 2009-11-03 10:34:03 +0000 +++ b/storage/innodb_plugin/ChangeLog 2009-11-30 13:42:26 +0000 @@ -1,3 +1,87 @@ +2009-11-20 The InnoDB Team + + * handler/ha_innodb.cc: + Add a workaround to prevent a crash due to Bug#45961 DDL on + partitioned innodb tables leaves data dictionary in an inconsistent + state + +2009-11-19 The InnoDB Team + + * btr/btr0btr.c: + Fix Bug#48469 when innodb tablespace is configured too small, crash + and corruption! + +2009-11-19 The InnoDB Team + + * data/data0type.c: + Fix Bug#48526 Data type for float and double is incorrectly reported + in InnoDB table monitor + +2009-11-19 The InnoDB Team + + * CMakeLists.txt: + Fix Bug#48317 cannot build innodb as static library + +2009-11-18 The InnoDB Team + + * handler/handler0alter.cc: + Fix Bug#48782 On lock wait timeout, CREATE INDEX (creating primary key) + attempts DROP TABLE + +2009-11-17 The InnoDB Team + + * handler/ha_innodb.cc, mysql-test/innodb.result, + mysql-test/innodb.test, mysql-test/innodb_bug44369.result, + mysql-test/innodb_bug44369.test, mysql-test/patches/innodb-index.diff, + row/row0mysql.c: + Report duplicate table names to the client connection, not to the + error log. + +2009-11-12 The InnoDB Team + + * handler/ha_innodb.cc, include/db0err.h, row/row0merge.c, + row/row0mysql.c: + Allow CREATE INDEX to be interrupted. + Also, when CHECK TABLE is interrupted, report ER_QUERY_INTERRUPTED. + +2009-11-11 The InnoDB Team + + * handler/ha_innodb.cc, mysql-test/innodb_bug47167.result, + mysql-test/innodb_bug47167.test, mysql-test/innodb_file_format.result: + Fix Bug#47167 "set global innodb_file_format_check" cannot set value + by User-Defined Variable + +2009-11-11 The InnoDB Team + + * include/os0file.h, os/os0file.c: + Fix Bug#3139 Mysql crashes: 'windows error 995' after several selects + on a large DB + +2009-11-04 The InnoDB Team + + * handler/ha_innodb.cc: + Fix Bug#32430 'show innodb status' causes errors + Invalid (old?) table or database name in logs + +2009-11-02 The InnoDB Team + + * btr/btr0sea.c, buf/buf0buf.c, dict/dict0dict.c, fil/fil0fil.c, + ibuf/ibuf0ibuf.c, include/btr0sea.h, include/dict0dict.h, + include/fil0fil.h, include/ibuf0ibuf.h, include/lock0lock.h, + include/log0log.h, include/log0recv.h, include/mem0mem.h, + include/mem0pool.h, include/os0file.h, include/pars0pars.h, + include/srv0srv.h, include/thr0loc.h, include/trx0i_s.h, + include/trx0purge.h, include/trx0rseg.h, include/trx0sys.h, + include/trx0undo.h, include/usr0sess.h, lock/lock0lock.c, + log/log0log.c, log/log0recv.c, mem/mem0dbg.c, mem/mem0pool.c, + os/os0file.c, os/os0sync.c, os/os0thread.c, pars/lexyy.c, + pars/pars0lex.l, que/que0que.c, srv/srv0srv.c, srv/srv0start.c, + sync/sync0arr.c, sync/sync0sync.c, thr/thr0loc.c, trx/trx0i_s.c, + trx/trx0purge.c, trx/trx0rseg.c, trx/trx0sys.c, trx/trx0undo.c, + usr/usr0sess.c, ut/ut0mem.c: + Fix Bug #45992 innodb memory not freed after shutdown + Fix Bug #46656 InnoDB plugin: memory leaks (Valgrind) + 2009-10-29 The InnoDB Team * handler/ha_innodb.cc, mysql-test/innodb-autoinc.result, @@ -66,6 +150,12 @@ Fix Bug#47058 Failure to compile innodb_plugin on solaris 10u7 + spro cc/CC 5.10 +2009-10-13 The InnoDB Team + + * buf/buf0flu.c: + Call fsync() on datafiles after a batch of pages is written to disk + even when skip_innodb_doublewrite is set. + 2009-10-05 The InnoDB Team * buf/buf0buf.c: === modified file 'storage/innodb_plugin/btr/btr0btr.c' --- a/storage/innodb_plugin/btr/btr0btr.c 2009-10-12 12:00:56 +0000 +++ b/storage/innodb_plugin/btr/btr0btr.c 2009-11-30 13:42:26 +0000 @@ -790,8 +790,15 @@ btr_create( } else { /* It is a non-ibuf tree: create a file segment for leaf pages */ - fseg_create(space, page_no, - PAGE_HEADER + PAGE_BTR_SEG_LEAF, mtr); + if (!fseg_create(space, page_no, + PAGE_HEADER + PAGE_BTR_SEG_LEAF, mtr)) { + /* Not enough space for new segment, free root + segment before return. */ + btr_free_root(space, zip_size, page_no, mtr); + + return(FIL_NULL); + } + /* The fseg create acquires a second latch on the page, therefore we must declare it: */ buf_block_dbg_add_level(block, SYNC_TREE_NODE_NEW); === modified file 'storage/innodb_plugin/btr/btr0sea.c' --- a/storage/innodb_plugin/btr/btr0sea.c 2009-10-08 11:28:37 +0000 +++ b/storage/innodb_plugin/btr/btr0sea.c 2009-11-30 11:32:05 +0000 @@ -175,6 +175,21 @@ btr_search_sys_create( btr_search_sys->hash_index = ha_create(hash_size, 0, 0); } +/*****************************************************************//** +Frees the adaptive search system at a database shutdown. */ +UNIV_INTERN +void +btr_search_sys_free(void) +/*=====================*/ +{ + mem_free(btr_search_latch_temp); + btr_search_latch_temp = NULL; + mem_heap_free(btr_search_sys->hash_index->heap); + hash_table_free(btr_search_sys->hash_index); + mem_free(btr_search_sys); + btr_search_sys = NULL; +} + /********************************************************************//** Disable the adaptive hash search system and empty the index. */ UNIV_INTERN === modified file 'storage/innodb_plugin/buf/buf0buf.c' --- a/storage/innodb_plugin/buf/buf0buf.c 2009-11-03 10:26:07 +0000 +++ b/storage/innodb_plugin/buf/buf0buf.c 2009-11-30 11:32:05 +0000 @@ -1020,7 +1020,11 @@ buf_pool_free(void) os_mem_free_large(chunk->mem, chunk->mem_size); } - buf_pool->n_chunks = 0; + mem_free(buf_pool->chunks); + hash_table_free(buf_pool->page_hash); + hash_table_free(buf_pool->zip_hash); + mem_free(buf_pool); + buf_pool = NULL; } /********************************************************************//** === modified file 'storage/innodb_plugin/data/data0type.c' --- a/storage/innodb_plugin/data/data0type.c 2009-07-30 12:42:56 +0000 +++ b/storage/innodb_plugin/data/data0type.c 2009-11-30 13:35:20 +0000 @@ -237,6 +237,22 @@ dtype_print( fputs("DATA_SYS", stderr); break; + case DATA_FLOAT: + fputs("DATA_FLOAT", stderr); + break; + + case DATA_DOUBLE: + fputs("DATA_DOUBLE", stderr); + break; + + case DATA_DECIMAL: + fputs("DATA_DECIMAL", stderr); + break; + + case DATA_VARMYSQL: + fputs("DATA_VARMYSQL", stderr); + break; + default: fprintf(stderr, "type %lu", (ulong) mtype); break; === modified file 'storage/innodb_plugin/dict/dict0dict.c' --- a/storage/innodb_plugin/dict/dict0dict.c 2009-10-09 12:52:18 +0000 +++ b/storage/innodb_plugin/dict/dict0dict.c 2009-11-30 11:42:51 +0000 @@ -1200,7 +1200,7 @@ dict_index_too_big_for_undo( = TRX_UNDO_PAGE_HDR - TRX_UNDO_PAGE_HDR_SIZE + 2 /* next record pointer */ + 1 /* type_cmpl */ - + 11 /* trx->undo_no */ - 11 /* table->id */ + + 11 /* trx->undo_no */ + 11 /* table->id */ + 1 /* rec_get_info_bits() */ + 11 /* DB_TRX_ID */ + 11 /* DB_ROLL_PTR */ @@ -4652,6 +4652,26 @@ dict_ind_init(void) dict_ind_redundant->cached = dict_ind_compact->cached = TRUE; } +/**********************************************************************//** +Frees dict_ind_redundant and dict_ind_compact. */ +static +void +dict_ind_free(void) +/*===============*/ +{ + dict_table_t* table; + + table = dict_ind_compact->table; + dict_mem_index_free(dict_ind_compact); + dict_ind_compact = NULL; + dict_mem_table_free(table); + + table = dict_ind_redundant->table; + dict_mem_index_free(dict_ind_redundant); + dict_ind_redundant = NULL; + dict_mem_table_free(table); +} + #ifndef UNIV_HOTBACKUP /**********************************************************************//** Get index by name @@ -4777,4 +4797,55 @@ dict_table_check_for_dup_indexes( } } #endif /* UNIV_DEBUG */ + +/************************************************************************** +Closes the data dictionary module. */ +UNIV_INTERN +void +dict_close(void) +/*============*/ +{ + ulint i; + + /* Free the hash elements. We don't remove them from the table + because we are going to destroy the table anyway. */ + for (i = 0; i < hash_get_n_cells(dict_sys->table_hash); i++) { + dict_table_t* table; + + table = HASH_GET_FIRST(dict_sys->table_hash, i); + + while (table) { + dict_table_t* prev_table = table; + + table = HASH_GET_NEXT(name_hash, prev_table); +#ifdef UNIV_DEBUG + ut_a(prev_table->magic_n == DICT_TABLE_MAGIC_N); +#endif + /* Acquire only because it's a pre-condition. */ + mutex_enter(&dict_sys->mutex); + + dict_table_remove_from_cache(prev_table); + + mutex_exit(&dict_sys->mutex); + } + } + + hash_table_free(dict_sys->table_hash); + + /* The elements are the same instance as in dict_sys->table_hash, + therefore we don't delete the individual elements. */ + hash_table_free(dict_sys->table_id_hash); + + dict_ind_free(); + + mutex_free(&dict_sys->mutex); + + rw_lock_free(&dict_operation_lock); + memset(&dict_operation_lock, 0x0, sizeof(dict_operation_lock)); + + mutex_free(&dict_foreign_err_mutex); + + mem_free(dict_sys); + dict_sys = NULL; +} #endif /* !UNIV_HOTBACKUP */ === modified file 'storage/innodb_plugin/fil/fil0fil.c' --- a/storage/innodb_plugin/fil/fil0fil.c 2009-11-03 10:24:21 +0000 +++ b/storage/innodb_plugin/fil/fil0fil.c 2009-11-30 11:32:05 +0000 @@ -321,6 +321,17 @@ fil_get_space_id_for_table( /*=======================*/ const char* name); /*!< in: table name in the standard 'databasename/tablename' format */ +/*******************************************************************//** +Frees a space object from the tablespace memory cache. Closes the files in +the chain but does not delete them. There must not be any pending i/o's or +flushes on the files. */ +static +ibool +fil_space_free( +/*===========*/ + /* out: TRUE if success */ + ulint id, /* in: space id */ + ibool own_mutex);/* in: TRUE if own system->mutex */ /********************************************************************//** Reads data from a space to a buffer. Remember that the possible incomplete blocks at the end of file are ignored: they are not taken into account when @@ -1144,7 +1155,7 @@ try_again: mutex_exit(&fil_system->mutex); - fil_space_free(namesake_id); + fil_space_free(namesake_id, FALSE); goto try_again; } @@ -1269,17 +1280,21 @@ Frees a space object from the tablespace the chain but does not delete them. There must not be any pending i/o's or flushes on the files. @return TRUE if success */ -UNIV_INTERN +static ibool fil_space_free( /*===========*/ - ulint id) /*!< in: space id */ + /* out: TRUE if success */ + ulint id, /* in: space id */ + ibool own_mutex) /* in: TRUE if own system->mutex */ { fil_space_t* space; fil_space_t* namespace; fil_node_t* fil_node; - mutex_enter(&fil_system->mutex); + if (!own_mutex) { + mutex_enter(&fil_system->mutex); + } space = fil_space_get_by_id(id); @@ -1326,7 +1341,9 @@ fil_space_free( ut_a(0 == UT_LIST_GET_LEN(space->chain)); - mutex_exit(&fil_system->mutex); + if (!own_mutex) { + mutex_exit(&fil_system->mutex); + } rw_lock_free(&(space->latch)); @@ -1586,6 +1603,8 @@ fil_close_all_files(void) space = UT_LIST_GET_FIRST(fil_system->space_list); while (space != NULL) { + fil_space_t* prev_space = space; + node = UT_LIST_GET_FIRST(space->chain); while (node != NULL) { @@ -1595,6 +1614,7 @@ fil_close_all_files(void) node = UT_LIST_GET_NEXT(chain, node); } space = UT_LIST_GET_NEXT(space_list, space); + fil_space_free(prev_space->id, TRUE); } mutex_exit(&fil_system->mutex); @@ -2226,7 +2246,7 @@ try_again: #endif /* printf("Deleting tablespace %s id %lu\n", space->name, id); */ - success = fil_space_free(id); + success = fil_space_free(id, FALSE); if (success) { success = os_file_delete(path); @@ -4753,3 +4773,26 @@ fil_page_get_type( return(mach_read_from_2(page + FIL_PAGE_TYPE)); } + +/******************************************************************** +Initializes the tablespace memory cache. */ +UNIV_INTERN +void +fil_close(void) +/*===========*/ +{ + /* The mutex should already have been freed. */ + ut_ad(fil_system->mutex.magic_n == 0); + + hash_table_free(fil_system->spaces); + + hash_table_free(fil_system->name_hash); + + ut_a(UT_LIST_GET_LEN(fil_system->LRU) == 0); + ut_a(UT_LIST_GET_LEN(fil_system->unflushed_spaces) == 0); + ut_a(UT_LIST_GET_LEN(fil_system->space_list) == 0); + + mem_free(fil_system); + + fil_system = NULL; +} === modified file 'storage/innodb_plugin/handler/ha_innodb.cc' --- a/storage/innodb_plugin/handler/ha_innodb.cc 2009-11-03 10:34:38 +0000 +++ b/storage/innodb_plugin/handler/ha_innodb.cc 2009-12-08 09:26:11 +0000 @@ -269,10 +269,10 @@ innobase_file_format_check_on_off( /************************************************************//** Validate the file format check config parameters, as a side effect it sets the srv_check_file_format_at_startup variable. -@return true if valid config value */ +@return the format_id if valid config value, otherwise, return -1 */ static -bool -innobase_file_format_check_validate( +int +innobase_file_format_validate_and_set( /*================================*/ const char* format_check); /*!< in: parameter value */ /****************************************************************//** @@ -785,11 +785,20 @@ convert_error_code_to_mysql( case DB_SUCCESS: return(0); + case DB_INTERRUPTED: + my_error(ER_QUERY_INTERRUPTED, MYF(0)); + /* fall through */ case DB_ERROR: default: return(-1); /* unspecified error */ case DB_DUPLICATE_KEY: + /* Be cautious with returning this error, since + mysql could re-enter the storage layer to get + duplicated key info, the operation requires a + valid table handle and/or transaction information, + which might not always be available in the error + handling stage. */ return(HA_ERR_FOUND_DUPP_KEY); case DB_FOREIGN_DUPLICATE_KEY: @@ -890,36 +899,6 @@ convert_error_code_to_mysql( } /*************************************************************//** -If you want to print a thd that is not associated with the current thread, -you must call this function before reserving the InnoDB kernel_mutex, to -protect MySQL from setting thd->query NULL. If you print a thd of the current -thread, we know that MySQL cannot modify thd->query, and it is not necessary -to call this. Call innobase_mysql_end_print_arbitrary_thd() after you release -the kernel_mutex. */ -extern "C" UNIV_INTERN -void -innobase_mysql_prepare_print_arbitrary_thd(void) -/*============================================*/ -{ - ut_ad(!mutex_own(&kernel_mutex)); - VOID(pthread_mutex_lock(&LOCK_thread_count)); -} - -/*************************************************************//** -Releases the mutex reserved by innobase_mysql_prepare_print_arbitrary_thd(). -In the InnoDB latching order, the mutex sits right above the -kernel_mutex. In debug builds, we assert that the kernel_mutex is -released before this function is invoked. */ -extern "C" UNIV_INTERN -void -innobase_mysql_end_print_arbitrary_thd(void) -/*========================================*/ -{ - ut_ad(!mutex_own(&kernel_mutex)); - VOID(pthread_mutex_unlock(&LOCK_thread_count)); -} - -/*************************************************************//** Prints info of a THD object (== user session thread) to the given file. */ extern "C" UNIV_INTERN void @@ -1707,15 +1686,19 @@ innobase_convert_identifier( FALSE=id is an UTF-8 string */ { char nz[NAME_LEN + 1]; +#if MYSQL_VERSION_ID >= 50141 + char nz2[NAME_LEN + 1 + EXPLAIN_FILENAME_MAX_EXTRA_LENGTH]; +#else /* MYSQL_VERSION_ID >= 50141 */ char nz2[NAME_LEN + 1 + sizeof srv_mysql50_table_name_prefix]; +#endif /* MYSQL_VERSION_ID >= 50141 */ const char* s = id; int q; if (file_id) { - /* Decode the table name. The filename_to_tablename() - function expects a NUL-terminated string. The input and - output strings buffers must not be shared. */ + /* Decode the table name. The MySQL function expects + a NUL-terminated string. The input and output strings + buffers must not be shared. */ if (UNIV_UNLIKELY(idlen > (sizeof nz) - 1)) { idlen = (sizeof nz) - 1; @@ -1725,7 +1708,13 @@ innobase_convert_identifier( nz[idlen] = 0; s = nz2; +#if MYSQL_VERSION_ID >= 50141 + idlen = explain_filename((THD*) thd, nz, nz2, sizeof nz2, + EXPLAIN_PARTITIONS_AS_COMMENT); + goto no_quote; +#else /* MYSQL_VERSION_ID >= 50141 */ idlen = filename_to_tablename(nz, nz2, sizeof nz2); +#endif /* MYSQL_VERSION_ID >= 50141 */ } /* See if the identifier needs to be quoted. */ @@ -1736,6 +1725,9 @@ innobase_convert_identifier( } if (q == EOF) { +#if MYSQL_VERSION_ID >= 50141 +no_quote: +#endif /* MYSQL_VERSION_ID >= 50141 */ if (UNIV_UNLIKELY(idlen > buflen)) { idlen = buflen; } @@ -2133,8 +2125,8 @@ mem_free_and_error: /* Did the user specify a format name that we support ? As a side effect it will update the variable srv_check_file_format_at_startup */ - if (!innobase_file_format_check_validate( - innobase_file_format_check)) { + if (innobase_file_format_validate_and_set( + innobase_file_format_check) < 0) { sql_print_error("InnoDB: invalid " "innodb_file_format_check value: " @@ -5225,8 +5217,10 @@ ha_innobase::change_active_index( prebuilt->index); if (UNIV_UNLIKELY(!prebuilt->index_usable)) { - sql_print_warning("InnoDB: insufficient history for index %u", - keynr); + push_warning_printf(user_thd, MYSQL_ERROR::WARN_LEVEL_WARN, + HA_ERR_TABLE_DEF_CHANGED, + "InnoDB: insufficient history for index %u", + keynr); /* The caller seems to ignore this. Thus, we must check this again in row_search_for_mysql(). */ DBUG_RETURN(2); @@ -5713,17 +5707,8 @@ create_table_def( /* First check whether the column to be added has a system reserved name. */ if (dict_col_name_is_reserved(field->field_name)){ - push_warning_printf( - (THD*) trx->mysql_thd, - MYSQL_ERROR::WARN_LEVEL_WARN, - ER_CANT_CREATE_TABLE, - "Error creating table '%s' with " - "column name '%s'. '%s' is a " - "reserved name. Please try to " - "re-create the table with a " - "different column name.", - table->name, (char*) field->field_name, - (char*) field->field_name); + my_error(ER_WRONG_COLUMN_NAME, MYF(0), + field->field_name); dict_mem_table_free(table); trx_commit_for_mysql(trx); @@ -5745,6 +5730,14 @@ create_table_def( error = row_create_table_for_mysql(table, trx); + if (error == DB_DUPLICATE_KEY) { + char buf[100]; + innobase_convert_identifier(buf, sizeof buf, + table_name, strlen(table_name), + trx->mysql_thd, TRUE); + my_error(ER_TABLE_EXISTS_ERROR, MYF(0), buf); + } + error_ret: error = convert_error_code_to_mysql(error, flags, NULL); @@ -6798,6 +6791,24 @@ ha_innobase::rename_table( innobase_commit_low(trx); trx_free_for_mysql(trx); + /* Add a special case to handle the Duplicated Key error + and return DB_ERROR instead. + This is to avoid a possible SIGSEGV error from mysql error + handling code. Currently, mysql handles the Duplicated Key + error by re-entering the storage layer and getting dup key + info by calling get_dup_key(). This operation requires a valid + table handle ('row_prebuilt_t' structure) which could no + longer be available in the error handling stage. The suggested + solution is to report a 'table exists' error message (since + the dup key error here is due to an existing table whose name + is the one we are trying to rename to) and return the generic + error code. */ + if (error == (int) DB_DUPLICATE_KEY) { + my_error(ER_TABLE_EXISTS_ERROR, MYF(0), to); + + error = DB_ERROR; + } + error = convert_error_code_to_mysql(error, 0, NULL); DBUG_RETURN(error); @@ -7348,11 +7359,15 @@ ha_innobase::check( ret = row_check_table_for_mysql(prebuilt); - if (ret == DB_SUCCESS) { + switch (ret) { + case DB_SUCCESS: return(HA_ADMIN_OK); + case DB_INTERRUPTED: + my_error(ER_QUERY_INTERRUPTED, MYF(0)); + return(-1); + default: + return(HA_ADMIN_CORRUPT); } - - return(HA_ADMIN_CORRUPT); } /*************************************************************//** @@ -7899,7 +7914,10 @@ ha_innobase::external_lock( ulong const tx_isolation = thd_tx_isolation(ha_thd()); if (tx_isolation <= ISO_READ_COMMITTED && binlog_format == BINLOG_FORMAT_STMT - && thd_binlog_filter_ok(thd)) +#if MYSQL_VERSION_ID > 50140 + && thd_binlog_filter_ok(thd) +#endif /* MYSQL_VERSION_ID > 50140 */ + ) { char buf[256]; my_snprintf(buf, sizeof(buf), @@ -9148,8 +9166,7 @@ innobase_xa_prepare( executing XA PREPARE and XA COMMIT commands. In this case we cannot know how many minutes or hours will be between XA PREPARE and XA COMMIT, and we don't want - to block for undefined period of time. - */ + to block for undefined period of time. */ pthread_mutex_lock(&prepare_commit_mutex); trx->active_trans = 2; } @@ -9491,25 +9508,24 @@ innobase_file_format_check_on_off( /************************************************************//** Validate the file format check config parameters, as a side effect it sets the srv_check_file_format_at_startup variable. -@return true if valid config value */ +@return the format_id if valid config value, otherwise, return -1 */ static -bool -innobase_file_format_check_validate( +int +innobase_file_format_validate_and_set( /*================================*/ const char* format_check) /*!< in: parameter value */ { uint format_id; - bool ret = true; format_id = innobase_file_format_name_lookup(format_check); if (format_id < DICT_TF_FORMAT_MAX + 1) { srv_check_file_format_at_startup = format_id; + + return((int) format_id); } else { - ret = false; + return(-1); } - - return(ret); } /*************************************************************//** @@ -9544,7 +9560,11 @@ innodb_file_format_name_validate( if (format_id <= DICT_TF_FORMAT_MAX) { - *static_cast<const char**>(save) = file_format_input; + /* Save a pointer to the name in the + 'file_format_name_map' constant array. */ + *static_cast<const char**>(save) = + trx_sys_file_format_id_to_name(format_id); + return(0); } } @@ -9607,6 +9627,7 @@ innodb_file_format_check_validate( const char* file_format_input; char buff[STRING_BUFFER_USUAL_SIZE]; int len = sizeof(buff); + int format_id; ut_a(save != NULL); ut_a(value != NULL); @@ -9619,24 +9640,35 @@ innodb_file_format_check_validate( message if they did so. */ if (innobase_file_format_check_on_off(file_format_input)) { - sql_print_warning( + push_warning_printf(thd, + MYSQL_ERROR::WARN_LEVEL_WARN, + ER_WRONG_ARGUMENTS, "InnoDB: invalid innodb_file_format_check " "value; on/off can only be set at startup or " "in the configuration file"); - } else if (innobase_file_format_check_validate( - file_format_input)) { + } else { + format_id = innobase_file_format_validate_and_set( + file_format_input); - *static_cast<const char**>(save) = file_format_input; + if (format_id >= 0) { + /* Save a pointer to the name in the + 'file_format_name_map' constant array. */ + *static_cast<const char**>(save) = + trx_sys_file_format_id_to_name( + (uint)format_id); - return(0); + return(0); - } else { - sql_print_warning( - "InnoDB: invalid innodb_file_format_check " - "value; can be any format up to %s " - "or its equivalent numeric id", - trx_sys_file_format_id_to_name( - DICT_TF_FORMAT_MAX)); + } else { + push_warning_printf(thd, + MYSQL_ERROR::WARN_LEVEL_WARN, + ER_WRONG_ARGUMENTS, + "InnoDB: invalid innodb_file_format_check " + "value; can be any format up to %s " + "or its equivalent numeric id", + trx_sys_file_format_id_to_name( + DICT_TF_FORMAT_MAX)); + } } } @@ -9906,12 +9938,15 @@ static MYSQL_SYSVAR_STR(file_format, inn innodb_file_format_name_validate, innodb_file_format_name_update, "Antelope"); +/* If a new file format is introduced, the file format +name needs to be updated accordingly. Please refer to +file_format_name_map[] defined in trx0sys.c for the next +file format name. */ static MYSQL_SYSVAR_STR(file_format_check, innobase_file_format_check, PLUGIN_VAR_OPCMDARG, "The highest file format in the tablespace.", innodb_file_format_check_validate, - innodb_file_format_check_update, - "on"); + innodb_file_format_check_update, "Barracuda"); static MYSQL_SYSVAR_ULONG(flush_log_at_trx_commit, srv_flush_log_at_trx_commit, PLUGIN_VAR_OPCMDARG, === modified file 'storage/innodb_plugin/handler/ha_innodb.h' --- a/storage/innodb_plugin/handler/ha_innodb.h 2009-11-03 10:07:51 +0000 +++ b/storage/innodb_plugin/handler/ha_innodb.h 2009-11-30 12:11:36 +0000 @@ -258,12 +258,14 @@ int thd_binlog_format(const MYSQL_THD th */ void thd_mark_transaction_to_rollback(MYSQL_THD thd, bool all); +#if MYSQL_VERSION_ID > 50140 /** Check if binary logging is filtered for thread's current db. @param thd Thread handle @retval 1 the query is not filtered, 0 otherwise. */ bool thd_binlog_filter_ok(const MYSQL_THD thd); +#endif /* MYSQL_VERSION_ID > 50140 */ } typedef struct trx_struct trx_t; === modified file 'storage/innodb_plugin/handler/handler0alter.cc' --- a/storage/innodb_plugin/handler/handler0alter.cc 2009-11-03 10:07:51 +0000 +++ b/storage/innodb_plugin/handler/handler0alter.cc 2009-11-30 13:42:26 +0000 @@ -765,10 +765,11 @@ err_exit: ut_ad(error == DB_SUCCESS); /* Commit the data dictionary transaction in order to release - the table locks on the system tables. Unfortunately, this - means that if MySQL crashes while creating a new primary key - inside row_merge_build_indexes(), indexed_table will not be - dropped on crash recovery. Thus, it will become orphaned. */ + the table locks on the system tables. This means that if + MySQL crashes while creating a new primary key inside + row_merge_build_indexes(), indexed_table will not be dropped + by trx_rollback_active(). It will have to be recovered or + dropped by the database administrator. */ trx_commit_for_mysql(trx); row_mysql_unlock_data_dictionary(trx); @@ -882,7 +883,9 @@ error: /* fall through */ default: if (new_primary) { - row_merge_drop_table(trx, indexed_table); + if (indexed_table != innodb_table) { + row_merge_drop_table(trx, indexed_table); + } } else { if (!dict_locked) { row_mysql_lock_data_dictionary(trx); === modified file 'storage/innodb_plugin/ibuf/ibuf0ibuf.c' --- a/storage/innodb_plugin/ibuf/ibuf0ibuf.c 2009-07-30 12:42:56 +0000 +++ b/storage/innodb_plugin/ibuf/ibuf0ibuf.c 2009-11-30 11:32:05 +0000 @@ -390,6 +390,27 @@ ibuf_count_set( #endif /******************************************************************//** +Closes insert buffer and frees the data structures. */ +UNIV_INTERN +void +ibuf_close(void) +/*============*/ +{ + mutex_free(&ibuf_pessimistic_insert_mutex); + memset(&ibuf_pessimistic_insert_mutex, + 0x0, sizeof(ibuf_pessimistic_insert_mutex)); + + mutex_free(&ibuf_mutex); + memset(&ibuf_mutex, 0x0, sizeof(ibuf_mutex)); + + mutex_free(&ibuf_bitmap_mutex); + memset(&ibuf_bitmap_mutex, 0x0, sizeof(ibuf_mutex)); + + mem_free(ibuf); + ibuf = NULL; +} + +/******************************************************************//** Updates the size information of the ibuf, assuming the segment size has not changed. */ static === modified file 'storage/innodb_plugin/include/btr0sea.h' --- a/storage/innodb_plugin/include/btr0sea.h 2009-05-27 09:45:59 +0000 +++ b/storage/innodb_plugin/include/btr0sea.h 2009-11-30 11:32:05 +0000 @@ -41,6 +41,12 @@ void btr_search_sys_create( /*==================*/ ulint hash_size); /*!< in: hash index hash table size */ +/*****************************************************************//** +Frees the adaptive search system at a database shutdown. */ +UNIV_INTERN +void +btr_search_sys_free(void); +/*=====================*/ /********************************************************************//** Disable the adaptive hash search system and empty the index. */ === modified file 'storage/innodb_plugin/include/db0err.h' --- a/storage/innodb_plugin/include/db0err.h 2009-05-27 09:45:59 +0000 +++ b/storage/innodb_plugin/include/db0err.h 2009-11-30 12:24:54 +0000 @@ -32,6 +32,7 @@ enum db_err { /* The following are error codes */ DB_ERROR, + DB_INTERRUPTED, DB_OUT_OF_MEMORY, DB_OUT_OF_FILE_SPACE, DB_LOCK_WAIT, === modified file 'storage/innodb_plugin/include/dict0dict.h' --- a/storage/innodb_plugin/include/dict0dict.h 2009-10-08 11:28:37 +0000 +++ b/storage/innodb_plugin/include/dict0dict.h 2009-11-30 11:32:05 +0000 @@ -1151,6 +1151,13 @@ void dict_ind_init(void); /*===============*/ +/**********************************************************************//** +Closes the data dictionary module. */ +UNIV_INTERN +void +dict_close(void); +/*============*/ + #ifndef UNIV_NONINL #include "dict0dict.ic" #endif === modified file 'storage/innodb_plugin/include/fil0fil.h' --- a/storage/innodb_plugin/include/fil0fil.h 2009-05-27 09:45:59 +0000 +++ b/storage/innodb_plugin/include/fil0fil.h 2009-11-30 11:32:05 +0000 @@ -224,15 +224,6 @@ fil_space_create( 0 for uncompressed tablespaces */ ulint purpose);/*!< in: FIL_TABLESPACE, or FIL_LOG if log */ /*******************************************************************//** -Frees a space object from a the tablespace memory cache. Closes the files in -the chain but does not delete them. -@return TRUE if success */ -UNIV_INTERN -ibool -fil_space_free( -/*===========*/ - ulint id); /*!< in: space id */ -/*******************************************************************//** Returns the size of the space in pages. The tablespace must be cached in the memory cache. @return space size, 0 if space not found */ @@ -278,6 +269,12 @@ fil_init( ulint hash_size, /*!< in: hash table size */ ulint max_n_open); /*!< in: max number of open files */ /*******************************************************************//** +Initializes the tablespace memory cache. */ +UNIV_INTERN +void +fil_close(void); +/*===========*/ +/*******************************************************************//** Opens all log files and system tablespace data files. They stay open until the database server shutdown. This should be called at a server startup after the space objects for the log and the system tablespace have been created. The === modified file 'storage/innodb_plugin/include/ha_prototypes.h' --- a/storage/innodb_plugin/include/ha_prototypes.h 2009-05-27 09:45:59 +0000 +++ b/storage/innodb_plugin/include/ha_prototypes.h 2009-12-01 10:38:40 +0000 @@ -153,28 +153,6 @@ get_innobase_type_from_mysql_type( const void* field) /*!< in: MySQL Field */ __attribute__((nonnull)); -/*************************************************************//** -If you want to print a thd that is not associated with the current thread, -you must call this function before reserving the InnoDB kernel_mutex, to -protect MySQL from setting thd->query NULL. If you print a thd of the current -thread, we know that MySQL cannot modify thd->query, and it is not necessary -to call this. Call innobase_mysql_end_print_arbitrary_thd() after you release -the kernel_mutex. */ -UNIV_INTERN -void -innobase_mysql_prepare_print_arbitrary_thd(void); -/*============================================*/ - -/*************************************************************//** -Releases the mutex reserved by innobase_mysql_prepare_print_arbitrary_thd(). -In the InnoDB latching order, the mutex sits right above the -kernel_mutex. In debug builds, we assert that the kernel_mutex is -released before this function is invoked. */ -UNIV_INTERN -void -innobase_mysql_end_print_arbitrary_thd(void); -/*========================================*/ - /******************************************************************//** Get the variable length bounds of the given character set. */ UNIV_INTERN === modified file 'storage/innodb_plugin/include/ibuf0ibuf.h' --- a/storage/innodb_plugin/include/ibuf0ibuf.h 2009-05-27 09:45:59 +0000 +++ b/storage/innodb_plugin/include/ibuf0ibuf.h 2009-11-30 11:32:05 +0000 @@ -356,6 +356,12 @@ void ibuf_print( /*=======*/ FILE* file); /*!< in: file where to print */ +/******************************************************************//** +Closes insert buffer and frees the data structures. */ +UNIV_INTERN +void +ibuf_close(void); +/*============*/ #define IBUF_HEADER_PAGE_NO FSP_IBUF_HEADER_PAGE_NO #define IBUF_TREE_ROOT_PAGE_NO FSP_IBUF_TREE_ROOT_PAGE_NO === modified file 'storage/innodb_plugin/include/lock0lock.h' --- a/storage/innodb_plugin/include/lock0lock.h 2009-10-08 12:18:19 +0000 +++ b/storage/innodb_plugin/include/lock0lock.h 2009-11-30 11:32:05 +0000 @@ -59,6 +59,12 @@ lock_sys_create( /*============*/ ulint n_cells); /*!< in: number of slots in lock hash table */ /*********************************************************************//** +Closes the lock system at database shutdown. */ +UNIV_INTERN +void +lock_sys_close(void); +/*================*/ +/*********************************************************************//** Checks if some transaction has an implicit x-lock on a record in a clustered index. @return transaction which has the x-lock, or NULL */ === modified file 'storage/innodb_plugin/include/log0log.h' --- a/storage/innodb_plugin/include/log0log.h 2009-10-08 12:18:19 +0000 +++ b/storage/innodb_plugin/include/log0log.h 2009-11-30 11:32:05 +0000 @@ -572,6 +572,18 @@ UNIV_INTERN void log_refresh_stats(void); /*===================*/ +/********************************************************** +Shutdown the log system but do not release all the memory. */ +UNIV_INTERN +void +log_shutdown(void); +/*==============*/ +/********************************************************** +Free the log system data structures. */ +UNIV_INTERN +void +log_mem_free(void); +/*==============*/ extern log_t* log_sys; @@ -584,7 +596,7 @@ extern log_t* log_sys; #define LOG_RECOVER 98887331 /* The counting of lsn's starts from this value: this must be non-zero */ -#define LOG_START_LSN ((ib_uint64_t) (16 * OS_FILE_LOG_BLOCK_SIZE)) +#define LOG_START_LSN ((ib_uint64_t) (16 * OS_FILE_LOG_BLOCK_SIZE)) #define LOG_BUFFER_SIZE (srv_log_buffer_size * UNIV_PAGE_SIZE) #define LOG_ARCHIVE_BUF_SIZE (srv_log_buffer_size * UNIV_PAGE_SIZE / 4) @@ -721,9 +733,12 @@ struct log_group_struct{ ulint lsn_offset; /*!< the offset of the above lsn */ ulint n_pending_writes;/*!< number of currently pending flush writes for this log group */ + byte** file_header_bufs_ptr;/*!< unaligned buffers */ byte** file_header_bufs;/*!< buffers for each file header in the group */ +#ifdef UNIV_LOG_ARCHIVE /*-----------------------------*/ + byte** archive_file_header_bufs_ptr;/*!< unaligned buffers */ byte** archive_file_header_bufs;/*!< buffers for each file header in the group */ ulint archive_space_id;/*!< file space which @@ -742,10 +757,12 @@ struct log_group_struct{ completion function then sets the new value to ..._file_no */ ulint next_archived_offset; /*!< like the preceding field */ +#endif /* UNIV_LOG_ARCHIVE */ /*-----------------------------*/ ib_uint64_t scanned_lsn; /*!< used only in recovery: recovery scan succeeded up to this lsn in this log group */ + byte* checkpoint_buf_ptr;/*!< unaligned checkpoint header */ byte* checkpoint_buf; /*!< checkpoint header is written from this buffer to the group */ UT_LIST_NODE_T(log_group_t) @@ -763,6 +780,7 @@ struct log_struct{ #ifndef UNIV_HOTBACKUP mutex_t mutex; /*!< mutex protecting the log */ #endif /* !UNIV_HOTBACKUP */ + byte* buf_ptr; /* unaligned log buffer */ byte* buf; /*!< log buffer */ ulint buf_size; /*!< log buffer size in bytes */ ulint max_buf_free; /*!< recommended maximum value of @@ -899,6 +917,7 @@ struct log_struct{ should wait for this without owning the log mutex */ #endif /* !UNIV_HOTBACKUP */ + byte* checkpoint_buf_ptr;/* unaligned checkpoint header */ byte* checkpoint_buf; /*!< checkpoint header is read to this buffer */ /* @} */ === modified file 'storage/innodb_plugin/include/log0recv.h' --- a/storage/innodb_plugin/include/log0recv.h 2009-10-09 14:13:15 +0000 +++ b/storage/innodb_plugin/include/log0recv.h 2009-11-30 11:32:05 +0000 @@ -239,6 +239,18 @@ UNIV_INTERN void recv_sys_create(void); /*=================*/ +/**********************************************************//** +Release recovery system mutexes. */ +UNIV_INTERN +void +recv_sys_close(void); +/*================*/ +/********************************************************//** +Frees the recovery system memory. */ +UNIV_INTERN +void +recv_sys_mem_free(void); +/*===================*/ /********************************************************//** Inits the recovery system for a recovery operation. */ UNIV_INTERN @@ -246,6 +258,12 @@ void recv_sys_init( /*==========*/ ulint available_memory); /*!< in: available memory in bytes */ +/********************************************************//** +Reset the state of the recovery system variables. */ +UNIV_INTERN +void +recv_sys_var_init(void); +/*===================*/ /*******************************************************************//** Empties the hash table of stored log records, applying them to appropriate pages. */ === modified file 'storage/innodb_plugin/include/mem0mem.h' --- a/storage/innodb_plugin/include/mem0mem.h 2009-07-30 12:42:56 +0000 +++ b/storage/innodb_plugin/include/mem0mem.h 2009-11-30 11:32:05 +0000 @@ -82,6 +82,13 @@ void mem_init( /*=====*/ ulint size); /*!< in: common pool size in bytes */ +/******************************************************************//** +Closes the memory system. */ +UNIV_INTERN +void +mem_close(void); +/*===========*/ + /**************************************************************//** Use this macro instead of the corresponding function! Macro for memory heap creation. */ === modified file 'storage/innodb_plugin/include/mem0pool.h' --- a/storage/innodb_plugin/include/mem0pool.h 2009-05-27 09:45:59 +0000 +++ b/storage/innodb_plugin/include/mem0pool.h 2009-11-30 11:32:05 +0000 @@ -62,6 +62,13 @@ mem_pool_create( /*============*/ ulint size); /*!< in: pool size in bytes */ /********************************************************************//** +Frees a memory pool. */ +UNIV_INTERN +void +mem_pool_free( +/*==========*/ + mem_pool_t* pool); /*!< in, own: memory pool */ +/********************************************************************//** Allocates memory from a pool. NOTE: This low-level function should only be used in mem0mem.*! @return own: allocated memory buffer */ === modified file 'storage/innodb_plugin/include/os0file.h' --- a/storage/innodb_plugin/include/os0file.h 2009-11-03 09:59:06 +0000 +++ b/storage/innodb_plugin/include/os0file.h 2009-11-30 12:04:09 +0000 @@ -158,6 +158,7 @@ log. */ #define OS_FILE_SHARING_VIOLATION 76 #define OS_FILE_ERROR_NOT_SPECIFIED 77 #define OS_FILE_INSUFFICIENT_RESOURCE 78 +#define OS_FILE_OPERATION_ABORTED 79 /* @} */ /** Types for aio operations @{ */ @@ -620,6 +621,13 @@ os_aio_init( ulint n_write_segs, /*<! in: number of writer threads */ ulint n_slots_sync); /*<! in: number of slots in the sync aio array */ +/*********************************************************************** +Frees the asynchronous io system. */ +UNIV_INTERN +void +os_aio_free(void); +/*=============*/ + /*******************************************************************//** Requests an asynchronous i/o operation. @return TRUE if request was queued successfully, FALSE if fail */ === modified file 'storage/innodb_plugin/include/pars0pars.h' --- a/storage/innodb_plugin/include/pars0pars.h 2009-05-27 09:45:59 +0000 +++ b/storage/innodb_plugin/include/pars0pars.h 2009-11-30 11:32:05 +0000 @@ -583,6 +583,12 @@ pars_info_get_bound_id( pars_info_t* info, /*!< in: info struct */ const char* name); /*!< in: bound id name to find */ +/******************************************************************//** +Release any resources used by the lexer. */ +UNIV_INTERN +void +pars_lexer_close(void); +/*==================*/ /** Extra information supplied for pars_sql(). */ struct pars_info_struct { === modified file 'storage/innodb_plugin/include/srv0srv.h' --- a/storage/innodb_plugin/include/srv0srv.h 2009-10-08 11:28:37 +0000 +++ b/storage/innodb_plugin/include/srv0srv.h 2009-11-30 11:32:05 +0000 @@ -411,7 +411,7 @@ void srv_init(void); /*==========*/ /*********************************************************************//** -Frees the OS fast mutex created in srv_boot(). */ +Frees the data structures created in srv_init(). */ UNIV_INTERN void srv_free(void); === modified file 'storage/innodb_plugin/include/thr0loc.h' --- a/storage/innodb_plugin/include/thr0loc.h 2009-05-27 09:45:59 +0000 +++ b/storage/innodb_plugin/include/thr0loc.h 2009-11-30 11:32:05 +0000 @@ -39,6 +39,12 @@ UNIV_INTERN void thr_local_init(void); /*================*/ + /****************************************************************//** +Close the thread local storage module. */ +UNIV_INTERN +void +thr_local_close(void); +/*=================*/ /*******************************************************************//** Creates a local storage struct for the calling new thread. */ UNIV_INTERN === modified file 'storage/innodb_plugin/include/trx0i_s.h' --- a/storage/innodb_plugin/include/trx0i_s.h 2009-05-27 09:45:59 +0000 +++ b/storage/innodb_plugin/include/trx0i_s.h 2009-11-30 11:32:05 +0000 @@ -141,6 +141,13 @@ void trx_i_s_cache_init( /*===============*/ trx_i_s_cache_t* cache); /*!< out: cache to init */ +/*******************************************************************//** +Free the INFORMATION SCHEMA trx related cache. */ +UNIV_INTERN +void +trx_i_s_cache_free( +/*===============*/ + trx_i_s_cache_t* cache); /*!< in/out: cache to free */ /*******************************************************************//** Issue a shared/read lock on the tables cache. */ === modified file 'storage/innodb_plugin/include/trx0purge.h' --- a/storage/innodb_plugin/include/trx0purge.h 2009-05-27 09:45:59 +0000 +++ b/storage/innodb_plugin/include/trx0purge.h 2009-11-30 11:32:05 +0000 @@ -71,6 +71,12 @@ void trx_purge_sys_create(void); /*======================*/ /********************************************************************//** +Frees the global purge system control structure. */ +UNIV_INTERN +void +trx_purge_sys_close(void); +/*======================*/ +/************************************************************************ Adds the update undo log as the first log in the history list. Removes the update undo log segment from the rseg slot if it is too big for reuse. */ UNIV_INTERN === modified file 'storage/innodb_plugin/include/trx0rseg.h' --- a/storage/innodb_plugin/include/trx0rseg.h 2009-05-27 09:45:59 +0000 +++ b/storage/innodb_plugin/include/trx0rseg.h 2009-11-30 11:32:05 +0000 @@ -125,6 +125,13 @@ trx_rseg_create( ulint max_size, /*!< in: max size in pages */ ulint* id, /*!< out: rseg id */ mtr_t* mtr); /*!< in: mtr */ +/*************************************************************************** +Free's an instance of the rollback segment in memory. */ +UNIV_INTERN +void +trx_rseg_mem_free( +/*==============*/ + trx_rseg_t* rseg); /* in, own: instance to free */ /* Number of undo log slots in a rollback segment file copy */ === modified file 'storage/innodb_plugin/include/trx0sys.h' --- a/storage/innodb_plugin/include/trx0sys.h 2009-07-30 12:42:56 +0000 +++ b/storage/innodb_plugin/include/trx0sys.h 2009-11-30 11:32:05 +0000 @@ -334,6 +334,12 @@ void trx_sys_file_format_tag_init(void); /*==============================*/ /*****************************************************************//** +Shutdown/Close the transaction system. */ +UNIV_INTERN +void +trx_sys_close(void); +/*===============*/ +/*****************************************************************//** Get the name representation of the file format from its id. @return pointer to the name */ UNIV_INTERN === modified file 'storage/innodb_plugin/include/trx0trx.h' --- a/storage/innodb_plugin/include/trx0trx.h 2009-10-08 13:05:59 +0000 +++ b/storage/innodb_plugin/include/trx0trx.h 2009-12-01 10:38:40 +0000 @@ -338,9 +338,7 @@ trx_commit_step( /**********************************************************************//** Prints info about a transaction to the given file. The caller must own the -kernel mutex and must have called -innobase_mysql_prepare_print_arbitrary_thd(), unless he knows that MySQL -or InnoDB cannot meanwhile change the info printed here. */ +kernel mutex. */ UNIV_INTERN void trx_print( === modified file 'storage/innodb_plugin/include/trx0undo.h' --- a/storage/innodb_plugin/include/trx0undo.h 2009-05-27 09:45:59 +0000 +++ b/storage/innodb_plugin/include/trx0undo.h 2009-11-30 11:32:05 +0000 @@ -333,6 +333,13 @@ trx_undo_parse_discard_latest( byte* end_ptr,/*!< in: buffer end */ page_t* page, /*!< in: page or NULL */ mtr_t* mtr); /*!< in: mtr or NULL */ +/************************************************************************ +Frees an undo log memory copy. */ +UNIV_INTERN +void +trx_undo_mem_free( +/*==============*/ + trx_undo_t* undo); /* in: the undo object to be freed */ /* Types of an undo log segment */ #define TRX_UNDO_INSERT 1 /* contains undo entries for inserts */ === modified file 'storage/innodb_plugin/include/univ.i' --- a/storage/innodb_plugin/include/univ.i 2009-11-03 10:26:39 +0000 +++ b/storage/innodb_plugin/include/univ.i 2009-11-30 13:13:34 +0000 @@ -46,7 +46,7 @@ Created 1/20/1994 Heikki Tuuri #define INNODB_VERSION_MAJOR 1 #define INNODB_VERSION_MINOR 0 -#define INNODB_VERSION_BUGFIX 5 +#define INNODB_VERSION_BUGFIX 6 /* The following is the InnoDB version as shown in SELECT plugin_version FROM information_schema.plugins; === modified file 'storage/innodb_plugin/include/usr0sess.h' --- a/storage/innodb_plugin/include/usr0sess.h 2009-05-27 09:45:59 +0000 +++ b/storage/innodb_plugin/include/usr0sess.h 2009-11-30 11:32:05 +0000 @@ -44,14 +44,12 @@ sess_t* sess_open(void); /*============*/ /*********************************************************************//** -Closes a session, freeing the memory occupied by it, if it is in a state -where it should be closed. -@return TRUE if closed */ +Closes a session, freeing the memory occupied by it. */ UNIV_INTERN -ibool -sess_try_close( -/*===========*/ - sess_t* sess); /*!< in, own: session object */ +void +sess_close( +/*=======*/ + sess_t* sess); /* in, own: session object */ /* The session handle. All fields are protected by the kernel mutex */ struct sess_struct{ === modified file 'storage/innodb_plugin/lock/lock0lock.c' --- a/storage/innodb_plugin/lock/lock0lock.c 2009-10-09 14:13:15 +0000 +++ b/storage/innodb_plugin/lock/lock0lock.c 2009-12-01 10:38:40 +0000 @@ -578,6 +578,23 @@ lock_sys_create( } /*********************************************************************//** +Closes the lock system at database shutdown. */ +UNIV_INTERN +void +lock_sys_close(void) +/*================*/ +{ + if (lock_latest_err_file != NULL) { + fclose(lock_latest_err_file); + lock_latest_err_file = NULL; + } + + hash_table_free(lock_sys->rec_hash); + mem_free(lock_sys); + lock_sys = NULL; +} + +/*********************************************************************//** Gets the size of a lock struct. @return size in bytes */ UNIV_INTERN @@ -4307,11 +4324,6 @@ lock_print_info_summary( /*====================*/ FILE* file) /*!< in: file where to print */ { - /* We must protect the MySQL thd->query field with a MySQL mutex, and - because the MySQL mutex must be reserved before the kernel_mutex of - InnoDB, we call innobase_mysql_prepare_print_arbitrary_thd() here. */ - - innobase_mysql_prepare_print_arbitrary_thd(); lock_mutex_enter_kernel(); if (lock_deadlock_found) { @@ -4394,7 +4406,6 @@ loop: if (trx == NULL) { lock_mutex_exit_kernel(); - innobase_mysql_end_print_arbitrary_thd(); ut_ad(lock_validate()); @@ -4478,7 +4489,6 @@ loop: } lock_mutex_exit_kernel(); - innobase_mysql_end_print_arbitrary_thd(); mtr_start(&mtr); @@ -4489,7 +4499,6 @@ loop: load_page_first = FALSE; - innobase_mysql_prepare_print_arbitrary_thd(); lock_mutex_enter_kernel(); goto loop; === modified file 'storage/innodb_plugin/log/log0log.c' --- a/storage/innodb_plugin/log/log0log.c 2009-10-09 14:13:15 +0000 +++ b/storage/innodb_plugin/log/log0log.c 2009-11-30 11:32:05 +0000 @@ -771,8 +771,6 @@ void log_init(void) /*==========*/ { - byte* buf; - log_sys = mem_alloc(sizeof(log_t)); mutex_create(&log_sys->mutex, SYNC_LOG); @@ -787,8 +785,8 @@ log_init(void) ut_a(LOG_BUFFER_SIZE >= 16 * OS_FILE_LOG_BLOCK_SIZE); ut_a(LOG_BUFFER_SIZE >= 4 * UNIV_PAGE_SIZE); - buf = mem_alloc(LOG_BUFFER_SIZE + OS_FILE_LOG_BLOCK_SIZE); - log_sys->buf = ut_align(buf, OS_FILE_LOG_BLOCK_SIZE); + log_sys->buf_ptr = mem_alloc(LOG_BUFFER_SIZE + OS_FILE_LOG_BLOCK_SIZE); + log_sys->buf = ut_align(log_sys->buf_ptr, OS_FILE_LOG_BLOCK_SIZE); log_sys->buf_size = LOG_BUFFER_SIZE; @@ -833,9 +831,9 @@ log_init(void) rw_lock_create(&log_sys->checkpoint_lock, SYNC_NO_ORDER_CHECK); - log_sys->checkpoint_buf - = ut_align(mem_alloc(2 * OS_FILE_LOG_BLOCK_SIZE), - OS_FILE_LOG_BLOCK_SIZE); + log_sys->checkpoint_buf_ptr = mem_alloc(2 * OS_FILE_LOG_BLOCK_SIZE); + log_sys->checkpoint_buf = ut_align(log_sys->checkpoint_buf_ptr, + OS_FILE_LOG_BLOCK_SIZE); memset(log_sys->checkpoint_buf, '\0', OS_FILE_LOG_BLOCK_SIZE); /*----------------------------*/ @@ -918,23 +916,33 @@ log_group_init( group->lsn_offset = LOG_FILE_HDR_SIZE; group->n_pending_writes = 0; + group->file_header_bufs_ptr = mem_alloc(sizeof(byte*) * n_files); group->file_header_bufs = mem_alloc(sizeof(byte*) * n_files); #ifdef UNIV_LOG_ARCHIVE + group->archive_file_header_bufs_ptr = mem_alloc( + sizeof(byte*) * n_files); group->archive_file_header_bufs = mem_alloc(sizeof(byte*) * n_files); #endif /* UNIV_LOG_ARCHIVE */ for (i = 0; i < n_files; i++) { - *(group->file_header_bufs + i) = ut_align( - mem_alloc(LOG_FILE_HDR_SIZE + OS_FILE_LOG_BLOCK_SIZE), + group->file_header_bufs_ptr[i] = mem_alloc( + LOG_FILE_HDR_SIZE + OS_FILE_LOG_BLOCK_SIZE); + + group->file_header_bufs[i] = ut_align( + group->file_header_bufs_ptr[i], OS_FILE_LOG_BLOCK_SIZE); memset(*(group->file_header_bufs + i), '\0', LOG_FILE_HDR_SIZE); #ifdef UNIV_LOG_ARCHIVE - *(group->archive_file_header_bufs + i) = ut_align( - mem_alloc(LOG_FILE_HDR_SIZE + OS_FILE_LOG_BLOCK_SIZE), + group->archive_file_header_bufs_ptr[i] = mem_alloc( + LOG_FILE_HDR_SIZE + OS_FILE_LOG_BLOCK_SIZE); + + group->archive_file_header_bufs[i] = ut_align( + group->archive_file_header_bufs_ptr[i], OS_FILE_LOG_BLOCK_SIZE); + memset(*(group->archive_file_header_bufs + i), '\0', LOG_FILE_HDR_SIZE); #endif /* UNIV_LOG_ARCHIVE */ @@ -947,8 +955,9 @@ log_group_init( group->archived_offset = 0; #endif /* UNIV_LOG_ARCHIVE */ - group->checkpoint_buf = ut_align( - mem_alloc(2 * OS_FILE_LOG_BLOCK_SIZE), OS_FILE_LOG_BLOCK_SIZE); + group->checkpoint_buf_ptr = mem_alloc(2 * OS_FILE_LOG_BLOCK_SIZE); + group->checkpoint_buf = ut_align(group->checkpoint_buf_ptr, + OS_FILE_LOG_BLOCK_SIZE); memset(group->checkpoint_buf, '\0', OS_FILE_LOG_BLOCK_SIZE); @@ -3364,4 +3373,95 @@ log_refresh_stats(void) log_sys->n_log_ios_old = log_sys->n_log_ios; log_sys->last_printout_time = time(NULL); } + +/********************************************************************** +Closes a log group. */ +static +void +log_group_close( +/*===========*/ + log_group_t* group) /* in,own: log group to close */ +{ + ulint i; + + for (i = 0; i < group->n_files; i++) { + mem_free(group->file_header_bufs_ptr[i]); +#ifdef UNIV_LOG_ARCHIVE + mem_free(group->archive_file_header_bufs_ptr[i]); +#endif /* UNIV_LOG_ARCHIVE */ + } + + mem_free(group->file_header_bufs_ptr); + mem_free(group->file_header_bufs); + +#ifdef UNIV_LOG_ARCHIVE + mem_free(group->archive_file_header_bufs_ptr); + mem_free(group->archive_file_header_bufs); +#endif /* UNIV_LOG_ARCHIVE */ + + mem_free(group->checkpoint_buf_ptr); + + mem_free(group); +} + +/********************************************************** +Shutdown the log system but do not release all the memory. */ +UNIV_INTERN +void +log_shutdown(void) +/*==============*/ +{ + log_group_t* group; + + group = UT_LIST_GET_FIRST(log_sys->log_groups); + + while (UT_LIST_GET_LEN(log_sys->log_groups) > 0) { + log_group_t* prev_group = group; + + group = UT_LIST_GET_NEXT(log_groups, group); + UT_LIST_REMOVE(log_groups, log_sys->log_groups, prev_group); + + log_group_close(prev_group); + } + + mem_free(log_sys->buf_ptr); + log_sys->buf_ptr = NULL; + log_sys->buf = NULL; + mem_free(log_sys->checkpoint_buf_ptr); + log_sys->checkpoint_buf_ptr = NULL; + log_sys->checkpoint_buf = NULL; + + os_event_free(log_sys->no_flush_event); + os_event_free(log_sys->one_flushed_event); + + rw_lock_free(&log_sys->checkpoint_lock); + + mutex_free(&log_sys->mutex); + +#ifdef UNIV_LOG_ARCHIVE + rw_lock_free(&log_sys->archive_lock); + os_event_create(log_sys->archiving_on); +#endif /* UNIV_LOG_ARCHIVE */ + +#ifdef UNIV_LOG_DEBUG + recv_sys_debug_free(); +#endif + + recv_sys_close(); +} + +/********************************************************** +Free the log system data structures. */ +UNIV_INTERN +void +log_mem_free(void) +/*==============*/ +{ + if (log_sys != NULL) { + recv_sys_mem_free(); + mem_free(log_sys); + + log_sys = NULL; + } +} #endif /* !UNIV_HOTBACKUP */ === modified file 'storage/innodb_plugin/log/log0recv.c' --- a/storage/innodb_plugin/log/log0recv.c 2009-10-09 14:13:15 +0000 +++ b/storage/innodb_plugin/log/log0recv.c 2009-11-30 11:32:05 +0000 @@ -69,15 +69,15 @@ UNIV_INTERN recv_sys_t* recv_sys = NULL; /** TRUE when applying redo log records during crash recovery; FALSE otherwise. Note that this is FALSE while a background thread is rolling back incomplete transactions. */ -UNIV_INTERN ibool recv_recovery_on = FALSE; +UNIV_INTERN ibool recv_recovery_on; #ifdef UNIV_LOG_ARCHIVE /** TRUE when applying redo log records from an archived log file */ -UNIV_INTERN ibool recv_recovery_from_backup_on = FALSE; +UNIV_INTERN ibool recv_recovery_from_backup_on; #endif /* UNIV_LOG_ARCHIVE */ #ifndef UNIV_HOTBACKUP /** TRUE when recv_init_crash_recovery() has been called. */ -UNIV_INTERN ibool recv_needed_recovery = FALSE; +UNIV_INTERN ibool recv_needed_recovery; # ifdef UNIV_DEBUG /** TRUE if writing to the redo log (mtr_commit) is forbidden. Protected by log_sys->mutex. */ @@ -87,7 +87,7 @@ UNIV_INTERN ibool recv_no_log_write = FA /** TRUE if buf_page_is_corrupted() should check if the log sequence number (FIL_PAGE_LSN) is in the future. Initially FALSE, and set by recv_recovery_from_checkpoint_start_func(). */ -UNIV_INTERN ibool recv_lsn_checks_on = FALSE; +UNIV_INTERN ibool recv_lsn_checks_on; /** There are two conditions under which we scan the logs, the first is normal startup and the second is when we do a recovery from an @@ -97,7 +97,7 @@ startup. If we find log entries that wer we know that the server was not cleanly shutdown. We must then initialize the crash recovery environment before attempting to store these entries in the log hash table. */ -static ibool recv_log_scan_is_startup_type = FALSE; +static ibool recv_log_scan_is_startup_type; /** If the following is TRUE, the buffer pool file pages must be invalidated after recovery and no ibuf operations are allowed; this becomes TRUE if @@ -108,7 +108,7 @@ buffer pool before the pages have been r TRUE means that recovery is running and no operations on the log files are allowed yet: the variable name is misleading. */ -UNIV_INTERN ibool recv_no_ibuf_operations = FALSE; +UNIV_INTERN ibool recv_no_ibuf_operations; /** TRUE when the redo log is being backed up */ # define recv_is_making_a_backup FALSE /** TRUE when recovering from a backed up redo log file */ @@ -116,30 +116,30 @@ UNIV_INTERN ibool recv_no_ibuf_operation #else /* !UNIV_HOTBACKUP */ # define recv_needed_recovery FALSE /** TRUE when the redo log is being backed up */ -UNIV_INTERN ibool recv_is_making_a_backup = FALSE; +UNIV_INTERN ibool recv_is_making_a_backup = FALSE; /** TRUE when recovering from a backed up redo log file */ UNIV_INTERN ibool recv_is_from_backup = FALSE; # define buf_pool_get_curr_size() (5 * 1024 * 1024) #endif /* !UNIV_HOTBACKUP */ /** The following counter is used to decide when to print info on log scan */ -static ulint recv_scan_print_counter = 0; +static ulint recv_scan_print_counter; /** The type of the previous parsed redo log record */ -static ulint recv_previous_parsed_rec_type = 999999; +static ulint recv_previous_parsed_rec_type; /** The offset of the previous parsed redo log record */ -static ulint recv_previous_parsed_rec_offset = 0; +static ulint recv_previous_parsed_rec_offset; /** The 'multi' flag of the previous parsed redo log record */ -static ulint recv_previous_parsed_rec_is_multi = 0; +static ulint recv_previous_parsed_rec_is_multi; /** Maximum page number encountered in the redo log */ -UNIV_INTERN ulint recv_max_parsed_page_no = 0; +UNIV_INTERN ulint recv_max_parsed_page_no; /** This many frames must be left free in the buffer pool when we scan the log and store the scanned log records in the buffer pool: we will use these free frames to read in pages when we start applying the log records to the database. */ -UNIV_INTERN ulint recv_n_pool_free_frames = 256; +UNIV_INTERN ulint recv_n_pool_free_frames; /** The maximum lsn we see for a page during the recovery process. If this is bigger than the lsn we are able to scan up to, that is an indication that @@ -170,7 +170,8 @@ recv_sys_create(void) return; } - recv_sys = mem_alloc(sizeof(recv_sys_t)); + recv_sys = mem_alloc(sizeof(*recv_sys)); + memset(recv_sys, 0x0, sizeof(*recv_sys)); mutex_create(&recv_sys->mutex, SYNC_RECV); @@ -179,6 +180,106 @@ recv_sys_create(void) } /********************************************************//** +Release recovery system mutexes. */ +UNIV_INTERN +void +recv_sys_close(void) +/*================*/ +{ + if (recv_sys != NULL) { + if (recv_sys->addr_hash != NULL) { + hash_table_free(recv_sys->addr_hash); + } + + if (recv_sys->heap != NULL) { + mem_heap_free(recv_sys->heap); + } + + if (recv_sys->buf != NULL) { + ut_free(recv_sys->buf); + } + + if (recv_sys->last_block_buf_start != NULL) { + mem_free(recv_sys->last_block_buf_start); + } + + mutex_free(&recv_sys->mutex); + + mem_free(recv_sys); + recv_sys = NULL; + } +} + +/********************************************************//** +Frees the recovery system memory. */ +UNIV_INTERN +void +recv_sys_mem_free(void) +/*===================*/ +{ + if (recv_sys != NULL) { + if (recv_sys->addr_hash != NULL) { + hash_table_free(recv_sys->addr_hash); + } + + if (recv_sys->heap != NULL) { + mem_heap_free(recv_sys->heap); + } + + if (recv_sys->buf != NULL) { + ut_free(recv_sys->buf); + } + + if (recv_sys->last_block_buf_start != NULL) { + mem_free(recv_sys->last_block_buf_start); + } + + mem_free(recv_sys); + recv_sys = NULL; + } +} + +/************************************************************ +Reset the state of the recovery system variables. */ +UNIV_INTERN +void +recv_sys_var_init(void) +/*===================*/ +{ + recv_lsn_checks_on = FALSE; + + recv_n_pool_free_frames = 256; + + recv_recovery_on = FALSE; + +#ifdef UNIV_LOG_ARCHIVE + recv_recovery_from_backup_on = FALSE; +#endif /* UNIV_LOG_ARCHIVE */ + + recv_needed_recovery = FALSE; + + recv_lsn_checks_on = FALSE; + + recv_log_scan_is_startup_type = FALSE; + + recv_no_ibuf_operations = FALSE; + + recv_scan_print_counter = 0; + + recv_previous_parsed_rec_type = 999999; + + recv_previous_parsed_rec_offset = 0; + + recv_previous_parsed_rec_is_multi = 0; + + recv_max_parsed_page_no = 0; + + recv_n_pool_free_frames = 256; + + recv_max_page_lsn = 0; +} + +/************************************************************ Inits the recovery system for a recovery operation. */ UNIV_INTERN void @@ -253,8 +354,8 @@ recv_sys_empty_hash(void) Frees the recovery system. */ static void -recv_sys_free(void) -/*===============*/ +recv_sys_debug_free(void) +/*=====================*/ { mutex_enter(&(recv_sys->mutex)); @@ -263,8 +364,10 @@ recv_sys_free(void) ut_free(recv_sys->buf); mem_free(recv_sys->last_block_buf_start); - recv_sys->addr_hash = NULL; + recv_sys->buf = NULL; recv_sys->heap = NULL; + recv_sys->addr_hash = NULL; + recv_sys->last_block_buf_start = NULL; mutex_exit(&(recv_sys->mutex)); } @@ -3149,7 +3252,7 @@ recv_recovery_from_checkpoint_finish(voi recv_recovery_on = FALSE; #ifndef UNIV_LOG_DEBUG - recv_sys_free(); + recv_sys_debug_free(); #endif /* Roll back any recovered data dictionary transactions, so that the data dictionary tables will be free of any locks. === modified file 'storage/innodb_plugin/mem/mem0dbg.c' --- a/storage/innodb_plugin/mem/mem0dbg.c 2009-05-27 09:45:59 +0000 +++ b/storage/innodb_plugin/mem/mem0dbg.c 2009-11-30 11:32:05 +0000 @@ -170,6 +170,17 @@ mem_init( mem_comm_pool = mem_pool_create(size); } + +/******************************************************************//** +Closes the memory system. */ +UNIV_INTERN +void +mem_close(void) +/*===========*/ +{ + mem_pool_free(mem_comm_pool); + mem_comm_pool = NULL; +} #endif /* !UNIV_HOTBACKUP */ #ifdef UNIV_MEM_DEBUG === modified file 'storage/innodb_plugin/mem/mem0pool.c' --- a/storage/innodb_plugin/mem/mem0pool.c 2009-05-27 09:45:59 +0000 +++ b/storage/innodb_plugin/mem/mem0pool.c 2009-11-30 11:32:05 +0000 @@ -261,6 +261,18 @@ mem_pool_create( } /********************************************************************//** +Frees a memory pool. */ +UNIV_INTERN +void +mem_pool_free( +/*==========*/ + mem_pool_t* pool) /*!< in, own: memory pool */ +{ + ut_free(pool->buf); + ut_free(pool); +} + +/********************************************************************//** Fills the specified free list. @return TRUE if we were able to insert a block to the free list */ static === modified file 'storage/innodb_plugin/os/os0file.c' --- a/storage/innodb_plugin/os/os0file.c 2009-11-03 09:59:31 +0000 +++ b/storage/innodb_plugin/os/os0file.c 2009-11-30 12:04:09 +0000 @@ -323,6 +323,13 @@ os_file_get_last_error( "InnoDB: The error means that there are no" " sufficient system resources or quota to" " complete the operation.\n"); + } else if (err == ERROR_OPERATION_ABORTED) { + fprintf(stderr, + "InnoDB: The error means that the I/O" + " operation has been aborted\n" + "InnoDB: because of either a thread exit" + " or an application request.\n" + "InnoDB: Retry attempt is made.\n"); } else { fprintf(stderr, "InnoDB: Some operating system error numbers" @@ -347,6 +354,8 @@ os_file_get_last_error( } else if (err == ERROR_WORKING_SET_QUOTA || err == ERROR_NO_SYSTEM_RESOURCES) { return(OS_FILE_INSUFFICIENT_RESOURCE); + } else if (err == ERROR_OPERATION_ABORTED) { + return(OS_FILE_OPERATION_ABORTED); } else { return(100 + err); } @@ -469,6 +478,10 @@ os_file_handle_error_cond_exit( os_thread_sleep(100000); /* 100 ms */ return(TRUE); + } else if (err == OS_FILE_OPERATION_ABORTED) { + + os_thread_sleep(100000); /* 100 ms */ + return(TRUE); } else { if (name) { fprintf(stderr, "InnoDB: File name %s\n", name); @@ -3029,6 +3042,34 @@ os_aio_array_create( return(array); } +/************************************************************************//** +Frees an aio wait array. */ +static +void +os_aio_array_free( +/*==============*/ + os_aio_array_t* array) /*!< in, own: array to free */ +{ +#ifdef WIN_ASYNC_IO + ulint i; + + for (i = 0; i < array->n_slots; i++) { + os_aio_slot_t* slot = os_aio_array_get_nth_slot(array, i); + os_event_free(slot->event); + } +#endif /* WIN_ASYNC_IO */ + +#ifdef __WIN__ + ut_free(array->native_events); +#endif /* __WIN__ */ + os_mutex_free(array->mutex); + os_event_free(array->not_full); + os_event_free(array->is_empty); + + ut_free(array->slots); + ut_free(array); +} + /*********************************************************************** Initializes the asynchronous io system. Creates one array each for ibuf and log i/o. Also creates one array each for read and write where each @@ -3099,6 +3140,35 @@ os_aio_init( } +/*********************************************************************** +Frees the asynchronous io system. */ +UNIV_INTERN +void +os_aio_free(void) +/*=============*/ +{ + ulint i; + + os_aio_array_free(os_aio_ibuf_array); + os_aio_ibuf_array = NULL; + os_aio_array_free(os_aio_log_array); + os_aio_log_array = NULL; + os_aio_array_free(os_aio_read_array); + os_aio_read_array = NULL; + os_aio_array_free(os_aio_write_array); + os_aio_write_array = NULL; + os_aio_array_free(os_aio_sync_array); + os_aio_sync_array = NULL; + + for (i = 0; i < os_aio_n_segments; i++) { + os_event_free(os_aio_segment_wait_events[i]); + } + + ut_free(os_aio_segment_wait_events); + os_aio_segment_wait_events = 0; + os_aio_n_segments = 0; +} + #ifdef WIN_ASYNC_IO /************************************************************************//** Wakes up all async i/o threads in the array in Windows async i/o at @@ -3709,6 +3779,7 @@ os_aio_windows_handle( ibool ret_val; BOOL ret; DWORD len; + BOOL retry = FALSE; if (segment == ULINT_UNDEFINED) { array = os_aio_sync_array; @@ -3762,14 +3833,52 @@ os_aio_windows_handle( ut_a(TRUE == os_file_flush(slot->file)); } #endif /* UNIV_DO_FLUSH */ + } else if (os_file_handle_error(slot->name, "Windows aio")) { + + retry = TRUE; } else { - os_file_handle_error(slot->name, "Windows aio"); ret_val = FALSE; } os_mutex_exit(array->mutex); + if (retry) { + /* retry failed read/write operation synchronously. + No need to hold array->mutex. */ + + switch (slot->type) { + case OS_FILE_WRITE: + ret = WriteFile(slot->file, slot->buf, + slot->len, &len, + &(slot->control)); + + break; + case OS_FILE_READ: + ret = ReadFile(slot->file, slot->buf, + slot->len, &len, + &(slot->control)); + + break; + default: + ut_error; + } + + if (!ret && GetLastError() == ERROR_IO_PENDING) { + /* aio was queued successfully! + We want a synchronous i/o operation on a + file where we also use async i/o: in Windows + we must use the same wait mechanism as for + async i/o */ + + ret = GetOverlappedResult(slot->file, + &(slot->control), + &len, TRUE); + } + + ret_val = ret && len == slot->len; + } + os_aio_array_free_slot(array, slot); return(ret_val); === modified file 'storage/innodb_plugin/os/os0sync.c' --- a/storage/innodb_plugin/os/os0sync.c 2009-07-30 12:42:56 +0000 +++ b/storage/innodb_plugin/os/os0sync.c 2009-11-30 11:32:05 +0000 @@ -86,6 +86,9 @@ os_sync_init(void) UT_LIST_INIT(os_event_list); UT_LIST_INIT(os_mutex_list); + os_sync_mutex = NULL; + os_sync_mutex_inited = FALSE; + os_sync_mutex = os_mutex_create(NULL); os_sync_mutex_inited = TRUE; @@ -713,6 +716,7 @@ os_fast_mutex_free( os_mutex_enter(os_sync_mutex); } + ut_ad(os_fast_mutex_count > 0); os_fast_mutex_count--; if (UNIV_LIKELY(os_sync_mutex_inited)) { === modified file 'storage/innodb_plugin/os/os0thread.c' --- a/storage/innodb_plugin/os/os0thread.c 2009-05-27 09:45:59 +0000 +++ b/storage/innodb_plugin/os/os0thread.c 2009-11-30 11:32:05 +0000 @@ -233,6 +233,7 @@ os_thread_exit( #ifdef __WIN__ ExitThread((DWORD)exit_value); #else + pthread_detach(pthread_self()); pthread_exit(exit_value); #endif } === modified file 'storage/innodb_plugin/pars/lexyy.c' --- a/storage/innodb_plugin/pars/lexyy.c 2009-05-27 09:45:59 +0000 +++ b/storage/innodb_plugin/pars/lexyy.c 2009-11-30 11:32:05 +0000 @@ -2778,3 +2778,16 @@ static void yyfree (void * ptr ) + +/********************************************************************** +Release any resources used by the lexer. */ +UNIV_INTERN +void +pars_lexer_close(void) +/*==================*/ +{ + yylex_destroy(); + free(stringbuf); + stringbuf = NULL; + stringbuf_len_alloc = stringbuf_len = 0; +} === modified file 'storage/innodb_plugin/pars/pars0lex.l' --- a/storage/innodb_plugin/pars/pars0lex.l 2009-05-27 09:45:59 +0000 +++ b/storage/innodb_plugin/pars/pars0lex.l 2009-11-30 11:32:05 +0000 @@ -661,3 +661,16 @@ In the state 'id', only two actions are } %% + +/********************************************************************** +Release any resources used by the lexer. */ +UNIV_INTERN +void +pars_lexer_close(void) +/*==================*/ +{ + yylex_destroy(); + free(stringbuf); + stringbuf = NULL; + stringbuf_len_alloc = stringbuf_len = 0; +} === modified file 'storage/innodb_plugin/que/que0que.c' --- a/storage/innodb_plugin/que/que0que.c 2009-07-30 12:42:56 +0000 +++ b/storage/innodb_plugin/que/que0que.c 2009-11-30 11:32:05 +0000 @@ -518,6 +518,7 @@ que_graph_free_recursive( upd_node_t* upd; tab_node_t* cre_tab; ind_node_t* cre_ind; + purge_node_t* purge; if (node == NULL) { @@ -579,6 +580,13 @@ que_graph_free_recursive( mem_heap_free(ins->entry_sys_heap); break; + case QUE_NODE_PURGE: + purge = node; + + mem_heap_free(purge->heap); + + break; + case QUE_NODE_UPDATE: upd = node; === modified file 'storage/innodb_plugin/row/row0merge.c' --- a/storage/innodb_plugin/row/row0merge.c 2009-10-09 14:13:15 +0000 +++ b/storage/innodb_plugin/row/row0merge.c 2009-11-30 12:24:54 +0000 @@ -1200,6 +1200,12 @@ row_merge_read_clustered_index( in order to release the latch on the old page. */ if (btr_pcur_is_after_last_on_page(&pcur)) { + if (UNIV_UNLIKELY(trx_is_interrupted(trx))) { + i = 0; + err = DB_INTERRUPTED; + goto err_exit; + } + btr_pcur_store_position(&pcur, &mtr); mtr_commit(&mtr); mtr_start(&mtr); @@ -1557,6 +1563,7 @@ static __attribute__((nonnull)) ulint row_merge( /*======*/ + trx_t* trx, /*!< in: transaction */ const dict_index_t* index, /*!< in: index being created */ merge_file_t* file, /*!< in/out: file containing index entries */ @@ -1590,6 +1597,10 @@ row_merge( for (; foffs0 < ihalf && foffs1 < file->offset; foffs0++, foffs1++) { ulint ahalf; /*!< arithmetic half the input file */ + if (UNIV_UNLIKELY(trx_is_interrupted(trx))) { + return(DB_INTERRUPTED); + } + error = row_merge_blocks(index, file, block, &foffs0, &foffs1, &of, table); @@ -1617,6 +1628,10 @@ row_merge( /* Copy the last blocks, if there are any. */ while (foffs0 < ihalf) { + if (UNIV_UNLIKELY(trx_is_interrupted(trx))) { + return(DB_INTERRUPTED); + } + if (!row_merge_blocks_copy(index, file, block, &foffs0, &of)) { return(DB_CORRUPTION); } @@ -1625,6 +1640,10 @@ row_merge( ut_ad(foffs0 == ihalf); while (foffs1 < file->offset) { + if (UNIV_UNLIKELY(trx_is_interrupted(trx))) { + return(DB_INTERRUPTED); + } + if (!row_merge_blocks_copy(index, file, block, &foffs1, &of)) { return(DB_CORRUPTION); } @@ -1653,6 +1672,7 @@ static ulint row_merge_sort( /*===========*/ + trx_t* trx, /*!< in: transaction */ const dict_index_t* index, /*!< in: index being created */ merge_file_t* file, /*!< in/out: file containing index entries */ @@ -1671,7 +1691,8 @@ row_merge_sort( do { ulint error; - error = row_merge(index, file, &half, block, tmpfd, table); + error = row_merge(trx, index, file, &half, + block, tmpfd, table); if (error != DB_SUCCESS) { return(error); @@ -2490,7 +2511,7 @@ row_merge_build_indexes( sorting and inserting. */ for (i = 0; i < n_indexes; i++) { - error = row_merge_sort(indexes[i], &merge_files[i], + error = row_merge_sort(trx, indexes[i], &merge_files[i], block, &tmpfd, table); if (error == DB_SUCCESS) { === modified file 'storage/innodb_plugin/row/row0mysql.c' --- a/storage/innodb_plugin/row/row0mysql.c 2009-11-03 10:32:33 +0000 +++ b/storage/innodb_plugin/row/row0mysql.c 2009-11-30 13:13:34 +0000 @@ -1880,6 +1880,8 @@ err_exit: if (UNIV_UNLIKELY(err != DB_SUCCESS)) { trx->error_state = DB_SUCCESS; trx_general_rollback_for_mysql(trx, NULL); + /* TO DO: free table? The code below will dereference + table->name, though. */ } switch (err) { @@ -1898,31 +1900,6 @@ err_exit: break; case DB_DUPLICATE_KEY: - ut_print_timestamp(stderr); - fputs(" InnoDB: Error: table ", stderr); - ut_print_name(stderr, trx, TRUE, table->name); - fputs(" already exists in InnoDB internal\n" - "InnoDB: data dictionary. Have you deleted" - " the .frm file\n" - "InnoDB: and not used DROP TABLE?" - " Have you used DROP DATABASE\n" - "InnoDB: for InnoDB tables in" - " MySQL version <= 3.23.43?\n" - "InnoDB: See the Restrictions section" - " of the InnoDB manual.\n" - "InnoDB: You can drop the orphaned table" - " inside InnoDB by\n" - "InnoDB: creating an InnoDB table with" - " the same name in another\n" - "InnoDB: database and copying the .frm file" - " to the current database.\n" - "InnoDB: Then MySQL thinks the table exists," - " and DROP TABLE will\n" - "InnoDB: succeed.\n" - "InnoDB: You can look for further help from\n" - "InnoDB: " REFMAN "innodb-troubleshooting.html\n", - stderr); - /* We may also get err == DB_ERROR if the .ibd file for the table already exists */ @@ -4157,6 +4134,7 @@ row_check_table_for_mysql( } if (trx_is_interrupted(prebuilt->trx)) { + ret = DB_INTERRUPTED; break; } === modified file 'storage/innodb_plugin/srv/srv0srv.c' --- a/storage/innodb_plugin/srv/srv0srv.c 2009-10-09 12:19:13 +0000 +++ b/storage/innodb_plugin/srv/srv0srv.c 2009-11-30 11:32:05 +0000 @@ -1006,13 +1006,26 @@ srv_init(void) } /*********************************************************************//** -Frees the OS fast mutex created in srv_init(). */ +Frees the data structures created in srv_init(). */ UNIV_INTERN void srv_free(void) /*==========*/ { os_fast_mutex_free(&srv_conc_mutex); + mem_free(srv_conc_slots); + srv_conc_slots = NULL; + + mem_free(srv_sys->threads); + mem_free(srv_sys); + srv_sys = NULL; + + mem_free(kernel_mutex_temp); + kernel_mutex_temp = NULL; + mem_free(srv_mysql_table); + srv_mysql_table = NULL; + + trx_i_s_cache_free(trx_i_s_cache); } /*********************************************************************//** @@ -1024,6 +1037,8 @@ srv_general_init(void) /*==================*/ { ut_mem_init(); + /* Reset the system variables in the recovery module. */ + recv_sys_var_init(); os_sync_init(); sync_init(); mem_init(srv_mem_pool_size); === modified file 'storage/innodb_plugin/srv/srv0start.c' --- a/storage/innodb_plugin/srv/srv0start.c 2009-11-03 10:23:22 +0000 +++ b/storage/innodb_plugin/srv/srv0start.c 2009-11-30 11:32:05 +0000 @@ -103,6 +103,7 @@ Created 2/16/1996 Heikki Tuuri # include "row0row.h" # include "row0mysql.h" # include "btr0pcur.h" +# include "thr0loc.h" # include "os0sync.h" /* for INNODB_RW_LOCKS_USE_ATOMICS */ /** Log sequence number immediately after startup */ @@ -495,6 +496,8 @@ io_handler_thread( mutex_exit(&ios_mutex); } + thr_local_free(os_thread_get_curr_id()); + /* We count the number of threads in os_thread_exit(). A created thread should always use that to exit and not use return() to exit. The thread actually never comes here because it is exited in an @@ -531,32 +534,6 @@ srv_normalize_path_for_win( #endif } -/*********************************************************************//** -Adds a slash or a backslash to the end of a string if it is missing -and the string is not empty. -@return string which has the separator if the string is not empty */ -UNIV_INTERN -char* -srv_add_path_separator_if_needed( -/*=============================*/ - char* str) /*!< in: null-terminated character string */ -{ - char* out_str; - ulint len = ut_strlen(str); - - if (len == 0 || str[len - 1] == SRV_PATH_SEPARATOR) { - - return(str); - } - - out_str = ut_malloc(len + 2); - memcpy(out_str, str, len); - out_str[len] = SRV_PATH_SEPARATOR; - out_str[len + 1] = 0; - - return(out_str); -} - #ifndef UNIV_HOTBACKUP /*********************************************************************//** Calculates the low 32 bits when a file size which is given as a number @@ -605,19 +582,24 @@ open_or_create_log_file( ulint size; ulint size_high; char name[10000]; + ulint dirnamelen; UT_NOT_USED(create_new_db); *log_file_created = FALSE; srv_normalize_path_for_win(srv_log_group_home_dirs[k]); - srv_log_group_home_dirs[k] = srv_add_path_separator_if_needed( - srv_log_group_home_dirs[k]); - ut_a(strlen(srv_log_group_home_dirs[k]) - < (sizeof name) - 10 - sizeof "ib_logfile"); - sprintf(name, "%s%s%lu", srv_log_group_home_dirs[k], - "ib_logfile", (ulong) i); + dirnamelen = strlen(srv_log_group_home_dirs[k]); + ut_a(dirnamelen < (sizeof name) - 10 - sizeof "ib_logfile"); + memcpy(name, srv_log_group_home_dirs[k], dirnamelen); + + /* Add a path separator if needed. */ + if (dirnamelen && name[dirnamelen - 1] != SRV_PATH_SEPARATOR) { + name[dirnamelen++] = SRV_PATH_SEPARATOR; + } + + sprintf(name + dirnamelen, "%s%lu", "ib_logfile", (ulong) i); files[i] = os_file_create(name, OS_FILE_CREATE, OS_FILE_NORMAL, OS_LOG_FILE, &ret); @@ -780,14 +762,22 @@ open_or_create_data_files( *create_new_db = FALSE; srv_normalize_path_for_win(srv_data_home); - srv_data_home = srv_add_path_separator_if_needed(srv_data_home); for (i = 0; i < srv_n_data_files; i++) { + ulint dirnamelen; + srv_normalize_path_for_win(srv_data_file_names[i]); + dirnamelen = strlen(srv_data_home); - ut_a(strlen(srv_data_home) + strlen(srv_data_file_names[i]) + ut_a(dirnamelen + strlen(srv_data_file_names[i]) < (sizeof name) - 1); - sprintf(name, "%s%s", srv_data_home, srv_data_file_names[i]); + memcpy(name, srv_data_home, dirnamelen); + /* Add a path separator if needed. */ + if (dirnamelen && name[dirnamelen - 1] != SRV_PATH_SEPARATOR) { + name[dirnamelen++] = SRV_PATH_SEPARATOR; + } + + strcpy(name + dirnamelen, srv_data_file_names[i]); if (srv_data_file_is_raw_partition[i] == 0) { @@ -1009,7 +999,7 @@ skip_size_check: return(DB_SUCCESS); } -/****************************************************************//** +/******************************************************************** Starts InnoDB and creates a new database if database files are not found and the user wants. @return DB_SUCCESS or error code */ @@ -1120,7 +1110,7 @@ innobase_start_or_create_for_mysql(void) if (srv_start_has_been_called) { fprintf(stderr, - "InnoDB: Error:startup called second time" + "InnoDB: Error: startup called second time" " during the process lifetime.\n" "InnoDB: In the MySQL Embedded Server Library" " you cannot call server_init()\n" @@ -1959,8 +1949,10 @@ innobase_shutdown_for_mysql(void) /* All the threads have exited or are just exiting; NOTE that the threads may not have completed their exit yet. Should we use pthread_join() to make sure - they have exited? Now we just sleep 0.1 seconds and - hope that is enough! */ + they have exited? If we did, we would have to + remove the pthread_detach() from + os_thread_exit(). Now we just sleep 0.1 + seconds and hope that is enough! */ os_mutex_exit(os_sync_mutex); @@ -1999,37 +1991,41 @@ innobase_shutdown_for_mysql(void) srv_misc_tmpfile = 0; } + /* This must be disabled before closing the buffer pool + and closing the data dictionary. */ + btr_search_disable(); + + ibuf_close(); + log_shutdown(); + lock_sys_close(); + thr_local_close(); trx_sys_file_format_close(); + trx_sys_close(); mutex_free(&srv_monitor_file_mutex); mutex_free(&srv_dict_tmpfile_mutex); mutex_free(&srv_misc_tmpfile_mutex); + dict_close(); + btr_search_sys_free(); /* 3. Free all InnoDB's own mutexes and the os_fast_mutexes inside them */ + os_aio_free(); sync_close(); + srv_free(); + fil_close(); /* 4. Free the os_conc_mutex and all os_events and os_mutexes */ - srv_free(); os_sync_free(); - /* Check that all read views are closed except read view owned - by a purge. */ - - if (UT_LIST_GET_LEN(trx_sys->view_list) > 1) { - fprintf(stderr, - "InnoDB: Error: all read views were not closed" - " before shutdown:\n" - "InnoDB: %lu read views open \n", - UT_LIST_GET_LEN(trx_sys->view_list) - 1); - } - - /* 5. Free all allocated memory and the os_fast_mutex created in - ut0mem.c */ + /* 5. Free all allocated memory */ + pars_lexer_close(); + log_mem_free(); buf_pool_free(); ut_free_all_mem(); + mem_close(); if (os_thread_count != 0 || os_event_count != 0 @@ -2060,6 +2056,7 @@ innobase_shutdown_for_mysql(void) } srv_was_started = FALSE; + srv_start_has_been_called = FALSE; return((int) DB_SUCCESS); } === modified file 'storage/innodb_plugin/sync/sync0arr.c' --- a/storage/innodb_plugin/sync/sync0arr.c 2009-05-27 09:45:59 +0000 +++ b/storage/innodb_plugin/sync/sync0arr.c 2009-11-30 11:32:05 +0000 @@ -227,24 +227,21 @@ sync_array_create( SYNC_ARRAY_MUTEX: determines the type of mutex protecting the data structure */ { + ulint sz; sync_array_t* arr; - sync_cell_t* cell_array; - sync_cell_t* cell; - ulint i; ut_a(n_cells > 0); /* Allocate memory for the data structures */ arr = ut_malloc(sizeof(sync_array_t)); + memset(arr, 0x0, sizeof(*arr)); - cell_array = ut_malloc(sizeof(sync_cell_t) * n_cells); + sz = sizeof(sync_cell_t) * n_cells; + arr->array = ut_malloc(sz); + memset(arr->array, 0x0, sz); arr->n_cells = n_cells; - arr->n_reserved = 0; - arr->array = cell_array; arr->protection = protection; - arr->sg_count = 0; - arr->res_count = 0; /* Then create the mutex to protect the wait array complex */ if (protection == SYNC_ARRAY_OS_MUTEX) { @@ -255,13 +252,6 @@ sync_array_create( ut_error; } - for (i = 0; i < n_cells; i++) { - cell = sync_array_get_nth_cell(arr, i); - cell->wait_object = NULL; - cell->waiting = FALSE; - cell->signal_count = 0; - } - return(arr); } === modified file 'storage/innodb_plugin/sync/sync0sync.c' --- a/storage/innodb_plugin/sync/sync0sync.c 2009-10-12 12:00:56 +0000 +++ b/storage/innodb_plugin/sync/sync0sync.c 2009-11-30 11:32:05 +0000 @@ -1377,7 +1377,12 @@ sync_close(void) mutex_free(&mutex_list_mutex); #ifdef UNIV_SYNC_DEBUG mutex_free(&sync_thread_mutex); + + /* Switch latching order checks on in sync0sync.c */ + sync_order_checks_on = FALSE; #endif /* UNIV_SYNC_DEBUG */ + + sync_initialized = FALSE; } /*******************************************************************//** === modified file 'storage/innodb_plugin/thr/thr0loc.c' --- a/storage/innodb_plugin/thr/thr0loc.c 2009-10-08 10:00:49 +0000 +++ b/storage/innodb_plugin/thr/thr0loc.c 2009-11-30 11:32:05 +0000 @@ -246,3 +246,34 @@ thr_local_init(void) mutex_create(&thr_local_mutex, SYNC_THR_LOCAL); } + +/******************************************************************** +Close the thread local storage module. */ +UNIV_INTERN +void +thr_local_close(void) +/*=================*/ +{ + ulint i; + + ut_a(thr_local_hash != NULL); + + /* Free the hash elements. We don't remove them from the table + because we are going to destroy the table anyway. */ + for (i = 0; i < hash_get_n_cells(thr_local_hash); i++) { + thr_local_t* local; + + local = HASH_GET_FIRST(thr_local_hash, i); + + while (local) { + thr_local_t* prev_local = local; + + local = HASH_GET_NEXT(hash, prev_local); + ut_a(prev_local->magic_n == THR_LOCAL_MAGIC_N); + mem_free(prev_local); + } + } + + hash_table_free(thr_local_hash); + thr_local_hash = NULL; +} === modified file 'storage/innodb_plugin/trx/trx0i_s.c' --- a/storage/innodb_plugin/trx/trx0i_s.c 2009-05-27 09:45:59 +0000 +++ b/storage/innodb_plugin/trx/trx0i_s.c 2009-12-01 10:38:40 +0000 @@ -60,7 +60,7 @@ Created July 17, 2007 Vasil Dimov /** @brief The maximum number of chunks to allocate for a table cache. The rows of a table cache are stored in a set of chunks. When a new -row is added a new chunk is allocated if necessary. Assuming that the +row is added a new chunk is allocated if necessary. Assuming that the first one is 1024 rows (TABLE_CACHE_INITIAL_ROWSNUM) and each subsequent is N/2 where N is the number of rows we have allocated till now, then 39th chunk would accommodate 1677416425 rows and all chunks @@ -238,6 +238,27 @@ table_cache_init( } /*******************************************************************//** +Frees a table cache. */ +static +void +table_cache_free( +/*=============*/ + i_s_table_cache_t* table_cache) /*!< in/out: table cache */ +{ + ulint i; + + for (i = 0; i < MEM_CHUNKS_IN_TABLE_CACHE; i++) { + + /* the memory is actually allocated in + table_cache_create_empty_row() */ + if (table_cache->chunks[i].base) { + mem_free(table_cache->chunks[i].base); + table_cache->chunks[i].base = NULL; + } + } +} + +/*******************************************************************//** Returns an empty row from a table cache. The row is allocated if no more empty rows are available. The number of used rows is incremented. If the memory limit is hit then NULL is returned and nothing is @@ -1184,9 +1205,6 @@ trx_i_s_possibly_fetch_data_into_cache( return(1); } - /* We are going to access trx->query in all transactions */ - innobase_mysql_prepare_print_arbitrary_thd(); - /* We need to read trx_sys and record/table lock queues */ mutex_enter(&kernel_mutex); @@ -1194,8 +1212,6 @@ trx_i_s_possibly_fetch_data_into_cache( mutex_exit(&kernel_mutex); - innobase_mysql_end_print_arbitrary_thd(); - return(0); } @@ -1252,6 +1268,22 @@ trx_i_s_cache_init( } /*******************************************************************//** +Free the INFORMATION SCHEMA trx related cache. */ +UNIV_INTERN +void +trx_i_s_cache_free( +/*===============*/ + trx_i_s_cache_t* cache) /*!< in, own: cache to free */ +{ + hash_table_free(cache->locks_hash); + ha_storage_free(cache->storage); + table_cache_free(&cache->innodb_trx); + table_cache_free(&cache->innodb_locks); + table_cache_free(&cache->innodb_lock_waits); + memset(cache, 0, sizeof *cache); +} + +/*******************************************************************//** Issue a shared/read lock on the tables cache. */ UNIV_INTERN void === modified file 'storage/innodb_plugin/trx/trx0purge.c' --- a/storage/innodb_plugin/trx/trx0purge.c 2009-07-30 12:42:56 +0000 +++ b/storage/innodb_plugin/trx/trx0purge.c 2009-11-30 11:32:05 +0000 @@ -249,6 +249,44 @@ trx_purge_sys_create(void) purge_sys->heap); } +/************************************************************************ +Frees the global purge system control structure. */ +UNIV_INTERN +void +trx_purge_sys_close(void) +/*======================*/ +{ + ut_ad(!mutex_own(&kernel_mutex)); + + que_graph_free(purge_sys->query); + + ut_a(purge_sys->sess->trx->is_purge); + purge_sys->sess->trx->conc_state = TRX_NOT_STARTED; + sess_close(purge_sys->sess); + purge_sys->sess = NULL; + + if (purge_sys->view != NULL) { + /* Because acquiring the kernel mutex is a pre-condition + of read_view_close(). We don't really need it here. */ + mutex_enter(&kernel_mutex); + + read_view_close(purge_sys->view); + purge_sys->view = NULL; + + mutex_exit(&kernel_mutex); + } + + trx_undo_arr_free(purge_sys->arr); + + rw_lock_free(&purge_sys->latch); + mutex_free(&purge_sys->mutex); + + mem_heap_free(purge_sys->heap); + mem_free(purge_sys); + + purge_sys = NULL; +} + /*================ UNDO LOG HISTORY LIST =============================*/ /********************************************************************//** === modified file 'storage/innodb_plugin/trx/trx0rseg.c' --- a/storage/innodb_plugin/trx/trx0rseg.c 2009-05-27 09:45:59 +0000 +++ b/storage/innodb_plugin/trx/trx0rseg.c 2009-11-30 11:32:05 +0000 @@ -132,6 +132,49 @@ trx_rseg_header_create( } /***********************************************************************//** +Free's an instance of the rollback segment in memory. */ +UNIV_INTERN +void +trx_rseg_mem_free( +/*==============*/ + trx_rseg_t* rseg) /* in, own: instance to free */ +{ + trx_undo_t* undo; + + mutex_free(&rseg->mutex); + + /* There can't be any active transactions. */ + ut_a(UT_LIST_GET_LEN(rseg->update_undo_list) == 0); + ut_a(UT_LIST_GET_LEN(rseg->insert_undo_list) == 0); + + undo = UT_LIST_GET_FIRST(rseg->update_undo_cached); + + while (undo != NULL) { + trx_undo_t* prev_undo = undo; + + undo = UT_LIST_GET_NEXT(undo_list, undo); + UT_LIST_REMOVE(undo_list, rseg->update_undo_cached, prev_undo); + + trx_undo_mem_free(prev_undo); + } + + undo = UT_LIST_GET_FIRST(rseg->insert_undo_cached); + + while (undo != NULL) { + trx_undo_t* prev_undo = undo; + + undo = UT_LIST_GET_NEXT(undo_list, undo); + UT_LIST_REMOVE(undo_list, rseg->insert_undo_cached, prev_undo); + + trx_undo_mem_free(prev_undo); + } + + trx_sys_set_nth_rseg(trx_sys, rseg->id, NULL); + + mem_free(rseg); +} + +/*************************************************************************** Creates and initializes a rollback segment object. The values for the fields are read from the header. The object is inserted to the rseg list of the trx system object and a pointer is inserted in the rseg === modified file 'storage/innodb_plugin/trx/trx0sys.c' --- a/storage/innodb_plugin/trx/trx0sys.c 2009-07-30 12:42:56 +0000 +++ b/storage/innodb_plugin/trx/trx0sys.c 2009-11-30 11:32:05 +0000 @@ -40,6 +40,7 @@ Created 3/26/1996 Heikki Tuuri #include "trx0purge.h" #include "log0log.h" #include "os0file.h" +#include "read0read.h" /** The file format tag structure with id and name. */ struct file_format_struct { @@ -1533,3 +1534,80 @@ trx_sys_file_format_id_to_name( } #endif /* !UNIV_HOTBACKUP */ + +/********************************************************************* +Shutdown/Close the transaction system. */ +UNIV_INTERN +void +trx_sys_close(void) +/*===============*/ +{ + trx_rseg_t* rseg; + read_view_t* view; + + ut_ad(trx_sys != NULL); + + /* Check that all read views are closed except read view owned + by a purge. */ + + if (UT_LIST_GET_LEN(trx_sys->view_list) > 1) { + fprintf(stderr, + "InnoDB: Error: all read views were not closed" + " before shutdown:\n" + "InnoDB: %lu read views open \n", + UT_LIST_GET_LEN(trx_sys->view_list) - 1); + } + + sess_close(trx_dummy_sess); + trx_dummy_sess = NULL; + + trx_purge_sys_close(); + + mutex_enter(&kernel_mutex); + + /* Free the double write data structures. */ + ut_a(trx_doublewrite != NULL); + ut_free(trx_doublewrite->write_buf_unaligned); + trx_doublewrite->write_buf_unaligned = NULL; + + mem_free(trx_doublewrite->buf_block_arr); + trx_doublewrite->buf_block_arr = NULL; + + mutex_free(&trx_doublewrite->mutex); + mem_free(trx_doublewrite); + trx_doublewrite = NULL; + + /* There can't be any active transactions. */ + rseg = UT_LIST_GET_FIRST(trx_sys->rseg_list); + + while (rseg != NULL) { + trx_rseg_t* prev_rseg = rseg; + + rseg = UT_LIST_GET_NEXT(rseg_list, prev_rseg); + UT_LIST_REMOVE(rseg_list, trx_sys->rseg_list, prev_rseg); + + trx_rseg_mem_free(prev_rseg); + } + + view = UT_LIST_GET_FIRST(trx_sys->view_list); + + while (view != NULL) { + read_view_t* prev_view = view; + + view = UT_LIST_GET_NEXT(view_list, prev_view); + + /* Views are allocated from the trx_sys->global_read_view_heap. + So, we simply remove the element here. */ + UT_LIST_REMOVE(view_list, trx_sys->view_list, prev_view); + } + + ut_a(UT_LIST_GET_LEN(trx_sys->trx_list) == 0); + ut_a(UT_LIST_GET_LEN(trx_sys->rseg_list) == 0); + ut_a(UT_LIST_GET_LEN(trx_sys->view_list) == 0); + ut_a(UT_LIST_GET_LEN(trx_sys->mysql_trx_list) == 0); + + mem_free(trx_sys); + + trx_sys = NULL; + mutex_exit(&kernel_mutex); +} === modified file 'storage/innodb_plugin/trx/trx0trx.c' --- a/storage/innodb_plugin/trx/trx0trx.c 2009-10-09 14:13:15 +0000 +++ b/storage/innodb_plugin/trx/trx0trx.c 2009-12-01 10:38:40 +0000 @@ -1636,9 +1636,7 @@ trx_mark_sql_stat_end( /**********************************************************************//** Prints info about a transaction to the given file. The caller must own the -kernel mutex and must have called -innobase_mysql_prepare_print_arbitrary_thd(), unless he knows that MySQL -or InnoDB cannot meanwhile change the info printed here. */ +kernel mutex. */ UNIV_INTERN void trx_print( === modified file 'storage/innodb_plugin/trx/trx0undo.c' --- a/storage/innodb_plugin/trx/trx0undo.c 2009-07-30 12:42:56 +0000 +++ b/storage/innodb_plugin/trx/trx0undo.c 2009-11-30 11:32:05 +0000 @@ -1522,7 +1522,7 @@ trx_undo_mem_init_for_reuse( /********************************************************************//** Frees an undo log memory copy. */ -static +UNIV_INTERN void trx_undo_mem_free( /*==============*/ === modified file 'storage/innodb_plugin/usr/usr0sess.c' --- a/storage/innodb_plugin/usr/usr0sess.c 2009-05-27 09:45:59 +0000 +++ b/storage/innodb_plugin/usr/usr0sess.c 2009-11-30 11:32:05 +0000 @@ -32,14 +32,6 @@ Created 6/25/1996 Heikki Tuuri #include "trx0trx.h" /*********************************************************************//** -Closes a session, freeing the memory occupied by it. */ -static -void -sess_close( -/*=======*/ - sess_t* sess); /*!< in, own: session object */ - -/*********************************************************************//** Opens a session. @return own: session object */ UNIV_INTERN @@ -64,35 +56,16 @@ sess_open(void) /*********************************************************************//** Closes a session, freeing the memory occupied by it. */ -static +UNIV_INTERN void sess_close( /*=======*/ sess_t* sess) /*!< in, own: session object */ { - ut_ad(mutex_own(&kernel_mutex)); - ut_ad(sess->trx == NULL); - - mem_free(sess); -} - -/*********************************************************************//** -Closes a session, freeing the memory occupied by it, if it is in a state -where it should be closed. -@return TRUE if closed */ -UNIV_INTERN -ibool -sess_try_close( -/*===========*/ - sess_t* sess) /*!< in, own: session object */ -{ - ut_ad(mutex_own(&kernel_mutex)); + ut_ad(!mutex_own(&kernel_mutex)); - if (UT_LIST_GET_LEN(sess->graphs) == 0) { - sess_close(sess); + ut_a(UT_LIST_GET_LEN(sess->graphs) == 0); - return(TRUE); - } - - return(FALSE); + trx_free_for_background(sess->trx); + mem_free(sess); } === modified file 'storage/innodb_plugin/ut/ut0mem.c' --- a/storage/innodb_plugin/ut/ut0mem.c 2009-07-30 12:42:56 +0000 +++ b/storage/innodb_plugin/ut/ut0mem.c 2009-11-30 11:32:05 +0000 @@ -433,6 +433,8 @@ ut_free_all_mem(void) " total allocated memory is %lu\n", (ulong) ut_total_allocated_memory); } + + ut_mem_block_list_inited = FALSE; } #endif /* !UNIV_HOTBACKUP */ === modified file 'storage/maria/ma_check.c' --- a/storage/maria/ma_check.c 2009-11-29 23:08:56 +0000 +++ b/storage/maria/ma_check.c 2010-01-14 16:51:00 +0000 @@ -6018,7 +6018,7 @@ int maria_update_state_info(HA_CHECK *pa { if (update & UPDATE_TIME) { - share->state.check_time= (long) time((time_t*) 0); + share->state.check_time= time((time_t*) 0); if (!share->state.create_time) share->state.create_time= share->state.check_time; } === modified file 'storage/maria/ma_create.c' --- a/storage/maria/ma_create.c 2009-02-19 09:01:25 +0000 +++ b/storage/maria/ma_create.c 2010-01-14 16:51:00 +0000 @@ -772,7 +772,7 @@ int maria_create(const char *name, enum share.base.min_block_length= share.base.pack_reclength; if (! (flags & HA_DONT_TOUCH_DATA)) - share.state.create_time= (long) time((time_t*) 0); + share.state.create_time= time((time_t*) 0); pthread_mutex_lock(&THR_LOCK_maria); === modified file 'storage/myisam/ft_boolean_search.c' --- a/storage/myisam/ft_boolean_search.c 2009-11-30 13:36:06 +0000 +++ b/storage/myisam/ft_boolean_search.c 2010-01-15 15:27:55 +0000 @@ -475,8 +475,7 @@ static void _ftb_init_index_search(FT_IN int i; FTB_WORD *ftbw; - if ((ftb->state != READY && ftb->state !=INDEX_DONE) || - ftb->keynr == NO_SUCH_KEY) + if (ftb->state == UNINITIALIZED || ftb->keynr == NO_SUCH_KEY) return; ftb->state=INDEX_SEARCH; === modified file 'storage/myisam/mi_check.c' --- a/storage/myisam/mi_check.c 2009-12-03 11:34:11 +0000 +++ b/storage/myisam/mi_check.c 2010-01-14 16:51:00 +0000 @@ -4447,7 +4447,7 @@ int update_state_info(HA_CHECK *param, M { if (update & UPDATE_TIME) { - share->state.check_time= (long) time((time_t*) 0); + share->state.check_time= time((time_t*) 0); if (!share->state.create_time) share->state.create_time=share->state.check_time; } === modified file 'storage/myisam/mi_create.c' --- a/storage/myisam/mi_create.c 2009-10-15 21:38:29 +0000 +++ b/storage/myisam/mi_create.c 2010-01-14 16:51:00 +0000 @@ -575,7 +575,7 @@ int mi_create(const char *name,uint keys max(share.base.pack_reclength,MI_MIN_BLOCK_LENGTH) : MI_EXTEND_BLOCK_LENGTH; if (! (flags & HA_DONT_TOUCH_DATA)) - share.state.create_time= (long) time((time_t*) 0); + share.state.create_time= time((time_t*) 0); pthread_mutex_lock(&THR_LOCK_myisam); === modified file 'storage/xtradb/CMakeLists.txt' --- a/storage/xtradb/CMakeLists.txt 2009-11-14 09:53:18 +0000 +++ b/storage/xtradb/CMakeLists.txt 2010-01-15 15:58:25 +0000 @@ -13,36 +13,41 @@ # along with this program; if not, write to the Free Software # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA -IF (CMAKE_SIZEOF_VOID_P MATCHES 8) - SET(WIN64 TRUE) -ENDIF (CMAKE_SIZEOF_VOID_P MATCHES 8) +# This is the CMakeLists for InnoDB Plugin -# Check type sizes -include(CheckTypeSize) - -# Currently, the checked results are not used. -CHECK_TYPE_SIZE(int SIZEOF_INT) -CHECK_TYPE_SIZE(long SIZEOF_LONG) -CHECK_TYPE_SIZE(void* SIZEOF_VOID_P) SET(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} -DSAFEMALLOC -DSAFE_MUTEX") SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} -DSAFEMALLOC -DSAFE_MUTEX") -INCLUDE("${PROJECT_SOURCE_DIR}/storage/mysql_storage_engine.cmake") + +# Starting at 5.1.38, MySQL CMake files are simplified. But the plugin +# CMakeLists.txt still needs to work with previous versions of MySQL. +IF (MYSQL_VERSION_ID GREATER "50137") + INCLUDE("${PROJECT_SOURCE_DIR}/storage/mysql_storage_engine.cmake") +ENDIF (MYSQL_VERSION_ID GREATER "50137") + +IF (CMAKE_SIZEOF_VOID_P MATCHES 8) + SET(WIN64 TRUE) +ENDIF (CMAKE_SIZEOF_VOID_P MATCHES 8) + ADD_DEFINITIONS(-D_WIN32 -D_LIB -DMYSQL_SERVER) -INCLUDE_DIRECTORIES(${CMAKE_SOURCE_DIR}/include ${CMAKE_SOURCE_DIR}/zlib - ${CMAKE_SOURCE_DIR}/storage/xtradb/include - ${CMAKE_SOURCE_DIR}/storage/xtradb/handler - ${CMAKE_SOURCE_DIR}/sql - ${CMAKE_SOURCE_DIR}/regex - ${CMAKE_SOURCE_DIR}/extra/yassl/include) +# Include directories under innobase +INCLUDE_DIRECTORIES(${CMAKE_SOURCE_DIR}/storage/xtradb/include + ${CMAKE_SOURCE_DIR}/storage/xtradb/handler) + +# Include directories under mysql +INCLUDE_DIRECTORIES(${CMAKE_SOURCE_DIR}/include + ${CMAKE_SOURCE_DIR}/sql + ${CMAKE_SOURCE_DIR}/regex + ${CMAKE_SOURCE_DIR}/zlib + ${CMAKE_SOURCE_DIR}/extra/yassl/include) # Removing compiler optimizations for innodb/mem/* files on 64-bit Windows # due to 64-bit compiler error, See MySQL Bug #19424, #36366, #34297 -IF(MSVC AND $(WIN64)) +IF (MSVC AND $(WIN64)) SET_SOURCE_FILES_PROPERTIES(mem/mem0mem.c mem/mem0pool.c PROPERTIES COMPILE_FLAGS -Od) -ENDIF(MSVC AND $(WIN64)) +ENDIF (MSVC AND $(WIN64)) SET(INNOBASE_SOURCES btr/btr0btr.c btr/btr0cur.c btr/btr0pcur.c btr/btr0sea.c buf/buf0buddy.c buf/buf0buf.c buf/buf0flu.c buf/buf0lru.c buf/buf0rea.c @@ -77,5 +82,20 @@ SET(INNOBASE_SOURCES btr/btr0btr.c btr/b usr/usr0sess.c ut/ut0byte.c ut/ut0dbg.c ut/ut0mem.c ut/ut0rnd.c ut/ut0ut.c ut/ut0vec.c ut/ut0list.c ut/ut0wqueue.c) +ADD_DEFINITIONS(-DHAVE_WINDOWS_ATOMICS -DIB_HAVE_PAUSE_INSTRUCTION) -MYSQL_STORAGE_ENGINE(INNOBASE) +IF (MYSQL_VERSION_ID GREATER "50137") + MYSQL_STORAGE_ENGINE(INNOBASE) + # Use ha_innodb for plugin name, if plugin is built + GET_TARGET_PROPERTY(LIB_LOCATION ha_innobase LOCATION) + IF(LIB_LOCATION) + SET_TARGET_PROPERTIES(ha_innobase PROPERTIES OUTPUT_NAME ha_innodb) + ENDIF(LIB_LOCATION) +ELSE (MYSQL_VERSION_ID GREATER "50137") + IF (NOT SOURCE_SUBLIBS) + ADD_DEFINITIONS(-D_WIN32 -DMYSQL_SERVER) + ADD_LIBRARY(innobase STATIC ${INNOBASE_SOURCES}) + # Require mysqld_error.h, which is built as part of the GenError + ADD_DEPENDENCIES(innobase GenError) + ENDIF (NOT SOURCE_SUBLIBS) +ENDIF (MYSQL_VERSION_ID GREATER "50137") === modified file 'storage/xtradb/ChangeLog' --- a/storage/xtradb/ChangeLog 2009-11-13 21:26:08 +0000 +++ b/storage/xtradb/ChangeLog 2010-01-15 15:58:25 +0000 @@ -1,3 +1,319 @@ +2009-11-20 The InnoDB Team + + * handler/ha_innodb.cc: + Add a workaround to prevent a crash due to Bug#45961 DDL on + partitioned innodb tables leaves data dictionary in an inconsistent + state + +2009-11-19 The InnoDB Team + + * btr/btr0btr.c: + Fix Bug#48469 when innodb tablespace is configured too small, crash + and corruption! + +2009-11-19 The InnoDB Team + + * data/data0type.c: + Fix Bug#48526 Data type for float and double is incorrectly reported + in InnoDB table monitor + +2009-11-19 The InnoDB Team + + * CMakeLists.txt: + Fix Bug#48317 cannot build innodb as static library + +2009-11-18 The InnoDB Team + + * handler/handler0alter.cc: + Fix Bug#48782 On lock wait timeout, CREATE INDEX (creating primary key) + attempts DROP TABLE + +2009-11-17 The InnoDB Team + + * handler/ha_innodb.cc, mysql-test/innodb.result, + mysql-test/innodb.test, mysql-test/innodb_bug44369.result, + mysql-test/innodb_bug44369.test, mysql-test/patches/innodb-index.diff, + row/row0mysql.c: + Report duplicate table names to the client connection, not to the + error log. + +2009-11-12 The InnoDB Team + + * handler/ha_innodb.cc, include/db0err.h, row/row0merge.c, + row/row0mysql.c: + Allow CREATE INDEX to be interrupted. + Also, when CHECK TABLE is interrupted, report ER_QUERY_INTERRUPTED. + +2009-11-11 The InnoDB Team + + * handler/ha_innodb.cc, mysql-test/innodb_bug47167.result, + mysql-test/innodb_bug47167.test, mysql-test/innodb_file_format.result: + Fix Bug#47167 "set global innodb_file_format_check" cannot set value + by User-Defined Variable + +2009-11-11 The InnoDB Team + + * include/os0file.h, os/os0file.c: + Fix Bug#3139 Mysql crashes: 'windows error 995' after several selects + on a large DB + +2009-11-04 The InnoDB Team + + * handler/ha_innodb.cc: + Fix Bug#32430 'show innodb status' causes errors + Invalid (old?) table or database name in logs + +2009-11-02 The InnoDB Team + + * btr/btr0sea.c, buf/buf0buf.c, dict/dict0dict.c, fil/fil0fil.c, + ibuf/ibuf0ibuf.c, include/btr0sea.h, include/dict0dict.h, + include/fil0fil.h, include/ibuf0ibuf.h, include/lock0lock.h, + include/log0log.h, include/log0recv.h, include/mem0mem.h, + include/mem0pool.h, include/os0file.h, include/pars0pars.h, + include/srv0srv.h, include/thr0loc.h, include/trx0i_s.h, + include/trx0purge.h, include/trx0rseg.h, include/trx0sys.h, + include/trx0undo.h, include/usr0sess.h, lock/lock0lock.c, + log/log0log.c, log/log0recv.c, mem/mem0dbg.c, mem/mem0pool.c, + os/os0file.c, os/os0sync.c, os/os0thread.c, pars/lexyy.c, + pars/pars0lex.l, que/que0que.c, srv/srv0srv.c, srv/srv0start.c, + sync/sync0arr.c, sync/sync0sync.c, thr/thr0loc.c, trx/trx0i_s.c, + trx/trx0purge.c, trx/trx0rseg.c, trx/trx0sys.c, trx/trx0undo.c, + usr/usr0sess.c, ut/ut0mem.c: + Fix Bug #45992 innodb memory not freed after shutdown + Fix Bug #46656 InnoDB plugin: memory leaks (Valgrind) + +2009-10-29 The InnoDB Team + + * handler/ha_innodb.cc, mysql-test/innodb-autoinc.result, + mysql-test/innodb-autoinc.test: + Fix Bug#47125 auto_increment start value is ignored if an index is + created and engine=innodb + +2009-10-29 The InnoDB Team + + * handler/ha_innodb.cc, mysql-test/innodb_bug47777.result, + mysql-test/innodb_bug47777.test: + Fix Bug#47777 innodb dies with spatial pk: Failing assertion: buf <= + original_buf + buf_len + +2009-10-29 The InnoDB Team + + * handler/ha_innodb.cc: + Fix Bug#38996 Race condition in ANALYZE TABLE + +2009-10-29 The InnoDB Team + + * handler/ha_innodb.cc: + Fix bug#42383: Can't create table 'test.bug39438' + +2009-10-29 The InnoDB Team + + * os/os0proc.c: + Fix Bug#48237 Error handling in os_mem_alloc_large appears to + be incorrect + +2009-10-29 The InnoDB Team + + * buf/buf0buf.c, buf/buf0lru.c, include/buf0buf.h, include/buf0buf.ic: + Fix corruption of the buf_pool->LRU_old list and improve debug + assertions. + +2009-10-28 The InnoDB Team + + * srv/srv0start.c: + Fix Bug#41490 After enlargement of InnoDB page size, the error message + become inaccurate + +2009-10-26 The InnoDB Team + + * row/row0ins.c: + When allocating a data tuple, zero out the system fields in order + to avoid Valgrind warnings about uninitialized fields in + dtuple_validate(). + +2009-10-22 The InnoDB Team + + * handler/ha_innodb.cc, mysql-test/innodb-zip.result, + mysql-test/innodb-zip.test, mysql-test/innodb_bug44369.result, + mysql-test/innodb_bug44369.test: + Fix Bug#47233 Innodb calls push_warning(MYSQL_ERROR::WARN_LEVEL_ERROR) + +2009-10-19 The InnoDB Team + + * mysql-test/innodb_information_schema.test: + Fix Bug#47808 innodb_information_schema.test fails when run under + valgrind + +2009-10-15 The InnoDB Team + + * include/page0page.ic: + Fix Bug#47058 Failure to compile innodb_plugin on solaris 10u7 + spro + cc/CC 5.10 + +2009-10-13 The InnoDB Team + + * buf/buf0flu.c: + Call fsync() on datafiles after a batch of pages is written to disk + even when skip_innodb_doublewrite is set. + +2009-10-05 The InnoDB Team + + * buf/buf0buf.c: + Do not invalidate buffer pool while an LRU batch is active. Added code + to buf_pool_invalidate() to wait for the running batches to finish. + +2009-10-01 The InnoDB Team + + * handler/ha_innodb.cc: + Fix Bug#47763 typo in error message: Failed to open table %s after %lu + attemtps. + +2009-10-01 The InnoDB Team + + * fsp/fsp0fsp.c, row/row0merge.c: + Clean up after a crash during DROP INDEX. When InnoDB crashes + while dropping an index, ensure that the index will be completely + dropped during crash recovery. The MySQL .frm file may still + contain the dropped index, but there is little that we can do + about it. + +2009-09-28 The InnoDB Team + + * handler/ha_innodb.cc: + When a secondary index exists in the MySQL .frm file but not in + the InnoDB data dictionary, return an error instead of letting an + assertion fail in index_read. + +2009-09-28 The InnoDB Team + + * btr/btr0btr.c, buf/buf0buf.c, include/page0page.h, + include/page0zip.h, page/page0cur.c, page/page0page.c, + page/page0zip.c: + Do not write to PAGE_INDEX_ID when restoring an uncompressed page + after a compression failure. The field should only be written + when creating a B-tree page. This fix addresses a race condition + in a debug assertion. + +2009-09-28 The InnoDB Team + + * fil/fil0fil.c: + Try to prevent the reuse of tablespace identifiers after InnoDB + has crashed during table creation. Also, refuse to start if files + with duplicate tablespace identifiers are encountered. + +2009-09-25 The InnoDB Team + + * include/os0file.h, os/os0file.c: + Fix Bug#47055 unconditional exit(1) on ERROR_WORKING_SET_QUOTA + 1453 (0x5AD) for InnoDB backend + +2009-09-19 The InnoDB Team + + * handler/ha_innodb.cc, mysql-test/innodb-consistent-master.opt, + mysql-test/innodb-consistent.result, + mysql-test/innodb-consistent.test: + Fix Bug#37232 Innodb might get too many read locks for DML with + repeatable-read + +2009-09-19 The InnoDB Team + + * fsp/fsp0fsp.c: + Fix Bug#31183 Tablespace full problems not reported in error log, + error message unclear + +2009-09-17 The InnoDB Team + + * mysql-test/innodb-zip.result, mysql-test/innodb-zip.test: + Make the test pass with zlib 1.2.3.3. Apparently, the definition + of compressBound() has changed between zlib versions, and the + maximum record size of a table with 1K compressed page size has + been reduced by one byte. This is an arbitrary test. In practical + applications, for good write performance, the compressed page size + should be chosen to be bigger than the absolute minimum. + +2009-09-16 The InnoDB Team + + * handler/ha_innodb.cc: + Fix Bug#46256 drop table with unknown collation crashes innodb + +2009-09-16 The InnoDB Team + + * dict/dict0dict.c, handler/ha_innodb.cc, + mysql-test/innodb_bug44369.result, mysql-test/innodb_bug44369.test, + row/row0mysql.c: + Fix Bug#44369 InnoDB: Does not uniformly disallow disallowed column + names + +2009-09-16 The InnoDB Team + + * handler/ha_innodb.cc, include/db0err.h, + mysql-test/innodb_bug46000.result, mysql-test/innodb_bug46000.test: + Fix Bug#46000 using index called GEN_CLUST_INDEX crashes server + +2009-09-02 The InnoDB Team + + * include/lock0lock.h, include/row0mysql.h, lock/lock0lock.c, + row/row0mysql.c: + Fix a regression introduced by the fix for MySQL bug#26316. We check + whether a transaction holds any AUTOINC locks before we acquire + the kernel mutex and release those locks. + +2009-08-27 The InnoDB Team + + * dict/dict0dict.c, include/dict0dict.h, + mysql-test/innodb_bug44571.result, mysql-test/innodb_bug44571.test: + Fix Bug#44571 InnoDB Plugin crashes on ADD INDEX + +2009-08-27 The InnoDB Team + + * row/row0merge.c: + Fix a bug in the merge sort that can corrupt indexes in fast index + creation. Add some consistency checks. Check that the number of + records remains constant in every merge sort pass. + +2009-08-27 The InnoDB Team + + * buf/buf0buf.c, buf/buf0lru.c, buf/buf0rea.c, handler/ha_innodb.cc, + include/buf0buf.h, include/buf0buf.ic, include/buf0lru.h, + include/ut0ut.h, ut/ut0ut.c: + Make it possible to tune the buffer pool LRU eviction policy to be + more resistant against index scans. Introduce the settable global + variables innodb_old_blocks_pct and innodb_old_blocks_time for + controlling the buffer pool eviction policy. The parameter + innodb_old_blocks_pct (5..95) controls the desired amount of "old" + blocks in the LRU list. The default is 37, corresponding to the + old fixed ratio of 3/8. Each time a block is accessed, it will be + moved to the "new" blocks if its first access was at least + innodb_old_blocks_time milliseconds ago (default 0, meaning every + block). The idea is that in index scans, blocks will be accessed + a few times within innodb_old_blocks_time, and they will remain in + the "old" section of the LRU list. Thus, when innodb_old_blocks_time + is nonzero, blocks retrieved for one-time index scans will be more + likely candidates for eviction than blocks that are accessed in + random patterns. + +2009-08-26 The InnoDB Team + + * handler/ha_innodb.cc, os/os0file.c: + Fix Bug#42885 buf_read_ahead_random, buf_read_ahead_linear counters, + thread wakeups + +2009-08-20 The InnoDB Team + + * lock/lock0lock.c: + Fix Bug#46650 Innodb assertion autoinc_lock == lock in + lock_table_remove_low on INSERT SELECT + +2009-08-13 The InnoDB Team + + * handler/handler0alter.cc: + Fix Bug#46657 InnoDB plugin: invalid read in index_merge_innodb test + (Valgrind) + +2009-08-11 The InnoDB Team + + InnoDB Plugin 1.0.4 released + 2009-07-20 The InnoDB Team * buf/buf0rea.c, handler/ha_innodb.cc, include/srv0srv.h, === modified file 'storage/xtradb/Makefile.am' --- a/storage/xtradb/Makefile.am 2009-11-13 22:53:04 +0000 +++ b/storage/xtradb/Makefile.am 2010-01-15 15:58:25 +0000 @@ -22,7 +22,7 @@ MYSQLLIBdir= $(pkglibdir) pkgplugindir= $(pkglibdir)/plugin INCLUDES= -I$(top_srcdir)/include -I$(top_builddir)/include \ -I$(top_srcdir)/regex \ - -I$(top_srcdir)/storage/xtradb/include \ + -I$(srcdir)/include \ -I$(top_srcdir)/sql \ -I$(srcdir) @ZLIB_INCLUDES@ @@ -31,7 +31,6 @@ DEFS= @DEFS@ noinst_HEADERS= \ handler/ha_innodb.h \ - handler/handler0vars.h \ handler/i_s.h \ include/btr0btr.h \ include/btr0btr.ic \ === modified file 'storage/xtradb/btr/btr0btr.c' --- a/storage/xtradb/btr/btr0btr.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/btr/btr0btr.c 2010-01-06 12:00:14 +0000 @@ -790,14 +790,21 @@ btr_create( } else { /* It is a non-ibuf tree: create a file segment for leaf pages */ - fseg_create(space, page_no, - PAGE_HEADER + PAGE_BTR_SEG_LEAF, mtr); + if (!fseg_create(space, page_no, + PAGE_HEADER + PAGE_BTR_SEG_LEAF, mtr)) { + /* Not enough space for new segment, free root + segment before return. */ + btr_free_root(space, zip_size, page_no, mtr); + + return(FIL_NULL); + } + /* The fseg create acquires a second latch on the page, therefore we must declare it: */ buf_block_dbg_add_level(block, SYNC_TREE_NODE_NEW); } - /* Create a new index page on the the allocated segment page */ + /* Create a new index page on the allocated segment page */ page_zip = buf_block_get_page_zip(block); if (UNIV_LIKELY_NULL(page_zip)) { @@ -1011,7 +1018,26 @@ btr_page_reorganize_low( (!page_zip_compress(page_zip, page, index, NULL))) { /* Restore the old page and exit. */ - buf_frame_copy(page, temp_page); + +#if defined UNIV_DEBUG || defined UNIV_ZIP_DEBUG + /* Check that the bytes that we skip are identical. */ + ut_a(!memcmp(page, temp_page, PAGE_HEADER)); + ut_a(!memcmp(PAGE_HEADER + PAGE_N_RECS + page, + PAGE_HEADER + PAGE_N_RECS + temp_page, + PAGE_DATA - (PAGE_HEADER + PAGE_N_RECS))); + ut_a(!memcmp(UNIV_PAGE_SIZE - FIL_PAGE_DATA_END + page, + UNIV_PAGE_SIZE - FIL_PAGE_DATA_END + temp_page, + FIL_PAGE_DATA_END)); +#endif /* UNIV_DEBUG || UNIV_ZIP_DEBUG */ + + memcpy(PAGE_HEADER + page, PAGE_HEADER + temp_page, + PAGE_N_RECS - PAGE_N_DIR_SLOTS); + memcpy(PAGE_DATA + page, PAGE_DATA + temp_page, + UNIV_PAGE_SIZE - PAGE_DATA - FIL_PAGE_DATA_END); + +#if defined UNIV_DEBUG || defined UNIV_ZIP_DEBUG + ut_a(!memcmp(page, temp_page, UNIV_PAGE_SIZE)); +#endif /* UNIV_DEBUG || UNIV_ZIP_DEBUG */ goto func_exit; } @@ -1902,7 +1928,7 @@ func_start: n_uniq, &heap); /* If the new record is less than the existing record - the the split in the middle will copy the existing + the split in the middle will copy the existing record to the new node. */ if (cmp_dtuple_rec(tuple, first_rec, offsets) < 0) { split_rec = page_get_middle_rec(page); === modified file 'storage/xtradb/btr/btr0sea.c' --- a/storage/xtradb/btr/btr0sea.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/btr/btr0sea.c 2010-01-06 12:00:14 +0000 @@ -175,6 +175,21 @@ btr_search_sys_create( btr_search_sys->hash_index = ha_create(hash_size, 0, 0); } +/*****************************************************************//** +Frees the adaptive search system at a database shutdown. */ +UNIV_INTERN +void +btr_search_sys_free(void) +/*=====================*/ +{ + mem_free(btr_search_latch_temp); + btr_search_latch_temp = NULL; + mem_heap_free(btr_search_sys->hash_index->heap); + hash_table_free(btr_search_sys->hash_index); + mem_free(btr_search_sys); + btr_search_sys = NULL; +} + /********************************************************************//** Disable the adaptive hash search system and empty the index. */ UNIV_INTERN @@ -957,7 +972,7 @@ btr_search_guess_on_hash( /* Increment the page get statistics though we did not really fix the page: for user info only */ - buf_pool->n_page_gets++; + buf_pool->stat.n_page_gets++; return(TRUE); === modified file 'storage/xtradb/buf/buf0buddy.c' --- a/storage/xtradb/buf/buf0buddy.c 2009-11-13 21:26:08 +0000 +++ b/storage/xtradb/buf/buf0buddy.c 2010-01-15 15:58:25 +0000 @@ -531,11 +531,10 @@ buf_buddy_relocate( UNIV_MEM_ASSERT_W(src, size); mutex = buf_page_get_mutex_enter(bpage); - ut_a(mutex); mutex_enter(&zip_free_mutex); - if (buf_page_can_relocate(bpage)) { + if (mutex && buf_page_can_relocate(bpage)) { /* Relocate the compressed page. */ ut_a(bpage->zip.data == src); memcpy(dst, src, size); @@ -563,7 +562,9 @@ success: rw_lock_x_unlock(&page_hash_latch); } - mutex_exit(mutex); + if (mutex) { + mutex_exit(mutex); + } } else if (i == buf_buddy_get_slot(sizeof(buf_page_t))) { /* This must be a buf_page_t object. */ UNIV_MEM_ASSERT_RW(src, size); === modified file 'storage/xtradb/buf/buf0buf.c' --- a/storage/xtradb/buf/buf0buf.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/buf/buf0buf.c 2010-01-06 12:00:14 +0000 @@ -51,6 +51,40 @@ Created 11/5/1995 Heikki Tuuri #include "dict0dict.h" #include "log0recv.h" #include "page0zip.h" +#include "trx0trx.h" + +/* prototypes for new functions added to ha_innodb.cc */ +trx_t* innobase_get_trx(); + +inline void _increment_page_get_statistics(buf_block_t* block, trx_t* trx) +{ + ulint block_hash; + ulint block_hash_byte; + byte block_hash_offset; + + ut_ad(block); + + if (!innobase_get_slow_log() || !trx || !trx->take_stats) + return; + + if (!trx->distinct_page_access_hash) { + trx->distinct_page_access_hash = mem_alloc(DPAH_SIZE); + memset(trx->distinct_page_access_hash, 0, DPAH_SIZE); + } + + block_hash = ut_hash_ulint((block->page.space << 20) + block->page.space + + block->page.offset, DPAH_SIZE << 3); + block_hash_byte = block_hash >> 3; + block_hash_offset = (byte) block_hash & 0x07; + if (block_hash_byte < 0 || block_hash_byte >= DPAH_SIZE) + fprintf(stderr, "!!! block_hash_byte = %lu block_hash_offset = %lu !!!\n", block_hash_byte, block_hash_offset); + if (block_hash_offset < 0 || block_hash_offset > 7) + fprintf(stderr, "!!! block_hash_byte = %lu block_hash_offset = %lu !!!\n", block_hash_byte, block_hash_offset); + if ((trx->distinct_page_access_hash[block_hash_byte] & ((byte) 0x01 << block_hash_offset)) == 0) + trx->distinct_page_access++; + trx->distinct_page_access_hash[block_hash_byte] |= (byte) 0x01 << block_hash_offset; + return; +} /* IMPLEMENTATION OF THE BUFFER POOL @@ -845,16 +879,35 @@ buf_chunk_not_freed( block = chunk->blocks; for (i = chunk->size; i--; block++) { - mutex_enter(&block->mutex); - - if (buf_block_get_state(block) == BUF_BLOCK_FILE_PAGE - && !buf_flush_ready_for_replace(&block->page)) { + ibool ready; + switch (buf_block_get_state(block)) { + case BUF_BLOCK_ZIP_FREE: + case BUF_BLOCK_ZIP_PAGE: + case BUF_BLOCK_ZIP_DIRTY: + /* The uncompressed buffer pool should never + contain compressed block descriptors. */ + ut_error; + break; + case BUF_BLOCK_NOT_USED: + case BUF_BLOCK_READY_FOR_USE: + case BUF_BLOCK_MEMORY: + case BUF_BLOCK_REMOVE_HASH: + /* Skip blocks that are not being used for + file pages. */ + break; + case BUF_BLOCK_FILE_PAGE: + mutex_enter(&block->mutex); + ready = buf_flush_ready_for_replace(&block->page); mutex_exit(&block->mutex); - return(block); - } - mutex_exit(&block->mutex); + if (!ready) { + + return(block); + } + + break; + } } return(NULL); @@ -985,8 +1038,6 @@ buf_pool_init(void) buf_pool->no_flush[i] = os_event_create(NULL); } - buf_pool->ulint_clock = 1; - /* 3. Initialize LRU fields --------------------------- */ /* All fields are initialized by mem_zalloc(). */ @@ -1024,7 +1075,11 @@ buf_pool_free(void) os_mem_free_large(chunk->mem, chunk->mem_size); } - buf_pool->n_chunks = 0; + mem_free(buf_pool->chunks); + hash_table_free(buf_pool->page_hash); + hash_table_free(buf_pool->zip_hash); + mem_free(buf_pool); + buf_pool = NULL; } /********************************************************************//** @@ -1171,10 +1226,15 @@ buf_relocate( #ifdef UNIV_LRU_DEBUG /* buf_pool->LRU_old must be the first item in the LRU list whose "old" flag is set. */ + ut_a(buf_pool->LRU_old->old); ut_a(!UT_LIST_GET_PREV(LRU, buf_pool->LRU_old) || !UT_LIST_GET_PREV(LRU, buf_pool->LRU_old)->old); ut_a(!UT_LIST_GET_NEXT(LRU, buf_pool->LRU_old) || UT_LIST_GET_NEXT(LRU, buf_pool->LRU_old)->old); + } else { + /* Check that the "old" flag is consistent in + the block and its neighbours. */ + buf_page_set_old(dpage, buf_page_is_old(dpage)); #endif /* UNIV_LRU_DEBUG */ } @@ -1510,35 +1570,8 @@ buf_pool_resize(void) } /********************************************************************//** -Moves the block to the start of the LRU list if there is a danger -that the block would drift out of the buffer pool. */ -UNIV_INLINE -void -buf_block_make_young( -/*=================*/ - buf_page_t* bpage) /*!< in: block to make younger */ -{ - ut_ad(!buf_pool_mutex_own()); - - /* Note that we read freed_page_clock's without holding any mutex: - this is allowed since the result is used only in heuristics */ - - if (buf_page_peek_if_too_old(bpage)) { - - //buf_pool_mutex_enter(); - mutex_enter(&LRU_list_mutex); - /* There has been freeing activity in the LRU list: - best to move to the head of the LRU list */ - - buf_LRU_make_block_young(bpage); - //buf_pool_mutex_exit(); - mutex_exit(&LRU_list_mutex); - } -} - -/********************************************************************//** Moves a page to the start of the buffer pool LRU list. This high-level -function can be used to prevent an important page from from slipping out of +function can be used to prevent an important page from slipping out of the buffer pool. */ UNIV_INTERN void @@ -1558,6 +1591,42 @@ buf_page_make_young( } /********************************************************************//** +Sets the time of the first access of a page and moves a page to the +start of the buffer pool LRU list if it is too old. This high-level +function can be used to prevent an important page from slipping +out of the buffer pool. */ +static +void +buf_page_set_accessed_make_young( +/*=============================*/ + buf_page_t* bpage, /*!< in/out: buffer block of a + file page */ + unsigned access_time) /*!< in: bpage->access_time + read under mutex protection, + or 0 if unknown */ +{ + ut_ad(!buf_pool_mutex_own()); + ut_a(buf_page_in_file(bpage)); + + if (buf_page_peek_if_too_old(bpage)) { + //buf_pool_mutex_enter(); + mutex_enter(&LRU_list_mutex); + buf_LRU_make_block_young(bpage); + //buf_pool_mutex_exit(); + mutex_exit(&LRU_list_mutex); + } else if (!access_time) { + ulint time_ms = ut_time_ms(); + mutex_t* block_mutex = buf_page_get_mutex_enter(bpage); + //buf_pool_mutex_enter(); + if (block_mutex) { + buf_page_set_accessed(bpage, time_ms); + mutex_exit(block_mutex); + } + //buf_pool_mutex_exit(); + } +} + +/********************************************************************//** Resets the check_index_page_at_flush field of a page if found in the buffer pool. */ UNIV_INTERN @@ -1696,11 +1765,20 @@ buf_page_get_zip( buf_page_t* bpage; mutex_t* block_mutex; ibool must_read; + unsigned access_time; + trx_t* trx = NULL; + ulint sec; + ulint ms; + ib_uint64_t start_time; + ib_uint64_t finish_time; #ifndef UNIV_LOG_DEBUG ut_ad(!ibuf_inside()); #endif - buf_pool->n_page_gets++; + if (innobase_get_slow_log()) { + trx = innobase_get_trx(); + } + buf_pool->stat.n_page_gets++; for (;;) { //buf_pool_mutex_enter(); @@ -1716,7 +1794,7 @@ lookup: //buf_pool_mutex_exit(); rw_lock_s_unlock(&page_hash_latch); - buf_read_page(space, zip_size, offset); + buf_read_page(space, zip_size, offset, trx); #if defined UNIV_DEBUG || defined UNIV_BUF_DEBUG ut_a(++buf_dbg_counter % 37 || buf_validate()); @@ -1770,14 +1848,13 @@ err_exit: got_block: must_read = buf_page_get_io_fix(bpage) == BUF_IO_READ; + access_time = buf_page_is_accessed(bpage); //buf_pool_mutex_exit(); - buf_page_set_accessed(bpage, TRUE); - mutex_exit(block_mutex); - buf_block_make_young(bpage); + buf_page_set_accessed_make_young(bpage, access_time); #ifdef UNIV_DEBUG_FILE_ACCESSES ut_a(!bpage->file_page_was_freed); @@ -1793,6 +1870,13 @@ got_block: /* Let us wait until the read operation completes */ + if (innobase_get_slow_log() && trx && trx->take_stats) + { + ut_usectime(&sec, &ms); + start_time = (ib_uint64_t)sec * 1000000 + ms; + } else { + start_time = 0; + } for (;;) { enum buf_io_fix io_fix; @@ -1807,6 +1891,12 @@ got_block: break; } } + if (innobase_get_slow_log() && trx && trx->take_stats && start_time) + { + ut_usectime(&sec, &ms); + finish_time = (ib_uint64_t)sec * 1000000 + ms; + trx->io_reads_wait_timer += (ulint)(finish_time - start_time); + } } #ifdef UNIV_IBUF_COUNT_DEBUG @@ -1870,7 +1960,7 @@ buf_zip_decompress( switch (fil_page_get_type(frame)) { case FIL_PAGE_INDEX: if (page_zip_decompress(&block->page.zip, - block->frame)) { + block->frame, TRUE)) { return(TRUE); } @@ -2058,10 +2148,15 @@ buf_page_get_gen( mtr_t* mtr) /*!< in: mini-transaction */ { buf_block_t* block; - ibool accessed; + unsigned access_time; ulint fix_type; ibool must_read; mutex_t* block_mutex; + trx_t* trx = NULL; + ulint sec; + ulint ms; + ib_uint64_t start_time; + ib_uint64_t finish_time; ut_ad(mtr); ut_ad((rw_latch == RW_S_LATCH) @@ -2075,14 +2170,16 @@ buf_page_get_gen( #ifndef UNIV_LOG_DEBUG ut_ad(!ibuf_inside() || ibuf_page(space, zip_size, offset, NULL)); #endif - buf_pool->n_page_gets++; + if (innobase_get_slow_log()) { + trx = innobase_get_trx(); + } + buf_pool->stat.n_page_gets++; loop: block = guess; //buf_pool_mutex_enter(); if (block) { block_mutex = buf_page_get_mutex_enter((buf_page_t*)block); - ut_a(block_mutex); /* If the guess is a compressed page descriptor that has been allocated by buf_buddy_alloc(), it may have @@ -2092,7 +2189,9 @@ loop: the guess may be pointing to a buffer pool chunk that has been released when resizing the buffer pool. */ - if (!buf_block_is_uncompressed(block) + if (!block_mutex) { + block = guess = NULL; + } else if (!buf_block_is_uncompressed(block) || offset != block->page.offset || space != block->page.space || buf_block_get_state(block) != BUF_BLOCK_FILE_PAGE) { @@ -2127,7 +2226,7 @@ loop2: return(NULL); } - buf_read_page(space, zip_size, offset); + buf_read_page(space, zip_size, offset, trx); #if defined UNIV_DEBUG || defined UNIV_BUF_DEBUG ut_a(++buf_dbg_counter % 37 || buf_validate()); @@ -2351,17 +2450,17 @@ wait_until_unfixed: UNIV_MEM_ASSERT_RW(&block->page, sizeof block->page); buf_block_buf_fix_inc(block, file, line); - //buf_pool_mutex_exit(); - /* Check if this is the first access to the page */ + //mutex_exit(&block->mutex); - accessed = buf_page_is_accessed(&block->page); + /* Check if this is the first access to the page */ - buf_page_set_accessed(&block->page, TRUE); + access_time = buf_page_is_accessed(&block->page); + //buf_pool_mutex_exit(); mutex_exit(block_mutex); - buf_block_make_young(&block->page); + buf_page_set_accessed_make_young(&block->page, access_time); #ifdef UNIV_DEBUG_FILE_ACCESSES ut_a(!block->page.file_page_was_freed); @@ -2379,6 +2478,13 @@ wait_until_unfixed: /* Let us wait until the read operation completes */ + if (innobase_get_slow_log() && trx && trx->take_stats) + { + ut_usectime(&sec, &ms); + start_time = (ib_uint64_t)sec * 1000000 + ms; + } else { + start_time = 0; + } for (;;) { enum buf_io_fix io_fix; @@ -2393,6 +2499,12 @@ wait_until_unfixed: break; } } + if (innobase_get_slow_log() && trx && trx->take_stats && start_time) + { + ut_usectime(&sec, &ms); + finish_time = (ib_uint64_t)sec * 1000000 + ms; + trx->io_reads_wait_timer += (ulint)(finish_time - start_time); + } } fix_type = MTR_MEMO_BUF_FIX; @@ -2414,17 +2526,21 @@ wait_until_unfixed: mtr_memo_push(mtr, block, fix_type); - if (!accessed) { + if (!access_time) { /* In the case of a first access, try to apply linear read-ahead */ - buf_read_ahead_linear(space, zip_size, offset); + buf_read_ahead_linear(space, zip_size, offset, trx); } #ifdef UNIV_IBUF_COUNT_DEBUG ut_a(ibuf_count_get(buf_block_get_space(block), buf_block_get_page_no(block)) == 0); #endif + if (innobase_get_slow_log()) { + _increment_page_get_statistics(block, trx); + } + return(block); } @@ -2444,9 +2560,10 @@ buf_page_optimistic_get_func( ulint line, /*!< in: line where called */ mtr_t* mtr) /*!< in: mini-transaction */ { - ibool accessed; + unsigned access_time; ibool success; ulint fix_type; + trx_t* trx = NULL; ut_ad(mtr && block); ut_ad((rw_latch == RW_S_LATCH) || (rw_latch == RW_X_LATCH)); @@ -2461,14 +2578,16 @@ buf_page_optimistic_get_func( } buf_block_buf_fix_inc(block, file, line); - accessed = buf_page_is_accessed(&block->page); - buf_page_set_accessed(&block->page, TRUE); mutex_exit(&block->mutex); - buf_block_make_young(&block->page); + /* Check if this is the first access to the page. + We do a dirty read on purpose, to avoid mutex contention. + This field is only used for heuristic purposes; it does not + affect correctness. */ - /* Check if this is the first access to the page */ + access_time = buf_page_is_accessed(&block->page); + buf_page_set_accessed_make_young(&block->page, access_time); ut_ad(!ibuf_inside() || ibuf_page(buf_block_get_space(block), @@ -2520,21 +2639,28 @@ buf_page_optimistic_get_func( #ifdef UNIV_DEBUG_FILE_ACCESSES ut_a(block->page.file_page_was_freed == FALSE); #endif - if (UNIV_UNLIKELY(!accessed)) { + if (innobase_get_slow_log()) { + trx = innobase_get_trx(); + } + + if (UNIV_UNLIKELY(!access_time)) { /* In the case of a first access, try to apply linear read-ahead */ buf_read_ahead_linear(buf_block_get_space(block), buf_block_get_zip_size(block), - buf_block_get_page_no(block)); + buf_block_get_page_no(block), trx); } #ifdef UNIV_IBUF_COUNT_DEBUG ut_a(ibuf_count_get(buf_block_get_space(block), buf_block_get_page_no(block)) == 0); #endif - buf_pool->n_page_gets++; + buf_pool->stat.n_page_gets++; + if (innobase_get_slow_log()) { + _increment_page_get_statistics(block, trx); + } return(TRUE); } @@ -2556,6 +2682,7 @@ buf_page_get_known_nowait( { ibool success; ulint fix_type; + trx_t* trx = NULL; ut_ad(mtr); ut_ad((rw_latch == RW_S_LATCH) || (rw_latch == RW_X_LATCH)); @@ -2581,8 +2708,24 @@ buf_page_get_known_nowait( mutex_exit(&block->mutex); - if (mode == BUF_MAKE_YOUNG) { - buf_block_make_young(&block->page); + if (mode == BUF_MAKE_YOUNG && buf_page_peek_if_too_old(&block->page)) { + //buf_pool_mutex_enter(); + mutex_enter(&LRU_list_mutex); + buf_LRU_make_block_young(&block->page); + //buf_pool_mutex_exit(); + mutex_exit(&LRU_list_mutex); + } else if (!buf_page_is_accessed(&block->page)) { + /* Above, we do a dirty read on purpose, to avoid + mutex contention. The field buf_page_t::access_time + is only used for heuristic purposes. Writes to the + field must be protected by mutex, however. */ + ulint time_ms = ut_time_ms(); + + //buf_pool_mutex_enter(); + mutex_enter(&block->mutex); + buf_page_set_accessed(&block->page, time_ms); + //buf_pool_mutex_exit(); + mutex_exit(&block->mutex); } ut_ad(!ibuf_inside() || (mode == BUF_KEEP_OLD)); @@ -2621,7 +2764,12 @@ buf_page_get_known_nowait( || (ibuf_count_get(buf_block_get_space(block), buf_block_get_page_no(block)) == 0)); #endif - buf_pool->n_page_gets++; + buf_pool->stat.n_page_gets++; + + if (innobase_get_slow_log()) { + trx = innobase_get_trx(); + _increment_page_get_statistics(block, trx); + } return(TRUE); } @@ -2700,7 +2848,7 @@ buf_page_try_get_func( #endif /* UNIV_DEBUG_FILE_ACCESSES */ buf_block_dbg_add_level(block, SYNC_NO_ORDER_CHECK); - buf_pool->n_page_gets++; + buf_pool->stat.n_page_gets++; #ifdef UNIV_IBUF_COUNT_DEBUG ut_a(ibuf_count_get(buf_block_get_space(block), @@ -2719,10 +2867,10 @@ buf_page_init_low( buf_page_t* bpage) /*!< in: block to init */ { bpage->flush_type = BUF_FLUSH_LRU; - bpage->accessed = FALSE; bpage->io_fix = BUF_IO_NONE; bpage->buf_fix_count = 0; bpage->freed_page_clock = 0; + bpage->access_time = 0; bpage->newest_modification = 0; bpage->oldest_modification = 0; HASH_INVALIDATE(bpage, hash); @@ -3044,6 +3192,7 @@ buf_page_create( buf_frame_t* frame; buf_block_t* block; buf_block_t* free_block = NULL; + ulint time_ms = ut_time_ms(); ut_ad(mtr); ut_ad(space || !zip_size); @@ -3095,7 +3244,7 @@ buf_page_create( buf_LRU_add_block(&block->page, FALSE); buf_block_buf_fix_inc(block, __FILE__, __LINE__); - buf_pool->n_pages_created++; + buf_pool->stat.n_pages_created++; if (zip_size) { void* data; @@ -3132,13 +3281,13 @@ buf_page_create( rw_lock_x_unlock(&block->lock); } + buf_page_set_accessed(&block->page, time_ms); + //buf_pool_mutex_exit(); mutex_exit(&LRU_list_mutex); mtr_memo_push(mtr, block, MTR_MEMO_BUF_FIX); - buf_page_set_accessed(&block->page, TRUE); - mutex_exit(&block->mutex); /* Delete possible entries for the page from the insert buffer: @@ -3355,7 +3504,7 @@ corrupt: ut_ad(buf_pool->n_pend_reads > 0); buf_pool->n_pend_reads--; - buf_pool->n_pages_read++; + buf_pool->stat.n_pages_read++; if (uncompressed) { rw_lock_x_unlock_gen(&((buf_block_t*) bpage)->lock, @@ -3380,7 +3529,7 @@ corrupt: BUF_IO_WRITE); } - buf_pool->n_pages_written++; + buf_pool->stat.n_pages_written++; break; @@ -3411,7 +3560,32 @@ void buf_pool_invalidate(void) /*=====================*/ { - ibool freed; + ibool freed; + enum buf_flush i; + + buf_pool_mutex_enter(); + + for (i = BUF_FLUSH_LRU; i < BUF_FLUSH_N_TYPES; i++) { + + /* As this function is called during startup and + during redo application phase during recovery, InnoDB + is single threaded (apart from IO helper threads) at + this stage. No new write batch can be in intialization + stage at this point. */ + ut_ad(buf_pool->init_flush[i] == FALSE); + + /* However, it is possible that a write batch that has + been posted earlier is still not complete. For buffer + pool invalidation to proceed we must ensure there is NO + write activity happening. */ + if (buf_pool->n_flush[i] > 0) { + buf_pool_mutex_exit(); + buf_flush_wait_batch_end(i); + buf_pool_mutex_enter(); + } + } + + buf_pool_mutex_exit(); ut_ad(buf_all_freed()); @@ -3427,6 +3601,14 @@ buf_pool_invalidate(void) ut_ad(UT_LIST_GET_LEN(buf_pool->LRU) == 0); ut_ad(UT_LIST_GET_LEN(buf_pool->unzip_LRU) == 0); + buf_pool->freed_page_clock = 0; + buf_pool->LRU_old = NULL; + buf_pool->LRU_old_len = 0; + buf_pool->LRU_flush_ended = 0; + + memset(&buf_pool->stat, 0x00, sizeof(buf_pool->stat)); + buf_refresh_io_stats(); + //buf_pool_mutex_exit(); mutex_exit(&LRU_list_mutex); } @@ -3706,6 +3888,7 @@ buf_print(void) "n pending decompressions %lu\n" "n pending reads %lu\n" "n pending flush LRU %lu list %lu single page %lu\n" + "pages made young %lu, not young %lu\n" "pages read %lu, created %lu, written %lu\n", (ulong) size, (ulong) UT_LIST_GET_LEN(buf_pool->LRU), @@ -3716,8 +3899,11 @@ buf_print(void) (ulong) buf_pool->n_flush[BUF_FLUSH_LRU], (ulong) buf_pool->n_flush[BUF_FLUSH_LIST], (ulong) buf_pool->n_flush[BUF_FLUSH_SINGLE_PAGE], - (ulong) buf_pool->n_pages_read, buf_pool->n_pages_created, - (ulong) buf_pool->n_pages_written); + (ulong) buf_pool->stat.n_pages_made_young, + (ulong) buf_pool->stat.n_pages_not_made_young, + (ulong) buf_pool->stat.n_pages_read, + (ulong) buf_pool->stat.n_pages_created, + (ulong) buf_pool->stat.n_pages_written); /* Count the number of blocks belonging to each index in the buffer */ @@ -3927,10 +4113,9 @@ buf_print_io( { time_t current_time; double time_elapsed; - ulint size; + ulint n_gets_diff; ut_ad(buf_pool); - size = buf_pool->curr_size; //buf_pool_mutex_enter(); mutex_enter(&LRU_list_mutex); @@ -3943,13 +4128,15 @@ buf_print_io( "Buffer pool size, bytes %lu\n" "Free buffers %lu\n" "Database pages %lu\n" + "Old database pages %lu\n" "Modified db pages %lu\n" "Pending reads %lu\n" "Pending writes: LRU %lu, flush list %lu, single page %lu\n", - (ulong) size, - (ulong) size * UNIV_PAGE_SIZE, + (ulong) buf_pool->curr_size, + (ulong) buf_pool->curr_size * UNIV_PAGE_SIZE, (ulong) UT_LIST_GET_LEN(buf_pool->free), (ulong) UT_LIST_GET_LEN(buf_pool->LRU), + (ulong) buf_pool->LRU_old_len, (ulong) UT_LIST_GET_LEN(buf_pool->flush_list), (ulong) buf_pool->n_pend_reads, (ulong) buf_pool->n_flush[BUF_FLUSH_LRU] @@ -3961,37 +4148,66 @@ buf_print_io( current_time = time(NULL); time_elapsed = 0.001 + difftime(current_time, buf_pool->last_printout_time); - buf_pool->last_printout_time = current_time; fprintf(file, + "Pages made young %lu, not young %lu\n" + "%.2f youngs/s, %.2f non-youngs/s\n" "Pages read %lu, created %lu, written %lu\n" "%.2f reads/s, %.2f creates/s, %.2f writes/s\n", - (ulong) buf_pool->n_pages_read, - (ulong) buf_pool->n_pages_created, - (ulong) buf_pool->n_pages_written, - (buf_pool->n_pages_read - buf_pool->n_pages_read_old) + (ulong) buf_pool->stat.n_pages_made_young, + (ulong) buf_pool->stat.n_pages_not_made_young, + (buf_pool->stat.n_pages_made_young + - buf_pool->old_stat.n_pages_made_young) + / time_elapsed, + (buf_pool->stat.n_pages_not_made_young + - buf_pool->old_stat.n_pages_not_made_young) / time_elapsed, - (buf_pool->n_pages_created - buf_pool->n_pages_created_old) + (ulong) buf_pool->stat.n_pages_read, + (ulong) buf_pool->stat.n_pages_created, + (ulong) buf_pool->stat.n_pages_written, + (buf_pool->stat.n_pages_read + - buf_pool->old_stat.n_pages_read) / time_elapsed, - (buf_pool->n_pages_written - buf_pool->n_pages_written_old) + (buf_pool->stat.n_pages_created + - buf_pool->old_stat.n_pages_created) + / time_elapsed, + (buf_pool->stat.n_pages_written + - buf_pool->old_stat.n_pages_written) / time_elapsed); - if (buf_pool->n_page_gets > buf_pool->n_page_gets_old) { - fprintf(file, "Buffer pool hit rate %lu / 1000\n", + n_gets_diff = buf_pool->stat.n_page_gets - buf_pool->old_stat.n_page_gets; + + if (n_gets_diff) { + fprintf(file, + "Buffer pool hit rate %lu / 1000," + " young-making rate %lu / 1000 not %lu / 1000\n", (ulong) - (1000 - ((1000 * (buf_pool->n_pages_read - - buf_pool->n_pages_read_old)) - / (buf_pool->n_page_gets - - buf_pool->n_page_gets_old)))); + (1000 - ((1000 * (buf_pool->stat.n_pages_read + - buf_pool->old_stat.n_pages_read)) + / (buf_pool->stat.n_page_gets + - buf_pool->old_stat.n_page_gets))), + (ulong) + (1000 * (buf_pool->stat.n_pages_made_young + - buf_pool->old_stat.n_pages_made_young) + / n_gets_diff), + (ulong) + (1000 * (buf_pool->stat.n_pages_not_made_young + - buf_pool->old_stat.n_pages_not_made_young) + / n_gets_diff)); } else { fputs("No buffer pool page gets since the last printout\n", file); } - buf_pool->n_page_gets_old = buf_pool->n_page_gets; - buf_pool->n_pages_read_old = buf_pool->n_pages_read; - buf_pool->n_pages_created_old = buf_pool->n_pages_created; - buf_pool->n_pages_written_old = buf_pool->n_pages_written; + /* Statistics about read ahead algorithm */ + fprintf(file, "Pages read ahead %.2f/s," + " evicted without access %.2f/s\n", + (buf_pool->stat.n_ra_pages_read + - buf_pool->old_stat.n_ra_pages_read) + / time_elapsed, + (buf_pool->stat.n_ra_pages_evicted + - buf_pool->old_stat.n_ra_pages_evicted) + / time_elapsed); /* Print some values to help us with visualizing what is happening with LRU eviction. */ @@ -4003,6 +4219,7 @@ buf_print_io( buf_LRU_stat_sum.io, buf_LRU_stat_cur.io, buf_LRU_stat_sum.unzip, buf_LRU_stat_cur.unzip); + buf_refresh_io_stats(); //buf_pool_mutex_exit(); mutex_exit(&LRU_list_mutex); mutex_exit(&free_list_mutex); @@ -4018,10 +4235,7 @@ buf_refresh_io_stats(void) /*======================*/ { buf_pool->last_printout_time = time(NULL); - buf_pool->n_page_gets_old = buf_pool->n_page_gets; - buf_pool->n_pages_read_old = buf_pool->n_pages_read; - buf_pool->n_pages_created_old = buf_pool->n_pages_created; - buf_pool->n_pages_written_old = buf_pool->n_pages_written; + buf_pool->old_stat = buf_pool->stat; } /*********************************************************************//** === modified file 'storage/xtradb/buf/buf0flu.c' --- a/storage/xtradb/buf/buf0flu.c 2009-11-13 21:26:08 +0000 +++ b/storage/xtradb/buf/buf0flu.c 2010-01-15 15:58:25 +0000 @@ -331,6 +331,28 @@ buf_flush_write_complete( } /********************************************************************//** +Flush a batch of writes to the datafiles that have already been +written by the OS. */ +static +void +buf_flush_sync_datafiles(void) +/*==========================*/ +{ + /* Wake possible simulated aio thread to actually post the + writes to the operating system */ + os_aio_simulated_wake_handler_threads(); + + /* Wait that all async writes to tablespaces have been posted to + the OS */ + os_aio_wait_until_no_pending_writes(); + + /* Now we flush the data to disk (for example, with fsync) */ + fil_flush_file_spaces(FIL_TABLESPACE); + + return; +} + +/********************************************************************//** Flushes possible buffered writes from the doublewrite memory buffer to disk, and also wakes up the aio thread if simulated aio is used. It is very important to call this function after a batch of writes has been posted, @@ -347,8 +369,8 @@ buf_flush_buffered_writes(void) ulint i; if (!srv_use_doublewrite_buf || trx_doublewrite == NULL) { - os_aio_simulated_wake_handler_threads(); - + /* Sync the writes to the disk. */ + buf_flush_sync_datafiles(); return; } @@ -556,22 +578,10 @@ flush: buf_LRU_stat_inc_io(); } - /* Wake possible simulated aio thread to actually post the - writes to the operating system */ - - os_aio_simulated_wake_handler_threads(); - - /* Wait that all async writes to tablespaces have been posted to - the OS */ - - os_aio_wait_until_no_pending_writes(); - - /* Now we flush the data to disk (for example, with fsync) */ - - fil_flush_file_spaces(FIL_TABLESPACE); + /* Sync the writes to the disk. */ + buf_flush_sync_datafiles(); /* We can now reuse the doublewrite memory buffer: */ - trx_doublewrite->first_free = 0; mutex_exit(&(trx_doublewrite->mutex)); @@ -994,9 +1004,7 @@ buf_flush_try_neighbors( || buf_page_is_old(bpage)) { mutex_t* block_mutex = buf_page_get_mutex_enter(bpage); - ut_a(block_mutex); - - if (buf_flush_ready_for_flush(bpage, flush_type) + if (block_mutex && buf_flush_ready_for_flush(bpage, flush_type) && (i == offset || !bpage->buf_fix_count)) { /* We only try to flush those neighbors != offset where the buf fix count is @@ -1012,7 +1020,7 @@ buf_flush_try_neighbors( //buf_pool_mutex_enter(); rw_lock_s_lock(&page_hash_latch); - } else { + } else if (block_mutex) { mutex_exit(block_mutex); } } @@ -1050,6 +1058,7 @@ buf_flush_batch( min_n), otherwise ignored */ { buf_page_t* bpage; + buf_page_t* prev_bpage = NULL; ulint page_count = 0; ulint old_page_count; ulint space; @@ -1103,6 +1112,9 @@ flush_next: mutex_enter(&flush_list_mutex); remaining = UT_LIST_GET_LEN(buf_pool->flush_list); bpage = UT_LIST_GET_LAST(buf_pool->flush_list); + if (bpage) { + prev_bpage = UT_LIST_GET_PREV(flush_list, bpage); + } mutex_exit(&flush_list_mutex); if (!bpage || bpage->oldest_modification >= lsn_limit) { @@ -1123,11 +1135,14 @@ flush_next: mutex_t*block_mutex = buf_page_get_mutex_enter(bpage); ibool ready; - ut_a(buf_page_in_file(bpage)); + //ut_a(buf_page_in_file(bpage)); - ut_a(block_mutex); - ready = buf_flush_ready_for_flush(bpage, flush_type); - mutex_exit(block_mutex); + if (block_mutex) { + ready = buf_flush_ready_for_flush(bpage, flush_type); + mutex_exit(block_mutex); + } else { + ready = FALSE; + } if (ready) { space = buf_page_get_space(bpage); @@ -1162,6 +1177,13 @@ flush_next: mutex_enter(&flush_list_mutex); bpage = UT_LIST_GET_PREV(flush_list, bpage); //ut_ad(!bpage || bpage->in_flush_list); /* optimistic */ + if (bpage != prev_bpage) { + /* the search may warp.. retrying */ + bpage = NULL; + } + if (bpage) { + prev_bpage = UT_LIST_GET_PREV(flush_list, bpage); + } mutex_exit(&flush_list_mutex); remaining--; } @@ -1271,13 +1293,13 @@ buf_flush_LRU_recommendation(void) } block_mutex = buf_page_get_mutex_enter(bpage); - ut_a(block_mutex); - - if (buf_flush_ready_for_replace(bpage)) { + if (block_mutex && buf_flush_ready_for_replace(bpage)) { n_replaceable++; } - mutex_exit(block_mutex); + if (block_mutex) { + mutex_exit(block_mutex); + } distance++; === modified file 'storage/xtradb/buf/buf0lru.c' --- a/storage/xtradb/buf/buf0lru.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/buf/buf0lru.c 2010-01-06 12:00:14 +0000 @@ -49,18 +49,22 @@ Created 11/5/1995 Heikki Tuuri #include "log0recv.h" #include "srv0srv.h" -/** The number of blocks from the LRU_old pointer onward, including the block -pointed to, must be 3/8 of the whole LRU list length, except that the -tolerance defined below is allowed. Note that the tolerance must be small -enough such that for even the BUF_LRU_OLD_MIN_LEN long LRU list, the -LRU_old pointer is not allowed to point to either end of the LRU list. */ +/** The number of blocks from the LRU_old pointer onward, including +the block pointed to, must be buf_LRU_old_ratio/BUF_LRU_OLD_RATIO_DIV +of the whole LRU list length, except that the tolerance defined below +is allowed. Note that the tolerance must be small enough such that for +even the BUF_LRU_OLD_MIN_LEN long LRU list, the LRU_old pointer is not +allowed to point to either end of the LRU list. */ #define BUF_LRU_OLD_TOLERANCE 20 -/** The whole LRU list length is divided by this number to determine an -initial segment in buf_LRU_get_recent_limit */ - -#define BUF_LRU_INITIAL_RATIO 8 +/** The minimum amount of non-old blocks when the LRU_old list exists +(that is, when there are more than BUF_LRU_OLD_MIN_LEN blocks). +@see buf_LRU_old_adjust_len */ +#define BUF_LRU_NON_OLD_MIN_LEN 5 +#if BUF_LRU_NON_OLD_MIN_LEN >= BUF_LRU_OLD_MIN_LEN +# error "BUF_LRU_NON_OLD_MIN_LEN >= BUF_LRU_OLD_MIN_LEN" +#endif /** When dropping the search hash index entries before deleting an ibd file, we build a local array of pages belonging to that tablespace @@ -107,6 +111,15 @@ UNIV_INTERN buf_LRU_stat_t buf_LRU_stat_ /* @} */ +/** @name Heuristics for detecting index scan @{ */ +/** Reserve this much/BUF_LRU_OLD_RATIO_DIV of the buffer pool for +"old" blocks. Protected by buf_pool_mutex. */ +UNIV_INTERN uint buf_LRU_old_ratio; +/** Move blocks to "new" LRU list only if the first access was at +least this many milliseconds ago. Not protected by any mutex or latch. */ +UNIV_INTERN uint buf_LRU_old_threshold_ms; +/* @} */ + /******************************************************************//** Takes a block out of the LRU list and page hash table. If the block is compressed-only (BUF_BLOCK_ZIP_PAGE), @@ -255,9 +268,12 @@ scan_again: mutex_t* block_mutex = buf_page_get_mutex_enter(bpage); buf_page_t* prev_bpage; - ut_a(block_mutex); prev_bpage = UT_LIST_GET_PREV(LRU, bpage); + if (!block_mutex) { + goto next_page; + } + ut_a(buf_page_in_file(bpage)); if (buf_page_get_state(bpage) != BUF_BLOCK_FILE_PAGE @@ -360,9 +376,13 @@ scan_again: ut_a(buf_page_in_file(bpage)); - ut_a(block_mutex); prev_bpage = UT_LIST_GET_PREV(LRU, bpage); + if (!block_mutex) { + bpage = prev_bpage; + continue; + } + if (buf_page_get_space(bpage) == id) { if (bpage->buf_fix_count > 0 || buf_page_get_io_fix(bpage) != BUF_IO_NONE) { @@ -450,45 +470,6 @@ next_page: } } -/******************************************************************//** -Gets the minimum LRU_position field for the blocks in an initial segment -(determined by BUF_LRU_INITIAL_RATIO) of the LRU list. The limit is not -guaranteed to be precise, because the ulint_clock may wrap around. -@return the limit; zero if could not determine it */ -UNIV_INTERN -ulint -buf_LRU_get_recent_limit(void) -/*==========================*/ -{ - const buf_page_t* bpage; - ulint len; - ulint limit; - - //buf_pool_mutex_enter(); - mutex_enter(&LRU_list_mutex); - - len = UT_LIST_GET_LEN(buf_pool->LRU); - - if (len < BUF_LRU_OLD_MIN_LEN) { - /* The LRU list is too short to do read-ahead */ - - //buf_pool_mutex_exit(); - mutex_exit(&LRU_list_mutex); - - return(0); - } - - bpage = UT_LIST_GET_FIRST(buf_pool->LRU); - - limit = buf_page_get_LRU_position(bpage); - len /= BUF_LRU_INITIAL_RATIO; - - //buf_pool_mutex_exit(); - mutex_exit(&LRU_list_mutex); - - return(limit > len ? (limit - len) : 0); -} - /********************************************************************//** Insert a compressed block into buf_pool->zip_clean in the LRU order. */ UNIV_INTERN @@ -631,10 +612,13 @@ restart: bpage = UT_LIST_GET_PREV(LRU, bpage), distance--) { enum buf_lru_free_block_status freed; + unsigned accessed; mutex_t* block_mutex = buf_page_get_mutex_enter(bpage); - ut_a(block_mutex); + if (!block_mutex) { + goto restart; + } if (!bpage->in_LRU_list || !buf_page_in_file(bpage)) { @@ -645,11 +629,18 @@ restart: ut_ad(buf_page_in_file(bpage)); ut_ad(bpage->in_LRU_list); + accessed = buf_page_is_accessed(bpage); freed = buf_LRU_free_block(bpage, TRUE, NULL, have_LRU_mutex); mutex_exit(block_mutex); switch (freed) { case BUF_LRU_FREED: + /* Keep track of pages that are evicted without + ever being accessed. This gives us a measure of + the effectiveness of readahead */ + if (!accessed) { + ++buf_pool->stat.n_ra_pages_evicted; + } return(TRUE); case BUF_LRU_NOT_FREED: @@ -1027,8 +1018,10 @@ buf_LRU_old_adjust_len(void) ut_a(buf_pool->LRU_old); //ut_ad(buf_pool_mutex_own()); ut_ad(mutex_own(&LRU_list_mutex)); -#if 3 * (BUF_LRU_OLD_MIN_LEN / 8) <= BUF_LRU_OLD_TOLERANCE + 5 -# error "3 * (BUF_LRU_OLD_MIN_LEN / 8) <= BUF_LRU_OLD_TOLERANCE + 5" + ut_ad(buf_LRU_old_ratio >= BUF_LRU_OLD_RATIO_MIN); + ut_ad(buf_LRU_old_ratio <= BUF_LRU_OLD_RATIO_MAX); +#if BUF_LRU_OLD_RATIO_MIN * BUF_LRU_OLD_MIN_LEN <= BUF_LRU_OLD_RATIO_DIV * (BUF_LRU_OLD_TOLERANCE + 5) +# error "BUF_LRU_OLD_RATIO_MIN * BUF_LRU_OLD_MIN_LEN <= BUF_LRU_OLD_RATIO_DIV * (BUF_LRU_OLD_TOLERANCE + 5)" #endif #ifdef UNIV_LRU_DEBUG /* buf_pool->LRU_old must be the first item in the LRU list @@ -1040,34 +1033,39 @@ buf_LRU_old_adjust_len(void) || UT_LIST_GET_NEXT(LRU, buf_pool->LRU_old)->old); #endif /* UNIV_LRU_DEBUG */ + old_len = buf_pool->LRU_old_len; + new_len = ut_min(UT_LIST_GET_LEN(buf_pool->LRU) + * buf_LRU_old_ratio / BUF_LRU_OLD_RATIO_DIV, + UT_LIST_GET_LEN(buf_pool->LRU) + - (BUF_LRU_OLD_TOLERANCE + + BUF_LRU_NON_OLD_MIN_LEN)); + for (;;) { - old_len = buf_pool->LRU_old_len; - new_len = 3 * (UT_LIST_GET_LEN(buf_pool->LRU) / 8); + buf_page_t* LRU_old = buf_pool->LRU_old; - ut_ad(buf_pool->LRU_old->in_LRU_list); - ut_a(buf_pool->LRU_old); + ut_a(LRU_old); + ut_ad(LRU_old->in_LRU_list); #ifdef UNIV_LRU_DEBUG - ut_a(buf_pool->LRU_old->old); + ut_a(LRU_old->old); #endif /* UNIV_LRU_DEBUG */ /* Update the LRU_old pointer if necessary */ - if (old_len < new_len - BUF_LRU_OLD_TOLERANCE) { + if (old_len + BUF_LRU_OLD_TOLERANCE < new_len) { - buf_pool->LRU_old = UT_LIST_GET_PREV( - LRU, buf_pool->LRU_old); + buf_pool->LRU_old = LRU_old = UT_LIST_GET_PREV( + LRU, LRU_old); #ifdef UNIV_LRU_DEBUG - ut_a(!buf_pool->LRU_old->old); + ut_a(!LRU_old->old); #endif /* UNIV_LRU_DEBUG */ - buf_page_set_old(buf_pool->LRU_old, TRUE); - buf_pool->LRU_old_len++; + old_len = ++buf_pool->LRU_old_len; + buf_page_set_old(LRU_old, TRUE); } else if (old_len > new_len + BUF_LRU_OLD_TOLERANCE) { - buf_page_set_old(buf_pool->LRU_old, FALSE); - buf_pool->LRU_old = UT_LIST_GET_NEXT( - LRU, buf_pool->LRU_old); - buf_pool->LRU_old_len--; + buf_pool->LRU_old = UT_LIST_GET_NEXT(LRU, LRU_old); + old_len = --buf_pool->LRU_old_len; + buf_page_set_old(LRU_old, FALSE); } else { return; } @@ -1092,12 +1090,13 @@ buf_LRU_old_init(void) the adjust function to move the LRU_old pointer to the right position */ - bpage = UT_LIST_GET_FIRST(buf_pool->LRU); - - while (bpage != NULL) { + for (bpage = UT_LIST_GET_LAST(buf_pool->LRU); bpage != NULL; + bpage = UT_LIST_GET_PREV(LRU, bpage)) { ut_ad(bpage->in_LRU_list); - buf_page_set_old(bpage, TRUE); - bpage = UT_LIST_GET_NEXT(LRU, bpage); + ut_ad(buf_page_in_file(bpage)); + /* This loop temporarily violates the + assertions of buf_page_set_old(). */ + bpage->old = TRUE; } buf_pool->LRU_old = UT_LIST_GET_FIRST(buf_pool->LRU); @@ -1152,16 +1151,19 @@ buf_LRU_remove_block( if (UNIV_UNLIKELY(bpage == buf_pool->LRU_old)) { - /* Below: the previous block is guaranteed to exist, because - the LRU_old pointer is only allowed to differ by the - tolerance value from strict 3/8 of the LRU list length. */ + /* Below: the previous block is guaranteed to exist, + because the LRU_old pointer is only allowed to differ + by BUF_LRU_OLD_TOLERANCE from strict + buf_LRU_old_ratio/BUF_LRU_OLD_RATIO_DIV of the LRU + list length. */ + buf_page_t* prev_bpage = UT_LIST_GET_PREV(LRU, bpage); - buf_pool->LRU_old = UT_LIST_GET_PREV(LRU, bpage); - ut_a(buf_pool->LRU_old); + ut_a(prev_bpage); #ifdef UNIV_LRU_DEBUG - ut_a(!buf_pool->LRU_old->old); + ut_a(!prev_bpage->old); #endif /* UNIV_LRU_DEBUG */ - buf_page_set_old(buf_pool->LRU_old, TRUE); + buf_pool->LRU_old = prev_bpage; + buf_page_set_old(prev_bpage, TRUE); buf_pool->LRU_old_len++; } @@ -1172,10 +1174,19 @@ buf_LRU_remove_block( buf_unzip_LRU_remove_block_if_needed(bpage); - /* If the LRU list is so short that LRU_old not defined, return */ + /* If the LRU list is so short that LRU_old is not defined, + clear the "old" flags and return */ if (UT_LIST_GET_LEN(buf_pool->LRU) < BUF_LRU_OLD_MIN_LEN) { + for (bpage = UT_LIST_GET_FIRST(buf_pool->LRU); bpage != NULL; + bpage = UT_LIST_GET_NEXT(LRU, bpage)) { + /* This loop temporarily violates the + assertions of buf_page_set_old(). */ + bpage->old = FALSE; + } + buf_pool->LRU_old = NULL; + buf_pool->LRU_old_len = 0; return; } @@ -1227,8 +1238,6 @@ buf_LRU_add_block_to_end_low( /*=========================*/ buf_page_t* bpage) /*!< in: control block */ { - buf_page_t* last_bpage; - ut_ad(buf_pool); ut_ad(bpage); //ut_ad(buf_pool_mutex_own()); @@ -1236,31 +1245,18 @@ buf_LRU_add_block_to_end_low( ut_a(buf_page_in_file(bpage)); - last_bpage = UT_LIST_GET_LAST(buf_pool->LRU); - - if (last_bpage) { - bpage->LRU_position = last_bpage->LRU_position; - } else { - bpage->LRU_position = buf_pool_clock_tic(); - } - ut_ad(!bpage->in_LRU_list); UT_LIST_ADD_LAST(LRU, buf_pool->LRU, bpage); bpage->in_LRU_list = TRUE; - buf_page_set_old(bpage, TRUE); - - if (UT_LIST_GET_LEN(buf_pool->LRU) >= BUF_LRU_OLD_MIN_LEN) { - - buf_pool->LRU_old_len++; - } - if (UT_LIST_GET_LEN(buf_pool->LRU) > BUF_LRU_OLD_MIN_LEN) { ut_ad(buf_pool->LRU_old); /* Adjust the length of the old block list if necessary */ + buf_page_set_old(bpage, TRUE); + buf_pool->LRU_old_len++; buf_LRU_old_adjust_len(); } else if (UT_LIST_GET_LEN(buf_pool->LRU) == BUF_LRU_OLD_MIN_LEN) { @@ -1269,6 +1265,8 @@ buf_LRU_add_block_to_end_low( defined: init it */ buf_LRU_old_init(); + } else { + buf_page_set_old(bpage, buf_pool->LRU_old != NULL); } /* If this is a zipped block with decompressed frame as well @@ -1302,7 +1300,6 @@ buf_LRU_add_block_low( UT_LIST_ADD_FIRST(LRU, buf_pool->LRU, bpage); - bpage->LRU_position = buf_pool_clock_tic(); bpage->freed_page_clock = buf_pool->freed_page_clock; } else { #ifdef UNIV_LRU_DEBUG @@ -1317,23 +1314,17 @@ buf_LRU_add_block_low( UT_LIST_INSERT_AFTER(LRU, buf_pool->LRU, buf_pool->LRU_old, bpage); buf_pool->LRU_old_len++; - - /* We copy the LRU position field of the previous block - to the new block */ - - bpage->LRU_position = (buf_pool->LRU_old)->LRU_position; } bpage->in_LRU_list = TRUE; - buf_page_set_old(bpage, old); - if (UT_LIST_GET_LEN(buf_pool->LRU) > BUF_LRU_OLD_MIN_LEN) { ut_ad(buf_pool->LRU_old); /* Adjust the length of the old block list if necessary */ + buf_page_set_old(bpage, old); buf_LRU_old_adjust_len(); } else if (UT_LIST_GET_LEN(buf_pool->LRU) == BUF_LRU_OLD_MIN_LEN) { @@ -1342,6 +1333,8 @@ buf_LRU_add_block_low( defined: init it */ buf_LRU_old_init(); + } else { + buf_page_set_old(bpage, buf_pool->LRU_old != NULL); } /* If this is a zipped block with decompressed frame as well @@ -1375,6 +1368,13 @@ buf_LRU_make_block_young( /*=====================*/ buf_page_t* bpage) /*!< in: control block */ { + //ut_ad(buf_pool_mutex_own()); + ut_ad(mutex_own(&LRU_list_mutex)); + + if (bpage->old) { + buf_pool->stat.n_pages_made_young++; + } + buf_LRU_remove_block(bpage); buf_LRU_add_block_low(bpage, FALSE); } @@ -1571,15 +1571,6 @@ not_freed: buf_pool->LRU_old = b; } -#ifdef UNIV_LRU_DEBUG - ut_a(prev_b->old - || !UT_LIST_GET_NEXT(LRU, b) - || UT_LIST_GET_NEXT(LRU, b)->old); - } else { - ut_a(!prev_b->old - || !UT_LIST_GET_NEXT(LRU, b) - || !UT_LIST_GET_NEXT(LRU, b)->old); -#endif /* UNIV_LRU_DEBUG */ } lru_len = UT_LIST_GET_LEN(buf_pool->LRU); @@ -1595,6 +1586,11 @@ not_freed: defined: init it */ buf_LRU_old_init(); } +#ifdef UNIV_LRU_DEBUG + /* Check that the "old" flag is consistent + in the block and its neighbours. */ + buf_page_set_old(b, buf_page_is_old(b)); +#endif /* UNIV_LRU_DEBUG */ } else { b->in_LRU_list = FALSE; buf_LRU_add_block_low(b, buf_page_is_old(b)); @@ -1985,6 +1981,52 @@ buf_LRU_block_free_hashed_page( buf_LRU_block_free_non_file_page(block, have_page_hash_mutex); } +/**********************************************************************//** +Updates buf_LRU_old_ratio. +@return updated old_pct */ +UNIV_INTERN +uint +buf_LRU_old_ratio_update( +/*=====================*/ + uint old_pct,/*!< in: Reserve this percentage of + the buffer pool for "old" blocks. */ + ibool adjust) /*!< in: TRUE=adjust the LRU list; + FALSE=just assign buf_LRU_old_ratio + during the initialization of InnoDB */ +{ + uint ratio; + + ratio = old_pct * BUF_LRU_OLD_RATIO_DIV / 100; + if (ratio < BUF_LRU_OLD_RATIO_MIN) { + ratio = BUF_LRU_OLD_RATIO_MIN; + } else if (ratio > BUF_LRU_OLD_RATIO_MAX) { + ratio = BUF_LRU_OLD_RATIO_MAX; + } + + if (adjust) { + //buf_pool_mutex_enter(); + mutex_enter(&LRU_list_mutex); + + if (ratio != buf_LRU_old_ratio) { + buf_LRU_old_ratio = ratio; + + if (UT_LIST_GET_LEN(buf_pool->LRU) + >= BUF_LRU_OLD_MIN_LEN) { + buf_LRU_old_adjust_len(); + } + } + + //buf_pool_mutex_exit(); + mutex_exit(&LRU_list_mutex); + } else { + buf_LRU_old_ratio = ratio; + } + + /* the reverse of + ratio = old_pct * BUF_LRU_OLD_RATIO_DIV / 100 */ + return((uint) (ratio * 100 / (double) BUF_LRU_OLD_RATIO_DIV + 0.5)); +} + /********************************************************************//** Update the historical stats that we are collecting for LRU eviction policy at the end of each interval. */ @@ -2023,6 +2065,218 @@ func_exit: memset(&buf_LRU_stat_cur, 0, sizeof buf_LRU_stat_cur); } +/********************************************************************//** +Dump the LRU page list to the specific file. */ +#define LRU_DUMP_FILE "ib_lru_dump" + +UNIV_INTERN +ibool +buf_LRU_file_dump(void) +/*===================*/ +{ + os_file_t dump_file = -1; + ibool success; + byte* buffer_base = NULL; + byte* buffer = NULL; + buf_page_t* bpage; + ulint buffers; + ulint offset; + ibool ret = FALSE; + ulint i; + + for (i = 0; i < srv_n_data_files; i++) { + if (strstr(srv_data_file_names[i], LRU_DUMP_FILE) != NULL) { + fprintf(stderr, + " InnoDB: The name '%s' seems to be used for" + " innodb_data_file_path. Dumping LRU list is not" + " done for safeness.\n", LRU_DUMP_FILE); + goto end; + } + } + + buffer_base = ut_malloc(2 * UNIV_PAGE_SIZE); + buffer = ut_align(buffer_base, UNIV_PAGE_SIZE); + if (!buffer) { + fprintf(stderr, + " InnoDB: cannot allocate buffer.\n"); + goto end; + } + + dump_file = os_file_create(LRU_DUMP_FILE, OS_FILE_OVERWRITE, + OS_FILE_NORMAL, OS_DATA_FILE, &success); + if (!success) { + os_file_get_last_error(TRUE); + fprintf(stderr, + " InnoDB: cannot open %s\n", LRU_DUMP_FILE); + goto end; + } + + mutex_enter(&LRU_list_mutex); + bpage = UT_LIST_GET_LAST(buf_pool->LRU); + + buffers = offset = 0; + while (bpage != NULL) { + if (offset == 0) { + memset(buffer, 0, UNIV_PAGE_SIZE); + } + + mach_write_to_4(buffer + offset * 4, bpage->space); + offset++; + mach_write_to_4(buffer + offset * 4, bpage->offset); + offset++; + + if (offset == UNIV_PAGE_SIZE/4) { + success = os_file_write(LRU_DUMP_FILE, dump_file, buffer, + (buffers << UNIV_PAGE_SIZE_SHIFT) & 0xFFFFFFFFUL, + (buffers >> (32 - UNIV_PAGE_SIZE_SHIFT)), + UNIV_PAGE_SIZE); + if (!success) { + mutex_exit(&LRU_list_mutex); + fprintf(stderr, + " InnoDB: cannot write page %lu of %s\n", + buffers, LRU_DUMP_FILE); + goto end; + } + buffers++; + offset = 0; + } + + bpage = UT_LIST_GET_PREV(LRU, bpage); + } + mutex_exit(&LRU_list_mutex); + + if (offset == 0) { + memset(buffer, 0, UNIV_PAGE_SIZE); + } + + mach_write_to_4(buffer + offset * 4, 0xFFFFFFFFUL); + offset++; + mach_write_to_4(buffer + offset * 4, 0xFFFFFFFFUL); + offset++; + + success = os_file_write(LRU_DUMP_FILE, dump_file, buffer, + (buffers << UNIV_PAGE_SIZE_SHIFT) & 0xFFFFFFFFUL, + (buffers >> (32 - UNIV_PAGE_SIZE_SHIFT)), + UNIV_PAGE_SIZE); + if (!success) { + goto end; + } + + ret = TRUE; +end: + if (dump_file != -1) + os_file_close(dump_file); + if (buffer_base) + ut_free(buffer_base); + + return(ret); +} +/********************************************************************//** +Read the pages based on the specific file.*/ +UNIV_INTERN +ibool +buf_LRU_file_restore(void) +/*======================*/ +{ + os_file_t dump_file = -1; + ibool success; + byte* buffer_base = NULL; + byte* buffer = NULL; + ulint buffers; + ulint offset; + ulint reads = 0; + ulint req = 0; + ibool terminated = FALSE; + ibool ret = FALSE; + + buffer_base = ut_malloc(2 * UNIV_PAGE_SIZE); + buffer = ut_align(buffer_base, UNIV_PAGE_SIZE); + if (!buffer) { + fprintf(stderr, + " InnoDB: cannot allocate buffer.\n"); + goto end; + } + + dump_file = os_file_create_simple_no_error_handling( + LRU_DUMP_FILE, OS_FILE_OPEN, OS_FILE_READ_ONLY, &success); + if (!success) { + os_file_get_last_error(TRUE); + fprintf(stderr, + " InnoDB: cannot open %s\n", LRU_DUMP_FILE); + goto end; + } + + buffers = 0; + while (!terminated) { + success = os_file_read(dump_file, buffer, + (buffers << UNIV_PAGE_SIZE_SHIFT) & 0xFFFFFFFFUL, + (buffers >> (32 - UNIV_PAGE_SIZE_SHIFT)), + UNIV_PAGE_SIZE); + if (!success) { + fprintf(stderr, + " InnoDB: cannot read page %lu of %s," + " or meet unexpected terminal.", + buffers, LRU_DUMP_FILE); + goto end; + } + + for (offset = 0; offset < UNIV_PAGE_SIZE/4; offset += 2) { + ulint space_id, zip_size, page_no; + ulint err; + ib_int64_t tablespace_version; + + space_id = mach_read_from_4(buffer + offset * 4); + page_no = mach_read_from_4(buffer + (offset + 1) * 4); + if (space_id == 0xFFFFFFFFUL + || page_no == 0xFFFFFFFFUL) { + terminated = TRUE; + break; + } + + if (offset % 16 == 15) { + os_aio_simulated_wake_handler_threads(); + buf_flush_free_margin(FALSE); + } + + zip_size = fil_space_get_zip_size(space_id); + if (UNIV_UNLIKELY(zip_size == ULINT_UNDEFINED)) { + continue; + } + + if (fil_area_is_exist(space_id, zip_size, page_no, 0, + zip_size ? zip_size : UNIV_PAGE_SIZE)) { + + tablespace_version = fil_space_get_version(space_id); + + req++; + reads += buf_read_page_low(&err, FALSE, BUF_READ_ANY_PAGE + | OS_AIO_SIMULATED_WAKE_LATER, + space_id, zip_size, TRUE, + tablespace_version, page_no, NULL); + buf_LRU_stat_inc_io(); + } + } + + buffers++; + } + + os_aio_simulated_wake_handler_threads(); + buf_flush_free_margin(FALSE); + + ut_print_timestamp(stderr); + fprintf(stderr, + " InnoDB: reading pages based on the dumped LRU list was done." + " (requested: %lu, read: %lu)\n", req, reads); + ret = TRUE; +end: + if (dump_file != -1) + os_file_close(dump_file); + if (buffer_base) + ut_free(buffer_base); + + return(ret); +} + #if defined UNIV_DEBUG || defined UNIV_BUF_DEBUG /**********************************************************************//** Validates the LRU list. @@ -2036,7 +2290,6 @@ buf_LRU_validate(void) buf_block_t* block; ulint old_len; ulint new_len; - ulint LRU_pos; ut_ad(buf_pool); //buf_pool_mutex_enter(); @@ -2046,7 +2299,11 @@ buf_LRU_validate(void) ut_a(buf_pool->LRU_old); old_len = buf_pool->LRU_old_len; - new_len = 3 * (UT_LIST_GET_LEN(buf_pool->LRU) / 8); + new_len = ut_min(UT_LIST_GET_LEN(buf_pool->LRU) + * buf_LRU_old_ratio / BUF_LRU_OLD_RATIO_DIV, + UT_LIST_GET_LEN(buf_pool->LRU) + - (BUF_LRU_OLD_TOLERANCE + + BUF_LRU_NON_OLD_MIN_LEN)); ut_a(old_len >= new_len - BUF_LRU_OLD_TOLERANCE); ut_a(old_len <= new_len + BUF_LRU_OLD_TOLERANCE); } @@ -2077,28 +2334,24 @@ buf_LRU_validate(void) } if (buf_page_is_old(bpage)) { - old_len++; - } + const buf_page_t* prev + = UT_LIST_GET_PREV(LRU, bpage); + const buf_page_t* next + = UT_LIST_GET_NEXT(LRU, bpage); - if (buf_pool->LRU_old && (old_len == 1)) { - ut_a(buf_pool->LRU_old == bpage); - } + if (!old_len++) { + ut_a(buf_pool->LRU_old == bpage); + } else { + ut_a(!prev || buf_page_is_old(prev)); + } - LRU_pos = buf_page_get_LRU_position(bpage); + ut_a(!next || buf_page_is_old(next)); + } bpage = UT_LIST_GET_NEXT(LRU, bpage); - - if (bpage) { - /* If the following assert fails, it may - not be an error: just the buf_pool clock - has wrapped around */ - ut_a(LRU_pos >= buf_page_get_LRU_position(bpage)); - } } - if (buf_pool->LRU_old) { - ut_a(buf_pool->LRU_old_len == old_len); - } + ut_a(buf_pool->LRU_old_len == old_len); mutex_exit(&LRU_list_mutex); mutex_enter(&free_list_mutex); @@ -2149,9 +2402,6 @@ buf_LRU_print(void) //buf_pool_mutex_enter(); mutex_enter(&LRU_list_mutex); - fprintf(stderr, "Pool ulint clock %lu\n", - (ulong) buf_pool->ulint_clock); - bpage = UT_LIST_GET_FIRST(buf_pool->LRU); while (bpage != NULL) { @@ -2182,18 +2432,16 @@ buf_LRU_print(void) const byte* frame; case BUF_BLOCK_FILE_PAGE: frame = buf_block_get_frame((buf_block_t*) bpage); - fprintf(stderr, "\nLRU pos %lu type %lu" + fprintf(stderr, "\ntype %lu" " index id %lu\n", - (ulong) buf_page_get_LRU_position(bpage), (ulong) fil_page_get_type(frame), (ulong) ut_dulint_get_low( btr_page_get_index_id(frame))); break; case BUF_BLOCK_ZIP_PAGE: frame = bpage->zip.data; - fprintf(stderr, "\nLRU pos %lu type %lu size %lu" + fprintf(stderr, "\ntype %lu size %lu" " index id %lu\n", - (ulong) buf_page_get_LRU_position(bpage), (ulong) fil_page_get_type(frame), (ulong) buf_page_get_zip_size(bpage), (ulong) ut_dulint_get_low( @@ -2201,8 +2449,7 @@ buf_LRU_print(void) break; default: - fprintf(stderr, "\nLRU pos %lu !state %lu!\n", - (ulong) buf_page_get_LRU_position(bpage), + fprintf(stderr, "\n!state %lu!\n", (ulong) buf_page_get_state(bpage)); break; } === modified file 'storage/xtradb/buf/buf0rea.c' --- a/storage/xtradb/buf/buf0rea.c 2009-11-13 21:26:08 +0000 +++ b/storage/xtradb/buf/buf0rea.c 2010-01-15 15:58:25 +0000 @@ -38,14 +38,6 @@ Created 11/5/1995 Heikki Tuuri #include "srv0start.h" #include "srv0srv.h" -/** The size in blocks of the area where the random read-ahead algorithm counts -the accessed pages when deciding whether to read-ahead */ -#define BUF_READ_AHEAD_RANDOM_AREA BUF_READ_AHEAD_AREA - -/** There must be at least this many pages in buf_pool in the area to start -a random read-ahead */ -#define BUF_READ_AHEAD_RANDOM_THRESHOLD (1 + BUF_READ_AHEAD_RANDOM_AREA / 2) - /** The linear read-ahead area size */ #define BUF_READ_AHEAD_LINEAR_AREA BUF_READ_AHEAD_AREA @@ -62,8 +54,9 @@ flag is cleared and the x-lock released @return 1 if a read request was queued, 0 if the page already resided in buf_pool, or if the page is in the doublewrite buffer blocks in which case it is never read into the pool, or if the tablespace does -not exist or is being dropped */ -static +not exist or is being dropped +@return 1 if read request is issued. 0 if it is not */ +UNIV_INTERN ulint buf_read_page_low( /*==============*/ @@ -82,7 +75,8 @@ buf_read_page_low( treat the tablespace as dropped; this is a timestamp we use to stop dangling page reads from a tablespace which we have DISCARDed + IMPORTed back */ - ulint offset) /*!< in: page number */ + ulint offset, /*!< in: page number */ + trx_t* trx) { buf_page_t* bpage; ulint wake_later; @@ -183,15 +177,15 @@ not_to_recover: ut_ad(buf_page_in_file(bpage)); if (zip_size) { - *err = fil_io(OS_FILE_READ | wake_later, + *err = _fil_io(OS_FILE_READ | wake_later, sync, space, zip_size, offset, 0, zip_size, - bpage->zip.data, bpage); + bpage->zip.data, bpage, trx); } else { ut_a(buf_page_get_state(bpage) == BUF_BLOCK_FILE_PAGE); - *err = fil_io(OS_FILE_READ | wake_later, + *err = _fil_io(OS_FILE_READ | wake_later, sync, space, 0, offset, 0, UNIV_PAGE_SIZE, - ((buf_block_t*) bpage)->frame, bpage); + ((buf_block_t*) bpage)->frame, bpage, trx); } ut_a(*err == DB_SUCCESS); @@ -205,206 +199,33 @@ not_to_recover: } /********************************************************************//** -Applies a random read-ahead in buf_pool if there are at least a threshold -value of accessed pages from the random read-ahead area. Does not read any -page, not even the one at the position (space, offset), if the read-ahead -mechanism is not activated. NOTE 1: the calling thread may own latches on -pages: to avoid deadlocks this function must be written such that it cannot -end up waiting for these latches! NOTE 2: the calling thread must want -access to the page given: this rule is set to prevent unintended read-aheads -performed by ibuf routines, a situation which could result in a deadlock if -the OS does not support asynchronous i/o. -@return number of page read requests issued; NOTE that if we read ibuf -pages, it may happen that the page at the given page number does not -get read even if we return a positive value! */ -static -ulint -buf_read_ahead_random( -/*==================*/ - ulint space, /*!< in: space id */ - ulint zip_size,/*!< in: compressed page size in bytes, or 0 */ - ulint offset) /*!< in: page number of a page which the current thread - wants to access */ -{ - ib_int64_t tablespace_version; - ulint recent_blocks = 0; - ulint count; - ulint LRU_recent_limit; - ulint ibuf_mode; - ulint low, high; - ulint err; - ulint i; - ulint buf_read_ahead_random_area; - -// /* We have currently disabled random readahead */ -// return(0); - - if (!(srv_read_ahead & 1)) { - return(0); - } - - if (srv_startup_is_before_trx_rollback_phase) { - /* No read-ahead to avoid thread deadlocks */ - return(0); - } - - if (ibuf_bitmap_page(zip_size, offset) - || trx_sys_hdr_page(space, offset)) { - - /* If it is an ibuf bitmap page or trx sys hdr, we do - no read-ahead, as that could break the ibuf page access - order */ - - return(0); - } - - /* Remember the tablespace version before we ask te tablespace size - below: if DISCARD + IMPORT changes the actual .ibd file meanwhile, we - do not try to read outside the bounds of the tablespace! */ - - tablespace_version = fil_space_get_version(space); - - buf_read_ahead_random_area = BUF_READ_AHEAD_RANDOM_AREA; - - low = (offset / buf_read_ahead_random_area) - * buf_read_ahead_random_area; - high = (offset / buf_read_ahead_random_area + 1) - * buf_read_ahead_random_area; - if (high > fil_space_get_size(space)) { - - high = fil_space_get_size(space); - } - - /* Get the minimum LRU_position field value for an initial segment - of the LRU list, to determine which blocks have recently been added - to the start of the list. */ - - LRU_recent_limit = buf_LRU_get_recent_limit(); - - //buf_pool_mutex_enter(); - mutex_enter(&buf_pool_mutex); - - if (buf_pool->n_pend_reads - > buf_pool->curr_size / BUF_READ_AHEAD_PEND_LIMIT) { - //buf_pool_mutex_exit(); - mutex_exit(&buf_pool_mutex); - - return(0); - } - mutex_exit(&buf_pool_mutex); - - /* Count how many blocks in the area have been recently accessed, - that is, reside near the start of the LRU list. */ - - rw_lock_s_lock(&page_hash_latch); - for (i = low; i < high; i++) { - const buf_page_t* bpage = buf_page_hash_get(space, i); - - if (bpage - && buf_page_is_accessed(bpage) - && (buf_page_get_LRU_position(bpage) > LRU_recent_limit)) { - - recent_blocks++; - - if (recent_blocks >= BUF_READ_AHEAD_RANDOM_THRESHOLD) { - - //buf_pool_mutex_exit(); - rw_lock_s_unlock(&page_hash_latch); - goto read_ahead; - } - } - } - - //buf_pool_mutex_exit(); - rw_lock_s_unlock(&page_hash_latch); - /* Do nothing */ - return(0); - -read_ahead: - /* Read all the suitable blocks within the area */ - - if (ibuf_inside()) { - ibuf_mode = BUF_READ_IBUF_PAGES_ONLY; - } else { - ibuf_mode = BUF_READ_ANY_PAGE; - } - - count = 0; - - for (i = low; i < high; i++) { - /* It is only sensible to do read-ahead in the non-sync aio - mode: hence FALSE as the first parameter */ - - if (!ibuf_bitmap_page(zip_size, i)) { - count += buf_read_page_low( - &err, FALSE, - ibuf_mode | OS_AIO_SIMULATED_WAKE_LATER, - space, zip_size, FALSE, - tablespace_version, i); - if (err == DB_TABLESPACE_DELETED) { - ut_print_timestamp(stderr); - fprintf(stderr, - " InnoDB: Warning: in random" - " readahead trying to access\n" - "InnoDB: tablespace %lu page %lu,\n" - "InnoDB: but the tablespace does not" - " exist or is just being dropped.\n", - (ulong) space, (ulong) i); - } - } - } - - /* In simulated aio we wake the aio handler threads only after - queuing all aio requests, in native aio the following call does - nothing: */ - - os_aio_simulated_wake_handler_threads(); - -#ifdef UNIV_DEBUG - if (buf_debug_prints && (count > 0)) { - fprintf(stderr, - "Random read-ahead space %lu offset %lu pages %lu\n", - (ulong) space, (ulong) offset, - (ulong) count); - } -#endif /* UNIV_DEBUG */ - - ++srv_read_ahead_rnd; - return(count); -} - -/********************************************************************//** High-level function which reads a page asynchronously from a file to the buffer buf_pool if it is not already there. Sets the io_fix flag and sets an exclusive lock on the buffer frame. The flag is cleared and the x-lock -released by the i/o-handler thread. Does a random read-ahead if it seems -sensible. -@return number of page read requests issued: this can be greater than -1 if read-ahead occurred */ +released by the i/o-handler thread. +@return TRUE if page has been read in, FALSE in case of failure */ UNIV_INTERN -ulint +ibool buf_read_page( /*==========*/ ulint space, /*!< in: space id */ ulint zip_size,/*!< in: compressed page size in bytes, or 0 */ - ulint offset) /*!< in: page number */ + ulint offset, /*!< in: page number */ + trx_t* trx) { ib_int64_t tablespace_version; ulint count; - ulint count2; ulint err; tablespace_version = fil_space_get_version(space); - count = buf_read_ahead_random(space, zip_size, offset); - /* We do the i/o in the synchronous aio mode to save thread switches: hence TRUE */ - count2 = buf_read_page_low(&err, TRUE, BUF_READ_ANY_PAGE, space, - zip_size, FALSE, - tablespace_version, offset); - srv_buf_pool_reads+= count2; + count = buf_read_page_low(&err, TRUE, BUF_READ_ANY_PAGE, space, + zip_size, FALSE, + tablespace_version, offset, trx); + srv_buf_pool_reads += count; if (err == DB_TABLESPACE_DELETED) { ut_print_timestamp(stderr); fprintf(stderr, @@ -421,14 +242,14 @@ buf_read_page( /* Increment number of I/O operations used for LRU policy. */ buf_LRU_stat_inc_io(); - return(count + count2); + return(count > 0); } /********************************************************************//** Applies linear read-ahead if in the buf_pool the page is a border page of a linear read-ahead area and all the pages in the area have been accessed. Does not read any page if the read-ahead mechanism is not activated. Note -that the the algorithm looks at the 'natural' adjacent successor and +that the algorithm looks at the 'natural' adjacent successor and predecessor of the page, which on the leaf level of a B-tree are the next and previous page in the chain of leaves. To know these, the page specified in (space, offset) must already be present in the buf_pool. Thus, the @@ -454,8 +275,9 @@ buf_read_ahead_linear( /*==================*/ ulint space, /*!< in: space id */ ulint zip_size,/*!< in: compressed page size in bytes, or 0 */ - ulint offset) /*!< in: page number of a page; NOTE: the current thread + ulint offset, /*!< in: page number of a page; NOTE: the current thread must want access to this page (see NOTE 3 above) */ + trx_t* trx) { ib_int64_t tablespace_version; buf_page_t* bpage; @@ -557,9 +379,17 @@ buf_read_ahead_linear( fail_count++; } else if (pred_bpage) { - int res = (ut_ulint_cmp( - buf_page_get_LRU_position(bpage), - buf_page_get_LRU_position(pred_bpage))); + /* Note that buf_page_is_accessed() returns + the time of the first access. If some blocks + of the extent existed in the buffer pool at + the time of a linear access pattern, the first + access times may be nonmonotonic, even though + the latest access times were linear. The + threshold (srv_read_ahead_factor) should help + a little against this. */ + int res = ut_ulint_cmp( + buf_page_is_accessed(bpage), + buf_page_is_accessed(pred_bpage)); /* Accesses not in the right order */ if (res != 0 && res != asc_or_desc) { fail_count++; @@ -670,7 +500,7 @@ buf_read_ahead_linear( count += buf_read_page_low( &err, FALSE, ibuf_mode | OS_AIO_SIMULATED_WAKE_LATER, - space, zip_size, FALSE, tablespace_version, i); + space, zip_size, FALSE, tablespace_version, i, trx); if (err == DB_TABLESPACE_DELETED) { ut_print_timestamp(stderr); fprintf(stderr, @@ -705,7 +535,7 @@ buf_read_ahead_linear( LRU policy decision. */ buf_LRU_stat_inc_io(); - ++srv_read_ahead_seq; + buf_pool->stat.n_ra_pages_read += count; return(count); } @@ -760,7 +590,7 @@ buf_read_ibuf_merge_pages( buf_read_page_low(&err, sync && (i + 1 == n_stored), BUF_READ_ANY_PAGE, space_ids[i], zip_size, TRUE, space_versions[i], - page_nos[i]); + page_nos[i], NULL); if (UNIV_UNLIKELY(err == DB_TABLESPACE_DELETED)) { tablespace_deleted: @@ -857,12 +687,12 @@ buf_read_recv_pages( if ((i + 1 == n_stored) && sync) { buf_read_page_low(&err, TRUE, BUF_READ_ANY_PAGE, space, zip_size, TRUE, tablespace_version, - page_nos[i]); + page_nos[i], NULL); } else { buf_read_page_low(&err, FALSE, BUF_READ_ANY_PAGE | OS_AIO_SIMULATED_WAKE_LATER, space, zip_size, TRUE, - tablespace_version, page_nos[i]); + tablespace_version, page_nos[i], NULL); } } === modified file 'storage/xtradb/data/data0type.c' --- a/storage/xtradb/data/data0type.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/data/data0type.c 2010-01-06 12:00:14 +0000 @@ -237,6 +237,22 @@ dtype_print( fputs("DATA_SYS", stderr); break; + case DATA_FLOAT: + fputs("DATA_FLOAT", stderr); + break; + + case DATA_DOUBLE: + fputs("DATA_DOUBLE", stderr); + break; + + case DATA_DECIMAL: + fputs("DATA_DECIMAL", stderr); + break; + + case DATA_VARMYSQL: + fputs("DATA_VARMYSQL", stderr); + break; + default: fprintf(stderr, "type %lu", (ulong) mtype); break; === modified file 'storage/xtradb/dict/dict0crea.c' --- a/storage/xtradb/dict/dict0crea.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/dict/dict0crea.c 2010-01-06 12:00:14 +0000 @@ -1387,7 +1387,7 @@ dict_create_add_foreign_field_to_diction Add a single foreign key definition to the data dictionary tables in the database. We also generate names to constraints that were not named by the user. A generated constraint has a name of the format -databasename/tablename_ibfk_<number>, where the numbers start from 1, and +databasename/tablename_ibfk_NUMBER, where the numbers start from 1, and are given locally for this table, that is, the number is not global, as in the old format constraints < 4.0.18 it used to be. @return error code or DB_SUCCESS */ === modified file 'storage/xtradb/dict/dict0dict.c' --- a/storage/xtradb/dict/dict0dict.c 2009-12-03 11:34:11 +0000 +++ b/storage/xtradb/dict/dict0dict.c 2010-01-15 15:58:25 +0000 @@ -82,9 +82,10 @@ static char dict_ibfk[] = "_ibfk_"; /*******************************************************************//** Tries to find column names for the index and sets the col field of the -index. */ +index. +@return TRUE if the column names were found */ static -void +ibool dict_index_find_cols( /*=================*/ dict_table_t* table, /*!< in: table */ @@ -1261,7 +1262,7 @@ dict_index_too_big_for_undo( = TRX_UNDO_PAGE_HDR - TRX_UNDO_PAGE_HDR_SIZE + 2 /* next record pointer */ + 1 /* type_cmpl */ - + 11 /* trx->undo_no */ - 11 /* table->id */ + + 11 /* trx->undo_no */ + 11 /* table->id */ + 1 /* rec_get_info_bits() */ + 11 /* DB_TRX_ID */ + 11 /* DB_ROLL_PTR */ @@ -1440,7 +1441,11 @@ dict_index_too_big_for_tree( goto add_field_size; } + if (srv_relax_table_creation) { + field_max_size = dict_col_get_min_size(col); + } else { field_max_size = dict_col_get_max_size(col); + } field_ext_max_size = field_max_size < 256 ? 1 : 2; if (field->prefix_len) { @@ -1493,7 +1498,7 @@ add_field_size: /**********************************************************************//** Adds an index to the dictionary cache. -@return DB_SUCCESS or DB_TOO_BIG_RECORD */ +@return DB_SUCCESS, DB_TOO_BIG_RECORD, or DB_CORRUPTION */ UNIV_INTERN ulint dict_index_add_to_cache( @@ -1519,7 +1524,10 @@ dict_index_add_to_cache( ut_a(!dict_index_is_clust(index) || UT_LIST_GET_LEN(table->indexes) == 0); - dict_index_find_cols(table, index); + if (!dict_index_find_cols(table, index)) { + + return(DB_CORRUPTION); + } /* Build the cache internal representation of the index, containing also the added system fields */ @@ -1732,9 +1740,10 @@ dict_index_remove_from_cache( /*******************************************************************//** Tries to find column names for the index and sets the col field of the -index. */ +index. +@return TRUE if the column names were found */ static -void +ibool dict_index_find_cols( /*=================*/ dict_table_t* table, /*!< in: table */ @@ -1759,17 +1768,21 @@ dict_index_find_cols( } } +#ifdef UNIV_DEBUG /* It is an error not to find a matching column. */ fputs("InnoDB: Error: no matching column for ", stderr); ut_print_name(stderr, NULL, FALSE, field->name); fputs(" in ", stderr); dict_index_name_print(stderr, NULL, index); fputs("!\n", stderr); - ut_error; +#endif /* UNIV_DEBUG */ + return(FALSE); found: ; } + + return(TRUE); } #endif /* !UNIV_HOTBACKUP */ @@ -4711,6 +4724,26 @@ dict_ind_init(void) dict_ind_redundant->cached = dict_ind_compact->cached = TRUE; } +/**********************************************************************//** +Frees dict_ind_redundant and dict_ind_compact. */ +static +void +dict_ind_free(void) +/*===============*/ +{ + dict_table_t* table; + + table = dict_ind_compact->table; + dict_mem_index_free(dict_ind_compact); + dict_ind_compact = NULL; + dict_mem_table_free(table); + + table = dict_ind_redundant->table; + dict_mem_index_free(dict_ind_redundant); + dict_ind_redundant = NULL; + dict_mem_table_free(table); +} + #ifndef UNIV_HOTBACKUP /**********************************************************************//** Get index by name @@ -4836,4 +4869,55 @@ dict_table_check_for_dup_indexes( } } #endif /* UNIV_DEBUG */ + +/************************************************************************** +Closes the data dictionary module. */ +UNIV_INTERN +void +dict_close(void) +/*============*/ +{ + ulint i; + + /* Free the hash elements. We don't remove them from the table + because we are going to destroy the table anyway. */ + for (i = 0; i < hash_get_n_cells(dict_sys->table_hash); i++) { + dict_table_t* table; + + table = HASH_GET_FIRST(dict_sys->table_hash, i); + + while (table) { + dict_table_t* prev_table = table; + + table = HASH_GET_NEXT(name_hash, prev_table); +#ifdef UNIV_DEBUG + ut_a(prev_table->magic_n == DICT_TABLE_MAGIC_N); +#endif + /* Acquire only because it's a pre-condition. */ + mutex_enter(&dict_sys->mutex); + + dict_table_remove_from_cache(prev_table); + + mutex_exit(&dict_sys->mutex); + } + } + + hash_table_free(dict_sys->table_hash); + + /* The elements are the same instance as in dict_sys->table_hash, + therefore we don't delete the individual elements. */ + hash_table_free(dict_sys->table_id_hash); + + dict_ind_free(); + + mutex_free(&dict_sys->mutex); + + rw_lock_free(&dict_operation_lock); + memset(&dict_operation_lock, 0x0, sizeof(dict_operation_lock)); + + mutex_free(&dict_foreign_err_mutex); + + mem_free(dict_sys); + dict_sys = NULL; +} #endif /* !UNIV_HOTBACKUP */ === modified file 'storage/xtradb/fil/fil0fil.c' --- a/storage/xtradb/fil/fil0fil.c 2009-11-29 23:08:56 +0000 +++ b/storage/xtradb/fil/fil0fil.c 2010-01-15 15:58:25 +0000 @@ -327,6 +327,17 @@ fil_get_space_id_for_table( /*=======================*/ const char* name); /*!< in: table name in the standard 'databasename/tablename' format */ +/*******************************************************************//** +Frees a space object from the tablespace memory cache. Closes the files in +the chain but does not delete them. There must not be any pending i/o's or +flushes on the files. */ +static +ibool +fil_space_free( +/*===========*/ + /* out: TRUE if success */ + ulint id, /* in: space id */ + ibool own_mutex);/* in: TRUE if own system->mutex */ /********************************************************************//** Reads data from a space to a buffer. Remember that the possible incomplete blocks at the end of file are ignored: they are not taken into account when @@ -600,6 +611,11 @@ fil_node_create( UT_LIST_ADD_LAST(chain, space->chain, node); + if (id < SRV_LOG_SPACE_FIRST_ID && fil_system->max_assigned_id < id) { + + fil_system->max_assigned_id = id; + } + mutex_exit(&fil_system->mutex); } @@ -619,12 +635,10 @@ fil_node_open_file( ulint size_high; ibool ret; ibool success; -#ifndef UNIV_HOTBACKUP byte* buf2; byte* page; ulint space_id; ulint flags; -#endif /* !UNIV_HOTBACKUP */ ut_ad(mutex_own(&(system->mutex))); ut_a(node->n_pending == 0); @@ -660,9 +674,12 @@ fil_node_open_file( size_bytes = (((ib_int64_t)size_high) << 32) + (ib_int64_t)size_low; #ifdef UNIV_HOTBACKUP - node->size = (ulint) (size_bytes / UNIV_PAGE_SIZE); - /* TODO: adjust to zip_size, like below? */ -#else + if (space->id == 0) { + node->size = (ulint) (size_bytes / UNIV_PAGE_SIZE); + os_file_close(node->handle); + goto add_size; + } +#endif /* UNIV_HOTBACKUP */ ut_a(space->purpose != FIL_LOG); ut_a(space->id != 0); @@ -741,7 +758,10 @@ fil_node_open_file( (size_bytes / dict_table_flags_to_zip_size(flags)); } -#endif + +#ifdef UNIV_HOTBACKUP +add_size: +#endif /* UNIV_HOTBACKUP */ space->size += node->size; } @@ -961,7 +981,7 @@ close_more: " while the maximum\n" "InnoDB: allowed value would be %lu.\n" "InnoDB: You may need to raise the value of" - " innodb_max_files_open in\n" + " innodb_open_files in\n" "InnoDB: my.cnf.\n", (ulong) fil_system->n_open, (ulong) fil_system->max_n_open); @@ -1141,7 +1161,7 @@ try_again: mutex_exit(&fil_system->mutex); - fil_space_free(namesake_id); + fil_space_free(namesake_id, FALSE); goto try_again; } @@ -1266,17 +1286,21 @@ Frees a space object from the tablespace the chain but does not delete them. There must not be any pending i/o's or flushes on the files. @return TRUE if success */ -UNIV_INTERN +static ibool fil_space_free( /*===========*/ - ulint id) /*!< in: space id */ + /* out: TRUE if success */ + ulint id, /* in: space id */ + ibool own_mutex) /* in: TRUE if own system->mutex */ { fil_space_t* space; fil_space_t* namespace; fil_node_t* fil_node; - mutex_enter(&fil_system->mutex); + if (!own_mutex) { + mutex_enter(&fil_system->mutex); + } space = fil_space_get_by_id(id); @@ -1323,7 +1347,9 @@ fil_space_free( ut_a(0 == UT_LIST_GET_LEN(space->chain)); - mutex_exit(&fil_system->mutex); + if (!own_mutex) { + mutex_exit(&fil_system->mutex); + } rw_lock_free(&(space->latch)); @@ -1541,7 +1567,7 @@ fil_open_log_and_system_tablespace_files fprintf(stderr, "InnoDB: Warning: you must" " raise the value of" - " innodb_max_open_files in\n" + " innodb_open_files in\n" "InnoDB: my.cnf! Remember that" " InnoDB keeps all log files" " and all system\n" @@ -1583,6 +1609,8 @@ fil_close_all_files(void) space = UT_LIST_GET_FIRST(fil_system->space_list); while (space != NULL) { + fil_space_t* prev_space = space; + node = UT_LIST_GET_FIRST(space->chain); while (node != NULL) { @@ -1592,6 +1620,7 @@ fil_close_all_files(void) node = UT_LIST_GET_NEXT(chain, node); } space = UT_LIST_GET_NEXT(space_list, space); + fil_space_free(prev_space->id, TRUE); } mutex_exit(&fil_system->mutex); @@ -2223,7 +2252,7 @@ try_again: #endif /* printf("Deleting tablespace %s id %lu\n", space->name, id); */ - success = fil_space_free(id); + success = fil_space_free(id, FALSE); if (success) { success = os_file_delete(path); @@ -2929,7 +2958,6 @@ fil_open_single_table_tablespace( byte* page; ulint space_id; ulint space_flags; - ibool ret = TRUE; filepath = fil_make_ibd_name(name, FALSE); @@ -3330,7 +3358,7 @@ skip_write: (ulong) space_id, (ulong) space_flags, (ulong) id, (ulong) flags); - ret = FALSE; + success = FALSE; goto func_exit; } @@ -3350,7 +3378,7 @@ func_exit: os_file_close(file); mem_free(filepath); - return(ret); + return(success); } #endif /* !UNIV_HOTBACKUP */ @@ -3566,7 +3594,7 @@ fil_load_single_table_tablespace( fprintf(stderr, "InnoDB: Renaming tablespace %s of id %lu,\n" "InnoDB: to %s_ibbackup_old_vers_<timestamp>\n" - "InnoDB: because its size %lld is too small" + "InnoDB: because its size %" PRId64 " is too small" " (< 4 pages 16 kB each),\n" "InnoDB: or the space id in the file header" " is not sensible.\n" @@ -3628,7 +3656,17 @@ fil_load_single_table_tablespace( if (!success) { - goto func_exit; + if (srv_force_recovery > 0) { + fprintf(stderr, + "InnoDB: innodb_force_recovery" + " was set to %lu. Continuing crash recovery\n" + "InnoDB: even though the tablespace creation" + " of this table failed.\n", + srv_force_recovery); + goto func_exit; + } + + exit(1); } /* We do not use the size information we have about the file, because @@ -4163,7 +4201,7 @@ fil_extend_space_to_desired_size( node->name, node->handle, buf, offset_low, offset_high, page_size * n_pages, - NULL, NULL); + NULL, NULL, NULL); #endif if (success) { node->size += n_pages; @@ -4490,7 +4528,7 @@ Reads or writes data. This operation is i/o on a tablespace which does not exist */ UNIV_INTERN ulint -fil_io( +_fil_io( /*===*/ ulint type, /*!< in: OS_FILE_READ or OS_FILE_WRITE, ORed to OS_FILE_LOG, if a log i/o @@ -4515,8 +4553,9 @@ fil_io( void* buf, /*!< in/out: buffer where to store read data or from where to write; in aio this must be appropriately aligned */ - void* message) /*!< in: message for aio handler if non-sync + void* message, /*!< in: message for aio handler if non-sync aio used, else ignored */ + trx_t* trx) { ulint mode; fil_space_t* space; @@ -4686,7 +4725,7 @@ fil_io( #else /* Queue the aio request */ ret = os_aio(type, mode | wake_later, node->name, node->handle, buf, - offset_low, offset_high, len, node, message); + offset_low, offset_high, len, node, message, trx); #endif ut_a(ret); @@ -4706,6 +4745,78 @@ fil_io( return(DB_SUCCESS); } +/********************************************************************//** +Confirm whether the parameters are valid or not */ +UNIV_INTERN +ibool +fil_area_is_exist( +/*==============*/ + ulint space_id, /*!< in: space id */ + ulint zip_size, /*!< in: compressed page size in bytes; + 0 for uncompressed pages */ + ulint block_offset, /*!< in: offset in number of blocks */ + ulint byte_offset, /*!< in: remainder of offset in bytes; in + aio this must be divisible by the OS block + size */ + ulint len) /*!< in: how many bytes to read or write; this + must not cross a file boundary; in aio this + must be a block size multiple */ +{ + fil_space_t* space; + fil_node_t* node; + + /* Reserve the fil_system mutex and make sure that we can open at + least one file while holding it, if the file is not already open */ + + fil_mutex_enter_and_prepare_for_io(space_id); + + space = fil_space_get_by_id(space_id); + + if (!space) { + mutex_exit(&fil_system->mutex); + return(FALSE); + } + + node = UT_LIST_GET_FIRST(space->chain); + + for (;;) { + if (UNIV_UNLIKELY(node == NULL)) { + mutex_exit(&fil_system->mutex); + return(FALSE); + } + + if (space->id != 0 && node->size == 0) { + /* We do not know the size of a single-table tablespace + before we open the file */ + + break; + } + + if (node->size > block_offset) { + /* Found! */ + break; + } else { + block_offset -= node->size; + node = UT_LIST_GET_NEXT(chain, node); + } + } + + /* Open file if closed */ + fil_node_prepare_for_io(node, fil_system, space); + fil_node_complete_io(node, fil_system, OS_FILE_READ); + + /* Check that at least the start offset is within the bounds of a + single-table tablespace */ + if (UNIV_UNLIKELY(node->size <= block_offset) + && space->id != 0 && space->purpose == FIL_TABLESPACE) { + mutex_exit(&fil_system->mutex); + return(FALSE); + } + + mutex_exit(&fil_system->mutex); + return(TRUE); +} + #ifndef UNIV_HOTBACKUP /**********************************************************************//** Waits for an aio operation to complete. This function is used to write the @@ -5065,6 +5176,29 @@ fil_page_get_type( return(mach_read_from_2(page + FIL_PAGE_TYPE)); } +/******************************************************************** +Initializes the tablespace memory cache. */ +UNIV_INTERN +void +fil_close(void) +/*===========*/ +{ + /* The mutex should already have been freed. */ + ut_ad(fil_system->mutex.magic_n == 0); + + hash_table_free(fil_system->spaces); + + hash_table_free(fil_system->name_hash); + + ut_a(UT_LIST_GET_LEN(fil_system->LRU) == 0); + ut_a(UT_LIST_GET_LEN(fil_system->unflushed_spaces) == 0); + ut_a(UT_LIST_GET_LEN(fil_system->space_list) == 0); + + mem_free(fil_system); + + fil_system = NULL; +} + /************************************************************************* Return local hash table informations. */ === modified file 'storage/xtradb/fsp/fsp0fsp.c' --- a/storage/xtradb/fsp/fsp0fsp.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/fsp/fsp0fsp.c 2010-01-06 12:00:14 +0000 @@ -232,6 +232,9 @@ the extent are free and which contain ol #define XDES_ARR_OFFSET (FSP_HEADER_OFFSET + FSP_HEADER_SIZE) #ifndef UNIV_HOTBACKUP +/* Flag to indicate if we have printed the tablespace full error. */ +static ibool fsp_tbs_full_error_printed = FALSE; + /**********************************************************************//** Returns an extent to the free list of a space. */ static @@ -1099,7 +1102,7 @@ fsp_header_inc_size( /**********************************************************************//** Gets the current free limit of the system tablespace. The free limit -means the place of the first page which has never been put to the the +means the place of the first page which has never been put to the free list for allocation. The space above that address is initialized to zero. Sets also the global variable log_fsp_current_free_limit. @return free limit in megabytes */ @@ -1218,6 +1221,19 @@ fsp_try_extend_data_file( if (space == 0 && !srv_auto_extend_last_data_file) { + /* We print the error message only once to avoid + spamming the error log. Note that we don't need + to reset the flag to FALSE as dealing with this + error requires server restart. */ + if (fsp_tbs_full_error_printed == FALSE) { + fprintf(stderr, + "InnoDB: Error: Data file(s) ran" + " out of space.\n" + "Please add another data file or" + " use \'autoextend\' for the last" + " data file.\n"); + fsp_tbs_full_error_printed = TRUE; + } return(FALSE); } @@ -1832,6 +1848,8 @@ fsp_seg_inode_page_find_used( if (!ut_dulint_is_zero(mach_read_from_8(inode + FSEG_ID))) { /* This is used */ + ut_ad(mach_read_from_4(inode + FSEG_MAGIC_N) + == FSEG_MAGIC_N_VALUE); return(i); } } @@ -1863,6 +1881,9 @@ fsp_seg_inode_page_find_free( return(i); } + + ut_ad(mach_read_from_4(inode + FSEG_MAGIC_N) + == FSEG_MAGIC_N_VALUE); } return(ULINT_UNDEFINED); @@ -1981,6 +2002,8 @@ fsp_alloc_seg_inode( page + FSEG_INODE_PAGE_NODE, mtr); } + ut_ad(ut_dulint_is_zero(mach_read_from_8(inode + FSEG_ID)) + || mach_read_from_4(inode + FSEG_MAGIC_N) == FSEG_MAGIC_N_VALUE); return(inode); } @@ -2018,7 +2041,7 @@ fsp_free_seg_inode( } mlog_write_dulint(inode + FSEG_ID, ut_dulint_zero, mtr); - mlog_write_ulint(inode + FSEG_MAGIC_N, 0, MLOG_4BYTES, mtr); + mlog_write_ulint(inode + FSEG_MAGIC_N, 0xfa051ce3, MLOG_4BYTES, mtr); if (ULINT_UNDEFINED == fsp_seg_inode_page_find_used(page, zip_size, mtr)) { @@ -2034,11 +2057,11 @@ fsp_free_seg_inode( /**********************************************************************//** Returns the file segment inode, page x-latched. -@return segment inode, page x-latched */ +@return segment inode, page x-latched; NULL if the inode is free */ static fseg_inode_t* -fseg_inode_get( -/*===========*/ +fseg_inode_try_get( +/*===============*/ fseg_header_t* header, /*!< in: segment header */ ulint space, /*!< in: space id */ ulint zip_size,/*!< in: compressed page size in bytes @@ -2054,12 +2077,38 @@ fseg_inode_get( inode = fut_get_ptr(space, zip_size, inode_addr, RW_X_LATCH, mtr); - ut_ad(mach_read_from_4(inode + FSEG_MAGIC_N) == FSEG_MAGIC_N_VALUE); + if (UNIV_UNLIKELY + (ut_dulint_is_zero(mach_read_from_8(inode + FSEG_ID)))) { + + inode = NULL; + } else { + ut_ad(mach_read_from_4(inode + FSEG_MAGIC_N) + == FSEG_MAGIC_N_VALUE); + } return(inode); } /**********************************************************************//** +Returns the file segment inode, page x-latched. +@return segment inode, page x-latched */ +static +fseg_inode_t* +fseg_inode_get( +/*===========*/ + fseg_header_t* header, /*!< in: segment header */ + ulint space, /*!< in: space id */ + ulint zip_size,/*!< in: compressed page size in bytes + or 0 for uncompressed pages */ + mtr_t* mtr) /*!< in: mtr handle */ +{ + fseg_inode_t* inode + = fseg_inode_try_get(header, space, zip_size, mtr); + ut_a(inode); + return(inode); +} + +/**********************************************************************//** Gets the page number from the nth fragment page slot. @return page number, FIL_NULL if not in use */ UNIV_INLINE @@ -2073,6 +2122,7 @@ fseg_get_nth_frag_page_no( ut_ad(inode && mtr); ut_ad(n < FSEG_FRAG_ARR_N_SLOTS); ut_ad(mtr_memo_contains_page(mtr, inode, MTR_MEMO_PAGE_X_FIX)); + ut_ad(mach_read_from_4(inode + FSEG_MAGIC_N) == FSEG_MAGIC_N_VALUE); return(mach_read_from_4(inode + FSEG_FRAG_ARR + n * FSEG_FRAG_SLOT_SIZE)); } @@ -2091,6 +2141,7 @@ fseg_set_nth_frag_page_no( ut_ad(inode && mtr); ut_ad(n < FSEG_FRAG_ARR_N_SLOTS); ut_ad(mtr_memo_contains_page(mtr, inode, MTR_MEMO_PAGE_X_FIX)); + ut_ad(mach_read_from_4(inode + FSEG_MAGIC_N) == FSEG_MAGIC_N_VALUE); mlog_write_ulint(inode + FSEG_FRAG_ARR + n * FSEG_FRAG_SLOT_SIZE, page_no, MLOG_4BYTES, mtr); @@ -2451,6 +2502,8 @@ fseg_fill_free_list( xdes_set_state(descr, XDES_FSEG, mtr); seg_id = mtr_read_dulint(inode + FSEG_ID, mtr); + ut_ad(mach_read_from_4(inode + FSEG_MAGIC_N) + == FSEG_MAGIC_N_VALUE); mlog_write_dulint(descr + XDES_ID, seg_id, mtr); flst_add_last(inode + FSEG_FREE, descr + XDES_FLST_NODE, mtr); @@ -2479,6 +2532,7 @@ fseg_alloc_free_extent( fil_addr_t first; ut_ad(!((page_offset(inode) - FSEG_ARR_OFFSET) % FSEG_INODE_SIZE)); + ut_ad(mach_read_from_4(inode + FSEG_MAGIC_N) == FSEG_MAGIC_N_VALUE); if (flst_get_len(inode + FSEG_FREE, mtr) > 0) { /* Segment free list is not empty, allocate from it */ @@ -3136,6 +3190,8 @@ fseg_mark_page_used( ut_ad(seg_inode && mtr); ut_ad(!((page_offset(seg_inode) - FSEG_ARR_OFFSET) % FSEG_INODE_SIZE)); + ut_ad(mach_read_from_4(seg_inode + FSEG_MAGIC_N) + == FSEG_MAGIC_N_VALUE); descr = xdes_get_descriptor(space, zip_size, page, mtr); @@ -3373,6 +3429,8 @@ fseg_free_extent( ut_a(xdes_get_state(descr, mtr) == XDES_FSEG); ut_a(0 == ut_dulint_cmp(mtr_read_dulint(descr + XDES_ID, mtr), mtr_read_dulint(seg_inode + FSEG_ID, mtr))); + ut_ad(mach_read_from_4(seg_inode + FSEG_MAGIC_N) + == FSEG_MAGIC_N_VALUE); first_page_in_extent = page - (page % FSP_EXTENT_SIZE); @@ -3463,7 +3521,13 @@ fseg_free_step( ut_a(descr); ut_a(xdes_get_bit(descr, XDES_FREE_BIT, header_page % FSP_EXTENT_SIZE, mtr) == FALSE); - inode = fseg_inode_get(header, space, zip_size, mtr); + inode = fseg_inode_try_get(header, space, zip_size, mtr); + + if (UNIV_UNLIKELY(inode == NULL)) { + fprintf(stderr, "double free of inode from %u:%u\n", + (unsigned) space, (unsigned) header_page); + return(TRUE); + } descr = fseg_get_first_extent(inode, space, zip_size, mtr); @@ -3587,6 +3651,7 @@ fseg_get_first_extent( ut_ad(inode && mtr); ut_ad(space == page_get_space_id(page_align(inode))); + ut_ad(mach_read_from_4(inode + FSEG_MAGIC_N) == FSEG_MAGIC_N_VALUE); first = fil_addr_null; @@ -3801,6 +3866,7 @@ fseg_print_low( (ulong) reserved, (ulong) used, (ulong) n_full, (ulong) n_frag, (ulong) n_free, (ulong) n_not_full, (ulong) n_used); + ut_ad(mach_read_from_4(inode + FSEG_MAGIC_N) == FSEG_MAGIC_N_VALUE); } #ifdef UNIV_BTR_PRINT === modified file 'storage/xtradb/handler/ha_innodb.cc' --- a/storage/xtradb/handler/ha_innodb.cc 2009-12-03 11:34:11 +0000 +++ b/storage/xtradb/handler/ha_innodb.cc 2010-01-15 21:12:30 +0000 @@ -61,7 +61,9 @@ with this program; if not, write to the #pragma implementation // gcc: Class implementation #endif +#ifndef MYSQL_SERVER #define MYSQL_SERVER +#endif #include <mysql_priv.h> #ifdef MYSQL_SERVER @@ -77,6 +79,7 @@ with this program; if not, write to the /* Include necessary InnoDB headers */ extern "C" { #include "univ.i" +#include "buf0lru.h" #include "btr0sea.h" #include "os0file.h" #include "os0thread.h" @@ -109,7 +112,6 @@ extern "C" { #include "ha_innodb.h" #include "i_s.h" -#include "handler0vars.h" #ifdef MYSQL_SERVER // Defined in trx0sys.c @@ -120,9 +122,12 @@ extern ib_int64_t trx_sys_mysql_relay_lo #endif /* MYSQL_SERVER */ #ifndef MYSQL_SERVER -/* This is needed because of Bug #3596. Let us hope that pthread_mutex_t +# ifndef MYSQL_PLUGIN_IMPORT +# define MYSQL_PLUGIN_IMPORT /* nothing */ +# endif /* MYSQL_PLUGIN_IMPORT */ +/* This is needed because of Bug #3596. Let us hope that pthread_mutex_t is defined the same in both builds: the MySQL server and the InnoDB plugin. */ -extern pthread_mutex_t LOCK_thread_count; +extern MYSQL_PLUGIN_IMPORT pthread_mutex_t LOCK_thread_count; #if MYSQL_VERSION_ID < 50124 /* this is defined in mysql_priv.h inside #ifdef MYSQL_SERVER @@ -146,13 +151,9 @@ static bool innodb_inited = 0; /* In the Windows plugin, the return value of current_thd is undefined. Map it to NULL. */ -#if defined MYSQL_DYNAMIC_PLUGIN && defined __WIN__ -# undef current_thd -# define current_thd NULL -# define EQ_CURRENT_THD(thd) TRUE -#else /* MYSQL_DYNAMIC_PLUGIN && __WIN__ */ -# define EQ_CURRENT_THD(thd) ((thd) == current_thd) -#endif /* MYSQL_DYNAMIC_PLUGIN && __WIN__ */ + +#define EQ_CURRENT_THD(thd) ((thd) == current_thd) + static struct handlerton* innodb_hton_ptr; @@ -172,6 +173,10 @@ static ulong innobase_write_io_threads; static my_bool innobase_thread_concurrency_timer_based; static long long innobase_buffer_pool_size, innobase_log_file_size; +/** Percentage of the buffer pool to reserve for 'old' blocks. +Connected to buf_LRU_old_ratio. */ +static uint innobase_old_blocks_pct; + /* The default values for the following char* start-up parameters are determined in innobase_init below: */ @@ -186,9 +191,7 @@ file formats in the configuration file, of the supported file formats during runtime. */ static char* innobase_file_format_check = NULL; -/* The following has a misleading name: starting from 4.0.5, this also -affects Windows: */ -static char* innobase_unix_file_flush_method = NULL; +static char* innobase_file_flush_method = NULL; /* Below we have boolean-valued start-up parameters, and their default values */ @@ -202,7 +205,7 @@ static my_bool innobase_use_doublewrite static my_bool innobase_use_checksums = TRUE; static my_bool innobase_extra_undoslots = FALSE; static my_bool innobase_fast_recovery = FALSE; -static my_bool innobase_use_purge_thread = FALSE; +static my_bool innobase_recovery_stats = TRUE; static my_bool innobase_locks_unsafe_for_binlog = FALSE; static my_bool innobase_overwrite_relay_log_info = FALSE; static my_bool innobase_rollback_on_timeout = FALSE; @@ -238,10 +241,10 @@ static void free_share(INNOBASE_SHARE *s static int innobase_close_connection(handlerton *hton, THD* thd); static int innobase_commit(handlerton *hton, THD* thd, bool all); static int innobase_rollback(handlerton *hton, THD* thd, bool all); -static int innobase_rollback_to_savepoint(handlerton *hton, THD* thd, +static int innobase_rollback_to_savepoint(handlerton *hton, THD* thd, void *savepoint); static int innobase_savepoint(handlerton *hton, THD* thd, void *savepoint); -static int innobase_release_savepoint(handlerton *hton, THD* thd, +static int innobase_release_savepoint(handlerton *hton, THD* thd, void *savepoint); static handler *innobase_create_handler(handlerton *hton, TABLE_SHARE *table, @@ -285,10 +288,10 @@ innobase_file_format_check_on_off( /************************************************************//** Validate the file format check config parameters, as a side effect it sets the srv_check_file_format_at_startup variable. -@return true if valid config value */ +@return the format_id if valid config value, otherwise, return -1 */ static -bool -innobase_file_format_check_validate( +int +innobase_file_format_validate_and_set( /*================================*/ const char* format_check); /*!< in: parameter value */ /****************************************************************//** @@ -519,10 +522,10 @@ static SHOW_VAR innodb_status_variables[ (char*) &export_vars.innodb_buffer_pool_pages_misc, SHOW_LONG}, {"buffer_pool_pages_total", (char*) &export_vars.innodb_buffer_pool_pages_total, SHOW_LONG}, - {"buffer_pool_read_ahead_rnd", - (char*) &export_vars.innodb_buffer_pool_read_ahead_rnd, SHOW_LONG}, - {"buffer_pool_read_ahead_seq", - (char*) &export_vars.innodb_buffer_pool_read_ahead_seq, SHOW_LONG}, + {"buffer_pool_read_ahead", + (char*) &export_vars.innodb_buffer_pool_read_ahead, SHOW_LONG}, + {"buffer_pool_read_ahead_evicted", + (char*) &export_vars.innodb_buffer_pool_read_ahead_evicted, SHOW_LONG}, {"buffer_pool_read_requests", (char*) &export_vars.innodb_buffer_pool_read_requests, SHOW_LONG}, {"buffer_pool_reads", @@ -803,11 +806,20 @@ convert_error_code_to_mysql( case DB_SUCCESS: return(0); + case DB_INTERRUPTED: + my_error(ER_QUERY_INTERRUPTED, MYF(0)); + /* fall through */ case DB_ERROR: default: return(-1); /* unspecified error */ case DB_DUPLICATE_KEY: + /* Be cautious with returning this error, since + mysql could re-enter the storage layer to get + duplicated key info, the operation requires a + valid table handle and/or transaction information, + which might not always be available in the error + handling stage. */ return(HA_ERR_FOUND_DUPP_KEY); case DB_FOREIGN_DUPLICATE_KEY: @@ -894,17 +906,14 @@ convert_error_code_to_mysql( return(ER_PRIMARY_CANT_HAVE_NULL); case DB_TOO_MANY_CONCURRENT_TRXS: - /* Once MySQL add the appropriate code to errmsg.txt then - we can get rid of this #ifdef. NOTE: The code checked by - the #ifdef is the suggested name for the error condition - and the actual error code name could very well be different. - This will require some monitoring, ie. the status - of this request on our part.*/ -#ifdef ER_TOO_MANY_CONCURRENT_TRXS - return(ER_TOO_MANY_CONCURRENT_TRXS); -#else + /* New error code HA_ERR_TOO_MANY_CONCURRENT_TRXS is only + available in 5.1.38 and later, but the plugin should still + work with previous versions of MySQL. */ +#ifdef HA_ERR_TOO_MANY_CONCURRENT_TRXS + return(HA_ERR_TOO_MANY_CONCURRENT_TRXS); +#else /* HA_ERR_TOO_MANY_CONCURRENT_TRXS */ return(HA_ERR_RECORD_FILE_FULL); -#endif +#endif /* HA_ERR_TOO_MANY_CONCURRENT_TRXS */ case DB_UNSUPPORTED: return(HA_ERR_UNSUPPORTED); } @@ -978,7 +987,23 @@ innobase_get_cset_width( *mbminlen = cs->mbminlen; *mbmaxlen = cs->mbmaxlen; } else { - ut_a(cset == 0); + THD* thd = current_thd; + + if (thd && thd_sql_command(thd) == SQLCOM_DROP_TABLE) { + + /* Fix bug#46256: allow tables to be dropped if the + collation is not found, but issue a warning. */ + if ((global_system_variables.log_warnings) + && (cset != 0)){ + + sql_print_warning( + "Unknown collation #%lu.", cset); + } + } else { + + ut_a(cset == 0); + } + *mbminlen = *mbmaxlen = 0; } } @@ -1052,6 +1077,7 @@ innobase_get_charset( } #if defined (__WIN__) && defined (MYSQL_DYNAMIC_PLUGIN) +extern MYSQL_PLUGIN_IMPORT MY_TMPDIR mysql_tmpdir_list; /*******************************************************************//** Map an OS error to an errno value. The OS error number is stored in _doserrno and the mapped value is stored in errno) */ @@ -1339,6 +1365,16 @@ innobase_trx_init( trx->check_unique_secondary = !thd_test_options( thd, OPTION_RELAXED_UNIQUE_CHECKS); +#ifdef EXTENDED_SLOWLOG + if (thd_log_slow_verbosity(thd) & SLOG_V_INNODB) { + trx->take_stats = TRUE; + } else { + trx->take_stats = FALSE; + } +#else + trx->take_stats = FALSE; +#endif + DBUG_VOID_RETURN; } @@ -1395,6 +1431,32 @@ check_trx_exists( } +/************************************************************************* +Gets current trx. */ +extern "C" +trx_t* +innobase_get_trx() +{ + THD *thd=current_thd; + if (likely(thd != 0)) { + trx_t*& trx = thd_to_trx(thd); + return(trx); + } else { + return(NULL); + } +} + +extern "C" +ibool +innobase_get_slow_log() +{ +#ifdef EXTENDED_SLOWLOG + return((ibool) thd_opt_slow_log()); +#else + return(FALSE); +#endif +} + /*********************************************************************//** Construct ha_innobase handler. */ UNIV_INTERN @@ -1711,15 +1773,19 @@ innobase_convert_identifier( FALSE=id is an UTF-8 string */ { char nz[NAME_LEN + 1]; +#if MYSQL_VERSION_ID >= 50141 + char nz2[NAME_LEN + 1 + EXPLAIN_FILENAME_MAX_EXTRA_LENGTH]; +#else /* MYSQL_VERSION_ID >= 50141 */ char nz2[NAME_LEN + 1 + sizeof srv_mysql50_table_name_prefix]; +#endif /* MYSQL_VERSION_ID >= 50141 */ const char* s = id; int q; if (file_id) { - /* Decode the table name. The filename_to_tablename() - function expects a NUL-terminated string. The input and - output strings buffers must not be shared. */ + /* Decode the table name. The MySQL function expects + a NUL-terminated string. The input and output strings + buffers must not be shared. */ if (UNIV_UNLIKELY(idlen > (sizeof nz) - 1)) { idlen = (sizeof nz) - 1; @@ -1729,7 +1795,13 @@ innobase_convert_identifier( nz[idlen] = 0; s = nz2; +#if MYSQL_VERSION_ID >= 50141 + idlen = explain_filename((THD*) thd, nz, nz2, sizeof nz2, + EXPLAIN_PARTITIONS_AS_COMMENT); + goto no_quote; +#else /* MYSQL_VERSION_ID >= 50141 */ idlen = filename_to_tablename(nz, nz2, sizeof nz2); +#endif /* MYSQL_VERSION_ID >= 50141 */ } /* See if the identifier needs to be quoted. */ @@ -1740,6 +1812,9 @@ innobase_convert_identifier( } if (q == EOF) { +#if MYSQL_VERSION_ID >= 50141 +no_quote: +#endif /* MYSQL_VERSION_ID >= 50141 */ if (UNIV_UNLIKELY(idlen > buflen)) { idlen = buflen; } @@ -2222,8 +2297,8 @@ mem_free_and_error: /* Did the user specify a format name that we support ? As a side effect it will update the variable srv_check_file_format_at_startup */ - if (!innobase_file_format_check_validate( - innobase_file_format_check)) { + if (innobase_file_format_validate_and_set( + innobase_file_format_check) < 0) { sql_print_error("InnoDB: invalid " "innodb_file_format_check value: " @@ -2264,7 +2339,7 @@ innobase_change_buffering_inited_ok: /* --------------------------------------------------*/ - srv_file_flush_method_str = innobase_unix_file_flush_method; + srv_file_flush_method_str = innobase_file_flush_method; srv_n_log_groups = (ulint) innobase_mirrored_log_groups; srv_n_log_files = (ulint) innobase_log_files_in_group; @@ -2292,8 +2367,7 @@ innobase_change_buffering_inited_ok: srv_force_recovery = (ulint) innobase_force_recovery; srv_fast_recovery = (ibool) innobase_fast_recovery; - - srv_use_purge_thread = (ibool) innobase_use_purge_thread; + srv_recovery_stats = (ibool) innobase_recovery_stats; srv_use_doublewrite_buf = (ibool) innobase_use_doublewrite; srv_use_checksums = (ibool) innobase_use_checksums; @@ -2329,6 +2403,9 @@ innobase_change_buffering_inited_ok: ut_a(0 == strcmp(my_charset_latin1.name, "latin1_swedish_ci")); srv_latin1_ordering = my_charset_latin1.sort_order; + innobase_old_blocks_pct = buf_LRU_old_ratio_update( + innobase_old_blocks_pct, FALSE); + innobase_commit_concurrency_init_default(); /* Since we in this module access directly the fields of a trx @@ -2510,7 +2587,10 @@ innobase_alter_table_flags( { return(HA_ONLINE_ADD_INDEX_NO_WRITES | HA_ONLINE_DROP_INDEX_NO_WRITES - | HA_ONLINE_ADD_UNIQUE_INDEX_NO_WRITES + /* Current InnoDB doesn't sort unique indexes along mysqld's order + It is dangerous to use index. So it is disabled until + the bug http://bugs.mysql.com/47622 */ + /* | HA_ONLINE_ADD_UNIQUE_INDEX_NO_WRITES */ | HA_ONLINE_DROP_UNIQUE_INDEX_NO_WRITES | HA_ONLINE_ADD_PK_INDEX_NO_WRITES); } @@ -2676,6 +2756,19 @@ retry: } } + /* The following calls to read the MySQL binary log + file name and the position return consistent results: + 1) Other InnoDB transactions cannot intervene between + these calls as we are holding prepare_commit_mutex. + 2) Binary logging of other engines is not relevant + to InnoDB as all InnoDB requires is that committing + InnoDB transactions appear in the same order in the + MySQL binary log as they appear in InnoDB logs. + 3) A MySQL log file rotation cannot happen because + MySQL protects against this by having a counter of + transactions in prepared state and it only allows + a rotation when the counter drops to zero. See + LOCK_prep_xids and COND_prep_xids in log.cc. */ trx->mysql_log_file_name = mysql_bin_log_file_name(); trx->mysql_log_offset = (ib_int64_t) mysql_bin_log_file_pos(); @@ -2761,6 +2854,8 @@ innobase_rollback( innobase_release_stat_resources(trx); + trx->n_autoinc_rows = 0; /* Reset the number AUTO-INC rows required */ + /* If we had reserved the auto-inc lock for some table (if we come here to roll back the latest SQL statement) we release it now before a possibly lengthy rollback */ @@ -3322,7 +3417,7 @@ retry: if (is_part) { sql_print_error("Failed to open table %s after " - "%lu attemtps.\n", norm_name, + "%lu attempts.\n", norm_name, retries); } @@ -3926,7 +4021,6 @@ ha_innobase::store_key_val_for_row( as BLOB data in innodb. */ || mysql_type == MYSQL_TYPE_GEOMETRY) { - CHARSET_INFO* cs; ulint key_len; ulint true_len; @@ -4648,24 +4742,29 @@ no_commit: update the table upper limit. Note: last_value will be 0 if get_auto_increment() was not called.*/ - if (auto_inc <= col_max_value - && auto_inc >= prebuilt->autoinc_last_value) { + if (auto_inc >= prebuilt->autoinc_last_value) { set_max_autoinc: - ut_a(prebuilt->autoinc_increment > 0); - - ulonglong need; - ulonglong offset; - - offset = prebuilt->autoinc_offset; - need = prebuilt->autoinc_increment; - - auto_inc = innobase_next_autoinc( - auto_inc, need, offset, col_max_value); - - err = innobase_set_max_autoinc(auto_inc); - - if (err != DB_SUCCESS) { - error = err; + /* This should filter out the negative + values set explicitly by the user. */ + if (auto_inc <= col_max_value) { + ut_a(prebuilt->autoinc_increment > 0); + + ulonglong need; + ulonglong offset; + + offset = prebuilt->autoinc_offset; + need = prebuilt->autoinc_increment; + + auto_inc = innobase_next_autoinc( + auto_inc, + need, offset, col_max_value); + + err = innobase_set_max_autoinc( + auto_inc); + + if (err != DB_SUCCESS) { + error = err; + } } } break; @@ -5227,6 +5326,11 @@ ha_innobase::index_read( index = prebuilt->index; + if (UNIV_UNLIKELY(index == NULL)) { + prebuilt->index_usable = FALSE; + DBUG_RETURN(HA_ERR_CRASHED); + } + /* Note that if the index for which the search template is built is not necessarily prebuilt->index, but can also be the clustered index */ @@ -5386,6 +5490,7 @@ ha_innobase::change_active_index( if (UNIV_UNLIKELY(!prebuilt->index)) { sql_print_warning("InnoDB: change_active_index(%u) failed", keynr); + prebuilt->index_usable = FALSE; DBUG_RETURN(1); } @@ -5393,8 +5498,10 @@ ha_innobase::change_active_index( prebuilt->index); if (UNIV_UNLIKELY(!prebuilt->index_usable)) { - sql_print_warning("InnoDB: insufficient history for index %u", - keynr); + push_warning_printf(user_thd, MYSQL_ERROR::WARN_LEVEL_WARN, + HA_ERR_TABLE_DEF_CHANGED, + "InnoDB: insufficient history for index %u", + keynr); /* The caller seems to ignore this. Thus, we must check this again in row_search_for_mysql(). */ DBUG_RETURN(convert_error_code_to_mysql(DB_MISSING_HISTORY, @@ -5850,7 +5957,7 @@ create_table_def( number fits in one byte in prtype */ push_warning_printf( (THD*) trx->mysql_thd, - MYSQL_ERROR::WARN_LEVEL_ERROR, + MYSQL_ERROR::WARN_LEVEL_WARN, ER_CANT_CREATE_TABLE, "In InnoDB, charset-collation codes" " must be below 256." @@ -5882,17 +5989,8 @@ create_table_def( /* First check whether the column to be added has a system reserved name. */ if (dict_col_name_is_reserved(field->field_name)){ - push_warning_printf( - (THD*) trx->mysql_thd, - MYSQL_ERROR::WARN_LEVEL_WARN, - ER_CANT_CREATE_TABLE, - "Error creating table '%s' with " - "column name '%s'. '%s' is a " - "reserved name. Please try to " - "re-create the table with a " - "different column name.", - table->name, (char*) field->field_name, - (char*) field->field_name); + my_error(ER_WRONG_COLUMN_NAME, MYF(0), + field->field_name); dict_mem_table_free(table); trx_commit_for_mysql(trx); @@ -5914,6 +6012,14 @@ create_table_def( error = row_create_table_for_mysql(table, trx); + if (error == DB_DUPLICATE_KEY) { + char buf[100]; + innobase_convert_identifier(buf, sizeof buf, + table_name, strlen(table_name), + trx->mysql_thd, TRUE); + my_error(ER_TABLE_EXISTS_ERROR, MYF(0), buf); + } + error_ret: error = convert_error_code_to_mysql(error, flags, NULL); @@ -6064,7 +6170,6 @@ create_clustered_index_when_no_primary( /* We pass 0 as the space id, and determine at a lower level the space id where to store the table */ - index = dict_mem_index_create(table_name, innobase_index_reserve_name, 0, DICT_CLUSTERED, 0); @@ -6119,7 +6224,7 @@ create_options_are_valid( /* Valid value. */ break; default: - push_warning_printf(thd, MYSQL_ERROR::WARN_LEVEL_ERROR, + push_warning_printf(thd, MYSQL_ERROR::WARN_LEVEL_WARN, ER_ILLEGAL_HA_CREATE_OPTION, "InnoDB: invalid" " KEY_BLOCK_SIZE = %lu." @@ -6133,7 +6238,7 @@ create_options_are_valid( /* If KEY_BLOCK_SIZE was specified, check for its dependencies. */ if (kbs_specified && !srv_file_per_table) { - push_warning(thd, MYSQL_ERROR::WARN_LEVEL_ERROR, + push_warning(thd, MYSQL_ERROR::WARN_LEVEL_WARN, ER_ILLEGAL_HA_CREATE_OPTION, "InnoDB: KEY_BLOCK_SIZE" " requires innodb_file_per_table."); @@ -6141,7 +6246,7 @@ create_options_are_valid( } if (kbs_specified && srv_file_format < DICT_TF_FORMAT_ZIP) { - push_warning(thd, MYSQL_ERROR::WARN_LEVEL_ERROR, + push_warning(thd, MYSQL_ERROR::WARN_LEVEL_WARN, ER_ILLEGAL_HA_CREATE_OPTION, "InnoDB: KEY_BLOCK_SIZE" " requires innodb_file_format >" @@ -6165,7 +6270,7 @@ create_options_are_valid( if (!srv_file_per_table) { push_warning_printf( thd, - MYSQL_ERROR::WARN_LEVEL_ERROR, + MYSQL_ERROR::WARN_LEVEL_WARN, ER_ILLEGAL_HA_CREATE_OPTION, "InnoDB: ROW_FORMAT=%s" " requires innodb_file_per_table.", @@ -6177,7 +6282,7 @@ create_options_are_valid( if (srv_file_format < DICT_TF_FORMAT_ZIP) { push_warning_printf( thd, - MYSQL_ERROR::WARN_LEVEL_ERROR, + MYSQL_ERROR::WARN_LEVEL_WARN, ER_ILLEGAL_HA_CREATE_OPTION, "InnoDB: ROW_FORMAT=%s" " requires innodb_file_format >" @@ -6194,7 +6299,7 @@ create_options_are_valid( && form->s->row_type == ROW_TYPE_DYNAMIC) { push_warning_printf( thd, - MYSQL_ERROR::WARN_LEVEL_ERROR, + MYSQL_ERROR::WARN_LEVEL_WARN, ER_ILLEGAL_HA_CREATE_OPTION, "InnoDB: cannot specify" " ROW_FORMAT = DYNAMIC with" @@ -6218,7 +6323,7 @@ create_options_are_valid( if (kbs_specified) { push_warning_printf( thd, - MYSQL_ERROR::WARN_LEVEL_ERROR, + MYSQL_ERROR::WARN_LEVEL_WARN, ER_ILLEGAL_HA_CREATE_OPTION, "InnoDB: cannot specify" " ROW_FORMAT = %s with" @@ -6231,7 +6336,7 @@ create_options_are_valid( default: push_warning(thd, - MYSQL_ERROR::WARN_LEVEL_ERROR, + MYSQL_ERROR::WARN_LEVEL_WARN, ER_ILLEGAL_HA_CREATE_OPTION, "InnoDB: invalid ROW_FORMAT specifier."); ret = FALSE; @@ -6295,13 +6400,15 @@ ha_innobase::create( 1. <database_name>/<table_name>: for normal table creation 2. full path: for temp table creation, or sym link - When srv_file_per_table is on, check for full path pattern, i.e. + When srv_file_per_table is on and mysqld_embedded is off, + check for full path pattern, i.e. X:\dir\..., X is a driver letter, or \\dir1\dir2\..., UNC path returns error if it is in full path format, but not creating a temp. table. Currently InnoDB does not support symbolic link on Windows. */ if (srv_file_per_table + && !mysqld_embedded && (!create_info->options & HA_LEX_CREATE_TMP_TABLE)) { if ((name[1] == ':') @@ -6519,6 +6626,7 @@ ha_innobase::create( goto cleanup; } + /* Create the keys */ if (form->s->keys == 0 || primary_key_no == -1) { @@ -6967,6 +7075,24 @@ ha_innobase::rename_table( innobase_commit_low(trx); trx_free_for_mysql(trx); + /* Add a special case to handle the Duplicated Key error + and return DB_ERROR instead. + This is to avoid a possible SIGSEGV error from mysql error + handling code. Currently, mysql handles the Duplicated Key + error by re-entering the storage layer and getting dup key + info by calling get_dup_key(). This operation requires a valid + table handle ('row_prebuilt_t' structure) which could no + longer be available in the error handling stage. The suggested + solution is to report a 'table exists' error message (since + the dup key error here is due to an existing table whose name + is the one we are trying to rename to) and return the generic + error code. */ + if (error == (int) DB_DUPLICATE_KEY) { + my_error(ER_TABLE_EXISTS_ERROR, MYF(0), to); + + error = DB_ERROR; + } + error = convert_error_code_to_mysql(error, 0, NULL); DBUG_RETURN(error); @@ -7519,11 +7645,15 @@ ha_innobase::check( ret = row_check_table_for_mysql(prebuilt); - if (ret == DB_SUCCESS) { + switch (ret) { + case DB_SUCCESS: return(HA_ADMIN_OK); + case DB_INTERRUPTED: + my_error(ER_QUERY_INTERRUPTED, MYF(0)); + return(-1); + default: + return(HA_ADMIN_CORRUPT); } - - return(HA_ADMIN_CORRUPT); } /*************************************************************//** @@ -8069,8 +8199,11 @@ ha_innobase::external_lock( ulong const binlog_format= thd_binlog_format(thd); ulong const tx_isolation = thd_tx_isolation(ha_thd()); if (tx_isolation <= ISO_READ_COMMITTED - && binlog_format == BINLOG_FORMAT_STMT - && thd_binlog_filter_ok(thd)) + && binlog_format == BINLOG_FORMAT_STMT +#if MYSQL_VERSION_ID > 50140 + && thd_binlog_filter_ok(thd) +#endif /* MYSQL_VERSION_ID > 50140 */ + ) { char buf[256]; my_snprintf(buf, sizeof(buf), @@ -8183,6 +8316,23 @@ ha_innobase::external_lock( statement has ended */ if (trx->n_mysql_tables_in_use == 0) { +#ifdef EXTENDED_SLOWLOG + increment_thd_innodb_stats(thd, trx->io_reads, + trx->io_read, + trx->io_reads_wait_timer, + trx->lock_que_wait_timer, + trx->innodb_que_wait_timer, + trx->distinct_page_access); + + trx->io_reads = 0; + trx->io_read = 0; + trx->io_reads_wait_timer = 0; + trx->lock_que_wait_timer = 0; + trx->innodb_que_wait_timer = 0; + trx->distinct_page_access = 0; + if (trx->distinct_page_access_hash) + memset(trx->distinct_page_access_hash, 0, DPAH_SIZE); +#endif trx->mysql_n_tables_locked = 0; prebuilt->used_in_HANDLER = FALSE; @@ -8470,8 +8620,8 @@ innodb_mutex_show_status( rw_lock_wait_time += mutex->lspent_time; } #else /* UNIV_DEBUG */ - buf1len= (uint) my_snprintf(buf1, sizeof(buf1), "%s:%lu", - mutex->cfile_name, (ulong) mutex->cline); + buf1len= (uint) my_snprintf(buf1, sizeof(buf1), "%s", + mutex->cmutex_name); buf2len= (uint) my_snprintf(buf2, sizeof(buf2), "os_waits=%lu", mutex->count_os_wait); @@ -8496,8 +8646,8 @@ next_mutex: while (lock != NULL) { if (lock->count_os_wait && !buf_pool_is_block_lock(lock)) { - buf1len= my_snprintf(buf1, sizeof(buf1), "%s:%lu", - lock->cfile_name, (ulong) lock->cline); + buf1len= my_snprintf(buf1, sizeof(buf1), "%s", + lock->lock_name); buf2len= my_snprintf(buf2, sizeof(buf2), "os_waits=%lu", lock->count_os_wait); @@ -8718,6 +8868,7 @@ ha_innobase::store_lock( && isolation_level != TRX_ISO_SERIALIZABLE && (lock_type == TL_READ || lock_type == TL_READ_NO_INSERT) && (sql_command == SQLCOM_INSERT_SELECT + || sql_command == SQLCOM_REPLACE_SELECT || sql_command == SQLCOM_UPDATE || sql_command == SQLCOM_CREATE_TABLE)) { @@ -8725,10 +8876,11 @@ ha_innobase::store_lock( option set or this session is using READ COMMITTED isolation level and isolation level of the transaction is not set to serializable and MySQL is doing - INSERT INTO...SELECT or UPDATE ... = (SELECT ...) or - CREATE ... SELECT... without FOR UPDATE or - IN SHARE MODE in select, then we use consistent - read for select. */ + INSERT INTO...SELECT or REPLACE INTO...SELECT + or UPDATE ... = (SELECT ...) or CREATE ... + SELECT... without FOR UPDATE or IN SHARE + MODE in select, then we use consistent read + for select. */ prebuilt->select_lock_type = LOCK_NONE; prebuilt->stored_select_lock_type = LOCK_NONE; @@ -8957,8 +9109,7 @@ ha_innobase::get_auto_increment( col_max_value = innobase_get_int_col_max_value( table->next_number_field); - current = *first_value > col_max_value ? autoinc : *first_value; - + current = *first_value > col_max_value ? autoinc : *first_value; need = *nb_reserved_values * increment; /* Compute the last value in the interval */ @@ -9322,8 +9473,7 @@ innobase_xa_prepare( executing XA PREPARE and XA COMMIT commands. In this case we cannot know how many minutes or hours will be between XA PREPARE and XA COMMIT, and we don't want - to block for undefined period of time. - */ + to block for undefined period of time. */ pthread_mutex_lock(&prepare_commit_mutex); trx->active_trans = 2; } @@ -9575,6 +9725,20 @@ ha_innobase::check_if_incompatible_data( DBUG_RETURN(COMPATIBLE_DATA_NO); } + /* Renaming column asynchronizes dictionary between mysqld and InnoDB... + If not synchronized, treat as COMPATIBLE_DATA_NO + until the bug http://bugs.mysql.com/47621 is fixed officialily */ + { + uint i; + for (i = 0; i < table->s->fields; i++) { + if (table->field[i]->flags & FIELD_IN_ADD_INDEX + && innobase_strcasecmp(table->field[i]->field_name, + dict_table_get_col_name(prebuilt->table, i))) { + DBUG_RETURN(COMPATIBLE_DATA_NO); + } + } + } + /* Check if a column participating in a foreign key is being renamed. There is no mechanism for updating InnoDB foreign key definitions. */ if (foreign_key_column_is_being_renamed(prebuilt, table)) { @@ -9683,25 +9847,24 @@ innobase_file_format_check_on_off( /************************************************************//** Validate the file format check config parameters, as a side effect it sets the srv_check_file_format_at_startup variable. -@return true if valid config value */ +@return the format_id if valid config value, otherwise, return -1 */ static -bool -innobase_file_format_check_validate( +int +innobase_file_format_validate_and_set( /*================================*/ const char* format_check) /*!< in: parameter value */ { uint format_id; - bool ret = true; format_id = innobase_file_format_name_lookup(format_check); if (format_id < DICT_TF_FORMAT_MAX + 1) { srv_check_file_format_at_startup = format_id; + + return((int) format_id); } else { - ret = false; + return(-1); } - - return(ret); } /*************************************************************//** @@ -9720,7 +9883,6 @@ innodb_file_format_name_validate( struct st_mysql_value* value) /*!< in: incoming string */ { const char* file_format_input; - char* file_format_input_strdup; char buff[STRING_BUFFER_USUAL_SIZE]; int len = sizeof(buff); @@ -9737,18 +9899,12 @@ innodb_file_format_name_validate( if (format_id <= DICT_TF_FORMAT_MAX) { - /* Copy out from stack-allocated memory (which will not - survive return from this function). The memory will be - freed in innodb_file_format_check_update(). */ - file_format_input_strdup = thd_strmake(thd, file_format_input, len); + /* Save a pointer to the name in the + 'file_format_name_map' constant array. */ + *static_cast<const char**>(save) = + trx_sys_file_format_id_to_name(format_id); - *static_cast<char**>(save) = file_format_input_strdup; - - if (file_format_input_strdup == NULL) { - return(1); - } else { - return(0); - } + return(0); } } @@ -9808,9 +9964,9 @@ innodb_file_format_check_validate( struct st_mysql_value* value) /*!< in: incoming string */ { const char* file_format_input; - char* file_format_input_strdup; char buff[STRING_BUFFER_USUAL_SIZE]; int len = sizeof(buff); + int format_id; ut_a(save != NULL); ut_a(value != NULL); @@ -9823,33 +9979,35 @@ innodb_file_format_check_validate( message if they did so. */ if (innobase_file_format_check_on_off(file_format_input)) { - sql_print_warning( + push_warning_printf(thd, + MYSQL_ERROR::WARN_LEVEL_WARN, + ER_WRONG_ARGUMENTS, "InnoDB: invalid innodb_file_format_check " "value; on/off can only be set at startup or " "in the configuration file"); - } else if (innobase_file_format_check_validate( - file_format_input)) { + } else { + format_id = innobase_file_format_validate_and_set( + file_format_input); - /* Copy out from stack-allocated memory (which will not - survive return from this function). The memory will be - freed in innodb_file_format_check_update(). */ - file_format_input_strdup = thd_strmake(thd, file_format_input, len); + if (format_id >= 0) { + /* Save a pointer to the name in the + 'file_format_name_map' constant array. */ + *static_cast<const char**>(save) = + trx_sys_file_format_id_to_name( + (uint)format_id); - *static_cast<char**>(save) = file_format_input_strdup; + return(0); - if (file_format_input_strdup == NULL) { - return(1); } else { - return(0); + push_warning_printf(thd, + MYSQL_ERROR::WARN_LEVEL_WARN, + ER_WRONG_ARGUMENTS, + "InnoDB: invalid innodb_file_format_check " + "value; can be any format up to %s " + "or its equivalent numeric id", + trx_sys_file_format_id_to_name( + DICT_TF_FORMAT_MAX)); } - - } else { - sql_print_warning( - "InnoDB: invalid innodb_file_format_check " - "value; can be any format up to %s " - "or its equivalent numeric id", - trx_sys_file_format_id_to_name( - DICT_TF_FORMAT_MAX)); } } @@ -9880,6 +10038,7 @@ innodb_file_format_check_update( ut_a(var_ptr != NULL); format_name_in = *static_cast<const char*const*>(save); + if (!format_name_in) { return; @@ -9929,6 +10088,25 @@ innodb_adaptive_hash_index_update( } } +/****************************************************************//** +Update the system variable innodb_old_blocks_pct using the "saved" +value. This function is registered as a callback with MySQL. */ +static +void +innodb_old_blocks_pct_update( +/*=========================*/ + THD* thd, /*!< in: thread handle */ + struct st_mysql_sys_var* var, /*!< in: pointer to + system variable */ + void* var_ptr,/*!< out: where the + formal string goes */ + const void* save) /*!< in: immediate result + from check function */ +{ + innobase_old_blocks_pct = buf_LRU_old_ratio_update( + *static_cast<const uint*>(save), TRUE); +} + /*************************************************************//** Check if it is a valid value of innodb_change_buffering. This function is registered as a callback with MySQL. @@ -10075,13 +10253,18 @@ static MYSQL_SYSVAR_BOOL(extra_undoslots static MYSQL_SYSVAR_BOOL(fast_recovery, innobase_fast_recovery, PLUGIN_VAR_NOCMDARG | PLUGIN_VAR_READONLY, "Enable to use speed hack of recovery avoiding flush list sorting.", - NULL, NULL, FALSE); + NULL, NULL, TRUE); -static MYSQL_SYSVAR_BOOL(use_purge_thread, innobase_use_purge_thread, +static MYSQL_SYSVAR_BOOL(recovery_stats, innobase_recovery_stats, PLUGIN_VAR_NOCMDARG | PLUGIN_VAR_READONLY, - "Enable to use purge devoted thread.", + "Output statistics of recovery process after it.", NULL, NULL, FALSE); +static MYSQL_SYSVAR_ULONG(use_purge_thread, srv_use_purge_thread, + PLUGIN_VAR_RQCMDARG | PLUGIN_VAR_READONLY, + "Number of purge devoted threads. #### over 1 is EXPERIMENTAL ####", + NULL, NULL, 1, 0, 64, 0); + static MYSQL_SYSVAR_BOOL(overwrite_relay_log_info, innobase_overwrite_relay_log_info, PLUGIN_VAR_NOCMDARG | PLUGIN_VAR_READONLY, "During InnoDB crash recovery on slave overwrite relay-log.info " @@ -10122,12 +10305,15 @@ static MYSQL_SYSVAR_STR(file_format, inn innodb_file_format_name_validate, innodb_file_format_name_update, "Antelope"); +/* If a new file format is introduced, the file format +name needs to be updated accordingly. Please refer to +file_format_name_map[] defined in trx0sys.c for the next +file format name. */ static MYSQL_SYSVAR_STR(file_format_check, innobase_file_format_check, PLUGIN_VAR_OPCMDARG, "The highest file format in the tablespace.", innodb_file_format_check_validate, - innodb_file_format_check_update, - "on"); + innodb_file_format_check_update, "Barracuda"); static MYSQL_SYSVAR_ULONG(flush_log_at_trx_commit, srv_flush_log_at_trx_commit, PLUGIN_VAR_OPCMDARG, @@ -10136,7 +10322,7 @@ static MYSQL_SYSVAR_ULONG(flush_log_at_t " or 2 (write at commit, flush once per second).", NULL, NULL, 1, 0, 2, 0); -static MYSQL_SYSVAR_STR(flush_method, innobase_unix_file_flush_method, +static MYSQL_SYSVAR_STR(flush_method, innobase_file_flush_method, PLUGIN_VAR_RQCMDARG | PLUGIN_VAR_READONLY, "With which method to flush data.", NULL, NULL, NULL); @@ -10177,7 +10363,7 @@ static MYSQL_SYSVAR_ULONG(max_dirty_page static MYSQL_SYSVAR_BOOL(adaptive_flushing, srv_adaptive_flushing, PLUGIN_VAR_NOCMDARG, "Attempt flushing dirty pages to avoid IO bursts at checkpoints.", - NULL, NULL, TRUE); + NULL, NULL, FALSE); static MYSQL_SYSVAR_ULONG(max_purge_lag, srv_max_purge_lag, PLUGIN_VAR_RQCMDARG, @@ -10273,7 +10459,7 @@ static MYSQL_SYSVAR_ULONG(concurrency_ti NULL, NULL, 500L, 1L, ~0L, 0); static MYSQL_SYSVAR_LONG(file_io_threads, innobase_file_io_threads, - PLUGIN_VAR_RQCMDARG | PLUGIN_VAR_READONLY, + PLUGIN_VAR_RQCMDARG | PLUGIN_VAR_READONLY | PLUGIN_VAR_NOSYSVAR, "Number of file I/O threads in InnoDB.", NULL, NULL, 4, 4, 64, 0); @@ -10312,6 +10498,18 @@ static MYSQL_SYSVAR_LONG(mirrored_log_gr "Number of identical copies of log groups we keep for the database. Currently this should be set to 1.", NULL, NULL, 1, 1, 10, 0); +static MYSQL_SYSVAR_UINT(old_blocks_pct, innobase_old_blocks_pct, + PLUGIN_VAR_RQCMDARG, + "Percentage of the buffer pool to reserve for 'old' blocks.", + NULL, innodb_old_blocks_pct_update, 100 * 3 / 8, 5, 95, 0); + +static MYSQL_SYSVAR_UINT(old_blocks_time, buf_LRU_old_threshold_ms, + PLUGIN_VAR_RQCMDARG, + "Move blocks to the 'new' end of the buffer pool if the first access" + " was at least this many milliseconds ago." + " The timeout is disabled if 0 (the default).", + NULL, NULL, 0, 0, UINT_MAX32, 0); + static MYSQL_SYSVAR_LONG(open_files, innobase_open_files, PLUGIN_VAR_RQCMDARG | PLUGIN_VAR_READONLY, "How many files at the maximum InnoDB keeps open at the same time.", @@ -10390,13 +10588,18 @@ static MYSQL_SYSVAR_LONGLONG(ibuf_max_si static MYSQL_SYSVAR_ULONG(ibuf_active_contract, srv_ibuf_active_contract, PLUGIN_VAR_RQCMDARG, "Enable/Disable active_contract of insert buffer. 0:disable 1:enable", - NULL, NULL, 0, 0, 1, 0); + NULL, NULL, 1, 0, 1, 0); static MYSQL_SYSVAR_ULONG(ibuf_accel_rate, srv_ibuf_accel_rate, PLUGIN_VAR_RQCMDARG, "Tunes amount of insert buffer processing of background, in addition to innodb_io_capacity. (in percentage)", NULL, NULL, 100, 100, 999999999, 0); +static MYSQL_SYSVAR_ULONG(checkpoint_age_target, srv_checkpoint_age_target, + PLUGIN_VAR_RQCMDARG, + "Control soft limit of checkpoint age. (0 : not control)", + NULL, NULL, 0, 0, ~0UL, 0); + static MYSQL_SYSVAR_ULONG(flush_neighbor_pages, srv_flush_neighbor_pages, PLUGIN_VAR_RQCMDARG, "Enable/Disable flushing also neighbor pages. 0:disable 1:enable", @@ -10432,7 +10635,7 @@ TYPELIB read_ahead_typelib= }; static MYSQL_SYSVAR_ENUM(read_ahead, srv_read_ahead, PLUGIN_VAR_RQCMDARG, - "Control read ahead activity. (none, random, [linear], both)", + "Control read ahead activity (none, random, [linear], both). [from 1.0.5: random read ahead is ignored]", NULL, innodb_read_ahead_update, 2, &read_ahead_typelib); static @@ -10463,8 +10666,8 @@ TYPELIB adaptive_checkpoint_typelib= }; static MYSQL_SYSVAR_ENUM(adaptive_checkpoint, srv_adaptive_checkpoint, PLUGIN_VAR_RQCMDARG, - "Enable/Disable flushing along modified age. ([none], reflex, estimate)", - NULL, innodb_adaptive_checkpoint_update, 0, &adaptive_checkpoint_typelib); + "Enable/Disable flushing along modified age. (none, reflex, [estimate])", + NULL, innodb_adaptive_checkpoint_update, 2, &adaptive_checkpoint_typelib); static MYSQL_SYSVAR_ULONG(enable_unsafe_group_commit, srv_enable_unsafe_group_commit, PLUGIN_VAR_RQCMDARG, @@ -10486,6 +10689,11 @@ static MYSQL_SYSVAR_ULONG(dict_size_limi "Limit the allocated memory for dictionary cache. (0: unlimited)", NULL, NULL, 0, 0, LONG_MAX, 0); +static MYSQL_SYSVAR_ULONG(relax_table_creation, srv_relax_table_creation, + PLUGIN_VAR_RQCMDARG, + "Relax limitation of column size at table creation as builtin InnoDB.", + NULL, NULL, 0, 0, 1, 0); + static struct st_mysql_sys_var* innobase_system_variables[]= { MYSQL_SYSVAR(additional_mem_pool_size), MYSQL_SYSVAR(autoextend_increment), @@ -10498,6 +10706,7 @@ static struct st_mysql_sys_var* innobase MYSQL_SYSVAR(doublewrite), MYSQL_SYSVAR(extra_undoslots), MYSQL_SYSVAR(fast_recovery), + MYSQL_SYSVAR(recovery_stats), MYSQL_SYSVAR(fast_shutdown), MYSQL_SYSVAR(file_io_threads), MYSQL_SYSVAR(read_io_threads), @@ -10522,6 +10731,8 @@ static struct st_mysql_sys_var* innobase MYSQL_SYSVAR(adaptive_flushing), MYSQL_SYSVAR(max_purge_lag), MYSQL_SYSVAR(mirrored_log_groups), + MYSQL_SYSVAR(old_blocks_pct), + MYSQL_SYSVAR(old_blocks_time), MYSQL_SYSVAR(open_files), MYSQL_SYSVAR(overwrite_relay_log_info), MYSQL_SYSVAR(rollback_on_timeout), @@ -10548,6 +10759,7 @@ static struct st_mysql_sys_var* innobase MYSQL_SYSVAR(ibuf_max_size), MYSQL_SYSVAR(ibuf_active_contract), MYSQL_SYSVAR(ibuf_accel_rate), + MYSQL_SYSVAR(checkpoint_age_target), MYSQL_SYSVAR(flush_neighbor_pages), MYSQL_SYSVAR(read_ahead), MYSQL_SYSVAR(adaptive_checkpoint), @@ -10560,6 +10772,7 @@ static struct st_mysql_sys_var* innobase MYSQL_SYSVAR(read_ahead_threshold), MYSQL_SYSVAR(io_capacity), MYSQL_SYSVAR(use_purge_thread), + MYSQL_SYSVAR(relax_table_creation), NULL }; @@ -10591,6 +10804,7 @@ i_s_innodb_cmpmem, i_s_innodb_cmpmem_reset, i_s_innodb_table_stats, i_s_innodb_index_stats, +i_s_innodb_admin_command, i_s_innodb_patches mysql_declare_plugin_end; === modified file 'storage/xtradb/handler/ha_innodb.h' --- a/storage/xtradb/handler/ha_innodb.h 2009-12-03 11:34:11 +0000 +++ b/storage/xtradb/handler/ha_innodb.h 2010-01-15 15:58:25 +0000 @@ -258,12 +258,14 @@ int thd_binlog_format(const MYSQL_THD th */ void thd_mark_transaction_to_rollback(MYSQL_THD thd, bool all); +#if MYSQL_VERSION_ID > 50140 /** Check if binary logging is filtered for thread's current db. @param thd Thread handle @retval 1 the query is not filtered, 0 otherwise. */ bool thd_binlog_filter_ok(const MYSQL_THD thd); +#endif /* MYSQL_VERSION_ID > 50140 */ } typedef struct trx_struct trx_t; @@ -289,6 +291,8 @@ trx_t* innobase_trx_allocate( /*==================*/ MYSQL_THD thd); /*!< in: user thread handle */ + + /*********************************************************************//** This function checks each index name for a table against reserved system default primary index name 'GEN_CLUST_INDEX'. If a name === modified file 'storage/xtradb/handler/handler0alter.cc' --- a/storage/xtradb/handler/handler0alter.cc 2009-12-03 11:34:11 +0000 +++ b/storage/xtradb/handler/handler0alter.cc 2010-01-15 15:58:25 +0000 @@ -35,7 +35,6 @@ extern "C" { } #include "ha_innodb.h" -#include "handler0vars.h" /*************************************************************//** Copies an InnoDB column to a MySQL field. This function is @@ -629,7 +628,7 @@ ha_innobase::add_index( ulint num_created = 0; ibool dict_locked = FALSE; ulint new_primary; - ulint error; + int error; DBUG_ENTER("ha_innobase::add_index"); ut_a(table); @@ -668,7 +667,7 @@ ha_innobase::add_index( if (UNIV_UNLIKELY(error)) { err_exit: mem_heap_free(heap); - trx_general_rollback_for_mysql(trx, FALSE, NULL); + trx_general_rollback_for_mysql(trx, NULL); trx_free_for_mysql(trx); trx_commit_for_mysql(prebuilt->trx); DBUG_RETURN(error); @@ -766,10 +765,11 @@ err_exit: ut_ad(error == DB_SUCCESS); /* Commit the data dictionary transaction in order to release - the table locks on the system tables. Unfortunately, this - means that if MySQL crashes while creating a new primary key - inside row_merge_build_indexes(), indexed_table will not be - dropped on crash recovery. Thus, it will become orphaned. */ + the table locks on the system tables. This means that if + MySQL crashes while creating a new primary key inside + row_merge_build_indexes(), indexed_table will not be dropped + by trx_rollback_active(). It will have to be recovered or + dropped by the database administrator. */ trx_commit_for_mysql(trx); row_mysql_unlock_data_dictionary(trx); @@ -806,7 +806,7 @@ error_handling: alter table t drop index b, add index (b); The fix will have to parse the SQL and note that the index - being added has the same name as the the one being dropped and + being added has the same name as the one being dropped and ignore that in the dup index check.*/ //dict_table_check_for_dup_indexes(prebuilt->table); #endif @@ -868,6 +868,7 @@ error_handling: indexed_table->n_mysql_handles_opened++; error = row_merge_drop_table(trx, innodb_table); + innodb_table = indexed_table; goto convert_error; case DB_TOO_BIG_RECORD: @@ -882,7 +883,9 @@ error: /* fall through */ default: if (new_primary) { - row_merge_drop_table(trx, indexed_table); + if (indexed_table != innodb_table) { + row_merge_drop_table(trx, indexed_table); + } } else { if (!dict_locked) { row_mysql_lock_data_dictionary(trx); === removed file 'storage/xtradb/handler/handler0vars.h' --- a/storage/xtradb/handler/handler0vars.h 2009-11-13 21:26:08 +0000 +++ b/storage/xtradb/handler/handler0vars.h 1970-01-01 00:00:00 +0000 @@ -1,73 +0,0 @@ -/***************************************************************************** - -Copyright (c) 2008, 2009, Innobase Oy. All Rights Reserved. - -This program is free software; you can redistribute it and/or modify it under -the terms of the GNU General Public License as published by the Free Software -Foundation; version 2 of the License. - -This program is distributed in the hope that it will be useful, but WITHOUT -ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS -FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. - -You should have received a copy of the GNU General Public License along with -this program; if not, write to the Free Software Foundation, Inc., 59 Temple -Place, Suite 330, Boston, MA 02111-1307 USA - -*****************************************************************************/ - -/*******************************************************************//** -@file handler/handler0vars.h -This file contains accessor functions for dynamic plugin on Windows. -***********************************************************************/ - -#if defined __WIN__ && defined MYSQL_DYNAMIC_PLUGIN -/*******************************************************************//** -This is a list of externals that can not be resolved by delay loading. -They have to be resolved indirectly via their addresses in the .map file. -All of them are external variables. */ -extern MYSQL_PLUGIN_IMPORT CHARSET_INFO my_charset_bin; -extern MYSQL_PLUGIN_IMPORT CHARSET_INFO my_charset_latin1; -extern MYSQL_PLUGIN_IMPORT CHARSET_INFO my_charset_filename; -extern MYSQL_PLUGIN_IMPORT CHARSET_INFO* system_charset_info; -extern MYSQL_PLUGIN_IMPORT CHARSET_INFO* default_charset_info; -//extern MYSQL_PLUGIN_IMPORT CHARSET_INFO** all_charsets; -extern MYSQL_PLUGIN_IMPORT system_variables global_system_variables; -//extern MYSQL_PLUGIN_IMPORT char* mysql_real_data_home; -extern MYSQL_PLUGIN_IMPORT char* mysql_data_home; -//extern MYSQL_PLUGIN_IMPORT char** tx_isolation_names; -//extern MYSQL_PLUGIN_IMPORT char** binlog_format_names; -//extern MYSQL_PLUGIN_IMPORT char reg_ext; -extern MYSQL_PLUGIN_IMPORT pthread_mutex_t LOCK_thread_count; -extern MYSQL_PLUGIN_IMPORT key_map key_map_full; -extern MYSQL_PLUGIN_IMPORT MY_TMPDIR mysql_tmpdir_list; -extern MYSQL_PLUGIN_IMPORT bool mysqld_embedded; -extern MYSQL_PLUGIN_IMPORT uint lower_case_table_names; -extern MYSQL_PLUGIN_IMPORT ulong specialflag; -extern MYSQL_PLUGIN_IMPORT int my_umask; - -extern MYSQL_PLUGIN_IMPORT char *relay_log_info_file; - -/* -#define my_charset_bin (*wdl_my_charset_bin) -#define my_charset_latin1 (*wdl_my_charset_latin1) -#define my_charset_filename (*wdl_my_charset_filename) -#define system_charset_info (*wdl_system_charset_info) -#define default_charset_info (*wdl_default_charset_info) -#define all_charsets (wdl_all_charsets) -#define global_system_variables (*wdl_global_system_variables) -#define mysql_real_data_home (wdl_mysql_real_data_home) -#define mysql_data_home (*wdl_mysql_data_home) -#define tx_isolation_names (wdl_tx_isolation_names) -#define binlog_format_names (wdl_binlog_format_names) -#define reg_ext (wdl_reg_ext) -#define LOCK_thread_count (*wdl_LOCK_thread_count) -#define key_map_full (*wdl_key_map_full) -#define mysql_tmpdir_list (*wdl_mysql_tmpdir_list) -#define mysqld_embedded (*wdl_mysqld_embedded) -*/ -//#define lower_case_table_names (*wdl_lower_case_table_names) -//#define specialflag (*wdl_specialflag) -//#define my_umask (*wdl_my_umask) - -#endif === modified file 'storage/xtradb/handler/i_s.cc' --- a/storage/xtradb/handler/i_s.cc 2009-11-13 21:26:08 +0000 +++ b/storage/xtradb/handler/i_s.cc 2010-01-15 15:58:25 +0000 @@ -47,6 +47,7 @@ extern "C" { #include "trx0rseg.h" /* for trx_rseg_struct */ #include "trx0sys.h" /* for trx_sys */ #include "dict0dict.h" /* for dict_sys */ +#include "buf0lru.h" /* for XTRA_LRU_[DUMP/RESTORE] */ /* from buf0buf.c */ struct buf_chunk_struct{ ulint mem_size; /* allocated size of the chunk */ @@ -56,7 +57,6 @@ struct buf_chunk_struct{ buf_block_t* blocks; /* array of buffer control blocks */ }; } -#include "handler0vars.h" static const char plugin_author[] = "Innobase Oy"; @@ -84,14 +84,16 @@ do { \ #define STRUCT_FLD(name, value) value #endif -static const ST_FIELD_INFO END_OF_ST_FIELD_INFO = - {STRUCT_FLD(field_name, NULL), - STRUCT_FLD(field_length, 0), - STRUCT_FLD(field_type, MYSQL_TYPE_NULL), - STRUCT_FLD(value, 0), - STRUCT_FLD(field_flags, 0), - STRUCT_FLD(old_name, ""), - STRUCT_FLD(open_method, SKIP_OPEN_TABLE)}; +/* Don't use a static const variable here, as some C++ compilers (notably +HPUX aCC: HP ANSI C++ B3910B A.03.65) can't handle it. */ +#define END_OF_ST_FIELD_INFO \ + {STRUCT_FLD(field_name, NULL), \ + STRUCT_FLD(field_length, 0), \ + STRUCT_FLD(field_type, MYSQL_TYPE_NULL), \ + STRUCT_FLD(value, 0), \ + STRUCT_FLD(field_flags, 0), \ + STRUCT_FLD(old_name, ""), \ + STRUCT_FLD(open_method, SKIP_OPEN_TABLE)} /* Use the following types mapping: @@ -511,7 +513,7 @@ static ST_FIELD_INFO i_s_innodb_buffer_p STRUCT_FLD(old_name, ""), STRUCT_FLD(open_method, SKIP_OPEN_TABLE)}, - {STRUCT_FLD(field_name, "accessed"), + {STRUCT_FLD(field_name, "access_time"), STRUCT_FLD(field_length, MY_INT64_NUM_DECIMAL_DIGITS), STRUCT_FLD(field_type, MYSQL_TYPE_LONGLONG), STRUCT_FLD(value, 0), @@ -728,7 +730,7 @@ i_s_innodb_buffer_pool_pages_fill( field_store_string(table->field[0], page_type); table->field[1]->store(block->page.space); table->field[2]->store(block->page.offset); - table->field[3]->store(block->page.LRU_position); + table->field[3]->store(0); table->field[4]->store(block->page.buf_fix_count); table->field[5]->store(block->page.flush_type); @@ -817,11 +819,11 @@ i_s_innodb_buffer_pool_pages_index_fill( table->field[5]->store(page_get_n_recs(frame)); table->field[6]->store(page_get_data_size(frame)); table->field[7]->store(block->is_hashed); - table->field[8]->store(block->page.accessed); + table->field[8]->store(block->page.access_time); table->field[9]->store(block->page.newest_modification != 0); table->field[10]->store(block->page.oldest_modification != 0); table->field[11]->store(block->page.old); - table->field[12]->store(block->page.LRU_position); + table->field[12]->store(0); table->field[13]->store(block->page.buf_fix_count); table->field[14]->store(block->page.flush_type); @@ -915,7 +917,7 @@ i_s_innodb_buffer_pool_pages_blob_fill( table->field[4]->store(block->page.offset); } - table->field[5]->store(block->page.LRU_position); + table->field[5]->store(0); table->field[6]->store(block->page.buf_fix_count); table->field[7]->store(block->page.flush_type); @@ -2953,3 +2955,170 @@ UNIV_INTERN struct st_mysql_plugin i_s_i STRUCT_FLD(system_vars, NULL), STRUCT_FLD(__reserved1, NULL) }; + +/*********************************************************************** +*/ +static ST_FIELD_INFO i_s_innodb_admin_command_info[] = +{ + {STRUCT_FLD(field_name, "result_message"), + STRUCT_FLD(field_length, 1024), + STRUCT_FLD(field_type, MYSQL_TYPE_STRING), + STRUCT_FLD(value, 0), + STRUCT_FLD(field_flags, 0), + STRUCT_FLD(old_name, ""), + STRUCT_FLD(open_method, SKIP_OPEN_TABLE)}, + + END_OF_ST_FIELD_INFO +}; + +#ifndef INNODB_COMPATIBILITY_HOOKS +#error InnoDB needs MySQL to be built with #define INNODB_COMPATIBILITY_HOOKS +#endif + +extern "C" { +char **thd_query(MYSQL_THD thd); +} + +static +int +i_s_innodb_admin_command_fill( +/*==========================*/ + THD* thd, + TABLE_LIST* tables, + COND* cond) +{ + TABLE* i_s_table = (TABLE *) tables->table; + CHARSET_INFO *cs= system_charset_info; + char** query_str; + char* ptr; + char quote = '\0'; + char* command_head = "XTRA_"; + + DBUG_ENTER("i_s_innodb_admin_command_fill"); + + /* deny access to non-superusers */ + if (check_global_access(thd, PROCESS_ACL)) { + DBUG_RETURN(0); + } + + if(thd_sql_command(thd) != SQLCOM_SELECT) { + field_store_string(i_s_table->field[0], + "SELECT command is only accepted."); + goto end_func; + } + + query_str = thd_query(thd); + ptr = *query_str; + + for (; *ptr; ptr++) { + if (*ptr == quote) { + quote = '\0'; + } else if (quote) { + } else if (*ptr == '`' || *ptr == '"') { + quote = *ptr; + } else { + long i; + for (i = 0; command_head[i]; i++) { + if (toupper((int)(unsigned char)(ptr[i])) + != toupper((int)(unsigned char) + (command_head[i]))) { + goto nomatch; + } + } + break; +nomatch: + ; + } + } + + if (!*ptr) { + field_store_string(i_s_table->field[0], + "No XTRA_* command in the SQL statement." + " Please add /*!XTRA_xxxx*/ to the SQL."); + goto end_func; + } + + if (!strncasecmp("XTRA_HELLO", ptr, 10)) { + /* This is example command XTRA_HELLO */ + + ut_print_timestamp(stderr); + fprintf(stderr, " InnoDB: administration command test for XtraDB" + " 'XTRA_HELLO' was detected.\n"); + + field_store_string(i_s_table->field[0], + "Hello!"); + goto end_func; + } + else if (!strncasecmp("XTRA_LRU_DUMP", ptr, 13)) { + ut_print_timestamp(stderr); + fprintf(stderr, " InnoDB: administration command 'XTRA_LRU_DUMP'" + " was detected.\n"); + + if (buf_LRU_file_dump()) { + field_store_string(i_s_table->field[0], + "XTRA_LRU_DUMP was succeeded."); + } else { + field_store_string(i_s_table->field[0], + "XTRA_LRU_DUMP was failed."); + } + + goto end_func; + } + else if (!strncasecmp("XTRA_LRU_RESTORE", ptr, 16)) { + ut_print_timestamp(stderr); + fprintf(stderr, " InnoDB: administration command 'XTRA_LRU_RESTORE'" + " was detected.\n"); + + if (buf_LRU_file_restore()) { + field_store_string(i_s_table->field[0], + "XTRA_LRU_RESTORE was succeeded."); + } else { + field_store_string(i_s_table->field[0], + "XTRA_LRU_RESTORE was failed."); + } + + goto end_func; + } + + field_store_string(i_s_table->field[0], + "Undefined XTRA_* command."); + goto end_func; + +end_func: + if (schema_table_store_record(thd, i_s_table)) { + DBUG_RETURN(1); + } else { + DBUG_RETURN(0); + } +} + +static +int +i_s_innodb_admin_command_init( +/*==========================*/ + void* p) +{ + DBUG_ENTER("i_s_innodb_admin_command_init"); + ST_SCHEMA_TABLE* schema = (ST_SCHEMA_TABLE*) p; + + schema->fields_info = i_s_innodb_admin_command_info; + schema->fill_table = i_s_innodb_admin_command_fill; + + DBUG_RETURN(0); +} + +UNIV_INTERN struct st_mysql_plugin i_s_innodb_admin_command = +{ + STRUCT_FLD(type, MYSQL_INFORMATION_SCHEMA_PLUGIN), + STRUCT_FLD(info, &i_s_info), + STRUCT_FLD(name, "XTRADB_ADMIN_COMMAND"), + STRUCT_FLD(author, plugin_author), + STRUCT_FLD(descr, "XtraDB specific command acceptor"), + STRUCT_FLD(license, PLUGIN_LICENSE_GPL), + STRUCT_FLD(init, i_s_innodb_admin_command_init), + STRUCT_FLD(deinit, i_s_common_deinit), + STRUCT_FLD(version, 0x0100 /* 1.0 */), + STRUCT_FLD(status_vars, NULL), + STRUCT_FLD(system_vars, NULL), + STRUCT_FLD(__reserved1, NULL) +}; === modified file 'storage/xtradb/handler/i_s.h' --- a/storage/xtradb/handler/i_s.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/handler/i_s.h 2009-11-04 20:11:12 +0000 @@ -40,5 +40,6 @@ extern struct st_mysql_plugin i_s_innodb extern struct st_mysql_plugin i_s_innodb_rseg; extern struct st_mysql_plugin i_s_innodb_table_stats; extern struct st_mysql_plugin i_s_innodb_index_stats; +extern struct st_mysql_plugin i_s_innodb_admin_command; #endif /* i_s_h */ === modified file 'storage/xtradb/handler/innodb_patch_info.h' --- a/storage/xtradb/handler/innodb_patch_info.h 2009-11-13 21:26:08 +0000 +++ b/storage/xtradb/handler/innodb_patch_info.h 2010-01-15 15:58:25 +0000 @@ -38,5 +38,10 @@ struct innodb_enhancement { {"innodb_stats","Additional features about InnoDB statistics/optimizer","","http://www.percona.com/docs/wiki/percona-xtradb"}, {"innodb_recovery_patches","Bugfixes and adjustments about recovery process","","http://www.percona.com/docs/wiki/percona-xtradb"}, {"innodb_purge_thread","Enable to use purge devoted thread","","http://www.percona.com/docs/wiki/percona-xtradb"}, +{"innodb_admin_command_base","XtraDB specific command interface through i_s","","http://www.percona.com/docs/wiki/percona-xtradb"}, +{"innodb_show_lock_name","Show mutex/lock name instead of crated file/line","","http://www.percona.com/docs/wiki/percona-xtradb"}, +{"innodb_extend_slow","Extended statistics in slow.log","It is InnoDB-part only. It needs to patch also to mysqld.","http://www.percona.com/docs/wiki/percona-xtradb"}, +{"innodb_relax_table_creation","Relax limitation of column size at table creation as builtin InnoDB.","","http://www.percona.com/docs/wiki/percona-xtradb"}, +{"innodb_lru_dump_restore","Dump and restore command for content of buffer pool","","http://www.percona.com/docs/wiki/percona-xtradb"}, {NULL, NULL, NULL, NULL} }; === removed file 'storage/xtradb/handler/win_delay_loader.cc' --- a/storage/xtradb/handler/win_delay_loader.cc 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/handler/win_delay_loader.cc 1970-01-01 00:00:00 +0000 @@ -1,1024 +0,0 @@ -/***************************************************************************** - -Copyright (c) 2008, 2009, Innobase Oy. All Rights Reserved. - -This program is free software; you can redistribute it and/or modify it under -the terms of the GNU General Public License as published by the Free Software -Foundation; version 2 of the License. - -This program is distributed in the hope that it will be useful, but WITHOUT -ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS -FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. - -You should have received a copy of the GNU General Public License along with -this program; if not, write to the Free Software Foundation, Inc., 59 Temple -Place, Suite 330, Boston, MA 02111-1307 USA - -*****************************************************************************/ - -/*******************************************************************//** -@file handler/win_delay_loader.cc -This file contains functions that implement the delay loader on Windows. - -This is a customized version of delay loader with limited functionalities. -It does not support: - -* (manual) unloading -* multiple delay loaded DLLs -* multiple loading of the same DLL - -This delay loader is used only by the InnoDB plugin. Other components (DLLs) -can still use the default delay loader, provided by MSVC. - -Several acronyms used by Microsoft: - * IAT: import address table - * INT: import name table - * RVA: Relative Virtual Address - -See http://msdn.microsoft.com/en-us/magazine/bb985992.aspx for details of -PE format. -***********************************************************************/ -#if defined (__WIN__) && defined (MYSQL_DYNAMIC_PLUGIN) -# define WIN32_LEAN_AND_MEAN -# include <windows.h> -# include <delayimp.h> -# include <mysql_priv.h> - -extern "C" { -# include "univ.i" -# include "hash0hash.h" -} - -/*******************************************************************//** -This following contains a list of externals that can not be resolved by -delay loading. They have to be resolved indirectly via their addresses -in the .map file. All of them are external variables. */ -CHARSET_INFO* wdl_my_charset_bin; -CHARSET_INFO* wdl_my_charset_latin1; -CHARSET_INFO* wdl_my_charset_filename; -CHARSET_INFO** wdl_system_charset_info; -CHARSET_INFO** wdl_default_charset_info; -CHARSET_INFO** wdl_all_charsets; -system_variables* wdl_global_system_variables; -char* wdl_mysql_real_data_home; -char** wdl_mysql_data_home; -char** wdl_tx_isolation_names; -char** wdl_binlog_format_names; -char* wdl_reg_ext; -pthread_mutex_t* wdl_LOCK_thread_count; -key_map* wdl_key_map_full; -MY_TMPDIR* wdl_mysql_tmpdir_list; -bool* wdl_mysqld_embedded; -uint* wdl_lower_case_table_names; -ulong* wdl_specialflag; -int* wdl_my_umask; - -/*******************************************************************//** -The preferred load-address defined in PE (portable executable format). */ -#if defined(_M_IA64) -#pragma section(".base", long, read) -extern "C" -__declspec(allocate(".base")) -const IMAGE_DOS_HEADER __ImageBase; -#else -extern "C" -const IMAGE_DOS_HEADER __ImageBase; -#endif - -/*******************************************************************//** -A template function for converting a relative address (RVA) to an -absolute address (VA). This is due to the pointers in the delay -descriptor (ImgDelayDescr in delayimp.h) have been changed from -VAs to RVAs to work on both 32- and 64-bit platforms. -@return absolute virtual address */ -template <class X> -X PFromRva( -/*=======*/ - RVA rva) /*!< in: relative virtual address */ -{ - return X(PBYTE(&__ImageBase) + rva); -} - -/*******************************************************************//** -Convert to the old format for convenience. The structure as well as its -element names follow the definition of ImgDelayDescr in delayimp.h. */ -struct InternalImgDelayDescr -{ - DWORD grAttrs; /*!< attributes */ - LPCSTR szName; /*!< pointer to dll name */ - HMODULE* phmod; /*!< address of module handle */ - PImgThunkData pIAT; /*!< address of the IAT */ - PCImgThunkData pINT; /*!< address of the INT */ - PCImgThunkData pBoundIAT; /*!< address of the optional bound IAT */ - PCImgThunkData pUnloadIAT; /*!< address of optional copy of - original IAT */ - DWORD dwTimeStamp; /*!< 0 if not bound, - otherwise date/time stamp of DLL - bound to (Old BIND) */ -}; - -typedef struct map_hash_chain_struct map_hash_chain_t; - -struct map_hash_chain_struct { - char* symbol; /*!< pointer to a symbol */ - ulint value; /*!< address of the symbol */ - map_hash_chain_t* next; /*!< pointer to the next cell - in the same folder. */ - map_hash_chain_t* chain; /*!< a linear chain used for - cleanup. */ -}; - -static HMODULE my_hmod = 0; -static struct hash_table_struct* m_htbl = NULL ; -static map_hash_chain_t* chain_header = NULL; -static ibool wdl_init = FALSE; -const ulint MAP_HASH_CELLS_NUM = 10000; - -#ifndef DBUG_OFF -/*******************************************************************//** -In the dynamic plugin, it is required to call the following dbug functions -in the server: - _db_pargs_ - _db_doprnt_ - _db_enter_ - _db_return_ - _db_dump_ - -The plugin will get those function pointers during the initialization. */ -typedef void (__cdecl* pfn_db_enter_)( - const char* _func_, - const char* _file_, - uint _line_, - const char** _sfunc_, - const char** _sfile_, - uint* _slevel_, - char***); - -typedef void (__cdecl* pfn_db_return_)( - uint _line_, - const char** _sfunc_, - const char** _sfile_, - uint* _slevel_); - -typedef void (__cdecl* pfn_db_pargs_)( - uint _line_, - const char* keyword); - -typedef void (__cdecl* pfn_db_doprnt_)( - const char* format, - ...); - -typedef void (__cdecl* pfn_db_dump_)( - uint _line_, - const char* keyword, - const unsigned char* memory, - size_t length); - -static pfn_db_enter_ wdl_db_enter_; -static pfn_db_return_ wdl_db_return_; -static pfn_db_pargs_ wdl_db_pargs_; -static pfn_db_doprnt_ wdl_db_doprnt_; -static pfn_db_dump_ wdl_db_dump_; -#endif /* !DBUG_OFF */ - -/*************************************************************//** -Creates a hash table with >= n array cells. The actual number of cells is -chosen to be a prime number slightly bigger than n. - -This is the same function as hash_create in hash0hash.c, except the -memory allocation. This function is invoked before the engine is -initialized, and buffer pools are not ready yet. -@return own: created hash table */ -static -hash_table_t* -wdl_hash_create( -/*============*/ - ulint n) /*!< in: number of array cells */ -{ - hash_cell_t* array; - ulint prime; - hash_table_t* table; - - prime = ut_find_prime(n); - - table = (hash_table_t*) malloc(sizeof(hash_table_t)); - if (table == NULL) { - return(NULL); - } - - array = (hash_cell_t*) malloc(sizeof(hash_cell_t) * prime); - if (array == NULL) { - free(table); - return(NULL); - } - - table->array = array; - table->n_cells = prime; - table->n_mutexes = 0; - table->mutexes = NULL; - table->heaps = NULL; - table->heap = NULL; - table->magic_n = HASH_TABLE_MAGIC_N; - - /* Initialize the cell array */ - hash_table_clear(table); - - return(table); -} - -/*************************************************************//** -Frees a hash table. */ -static -void -wdl_hash_table_free( -/*================*/ - hash_table_t* table) /*!< in, own: hash table */ -{ - ut_a(table != NULL); - ut_a(table->mutexes == NULL); - - free(table->array); - free(table); -} - -/*******************************************************************//** -Function for calculating the count of imports given the base of the IAT. -@return number of imports */ -static -ulint -wdl_import_count( -/*=============*/ - PCImgThunkData pitd_base) /*!< in: base of the IAT */ -{ - ulint ret = 0; - PCImgThunkData pitd = pitd_base; - - while (pitd->u1.Function) { - pitd++; - ret++; - } - - return(ret); -} - -/*******************************************************************//** -Read Mapfile to a hashtable for faster access -@return TRUE if the mapfile is loaded successfully. */ -static -ibool -wdl_load_mapfile( -/*=============*/ - const char* filename) /*!< in: name of the mapfile. */ -{ - FILE* fp; - const size_t nSize = 256; - char tmp_buf[nSize]; - char* func_name; - char* func_addr; - ulint load_addr = 0; - ibool valid_load_addr = FALSE; -#ifdef _WIN64 - const char* tmp_string = " Preferred load address is %16llx"; -#else - const char* tmp_string = " Preferred load address is %08x"; -#endif - - fp = fopen(filename, "r"); - if (fp == NULL) { - - return(FALSE); - } - - /* Check whether to create the hashtable */ - if (m_htbl == NULL) { - - m_htbl = wdl_hash_create(MAP_HASH_CELLS_NUM); - - if (m_htbl == NULL) { - - fclose(fp); - return(FALSE); - } - } - - /* Search start of symbol list and get the preferred load address */ - while (fgets(tmp_buf, sizeof(tmp_buf), fp)) { - - if (sscanf(tmp_buf, tmp_string, &load_addr) == 1) { - - valid_load_addr = TRUE; - } - - if (strstr(tmp_buf, "Rva+Base") != NULL) { - - break; - } - } - - if (valid_load_addr == FALSE) { - - /* No "Preferred load address", the map file is wrong. */ - fclose(fp); - return(FALSE); - } - - /* Read symbol list */ - while (fgets(tmp_buf, sizeof(tmp_buf), fp)) - { - map_hash_chain_t* map_cell; - ulint map_fold; - - if (*tmp_buf == 0) { - - continue; - } - - func_name = strtok(tmp_buf, " "); - func_name = strtok(NULL, " "); - func_addr = strtok(NULL, " "); - - if (func_name && func_addr) { - - ut_snprintf(tmp_buf, nSize, "0x%s", func_addr); - if (*func_name == '_') { - - func_name++; - } - - map_cell = (map_hash_chain_t*) - malloc(sizeof(map_hash_chain_t)); - if (map_cell == NULL) { - return(FALSE); - } - - /* Chain all cells together */ - map_cell->chain = chain_header; - chain_header = map_cell; - - map_cell->symbol = strdup(func_name); - map_cell->value = (ulint) _strtoui64(tmp_buf, NULL, 0) - - load_addr; - map_fold = ut_fold_string(map_cell->symbol); - - HASH_INSERT(map_hash_chain_t, - next, - m_htbl, - map_fold, - map_cell); - } - } - - fclose(fp); - - return(TRUE); -} - -/*************************************************************//** -Cleanup.during DLL unload */ -static -void -wdl_cleanup(void) -/*=============*/ -{ - while (chain_header != NULL) { - map_hash_chain_t* tmp; - - tmp = chain_header->chain; - free(chain_header->symbol); - free(chain_header); - chain_header = tmp; - } - - if (m_htbl != NULL) { - - wdl_hash_table_free(m_htbl); - } -} - -/*******************************************************************//** -Load the mapfile mysqld.map. -@return the module handle */ -static -HMODULE -wdl_get_mysqld_mapfile(void) -/*========================*/ -{ - char file_name[MAX_PATH]; - char* ext; - ulint err; - - if (my_hmod == 0) { - - size_t nSize = MAX_PATH - strlen(".map") -1; - - /* First find out the name of current executable */ - my_hmod = GetModuleHandle(NULL); - if (my_hmod == 0) { - - return(my_hmod); - } - - err = GetModuleFileName(my_hmod, file_name, nSize); - if (err == 0) { - - my_hmod = 0; - return(my_hmod); - } - - ext = strrchr(file_name, '.'); - if (ext != NULL) { - - *ext = 0; - strcat(file_name, ".map"); - - err = wdl_load_mapfile(file_name); - if (err == 0) { - - my_hmod = 0; - } - } else { - - my_hmod = 0; - } - } - - return(my_hmod); -} - -/*******************************************************************//** -Retrieves the address of an exported function. It follows the convention -of GetProcAddress(). -@return address of exported function. */ -static -FARPROC -wdl_get_procaddr_from_map( -/*======================*/ - HANDLE m_handle, /*!< in: module handle */ - const char* import_proc) /*!< in: procedure name */ -{ - map_hash_chain_t* hash_chain; - ulint map_fold; - - map_fold = ut_fold_string(import_proc); - HASH_SEARCH( - next, - m_htbl, - map_fold, - map_hash_chain_t*, - hash_chain, - , - (ut_strcmp(hash_chain->symbol, import_proc) == 0)); - - if (hash_chain == NULL) { - -#ifdef _WIN64 - /* On Win64, the leading '_' may not be taken out. In this - case, search again without the leading '_'. */ - if (*import_proc == '_') { - - import_proc++; - } - - map_fold = ut_fold_string(import_proc); - HASH_SEARCH( - next, - m_htbl, - map_fold, - map_hash_chain_t*, - hash_chain, - , - (ut_strcmp(hash_chain->symbol, import_proc) == 0)); - - if (hash_chain == NULL) { -#endif - if (wdl_init == TRUE) { - - sql_print_error( - "InnoDB: the procedure pointer of %s" - " is not found.", - import_proc); - } - - return(0); -#ifdef _WIN64 - } -#endif - } - - return((FARPROC) ((ulint) m_handle + hash_chain->value)); -} - -/*******************************************************************//** -Retrieves the address of an exported variable. -Note: It does not follow the Windows call convention FARPROC. -@return address of exported variable. */ -static -void* -wdl_get_varaddr_from_map( -/*=====================*/ - HANDLE m_handle, /*!< in: module handle */ - const char* import_variable) /*!< in: variable name */ -{ - map_hash_chain_t* hash_chain; - ulint map_fold; - - map_fold = ut_fold_string(import_variable); - HASH_SEARCH( - next, - m_htbl, - map_fold, - map_hash_chain_t*, - hash_chain, - , - (ut_strcmp(hash_chain->symbol, import_variable) == 0)); - - if (hash_chain == NULL) { - -#ifdef _WIN64 - /* On Win64, the leading '_' may not be taken out. In this - case, search again without the leading '_'. */ - if (*import_variable == '_') { - - import_variable++; - } - - map_fold = ut_fold_string(import_variable); - HASH_SEARCH( - next, - m_htbl, - map_fold, - map_hash_chain_t*, - hash_chain, - , - (ut_strcmp(hash_chain->symbol, import_variable) == 0)); - - if (hash_chain == NULL) { -#endif - if (wdl_init == TRUE) { - - sql_print_error( - "InnoDB: the variable address of %s" - " is not found.", - import_variable); - } - - return(0); -#ifdef _WIN64 - } -#endif - } - - return((void*) ((ulint) m_handle + hash_chain->value)); -} - -/*******************************************************************//** -Bind all unresolved external variables from the MySQL executable. -@return TRUE if successful */ -static -bool -wdl_get_external_variables(void) -/*============================*/ -{ - HMODULE hmod = wdl_get_mysqld_mapfile(); - - if (hmod == 0) { - - return(FALSE); - } - -#define GET_SYM(sym, var, type) \ - var = (type*) wdl_get_varaddr_from_map(hmod, sym); \ - if (var == NULL) return(FALSE) -#ifdef _WIN64 -#define GET_SYM2(sym1, sym2, var, type) \ - var = (type*) wdl_get_varaddr_from_map(hmod, sym1); \ - if (var == NULL) return(FALSE) -#else -#define GET_SYM2(sym1, sym2, var, type) \ - var = (type*) wdl_get_varaddr_from_map(hmod, sym2); \ - if (var == NULL) return(FALSE) -#endif // (_WIN64) -#define GET_C_SYM(sym, type) GET_SYM(#sym, wdl_##sym, type) -#define GET_PROC_ADDR(sym) \ - wdl##sym = (pfn##sym) wdl_get_procaddr_from_map(hmod, #sym) - - GET_C_SYM(my_charset_bin, CHARSET_INFO); - GET_C_SYM(my_charset_latin1, CHARSET_INFO); - GET_C_SYM(my_charset_filename, CHARSET_INFO); - GET_C_SYM(default_charset_info, CHARSET_INFO*); - GET_C_SYM(all_charsets, CHARSET_INFO*); - GET_C_SYM(my_umask, int); - - GET_SYM("?global_system_variables@@3Usystem_variables@@A", - wdl_global_system_variables, struct system_variables); - GET_SYM("?mysql_real_data_home@@3PADA", - wdl_mysql_real_data_home, char); - GET_SYM("?reg_ext@@3PADA", wdl_reg_ext, char); - GET_SYM("?LOCK_thread_count@@3U_RTL_CRITICAL_SECTION@@A", - wdl_LOCK_thread_count, pthread_mutex_t); - GET_SYM("?key_map_full@@3V?$Bitmap@$0EA@@@A", - wdl_key_map_full, key_map); - GET_SYM("?mysql_tmpdir_list@@3Ust_my_tmpdir@@A", - wdl_mysql_tmpdir_list, MY_TMPDIR); - GET_SYM("?mysqld_embedded@@3_NA", - wdl_mysqld_embedded, bool); - GET_SYM("?lower_case_table_names@@3IA", - wdl_lower_case_table_names, uint); - GET_SYM("?specialflag@@3KA", wdl_specialflag, ulong); - - GET_SYM2("?system_charset_info@@3PEAUcharset_info_st@@EA", - "?system_charset_info@@3PAUcharset_info_st@@A", - wdl_system_charset_info, CHARSET_INFO*); - GET_SYM2("?mysql_data_home@@3PEADEA", - "?mysql_data_home@@3PADA", - wdl_mysql_data_home, char*); - GET_SYM2("?tx_isolation_names@@3PAPEBDA", - "?tx_isolation_names@@3PAPBDA", - wdl_tx_isolation_names, char*); - GET_SYM2("?binlog_format_names@@3PAPEBDA", - "?binlog_format_names@@3PAPBDA", - wdl_binlog_format_names, char*); - -#ifndef DBUG_OFF - GET_PROC_ADDR(_db_enter_); - GET_PROC_ADDR(_db_return_); - GET_PROC_ADDR(_db_pargs_); - GET_PROC_ADDR(_db_doprnt_); - GET_PROC_ADDR(_db_dump_); - - /* If any of the dbug functions is not available, just make them - all invalid. This is the case when working with a non-debug - version of the server. */ - if (wdl_db_enter_ == NULL || wdl_db_return_ == NULL - || wdl_db_pargs_ == NULL || wdl_db_doprnt_ == NULL - || wdl_db_dump_ == NULL) { - - wdl_db_enter_ = NULL; - wdl_db_return_ = NULL; - wdl_db_pargs_ = NULL; - wdl_db_doprnt_ = NULL; - wdl_db_dump_ = NULL; - } -#endif /* !DBUG_OFF */ - - wdl_init = TRUE; - return(TRUE); - -#undef GET_SYM -#undef GET_SYM2 -#undef GET_C_SYM -#undef GET_PROC_ADDR -} - -/*******************************************************************//** -The DLL Delayed Loading Helper Function for resolving externals. - -The function may fail due to one of the three reasons: - -* Invalid parameter, which happens if the attributes in pidd aren't - specified correctly. -* Failed to load the map file mysqld.map. -* Failed to find an external name in the map file mysqld.map. - -Note: this function is called by run-time as well as __HrLoadAllImportsForDll. -So, it has to follow Windows call convention. -@return the address of the imported function */ -extern "C" -FARPROC WINAPI -__delayLoadHelper2( -/*===============*/ - PCImgDelayDescr pidd, /*!< in: a const pointer to a - ImgDelayDescr, see delayimp.h. */ - FARPROC* iat_entry) /*!< in/out: A pointer to the slot in - the delay load import address table - to be updated with the address of the - imported function. */ -{ - ulint iIAT, iINT; - HMODULE hmod; - PCImgThunkData pitd; - FARPROC fun = NULL; - - /* Set up data used for the hook procs */ - InternalImgDelayDescr idd = { - pidd->grAttrs, - PFromRva<LPCSTR>(pidd->rvaDLLName), - PFromRva<HMODULE*>(pidd->rvaHmod), - PFromRva<PImgThunkData>(pidd->rvaIAT), - PFromRva<PCImgThunkData>(pidd->rvaINT), - PFromRva<PCImgThunkData>(pidd->rvaBoundIAT), - PFromRva<PCImgThunkData>(pidd->rvaUnloadIAT), - pidd->dwTimeStamp - }; - - DelayLoadInfo dli = { - sizeof(DelayLoadInfo), - pidd, - iat_entry, - idd.szName, - {0}, - 0, - 0, - 0 - }; - - /* Check the Delay Load Attributes, log an error of invalid - parameter, which happens if the attributes in pidd are not - specified correctly. */ - if ((idd.grAttrs & dlattrRva) == 0) { - - sql_print_error("InnoDB: invalid parameter for delay loader."); - return(0); - } - - hmod = *idd.phmod; - - /* Calculate the index for the IAT entry in the import address table. - The INT entries are ordered the same as the IAT entries so the - calculation can be done on the IAT side. */ - iIAT = (PCImgThunkData) iat_entry - idd.pIAT; - iINT = iIAT; - - pitd = &(idd.pINT[iINT]); - - dli.dlp.fImportByName = !IMAGE_SNAP_BY_ORDINAL(pitd->u1.Ordinal); - - if (dli.dlp.fImportByName) { - - dli.dlp.szProcName = (LPCSTR) (PFromRva<PIMAGE_IMPORT_BY_NAME> - ((RVA) ((UINT_PTR) pitd->u1.AddressOfData))->Name); - } else { - - dli.dlp.dwOrdinal = (ulint) IMAGE_ORDINAL(pitd->u1.Ordinal); - } - - /* Now, load the mapfile, if it has not been done yet */ - if (hmod == 0) { - - hmod = wdl_get_mysqld_mapfile(); - } - - if (hmod == 0) { - /* LoadLibrary failed. */ - PDelayLoadInfo rgpdli[1] = {&dli}; - - dli.dwLastError = ::GetLastError(); - - sql_print_error( - "InnoDB: failed to load mysqld.map with error %d.", - dli.dwLastError); - - return(0); - } - - /* Store the library handle. */ - idd.phmod = &hmod; - - /* Go for the procedure now. */ - dli.hmodCur = hmod; - - if (pidd->rvaBoundIAT && pidd->dwTimeStamp) { - - /* Bound imports exist, check the timestamp from the target - image */ - PIMAGE_NT_HEADERS pinh; - - pinh = (PIMAGE_NT_HEADERS) ((byte*) hmod - + ((PIMAGE_DOS_HEADER) hmod)->e_lfanew); - - if (pinh->Signature == IMAGE_NT_SIGNATURE - && pinh->FileHeader.TimeDateStamp == idd.dwTimeStamp - && (DWORD) hmod == pinh->OptionalHeader.ImageBase) { - - /* We have a decent address in the bound IAT. */ - fun = (FARPROC) (UINT_PTR) - idd.pBoundIAT[iIAT].u1.Function; - - if (fun) { - - *iat_entry = fun; - return(fun); - } - } - } - - fun = wdl_get_procaddr_from_map(hmod, dli.dlp.szProcName); - - if (fun == 0) { - - return(0); - } - - *iat_entry = fun; - return(fun); -} - -/*******************************************************************//** -Unload a DLL that was delay loaded. This function is called by run-time. -@return TRUE is returned if the DLL is found and the IAT matches the -original one. */ -extern "C" -BOOL WINAPI -__FUnloadDelayLoadedDLL2( -/*=====================*/ - LPCSTR module_name) /*!< in: DLL name */ -{ - return(TRUE); -} - -/**************************************************************//** -Load all imports from a DLL that was specified with the /delayload linker -option. -Note: this function is called by run-time. So, it has to follow Windows call -convention. -@return S_OK if the DLL matches, otherwise ERROR_MOD_NOT_FOUND is returned. */ -extern "C" -HRESULT WINAPI -__HrLoadAllImportsForDll( -/*=====================*/ - LPCSTR module_name) /*!< in: DLL name */ -{ - PIMAGE_NT_HEADERS img; - PCImgDelayDescr pidd; - IMAGE_DATA_DIRECTORY* image_data; - LPCSTR current_module; - HRESULT ret = ERROR_MOD_NOT_FOUND; - HMODULE hmod = (HMODULE) &__ImageBase; - - img = (PIMAGE_NT_HEADERS) ((byte*) hmod - + ((PIMAGE_DOS_HEADER) hmod)->e_lfanew); - image_data = - &img->OptionalHeader.DataDirectory[IMAGE_DIRECTORY_ENTRY_DELAY_IMPORT]; - - /* Scan the delay load IAT/INT for the DLL */ - if (image_data->Size) { - - pidd = PFromRva<PCImgDelayDescr>(image_data->VirtualAddress); - - /* Check all of the listed DLLs we want to load. */ - while (pidd->rvaDLLName) { - - current_module = PFromRva<LPCSTR>(pidd->rvaDLLName); - - if (stricmp(module_name, current_module) == 0) { - - /* Found it, break out with pidd and - current_module set appropriately */ - break; - } - - /* To the next delay import descriptor */ - pidd++; - } - - if (pidd->rvaDLLName) { - - /* Found a matching DLL, now process it. */ - FARPROC* iat_entry; - size_t count; - - iat_entry = PFromRva<FARPROC*>(pidd->rvaIAT); - count = wdl_import_count((PCImgThunkData) iat_entry); - - /* now load all the imports from the DLL */ - while (count > 0) { - - /* No need to check the return value */ - __delayLoadHelper2(pidd, iat_entry); - iat_entry++; - count--; - } - - ret = S_OK; - } - } - - return ret; -} - -/**************************************************************//** -The main function of a DLL -@return TRUE if the call succeeds */ -BOOL -WINAPI -DllMain( -/*====*/ - HINSTANCE hinstDLL, /*!< in: handle to the DLL module */ - DWORD fdwReason, /*!< Reason code that indicates why the - DLL entry-point function is being - called.*/ - LPVOID lpvReserved) /*!< in: additional parameter based on - fdwReason */ -{ - BOOL success = TRUE; - - switch (fdwReason) { - - case DLL_PROCESS_ATTACH: - success = wdl_get_external_variables(); - break; - - case DLL_PROCESS_DETACH: - wdl_cleanup(); - break; - } - - return(success); -} - -#ifndef DBUG_OFF -/**************************************************************//** -Process entry point to user function. It makes the call to _db_enter_ -in mysqld.exe. The DBUG functions are defined in my_dbug.h. */ -extern "C" UNIV_INTERN -void -_db_enter_( - const char* _func_, /*!< in: current function name */ - const char* _file_, /*!< in: current file name */ - uint _line_, /*!< in: current source line number */ - const char** _sfunc_, /*!< out: previous _func_ */ - const char** _sfile_, /*!< out: previous _file_ */ - uint* _slevel_, /*!< out: previous nesting level */ - char*** _sframep_) /*!< out: previous frame pointer */ -{ - if (wdl_db_enter_ != NULL) { - - wdl_db_enter_(_func_, _file_, _line_, _sfunc_, _sfile_, - _slevel_, _sframep_); - } -} - -/**************************************************************//** -Process exit from user function. It makes the call to _db_return_() -in the server. */ -extern "C" UNIV_INTERN -void -_db_return_( - uint _line_, /*!< in: current source line number */ - const char** _sfunc_, /*!< out: previous _func_ */ - const char** _sfile_, /*!< out: previous _file_ */ - uint* _slevel_) /*!< out: previous level */ -{ - if (wdl_db_return_ != NULL) { - - wdl_db_return_(_line_, _sfunc_, _sfile_, _slevel_); - } -} - -/**************************************************************//** -Log arguments for subsequent use. It makes the call to _db_pargs_() -in the server. */ -extern "C" UNIV_INTERN -void -_db_pargs_( - uint _line_, /*!< in: current source line number */ - const char* keyword) /*!< in: keyword for current macro */ -{ - if (wdl_db_pargs_ != NULL) { - - wdl_db_pargs_(_line_, keyword); - } -} - -/**************************************************************//** -Handle print of debug lines. It saves the text into a buffer first, -then makes the call to _db_doprnt_() in the server. The text is -truncated to the size of buffer. */ -extern "C" UNIV_INTERN -void -_db_doprnt_( - const char* format, /*!< in: the format string */ - ...) /*!< in: list of arguments */ -{ - va_list argp; - char buffer[512]; - - if (wdl_db_doprnt_ != NULL) { - - va_start(argp, format); - /* it is ok to ignore the trunction. */ - _vsnprintf(buffer, sizeof(buffer), format, argp); - wdl_db_doprnt_(buffer); - va_end(argp); - } -} - -/**************************************************************//** -Dump a string in hex. It makes the call to _db_dump_() in the server. */ -extern "C" UNIV_INTERN -void -_db_dump_( - uint _line_, /*!< in: current source line - number */ - const char* keyword, /*!< in: keyword list */ - const unsigned char* memory, /*!< in: memory to dump */ - size_t length) /*!< in: bytes to dump */ -{ - if (wdl_db_dump_ != NULL) { - - wdl_db_dump_(_line_, keyword, memory, length); - } -} - -#endif /* !DBUG_OFF */ -#endif /* defined (__WIN__) && defined (MYSQL_DYNAMIC_PLUGIN) */ === modified file 'storage/xtradb/ibuf/ibuf0ibuf.c' --- a/storage/xtradb/ibuf/ibuf0ibuf.c 2009-11-13 21:26:08 +0000 +++ b/storage/xtradb/ibuf/ibuf0ibuf.c 2010-01-15 15:58:25 +0000 @@ -390,6 +390,27 @@ ibuf_count_set( #endif /******************************************************************//** +Closes insert buffer and frees the data structures. */ +UNIV_INTERN +void +ibuf_close(void) +/*============*/ +{ + mutex_free(&ibuf_pessimistic_insert_mutex); + memset(&ibuf_pessimistic_insert_mutex, + 0x0, sizeof(ibuf_pessimistic_insert_mutex)); + + mutex_free(&ibuf_mutex); + memset(&ibuf_mutex, 0x0, sizeof(ibuf_mutex)); + + mutex_free(&ibuf_bitmap_mutex); + memset(&ibuf_bitmap_mutex, 0x0, sizeof(ibuf_mutex)); + + mem_free(ibuf); + ibuf = NULL; +} + +/******************************************************************//** Updates the size information of the ibuf, assuming the segment size has not changed. */ static === modified file 'storage/xtradb/include/btr0cur.h' --- a/storage/xtradb/include/btr0cur.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/btr0cur.h 2010-01-06 12:00:14 +0000 @@ -618,7 +618,7 @@ enum btr_cur_method { hash_node, and might be necessary to update */ BTR_CUR_BINARY, /*!< success using the binary search */ - BTR_CUR_INSERT_TO_IBUF, /*!< performed the intended insert to + BTR_CUR_INSERT_TO_IBUF /*!< performed the intended insert to the insert buffer */ }; === modified file 'storage/xtradb/include/btr0sea.h' --- a/storage/xtradb/include/btr0sea.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/btr0sea.h 2010-01-06 12:00:14 +0000 @@ -41,6 +41,12 @@ void btr_search_sys_create( /*==================*/ ulint hash_size); /*!< in: hash index hash table size */ +/*****************************************************************//** +Frees the adaptive search system at a database shutdown. */ +UNIV_INTERN +void +btr_search_sys_free(void); +/*=====================*/ /********************************************************************//** Disable the adaptive hash search system and empty the index. */ === modified file 'storage/xtradb/include/buf0buf.h' --- a/storage/xtradb/include/buf0buf.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/buf0buf.h 2010-01-06 12:00:14 +0000 @@ -346,7 +346,7 @@ buf_page_release( mtr_t* mtr); /*!< in: mtr */ /********************************************************************//** Moves a page to the start of the buffer pool LRU list. This high-level -function can be used to prevent an important page from from slipping out of +function can be used to prevent an important page from slipping out of the buffer pool. */ UNIV_INTERN void @@ -707,15 +707,6 @@ buf_page_belongs_to_unzip_LRU( /*==========================*/ const buf_page_t* bpage) /*!< in: pointer to control block */ __attribute__((pure)); -/*********************************************************************//** -Determine the approximate LRU list position of a block. -@return LRU list position */ -UNIV_INLINE -ulint -buf_page_get_LRU_position( -/*======================*/ - const buf_page_t* bpage) /*!< in: control block */ - __attribute__((pure)); /*********************************************************************//** Gets the mutex of a block. @@ -825,14 +816,14 @@ buf_page_set_old( buf_page_t* bpage, /*!< in/out: control block */ ibool old); /*!< in: old */ /*********************************************************************//** -Determine if a block has been accessed in the buffer pool. -@return TRUE if accessed */ +Determine the time of first access of a block in the buffer pool. +@return ut_time_ms() at the time of first access, 0 if not accessed */ UNIV_INLINE -ibool +unsigned buf_page_is_accessed( /*=================*/ const buf_page_t* bpage) /*!< in: control block */ - __attribute__((pure)); + __attribute__((nonnull, pure)); /*********************************************************************//** Flag a block accessed. */ UNIV_INLINE @@ -840,7 +831,8 @@ void buf_page_set_accessed( /*==================*/ buf_page_t* bpage, /*!< in/out: control block */ - ibool accessed); /*!< in: accessed */ + ulint time_ms) /*!< in: ut_time_ms() */ + __attribute__((nonnull)); /*********************************************************************//** Gets the buf_block_t handle of a buffered file block if an uncompressed page frame exists, or NULL. @@ -1026,14 +1018,6 @@ buf_block_hash_get( /*===============*/ ulint space, /*!< in: space id */ ulint offset);/*!< in: offset of the page within space */ -/*******************************************************************//** -Increments the pool clock by one and returns its new value. Remember that -in the 32 bit version the clock wraps around at 4 billion! -@return new clock value */ -UNIV_INLINE -ulint -buf_pool_clock_tic(void); -/*====================*/ /*********************************************************************//** Gets the current length of the free list of buffer blocks. @return length of the free list */ @@ -1073,16 +1057,10 @@ struct buf_page_struct{ flushed to disk, this tells the flush_type. @see enum buf_flush */ - unsigned accessed:1; /*!< TRUE if the page has been accessed - while in the buffer pool: read-ahead - may read in pages which have not been - accessed yet; a thread is allowed to - read this for heuristic purposes - without holding any mutex or latch */ unsigned io_fix:2; /*!< type of pending I/O operation; also protected by buf_pool_mutex @see enum buf_io_fix */ - unsigned buf_fix_count:24;/*!< count of how manyfold this block + unsigned buf_fix_count:25;/*!< count of how manyfold this block is currently bufferfixed */ /* @} */ #endif /* !UNIV_HOTBACKUP */ @@ -1112,7 +1090,16 @@ struct buf_page_struct{ - BUF_BLOCK_FILE_PAGE: flush_list - BUF_BLOCK_ZIP_DIRTY: flush_list - BUF_BLOCK_ZIP_PAGE: zip_clean - - BUF_BLOCK_ZIP_FREE: zip_free[] */ + - BUF_BLOCK_ZIP_FREE: zip_free[] + + The contents of the list node + is undefined if !in_flush_list + && state == BUF_BLOCK_FILE_PAGE, + or if state is one of + BUF_BLOCK_MEMORY, + BUF_BLOCK_REMOVE_HASH or + BUF_BLOCK_READY_IN_USE. */ + /* resplit for optimistic use */ UT_LIST_NODE_T(buf_page_t) free; UT_LIST_NODE_T(buf_page_t) flush_list; @@ -1155,18 +1142,8 @@ struct buf_page_struct{ debugging */ //#endif /* UNIV_DEBUG */ unsigned old:1; /*!< TRUE if the block is in the old - blocks in the LRU list */ - unsigned LRU_position:31;/*!< value which monotonically - decreases (or may stay - constant if old==TRUE) toward - the end of the LRU list, if - buf_pool->ulint_clock has not - wrapped around: NOTE that this - value can only be used in - heuristic algorithms, because - of the possibility of a - wrap-around! */ - unsigned freed_page_clock:32;/*!< the value of + blocks in buf_pool->LRU_old */ + unsigned freed_page_clock:31;/*!< the value of buf_pool->freed_page_clock when this block was the last time put to the head of the @@ -1174,6 +1151,9 @@ struct buf_page_struct{ to read this for heuristic purposes without holding any mutex or latch */ + unsigned access_time:32; /*!< time of first access, or + 0 if the block was never accessed + in the buffer pool */ /* @} */ # ifdef UNIV_DEBUG_FILE_ACCESSES ibool file_page_was_freed; @@ -1318,6 +1298,31 @@ Compute the hash fold value for blocks i #define BUF_POOL_ZIP_FOLD_BPAGE(b) BUF_POOL_ZIP_FOLD((buf_block_t*) (b)) /* @} */ +/** @brief The buffer pool statistics structure. */ +struct buf_pool_stat_struct{ + ulint n_page_gets; /*!< number of page gets performed; + also successful searches through + the adaptive hash index are + counted as page gets; this field + is NOT protected by the buffer + pool mutex */ + ulint n_pages_read; /*!< number read operations */ + ulint n_pages_written;/*!< number write operations */ + ulint n_pages_created;/*!< number of pages created + in the pool with no read */ + ulint n_ra_pages_read;/*!< number of pages read in + as part of read ahead */ + ulint n_ra_pages_evicted;/*!< number of read ahead + pages that are evicted without + being accessed */ + ulint n_pages_made_young; /*!< number of pages made young, in + calls to buf_LRU_make_block_young() */ + ulint n_pages_not_made_young; /*!< number of pages not made + young because the first access + was not long enough ago, in + buf_page_peek_if_too_old() */ +}; + /** @brief The buffer pool structure. NOTE! The definition appears here only for other modules of this @@ -1342,28 +1347,16 @@ struct buf_pool_struct{ ulint n_pend_reads; /*!< number of pending read operations */ ulint n_pend_unzip; /*!< number of pending decompressions */ - time_t last_printout_time; /*!< when buf_print was last time + time_t last_printout_time; + /*!< when buf_print_io was last time called */ - ulint n_pages_read; /*!< number read operations */ - ulint n_pages_written;/*!< number write operations */ - ulint n_pages_created;/*!< number of pages created - in the pool with no read */ - ulint n_page_gets; /*!< number of page gets performed; - also successful searches through - the adaptive hash index are - counted as page gets; this field - is NOT protected by the buffer - pool mutex */ - ulint n_page_gets_old;/*!< n_page_gets when buf_print was - last time called: used to calculate - hit rate */ - ulint n_pages_read_old;/*!< n_pages_read when buf_print was - last time called */ - ulint n_pages_written_old;/*!< number write operations */ - ulint n_pages_created_old;/*!< number of pages created in - the pool with no read */ + buf_pool_stat_t stat; /*!< current statistics */ + buf_pool_stat_t old_stat; /*!< old statistics */ + /* @} */ + /** @name Page flushing algorithm fields */ + /* @{ */ UT_LIST_BASE_NODE_T(buf_page_t) flush_list; @@ -1379,10 +1372,6 @@ struct buf_pool_struct{ /*!< this is in the set state when there is no flush batch of the given type running */ - ulint ulint_clock; /*!< a sequence number used to count - time. NOTE! This counter wraps - around at 4 billion (if ulint == - 32 bits)! */ ulint freed_page_clock;/*!< a sequence number used to count the number of buffer blocks removed from the end of @@ -1406,17 +1395,18 @@ struct buf_pool_struct{ block list */ UT_LIST_BASE_NODE_T(buf_page_t) LRU; /*!< base node of the LRU list */ - buf_page_t* LRU_old; /*!< pointer to the about 3/8 oldest - blocks in the LRU list; NULL if LRU - length less than BUF_LRU_OLD_MIN_LEN; + buf_page_t* LRU_old; /*!< pointer to the about + buf_LRU_old_ratio/BUF_LRU_OLD_RATIO_DIV + oldest blocks in the LRU list; + NULL if LRU length less than + BUF_LRU_OLD_MIN_LEN; NOTE: when LRU_old != NULL, its length should always equal LRU_old_len */ ulint LRU_old_len; /*!< length of the LRU list from the block to which LRU_old points onward, including that block; see buf0lru.c for the restrictions - on this value; not defined if - LRU_old == NULL; + on this value; 0 if LRU_old == NULL; NOTE: LRU_old_len must be adjusted whenever LRU_old shrinks or grows! */ === modified file 'storage/xtradb/include/buf0buf.ic' --- a/storage/xtradb/include/buf0buf.ic 2009-11-13 21:26:08 +0000 +++ b/storage/xtradb/include/buf0buf.ic 2010-01-15 15:58:25 +0000 @@ -72,9 +72,30 @@ buf_page_peek_if_too_old( /*=====================*/ const buf_page_t* bpage) /*!< in: block to make younger */ { - return(buf_pool->freed_page_clock - >= buf_page_get_freed_page_clock(bpage) - + 1 + (buf_pool->curr_size / 4)); + if (UNIV_UNLIKELY(buf_pool->freed_page_clock == 0)) { + /* If eviction has not started yet, do not update the + statistics or move blocks in the LRU list. This is + either the warm-up phase or an in-memory workload. */ + return(FALSE); + } else if (buf_LRU_old_threshold_ms && bpage->old) { + unsigned access_time = buf_page_is_accessed(bpage); + + if (access_time > 0 + && (ut_time_ms() - access_time) + >= buf_LRU_old_threshold_ms) { + return(TRUE); + } + + buf_pool->stat.n_pages_not_made_young++; + return(FALSE); + } else { + /* FIXME: bpage->freed_page_clock is 31 bits */ + return((buf_pool->freed_page_clock & ((1UL << 31) - 1)) + > ((ulint) bpage->freed_page_clock + + (buf_pool->curr_size + * (BUF_LRU_OLD_RATIO_DIV - buf_LRU_old_ratio) + / (BUF_LRU_OLD_RATIO_DIV * 4)))); + } } /*********************************************************************//** @@ -125,23 +146,6 @@ try_again: return(lsn); } - -/*******************************************************************//** -Increments the buf_pool clock by one and returns its new value. Remember -that in the 32 bit version the clock wraps around at 4 billion! -@return new clock value */ -UNIV_INLINE -ulint -buf_pool_clock_tic(void) -/*====================*/ -{ - //ut_ad(buf_pool_mutex_own()); - ut_ad(mutex_own(&LRU_list_mutex)); - - buf_pool->ulint_clock++; - - return(buf_pool->ulint_clock); -} #endif /* !UNIV_HOTBACKUP */ /*********************************************************************//** @@ -288,21 +292,6 @@ buf_page_belongs_to_unzip_LRU( } /*********************************************************************//** -Determine the approximate LRU list position of a block. -@return LRU list position */ -UNIV_INLINE -ulint -buf_page_get_LRU_position( -/*======================*/ - const buf_page_t* bpage) /*!< in: control block */ -{ - ut_ad(buf_page_in_file(bpage)); - //ut_ad(buf_pool_mutex_own()); /* This is used in optimistic */ - - return(bpage->LRU_position); -} - -/*********************************************************************//** Gets the mutex of a block. @return pointer to mutex protecting bpage */ UNIV_INLINE @@ -508,10 +497,19 @@ buf_page_set_old( ut_ad(bpage->in_LRU_list); #ifdef UNIV_LRU_DEBUG - if (UT_LIST_GET_PREV(LRU, bpage) && UT_LIST_GET_NEXT(LRU, bpage) - && UT_LIST_GET_PREV(LRU, bpage)->old - == UT_LIST_GET_NEXT(LRU, bpage)->old) { - ut_a(UT_LIST_GET_PREV(LRU, bpage)->old == old); + ut_a((buf_pool->LRU_old_len == 0) == (buf_pool->LRU_old == NULL)); + /* If a block is flagged "old", the LRU_old list must exist. */ + ut_a(!old || buf_pool->LRU_old); + + if (UT_LIST_GET_PREV(LRU, bpage) && UT_LIST_GET_NEXT(LRU, bpage)) { + const buf_page_t* prev = UT_LIST_GET_PREV(LRU, bpage); + const buf_page_t* next = UT_LIST_GET_NEXT(LRU, bpage); + if (prev->old == next->old) { + ut_a(prev->old == old); + } else { + ut_a(!prev->old); + ut_a(buf_pool->LRU_old == (old ? bpage : next)); + } } #endif /* UNIV_LRU_DEBUG */ @@ -519,17 +517,17 @@ buf_page_set_old( } /*********************************************************************//** -Determine if a block has been accessed in the buffer pool. -@return TRUE if accessed */ +Determine the time of first access of a block in the buffer pool. +@return ut_time_ms() at the time of first access, 0 if not accessed */ UNIV_INLINE -ibool +unsigned buf_page_is_accessed( /*=================*/ const buf_page_t* bpage) /*!< in: control block */ { ut_ad(buf_page_in_file(bpage)); - return(bpage->accessed); + return(bpage->access_time); } /*********************************************************************//** @@ -539,12 +537,16 @@ void buf_page_set_accessed( /*==================*/ buf_page_t* bpage, /*!< in/out: control block */ - ibool accessed) /*!< in: accessed */ + ulint time_ms) /*!< in: ut_time_ms() */ { ut_a(buf_page_in_file(bpage)); + //ut_ad(buf_pool_mutex_own()); ut_ad(mutex_own(buf_page_get_mutex(bpage))); - bpage->accessed = accessed; + if (!bpage->access_time) { + /* Make this the time of the first access. */ + bpage->access_time = time_ms; + } } /*********************************************************************//** @@ -825,15 +827,15 @@ buf_page_get_newest_modification( ib_uint64_t lsn; mutex_t* block_mutex = buf_page_get_mutex_enter(bpage); - ut_a(block_mutex); - - if (buf_page_in_file(bpage)) { + if (block_mutex && buf_page_in_file(bpage)) { lsn = bpage->newest_modification; } else { lsn = 0; } - mutex_exit(block_mutex); + if (block_mutex) { + mutex_exit(block_mutex); + } return(lsn); } === modified file 'storage/xtradb/include/buf0lru.h' --- a/storage/xtradb/include/buf0lru.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/buf0lru.h 2010-01-06 12:00:14 +0000 @@ -69,7 +69,7 @@ These are low-level functions #########################################################################*/ /** Minimum LRU list length for which the LRU_old pointer is defined */ -#define BUF_LRU_OLD_MIN_LEN 80 +#define BUF_LRU_OLD_MIN_LEN 512 /* 8 megabytes of 16k pages */ /** Maximum LRU list search length in buf_flush_LRU_recommendation() */ #define BUF_LRU_FREE_SEARCH_LEN (5 + 2 * BUF_READ_AHEAD_AREA) @@ -84,15 +84,6 @@ void buf_LRU_invalidate_tablespace( /*==========================*/ ulint id); /*!< in: space id */ -/******************************************************************//** -Gets the minimum LRU_position field for the blocks in an initial segment -(determined by BUF_LRU_INITIAL_RATIO) of the LRU list. The limit is not -guaranteed to be precise, because the ulint_clock may wrap around. -@return the limit; zero if could not determine it */ -UNIV_INTERN -ulint -buf_LRU_get_recent_limit(void); -/*==========================*/ /********************************************************************//** Insert a compressed block into buf_pool->zip_clean in the LRU order. */ UNIV_INTERN @@ -203,6 +194,18 @@ void buf_LRU_make_block_old( /*===================*/ buf_page_t* bpage); /*!< in: control block */ +/**********************************************************************//** +Updates buf_LRU_old_ratio. +@return updated old_pct */ +UNIV_INTERN +uint +buf_LRU_old_ratio_update( +/*=====================*/ + uint old_pct,/*!< in: Reserve this percentage of + the buffer pool for "old" blocks. */ + ibool adjust);/*!< in: TRUE=adjust the LRU list; + FALSE=just assign buf_LRU_old_ratio + during the initialization of InnoDB */ /********************************************************************//** Update the historical stats that we are collecting for LRU eviction policy at the end of each interval. */ @@ -210,6 +213,18 @@ UNIV_INTERN void buf_LRU_stat_update(void); /*=====================*/ +/********************************************************************//** +Dump the LRU page list to the specific file. */ +UNIV_INTERN +ibool +buf_LRU_file_dump(void); +/*===================*/ +/********************************************************************//** +Read the pages based on the specific file.*/ +UNIV_INTERN +ibool +buf_LRU_file_restore(void); +/*======================*/ #if defined UNIV_DEBUG || defined UNIV_BUF_DEBUG /**********************************************************************//** @@ -229,6 +244,35 @@ buf_LRU_print(void); /*===============*/ #endif /* UNIV_DEBUG_PRINT || UNIV_DEBUG || UNIV_BUF_DEBUG */ +/** @name Heuristics for detecting index scan @{ */ +/** Reserve this much/BUF_LRU_OLD_RATIO_DIV of the buffer pool for +"old" blocks. Protected by buf_pool_mutex. */ +extern uint buf_LRU_old_ratio; +/** The denominator of buf_LRU_old_ratio. */ +#define BUF_LRU_OLD_RATIO_DIV 1024 +/** Maximum value of buf_LRU_old_ratio. +@see buf_LRU_old_adjust_len +@see buf_LRU_old_ratio_update */ +#define BUF_LRU_OLD_RATIO_MAX BUF_LRU_OLD_RATIO_DIV +/** Minimum value of buf_LRU_old_ratio. +@see buf_LRU_old_adjust_len +@see buf_LRU_old_ratio_update +The minimum must exceed +(BUF_LRU_OLD_TOLERANCE + 5) * BUF_LRU_OLD_RATIO_DIV / BUF_LRU_OLD_MIN_LEN. */ +#define BUF_LRU_OLD_RATIO_MIN 51 + +#if BUF_LRU_OLD_RATIO_MIN >= BUF_LRU_OLD_RATIO_MAX +# error "BUF_LRU_OLD_RATIO_MIN >= BUF_LRU_OLD_RATIO_MAX" +#endif +#if BUF_LRU_OLD_RATIO_MAX > BUF_LRU_OLD_RATIO_DIV +# error "BUF_LRU_OLD_RATIO_MAX > BUF_LRU_OLD_RATIO_DIV" +#endif + +/** Move blocks to "new" LRU list only if the first access was at +least this many milliseconds ago. Not protected by any mutex or latch. */ +extern uint buf_LRU_old_threshold_ms; +/* @} */ + /** @brief Statistics for selecting the LRU list for eviction. These statistics are not 'of' LRU but 'for' LRU. We keep count of I/O === modified file 'storage/xtradb/include/buf0rea.h' --- a/storage/xtradb/include/buf0rea.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/buf0rea.h 2010-01-06 12:00:14 +0000 @@ -27,28 +27,59 @@ Created 11/5/1995 Heikki Tuuri #define buf0rea_h #include "univ.i" +#include "trx0types.h" #include "buf0types.h" /********************************************************************//** +Low-level function which reads a page asynchronously from a file to the +buffer buf_pool if it is not already there, in which case does nothing. +Sets the io_fix flag and sets an exclusive lock on the buffer frame. The +flag is cleared and the x-lock released by an i/o-handler thread. +@return 1 if a read request was queued, 0 if the page already resided +in buf_pool, or if the page is in the doublewrite buffer blocks in +which case it is never read into the pool, or if the tablespace does +not exist or is being dropped +@return 1 if read request is issued. 0 if it is not */ +UNIV_INTERN +ulint +buf_read_page_low( +/*==============*/ + ulint* err, /*!< out: DB_SUCCESS or DB_TABLESPACE_DELETED if we are + trying to read from a non-existent tablespace, or a + tablespace which is just now being dropped */ + ibool sync, /*!< in: TRUE if synchronous aio is desired */ + ulint mode, /*!< in: BUF_READ_IBUF_PAGES_ONLY, ..., + ORed to OS_AIO_SIMULATED_WAKE_LATER (see below + at read-ahead functions) */ + ulint space, /*!< in: space id */ + ulint zip_size,/*!< in: compressed page size, or 0 */ + ibool unzip, /*!< in: TRUE=request uncompressed page */ + ib_int64_t tablespace_version, /*!< in: if the space memory object has + this timestamp different from what we are giving here, + treat the tablespace as dropped; this is a timestamp we + use to stop dangling page reads from a tablespace + which we have DISCARDed + IMPORTed back */ + ulint offset, /*!< in: page number */ + trx_t* trx); +/********************************************************************//** High-level function which reads a page asynchronously from a file to the buffer buf_pool if it is not already there. Sets the io_fix flag and sets an exclusive lock on the buffer frame. The flag is cleared and the x-lock -released by the i/o-handler thread. Does a random read-ahead if it seems -sensible. -@return number of page read requests issued: this can be greater than -1 if read-ahead occurred */ +released by the i/o-handler thread. +@return TRUE if page has been read in, FALSE in case of failure */ UNIV_INTERN -ulint +ibool buf_read_page( /*==========*/ ulint space, /*!< in: space id */ ulint zip_size,/*!< in: compressed page size in bytes, or 0 */ - ulint offset);/*!< in: page number */ + ulint offset, /*!< in: page number */ + trx_t* trx); /********************************************************************//** Applies linear read-ahead if in the buf_pool the page is a border page of a linear read-ahead area and all the pages in the area have been accessed. Does not read any page if the read-ahead mechanism is not activated. Note -that the the algorithm looks at the 'natural' adjacent successor and +that the algorithm looks at the 'natural' adjacent successor and predecessor of the page, which on the leaf level of a B-tree are the next and previous page in the chain of leaves. To know these, the page specified in (space, offset) must already be present in the buf_pool. Thus, the @@ -74,8 +105,9 @@ buf_read_ahead_linear( /*==================*/ ulint space, /*!< in: space id */ ulint zip_size,/*!< in: compressed page size in bytes, or 0 */ - ulint offset);/*!< in: page number of a page; NOTE: the current thread + ulint offset, /*!< in: page number of a page; NOTE: the current thread must want access to this page (see NOTE 3 above) */ + trx_t* trx); /********************************************************************//** Issues read requests for pages which the ibuf module wants to read in, in order to contract the insert buffer tree. Technically, this function is like === modified file 'storage/xtradb/include/buf0types.h' --- a/storage/xtradb/include/buf0types.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/buf0types.h 2010-01-06 12:00:14 +0000 @@ -34,6 +34,8 @@ typedef struct buf_block_struct buf_blo typedef struct buf_chunk_struct buf_chunk_t; /** Buffer pool comprising buf_chunk_t */ typedef struct buf_pool_struct buf_pool_t; +/** Buffer pool statistics struct */ +typedef struct buf_pool_stat_struct buf_pool_stat_t; /** A buffer frame. @see page_t */ typedef byte buf_frame_t; === modified file 'storage/xtradb/include/db0err.h' --- a/storage/xtradb/include/db0err.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/db0err.h 2010-01-06 12:00:14 +0000 @@ -32,6 +32,7 @@ enum db_err { /* The following are error codes */ DB_ERROR, + DB_INTERRUPTED, DB_OUT_OF_MEMORY, DB_OUT_OF_FILE_SPACE, DB_LOCK_WAIT, === modified file 'storage/xtradb/include/dict0crea.h' --- a/storage/xtradb/include/dict0crea.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/dict0crea.h 2010-01-06 12:00:14 +0000 @@ -110,7 +110,7 @@ dict_create_or_check_foreign_constraint_ Adds foreign key definitions to data dictionary tables in the database. We look at table->foreign_list, and also generate names to constraints that were not named by the user. A generated constraint has a name of the format -databasename/tablename_ibfk_<number>, where the numbers start from 1, and are +databasename/tablename_ibfk_NUMBER, where the numbers start from 1, and are given locally for this table, that is, the number is not global, as in the old format constraints < 4.0.18 it used to be. @return error code or DB_SUCCESS */ === modified file 'storage/xtradb/include/dict0dict.h' --- a/storage/xtradb/include/dict0dict.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/dict0dict.h 2010-01-06 12:00:14 +0000 @@ -712,7 +712,7 @@ dict_index_find_on_id_low( dulint id); /*!< in: index id */ /**********************************************************************//** Adds an index to the dictionary cache. -@return DB_SUCCESS or error code */ +@return DB_SUCCESS, DB_TOO_BIG_RECORD, or DB_CORRUPTION */ UNIV_INTERN ulint dict_index_add_to_cache( @@ -1157,6 +1157,13 @@ void dict_ind_init(void); /*===============*/ +/**********************************************************************//** +Closes the data dictionary module. */ +UNIV_INTERN +void +dict_close(void); +/*============*/ + #ifndef UNIV_NONINL #include "dict0dict.ic" #endif === modified file 'storage/xtradb/include/dict0mem.h' --- a/storage/xtradb/include/dict0mem.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/dict0mem.h 2010-01-06 12:00:14 +0000 @@ -317,7 +317,7 @@ struct dict_foreign_struct{ char* id; /*!< id of the constraint as a null-terminated string */ unsigned n_fields:10; /*!< number of indexes' first fields - for which the the foreign key + for which the foreign key constraint is defined: we allow the indexes to contain more fields than mentioned in the constraint, as long === modified file 'storage/xtradb/include/fil0fil.h' --- a/storage/xtradb/include/fil0fil.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/fil0fil.h 2010-01-06 12:00:14 +0000 @@ -224,15 +224,6 @@ fil_space_create( 0 for uncompressed tablespaces */ ulint purpose);/*!< in: FIL_TABLESPACE, or FIL_LOG if log */ /*******************************************************************//** -Frees a space object from a the tablespace memory cache. Closes the files in -the chain but does not delete them. -@return TRUE if success */ -UNIV_INTERN -ibool -fil_space_free( -/*===========*/ - ulint id); /*!< in: space id */ -/*******************************************************************//** Returns the size of the space in pages. The tablespace must be cached in the memory cache. @return space size, 0 if space not found */ @@ -278,6 +269,12 @@ fil_init( ulint hash_size, /*!< in: hash table size */ ulint max_n_open); /*!< in: max number of open files */ /*******************************************************************//** +Initializes the tablespace memory cache. */ +UNIV_INTERN +void +fil_close(void); +/*===========*/ +/*******************************************************************//** Opens all log files and system tablespace data files. They stay open until the database server shutdown. This should be called at a server startup after the space objects for the log and the system tablespace have been created. The @@ -614,9 +611,12 @@ fil_space_get_n_reserved_extents( Reads or writes data. This operation is asynchronous (aio). @return DB_SUCCESS, or DB_TABLESPACE_DELETED if we are trying to do i/o on a tablespace which does not exist */ +#define fil_io(type, sync, space_id, zip_size, block_offset, byte_offset, len, buf, message) \ + _fil_io(type, sync, space_id, zip_size, block_offset, byte_offset, len, buf, message, NULL) + UNIV_INTERN ulint -fil_io( +_fil_io( /*===*/ ulint type, /*!< in: OS_FILE_READ or OS_FILE_WRITE, ORed to OS_FILE_LOG, if a log i/o @@ -641,8 +641,25 @@ fil_io( void* buf, /*!< in/out: buffer where to store read data or from where to write; in aio this must be appropriately aligned */ - void* message); /*!< in: message for aio handler if non-sync + void* message, /*!< in: message for aio handler if non-sync aio used, else ignored */ + trx_t* trx); +/********************************************************************//** +Confirm whether the parameters are valid or not */ +UNIV_INTERN +ibool +fil_area_is_exist( +/*==============*/ + ulint space_id, /*!< in: space id */ + ulint zip_size, /*!< in: compressed page size in bytes; + 0 for uncompressed pages */ + ulint block_offset, /*!< in: offset in number of blocks */ + ulint byte_offset, /*!< in: remainder of offset in bytes; in + aio this must be divisible by the OS block + size */ + ulint len); /*!< in: how many bytes to read or write; this + must not cross a file boundary; in aio this + must be a block size multiple */ /**********************************************************************//** Waits for an aio operation to complete. This function is used to write the handler for completed requests. The aio array of pending requests is divided === modified file 'storage/xtradb/include/fsp0fsp.h' --- a/storage/xtradb/include/fsp0fsp.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/fsp0fsp.h 2010-01-06 12:00:14 +0000 @@ -42,7 +42,7 @@ fsp_init(void); /*==========*/ /**********************************************************************//** Gets the current free limit of the system tablespace. The free limit -means the place of the first page which has never been put to the the +means the place of the first page which has never been put to the free list for allocation. The space above that address is initialized to zero. Sets also the global variable log_fsp_current_free_limit. @return free limit in megabytes */ === modified file 'storage/xtradb/include/ibuf0ibuf.h' --- a/storage/xtradb/include/ibuf0ibuf.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/ibuf0ibuf.h 2010-01-06 12:00:14 +0000 @@ -356,6 +356,12 @@ void ibuf_print( /*=======*/ FILE* file); /*!< in: file where to print */ +/******************************************************************//** +Closes insert buffer and frees the data structures. */ +UNIV_INTERN +void +ibuf_close(void); +/*============*/ #define IBUF_HEADER_PAGE_NO FSP_IBUF_HEADER_PAGE_NO #define IBUF_TREE_ROOT_PAGE_NO FSP_IBUF_TREE_ROOT_PAGE_NO === modified file 'storage/xtradb/include/lock0lock.h' --- a/storage/xtradb/include/lock0lock.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/lock0lock.h 2010-01-06 12:00:14 +0000 @@ -59,6 +59,12 @@ lock_sys_create( /*============*/ ulint n_cells); /*!< in: number of slots in lock hash table */ /*********************************************************************//** +Closes the lock system at database shutdown. */ +UNIV_INTERN +void +lock_sys_close(void); +/*================*/ +/*********************************************************************//** Checks if some transaction has an implicit x-lock on a record in a clustered index. @return transaction which has the x-lock, or NULL */ @@ -630,6 +636,14 @@ lock_number_of_rows_locked( /*=======================*/ trx_t* trx); /*!< in: transaction */ /*******************************************************************//** +Check if a transaction holds any autoinc locks. +@return TRUE if the transaction holds any AUTOINC locks. */ +UNIV_INTERN +ibool +lock_trx_holds_autoinc_locks( +/*=========================*/ + const trx_t* trx); /*!< in: transaction */ +/*******************************************************************//** Release all the transaction's autoinc locks. */ UNIV_INTERN void === modified file 'storage/xtradb/include/log0log.h' --- a/storage/xtradb/include/log0log.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/log0log.h 2010-01-06 12:00:14 +0000 @@ -118,10 +118,9 @@ UNIV_INLINE ib_uint64_t log_reserve_and_write_fast( /*=======================*/ - byte* str, /*!< in: string */ + const void* str, /*!< in: string */ ulint len, /*!< in: string length */ - ib_uint64_t* start_lsn,/*!< out: start lsn of the log record */ - ibool* success);/*!< out: TRUE if success */ + ib_uint64_t* start_lsn);/*!< out: start lsn of the log record */ /***********************************************************************//** Releases the log mutex. */ UNIV_INLINE @@ -283,7 +282,7 @@ log_make_checkpoint_at( later lsn, if IB_ULONGLONG_MAX, makes a checkpoint at the latest lsn */ ibool write_always); /*!< in: the function normally checks if - the the new checkpoint would have a + the new checkpoint would have a greater lsn than the previous one: if not, then no physical write is done; by setting this parameter TRUE, a @@ -573,6 +572,18 @@ UNIV_INTERN void log_refresh_stats(void); /*===================*/ +/********************************************************** +Shutdown the log system but do not release all the memory. */ +UNIV_INTERN +void +log_shutdown(void); +/*==============*/ +/********************************************************** +Free the log system data structures. */ +UNIV_INTERN +void +log_mem_free(void); +/*==============*/ extern log_t* log_sys; @@ -585,7 +596,7 @@ extern log_t* log_sys; #define LOG_RECOVER 98887331 /* The counting of lsn's starts from this value: this must be non-zero */ -#define LOG_START_LSN ((ib_uint64_t) (16 * OS_FILE_LOG_BLOCK_SIZE)) +#define LOG_START_LSN ((ib_uint64_t) (16 * OS_FILE_LOG_BLOCK_SIZE)) #define LOG_BUFFER_SIZE (srv_log_buffer_size * UNIV_PAGE_SIZE) #define LOG_ARCHIVE_BUF_SIZE (srv_log_buffer_size * UNIV_PAGE_SIZE / 4) @@ -722,9 +733,12 @@ struct log_group_struct{ ulint lsn_offset; /*!< the offset of the above lsn */ ulint n_pending_writes;/*!< number of currently pending flush writes for this log group */ + byte** file_header_bufs_ptr;/*!< unaligned buffers */ byte** file_header_bufs;/*!< buffers for each file header in the group */ +#ifdef UNIV_LOG_ARCHIVE /*-----------------------------*/ + byte** archive_file_header_bufs_ptr;/*!< unaligned buffers */ byte** archive_file_header_bufs;/*!< buffers for each file header in the group */ ulint archive_space_id;/*!< file space which @@ -743,10 +757,12 @@ struct log_group_struct{ completion function then sets the new value to ..._file_no */ ulint next_archived_offset; /*!< like the preceding field */ +#endif /* UNIV_LOG_ARCHIVE */ /*-----------------------------*/ ib_uint64_t scanned_lsn; /*!< used only in recovery: recovery scan succeeded up to this lsn in this log group */ + byte* checkpoint_buf_ptr;/*!< unaligned checkpoint header */ byte* checkpoint_buf; /*!< checkpoint header is written from this buffer to the group */ UT_LIST_NODE_T(log_group_t) @@ -764,6 +780,7 @@ struct log_struct{ #ifndef UNIV_HOTBACKUP mutex_t mutex; /*!< mutex protecting the log */ #endif /* !UNIV_HOTBACKUP */ + byte* buf_ptr; /* unaligned log buffer */ byte* buf; /*!< log buffer */ ulint buf_size; /*!< log buffer size in bytes */ ulint max_buf_free; /*!< recommended maximum value of @@ -900,6 +917,7 @@ struct log_struct{ should wait for this without owning the log mutex */ #endif /* !UNIV_HOTBACKUP */ + byte* checkpoint_buf_ptr;/* unaligned checkpoint header */ byte* checkpoint_buf; /*!< checkpoint header is read to this buffer */ /* @} */ === modified file 'storage/xtradb/include/log0log.ic' --- a/storage/xtradb/include/log0log.ic 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/log0log.ic 2010-01-06 12:00:14 +0000 @@ -27,6 +27,7 @@ Created 12/9/1995 Heikki Tuuri #include "mach0data.h" #include "mtr0mtr.h" +#ifdef UNIV_LOG_DEBUG /******************************************************//** Checks by parsing that the catenated log segment for a single mtr is consistent. */ @@ -34,11 +35,12 @@ UNIV_INTERN ibool log_check_log_recs( /*===============*/ - byte* buf, /*!< in: pointer to the start of + const byte* buf, /*!< in: pointer to the start of the log segment in the log_sys->buf log buffer */ ulint len, /*!< in: segment length in bytes */ ib_uint64_t buf_start_lsn); /*!< in: buffer start lsn */ +#endif /* UNIV_LOG_DEBUG */ /************************************************************//** Gets a log block flush bit. @@ -305,55 +307,76 @@ UNIV_INLINE ib_uint64_t log_reserve_and_write_fast( /*=======================*/ - byte* str, /*!< in: string */ + const void* str, /*!< in: string */ ulint len, /*!< in: string length */ - ib_uint64_t* start_lsn,/*!< out: start lsn of the log record */ - ibool* success)/*!< out: TRUE if success */ + ib_uint64_t* start_lsn)/*!< out: start lsn of the log record */ { - log_t* log = log_sys; ulint data_len; - ib_uint64_t lsn; - - *success = TRUE; - - mutex_enter(&(log->mutex)); - - data_len = len + log->buf_free % OS_FILE_LOG_BLOCK_SIZE; +#ifdef UNIV_LOG_LSN_DEBUG + /* length of the LSN pseudo-record */ + ulint lsn_len = 1 + + mach_get_compressed_size(log_sys->lsn >> 32) + + mach_get_compressed_size(log_sys->lsn & 0xFFFFFFFFUL); +#endif /* UNIV_LOG_LSN_DEBUG */ + + mutex_enter(&log_sys->mutex); + + data_len = len +#ifdef UNIV_LOG_LSN_DEBUG + + lsn_len +#endif /* UNIV_LOG_LSN_DEBUG */ + + log_sys->buf_free % OS_FILE_LOG_BLOCK_SIZE; if (data_len >= OS_FILE_LOG_BLOCK_SIZE - LOG_BLOCK_TRL_SIZE) { /* The string does not fit within the current log block or the log block would become full */ - *success = FALSE; - - mutex_exit(&(log->mutex)); + mutex_exit(&log_sys->mutex); return(0); } - *start_lsn = log->lsn; + *start_lsn = log_sys->lsn; + +#ifdef UNIV_LOG_LSN_DEBUG + { + /* Write the LSN pseudo-record. */ + byte* b = &log_sys->buf[log_sys->buf_free]; + *b++ = MLOG_LSN | (MLOG_SINGLE_REC_FLAG & *(const byte*) str); + /* Write the LSN in two parts, + as a pseudo page number and space id. */ + b += mach_write_compressed(b, log_sys->lsn >> 32); + b += mach_write_compressed(b, log_sys->lsn & 0xFFFFFFFFUL); + ut_a(b - lsn_len == &log_sys->buf[log_sys->buf_free]); - ut_memcpy(log->buf + log->buf_free, str, len); + memcpy(b, str, len); + len += lsn_len; + } +#else /* UNIV_LOG_LSN_DEBUG */ + memcpy(log_sys->buf + log_sys->buf_free, str, len); +#endif /* UNIV_LOG_LSN_DEBUG */ - log_block_set_data_len((byte*) ut_align_down(log->buf + log->buf_free, + log_block_set_data_len((byte*) ut_align_down(log_sys->buf + + log_sys->buf_free, OS_FILE_LOG_BLOCK_SIZE), data_len); #ifdef UNIV_LOG_DEBUG - log->old_buf_free = log->buf_free; - log->old_lsn = log->lsn; + log_sys->old_buf_free = log_sys->buf_free; + log_sys->old_lsn = log_sys->lsn; #endif - log->buf_free += len; + log_sys->buf_free += len; - ut_ad(log->buf_free <= log->buf_size); + ut_ad(log_sys->buf_free <= log_sys->buf_size); - lsn = log->lsn += len; + log_sys->lsn += len; #ifdef UNIV_LOG_DEBUG - log_check_log_recs(log->buf + log->old_buf_free, - log->buf_free - log->old_buf_free, log->old_lsn); + log_check_log_recs(log_sys->buf + log_sys->old_buf_free, + log_sys->buf_free - log_sys->old_buf_free, + log_sys->old_lsn); #endif - return(lsn); + return(log_sys->lsn); } /***********************************************************************//** === modified file 'storage/xtradb/include/log0recv.h' --- a/storage/xtradb/include/log0recv.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/log0recv.h 2010-01-06 12:00:14 +0000 @@ -239,6 +239,18 @@ UNIV_INTERN void recv_sys_create(void); /*=================*/ +/**********************************************************//** +Release recovery system mutexes. */ +UNIV_INTERN +void +recv_sys_close(void); +/*================*/ +/********************************************************//** +Frees the recovery system memory. */ +UNIV_INTERN +void +recv_sys_mem_free(void); +/*===================*/ /********************************************************//** Inits the recovery system for a recovery operation. */ UNIV_INTERN @@ -246,6 +258,12 @@ void recv_sys_init( /*==========*/ ulint available_memory); /*!< in: available memory in bytes */ +/********************************************************//** +Reset the state of the recovery system variables. */ +UNIV_INTERN +void +recv_sys_var_init(void); +/*===================*/ /*******************************************************************//** Empties the hash table of stored log records, applying them to appropriate pages. */ @@ -412,6 +430,39 @@ struct recv_sys_struct{ hash_table_t* addr_hash;/*!< hash table of file addresses of pages */ ulint n_addrs;/*!< number of not processed hashed file addresses in the hash table */ + +/* If you modified the following defines at original file, + You should also modify them. */ +/* defined in os0file.c */ +#define OS_AIO_MERGE_N_CONSECUTIVE 64 +/* defined in log0recv.c */ +#define RECV_READ_AHEAD_AREA 32 + time_t stats_recv_start_time; + ulint stats_recv_turns; + + ulint stats_read_requested_pages; + ulint stats_read_in_area[RECV_READ_AHEAD_AREA]; + + ulint stats_read_io_pages; + ulint stats_read_io_consecutive[OS_AIO_MERGE_N_CONSECUTIVE]; + ulint stats_write_io_pages; + ulint stats_write_io_consecutive[OS_AIO_MERGE_N_CONSECUTIVE]; + + ulint stats_doublewrite_check_pages; + ulint stats_doublewrite_overwrite_pages; + + ulint stats_recover_pages_with_read; + ulint stats_recover_pages_without_read; + + ulint stats_log_recs; + ulint stats_log_len_sum; + + ulint stats_applied_log_recs; + ulint stats_applied_log_len_sum; + ulint stats_pages_already_new; + + ib_uint64_t stats_oldest_modified_lsn; + ib_uint64_t stats_newest_modified_lsn; }; /** The recovery system */ @@ -433,6 +484,11 @@ are allowed yet: the variable name is mi extern ibool recv_no_ibuf_operations; /** TRUE when recv_init_crash_recovery() has been called. */ extern ibool recv_needed_recovery; +#ifdef UNIV_DEBUG +/** TRUE if writing to the redo log (mtr_commit) is forbidden. +Protected by log_sys->mutex. */ +extern ibool recv_no_log_write; +#endif /* UNIV_DEBUG */ /** TRUE if buf_page_is_corrupted() should check if the log sequence number (FIL_PAGE_LSN) is in the future. Initially FALSE, and set by === modified file 'storage/xtradb/include/mem0mem.h' --- a/storage/xtradb/include/mem0mem.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/mem0mem.h 2010-01-06 12:00:14 +0000 @@ -82,6 +82,13 @@ void mem_init( /*=====*/ ulint size); /*!< in: common pool size in bytes */ +/******************************************************************//** +Closes the memory system. */ +UNIV_INTERN +void +mem_close(void); +/*===========*/ + /**************************************************************//** Use this macro instead of the corresponding function! Macro for memory heap creation. */ === modified file 'storage/xtradb/include/mem0pool.h' --- a/storage/xtradb/include/mem0pool.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/mem0pool.h 2010-01-06 12:00:14 +0000 @@ -62,6 +62,13 @@ mem_pool_create( /*============*/ ulint size); /*!< in: pool size in bytes */ /********************************************************************//** +Frees a memory pool. */ +UNIV_INTERN +void +mem_pool_free( +/*==========*/ + mem_pool_t* pool); /*!< in, own: memory pool */ +/********************************************************************//** Allocates memory from a pool. NOTE: This low-level function should only be used in mem0mem.*! @return own: allocated memory buffer */ === modified file 'storage/xtradb/include/mtr0mtr.h' --- a/storage/xtradb/include/mtr0mtr.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/mtr0mtr.h 2010-01-06 12:00:14 +0000 @@ -106,6 +106,9 @@ For 1 - 8 bytes, the flag value must giv #define MLOG_IBUF_BITMAP_INIT ((byte)27) /*!< initialize an ibuf bitmap page */ /*#define MLOG_FULL_PAGE ((byte)28) full contents of a page */ +#ifdef UNIV_LOG_LSN_DEBUG +# define MLOG_LSN ((byte)28) /* current LSN */ +#endif #define MLOG_INIT_FILE_PAGE ((byte)29) /*!< this means that a file page is taken into use and the prior @@ -118,7 +121,7 @@ For 1 - 8 bytes, the flag value must giv #define MLOG_WRITE_STRING ((byte)30) /*!< write a string to a page */ #define MLOG_MULTI_REC_END ((byte)31) /*!< if a single mtr writes - log records for several pages, + several log records, this log record ends the sequence of these records */ #define MLOG_DUMMY_RECORD ((byte)32) /*!< dummy log record used to === modified file 'storage/xtradb/include/os0file.h' --- a/storage/xtradb/include/os0file.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/os0file.h 2010-01-06 12:00:14 +0000 @@ -53,6 +53,7 @@ Created 10/21/1995 Heikki Tuuri #define os0file_h #include "univ.i" +#include "trx0types.h" #ifndef __WIN__ #include <dirent.h> @@ -157,6 +158,8 @@ log. */ to become available again */ #define OS_FILE_SHARING_VIOLATION 76 #define OS_FILE_ERROR_NOT_SPECIFIED 77 +#define OS_FILE_INSUFFICIENT_RESOURCE 78 +#define OS_FILE_OPERATION_ABORTED 79 /* @} */ /** Types for aio operations @{ */ @@ -497,9 +500,12 @@ os_file_get_last_error( /*******************************************************************//** Requests a synchronous read operation. @return TRUE if request was successful, FALSE if fail */ +#define os_file_read(file, buf, offset, offset_high, n) \ + _os_file_read(file, buf, offset, offset_high, n, NULL) + UNIV_INTERN ibool -os_file_read( +_os_file_read( /*=========*/ os_file_t file, /*!< in: handle to a file */ void* buf, /*!< in: buffer where to read */ @@ -507,7 +513,8 @@ os_file_read( offset where to read */ ulint offset_high,/*!< in: most significant 32 bits of offset */ - ulint n); /*!< in: number of bytes to read */ + ulint n, /*!< in: number of bytes to read */ + trx_t* trx); /*******************************************************************//** Rewind file to its start, read at most size - 1 bytes from it to str, and NUL-terminate str. All errors are silently ignored. This function is @@ -619,6 +626,13 @@ os_aio_init( ulint n_write_segs, /*<! in: number of writer threads */ ulint n_slots_sync); /*<! in: number of slots in the sync aio array */ +/*********************************************************************** +Frees the asynchronous io system. */ +UNIV_INTERN +void +os_aio_free(void); +/*=============*/ + /*******************************************************************//** Requests an asynchronous i/o operation. @return TRUE if request was queued successfully, FALSE if fail */ @@ -654,10 +668,11 @@ os_aio( (can be used to identify a completed aio operation); ignored if mode is OS_AIO_SYNC */ - void* message2);/*!< in: message for the aio handler + void* message2,/*!< in: message for the aio handler (can be used to identify a completed aio operation); ignored if mode is OS_AIO_SYNC */ + trx_t* trx); /************************************************************************//** Wakes up all async i/o threads so that they know to exit themselves in shutdown. */ === modified file 'storage/xtradb/include/os0sync.h' --- a/storage/xtradb/include/os0sync.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/os0sync.h 2010-01-06 12:00:14 +0000 @@ -285,44 +285,74 @@ os_fast_mutex_free( /**********************************************************//** Atomic compare-and-swap and increment for InnoDB. */ -#ifdef HAVE_GCC_ATOMIC_BUILTINS +#if defined(HAVE_IB_GCC_ATOMIC_BUILTINS) + +#define HAVE_ATOMIC_BUILTINS + /**********************************************************//** Returns true if swapped, ptr is pointer to target, old_val is value to compare to, new_val is the value to swap in. */ + # define os_compare_and_swap(ptr, old_val, new_val) \ __sync_bool_compare_and_swap(ptr, old_val, new_val) + # define os_compare_and_swap_ulint(ptr, old_val, new_val) \ os_compare_and_swap(ptr, old_val, new_val) + # define os_compare_and_swap_lint(ptr, old_val, new_val) \ os_compare_and_swap(ptr, old_val, new_val) -# define os_compare_and_swap_thread_id(ptr, old_val, new_val) \ + +# ifdef HAVE_IB_ATOMIC_PTHREAD_T_GCC +# define os_compare_and_swap_thread_id(ptr, old_val, new_val) \ os_compare_and_swap(ptr, old_val, new_val) +# define INNODB_RW_LOCKS_USE_ATOMICS +# define IB_ATOMICS_STARTUP_MSG \ + "Mutexes and rw_locks use GCC atomic builtins" +# else /* HAVE_IB_ATOMIC_PTHREAD_T_GCC */ +# define IB_ATOMICS_STARTUP_MSG \ + "Mutexes use GCC atomic builtins, rw_locks do not" +# endif /* HAVE_IB_ATOMIC_PTHREAD_T_GCC */ + /**********************************************************//** Returns the resulting value, ptr is pointer to target, amount is the amount of increment. */ + # define os_atomic_increment(ptr, amount) \ __sync_add_and_fetch(ptr, amount) + # define os_atomic_increment_lint(ptr, amount) \ os_atomic_increment(ptr, amount) + # define os_atomic_increment_ulint(ptr, amount) \ os_atomic_increment(ptr, amount) + /**********************************************************//** Returns the old value of *ptr, atomically sets *ptr to new_val */ + # define os_atomic_test_and_set_byte(ptr, new_val) \ __sync_lock_test_and_set(ptr, new_val) + +#elif defined(HAVE_IB_SOLARIS_ATOMICS) + +#define HAVE_ATOMIC_BUILTINS + /* If not compiling with GCC or GCC doesn't support the atomic intrinsics and running on Solaris >= 10 use Solaris atomics */ -#elif defined(HAVE_SOLARIS_ATOMICS) + #include <atomic.h> + /**********************************************************//** Returns true if swapped, ptr is pointer to target, old_val is value to compare to, new_val is the value to swap in. */ + # define os_compare_and_swap_ulint(ptr, old_val, new_val) \ (atomic_cas_ulong(ptr, old_val, new_val) == old_val) + # define os_compare_and_swap_lint(ptr, old_val, new_val) \ ((lint)atomic_cas_ulong((ulong_t*) ptr, old_val, new_val) == old_val) -# ifdef INNODB_RW_LOCKS_USE_ATOMICS -# if SIZEOF_PTHREAD_T == 4 + +# ifdef HAVE_IB_ATOMIC_PTHREAD_T_SOLARIS +# if SIZEOF_PTHREAD_T == 4 # define os_compare_and_swap_thread_id(ptr, old_val, new_val) \ ((pthread_t)atomic_cas_32(ptr, old_val, new_val) == old_val) # elif SIZEOF_PTHREAD_T == 8 @@ -331,21 +361,35 @@ compare to, new_val is the value to swap # else # error "SIZEOF_PTHREAD_T != 4 or 8" # endif /* SIZEOF_PTHREAD_T CHECK */ -# endif /* INNODB_RW_LOCKS_USE_ATOMICS */ +# define INNODB_RW_LOCKS_USE_ATOMICS +# define IB_ATOMICS_STARTUP_MSG \ + "Mutexes and rw_locks use Solaris atomic functions" +# else /* HAVE_IB_ATOMIC_PTHREAD_T_SOLARIS */ +# define IB_ATOMICS_STARTUP_MSG \ + "Mutexes use Solaris atomic functions, rw_locks do not" +# endif /* HAVE_IB_ATOMIC_PTHREAD_T_SOLARIS */ /**********************************************************//** Returns the resulting value, ptr is pointer to target, amount is the amount of increment. */ + # define os_atomic_increment_lint(ptr, amount) \ atomic_add_long_nv((ulong_t*) ptr, amount) + # define os_atomic_increment_ulint(ptr, amount) \ atomic_add_long_nv(ptr, amount) + /**********************************************************//** Returns the old value of *ptr, atomically sets *ptr to new_val */ + # define os_atomic_test_and_set_byte(ptr, new_val) \ atomic_swap_uchar(ptr, new_val) -/* On Windows, use Windows atomics / interlocked */ + #elif defined(HAVE_WINDOWS_ATOMICS) + +#define HAVE_ATOMIC_BUILTINS + +/* On Windows, use Windows atomics / interlocked */ # ifdef _WIN64 # define win_cmp_and_xchg InterlockedCompareExchange64 # define win_xchg_and_add InterlockedExchangeAdd64 @@ -353,31 +397,46 @@ Returns the old value of *ptr, atomicall # define win_cmp_and_xchg InterlockedCompareExchange # define win_xchg_and_add InterlockedExchangeAdd # endif + /**********************************************************//** Returns true if swapped, ptr is pointer to target, old_val is value to compare to, new_val is the value to swap in. */ + # define os_compare_and_swap_ulint(ptr, old_val, new_val) \ (win_cmp_and_xchg(ptr, new_val, old_val) == old_val) + # define os_compare_and_swap_lint(ptr, old_val, new_val) \ (win_cmp_and_xchg(ptr, new_val, old_val) == old_val) -# ifdef INNODB_RW_LOCKS_USE_ATOMICS -# define os_compare_and_swap_thread_id(ptr, old_val, new_val) \ + +/* windows thread objects can always be passed to windows atomic functions */ +# define os_compare_and_swap_thread_id(ptr, old_val, new_val) \ (InterlockedCompareExchange(ptr, new_val, old_val) == old_val) -# endif /* INNODB_RW_LOCKS_USE_ATOMICS */ +# define INNODB_RW_LOCKS_USE_ATOMICS +# define IB_ATOMICS_STARTUP_MSG \ + "Mutexes and rw_locks use Windows interlocked functions" + /**********************************************************//** Returns the resulting value, ptr is pointer to target, amount is the amount of increment. */ + # define os_atomic_increment_lint(ptr, amount) \ (win_xchg_and_add(ptr, amount) + amount) + # define os_atomic_increment_ulint(ptr, amount) \ ((ulint) (win_xchg_and_add(ptr, amount) + amount)) + /**********************************************************//** Returns the old value of *ptr, atomically sets *ptr to new_val. InterlockedExchange() operates on LONG, and the LONG will be clobbered */ + # define os_atomic_test_and_set_byte(ptr, new_val) \ ((byte) InterlockedExchange(ptr, new_val)) -#endif /* HAVE_GCC_ATOMIC_BUILTINS */ + +#else +# define IB_ATOMICS_STARTUP_MSG \ + "Mutexes and rw_locks use InnoDB's own implementation" +#endif #ifndef UNIV_NONINL #include "os0sync.ic" === modified file 'storage/xtradb/include/page0page.h' --- a/storage/xtradb/include/page0page.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/page0page.h 2010-01-06 12:00:14 +0000 @@ -76,8 +76,11 @@ typedef byte page_header_t; header which are set in a page create */ /*----*/ #define PAGE_LEVEL 26 /* level of the node in an index tree; the - leaf level is the level 0 */ -#define PAGE_INDEX_ID 28 /* index id where the page belongs */ + leaf level is the level 0. This field should + not be written to after page creation. */ +#define PAGE_INDEX_ID 28 /* index id where the page belongs. + This field should not be written to after + page creation. */ #define PAGE_BTR_SEG_LEAF 36 /* file segment header for the leaf pages in a B-tree: defined only on the root page of a B-tree, but not in the root of an ibuf tree */ === modified file 'storage/xtradb/include/page0page.ic' --- a/storage/xtradb/include/page0page.ic 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/page0page.ic 2010-01-06 12:00:14 +0000 @@ -907,7 +907,7 @@ page_get_data_size( /************************************************************//** Allocates a block of memory from the free list of an index page. */ -UNIV_INTERN +UNIV_INLINE void page_mem_alloc_free( /*================*/ === modified file 'storage/xtradb/include/page0zip.h' --- a/storage/xtradb/include/page0zip.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/page0zip.h 2010-01-06 12:00:14 +0000 @@ -127,8 +127,12 @@ page_zip_decompress( /*================*/ page_zip_des_t* page_zip,/*!< in: data, ssize; out: m_start, m_end, m_nonempty, n_blobs */ - page_t* page) /*!< out: uncompressed page, may be trashed */ - __attribute__((nonnull)); + page_t* page, /*!< out: uncompressed page, may be trashed */ + ibool all) /*!< in: TRUE=decompress the whole page; + FALSE=verify but do not copy some + page header fields that should not change + after page creation */ + __attribute__((nonnull(1,2))); #ifdef UNIV_DEBUG /**********************************************************************//** @@ -385,8 +389,8 @@ IMPORTANT: if page_zip_reorganize() is i non-clustered index, the caller must update the insert buffer free bits in the same mini-transaction in such a way that the modification will be redo-logged. -@return TRUE on success, FALSE on failure; page and page_zip will be -left intact on failure. */ +@return TRUE on success, FALSE on failure; page_zip will be left +intact on failure, but page will be overwritten. */ UNIV_INTERN ibool page_zip_reorganize( === modified file 'storage/xtradb/include/pars0pars.h' --- a/storage/xtradb/include/pars0pars.h 2009-11-13 21:26:08 +0000 +++ b/storage/xtradb/include/pars0pars.h 2010-01-15 15:58:25 +0000 @@ -583,6 +583,12 @@ pars_info_get_bound_id( pars_info_t* info, /*!< in: info struct */ const char* name); /*!< in: bound id name to find */ +/******************************************************************//** +Release any resources used by the lexer. */ +UNIV_INTERN +void +pars_lexer_close(void); +/*==================*/ /** Extra information supplied for pars_sql(). */ struct pars_info_struct { === modified file 'storage/xtradb/include/rem0cmp.h' --- a/storage/xtradb/include/rem0cmp.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/rem0cmp.h 2010-01-06 12:00:14 +0000 @@ -89,7 +89,7 @@ cmp_dfield_dfield( /*************************************************************//** This function is used to compare a data tuple to a physical record. Only dtuple->n_fields_cmp first fields are taken into account for -the the data tuple! If we denote by n = n_fields_cmp, then rec must +the data tuple! If we denote by n = n_fields_cmp, then rec must have either m >= n fields, or it must differ from dtuple in some of the m fields rec has. If rec has an externally stored field we do not compare it but return with value 0 if such a comparison should be === modified file 'storage/xtradb/include/rem0rec.ic' --- a/storage/xtradb/include/rem0rec.ic 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/rem0rec.ic 2010-01-06 12:00:14 +0000 @@ -65,7 +65,7 @@ most significant bytes and bits are writ - offset_of_this_record) mod 64Ki, where mod is the modulo as a non-negative number; - we can calculate the the offset of the next + we can calculate the offset of the next record with the formula: relative_offset + offset_of_this_record mod UNIV_PAGE_SIZE === modified file 'storage/xtradb/include/row0ins.h' --- a/storage/xtradb/include/row0ins.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/row0ins.h 2010-01-06 12:00:14 +0000 @@ -45,7 +45,7 @@ row_ins_check_foreign_constraint( /*=============================*/ ibool check_ref,/*!< in: TRUE If we want to check that the referenced table is ok, FALSE if we - want to to check the foreign key table */ + want to check the foreign key table */ dict_foreign_t* foreign,/*!< in: foreign constraint; NOTE that the tables mentioned in it must be in the dictionary cache if they exist at all */ === modified file 'storage/xtradb/include/row0mysql.h' --- a/storage/xtradb/include/row0mysql.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/row0mysql.h 2010-01-06 12:00:14 +0000 @@ -177,7 +177,9 @@ row_update_prebuilt_trx( in MySQL handle */ trx_t* trx); /*!< in: transaction handle */ /*********************************************************************//** -Unlocks AUTO_INC type locks that were possibly reserved by a trx. */ +Unlocks AUTO_INC type locks that were possibly reserved by a trx. This +function should be called at the the end of an SQL statement, by the +connection thread that owns the transaction (trx->mysql_thd). */ UNIV_INTERN void row_unlock_table_autoinc_for_mysql( @@ -754,8 +756,6 @@ struct row_prebuilt_struct { store it here so that we can return it to MySQL */ /*----------------------*/ - UT_LIST_NODE_T(row_prebuilt_t) prebuilts; - /*!< list node of table->prebuilts */ ulint magic_n2; /*!< this should be the same as magic_n */ }; === modified file 'storage/xtradb/include/srv0srv.h' --- a/storage/xtradb/include/srv0srv.h 2009-11-13 21:26:08 +0000 +++ b/storage/xtradb/include/srv0srv.h 2010-01-15 15:58:25 +0000 @@ -80,6 +80,9 @@ at a time */ #define SRV_AUTO_EXTEND_INCREMENT \ (srv_auto_extend_increment * ((1024 * 1024) / UNIV_PAGE_SIZE)) +/* prototypes for new functions added to ha_innodb.cc */ +ibool innobase_get_slow_log(); + /* This is set to TRUE if the MySQL user has set it in MySQL */ extern ibool srv_lower_case_table_names; @@ -133,8 +136,9 @@ extern ulint* srv_data_file_is_raw_parti extern ibool srv_extra_undoslots; extern ibool srv_fast_recovery; +extern ibool srv_recovery_stats; -extern ibool srv_use_purge_thread; +extern ulint srv_use_purge_thread; extern ibool srv_auto_extend_last_data_file; extern ulint srv_last_file_size_max; @@ -235,12 +239,14 @@ extern ulong srv_replication_delay; extern long long srv_ibuf_max_size; extern ulong srv_ibuf_active_contract; extern ulong srv_ibuf_accel_rate; +extern ulint srv_checkpoint_age_target; extern ulong srv_flush_neighbor_pages; extern ulong srv_enable_unsafe_group_commit; extern ulong srv_read_ahead; extern ulong srv_adaptive_checkpoint; extern ulong srv_expand_import; +extern ulint srv_relax_table_creation; extern ulong srv_extra_rsegments; extern ulong srv_dict_size_limit; @@ -345,10 +351,6 @@ extern ulint srv_buf_pool_flushed; /** Number of buffer pool reads that led to the reading of a disk page */ extern ulint srv_buf_pool_reads; -/** Number of sequential read-aheads */ -extern ulint srv_read_ahead_seq; -/** Number of random read-aheads */ -extern ulint srv_read_ahead_rnd; /** Status variables to be passed to MySQL */ typedef struct export_var_struct export_struc; @@ -428,6 +430,7 @@ enum srv_thread_type { SRV_INSERT, /**< thread flushing the insert buffer to disk */ #endif SRV_PURGE, /* thread purging undo records */ + SRV_PURGE_WORKER, /* thread purging undo records */ SRV_MASTER /**< the master thread, (whose type number must be biggest) */ }; @@ -446,7 +449,7 @@ void srv_init(void); /*==========*/ /*********************************************************************//** -Frees the OS fast mutex created in srv_boot(). */ +Frees the data structures created in srv_init(). */ UNIV_INTERN void srv_free(void); @@ -509,6 +512,13 @@ srv_purge_thread( /*=============*/ void* arg); /* in: a dummy parameter required by os_thread_create */ +/************************************************************************* +The undo purge thread. */ +UNIV_INTERN +os_thread_ret_t +srv_purge_worker_thread( +/*====================*/ + void* arg); /*******************************************************************//** Tells the Innobase server that there has been activity in the database and wakes up the master thread if it is suspended (not sleeping). Used @@ -645,13 +655,13 @@ struct export_var_struct{ #ifdef UNIV_DEBUG ulint innodb_buffer_pool_pages_latched; /*!< Latched pages */ #endif /* UNIV_DEBUG */ - ulint innodb_buffer_pool_read_requests; /*!< buf_pool->n_page_gets */ + ulint innodb_buffer_pool_read_requests; /*!< buf_pool->stat.n_page_gets */ ulint innodb_buffer_pool_reads; /*!< srv_buf_pool_reads */ ulint innodb_buffer_pool_wait_free; /*!< srv_buf_pool_wait_free */ ulint innodb_buffer_pool_pages_flushed; /*!< srv_buf_pool_flushed */ ulint innodb_buffer_pool_write_requests;/*!< srv_buf_pool_write_requests */ - ulint innodb_buffer_pool_read_ahead_seq;/*!< srv_read_ahead_seq */ - ulint innodb_buffer_pool_read_ahead_rnd;/*!< srv_read_ahead_rnd */ + ulint innodb_buffer_pool_read_ahead; /*!< srv_read_ahead */ + ulint innodb_buffer_pool_read_ahead_evicted;/*!< srv_read_ahead evicted*/ ulint innodb_dblwr_pages_written; /*!< srv_dblwr_pages_written */ ulint innodb_dblwr_writes; /*!< srv_dblwr_writes */ ibool innodb_have_atomic_builtins; /*!< HAVE_ATOMIC_BUILTINS */ @@ -663,9 +673,9 @@ struct export_var_struct{ ulint innodb_os_log_pending_writes; /*!< srv_os_log_pending_writes */ ulint innodb_os_log_pending_fsyncs; /*!< fil_n_pending_log_flushes */ ulint innodb_page_size; /*!< UNIV_PAGE_SIZE */ - ulint innodb_pages_created; /*!< buf_pool->n_pages_created */ - ulint innodb_pages_read; /*!< buf_pool->n_pages_read */ - ulint innodb_pages_written; /*!< buf_pool->n_pages_written */ + ulint innodb_pages_created; /*!< buf_pool->stat.n_pages_created */ + ulint innodb_pages_read; /*!< buf_pool->stat.n_pages_read */ + ulint innodb_pages_written; /*!< buf_pool->stat.n_pages_written */ ulint innodb_row_lock_waits; /*!< srv_n_lock_wait_count */ ulint innodb_row_lock_current_waits; /*!< srv_n_lock_wait_current_count */ ib_int64_t innodb_row_lock_time; /*!< srv_n_lock_wait_time === modified file 'storage/xtradb/include/sync0rw.h' --- a/storage/xtradb/include/sync0rw.h 2009-11-13 21:26:08 +0000 +++ b/storage/xtradb/include/sync0rw.h 2010-01-15 15:58:25 +0000 @@ -120,7 +120,7 @@ is necessary only if the memory block co # endif /* UNIV_SYNC_DEBUG */ #else /* UNIV_DEBUG */ # define rw_lock_create(L, level) \ - rw_lock_create_func((L), __FILE__, __LINE__) + rw_lock_create_func((L), #L, NULL, 0) #endif /* UNIV_DEBUG */ /******************************************************************//** @@ -137,8 +137,8 @@ rw_lock_create_func( # ifdef UNIV_SYNC_DEBUG ulint level, /*!< in: level */ # endif /* UNIV_SYNC_DEBUG */ - const char* cmutex_name, /*!< in: mutex name */ #endif /* UNIV_DEBUG */ + const char* cmutex_name, /*!< in: mutex name */ const char* cfile_name, /*!< in: file name where created */ ulint cline); /*!< in: file line where created */ /******************************************************************//** @@ -540,7 +540,8 @@ struct rw_lock_struct { ulint level; /*!< Level in the global latching order. */ #endif /* UNIV_SYNC_DEBUG */ ulint count_os_wait; /*!< Count of os_waits. May not be accurate */ - const char* cfile_name;/*!< File name where lock created */ + //const char* cfile_name;/*!< File name where lock created */ + const char* lock_name;/*!< lock name */ /* last s-lock file/line is not guaranteed to be correct */ const char* last_s_file_name;/*!< File name where last s-locked */ const char* last_x_file_name;/*!< File name where last x-locked */ @@ -551,7 +552,7 @@ struct rw_lock_struct { are at the start of this struct, thus we can peek this field without causing much memory bus traffic */ - unsigned cline:14; /*!< Line where created */ + //unsigned cline:14; /*!< Line where created */ unsigned last_s_line:14; /*!< Line number where last time s-locked */ unsigned last_x_line:14; /*!< Line number where last time x-locked */ ulint magic_n; /*!< RW_LOCK_MAGIC_N */ === modified file 'storage/xtradb/include/sync0sync.h' --- a/storage/xtradb/include/sync0sync.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/sync0sync.h 2010-01-06 12:00:14 +0000 @@ -80,7 +80,7 @@ necessary only if the memory block conta # endif #else # define mutex_create(M, level) \ - mutex_create_func((M), __FILE__, __LINE__) + mutex_create_func((M), #M, NULL, 0) #endif /******************************************************************//** @@ -93,8 +93,8 @@ void mutex_create_func( /*==============*/ mutex_t* mutex, /*!< in: pointer to memory */ -#ifdef UNIV_DEBUG const char* cmutex_name, /*!< in: mutex name */ +#ifdef UNIV_DEBUG # ifdef UNIV_SYNC_DEBUG ulint level, /*!< in: level */ # endif /* UNIV_SYNC_DEBUG */ @@ -513,7 +513,7 @@ struct mutex_struct { os_fast_mutex; /*!< We use this OS mutex in place of lock_word when atomic operations are not enabled */ #endif - ulint waiters; /*!< This ulint is set to 1 if there are (or + volatile ulint waiters; /*!< This ulint is set to 1 if there are (or may be) threads waiting in the global wait array for this mutex to be released. Otherwise, this is 0. */ @@ -524,9 +524,9 @@ struct mutex_struct { ulint line; /*!< Line where the mutex was locked */ ulint level; /*!< Level in the global latching order */ #endif /* UNIV_SYNC_DEBUG */ +#ifdef UNIV_DEBUG const char* cfile_name;/*!< File name where mutex created */ ulint cline; /*!< Line where created */ -#ifdef UNIV_DEBUG os_thread_id_t thread_id; /*!< The thread id of the thread which locked the mutex. */ ulint magic_n; /*!< MUTEX_MAGIC_N */ @@ -541,9 +541,9 @@ struct mutex_struct { ulong count_os_yield; /*!< count of os_wait */ ulonglong lspent_time; /*!< mutex os_wait timer msec */ ulonglong lmax_spent_time;/*!< mutex os_wait timer msec */ - const char* cmutex_name; /*!< mutex name */ ulint mutex_type; /*!< 0=usual mutex, 1=rw_lock mutex */ #endif /* UNIV_DEBUG */ + const char* cmutex_name; /*!< mutex name */ }; /** The global array of wait cells for implementation of the databases own === modified file 'storage/xtradb/include/thr0loc.h' --- a/storage/xtradb/include/thr0loc.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/thr0loc.h 2010-01-06 12:00:14 +0000 @@ -39,6 +39,12 @@ UNIV_INTERN void thr_local_init(void); /*================*/ + /****************************************************************//** +Close the thread local storage module. */ +UNIV_INTERN +void +thr_local_close(void); +/*=================*/ /*******************************************************************//** Creates a local storage struct for the calling new thread. */ UNIV_INTERN === modified file 'storage/xtradb/include/trx0i_s.h' --- a/storage/xtradb/include/trx0i_s.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/trx0i_s.h 2010-01-06 12:00:14 +0000 @@ -141,6 +141,13 @@ void trx_i_s_cache_init( /*===============*/ trx_i_s_cache_t* cache); /*!< out: cache to init */ +/*******************************************************************//** +Free the INFORMATION SCHEMA trx related cache. */ +UNIV_INTERN +void +trx_i_s_cache_free( +/*===============*/ + trx_i_s_cache_t* cache); /*!< in/out: cache to free */ /*******************************************************************//** Issue a shared/read lock on the tables cache. */ === modified file 'storage/xtradb/include/trx0purge.h' --- a/storage/xtradb/include/trx0purge.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/trx0purge.h 2010-01-06 12:00:14 +0000 @@ -71,6 +71,12 @@ void trx_purge_sys_create(void); /*======================*/ /********************************************************************//** +Frees the global purge system control structure. */ +UNIV_INTERN +void +trx_purge_sys_close(void); +/*======================*/ +/************************************************************************ Adds the update undo log as the first log in the history list. Removes the update undo log segment from the rseg slot if it is too big for reuse. */ UNIV_INTERN @@ -108,6 +114,25 @@ UNIV_INTERN ulint trx_purge(void); /*===========*/ +/********************************************************************** +This function runs a purge worker batch */ +UNIV_INTERN +void +trx_purge_worker( +/*=============*/ + ulint worker_id); +/********************************************************************** +This function waits the event for worker batch */ +UNIV_INTERN +void +trx_purge_worker_wait(void); +/*========================*/ +/********************************************************************** +This function wakes the waiting worker batch */ +UNIV_INTERN +void +trx_purge_worker_wake(void); +/*========================*/ /******************************************************************//** Prints information of the purge system to stderr. */ UNIV_INTERN @@ -125,6 +150,11 @@ struct trx_purge_struct{ of the trx system and it never ends */ que_t* query; /*!< The query graph which will do the parallelized purge operation */ + ulint n_worker; + os_event_t worker_event; + sess_t** sess_arr; + trx_t** trx_arr; + que_t** query_arr; rw_lock_t latch; /*!< The latch protecting the purge view. A purge operation must acquire an x-latch here for the instant at which === modified file 'storage/xtradb/include/trx0rec.h' --- a/storage/xtradb/include/trx0rec.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/trx0rec.h 2010-01-06 12:00:14 +0000 @@ -44,8 +44,8 @@ UNIV_INLINE trx_undo_rec_t* trx_undo_rec_copy( /*==============*/ - trx_undo_rec_t* undo_rec, /*!< in: undo log record */ - mem_heap_t* heap); /*!< in: heap where copied */ + const trx_undo_rec_t* undo_rec, /*!< in: undo log record */ + mem_heap_t* heap); /*!< in: heap where copied */ /**********************************************************************//** Reads the undo log record type. @return record type */ === modified file 'storage/xtradb/include/trx0rec.ic' --- a/storage/xtradb/include/trx0rec.ic 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/trx0rec.ic 2010-01-06 12:00:14 +0000 @@ -100,8 +100,8 @@ UNIV_INLINE trx_undo_rec_t* trx_undo_rec_copy( /*==============*/ - trx_undo_rec_t* undo_rec, /*!< in: undo log record */ - mem_heap_t* heap) /*!< in: heap where copied */ + const trx_undo_rec_t* undo_rec, /*!< in: undo log record */ + mem_heap_t* heap) /*!< in: heap where copied */ { ulint len; === modified file 'storage/xtradb/include/trx0roll.h' --- a/storage/xtradb/include/trx0roll.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/trx0roll.h 2010-01-06 12:00:14 +0000 @@ -133,6 +133,17 @@ trx_rollback( Rollback or clean up any incomplete transactions which were encountered in crash recovery. If the transaction already was committed, then we clean up a possible insert undo log. If the +transaction was not yet committed, then we roll it back. */ +UNIV_INTERN +void +trx_rollback_or_clean_recovered( +/*============================*/ + ibool all); /*!< in: FALSE=roll back dictionary transactions; + TRUE=roll back all non-PREPARED transactions */ +/*******************************************************************//** +Rollback or clean up any incomplete transactions which were +encountered in crash recovery. If the transaction already was +committed, then we clean up a possible insert undo log. If the transaction was not yet committed, then we roll it back. Note: this is done in a background thread. @return a dummy parameter */ @@ -208,9 +219,9 @@ int trx_general_rollback_for_mysql( /*===========================*/ trx_t* trx, /*!< in: transaction handle */ - ibool partial,/*!< in: TRUE if partial rollback requested */ trx_savept_t* savept);/*!< in: pointer to savepoint undo number, if - partial rollback requested */ + partial rollback requested, or NULL for + complete rollback */ /*******************************************************************//** Rolls back a transaction back to a named savepoint. Modifications after the savepoint are undone but InnoDB does NOT release the corresponding locks === modified file 'storage/xtradb/include/trx0rseg.h' --- a/storage/xtradb/include/trx0rseg.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/trx0rseg.h 2010-01-06 12:00:14 +0000 @@ -125,6 +125,13 @@ trx_rseg_create( ulint max_size, /*!< in: max size in pages */ ulint* id, /*!< out: rseg id */ mtr_t* mtr); /*!< in: mtr */ +/*************************************************************************** +Free's an instance of the rollback segment in memory. */ +UNIV_INTERN +void +trx_rseg_mem_free( +/*==============*/ + trx_rseg_t* rseg); /* in, own: instance to free */ /* Real max value may be 4076 in usual. But reserve 4 slot for safety or etc... */ === modified file 'storage/xtradb/include/trx0sys.h' --- a/storage/xtradb/include/trx0sys.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/trx0sys.h 2010-01-06 12:00:14 +0000 @@ -344,6 +344,12 @@ void trx_sys_file_format_tag_init(void); /*==============================*/ /*****************************************************************//** +Shutdown/Close the transaction system. */ +UNIV_INTERN +void +trx_sys_close(void); +/*===============*/ +/*****************************************************************//** Get the name representation of the file format from its id. @return pointer to the name */ UNIV_INTERN === modified file 'storage/xtradb/include/trx0sys.ic' --- a/storage/xtradb/include/trx0sys.ic 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/trx0sys.ic 2010-01-06 12:00:14 +0000 @@ -34,11 +34,11 @@ typedef byte trx_sysf_rseg_t; /* Rollback segment specification slot offsets */ /*-------------------------------------------------------------*/ -#define TRX_SYS_RSEG_SPACE 0 /* space where the the segment +#define TRX_SYS_RSEG_SPACE 0 /* space where the segment header is placed; starting with MySQL/InnoDB 5.1.7, this is UNIV_UNDEFINED if the slot is unused */ -#define TRX_SYS_RSEG_PAGE_NO 4 /* page number where the the segment +#define TRX_SYS_RSEG_PAGE_NO 4 /* page number where the segment header is placed; this is FIL_NULL if the slot is unused */ /*-------------------------------------------------------------*/ === modified file 'storage/xtradb/include/trx0trx.h' --- a/storage/xtradb/include/trx0trx.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/trx0trx.h 2010-01-06 12:00:14 +0000 @@ -179,7 +179,7 @@ trx_commit_off_kernel( /****************************************************************//** Cleans up a transaction at database startup. The cleanup is needed if the transaction already got to the middle of a commit when the database -crashed, andf we cannot roll it back. */ +crashed, and we cannot roll it back. */ UNIV_INTERN void trx_cleanup_at_db_startup( @@ -360,7 +360,7 @@ enum trx_dict_op { operation modes in crash recovery. */ TRX_DICT_OP_TABLE = 1, /** The transaction is creating or dropping an index in an - existing table. In crash recovery, the the data dictionary + existing table. In crash recovery, the data dictionary must be locked, but the table must not be dropped. */ TRX_DICT_OP_INDEX = 2 }; @@ -729,6 +729,17 @@ struct trx_struct{ /*------------------------------*/ char detailed_error[256]; /*!< detailed error message for last error, or empty. */ + /*------------------------------*/ + ulint io_reads; + ib_uint64_t io_read; + ulint io_reads_wait_timer; + ib_uint64_t lock_que_wait_ustarted; + ulint lock_que_wait_timer; + ulint innodb_que_wait_timer; + ulint distinct_page_access; +#define DPAH_SIZE 8192 + byte* distinct_page_access_hash; + ibool take_stats; }; #define TRX_MAX_N_THREADS 32 /* maximum number of === modified file 'storage/xtradb/include/trx0types.h' --- a/storage/xtradb/include/trx0types.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/trx0types.h 2010-01-06 12:00:14 +0000 @@ -70,7 +70,7 @@ typedef struct trx_named_savept_struct t enum trx_rb_ctx { RB_NONE = 0, /*!< no rollback */ RB_NORMAL, /*!< normal rollback */ - RB_RECOVERY, /*!< rolling back an incomplete transaction, + RB_RECOVERY /*!< rolling back an incomplete transaction, in crash recovery */ }; === modified file 'storage/xtradb/include/trx0undo.h' --- a/storage/xtradb/include/trx0undo.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/trx0undo.h 2010-01-06 12:00:14 +0000 @@ -333,6 +333,13 @@ trx_undo_parse_discard_latest( byte* end_ptr,/*!< in: buffer end */ page_t* page, /*!< in: page or NULL */ mtr_t* mtr); /*!< in: mtr or NULL */ +/************************************************************************ +Frees an undo log memory copy. */ +UNIV_INTERN +void +trx_undo_mem_free( +/*==============*/ + trx_undo_t* undo); /* in: the undo object to be freed */ /* Types of an undo log segment */ #define TRX_UNDO_INSERT 1 /* contains undo entries for inserts */ === modified file 'storage/xtradb/include/univ.i' --- a/storage/xtradb/include/univ.i 2009-11-13 21:26:08 +0000 +++ b/storage/xtradb/include/univ.i 2010-01-15 15:58:25 +0000 @@ -46,12 +46,12 @@ Created 1/20/1994 Heikki Tuuri #define INNODB_VERSION_MAJOR 1 #define INNODB_VERSION_MINOR 0 -#define INNODB_VERSION_BUGFIX 4 -#define PERCONA_INNODB_VERSION 8 +#define INNODB_VERSION_BUGFIX 6 +#define PERCONA_INNODB_VERSION 9 /* The following is the InnoDB version as shown in SELECT plugin_version FROM information_schema.plugins; -calculated in in make_version_string() in sql/sql_show.cc like this: +calculated in make_version_string() in sql/sql_show.cc like this: "version >> 8" . "version & 0xff" because the version is shown with only one dot, we skip the last component, i.e. we show M.N.P as M.N */ @@ -59,13 +59,14 @@ component, i.e. we show M.N.P as M.N */ (INNODB_VERSION_MAJOR << 8 | INNODB_VERSION_MINOR) /* auxiliary macros to help creating the version as string */ -#define __INNODB_VERSION(a, b, c, d) (#a "." #b "." #c "-" #d) -#define _INNODB_VERSION(a, b, c, d) __INNODB_VERSION(a, b, c, d) +#define __INNODB_VERSION(a, b, c, d) (#a "." #b "." #c "-" #d) +#define _INNODB_VERSION(a, b, c, d) __INNODB_VERSION(a, b, c, d) + #define INNODB_VERSION_STR \ _INNODB_VERSION(INNODB_VERSION_MAJOR, \ INNODB_VERSION_MINOR, \ - INNODB_VERSION_BUGFIX, \ + INNODB_VERSION_BUGFIX, \ PERCONA_INNODB_VERSION) #define REFMAN "http://dev.mysql.com/doc/refman/5.1/en/" @@ -80,17 +81,25 @@ the virtual method table (vtable) in GCC # define ha_innobase ha_innodb #endif /* MYSQL_DYNAMIC_PLUGIN */ +/* if any of the following macros is defined at this point this means +that the code from the "right" plug.in was executed and we do not +need to include ut0auxconf.h which would either define the same macros +or will be empty */ +#if !defined(HAVE_IB_GCC_ATOMIC_BUILTINS) \ + && !defined(HAVE_IB_ATOMIC_PTHREAD_T_GCC) \ + && !defined(HAVE_IB_SOLARIS_ATOMICS) \ + && !defined(HAVE_IB_ATOMIC_PTHREAD_T_SOLARIS) \ + && !defined(SIZEOF_PTHREAD_T) \ + && !defined(HAVE_IB_PAUSE_INSTRUCTION) +# include "ut0auxconf.h" +#endif + #if (defined(WIN32) || defined(_WIN32) || defined(WIN64) || defined(_WIN64)) && !defined(MYSQL_SERVER) && !defined(__WIN__) # undef __WIN__ # define __WIN__ # include <windows.h> -# if defined(HAVE_WINDOWS_ATOMICS) -/* If atomics are defined we use them in InnoDB mutex implementation */ -# define HAVE_ATOMIC_BUILTINS -# endif /* HAVE_WINDOWS_ATOMICS */ - # ifdef _NT_ # define __NT__ # endif @@ -113,45 +122,17 @@ if we are compiling on Windows. */ # include <sys/mman.h> /* mmap() for os0proc.c */ # endif -# undef PACKAGE -# undef VERSION - /* Include the header file generated by GNU autoconf */ # ifndef __WIN__ -#ifndef UNIV_HOTBACKUP -# include "config.h" -#endif /* UNIV_HOTBACKUP */ +# ifndef UNIV_HOTBACKUP +# include "config.h" +# endif /* UNIV_HOTBACKUP */ # endif # ifdef HAVE_SCHED_H # include <sched.h> # endif -# if defined(HAVE_GCC_ATOMIC_BUILTINS) || defined(HAVE_SOLARIS_ATOMICS) \ - || defined(HAVE_WINDOWS_ATOMICS) -/* If atomics are defined we use them in InnoDB mutex implementation */ -# define HAVE_ATOMIC_BUILTINS -# endif /* (HAVE_GCC_ATOMIC_BUILTINS) || (HAVE_SOLARIS_ATOMICS) - || (HAVE_WINDOWS_ATOMICS) */ - -/* For InnoDB rw_locks to work with atomics we need the thread_id -to be no more than machine word wide. The following enables using -atomics for InnoDB rw_locks where these conditions are met. */ -#ifdef HAVE_ATOMIC_BUILTINS -/* if HAVE_ATOMIC_PTHREAD_T is defined at this point that means that -the code from plug.in has defined it and we do not need to include -ut0auxconf.h which would either define HAVE_ATOMIC_PTHREAD_T or will -be empty */ -# ifndef HAVE_ATOMIC_PTHREAD_T -# include "ut0auxconf.h" -# endif /* HAVE_ATOMIC_PTHREAD_T */ -/* now HAVE_ATOMIC_PTHREAD_T is eventually defined either by plug.in or -from Makefile.in->ut0auxconf.h */ -# ifdef HAVE_ATOMIC_PTHREAD_T -# define INNODB_RW_LOCKS_USE_ATOMICS -# endif /* HAVE_ATOMIC_PTHREAD_T */ -#endif /* HAVE_ATOMIC_BUILTINS */ - /* We only try to do explicit inlining of functions with gcc and Sun Studio */ @@ -198,12 +179,18 @@ command. Not tested on Windows. */ debugging without UNIV_DEBUG */ #define UNIV_DEBUG /* Enable ut_ad() assertions and disable UNIV_INLINE */ +#define UNIV_DEBUG_LOCK_VALIDATE /* Enable + ut_ad(lock_rec_validate_page()) + assertions. */ #define UNIV_DEBUG_FILE_ACCESSES /* Debug .ibd file access (field file_page_was_freed in buf_page_t) */ #define UNIV_LRU_DEBUG /* debug the buffer pool LRU */ #define UNIV_HASH_DEBUG /* debug HASH_ macros */ #define UNIV_LIST_DEBUG /* debug UT_LIST_ macros */ +#define UNIV_LOG_LSN_DEBUG /* write LSN to the redo log; +this will break redo log file compatibility, but it may be useful when +debugging redo log application problems. */ #define UNIV_MEM_DEBUG /* detect memory leaks etc */ #define UNIV_IBUF_DEBUG /* debug the insert buffer */ #define UNIV_IBUF_COUNT_DEBUG /* debug the insert buffer; @@ -253,7 +240,7 @@ by one. */ /* Linkage specifier for non-static InnoDB symbols (variables and functions) that are only referenced from within InnoDB, not from MySQL */ -#if defined(__GNUC__) && (__GNUC__ >= 4) && !defined(UNIV_HOTBACKUP) +#if defined(__GNUC__) && (__GNUC__ >= 4) || defined(__INTEL_COMPILER) # define UNIV_INTERN __attribute__((visibility ("hidden"))) #else # define UNIV_INTERN @@ -410,7 +397,9 @@ it is read. */ /* Minimize cache-miss latency by moving data at addr into a cache before it is read or written. */ # define UNIV_PREFETCH_RW(addr) __builtin_prefetch(addr, 1, 3) -#elif defined(__SUNPRO_C) || defined(__SUNPRO_CC) +/* Sun Studio includes sun_prefetch.h as of version 5.9 */ +#elif (defined(__SUNPRO_C) && __SUNPRO_C >= 0x590) \ + || (defined(__SUNPRO_CC) && __SUNPRO_CC >= 0x590) # include <sun_prefetch.h> #if __SUNPRO_C >= 0x550 # undef UNIV_INTERN === modified file 'storage/xtradb/include/usr0sess.h' --- a/storage/xtradb/include/usr0sess.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/usr0sess.h 2010-01-06 12:00:14 +0000 @@ -44,14 +44,12 @@ sess_t* sess_open(void); /*============*/ /*********************************************************************//** -Closes a session, freeing the memory occupied by it, if it is in a state -where it should be closed. -@return TRUE if closed */ +Closes a session, freeing the memory occupied by it. */ UNIV_INTERN -ibool -sess_try_close( -/*===========*/ - sess_t* sess); /*!< in, own: session object */ +void +sess_close( +/*=======*/ + sess_t* sess); /* in, own: session object */ /* The session handle. All fields are protected by the kernel mutex */ struct sess_struct{ === modified file 'storage/xtradb/include/ut0auxconf.h' --- a/storage/xtradb/include/ut0auxconf.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/ut0auxconf.h 2010-01-06 12:00:14 +0000 @@ -1,14 +1,14 @@ /* Do not remove this file even though it is empty. This file is included in univ.i and will cause compilation failure if not present. -A custom check has been added in the generated +A custom checks have been added in the generated storage/innobase/Makefile.in that is shipped with the InnoDB Plugin -source archive. This check tries to compile a test program and if -successful then adds "#define HAVE_ATOMIC_PTHREAD_T" to this file. -This is a hack that has been developed in order to check for pthread_t -atomicity without the need to regenerate the ./configure script that is +source archive. These checks eventually define some macros and put +them in this file. +This is a hack that has been developed in order to deploy new compile +time checks without the need to regenerate the ./configure script that is distributed in the MySQL 5.1 official source archives. If by any chance Makefile.in and ./configure are regenerated and thus -the hack from Makefile.in wiped away then the "real" check from plug.in +the hack from Makefile.in wiped away then the "real" checks from plug.in will take over. */ === modified file 'storage/xtradb/include/ut0byte.h' --- a/storage/xtradb/include/ut0byte.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/ut0byte.h 2010-01-06 12:00:14 +0000 @@ -219,8 +219,8 @@ UNIV_INLINE void* ut_align( /*=====*/ - void* ptr, /*!< in: pointer */ - ulint align_no); /*!< in: align by this number */ + const void* ptr, /*!< in: pointer */ + ulint align_no); /*!< in: align by this number */ /*********************************************************//** The following function rounds down a pointer to the nearest aligned address. === modified file 'storage/xtradb/include/ut0byte.ic' --- a/storage/xtradb/include/ut0byte.ic 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/ut0byte.ic 2010-01-06 12:00:14 +0000 @@ -319,8 +319,8 @@ UNIV_INLINE void* ut_align( /*=====*/ - void* ptr, /*!< in: pointer */ - ulint align_no) /*!< in: align by this number */ + const void* ptr, /*!< in: pointer */ + ulint align_no) /*!< in: align by this number */ { ut_ad(align_no > 0); ut_ad(((align_no - 1) & align_no) == 0); === modified file 'storage/xtradb/include/ut0ut.h' --- a/storage/xtradb/include/ut0ut.h 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/include/ut0ut.h 2010-01-06 12:00:14 +0000 @@ -34,6 +34,11 @@ Created 1/20/1994 Heikki Tuuri #define ut0ut_h #include "univ.i" + +#ifndef UNIV_HOTBACKUP +# include "os0sync.h" /* for HAVE_ATOMIC_BUILTINS */ +#endif /* UNIV_HOTBACKUP */ + #include <time.h> #ifndef MYSQL_SERVER #include <ctype.h> @@ -47,7 +52,8 @@ Created 1/20/1994 Heikki Tuuri /** Time stamp */ typedef time_t ib_time_t; -#if defined(IB_HAVE_PAUSE_INSTRUCTION) +#ifndef UNIV_HOTBACKUP +#if defined(HAVE_IB_PAUSE_INSTRUCTION) # ifdef WIN32 /* In the Win32 API, the x86 PAUSE instruction is executed by calling the YieldProcessor macro defined in WinNT.h. It is a CPU architecture- @@ -84,6 +90,7 @@ do { \ os_thread_sleep(2000 /* 2 ms */); \ } \ } while (0) +#endif /* !UNIV_HOTBACKUP */ /********************************************************//** Gets the high 32 bits in a ulint. That is makes a shift >> 32, @@ -216,6 +223,7 @@ UNIV_INTERN ib_time_t ut_time(void); /*=========*/ +#ifndef UNIV_HOTBACKUP /**********************************************************//** Returns system time. Upon successful completion, the value 0 is returned; otherwise the @@ -239,6 +247,16 @@ ullint ut_time_us( /*=======*/ ullint* tloc); /*!< out: us since epoch, if non-NULL */ +/**********************************************************//** +Returns the number of milliseconds since some epoch. The +value may wrap around. It should only be used for heuristic +purposes. +@return ms since epoch */ +UNIV_INTERN +ulint +ut_time_ms(void); +/*============*/ +#endif /* !UNIV_HOTBACKUP */ /**********************************************************//** Returns the difference of two times in seconds. === modified file 'storage/xtradb/lock/lock0lock.c' --- a/storage/xtradb/lock/lock0lock.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/lock/lock0lock.c 2010-01-06 12:00:14 +0000 @@ -214,7 +214,7 @@ a waiting s-lock request on the next rec by a read cursor moving in the ascending order in the index, we cannot do the insert immediately, because when we finally commit our transaction, the read cursor should see also the new inserted record. So we should -move the read cursor backward from the the next record for it to pass over +move the read cursor backward from the next record for it to pass over the new inserted record. This move backward may be too cumbersome to implement. If we in this situation just enqueue a second x-lock request for our transaction on the next record, then the deadlock mechanism @@ -360,10 +360,9 @@ ibool lock_rec_validate_page( /*===================*/ ulint space, /*!< in: space id */ + ulint zip_size,/*!< in: compressed page size in bytes + or 0 for uncompressed pages */ ulint page_no);/*!< in: page number */ - -/* Define the following in order to enable lock_rec_validate_page() checks. */ -# undef UNIV_DEBUG_LOCK_VALIDATE #endif /* UNIV_DEBUG */ /* The lock system */ @@ -579,6 +578,23 @@ lock_sys_create( } /*********************************************************************//** +Closes the lock system at database shutdown. */ +UNIV_INTERN +void +lock_sys_close(void) +/*================*/ +{ + if (lock_latest_err_file != NULL) { + fclose(lock_latest_err_file); + lock_latest_err_file = NULL; + } + + hash_table_free(lock_sys->rec_hash); + mem_free(lock_sys); + lock_sys = NULL; +} + +/*********************************************************************//** Gets the size of a lock struct. @return size in bytes */ UNIV_INTERN @@ -1739,6 +1755,8 @@ lock_rec_enqueue_waiting( { lock_t* lock; trx_t* trx; + ulint sec; + ulint ms; ut_ad(mutex_own(&kernel_mutex)); @@ -1797,6 +1815,10 @@ lock_rec_enqueue_waiting( trx->que_state = TRX_QUE_LOCK_WAIT; trx->was_chosen_as_deadlock_victim = FALSE; trx->wait_started = time(NULL); + if (innobase_get_slow_log() && trx->take_stats) { + ut_usectime(&sec, &ms); + trx->lock_que_wait_ustarted = (ib_uint64_t)sec * 1000000 + ms; + } ut_a(que_thr_stop(thr)); @@ -2622,6 +2644,7 @@ lock_move_reorganize_page( #ifdef UNIV_DEBUG_LOCK_VALIDATE ut_ad(lock_rec_validate_page(buf_block_get_space(block), + buf_block_get_zip_size(block), buf_block_get_page_no(block))); #endif } @@ -2711,8 +2734,10 @@ lock_move_rec_list_end( #ifdef UNIV_DEBUG_LOCK_VALIDATE ut_ad(lock_rec_validate_page(buf_block_get_space(block), + buf_block_get_zip_size(block), buf_block_get_page_no(block))); ut_ad(lock_rec_validate_page(buf_block_get_space(new_block), + buf_block_get_zip_size(block), buf_block_get_page_no(new_block))); #endif } @@ -2822,6 +2847,7 @@ lock_move_rec_list_start( #ifdef UNIV_DEBUG_LOCK_VALIDATE ut_ad(lock_rec_validate_page(buf_block_get_space(block), + buf_block_get_zip_size(block), buf_block_get_page_no(block))); #endif } @@ -3574,7 +3600,8 @@ lock_table_remove_low( and lock_grant()). Therefore it can be empty and we need to check for that. */ - if (!ib_vector_is_empty(trx->autoinc_locks)) { + if (!lock_get_wait(lock) + && !ib_vector_is_empty(trx->autoinc_locks)) { lock_t* autoinc_lock; autoinc_lock = ib_vector_pop(trx->autoinc_locks); @@ -3607,6 +3634,8 @@ lock_table_enqueue_waiting( { lock_t* lock; trx_t* trx; + ulint sec; + ulint ms; ut_ad(mutex_own(&kernel_mutex)); @@ -3647,8 +3676,10 @@ lock_table_enqueue_waiting( if (lock_deadlock_occurs(lock, trx)) { - lock_reset_lock_and_trx_wait(lock); + /* The order here is important, we don't want to + lose the state of the lock before calling remove. */ lock_table_remove_low(lock); + lock_reset_lock_and_trx_wait(lock); return(DB_DEADLOCK); } @@ -3660,6 +3691,10 @@ lock_table_enqueue_waiting( return(DB_SUCCESS); } + if (innobase_get_slow_log() && trx->take_stats) { + ut_usectime(&sec, &ms); + trx->lock_que_wait_ustarted = (ib_uint64_t)sec * 1000000 + ms; + } trx->que_state = TRX_QUE_LOCK_WAIT; trx->was_chosen_as_deadlock_victim = FALSE; trx->wait_started = time(NULL); @@ -4627,6 +4662,10 @@ lock_rec_queue_validate( next function call: we have to release lock table mutex to obey the latching order */ + /* If this thread is holding the file space latch + (fil_space_t::latch), the following check WILL break + latching order and may cause a deadlock of threads. */ + impl_trx = lock_sec_rec_some_has_impl_off_kernel( rec, index, offsets); @@ -4684,6 +4723,8 @@ ibool lock_rec_validate_page( /*===================*/ ulint space, /*!< in: space id */ + ulint zip_size,/*!< in: compressed page size in bytes + or 0 for uncompressed pages */ ulint page_no)/*!< in: page number */ { dict_index_t* index; @@ -4694,7 +4735,6 @@ lock_rec_validate_page( ulint nth_lock = 0; ulint nth_bit = 0; ulint i; - ulint zip_size; mtr_t mtr; mem_heap_t* heap = NULL; ulint offsets_[REC_OFFS_NORMAL_SIZE]; @@ -4705,7 +4745,6 @@ lock_rec_validate_page( mtr_start(&mtr); - zip_size = fil_space_get_zip_size(space); ut_ad(zip_size != ULINT_UNDEFINED); block = buf_page_get(space, zip_size, page_no, RW_X_LATCH, &mtr); buf_block_dbg_add_level(block, SYNC_NO_ORDER_CHECK); @@ -4750,6 +4789,11 @@ loop: lock_mutex_exit_kernel(); + /* If this thread is holding the file space + latch (fil_space_t::latch), the following + check WILL break the latching order and may + cause a deadlock of threads. */ + lock_rec_queue_validate(block, rec, index, offsets); lock_mutex_enter_kernel(); @@ -4840,7 +4884,9 @@ lock_validate(void) lock_mutex_exit_kernel(); - lock_rec_validate_page(space, page_no); + lock_rec_validate_page(space, + fil_space_get_zip_size(space), + page_no); lock_mutex_enter_kernel(); @@ -5364,6 +5410,20 @@ lock_release_autoinc_last_lock( } /*******************************************************************//** +Check if a transaction holds any autoinc locks. +@return TRUE if the transaction holds any AUTOINC locks. */ +UNIV_INTERN +ibool +lock_trx_holds_autoinc_locks( +/*=========================*/ + const trx_t* trx) /*!< in: transaction */ +{ + ut_a(trx->autoinc_locks != NULL); + + return(!ib_vector_is_empty(trx->autoinc_locks)); +} + +/*******************************************************************//** Release all the transaction's autoinc locks. */ UNIV_INTERN void === modified file 'storage/xtradb/log/log0log.c' --- a/storage/xtradb/log/log0log.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/log/log0log.c 2010-01-06 12:00:14 +0000 @@ -241,6 +241,7 @@ log_reserve_and_open( ut_a(len < log->buf_size / 2); loop: mutex_enter(&(log->mutex)); + ut_ad(!recv_no_log_write); /* Calculate an upper limit for the space the string may take in the log buffer */ @@ -309,6 +310,7 @@ log_write_low( ut_ad(mutex_own(&(log->mutex))); part_loop: + ut_ad(!recv_no_log_write); /* Calculate a part length */ data_len = (log->buf_free % OS_FILE_LOG_BLOCK_SIZE) + str_len; @@ -362,6 +364,33 @@ part_loop: } /************************************************************//** +*/ +UNIV_INLINE +ulint +log_max_modified_age_async() +{ + if (srv_checkpoint_age_target) { + return(ut_min(log_sys->max_modified_age_async, + srv_checkpoint_age_target + - srv_checkpoint_age_target / 8)); + } else { + return(log_sys->max_modified_age_async); + } +} + +UNIV_INLINE +ulint +log_max_checkpoint_age_async() +{ + if (srv_checkpoint_age_target) { + return(ut_min(log_sys->max_checkpoint_age_async, + srv_checkpoint_age_target)); + } else { + return(log_sys->max_checkpoint_age_async); + } +} + +/************************************************************//** Closes the log. @return lsn */ UNIV_INTERN @@ -377,6 +406,7 @@ log_close(void) ib_uint64_t checkpoint_age; ut_ad(mutex_own(&(log->mutex))); + ut_ad(!recv_no_log_write); lsn = log->lsn; @@ -429,7 +459,7 @@ log_close(void) } } - if (checkpoint_age <= log->max_modified_age_async) { + if (checkpoint_age <= log_max_modified_age_async()) { goto function_exit; } @@ -437,8 +467,8 @@ log_close(void) oldest_lsn = buf_pool_get_oldest_modification(); if (!oldest_lsn - || lsn - oldest_lsn > log->max_modified_age_async - || checkpoint_age > log->max_checkpoint_age_async) { + || lsn - oldest_lsn > log_max_modified_age_async() + || checkpoint_age > log_max_checkpoint_age_async()) { log->check_flush_or_checkpoint = TRUE; } @@ -668,8 +698,6 @@ log_calc_max_ages(void) ulint archive_margin; ulint smallest_archive_margin; - ut_ad(!mutex_own(&(log_sys->mutex))); - mutex_enter(&(log_sys->mutex)); group = UT_LIST_GET_FIRST(log_sys->log_groups); @@ -770,8 +798,6 @@ void log_init(void) /*==========*/ { - byte* buf; - log_sys = mem_alloc(sizeof(log_t)); mutex_create(&log_sys->mutex, SYNC_LOG); @@ -786,8 +812,8 @@ log_init(void) ut_a(LOG_BUFFER_SIZE >= 16 * OS_FILE_LOG_BLOCK_SIZE); ut_a(LOG_BUFFER_SIZE >= 4 * UNIV_PAGE_SIZE); - buf = mem_alloc(LOG_BUFFER_SIZE + OS_FILE_LOG_BLOCK_SIZE); - log_sys->buf = ut_align(buf, OS_FILE_LOG_BLOCK_SIZE); + log_sys->buf_ptr = mem_alloc(LOG_BUFFER_SIZE + OS_FILE_LOG_BLOCK_SIZE); + log_sys->buf = ut_align(log_sys->buf_ptr, OS_FILE_LOG_BLOCK_SIZE); log_sys->buf_size = LOG_BUFFER_SIZE; @@ -832,9 +858,9 @@ log_init(void) rw_lock_create(&log_sys->checkpoint_lock, SYNC_NO_ORDER_CHECK); - log_sys->checkpoint_buf - = ut_align(mem_alloc(2 * OS_FILE_LOG_BLOCK_SIZE), - OS_FILE_LOG_BLOCK_SIZE); + log_sys->checkpoint_buf_ptr = mem_alloc(2 * OS_FILE_LOG_BLOCK_SIZE); + log_sys->checkpoint_buf = ut_align(log_sys->checkpoint_buf_ptr, + OS_FILE_LOG_BLOCK_SIZE); memset(log_sys->checkpoint_buf, '\0', OS_FILE_LOG_BLOCK_SIZE); /*----------------------------*/ @@ -917,23 +943,33 @@ log_group_init( group->lsn_offset = LOG_FILE_HDR_SIZE; group->n_pending_writes = 0; + group->file_header_bufs_ptr = mem_alloc(sizeof(byte*) * n_files); group->file_header_bufs = mem_alloc(sizeof(byte*) * n_files); #ifdef UNIV_LOG_ARCHIVE + group->archive_file_header_bufs_ptr = mem_alloc( + sizeof(byte*) * n_files); group->archive_file_header_bufs = mem_alloc(sizeof(byte*) * n_files); #endif /* UNIV_LOG_ARCHIVE */ for (i = 0; i < n_files; i++) { - *(group->file_header_bufs + i) = ut_align( - mem_alloc(LOG_FILE_HDR_SIZE + OS_FILE_LOG_BLOCK_SIZE), + group->file_header_bufs_ptr[i] = mem_alloc( + LOG_FILE_HDR_SIZE + OS_FILE_LOG_BLOCK_SIZE); + + group->file_header_bufs[i] = ut_align( + group->file_header_bufs_ptr[i], OS_FILE_LOG_BLOCK_SIZE); memset(*(group->file_header_bufs + i), '\0', LOG_FILE_HDR_SIZE); #ifdef UNIV_LOG_ARCHIVE - *(group->archive_file_header_bufs + i) = ut_align( - mem_alloc(LOG_FILE_HDR_SIZE + OS_FILE_LOG_BLOCK_SIZE), + group->archive_file_header_bufs_ptr[i] = mem_alloc( + LOG_FILE_HDR_SIZE + OS_FILE_LOG_BLOCK_SIZE); + + group->archive_file_header_bufs[i] = ut_align( + group->archive_file_header_bufs_ptr[i], OS_FILE_LOG_BLOCK_SIZE); + memset(*(group->archive_file_header_bufs + i), '\0', LOG_FILE_HDR_SIZE); #endif /* UNIV_LOG_ARCHIVE */ @@ -946,8 +982,9 @@ log_group_init( group->archived_offset = 0; #endif /* UNIV_LOG_ARCHIVE */ - group->checkpoint_buf = ut_align( - mem_alloc(2 * OS_FILE_LOG_BLOCK_SIZE), OS_FILE_LOG_BLOCK_SIZE); + group->checkpoint_buf_ptr = mem_alloc(2 * OS_FILE_LOG_BLOCK_SIZE); + group->checkpoint_buf = ut_align(group->checkpoint_buf_ptr, + OS_FILE_LOG_BLOCK_SIZE); memset(group->checkpoint_buf, '\0', OS_FILE_LOG_BLOCK_SIZE); @@ -1117,6 +1154,7 @@ log_io_complete( } mutex_enter(&(log_sys->mutex)); + ut_ad(!recv_no_log_write); ut_a(group->n_pending_writes > 0); ut_a(log_sys->n_pending_writes > 0); @@ -1148,6 +1186,7 @@ log_group_file_header_flush( ulint dest_offset; ut_ad(mutex_own(&(log_sys->mutex))); + ut_ad(!recv_no_log_write); ut_a(nth_file < group->n_files); buf = *(group->file_header_bufs + nth_file); @@ -1219,6 +1258,7 @@ log_group_write_buf( ulint i; ut_ad(mutex_own(&(log_sys->mutex))); + ut_ad(!recv_no_log_write); ut_a(len % OS_FILE_LOG_BLOCK_SIZE == 0); ut_a(((ulint) start_lsn) % OS_FILE_LOG_BLOCK_SIZE == 0); @@ -1361,6 +1401,7 @@ loop: #endif mutex_enter(&(log_sys->mutex)); + ut_ad(!recv_no_log_write); if (flush_to_disk && log_sys->flushed_to_disk_lsn >= lsn) { @@ -1974,6 +2015,7 @@ log_checkpoint( mutex_enter(&(log_sys->mutex)); + ut_ad(!recv_no_log_write); oldest_lsn = log_buf_pool_get_oldest_modification(); mutex_exit(&(log_sys->mutex)); @@ -2047,7 +2089,7 @@ log_make_checkpoint_at( later lsn, if IB_ULONGLONG_MAX, makes a checkpoint at the latest lsn */ ibool write_always) /*!< in: the function normally checks if - the the new checkpoint would have a + the new checkpoint would have a greater lsn than the previous one: if not, then no physical write is done; by setting this parameter TRUE, a @@ -2086,6 +2128,7 @@ loop: do_checkpoint = FALSE; mutex_enter(&(log->mutex)); + ut_ad(!recv_no_log_write); if (log->check_flush_or_checkpoint == FALSE) { mutex_exit(&(log->mutex)); @@ -2103,10 +2146,10 @@ loop: sync = TRUE; advance = 2 * (age - log->max_modified_age_sync); - } else if (age > log->max_modified_age_async) { + } else if (age > log_max_modified_age_async()) { /* A flush is not urgent: we do an asynchronous preflush */ - advance = age - log->max_modified_age_async; + advance = age - log_max_modified_age_async(); } else { advance = 0; } @@ -2120,7 +2163,7 @@ loop: do_checkpoint = TRUE; - } else if (checkpoint_age > log->max_checkpoint_age_async) { + } else if (checkpoint_age > log_max_checkpoint_age_async()) { /* A checkpoint is not urgent: do it asynchronously */ do_checkpoint = TRUE; @@ -3035,6 +3078,7 @@ loop: #endif /* UNIV_LOG_ARCHIVE */ mutex_enter(&(log_sys->mutex)); + ut_ad(!recv_no_log_write); if (log_sys->check_flush_or_checkpoint) { @@ -3120,6 +3164,16 @@ loop: goto loop; } + /* Check that the purge threads ended */ + if (srv_use_purge_thread + && (srv_n_threads_active[SRV_PURGE] != 0 + || srv_n_threads_active[SRV_PURGE_WORKER] != 0)) { + + mutex_exit(&kernel_mutex); + + goto loop; + } + mutex_exit(&kernel_mutex); mutex_enter(&(log_sys->mutex)); @@ -3234,6 +3288,7 @@ loop: ut_a(lsn == log_sys->lsn); } +#ifdef UNIV_LOG_DEBUG /******************************************************//** Checks by parsing that the catenated log segment for a single mtr is consistent. */ @@ -3241,7 +3296,7 @@ UNIV_INTERN ibool log_check_log_recs( /*===============*/ - byte* buf, /*!< in: pointer to the start of + const byte* buf, /*!< in: pointer to the start of the log segment in the log_sys->buf log buffer */ ulint len, /*!< in: segment length in bytes */ @@ -3249,8 +3304,8 @@ log_check_log_recs( { ib_uint64_t contiguous_lsn; ib_uint64_t scanned_lsn; - byte* start; - byte* end; + const byte* start; + const byte* end; byte* buf1; byte* scan_buf; @@ -3283,6 +3338,7 @@ log_check_log_recs( return(TRUE); } +#endif /* UNIV_LOG_DEBUG */ /******************************************************//** Peeks the current lsn. @@ -3326,10 +3382,12 @@ log_print( log_sys->last_checkpoint_lsn); fprintf(file, - "Max checkpoint age %lu\n" - "Modified age %lu\n" - "Checkpoint age %lu\n", + "Max checkpoint age %lu\n" + "Checkpoint age target %lu\n" + "Modified age %lu\n" + "Checkpoint age %lu\n", (ulong) log_sys->max_checkpoint_age, + (ulong) log_max_checkpoint_age_async(), (ulong) (log_sys->lsn - log_buf_pool_get_oldest_modification()), (ulong) (log_sys->lsn - log_sys->last_checkpoint_lsn)); @@ -3363,4 +3421,95 @@ log_refresh_stats(void) log_sys->n_log_ios_old = log_sys->n_log_ios; log_sys->last_printout_time = time(NULL); } + +/********************************************************************** +Closes a log group. */ +static +void +log_group_close( +/*===========*/ + log_group_t* group) /* in,own: log group to close */ +{ + ulint i; + + for (i = 0; i < group->n_files; i++) { + mem_free(group->file_header_bufs_ptr[i]); +#ifdef UNIV_LOG_ARCHIVE + mem_free(group->archive_file_header_bufs_ptr[i]); +#endif /* UNIV_LOG_ARCHIVE */ + } + + mem_free(group->file_header_bufs_ptr); + mem_free(group->file_header_bufs); + +#ifdef UNIV_LOG_ARCHIVE + mem_free(group->archive_file_header_bufs_ptr); + mem_free(group->archive_file_header_bufs); +#endif /* UNIV_LOG_ARCHIVE */ + + mem_free(group->checkpoint_buf_ptr); + + mem_free(group); +} + +/********************************************************** +Shutdown the log system but do not release all the memory. */ +UNIV_INTERN +void +log_shutdown(void) +/*==============*/ +{ + log_group_t* group; + + group = UT_LIST_GET_FIRST(log_sys->log_groups); + + while (UT_LIST_GET_LEN(log_sys->log_groups) > 0) { + log_group_t* prev_group = group; + + group = UT_LIST_GET_NEXT(log_groups, group); + UT_LIST_REMOVE(log_groups, log_sys->log_groups, prev_group); + + log_group_close(prev_group); + } + + mem_free(log_sys->buf_ptr); + log_sys->buf_ptr = NULL; + log_sys->buf = NULL; + mem_free(log_sys->checkpoint_buf_ptr); + log_sys->checkpoint_buf_ptr = NULL; + log_sys->checkpoint_buf = NULL; + + os_event_free(log_sys->no_flush_event); + os_event_free(log_sys->one_flushed_event); + + rw_lock_free(&log_sys->checkpoint_lock); + + mutex_free(&log_sys->mutex); + +#ifdef UNIV_LOG_ARCHIVE + rw_lock_free(&log_sys->archive_lock); + os_event_create(log_sys->archiving_on); +#endif /* UNIV_LOG_ARCHIVE */ + +#ifdef UNIV_LOG_DEBUG + recv_sys_debug_free(); +#endif + + recv_sys_close(); +} + +/********************************************************** +Free the log system data structures. */ +UNIV_INTERN +void +log_mem_free(void) +/*==============*/ +{ + if (log_sys != NULL) { + recv_sys_mem_free(); + mem_free(log_sys); + + log_sys = NULL; + } +} #endif /* !UNIV_HOTBACKUP */ === modified file 'storage/xtradb/log/log0recv.c' --- a/storage/xtradb/log/log0recv.c 2009-11-13 21:26:08 +0000 +++ b/storage/xtradb/log/log0recv.c 2010-01-15 15:58:25 +0000 @@ -69,20 +69,25 @@ UNIV_INTERN recv_sys_t* recv_sys = NULL; /** TRUE when applying redo log records during crash recovery; FALSE otherwise. Note that this is FALSE while a background thread is rolling back incomplete transactions. */ -UNIV_INTERN ibool recv_recovery_on = FALSE; +UNIV_INTERN ibool recv_recovery_on; #ifdef UNIV_LOG_ARCHIVE /** TRUE when applying redo log records from an archived log file */ -UNIV_INTERN ibool recv_recovery_from_backup_on = FALSE; +UNIV_INTERN ibool recv_recovery_from_backup_on; #endif /* UNIV_LOG_ARCHIVE */ #ifndef UNIV_HOTBACKUP /** TRUE when recv_init_crash_recovery() has been called. */ -UNIV_INTERN ibool recv_needed_recovery = FALSE; +UNIV_INTERN ibool recv_needed_recovery; +# ifdef UNIV_DEBUG +/** TRUE if writing to the redo log (mtr_commit) is forbidden. +Protected by log_sys->mutex. */ +UNIV_INTERN ibool recv_no_log_write = FALSE; +# endif /* UNIV_DEBUG */ /** TRUE if buf_page_is_corrupted() should check if the log sequence number (FIL_PAGE_LSN) is in the future. Initially FALSE, and set by recv_recovery_from_checkpoint_start_func(). */ -UNIV_INTERN ibool recv_lsn_checks_on = FALSE; +UNIV_INTERN ibool recv_lsn_checks_on; /** There are two conditions under which we scan the logs, the first is normal startup and the second is when we do a recovery from an @@ -92,7 +97,7 @@ startup. If we find log entries that wer we know that the server was not cleanly shutdown. We must then initialize the crash recovery environment before attempting to store these entries in the log hash table. */ -static ibool recv_log_scan_is_startup_type = FALSE; +static ibool recv_log_scan_is_startup_type; /** If the following is TRUE, the buffer pool file pages must be invalidated after recovery and no ibuf operations are allowed; this becomes TRUE if @@ -103,7 +108,7 @@ buffer pool before the pages have been r TRUE means that recovery is running and no operations on the log files are allowed yet: the variable name is misleading. */ -UNIV_INTERN ibool recv_no_ibuf_operations = FALSE; +UNIV_INTERN ibool recv_no_ibuf_operations; /** TRUE when the redo log is being backed up */ # define recv_is_making_a_backup FALSE /** TRUE when recovering from a backed up redo log file */ @@ -111,30 +116,30 @@ UNIV_INTERN ibool recv_no_ibuf_operation #else /* !UNIV_HOTBACKUP */ # define recv_needed_recovery FALSE /** TRUE when the redo log is being backed up */ -UNIV_INTERN ibool recv_is_making_a_backup = FALSE; +UNIV_INTERN ibool recv_is_making_a_backup = FALSE; /** TRUE when recovering from a backed up redo log file */ UNIV_INTERN ibool recv_is_from_backup = FALSE; # define buf_pool_get_curr_size() (5 * 1024 * 1024) #endif /* !UNIV_HOTBACKUP */ /** The following counter is used to decide when to print info on log scan */ -static ulint recv_scan_print_counter = 0; +static ulint recv_scan_print_counter; /** The type of the previous parsed redo log record */ -static ulint recv_previous_parsed_rec_type = 999999; +static ulint recv_previous_parsed_rec_type; /** The offset of the previous parsed redo log record */ -static ulint recv_previous_parsed_rec_offset = 0; +static ulint recv_previous_parsed_rec_offset; /** The 'multi' flag of the previous parsed redo log record */ -static ulint recv_previous_parsed_rec_is_multi = 0; +static ulint recv_previous_parsed_rec_is_multi; /** Maximum page number encountered in the redo log */ -UNIV_INTERN ulint recv_max_parsed_page_no = 0; +UNIV_INTERN ulint recv_max_parsed_page_no; /** This many frames must be left free in the buffer pool when we scan the log and store the scanned log records in the buffer pool: we will use these free frames to read in pages when we start applying the log records to the database. */ -UNIV_INTERN ulint recv_n_pool_free_frames = 1024; +UNIV_INTERN ulint recv_n_pool_free_frames; /** The maximum lsn we see for a page during the recovery process. If this is bigger than the lsn we are able to scan up to, that is an indication that @@ -165,15 +170,119 @@ recv_sys_create(void) return; } - recv_sys = mem_alloc(sizeof(recv_sys_t)); + recv_sys = mem_alloc(sizeof(*recv_sys)); + memset(recv_sys, 0x0, sizeof(*recv_sys)); mutex_create(&recv_sys->mutex, SYNC_RECV); recv_sys->heap = NULL; recv_sys->addr_hash = NULL; + + recv_sys->stats_recv_start_time = time(NULL); + recv_sys->stats_oldest_modified_lsn = IB_ULONGLONG_MAX; } /********************************************************//** +Release recovery system mutexes. */ +UNIV_INTERN +void +recv_sys_close(void) +/*================*/ +{ + if (recv_sys != NULL) { + if (recv_sys->addr_hash != NULL) { + hash_table_free(recv_sys->addr_hash); + } + + if (recv_sys->heap != NULL) { + mem_heap_free(recv_sys->heap); + } + + if (recv_sys->buf != NULL) { + ut_free(recv_sys->buf); + } + + if (recv_sys->last_block_buf_start != NULL) { + mem_free(recv_sys->last_block_buf_start); + } + + mutex_free(&recv_sys->mutex); + + mem_free(recv_sys); + recv_sys = NULL; + } +} + +/********************************************************//** +Frees the recovery system memory. */ +UNIV_INTERN +void +recv_sys_mem_free(void) +/*===================*/ +{ + if (recv_sys != NULL) { + if (recv_sys->addr_hash != NULL) { + hash_table_free(recv_sys->addr_hash); + } + + if (recv_sys->heap != NULL) { + mem_heap_free(recv_sys->heap); + } + + if (recv_sys->buf != NULL) { + ut_free(recv_sys->buf); + } + + if (recv_sys->last_block_buf_start != NULL) { + mem_free(recv_sys->last_block_buf_start); + } + + mem_free(recv_sys); + recv_sys = NULL; + } +} + +/************************************************************ +Reset the state of the recovery system variables. */ +UNIV_INTERN +void +recv_sys_var_init(void) +/*===================*/ +{ + recv_lsn_checks_on = FALSE; + + recv_n_pool_free_frames = 1024; + + recv_recovery_on = FALSE; + +#ifdef UNIV_LOG_ARCHIVE + recv_recovery_from_backup_on = FALSE; +#endif /* UNIV_LOG_ARCHIVE */ + + recv_needed_recovery = FALSE; + + recv_lsn_checks_on = FALSE; + + recv_log_scan_is_startup_type = FALSE; + + recv_no_ibuf_operations = FALSE; + + recv_scan_print_counter = 0; + + recv_previous_parsed_rec_type = 999999; + + recv_previous_parsed_rec_offset = 0; + + recv_previous_parsed_rec_is_multi = 0; + + recv_max_parsed_page_no = 0; + + recv_n_pool_free_frames = 1024; + + recv_max_page_lsn = 0; +} + +/************************************************************ Inits the recovery system for a recovery operation. */ UNIV_INTERN void @@ -248,8 +357,8 @@ recv_sys_empty_hash(void) Frees the recovery system. */ static void -recv_sys_free(void) -/*===============*/ +recv_sys_debug_free(void) +/*=====================*/ { mutex_enter(&(recv_sys->mutex)); @@ -258,8 +367,10 @@ recv_sys_free(void) ut_free(recv_sys->buf); mem_free(recv_sys->last_block_buf_start); - recv_sys->addr_hash = NULL; + recv_sys->buf = NULL; recv_sys->heap = NULL; + recv_sys->addr_hash = NULL; + recv_sys->last_block_buf_start = NULL; mutex_exit(&(recv_sys->mutex)); } @@ -853,6 +964,11 @@ recv_parse_or_apply_log_rec_body( } switch (type) { +#ifdef UNIV_LOG_LSN_DEBUG + case MLOG_LSN: + /* The LSN is checked in recv_parse_log_rec(). */ + break; +#endif /* UNIV_LOG_LSN_DEBUG */ case MLOG_1BYTE: case MLOG_2BYTES: case MLOG_4BYTES: case MLOG_8BYTES: #ifdef UNIV_DEBUG if (page && page_type == FIL_PAGE_TYPE_ALLOCATED @@ -1223,6 +1339,11 @@ recv_add_to_hash_table( len = rec_end - body; + if (srv_recovery_stats) { + recv_sys->stats_log_recs++; + recv_sys->stats_log_len_sum += len; + } + recv = mem_heap_alloc(recv_sys->heap, sizeof(recv_t)); recv->type = type; recv->len = rec_end - body; @@ -1269,7 +1390,7 @@ recv_add_to_hash_table( sizeof(recv_data_t) + len); *prev_field = recv_data; - ut_memcpy(((byte*)recv_data) + sizeof(recv_data_t), body, len); + memcpy(recv_data + 1, body, len); prev_field = &(recv_data->next); @@ -1327,12 +1448,14 @@ recv_recover_page_func( buf_block_t* block) /*!< in/out: buffer block */ { page_t* page; + page_zip_des_t* page_zip; recv_addr_t* recv_addr; recv_t* recv; byte* buf; ib_uint64_t start_lsn; ib_uint64_t end_lsn; ib_uint64_t page_lsn; + ib_uint64_t page_lsn_orig; ib_uint64_t page_newest_lsn; ibool modification_to_page; #ifndef UNIV_HOTBACKUP @@ -1372,12 +1495,21 @@ recv_recover_page_func( recv_addr->state = RECV_BEING_PROCESSED; + if (srv_recovery_stats) { + if (just_read_in) { + recv_sys->stats_recover_pages_with_read++; + } else { + recv_sys->stats_recover_pages_without_read++; + } + } + mutex_exit(&(recv_sys->mutex)); mtr_start(&mtr); mtr_set_log_mode(&mtr, MTR_LOG_NONE); page = block->frame; + page_zip = buf_block_get_page_zip(block); #ifndef UNIV_HOTBACKUP if (just_read_in) { @@ -1400,6 +1532,7 @@ recv_recover_page_func( /* Read the newest modification lsn from the page */ page_lsn = mach_read_ull(page + FIL_PAGE_LSN); + page_lsn_orig = page_lsn; #ifndef UNIV_HOTBACKUP /* It may be that the page has been modified in the buffer @@ -1419,6 +1552,21 @@ recv_recover_page_func( modification_to_page = FALSE; start_lsn = end_lsn = 0; + if (srv_recovery_stats) { + mutex_enter(&(recv_sys->mutex)); + if (page_lsn_orig && recv_sys->stats_oldest_modified_lsn > page_lsn_orig) { + recv_sys->stats_oldest_modified_lsn = page_lsn_orig; + } + if (page_lsn_orig && recv_sys->stats_newest_modified_lsn < page_lsn_orig) { + recv_sys->stats_newest_modified_lsn = page_lsn_orig; + } + if (UT_LIST_GET_LAST(recv_addr->rec_list)->start_lsn + < page_lsn_orig) { + recv_sys->stats_pages_already_new++; + } + mutex_exit(&(recv_sys->mutex)); + } + recv = UT_LIST_GET_FIRST(recv_addr->rec_list); while (recv) { @@ -1438,13 +1586,19 @@ recv_recover_page_func( if (recv->type == MLOG_INIT_FILE_PAGE) { page_lsn = page_newest_lsn; - mach_write_ull(page + UNIV_PAGE_SIZE - - FIL_PAGE_END_LSN_OLD_CHKSUM, 0); - mach_write_ull(page + FIL_PAGE_LSN, 0); + memset(FIL_PAGE_LSN + page, 0, 8); + memset(UNIV_PAGE_SIZE - FIL_PAGE_END_LSN_OLD_CHKSUM + + page, 0, 8); + + if (page_zip) { + memset(FIL_PAGE_LSN + page_zip->data, 0, 8); + } } if (recv->start_lsn >= page_lsn) { + ib_uint64_t end_lsn; + if (!modification_to_page) { modification_to_page = TRUE; @@ -1466,11 +1620,24 @@ recv_recover_page_func( recv_parse_or_apply_log_rec_body(recv->type, buf, buf + recv->len, block, &mtr); - mach_write_ull(page + UNIV_PAGE_SIZE - - FIL_PAGE_END_LSN_OLD_CHKSUM, - recv->start_lsn + recv->len); - mach_write_ull(page + FIL_PAGE_LSN, - recv->start_lsn + recv->len); + + if (srv_recovery_stats) { + mutex_enter(&(recv_sys->mutex)); + recv_sys->stats_applied_log_recs++; + recv_sys->stats_applied_log_len_sum += recv->len; + mutex_exit(&(recv_sys->mutex)); + } + + end_lsn = recv->start_lsn + recv->len; + mach_write_ull(FIL_PAGE_LSN + page, end_lsn); + mach_write_ull(UNIV_PAGE_SIZE + - FIL_PAGE_END_LSN_OLD_CHKSUM + + page, end_lsn); + + if (page_zip) { + mach_write_ull(FIL_PAGE_LSN + + page_zip->data, end_lsn); + } } if (recv->len > RECV_DATA_BLOCK_SIZE) { @@ -1561,6 +1728,13 @@ recv_read_in_area( } } + if (srv_recovery_stats && n) { + mutex_enter(&(recv_sys->mutex)); + recv_sys->stats_read_requested_pages += n; + recv_sys->stats_read_in_area[n - 1]++; + mutex_exit(&(recv_sys->mutex)); + } + buf_read_recv_pages(FALSE, space, zip_size, page_nos, n); /* fprintf(stderr, "Recv pages at %lu n %lu\n", page_nos[0], n); @@ -1688,6 +1862,7 @@ loop: /* Flush all the file pages to disk and invalidate them in the buffer pool */ + ut_d(recv_no_log_write = TRUE); mutex_exit(&(recv_sys->mutex)); mutex_exit(&(log_sys->mutex)); @@ -1701,6 +1876,7 @@ loop: mutex_enter(&(log_sys->mutex)); mutex_enter(&(recv_sys->mutex)); + ut_d(recv_no_log_write = FALSE); recv_no_ibuf_operations = FALSE; } @@ -1712,6 +1888,10 @@ loop: if (has_printed) { fprintf(stderr, "InnoDB: Apply batch completed\n"); + + if (srv_recovery_stats) { + recv_sys->stats_recv_turns++; + } } mutex_exit(&(recv_sys->mutex)); @@ -1912,6 +2092,17 @@ recv_parse_log_rec( return(0); } +#ifdef UNIV_LOG_LSN_DEBUG + if (*type == MLOG_LSN) { + ib_uint64_t lsn = (ib_uint64_t) *space << 32 | *page_no; +# ifdef UNIV_LOG_DEBUG + ut_a(lsn == log_sys->old_lsn); +# else /* UNIV_LOG_DEBUG */ + ut_a(lsn == recv_sys->recovered_lsn); +# endif /* UNIV_LOG_DEBUG */ + } +#endif /* UNIV_LOG_LSN_DEBUG */ + /* Check that page_no is sensible */ if (UNIV_UNLIKELY(*page_no > 0x8FFFFFFFUL)) { @@ -2169,6 +2360,12 @@ loop: #endif /* In normal mysqld crash recovery we do not try to replay file operations */ +#ifdef UNIV_LOG_LSN_DEBUG + } else if (type == MLOG_LSN) { + /* Do not add these records to the hash table. + The page number and space id fields are misused + for something else. */ +#endif /* UNIV_LOG_LSN_DEBUG */ } else { recv_add_to_hash_table(type, space, page_no, body, ptr + len, old_lsn, @@ -2200,11 +2397,11 @@ loop: = recv_sys->recovered_offset + total_len; recv_previous_parsed_rec_is_multi = 1; - if ((!store_to_hash) && (type != MLOG_MULTI_REC_END)) { #ifdef UNIV_LOG_DEBUG + if ((!store_to_hash) && (type != MLOG_MULTI_REC_END)) { recv_check_incomplete_log_recs(ptr, len); -#endif /* UNIV_LOG_DEBUG */ } +#endif /* UNIV_LOG_DEBUG */ #ifdef UNIV_DEBUG if (log_debug_writes) { @@ -2268,7 +2465,11 @@ loop: break; } - if (store_to_hash) { + if (store_to_hash +#ifdef UNIV_LOG_LSN_DEBUG + && type != MLOG_LSN +#endif /* UNIV_LOG_LSN_DEBUG */ + ) { recv_add_to_hash_table(type, space, page_no, body, ptr + len, old_lsn, @@ -2417,8 +2618,7 @@ recv_scan_log_recs( scanned_lsn = start_lsn; more_data = FALSE; - while (log_block < buf + len && !finished) { - + do { no = log_block_get_hdr_no(log_block); /* fprintf(stderr, "Log block header no %lu\n", no); @@ -2548,10 +2748,11 @@ recv_scan_log_recs( /* Log data for this group ends here */ finished = TRUE; + break; } else { log_block += OS_FILE_LOG_BLOCK_SIZE; } - } + } while (log_block < buf + len && !finished); *group_scanned_lsn = scanned_lsn; @@ -3078,6 +3279,84 @@ recv_recovery_from_checkpoint_finish(voi } #endif /* UNIV_DEBUG */ + if (recv_needed_recovery && srv_recovery_stats) { + FILE* file = stderr; + ulint i; + + fprintf(stderr, + "InnoDB: Applying log records was done. Its statistics are followings.\n"); + + fprintf(stderr, + "============================================================\n" + "-------------------\n" + "RECOVERY STATISTICS\n" + "-------------------\n"); + fprintf(stderr, + "Recovery time: %g sec. (%lu turns)\n", + difftime(time(NULL), recv_sys->stats_recv_start_time), + recv_sys->stats_recv_turns); + + fprintf(stderr, + "\n" + "Data page IO statistics\n" + " Requested pages: %lu\n" + " Read pages: %lu\n" + " Written pages: %lu\n" + " (Dirty blocks): %lu\n", + recv_sys->stats_read_requested_pages, + recv_sys->stats_read_io_pages, + recv_sys->stats_write_io_pages, + UT_LIST_GET_LEN(buf_pool->flush_list)); + + fprintf(stderr, + " Grouping IO [times]:\n" + "\tnumber of pages,\n" + "\t\tread request neighbors (in %d pages chunk),\n" + "\t\t\tcombined read IO,\n" + "\t\t\t\tcombined write IO\n", + RECV_READ_AHEAD_AREA); + for (i = 0; i < ut_max(RECV_READ_AHEAD_AREA, + OS_AIO_MERGE_N_CONSECUTIVE); i++) { + fprintf(stderr, + "\t%3lu,\t%lu,\t%lu,\t%lu\n", i + 1, + (i < RECV_READ_AHEAD_AREA) ? + recv_sys->stats_read_in_area[i] : 0, + (i < OS_AIO_MERGE_N_CONSECUTIVE) ? + recv_sys->stats_read_io_consecutive[i] : 0, + (i < OS_AIO_MERGE_N_CONSECUTIVE) ? + recv_sys->stats_write_io_consecutive[i] : 0); + } + + fprintf(stderr, + "\n" + "Recovery process statistics\n" + " Checked pages by doublewrite buffer: %lu\n" + " Overwritten pages from doublewrite: %lu\n" + " Recovered pages by io_thread: %lu\n" + " Recovered pages by main thread: %lu\n" + " Parsed log records to apply: %lu\n" + " Sum of the length: %lu\n" + " Applied log records: %lu\n" + " Sum of the length: %lu\n" + " Pages which are already new enough: %lu (It may not be accurate, if turns > 1)\n" + " Oldest page's LSN: %llu\n" + " Newest page's LSN: %llu\n", + recv_sys->stats_doublewrite_check_pages, + recv_sys->stats_doublewrite_overwrite_pages, + recv_sys->stats_recover_pages_with_read, + recv_sys->stats_recover_pages_without_read, + recv_sys->stats_log_recs, + recv_sys->stats_log_len_sum, + recv_sys->stats_applied_log_recs, + recv_sys->stats_applied_log_len_sum, + recv_sys->stats_pages_already_new, + recv_sys->stats_oldest_modified_lsn, + recv_sys->stats_newest_modified_lsn); + + fprintf(stderr, + "============================================================\n"); + } + if (recv_needed_recovery) { trx_sys_print_mysql_master_log_pos(); trx_sys_print_mysql_binlog_offset(); @@ -3104,8 +3383,13 @@ recv_recovery_from_checkpoint_finish(voi recv_recovery_on = FALSE; #ifndef UNIV_LOG_DEBUG - recv_sys_free(); + recv_sys_debug_free(); #endif + /* Roll back any recovered data dictionary transactions, so + that the data dictionary tables will be free of any locks. + The data dictionary latch should guarantee that there is at + most one data dictionary transaction active at a time. */ + trx_rollback_or_clean_recovered(FALSE); /* Drop partially created indexes. */ row_merge_drop_temp_indexes(); === modified file 'storage/xtradb/mem/mem0dbg.c' --- a/storage/xtradb/mem/mem0dbg.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/mem/mem0dbg.c 2010-01-06 12:00:14 +0000 @@ -170,6 +170,17 @@ mem_init( mem_comm_pool = mem_pool_create(size); } + +/******************************************************************//** +Closes the memory system. */ +UNIV_INTERN +void +mem_close(void) +/*===========*/ +{ + mem_pool_free(mem_comm_pool); + mem_comm_pool = NULL; +} #endif /* !UNIV_HOTBACKUP */ #ifdef UNIV_MEM_DEBUG === modified file 'storage/xtradb/mem/mem0mem.c' --- a/storage/xtradb/mem/mem0mem.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/mem/mem0mem.c 2010-01-06 12:00:14 +0000 @@ -475,16 +475,18 @@ mem_heap_block_free( len = block->len; block->magic_n = MEM_FREED_BLOCK_MAGIC_N; +#ifndef UNIV_HOTBACKUP + if (!srv_use_sys_malloc) { #ifdef UNIV_MEM_DEBUG - /* In the debug version we set the memory to a random combination - of hex 0xDE and 0xAD. */ + /* In the debug version we set the memory to a random + combination of hex 0xDE and 0xAD. */ - mem_erase_buf((byte*)block, len); + mem_erase_buf((byte*)block, len); #else /* UNIV_MEM_DEBUG */ - UNIV_MEM_ASSERT_AND_FREE(block, len); + UNIV_MEM_ASSERT_AND_FREE(block, len); #endif /* UNIV_MEM_DEBUG */ -#ifndef UNIV_HOTBACKUP + } if (type == MEM_HEAP_DYNAMIC || len < UNIV_PAGE_SIZE / 2) { ut_ad(!buf_block); @@ -495,6 +497,14 @@ mem_heap_block_free( buf_block_free(buf_block); } #else /* !UNIV_HOTBACKUP */ +#ifdef UNIV_MEM_DEBUG + /* In the debug version we set the memory to a random + combination of hex 0xDE and 0xAD. */ + + mem_erase_buf((byte*)block, len); +#else /* UNIV_MEM_DEBUG */ + UNIV_MEM_ASSERT_AND_FREE(block, len); +#endif /* UNIV_MEM_DEBUG */ ut_free(block); #endif /* !UNIV_HOTBACKUP */ } === modified file 'storage/xtradb/mem/mem0pool.c' --- a/storage/xtradb/mem/mem0pool.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/mem/mem0pool.c 2010-01-06 12:00:14 +0000 @@ -261,6 +261,18 @@ mem_pool_create( } /********************************************************************//** +Frees a memory pool. */ +UNIV_INTERN +void +mem_pool_free( +/*==========*/ + mem_pool_t* pool) /*!< in, own: memory pool */ +{ + ut_free(pool->buf); + ut_free(pool); +} + +/********************************************************************//** Fills the specified free list. @return TRUE if we were able to insert a block to the free list */ static === modified file 'storage/xtradb/mtr/mtr0mtr.c' --- a/storage/xtradb/mtr/mtr0mtr.c 2009-11-13 21:26:08 +0000 +++ b/storage/xtradb/mtr/mtr0mtr.c 2010-01-15 15:58:25 +0000 @@ -36,6 +36,7 @@ Created 11/26/1995 Heikki Tuuri #include "buf0flu.h" #ifndef UNIV_HOTBACKUP +# include "log0recv.h" /*****************************************************************//** Releases the item in the slot given. */ UNIV_INLINE @@ -148,7 +149,6 @@ mtr_log_reserve_and_write( dyn_array_t* mlog; dyn_block_t* block; ulint data_size; - ibool success; byte* first_data; ut_ad(mtr); @@ -167,8 +167,8 @@ mtr_log_reserve_and_write( if (mlog->heap == NULL) { mtr->end_lsn = log_reserve_and_write_fast( first_data, dyn_block_get_used(mlog), - &(mtr->start_lsn), &success); - if (success) { + &mtr->start_lsn); + if (mtr->end_lsn) { return; } @@ -215,6 +215,8 @@ mtr_commit( ut_d(mtr->state = MTR_COMMITTING); #ifndef UNIV_HOTBACKUP + /* This is a dirty read, for debugging. */ + ut_ad(!recv_no_log_write); write_log = mtr->modifications && mtr->n_log_recs; if (write_log) { === modified file 'storage/xtradb/os/os0file.c' --- a/storage/xtradb/os/os0file.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/os/os0file.c 2010-01-06 12:00:14 +0000 @@ -55,6 +55,9 @@ Created 10/21/1995 Heikki Tuuri #include "srv0start.h" #include "fil0fil.h" #include "buf0buf.h" +#include "trx0sys.h" +#include "trx0trx.h" +#include "log0recv.h" #ifndef UNIV_HOTBACKUP # include "os0sync.h" # include "os0thread.h" @@ -88,7 +91,9 @@ UNIV_INTERN ibool os_do_not_call_flush_a /* We do not call os_file_flush in every os_file_write. */ #endif /* UNIV_DO_FLUSH */ -#ifndef UNIV_HOTBACKUP +#ifdef UNIV_HOTBACKUP +# define os_aio_use_native_aio FALSE +#else /* UNIV_HOTBACKUP */ /* We use these mutexes to protect lseek + file i/o operation, if the OS does not provide an atomic pread or pwrite, or similar */ #define OS_FILE_N_SEEK_MUTEXES 16 @@ -235,7 +240,7 @@ static ulint os_aio_n_segments = ULINT_U /** If the following is TRUE, read i/o handler threads try to wait until a batch of new read requests have been posted */ static volatile ibool os_aio_recommend_sleep_for_read_threads = FALSE; -#endif /* !UNIV_HOTBACKUP */ +#endif /* UNIV_HOTBACKUP */ UNIV_INTERN ulint os_n_file_reads = 0; UNIV_INTERN ulint os_bytes_read_since_printout = 0; @@ -352,6 +357,19 @@ os_file_get_last_error( " software or another instance\n" "InnoDB: of MySQL." " Please close it to get rid of this error.\n"); + } else if (err == ERROR_WORKING_SET_QUOTA + || err == ERROR_NO_SYSTEM_RESOURCES) { + fprintf(stderr, + "InnoDB: The error means that there are no" + " sufficient system resources or quota to" + " complete the operation.\n"); + } else if (err == ERROR_OPERATION_ABORTED) { + fprintf(stderr, + "InnoDB: The error means that the I/O" + " operation has been aborted\n" + "InnoDB: because of either a thread exit" + " or an application request.\n" + "InnoDB: Retry attempt is made.\n"); } else { fprintf(stderr, "InnoDB: Some operating system error numbers" @@ -373,6 +391,11 @@ os_file_get_last_error( } else if (err == ERROR_SHARING_VIOLATION || err == ERROR_LOCK_VIOLATION) { return(OS_FILE_SHARING_VIOLATION); + } else if (err == ERROR_WORKING_SET_QUOTA + || err == ERROR_NO_SYSTEM_RESOURCES) { + return(OS_FILE_INSUFFICIENT_RESOURCE); + } else if (err == ERROR_OPERATION_ABORTED) { + return(OS_FILE_OPERATION_ABORTED); } else { return(100 + err); } @@ -491,6 +514,14 @@ os_file_handle_error_cond_exit( os_thread_sleep(10000000); /* 10 sec */ return(TRUE); + } else if (err == OS_FILE_INSUFFICIENT_RESOURCE) { + + os_thread_sleep(100000); /* 100 ms */ + return(TRUE); + } else if (err == OS_FILE_OPERATION_ABORTED) { + + os_thread_sleep(100000); /* 100 ms */ + return(TRUE); } else { if (name) { fprintf(stderr, "InnoDB: File name %s\n", name); @@ -854,6 +885,23 @@ next_file: ret = stat(full_path, &statinfo); if (ret) { + + if (errno == ENOENT) { + /* readdir() returned a file that does not exist, + it must have been deleted in the meantime. Do what + would have happened if the file was deleted before + readdir() - ignore and go to the next entry. + If this is the last entry then info->name will still + contain the name of the deleted file when this + function returns, but this is not an issue since the + caller shouldn't be looking at info when end of + directory is returned. */ + + ut_free(full_path); + + goto next_file; + } + os_file_handle_error_no_exit(full_path, "stat"); ut_free(full_path); @@ -1282,6 +1330,7 @@ try_again: } #endif #ifdef UNIV_NON_BUFFERED_IO +# ifndef UNIV_HOTBACKUP if (type == OS_LOG_FILE && srv_flush_log_at_trx_commit == 2) { /* Do not use unbuffered i/o to log files because value 2 denotes that we do not flush the log at every @@ -1290,10 +1339,14 @@ try_again: == SRV_WIN_IO_UNBUFFERED) { attributes = attributes | FILE_FLAG_NO_BUFFERING; } -#endif +# else /* !UNIV_HOTBACKUP */ + attributes = attributes | FILE_FLAG_NO_BUFFERING; +# endif /* !UNIV_HOTBACKUP */ +#endif /* UNIV_NON_BUFFERED_IO */ } else if (purpose == OS_FILE_NORMAL) { attributes = 0; #ifdef UNIV_NON_BUFFERED_IO +# ifndef UNIV_HOTBACKUP if (type == OS_LOG_FILE && srv_flush_log_at_trx_commit == 2) { /* Do not use unbuffered i/o to log files because value 2 denotes that we do not flush the log at every @@ -1302,7 +1355,10 @@ try_again: == SRV_WIN_IO_UNBUFFERED) { attributes = attributes | FILE_FLAG_NO_BUFFERING; } -#endif +# else /* !UNIV_HOTBACKUP */ + attributes = attributes | FILE_FLAG_NO_BUFFERING; +# endif /* !UNIV_HOTBACKUP */ +#endif /* UNIV_NON_BUFFERED_IO */ } else { attributes = 0; ut_error; @@ -2046,20 +2102,30 @@ os_file_flush( /*******************************************************************//** Does a synchronous read operation in Posix. @return number of bytes read, -1 if error */ +#define os_file_pread(file, buf, n, offset, offset_high) \ + _os_file_pread(file, buf, n, offset, offset_high, NULL); + static ssize_t -os_file_pread( +_os_file_pread( /*==========*/ os_file_t file, /*!< in: handle to a file */ void* buf, /*!< in: buffer where to read */ ulint n, /*!< in: number of bytes to read */ ulint offset, /*!< in: least significant 32 bits of file offset from where to read */ - ulint offset_high) /*!< in: most significant 32 bits of + ulint offset_high, /*!< in: most significant 32 bits of offset */ + trx_t* trx) { off_t offs; +#if defined(HAVE_PREAD) && !defined(HAVE_BROKEN_PREAD) ssize_t n_bytes; +#endif /* HAVE_PREAD && !HAVE_BROKEN_PREAD */ + ulint sec; + ulint ms; + ib_uint64_t start_time; + ib_uint64_t finish_time; ut_a((offset & 0xFFFFFFFFUL) == offset); @@ -2080,6 +2146,15 @@ os_file_pread( os_n_file_reads++; + if (innobase_get_slow_log() && trx && trx->take_stats) + { + trx->io_reads++; + trx->io_read += n; + ut_usectime(&sec, &ms); + start_time = (ib_uint64_t)sec * 1000000 + ms; + } else { + start_time = 0; + } #if defined(HAVE_PREAD) && !defined(HAVE_BROKEN_PREAD) os_mutex_enter(os_file_count_mutex); os_file_n_pending_preads++; @@ -2093,21 +2168,32 @@ os_file_pread( os_n_pending_reads--; os_mutex_exit(os_file_count_mutex); + if (innobase_get_slow_log() && trx && trx->take_stats && start_time) + { + ut_usectime(&sec, &ms); + finish_time = (ib_uint64_t)sec * 1000000 + ms; + trx->io_reads_wait_timer += (ulint)(finish_time - start_time); + } + return(n_bytes); #else { off_t ret_offset; ssize_t ret; +#ifndef UNIV_HOTBACKUP ulint i; +#endif /* !UNIV_HOTBACKUP */ os_mutex_enter(os_file_count_mutex); os_n_pending_reads++; os_mutex_exit(os_file_count_mutex); +#ifndef UNIV_HOTBACKUP /* Protect the seek / read operation with a mutex */ i = ((ulint) file) % OS_FILE_N_SEEK_MUTEXES; os_mutex_enter(os_file_seek_mutexes[i]); +#endif /* !UNIV_HOTBACKUP */ ret_offset = lseek(file, offs, SEEK_SET); @@ -2117,12 +2203,21 @@ os_file_pread( ret = read(file, buf, (ssize_t)n); } +#ifndef UNIV_HOTBACKUP os_mutex_exit(os_file_seek_mutexes[i]); +#endif /* !UNIV_HOTBACKUP */ os_mutex_enter(os_file_count_mutex); os_n_pending_reads--; os_mutex_exit(os_file_count_mutex); + if (innobase_get_slow_log() && trx && trx->take_stats && start_time) + { + ut_usectime(&sec, &ms); + finish_time = (ib_uint64_t)sec * 1000000 + ms; + trx->io_reads_wait_timer += (ulint)(finish_time - start_time); + } + return(ret); } #endif @@ -2195,16 +2290,20 @@ os_file_pwrite( #else { off_t ret_offset; +# ifndef UNIV_HOTBACKUP ulint i; +# endif /* !UNIV_HOTBACKUP */ os_mutex_enter(os_file_count_mutex); os_n_pending_writes++; os_mutex_exit(os_file_count_mutex); +# ifndef UNIV_HOTBACKUP /* Protect the seek / write operation with a mutex */ i = ((ulint) file) % OS_FILE_N_SEEK_MUTEXES; os_mutex_enter(os_file_seek_mutexes[i]); +# endif /* UNIV_HOTBACKUP */ ret_offset = lseek(file, offs, SEEK_SET); @@ -2230,7 +2329,9 @@ os_file_pwrite( # endif /* UNIV_DO_FLUSH */ func_exit: +# ifndef UNIV_HOTBACKUP os_mutex_exit(os_file_seek_mutexes[i]); +# endif /* !UNIV_HOTBACKUP */ os_mutex_enter(os_file_count_mutex); os_n_pending_writes--; @@ -2247,7 +2348,7 @@ Requests a synchronous positioned read o @return TRUE if request was successful, FALSE if fail */ UNIV_INTERN ibool -os_file_read( +_os_file_read( /*=========*/ os_file_t file, /*!< in: handle to a file */ void* buf, /*!< in: buffer where to read */ @@ -2255,7 +2356,8 @@ os_file_read( offset where to read */ ulint offset_high, /*!< in: most significant 32 bits of offset */ - ulint n) /*!< in: number of bytes to read */ + ulint n, /*!< in: number of bytes to read */ + trx_t* trx) { #ifdef __WIN__ BOOL ret; @@ -2264,7 +2366,9 @@ os_file_read( DWORD low; DWORD high; ibool retry; +#ifndef UNIV_HOTBACKUP ulint i; +#endif /* !UNIV_HOTBACKUP */ ut_a((offset & 0xFFFFFFFFUL) == offset); @@ -2283,16 +2387,20 @@ try_again: os_n_pending_reads++; os_mutex_exit(os_file_count_mutex); +#ifndef UNIV_HOTBACKUP /* Protect the seek / read operation with a mutex */ i = ((ulint) file) % OS_FILE_N_SEEK_MUTEXES; os_mutex_enter(os_file_seek_mutexes[i]); +#endif /* !UNIV_HOTBACKUP */ ret2 = SetFilePointer(file, low, &high, FILE_BEGIN); if (ret2 == 0xFFFFFFFF && GetLastError() != NO_ERROR) { +#ifndef UNIV_HOTBACKUP os_mutex_exit(os_file_seek_mutexes[i]); +#endif /* !UNIV_HOTBACKUP */ os_mutex_enter(os_file_count_mutex); os_n_pending_reads--; @@ -2303,7 +2411,9 @@ try_again: ret = ReadFile(file, buf, (DWORD) n, &len, NULL); +#ifndef UNIV_HOTBACKUP os_mutex_exit(os_file_seek_mutexes[i]); +#endif /* !UNIV_HOTBACKUP */ os_mutex_enter(os_file_count_mutex); os_n_pending_reads--; @@ -2312,14 +2422,14 @@ try_again: if (ret && len == n) { return(TRUE); } -#else +#else /* __WIN__ */ ibool retry; ssize_t ret; os_bytes_read_since_printout += n; try_again: - ret = os_file_pread(file, buf, n, offset, offset_high); + ret = _os_file_pread(file, buf, n, offset, offset_high, trx); if ((ulint)ret == n) { @@ -2331,7 +2441,7 @@ try_again: "InnoDB: Was only able to read %ld.\n", (ulong)n, (ulong)offset_high, (ulong)offset, (long)ret); -#endif +#endif /* __WIN__ */ #ifdef __WIN__ error_handling: #endif @@ -2380,7 +2490,9 @@ os_file_read_no_error_handling( DWORD low; DWORD high; ibool retry; +#ifndef UNIV_HOTBACKUP ulint i; +#endif /* !UNIV_HOTBACKUP */ ut_a((offset & 0xFFFFFFFFUL) == offset); @@ -2399,16 +2511,20 @@ try_again: os_n_pending_reads++; os_mutex_exit(os_file_count_mutex); +#ifndef UNIV_HOTBACKUP /* Protect the seek / read operation with a mutex */ i = ((ulint) file) % OS_FILE_N_SEEK_MUTEXES; os_mutex_enter(os_file_seek_mutexes[i]); +#endif /* !UNIV_HOTBACKUP */ ret2 = SetFilePointer(file, low, &high, FILE_BEGIN); if (ret2 == 0xFFFFFFFF && GetLastError() != NO_ERROR) { +#ifndef UNIV_HOTBACKUP os_mutex_exit(os_file_seek_mutexes[i]); +#endif /* !UNIV_HOTBACKUP */ os_mutex_enter(os_file_count_mutex); os_n_pending_reads--; @@ -2419,7 +2535,9 @@ try_again: ret = ReadFile(file, buf, (DWORD) n, &len, NULL); +#ifndef UNIV_HOTBACKUP os_mutex_exit(os_file_seek_mutexes[i]); +#endif /* !UNIV_HOTBACKUP */ os_mutex_enter(os_file_count_mutex); os_n_pending_reads--; @@ -2428,7 +2546,7 @@ try_again: if (ret && len == n) { return(TRUE); } -#else +#else /* __WIN__ */ ibool retry; ssize_t ret; @@ -2441,7 +2559,7 @@ try_again: return(TRUE); } -#endif +#endif /* __WIN__ */ #ifdef __WIN__ error_handling: #endif @@ -2500,9 +2618,11 @@ os_file_write( DWORD ret2; DWORD low; DWORD high; - ulint i; ulint n_retries = 0; ulint err; +#ifndef UNIV_HOTBACKUP + ulint i; +#endif /* !UNIV_HOTBACKUP */ ut_a((offset & 0xFFFFFFFF) == offset); @@ -2519,16 +2639,20 @@ retry: os_n_pending_writes++; os_mutex_exit(os_file_count_mutex); +#ifndef UNIV_HOTBACKUP /* Protect the seek / write operation with a mutex */ i = ((ulint) file) % OS_FILE_N_SEEK_MUTEXES; os_mutex_enter(os_file_seek_mutexes[i]); +#endif /* !UNIV_HOTBACKUP */ ret2 = SetFilePointer(file, low, &high, FILE_BEGIN); if (ret2 == 0xFFFFFFFF && GetLastError() != NO_ERROR) { +#ifndef UNIV_HOTBACKUP os_mutex_exit(os_file_seek_mutexes[i]); +#endif /* !UNIV_HOTBACKUP */ os_mutex_enter(os_file_count_mutex); os_n_pending_writes--; @@ -2562,7 +2686,9 @@ retry: } # endif /* UNIV_DO_FLUSH */ +#ifndef UNIV_HOTBACKUP os_mutex_exit(os_file_seek_mutexes[i]); +#endif /* !UNIV_HOTBACKUP */ os_mutex_enter(os_file_count_mutex); os_n_pending_writes--; @@ -2988,6 +3114,34 @@ os_aio_array_create( return(array); } +/************************************************************************//** +Frees an aio wait array. */ +static +void +os_aio_array_free( +/*==============*/ + os_aio_array_t* array) /*!< in, own: array to free */ +{ +#ifdef WIN_ASYNC_IO + ulint i; + + for (i = 0; i < array->n_slots; i++) { + os_aio_slot_t* slot = os_aio_array_get_nth_slot(array, i); + os_event_free(slot->event); + } +#endif /* WIN_ASYNC_IO */ + +#ifdef __WIN__ + ut_free(array->native_events); +#endif /* __WIN__ */ + os_mutex_free(array->mutex); + os_event_free(array->not_full); + os_event_free(array->is_empty); + + ut_free(array->slots); + ut_free(array); +} + /*********************************************************************** Initializes the asynchronous io system. Creates one array each for ibuf and log i/o. Also creates one array each for read and write where each @@ -3061,6 +3215,35 @@ os_aio_init( } +/*********************************************************************** +Frees the asynchronous io system. */ +UNIV_INTERN +void +os_aio_free(void) +/*=============*/ +{ + ulint i; + + os_aio_array_free(os_aio_ibuf_array); + os_aio_ibuf_array = NULL; + os_aio_array_free(os_aio_log_array); + os_aio_log_array = NULL; + os_aio_array_free(os_aio_read_array); + os_aio_read_array = NULL; + os_aio_array_free(os_aio_write_array); + os_aio_write_array = NULL; + os_aio_array_free(os_aio_sync_array); + os_aio_sync_array = NULL; + + for (i = 0; i < os_aio_n_segments; i++) { + os_event_free(os_aio_segment_wait_events[i]); + } + + ut_free(os_aio_segment_wait_events); + os_aio_segment_wait_events = 0; + os_aio_n_segments = 0; +} + #ifdef WIN_ASYNC_IO /************************************************************************//** Wakes up all async i/o threads in the array in Windows async i/o at @@ -3211,7 +3394,8 @@ os_aio_array_reserve_slot( offset */ ulint offset_high, /*!< in: most significant 32 bits of offset */ - ulint len) /*!< in: length of the block to read or write */ + ulint len, /*!< in: length of the block to read or write */ + trx_t* trx) { os_aio_slot_t* slot; #ifdef WIN_ASYNC_IO @@ -3432,9 +3616,21 @@ void os_aio_simulated_put_read_threads_to_sleep(void) /*============================================*/ { + +/* The idea of putting background IO threads to sleep is only for +Windows when using simulated AIO. Windows XP seems to schedule +background threads too eagerly to allow for coalescing during +readahead requests. */ +#ifdef __WIN__ os_aio_array_t* array; ulint g; + if (os_aio_use_native_aio) { + /* We do not use simulated aio: do nothing */ + + return; + } + os_aio_recommend_sleep_for_read_threads = TRUE; for (g = 0; g < os_aio_n_segments; g++) { @@ -3445,6 +3641,7 @@ os_aio_simulated_put_read_threads_to_sle os_event_reset(os_aio_segment_wait_events[g]); } } +#endif /* __WIN__ */ } /*******************************************************************//** @@ -3482,10 +3679,11 @@ os_aio( (can be used to identify a completed aio operation); ignored if mode is OS_AIO_SYNC */ - void* message2)/*!< in: message for the aio handler + void* message2,/*!< in: message for the aio handler (can be used to identify a completed aio operation); ignored if mode is OS_AIO_SYNC */ + trx_t* trx) { os_aio_array_t* array; os_aio_slot_t* slot; @@ -3524,8 +3722,8 @@ os_aio( wait in the Windows case. */ if (type == OS_FILE_READ) { - return(os_file_read(file, buf, offset, - offset_high, n)); + return(_os_file_read(file, buf, offset, + offset_high, n, trx)); } ut_a(type == OS_FILE_WRITE); @@ -3558,8 +3756,13 @@ try_again: ut_error; } + if (trx && type == OS_FILE_READ) + { + trx->io_reads++; + trx->io_read += n; + } slot = os_aio_array_reserve_slot(type, array, message1, message2, file, - name, buf, offset, offset_high, n); + name, buf, offset, offset_high, n, trx); if (type == OS_FILE_READ) { if (os_aio_use_native_aio) { #ifdef WIN_ASYNC_IO @@ -3679,6 +3882,7 @@ os_aio_windows_handle( ibool ret_val; BOOL ret; DWORD len; + BOOL retry = FALSE; if (segment == ULINT_UNDEFINED) { array = os_aio_sync_array; @@ -3732,14 +3936,52 @@ os_aio_windows_handle( ut_a(TRUE == os_file_flush(slot->file)); } #endif /* UNIV_DO_FLUSH */ + } else if (os_file_handle_error(slot->name, "Windows aio")) { + + retry = TRUE; } else { - os_file_handle_error(slot->name, "Windows aio"); ret_val = FALSE; } os_mutex_exit(array->mutex); + if (retry) { + /* retry failed read/write operation synchronously. + No need to hold array->mutex. */ + + switch (slot->type) { + case OS_FILE_WRITE: + ret = WriteFile(slot->file, slot->buf, + slot->len, &len, + &(slot->control)); + + break; + case OS_FILE_READ: + ret = ReadFile(slot->file, slot->buf, + slot->len, &len, + &(slot->control)); + + break; + default: + ut_error; + } + + if (!ret && GetLastError() == ERROR_IO_PENDING) { + /* aio was queued successfully! + We want a synchronous i/o operation on a + file where we also use async i/o: in Windows + we must use the same wait mechanism as for + async i/o */ + + ret = GetOverlappedResult(slot->file, + &(slot->control), + &len, TRUE); + } + + ret_val = ret && len == slot->len; + } + os_aio_array_free_slot(array, slot); return(ret_val); @@ -4018,6 +4260,18 @@ consecutive_loop: } } + if (srv_recovery_stats && recv_recovery_is_on() && n_consecutive) { + mutex_enter(&(recv_sys->mutex)); + if (slot->type == OS_FILE_READ) { + recv_sys->stats_read_io_pages += n_consecutive; + recv_sys->stats_read_io_consecutive[n_consecutive - 1]++; + } else if (slot->type == OS_FILE_WRITE) { + recv_sys->stats_write_io_pages += n_consecutive; + recv_sys->stats_write_io_consecutive[n_consecutive - 1]++; + } + mutex_exit(&(recv_sys->mutex)); + } + os_mutex_enter(array->mutex); /* Mark the i/os done in slots */ === modified file 'storage/xtradb/os/os0proc.c' --- a/storage/xtradb/os/os0proc.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/os/os0proc.c 2010-01-06 12:00:14 +0000 @@ -97,6 +97,7 @@ os_mem_alloc_large( fprintf(stderr, "InnoDB: HugeTLB: Warning: Failed to" " attach shared memory segment, errno %d\n", errno); + ptr = NULL; } /* Remove the shared memory segment so that it will be === modified file 'storage/xtradb/os/os0sync.c' --- a/storage/xtradb/os/os0sync.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/os/os0sync.c 2010-01-06 12:00:14 +0000 @@ -86,6 +86,9 @@ os_sync_init(void) UT_LIST_INIT(os_event_list); UT_LIST_INIT(os_mutex_list); + os_sync_mutex = NULL; + os_sync_mutex_inited = FALSE; + os_sync_mutex = os_mutex_create(NULL); os_sync_mutex_inited = TRUE; @@ -713,6 +716,7 @@ os_fast_mutex_free( os_mutex_enter(os_sync_mutex); } + ut_ad(os_fast_mutex_count > 0); os_fast_mutex_count--; if (UNIV_LIKELY(os_sync_mutex_inited)) { === modified file 'storage/xtradb/os/os0thread.c' --- a/storage/xtradb/os/os0thread.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/os/os0thread.c 2010-01-06 12:00:14 +0000 @@ -233,6 +233,7 @@ os_thread_exit( #ifdef __WIN__ ExitThread((DWORD)exit_value); #else + pthread_detach(pthread_self()); pthread_exit(exit_value); #endif } === modified file 'storage/xtradb/page/page0cur.c' --- a/storage/xtradb/page/page0cur.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/page/page0cur.c 2010-01-06 12:00:14 +0000 @@ -1195,7 +1195,7 @@ page_cur_insert_rec_zip_reorg( } /* Out of space: restore the page */ - if (!page_zip_decompress(page_zip, page)) { + if (!page_zip_decompress(page_zip, page, FALSE)) { ut_error; /* Memory corrupted? */ } ut_ad(page_validate(page, index)); === modified file 'storage/xtradb/page/page0page.c' --- a/storage/xtradb/page/page0page.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/page/page0page.c 2010-01-06 12:00:14 +0000 @@ -45,7 +45,7 @@ Created 2/2/1994 Heikki Tuuri ============== The index page consists of a page header which contains the page's -id and other information. On top of it are the the index records +id and other information. On top of it are the index records in a heap linked into a one way linear list according to alphabetic order. Just below page end is an array of pointers which we call page directory, @@ -679,7 +679,7 @@ page_copy_rec_list_end( if (UNIV_UNLIKELY (!page_zip_decompress(new_page_zip, - new_page))) { + new_page, FALSE))) { ut_error; } ut_ad(page_validate(new_page, index)); @@ -792,7 +792,7 @@ page_copy_rec_list_start( if (UNIV_UNLIKELY (!page_zip_decompress(new_page_zip, - new_page))) { + new_page, FALSE))) { ut_error; } ut_ad(page_validate(new_page, index)); === modified file 'storage/xtradb/page/page0zip.c' --- a/storage/xtradb/page/page0zip.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/page/page0zip.c 2010-01-06 12:00:14 +0000 @@ -47,8 +47,10 @@ Created June 2005 by Marko Makela # define buf_LRU_stat_inc_unzip() ((void) 0) #endif /* !UNIV_HOTBACKUP */ +#ifndef UNIV_HOTBACKUP /** Statistics on compression, indexed by page_zip_des_t::ssize - 1 */ UNIV_INTERN page_zip_stat_t page_zip_stat[PAGE_ZIP_NUM_SSIZE - 1]; +#endif /* !UNIV_HOTBACKUP */ /* Please refer to ../include/page0zip.ic for a description of the compressed page format. */ @@ -1144,7 +1146,9 @@ page_zip_compress( ulint* offsets = NULL; ulint n_blobs = 0; byte* storage;/* storage of uncompressed columns */ +#ifndef UNIV_HOTBACKUP ullint usec = ut_time_us(NULL); +#endif /* !UNIV_HOTBACKUP */ #ifdef PAGE_ZIP_COMPRESS_DBG FILE* logfile = NULL; #endif @@ -1208,7 +1212,9 @@ page_zip_compress( } } #endif /* PAGE_ZIP_COMPRESS_DBG */ +#ifndef UNIV_HOTBACKUP page_zip_stat[page_zip->ssize - 1].compressed++; +#endif /* !UNIV_HOTBACKUP */ if (UNIV_UNLIKELY(n_dense * PAGE_ZIP_DIR_SLOT_SIZE >= page_zip_get_size(page_zip))) { @@ -1345,8 +1351,10 @@ err_exit: fclose(logfile); } #endif /* PAGE_ZIP_COMPRESS_DBG */ +#ifndef UNIV_HOTBACKUP page_zip_stat[page_zip->ssize - 1].compressed_usec += ut_time_us(NULL) - usec; +#endif /* !UNIV_HOTBACKUP */ return(FALSE); } @@ -1404,12 +1412,14 @@ err_exit: fclose(logfile); } #endif /* PAGE_ZIP_COMPRESS_DBG */ +#ifndef UNIV_HOTBACKUP { page_zip_stat_t* zip_stat = &page_zip_stat[page_zip->ssize - 1]; zip_stat->compressed_ok++; zip_stat->compressed_usec += ut_time_us(NULL) - usec; } +#endif /* !UNIV_HOTBACKUP */ return(TRUE); } @@ -2811,7 +2821,11 @@ page_zip_decompress( /*================*/ page_zip_des_t* page_zip,/*!< in: data, ssize; out: m_start, m_end, m_nonempty, n_blobs */ - page_t* page) /*!< out: uncompressed page, may be trashed */ + page_t* page, /*!< out: uncompressed page, may be trashed */ + ibool all) /*!< in: TRUE=decompress the whole page; + FALSE=verify but do not copy some + page header fields that should not change + after page creation */ { z_stream d_stream; dict_index_t* index = NULL; @@ -2820,7 +2834,9 @@ page_zip_decompress( ulint trx_id_col = ULINT_UNDEFINED; mem_heap_t* heap; ulint* offsets; +#ifndef UNIV_HOTBACKUP ullint usec = ut_time_us(NULL); +#endif /* !UNIV_HOTBACKUP */ ut_ad(page_zip_simple_validate(page_zip)); UNIV_MEM_ASSERT_W(page, UNIV_PAGE_SIZE); @@ -2839,13 +2855,36 @@ page_zip_decompress( heap = mem_heap_create(n_dense * (3 * sizeof *recs) + UNIV_PAGE_SIZE); recs = mem_heap_alloc(heap, n_dense * (2 * sizeof *recs)); + if (all) { + /* Copy the page header. */ + memcpy(page, page_zip->data, PAGE_DATA); + } else { + /* Check that the bytes that we skip are identical. */ +#if defined UNIV_DEBUG || defined UNIV_ZIP_DEBUG + ut_a(!memcmp(FIL_PAGE_TYPE + page, + FIL_PAGE_TYPE + page_zip->data, + PAGE_HEADER - FIL_PAGE_TYPE)); + ut_a(!memcmp(PAGE_HEADER + PAGE_LEVEL + page, + PAGE_HEADER + PAGE_LEVEL + page_zip->data, + PAGE_DATA - (PAGE_HEADER + PAGE_LEVEL))); +#endif /* UNIV_DEBUG || UNIV_ZIP_DEBUG */ + + /* Copy the mutable parts of the page header. */ + memcpy(page, page_zip->data, FIL_PAGE_TYPE); + memcpy(PAGE_HEADER + page, PAGE_HEADER + page_zip->data, + PAGE_LEVEL - PAGE_N_DIR_SLOTS); + +#if defined UNIV_DEBUG || defined UNIV_ZIP_DEBUG + /* Check that the page headers match after copying. */ + ut_a(!memcmp(page, page_zip->data, PAGE_DATA)); +#endif /* UNIV_DEBUG || UNIV_ZIP_DEBUG */ + } + #ifdef UNIV_ZIP_DEBUG - /* Clear the page. */ - memset(page, 0x55, UNIV_PAGE_SIZE); + /* Clear the uncompressed page, except the header. */ + memset(PAGE_DATA + page, 0x55, UNIV_PAGE_SIZE - PAGE_DATA); #endif /* UNIV_ZIP_DEBUG */ - UNIV_MEM_INVALID(page, UNIV_PAGE_SIZE); - /* Copy the page header. */ - memcpy(page, page_zip->data, PAGE_DATA); + UNIV_MEM_INVALID(PAGE_DATA + page, UNIV_PAGE_SIZE - PAGE_DATA); /* Copy the page directory. */ if (UNIV_UNLIKELY(!page_zip_dir_decode(page_zip, page, recs, @@ -2976,12 +3015,14 @@ err_exit: page_zip_fields_free(index); mem_heap_free(heap); +#ifndef UNIV_HOTBACKUP { page_zip_stat_t* zip_stat = &page_zip_stat[page_zip->ssize - 1]; zip_stat->decompressed++; zip_stat->decompressed_usec += ut_time_us(NULL) - usec; } +#endif /* !UNIV_HOTBACKUP */ /* Update the stat counter for LRU policy. */ buf_LRU_stat_inc_unzip(); @@ -3084,7 +3125,7 @@ page_zip_validate_low( #endif /* UNIV_DEBUG_VALGRIND */ temp_page_zip = *page_zip; - valid = page_zip_decompress(&temp_page_zip, temp_page); + valid = page_zip_decompress(&temp_page_zip, temp_page, TRUE); if (!valid) { fputs("page_zip_validate(): failed to decompress\n", stderr); goto func_exit; @@ -4362,8 +4403,8 @@ IMPORTANT: if page_zip_reorganize() is i non-clustered index, the caller must update the insert buffer free bits in the same mini-transaction in such a way that the modification will be redo-logged. -@return TRUE on success, FALSE on failure; page and page_zip will be -left intact on failure. */ +@return TRUE on success, FALSE on failure; page_zip will be left +intact on failure, but page will be overwritten. */ UNIV_INTERN ibool page_zip_reorganize( @@ -4428,9 +4469,6 @@ page_zip_reorganize( if (UNIV_UNLIKELY(!page_zip_compress(page_zip, page, index, mtr))) { - /* Restore the old page and exit. */ - buf_frame_copy(page, temp_page); - #ifndef UNIV_HOTBACKUP buf_block_free(temp_block); #endif /* !UNIV_HOTBACKUP */ @@ -4591,7 +4629,8 @@ corrupt: memcpy(page_zip->data + page_zip_get_size(page_zip) - trailer_size, ptr + 8 + size, trailer_size); - if (UNIV_UNLIKELY(!page_zip_decompress(page_zip, page))) { + if (UNIV_UNLIKELY(!page_zip_decompress(page_zip, page, + TRUE))) { goto corrupt; } === modified file 'storage/xtradb/pars/lexyy.c' --- a/storage/xtradb/pars/lexyy.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/pars/lexyy.c 2010-01-17 11:41:32 +0000 @@ -2778,3 +2778,18 @@ static void yyfree (void * ptr ) + +/********************************************************************** +Release any resources used by the lexer. */ +UNIV_INTERN +void +pars_lexer_close(void) +/*==================*/ +{ + if (yy_buffer_stack) + yylex_destroy(); + if (stringbuf) + free(stringbuf); + stringbuf = NULL; + stringbuf_len_alloc = stringbuf_len = 0; +} === modified file 'storage/xtradb/pars/pars0lex.l' --- a/storage/xtradb/pars/pars0lex.l 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/pars/pars0lex.l 2010-01-17 11:41:32 +0000 @@ -661,3 +661,18 @@ In the state 'id', only two actions are } %% + +/********************************************************************** +Release any resources used by the lexer. */ +UNIV_INTERN +void +pars_lexer_close(void) +/*==================*/ +{ + if (yy_buffer_stack) + yylex_destroy(); + if (stringbuf) + free(stringbuf); + stringbuf = NULL; + stringbuf_len_alloc = stringbuf_len = 0; +} === modified file 'storage/xtradb/plug.in' --- a/storage/xtradb/plug.in 2009-11-13 21:26:08 +0000 +++ b/storage/xtradb/plug.in 2010-01-15 15:58:25 +0000 @@ -40,19 +40,11 @@ MYSQL_PLUGIN_ACTIONS(innobase, [ irix*|osf*|sysv5uw7*|openbsd*) CFLAGS="$CFLAGS -DUNIV_MUST_NOT_INLINE";; *solaris*|*SunOS*) - # Begin Solaris atomic function checks - AC_CHECK_FUNCS(atomic_cas_ulong atomic_cas_32 \ - atomic_cas_64 atomic_add_long, - AC_DEFINE( - [HAVE_SOLARIS_ATOMICS], - [1], - [Define to 1 if Solaris supports \ - atomic functions.])) - ### End Solaris atomic function checks - CFLAGS="$CFLAGS -DUNIV_SOLARIS";; esac + INNODB_DYNAMIC_CFLAGS="-DMYSQL_DYNAMIC_PLUGIN" + case "$target_cpu" in x86_64) # The AMD64 ABI forbids absolute addresses in shared libraries @@ -63,7 +55,60 @@ MYSQL_PLUGIN_ACTIONS(innobase, [ ;; esac AC_SUBST(INNODB_DYNAMIC_CFLAGS) + + AC_MSG_CHECKING(whether GCC atomic builtins are available) + # either define HAVE_IB_GCC_ATOMIC_BUILTINS or not + AC_TRY_RUN( + [ + int main() + { + long x; + long y; + long res; + char c; + + x = 10; + y = 123; + res = __sync_bool_compare_and_swap(&x, x, y); + if (!res || x != y) { + return(1); + } + + x = 10; + y = 123; + res = __sync_bool_compare_and_swap(&x, x + 1, y); + if (res || x != 10) { + return(1); + } + + x = 10; + y = 123; + res = __sync_add_and_fetch(&x, y); + if (res != 123 + 10 || x != 123 + 10) { + return(1); + } + + c = 10; + res = __sync_lock_test_and_set(&c, 123); + if (res != 10 || c != 123) { + return(1); + } + + return(0); + } + ], + [ + AC_DEFINE([HAVE_IB_GCC_ATOMIC_BUILTINS], [1], + [GCC atomic builtins are available]) + AC_MSG_RESULT(yes) + ], + [ + AC_MSG_RESULT(no) + ] + ) + AC_MSG_CHECKING(whether pthread_t can be used by GCC atomic builtins) + # either define HAVE_IB_ATOMIC_PTHREAD_T_GCC or not AC_TRY_RUN( [ #include <pthread.h> @@ -84,47 +129,73 @@ MYSQL_PLUGIN_ACTIONS(innobase, [ } ], [ - AC_DEFINE([HAVE_ATOMIC_PTHREAD_T], [1], + AC_DEFINE([HAVE_IB_ATOMIC_PTHREAD_T_GCC], [1], [pthread_t can be used by GCC atomic builtins]) AC_MSG_RESULT(yes) ], [ AC_MSG_RESULT(no) ] - ) + ) + + AC_MSG_CHECKING(whether Solaris libc atomic functions are available) + # either define HAVE_IB_SOLARIS_ATOMICS or not + AC_CHECK_FUNCS(atomic_add_long \ + atomic_cas_32 \ + atomic_cas_64 \ + atomic_cas_ulong, + + AC_DEFINE([HAVE_IB_SOLARIS_ATOMICS], [1], + [Define to 1 if Solaris libc atomic functions \ + are available]) + ) + + AC_MSG_CHECKING(whether pthread_t can be used by Solaris libc atomic functions) + # either define HAVE_IB_ATOMIC_PTHREAD_T_SOLARIS or not + AC_TRY_RUN( + [ + #include <pthread.h> + #include <string.h> - # Try using solaris atomics on SunOS if GCC atomics are not available - AC_CHECK_DECLS( - [HAVE_ATOMIC_PTHREAD_T], - [ - AC_MSG_NOTICE(no need to check pthread_t size) - ], - [ - AC_CHECK_DECLS( - [HAVE_SOLARIS_ATOMICS], - [ - AC_MSG_CHECKING(checking if pthread_t size is integral) - AC_TRY_RUN( - [ - #include <pthread.h> - int main() - { - pthread_t x = 0; - return(0); - } - ], - [ - AC_DEFINE([HAVE_ATOMIC_PTHREAD_T], [1], + int main(int argc, char** argv) { + pthread_t x1; + pthread_t x2; + pthread_t x3; + + memset(&x1, 0x0, sizeof(x1)); + memset(&x2, 0x0, sizeof(x2)); + memset(&x3, 0x0, sizeof(x3)); + + if (sizeof(pthread_t) == 4) { + + atomic_cas_32(&x1, x2, x3); + + } else if (sizeof(pthread_t) == 8) { + + atomic_cas_64(&x1, x2, x3); + + } else { + + return(1); + } + + return(0); + } + ], + [ + AC_DEFINE([HAVE_IB_ATOMIC_PTHREAD_T_SOLARIS], [1], [pthread_t can be used by solaris atomics]) - AC_MSG_RESULT(yes) - # size of pthread_t is needed for typed solaris atomics - AC_CHECK_SIZEOF([pthread_t], [], [#include <pthread.h>]) - ], - [ - AC_MSG_RESULT(no) - ]) - ]) - ]) + AC_MSG_RESULT(yes) + ], + [ + AC_MSG_RESULT(no) + ] + ) + + # this is needed to know which one of atomic_cas_32() or atomic_cas_64() + # to use in the source + AC_CHECK_SIZEOF([pthread_t], [], [#include <pthread.h>]) + # Check for x86 PAUSE instruction AC_MSG_CHECKING(for x86 PAUSE instruction) # We have to actually try running the test program, because of a bug @@ -141,7 +212,7 @@ MYSQL_PLUGIN_ACTIONS(innobase, [ } ], [ - AC_DEFINE([IB_HAVE_PAUSE_INSTRUCTION], [1], [Does x86 PAUSE instruction exist]) + AC_DEFINE([HAVE_IB_PAUSE_INSTRUCTION], [1], [Does x86 PAUSE instruction exist]) AC_MSG_RESULT(yes) ], [ === modified file 'storage/xtradb/que/que0que.c' --- a/storage/xtradb/que/que0que.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/que/que0que.c 2010-01-06 12:00:14 +0000 @@ -518,6 +518,7 @@ que_graph_free_recursive( upd_node_t* upd; tab_node_t* cre_tab; ind_node_t* cre_ind; + purge_node_t* purge; if (node == NULL) { @@ -579,6 +580,13 @@ que_graph_free_recursive( mem_heap_free(ins->entry_sys_heap); break; + case QUE_NODE_PURGE: + purge = node; + + mem_heap_free(purge->heap); + + break; + case QUE_NODE_UPDATE: upd = node; === modified file 'storage/xtradb/rem/rem0cmp.c' --- a/storage/xtradb/rem/rem0cmp.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/rem/rem0cmp.c 2010-01-06 12:00:14 +0000 @@ -36,7 +36,7 @@ Created 7/1/1994 Heikki Tuuri The records are put into alphabetical order in the following way: let F be the first field where two records disagree. -If there is a character in some position n where the the +If there is a character in some position n where the records disagree, the order is determined by comparison of the characters at position n, possibly after collating transformation. If there is no such character, @@ -76,7 +76,7 @@ cmp_debug_dtuple_rec_with_match( /*************************************************************//** This function is used to compare two data fields for which the data type is such that we must use MySQL code to compare them. The prototype here -must be a copy of the the one in ha_innobase.cc! +must be a copy of the one in ha_innobase.cc! @return 1, 0, -1, if a is greater, equal, less than b, respectively */ extern int @@ -399,7 +399,7 @@ next_byte: /*************************************************************//** This function is used to compare a data tuple to a physical record. Only dtuple->n_fields_cmp first fields are taken into account for -the the data tuple! If we denote by n = n_fields_cmp, then rec must +the data tuple! If we denote by n = n_fields_cmp, then rec must have either m >= n fields, or it must differ from dtuple in some of the m fields rec has. If rec has an externally stored field we do not compare it but return with value 0 if such a comparison should be === modified file 'storage/xtradb/row/row0ins.c' --- a/storage/xtradb/row/row0ins.c 2009-11-13 21:26:08 +0000 +++ b/storage/xtradb/row/row0ins.c 2010-01-15 15:58:25 +0000 @@ -141,7 +141,7 @@ row_ins_alloc_sys_fields( dfield = dtuple_get_nth_field(row, dict_col_get_no(col)); - ptr = mem_heap_alloc(heap, DATA_ROW_ID_LEN); + ptr = mem_heap_zalloc(heap, DATA_ROW_ID_LEN); dfield_set_data(dfield, ptr, DATA_ROW_ID_LEN); @@ -152,7 +152,7 @@ row_ins_alloc_sys_fields( col = dict_table_get_sys_col(table, DATA_TRX_ID); dfield = dtuple_get_nth_field(row, dict_col_get_no(col)); - ptr = mem_heap_alloc(heap, DATA_TRX_ID_LEN); + ptr = mem_heap_zalloc(heap, DATA_TRX_ID_LEN); dfield_set_data(dfield, ptr, DATA_TRX_ID_LEN); @@ -163,7 +163,7 @@ row_ins_alloc_sys_fields( col = dict_table_get_sys_col(table, DATA_ROLL_PTR); dfield = dtuple_get_nth_field(row, dict_col_get_no(col)); - ptr = mem_heap_alloc(heap, DATA_ROLL_PTR_LEN); + ptr = mem_heap_zalloc(heap, DATA_ROLL_PTR_LEN); dfield_set_data(dfield, ptr, DATA_ROLL_PTR_LEN); } @@ -1191,7 +1191,7 @@ row_ins_check_foreign_constraint( /*=============================*/ ibool check_ref,/*!< in: TRUE if we want to check that the referenced table is ok, FALSE if we - want to to check the foreign key table */ + want to check the foreign key table */ dict_foreign_t* foreign,/*!< in: foreign constraint; NOTE that the tables mentioned in it must be in the dictionary cache if they exist at all */ === modified file 'storage/xtradb/row/row0merge.c' --- a/storage/xtradb/row/row0merge.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/row/row0merge.c 2010-01-06 12:00:14 +0000 @@ -60,9 +60,19 @@ Completed by Sunny Bains and Marko Makel #ifdef UNIV_DEBUG /** Set these in order ot enable debug printout. */ /* @{ */ +/** Log the outcome of each row_merge_cmp() call, comparing records. */ static ibool row_merge_print_cmp; +/** Log each record read from temporary file. */ static ibool row_merge_print_read; +/** Log each record write to temporary file. */ static ibool row_merge_print_write; +/** Log each row_merge_blocks() call, merging two blocks of records to +a bigger one. */ +static ibool row_merge_print_block; +/** Log each block read from temporary file. */ +static ibool row_merge_print_block_read; +/** Log each block read from temporary file. */ +static ibool row_merge_print_block_write; /* @} */ #endif /* UNIV_DEBUG */ @@ -109,8 +119,9 @@ typedef struct row_merge_buf_struct row_ /** Information about temporary files used in merge sort */ struct merge_file_struct { - int fd; /*!< file descriptor */ - ulint offset; /*!< file offset */ + int fd; /*!< file descriptor */ + ulint offset; /*!< file offset (end of file) */ + ib_uint64_t n_rec; /*!< number of records in the file */ }; /** Information about temporary files used in merge sort */ @@ -682,6 +693,13 @@ row_merge_read( ib_uint64_t ofs = ((ib_uint64_t) offset) * sizeof *buf; ibool success; +#ifdef UNIV_DEBUG + if (row_merge_print_block_read) { + fprintf(stderr, "row_merge_read fd=%d ofs=%lu\n", + fd, (ulong) offset); + } +#endif /* UNIV_DEBUG */ + success = os_file_read_no_error_handling(OS_FILE_FROM_FD(fd), buf, (ulint) (ofs & 0xFFFFFFFF), (ulint) (ofs >> 32), @@ -709,6 +727,13 @@ row_merge_write( ib_uint64_t ofs = ((ib_uint64_t) offset) * sizeof(row_merge_block_t); +#ifdef UNIV_DEBUG + if (row_merge_print_block_write) { + fprintf(stderr, "row_merge_write fd=%d ofs=%lu\n", + fd, (ulong) offset); + } +#endif /* UNIV_DEBUG */ + return(UNIV_LIKELY(os_file_write("(merge)", OS_FILE_FROM_FD(fd), buf, (ulint) (ofs & 0xFFFFFFFF), (ulint) (ofs >> 32), @@ -718,7 +743,7 @@ row_merge_write( /********************************************************************//** Read a merge record. @return pointer to next record, or NULL on I/O error or end of list */ -static +static __attribute__((nonnull)) const byte* row_merge_read_rec( /*===============*/ @@ -1070,7 +1095,7 @@ row_merge_cmp( Reads clustered index of the table and create temporary files containing the index entries for the indexes to be built. @return DB_SUCCESS or error */ -static +static __attribute__((nonnull)) ulint row_merge_read_clustered_index( /*===========================*/ @@ -1175,6 +1200,12 @@ row_merge_read_clustered_index( in order to release the latch on the old page. */ if (btr_pcur_is_after_last_on_page(&pcur)) { + if (UNIV_UNLIKELY(trx_is_interrupted(trx))) { + i = 0; + err = DB_INTERRUPTED; + goto err_exit; + } + btr_pcur_store_position(&pcur, &mtr); mtr_commit(&mtr); mtr_start(&mtr); @@ -1233,6 +1264,7 @@ row_merge_read_clustered_index( if (UNIV_LIKELY (row && row_merge_buf_add(buf, row, ext))) { + file->n_rec++; continue; } @@ -1274,14 +1306,19 @@ err_exit: UNIV_MEM_INVALID(block[0], sizeof block[0]); merge_buf[i] = row_merge_buf_empty(buf); - /* Try writing the record again, now that - the buffer has been written out and emptied. */ + if (UNIV_LIKELY(row != NULL)) { + /* Try writing the record again, now + that the buffer has been written out + and emptied. */ + + if (UNIV_UNLIKELY + (!row_merge_buf_add(buf, row, ext))) { + /* An empty buffer should have enough + room for at least one record. */ + ut_error; + } - if (UNIV_UNLIKELY - (row && !row_merge_buf_add(buf, row, ext))) { - /* An empty buffer should have enough - room for at least one record. */ - ut_error; + file->n_rec++; } } @@ -1320,7 +1357,7 @@ func_exit: b2 = row_merge_write_rec(&block[2], &buf[2], b2, \ of->fd, &of->offset, \ mrec##N, offsets##N); \ - if (UNIV_UNLIKELY(!b2)) { \ + if (UNIV_UNLIKELY(!b2 || ++of->n_rec > file->n_rec)) { \ goto corrupt; \ } \ b##N = row_merge_read_rec(&block[N], &buf[N], \ @@ -1336,14 +1373,14 @@ func_exit: } while (0) /*************************************************************//** -Merge two blocks of linked lists on disk and write a bigger block. +Merge two blocks of records on disk and write a bigger block. @return DB_SUCCESS or error code */ static ulint row_merge_blocks( /*=============*/ const dict_index_t* index, /*!< in: index being created */ - merge_file_t* file, /*!< in/out: file containing + const merge_file_t* file, /*!< in: file containing index entries */ row_merge_block_t* block, /*!< in/out: 3 buffers */ ulint* foffs0, /*!< in/out: offset of first @@ -1366,6 +1403,17 @@ row_merge_blocks( ulint* offsets0;/* offsets of mrec0 */ ulint* offsets1;/* offsets of mrec1 */ +#ifdef UNIV_DEBUG + if (row_merge_print_block) { + fprintf(stderr, + "row_merge_blocks fd=%d ofs=%lu + fd=%d ofs=%lu" + " = fd=%d ofs=%lu\n", + file->fd, (ulong) *foffs0, + file->fd, (ulong) *foffs1, + of->fd, (ulong) of->offset); + } +#endif /* UNIV_DEBUG */ + heap = row_merge_heap_create(index, &offsets0, &offsets1); /* Write a record and read the next record. Split the output @@ -1438,16 +1486,88 @@ done1: } /*************************************************************//** +Copy a block of index entries. +@return TRUE on success, FALSE on failure */ +static __attribute__((nonnull)) +ibool +row_merge_blocks_copy( +/*==================*/ + const dict_index_t* index, /*!< in: index being created */ + const merge_file_t* file, /*!< in: input file */ + row_merge_block_t* block, /*!< in/out: 3 buffers */ + ulint* foffs0, /*!< in/out: input file offset */ + merge_file_t* of) /*!< in/out: output file */ +{ + mem_heap_t* heap; /*!< memory heap for offsets0, offsets1 */ + + mrec_buf_t buf[3]; /*!< buffer for handling + split mrec in block[] */ + const byte* b0; /*!< pointer to block[0] */ + byte* b2; /*!< pointer to block[2] */ + const mrec_t* mrec0; /*!< merge rec, points to block[0] */ + ulint* offsets0;/* offsets of mrec0 */ + ulint* offsets1;/* dummy offsets */ + +#ifdef UNIV_DEBUG + if (row_merge_print_block) { + fprintf(stderr, + "row_merge_blocks_copy fd=%d ofs=%lu" + " = fd=%d ofs=%lu\n", + file->fd, (ulong) foffs0, + of->fd, (ulong) of->offset); + } +#endif /* UNIV_DEBUG */ + + heap = row_merge_heap_create(index, &offsets0, &offsets1); + + /* Write a record and read the next record. Split the output + file in two halves, which can be merged on the following pass. */ + + if (!row_merge_read(file->fd, *foffs0, &block[0])) { +corrupt: + mem_heap_free(heap); + return(FALSE); + } + + b0 = block[0]; + b2 = block[2]; + + b0 = row_merge_read_rec(&block[0], &buf[0], b0, index, file->fd, + foffs0, &mrec0, offsets0); + if (UNIV_UNLIKELY(!b0 && mrec0)) { + + goto corrupt; + } + + if (mrec0) { + /* append all mrec0 to output */ + for (;;) { + ROW_MERGE_WRITE_GET_NEXT(0, goto done0); + } + } +done0: + + /* The file offset points to the beginning of the last page + that has been read. Update it to point to the next block. */ + (*foffs0)++; + + mem_heap_free(heap); + return(row_merge_write_eof(&block[2], b2, of->fd, &of->offset) + != NULL); +} + +/*************************************************************//** Merge disk files. @return DB_SUCCESS or error code */ -static +static __attribute__((nonnull)) ulint row_merge( /*======*/ + trx_t* trx, /*!< in: transaction */ const dict_index_t* index, /*!< in: index being created */ merge_file_t* file, /*!< in/out: file containing index entries */ - ulint half, /*!< in: half the file */ + ulint* half, /*!< in/out: half the file */ row_merge_block_t* block, /*!< in/out: 3 buffers */ int* tmpfd, /*!< in/out: temporary file handle */ TABLE* table) /*!< in/out: MySQL table, for @@ -1458,43 +1578,87 @@ row_merge( ulint foffs1; /*!< second input offset */ ulint error; /*!< error code */ merge_file_t of; /*!< output file */ + const ulint ihalf = *half; + /*!< half the input file */ + ulint ohalf; /*!< half the output file */ UNIV_MEM_ASSERT_W(block[0], 3 * sizeof block[0]); - ut_ad(half > 0); + ut_ad(ihalf < file->offset); of.fd = *tmpfd; of.offset = 0; + of.n_rec = 0; /* Merge blocks to the output file. */ + ohalf = 0; foffs0 = 0; - foffs1 = half; + foffs1 = ihalf; + + for (; foffs0 < ihalf && foffs1 < file->offset; foffs0++, foffs1++) { + ulint ahalf; /*!< arithmetic half the input file */ + + if (UNIV_UNLIKELY(trx_is_interrupted(trx))) { + return(DB_INTERRUPTED); + } - for (; foffs0 < half && foffs1 < file->offset; foffs0++, foffs1++) { error = row_merge_blocks(index, file, block, &foffs0, &foffs1, &of, table); if (error != DB_SUCCESS) { return(error); } + + /* Record the offset of the output file when + approximately half the output has been generated. In + this way, the next invocation of row_merge() will + spend most of the time in this loop. The initial + estimate is ohalf==0. */ + ahalf = file->offset / 2; + ut_ad(ohalf <= of.offset); + + /* Improve the estimate until reaching half the input + file size, or we can not get any closer to it. All + comparands should be non-negative when !(ohalf < ahalf) + because ohalf <= of.offset. */ + if (ohalf < ahalf || of.offset - ahalf < ohalf - ahalf) { + ohalf = of.offset; + } } - /* Copy the last block, if there is one. */ - while (foffs0 < half) { - if (!row_merge_read(file->fd, foffs0++, block) - || !row_merge_write(of.fd, of.offset++, block)) { + /* Copy the last blocks, if there are any. */ + + while (foffs0 < ihalf) { + if (UNIV_UNLIKELY(trx_is_interrupted(trx))) { + return(DB_INTERRUPTED); + } + + if (!row_merge_blocks_copy(index, file, block, &foffs0, &of)) { return(DB_CORRUPTION); } } + + ut_ad(foffs0 == ihalf); + while (foffs1 < file->offset) { - if (!row_merge_read(file->fd, foffs1++, block) - || !row_merge_write(of.fd, of.offset++, block)) { + if (UNIV_UNLIKELY(trx_is_interrupted(trx))) { + return(DB_INTERRUPTED); + } + + if (!row_merge_blocks_copy(index, file, block, &foffs1, &of)) { return(DB_CORRUPTION); } } + ut_ad(foffs1 == file->offset); + + if (UNIV_UNLIKELY(of.n_rec != file->n_rec)) { + return(DB_CORRUPTION); + } + /* Swap file descriptors for the next pass. */ *tmpfd = file->fd; *file = of; + *half = ohalf; UNIV_MEM_INVALID(block[0], 3 * sizeof block[0]); @@ -1508,6 +1672,7 @@ static ulint row_merge_sort( /*===========*/ + trx_t* trx, /*!< in: transaction */ const dict_index_t* index, /*!< in: index being created */ merge_file_t* file, /*!< in/out: file containing index entries */ @@ -1517,20 +1682,26 @@ row_merge_sort( reporting erroneous key value if applicable */ { - ulint blksz; /*!< block size */ + ulint half = file->offset / 2; + + /* The file should always contain at least one byte (the end + of file marker). Thus, it must be at least one block. */ + ut_ad(file->offset > 0); - for (blksz = 1; blksz < file->offset; blksz *= 2) { - ulint half; + do { ulint error; - ut_ad(ut_is_2pow(blksz)); - half = ut_2pow_round((file->offset + (blksz - 1)) / 2, blksz); - error = row_merge(index, file, half, block, tmpfd, table); + error = row_merge(trx, index, file, &half, + block, tmpfd, table); if (error != DB_SUCCESS) { return(error); } - } + + /* half > 0 should hold except when the file consists + of one block. No need to merge further then. */ + ut_ad(half > 0 || file->offset == 1); + } while (half < file->offset && half > 0); return(DB_SUCCESS); } @@ -1797,7 +1968,15 @@ row_merge_drop_index( static const char str1[] = "PROCEDURE DROP_INDEX_PROC () IS\n" "BEGIN\n" + /* Rename the index, so that it will be dropped by + row_merge_drop_temp_indexes() at crash recovery + if the server crashes before this trx is committed. */ + "UPDATE SYS_INDEXES SET NAME=CONCAT('" + TEMP_INDEX_PREFIX_STR "', NAME) WHERE ID = :indexid;\n" + "COMMIT WORK;\n" + /* Drop the field definitions of the index. */ "DELETE FROM SYS_FIELDS WHERE INDEX_ID = :indexid;\n" + /* Drop the index definition and the B-tree. */ "DELETE FROM SYS_INDEXES WHERE ID = :indexid\n" " AND TABLE_ID = :tableid;\n" "END;\n"; @@ -1909,6 +2088,7 @@ row_merge_file_create( { merge_file->fd = innobase_mysql_tmpfile(); merge_file->offset = 0; + merge_file->n_rec = 0; } /*********************************************************************//** @@ -2129,7 +2309,7 @@ row_merge_rename_tables( if (err != DB_SUCCESS) { err_exit: trx->error_state = DB_SUCCESS; - trx_general_rollback_for_mysql(trx, FALSE, NULL); + trx_general_rollback_for_mysql(trx, NULL); trx->error_state = DB_SUCCESS; } @@ -2331,7 +2511,7 @@ row_merge_build_indexes( sorting and inserting. */ for (i = 0; i < n_indexes; i++) { - error = row_merge_sort(indexes[i], &merge_files[i], + error = row_merge_sort(trx, indexes[i], &merge_files[i], block, &tmpfd, table); if (error == DB_SUCCESS) { === modified file 'storage/xtradb/row/row0mysql.c' --- a/storage/xtradb/row/row0mysql.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/row/row0mysql.c 2010-01-06 12:00:14 +0000 @@ -510,7 +510,7 @@ handle_new_error: switch (err) { case DB_LOCK_WAIT_TIMEOUT: if (row_rollback_on_timeout) { - trx_general_rollback_for_mysql(trx, FALSE, NULL); + trx_general_rollback_for_mysql(trx, NULL); break; } /* fall through */ @@ -526,7 +526,7 @@ handle_new_error: /* Roll back the latest, possibly incomplete insertion or update */ - trx_general_rollback_for_mysql(trx, TRUE, savept); + trx_general_rollback_for_mysql(trx, savept); } /* MySQL will roll back the latest SQL statement */ break; @@ -548,7 +548,7 @@ handle_new_error: /* Roll back the whole transaction; this resolution was added to version 3.23.43 */ - trx_general_rollback_for_mysql(trx, FALSE, NULL); + trx_general_rollback_for_mysql(trx, NULL); break; case DB_MUST_GET_MORE_FILE_SPACE: @@ -869,18 +869,22 @@ row_update_statistics_if_needed( } /*********************************************************************//** -Unlocks AUTO_INC type locks that were possibly reserved by a trx. */ +Unlocks AUTO_INC type locks that were possibly reserved by a trx. This +function should be called at the the end of an SQL statement, by the +connection thread that owns the transaction (trx->mysql_thd). */ UNIV_INTERN void row_unlock_table_autoinc_for_mysql( /*===============================*/ trx_t* trx) /*!< in/out: transaction */ { - mutex_enter(&kernel_mutex); + if (lock_trx_holds_autoinc_locks(trx)) { + mutex_enter(&kernel_mutex); - lock_release_autoinc_locks(trx); + lock_release_autoinc_locks(trx); - mutex_exit(&kernel_mutex); + mutex_exit(&kernel_mutex); + } } /*********************************************************************//** @@ -1770,7 +1774,6 @@ row_create_table_for_mysql( const char* table_name; ulint table_name_len; ulint err; - ulint i; ut_ad(trx->mysql_thread_id == os_thread_get_curr_id()); #ifdef UNIV_SYNC_DEBUG @@ -1805,15 +1808,6 @@ err_exit: goto err_exit; } - /* Check that no reserved column names are used. */ - for (i = 0; i < dict_table_get_n_user_cols(table); i++) { - if (dict_col_name_is_reserved( - dict_table_get_col_name(table, i))) { - - goto err_exit; - } - } - trx_start_if_not_started(trx); /* The table name is prefixed with the database name and a '/'. @@ -1888,7 +1882,9 @@ err_exit: if (UNIV_UNLIKELY(err != DB_SUCCESS)) { trx->error_state = DB_SUCCESS; - trx_general_rollback_for_mysql(trx, FALSE, NULL); + trx_general_rollback_for_mysql(trx, NULL); + /* TO DO: free table? The code below will dereference + table->name, though. */ } switch (err) { @@ -1907,31 +1903,6 @@ err_exit: break; case DB_DUPLICATE_KEY: - ut_print_timestamp(stderr); - fputs(" InnoDB: Error: table ", stderr); - ut_print_name(stderr, trx, TRUE, table->name); - fputs(" already exists in InnoDB internal\n" - "InnoDB: data dictionary. Have you deleted" - " the .frm file\n" - "InnoDB: and not used DROP TABLE?" - " Have you used DROP DATABASE\n" - "InnoDB: for InnoDB tables in" - " MySQL version <= 3.23.43?\n" - "InnoDB: See the Restrictions section" - " of the InnoDB manual.\n" - "InnoDB: You can drop the orphaned table" - " inside InnoDB by\n" - "InnoDB: creating an InnoDB table with" - " the same name in another\n" - "InnoDB: database and copying the .frm file" - " to the current database.\n" - "InnoDB: Then MySQL thinks the table exists," - " and DROP TABLE will\n" - "InnoDB: succeed.\n" - "InnoDB: You can look for further help from\n" - "InnoDB: " REFMAN "innodb-troubleshooting.html\n", - stderr); - /* We may also get err == DB_ERROR if the .ibd file for the table already exists */ @@ -2056,7 +2027,7 @@ error_handling: trx->error_state = DB_SUCCESS; - trx_general_rollback_for_mysql(trx, FALSE, NULL); + trx_general_rollback_for_mysql(trx, NULL); row_drop_table_for_mysql(table_name, trx, FALSE); @@ -2077,7 +2048,7 @@ Scans a table create SQL string and adds the foreign key constraints declared in the string. This function should be called after the indexes for a table have been created. Each foreign key constraint must be accompanied with indexes in -bot participating tables. The indexes are allowed to contain more +both participating tables. The indexes are allowed to contain more fields than mentioned in the constraint. Check also that foreign key constraints which reference this table are ok. @return error code or DB_SUCCESS */ @@ -2124,7 +2095,7 @@ row_table_add_foreign_constraints( trx->error_state = DB_SUCCESS; - trx_general_rollback_for_mysql(trx, FALSE, NULL); + trx_general_rollback_for_mysql(trx, NULL); row_drop_table_for_mysql(name, trx, FALSE); @@ -2491,7 +2462,7 @@ row_discard_tablespace_for_mysql( if (err != DB_SUCCESS) { trx->error_state = DB_SUCCESS; - trx_general_rollback_for_mysql(trx, FALSE, NULL); + trx_general_rollback_for_mysql(trx, NULL); trx->error_state = DB_SUCCESS; } else { dict_table_change_id_in_cache(table, new_id); @@ -2500,7 +2471,7 @@ row_discard_tablespace_for_mysql( if (!success) { trx->error_state = DB_SUCCESS; - trx_general_rollback_for_mysql(trx, FALSE, NULL); + trx_general_rollback_for_mysql(trx, NULL); trx->error_state = DB_SUCCESS; err = DB_ERROR; @@ -2952,7 +2923,7 @@ next_rec: if (err != DB_SUCCESS) { trx->error_state = DB_SUCCESS; - trx_general_rollback_for_mysql(trx, FALSE, NULL); + trx_general_rollback_for_mysql(trx, NULL); trx->error_state = DB_SUCCESS; ut_print_timestamp(stderr); fputs(" InnoDB: Unable to assign a new identifier to table ", @@ -3593,7 +3564,7 @@ row_delete_constraint( if ((err == DB_SUCCESS) && !strchr(id, '/')) { /* Old format < 4.0.18 constraints have constraint ids - <number>_<number>. We only try deleting them if the + NUMBER_NUMBER. We only try deleting them if the constraint name does not contain a '/' character, otherwise deleting a new format constraint named 'foo/bar' from database 'baz' would remove constraint 'bar' from database @@ -3857,7 +3828,7 @@ end: "InnoDB: succeed.\n", stderr); } trx->error_state = DB_SUCCESS; - trx_general_rollback_for_mysql(trx, FALSE, NULL); + trx_general_rollback_for_mysql(trx, NULL); trx->error_state = DB_SUCCESS; } else { /* The following call will also rename the .ibd data file if @@ -3866,7 +3837,7 @@ end: if (!dict_table_rename_in_cache(table, new_name, !new_is_tmp)) { trx->error_state = DB_SUCCESS; - trx_general_rollback_for_mysql(trx, FALSE, NULL); + trx_general_rollback_for_mysql(trx, NULL); trx->error_state = DB_SUCCESS; goto funct_exit; } @@ -3906,7 +3877,7 @@ end: ut_a(dict_table_rename_in_cache(table, old_name, FALSE)); trx->error_state = DB_SUCCESS; - trx_general_rollback_for_mysql(trx, FALSE, NULL); + trx_general_rollback_for_mysql(trx, NULL); trx->error_state = DB_SUCCESS; } } @@ -4166,6 +4137,7 @@ row_check_table_for_mysql( } if (trx_is_interrupted(prebuilt->trx)) { + ret = DB_INTERRUPTED; break; } === modified file 'storage/xtradb/row/row0sel.c' --- a/storage/xtradb/row/row0sel.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/row/row0sel.c 2010-01-15 21:12:30 +0000 @@ -4616,6 +4616,7 @@ row_search_autoinc_read_column( dict_index_t* index, /*!< in: index to read from */ const rec_t* rec, /*!< in: current rec */ ulint col_no, /*!< in: column number */ + ulint mtype, /*!< in: column main type */ ibool unsigned_type) /*!< in: signed or unsigned flag */ { ulint len; @@ -4632,10 +4633,27 @@ row_search_autoinc_read_column( data = rec_get_nth_field(rec, offsets, col_no, &len); ut_a(len != UNIV_SQL_NULL); - ut_a(len <= sizeof value); /* we assume AUTOINC value cannot be negative */ - value = mach_read_int_type(data, len, unsigned_type); + switch (mtype) { + case DATA_INT: + ut_a(len <= sizeof value); + value = mach_read_int_type(data, len, unsigned_type); + break; + + case DATA_FLOAT: + ut_a(len == sizeof(float)); + value = mach_float_read(data); + break; + + case DATA_DOUBLE: + ut_a(len == sizeof(double)); + value = mach_double_read(data); + break; + + default: + ut_error; + } if (UNIV_LIKELY_NULL(heap)) { mem_heap_free(heap); @@ -4721,7 +4739,8 @@ row_search_max_autoinc( dfield->col->prtype & DATA_UNSIGNED); *value = row_search_autoinc_read_column( - index, rec, i, unsigned_type); + index, rec, i, + dfield->col->mtype, unsigned_type); } } === modified file 'storage/xtradb/scripts/install_innodb_plugins.sql' --- a/storage/xtradb/scripts/install_innodb_plugins.sql 2009-06-25 01:43:25 +0000 +++ b/storage/xtradb/scripts/install_innodb_plugins.sql 2010-01-06 12:00:14 +0000 @@ -14,3 +14,4 @@ INSTALL PLUGIN INNODB_BUFFER_POOL_PAGES_ INSTALL PLUGIN innodb_rseg SONAME 'ha_innodb.so'; INSTALL PLUGIN innodb_table_stats SONAME 'ha_innodb.so'; INSTALL PLUGIN innodb_index_stats SONAME 'ha_innodb.so'; +INSTALL PLUGIN xtradb_admin_command SONAME 'ha_innodb.so'; === modified file 'storage/xtradb/scripts/install_innodb_plugins_win.sql' --- a/storage/xtradb/scripts/install_innodb_plugins_win.sql 2008-12-03 05:06:00 +0000 +++ b/storage/xtradb/scripts/install_innodb_plugins_win.sql 2010-01-06 12:00:14 +0000 @@ -7,3 +7,11 @@ INSTALL PLUGIN innodb_cmp SONAME 'ha_inn INSTALL PLUGIN innodb_cmp_reset SONAME 'ha_innodb.dll'; INSTALL PLUGIN innodb_cmpmem SONAME 'ha_innodb.dll'; INSTALL PLUGIN innodb_cmpmem_reset SONAME 'ha_innodb.dll'; +INSTALL PLUGIN XTRADB_ENHANCEMENTS SONAME 'ha_innodb.dll'; +INSTALL PLUGIN INNODB_BUFFER_POOL_PAGES SONAME 'ha_innodb.dll'; +INSTALL PLUGIN INNODB_BUFFER_POOL_PAGES_BLOB SONAME 'ha_innodb.dll'; +INSTALL PLUGIN INNODB_BUFFER_POOL_PAGES_INDEX SONAME 'ha_innodb.dll'; +INSTALL PLUGIN innodb_rseg SONAME 'ha_innodb.dll'; +INSTALL PLUGIN innodb_table_stats SONAME 'ha_innodb.dll'; +INSTALL PLUGIN innodb_index_stats SONAME 'ha_innodb.dll'; +INSTALL PLUGIN xtradb_admin_command SONAME 'ha_innodb.dll'; === modified file 'storage/xtradb/srv/srv0srv.c' --- a/storage/xtradb/srv/srv0srv.c 2010-01-06 21:27:53 +0000 +++ b/storage/xtradb/srv/srv0srv.c 2010-01-15 19:48:33 +0000 @@ -81,7 +81,6 @@ Created 10/8/1995 Heikki Tuuri #include "ut0mem.h" #include "ut0ut.h" #include "os0proc.h" -#include "os0sync.h" #include "mem0mem.h" #include "mem0pool.h" #include "sync0sync.h" @@ -103,6 +102,10 @@ Created 10/8/1995 Heikki Tuuri #include "row0mysql.h" #include "ha_prototypes.h" #include "trx0i_s.h" +#include "os0sync.h" /* for HAVE_ATOMIC_BUILTINS */ + +/* prototypes for new functions added to ha_innodb.cc */ +ibool innobase_get_slow_log(); /* This is set to TRUE if the MySQL user has set it in MySQL; currently affects only FOREIGN KEY definition parsing */ @@ -162,8 +165,9 @@ UNIV_INTERN ulint* srv_data_file_sizes = UNIV_INTERN ibool srv_extra_undoslots = FALSE; UNIV_INTERN ibool srv_fast_recovery = FALSE; +UNIV_INTERN ibool srv_recovery_stats = FALSE; -UNIV_INTERN ibool srv_use_purge_thread = FALSE; +UNIV_INTERN ulint srv_use_purge_thread = 0; /* if TRUE, then we auto-extend the last data file */ UNIV_INTERN ibool srv_auto_extend_last_data_file = FALSE; @@ -303,12 +307,6 @@ UNIV_INTERN ulint srv_buf_pool_flushed = reading of a disk page */ UNIV_INTERN ulint srv_buf_pool_reads = 0; -/** Number of sequential read-aheads */ -UNIV_INTERN ulint srv_read_ahead_seq = 0; - -/** Number of random read-aheads */ -UNIV_INTERN ulint srv_read_ahead_rnd = 0; - /* structure to pass status variables to MySQL */ UNIV_INTERN export_struc export_vars; @@ -399,6 +397,7 @@ UNIV_INTERN ulong srv_ibuf_active_contra UNIV_INTERN ulong srv_ibuf_accel_rate = 100; #define PCT_IBUF_IO(pct) ((ulint) (srv_io_capacity * srv_ibuf_accel_rate * ((double) pct / 10000.0))) +UNIV_INTERN ulint srv_checkpoint_age_target = 0; UNIV_INTERN ulong srv_flush_neighbor_pages = 1; /* 0:disable 1:enable */ UNIV_INTERN ulong srv_enable_unsafe_group_commit = 0; /* 0:disable 1:enable */ @@ -406,6 +405,7 @@ UNIV_INTERN ulong srv_read_ahead = 3; /* UNIV_INTERN ulong srv_adaptive_checkpoint = 0; /* 0: none 1: reflex 2: estimate */ UNIV_INTERN ulong srv_expand_import = 0; /* 0:disable 1:enable */ +UNIV_INTERN ulint srv_relax_table_creation = 0; /* 0:disable 1:enable */ UNIV_INTERN ulong srv_extra_rsegments = 0; /* extra rseg for users */ UNIV_INTERN ulong srv_dict_size_limit = 0; @@ -494,8 +494,6 @@ static ulint srv_main_background_loops static ulint srv_main_flush_loops = 0; /* Log writes involving flush. */ static ulint srv_log_writes_and_flush = 0; -/* Log writes not including flush. */ -static ulint srv_log_buffer_writes = 0; /* This is only ever touched by the master thread. It records the time when the last flush of log file has happened. The master @@ -644,7 +642,7 @@ future, but at the moment we plan to imp which could be called a global priority inheritance. If a thread has to wait for a long time, say 300 milliseconds, for a resource, we just guess that it may be waiting for a resource owned by a background -thread, and boost the the priority of all runnable background threads +thread, and boost the priority of all runnable background threads to the normal level. The background threads then themselves adjust their fixed priority back to background after releasing all resources they had (or, at some fixed points in their program code). @@ -744,9 +742,8 @@ srv_print_master_thread_info( srv_main_1_second_loops, srv_main_sleeps, srv_main_10_second_loops, srv_main_background_loops, srv_main_flush_loops); - fprintf(file, "srv_master_thread log flush and writes: %lu " - " log writes only: %lu\n", - srv_log_writes_and_flush, srv_log_buffer_writes); + fprintf(file, "srv_master_thread log flush and writes: %lu\n", + srv_log_writes_and_flush); } /*********************************************************************//** @@ -1044,13 +1041,26 @@ srv_init(void) } /*********************************************************************//** -Frees the OS fast mutex created in srv_init(). */ +Frees the data structures created in srv_init(). */ UNIV_INTERN void srv_free(void) /*==========*/ { os_fast_mutex_free(&srv_conc_mutex); + mem_free(srv_conc_slots); + srv_conc_slots = NULL; + + mem_free(srv_sys->threads); + mem_free(srv_sys); + srv_sys = NULL; + + mem_free(kernel_mutex_temp); + kernel_mutex_temp = NULL; + mem_free(srv_mysql_table); + srv_mysql_table = NULL; + + trx_i_s_cache_free(trx_i_s_cache); } /*********************************************************************//** @@ -1062,6 +1072,8 @@ srv_general_init(void) /*==================*/ { ut_mem_init(); + /* Reset the system variables in the recovery module. */ + recv_sys_var_init(); os_sync_init(); sync_init(); mem_init(srv_mem_pool_size); @@ -1077,7 +1089,7 @@ UNIV_INTERN ulong srv_max_purge_lag = 0 Puts an OS thread to wait if there are too many concurrent threads (>= srv_thread_concurrency) inside InnoDB. The threads wait in a FIFO queue. */ -#ifdef INNODB_RW_LOCKS_USE_ATOMICS +#ifdef HAVE_ATOMIC_BUILTINS static void enter_innodb_with_tickets(trx_t* trx) { @@ -1155,6 +1167,10 @@ srv_conc_enter_innodb( ibool has_slept = FALSE; srv_conc_slot_t* slot = NULL; ulint i; + ib_uint64_t start_time = 0L; + ib_uint64_t finish_time = 0L; + ulint sec; + ulint ms; if (trx->mysql_thd != NULL && thd_is_replication_slave_thread(trx->mysql_thd)) { @@ -1175,7 +1191,7 @@ srv_conc_enter_innodb( return; } -#ifdef INNODB_RW_LOCKS_USE_ATOMICS +#ifdef HAVE_ATOMIC_BUILTINS if (srv_thread_concurrency_timer_based) { srv_conc_enter_innodb_timer_based(trx); return; @@ -1231,6 +1247,7 @@ retry: switches. */ if (SRV_THREAD_SLEEP_DELAY > 0) { os_thread_sleep(SRV_THREAD_SLEEP_DELAY); + trx->innodb_que_wait_timer += SRV_THREAD_SLEEP_DELAY; } trx->op_info = ""; @@ -1286,12 +1303,25 @@ retry: /* Go to wait for the event; when a thread leaves InnoDB it will release this thread */ + if (innobase_get_slow_log() && trx->take_stats) { + ut_usectime(&sec, &ms); + start_time = (ib_uint64_t)sec * 1000000 + ms; + } else { + start_time = 0; + } + trx->op_info = "waiting in InnoDB queue"; os_event_wait(slot->event); trx->op_info = ""; + if (innobase_get_slow_log() && trx->take_stats && start_time) { + ut_usectime(&sec, &ms); + finish_time = (ib_uint64_t)sec * 1000000 + ms; + trx->innodb_que_wait_timer += (ulint)(finish_time - start_time); + } + os_fast_mutex_lock(&srv_conc_mutex); srv_conc_n_waiting_threads--; @@ -1325,7 +1355,7 @@ srv_conc_force_enter_innodb( } ut_ad(srv_conc_n_threads >= 0); -#ifdef INNODB_RW_LOCKS_USE_ATOMICS +#ifdef HAVE_ATOMIC_BUILTINS if (srv_thread_concurrency_timer_based) { os_atomic_increment_lint(&srv_conc_n_threads, 1); trx->declared_to_be_inside_innodb = TRUE; @@ -1366,7 +1396,7 @@ srv_conc_force_exit_innodb( return; } -#ifdef INNODB_RW_LOCKS_USE_ATOMICS +#ifdef HAVE_ATOMIC_BUILTINS if (srv_thread_concurrency_timer_based) { srv_conc_exit_innodb_timer_based(trx); return; @@ -2081,14 +2111,16 @@ srv_export_innodb_status(void) export_vars.innodb_data_writes = os_n_file_writes; export_vars.innodb_data_written = srv_data_written; export_vars.innodb_dict_tables= (dict_sys ? UT_LIST_GET_LEN(dict_sys->table_LRU) : 0); - export_vars.innodb_buffer_pool_read_requests = buf_pool->n_page_gets; + export_vars.innodb_buffer_pool_read_requests = buf_pool->stat.n_page_gets; export_vars.innodb_buffer_pool_write_requests = srv_buf_pool_write_requests; export_vars.innodb_buffer_pool_wait_free = srv_buf_pool_wait_free; export_vars.innodb_buffer_pool_pages_flushed = srv_buf_pool_flushed; export_vars.innodb_buffer_pool_reads = srv_buf_pool_reads; - export_vars.innodb_buffer_pool_read_ahead_rnd = srv_read_ahead_rnd; - export_vars.innodb_buffer_pool_read_ahead_seq = srv_read_ahead_seq; + export_vars.innodb_buffer_pool_read_ahead + = buf_pool->stat.n_ra_pages_read; + export_vars.innodb_buffer_pool_read_ahead_evicted + = buf_pool->stat.n_ra_pages_evicted; export_vars.innodb_buffer_pool_pages_data = UT_LIST_GET_LEN(buf_pool->LRU); export_vars.innodb_buffer_pool_pages_dirty @@ -2119,9 +2151,9 @@ srv_export_innodb_status(void) export_vars.innodb_log_writes = srv_log_writes; export_vars.innodb_dblwr_pages_written = srv_dblwr_pages_written; export_vars.innodb_dblwr_writes = srv_dblwr_writes; - export_vars.innodb_pages_created = buf_pool->n_pages_created; - export_vars.innodb_pages_read = buf_pool->n_pages_read; - export_vars.innodb_pages_written = buf_pool->n_pages_written; + export_vars.innodb_pages_created = buf_pool->stat.n_pages_created; + export_vars.innodb_pages_read = buf_pool->stat.n_pages_read; + export_vars.innodb_pages_written = buf_pool->stat.n_pages_written; export_vars.innodb_row_lock_waits = srv_n_lock_wait_count; export_vars.innodb_row_lock_current_waits = srv_n_lock_wait_current_count; @@ -2488,12 +2520,6 @@ srv_sync_log_buffer_in_background(void) log_buffer_sync_in_background(TRUE); srv_last_log_flush_time = current_time; srv_log_writes_and_flush++; - } else { - /* Actually we don't need to write logs here. - We are just being extra safe here by forcing - the log buffer to log file. */ - log_buffer_sync_in_background(FALSE); - srv_log_buffer_writes++; } } @@ -2533,10 +2559,9 @@ srv_master_thread( srv_main_thread_process_no = os_proc_get_number(); srv_main_thread_id = os_thread_pf(os_thread_get_curr_id()); - srv_table_reserve_slot(SRV_MASTER); - mutex_enter(&kernel_mutex); + srv_table_reserve_slot(SRV_MASTER); srv_n_threads_active[SRV_MASTER]++; mutex_exit(&kernel_mutex); @@ -2551,8 +2576,8 @@ loop: srv_main_thread_op_info = "reserving kernel mutex"; - n_ios_very_old = log_sys->n_log_ios + buf_pool->n_pages_read - + buf_pool->n_pages_written; + n_ios_very_old = log_sys->n_log_ios + buf_pool->stat.n_pages_read + + buf_pool->stat.n_pages_written; mutex_enter(&kernel_mutex); /* Store the user activity counter at the start of this loop */ @@ -2572,8 +2597,8 @@ loop: skip_sleep = FALSE; for (i = 0; i < 10; i++) { - n_ios_old = log_sys->n_log_ios + buf_pool->n_pages_read - + buf_pool->n_pages_written; + n_ios_old = log_sys->n_log_ios + buf_pool->stat.n_pages_read + + buf_pool->stat.n_pages_written; srv_main_thread_op_info = "sleeping"; srv_main_1_second_loops++; @@ -2625,8 +2650,8 @@ loop: n_pend_ios = buf_get_n_pending_ios() + log_sys->n_pending_writes; - n_ios = log_sys->n_log_ios + buf_pool->n_pages_read - + buf_pool->n_pages_written; + n_ios = log_sys->n_log_ios + buf_pool->stat.n_pages_read + + buf_pool->stat.n_pages_written; if (n_pend_ios < SRV_PEND_IO_THRESHOLD && (n_ios - n_ios_old < SRV_RECENT_IO_ACTIVITY)) { srv_main_thread_op_info = "doing insert buffer merge"; @@ -2642,6 +2667,8 @@ loop: /* Try to keep the number of modified pages in the buffer pool under the limit wished by the user */ + srv_main_thread_op_info = + "flushing buffer pool pages"; n_pages_flushed = buf_flush_batch(BUF_FLUSH_LIST, PCT_IO(100), IB_ULONGLONG_MAX); @@ -2664,6 +2691,8 @@ loop: ulint n_flush = buf_flush_get_desired_flush_rate(); if (n_flush) { + srv_main_thread_op_info = + "flushing buffer pool pages"; n_flush = ut_min(PCT_IO(100), n_flush); n_pages_flushed = buf_flush_batch( @@ -2835,8 +2864,8 @@ retry_flush_batch: are not required, and may be disabled. */ n_pend_ios = buf_get_n_pending_ios() + log_sys->n_pending_writes; - n_ios = log_sys->n_log_ios + buf_pool->n_pages_read - + buf_pool->n_pages_written; + n_ios = log_sys->n_log_ios + buf_pool->stat.n_pages_read + + buf_pool->stat.n_pages_written; srv_main_10_second_loops++; if (n_pend_ios < SRV_PEND_IO_THRESHOLD @@ -3130,20 +3159,34 @@ srv_purge_thread( ulint n_pages_purged_sum = 1; /* dummy */ ulint history_len; ulint sleep_ms= 10000; /* initial: 10 sec. */ + ibool can_be_last = FALSE; #ifdef UNIV_DEBUG_THREAD_CREATION fprintf(stderr, "Purge thread starts, id %lu\n", os_thread_pf(os_thread_get_curr_id())); #endif - srv_table_reserve_slot(SRV_PURGE); mutex_enter(&kernel_mutex); + srv_table_reserve_slot(SRV_PURGE); srv_n_threads_active[SRV_PURGE]++; mutex_exit(&kernel_mutex); loop: - if (srv_fast_shutdown && srv_shutdown_state > 0) { - goto exit_func; + if (srv_shutdown_state > 0) { + if (srv_fast_shutdown) { + /* someone other should wait the end of the workers */ + goto exit_func; + } + + mutex_enter(&kernel_mutex); + if (srv_n_threads_active[SRV_PURGE_WORKER]) { + can_be_last = FALSE; + } else { + can_be_last = TRUE; + } + mutex_exit(&kernel_mutex); + + sleep_ms = 10; } os_thread_sleep( sleep_ms * 1000 ); @@ -3164,6 +3207,15 @@ loop: n_pages_purged_sum += n_pages_purged; } while (n_pages_purged); + if (srv_shutdown_state > 0 && can_be_last) { + /* the last trx_purge() is executed without workers */ + goto exit_func; + } + + if (n_pages_purged_sum) { + srv_active_wake_master_thread(); + } + if (n_pages_purged_sum == 0) sleep_ms *= 10; if (sleep_ms > 10000) @@ -3172,9 +3224,62 @@ loop: goto loop; exit_func: - /* We count the number of threads in os_thread_exit(). A created - thread should always use that to exit and not use return() to exit. */ + trx_purge_worker_wake(); /* It may not make sense. for safety only */ + /* wake master thread to flush the pages */ + srv_wake_master_thread(); + + mutex_enter(&kernel_mutex); + srv_n_threads_active[SRV_PURGE]--; + mutex_exit(&kernel_mutex); + os_thread_exit(NULL); + + OS_THREAD_DUMMY_RETURN; +} + +/************************************************************************* +A thread which is devoted to purge, for take over the master thread's +purging */ +UNIV_INTERN +os_thread_ret_t +srv_purge_worker_thread( +/*====================*/ + void* arg) +{ + ulint worker_id; /* index for array */ + + worker_id = *((ulint*)arg); + +#ifdef UNIV_DEBUG_THREAD_CREATION + fprintf(stderr, "Purge worker thread starts, id %lu\n", + os_thread_pf(os_thread_get_curr_id())); +#endif + mutex_enter(&kernel_mutex); + srv_table_reserve_slot(SRV_PURGE_WORKER); + srv_n_threads_active[SRV_PURGE_WORKER]++; + mutex_exit(&kernel_mutex); + +loop: + /* purge worker threads only works when srv_shutdown_state==0 */ + /* for safety and exactness. */ + if (srv_shutdown_state > 0) { + goto exit_func; + } + + trx_purge_worker_wait(); + + if (srv_shutdown_state > 0) { + goto exit_func; + } + + trx_purge_worker(worker_id); + + goto loop; + +exit_func: + mutex_enter(&kernel_mutex); + srv_n_threads_active[SRV_PURGE_WORKER]--; + mutex_exit(&kernel_mutex); os_thread_exit(NULL); OS_THREAD_DUMMY_RETURN; === modified file 'storage/xtradb/srv/srv0start.c' --- a/storage/xtradb/srv/srv0start.c 2009-11-13 21:26:08 +0000 +++ b/storage/xtradb/srv/srv0start.c 2010-01-17 08:41:43 +0000 @@ -103,6 +103,8 @@ Created 2/16/1996 Heikki Tuuri # include "row0row.h" # include "row0mysql.h" # include "btr0pcur.h" +# include "thr0loc.h" +# include "os0sync.h" /* for INNODB_RW_LOCKS_USE_ATOMICS */ /** Log sequence number immediately after startup */ UNIV_INTERN ib_uint64_t srv_start_lsn; @@ -141,9 +143,9 @@ static mutex_t ios_mutex; static ulint ios; /** io_handler_thread parameters for thread identification */ -static ulint n[SRV_MAX_N_IO_THREADS + 5]; +static ulint n[SRV_MAX_N_IO_THREADS + 5 + 64]; /** io_handler_thread identifiers */ -static os_thread_id_t thread_ids[SRV_MAX_N_IO_THREADS + 5]; +static os_thread_id_t thread_ids[SRV_MAX_N_IO_THREADS + 5 + 64]; /** We use this mutex to test the return value of pthread_mutex_trylock on successful locking. HP-UX does NOT return 0, though Linux et al do. */ @@ -494,6 +496,8 @@ io_handler_thread( mutex_exit(&ios_mutex); } + thr_local_free(os_thread_get_curr_id()); + /* We count the number of threads in os_thread_exit(). A created thread should always use that to exit and not use return() to exit. The thread actually never comes here because it is exited in an @@ -530,32 +534,6 @@ srv_normalize_path_for_win( #endif } -/*********************************************************************//** -Adds a slash or a backslash to the end of a string if it is missing -and the string is not empty. -@return string which has the separator if the string is not empty */ -UNIV_INTERN -char* -srv_add_path_separator_if_needed( -/*=============================*/ - char* str) /*!< in: null-terminated character string */ -{ - char* out_str; - ulint len = ut_strlen(str); - - if (len == 0 || str[len - 1] == SRV_PATH_SEPARATOR) { - - return(str); - } - - out_str = ut_malloc(len + 2); - memcpy(out_str, str, len); - out_str[len] = SRV_PATH_SEPARATOR; - out_str[len + 1] = 0; - - return(out_str); -} - #ifndef UNIV_HOTBACKUP /*********************************************************************//** Calculates the low 32 bits when a file size which is given as a number @@ -604,19 +582,24 @@ open_or_create_log_file( ulint size; ulint size_high; char name[10000]; + ulint dirnamelen; UT_NOT_USED(create_new_db); *log_file_created = FALSE; srv_normalize_path_for_win(srv_log_group_home_dirs[k]); - srv_log_group_home_dirs[k] = srv_add_path_separator_if_needed( - srv_log_group_home_dirs[k]); - ut_a(strlen(srv_log_group_home_dirs[k]) - < (sizeof name) - 10 - sizeof "ib_logfile"); - sprintf(name, "%s%s%lu", srv_log_group_home_dirs[k], - "ib_logfile", (ulong) i); + dirnamelen = strlen(srv_log_group_home_dirs[k]); + ut_a(dirnamelen < (sizeof name) - 10 - sizeof "ib_logfile"); + memcpy(name, srv_log_group_home_dirs[k], dirnamelen); + + /* Add a path separator if needed. */ + if (dirnamelen && name[dirnamelen - 1] != SRV_PATH_SEPARATOR) { + name[dirnamelen++] = SRV_PATH_SEPARATOR; + } + + sprintf(name + dirnamelen, "%s%lu", "ib_logfile", (ulong) i); files[i] = os_file_create(name, OS_FILE_CREATE, OS_FILE_NORMAL, OS_LOG_FILE, &ret); @@ -779,14 +762,22 @@ open_or_create_data_files( *create_new_db = FALSE; srv_normalize_path_for_win(srv_data_home); - srv_data_home = srv_add_path_separator_if_needed(srv_data_home); for (i = 0; i < srv_n_data_files; i++) { + ulint dirnamelen; + srv_normalize_path_for_win(srv_data_file_names[i]); + dirnamelen = strlen(srv_data_home); - ut_a(strlen(srv_data_home) + strlen(srv_data_file_names[i]) + ut_a(dirnamelen + strlen(srv_data_file_names[i]) < (sizeof name) - 1); - sprintf(name, "%s%s", srv_data_home, srv_data_file_names[i]); + memcpy(name, srv_data_home, dirnamelen); + /* Add a path separator if needed. */ + if (dirnamelen && name[dirnamelen - 1] != SRV_PATH_SEPARATOR) { + name[dirnamelen++] = SRV_PATH_SEPARATOR; + } + + strcpy(name + dirnamelen, srv_data_file_names[i]); if (srv_data_file_is_raw_partition[i] == 0) { @@ -1008,7 +999,7 @@ skip_size_check: return(DB_SUCCESS); } -/****************************************************************//** +/******************************************************************** Starts InnoDB and creates a new database if database files are not found and the user wants. @return DB_SUCCESS or error code */ @@ -1096,6 +1087,10 @@ innobase_start_or_create_for_mysql(void) "InnoDB: !!!!!!!! UNIV_SEARCH_DEBUG switched on !!!!!!!!!\n"); #endif +#ifdef UNIV_LOG_LSN_DEBUG + fprintf(stderr, + "InnoDB: !!!!!!!! UNIV_LOG_LSN_DEBUG switched on !!!!!!!!!\n"); +#endif /* UNIV_LOG_LSN_DEBUG */ #ifdef UNIV_MEM_DEBUG fprintf(stderr, "InnoDB: !!!!!!!! UNIV_MEM_DEBUG switched on !!!!!!!!!\n"); @@ -1106,34 +1101,7 @@ innobase_start_or_create_for_mysql(void) "InnoDB: The InnoDB memory heap is disabled\n"); } -#ifdef HAVE_GCC_ATOMIC_BUILTINS -# ifdef INNODB_RW_LOCKS_USE_ATOMICS - fprintf(stderr, - "InnoDB: Mutexes and rw_locks use GCC atomic builtins.\n"); -# else /* INNODB_RW_LOCKS_USE_ATOMICS */ - fprintf(stderr, - "InnoDB: Mutexes use GCC atomic builtins, rw_locks do not.\n"); -# endif /* INNODB_RW_LOCKS_USE_ATOMICS */ -#elif defined(HAVE_SOLARIS_ATOMICS) -# ifdef INNODB_RW_LOCKS_USE_ATOMICS - fprintf(stderr, - "InnoDB: Mutexes and rw_locks use Solaris atomic functions.\n"); -# else - fprintf(stderr, - "InnoDB: Mutexes use Solaris atomic functions.\n"); -# endif /* INNODB_RW_LOCKS_USE_ATOMICS */ -#elif HAVE_WINDOWS_ATOMICS -# ifdef INNODB_RW_LOCKS_USE_ATOMICS - fprintf(stderr, - "InnoDB: Mutexes and rw_locks use Windows interlocked functions.\n"); -# else - fprintf(stderr, - "InnoDB: Mutexes use Windows interlocked functions.\n"); -# endif /* INNODB_RW_LOCKS_USE_ATOMICS */ -#else /* HAVE_GCC_ATOMIC_BUILTINS */ - fprintf(stderr, - "InnoDB: Neither mutexes nor rw_locks use GCC atomic builtins.\n"); -#endif /* HAVE_GCC_ATOMIC_BUILTINS */ + fprintf(stderr, "InnoDB: %s\n", IB_ATOMICS_STARTUP_MSG); /* Since InnoDB does not currently clean up all its internal data structures in MySQL Embedded Server Library server_end(), we @@ -1142,7 +1110,7 @@ innobase_start_or_create_for_mysql(void) if (srv_start_has_been_called) { fprintf(stderr, - "InnoDB: Error:startup called second time" + "InnoDB: Error: startup called second time" " during the process lifetime.\n" "InnoDB: In the MySQL Embedded Server Library" " you cannot call server_init()\n" @@ -1409,7 +1377,7 @@ innobase_start_or_create_for_mysql(void) sum_of_new_sizes += srv_data_file_sizes[i]; } - if (sum_of_new_sizes < 640) { + if (sum_of_new_sizes < 10485760 / UNIV_PAGE_SIZE) { fprintf(stderr, "InnoDB: Error: tablespace size must be" " at least 10 MB\n"); @@ -1739,8 +1707,17 @@ innobase_start_or_create_for_mysql(void) + (1 + SRV_MAX_N_IO_THREADS)); if (srv_use_purge_thread) { + ulint i; + os_thread_create(&srv_purge_thread, NULL, thread_ids + (4 + SRV_MAX_N_IO_THREADS)); + + for (i = 0; i < srv_use_purge_thread - 1; i++) { + n[5 + i + SRV_MAX_N_IO_THREADS] = i; /* using as index for arrays in purge_sys */ + os_thread_create(&srv_purge_worker_thread, + n + (5 + i + SRV_MAX_N_IO_THREADS), + thread_ids + (5 + i + SRV_MAX_N_IO_THREADS)); + } } #ifdef UNIV_DEBUG /* buf_debug_prints = TRUE; */ @@ -1853,7 +1830,7 @@ innobase_start_or_create_for_mysql(void) /* Actually, we did not change the undo log format between 4.0 and 4.1.1, and we would not need to run purge to completion. Note also that the purge algorithm in 4.1.1 - can process the the history list again even after a full + can process the history list again even after a full purge, because our algorithm does not cut the end of the history list in all cases so that it would become empty after a full purge. That mean that we may purge 4.0 type @@ -2005,8 +1982,10 @@ innobase_shutdown_for_mysql(void) /* All the threads have exited or are just exiting; NOTE that the threads may not have completed their exit yet. Should we use pthread_join() to make sure - they have exited? Now we just sleep 0.1 seconds and - hope that is enough! */ + they have exited? If we did, we would have to + remove the pthread_detach() from + os_thread_exit(). Now we just sleep 0.1 + seconds and hope that is enough! */ os_mutex_exit(os_sync_mutex); @@ -2045,36 +2024,40 @@ innobase_shutdown_for_mysql(void) srv_misc_tmpfile = 0; } + /* This must be disabled before closing the buffer pool + and closing the data dictionary. */ + btr_search_disable(); + + ibuf_close(); + log_shutdown(); + lock_sys_close(); + thr_local_close(); trx_sys_file_format_close(); + trx_sys_close(); mutex_free(&srv_monitor_file_mutex); mutex_free(&srv_dict_tmpfile_mutex); mutex_free(&srv_misc_tmpfile_mutex); + dict_close(); + btr_search_sys_free(); /* 3. Free all InnoDB's own mutexes and the os_fast_mutexes inside them */ + os_aio_free(); sync_close(); + srv_free(); + fil_close(); /* 4. Free the os_conc_mutex and all os_events and os_mutexes */ - srv_free(); os_sync_free(); - /* Check that all read views are closed except read view owned - by a purge. */ - - if (UT_LIST_GET_LEN(trx_sys->view_list) > 1) { - fprintf(stderr, - "InnoDB: Error: all read views were not closed" - " before shutdown:\n" - "InnoDB: %lu read views open \n", - UT_LIST_GET_LEN(trx_sys->view_list) - 1); - } - - /* 5. Free all allocated memory and the os_fast_mutex created in - ut0mem.c */ + /* 5. Free all allocated memory */ + pars_lexer_close(); + log_mem_free(); buf_pool_free(); + mem_close(); ut_free_all_mem(); if (os_thread_count != 0 @@ -2106,6 +2089,7 @@ innobase_shutdown_for_mysql(void) } srv_was_started = FALSE; + srv_start_has_been_called = FALSE; return((int) DB_SUCCESS); } === modified file 'storage/xtradb/sync/sync0arr.c' --- a/storage/xtradb/sync/sync0arr.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/sync/sync0arr.c 2010-01-06 12:00:14 +0000 @@ -227,24 +227,21 @@ sync_array_create( SYNC_ARRAY_MUTEX: determines the type of mutex protecting the data structure */ { + ulint sz; sync_array_t* arr; - sync_cell_t* cell_array; - sync_cell_t* cell; - ulint i; ut_a(n_cells > 0); /* Allocate memory for the data structures */ arr = ut_malloc(sizeof(sync_array_t)); + memset(arr, 0x0, sizeof(*arr)); - cell_array = ut_malloc(sizeof(sync_cell_t) * n_cells); + sz = sizeof(sync_cell_t) * n_cells; + arr->array = ut_malloc(sz); + memset(arr->array, 0x0, sz); arr->n_cells = n_cells; - arr->n_reserved = 0; - arr->array = cell_array; arr->protection = protection; - arr->sg_count = 0; - arr->res_count = 0; /* Then create the mutex to protect the wait array complex */ if (protection == SYNC_ARRAY_OS_MUTEX) { @@ -255,13 +252,6 @@ sync_array_create( ut_error; } - for (i = 0; i < n_cells; i++) { - cell = sync_array_get_nth_cell(arr, i); - cell->wait_object = NULL; - cell->waiting = FALSE; - cell->signal_count = 0; - } - return(arr); } @@ -492,12 +482,12 @@ sync_array_cell_print( mutex = cell->old_wait_mutex; fprintf(file, - "Mutex at %p created file %s line %lu, lock var %lu\n" + "Mutex at %p '%s', lock var %lu\n" #ifdef UNIV_SYNC_DEBUG "Last time reserved in file %s line %lu, " #endif /* UNIV_SYNC_DEBUG */ "waiters flag %lu\n", - (void*) mutex, mutex->cfile_name, (ulong) mutex->cline, + (void*) mutex, mutex->cmutex_name, (ulong) mutex->lock_word, #ifdef UNIV_SYNC_DEBUG mutex->file_name, (ulong) mutex->line, @@ -513,9 +503,8 @@ sync_array_cell_print( rwlock = cell->old_wait_rw_lock; fprintf(file, - " RW-latch at %p created in file %s line %lu\n", - (void*) rwlock, rwlock->cfile_name, - (ulong) rwlock->cline); + " RW-latch at %p '%s'\n", + (void*) rwlock, rwlock->lock_name); writer = rw_lock_get_writer(rwlock); if (writer != RW_LOCK_NOT_LOCKED) { fprintf(file, === modified file 'storage/xtradb/sync/sync0rw.c' --- a/storage/xtradb/sync/sync0rw.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/sync/sync0rw.c 2010-01-06 12:00:14 +0000 @@ -38,6 +38,7 @@ Created 9/11/1995 Heikki Tuuri #include "os0thread.h" #include "mem0mem.h" #include "srv0srv.h" +#include "os0sync.h" /* for INNODB_RW_LOCKS_USE_ATOMICS */ /* IMPLEMENTATION OF THE RW_LOCK @@ -230,8 +231,8 @@ rw_lock_create_func( # ifdef UNIV_SYNC_DEBUG ulint level, /*!< in: level */ # endif /* UNIV_SYNC_DEBUG */ - const char* cmutex_name, /*!< in: mutex name */ #endif /* UNIV_DEBUG */ + const char* cmutex_name, /*!< in: mutex name */ const char* cfile_name, /*!< in: file name where created */ ulint cline) /*!< in: file line where created */ { @@ -241,14 +242,15 @@ rw_lock_create_func( #ifndef INNODB_RW_LOCKS_USE_ATOMICS mutex_create(rw_lock_get_mutex(lock), SYNC_NO_ORDER_CHECK); - lock->mutex.cfile_name = cfile_name; - lock->mutex.cline = cline; + ut_d(lock->mutex.cfile_name = cfile_name); + ut_d(lock->mutex.cline = cline); - ut_d(lock->mutex.cmutex_name = cmutex_name); + lock->mutex.cmutex_name = cmutex_name; ut_d(lock->mutex.mutex_type = 1); #else /* INNODB_RW_LOCKS_USE_ATOMICS */ # ifdef UNIV_DEBUG - UT_NOT_USED(cmutex_name); + UT_NOT_USED(cfile_name); + UT_NOT_USED(cline); # endif #endif /* INNODB_RW_LOCKS_USE_ATOMICS */ @@ -268,8 +270,7 @@ rw_lock_create_func( lock->magic_n = RW_LOCK_MAGIC_N; - lock->cfile_name = cfile_name; - lock->cline = (unsigned int) cline; + lock->lock_name = cmutex_name; lock->count_os_wait = 0; lock->last_s_file_name = "not yet reserved"; @@ -304,8 +305,6 @@ rw_lock_free( ut_ad(rw_lock_validate(lock)); ut_a(lock->lock_word == X_LOCK_DECR); - lock->magic_n = 0; - #ifndef INNODB_RW_LOCKS_USE_ATOMICS mutex_free(rw_lock_get_mutex(lock)); #endif /* INNODB_RW_LOCKS_USE_ATOMICS */ @@ -325,6 +324,8 @@ rw_lock_free( UT_LIST_REMOVE(list, rw_lock_list, lock); mutex_exit(&rw_lock_list_mutex); + + lock->magic_n = 0; } #ifdef UNIV_DEBUG @@ -390,10 +391,10 @@ lock_loop: if (srv_print_latch_waits) { fprintf(stderr, "Thread %lu spin wait rw-s-lock at %p" - " cfile %s cline %lu rnds %lu\n", + " '%s' rnds %lu\n", (ulong) os_thread_pf(os_thread_get_curr_id()), (void*) lock, - lock->cfile_name, (ulong) lock->cline, (ulong) i); + lock->lock_name, (ulong) i); } /* We try once again to obtain the lock */ @@ -426,10 +427,9 @@ lock_loop: if (srv_print_latch_waits) { fprintf(stderr, "Thread %lu OS wait rw-s-lock at %p" - " cfile %s cline %lu\n", + " '%s'\n", os_thread_pf(os_thread_get_curr_id()), - (void*) lock, lock->cfile_name, - (ulong) lock->cline); + (void*) lock, lock->lock_name); } /* these stats may not be accurate */ @@ -648,9 +648,9 @@ lock_loop: if (srv_print_latch_waits) { fprintf(stderr, "Thread %lu spin wait rw-x-lock at %p" - " cfile %s cline %lu rnds %lu\n", + " '%s' rnds %lu\n", os_thread_pf(os_thread_get_curr_id()), (void*) lock, - lock->cfile_name, (ulong) lock->cline, (ulong) i); + lock->lock_name, (ulong) i); } sync_array_reserve_cell(sync_primary_wait_array, @@ -671,9 +671,9 @@ lock_loop: if (srv_print_latch_waits) { fprintf(stderr, "Thread %lu OS wait for rw-x-lock at %p" - " cfile %s cline %lu\n", + " '%s'\n", os_thread_pf(os_thread_get_curr_id()), (void*) lock, - lock->cfile_name, (ulong) lock->cline); + lock->lock_name); } /* these stats may not be accurate */ === modified file 'storage/xtradb/sync/sync0sync.c' --- a/storage/xtradb/sync/sync0sync.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/sync/sync0sync.c 2010-01-06 12:00:14 +0000 @@ -39,6 +39,7 @@ Created 9/5/1995 Heikki Tuuri #include "buf0buf.h" #include "srv0srv.h" #include "buf0types.h" +#include "os0sync.h" /* for HAVE_ATOMIC_BUILTINS */ /* REASONS FOR IMPLEMENTING THE SPIN LOCK MUTEX @@ -237,8 +238,8 @@ void mutex_create_func( /*==============*/ mutex_t* mutex, /*!< in: pointer to memory */ -#ifdef UNIV_DEBUG const char* cmutex_name, /*!< in: mutex name */ +#ifdef UNIV_DEBUG # ifdef UNIV_SYNC_DEBUG ulint level, /*!< in: level */ # endif /* UNIV_SYNC_DEBUG */ @@ -253,7 +254,7 @@ mutex_create_func( mutex->lock_word = 0; #endif mutex->event = os_event_create(NULL); - mutex_set_waiters(mutex, 0); + mutex->waiters = 0; #ifdef UNIV_DEBUG mutex->magic_n = MUTEX_MAGIC_N; #endif /* UNIV_DEBUG */ @@ -262,11 +263,13 @@ mutex_create_func( mutex->file_name = "not yet reserved"; mutex->level = level; #endif /* UNIV_SYNC_DEBUG */ +#ifdef UNIV_DEBUG mutex->cfile_name = cfile_name; mutex->cline = cline; +#endif /* UNIV_DEBUG */ mutex->count_os_wait = 0; -#ifdef UNIV_DEBUG mutex->cmutex_name= cmutex_name; +#ifdef UNIV_DEBUG mutex->count_using= 0; mutex->mutex_type= 0; mutex->lspent_time= 0; @@ -424,10 +427,18 @@ mutex_set_waiters( the value is stored to memory */ ut_ad(mutex); +#ifdef INNODB_RW_LOCKS_USE_ATOMICS + if (n) { + os_compare_and_swap_ulint(&mutex->waiters, 0, 1); + } else { + os_compare_and_swap_ulint(&mutex->waiters, 1, 0); + } +#else ptr = &(mutex->waiters); *ptr = n; /* Here we assume that the write of a single word in memory is atomic */ +#endif } /******************************************************************//** @@ -498,9 +509,9 @@ spin_loop: #ifdef UNIV_SRV_PRINT_LATCH_WAITS fprintf(stderr, "Thread %lu spin wait mutex at %p" - " cfile %s cline %lu rnds %lu\n", + " '%s' rnds %lu\n", (ulong) os_thread_pf(os_thread_get_curr_id()), (void*) mutex, - mutex->cfile_name, (ulong) mutex->cline, (ulong) i); + mutex->cmutex_name, (ulong) i); #endif mutex_spin_round_count += i; @@ -575,9 +586,9 @@ spin_loop: #ifdef UNIV_SRV_PRINT_LATCH_WAITS fprintf(stderr, - "Thread %lu OS wait mutex at %p cfile %s cline %lu rnds %lu\n", + "Thread %lu OS wait mutex at %p '%s' rnds %lu\n", (ulong) os_thread_pf(os_thread_get_curr_id()), (void*) mutex, - mutex->cfile_name, (ulong) mutex->cline, (ulong) i); + mutex->cmutex_name, (ulong) i); #endif mutex_os_wait_count++; @@ -849,7 +860,8 @@ sync_thread_levels_g( /*=================*/ sync_level_t* arr, /*!< in: pointer to level array for an OS thread */ - ulint limit) /*!< in: level limit */ + ulint limit, /*!< in: level limit */ + ulint warn) /*!< in: TRUE=display a diagnostic message */ { sync_level_t* slot; rw_lock_t* lock; @@ -863,6 +875,11 @@ sync_thread_levels_g( if (slot->latch != NULL) { if (slot->level <= limit) { + if (!warn) { + + return(FALSE); + } + lock = slot->latch; mutex = slot->latch; @@ -873,9 +890,8 @@ sync_thread_levels_g( if (mutex->magic_n == MUTEX_MAGIC_N) { fprintf(stderr, - "Mutex created at %s %lu\n", - mutex->cfile_name, - (ulong) mutex->cline); + "Mutex '%s'\n", + mutex->cmutex_name); if (mutex_get_lock_word(mutex) != 0) { const char* file_name; @@ -1106,7 +1122,7 @@ sync_thread_add_level( case SYNC_DICT_HEADER: case SYNC_TRX_I_S_RWLOCK: case SYNC_TRX_I_S_LAST_READ: - if (!sync_thread_levels_g(array, level)) { + if (!sync_thread_levels_g(array, level, TRUE)) { fprintf(stderr, "InnoDB: sync_thread_levels_g(array, %lu)" " does not hold!\n", level); @@ -1117,36 +1133,44 @@ sync_thread_add_level( /* Either the thread must own the buffer pool mutex (buf_pool_mutex), or it is allowed to latch only ONE buffer block (block->mutex or buf_pool_zip_mutex). */ - if (!sync_thread_levels_g(array, level)) { - ut_a(sync_thread_levels_g(array, level - 1)); + if (!sync_thread_levels_g(array, level, FALSE)) { + ut_a(sync_thread_levels_g(array, level - 1, TRUE)); ut_a(sync_thread_levels_contain(array, SYNC_BUF_LRU_LIST)); } break; case SYNC_REC_LOCK: - ut_a((sync_thread_levels_contain(array, SYNC_KERNEL) - && sync_thread_levels_g(array, SYNC_REC_LOCK - 1)) - || sync_thread_levels_g(array, SYNC_REC_LOCK)); + if (sync_thread_levels_contain(array, SYNC_KERNEL)) { + ut_a(sync_thread_levels_g(array, SYNC_REC_LOCK - 1, + TRUE)); + } else { + ut_a(sync_thread_levels_g(array, SYNC_REC_LOCK, TRUE)); + } break; case SYNC_IBUF_BITMAP: /* Either the thread must own the master mutex to all the bitmap pages, or it is allowed to latch only ONE bitmap page. */ - ut_a((sync_thread_levels_contain(array, SYNC_IBUF_BITMAP_MUTEX) - && sync_thread_levels_g(array, SYNC_IBUF_BITMAP - 1)) - || sync_thread_levels_g(array, SYNC_IBUF_BITMAP)); + if (sync_thread_levels_contain(array, + SYNC_IBUF_BITMAP_MUTEX)) { + ut_a(sync_thread_levels_g(array, SYNC_IBUF_BITMAP - 1, + TRUE)); + } else { + ut_a(sync_thread_levels_g(array, SYNC_IBUF_BITMAP, + TRUE)); + } break; case SYNC_FSP_PAGE: ut_a(sync_thread_levels_contain(array, SYNC_FSP)); break; case SYNC_FSP: ut_a(sync_thread_levels_contain(array, SYNC_FSP) - || sync_thread_levels_g(array, SYNC_FSP)); + || sync_thread_levels_g(array, SYNC_FSP, TRUE)); break; case SYNC_TRX_UNDO_PAGE: ut_a(sync_thread_levels_contain(array, SYNC_TRX_UNDO) || sync_thread_levels_contain(array, SYNC_RSEG) || sync_thread_levels_contain(array, SYNC_PURGE_SYS) - || sync_thread_levels_g(array, SYNC_TRX_UNDO_PAGE)); + || sync_thread_levels_g(array, SYNC_TRX_UNDO_PAGE, TRUE)); break; case SYNC_RSEG_HEADER: ut_a(sync_thread_levels_contain(array, SYNC_RSEG)); @@ -1158,37 +1182,41 @@ sync_thread_add_level( case SYNC_TREE_NODE: ut_a(sync_thread_levels_contain(array, SYNC_INDEX_TREE) || sync_thread_levels_contain(array, SYNC_DICT_OPERATION) - || sync_thread_levels_g(array, SYNC_TREE_NODE - 1)); + || sync_thread_levels_g(array, SYNC_TREE_NODE - 1, TRUE)); break; case SYNC_TREE_NODE_NEW: ut_a(sync_thread_levels_contain(array, SYNC_FSP_PAGE) || sync_thread_levels_contain(array, SYNC_IBUF_MUTEX)); break; case SYNC_INDEX_TREE: - ut_a((sync_thread_levels_contain(array, SYNC_IBUF_MUTEX) - && sync_thread_levels_contain(array, SYNC_FSP) - && sync_thread_levels_g(array, SYNC_FSP_PAGE - 1)) - || sync_thread_levels_g(array, SYNC_TREE_NODE - 1)); + if (sync_thread_levels_contain(array, SYNC_IBUF_MUTEX) + && sync_thread_levels_contain(array, SYNC_FSP)) { + ut_a(sync_thread_levels_g(array, SYNC_FSP_PAGE - 1, + TRUE)); + } else { + ut_a(sync_thread_levels_g(array, SYNC_TREE_NODE - 1, + TRUE)); + } break; case SYNC_IBUF_MUTEX: - ut_a(sync_thread_levels_g(array, SYNC_FSP_PAGE - 1)); + ut_a(sync_thread_levels_g(array, SYNC_FSP_PAGE - 1, TRUE)); break; case SYNC_IBUF_PESS_INSERT_MUTEX: - ut_a(sync_thread_levels_g(array, SYNC_FSP - 1) - && !sync_thread_levels_contain(array, SYNC_IBUF_MUTEX)); + ut_a(sync_thread_levels_g(array, SYNC_FSP - 1, TRUE)); + ut_a(!sync_thread_levels_contain(array, SYNC_IBUF_MUTEX)); break; case SYNC_IBUF_HEADER: - ut_a(sync_thread_levels_g(array, SYNC_FSP - 1) - && !sync_thread_levels_contain(array, SYNC_IBUF_MUTEX) - && !sync_thread_levels_contain( - array, SYNC_IBUF_PESS_INSERT_MUTEX)); + ut_a(sync_thread_levels_g(array, SYNC_FSP - 1, TRUE)); + ut_a(!sync_thread_levels_contain(array, SYNC_IBUF_MUTEX)); + ut_a(!sync_thread_levels_contain(array, + SYNC_IBUF_PESS_INSERT_MUTEX)); break; case SYNC_DICT: #ifdef UNIV_DEBUG ut_a(buf_debug_prints - || sync_thread_levels_g(array, SYNC_DICT)); + || sync_thread_levels_g(array, SYNC_DICT, TRUE)); #else /* UNIV_DEBUG */ - ut_a(sync_thread_levels_g(array, SYNC_DICT)); + ut_a(sync_thread_levels_g(array, SYNC_DICT, TRUE)); #endif /* UNIV_DEBUG */ break; default: @@ -1364,7 +1392,12 @@ sync_close(void) mutex_free(&mutex_list_mutex); #ifdef UNIV_SYNC_DEBUG mutex_free(&sync_thread_mutex); + + /* Switch latching order checks on in sync0sync.c */ + sync_order_checks_on = FALSE; #endif /* UNIV_SYNC_DEBUG */ + + sync_initialized = FALSE; } /*******************************************************************//** === modified file 'storage/xtradb/thr/thr0loc.c' --- a/storage/xtradb/thr/thr0loc.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/thr/thr0loc.c 2010-01-06 12:00:14 +0000 @@ -63,7 +63,7 @@ struct thr_local_struct{ os_thread_t handle; /*!< operating system handle to the thread */ ulint slot_no;/*!< the index of the slot in the thread table for this thread */ - ibool in_ibuf;/*!< TRUE if the the thread is doing an ibuf + ibool in_ibuf;/*!< TRUE if the thread is doing an ibuf operation */ hash_node_t hash; /*!< hash chain node */ ulint magic_n;/*!< magic number (THR_LOCAL_MAGIC_N) */ @@ -250,6 +250,37 @@ thr_local_init(void) mutex_create(&thr_local_mutex, SYNC_THR_LOCAL); } +/******************************************************************** +Close the thread local storage module. */ +UNIV_INTERN +void +thr_local_close(void) +/*=================*/ +{ + ulint i; + + ut_a(thr_local_hash != NULL); + + /* Free the hash elements. We don't remove them from the table + because we are going to destroy the table anyway. */ + for (i = 0; i < hash_get_n_cells(thr_local_hash); i++) { + thr_local_t* local; + + local = HASH_GET_FIRST(thr_local_hash, i); + + while (local) { + thr_local_t* prev_local = local; + + local = HASH_GET_NEXT(hash, prev_local); + ut_a(prev_local->magic_n == THR_LOCAL_MAGIC_N); + mem_free(prev_local); + } + } + + hash_table_free(thr_local_hash); + thr_local_hash = NULL; +} + /************************************************************************* Return local hash table informations. */ === modified file 'storage/xtradb/trx/trx0i_s.c' --- a/storage/xtradb/trx/trx0i_s.c 2009-11-29 23:08:56 +0000 +++ b/storage/xtradb/trx/trx0i_s.c 2010-01-15 15:58:25 +0000 @@ -60,7 +60,7 @@ Created July 17, 2007 Vasil Dimov /** @brief The maximum number of chunks to allocate for a table cache. The rows of a table cache are stored in a set of chunks. When a new -row is added a new chunk is allocated if necessary. Assuming that the +row is added a new chunk is allocated if necessary. Assuming that the first one is 1024 rows (TABLE_CACHE_INITIAL_ROWSNUM) and each subsequent is N/2 where N is the number of rows we have allocated till now, then 39th chunk would accommodate 1677416425 rows and all chunks @@ -238,6 +238,27 @@ table_cache_init( } /*******************************************************************//** +Frees a table cache. */ +static +void +table_cache_free( +/*=============*/ + i_s_table_cache_t* table_cache) /*!< in/out: table cache */ +{ + ulint i; + + for (i = 0; i < MEM_CHUNKS_IN_TABLE_CACHE; i++) { + + /* the memory is actually allocated in + table_cache_create_empty_row() */ + if (table_cache->chunks[i].base) { + mem_free(table_cache->chunks[i].base); + table_cache->chunks[i].base = NULL; + } + } +} + +/*******************************************************************//** Returns an empty row from a table cache. The row is allocated if no more empty rows are available. The number of used rows is incremented. If the memory limit is hit then NULL is returned and nothing is @@ -1252,6 +1273,22 @@ trx_i_s_cache_init( } /*******************************************************************//** +Free the INFORMATION SCHEMA trx related cache. */ +UNIV_INTERN +void +trx_i_s_cache_free( +/*===============*/ + trx_i_s_cache_t* cache) /*!< in, own: cache to free */ +{ + hash_table_free(cache->locks_hash); + ha_storage_free(cache->storage); + table_cache_free(&cache->innodb_trx); + table_cache_free(&cache->innodb_locks); + table_cache_free(&cache->innodb_lock_waits); + memset(cache, 0, sizeof *cache); +} + +/*******************************************************************//** Issue a shared/read lock on the tables cache. */ UNIV_INTERN void === modified file 'storage/xtradb/trx/trx0purge.c' --- a/storage/xtradb/trx/trx0purge.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/trx/trx0purge.c 2010-01-06 12:00:14 +0000 @@ -184,8 +184,9 @@ this query graph. @return own: the query graph */ static que_t* -trx_purge_graph_build(void) +trx_purge_graph_build( /*=======================*/ + trx_t* trx) { mem_heap_t* heap; que_fork_t* fork; @@ -194,7 +195,7 @@ trx_purge_graph_build(void) heap = mem_heap_create(512); fork = que_fork_create(NULL, NULL, QUE_FORK_PURGE, heap); - fork->trx = purge_sys->trx; + fork->trx = trx; thr = que_thr_create(fork, heap); @@ -243,10 +244,73 @@ trx_purge_sys_create(void) ut_a(trx_start_low(purge_sys->trx, ULINT_UNDEFINED)); - purge_sys->query = trx_purge_graph_build(); + purge_sys->query = trx_purge_graph_build(purge_sys->trx); purge_sys->view = read_view_oldest_copy_or_open_new(ut_dulint_zero, purge_sys->heap); + + purge_sys->n_worker = 0; + if (srv_use_purge_thread > 1) { + /* Use worker threads */ + ulint i; + + purge_sys->n_worker = srv_use_purge_thread - 1; + + purge_sys->sess_arr = mem_alloc(sizeof(sess_t*) * purge_sys->n_worker); + purge_sys->trx_arr = mem_alloc(sizeof(trx_t*) * purge_sys->n_worker); + purge_sys->query_arr = mem_alloc(sizeof(que_t*) * purge_sys->n_worker); + + purge_sys->worker_event = os_event_create(NULL); + os_event_reset(purge_sys->worker_event); + + for (i = 0; i < purge_sys->n_worker; i++) { + purge_sys->sess_arr[i] = sess_open(); + + purge_sys->trx_arr[i] = purge_sys->sess_arr[i]->trx; + purge_sys->trx_arr[i]->is_purge = 1; + ut_a(trx_start_low(purge_sys->trx_arr[i], ULINT_UNDEFINED)); + + purge_sys->query_arr[i] = trx_purge_graph_build(purge_sys->trx_arr[i]); + } + } +} + +/************************************************************************ +Frees the global purge system control structure. */ +UNIV_INTERN +void +trx_purge_sys_close(void) +/*======================*/ +{ + ut_ad(!mutex_own(&kernel_mutex)); + + que_graph_free(purge_sys->query); + + ut_a(purge_sys->sess->trx->is_purge); + purge_sys->sess->trx->conc_state = TRX_NOT_STARTED; + sess_close(purge_sys->sess); + purge_sys->sess = NULL; + + if (purge_sys->view != NULL) { + /* Because acquiring the kernel mutex is a pre-condition + of read_view_close(). We don't really need it here. */ + mutex_enter(&kernel_mutex); + + read_view_close(purge_sys->view); + purge_sys->view = NULL; + + mutex_exit(&kernel_mutex); + } + + trx_undo_arr_free(purge_sys->arr); + + rw_lock_free(&purge_sys->latch); + mutex_free(&purge_sys->mutex); + + mem_heap_free(purge_sys->heap); + mem_free(purge_sys); + + purge_sys = NULL; } /*================ UNDO LOG HISTORY LIST =============================*/ @@ -1110,7 +1174,7 @@ trx_purge(void) /* Handle at most 20 undo log pages in one purge batch */ - purge_sys->handle_limit = purge_sys->n_pages_handled + 20; + purge_sys->handle_limit = purge_sys->n_pages_handled + 20 * (srv_use_purge_thread + 1); old_pages_handled = purge_sys->n_pages_handled; @@ -1129,6 +1193,9 @@ trx_purge(void) mutex_exit(&kernel_mutex); + if (purge_sys->n_worker) + os_event_set(purge_sys->worker_event); + /* srv_que_task_enqueue(thr2); */ if (srv_print_thread_releases) { @@ -1138,6 +1205,9 @@ trx_purge(void) que_run_threads(thr); + if (purge_sys->n_worker) + os_event_reset(purge_sys->worker_event); + if (srv_print_thread_releases) { fprintf(stderr, @@ -1148,6 +1218,52 @@ trx_purge(void) return(purge_sys->n_pages_handled - old_pages_handled); } +/********************************************************************** +This function runs a purge worker batch */ +UNIV_INTERN +void +trx_purge_worker( +/*=============*/ + ulint worker_id) +{ + que_thr_t* thr; + + mutex_enter(&kernel_mutex); + + thr = que_fork_start_command(purge_sys->query_arr[worker_id]); + + ut_ad(thr); + + mutex_exit(&kernel_mutex); + + que_run_threads(thr); + + if (purge_sys->state == TRX_STOP_PURGE) { /* optimistic */ + os_event_reset(purge_sys->worker_event); + } +} + +/********************************************************************** +This function waits the event for worker batch */ +UNIV_INTERN +void +trx_purge_worker_wait(void) +/*=======================*/ +{ + os_event_wait(purge_sys->worker_event); +} + +/********************************************************************** +This function wakes the waiting worker batch */ +UNIV_INTERN +void +trx_purge_worker_wake(void) +/*=======================*/ +{ + if (purge_sys->n_worker) + os_event_set(purge_sys->worker_event); +} + /******************************************************************//** Prints information of the purge system to stderr. */ UNIV_INTERN === modified file 'storage/xtradb/trx/trx0rec.c' --- a/storage/xtradb/trx/trx0rec.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/trx/trx0rec.c 2010-01-06 12:00:14 +0000 @@ -1333,7 +1333,7 @@ trx_undo_get_undo_rec_low( ulint rseg_id; ulint page_no; ulint offset; - page_t* undo_page; + const page_t* undo_page; trx_rseg_t* rseg; ibool is_insert; mtr_t mtr; @@ -1572,7 +1572,7 @@ trx_undo_prev_version_build( /* We have to set the appropriate extern storage bits in the old version of the record: the extern bits in rec for those - fields that update does NOT update, as well as the the bits for + fields that update does NOT update, as well as the bits for those fields that update updates to become externally stored fields. Store the info: */ === modified file 'storage/xtradb/trx/trx0roll.c' --- a/storage/xtradb/trx/trx0roll.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/trx/trx0roll.c 2010-01-06 12:00:14 +0000 @@ -66,9 +66,9 @@ int trx_general_rollback_for_mysql( /*===========================*/ trx_t* trx, /*!< in: transaction handle */ - ibool partial,/*!< in: TRUE if partial rollback requested */ trx_savept_t* savept) /*!< in: pointer to savepoint undo number, if - partial rollback requested */ + partial rollback requested, or NULL for + complete rollback */ { mem_heap_t* heap; que_thr_t* thr; @@ -85,9 +85,8 @@ trx_general_rollback_for_mysql( roll_node = roll_node_create(heap); - roll_node->partial = partial; - - if (partial) { + if (savept) { + roll_node->partial = TRUE; roll_node->savept = *savept; } @@ -145,7 +144,7 @@ trx_rollback_for_mysql( the transaction object does not have an InnoDB session object, and we set a dummy session that we use for all MySQL transactions. */ - err = trx_general_rollback_for_mysql(trx, FALSE, NULL); + err = trx_general_rollback_for_mysql(trx, NULL); trx->op_info = ""; @@ -170,8 +169,7 @@ trx_rollback_last_sql_stat_for_mysql( trx->op_info = "rollback of SQL statement"; - err = trx_general_rollback_for_mysql(trx, TRUE, - &(trx->last_sql_stat_start)); + err = trx_general_rollback_for_mysql(trx, &trx->last_sql_stat_start); /* The following call should not be needed, but we play safe: */ trx_mark_sql_stat_end(trx); @@ -282,7 +280,7 @@ trx_rollback_to_savepoint_for_mysql( trx->op_info = "rollback to a savepoint"; - err = trx_general_rollback_for_mysql(trx, TRUE, &(savep->savept)); + err = trx_general_rollback_for_mysql(trx, &savep->savept); /* Store the current undo_no of the transaction so that we know where to roll back if we have to roll back the next SQL statement: */ @@ -534,28 +532,26 @@ trx_rollback_active( Rollback or clean up any incomplete transactions which were encountered in crash recovery. If the transaction already was committed, then we clean up a possible insert undo log. If the -transaction was not yet committed, then we roll it back. -Note: this is done in a background thread. -@return a dummy parameter */ +transaction was not yet committed, then we roll it back. */ UNIV_INTERN -os_thread_ret_t -trx_rollback_or_clean_all_recovered( -/*================================*/ - void* arg __attribute__((unused))) - /*!< in: a dummy parameter required by - os_thread_create */ +void +trx_rollback_or_clean_recovered( +/*============================*/ + ibool all) /*!< in: FALSE=roll back dictionary transactions; + TRUE=roll back all non-PREPARED transactions */ { trx_t* trx; mutex_enter(&kernel_mutex); - if (UT_LIST_GET_FIRST(trx_sys->trx_list)) { + if (!UT_LIST_GET_FIRST(trx_sys->trx_list)) { + goto leave_function; + } + if (all) { fprintf(stderr, "InnoDB: Starting in background the rollback" " of uncommitted transactions\n"); - } else { - goto leave_function; } mutex_exit(&kernel_mutex); @@ -584,18 +580,42 @@ loop: goto loop; case TRX_ACTIVE: - mutex_exit(&kernel_mutex); - trx_rollback_active(trx); - goto loop; + if (all || trx_get_dict_operation(trx) + != TRX_DICT_OP_NONE) { + mutex_exit(&kernel_mutex); + trx_rollback_active(trx); + goto loop; + } } } - ut_print_timestamp(stderr); - fprintf(stderr, - " InnoDB: Rollback of non-prepared transactions completed\n"); + if (all) { + ut_print_timestamp(stderr); + fprintf(stderr, + " InnoDB: Rollback of non-prepared" + " transactions completed\n"); + } leave_function: mutex_exit(&kernel_mutex); +} + +/*******************************************************************//** +Rollback or clean up any incomplete transactions which were +encountered in crash recovery. If the transaction already was +committed, then we clean up a possible insert undo log. If the +transaction was not yet committed, then we roll it back. +Note: this is done in a background thread. +@return a dummy parameter */ +UNIV_INTERN +os_thread_ret_t +trx_rollback_or_clean_all_recovered( +/*================================*/ + void* arg __attribute__((unused))) + /*!< in: a dummy parameter required by + os_thread_create */ +{ + trx_rollback_or_clean_recovered(TRUE); /* We count the number of threads in os_thread_exit(). A created thread should always use that to exit and not use return() to exit. */ === modified file 'storage/xtradb/trx/trx0rseg.c' --- a/storage/xtradb/trx/trx0rseg.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/trx/trx0rseg.c 2010-01-06 12:00:14 +0000 @@ -132,6 +132,49 @@ trx_rseg_header_create( } /***********************************************************************//** +Free's an instance of the rollback segment in memory. */ +UNIV_INTERN +void +trx_rseg_mem_free( +/*==============*/ + trx_rseg_t* rseg) /* in, own: instance to free */ +{ + trx_undo_t* undo; + + mutex_free(&rseg->mutex); + + /* There can't be any active transactions. */ + ut_a(UT_LIST_GET_LEN(rseg->update_undo_list) == 0); + ut_a(UT_LIST_GET_LEN(rseg->insert_undo_list) == 0); + + undo = UT_LIST_GET_FIRST(rseg->update_undo_cached); + + while (undo != NULL) { + trx_undo_t* prev_undo = undo; + + undo = UT_LIST_GET_NEXT(undo_list, undo); + UT_LIST_REMOVE(undo_list, rseg->update_undo_cached, prev_undo); + + trx_undo_mem_free(prev_undo); + } + + undo = UT_LIST_GET_FIRST(rseg->insert_undo_cached); + + while (undo != NULL) { + trx_undo_t* prev_undo = undo; + + undo = UT_LIST_GET_NEXT(undo_list, undo); + UT_LIST_REMOVE(undo_list, rseg->insert_undo_cached, prev_undo); + + trx_undo_mem_free(prev_undo); + } + + trx_sys_set_nth_rseg(trx_sys, rseg->id, NULL); + + mem_free(rseg); +} + +/*************************************************************************** Creates and initializes a rollback segment object. The values for the fields are read from the header. The object is inserted to the rseg list of the trx system object and a pointer is inserted in the rseg === modified file 'storage/xtradb/trx/trx0sys.c' --- a/storage/xtradb/trx/trx0sys.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/trx/trx0sys.c 2010-01-06 12:00:14 +0000 @@ -39,7 +39,9 @@ Created 3/26/1996 Heikki Tuuri #include "srv0srv.h" #include "trx0purge.h" #include "log0log.h" +#include "log0recv.h" #include "os0file.h" +#include "read0read.h" /** The file format tag structure with id and name. */ struct file_format_struct { @@ -552,6 +554,12 @@ trx_sys_doublewrite_init_or_restore_page zip_size ? zip_size : UNIV_PAGE_SIZE, read_buf, NULL); + if (srv_recovery_stats && recv_recovery_is_on()) { + mutex_enter(&(recv_sys->mutex)); + recv_sys->stats_doublewrite_check_pages++; + mutex_exit(&(recv_sys->mutex)); + } + /* Check if the page is corrupt */ if (UNIV_UNLIKELY @@ -599,6 +607,13 @@ trx_sys_doublewrite_init_or_restore_page zip_size, page_no, 0, zip_size ? zip_size : UNIV_PAGE_SIZE, page, NULL); + + if (srv_recovery_stats && recv_recovery_is_on()) { + mutex_enter(&(recv_sys->mutex)); + recv_sys->stats_doublewrite_overwrite_pages++; + mutex_exit(&(recv_sys->mutex)); + } + fprintf(stderr, "InnoDB: Recovered the page from" " the doublewrite buffer.\n"); @@ -1592,3 +1607,80 @@ trx_sys_file_format_id_to_name( } #endif /* !UNIV_HOTBACKUP */ + +/********************************************************************* +Shutdown/Close the transaction system. */ +UNIV_INTERN +void +trx_sys_close(void) +/*===============*/ +{ + trx_rseg_t* rseg; + read_view_t* view; + + ut_ad(trx_sys != NULL); + + /* Check that all read views are closed except read view owned + by a purge. */ + + if (UT_LIST_GET_LEN(trx_sys->view_list) > 1) { + fprintf(stderr, + "InnoDB: Error: all read views were not closed" + " before shutdown:\n" + "InnoDB: %lu read views open \n", + UT_LIST_GET_LEN(trx_sys->view_list) - 1); + } + + sess_close(trx_dummy_sess); + trx_dummy_sess = NULL; + + trx_purge_sys_close(); + + mutex_enter(&kernel_mutex); + + /* Free the double write data structures. */ + ut_a(trx_doublewrite != NULL); + ut_free(trx_doublewrite->write_buf_unaligned); + trx_doublewrite->write_buf_unaligned = NULL; + + mem_free(trx_doublewrite->buf_block_arr); + trx_doublewrite->buf_block_arr = NULL; + + mutex_free(&trx_doublewrite->mutex); + mem_free(trx_doublewrite); + trx_doublewrite = NULL; + + /* There can't be any active transactions. */ + rseg = UT_LIST_GET_FIRST(trx_sys->rseg_list); + + while (rseg != NULL) { + trx_rseg_t* prev_rseg = rseg; + + rseg = UT_LIST_GET_NEXT(rseg_list, prev_rseg); + UT_LIST_REMOVE(rseg_list, trx_sys->rseg_list, prev_rseg); + + trx_rseg_mem_free(prev_rseg); + } + + view = UT_LIST_GET_FIRST(trx_sys->view_list); + + while (view != NULL) { + read_view_t* prev_view = view; + + view = UT_LIST_GET_NEXT(view_list, prev_view); + + /* Views are allocated from the trx_sys->global_read_view_heap. + So, we simply remove the element here. */ + UT_LIST_REMOVE(view_list, trx_sys->view_list, prev_view); + } + + ut_a(UT_LIST_GET_LEN(trx_sys->trx_list) == 0); + ut_a(UT_LIST_GET_LEN(trx_sys->rseg_list) == 0); + ut_a(UT_LIST_GET_LEN(trx_sys->view_list) == 0); + ut_a(UT_LIST_GET_LEN(trx_sys->mysql_trx_list) == 0); + + mem_free(trx_sys); + + trx_sys = NULL; + mutex_exit(&kernel_mutex); +} === modified file 'storage/xtradb/trx/trx0trx.c' --- a/storage/xtradb/trx/trx0trx.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/trx/trx0trx.c 2010-01-06 12:00:14 +0000 @@ -178,6 +178,15 @@ trx_create( trx->global_read_view = NULL; trx->read_view = NULL; + trx->io_reads = 0; + trx->io_read = 0; + trx->io_reads_wait_timer = 0; + trx->lock_que_wait_timer = 0; + trx->innodb_que_wait_timer = 0; + trx->distinct_page_access = 0; + trx->distinct_page_access_hash = NULL; + trx->take_stats = FALSE; + /* Set X/Open XA transaction identification to NULL */ memset(&trx->xid, 0, sizeof(trx->xid)); trx->xid.formatID = -1; @@ -215,6 +224,11 @@ trx_allocate_for_mysql(void) trx->mysql_process_no = os_proc_get_number(); + if (innobase_get_slow_log() && trx->take_stats) { + trx->distinct_page_access_hash = mem_alloc(DPAH_SIZE); + memset(trx->distinct_page_access_hash, 0, DPAH_SIZE); + } + return(trx); } @@ -346,6 +360,12 @@ trx_free_for_mysql( /*===============*/ trx_t* trx) /*!< in, own: trx object */ { + if (trx->distinct_page_access_hash) + { + mem_free(trx->distinct_page_access_hash); + trx->distinct_page_access_hash= NULL; + } + mutex_enter(&kernel_mutex); UT_LIST_REMOVE(mysql_trx_list, trx_sys->mysql_trx_list, trx); @@ -367,6 +387,12 @@ trx_free_for_background( /*====================*/ trx_t* trx) /*!< in, own: trx object */ { + if (trx->distinct_page_access_hash) + { + mem_free(trx->distinct_page_access_hash); + trx->distinct_page_access_hash= NULL; + } + mutex_enter(&kernel_mutex); trx_free(trx); @@ -820,7 +846,7 @@ trx_commit_off_kernel( in exactly the same order as commit lsn's, if the transactions have different rollback segments. To get exactly the same order we should hold the kernel mutex up to this point, - adding to to the contention of the kernel mutex. However, if + adding to the contention of the kernel mutex. However, if a transaction T2 is able to see modifications made by a transaction T1, T2 will always get a bigger transaction number and a bigger commit lsn than T1. */ @@ -967,7 +993,7 @@ trx_commit_off_kernel( /****************************************************************//** Cleans up a transaction at database startup. The cleanup is needed if the transaction already got to the middle of a commit when the database -crashed, andf we cannot roll it back. */ +crashed, and we cannot roll it back. */ UNIV_INTERN void trx_cleanup_at_db_startup( @@ -1072,6 +1098,9 @@ trx_end_lock_wait( trx_t* trx) /*!< in: transaction */ { que_thr_t* thr; + ulint sec; + ulint ms; + ib_uint64_t now; ut_ad(mutex_own(&kernel_mutex)); ut_ad(trx->que_state == TRX_QUE_LOCK_WAIT); @@ -1086,6 +1115,11 @@ trx_end_lock_wait( thr = UT_LIST_GET_FIRST(trx->wait_thrs); } + if (innobase_get_slow_log() && trx->take_stats) { + ut_usectime(&sec, &ms); + now = (ib_uint64_t)sec * 1000000 + ms; + trx->lock_que_wait_timer += (ulint)(now - trx->lock_que_wait_ustarted); + } trx->que_state = TRX_QUE_RUNNING; } @@ -1099,6 +1133,9 @@ trx_lock_wait_to_suspended( trx_t* trx) /*!< in: transaction in the TRX_QUE_LOCK_WAIT state */ { que_thr_t* thr; + ulint sec; + ulint ms; + ib_uint64_t now; ut_ad(mutex_own(&kernel_mutex)); ut_ad(trx->que_state == TRX_QUE_LOCK_WAIT); @@ -1113,6 +1150,11 @@ trx_lock_wait_to_suspended( thr = UT_LIST_GET_FIRST(trx->wait_thrs); } + if (innobase_get_slow_log() && trx->take_stats) { + ut_usectime(&sec, &ms); + now = (ib_uint64_t)sec * 1000000 + ms; + trx->lock_que_wait_timer += (ulint)(now - trx->lock_que_wait_ustarted); + } trx->que_state = TRX_QUE_RUNNING; } === modified file 'storage/xtradb/trx/trx0undo.c' --- a/storage/xtradb/trx/trx0undo.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/trx/trx0undo.c 2010-01-06 12:00:14 +0000 @@ -1560,7 +1560,7 @@ trx_undo_mem_init_for_reuse( /********************************************************************//** Frees an undo log memory copy. */ -static +UNIV_INTERN void trx_undo_mem_free( /*==============*/ === modified file 'storage/xtradb/usr/usr0sess.c' --- a/storage/xtradb/usr/usr0sess.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/usr/usr0sess.c 2010-01-06 12:00:14 +0000 @@ -32,14 +32,6 @@ Created 6/25/1996 Heikki Tuuri #include "trx0trx.h" /*********************************************************************//** -Closes a session, freeing the memory occupied by it. */ -static -void -sess_close( -/*=======*/ - sess_t* sess); /*!< in, own: session object */ - -/*********************************************************************//** Opens a session. @return own: session object */ UNIV_INTERN @@ -64,35 +56,16 @@ sess_open(void) /*********************************************************************//** Closes a session, freeing the memory occupied by it. */ -static +UNIV_INTERN void sess_close( /*=======*/ sess_t* sess) /*!< in, own: session object */ { - ut_ad(mutex_own(&kernel_mutex)); - ut_ad(sess->trx == NULL); - - mem_free(sess); -} - -/*********************************************************************//** -Closes a session, freeing the memory occupied by it, if it is in a state -where it should be closed. -@return TRUE if closed */ -UNIV_INTERN -ibool -sess_try_close( -/*===========*/ - sess_t* sess) /*!< in, own: session object */ -{ - ut_ad(mutex_own(&kernel_mutex)); + ut_ad(!mutex_own(&kernel_mutex)); - if (UT_LIST_GET_LEN(sess->graphs) == 0) { - sess_close(sess); + ut_a(UT_LIST_GET_LEN(sess->graphs) == 0); - return(TRUE); - } - - return(FALSE); + trx_free_for_background(sess->trx); + mem_free(sess); } === modified file 'storage/xtradb/ut/ut0auxconf_atomic_pthread_t_solaris.c' --- a/storage/xtradb/ut/ut0auxconf_atomic_pthread_t_solaris.c 2009-09-23 00:06:02 +0000 +++ b/storage/xtradb/ut/ut0auxconf_atomic_pthread_t_solaris.c 2010-01-06 12:00:14 +0000 @@ -17,18 +17,38 @@ Place, Suite 330, Boston, MA 02111-1307 *****************************************************************************/ /***************************************************************************** -If this program compiles, then pthread_t objects can be used as arguments -to Solaris libc atomic functions. +If this program compiles and returns 0, then pthread_t objects can be used as +arguments to Solaris libc atomic functions. Created April 18, 2009 Vasil Dimov *****************************************************************************/ #include <pthread.h> +#include <string.h> int main(int argc, char** argv) { - pthread_t x = 0; + pthread_t x1; + pthread_t x2; + pthread_t x3; + + memset(&x1, 0x0, sizeof(x1)); + memset(&x2, 0x0, sizeof(x2)); + memset(&x3, 0x0, sizeof(x3)); + + if (sizeof(pthread_t) == 4) { + + atomic_cas_32(&x1, x2, x3); + + } else if (sizeof(pthread_t) == 8) { + + atomic_cas_64(&x1, x2, x3); + + } else { + + return(1); + } return(0); } === added file 'storage/xtradb/ut/ut0auxconf_have_gcc_atomics.c' --- a/storage/xtradb/ut/ut0auxconf_have_gcc_atomics.c 1970-01-01 00:00:00 +0000 +++ b/storage/xtradb/ut/ut0auxconf_have_gcc_atomics.c 2010-01-06 12:00:14 +0000 @@ -0,0 +1,61 @@ +/***************************************************************************** + +Copyright (c) 2009, Innobase Oy. All Rights Reserved. + +This program is free software; you can redistribute it and/or modify it under +the terms of the GNU General Public License as published by the Free Software +Foundation; version 2 of the License. + +This program is distributed in the hope that it will be useful, but WITHOUT +ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS +FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. + +You should have received a copy of the GNU General Public License along with +this program; if not, write to the Free Software Foundation, Inc., 59 Temple +Place, Suite 330, Boston, MA 02111-1307 USA + +*****************************************************************************/ + +/***************************************************************************** +If this program compiles and returns 0, then GCC atomic funcions are available. + +Created September 12, 2009 Vasil Dimov +*****************************************************************************/ + +int +main(int argc, char** argv) +{ + long x; + long y; + long res; + char c; + + x = 10; + y = 123; + res = __sync_bool_compare_and_swap(&x, x, y); + if (!res || x != y) { + return(1); + } + + x = 10; + y = 123; + res = __sync_bool_compare_and_swap(&x, x + 1, y); + if (res || x != 10) { + return(1); + } + + x = 10; + y = 123; + res = __sync_add_and_fetch(&x, y); + if (res != 123 + 10 || x != 123 + 10) { + return(1); + } + + c = 10; + res = __sync_lock_test_and_set(&c, 123); + if (res != 10 || c != 123) { + return(1); + } + + return(0); +} === modified file 'storage/xtradb/ut/ut0mem.c' --- a/storage/xtradb/ut/ut0mem.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/ut/ut0mem.c 2010-01-06 12:00:14 +0000 @@ -433,6 +433,8 @@ ut_free_all_mem(void) " total allocated memory is %lu\n", (ulong) ut_total_allocated_memory); } + + ut_mem_block_list_inited = FALSE; } #endif /* !UNIV_HOTBACKUP */ === modified file 'storage/xtradb/ut/ut0ut.c' --- a/storage/xtradb/ut/ut0ut.c 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/ut/ut0ut.c 2010-01-06 12:00:14 +0000 @@ -132,6 +132,7 @@ ut_time(void) return(time(NULL)); } +#ifndef UNIV_HOTBACKUP /**********************************************************//** Returns system time. Upon successful completion, the value 0 is returned; otherwise the @@ -200,6 +201,24 @@ ut_time_us( } /**********************************************************//** +Returns the number of milliseconds since some epoch. The +value may wrap around. It should only be used for heuristic +purposes. +@return ms since epoch */ +UNIV_INTERN +ulint +ut_time_ms(void) +/*============*/ +{ + struct timeval tv; + + ut_gettimeofday(&tv, NULL); + + return((ulint) tv.tv_sec * 1000 + tv.tv_usec / 1000); +} +#endif /* !UNIV_HOTBACKUP */ + +/**********************************************************//** Returns the difference of two times in seconds. @return time2 - time1 expressed in seconds */ UNIV_INTERN === removed directory 'storage/xtradb/win-plugin' === removed file 'storage/xtradb/win-plugin/README' --- a/storage/xtradb/win-plugin/README 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/win-plugin/README 1970-01-01 00:00:00 +0000 @@ -1,22 +0,0 @@ -This directory contains patches that need to be applied to the MySQL -source tree in order to build the dynamic plugin on Windows -- -HA_INNODB.DLL. Please note the followings when adding the patches: - -* The patch must be applied from the mysql top-level source directory. - patch -p0 < win-plugin.diff -* The patch filenames end in ".diff". -* All patches here are expected to apply cleanly to the latest MySQL 5.1 - tree when storage/innobase is replaced with this InnoDB branch. - -When applying the patch, the following files will be modified: - - * CMakeLists.txt - * sql/CMakeLists.txt - * win/configure.js - -Also, two new files will be added: - - * sql/mysqld.def - * sql/mysqld_x64.def - -You can get "patch" utility for Windows from http://unxutils.sourceforge.net/ === removed file 'storage/xtradb/win-plugin/win-plugin.diff' --- a/storage/xtradb/win-plugin/win-plugin.diff 2009-09-07 10:22:53 +0000 +++ b/storage/xtradb/win-plugin/win-plugin.diff 1970-01-01 00:00:00 +0000 @@ -1,279 +0,0 @@ -diff -Nur CMakeLists.txt.orig CMakeLists.txt ---- CMakeLists.txt.orig 2008-10-03 12:25:41 -05:00 -+++ CMakeLists.txt 2008-09-26 17:32:51 -05:00 -@@ -254,9 +254,9 @@ - IF(WITH_FEDERATED_STORAGE_ENGINE) - ADD_SUBDIRECTORY(storage/federated) - ENDIF(WITH_FEDERATED_STORAGE_ENGINE) --IF(WITH_INNOBASE_STORAGE_ENGINE) -+IF(WITH_INNOBASE_STORAGE_ENGINE OR INNODB_DYNAMIC_PLUGIN) - ADD_SUBDIRECTORY(storage/innobase) --ENDIF(WITH_INNOBASE_STORAGE_ENGINE) -+ENDIF(WITH_INNOBASE_STORAGE_ENGINE OR INNODB_DYNAMIC_PLUGIN) - ADD_SUBDIRECTORY(sql) - ADD_SUBDIRECTORY(server-tools/instance-manager) - ADD_SUBDIRECTORY(libmysql) - -diff -Nur sql/CMakeLists.txt.orig sql/CMakeLists.txt ---- sql/CMakeLists.txt.orig 2008-10-03 12:25:41 -05:00 -+++ sql/CMakeLists.txt 2008-09-24 03:58:19 -05:00 -@@ -98,6 +98,15 @@ - LINK_FLAGS "/PDB:${CMAKE_CFG_INTDIR}/mysqld${MYSQLD_EXE_SUFFIX}.pdb") - ENDIF(cmake_version EQUAL 20406) - -+# Checks for 64-bit version -+IF(CMAKE_SIZEOF_VOID_P MATCHES 8) -+SET_TARGET_PROPERTIES(mysqld PROPERTIES -+ LINK_FLAGS "/def:\"${PROJECT_SOURCE_DIR}/sql/mysqld_x64.def\"") -+ELSE(CMAKE_SIZEOF_VOID_P MATCHES 8) -+SET_TARGET_PROPERTIES(mysqld PROPERTIES -+ LINK_FLAGS "/def:\"${PROJECT_SOURCE_DIR}/sql/mysqld.def\"") -+ENDIF(CMAKE_SIZEOF_VOID_P MATCHES 8) -+ - IF(EMBED_MANIFESTS) - MYSQL_EMBED_MANIFEST("mysqld" "asInvoker") - ENDIF(EMBED_MANIFESTS) - -diff -Nur sql/mysqld.def.orig sql/mysqld.def ---- sql/mysqld.def.orig 1969-12-31 18:00:00 -06:00 -+++ sql/mysqld.def 2009-04-09 02:20:32 -05:00 -@@ -0,0 +1,111 @@ -+EXPORTS -+ ?use_hidden_primary_key@handler@@UAEXXZ -+ ?get_dynamic_partition_info@handler@@UAEXPAUPARTITION_INFO@@I@Z -+ ?read_first_row@handler@@UAEHPAEI@Z -+ ?read_range_next@handler@@UAEHXZ -+ ?read_range_first@handler@@UAEHPBUst_key_range@@0_N1@Z -+ ?read_multi_range_first@handler@@UAEHPAPAUst_key_multi_range@@PAU2@I_NPAUst_handler_buffer@@@Z -+ ?read_multi_range_next@handler@@UAEHPAPAUst_key_multi_range@@@Z -+ ?index_read_idx_map@handler@@UAEHPAEIPBEKW4ha_rkey_function@@@Z -+ ?print_error@handler@@UAEXHH@Z -+ ?clone@handler@@UAEPAV1@PAUst_mem_root@@@Z -+ ?get_auto_increment@handler@@UAEX_K00PA_K1@Z -+ ?index_next_same@handler@@UAEHPAEPBEI@Z -+ ?get_error_message@handler@@UAE_NHPAVString@@@Z -+ ?ha_thd@handler@@IBEPAVTHD@@XZ -+ ?update_auto_increment@handler@@QAEHXZ -+ ?ha_statistic_increment@handler@@IBEXPQsystem_status_var@@K@Z -+ ?trans_register_ha@@YAXPAVTHD@@_NPAUhandlerton@@@Z -+ ?cmp@Field_blob@@QAEHPBEI0I@Z -+ ?set_time@Field_timestamp@@QAEXXZ -+ ?sql_print_error@@YAXPBDZZ -+ ?sql_print_warning@@YAXPBDZZ -+ ?check_global_access@@YA_NPAVTHD@@K@Z -+ ?schema_table_store_record@@YA_NPAVTHD@@PAUst_table@@@Z -+ ?get_quote_char_for_identifier@@YAHPAVTHD@@PBDI@Z -+ ?copy@String@@QAE_NXZ -+ ?copy@String@@QAE_NABV1@@Z -+ ?copy@String@@QAE_NPBDIPAUcharset_info_st@@@Z -+ ?copy_and_convert@@YAIPADIPAUcharset_info_st@@PBDI1PAI@Z -+ ?filename_to_tablename@@YAIPBDPADI@Z -+ ?strconvert@@YAIPAUcharset_info_st@@PBD0PADIPAI@Z -+ ?calculate_key_len@@YAIPAUst_table@@IPBEK@Z -+ ?sql_alloc@@YAPAXI@Z -+ ?localtime_to_TIME@@YAXPAUst_mysql_time@@PAUtm@@@Z -+ ?push_warning@@YAPAVMYSQL_ERROR@@PAVTHD@@W4enum_warning_level@1@IPBD@Z -+ ?push_warning_printf@@YAXPAVTHD@@W4enum_warning_level@MYSQL_ERROR@@IPBDZZ -+ ?drop_table@handler@@EAEXPBD@Z -+ ?column_bitmaps_signal@handler@@UAEXXZ -+ ?delete_table@handler@@MAEHPBD@Z -+ ?rename_table@handler@@MAEHPBD0@Z -+ ?key_map_empty@@3V?$Bitmap@$0EA@@@B -+ ?THR_THD@@3PAVTHD@@A -+ ?end_of_list@@3Ulist_node@@A -+ ?mysql_tmpdir_list@@3Ust_my_tmpdir@@A -+ mysql_query_cache_invalidate4 -+ thd_query -+ thd_sql_command -+ thd_get_thread_id -+ thd_get_xid -+ thd_slave_thread -+ thd_non_transactional_update -+ thd_mark_transaction_to_rollback -+ thd_security_context -+ thd_charset -+ thd_test_options -+ thd_ha_data -+ thd_killed -+ thd_tx_isolation -+ thd_tablespace_op -+ thd_sql_command -+ thd_memdup -+ thd_make_lex_string -+ thd_in_lock_tables -+ thd_binlog_format -+ _my_hash_init -+ my_hash_free -+ my_tmpdir -+ check_if_legal_filename -+ my_filename -+ my_sync_dir_by_file -+ alloc_root -+ thr_lock_data_init -+ thr_lock_init -+ thr_lock_delete -+ my_multi_malloc -+ get_charset -+ unpack_filename -+ my_hash_insert -+ my_hash_search -+ my_hash_delete -+ mysql_bin_log_file_pos -+ mysql_bin_log_file_name -+ mysqld_embedded -+ my_thread_name -+ my_malloc -+ my_no_flags_free -+ _sanity -+ _mymalloc -+ _myfree -+ _my_strdup -+ _my_thread_var -+ my_error -+ pthread_cond_init -+ pthread_cond_signal -+ pthread_cond_wait -+ pthread_cond_destroy -+ localtime_r -+ my_strdup -+ deflate -+ deflateEnd -+ deflateReset -+ deflateInit2_ -+ inflateEnd -+ inflateInit_ -+ inflate -+ compressBound -+ inflateInit2_ -+ adler32 -+ longlong2str -+ strend -+ my_snprintf - -diff -Nur sql/mysqld_x64.def.orig sql/mysqld_x64.def ---- sql/mysqld_x64.def.orig 1969-12-31 18:00:00 -06:00 -+++ sql/mysqld_x64.def 2009-04-09 02:22:04 -05:00 -@@ -0,0 +1,111 @@ -+EXPORTS -+ ?use_hidden_primary_key@handler@@UEAAXXZ -+ ?get_dynamic_partition_info@handler@@UEAAXPEAUPARTITION_INFO@@I@Z -+ ?read_first_row@handler@@UEAAHPEAEI@Z -+ ?read_range_next@handler@@UEAAHXZ -+ ?read_range_first@handler@@UEAAHPEBUst_key_range@@0_N1@Z -+ ?read_multi_range_first@handler@@UEAAHPEAPEAUst_key_multi_range@@PEAU2@I_NPEAUst_handler_buffer@@@Z -+ ?read_multi_range_next@handler@@UEAAHPEAPEAUst_key_multi_range@@@Z -+ ?index_read_idx_map@handler@@UEAAHPEAEIPEBEKW4ha_rkey_function@@@Z -+ ?print_error@handler@@UEAAXHH@Z -+ ?clone@handler@@UEAAPEAV1@PEAUst_mem_root@@@Z -+ ?get_auto_increment@handler@@UEAAX_K00PEA_K1@Z -+ ?index_next_same@handler@@UEAAHPEAEPEBEI@Z -+ ?get_error_message@handler@@UEAA_NHPEAVString@@@Z -+ ?ha_thd@handler@@IEBAPEAVTHD@@XZ -+ ?update_auto_increment@handler@@QEAAHXZ -+ ?ha_statistic_increment@handler@@IEBAXPEQsystem_status_var@@K@Z -+ ?trans_register_ha@@YAXPEAVTHD@@_NPEAUhandlerton@@@Z -+ ?cmp@Field_blob@@QEAAHPEBEI0I@Z -+ ?set_time@Field_timestamp@@QEAAXXZ -+ ?sql_print_error@@YAXPEBDZZ -+ ?sql_print_warning@@YAXPEBDZZ -+ ?check_global_access@@YA_NPEAVTHD@@K@Z -+ ?schema_table_store_record@@YA_NPEAVTHD@@PEAUst_table@@@Z -+ ?get_quote_char_for_identifier@@YAHPEAVTHD@@PEBDI@Z -+ ?copy@String@@QEAA_NXZ -+ ?copy@String@@QEAA_NAEBV1@@Z -+ ?copy@String@@QEAA_NPEBDIPEAUcharset_info_st@@@Z -+ ?copy_and_convert@@YAIPEADIPEAUcharset_info_st@@PEBDI1PEAI@Z -+ ?filename_to_tablename@@YAIPEBDPEADI@Z -+ ?strconvert@@YAIPEAUcharset_info_st@@PEBD0PEADIPEAI@Z -+ ?calculate_key_len@@YAIPEAUst_table@@IPEBEK@Z -+ ?sql_alloc@@YAPEAX_K@Z -+ ?localtime_to_TIME@@YAXPEAUst_mysql_time@@PEAUtm@@@Z -+ ?push_warning@@YAPEAVMYSQL_ERROR@@PEAVTHD@@W4enum_warning_level@1@IPEBD@Z -+ ?push_warning_printf@@YAXPEAVTHD@@W4enum_warning_level@MYSQL_ERROR@@IPEBDZZ -+ ?drop_table@handler@@EEAAXPEBD@Z -+ ?column_bitmaps_signal@handler@@UEAAXXZ -+ ?delete_table@handler@@MEAAHPEBD@Z -+ ?rename_table@handler@@MEAAHPEBD0@Z -+ ?key_map_empty@@3V?$Bitmap@$0EA@@@B -+ ?THR_THD@@3PEAVTHD@@EA -+ ?end_of_list@@3Ulist_node@@A -+ ?mysql_tmpdir_list@@3Ust_my_tmpdir@@A -+ mysql_query_cache_invalidate4 -+ thd_query -+ thd_sql_command -+ thd_get_thread_id -+ thd_get_xid -+ thd_slave_thread -+ thd_non_transactional_update -+ thd_mark_transaction_to_rollback -+ thd_security_context -+ thd_charset -+ thd_test_options -+ thd_ha_data -+ thd_killed -+ thd_tx_isolation -+ thd_tablespace_op -+ thd_sql_command -+ thd_memdup -+ thd_make_lex_string -+ thd_in_lock_tables -+ thd_binlog_format -+ _my_hash_init -+ my_hash_free -+ my_tmpdir -+ check_if_legal_filename -+ my_filename -+ my_sync_dir_by_file -+ alloc_root -+ thr_lock_data_init -+ thr_lock_init -+ thr_lock_delete -+ my_multi_malloc -+ get_charset -+ unpack_filename -+ my_hash_insert -+ my_hash_search -+ my_hash_delete -+ mysql_bin_log_file_pos -+ mysql_bin_log_file_name -+ mysqld_embedded -+ my_thread_name -+ my_malloc -+ my_no_flags_free -+ _sanity -+ _mymalloc -+ _myfree -+ _my_strdup -+ _my_thread_var -+ my_error -+ pthread_cond_init -+ pthread_cond_signal -+ pthread_cond_wait -+ pthread_cond_destroy -+ localtime_r -+ my_strdup -+ deflate -+ deflateEnd -+ deflateReset -+ deflateInit2_ -+ inflateEnd -+ inflateInit_ -+ inflate -+ compressBound -+ inflateInit2_ -+ adler32 -+ longlong2str -+ strend -+ my_snprintf - -diff -Nur win/configure.js.orig win/configure.js ---- win/configure.js.orig 2008-09-26 21:18:37 -05:00 -+++ win/configure.js 2008-10-01 11:21:27 -05:00 -@@ -50,6 +50,7 @@ - case "EMBED_MANIFESTS": - case "EXTRA_DEBUG": - case "WITH_EMBEDDED_SERVER": -+ case "INNODB_DYNAMIC_PLUGIN": - configfile.WriteLine("SET (" + args.Item(i) + " TRUE)"); - break; - case "MYSQL_SERVER_SUFFIX": === modified file 'vio/vio.c' --- a/vio/vio.c 2009-11-02 22:19:58 +0000 +++ b/vio/vio.c 2009-11-20 12:09:50 +0000 @@ -62,10 +62,8 @@ static void vio_init(Vio* vio, enum enum vio->timeout=vio_win32_timeout; /* Set default timeout */ - vio->read_timeout_millis = INFINITE; - vio->write_timeout_millis = INFINITE; - - memset(&(vio->pipe_overlapped), 0, sizeof(OVERLAPPED)); + vio->read_timeout_ms= INFINITE; + vio->write_timeout_ms= INFINITE; vio->pipe_overlapped.hEvent= CreateEvent(NULL, TRUE, FALSE, NULL); DBUG_VOID_RETURN; } @@ -90,8 +88,8 @@ static void vio_init(Vio* vio, enum enum /* Currently, shared memory is on Windows only, hence the below is ok*/ vio->timeout= vio_win32_timeout; /* Set default timeout */ - vio->read_timeout_millis= INFINITE; - vio->write_timeout_millis= INFINITE; + vio->read_timeout_ms= INFINITE; + vio->write_timeout_ms= INFINITE; DBUG_VOID_RETURN; } #endif @@ -115,22 +113,20 @@ static void vio_init(Vio* vio, enum enum DBUG_VOID_RETURN; } #endif /* HAVE_OPENSSL */ - { - vio->viodelete =vio_delete; - vio->vioerrno =vio_errno; - vio->read= (flags & VIO_BUFFERED_READ) ? vio_read_buff : vio_read; - vio->write =vio_write; - vio->fastsend =vio_fastsend; - vio->viokeepalive =vio_keepalive; - vio->should_retry =vio_should_retry; - vio->was_interrupted=vio_was_interrupted; - vio->vioclose =vio_close; - vio->peer_addr =vio_peer_addr; - vio->in_addr =vio_in_addr; - vio->vioblocking =vio_blocking; - vio->is_blocking =vio_is_blocking; - vio->timeout =vio_timeout; - } + vio->viodelete =vio_delete; + vio->vioerrno =vio_errno; + vio->read= (flags & VIO_BUFFERED_READ) ? vio_read_buff : vio_read; + vio->write =vio_write; + vio->fastsend =vio_fastsend; + vio->viokeepalive =vio_keepalive; + vio->should_retry =vio_should_retry; + vio->was_interrupted=vio_was_interrupted; + vio->vioclose =vio_close; + vio->peer_addr =vio_peer_addr; + vio->in_addr =vio_in_addr; + vio->vioblocking =vio_blocking; + vio->is_blocking =vio_is_blocking; + vio->timeout =vio_timeout; DBUG_VOID_RETURN; } === modified file 'vio/viosocket.c' --- a/vio/viosocket.c 2009-12-03 11:19:05 +0000 +++ b/vio/viosocket.c 2010-01-15 15:27:55 +0000 @@ -428,14 +428,14 @@ void vio_timeout(Vio *vio, uint which, u /* Finish pending IO on pipe. Honor wait timeout */ -static int pipe_complete_io(Vio* vio, char* buf, size_t size, DWORD timeout_millis) +static size_t pipe_complete_io(Vio* vio, char* buf, size_t size, DWORD timeout_ms) { DWORD length; DWORD ret; DBUG_ENTER("pipe_complete_io"); - ret= WaitForSingleObject(vio->pipe_overlapped.hEvent, timeout_millis); + ret= WaitForSingleObject(vio->pipe_overlapped.hEvent, timeout_ms); /* WaitForSingleObjects will normally return WAIT_OBJECT_O (success, IO completed) or WAIT_TIMEOUT. @@ -444,14 +444,14 @@ static int pipe_complete_io(Vio* vio, ch { CancelIo(vio->hPipe); DBUG_PRINT("error",("WaitForSingleObject() returned %d", ret)); - DBUG_RETURN(-1); + DBUG_RETURN((size_t)-1); } if (!GetOverlappedResult(vio->hPipe,&(vio->pipe_overlapped),&length, FALSE)) { DBUG_PRINT("error",("GetOverlappedResult() returned last error %d", GetLastError())); - DBUG_RETURN(-1); + DBUG_RETURN((size_t)-1); } DBUG_RETURN(length); @@ -461,49 +461,58 @@ static int pipe_complete_io(Vio* vio, ch size_t vio_read_pipe(Vio * vio, uchar *buf, size_t size) { DWORD bytes_read; + size_t retval; DBUG_ENTER("vio_read_pipe"); DBUG_PRINT("enter", ("sd: %d buf: 0x%lx size: %u", vio->sd, (long) buf, (uint) size)); - if (!ReadFile(vio->hPipe, buf, (DWORD)size, &bytes_read, + if (ReadFile(vio->hPipe, buf, (DWORD)size, &bytes_read, &(vio->pipe_overlapped))) { + retval= bytes_read; + } + else + { if (GetLastError() != ERROR_IO_PENDING) { DBUG_PRINT("error",("ReadFile() returned last error %d", GetLastError())); DBUG_RETURN((size_t)-1); } - bytes_read= pipe_complete_io(vio, buf, size,vio->read_timeout_millis); + retval= pipe_complete_io(vio, buf, size,vio->read_timeout_ms); } - DBUG_PRINT("exit", ("%d", bytes_read)); - DBUG_RETURN(bytes_read); + DBUG_PRINT("exit", ("%lld", (longlong)retval)); + DBUG_RETURN(retval); } size_t vio_write_pipe(Vio * vio, const uchar* buf, size_t size) { DWORD bytes_written; + size_t retval; DBUG_ENTER("vio_write_pipe"); DBUG_PRINT("enter", ("sd: %d buf: 0x%lx size: %u", vio->sd, (long) buf, (uint) size)); - if (!WriteFile(vio->hPipe, buf, (DWORD)size, &bytes_written, + if (WriteFile(vio->hPipe, buf, (DWORD)size, &bytes_written, &(vio->pipe_overlapped))) { + retval= bytes_written; + } + else + { if (GetLastError() != ERROR_IO_PENDING) { DBUG_PRINT("vio_error",("WriteFile() returned last error %d", GetLastError())); DBUG_RETURN((size_t)-1); } - bytes_written = pipe_complete_io(vio, (char *)buf, size, - vio->write_timeout_millis); + retval= pipe_complete_io(vio, (char *)buf, size, vio->write_timeout_ms); } - DBUG_PRINT("exit", ("%d", bytes_written)); - DBUG_RETURN(bytes_written); + DBUG_PRINT("exit", ("%lld", (longlong)retval)); + DBUG_RETURN(retval); } @@ -528,21 +537,21 @@ int vio_close_pipe(Vio * vio) void vio_win32_timeout(Vio *vio, uint which , uint timeout_sec) { - DWORD timeout_millis; + DWORD timeout_ms; /* Windows is measuring timeouts in milliseconds. Check for possible int overflow. */ if (timeout_sec > UINT_MAX/1000) - timeout_millis= INFINITE; + timeout_ms= INFINITE; else - timeout_millis= timeout_sec * 1000; + timeout_ms= timeout_sec * 1000; /* which == 1 means "write", which == 0 means "read".*/ if(which) - vio->write_timeout_millis= timeout_millis; + vio->write_timeout_ms= timeout_ms; else - vio->read_timeout_millis= timeout_millis; + vio->read_timeout_ms= timeout_ms; } @@ -577,7 +586,7 @@ size_t vio_read_shared_memory(Vio * vio, WAIT_ABANDONED_0 and WAIT_TIMEOUT - fail. We can't read anything */ if (WaitForMultipleObjects(array_elements(events), events, FALSE, - vio->read_timeout_millis) != WAIT_OBJECT_0) + vio->read_timeout_ms) != WAIT_OBJECT_0) { DBUG_RETURN(-1); }; @@ -634,7 +643,7 @@ size_t vio_write_shared_memory(Vio * vio while (remain != 0) { if (WaitForMultipleObjects(array_elements(events), events, FALSE, - vio->write_timeout_millis) != WAIT_OBJECT_0) + vio->write_timeout_ms) != WAIT_OBJECT_0) { DBUG_RETURN((size_t) -1); }

1 0

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (knielsen:2802)
by knielsen＠knielsen-hq.org 19 Jan '10

19 Jan '10

#At lp:maria 2802 knielsen(a)knielsen-hq.org 2010-01-19 Fix Windows test failures in binlog tests in certain time zones. On Unix systems TZ can be set to change the time zone for specific processes only. But on Windows this does not fully work. It changes some aspects of time zones in the system but not others (notably localtime() vs. file system time stamps). This causes test failures in Windows in certain time zones (but not all), where PURGE BEFORE DATE statements get the wrong files when TZ is set to +03:00 by default. Fix by only setting TZ in the small number of tests that really need it, and leave it untouched in the rest. Thanks to Alex Budovski for helping with this. added: mysql-test/t/mysqlbinlog2-master.opt modified: mysql-test/include/ps_conv.inc mysql-test/mysql-test-run.pl mysql-test/suite/binlog/r/binlog_unsafe.result mysql-test/suite/binlog/t/binlog_unsafe.test mysql-test/suite/rpl/r/rpl_mixed_ddl_dml.result mysql-test/suite/rpl/t/rpl_mixed_ddl_dml.test per-file messages: mysql-test/include/ps_conv.inc Explicitly set TIME_ZONE, as mysql-test-run.pl now does not set it. mysql-test/mysql-test-run.pl Do not set TZ by default, as it causes problems on Windows. mysql-test/suite/binlog/r/binlog_unsafe.result Explicitly set TIME_ZONE, as mysql-test-run.pl now does not set it. mysql-test/suite/binlog/t/binlog_unsafe.test Explicitly set TIME_ZONE, as mysql-test-run.pl now does not set it. mysql-test/suite/rpl/r/rpl_mixed_ddl_dml.result Explicitly set TIME_ZONE, as mysql-test-run.pl now does not set it. mysql-test/suite/rpl/t/rpl_mixed_ddl_dml.test Explicitly set TIME_ZONE, as mysql-test-run.pl now does not set it. mysql-test/t/mysqlbinlog2-master.opt mysql-test-run.pl no longer sets TZ by default, so set it explicitly for this particular test. === modified file 'mysql-test/include/ps_conv.inc' --- a/mysql-test/include/ps_conv.inc 2007-07-04 09:20:53 +0000 +++ b/mysql-test/include/ps_conv.inc 2010-01-19 10:36:52 +0000 @@ -30,6 +30,9 @@ --disable_warnings drop table if exists t5 ; --enable_warnings +--disable_query_log +SET TIME_ZONE= '+03:00'; +--enable_query_log set @arg01= 8; set @arg02= 8.0; set @arg03= 80.00000000000e-1; === modified file 'mysql-test/mysql-test-run.pl' --- a/mysql-test/mysql-test-run.pl 2010-01-06 21:27:53 +0000 +++ b/mysql-test/mysql-test-run.pl 2010-01-19 10:36:52 +0000 @@ -3425,7 +3425,7 @@ sub restart_forced_by_test # Return timezone value of tinfo or default value sub timezone { my ($tinfo)= @_; - return $tinfo->{timezone} || "GMT-3"; + return $tinfo->{timezone} || "DEFAULT"; } @@ -3455,7 +3455,9 @@ sub run_testcase ($$) { # Init variables that can change between each test case # ------------------------------------------------------- my $timezone= timezone($tinfo); - $ENV{'TZ'}= $timezone; + if ($timezone ne 'DEFAULT') { + $ENV{'TZ'}= $timezone; + } mtr_verbose("Setting timezone: $timezone"); if ( ! using_extern() ) === modified file 'mysql-test/suite/binlog/r/binlog_unsafe.result' --- a/mysql-test/suite/binlog/r/binlog_unsafe.result 2010-01-15 15:27:55 +0000 +++ b/mysql-test/suite/binlog/r/binlog_unsafe.result 2010-01-19 10:36:52 +0000 @@ -380,6 +380,7 @@ INSERT INTO t1 VALUES (VERSION()); Warnings: Note 1592 Statement may not be safe to log in statement format. DELETE FROM t1; +SET TIME_ZONE= '+03:00'; SET TIMESTAMP=1000000; INSERT INTO t1 VALUES (CURDATE()), === modified file 'mysql-test/suite/binlog/t/binlog_unsafe.test' --- a/mysql-test/suite/binlog/t/binlog_unsafe.test 2010-01-15 15:27:55 +0000 +++ b/mysql-test/suite/binlog/t/binlog_unsafe.test 2010-01-19 10:36:52 +0000 @@ -422,6 +422,7 @@ DELETE FROM t1; # following following functions depend on the TIMESTAMP variable and # don't generate a warning. +SET TIME_ZONE= '+03:00'; SET TIMESTAMP=1000000; INSERT INTO t1 VALUES (CURDATE()), === modified file 'mysql-test/suite/rpl/r/rpl_mixed_ddl_dml.result' --- a/mysql-test/suite/rpl/r/rpl_mixed_ddl_dml.result 2008-11-04 17:07:14 +0000 +++ b/mysql-test/suite/rpl/r/rpl_mixed_ddl_dml.result 2010-01-19 10:36:52 +0000 @@ -18,6 +18,7 @@ Server_id Host Port Rpl_recovery_rank Ma drop table t1; stop slave; create table t2(id int auto_increment primary key, created datetime); +SET TIME_ZONE= '+03:00'; set timestamp=12345; insert into t2 set created=now(); select * from t2; === modified file 'mysql-test/suite/rpl/t/rpl_mixed_ddl_dml.test' --- a/mysql-test/suite/rpl/t/rpl_mixed_ddl_dml.test 2007-06-27 12:28:02 +0000 +++ b/mysql-test/suite/rpl/t/rpl_mixed_ddl_dml.test 2010-01-19 10:36:52 +0000 @@ -23,6 +23,7 @@ connection master; # Test replication of timestamp create table t2(id int auto_increment primary key, created datetime); +SET TIME_ZONE= '+03:00'; set timestamp=12345; insert into t2 set created=now(); select * from t2; === added file 'mysql-test/t/mysqlbinlog2-master.opt' --- a/mysql-test/t/mysqlbinlog2-master.opt 1970-01-01 00:00:00 +0000 +++ b/mysql-test/t/mysqlbinlog2-master.opt 2010-01-19 10:36:52 +0000 @@ -0,0 +1 @@ +--timezone=GMT-3

1 0

[Maria-developers] Any documentation on FederatedX
by Sergey Petrunya 18 Jan '10

18 Jan '10

Hi! It has been some time since FederatedX has been pushed into MariaDB. However, the documentation page here http://askmonty.org/wiki/index.php/Manual:FederatedX_storage_engine still has nothing. I also wasn't able to find any information about FederatedX anywhere else. Maybe it's only me, but for me it seems that there was little point in pushing this feature if we don't have even the basic documentation. How can we expect somebody to use it? Patrick, could you point us to information about: - Whether FederatedX is compatible with Federated, in particular, whether the information here http://dev.mysql.com/doc/refman/5.0/en/federated-storage-engine.html is applicable to FederateX - What are the differences of FederatedX from Federated (from user's point of view). Thanks in advance, Sergey -- Sergey Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

2 2

[Maria-developers] Updated (by Guest): options for CREATE TABLE (43)
by worklog-noreply＠askmonty.org 18 Jan '10

18 Jan '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: options for CREATE TABLE CREATION DATE..: Tue, 11 Aug 2009, 17:02 SUPERVISOR.....: Bothorsen IMPLEMENTOR....: Sanja COPIES TO......: Sergei, Monty CATEGORY.......: Server-BackLog TASK ID........: 43 (http://askmonty.org/worklog/?tid=43) VERSION........: Server-5.2 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 32 (hours remain) ORIG. ESTIMATE.: 32 PROGRESS NOTES: -=-=(Sergei - Mon, 18 Jan 2010, 22:12)=-=- Observers changed: Monty,Sergei -=-=(Monty - Tue, 12 Jan 2010, 17:04)=-=- Version updated. --- /tmp/wklog.43.old.21802 2010-01-12 17:04:31.000000000 +0200 +++ /tmp/wklog.43.new.21802 2010-01-12 17:04:31.000000000 +0200 @@ -1 +1 @@ -Server-5.1 +Server-5.2 -=-=(Guest - Fri, 18 Sep 2009, 14:24)=-=- Low Level Design modified. --- /tmp/wklog.43.old.31654 2009-09-18 14:24:41.000000000 +0300 +++ /tmp/wklog.43.new.31654 2009-09-18 14:24:41.000000000 +0300 @@ -2,4 +2,6 @@ Handler have access to the options during creation and after opening frm. +It should has C interface. + -=-=(Sanja - Fri, 18 Sep 2009, 13:32)=-=- Low Level Design modified. --- /tmp/wklog.43.old.29567 2009-09-18 13:32:41.000000000 +0300 +++ /tmp/wklog.43.new.29567 2009-09-18 13:32:41.000000000 +0300 @@ -1 +1,5 @@ +Options stored in a list as pairs key, value which are LEX_STRING. + +Handler have access to the options during creation and after opening frm. + -=-=(Sanja - Fri, 18 Sep 2009, 13:21)=-=- High-Level Specification modified. --- /tmp/wklog.43.old.29107 2009-09-18 13:21:19.000000000 +0300 +++ /tmp/wklog.43.new.29107 2009-09-18 13:21:19.000000000 +0300 @@ -6,7 +6,8 @@ table_option1=tval1, table_option2=tval2; Exclusion should be made for old table and key options where -'=' was not obligatory. +'=' was not obligatory. Behaviour and way of storage for existing options will +be left as is. Old key options: KEY_BLOCK_SIZE <num> -> KEY_BLOCK_SIZE=num @@ -42,6 +43,6 @@ For fields options can go with field attributes (NOT NULL, UNIQUE and so on) can be separated from them by '=' sign. - - +Incorrect option during creation/altering table lead only to worning. During +opening unrecognised options should be ignored. -=-=(Guest - Tue, 11 Aug 2009, 19:57)=-=- High-Level Specification modified. --- /tmp/wklog.43.old.31856 2009-08-11 19:57:38.000000000 +0300 +++ /tmp/wklog.43.new.31856 2009-08-11 19:57:38.000000000 +0300 @@ -5,8 +5,43 @@ key key1(field) key_opt1=kval1 key_opt2=kval2) table_option1=tval1, table_option2=tval2; -Exclusion should be made for old table and key (KEY_BLOCK_SIZE) options where +Exclusion should be made for old table and key options where '=' was not obligatory. +Old key options: +KEY_BLOCK_SIZE <num> -> KEY_BLOCK_SIZE=num +WITH PARSER <name> -> PARSER=name + +Old table options: +ENGINE name -> ENGINE=name +TYPE name -> TYPE=name +MAX_ROWS num -> MAX_ROWS=num +MIX_ROWS num -> MIX_ROWS=num +AVG_ROW_LENGTH num -> AVG_ROW_LENGTH=num +PASSWORD string -> PASSWORD=string +COMMENT string -> COMMENT=string +AUTO_INCREMENT num -> AUTO_INCREMENT=num +PACK_KEYS num/default -> PACK_KEYS=num/default +CHECKSUM num -> CHECKSUM=num +TABLE_CHECKSUM num -> TABLE_CHECKSUM=num +PAGE_CHECKSUM num -> PAGE_CHECKSUM=num +DELAY_KEY_WRITE num -> DELAY_KEY_WRITE=num +ROW_FORMAT name -> ROW_FORMAT=name +INSERT_METHOD name -> INSERT_METHOD=name +KEY_BLOCK_SIZE num -> KEY_BLOCK_SIZE=num +TRANSACTIONAL num -> TRANSACTIONAL=num + +Table options which will be left hardcoded +UNION +default charset +default collation +DATA DIRECTORY +TABLESPACE +STORAGE + For fields options can go with field attributes (NOT NULL, UNIQUE and so on) can be separated from them by '=' sign. + + + + -=-=(Guest - Tue, 11 Aug 2009, 19:36)=-=- High-Level Specification modified. --- /tmp/wklog.43.old.30883 2009-08-11 19:36:45.000000000 +0300 +++ /tmp/wklog.43.new.30883 2009-08-11 19:36:45.000000000 +0300 @@ -1 +1,12 @@ +Table definition ca looks like following +CREATE TABLE table + (field int ... field_opt1=fval1 field_opt2=fval2, + key key1(field) key_opt1=kval1 key_opt2=kval2) + table_option1=tval1, table_option2=tval2; + +Exclusion should be made for old table and key (KEY_BLOCK_SIZE) options where +'=' was not obligatory. + +For fields options can go with field attributes (NOT NULL, UNIQUE and so on) can +be separated from them by '=' sign. DESCRIPTION: Add ability to create table with additional option which can be passed to engine. Also make current options such as TRANSACTIONAL working via this mechanism. HIGH-LEVEL SPECIFICATION: Table definition ca looks like following CREATE TABLE table (field int ... field_opt1=fval1 field_opt2=fval2, key key1(field) key_opt1=kval1 key_opt2=kval2) table_option1=tval1, table_option2=tval2; Exclusion should be made for old table and key options where '=' was not obligatory. Behaviour and way of storage for existing options will be left as is. Old key options: KEY_BLOCK_SIZE <num> -> KEY_BLOCK_SIZE=num WITH PARSER <name> -> PARSER=name Old table options: ENGINE name -> ENGINE=name TYPE name -> TYPE=name MAX_ROWS num -> MAX_ROWS=num MIX_ROWS num -> MIX_ROWS=num AVG_ROW_LENGTH num -> AVG_ROW_LENGTH=num PASSWORD string -> PASSWORD=string COMMENT string -> COMMENT=string AUTO_INCREMENT num -> AUTO_INCREMENT=num PACK_KEYS num/default -> PACK_KEYS=num/default CHECKSUM num -> CHECKSUM=num TABLE_CHECKSUM num -> TABLE_CHECKSUM=num PAGE_CHECKSUM num -> PAGE_CHECKSUM=num DELAY_KEY_WRITE num -> DELAY_KEY_WRITE=num ROW_FORMAT name -> ROW_FORMAT=name INSERT_METHOD name -> INSERT_METHOD=name KEY_BLOCK_SIZE num -> KEY_BLOCK_SIZE=num TRANSACTIONAL num -> TRANSACTIONAL=num Table options which will be left hardcoded UNION default charset default collation DATA DIRECTORY TABLESPACE STORAGE For fields options can go with field attributes (NOT NULL, UNIQUE and so on) can be separated from them by '=' sign. Incorrect option during creation/altering table lead only to worning. During opening unrecognised options should be ignored. LOW-LEVEL DESIGN: Options stored in a list as pairs key, value which are LEX_STRING. Handler have access to the options during creation and after opening frm. It should has C interface. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Guest): options for CREATE TABLE (43)
by worklog-noreply＠askmonty.org 18 Jan '10

18 Jan '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: options for CREATE TABLE CREATION DATE..: Tue, 11 Aug 2009, 17:02 SUPERVISOR.....: Bothorsen IMPLEMENTOR....: Sanja COPIES TO......: Sergei, Monty CATEGORY.......: Server-BackLog TASK ID........: 43 (http://askmonty.org/worklog/?tid=43) VERSION........: Server-5.2 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 32 (hours remain) ORIG. ESTIMATE.: 32 PROGRESS NOTES: -=-=(Sergei - Mon, 18 Jan 2010, 22:12)=-=- Observers changed: Monty,Sergei -=-=(Monty - Tue, 12 Jan 2010, 17:04)=-=- Version updated. --- /tmp/wklog.43.old.21802 2010-01-12 17:04:31.000000000 +0200 +++ /tmp/wklog.43.new.21802 2010-01-12 17:04:31.000000000 +0200 @@ -1 +1 @@ -Server-5.1 +Server-5.2 -=-=(Guest - Fri, 18 Sep 2009, 14:24)=-=- Low Level Design modified. --- /tmp/wklog.43.old.31654 2009-09-18 14:24:41.000000000 +0300 +++ /tmp/wklog.43.new.31654 2009-09-18 14:24:41.000000000 +0300 @@ -2,4 +2,6 @@ Handler have access to the options during creation and after opening frm. +It should has C interface. + -=-=(Sanja - Fri, 18 Sep 2009, 13:32)=-=- Low Level Design modified. --- /tmp/wklog.43.old.29567 2009-09-18 13:32:41.000000000 +0300 +++ /tmp/wklog.43.new.29567 2009-09-18 13:32:41.000000000 +0300 @@ -1 +1,5 @@ +Options stored in a list as pairs key, value which are LEX_STRING. + +Handler have access to the options during creation and after opening frm. + -=-=(Sanja - Fri, 18 Sep 2009, 13:21)=-=- High-Level Specification modified. --- /tmp/wklog.43.old.29107 2009-09-18 13:21:19.000000000 +0300 +++ /tmp/wklog.43.new.29107 2009-09-18 13:21:19.000000000 +0300 @@ -6,7 +6,8 @@ table_option1=tval1, table_option2=tval2; Exclusion should be made for old table and key options where -'=' was not obligatory. +'=' was not obligatory. Behaviour and way of storage for existing options will +be left as is. Old key options: KEY_BLOCK_SIZE <num> -> KEY_BLOCK_SIZE=num @@ -42,6 +43,6 @@ For fields options can go with field attributes (NOT NULL, UNIQUE and so on) can be separated from them by '=' sign. - - +Incorrect option during creation/altering table lead only to worning. During +opening unrecognised options should be ignored. -=-=(Guest - Tue, 11 Aug 2009, 19:57)=-=- High-Level Specification modified. --- /tmp/wklog.43.old.31856 2009-08-11 19:57:38.000000000 +0300 +++ /tmp/wklog.43.new.31856 2009-08-11 19:57:38.000000000 +0300 @@ -5,8 +5,43 @@ key key1(field) key_opt1=kval1 key_opt2=kval2) table_option1=tval1, table_option2=tval2; -Exclusion should be made for old table and key (KEY_BLOCK_SIZE) options where +Exclusion should be made for old table and key options where '=' was not obligatory. +Old key options: +KEY_BLOCK_SIZE <num> -> KEY_BLOCK_SIZE=num +WITH PARSER <name> -> PARSER=name + +Old table options: +ENGINE name -> ENGINE=name +TYPE name -> TYPE=name +MAX_ROWS num -> MAX_ROWS=num +MIX_ROWS num -> MIX_ROWS=num +AVG_ROW_LENGTH num -> AVG_ROW_LENGTH=num +PASSWORD string -> PASSWORD=string +COMMENT string -> COMMENT=string +AUTO_INCREMENT num -> AUTO_INCREMENT=num +PACK_KEYS num/default -> PACK_KEYS=num/default +CHECKSUM num -> CHECKSUM=num +TABLE_CHECKSUM num -> TABLE_CHECKSUM=num +PAGE_CHECKSUM num -> PAGE_CHECKSUM=num +DELAY_KEY_WRITE num -> DELAY_KEY_WRITE=num +ROW_FORMAT name -> ROW_FORMAT=name +INSERT_METHOD name -> INSERT_METHOD=name +KEY_BLOCK_SIZE num -> KEY_BLOCK_SIZE=num +TRANSACTIONAL num -> TRANSACTIONAL=num + +Table options which will be left hardcoded +UNION +default charset +default collation +DATA DIRECTORY +TABLESPACE +STORAGE + For fields options can go with field attributes (NOT NULL, UNIQUE and so on) can be separated from them by '=' sign. + + + + -=-=(Guest - Tue, 11 Aug 2009, 19:36)=-=- High-Level Specification modified. --- /tmp/wklog.43.old.30883 2009-08-11 19:36:45.000000000 +0300 +++ /tmp/wklog.43.new.30883 2009-08-11 19:36:45.000000000 +0300 @@ -1 +1,12 @@ +Table definition ca looks like following +CREATE TABLE table + (field int ... field_opt1=fval1 field_opt2=fval2, + key key1(field) key_opt1=kval1 key_opt2=kval2) + table_option1=tval1, table_option2=tval2; + +Exclusion should be made for old table and key (KEY_BLOCK_SIZE) options where +'=' was not obligatory. + +For fields options can go with field attributes (NOT NULL, UNIQUE and so on) can +be separated from them by '=' sign. DESCRIPTION: Add ability to create table with additional option which can be passed to engine. Also make current options such as TRANSACTIONAL working via this mechanism. HIGH-LEVEL SPECIFICATION: Table definition ca looks like following CREATE TABLE table (field int ... field_opt1=fval1 field_opt2=fval2, key key1(field) key_opt1=kval1 key_opt2=kval2) table_option1=tval1, table_option2=tval2; Exclusion should be made for old table and key options where '=' was not obligatory. Behaviour and way of storage for existing options will be left as is. Old key options: KEY_BLOCK_SIZE <num> -> KEY_BLOCK_SIZE=num WITH PARSER <name> -> PARSER=name Old table options: ENGINE name -> ENGINE=name TYPE name -> TYPE=name MAX_ROWS num -> MAX_ROWS=num MIX_ROWS num -> MIX_ROWS=num AVG_ROW_LENGTH num -> AVG_ROW_LENGTH=num PASSWORD string -> PASSWORD=string COMMENT string -> COMMENT=string AUTO_INCREMENT num -> AUTO_INCREMENT=num PACK_KEYS num/default -> PACK_KEYS=num/default CHECKSUM num -> CHECKSUM=num TABLE_CHECKSUM num -> TABLE_CHECKSUM=num PAGE_CHECKSUM num -> PAGE_CHECKSUM=num DELAY_KEY_WRITE num -> DELAY_KEY_WRITE=num ROW_FORMAT name -> ROW_FORMAT=name INSERT_METHOD name -> INSERT_METHOD=name KEY_BLOCK_SIZE num -> KEY_BLOCK_SIZE=num TRANSACTIONAL num -> TRANSACTIONAL=num Table options which will be left hardcoded UNION default charset default collation DATA DIRECTORY TABLESPACE STORAGE For fields options can go with field attributes (NOT NULL, UNIQUE and so on) can be separated from them by '=' sign. Incorrect option during creation/altering table lead only to worning. During opening unrecognised options should be ignored. LOW-LEVEL DESIGN: Options stored in a list as pairs key, value which are LEX_STRING. Handler have access to the options during creation and after opening frm. It should has C interface. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Guest): options for CREATE TABLE (43)
by worklog-noreply＠askmonty.org 18 Jan '10

18 Jan '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: options for CREATE TABLE CREATION DATE..: Tue, 11 Aug 2009, 17:02 SUPERVISOR.....: Bothorsen IMPLEMENTOR....: Sanja COPIES TO......: Sergei, Monty CATEGORY.......: Server-BackLog TASK ID........: 43 (http://askmonty.org/worklog/?tid=43) VERSION........: Server-5.2 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 32 (hours remain) ORIG. ESTIMATE.: 32 PROGRESS NOTES: -=-=(Sergei - Mon, 18 Jan 2010, 22:12)=-=- Observers changed: Monty,Sergei -=-=(Monty - Tue, 12 Jan 2010, 17:04)=-=- Version updated. --- /tmp/wklog.43.old.21802 2010-01-12 17:04:31.000000000 +0200 +++ /tmp/wklog.43.new.21802 2010-01-12 17:04:31.000000000 +0200 @@ -1 +1 @@ -Server-5.1 +Server-5.2 -=-=(Guest - Fri, 18 Sep 2009, 14:24)=-=- Low Level Design modified. --- /tmp/wklog.43.old.31654 2009-09-18 14:24:41.000000000 +0300 +++ /tmp/wklog.43.new.31654 2009-09-18 14:24:41.000000000 +0300 @@ -2,4 +2,6 @@ Handler have access to the options during creation and after opening frm. +It should has C interface. + -=-=(Sanja - Fri, 18 Sep 2009, 13:32)=-=- Low Level Design modified. --- /tmp/wklog.43.old.29567 2009-09-18 13:32:41.000000000 +0300 +++ /tmp/wklog.43.new.29567 2009-09-18 13:32:41.000000000 +0300 @@ -1 +1,5 @@ +Options stored in a list as pairs key, value which are LEX_STRING. + +Handler have access to the options during creation and after opening frm. + -=-=(Sanja - Fri, 18 Sep 2009, 13:21)=-=- High-Level Specification modified. --- /tmp/wklog.43.old.29107 2009-09-18 13:21:19.000000000 +0300 +++ /tmp/wklog.43.new.29107 2009-09-18 13:21:19.000000000 +0300 @@ -6,7 +6,8 @@ table_option1=tval1, table_option2=tval2; Exclusion should be made for old table and key options where -'=' was not obligatory. +'=' was not obligatory. Behaviour and way of storage for existing options will +be left as is. Old key options: KEY_BLOCK_SIZE <num> -> KEY_BLOCK_SIZE=num @@ -42,6 +43,6 @@ For fields options can go with field attributes (NOT NULL, UNIQUE and so on) can be separated from them by '=' sign. - - +Incorrect option during creation/altering table lead only to worning. During +opening unrecognised options should be ignored. -=-=(Guest - Tue, 11 Aug 2009, 19:57)=-=- High-Level Specification modified. --- /tmp/wklog.43.old.31856 2009-08-11 19:57:38.000000000 +0300 +++ /tmp/wklog.43.new.31856 2009-08-11 19:57:38.000000000 +0300 @@ -5,8 +5,43 @@ key key1(field) key_opt1=kval1 key_opt2=kval2) table_option1=tval1, table_option2=tval2; -Exclusion should be made for old table and key (KEY_BLOCK_SIZE) options where +Exclusion should be made for old table and key options where '=' was not obligatory. +Old key options: +KEY_BLOCK_SIZE <num> -> KEY_BLOCK_SIZE=num +WITH PARSER <name> -> PARSER=name + +Old table options: +ENGINE name -> ENGINE=name +TYPE name -> TYPE=name +MAX_ROWS num -> MAX_ROWS=num +MIX_ROWS num -> MIX_ROWS=num +AVG_ROW_LENGTH num -> AVG_ROW_LENGTH=num +PASSWORD string -> PASSWORD=string +COMMENT string -> COMMENT=string +AUTO_INCREMENT num -> AUTO_INCREMENT=num +PACK_KEYS num/default -> PACK_KEYS=num/default +CHECKSUM num -> CHECKSUM=num +TABLE_CHECKSUM num -> TABLE_CHECKSUM=num +PAGE_CHECKSUM num -> PAGE_CHECKSUM=num +DELAY_KEY_WRITE num -> DELAY_KEY_WRITE=num +ROW_FORMAT name -> ROW_FORMAT=name +INSERT_METHOD name -> INSERT_METHOD=name +KEY_BLOCK_SIZE num -> KEY_BLOCK_SIZE=num +TRANSACTIONAL num -> TRANSACTIONAL=num + +Table options which will be left hardcoded +UNION +default charset +default collation +DATA DIRECTORY +TABLESPACE +STORAGE + For fields options can go with field attributes (NOT NULL, UNIQUE and so on) can be separated from them by '=' sign. + + + + -=-=(Guest - Tue, 11 Aug 2009, 19:36)=-=- High-Level Specification modified. --- /tmp/wklog.43.old.30883 2009-08-11 19:36:45.000000000 +0300 +++ /tmp/wklog.43.new.30883 2009-08-11 19:36:45.000000000 +0300 @@ -1 +1,12 @@ +Table definition ca looks like following +CREATE TABLE table + (field int ... field_opt1=fval1 field_opt2=fval2, + key key1(field) key_opt1=kval1 key_opt2=kval2) + table_option1=tval1, table_option2=tval2; + +Exclusion should be made for old table and key (KEY_BLOCK_SIZE) options where +'=' was not obligatory. + +For fields options can go with field attributes (NOT NULL, UNIQUE and so on) can +be separated from them by '=' sign. DESCRIPTION: Add ability to create table with additional option which can be passed to engine. Also make current options such as TRANSACTIONAL working via this mechanism. HIGH-LEVEL SPECIFICATION: Table definition ca looks like following CREATE TABLE table (field int ... field_opt1=fval1 field_opt2=fval2, key key1(field) key_opt1=kval1 key_opt2=kval2) table_option1=tval1, table_option2=tval2; Exclusion should be made for old table and key options where '=' was not obligatory. Behaviour and way of storage for existing options will be left as is. Old key options: KEY_BLOCK_SIZE <num> -> KEY_BLOCK_SIZE=num WITH PARSER <name> -> PARSER=name Old table options: ENGINE name -> ENGINE=name TYPE name -> TYPE=name MAX_ROWS num -> MAX_ROWS=num MIX_ROWS num -> MIX_ROWS=num AVG_ROW_LENGTH num -> AVG_ROW_LENGTH=num PASSWORD string -> PASSWORD=string COMMENT string -> COMMENT=string AUTO_INCREMENT num -> AUTO_INCREMENT=num PACK_KEYS num/default -> PACK_KEYS=num/default CHECKSUM num -> CHECKSUM=num TABLE_CHECKSUM num -> TABLE_CHECKSUM=num PAGE_CHECKSUM num -> PAGE_CHECKSUM=num DELAY_KEY_WRITE num -> DELAY_KEY_WRITE=num ROW_FORMAT name -> ROW_FORMAT=name INSERT_METHOD name -> INSERT_METHOD=name KEY_BLOCK_SIZE num -> KEY_BLOCK_SIZE=num TRANSACTIONAL num -> TRANSACTIONAL=num Table options which will be left hardcoded UNION default charset default collation DATA DIRECTORY TABLESPACE STORAGE For fields options can go with field attributes (NOT NULL, UNIQUE and so on) can be separated from them by '=' sign. Incorrect option during creation/altering table lead only to worning. During opening unrecognised options should be ignored. LOW-LEVEL DESIGN: Options stored in a list as pairs key, value which are LEX_STRING. Handler have access to the options during creation and after opening frm. It should has C interface. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Guest): options for CREATE TABLE (43)
by worklog-noreply＠askmonty.org 18 Jan '10

18 Jan '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: options for CREATE TABLE CREATION DATE..: Tue, 11 Aug 2009, 17:02 SUPERVISOR.....: Bothorsen IMPLEMENTOR....: Sanja COPIES TO......: Sergei, Monty CATEGORY.......: Server-BackLog TASK ID........: 43 (http://askmonty.org/worklog/?tid=43) VERSION........: Server-5.2 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 32 (hours remain) ORIG. ESTIMATE.: 32 PROGRESS NOTES: -=-=(Sergei - Mon, 18 Jan 2010, 22:12)=-=- Observers changed: Monty,Sergei -=-=(Monty - Tue, 12 Jan 2010, 17:04)=-=- Version updated. --- /tmp/wklog.43.old.21802 2010-01-12 17:04:31.000000000 +0200 +++ /tmp/wklog.43.new.21802 2010-01-12 17:04:31.000000000 +0200 @@ -1 +1 @@ -Server-5.1 +Server-5.2 -=-=(Guest - Fri, 18 Sep 2009, 14:24)=-=- Low Level Design modified. --- /tmp/wklog.43.old.31654 2009-09-18 14:24:41.000000000 +0300 +++ /tmp/wklog.43.new.31654 2009-09-18 14:24:41.000000000 +0300 @@ -2,4 +2,6 @@ Handler have access to the options during creation and after opening frm. +It should has C interface. + -=-=(Sanja - Fri, 18 Sep 2009, 13:32)=-=- Low Level Design modified. --- /tmp/wklog.43.old.29567 2009-09-18 13:32:41.000000000 +0300 +++ /tmp/wklog.43.new.29567 2009-09-18 13:32:41.000000000 +0300 @@ -1 +1,5 @@ +Options stored in a list as pairs key, value which are LEX_STRING. + +Handler have access to the options during creation and after opening frm. + -=-=(Sanja - Fri, 18 Sep 2009, 13:21)=-=- High-Level Specification modified. --- /tmp/wklog.43.old.29107 2009-09-18 13:21:19.000000000 +0300 +++ /tmp/wklog.43.new.29107 2009-09-18 13:21:19.000000000 +0300 @@ -6,7 +6,8 @@ table_option1=tval1, table_option2=tval2; Exclusion should be made for old table and key options where -'=' was not obligatory. +'=' was not obligatory. Behaviour and way of storage for existing options will +be left as is. Old key options: KEY_BLOCK_SIZE <num> -> KEY_BLOCK_SIZE=num @@ -42,6 +43,6 @@ For fields options can go with field attributes (NOT NULL, UNIQUE and so on) can be separated from them by '=' sign. - - +Incorrect option during creation/altering table lead only to worning. During +opening unrecognised options should be ignored. -=-=(Guest - Tue, 11 Aug 2009, 19:57)=-=- High-Level Specification modified. --- /tmp/wklog.43.old.31856 2009-08-11 19:57:38.000000000 +0300 +++ /tmp/wklog.43.new.31856 2009-08-11 19:57:38.000000000 +0300 @@ -5,8 +5,43 @@ key key1(field) key_opt1=kval1 key_opt2=kval2) table_option1=tval1, table_option2=tval2; -Exclusion should be made for old table and key (KEY_BLOCK_SIZE) options where +Exclusion should be made for old table and key options where '=' was not obligatory. +Old key options: +KEY_BLOCK_SIZE <num> -> KEY_BLOCK_SIZE=num +WITH PARSER <name> -> PARSER=name + +Old table options: +ENGINE name -> ENGINE=name +TYPE name -> TYPE=name +MAX_ROWS num -> MAX_ROWS=num +MIX_ROWS num -> MIX_ROWS=num +AVG_ROW_LENGTH num -> AVG_ROW_LENGTH=num +PASSWORD string -> PASSWORD=string +COMMENT string -> COMMENT=string +AUTO_INCREMENT num -> AUTO_INCREMENT=num +PACK_KEYS num/default -> PACK_KEYS=num/default +CHECKSUM num -> CHECKSUM=num +TABLE_CHECKSUM num -> TABLE_CHECKSUM=num +PAGE_CHECKSUM num -> PAGE_CHECKSUM=num +DELAY_KEY_WRITE num -> DELAY_KEY_WRITE=num +ROW_FORMAT name -> ROW_FORMAT=name +INSERT_METHOD name -> INSERT_METHOD=name +KEY_BLOCK_SIZE num -> KEY_BLOCK_SIZE=num +TRANSACTIONAL num -> TRANSACTIONAL=num + +Table options which will be left hardcoded +UNION +default charset +default collation +DATA DIRECTORY +TABLESPACE +STORAGE + For fields options can go with field attributes (NOT NULL, UNIQUE and so on) can be separated from them by '=' sign. + + + + -=-=(Guest - Tue, 11 Aug 2009, 19:36)=-=- High-Level Specification modified. --- /tmp/wklog.43.old.30883 2009-08-11 19:36:45.000000000 +0300 +++ /tmp/wklog.43.new.30883 2009-08-11 19:36:45.000000000 +0300 @@ -1 +1,12 @@ +Table definition ca looks like following +CREATE TABLE table + (field int ... field_opt1=fval1 field_opt2=fval2, + key key1(field) key_opt1=kval1 key_opt2=kval2) + table_option1=tval1, table_option2=tval2; + +Exclusion should be made for old table and key (KEY_BLOCK_SIZE) options where +'=' was not obligatory. + +For fields options can go with field attributes (NOT NULL, UNIQUE and so on) can +be separated from them by '=' sign. DESCRIPTION: Add ability to create table with additional option which can be passed to engine. Also make current options such as TRANSACTIONAL working via this mechanism. HIGH-LEVEL SPECIFICATION: Table definition ca looks like following CREATE TABLE table (field int ... field_opt1=fval1 field_opt2=fval2, key key1(field) key_opt1=kval1 key_opt2=kval2) table_option1=tval1, table_option2=tval2; Exclusion should be made for old table and key options where '=' was not obligatory. Behaviour and way of storage for existing options will be left as is. Old key options: KEY_BLOCK_SIZE <num> -> KEY_BLOCK_SIZE=num WITH PARSER <name> -> PARSER=name Old table options: ENGINE name -> ENGINE=name TYPE name -> TYPE=name MAX_ROWS num -> MAX_ROWS=num MIX_ROWS num -> MIX_ROWS=num AVG_ROW_LENGTH num -> AVG_ROW_LENGTH=num PASSWORD string -> PASSWORD=string COMMENT string -> COMMENT=string AUTO_INCREMENT num -> AUTO_INCREMENT=num PACK_KEYS num/default -> PACK_KEYS=num/default CHECKSUM num -> CHECKSUM=num TABLE_CHECKSUM num -> TABLE_CHECKSUM=num PAGE_CHECKSUM num -> PAGE_CHECKSUM=num DELAY_KEY_WRITE num -> DELAY_KEY_WRITE=num ROW_FORMAT name -> ROW_FORMAT=name INSERT_METHOD name -> INSERT_METHOD=name KEY_BLOCK_SIZE num -> KEY_BLOCK_SIZE=num TRANSACTIONAL num -> TRANSACTIONAL=num Table options which will be left hardcoded UNION default charset default collation DATA DIRECTORY TABLESPACE STORAGE For fields options can go with field attributes (NOT NULL, UNIQUE and so on) can be separated from them by '=' sign. Incorrect option during creation/altering table lead only to worning. During opening unrecognised options should be ignored. LOW-LEVEL DESIGN: Options stored in a list as pairs key, value which are LEX_STRING. Handler have access to the options during creation and after opening frm. It should has C interface. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Sergei): options for CREATE TABLE (43)
by worklog-noreply＠askmonty.org 18 Jan '10

18 Jan '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: options for CREATE TABLE CREATION DATE..: Tue, 11 Aug 2009, 17:02 SUPERVISOR.....: Bothorsen IMPLEMENTOR....: Sanja COPIES TO......: Sergei, Monty CATEGORY.......: Server-BackLog TASK ID........: 43 (http://askmonty.org/worklog/?tid=43) VERSION........: Server-5.2 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 32 (hours remain) ORIG. ESTIMATE.: 32 PROGRESS NOTES: -=-=(Sergei - Mon, 18 Jan 2010, 22:12)=-=- Observers changed: Monty,Sergei -=-=(Monty - Tue, 12 Jan 2010, 17:04)=-=- Version updated. --- /tmp/wklog.43.old.21802 2010-01-12 17:04:31.000000000 +0200 +++ /tmp/wklog.43.new.21802 2010-01-12 17:04:31.000000000 +0200 @@ -1 +1 @@ -Server-5.1 +Server-5.2 -=-=(Guest - Fri, 18 Sep 2009, 14:24)=-=- Low Level Design modified. --- /tmp/wklog.43.old.31654 2009-09-18 14:24:41.000000000 +0300 +++ /tmp/wklog.43.new.31654 2009-09-18 14:24:41.000000000 +0300 @@ -2,4 +2,6 @@ Handler have access to the options during creation and after opening frm. +It should has C interface. + -=-=(Sanja - Fri, 18 Sep 2009, 13:32)=-=- Low Level Design modified. --- /tmp/wklog.43.old.29567 2009-09-18 13:32:41.000000000 +0300 +++ /tmp/wklog.43.new.29567 2009-09-18 13:32:41.000000000 +0300 @@ -1 +1,5 @@ +Options stored in a list as pairs key, value which are LEX_STRING. + +Handler have access to the options during creation and after opening frm. + -=-=(Sanja - Fri, 18 Sep 2009, 13:21)=-=- High-Level Specification modified. --- /tmp/wklog.43.old.29107 2009-09-18 13:21:19.000000000 +0300 +++ /tmp/wklog.43.new.29107 2009-09-18 13:21:19.000000000 +0300 @@ -6,7 +6,8 @@ table_option1=tval1, table_option2=tval2; Exclusion should be made for old table and key options where -'=' was not obligatory. +'=' was not obligatory. Behaviour and way of storage for existing options will +be left as is. Old key options: KEY_BLOCK_SIZE <num> -> KEY_BLOCK_SIZE=num @@ -42,6 +43,6 @@ For fields options can go with field attributes (NOT NULL, UNIQUE and so on) can be separated from them by '=' sign. - - +Incorrect option during creation/altering table lead only to worning. During +opening unrecognised options should be ignored. -=-=(Guest - Tue, 11 Aug 2009, 19:57)=-=- High-Level Specification modified. --- /tmp/wklog.43.old.31856 2009-08-11 19:57:38.000000000 +0300 +++ /tmp/wklog.43.new.31856 2009-08-11 19:57:38.000000000 +0300 @@ -5,8 +5,43 @@ key key1(field) key_opt1=kval1 key_opt2=kval2) table_option1=tval1, table_option2=tval2; -Exclusion should be made for old table and key (KEY_BLOCK_SIZE) options where +Exclusion should be made for old table and key options where '=' was not obligatory. +Old key options: +KEY_BLOCK_SIZE <num> -> KEY_BLOCK_SIZE=num +WITH PARSER <name> -> PARSER=name + +Old table options: +ENGINE name -> ENGINE=name +TYPE name -> TYPE=name +MAX_ROWS num -> MAX_ROWS=num +MIX_ROWS num -> MIX_ROWS=num +AVG_ROW_LENGTH num -> AVG_ROW_LENGTH=num +PASSWORD string -> PASSWORD=string +COMMENT string -> COMMENT=string +AUTO_INCREMENT num -> AUTO_INCREMENT=num +PACK_KEYS num/default -> PACK_KEYS=num/default +CHECKSUM num -> CHECKSUM=num +TABLE_CHECKSUM num -> TABLE_CHECKSUM=num +PAGE_CHECKSUM num -> PAGE_CHECKSUM=num +DELAY_KEY_WRITE num -> DELAY_KEY_WRITE=num +ROW_FORMAT name -> ROW_FORMAT=name +INSERT_METHOD name -> INSERT_METHOD=name +KEY_BLOCK_SIZE num -> KEY_BLOCK_SIZE=num +TRANSACTIONAL num -> TRANSACTIONAL=num + +Table options which will be left hardcoded +UNION +default charset +default collation +DATA DIRECTORY +TABLESPACE +STORAGE + For fields options can go with field attributes (NOT NULL, UNIQUE and so on) can be separated from them by '=' sign. + + + + -=-=(Guest - Tue, 11 Aug 2009, 19:36)=-=- High-Level Specification modified. --- /tmp/wklog.43.old.30883 2009-08-11 19:36:45.000000000 +0300 +++ /tmp/wklog.43.new.30883 2009-08-11 19:36:45.000000000 +0300 @@ -1 +1,12 @@ +Table definition ca looks like following +CREATE TABLE table + (field int ... field_opt1=fval1 field_opt2=fval2, + key key1(field) key_opt1=kval1 key_opt2=kval2) + table_option1=tval1, table_option2=tval2; + +Exclusion should be made for old table and key (KEY_BLOCK_SIZE) options where +'=' was not obligatory. + +For fields options can go with field attributes (NOT NULL, UNIQUE and so on) can +be separated from them by '=' sign. DESCRIPTION: Add ability to create table with additional option which can be passed to engine. Also make current options such as TRANSACTIONAL working via this mechanism. HIGH-LEVEL SPECIFICATION: Table definition ca looks like following CREATE TABLE table (field int ... field_opt1=fval1 field_opt2=fval2, key key1(field) key_opt1=kval1 key_opt2=kval2) table_option1=tval1, table_option2=tval2; Exclusion should be made for old table and key options where '=' was not obligatory. Behaviour and way of storage for existing options will be left as is. Old key options: KEY_BLOCK_SIZE <num> -> KEY_BLOCK_SIZE=num WITH PARSER <name> -> PARSER=name Old table options: ENGINE name -> ENGINE=name TYPE name -> TYPE=name MAX_ROWS num -> MAX_ROWS=num MIX_ROWS num -> MIX_ROWS=num AVG_ROW_LENGTH num -> AVG_ROW_LENGTH=num PASSWORD string -> PASSWORD=string COMMENT string -> COMMENT=string AUTO_INCREMENT num -> AUTO_INCREMENT=num PACK_KEYS num/default -> PACK_KEYS=num/default CHECKSUM num -> CHECKSUM=num TABLE_CHECKSUM num -> TABLE_CHECKSUM=num PAGE_CHECKSUM num -> PAGE_CHECKSUM=num DELAY_KEY_WRITE num -> DELAY_KEY_WRITE=num ROW_FORMAT name -> ROW_FORMAT=name INSERT_METHOD name -> INSERT_METHOD=name KEY_BLOCK_SIZE num -> KEY_BLOCK_SIZE=num TRANSACTIONAL num -> TRANSACTIONAL=num Table options which will be left hardcoded UNION default charset default collation DATA DIRECTORY TABLESPACE STORAGE For fields options can go with field attributes (NOT NULL, UNIQUE and so on) can be separated from them by '=' sign. Incorrect option during creation/altering table lead only to worning. During opening unrecognised options should be ignored. LOW-LEVEL DESIGN: Options stored in a list as pairs key, value which are LEX_STRING. Handler have access to the options during creation and after opening frm. It should has C interface. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Sergei): options for CREATE TABLE (43)
by worklog-noreply＠askmonty.org 18 Jan '10

18 Jan '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: options for CREATE TABLE CREATION DATE..: Tue, 11 Aug 2009, 17:02 SUPERVISOR.....: Bothorsen IMPLEMENTOR....: Sanja COPIES TO......: Sergei, Monty CATEGORY.......: Server-BackLog TASK ID........: 43 (http://askmonty.org/worklog/?tid=43) VERSION........: Server-5.2 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 32 (hours remain) ORIG. ESTIMATE.: 32 PROGRESS NOTES: -=-=(Sergei - Mon, 18 Jan 2010, 22:12)=-=- Observers changed: Monty,Sergei -=-=(Monty - Tue, 12 Jan 2010, 17:04)=-=- Version updated. --- /tmp/wklog.43.old.21802 2010-01-12 17:04:31.000000000 +0200 +++ /tmp/wklog.43.new.21802 2010-01-12 17:04:31.000000000 +0200 @@ -1 +1 @@ -Server-5.1 +Server-5.2 -=-=(Guest - Fri, 18 Sep 2009, 14:24)=-=- Low Level Design modified. --- /tmp/wklog.43.old.31654 2009-09-18 14:24:41.000000000 +0300 +++ /tmp/wklog.43.new.31654 2009-09-18 14:24:41.000000000 +0300 @@ -2,4 +2,6 @@ Handler have access to the options during creation and after opening frm. +It should has C interface. + -=-=(Sanja - Fri, 18 Sep 2009, 13:32)=-=- Low Level Design modified. --- /tmp/wklog.43.old.29567 2009-09-18 13:32:41.000000000 +0300 +++ /tmp/wklog.43.new.29567 2009-09-18 13:32:41.000000000 +0300 @@ -1 +1,5 @@ +Options stored in a list as pairs key, value which are LEX_STRING. + +Handler have access to the options during creation and after opening frm. + -=-=(Sanja - Fri, 18 Sep 2009, 13:21)=-=- High-Level Specification modified. --- /tmp/wklog.43.old.29107 2009-09-18 13:21:19.000000000 +0300 +++ /tmp/wklog.43.new.29107 2009-09-18 13:21:19.000000000 +0300 @@ -6,7 +6,8 @@ table_option1=tval1, table_option2=tval2; Exclusion should be made for old table and key options where -'=' was not obligatory. +'=' was not obligatory. Behaviour and way of storage for existing options will +be left as is. Old key options: KEY_BLOCK_SIZE <num> -> KEY_BLOCK_SIZE=num @@ -42,6 +43,6 @@ For fields options can go with field attributes (NOT NULL, UNIQUE and so on) can be separated from them by '=' sign. - - +Incorrect option during creation/altering table lead only to worning. During +opening unrecognised options should be ignored. -=-=(Guest - Tue, 11 Aug 2009, 19:57)=-=- High-Level Specification modified. --- /tmp/wklog.43.old.31856 2009-08-11 19:57:38.000000000 +0300 +++ /tmp/wklog.43.new.31856 2009-08-11 19:57:38.000000000 +0300 @@ -5,8 +5,43 @@ key key1(field) key_opt1=kval1 key_opt2=kval2) table_option1=tval1, table_option2=tval2; -Exclusion should be made for old table and key (KEY_BLOCK_SIZE) options where +Exclusion should be made for old table and key options where '=' was not obligatory. +Old key options: +KEY_BLOCK_SIZE <num> -> KEY_BLOCK_SIZE=num +WITH PARSER <name> -> PARSER=name + +Old table options: +ENGINE name -> ENGINE=name +TYPE name -> TYPE=name +MAX_ROWS num -> MAX_ROWS=num +MIX_ROWS num -> MIX_ROWS=num +AVG_ROW_LENGTH num -> AVG_ROW_LENGTH=num +PASSWORD string -> PASSWORD=string +COMMENT string -> COMMENT=string +AUTO_INCREMENT num -> AUTO_INCREMENT=num +PACK_KEYS num/default -> PACK_KEYS=num/default +CHECKSUM num -> CHECKSUM=num +TABLE_CHECKSUM num -> TABLE_CHECKSUM=num +PAGE_CHECKSUM num -> PAGE_CHECKSUM=num +DELAY_KEY_WRITE num -> DELAY_KEY_WRITE=num +ROW_FORMAT name -> ROW_FORMAT=name +INSERT_METHOD name -> INSERT_METHOD=name +KEY_BLOCK_SIZE num -> KEY_BLOCK_SIZE=num +TRANSACTIONAL num -> TRANSACTIONAL=num + +Table options which will be left hardcoded +UNION +default charset +default collation +DATA DIRECTORY +TABLESPACE +STORAGE + For fields options can go with field attributes (NOT NULL, UNIQUE and so on) can be separated from them by '=' sign. + + + + -=-=(Guest - Tue, 11 Aug 2009, 19:36)=-=- High-Level Specification modified. --- /tmp/wklog.43.old.30883 2009-08-11 19:36:45.000000000 +0300 +++ /tmp/wklog.43.new.30883 2009-08-11 19:36:45.000000000 +0300 @@ -1 +1,12 @@ +Table definition ca looks like following +CREATE TABLE table + (field int ... field_opt1=fval1 field_opt2=fval2, + key key1(field) key_opt1=kval1 key_opt2=kval2) + table_option1=tval1, table_option2=tval2; + +Exclusion should be made for old table and key (KEY_BLOCK_SIZE) options where +'=' was not obligatory. + +For fields options can go with field attributes (NOT NULL, UNIQUE and so on) can +be separated from them by '=' sign. DESCRIPTION: Add ability to create table with additional option which can be passed to engine. Also make current options such as TRANSACTIONAL working via this mechanism. HIGH-LEVEL SPECIFICATION: Table definition ca looks like following CREATE TABLE table (field int ... field_opt1=fval1 field_opt2=fval2, key key1(field) key_opt1=kval1 key_opt2=kval2) table_option1=tval1, table_option2=tval2; Exclusion should be made for old table and key options where '=' was not obligatory. Behaviour and way of storage for existing options will be left as is. Old key options: KEY_BLOCK_SIZE <num> -> KEY_BLOCK_SIZE=num WITH PARSER <name> -> PARSER=name Old table options: ENGINE name -> ENGINE=name TYPE name -> TYPE=name MAX_ROWS num -> MAX_ROWS=num MIX_ROWS num -> MIX_ROWS=num AVG_ROW_LENGTH num -> AVG_ROW_LENGTH=num PASSWORD string -> PASSWORD=string COMMENT string -> COMMENT=string AUTO_INCREMENT num -> AUTO_INCREMENT=num PACK_KEYS num/default -> PACK_KEYS=num/default CHECKSUM num -> CHECKSUM=num TABLE_CHECKSUM num -> TABLE_CHECKSUM=num PAGE_CHECKSUM num -> PAGE_CHECKSUM=num DELAY_KEY_WRITE num -> DELAY_KEY_WRITE=num ROW_FORMAT name -> ROW_FORMAT=name INSERT_METHOD name -> INSERT_METHOD=name KEY_BLOCK_SIZE num -> KEY_BLOCK_SIZE=num TRANSACTIONAL num -> TRANSACTIONAL=num Table options which will be left hardcoded UNION default charset default collation DATA DIRECTORY TABLESPACE STORAGE For fields options can go with field attributes (NOT NULL, UNIQUE and so on) can be separated from them by '=' sign. Incorrect option during creation/altering table lead only to worning. During opening unrecognised options should be ignored. LOW-LEVEL DESIGN: Options stored in a list as pairs key, value which are LEX_STRING. Handler have access to the options during creation and after opening frm. It should has C interface. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Sergei): options for CREATE TABLE (43)
by worklog-noreply＠askmonty.org 18 Jan '10

18 Jan '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: options for CREATE TABLE CREATION DATE..: Tue, 11 Aug 2009, 17:02 SUPERVISOR.....: Bothorsen IMPLEMENTOR....: Sanja COPIES TO......: Sergei, Monty CATEGORY.......: Server-BackLog TASK ID........: 43 (http://askmonty.org/worklog/?tid=43) VERSION........: Server-5.2 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 32 (hours remain) ORIG. ESTIMATE.: 32 PROGRESS NOTES: -=-=(Sergei - Mon, 18 Jan 2010, 22:12)=-=- Observers changed: Monty,Sergei -=-=(Monty - Tue, 12 Jan 2010, 17:04)=-=- Version updated. --- /tmp/wklog.43.old.21802 2010-01-12 17:04:31.000000000 +0200 +++ /tmp/wklog.43.new.21802 2010-01-12 17:04:31.000000000 +0200 @@ -1 +1 @@ -Server-5.1 +Server-5.2 -=-=(Guest - Fri, 18 Sep 2009, 14:24)=-=- Low Level Design modified. --- /tmp/wklog.43.old.31654 2009-09-18 14:24:41.000000000 +0300 +++ /tmp/wklog.43.new.31654 2009-09-18 14:24:41.000000000 +0300 @@ -2,4 +2,6 @@ Handler have access to the options during creation and after opening frm. +It should has C interface. + -=-=(Sanja - Fri, 18 Sep 2009, 13:32)=-=- Low Level Design modified. --- /tmp/wklog.43.old.29567 2009-09-18 13:32:41.000000000 +0300 +++ /tmp/wklog.43.new.29567 2009-09-18 13:32:41.000000000 +0300 @@ -1 +1,5 @@ +Options stored in a list as pairs key, value which are LEX_STRING. + +Handler have access to the options during creation and after opening frm. + -=-=(Sanja - Fri, 18 Sep 2009, 13:21)=-=- High-Level Specification modified. --- /tmp/wklog.43.old.29107 2009-09-18 13:21:19.000000000 +0300 +++ /tmp/wklog.43.new.29107 2009-09-18 13:21:19.000000000 +0300 @@ -6,7 +6,8 @@ table_option1=tval1, table_option2=tval2; Exclusion should be made for old table and key options where -'=' was not obligatory. +'=' was not obligatory. Behaviour and way of storage for existing options will +be left as is. Old key options: KEY_BLOCK_SIZE <num> -> KEY_BLOCK_SIZE=num @@ -42,6 +43,6 @@ For fields options can go with field attributes (NOT NULL, UNIQUE and so on) can be separated from them by '=' sign. - - +Incorrect option during creation/altering table lead only to worning. During +opening unrecognised options should be ignored. -=-=(Guest - Tue, 11 Aug 2009, 19:57)=-=- High-Level Specification modified. --- /tmp/wklog.43.old.31856 2009-08-11 19:57:38.000000000 +0300 +++ /tmp/wklog.43.new.31856 2009-08-11 19:57:38.000000000 +0300 @@ -5,8 +5,43 @@ key key1(field) key_opt1=kval1 key_opt2=kval2) table_option1=tval1, table_option2=tval2; -Exclusion should be made for old table and key (KEY_BLOCK_SIZE) options where +Exclusion should be made for old table and key options where '=' was not obligatory. +Old key options: +KEY_BLOCK_SIZE <num> -> KEY_BLOCK_SIZE=num +WITH PARSER <name> -> PARSER=name + +Old table options: +ENGINE name -> ENGINE=name +TYPE name -> TYPE=name +MAX_ROWS num -> MAX_ROWS=num +MIX_ROWS num -> MIX_ROWS=num +AVG_ROW_LENGTH num -> AVG_ROW_LENGTH=num +PASSWORD string -> PASSWORD=string +COMMENT string -> COMMENT=string +AUTO_INCREMENT num -> AUTO_INCREMENT=num +PACK_KEYS num/default -> PACK_KEYS=num/default +CHECKSUM num -> CHECKSUM=num +TABLE_CHECKSUM num -> TABLE_CHECKSUM=num +PAGE_CHECKSUM num -> PAGE_CHECKSUM=num +DELAY_KEY_WRITE num -> DELAY_KEY_WRITE=num +ROW_FORMAT name -> ROW_FORMAT=name +INSERT_METHOD name -> INSERT_METHOD=name +KEY_BLOCK_SIZE num -> KEY_BLOCK_SIZE=num +TRANSACTIONAL num -> TRANSACTIONAL=num + +Table options which will be left hardcoded +UNION +default charset +default collation +DATA DIRECTORY +TABLESPACE +STORAGE + For fields options can go with field attributes (NOT NULL, UNIQUE and so on) can be separated from them by '=' sign. + + + + -=-=(Guest - Tue, 11 Aug 2009, 19:36)=-=- High-Level Specification modified. --- /tmp/wklog.43.old.30883 2009-08-11 19:36:45.000000000 +0300 +++ /tmp/wklog.43.new.30883 2009-08-11 19:36:45.000000000 +0300 @@ -1 +1,12 @@ +Table definition ca looks like following +CREATE TABLE table + (field int ... field_opt1=fval1 field_opt2=fval2, + key key1(field) key_opt1=kval1 key_opt2=kval2) + table_option1=tval1, table_option2=tval2; + +Exclusion should be made for old table and key (KEY_BLOCK_SIZE) options where +'=' was not obligatory. + +For fields options can go with field attributes (NOT NULL, UNIQUE and so on) can +be separated from them by '=' sign. DESCRIPTION: Add ability to create table with additional option which can be passed to engine. Also make current options such as TRANSACTIONAL working via this mechanism. HIGH-LEVEL SPECIFICATION: Table definition ca looks like following CREATE TABLE table (field int ... field_opt1=fval1 field_opt2=fval2, key key1(field) key_opt1=kval1 key_opt2=kval2) table_option1=tval1, table_option2=tval2; Exclusion should be made for old table and key options where '=' was not obligatory. Behaviour and way of storage for existing options will be left as is. Old key options: KEY_BLOCK_SIZE <num> -> KEY_BLOCK_SIZE=num WITH PARSER <name> -> PARSER=name Old table options: ENGINE name -> ENGINE=name TYPE name -> TYPE=name MAX_ROWS num -> MAX_ROWS=num MIX_ROWS num -> MIX_ROWS=num AVG_ROW_LENGTH num -> AVG_ROW_LENGTH=num PASSWORD string -> PASSWORD=string COMMENT string -> COMMENT=string AUTO_INCREMENT num -> AUTO_INCREMENT=num PACK_KEYS num/default -> PACK_KEYS=num/default CHECKSUM num -> CHECKSUM=num TABLE_CHECKSUM num -> TABLE_CHECKSUM=num PAGE_CHECKSUM num -> PAGE_CHECKSUM=num DELAY_KEY_WRITE num -> DELAY_KEY_WRITE=num ROW_FORMAT name -> ROW_FORMAT=name INSERT_METHOD name -> INSERT_METHOD=name KEY_BLOCK_SIZE num -> KEY_BLOCK_SIZE=num TRANSACTIONAL num -> TRANSACTIONAL=num Table options which will be left hardcoded UNION default charset default collation DATA DIRECTORY TABLESPACE STORAGE For fields options can go with field attributes (NOT NULL, UNIQUE and so on) can be separated from them by '=' sign. Incorrect option during creation/altering table lead only to worning. During opening unrecognised options should be ignored. LOW-LEVEL DESIGN: Options stored in a list as pairs key, value which are LEX_STRING. Handler have access to the options during creation and after opening frm. It should has C interface. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Sergei): options for CREATE TABLE (43)
by worklog-noreply＠askmonty.org 18 Jan '10

18 Jan '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: options for CREATE TABLE CREATION DATE..: Tue, 11 Aug 2009, 17:02 SUPERVISOR.....: Bothorsen IMPLEMENTOR....: Sanja COPIES TO......: Sergei, Monty CATEGORY.......: Server-BackLog TASK ID........: 43 (http://askmonty.org/worklog/?tid=43) VERSION........: Server-5.2 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 32 (hours remain) ORIG. ESTIMATE.: 32 PROGRESS NOTES: -=-=(Sergei - Mon, 18 Jan 2010, 22:12)=-=- Observers changed: Monty,Sergei -=-=(Monty - Tue, 12 Jan 2010, 17:04)=-=- Version updated. --- /tmp/wklog.43.old.21802 2010-01-12 17:04:31.000000000 +0200 +++ /tmp/wklog.43.new.21802 2010-01-12 17:04:31.000000000 +0200 @@ -1 +1 @@ -Server-5.1 +Server-5.2 -=-=(Guest - Fri, 18 Sep 2009, 14:24)=-=- Low Level Design modified. --- /tmp/wklog.43.old.31654 2009-09-18 14:24:41.000000000 +0300 +++ /tmp/wklog.43.new.31654 2009-09-18 14:24:41.000000000 +0300 @@ -2,4 +2,6 @@ Handler have access to the options during creation and after opening frm. +It should has C interface. + -=-=(Sanja - Fri, 18 Sep 2009, 13:32)=-=- Low Level Design modified. --- /tmp/wklog.43.old.29567 2009-09-18 13:32:41.000000000 +0300 +++ /tmp/wklog.43.new.29567 2009-09-18 13:32:41.000000000 +0300 @@ -1 +1,5 @@ +Options stored in a list as pairs key, value which are LEX_STRING. + +Handler have access to the options during creation and after opening frm. + -=-=(Sanja - Fri, 18 Sep 2009, 13:21)=-=- High-Level Specification modified. --- /tmp/wklog.43.old.29107 2009-09-18 13:21:19.000000000 +0300 +++ /tmp/wklog.43.new.29107 2009-09-18 13:21:19.000000000 +0300 @@ -6,7 +6,8 @@ table_option1=tval1, table_option2=tval2; Exclusion should be made for old table and key options where -'=' was not obligatory. +'=' was not obligatory. Behaviour and way of storage for existing options will +be left as is. Old key options: KEY_BLOCK_SIZE <num> -> KEY_BLOCK_SIZE=num @@ -42,6 +43,6 @@ For fields options can go with field attributes (NOT NULL, UNIQUE and so on) can be separated from them by '=' sign. - - +Incorrect option during creation/altering table lead only to worning. During +opening unrecognised options should be ignored. -=-=(Guest - Tue, 11 Aug 2009, 19:57)=-=- High-Level Specification modified. --- /tmp/wklog.43.old.31856 2009-08-11 19:57:38.000000000 +0300 +++ /tmp/wklog.43.new.31856 2009-08-11 19:57:38.000000000 +0300 @@ -5,8 +5,43 @@ key key1(field) key_opt1=kval1 key_opt2=kval2) table_option1=tval1, table_option2=tval2; -Exclusion should be made for old table and key (KEY_BLOCK_SIZE) options where +Exclusion should be made for old table and key options where '=' was not obligatory. +Old key options: +KEY_BLOCK_SIZE <num> -> KEY_BLOCK_SIZE=num +WITH PARSER <name> -> PARSER=name + +Old table options: +ENGINE name -> ENGINE=name +TYPE name -> TYPE=name +MAX_ROWS num -> MAX_ROWS=num +MIX_ROWS num -> MIX_ROWS=num +AVG_ROW_LENGTH num -> AVG_ROW_LENGTH=num +PASSWORD string -> PASSWORD=string +COMMENT string -> COMMENT=string +AUTO_INCREMENT num -> AUTO_INCREMENT=num +PACK_KEYS num/default -> PACK_KEYS=num/default +CHECKSUM num -> CHECKSUM=num +TABLE_CHECKSUM num -> TABLE_CHECKSUM=num +PAGE_CHECKSUM num -> PAGE_CHECKSUM=num +DELAY_KEY_WRITE num -> DELAY_KEY_WRITE=num +ROW_FORMAT name -> ROW_FORMAT=name +INSERT_METHOD name -> INSERT_METHOD=name +KEY_BLOCK_SIZE num -> KEY_BLOCK_SIZE=num +TRANSACTIONAL num -> TRANSACTIONAL=num + +Table options which will be left hardcoded +UNION +default charset +default collation +DATA DIRECTORY +TABLESPACE +STORAGE + For fields options can go with field attributes (NOT NULL, UNIQUE and so on) can be separated from them by '=' sign. + + + + -=-=(Guest - Tue, 11 Aug 2009, 19:36)=-=- High-Level Specification modified. --- /tmp/wklog.43.old.30883 2009-08-11 19:36:45.000000000 +0300 +++ /tmp/wklog.43.new.30883 2009-08-11 19:36:45.000000000 +0300 @@ -1 +1,12 @@ +Table definition ca looks like following +CREATE TABLE table + (field int ... field_opt1=fval1 field_opt2=fval2, + key key1(field) key_opt1=kval1 key_opt2=kval2) + table_option1=tval1, table_option2=tval2; + +Exclusion should be made for old table and key (KEY_BLOCK_SIZE) options where +'=' was not obligatory. + +For fields options can go with field attributes (NOT NULL, UNIQUE and so on) can +be separated from them by '=' sign. DESCRIPTION: Add ability to create table with additional option which can be passed to engine. Also make current options such as TRANSACTIONAL working via this mechanism. HIGH-LEVEL SPECIFICATION: Table definition ca looks like following CREATE TABLE table (field int ... field_opt1=fval1 field_opt2=fval2, key key1(field) key_opt1=kval1 key_opt2=kval2) table_option1=tval1, table_option2=tval2; Exclusion should be made for old table and key options where '=' was not obligatory. Behaviour and way of storage for existing options will be left as is. Old key options: KEY_BLOCK_SIZE <num> -> KEY_BLOCK_SIZE=num WITH PARSER <name> -> PARSER=name Old table options: ENGINE name -> ENGINE=name TYPE name -> TYPE=name MAX_ROWS num -> MAX_ROWS=num MIX_ROWS num -> MIX_ROWS=num AVG_ROW_LENGTH num -> AVG_ROW_LENGTH=num PASSWORD string -> PASSWORD=string COMMENT string -> COMMENT=string AUTO_INCREMENT num -> AUTO_INCREMENT=num PACK_KEYS num/default -> PACK_KEYS=num/default CHECKSUM num -> CHECKSUM=num TABLE_CHECKSUM num -> TABLE_CHECKSUM=num PAGE_CHECKSUM num -> PAGE_CHECKSUM=num DELAY_KEY_WRITE num -> DELAY_KEY_WRITE=num ROW_FORMAT name -> ROW_FORMAT=name INSERT_METHOD name -> INSERT_METHOD=name KEY_BLOCK_SIZE num -> KEY_BLOCK_SIZE=num TRANSACTIONAL num -> TRANSACTIONAL=num Table options which will be left hardcoded UNION default charset default collation DATA DIRECTORY TABLESPACE STORAGE For fields options can go with field attributes (NOT NULL, UNIQUE and so on) can be separated from them by '=' sign. Incorrect option during creation/altering table lead only to worning. During opening unrecognised options should be ignored. LOW-LEVEL DESIGN: Options stored in a list as pairs key, value which are LEX_STRING. Handler have access to the options during creation and after opening frm. It should has C interface. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Psergey): Subquery optimization: Avoid recalculating subquery if external fields values found in subquery cache (66)
by worklog-noreply＠askmonty.org 18 Jan '10

18 Jan '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Subquery optimization: Avoid recalculating subquery if external fields values found in subquery cache CREATION DATE..: Wed, 25 Nov 2009, 22:25 SUPERVISOR.....: Monty IMPLEMENTOR....: Sanja COPIES TO......: CATEGORY.......: Server-BackLog TASK ID........: 66 (http://askmonty.org/worklog/?tid=66) VERSION........: Server-5.2 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Psergey - Mon, 18 Jan 2010, 16:40)=-=- Low Level Design modified. --- /tmp/wklog.66.old.24899 2010-01-18 16:40:16.000000000 +0200 +++ /tmp/wklog.66.new.24899 2010-01-18 16:40:16.000000000 +0200 @@ -1,3 +1,5 @@ +* Target version: base on mysql-5.2 code + All items on which subquery depend could be collected in st_select_lex::mark_as_dependent (direct of indirect reference?) -=-=(Psergey - Mon, 18 Jan 2010, 16:37)=-=- Low Level Design modified. --- /tmp/wklog.66.old.24586 2010-01-18 16:37:07.000000000 +0200 +++ /tmp/wklog.66.new.24586 2010-01-18 16:37:07.000000000 +0200 @@ -4,6 +4,11 @@ Temporary table index should be created by all fields except result field (TMP_TABLE_PARAM::keyinfo). +How to fill the temptable +------------------------- +Can reuse approach from SJ-Materialization. Its code is in end_sj_materialize() +and is supposed to be quite trivial. + How to make lookups into temptable ---------------------------------- We'll reuse approach used by SJ-Materialization in 6.0. -=-=(Psergey - Mon, 18 Jan 2010, 16:34)=-=- Low Level Design modified. --- /tmp/wklog.66.old.24328 2010-01-18 16:34:19.000000000 +0200 +++ /tmp/wklog.66.new.24328 2010-01-18 16:34:19.000000000 +0200 @@ -32,8 +32,8 @@ Question: or perhaps that is not necessarry? </questionable> -Execution process -~~~~~~~~~~~~~~~~~ +Doing the lookup +~~~~~~~~~~~~~~~~ SJ-Materialization does lookup in sub_select_sjm(), with this code: /* Do index lookup in the materialized table */ @@ -42,4 +42,12 @@ if (res || !sjm->in_equality->val_int()) DBUG_RETURN(NESTED_LOOP_NO_MORE_ROWS); +The code in this WL will use the same approach +Extracting the value of the subquery predicate +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The goal of making the lookup is to get the value of subquery predicate. +This is done by creating an Item_field $I which refers to appropriate +temporary table's field and then subquery_predicate->val_int() will invoke +$I->val_int(), subquery_predicate->val_str() will invoke $I->val_str() and so +forth. -=-=(Psergey - Mon, 18 Jan 2010, 16:23)=-=- Low Level Design modified. --- /tmp/wklog.66.old.23203 2010-01-18 16:23:18.000000000 +0200 +++ /tmp/wklog.66.new.23203 2010-01-18 16:23:18.000000000 +0200 @@ -31,3 +31,15 @@ Question: or perhaps that is not necessarry? </questionable> + +Execution process +~~~~~~~~~~~~~~~~~ +SJ-Materialization does lookup in sub_select_sjm(), with this code: + + /* Do index lookup in the materialized table */ + if ((res= join_read_key2(join_tab, sjm->table, sjm->tab_ref)) == 1) + DBUG_RETURN(NESTED_LOOP_ERROR); /* purecov: inspected */ + if (res || !sjm->in_equality->val_int()) + DBUG_RETURN(NESTED_LOOP_NO_MORE_ROWS); + + -=-=(Psergey - Mon, 18 Jan 2010, 16:22)=-=- Low Level Design modified. --- /tmp/wklog.66.old.23076 2010-01-18 16:22:07.000000000 +0200 +++ /tmp/wklog.66.new.23076 2010-01-18 16:22:07.000000000 +0200 @@ -4,3 +4,30 @@ Temporary table index should be created by all fields except result field (TMP_TABLE_PARAM::keyinfo). +How to make lookups into temptable +---------------------------------- +We'll reuse approach used by SJ-Materialization in 6.0. + +Setup process +~~~~~~~~~~~~~ +Setup is performed in the same way as in setup_sj_materialization(), +see the code that starts these lines: + + /* + Create/initialize everything we will need to index lookups into the + temptable. + */ + +and ends at this line: + + Remove the injected semi-join IN-equalities from join_tab conds. This + +<questionable> +We'll also need to check equalities, i.e. do an equivalent of this: + + if (!(sjm->in_equality= create_subq_in_equalities(thd, sjm, + emb_sj_nest->sj_subq_pred))) + DBUG_RETURN(TRUE); /* purecov: inspected */ + +Question: or perhaps that is not necessarry? +</questionable> -=-=(Psergey - Tue, 12 Jan 2010, 18:39)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.31666 2010-01-12 18:39:43.000000000 +0200 +++ /tmp/wklog.66.new.31666 2010-01-12 18:39:43.000000000 +0200 @@ -4,3 +4,99 @@ To check/discuss: To put subquery cache on all levels of subqueries or on highest level only. + + +<contents> +1. Scope of the task +2. Data structure used for the cache +3. Cache size +4. Interplay with other subquery optimizations +5. User interface +</contents> + +1. Scope of the task +-------------------- +This WL should handle all subquery predicates, i.e. it should handle these +cases: + + outer_expr IN (SELECT correlated_select) + outer_expr $CMP$ ALL/ANY (SELECT correlated_select) + EXISTS (SELECT correlated_select) + scalar-context subquery: (SELECT correlated_select) + +The cache will maintain + + (outer_expr, correlation_references)-> subquery_item_result + +mapping, where +- correlation_references is a list of tablename.column_name that are referred + from the correlated_select but tablename is a table that is ouside the + subquery. +- subquery_item_result is 'bool' for subquery predicates, and is of +some scalar or ROW(scalar1,...scalarN) type for scalar-context subquery. + +We dont support cases when outer_expr or correlation_references are blobs. + +2. Data structure used for the cache +------------------------------------ +There are two data structures available in the codebase that will allow fast +equality lookups: + +1. HASH (mysys/hash.c) tables +2. Temporary tables (the ones that are used for e.g. GROUP BY) + +None of them has any support for element eviction on overflow (using LRU or +some other policy). + +Query cache and MyISAM/Maria's key/page cache ought to support some eviction +mechanism, but code-wise it is not readily reusable, one will need to factor +it out (or copy it). + +We choose to use #2, and not to have any eviction policy. See subsequent +sections for details and reasoning behind the decision. + +3. Cache size +------------- +Typically, a cache has some maximum size and a policy which is used to +select a cache entry for removal when the cache becomes full (e.g. find +and remove the least [recently] used entry) + +For this WL entry we will use a cache of infinite size. The reasoning behind +this is that: +- is is easy to do: we have temporary tables that can grow to arbitrarily + large size while still providing the same insert/lookup interface. +- it suits us: unless the subquery is resolved with one index lookup, + hitting the cache would be many times cheaper than re-running the + subquery, so cache is worth having. + +4. Interplay with other subquery optimizations +---------------------------------------------- +* This WL entry should not care about IN->EXISTS transformation: caching for + IN subquery and result of its conversion to EXISTS would work in the same + way. + +* This optimization is orthogonal to <=>ANY -> MIN/MAX rewrite (it will + work/be useful irrespectively of whether the rewrite has been performed or + not) + +* TODO: compare this with materialization for uncorrelated IN-subqueries. Is + this basically the same? + A: no, it is not: + - IN-Materialization has to perform full materialization before it can + do the first subquery evaluation. This WL's code has almost no startup + costs. + - This optimization has temp.table of (corr_reference, predicate_value), + while IN-materialization will have (corr_reference) only. + +5. User interface +----------------- +* There will be an @@optimizer_switch flag to turn this optimization on and + off (TODO: name of the flag?) + +* TODO: how do we show this in EXPLAIN [EXTENDED]? The most easiest is to + print something in the warning text of EXPLAIN EXTEDED that would indicate + use of cache. + +* temporary table sizing (max size for heap table, whether to use MyISAM or + Maria) will be controlled with common temp.table control variables. + -=-=(Psergey - Mon, 11 Jan 2010, 13:25)=-=- As of today, there is code that - collects outside references - creates a temporary table with index that would allow for fast lookups. there is no code to - fill the temporary table - make lookups into it Reported zero hours worked. Estimate unchanged. -=-=(Sanja - Fri, 11 Dec 2009, 15:09)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.28164 2009-12-11 15:09:15.000000000 +0200 +++ /tmp/wklog.66.new.28164 2009-12-11 15:09:15.000000000 +0200 @@ -3,4 +3,4 @@ To check/discuss: -Are there sens to put subquery cache on all levels of subqueries of on highest. + To put subquery cache on all levels of subqueries or on highest level only. -=-=(Sanja - Fri, 11 Dec 2009, 15:08)=-=- Low Level Design modified. --- /tmp/wklog.66.old.28072 2009-12-11 15:08:16.000000000 +0200 +++ /tmp/wklog.66.new.28072 2009-12-11 15:08:16.000000000 +0200 @@ -1 +1,6 @@ +All items on which subquery depend could be collected in +st_select_lex::mark_as_dependent (direct of indirect reference?) + +Temporary table index should be created by all fields except result field +(TMP_TABLE_PARAM::keyinfo). -=-=(Sanja - Fri, 11 Dec 2009, 15:05)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.27795 2009-12-11 15:05:04.000000000 +0200 +++ /tmp/wklog.66.new.27795 2009-12-11 15:05:04.000000000 +0200 @@ -1 +1,6 @@ +Attach subquery cache to each Item_subquery. Interface should allow to use hash +or temporary table inside. + +To check/discuss: +Are there sens to put subquery cache on all levels of subqueries of on highest. DESCRIPTION: Collect all outer items/references (left part of the subquiery and outer references inside the subquery) in key string. Compare the string (which represents certain value set of the references) against values in hash table and return cached result of subquery if the reference values combination has already been used. For example in the following subquery: (L1, L2) IN (SELECT A, B FROM T WHERE T.F1>OTER_FIELD) set of references to look into the subquery cache is (L1, L2, OTER_FIELD). The subquery cache should be implemented as simple LRU connected to the subquery. Size of the subquery cache (in number of results (but maybe in used memory amount)) is limited by session variable (query parameter?). HIGH-LEVEL SPECIFICATION: Attach subquery cache to each Item_subquery. Interface should allow to use hash or temporary table inside. To check/discuss: To put subquery cache on all levels of subqueries or on highest level only. <contents> 1. Scope of the task 2. Data structure used for the cache 3. Cache size 4. Interplay with other subquery optimizations 5. User interface </contents> 1. Scope of the task -------------------- This WL should handle all subquery predicates, i.e. it should handle these cases: outer_expr IN (SELECT correlated_select) outer_expr $CMP$ ALL/ANY (SELECT correlated_select) EXISTS (SELECT correlated_select) scalar-context subquery: (SELECT correlated_select) The cache will maintain (outer_expr, correlation_references)-> subquery_item_result mapping, where - correlation_references is a list of tablename.column_name that are referred from the correlated_select but tablename is a table that is ouside the subquery. - subquery_item_result is 'bool' for subquery predicates, and is of some scalar or ROW(scalar1,...scalarN) type for scalar-context subquery. We dont support cases when outer_expr or correlation_references are blobs. 2. Data structure used for the cache ------------------------------------ There are two data structures available in the codebase that will allow fast equality lookups: 1. HASH (mysys/hash.c) tables 2. Temporary tables (the ones that are used for e.g. GROUP BY) None of them has any support for element eviction on overflow (using LRU or some other policy). Query cache and MyISAM/Maria's key/page cache ought to support some eviction mechanism, but code-wise it is not readily reusable, one will need to factor it out (or copy it). We choose to use #2, and not to have any eviction policy. See subsequent sections for details and reasoning behind the decision. 3. Cache size ------------- Typically, a cache has some maximum size and a policy which is used to select a cache entry for removal when the cache becomes full (e.g. find and remove the least [recently] used entry) For this WL entry we will use a cache of infinite size. The reasoning behind this is that: - is is easy to do: we have temporary tables that can grow to arbitrarily large size while still providing the same insert/lookup interface. - it suits us: unless the subquery is resolved with one index lookup, hitting the cache would be many times cheaper than re-running the subquery, so cache is worth having. 4. Interplay with other subquery optimizations ---------------------------------------------- * This WL entry should not care about IN->EXISTS transformation: caching for IN subquery and result of its conversion to EXISTS would work in the same way. * This optimization is orthogonal to <=>ANY -> MIN/MAX rewrite (it will work/be useful irrespectively of whether the rewrite has been performed or not) * TODO: compare this with materialization for uncorrelated IN-subqueries. Is this basically the same? A: no, it is not: - IN-Materialization has to perform full materialization before it can do the first subquery evaluation. This WL's code has almost no startup costs. - This optimization has temp.table of (corr_reference, predicate_value), while IN-materialization will have (corr_reference) only. 5. User interface ----------------- * There will be an @@optimizer_switch flag to turn this optimization on and off (TODO: name of the flag?) * TODO: how do we show this in EXPLAIN [EXTENDED]? The most easiest is to print something in the warning text of EXPLAIN EXTEDED that would indicate use of cache. * temporary table sizing (max size for heap table, whether to use MyISAM or Maria) will be controlled with common temp.table control variables. LOW-LEVEL DESIGN: * Target version: base on mysql-5.2 code All items on which subquery depend could be collected in st_select_lex::mark_as_dependent (direct of indirect reference?) Temporary table index should be created by all fields except result field (TMP_TABLE_PARAM::keyinfo). How to fill the temptable ------------------------- Can reuse approach from SJ-Materialization. Its code is in end_sj_materialize() and is supposed to be quite trivial. How to make lookups into temptable ---------------------------------- We'll reuse approach used by SJ-Materialization in 6.0. Setup process ~~~~~~~~~~~~~ Setup is performed in the same way as in setup_sj_materialization(), see the code that starts these lines: /* Create/initialize everything we will need to index lookups into the temptable. */ and ends at this line: Remove the injected semi-join IN-equalities from join_tab conds. This <questionable> We'll also need to check equalities, i.e. do an equivalent of this: if (!(sjm->in_equality= create_subq_in_equalities(thd, sjm, emb_sj_nest->sj_subq_pred))) DBUG_RETURN(TRUE); /* purecov: inspected */ Question: or perhaps that is not necessarry? </questionable> Doing the lookup ~~~~~~~~~~~~~~~~ SJ-Materialization does lookup in sub_select_sjm(), with this code: /* Do index lookup in the materialized table */ if ((res= join_read_key2(join_tab, sjm->table, sjm->tab_ref)) == 1) DBUG_RETURN(NESTED_LOOP_ERROR); /* purecov: inspected */ if (res || !sjm->in_equality->val_int()) DBUG_RETURN(NESTED_LOOP_NO_MORE_ROWS); The code in this WL will use the same approach Extracting the value of the subquery predicate ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The goal of making the lookup is to get the value of subquery predicate. This is done by creating an Item_field $I which refers to appropriate temporary table's field and then subquery_predicate->val_int() will invoke $I->val_int(), subquery_predicate->val_str() will invoke $I->val_str() and so forth. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Psergey): Subquery optimization: Avoid recalculating subquery if external fields values found in subquery cache (66)
by worklog-noreply＠askmonty.org 18 Jan '10

18 Jan '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Subquery optimization: Avoid recalculating subquery if external fields values found in subquery cache CREATION DATE..: Wed, 25 Nov 2009, 22:25 SUPERVISOR.....: Monty IMPLEMENTOR....: Sanja COPIES TO......: CATEGORY.......: Server-BackLog TASK ID........: 66 (http://askmonty.org/worklog/?tid=66) VERSION........: Server-5.2 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Psergey - Mon, 18 Jan 2010, 16:40)=-=- Low Level Design modified. --- /tmp/wklog.66.old.24899 2010-01-18 16:40:16.000000000 +0200 +++ /tmp/wklog.66.new.24899 2010-01-18 16:40:16.000000000 +0200 @@ -1,3 +1,5 @@ +* Target version: base on mysql-5.2 code + All items on which subquery depend could be collected in st_select_lex::mark_as_dependent (direct of indirect reference?) -=-=(Psergey - Mon, 18 Jan 2010, 16:37)=-=- Low Level Design modified. --- /tmp/wklog.66.old.24586 2010-01-18 16:37:07.000000000 +0200 +++ /tmp/wklog.66.new.24586 2010-01-18 16:37:07.000000000 +0200 @@ -4,6 +4,11 @@ Temporary table index should be created by all fields except result field (TMP_TABLE_PARAM::keyinfo). +How to fill the temptable +------------------------- +Can reuse approach from SJ-Materialization. Its code is in end_sj_materialize() +and is supposed to be quite trivial. + How to make lookups into temptable ---------------------------------- We'll reuse approach used by SJ-Materialization in 6.0. -=-=(Psergey - Mon, 18 Jan 2010, 16:34)=-=- Low Level Design modified. --- /tmp/wklog.66.old.24328 2010-01-18 16:34:19.000000000 +0200 +++ /tmp/wklog.66.new.24328 2010-01-18 16:34:19.000000000 +0200 @@ -32,8 +32,8 @@ Question: or perhaps that is not necessarry? </questionable> -Execution process -~~~~~~~~~~~~~~~~~ +Doing the lookup +~~~~~~~~~~~~~~~~ SJ-Materialization does lookup in sub_select_sjm(), with this code: /* Do index lookup in the materialized table */ @@ -42,4 +42,12 @@ if (res || !sjm->in_equality->val_int()) DBUG_RETURN(NESTED_LOOP_NO_MORE_ROWS); +The code in this WL will use the same approach +Extracting the value of the subquery predicate +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The goal of making the lookup is to get the value of subquery predicate. +This is done by creating an Item_field $I which refers to appropriate +temporary table's field and then subquery_predicate->val_int() will invoke +$I->val_int(), subquery_predicate->val_str() will invoke $I->val_str() and so +forth. -=-=(Psergey - Mon, 18 Jan 2010, 16:23)=-=- Low Level Design modified. --- /tmp/wklog.66.old.23203 2010-01-18 16:23:18.000000000 +0200 +++ /tmp/wklog.66.new.23203 2010-01-18 16:23:18.000000000 +0200 @@ -31,3 +31,15 @@ Question: or perhaps that is not necessarry? </questionable> + +Execution process +~~~~~~~~~~~~~~~~~ +SJ-Materialization does lookup in sub_select_sjm(), with this code: + + /* Do index lookup in the materialized table */ + if ((res= join_read_key2(join_tab, sjm->table, sjm->tab_ref)) == 1) + DBUG_RETURN(NESTED_LOOP_ERROR); /* purecov: inspected */ + if (res || !sjm->in_equality->val_int()) + DBUG_RETURN(NESTED_LOOP_NO_MORE_ROWS); + + -=-=(Psergey - Mon, 18 Jan 2010, 16:22)=-=- Low Level Design modified. --- /tmp/wklog.66.old.23076 2010-01-18 16:22:07.000000000 +0200 +++ /tmp/wklog.66.new.23076 2010-01-18 16:22:07.000000000 +0200 @@ -4,3 +4,30 @@ Temporary table index should be created by all fields except result field (TMP_TABLE_PARAM::keyinfo). +How to make lookups into temptable +---------------------------------- +We'll reuse approach used by SJ-Materialization in 6.0. + +Setup process +~~~~~~~~~~~~~ +Setup is performed in the same way as in setup_sj_materialization(), +see the code that starts these lines: + + /* + Create/initialize everything we will need to index lookups into the + temptable. + */ + +and ends at this line: + + Remove the injected semi-join IN-equalities from join_tab conds. This + +<questionable> +We'll also need to check equalities, i.e. do an equivalent of this: + + if (!(sjm->in_equality= create_subq_in_equalities(thd, sjm, + emb_sj_nest->sj_subq_pred))) + DBUG_RETURN(TRUE); /* purecov: inspected */ + +Question: or perhaps that is not necessarry? +</questionable> -=-=(Psergey - Tue, 12 Jan 2010, 18:39)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.31666 2010-01-12 18:39:43.000000000 +0200 +++ /tmp/wklog.66.new.31666 2010-01-12 18:39:43.000000000 +0200 @@ -4,3 +4,99 @@ To check/discuss: To put subquery cache on all levels of subqueries or on highest level only. + + +<contents> +1. Scope of the task +2. Data structure used for the cache +3. Cache size +4. Interplay with other subquery optimizations +5. User interface +</contents> + +1. Scope of the task +-------------------- +This WL should handle all subquery predicates, i.e. it should handle these +cases: + + outer_expr IN (SELECT correlated_select) + outer_expr $CMP$ ALL/ANY (SELECT correlated_select) + EXISTS (SELECT correlated_select) + scalar-context subquery: (SELECT correlated_select) + +The cache will maintain + + (outer_expr, correlation_references)-> subquery_item_result + +mapping, where +- correlation_references is a list of tablename.column_name that are referred + from the correlated_select but tablename is a table that is ouside the + subquery. +- subquery_item_result is 'bool' for subquery predicates, and is of +some scalar or ROW(scalar1,...scalarN) type for scalar-context subquery. + +We dont support cases when outer_expr or correlation_references are blobs. + +2. Data structure used for the cache +------------------------------------ +There are two data structures available in the codebase that will allow fast +equality lookups: + +1. HASH (mysys/hash.c) tables +2. Temporary tables (the ones that are used for e.g. GROUP BY) + +None of them has any support for element eviction on overflow (using LRU or +some other policy). + +Query cache and MyISAM/Maria's key/page cache ought to support some eviction +mechanism, but code-wise it is not readily reusable, one will need to factor +it out (or copy it). + +We choose to use #2, and not to have any eviction policy. See subsequent +sections for details and reasoning behind the decision. + +3. Cache size +------------- +Typically, a cache has some maximum size and a policy which is used to +select a cache entry for removal when the cache becomes full (e.g. find +and remove the least [recently] used entry) + +For this WL entry we will use a cache of infinite size. The reasoning behind +this is that: +- is is easy to do: we have temporary tables that can grow to arbitrarily + large size while still providing the same insert/lookup interface. +- it suits us: unless the subquery is resolved with one index lookup, + hitting the cache would be many times cheaper than re-running the + subquery, so cache is worth having. + +4. Interplay with other subquery optimizations +---------------------------------------------- +* This WL entry should not care about IN->EXISTS transformation: caching for + IN subquery and result of its conversion to EXISTS would work in the same + way. + +* This optimization is orthogonal to <=>ANY -> MIN/MAX rewrite (it will + work/be useful irrespectively of whether the rewrite has been performed or + not) + +* TODO: compare this with materialization for uncorrelated IN-subqueries. Is + this basically the same? + A: no, it is not: + - IN-Materialization has to perform full materialization before it can + do the first subquery evaluation. This WL's code has almost no startup + costs. + - This optimization has temp.table of (corr_reference, predicate_value), + while IN-materialization will have (corr_reference) only. + +5. User interface +----------------- +* There will be an @@optimizer_switch flag to turn this optimization on and + off (TODO: name of the flag?) + +* TODO: how do we show this in EXPLAIN [EXTENDED]? The most easiest is to + print something in the warning text of EXPLAIN EXTEDED that would indicate + use of cache. + +* temporary table sizing (max size for heap table, whether to use MyISAM or + Maria) will be controlled with common temp.table control variables. + -=-=(Psergey - Mon, 11 Jan 2010, 13:25)=-=- As of today, there is code that - collects outside references - creates a temporary table with index that would allow for fast lookups. there is no code to - fill the temporary table - make lookups into it Reported zero hours worked. Estimate unchanged. -=-=(Sanja - Fri, 11 Dec 2009, 15:09)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.28164 2009-12-11 15:09:15.000000000 +0200 +++ /tmp/wklog.66.new.28164 2009-12-11 15:09:15.000000000 +0200 @@ -3,4 +3,4 @@ To check/discuss: -Are there sens to put subquery cache on all levels of subqueries of on highest. + To put subquery cache on all levels of subqueries or on highest level only. -=-=(Sanja - Fri, 11 Dec 2009, 15:08)=-=- Low Level Design modified. --- /tmp/wklog.66.old.28072 2009-12-11 15:08:16.000000000 +0200 +++ /tmp/wklog.66.new.28072 2009-12-11 15:08:16.000000000 +0200 @@ -1 +1,6 @@ +All items on which subquery depend could be collected in +st_select_lex::mark_as_dependent (direct of indirect reference?) + +Temporary table index should be created by all fields except result field +(TMP_TABLE_PARAM::keyinfo). -=-=(Sanja - Fri, 11 Dec 2009, 15:05)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.27795 2009-12-11 15:05:04.000000000 +0200 +++ /tmp/wklog.66.new.27795 2009-12-11 15:05:04.000000000 +0200 @@ -1 +1,6 @@ +Attach subquery cache to each Item_subquery. Interface should allow to use hash +or temporary table inside. + +To check/discuss: +Are there sens to put subquery cache on all levels of subqueries of on highest. DESCRIPTION: Collect all outer items/references (left part of the subquiery and outer references inside the subquery) in key string. Compare the string (which represents certain value set of the references) against values in hash table and return cached result of subquery if the reference values combination has already been used. For example in the following subquery: (L1, L2) IN (SELECT A, B FROM T WHERE T.F1>OTER_FIELD) set of references to look into the subquery cache is (L1, L2, OTER_FIELD). The subquery cache should be implemented as simple LRU connected to the subquery. Size of the subquery cache (in number of results (but maybe in used memory amount)) is limited by session variable (query parameter?). HIGH-LEVEL SPECIFICATION: Attach subquery cache to each Item_subquery. Interface should allow to use hash or temporary table inside. To check/discuss: To put subquery cache on all levels of subqueries or on highest level only. <contents> 1. Scope of the task 2. Data structure used for the cache 3. Cache size 4. Interplay with other subquery optimizations 5. User interface </contents> 1. Scope of the task -------------------- This WL should handle all subquery predicates, i.e. it should handle these cases: outer_expr IN (SELECT correlated_select) outer_expr $CMP$ ALL/ANY (SELECT correlated_select) EXISTS (SELECT correlated_select) scalar-context subquery: (SELECT correlated_select) The cache will maintain (outer_expr, correlation_references)-> subquery_item_result mapping, where - correlation_references is a list of tablename.column_name that are referred from the correlated_select but tablename is a table that is ouside the subquery. - subquery_item_result is 'bool' for subquery predicates, and is of some scalar or ROW(scalar1,...scalarN) type for scalar-context subquery. We dont support cases when outer_expr or correlation_references are blobs. 2. Data structure used for the cache ------------------------------------ There are two data structures available in the codebase that will allow fast equality lookups: 1. HASH (mysys/hash.c) tables 2. Temporary tables (the ones that are used for e.g. GROUP BY) None of them has any support for element eviction on overflow (using LRU or some other policy). Query cache and MyISAM/Maria's key/page cache ought to support some eviction mechanism, but code-wise it is not readily reusable, one will need to factor it out (or copy it). We choose to use #2, and not to have any eviction policy. See subsequent sections for details and reasoning behind the decision. 3. Cache size ------------- Typically, a cache has some maximum size and a policy which is used to select a cache entry for removal when the cache becomes full (e.g. find and remove the least [recently] used entry) For this WL entry we will use a cache of infinite size. The reasoning behind this is that: - is is easy to do: we have temporary tables that can grow to arbitrarily large size while still providing the same insert/lookup interface. - it suits us: unless the subquery is resolved with one index lookup, hitting the cache would be many times cheaper than re-running the subquery, so cache is worth having. 4. Interplay with other subquery optimizations ---------------------------------------------- * This WL entry should not care about IN->EXISTS transformation: caching for IN subquery and result of its conversion to EXISTS would work in the same way. * This optimization is orthogonal to <=>ANY -> MIN/MAX rewrite (it will work/be useful irrespectively of whether the rewrite has been performed or not) * TODO: compare this with materialization for uncorrelated IN-subqueries. Is this basically the same? A: no, it is not: - IN-Materialization has to perform full materialization before it can do the first subquery evaluation. This WL's code has almost no startup costs. - This optimization has temp.table of (corr_reference, predicate_value), while IN-materialization will have (corr_reference) only. 5. User interface ----------------- * There will be an @@optimizer_switch flag to turn this optimization on and off (TODO: name of the flag?) * TODO: how do we show this in EXPLAIN [EXTENDED]? The most easiest is to print something in the warning text of EXPLAIN EXTEDED that would indicate use of cache. * temporary table sizing (max size for heap table, whether to use MyISAM or Maria) will be controlled with common temp.table control variables. LOW-LEVEL DESIGN: * Target version: base on mysql-5.2 code All items on which subquery depend could be collected in st_select_lex::mark_as_dependent (direct of indirect reference?) Temporary table index should be created by all fields except result field (TMP_TABLE_PARAM::keyinfo). How to fill the temptable ------------------------- Can reuse approach from SJ-Materialization. Its code is in end_sj_materialize() and is supposed to be quite trivial. How to make lookups into temptable ---------------------------------- We'll reuse approach used by SJ-Materialization in 6.0. Setup process ~~~~~~~~~~~~~ Setup is performed in the same way as in setup_sj_materialization(), see the code that starts these lines: /* Create/initialize everything we will need to index lookups into the temptable. */ and ends at this line: Remove the injected semi-join IN-equalities from join_tab conds. This <questionable> We'll also need to check equalities, i.e. do an equivalent of this: if (!(sjm->in_equality= create_subq_in_equalities(thd, sjm, emb_sj_nest->sj_subq_pred))) DBUG_RETURN(TRUE); /* purecov: inspected */ Question: or perhaps that is not necessarry? </questionable> Doing the lookup ~~~~~~~~~~~~~~~~ SJ-Materialization does lookup in sub_select_sjm(), with this code: /* Do index lookup in the materialized table */ if ((res= join_read_key2(join_tab, sjm->table, sjm->tab_ref)) == 1) DBUG_RETURN(NESTED_LOOP_ERROR); /* purecov: inspected */ if (res || !sjm->in_equality->val_int()) DBUG_RETURN(NESTED_LOOP_NO_MORE_ROWS); The code in this WL will use the same approach Extracting the value of the subquery predicate ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The goal of making the lookup is to get the value of subquery predicate. This is done by creating an Item_field $I which refers to appropriate temporary table's field and then subquery_predicate->val_int() will invoke $I->val_int(), subquery_predicate->val_str() will invoke $I->val_str() and so forth. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Psergey): Subquery optimization: Avoid recalculating subquery if external fields values found in subquery cache (66)
by worklog-noreply＠askmonty.org 18 Jan '10

18 Jan '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Subquery optimization: Avoid recalculating subquery if external fields values found in subquery cache CREATION DATE..: Wed, 25 Nov 2009, 22:25 SUPERVISOR.....: Monty IMPLEMENTOR....: Sanja COPIES TO......: CATEGORY.......: Server-BackLog TASK ID........: 66 (http://askmonty.org/worklog/?tid=66) VERSION........: Server-5.2 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Psergey - Mon, 18 Jan 2010, 16:37)=-=- Low Level Design modified. --- /tmp/wklog.66.old.24586 2010-01-18 16:37:07.000000000 +0200 +++ /tmp/wklog.66.new.24586 2010-01-18 16:37:07.000000000 +0200 @@ -4,6 +4,11 @@ Temporary table index should be created by all fields except result field (TMP_TABLE_PARAM::keyinfo). +How to fill the temptable +------------------------- +Can reuse approach from SJ-Materialization. Its code is in end_sj_materialize() +and is supposed to be quite trivial. + How to make lookups into temptable ---------------------------------- We'll reuse approach used by SJ-Materialization in 6.0. -=-=(Psergey - Mon, 18 Jan 2010, 16:34)=-=- Low Level Design modified. --- /tmp/wklog.66.old.24328 2010-01-18 16:34:19.000000000 +0200 +++ /tmp/wklog.66.new.24328 2010-01-18 16:34:19.000000000 +0200 @@ -32,8 +32,8 @@ Question: or perhaps that is not necessarry? </questionable> -Execution process -~~~~~~~~~~~~~~~~~ +Doing the lookup +~~~~~~~~~~~~~~~~ SJ-Materialization does lookup in sub_select_sjm(), with this code: /* Do index lookup in the materialized table */ @@ -42,4 +42,12 @@ if (res || !sjm->in_equality->val_int()) DBUG_RETURN(NESTED_LOOP_NO_MORE_ROWS); +The code in this WL will use the same approach +Extracting the value of the subquery predicate +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The goal of making the lookup is to get the value of subquery predicate. +This is done by creating an Item_field $I which refers to appropriate +temporary table's field and then subquery_predicate->val_int() will invoke +$I->val_int(), subquery_predicate->val_str() will invoke $I->val_str() and so +forth. -=-=(Psergey - Mon, 18 Jan 2010, 16:23)=-=- Low Level Design modified. --- /tmp/wklog.66.old.23203 2010-01-18 16:23:18.000000000 +0200 +++ /tmp/wklog.66.new.23203 2010-01-18 16:23:18.000000000 +0200 @@ -31,3 +31,15 @@ Question: or perhaps that is not necessarry? </questionable> + +Execution process +~~~~~~~~~~~~~~~~~ +SJ-Materialization does lookup in sub_select_sjm(), with this code: + + /* Do index lookup in the materialized table */ + if ((res= join_read_key2(join_tab, sjm->table, sjm->tab_ref)) == 1) + DBUG_RETURN(NESTED_LOOP_ERROR); /* purecov: inspected */ + if (res || !sjm->in_equality->val_int()) + DBUG_RETURN(NESTED_LOOP_NO_MORE_ROWS); + + -=-=(Psergey - Mon, 18 Jan 2010, 16:22)=-=- Low Level Design modified. --- /tmp/wklog.66.old.23076 2010-01-18 16:22:07.000000000 +0200 +++ /tmp/wklog.66.new.23076 2010-01-18 16:22:07.000000000 +0200 @@ -4,3 +4,30 @@ Temporary table index should be created by all fields except result field (TMP_TABLE_PARAM::keyinfo). +How to make lookups into temptable +---------------------------------- +We'll reuse approach used by SJ-Materialization in 6.0. + +Setup process +~~~~~~~~~~~~~ +Setup is performed in the same way as in setup_sj_materialization(), +see the code that starts these lines: + + /* + Create/initialize everything we will need to index lookups into the + temptable. + */ + +and ends at this line: + + Remove the injected semi-join IN-equalities from join_tab conds. This + +<questionable> +We'll also need to check equalities, i.e. do an equivalent of this: + + if (!(sjm->in_equality= create_subq_in_equalities(thd, sjm, + emb_sj_nest->sj_subq_pred))) + DBUG_RETURN(TRUE); /* purecov: inspected */ + +Question: or perhaps that is not necessarry? +</questionable> -=-=(Psergey - Tue, 12 Jan 2010, 18:39)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.31666 2010-01-12 18:39:43.000000000 +0200 +++ /tmp/wklog.66.new.31666 2010-01-12 18:39:43.000000000 +0200 @@ -4,3 +4,99 @@ To check/discuss: To put subquery cache on all levels of subqueries or on highest level only. + + +<contents> +1. Scope of the task +2. Data structure used for the cache +3. Cache size +4. Interplay with other subquery optimizations +5. User interface +</contents> + +1. Scope of the task +-------------------- +This WL should handle all subquery predicates, i.e. it should handle these +cases: + + outer_expr IN (SELECT correlated_select) + outer_expr $CMP$ ALL/ANY (SELECT correlated_select) + EXISTS (SELECT correlated_select) + scalar-context subquery: (SELECT correlated_select) + +The cache will maintain + + (outer_expr, correlation_references)-> subquery_item_result + +mapping, where +- correlation_references is a list of tablename.column_name that are referred + from the correlated_select but tablename is a table that is ouside the + subquery. +- subquery_item_result is 'bool' for subquery predicates, and is of +some scalar or ROW(scalar1,...scalarN) type for scalar-context subquery. + +We dont support cases when outer_expr or correlation_references are blobs. + +2. Data structure used for the cache +------------------------------------ +There are two data structures available in the codebase that will allow fast +equality lookups: + +1. HASH (mysys/hash.c) tables +2. Temporary tables (the ones that are used for e.g. GROUP BY) + +None of them has any support for element eviction on overflow (using LRU or +some other policy). + +Query cache and MyISAM/Maria's key/page cache ought to support some eviction +mechanism, but code-wise it is not readily reusable, one will need to factor +it out (or copy it). + +We choose to use #2, and not to have any eviction policy. See subsequent +sections for details and reasoning behind the decision. + +3. Cache size +------------- +Typically, a cache has some maximum size and a policy which is used to +select a cache entry for removal when the cache becomes full (e.g. find +and remove the least [recently] used entry) + +For this WL entry we will use a cache of infinite size. The reasoning behind +this is that: +- is is easy to do: we have temporary tables that can grow to arbitrarily + large size while still providing the same insert/lookup interface. +- it suits us: unless the subquery is resolved with one index lookup, + hitting the cache would be many times cheaper than re-running the + subquery, so cache is worth having. + +4. Interplay with other subquery optimizations +---------------------------------------------- +* This WL entry should not care about IN->EXISTS transformation: caching for + IN subquery and result of its conversion to EXISTS would work in the same + way. + +* This optimization is orthogonal to <=>ANY -> MIN/MAX rewrite (it will + work/be useful irrespectively of whether the rewrite has been performed or + not) + +* TODO: compare this with materialization for uncorrelated IN-subqueries. Is + this basically the same? + A: no, it is not: + - IN-Materialization has to perform full materialization before it can + do the first subquery evaluation. This WL's code has almost no startup + costs. + - This optimization has temp.table of (corr_reference, predicate_value), + while IN-materialization will have (corr_reference) only. + +5. User interface +----------------- +* There will be an @@optimizer_switch flag to turn this optimization on and + off (TODO: name of the flag?) + +* TODO: how do we show this in EXPLAIN [EXTENDED]? The most easiest is to + print something in the warning text of EXPLAIN EXTEDED that would indicate + use of cache. + +* temporary table sizing (max size for heap table, whether to use MyISAM or + Maria) will be controlled with common temp.table control variables. + -=-=(Psergey - Mon, 11 Jan 2010, 13:25)=-=- As of today, there is code that - collects outside references - creates a temporary table with index that would allow for fast lookups. there is no code to - fill the temporary table - make lookups into it Reported zero hours worked. Estimate unchanged. -=-=(Sanja - Fri, 11 Dec 2009, 15:09)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.28164 2009-12-11 15:09:15.000000000 +0200 +++ /tmp/wklog.66.new.28164 2009-12-11 15:09:15.000000000 +0200 @@ -3,4 +3,4 @@ To check/discuss: -Are there sens to put subquery cache on all levels of subqueries of on highest. + To put subquery cache on all levels of subqueries or on highest level only. -=-=(Sanja - Fri, 11 Dec 2009, 15:08)=-=- Low Level Design modified. --- /tmp/wklog.66.old.28072 2009-12-11 15:08:16.000000000 +0200 +++ /tmp/wklog.66.new.28072 2009-12-11 15:08:16.000000000 +0200 @@ -1 +1,6 @@ +All items on which subquery depend could be collected in +st_select_lex::mark_as_dependent (direct of indirect reference?) + +Temporary table index should be created by all fields except result field +(TMP_TABLE_PARAM::keyinfo). -=-=(Sanja - Fri, 11 Dec 2009, 15:05)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.27795 2009-12-11 15:05:04.000000000 +0200 +++ /tmp/wklog.66.new.27795 2009-12-11 15:05:04.000000000 +0200 @@ -1 +1,6 @@ +Attach subquery cache to each Item_subquery. Interface should allow to use hash +or temporary table inside. + +To check/discuss: +Are there sens to put subquery cache on all levels of subqueries of on highest. DESCRIPTION: Collect all outer items/references (left part of the subquiery and outer references inside the subquery) in key string. Compare the string (which represents certain value set of the references) against values in hash table and return cached result of subquery if the reference values combination has already been used. For example in the following subquery: (L1, L2) IN (SELECT A, B FROM T WHERE T.F1>OTER_FIELD) set of references to look into the subquery cache is (L1, L2, OTER_FIELD). The subquery cache should be implemented as simple LRU connected to the subquery. Size of the subquery cache (in number of results (but maybe in used memory amount)) is limited by session variable (query parameter?). HIGH-LEVEL SPECIFICATION: Attach subquery cache to each Item_subquery. Interface should allow to use hash or temporary table inside. To check/discuss: To put subquery cache on all levels of subqueries or on highest level only. <contents> 1. Scope of the task 2. Data structure used for the cache 3. Cache size 4. Interplay with other subquery optimizations 5. User interface </contents> 1. Scope of the task -------------------- This WL should handle all subquery predicates, i.e. it should handle these cases: outer_expr IN (SELECT correlated_select) outer_expr $CMP$ ALL/ANY (SELECT correlated_select) EXISTS (SELECT correlated_select) scalar-context subquery: (SELECT correlated_select) The cache will maintain (outer_expr, correlation_references)-> subquery_item_result mapping, where - correlation_references is a list of tablename.column_name that are referred from the correlated_select but tablename is a table that is ouside the subquery. - subquery_item_result is 'bool' for subquery predicates, and is of some scalar or ROW(scalar1,...scalarN) type for scalar-context subquery. We dont support cases when outer_expr or correlation_references are blobs. 2. Data structure used for the cache ------------------------------------ There are two data structures available in the codebase that will allow fast equality lookups: 1. HASH (mysys/hash.c) tables 2. Temporary tables (the ones that are used for e.g. GROUP BY) None of them has any support for element eviction on overflow (using LRU or some other policy). Query cache and MyISAM/Maria's key/page cache ought to support some eviction mechanism, but code-wise it is not readily reusable, one will need to factor it out (or copy it). We choose to use #2, and not to have any eviction policy. See subsequent sections for details and reasoning behind the decision. 3. Cache size ------------- Typically, a cache has some maximum size and a policy which is used to select a cache entry for removal when the cache becomes full (e.g. find and remove the least [recently] used entry) For this WL entry we will use a cache of infinite size. The reasoning behind this is that: - is is easy to do: we have temporary tables that can grow to arbitrarily large size while still providing the same insert/lookup interface. - it suits us: unless the subquery is resolved with one index lookup, hitting the cache would be many times cheaper than re-running the subquery, so cache is worth having. 4. Interplay with other subquery optimizations ---------------------------------------------- * This WL entry should not care about IN->EXISTS transformation: caching for IN subquery and result of its conversion to EXISTS would work in the same way. * This optimization is orthogonal to <=>ANY -> MIN/MAX rewrite (it will work/be useful irrespectively of whether the rewrite has been performed or not) * TODO: compare this with materialization for uncorrelated IN-subqueries. Is this basically the same? A: no, it is not: - IN-Materialization has to perform full materialization before it can do the first subquery evaluation. This WL's code has almost no startup costs. - This optimization has temp.table of (corr_reference, predicate_value), while IN-materialization will have (corr_reference) only. 5. User interface ----------------- * There will be an @@optimizer_switch flag to turn this optimization on and off (TODO: name of the flag?) * TODO: how do we show this in EXPLAIN [EXTENDED]? The most easiest is to print something in the warning text of EXPLAIN EXTEDED that would indicate use of cache. * temporary table sizing (max size for heap table, whether to use MyISAM or Maria) will be controlled with common temp.table control variables. LOW-LEVEL DESIGN: All items on which subquery depend could be collected in st_select_lex::mark_as_dependent (direct of indirect reference?) Temporary table index should be created by all fields except result field (TMP_TABLE_PARAM::keyinfo). How to fill the temptable ------------------------- Can reuse approach from SJ-Materialization. Its code is in end_sj_materialize() and is supposed to be quite trivial. How to make lookups into temptable ---------------------------------- We'll reuse approach used by SJ-Materialization in 6.0. Setup process ~~~~~~~~~~~~~ Setup is performed in the same way as in setup_sj_materialization(), see the code that starts these lines: /* Create/initialize everything we will need to index lookups into the temptable. */ and ends at this line: Remove the injected semi-join IN-equalities from join_tab conds. This <questionable> We'll also need to check equalities, i.e. do an equivalent of this: if (!(sjm->in_equality= create_subq_in_equalities(thd, sjm, emb_sj_nest->sj_subq_pred))) DBUG_RETURN(TRUE); /* purecov: inspected */ Question: or perhaps that is not necessarry? </questionable> Doing the lookup ~~~~~~~~~~~~~~~~ SJ-Materialization does lookup in sub_select_sjm(), with this code: /* Do index lookup in the materialized table */ if ((res= join_read_key2(join_tab, sjm->table, sjm->tab_ref)) == 1) DBUG_RETURN(NESTED_LOOP_ERROR); /* purecov: inspected */ if (res || !sjm->in_equality->val_int()) DBUG_RETURN(NESTED_LOOP_NO_MORE_ROWS); The code in this WL will use the same approach Extracting the value of the subquery predicate ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The goal of making the lookup is to get the value of subquery predicate. This is done by creating an Item_field $I which refers to appropriate temporary table's field and then subquery_predicate->val_int() will invoke $I->val_int(), subquery_predicate->val_str() will invoke $I->val_str() and so forth. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Psergey): Subquery optimization: Avoid recalculating subquery if external fields values found in subquery cache (66)
by worklog-noreply＠askmonty.org 18 Jan '10

18 Jan '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Subquery optimization: Avoid recalculating subquery if external fields values found in subquery cache CREATION DATE..: Wed, 25 Nov 2009, 22:25 SUPERVISOR.....: Monty IMPLEMENTOR....: Sanja COPIES TO......: CATEGORY.......: Server-BackLog TASK ID........: 66 (http://askmonty.org/worklog/?tid=66) VERSION........: Server-5.2 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Psergey - Mon, 18 Jan 2010, 16:37)=-=- Low Level Design modified. --- /tmp/wklog.66.old.24586 2010-01-18 16:37:07.000000000 +0200 +++ /tmp/wklog.66.new.24586 2010-01-18 16:37:07.000000000 +0200 @@ -4,6 +4,11 @@ Temporary table index should be created by all fields except result field (TMP_TABLE_PARAM::keyinfo). +How to fill the temptable +------------------------- +Can reuse approach from SJ-Materialization. Its code is in end_sj_materialize() +and is supposed to be quite trivial. + How to make lookups into temptable ---------------------------------- We'll reuse approach used by SJ-Materialization in 6.0. -=-=(Psergey - Mon, 18 Jan 2010, 16:34)=-=- Low Level Design modified. --- /tmp/wklog.66.old.24328 2010-01-18 16:34:19.000000000 +0200 +++ /tmp/wklog.66.new.24328 2010-01-18 16:34:19.000000000 +0200 @@ -32,8 +32,8 @@ Question: or perhaps that is not necessarry? </questionable> -Execution process -~~~~~~~~~~~~~~~~~ +Doing the lookup +~~~~~~~~~~~~~~~~ SJ-Materialization does lookup in sub_select_sjm(), with this code: /* Do index lookup in the materialized table */ @@ -42,4 +42,12 @@ if (res || !sjm->in_equality->val_int()) DBUG_RETURN(NESTED_LOOP_NO_MORE_ROWS); +The code in this WL will use the same approach +Extracting the value of the subquery predicate +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The goal of making the lookup is to get the value of subquery predicate. +This is done by creating an Item_field $I which refers to appropriate +temporary table's field and then subquery_predicate->val_int() will invoke +$I->val_int(), subquery_predicate->val_str() will invoke $I->val_str() and so +forth. -=-=(Psergey - Mon, 18 Jan 2010, 16:23)=-=- Low Level Design modified. --- /tmp/wklog.66.old.23203 2010-01-18 16:23:18.000000000 +0200 +++ /tmp/wklog.66.new.23203 2010-01-18 16:23:18.000000000 +0200 @@ -31,3 +31,15 @@ Question: or perhaps that is not necessarry? </questionable> + +Execution process +~~~~~~~~~~~~~~~~~ +SJ-Materialization does lookup in sub_select_sjm(), with this code: + + /* Do index lookup in the materialized table */ + if ((res= join_read_key2(join_tab, sjm->table, sjm->tab_ref)) == 1) + DBUG_RETURN(NESTED_LOOP_ERROR); /* purecov: inspected */ + if (res || !sjm->in_equality->val_int()) + DBUG_RETURN(NESTED_LOOP_NO_MORE_ROWS); + + -=-=(Psergey - Mon, 18 Jan 2010, 16:22)=-=- Low Level Design modified. --- /tmp/wklog.66.old.23076 2010-01-18 16:22:07.000000000 +0200 +++ /tmp/wklog.66.new.23076 2010-01-18 16:22:07.000000000 +0200 @@ -4,3 +4,30 @@ Temporary table index should be created by all fields except result field (TMP_TABLE_PARAM::keyinfo). +How to make lookups into temptable +---------------------------------- +We'll reuse approach used by SJ-Materialization in 6.0. + +Setup process +~~~~~~~~~~~~~ +Setup is performed in the same way as in setup_sj_materialization(), +see the code that starts these lines: + + /* + Create/initialize everything we will need to index lookups into the + temptable. + */ + +and ends at this line: + + Remove the injected semi-join IN-equalities from join_tab conds. This + +<questionable> +We'll also need to check equalities, i.e. do an equivalent of this: + + if (!(sjm->in_equality= create_subq_in_equalities(thd, sjm, + emb_sj_nest->sj_subq_pred))) + DBUG_RETURN(TRUE); /* purecov: inspected */ + +Question: or perhaps that is not necessarry? +</questionable> -=-=(Psergey - Tue, 12 Jan 2010, 18:39)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.31666 2010-01-12 18:39:43.000000000 +0200 +++ /tmp/wklog.66.new.31666 2010-01-12 18:39:43.000000000 +0200 @@ -4,3 +4,99 @@ To check/discuss: To put subquery cache on all levels of subqueries or on highest level only. + + +<contents> +1. Scope of the task +2. Data structure used for the cache +3. Cache size +4. Interplay with other subquery optimizations +5. User interface +</contents> + +1. Scope of the task +-------------------- +This WL should handle all subquery predicates, i.e. it should handle these +cases: + + outer_expr IN (SELECT correlated_select) + outer_expr $CMP$ ALL/ANY (SELECT correlated_select) + EXISTS (SELECT correlated_select) + scalar-context subquery: (SELECT correlated_select) + +The cache will maintain + + (outer_expr, correlation_references)-> subquery_item_result + +mapping, where +- correlation_references is a list of tablename.column_name that are referred + from the correlated_select but tablename is a table that is ouside the + subquery. +- subquery_item_result is 'bool' for subquery predicates, and is of +some scalar or ROW(scalar1,...scalarN) type for scalar-context subquery. + +We dont support cases when outer_expr or correlation_references are blobs. + +2. Data structure used for the cache +------------------------------------ +There are two data structures available in the codebase that will allow fast +equality lookups: + +1. HASH (mysys/hash.c) tables +2. Temporary tables (the ones that are used for e.g. GROUP BY) + +None of them has any support for element eviction on overflow (using LRU or +some other policy). + +Query cache and MyISAM/Maria's key/page cache ought to support some eviction +mechanism, but code-wise it is not readily reusable, one will need to factor +it out (or copy it). + +We choose to use #2, and not to have any eviction policy. See subsequent +sections for details and reasoning behind the decision. + +3. Cache size +------------- +Typically, a cache has some maximum size and a policy which is used to +select a cache entry for removal when the cache becomes full (e.g. find +and remove the least [recently] used entry) + +For this WL entry we will use a cache of infinite size. The reasoning behind +this is that: +- is is easy to do: we have temporary tables that can grow to arbitrarily + large size while still providing the same insert/lookup interface. +- it suits us: unless the subquery is resolved with one index lookup, + hitting the cache would be many times cheaper than re-running the + subquery, so cache is worth having. + +4. Interplay with other subquery optimizations +---------------------------------------------- +* This WL entry should not care about IN->EXISTS transformation: caching for + IN subquery and result of its conversion to EXISTS would work in the same + way. + +* This optimization is orthogonal to <=>ANY -> MIN/MAX rewrite (it will + work/be useful irrespectively of whether the rewrite has been performed or + not) + +* TODO: compare this with materialization for uncorrelated IN-subqueries. Is + this basically the same? + A: no, it is not: + - IN-Materialization has to perform full materialization before it can + do the first subquery evaluation. This WL's code has almost no startup + costs. + - This optimization has temp.table of (corr_reference, predicate_value), + while IN-materialization will have (corr_reference) only. + +5. User interface +----------------- +* There will be an @@optimizer_switch flag to turn this optimization on and + off (TODO: name of the flag?) + +* TODO: how do we show this in EXPLAIN [EXTENDED]? The most easiest is to + print something in the warning text of EXPLAIN EXTEDED that would indicate + use of cache. + +* temporary table sizing (max size for heap table, whether to use MyISAM or + Maria) will be controlled with common temp.table control variables. + -=-=(Psergey - Mon, 11 Jan 2010, 13:25)=-=- As of today, there is code that - collects outside references - creates a temporary table with index that would allow for fast lookups. there is no code to - fill the temporary table - make lookups into it Reported zero hours worked. Estimate unchanged. -=-=(Sanja - Fri, 11 Dec 2009, 15:09)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.28164 2009-12-11 15:09:15.000000000 +0200 +++ /tmp/wklog.66.new.28164 2009-12-11 15:09:15.000000000 +0200 @@ -3,4 +3,4 @@ To check/discuss: -Are there sens to put subquery cache on all levels of subqueries of on highest. + To put subquery cache on all levels of subqueries or on highest level only. -=-=(Sanja - Fri, 11 Dec 2009, 15:08)=-=- Low Level Design modified. --- /tmp/wklog.66.old.28072 2009-12-11 15:08:16.000000000 +0200 +++ /tmp/wklog.66.new.28072 2009-12-11 15:08:16.000000000 +0200 @@ -1 +1,6 @@ +All items on which subquery depend could be collected in +st_select_lex::mark_as_dependent (direct of indirect reference?) + +Temporary table index should be created by all fields except result field +(TMP_TABLE_PARAM::keyinfo). -=-=(Sanja - Fri, 11 Dec 2009, 15:05)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.27795 2009-12-11 15:05:04.000000000 +0200 +++ /tmp/wklog.66.new.27795 2009-12-11 15:05:04.000000000 +0200 @@ -1 +1,6 @@ +Attach subquery cache to each Item_subquery. Interface should allow to use hash +or temporary table inside. + +To check/discuss: +Are there sens to put subquery cache on all levels of subqueries of on highest. DESCRIPTION: Collect all outer items/references (left part of the subquiery and outer references inside the subquery) in key string. Compare the string (which represents certain value set of the references) against values in hash table and return cached result of subquery if the reference values combination has already been used. For example in the following subquery: (L1, L2) IN (SELECT A, B FROM T WHERE T.F1>OTER_FIELD) set of references to look into the subquery cache is (L1, L2, OTER_FIELD). The subquery cache should be implemented as simple LRU connected to the subquery. Size of the subquery cache (in number of results (but maybe in used memory amount)) is limited by session variable (query parameter?). HIGH-LEVEL SPECIFICATION: Attach subquery cache to each Item_subquery. Interface should allow to use hash or temporary table inside. To check/discuss: To put subquery cache on all levels of subqueries or on highest level only. <contents> 1. Scope of the task 2. Data structure used for the cache 3. Cache size 4. Interplay with other subquery optimizations 5. User interface </contents> 1. Scope of the task -------------------- This WL should handle all subquery predicates, i.e. it should handle these cases: outer_expr IN (SELECT correlated_select) outer_expr $CMP$ ALL/ANY (SELECT correlated_select) EXISTS (SELECT correlated_select) scalar-context subquery: (SELECT correlated_select) The cache will maintain (outer_expr, correlation_references)-> subquery_item_result mapping, where - correlation_references is a list of tablename.column_name that are referred from the correlated_select but tablename is a table that is ouside the subquery. - subquery_item_result is 'bool' for subquery predicates, and is of some scalar or ROW(scalar1,...scalarN) type for scalar-context subquery. We dont support cases when outer_expr or correlation_references are blobs. 2. Data structure used for the cache ------------------------------------ There are two data structures available in the codebase that will allow fast equality lookups: 1. HASH (mysys/hash.c) tables 2. Temporary tables (the ones that are used for e.g. GROUP BY) None of them has any support for element eviction on overflow (using LRU or some other policy). Query cache and MyISAM/Maria's key/page cache ought to support some eviction mechanism, but code-wise it is not readily reusable, one will need to factor it out (or copy it). We choose to use #2, and not to have any eviction policy. See subsequent sections for details and reasoning behind the decision. 3. Cache size ------------- Typically, a cache has some maximum size and a policy which is used to select a cache entry for removal when the cache becomes full (e.g. find and remove the least [recently] used entry) For this WL entry we will use a cache of infinite size. The reasoning behind this is that: - is is easy to do: we have temporary tables that can grow to arbitrarily large size while still providing the same insert/lookup interface. - it suits us: unless the subquery is resolved with one index lookup, hitting the cache would be many times cheaper than re-running the subquery, so cache is worth having. 4. Interplay with other subquery optimizations ---------------------------------------------- * This WL entry should not care about IN->EXISTS transformation: caching for IN subquery and result of its conversion to EXISTS would work in the same way. * This optimization is orthogonal to <=>ANY -> MIN/MAX rewrite (it will work/be useful irrespectively of whether the rewrite has been performed or not) * TODO: compare this with materialization for uncorrelated IN-subqueries. Is this basically the same? A: no, it is not: - IN-Materialization has to perform full materialization before it can do the first subquery evaluation. This WL's code has almost no startup costs. - This optimization has temp.table of (corr_reference, predicate_value), while IN-materialization will have (corr_reference) only. 5. User interface ----------------- * There will be an @@optimizer_switch flag to turn this optimization on and off (TODO: name of the flag?) * TODO: how do we show this in EXPLAIN [EXTENDED]? The most easiest is to print something in the warning text of EXPLAIN EXTEDED that would indicate use of cache. * temporary table sizing (max size for heap table, whether to use MyISAM or Maria) will be controlled with common temp.table control variables. LOW-LEVEL DESIGN: All items on which subquery depend could be collected in st_select_lex::mark_as_dependent (direct of indirect reference?) Temporary table index should be created by all fields except result field (TMP_TABLE_PARAM::keyinfo). How to fill the temptable ------------------------- Can reuse approach from SJ-Materialization. Its code is in end_sj_materialize() and is supposed to be quite trivial. How to make lookups into temptable ---------------------------------- We'll reuse approach used by SJ-Materialization in 6.0. Setup process ~~~~~~~~~~~~~ Setup is performed in the same way as in setup_sj_materialization(), see the code that starts these lines: /* Create/initialize everything we will need to index lookups into the temptable. */ and ends at this line: Remove the injected semi-join IN-equalities from join_tab conds. This <questionable> We'll also need to check equalities, i.e. do an equivalent of this: if (!(sjm->in_equality= create_subq_in_equalities(thd, sjm, emb_sj_nest->sj_subq_pred))) DBUG_RETURN(TRUE); /* purecov: inspected */ Question: or perhaps that is not necessarry? </questionable> Doing the lookup ~~~~~~~~~~~~~~~~ SJ-Materialization does lookup in sub_select_sjm(), with this code: /* Do index lookup in the materialized table */ if ((res= join_read_key2(join_tab, sjm->table, sjm->tab_ref)) == 1) DBUG_RETURN(NESTED_LOOP_ERROR); /* purecov: inspected */ if (res || !sjm->in_equality->val_int()) DBUG_RETURN(NESTED_LOOP_NO_MORE_ROWS); The code in this WL will use the same approach Extracting the value of the subquery predicate ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The goal of making the lookup is to get the value of subquery predicate. This is done by creating an Item_field $I which refers to appropriate temporary table's field and then subquery_predicate->val_int() will invoke $I->val_int(), subquery_predicate->val_str() will invoke $I->val_str() and so forth. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Psergey): Subquery optimization: Avoid recalculating subquery if external fields values found in subquery cache (66)
by worklog-noreply＠askmonty.org 18 Jan '10

18 Jan '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Subquery optimization: Avoid recalculating subquery if external fields values found in subquery cache CREATION DATE..: Wed, 25 Nov 2009, 22:25 SUPERVISOR.....: Monty IMPLEMENTOR....: Sanja COPIES TO......: CATEGORY.......: Server-BackLog TASK ID........: 66 (http://askmonty.org/worklog/?tid=66) VERSION........: Server-5.2 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Psergey - Mon, 18 Jan 2010, 16:34)=-=- Low Level Design modified. --- /tmp/wklog.66.old.24328 2010-01-18 16:34:19.000000000 +0200 +++ /tmp/wklog.66.new.24328 2010-01-18 16:34:19.000000000 +0200 @@ -32,8 +32,8 @@ Question: or perhaps that is not necessarry? </questionable> -Execution process -~~~~~~~~~~~~~~~~~ +Doing the lookup +~~~~~~~~~~~~~~~~ SJ-Materialization does lookup in sub_select_sjm(), with this code: /* Do index lookup in the materialized table */ @@ -42,4 +42,12 @@ if (res || !sjm->in_equality->val_int()) DBUG_RETURN(NESTED_LOOP_NO_MORE_ROWS); +The code in this WL will use the same approach +Extracting the value of the subquery predicate +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The goal of making the lookup is to get the value of subquery predicate. +This is done by creating an Item_field $I which refers to appropriate +temporary table's field and then subquery_predicate->val_int() will invoke +$I->val_int(), subquery_predicate->val_str() will invoke $I->val_str() and so +forth. -=-=(Psergey - Mon, 18 Jan 2010, 16:23)=-=- Low Level Design modified. --- /tmp/wklog.66.old.23203 2010-01-18 16:23:18.000000000 +0200 +++ /tmp/wklog.66.new.23203 2010-01-18 16:23:18.000000000 +0200 @@ -31,3 +31,15 @@ Question: or perhaps that is not necessarry? </questionable> + +Execution process +~~~~~~~~~~~~~~~~~ +SJ-Materialization does lookup in sub_select_sjm(), with this code: + + /* Do index lookup in the materialized table */ + if ((res= join_read_key2(join_tab, sjm->table, sjm->tab_ref)) == 1) + DBUG_RETURN(NESTED_LOOP_ERROR); /* purecov: inspected */ + if (res || !sjm->in_equality->val_int()) + DBUG_RETURN(NESTED_LOOP_NO_MORE_ROWS); + + -=-=(Psergey - Mon, 18 Jan 2010, 16:22)=-=- Low Level Design modified. --- /tmp/wklog.66.old.23076 2010-01-18 16:22:07.000000000 +0200 +++ /tmp/wklog.66.new.23076 2010-01-18 16:22:07.000000000 +0200 @@ -4,3 +4,30 @@ Temporary table index should be created by all fields except result field (TMP_TABLE_PARAM::keyinfo). +How to make lookups into temptable +---------------------------------- +We'll reuse approach used by SJ-Materialization in 6.0. + +Setup process +~~~~~~~~~~~~~ +Setup is performed in the same way as in setup_sj_materialization(), +see the code that starts these lines: + + /* + Create/initialize everything we will need to index lookups into the + temptable. + */ + +and ends at this line: + + Remove the injected semi-join IN-equalities from join_tab conds. This + +<questionable> +We'll also need to check equalities, i.e. do an equivalent of this: + + if (!(sjm->in_equality= create_subq_in_equalities(thd, sjm, + emb_sj_nest->sj_subq_pred))) + DBUG_RETURN(TRUE); /* purecov: inspected */ + +Question: or perhaps that is not necessarry? +</questionable> -=-=(Psergey - Tue, 12 Jan 2010, 18:39)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.31666 2010-01-12 18:39:43.000000000 +0200 +++ /tmp/wklog.66.new.31666 2010-01-12 18:39:43.000000000 +0200 @@ -4,3 +4,99 @@ To check/discuss: To put subquery cache on all levels of subqueries or on highest level only. + + +<contents> +1. Scope of the task +2. Data structure used for the cache +3. Cache size +4. Interplay with other subquery optimizations +5. User interface +</contents> + +1. Scope of the task +-------------------- +This WL should handle all subquery predicates, i.e. it should handle these +cases: + + outer_expr IN (SELECT correlated_select) + outer_expr $CMP$ ALL/ANY (SELECT correlated_select) + EXISTS (SELECT correlated_select) + scalar-context subquery: (SELECT correlated_select) + +The cache will maintain + + (outer_expr, correlation_references)-> subquery_item_result + +mapping, where +- correlation_references is a list of tablename.column_name that are referred + from the correlated_select but tablename is a table that is ouside the + subquery. +- subquery_item_result is 'bool' for subquery predicates, and is of +some scalar or ROW(scalar1,...scalarN) type for scalar-context subquery. + +We dont support cases when outer_expr or correlation_references are blobs. + +2. Data structure used for the cache +------------------------------------ +There are two data structures available in the codebase that will allow fast +equality lookups: + +1. HASH (mysys/hash.c) tables +2. Temporary tables (the ones that are used for e.g. GROUP BY) + +None of them has any support for element eviction on overflow (using LRU or +some other policy). + +Query cache and MyISAM/Maria's key/page cache ought to support some eviction +mechanism, but code-wise it is not readily reusable, one will need to factor +it out (or copy it). + +We choose to use #2, and not to have any eviction policy. See subsequent +sections for details and reasoning behind the decision. + +3. Cache size +------------- +Typically, a cache has some maximum size and a policy which is used to +select a cache entry for removal when the cache becomes full (e.g. find +and remove the least [recently] used entry) + +For this WL entry we will use a cache of infinite size. The reasoning behind +this is that: +- is is easy to do: we have temporary tables that can grow to arbitrarily + large size while still providing the same insert/lookup interface. +- it suits us: unless the subquery is resolved with one index lookup, + hitting the cache would be many times cheaper than re-running the + subquery, so cache is worth having. + +4. Interplay with other subquery optimizations +---------------------------------------------- +* This WL entry should not care about IN->EXISTS transformation: caching for + IN subquery and result of its conversion to EXISTS would work in the same + way. + +* This optimization is orthogonal to <=>ANY -> MIN/MAX rewrite (it will + work/be useful irrespectively of whether the rewrite has been performed or + not) + +* TODO: compare this with materialization for uncorrelated IN-subqueries. Is + this basically the same? + A: no, it is not: + - IN-Materialization has to perform full materialization before it can + do the first subquery evaluation. This WL's code has almost no startup + costs. + - This optimization has temp.table of (corr_reference, predicate_value), + while IN-materialization will have (corr_reference) only. + +5. User interface +----------------- +* There will be an @@optimizer_switch flag to turn this optimization on and + off (TODO: name of the flag?) + +* TODO: how do we show this in EXPLAIN [EXTENDED]? The most easiest is to + print something in the warning text of EXPLAIN EXTEDED that would indicate + use of cache. + +* temporary table sizing (max size for heap table, whether to use MyISAM or + Maria) will be controlled with common temp.table control variables. + -=-=(Psergey - Mon, 11 Jan 2010, 13:25)=-=- As of today, there is code that - collects outside references - creates a temporary table with index that would allow for fast lookups. there is no code to - fill the temporary table - make lookups into it Reported zero hours worked. Estimate unchanged. -=-=(Sanja - Fri, 11 Dec 2009, 15:09)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.28164 2009-12-11 15:09:15.000000000 +0200 +++ /tmp/wklog.66.new.28164 2009-12-11 15:09:15.000000000 +0200 @@ -3,4 +3,4 @@ To check/discuss: -Are there sens to put subquery cache on all levels of subqueries of on highest. + To put subquery cache on all levels of subqueries or on highest level only. -=-=(Sanja - Fri, 11 Dec 2009, 15:08)=-=- Low Level Design modified. --- /tmp/wklog.66.old.28072 2009-12-11 15:08:16.000000000 +0200 +++ /tmp/wklog.66.new.28072 2009-12-11 15:08:16.000000000 +0200 @@ -1 +1,6 @@ +All items on which subquery depend could be collected in +st_select_lex::mark_as_dependent (direct of indirect reference?) + +Temporary table index should be created by all fields except result field +(TMP_TABLE_PARAM::keyinfo). -=-=(Sanja - Fri, 11 Dec 2009, 15:05)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.27795 2009-12-11 15:05:04.000000000 +0200 +++ /tmp/wklog.66.new.27795 2009-12-11 15:05:04.000000000 +0200 @@ -1 +1,6 @@ +Attach subquery cache to each Item_subquery. Interface should allow to use hash +or temporary table inside. + +To check/discuss: +Are there sens to put subquery cache on all levels of subqueries of on highest. DESCRIPTION: Collect all outer items/references (left part of the subquiery and outer references inside the subquery) in key string. Compare the string (which represents certain value set of the references) against values in hash table and return cached result of subquery if the reference values combination has already been used. For example in the following subquery: (L1, L2) IN (SELECT A, B FROM T WHERE T.F1>OTER_FIELD) set of references to look into the subquery cache is (L1, L2, OTER_FIELD). The subquery cache should be implemented as simple LRU connected to the subquery. Size of the subquery cache (in number of results (but maybe in used memory amount)) is limited by session variable (query parameter?). HIGH-LEVEL SPECIFICATION: Attach subquery cache to each Item_subquery. Interface should allow to use hash or temporary table inside. To check/discuss: To put subquery cache on all levels of subqueries or on highest level only. <contents> 1. Scope of the task 2. Data structure used for the cache 3. Cache size 4. Interplay with other subquery optimizations 5. User interface </contents> 1. Scope of the task -------------------- This WL should handle all subquery predicates, i.e. it should handle these cases: outer_expr IN (SELECT correlated_select) outer_expr $CMP$ ALL/ANY (SELECT correlated_select) EXISTS (SELECT correlated_select) scalar-context subquery: (SELECT correlated_select) The cache will maintain (outer_expr, correlation_references)-> subquery_item_result mapping, where - correlation_references is a list of tablename.column_name that are referred from the correlated_select but tablename is a table that is ouside the subquery. - subquery_item_result is 'bool' for subquery predicates, and is of some scalar or ROW(scalar1,...scalarN) type for scalar-context subquery. We dont support cases when outer_expr or correlation_references are blobs. 2. Data structure used for the cache ------------------------------------ There are two data structures available in the codebase that will allow fast equality lookups: 1. HASH (mysys/hash.c) tables 2. Temporary tables (the ones that are used for e.g. GROUP BY) None of them has any support for element eviction on overflow (using LRU or some other policy). Query cache and MyISAM/Maria's key/page cache ought to support some eviction mechanism, but code-wise it is not readily reusable, one will need to factor it out (or copy it). We choose to use #2, and not to have any eviction policy. See subsequent sections for details and reasoning behind the decision. 3. Cache size ------------- Typically, a cache has some maximum size and a policy which is used to select a cache entry for removal when the cache becomes full (e.g. find and remove the least [recently] used entry) For this WL entry we will use a cache of infinite size. The reasoning behind this is that: - is is easy to do: we have temporary tables that can grow to arbitrarily large size while still providing the same insert/lookup interface. - it suits us: unless the subquery is resolved with one index lookup, hitting the cache would be many times cheaper than re-running the subquery, so cache is worth having. 4. Interplay with other subquery optimizations ---------------------------------------------- * This WL entry should not care about IN->EXISTS transformation: caching for IN subquery and result of its conversion to EXISTS would work in the same way. * This optimization is orthogonal to <=>ANY -> MIN/MAX rewrite (it will work/be useful irrespectively of whether the rewrite has been performed or not) * TODO: compare this with materialization for uncorrelated IN-subqueries. Is this basically the same? A: no, it is not: - IN-Materialization has to perform full materialization before it can do the first subquery evaluation. This WL's code has almost no startup costs. - This optimization has temp.table of (corr_reference, predicate_value), while IN-materialization will have (corr_reference) only. 5. User interface ----------------- * There will be an @@optimizer_switch flag to turn this optimization on and off (TODO: name of the flag?) * TODO: how do we show this in EXPLAIN [EXTENDED]? The most easiest is to print something in the warning text of EXPLAIN EXTEDED that would indicate use of cache. * temporary table sizing (max size for heap table, whether to use MyISAM or Maria) will be controlled with common temp.table control variables. LOW-LEVEL DESIGN: All items on which subquery depend could be collected in st_select_lex::mark_as_dependent (direct of indirect reference?) Temporary table index should be created by all fields except result field (TMP_TABLE_PARAM::keyinfo). How to make lookups into temptable ---------------------------------- We'll reuse approach used by SJ-Materialization in 6.0. Setup process ~~~~~~~~~~~~~ Setup is performed in the same way as in setup_sj_materialization(), see the code that starts these lines: /* Create/initialize everything we will need to index lookups into the temptable. */ and ends at this line: Remove the injected semi-join IN-equalities from join_tab conds. This <questionable> We'll also need to check equalities, i.e. do an equivalent of this: if (!(sjm->in_equality= create_subq_in_equalities(thd, sjm, emb_sj_nest->sj_subq_pred))) DBUG_RETURN(TRUE); /* purecov: inspected */ Question: or perhaps that is not necessarry? </questionable> Doing the lookup ~~~~~~~~~~~~~~~~ SJ-Materialization does lookup in sub_select_sjm(), with this code: /* Do index lookup in the materialized table */ if ((res= join_read_key2(join_tab, sjm->table, sjm->tab_ref)) == 1) DBUG_RETURN(NESTED_LOOP_ERROR); /* purecov: inspected */ if (res || !sjm->in_equality->val_int()) DBUG_RETURN(NESTED_LOOP_NO_MORE_ROWS); The code in this WL will use the same approach Extracting the value of the subquery predicate ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The goal of making the lookup is to get the value of subquery predicate. This is done by creating an Item_field $I which refers to appropriate temporary table's field and then subquery_predicate->val_int() will invoke $I->val_int(), subquery_predicate->val_str() will invoke $I->val_str() and so forth. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Psergey): Subquery optimization: Avoid recalculating subquery if external fields values found in subquery cache (66)
by worklog-noreply＠askmonty.org 18 Jan '10

18 Jan '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Subquery optimization: Avoid recalculating subquery if external fields values found in subquery cache CREATION DATE..: Wed, 25 Nov 2009, 22:25 SUPERVISOR.....: Monty IMPLEMENTOR....: Sanja COPIES TO......: CATEGORY.......: Server-BackLog TASK ID........: 66 (http://askmonty.org/worklog/?tid=66) VERSION........: Server-5.2 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Psergey - Mon, 18 Jan 2010, 16:34)=-=- Low Level Design modified. --- /tmp/wklog.66.old.24328 2010-01-18 16:34:19.000000000 +0200 +++ /tmp/wklog.66.new.24328 2010-01-18 16:34:19.000000000 +0200 @@ -32,8 +32,8 @@ Question: or perhaps that is not necessarry? </questionable> -Execution process -~~~~~~~~~~~~~~~~~ +Doing the lookup +~~~~~~~~~~~~~~~~ SJ-Materialization does lookup in sub_select_sjm(), with this code: /* Do index lookup in the materialized table */ @@ -42,4 +42,12 @@ if (res || !sjm->in_equality->val_int()) DBUG_RETURN(NESTED_LOOP_NO_MORE_ROWS); +The code in this WL will use the same approach +Extracting the value of the subquery predicate +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The goal of making the lookup is to get the value of subquery predicate. +This is done by creating an Item_field $I which refers to appropriate +temporary table's field and then subquery_predicate->val_int() will invoke +$I->val_int(), subquery_predicate->val_str() will invoke $I->val_str() and so +forth. -=-=(Psergey - Mon, 18 Jan 2010, 16:23)=-=- Low Level Design modified. --- /tmp/wklog.66.old.23203 2010-01-18 16:23:18.000000000 +0200 +++ /tmp/wklog.66.new.23203 2010-01-18 16:23:18.000000000 +0200 @@ -31,3 +31,15 @@ Question: or perhaps that is not necessarry? </questionable> + +Execution process +~~~~~~~~~~~~~~~~~ +SJ-Materialization does lookup in sub_select_sjm(), with this code: + + /* Do index lookup in the materialized table */ + if ((res= join_read_key2(join_tab, sjm->table, sjm->tab_ref)) == 1) + DBUG_RETURN(NESTED_LOOP_ERROR); /* purecov: inspected */ + if (res || !sjm->in_equality->val_int()) + DBUG_RETURN(NESTED_LOOP_NO_MORE_ROWS); + + -=-=(Psergey - Mon, 18 Jan 2010, 16:22)=-=- Low Level Design modified. --- /tmp/wklog.66.old.23076 2010-01-18 16:22:07.000000000 +0200 +++ /tmp/wklog.66.new.23076 2010-01-18 16:22:07.000000000 +0200 @@ -4,3 +4,30 @@ Temporary table index should be created by all fields except result field (TMP_TABLE_PARAM::keyinfo). +How to make lookups into temptable +---------------------------------- +We'll reuse approach used by SJ-Materialization in 6.0. + +Setup process +~~~~~~~~~~~~~ +Setup is performed in the same way as in setup_sj_materialization(), +see the code that starts these lines: + + /* + Create/initialize everything we will need to index lookups into the + temptable. + */ + +and ends at this line: + + Remove the injected semi-join IN-equalities from join_tab conds. This + +<questionable> +We'll also need to check equalities, i.e. do an equivalent of this: + + if (!(sjm->in_equality= create_subq_in_equalities(thd, sjm, + emb_sj_nest->sj_subq_pred))) + DBUG_RETURN(TRUE); /* purecov: inspected */ + +Question: or perhaps that is not necessarry? +</questionable> -=-=(Psergey - Tue, 12 Jan 2010, 18:39)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.31666 2010-01-12 18:39:43.000000000 +0200 +++ /tmp/wklog.66.new.31666 2010-01-12 18:39:43.000000000 +0200 @@ -4,3 +4,99 @@ To check/discuss: To put subquery cache on all levels of subqueries or on highest level only. + + +<contents> +1. Scope of the task +2. Data structure used for the cache +3. Cache size +4. Interplay with other subquery optimizations +5. User interface +</contents> + +1. Scope of the task +-------------------- +This WL should handle all subquery predicates, i.e. it should handle these +cases: + + outer_expr IN (SELECT correlated_select) + outer_expr $CMP$ ALL/ANY (SELECT correlated_select) + EXISTS (SELECT correlated_select) + scalar-context subquery: (SELECT correlated_select) + +The cache will maintain + + (outer_expr, correlation_references)-> subquery_item_result + +mapping, where +- correlation_references is a list of tablename.column_name that are referred + from the correlated_select but tablename is a table that is ouside the + subquery. +- subquery_item_result is 'bool' for subquery predicates, and is of +some scalar or ROW(scalar1,...scalarN) type for scalar-context subquery. + +We dont support cases when outer_expr or correlation_references are blobs. + +2. Data structure used for the cache +------------------------------------ +There are two data structures available in the codebase that will allow fast +equality lookups: + +1. HASH (mysys/hash.c) tables +2. Temporary tables (the ones that are used for e.g. GROUP BY) + +None of them has any support for element eviction on overflow (using LRU or +some other policy). + +Query cache and MyISAM/Maria's key/page cache ought to support some eviction +mechanism, but code-wise it is not readily reusable, one will need to factor +it out (or copy it). + +We choose to use #2, and not to have any eviction policy. See subsequent +sections for details and reasoning behind the decision. + +3. Cache size +------------- +Typically, a cache has some maximum size and a policy which is used to +select a cache entry for removal when the cache becomes full (e.g. find +and remove the least [recently] used entry) + +For this WL entry we will use a cache of infinite size. The reasoning behind +this is that: +- is is easy to do: we have temporary tables that can grow to arbitrarily + large size while still providing the same insert/lookup interface. +- it suits us: unless the subquery is resolved with one index lookup, + hitting the cache would be many times cheaper than re-running the + subquery, so cache is worth having. + +4. Interplay with other subquery optimizations +---------------------------------------------- +* This WL entry should not care about IN->EXISTS transformation: caching for + IN subquery and result of its conversion to EXISTS would work in the same + way. + +* This optimization is orthogonal to <=>ANY -> MIN/MAX rewrite (it will + work/be useful irrespectively of whether the rewrite has been performed or + not) + +* TODO: compare this with materialization for uncorrelated IN-subqueries. Is + this basically the same? + A: no, it is not: + - IN-Materialization has to perform full materialization before it can + do the first subquery evaluation. This WL's code has almost no startup + costs. + - This optimization has temp.table of (corr_reference, predicate_value), + while IN-materialization will have (corr_reference) only. + +5. User interface +----------------- +* There will be an @@optimizer_switch flag to turn this optimization on and + off (TODO: name of the flag?) + +* TODO: how do we show this in EXPLAIN [EXTENDED]? The most easiest is to + print something in the warning text of EXPLAIN EXTEDED that would indicate + use of cache. + +* temporary table sizing (max size for heap table, whether to use MyISAM or + Maria) will be controlled with common temp.table control variables. + -=-=(Psergey - Mon, 11 Jan 2010, 13:25)=-=- As of today, there is code that - collects outside references - creates a temporary table with index that would allow for fast lookups. there is no code to - fill the temporary table - make lookups into it Reported zero hours worked. Estimate unchanged. -=-=(Sanja - Fri, 11 Dec 2009, 15:09)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.28164 2009-12-11 15:09:15.000000000 +0200 +++ /tmp/wklog.66.new.28164 2009-12-11 15:09:15.000000000 +0200 @@ -3,4 +3,4 @@ To check/discuss: -Are there sens to put subquery cache on all levels of subqueries of on highest. + To put subquery cache on all levels of subqueries or on highest level only. -=-=(Sanja - Fri, 11 Dec 2009, 15:08)=-=- Low Level Design modified. --- /tmp/wklog.66.old.28072 2009-12-11 15:08:16.000000000 +0200 +++ /tmp/wklog.66.new.28072 2009-12-11 15:08:16.000000000 +0200 @@ -1 +1,6 @@ +All items on which subquery depend could be collected in +st_select_lex::mark_as_dependent (direct of indirect reference?) + +Temporary table index should be created by all fields except result field +(TMP_TABLE_PARAM::keyinfo). -=-=(Sanja - Fri, 11 Dec 2009, 15:05)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.27795 2009-12-11 15:05:04.000000000 +0200 +++ /tmp/wklog.66.new.27795 2009-12-11 15:05:04.000000000 +0200 @@ -1 +1,6 @@ +Attach subquery cache to each Item_subquery. Interface should allow to use hash +or temporary table inside. + +To check/discuss: +Are there sens to put subquery cache on all levels of subqueries of on highest. DESCRIPTION: Collect all outer items/references (left part of the subquiery and outer references inside the subquery) in key string. Compare the string (which represents certain value set of the references) against values in hash table and return cached result of subquery if the reference values combination has already been used. For example in the following subquery: (L1, L2) IN (SELECT A, B FROM T WHERE T.F1>OTER_FIELD) set of references to look into the subquery cache is (L1, L2, OTER_FIELD). The subquery cache should be implemented as simple LRU connected to the subquery. Size of the subquery cache (in number of results (but maybe in used memory amount)) is limited by session variable (query parameter?). HIGH-LEVEL SPECIFICATION: Attach subquery cache to each Item_subquery. Interface should allow to use hash or temporary table inside. To check/discuss: To put subquery cache on all levels of subqueries or on highest level only. <contents> 1. Scope of the task 2. Data structure used for the cache 3. Cache size 4. Interplay with other subquery optimizations 5. User interface </contents> 1. Scope of the task -------------------- This WL should handle all subquery predicates, i.e. it should handle these cases: outer_expr IN (SELECT correlated_select) outer_expr $CMP$ ALL/ANY (SELECT correlated_select) EXISTS (SELECT correlated_select) scalar-context subquery: (SELECT correlated_select) The cache will maintain (outer_expr, correlation_references)-> subquery_item_result mapping, where - correlation_references is a list of tablename.column_name that are referred from the correlated_select but tablename is a table that is ouside the subquery. - subquery_item_result is 'bool' for subquery predicates, and is of some scalar or ROW(scalar1,...scalarN) type for scalar-context subquery. We dont support cases when outer_expr or correlation_references are blobs. 2. Data structure used for the cache ------------------------------------ There are two data structures available in the codebase that will allow fast equality lookups: 1. HASH (mysys/hash.c) tables 2. Temporary tables (the ones that are used for e.g. GROUP BY) None of them has any support for element eviction on overflow (using LRU or some other policy). Query cache and MyISAM/Maria's key/page cache ought to support some eviction mechanism, but code-wise it is not readily reusable, one will need to factor it out (or copy it). We choose to use #2, and not to have any eviction policy. See subsequent sections for details and reasoning behind the decision. 3. Cache size ------------- Typically, a cache has some maximum size and a policy which is used to select a cache entry for removal when the cache becomes full (e.g. find and remove the least [recently] used entry) For this WL entry we will use a cache of infinite size. The reasoning behind this is that: - is is easy to do: we have temporary tables that can grow to arbitrarily large size while still providing the same insert/lookup interface. - it suits us: unless the subquery is resolved with one index lookup, hitting the cache would be many times cheaper than re-running the subquery, so cache is worth having. 4. Interplay with other subquery optimizations ---------------------------------------------- * This WL entry should not care about IN->EXISTS transformation: caching for IN subquery and result of its conversion to EXISTS would work in the same way. * This optimization is orthogonal to <=>ANY -> MIN/MAX rewrite (it will work/be useful irrespectively of whether the rewrite has been performed or not) * TODO: compare this with materialization for uncorrelated IN-subqueries. Is this basically the same? A: no, it is not: - IN-Materialization has to perform full materialization before it can do the first subquery evaluation. This WL's code has almost no startup costs. - This optimization has temp.table of (corr_reference, predicate_value), while IN-materialization will have (corr_reference) only. 5. User interface ----------------- * There will be an @@optimizer_switch flag to turn this optimization on and off (TODO: name of the flag?) * TODO: how do we show this in EXPLAIN [EXTENDED]? The most easiest is to print something in the warning text of EXPLAIN EXTEDED that would indicate use of cache. * temporary table sizing (max size for heap table, whether to use MyISAM or Maria) will be controlled with common temp.table control variables. LOW-LEVEL DESIGN: All items on which subquery depend could be collected in st_select_lex::mark_as_dependent (direct of indirect reference?) Temporary table index should be created by all fields except result field (TMP_TABLE_PARAM::keyinfo). How to make lookups into temptable ---------------------------------- We'll reuse approach used by SJ-Materialization in 6.0. Setup process ~~~~~~~~~~~~~ Setup is performed in the same way as in setup_sj_materialization(), see the code that starts these lines: /* Create/initialize everything we will need to index lookups into the temptable. */ and ends at this line: Remove the injected semi-join IN-equalities from join_tab conds. This <questionable> We'll also need to check equalities, i.e. do an equivalent of this: if (!(sjm->in_equality= create_subq_in_equalities(thd, sjm, emb_sj_nest->sj_subq_pred))) DBUG_RETURN(TRUE); /* purecov: inspected */ Question: or perhaps that is not necessarry? </questionable> Doing the lookup ~~~~~~~~~~~~~~~~ SJ-Materialization does lookup in sub_select_sjm(), with this code: /* Do index lookup in the materialized table */ if ((res= join_read_key2(join_tab, sjm->table, sjm->tab_ref)) == 1) DBUG_RETURN(NESTED_LOOP_ERROR); /* purecov: inspected */ if (res || !sjm->in_equality->val_int()) DBUG_RETURN(NESTED_LOOP_NO_MORE_ROWS); The code in this WL will use the same approach Extracting the value of the subquery predicate ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The goal of making the lookup is to get the value of subquery predicate. This is done by creating an Item_field $I which refers to appropriate temporary table's field and then subquery_predicate->val_int() will invoke $I->val_int(), subquery_predicate->val_str() will invoke $I->val_str() and so forth. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Psergey): Subquery optimization: Avoid recalculating subquery if external fields values found in subquery cache (66)
by worklog-noreply＠askmonty.org 18 Jan '10

18 Jan '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Subquery optimization: Avoid recalculating subquery if external fields values found in subquery cache CREATION DATE..: Wed, 25 Nov 2009, 22:25 SUPERVISOR.....: Monty IMPLEMENTOR....: Sanja COPIES TO......: CATEGORY.......: Server-BackLog TASK ID........: 66 (http://askmonty.org/worklog/?tid=66) VERSION........: Server-5.2 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Psergey - Mon, 18 Jan 2010, 16:23)=-=- Low Level Design modified. --- /tmp/wklog.66.old.23203 2010-01-18 16:23:18.000000000 +0200 +++ /tmp/wklog.66.new.23203 2010-01-18 16:23:18.000000000 +0200 @@ -31,3 +31,15 @@ Question: or perhaps that is not necessarry? </questionable> + +Execution process +~~~~~~~~~~~~~~~~~ +SJ-Materialization does lookup in sub_select_sjm(), with this code: + + /* Do index lookup in the materialized table */ + if ((res= join_read_key2(join_tab, sjm->table, sjm->tab_ref)) == 1) + DBUG_RETURN(NESTED_LOOP_ERROR); /* purecov: inspected */ + if (res || !sjm->in_equality->val_int()) + DBUG_RETURN(NESTED_LOOP_NO_MORE_ROWS); + + -=-=(Psergey - Mon, 18 Jan 2010, 16:22)=-=- Low Level Design modified. --- /tmp/wklog.66.old.23076 2010-01-18 16:22:07.000000000 +0200 +++ /tmp/wklog.66.new.23076 2010-01-18 16:22:07.000000000 +0200 @@ -4,3 +4,30 @@ Temporary table index should be created by all fields except result field (TMP_TABLE_PARAM::keyinfo). +How to make lookups into temptable +---------------------------------- +We'll reuse approach used by SJ-Materialization in 6.0. + +Setup process +~~~~~~~~~~~~~ +Setup is performed in the same way as in setup_sj_materialization(), +see the code that starts these lines: + + /* + Create/initialize everything we will need to index lookups into the + temptable. + */ + +and ends at this line: + + Remove the injected semi-join IN-equalities from join_tab conds. This + +<questionable> +We'll also need to check equalities, i.e. do an equivalent of this: + + if (!(sjm->in_equality= create_subq_in_equalities(thd, sjm, + emb_sj_nest->sj_subq_pred))) + DBUG_RETURN(TRUE); /* purecov: inspected */ + +Question: or perhaps that is not necessarry? +</questionable> -=-=(Psergey - Tue, 12 Jan 2010, 18:39)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.31666 2010-01-12 18:39:43.000000000 +0200 +++ /tmp/wklog.66.new.31666 2010-01-12 18:39:43.000000000 +0200 @@ -4,3 +4,99 @@ To check/discuss: To put subquery cache on all levels of subqueries or on highest level only. + + +<contents> +1. Scope of the task +2. Data structure used for the cache +3. Cache size +4. Interplay with other subquery optimizations +5. User interface +</contents> + +1. Scope of the task +-------------------- +This WL should handle all subquery predicates, i.e. it should handle these +cases: + + outer_expr IN (SELECT correlated_select) + outer_expr $CMP$ ALL/ANY (SELECT correlated_select) + EXISTS (SELECT correlated_select) + scalar-context subquery: (SELECT correlated_select) + +The cache will maintain + + (outer_expr, correlation_references)-> subquery_item_result + +mapping, where +- correlation_references is a list of tablename.column_name that are referred + from the correlated_select but tablename is a table that is ouside the + subquery. +- subquery_item_result is 'bool' for subquery predicates, and is of +some scalar or ROW(scalar1,...scalarN) type for scalar-context subquery. + +We dont support cases when outer_expr or correlation_references are blobs. + +2. Data structure used for the cache +------------------------------------ +There are two data structures available in the codebase that will allow fast +equality lookups: + +1. HASH (mysys/hash.c) tables +2. Temporary tables (the ones that are used for e.g. GROUP BY) + +None of them has any support for element eviction on overflow (using LRU or +some other policy). + +Query cache and MyISAM/Maria's key/page cache ought to support some eviction +mechanism, but code-wise it is not readily reusable, one will need to factor +it out (or copy it). + +We choose to use #2, and not to have any eviction policy. See subsequent +sections for details and reasoning behind the decision. + +3. Cache size +------------- +Typically, a cache has some maximum size and a policy which is used to +select a cache entry for removal when the cache becomes full (e.g. find +and remove the least [recently] used entry) + +For this WL entry we will use a cache of infinite size. The reasoning behind +this is that: +- is is easy to do: we have temporary tables that can grow to arbitrarily + large size while still providing the same insert/lookup interface. +- it suits us: unless the subquery is resolved with one index lookup, + hitting the cache would be many times cheaper than re-running the + subquery, so cache is worth having. + +4. Interplay with other subquery optimizations +---------------------------------------------- +* This WL entry should not care about IN->EXISTS transformation: caching for + IN subquery and result of its conversion to EXISTS would work in the same + way. + +* This optimization is orthogonal to <=>ANY -> MIN/MAX rewrite (it will + work/be useful irrespectively of whether the rewrite has been performed or + not) + +* TODO: compare this with materialization for uncorrelated IN-subqueries. Is + this basically the same? + A: no, it is not: + - IN-Materialization has to perform full materialization before it can + do the first subquery evaluation. This WL's code has almost no startup + costs. + - This optimization has temp.table of (corr_reference, predicate_value), + while IN-materialization will have (corr_reference) only. + +5. User interface +----------------- +* There will be an @@optimizer_switch flag to turn this optimization on and + off (TODO: name of the flag?) + +* TODO: how do we show this in EXPLAIN [EXTENDED]? The most easiest is to + print something in the warning text of EXPLAIN EXTEDED that would indicate + use of cache. + +* temporary table sizing (max size for heap table, whether to use MyISAM or + Maria) will be controlled with common temp.table control variables. + -=-=(Psergey - Mon, 11 Jan 2010, 13:25)=-=- As of today, there is code that - collects outside references - creates a temporary table with index that would allow for fast lookups. there is no code to - fill the temporary table - make lookups into it Reported zero hours worked. Estimate unchanged. -=-=(Sanja - Fri, 11 Dec 2009, 15:09)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.28164 2009-12-11 15:09:15.000000000 +0200 +++ /tmp/wklog.66.new.28164 2009-12-11 15:09:15.000000000 +0200 @@ -3,4 +3,4 @@ To check/discuss: -Are there sens to put subquery cache on all levels of subqueries of on highest. + To put subquery cache on all levels of subqueries or on highest level only. -=-=(Sanja - Fri, 11 Dec 2009, 15:08)=-=- Low Level Design modified. --- /tmp/wklog.66.old.28072 2009-12-11 15:08:16.000000000 +0200 +++ /tmp/wklog.66.new.28072 2009-12-11 15:08:16.000000000 +0200 @@ -1 +1,6 @@ +All items on which subquery depend could be collected in +st_select_lex::mark_as_dependent (direct of indirect reference?) + +Temporary table index should be created by all fields except result field +(TMP_TABLE_PARAM::keyinfo). -=-=(Sanja - Fri, 11 Dec 2009, 15:05)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.27795 2009-12-11 15:05:04.000000000 +0200 +++ /tmp/wklog.66.new.27795 2009-12-11 15:05:04.000000000 +0200 @@ -1 +1,6 @@ +Attach subquery cache to each Item_subquery. Interface should allow to use hash +or temporary table inside. + +To check/discuss: +Are there sens to put subquery cache on all levels of subqueries of on highest. DESCRIPTION: Collect all outer items/references (left part of the subquiery and outer references inside the subquery) in key string. Compare the string (which represents certain value set of the references) against values in hash table and return cached result of subquery if the reference values combination has already been used. For example in the following subquery: (L1, L2) IN (SELECT A, B FROM T WHERE T.F1>OTER_FIELD) set of references to look into the subquery cache is (L1, L2, OTER_FIELD). The subquery cache should be implemented as simple LRU connected to the subquery. Size of the subquery cache (in number of results (but maybe in used memory amount)) is limited by session variable (query parameter?). HIGH-LEVEL SPECIFICATION: Attach subquery cache to each Item_subquery. Interface should allow to use hash or temporary table inside. To check/discuss: To put subquery cache on all levels of subqueries or on highest level only. <contents> 1. Scope of the task 2. Data structure used for the cache 3. Cache size 4. Interplay with other subquery optimizations 5. User interface </contents> 1. Scope of the task -------------------- This WL should handle all subquery predicates, i.e. it should handle these cases: outer_expr IN (SELECT correlated_select) outer_expr $CMP$ ALL/ANY (SELECT correlated_select) EXISTS (SELECT correlated_select) scalar-context subquery: (SELECT correlated_select) The cache will maintain (outer_expr, correlation_references)-> subquery_item_result mapping, where - correlation_references is a list of tablename.column_name that are referred from the correlated_select but tablename is a table that is ouside the subquery. - subquery_item_result is 'bool' for subquery predicates, and is of some scalar or ROW(scalar1,...scalarN) type for scalar-context subquery. We dont support cases when outer_expr or correlation_references are blobs. 2. Data structure used for the cache ------------------------------------ There are two data structures available in the codebase that will allow fast equality lookups: 1. HASH (mysys/hash.c) tables 2. Temporary tables (the ones that are used for e.g. GROUP BY) None of them has any support for element eviction on overflow (using LRU or some other policy). Query cache and MyISAM/Maria's key/page cache ought to support some eviction mechanism, but code-wise it is not readily reusable, one will need to factor it out (or copy it). We choose to use #2, and not to have any eviction policy. See subsequent sections for details and reasoning behind the decision. 3. Cache size ------------- Typically, a cache has some maximum size and a policy which is used to select a cache entry for removal when the cache becomes full (e.g. find and remove the least [recently] used entry) For this WL entry we will use a cache of infinite size. The reasoning behind this is that: - is is easy to do: we have temporary tables that can grow to arbitrarily large size while still providing the same insert/lookup interface. - it suits us: unless the subquery is resolved with one index lookup, hitting the cache would be many times cheaper than re-running the subquery, so cache is worth having. 4. Interplay with other subquery optimizations ---------------------------------------------- * This WL entry should not care about IN->EXISTS transformation: caching for IN subquery and result of its conversion to EXISTS would work in the same way. * This optimization is orthogonal to <=>ANY -> MIN/MAX rewrite (it will work/be useful irrespectively of whether the rewrite has been performed or not) * TODO: compare this with materialization for uncorrelated IN-subqueries. Is this basically the same? A: no, it is not: - IN-Materialization has to perform full materialization before it can do the first subquery evaluation. This WL's code has almost no startup costs. - This optimization has temp.table of (corr_reference, predicate_value), while IN-materialization will have (corr_reference) only. 5. User interface ----------------- * There will be an @@optimizer_switch flag to turn this optimization on and off (TODO: name of the flag?) * TODO: how do we show this in EXPLAIN [EXTENDED]? The most easiest is to print something in the warning text of EXPLAIN EXTEDED that would indicate use of cache. * temporary table sizing (max size for heap table, whether to use MyISAM or Maria) will be controlled with common temp.table control variables. LOW-LEVEL DESIGN: All items on which subquery depend could be collected in st_select_lex::mark_as_dependent (direct of indirect reference?) Temporary table index should be created by all fields except result field (TMP_TABLE_PARAM::keyinfo). How to make lookups into temptable ---------------------------------- We'll reuse approach used by SJ-Materialization in 6.0. Setup process ~~~~~~~~~~~~~ Setup is performed in the same way as in setup_sj_materialization(), see the code that starts these lines: /* Create/initialize everything we will need to index lookups into the temptable. */ and ends at this line: Remove the injected semi-join IN-equalities from join_tab conds. This <questionable> We'll also need to check equalities, i.e. do an equivalent of this: if (!(sjm->in_equality= create_subq_in_equalities(thd, sjm, emb_sj_nest->sj_subq_pred))) DBUG_RETURN(TRUE); /* purecov: inspected */ Question: or perhaps that is not necessarry? </questionable> Execution process ~~~~~~~~~~~~~~~~~ SJ-Materialization does lookup in sub_select_sjm(), with this code: /* Do index lookup in the materialized table */ if ((res= join_read_key2(join_tab, sjm->table, sjm->tab_ref)) == 1) DBUG_RETURN(NESTED_LOOP_ERROR); /* purecov: inspected */ if (res || !sjm->in_equality->val_int()) DBUG_RETURN(NESTED_LOOP_NO_MORE_ROWS); ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Psergey): Subquery optimization: Avoid recalculating subquery if external fields values found in subquery cache (66)
by worklog-noreply＠askmonty.org 18 Jan '10

18 Jan '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Subquery optimization: Avoid recalculating subquery if external fields values found in subquery cache CREATION DATE..: Wed, 25 Nov 2009, 22:25 SUPERVISOR.....: Monty IMPLEMENTOR....: Sanja COPIES TO......: CATEGORY.......: Server-BackLog TASK ID........: 66 (http://askmonty.org/worklog/?tid=66) VERSION........: Server-5.2 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Psergey - Mon, 18 Jan 2010, 16:23)=-=- Low Level Design modified. --- /tmp/wklog.66.old.23203 2010-01-18 16:23:18.000000000 +0200 +++ /tmp/wklog.66.new.23203 2010-01-18 16:23:18.000000000 +0200 @@ -31,3 +31,15 @@ Question: or perhaps that is not necessarry? </questionable> + +Execution process +~~~~~~~~~~~~~~~~~ +SJ-Materialization does lookup in sub_select_sjm(), with this code: + + /* Do index lookup in the materialized table */ + if ((res= join_read_key2(join_tab, sjm->table, sjm->tab_ref)) == 1) + DBUG_RETURN(NESTED_LOOP_ERROR); /* purecov: inspected */ + if (res || !sjm->in_equality->val_int()) + DBUG_RETURN(NESTED_LOOP_NO_MORE_ROWS); + + -=-=(Psergey - Mon, 18 Jan 2010, 16:22)=-=- Low Level Design modified. --- /tmp/wklog.66.old.23076 2010-01-18 16:22:07.000000000 +0200 +++ /tmp/wklog.66.new.23076 2010-01-18 16:22:07.000000000 +0200 @@ -4,3 +4,30 @@ Temporary table index should be created by all fields except result field (TMP_TABLE_PARAM::keyinfo). +How to make lookups into temptable +---------------------------------- +We'll reuse approach used by SJ-Materialization in 6.0. + +Setup process +~~~~~~~~~~~~~ +Setup is performed in the same way as in setup_sj_materialization(), +see the code that starts these lines: + + /* + Create/initialize everything we will need to index lookups into the + temptable. + */ + +and ends at this line: + + Remove the injected semi-join IN-equalities from join_tab conds. This + +<questionable> +We'll also need to check equalities, i.e. do an equivalent of this: + + if (!(sjm->in_equality= create_subq_in_equalities(thd, sjm, + emb_sj_nest->sj_subq_pred))) + DBUG_RETURN(TRUE); /* purecov: inspected */ + +Question: or perhaps that is not necessarry? +</questionable> -=-=(Psergey - Tue, 12 Jan 2010, 18:39)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.31666 2010-01-12 18:39:43.000000000 +0200 +++ /tmp/wklog.66.new.31666 2010-01-12 18:39:43.000000000 +0200 @@ -4,3 +4,99 @@ To check/discuss: To put subquery cache on all levels of subqueries or on highest level only. + + +<contents> +1. Scope of the task +2. Data structure used for the cache +3. Cache size +4. Interplay with other subquery optimizations +5. User interface +</contents> + +1. Scope of the task +-------------------- +This WL should handle all subquery predicates, i.e. it should handle these +cases: + + outer_expr IN (SELECT correlated_select) + outer_expr $CMP$ ALL/ANY (SELECT correlated_select) + EXISTS (SELECT correlated_select) + scalar-context subquery: (SELECT correlated_select) + +The cache will maintain + + (outer_expr, correlation_references)-> subquery_item_result + +mapping, where +- correlation_references is a list of tablename.column_name that are referred + from the correlated_select but tablename is a table that is ouside the + subquery. +- subquery_item_result is 'bool' for subquery predicates, and is of +some scalar or ROW(scalar1,...scalarN) type for scalar-context subquery. + +We dont support cases when outer_expr or correlation_references are blobs. + +2. Data structure used for the cache +------------------------------------ +There are two data structures available in the codebase that will allow fast +equality lookups: + +1. HASH (mysys/hash.c) tables +2. Temporary tables (the ones that are used for e.g. GROUP BY) + +None of them has any support for element eviction on overflow (using LRU or +some other policy). + +Query cache and MyISAM/Maria's key/page cache ought to support some eviction +mechanism, but code-wise it is not readily reusable, one will need to factor +it out (or copy it). + +We choose to use #2, and not to have any eviction policy. See subsequent +sections for details and reasoning behind the decision. + +3. Cache size +------------- +Typically, a cache has some maximum size and a policy which is used to +select a cache entry for removal when the cache becomes full (e.g. find +and remove the least [recently] used entry) + +For this WL entry we will use a cache of infinite size. The reasoning behind +this is that: +- is is easy to do: we have temporary tables that can grow to arbitrarily + large size while still providing the same insert/lookup interface. +- it suits us: unless the subquery is resolved with one index lookup, + hitting the cache would be many times cheaper than re-running the + subquery, so cache is worth having. + +4. Interplay with other subquery optimizations +---------------------------------------------- +* This WL entry should not care about IN->EXISTS transformation: caching for + IN subquery and result of its conversion to EXISTS would work in the same + way. + +* This optimization is orthogonal to <=>ANY -> MIN/MAX rewrite (it will + work/be useful irrespectively of whether the rewrite has been performed or + not) + +* TODO: compare this with materialization for uncorrelated IN-subqueries. Is + this basically the same? + A: no, it is not: + - IN-Materialization has to perform full materialization before it can + do the first subquery evaluation. This WL's code has almost no startup + costs. + - This optimization has temp.table of (corr_reference, predicate_value), + while IN-materialization will have (corr_reference) only. + +5. User interface +----------------- +* There will be an @@optimizer_switch flag to turn this optimization on and + off (TODO: name of the flag?) + +* TODO: how do we show this in EXPLAIN [EXTENDED]? The most easiest is to + print something in the warning text of EXPLAIN EXTEDED that would indicate + use of cache. + +* temporary table sizing (max size for heap table, whether to use MyISAM or + Maria) will be controlled with common temp.table control variables. + -=-=(Psergey - Mon, 11 Jan 2010, 13:25)=-=- As of today, there is code that - collects outside references - creates a temporary table with index that would allow for fast lookups. there is no code to - fill the temporary table - make lookups into it Reported zero hours worked. Estimate unchanged. -=-=(Sanja - Fri, 11 Dec 2009, 15:09)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.28164 2009-12-11 15:09:15.000000000 +0200 +++ /tmp/wklog.66.new.28164 2009-12-11 15:09:15.000000000 +0200 @@ -3,4 +3,4 @@ To check/discuss: -Are there sens to put subquery cache on all levels of subqueries of on highest. + To put subquery cache on all levels of subqueries or on highest level only. -=-=(Sanja - Fri, 11 Dec 2009, 15:08)=-=- Low Level Design modified. --- /tmp/wklog.66.old.28072 2009-12-11 15:08:16.000000000 +0200 +++ /tmp/wklog.66.new.28072 2009-12-11 15:08:16.000000000 +0200 @@ -1 +1,6 @@ +All items on which subquery depend could be collected in +st_select_lex::mark_as_dependent (direct of indirect reference?) + +Temporary table index should be created by all fields except result field +(TMP_TABLE_PARAM::keyinfo). -=-=(Sanja - Fri, 11 Dec 2009, 15:05)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.27795 2009-12-11 15:05:04.000000000 +0200 +++ /tmp/wklog.66.new.27795 2009-12-11 15:05:04.000000000 +0200 @@ -1 +1,6 @@ +Attach subquery cache to each Item_subquery. Interface should allow to use hash +or temporary table inside. + +To check/discuss: +Are there sens to put subquery cache on all levels of subqueries of on highest. DESCRIPTION: Collect all outer items/references (left part of the subquiery and outer references inside the subquery) in key string. Compare the string (which represents certain value set of the references) against values in hash table and return cached result of subquery if the reference values combination has already been used. For example in the following subquery: (L1, L2) IN (SELECT A, B FROM T WHERE T.F1>OTER_FIELD) set of references to look into the subquery cache is (L1, L2, OTER_FIELD). The subquery cache should be implemented as simple LRU connected to the subquery. Size of the subquery cache (in number of results (but maybe in used memory amount)) is limited by session variable (query parameter?). HIGH-LEVEL SPECIFICATION: Attach subquery cache to each Item_subquery. Interface should allow to use hash or temporary table inside. To check/discuss: To put subquery cache on all levels of subqueries or on highest level only. <contents> 1. Scope of the task 2. Data structure used for the cache 3. Cache size 4. Interplay with other subquery optimizations 5. User interface </contents> 1. Scope of the task -------------------- This WL should handle all subquery predicates, i.e. it should handle these cases: outer_expr IN (SELECT correlated_select) outer_expr $CMP$ ALL/ANY (SELECT correlated_select) EXISTS (SELECT correlated_select) scalar-context subquery: (SELECT correlated_select) The cache will maintain (outer_expr, correlation_references)-> subquery_item_result mapping, where - correlation_references is a list of tablename.column_name that are referred from the correlated_select but tablename is a table that is ouside the subquery. - subquery_item_result is 'bool' for subquery predicates, and is of some scalar or ROW(scalar1,...scalarN) type for scalar-context subquery. We dont support cases when outer_expr or correlation_references are blobs. 2. Data structure used for the cache ------------------------------------ There are two data structures available in the codebase that will allow fast equality lookups: 1. HASH (mysys/hash.c) tables 2. Temporary tables (the ones that are used for e.g. GROUP BY) None of them has any support for element eviction on overflow (using LRU or some other policy). Query cache and MyISAM/Maria's key/page cache ought to support some eviction mechanism, but code-wise it is not readily reusable, one will need to factor it out (or copy it). We choose to use #2, and not to have any eviction policy. See subsequent sections for details and reasoning behind the decision. 3. Cache size ------------- Typically, a cache has some maximum size and a policy which is used to select a cache entry for removal when the cache becomes full (e.g. find and remove the least [recently] used entry) For this WL entry we will use a cache of infinite size. The reasoning behind this is that: - is is easy to do: we have temporary tables that can grow to arbitrarily large size while still providing the same insert/lookup interface. - it suits us: unless the subquery is resolved with one index lookup, hitting the cache would be many times cheaper than re-running the subquery, so cache is worth having. 4. Interplay with other subquery optimizations ---------------------------------------------- * This WL entry should not care about IN->EXISTS transformation: caching for IN subquery and result of its conversion to EXISTS would work in the same way. * This optimization is orthogonal to <=>ANY -> MIN/MAX rewrite (it will work/be useful irrespectively of whether the rewrite has been performed or not) * TODO: compare this with materialization for uncorrelated IN-subqueries. Is this basically the same? A: no, it is not: - IN-Materialization has to perform full materialization before it can do the first subquery evaluation. This WL's code has almost no startup costs. - This optimization has temp.table of (corr_reference, predicate_value), while IN-materialization will have (corr_reference) only. 5. User interface ----------------- * There will be an @@optimizer_switch flag to turn this optimization on and off (TODO: name of the flag?) * TODO: how do we show this in EXPLAIN [EXTENDED]? The most easiest is to print something in the warning text of EXPLAIN EXTEDED that would indicate use of cache. * temporary table sizing (max size for heap table, whether to use MyISAM or Maria) will be controlled with common temp.table control variables. LOW-LEVEL DESIGN: All items on which subquery depend could be collected in st_select_lex::mark_as_dependent (direct of indirect reference?) Temporary table index should be created by all fields except result field (TMP_TABLE_PARAM::keyinfo). How to make lookups into temptable ---------------------------------- We'll reuse approach used by SJ-Materialization in 6.0. Setup process ~~~~~~~~~~~~~ Setup is performed in the same way as in setup_sj_materialization(), see the code that starts these lines: /* Create/initialize everything we will need to index lookups into the temptable. */ and ends at this line: Remove the injected semi-join IN-equalities from join_tab conds. This <questionable> We'll also need to check equalities, i.e. do an equivalent of this: if (!(sjm->in_equality= create_subq_in_equalities(thd, sjm, emb_sj_nest->sj_subq_pred))) DBUG_RETURN(TRUE); /* purecov: inspected */ Question: or perhaps that is not necessarry? </questionable> Execution process ~~~~~~~~~~~~~~~~~ SJ-Materialization does lookup in sub_select_sjm(), with this code: /* Do index lookup in the materialized table */ if ((res= join_read_key2(join_tab, sjm->table, sjm->tab_ref)) == 1) DBUG_RETURN(NESTED_LOOP_ERROR); /* purecov: inspected */ if (res || !sjm->in_equality->val_int()) DBUG_RETURN(NESTED_LOOP_NO_MORE_ROWS); ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Psergey): Subquery optimization: Avoid recalculating subquery if external fields values found in subquery cache (66)
by worklog-noreply＠askmonty.org 18 Jan '10

18 Jan '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Subquery optimization: Avoid recalculating subquery if external fields values found in subquery cache CREATION DATE..: Wed, 25 Nov 2009, 22:25 SUPERVISOR.....: Monty IMPLEMENTOR....: Sanja COPIES TO......: CATEGORY.......: Server-BackLog TASK ID........: 66 (http://askmonty.org/worklog/?tid=66) VERSION........: Server-5.2 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Psergey - Mon, 18 Jan 2010, 16:22)=-=- Low Level Design modified. --- /tmp/wklog.66.old.23076 2010-01-18 16:22:07.000000000 +0200 +++ /tmp/wklog.66.new.23076 2010-01-18 16:22:07.000000000 +0200 @@ -4,3 +4,30 @@ Temporary table index should be created by all fields except result field (TMP_TABLE_PARAM::keyinfo). +How to make lookups into temptable +---------------------------------- +We'll reuse approach used by SJ-Materialization in 6.0. + +Setup process +~~~~~~~~~~~~~ +Setup is performed in the same way as in setup_sj_materialization(), +see the code that starts these lines: + + /* + Create/initialize everything we will need to index lookups into the + temptable. + */ + +and ends at this line: + + Remove the injected semi-join IN-equalities from join_tab conds. This + +<questionable> +We'll also need to check equalities, i.e. do an equivalent of this: + + if (!(sjm->in_equality= create_subq_in_equalities(thd, sjm, + emb_sj_nest->sj_subq_pred))) + DBUG_RETURN(TRUE); /* purecov: inspected */ + +Question: or perhaps that is not necessarry? +</questionable> -=-=(Psergey - Tue, 12 Jan 2010, 18:39)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.31666 2010-01-12 18:39:43.000000000 +0200 +++ /tmp/wklog.66.new.31666 2010-01-12 18:39:43.000000000 +0200 @@ -4,3 +4,99 @@ To check/discuss: To put subquery cache on all levels of subqueries or on highest level only. + + +<contents> +1. Scope of the task +2. Data structure used for the cache +3. Cache size +4. Interplay with other subquery optimizations +5. User interface +</contents> + +1. Scope of the task +-------------------- +This WL should handle all subquery predicates, i.e. it should handle these +cases: + + outer_expr IN (SELECT correlated_select) + outer_expr $CMP$ ALL/ANY (SELECT correlated_select) + EXISTS (SELECT correlated_select) + scalar-context subquery: (SELECT correlated_select) + +The cache will maintain + + (outer_expr, correlation_references)-> subquery_item_result + +mapping, where +- correlation_references is a list of tablename.column_name that are referred + from the correlated_select but tablename is a table that is ouside the + subquery. +- subquery_item_result is 'bool' for subquery predicates, and is of +some scalar or ROW(scalar1,...scalarN) type for scalar-context subquery. + +We dont support cases when outer_expr or correlation_references are blobs. + +2. Data structure used for the cache +------------------------------------ +There are two data structures available in the codebase that will allow fast +equality lookups: + +1. HASH (mysys/hash.c) tables +2. Temporary tables (the ones that are used for e.g. GROUP BY) + +None of them has any support for element eviction on overflow (using LRU or +some other policy). + +Query cache and MyISAM/Maria's key/page cache ought to support some eviction +mechanism, but code-wise it is not readily reusable, one will need to factor +it out (or copy it). + +We choose to use #2, and not to have any eviction policy. See subsequent +sections for details and reasoning behind the decision. + +3. Cache size +------------- +Typically, a cache has some maximum size and a policy which is used to +select a cache entry for removal when the cache becomes full (e.g. find +and remove the least [recently] used entry) + +For this WL entry we will use a cache of infinite size. The reasoning behind +this is that: +- is is easy to do: we have temporary tables that can grow to arbitrarily + large size while still providing the same insert/lookup interface. +- it suits us: unless the subquery is resolved with one index lookup, + hitting the cache would be many times cheaper than re-running the + subquery, so cache is worth having. + +4. Interplay with other subquery optimizations +---------------------------------------------- +* This WL entry should not care about IN->EXISTS transformation: caching for + IN subquery and result of its conversion to EXISTS would work in the same + way. + +* This optimization is orthogonal to <=>ANY -> MIN/MAX rewrite (it will + work/be useful irrespectively of whether the rewrite has been performed or + not) + +* TODO: compare this with materialization for uncorrelated IN-subqueries. Is + this basically the same? + A: no, it is not: + - IN-Materialization has to perform full materialization before it can + do the first subquery evaluation. This WL's code has almost no startup + costs. + - This optimization has temp.table of (corr_reference, predicate_value), + while IN-materialization will have (corr_reference) only. + +5. User interface +----------------- +* There will be an @@optimizer_switch flag to turn this optimization on and + off (TODO: name of the flag?) + +* TODO: how do we show this in EXPLAIN [EXTENDED]? The most easiest is to + print something in the warning text of EXPLAIN EXTEDED that would indicate + use of cache. + +* temporary table sizing (max size for heap table, whether to use MyISAM or + Maria) will be controlled with common temp.table control variables. + -=-=(Psergey - Mon, 11 Jan 2010, 13:25)=-=- As of today, there is code that - collects outside references - creates a temporary table with index that would allow for fast lookups. there is no code to - fill the temporary table - make lookups into it Reported zero hours worked. Estimate unchanged. -=-=(Sanja - Fri, 11 Dec 2009, 15:09)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.28164 2009-12-11 15:09:15.000000000 +0200 +++ /tmp/wklog.66.new.28164 2009-12-11 15:09:15.000000000 +0200 @@ -3,4 +3,4 @@ To check/discuss: -Are there sens to put subquery cache on all levels of subqueries of on highest. + To put subquery cache on all levels of subqueries or on highest level only. -=-=(Sanja - Fri, 11 Dec 2009, 15:08)=-=- Low Level Design modified. --- /tmp/wklog.66.old.28072 2009-12-11 15:08:16.000000000 +0200 +++ /tmp/wklog.66.new.28072 2009-12-11 15:08:16.000000000 +0200 @@ -1 +1,6 @@ +All items on which subquery depend could be collected in +st_select_lex::mark_as_dependent (direct of indirect reference?) + +Temporary table index should be created by all fields except result field +(TMP_TABLE_PARAM::keyinfo). -=-=(Sanja - Fri, 11 Dec 2009, 15:05)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.27795 2009-12-11 15:05:04.000000000 +0200 +++ /tmp/wklog.66.new.27795 2009-12-11 15:05:04.000000000 +0200 @@ -1 +1,6 @@ +Attach subquery cache to each Item_subquery. Interface should allow to use hash +or temporary table inside. + +To check/discuss: +Are there sens to put subquery cache on all levels of subqueries of on highest. DESCRIPTION: Collect all outer items/references (left part of the subquiery and outer references inside the subquery) in key string. Compare the string (which represents certain value set of the references) against values in hash table and return cached result of subquery if the reference values combination has already been used. For example in the following subquery: (L1, L2) IN (SELECT A, B FROM T WHERE T.F1>OTER_FIELD) set of references to look into the subquery cache is (L1, L2, OTER_FIELD). The subquery cache should be implemented as simple LRU connected to the subquery. Size of the subquery cache (in number of results (but maybe in used memory amount)) is limited by session variable (query parameter?). HIGH-LEVEL SPECIFICATION: Attach subquery cache to each Item_subquery. Interface should allow to use hash or temporary table inside. To check/discuss: To put subquery cache on all levels of subqueries or on highest level only. <contents> 1. Scope of the task 2. Data structure used for the cache 3. Cache size 4. Interplay with other subquery optimizations 5. User interface </contents> 1. Scope of the task -------------------- This WL should handle all subquery predicates, i.e. it should handle these cases: outer_expr IN (SELECT correlated_select) outer_expr $CMP$ ALL/ANY (SELECT correlated_select) EXISTS (SELECT correlated_select) scalar-context subquery: (SELECT correlated_select) The cache will maintain (outer_expr, correlation_references)-> subquery_item_result mapping, where - correlation_references is a list of tablename.column_name that are referred from the correlated_select but tablename is a table that is ouside the subquery. - subquery_item_result is 'bool' for subquery predicates, and is of some scalar or ROW(scalar1,...scalarN) type for scalar-context subquery. We dont support cases when outer_expr or correlation_references are blobs. 2. Data structure used for the cache ------------------------------------ There are two data structures available in the codebase that will allow fast equality lookups: 1. HASH (mysys/hash.c) tables 2. Temporary tables (the ones that are used for e.g. GROUP BY) None of them has any support for element eviction on overflow (using LRU or some other policy). Query cache and MyISAM/Maria's key/page cache ought to support some eviction mechanism, but code-wise it is not readily reusable, one will need to factor it out (or copy it). We choose to use #2, and not to have any eviction policy. See subsequent sections for details and reasoning behind the decision. 3. Cache size ------------- Typically, a cache has some maximum size and a policy which is used to select a cache entry for removal when the cache becomes full (e.g. find and remove the least [recently] used entry) For this WL entry we will use a cache of infinite size. The reasoning behind this is that: - is is easy to do: we have temporary tables that can grow to arbitrarily large size while still providing the same insert/lookup interface. - it suits us: unless the subquery is resolved with one index lookup, hitting the cache would be many times cheaper than re-running the subquery, so cache is worth having. 4. Interplay with other subquery optimizations ---------------------------------------------- * This WL entry should not care about IN->EXISTS transformation: caching for IN subquery and result of its conversion to EXISTS would work in the same way. * This optimization is orthogonal to <=>ANY -> MIN/MAX rewrite (it will work/be useful irrespectively of whether the rewrite has been performed or not) * TODO: compare this with materialization for uncorrelated IN-subqueries. Is this basically the same? A: no, it is not: - IN-Materialization has to perform full materialization before it can do the first subquery evaluation. This WL's code has almost no startup costs. - This optimization has temp.table of (corr_reference, predicate_value), while IN-materialization will have (corr_reference) only. 5. User interface ----------------- * There will be an @@optimizer_switch flag to turn this optimization on and off (TODO: name of the flag?) * TODO: how do we show this in EXPLAIN [EXTENDED]? The most easiest is to print something in the warning text of EXPLAIN EXTEDED that would indicate use of cache. * temporary table sizing (max size for heap table, whether to use MyISAM or Maria) will be controlled with common temp.table control variables. LOW-LEVEL DESIGN: All items on which subquery depend could be collected in st_select_lex::mark_as_dependent (direct of indirect reference?) Temporary table index should be created by all fields except result field (TMP_TABLE_PARAM::keyinfo). How to make lookups into temptable ---------------------------------- We'll reuse approach used by SJ-Materialization in 6.0. Setup process ~~~~~~~~~~~~~ Setup is performed in the same way as in setup_sj_materialization(), see the code that starts these lines: /* Create/initialize everything we will need to index lookups into the temptable. */ and ends at this line: Remove the injected semi-join IN-equalities from join_tab conds. This <questionable> We'll also need to check equalities, i.e. do an equivalent of this: if (!(sjm->in_equality= create_subq_in_equalities(thd, sjm, emb_sj_nest->sj_subq_pred))) DBUG_RETURN(TRUE); /* purecov: inspected */ Question: or perhaps that is not necessarry? </questionable> ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Psergey): Subquery optimization: Avoid recalculating subquery if external fields values found in subquery cache (66)
by worklog-noreply＠askmonty.org 18 Jan '10

18 Jan '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Subquery optimization: Avoid recalculating subquery if external fields values found in subquery cache CREATION DATE..: Wed, 25 Nov 2009, 22:25 SUPERVISOR.....: Monty IMPLEMENTOR....: Sanja COPIES TO......: CATEGORY.......: Server-BackLog TASK ID........: 66 (http://askmonty.org/worklog/?tid=66) VERSION........: Server-5.2 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Psergey - Mon, 18 Jan 2010, 16:22)=-=- Low Level Design modified. --- /tmp/wklog.66.old.23076 2010-01-18 16:22:07.000000000 +0200 +++ /tmp/wklog.66.new.23076 2010-01-18 16:22:07.000000000 +0200 @@ -4,3 +4,30 @@ Temporary table index should be created by all fields except result field (TMP_TABLE_PARAM::keyinfo). +How to make lookups into temptable +---------------------------------- +We'll reuse approach used by SJ-Materialization in 6.0. + +Setup process +~~~~~~~~~~~~~ +Setup is performed in the same way as in setup_sj_materialization(), +see the code that starts these lines: + + /* + Create/initialize everything we will need to index lookups into the + temptable. + */ + +and ends at this line: + + Remove the injected semi-join IN-equalities from join_tab conds. This + +<questionable> +We'll also need to check equalities, i.e. do an equivalent of this: + + if (!(sjm->in_equality= create_subq_in_equalities(thd, sjm, + emb_sj_nest->sj_subq_pred))) + DBUG_RETURN(TRUE); /* purecov: inspected */ + +Question: or perhaps that is not necessarry? +</questionable> -=-=(Psergey - Tue, 12 Jan 2010, 18:39)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.31666 2010-01-12 18:39:43.000000000 +0200 +++ /tmp/wklog.66.new.31666 2010-01-12 18:39:43.000000000 +0200 @@ -4,3 +4,99 @@ To check/discuss: To put subquery cache on all levels of subqueries or on highest level only. + + +<contents> +1. Scope of the task +2. Data structure used for the cache +3. Cache size +4. Interplay with other subquery optimizations +5. User interface +</contents> + +1. Scope of the task +-------------------- +This WL should handle all subquery predicates, i.e. it should handle these +cases: + + outer_expr IN (SELECT correlated_select) + outer_expr $CMP$ ALL/ANY (SELECT correlated_select) + EXISTS (SELECT correlated_select) + scalar-context subquery: (SELECT correlated_select) + +The cache will maintain + + (outer_expr, correlation_references)-> subquery_item_result + +mapping, where +- correlation_references is a list of tablename.column_name that are referred + from the correlated_select but tablename is a table that is ouside the + subquery. +- subquery_item_result is 'bool' for subquery predicates, and is of +some scalar or ROW(scalar1,...scalarN) type for scalar-context subquery. + +We dont support cases when outer_expr or correlation_references are blobs. + +2. Data structure used for the cache +------------------------------------ +There are two data structures available in the codebase that will allow fast +equality lookups: + +1. HASH (mysys/hash.c) tables +2. Temporary tables (the ones that are used for e.g. GROUP BY) + +None of them has any support for element eviction on overflow (using LRU or +some other policy). + +Query cache and MyISAM/Maria's key/page cache ought to support some eviction +mechanism, but code-wise it is not readily reusable, one will need to factor +it out (or copy it). + +We choose to use #2, and not to have any eviction policy. See subsequent +sections for details and reasoning behind the decision. + +3. Cache size +------------- +Typically, a cache has some maximum size and a policy which is used to +select a cache entry for removal when the cache becomes full (e.g. find +and remove the least [recently] used entry) + +For this WL entry we will use a cache of infinite size. The reasoning behind +this is that: +- is is easy to do: we have temporary tables that can grow to arbitrarily + large size while still providing the same insert/lookup interface. +- it suits us: unless the subquery is resolved with one index lookup, + hitting the cache would be many times cheaper than re-running the + subquery, so cache is worth having. + +4. Interplay with other subquery optimizations +---------------------------------------------- +* This WL entry should not care about IN->EXISTS transformation: caching for + IN subquery and result of its conversion to EXISTS would work in the same + way. + +* This optimization is orthogonal to <=>ANY -> MIN/MAX rewrite (it will + work/be useful irrespectively of whether the rewrite has been performed or + not) + +* TODO: compare this with materialization for uncorrelated IN-subqueries. Is + this basically the same? + A: no, it is not: + - IN-Materialization has to perform full materialization before it can + do the first subquery evaluation. This WL's code has almost no startup + costs. + - This optimization has temp.table of (corr_reference, predicate_value), + while IN-materialization will have (corr_reference) only. + +5. User interface +----------------- +* There will be an @@optimizer_switch flag to turn this optimization on and + off (TODO: name of the flag?) + +* TODO: how do we show this in EXPLAIN [EXTENDED]? The most easiest is to + print something in the warning text of EXPLAIN EXTEDED that would indicate + use of cache. + +* temporary table sizing (max size for heap table, whether to use MyISAM or + Maria) will be controlled with common temp.table control variables. + -=-=(Psergey - Mon, 11 Jan 2010, 13:25)=-=- As of today, there is code that - collects outside references - creates a temporary table with index that would allow for fast lookups. there is no code to - fill the temporary table - make lookups into it Reported zero hours worked. Estimate unchanged. -=-=(Sanja - Fri, 11 Dec 2009, 15:09)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.28164 2009-12-11 15:09:15.000000000 +0200 +++ /tmp/wklog.66.new.28164 2009-12-11 15:09:15.000000000 +0200 @@ -3,4 +3,4 @@ To check/discuss: -Are there sens to put subquery cache on all levels of subqueries of on highest. + To put subquery cache on all levels of subqueries or on highest level only. -=-=(Sanja - Fri, 11 Dec 2009, 15:08)=-=- Low Level Design modified. --- /tmp/wklog.66.old.28072 2009-12-11 15:08:16.000000000 +0200 +++ /tmp/wklog.66.new.28072 2009-12-11 15:08:16.000000000 +0200 @@ -1 +1,6 @@ +All items on which subquery depend could be collected in +st_select_lex::mark_as_dependent (direct of indirect reference?) + +Temporary table index should be created by all fields except result field +(TMP_TABLE_PARAM::keyinfo). -=-=(Sanja - Fri, 11 Dec 2009, 15:05)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.27795 2009-12-11 15:05:04.000000000 +0200 +++ /tmp/wklog.66.new.27795 2009-12-11 15:05:04.000000000 +0200 @@ -1 +1,6 @@ +Attach subquery cache to each Item_subquery. Interface should allow to use hash +or temporary table inside. + +To check/discuss: +Are there sens to put subquery cache on all levels of subqueries of on highest. DESCRIPTION: Collect all outer items/references (left part of the subquiery and outer references inside the subquery) in key string. Compare the string (which represents certain value set of the references) against values in hash table and return cached result of subquery if the reference values combination has already been used. For example in the following subquery: (L1, L2) IN (SELECT A, B FROM T WHERE T.F1>OTER_FIELD) set of references to look into the subquery cache is (L1, L2, OTER_FIELD). The subquery cache should be implemented as simple LRU connected to the subquery. Size of the subquery cache (in number of results (but maybe in used memory amount)) is limited by session variable (query parameter?). HIGH-LEVEL SPECIFICATION: Attach subquery cache to each Item_subquery. Interface should allow to use hash or temporary table inside. To check/discuss: To put subquery cache on all levels of subqueries or on highest level only. <contents> 1. Scope of the task 2. Data structure used for the cache 3. Cache size 4. Interplay with other subquery optimizations 5. User interface </contents> 1. Scope of the task -------------------- This WL should handle all subquery predicates, i.e. it should handle these cases: outer_expr IN (SELECT correlated_select) outer_expr $CMP$ ALL/ANY (SELECT correlated_select) EXISTS (SELECT correlated_select) scalar-context subquery: (SELECT correlated_select) The cache will maintain (outer_expr, correlation_references)-> subquery_item_result mapping, where - correlation_references is a list of tablename.column_name that are referred from the correlated_select but tablename is a table that is ouside the subquery. - subquery_item_result is 'bool' for subquery predicates, and is of some scalar or ROW(scalar1,...scalarN) type for scalar-context subquery. We dont support cases when outer_expr or correlation_references are blobs. 2. Data structure used for the cache ------------------------------------ There are two data structures available in the codebase that will allow fast equality lookups: 1. HASH (mysys/hash.c) tables 2. Temporary tables (the ones that are used for e.g. GROUP BY) None of them has any support for element eviction on overflow (using LRU or some other policy). Query cache and MyISAM/Maria's key/page cache ought to support some eviction mechanism, but code-wise it is not readily reusable, one will need to factor it out (or copy it). We choose to use #2, and not to have any eviction policy. See subsequent sections for details and reasoning behind the decision. 3. Cache size ------------- Typically, a cache has some maximum size and a policy which is used to select a cache entry for removal when the cache becomes full (e.g. find and remove the least [recently] used entry) For this WL entry we will use a cache of infinite size. The reasoning behind this is that: - is is easy to do: we have temporary tables that can grow to arbitrarily large size while still providing the same insert/lookup interface. - it suits us: unless the subquery is resolved with one index lookup, hitting the cache would be many times cheaper than re-running the subquery, so cache is worth having. 4. Interplay with other subquery optimizations ---------------------------------------------- * This WL entry should not care about IN->EXISTS transformation: caching for IN subquery and result of its conversion to EXISTS would work in the same way. * This optimization is orthogonal to <=>ANY -> MIN/MAX rewrite (it will work/be useful irrespectively of whether the rewrite has been performed or not) * TODO: compare this with materialization for uncorrelated IN-subqueries. Is this basically the same? A: no, it is not: - IN-Materialization has to perform full materialization before it can do the first subquery evaluation. This WL's code has almost no startup costs. - This optimization has temp.table of (corr_reference, predicate_value), while IN-materialization will have (corr_reference) only. 5. User interface ----------------- * There will be an @@optimizer_switch flag to turn this optimization on and off (TODO: name of the flag?) * TODO: how do we show this in EXPLAIN [EXTENDED]? The most easiest is to print something in the warning text of EXPLAIN EXTEDED that would indicate use of cache. * temporary table sizing (max size for heap table, whether to use MyISAM or Maria) will be controlled with common temp.table control variables. LOW-LEVEL DESIGN: All items on which subquery depend could be collected in st_select_lex::mark_as_dependent (direct of indirect reference?) Temporary table index should be created by all fields except result field (TMP_TABLE_PARAM::keyinfo). How to make lookups into temptable ---------------------------------- We'll reuse approach used by SJ-Materialization in 6.0. Setup process ~~~~~~~~~~~~~ Setup is performed in the same way as in setup_sj_materialization(), see the code that starts these lines: /* Create/initialize everything we will need to index lookups into the temptable. */ and ends at this line: Remove the injected semi-join IN-equalities from join_tab conds. This <questionable> We'll also need to check equalities, i.e. do an equivalent of this: if (!(sjm->in_equality= create_subq_in_equalities(thd, sjm, emb_sj_nest->sj_subq_pred))) DBUG_RETURN(TRUE); /* purecov: inspected */ Question: or perhaps that is not necessarry? </questionable> ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] [Branch ~maria-captains/maria/5.1] Rev 2801: Fixes for some randomly occuring test failures in Buildbot.
by noreply＠launchpad.net 18 Jan '10

18 Jan '10

------------------------------------------------------------ revno: 2801 committer: knielsen(a)knielsen-hq.org branch nick: work-5.1 timestamp: Mon 2010-01-18 13:56:10 +0100 message: Fixes for some randomly occuring test failures in Buildbot. modified: mysql-test/r/innodb-timeout.result mysql-test/suite/rpl/r/rpl_relayspace.result mysql-test/suite/rpl/t/rpl_relayspace.test mysql-test/t/innodb-timeout.test mysql-test/valgrind.supp -- lp:maria https://code.launchpad.net/~maria-captains/maria/5.1 Your team Maria developers is subscribed to branch lp:maria. To unsubscribe from this branch go to https://code.launchpad.net/~maria-captains/maria/5.1/+edit-subscription.

1 0