[Maria-developers] Help with memory leak in optimiser code
Hi Sergey, Thanks for offering to help with the memory leak. Here are the details. Tree is here: lp:~maria-captains/maria/mariadb-5.1-knielsen This tree is the merge with MySQL-5.1.44, including your fix for the uninitialised variable in table elimination. The test case is the following, which is a simplified version of main.union (so main.union shows the same memory leak): -------------------------------- cut here -------------------------------- CREATE TABLE t1 (a VARCHAR(10), FULLTEXT KEY a (a)); INSERT INTO t1 VALUES (1),(2); CREATE TABLE t2 (b INT); INSERT INTO t2 VALUES (1),(2); EXPLAIN EXTENDED SELECT * FROM t1 UNION SELECT * FROM t1 ORDER BY (SELECT a FROM t2 WHERE b = 12); DROP TABLE t1,t2; -------------------------------- cut here -------------------------------- Here is the stack trace from Valgrind: main.knielsen [ pass ] 1604 ***Warnings generated in error logs during shutdown after running tests: main.knielsen ==11409== ==11409== 1,440 bytes in 1 blocks are definitely lost in loss record 7 of 7 ==11409== at 0x4C22FAB: malloc (vg_replace_malloc.c:207) ==11409== by 0xB3DD44: my_malloc (my_malloc.c:37) ==11409== by 0xB4D078: init_dynamic_array2 (array.c:64) ==11409== by 0x7146BA: update_ref_and_keys(THD*, st_dynamic_array*, st_join_table*, unsigned, Item*, COND_EQUAL*, unsigned long long, st_select_lex*, st_sargable_param**) (sql_select.cc:3930) ==11409== by 0x7156C5: make_join_statistics(JOIN*, TABLE_LIST*, Item*, st_dynamic_array*) (sql_select.cc:2721) ==11409== by 0x717998: JOIN::optimize() (sql_select.cc:1002) ==11409== by 0x71BC37: mysql_select(THD*, Item***, TABLE_LIST*, unsigned, List<Item>&, Item*, unsigned, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*) (sql_select.cc:2473) ==11409== by 0x71C1C5: mysql_explain_union(THD*, st_select_lex_unit*, select_result*) (sql_select.cc:16946) ==11409== by 0x71E942: select_describe(JOIN*, bool, bool, bool, char const*) (sql_select.cc:16887) ==11409== by 0x71F60F: JOIN::exec() (sql_select.cc:1837) ==11409== by 0x846859: st_select_lex_unit::exec() (sql_union.cc:513) ==11409== by 0x71C0A0: mysql_explain_union(THD*, st_select_lex_unit*, select_result*) (sql_select.cc:16929) ==11409== by 0x688108: execute_sqlcom_select(THD*, TABLE_LIST*) (sql_parse.cc:5091) ==11409== by 0x68A2AC: mysql_execute_command(THD*) (sql_parse.cc:2299) ==11409== by 0x692EFF: mysql_parse(THD*, char const*, unsigned, char const**) (sql_parse.cc:6034) ==11409== by 0x693D11: dispatch_command(enum_server_command, THD*, char*, unsigned) (sql_parse.cc:1247) - Kristian.
Hi Kristian, On Tue, Mar 09, 2010 at 04:11:26PM +0100, Kristian Nielsen wrote:
Thanks for offering to help with the memory leak. Here are the details.
Tree is here:
lp:~maria-captains/maria/mariadb-5.1-knielsen
This tree is the merge with MySQL-5.1.44, including your fix for the uninitialised variable in table elimination.
The test case is the following, which is a simplified version of main.union (so main.union shows the same memory leak):
-------------------------------- cut here -------------------------------- CREATE TABLE t1 (a VARCHAR(10), FULLTEXT KEY a (a)); INSERT INTO t1 VALUES (1),(2); CREATE TABLE t2 (b INT); INSERT INTO t2 VALUES (1),(2);
EXPLAIN EXTENDED SELECT * FROM t1 UNION SELECT * FROM t1 ORDER BY (SELECT a FROM t2 WHERE b = 12);
DROP TABLE t1,t2; -------------------------------- cut here --------------------------------
Please find the fix below. The fix has a relatively high chance of breaking something (I've discovered that this kind of subqueries is processed in a way that would surprise pretty much everyone I think), so it's better to wait for a full buildbot run before we include it in the main tree. ----- Forwarded message from Sergey Petrunya <psergey@askmonty.org> ----- From: Sergey Petrunya <psergey@askmonty.org> To: maria-developers@lists.launchpad.net X-Mailer: mail (GNU Mailutils 1.2) Date: Tue, 9 Mar 2010 21:53:58 +0300 (MSK) Subject: [Maria-developers] Rev 2798: Fix a buildbot memory leak due to JOIN::destroy() not being called for EXPLAIN in file:///home/psergey/dev/mariadb-5.1-knielsen/ At file:///home/psergey/dev/mariadb-5.1-knielsen/ ------------------------------------------------------------ revno: 2798 revision-id: psergey@askmonty.org-20100309185356-ns7o30p75xjjby35 parent: knielsen@knielsen-hq.org-20100309150559-07zcnnz8dh54m2ug committer: Sergey Petrunya <psergey@askmonty.org> branch nick: mariadb-5.1-knielsen timestamp: Tue 2010-03-09 21:53:56 +0300 message: Fix a buildbot memory leak due to JOIN::destroy() not being called for EXPLAIN query: - When subquery is located in ORDER BY, EXPLAIN will run as follows: select_describe() will run JOIN::prepare()/optimize() for the subquery; then at some point subselect_single_select_engine::prepare() will be called, which will create another join and run join->prepare(). In mainline mysql this is not a problem because subquery's join will be destroyed after the first call. In MariaDB, it won't (table elimination needs to keep JOIN objects around for longer in order to know which tables were eliminated when constructing EXPLAIN EXTENDED warning). Fix the problem of memory leak by calling select_lex->cleanup() in subselect_single_select_engine::prepare(). === modified file 'sql/item_subselect.cc' --- a/sql/item_subselect.cc 2010-03-09 15:03:54 +0000 +++ b/sql/item_subselect.cc 2010-03-09 18:53:56 +0000 @@ -1776,6 +1776,10 @@ { if (prepared) return 0; + if (select_lex->join) + { + select_lex->cleanup(); + } join= new JOIN(thd, select_lex->item_list, select_lex->options | SELECT_NO_UNLOCK, result); if (!join || !result) _______________________________________________ Mailing list: https://launchpad.net/~maria-developers Post to : maria-developers@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-developers More help : https://help.launchpad.net/ListHelp ----- End forwarded message ----- BR Sergey -- Sergey Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog
Hi!
"Sergey" == Sergey Petrunya <psergey@askmonty.org> writes:
Sergey> Hi Kristian, Sergey> On Tue, Mar 09, 2010 at 04:11:26PM +0100, Kristian Nielsen wrote:
Thanks for offering to help with the memory leak. Here are the details.
Tree is here:
lp:~maria-captains/maria/mariadb-5.1-knielsen
This tree is the merge with MySQL-5.1.44, including your fix for the uninitialised variable in table elimination.
The test case is the following, which is a simplified version of main.union (so main.union shows the same memory leak):
-------------------------------- cut here -------------------------------- CREATE TABLE t1 (a VARCHAR(10), FULLTEXT KEY a (a)); INSERT INTO t1 VALUES (1),(2); CREATE TABLE t2 (b INT); INSERT INTO t2 VALUES (1),(2);
EXPLAIN EXTENDED SELECT * FROM t1 UNION SELECT * FROM t1 ORDER BY (SELECT a FROM t2 WHERE b = 12);
DROP TABLE t1,t2; -------------------------------- cut here --------------------------------
<cut> Sergey> subselect_single_select_engine::prepare(). Sergey> === modified file 'sql/item_subselect.cc' Sergey> --- a/sql/item_subselect.cc 2010-03-09 15:03:54 +0000 Sergey> +++ b/sql/item_subselect.cc 2010-03-09 18:53:56 +0000 Sergey> @@ -1776,6 +1776,10 @@ Sergey> { Sergey> if (prepared) Sergey> return 0; Sergey> + if (select_lex->join) Sergey> + { Sergey> + select_lex->cleanup(); Sergey> + } Sergey> join= new JOIN(thd, select_lex->item_list, Sergey> select_lex->options | SELECT_NO_UNLOCK, result); Sergey> if (!join || !result) Sanja, as you know the code, could you please verify the above bug fix? Regards, Monty
Hi! 21 марта 2010, в 20:45, Michael Widenius написал(а): [skip]
Sergey> subselect_single_select_engine::prepare(). Sergey> === modified file 'sql/item_subselect.cc' Sergey> --- a/sql/item_subselect.cc 2010-03-09 15:03:54 +0000 Sergey> +++ b/sql/item_subselect.cc 2010-03-09 18:53:56 +0000 Sergey> @@ -1776,6 +1776,10 @@ Sergey> { Sergey> if (prepared) Sergey> return 0; Sergey> + if (select_lex->join) Sergey> + { Sergey> + select_lex->cleanup(); Sergey> + } Sergey> join= new JOIN(thd, select_lex->item_list, Sergey> select_lex->options | SELECT_NO_UNLOCK, result); Sergey> if (!join || !result)
Sanja, as you know the code, could you please verify the above bug fix?
On first look it is not correct, cleunup() should be called before. But I'll check it in details.
participants (4)
-
Kristian Nielsen -
Michael Widenius -
Oleksandr Byelkin -
Sergey Petrunya