- developers - lists.mariadb.org

[Maria-developers] Updated (by Knielsen): Enable the use of libstdc++ in MariaDB (63)
by worklog-noreply＠askmonty.org 14 Jan '10

14 Jan '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Enable the use of libstdc++ in MariaDB CREATION DATE..: Wed, 11 Nov 2009, 13:19 SUPERVISOR.....: Monty IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Server-BackLog TASK ID........: 63 (http://askmonty.org/worklog/?tid=63) VERSION........: Server-5.2 STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Knielsen - Thu, 14 Jan 2010, 13:45)=-=- High-Level Specification modified. --- /tmp/wklog.63.old.13967 2010-01-14 11:45:49.000000000 +0000 +++ /tmp/wklog.63.new.13967 2010-01-14 11:45:49.000000000 +0000 @@ -1 +1,37 @@ +I did some investigation into this. + +The simple way to do this is to simply use g++ to link C++ objects. So this +issue is really restricted to GCC compilation where we by default prefer to +link with gcc even for C++ code. So this means building with + + CXX=g++ + +The consequences of doing this for the binaries is the addition of two +additional run-time .so dependencies: libstdc++.so and libgcc_s.so. + +It still needs to be investigated if these additional dependencies are a +problem for binary tarball packages, or if the ABI for those libraries are now +as stable as libc.so. + +The libgcc_s.so is needed as a dependency to support exceptions between +different object files, as they need to use the same code for stack unwinding. +libstdc++.so is of course needed for access to C++ runtime. + +I researched into the possibility to instead link only specific plugins with +g++, and continue to link the rest of the server with gcc. Unfortunately, this +seems really hard to do in a proper way due to the way autotools works. At +configure time, a script ./libtool is created with hardcoded compiler commands +derived from $CC and $CXX. This script is then used to do the actual linking +in Makefiles generated by Automake. I thus did not find a way to change the +linker command on a per-makefile basis, as libtool is global to the project. + +One option would be to use separate configure.in for plugins, but this is +quite an intrusive change. + +My conclusion is that the best way is to start using g++ for linking the +entire server. This is no problem for binaries made for a specific +distribution (Ubuntu, Debian, Centos), where the dependencies are handled by +the package manager. If it is a big problem for binary tarball releases, at +worst we can build multiple binary tarball releases for the different library +versions we need to support. -=-=(Guest - Tue, 12 Jan 2010, 16:26)=-=- Version updated. --- /tmp/wklog.63.old.19522 2010-01-12 16:26:23.000000000 +0200 +++ /tmp/wklog.63.new.19522 2010-01-12 16:26:23.000000000 +0200 @@ -1 +1 @@ -Connector/.NET-5.2 +Server-5.2 -=-=(Guest - Tue, 12 Jan 2010, 16:26)=-=- Category updated. --- /tmp/wklog.63.old.19506 2010-01-12 16:26:15.000000000 +0200 +++ /tmp/wklog.63.new.19506 2010-01-12 16:26:15.000000000 +0200 @@ -1 +1 @@ -Server-RawIdeaBin +Server-BackLog DESCRIPTION: Enable the use of libstdc++ in MariaDB. As time goes on, more and more plugins and external code/library will need linking to libstdc++ for stuff it uses. I have already seen this happen several times, with extra work needed to integrate things properly. It would be nice to have a general solution for this so that it is not necessary to spend time on individual solutions in each case. It also needs to be considered what the impact of this will be for the server in terms of binary compatibility, performance etc. I think it should be mostly ok, except that it might introduce a problem for bintar packages with an external dependency on libstdc++. HIGH-LEVEL SPECIFICATION: I did some investigation into this. The simple way to do this is to simply use g++ to link C++ objects. So this issue is really restricted to GCC compilation where we by default prefer to link with gcc even for C++ code. So this means building with CXX=g++ The consequences of doing this for the binaries is the addition of two additional run-time .so dependencies: libstdc++.so and libgcc_s.so. It still needs to be investigated if these additional dependencies are a problem for binary tarball packages, or if the ABI for those libraries are now as stable as libc.so. The libgcc_s.so is needed as a dependency to support exceptions between different object files, as they need to use the same code for stack unwinding. libstdc++.so is of course needed for access to C++ runtime. I researched into the possibility to instead link only specific plugins with g++, and continue to link the rest of the server with gcc. Unfortunately, this seems really hard to do in a proper way due to the way autotools works. At configure time, a script ./libtool is created with hardcoded compiler commands derived from $CC and $CXX. This script is then used to do the actual linking in Makefiles generated by Automake. I thus did not find a way to change the linker command on a per-makefile basis, as libtool is global to the project. One option would be to use separate configure.in for plugins, but this is quite an intrusive change. My conclusion is that the best way is to start using g++ for linking the entire server. This is no problem for binaries made for a specific distribution (Ubuntu, Debian, Centos), where the dependencies are handled by the package manager. If it is a big problem for binary tarball releases, at worst we can build multiple binary tarball releases for the different library versions we need to support. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Knielsen): Enable the use of libstdc++ in MariaDB (63)
by worklog-noreply＠askmonty.org 14 Jan '10

14 Jan '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Enable the use of libstdc++ in MariaDB CREATION DATE..: Wed, 11 Nov 2009, 13:19 SUPERVISOR.....: Monty IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Server-BackLog TASK ID........: 63 (http://askmonty.org/worklog/?tid=63) VERSION........: Server-5.2 STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Knielsen - Thu, 14 Jan 2010, 13:45)=-=- High-Level Specification modified. --- /tmp/wklog.63.old.13967 2010-01-14 11:45:49.000000000 +0000 +++ /tmp/wklog.63.new.13967 2010-01-14 11:45:49.000000000 +0000 @@ -1 +1,37 @@ +I did some investigation into this. + +The simple way to do this is to simply use g++ to link C++ objects. So this +issue is really restricted to GCC compilation where we by default prefer to +link with gcc even for C++ code. So this means building with + + CXX=g++ + +The consequences of doing this for the binaries is the addition of two +additional run-time .so dependencies: libstdc++.so and libgcc_s.so. + +It still needs to be investigated if these additional dependencies are a +problem for binary tarball packages, or if the ABI for those libraries are now +as stable as libc.so. + +The libgcc_s.so is needed as a dependency to support exceptions between +different object files, as they need to use the same code for stack unwinding. +libstdc++.so is of course needed for access to C++ runtime. + +I researched into the possibility to instead link only specific plugins with +g++, and continue to link the rest of the server with gcc. Unfortunately, this +seems really hard to do in a proper way due to the way autotools works. At +configure time, a script ./libtool is created with hardcoded compiler commands +derived from $CC and $CXX. This script is then used to do the actual linking +in Makefiles generated by Automake. I thus did not find a way to change the +linker command on a per-makefile basis, as libtool is global to the project. + +One option would be to use separate configure.in for plugins, but this is +quite an intrusive change. + +My conclusion is that the best way is to start using g++ for linking the +entire server. This is no problem for binaries made for a specific +distribution (Ubuntu, Debian, Centos), where the dependencies are handled by +the package manager. If it is a big problem for binary tarball releases, at +worst we can build multiple binary tarball releases for the different library +versions we need to support. -=-=(Guest - Tue, 12 Jan 2010, 16:26)=-=- Version updated. --- /tmp/wklog.63.old.19522 2010-01-12 16:26:23.000000000 +0200 +++ /tmp/wklog.63.new.19522 2010-01-12 16:26:23.000000000 +0200 @@ -1 +1 @@ -Connector/.NET-5.2 +Server-5.2 -=-=(Guest - Tue, 12 Jan 2010, 16:26)=-=- Category updated. --- /tmp/wklog.63.old.19506 2010-01-12 16:26:15.000000000 +0200 +++ /tmp/wklog.63.new.19506 2010-01-12 16:26:15.000000000 +0200 @@ -1 +1 @@ -Server-RawIdeaBin +Server-BackLog DESCRIPTION: Enable the use of libstdc++ in MariaDB. As time goes on, more and more plugins and external code/library will need linking to libstdc++ for stuff it uses. I have already seen this happen several times, with extra work needed to integrate things properly. It would be nice to have a general solution for this so that it is not necessary to spend time on individual solutions in each case. It also needs to be considered what the impact of this will be for the server in terms of binary compatibility, performance etc. I think it should be mostly ok, except that it might introduce a problem for bintar packages with an external dependency on libstdc++. HIGH-LEVEL SPECIFICATION: I did some investigation into this. The simple way to do this is to simply use g++ to link C++ objects. So this issue is really restricted to GCC compilation where we by default prefer to link with gcc even for C++ code. So this means building with CXX=g++ The consequences of doing this for the binaries is the addition of two additional run-time .so dependencies: libstdc++.so and libgcc_s.so. It still needs to be investigated if these additional dependencies are a problem for binary tarball packages, or if the ABI for those libraries are now as stable as libc.so. The libgcc_s.so is needed as a dependency to support exceptions between different object files, as they need to use the same code for stack unwinding. libstdc++.so is of course needed for access to C++ runtime. I researched into the possibility to instead link only specific plugins with g++, and continue to link the rest of the server with gcc. Unfortunately, this seems really hard to do in a proper way due to the way autotools works. At configure time, a script ./libtool is created with hardcoded compiler commands derived from $CC and $CXX. This script is then used to do the actual linking in Makefiles generated by Automake. I thus did not find a way to change the linker command on a per-makefile basis, as libtool is global to the project. One option would be to use separate configure.in for plugins, but this is quite an intrusive change. My conclusion is that the best way is to start using g++ for linking the entire server. This is no problem for binaries made for a specific distribution (Ubuntu, Debian, Centos), where the dependencies are handled by the package manager. If it is a big problem for binary tarball releases, at worst we can build multiple binary tarball releases for the different library versions we need to support. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] [Branch ~maria-captains/maria/5.1] Rev 2796: Merged patch from Percona to get proper fix for compilation issue of srv0srv.c on Solaris
by noreply＠launchpad.net 12 Jan '10

12 Jan '10

------------------------------------------------------------ revno: 2796 committer: Michael Widenius <monty(a)askmonty.org> branch nick: maria-5.1 timestamp: Tue 2010-01-12 19:31:11 +0200 message: Merged patch from Percona to get proper fix for compilation issue of srv0srv.c on Solaris modified: sql/sql_cache.cc storage/xtradb/srv/srv0srv.c -- lp:maria https://code.launchpad.net/~maria-captains/maria/5.1 Your team Maria developers is subscribed to branch lp:maria. To unsubscribe from this branch go to https://code.launchpad.net/~maria-captains/maria/5.1/+edit-subscription.

1 0

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (monty:2796)
by Michael Widenius 12 Jan '10

12 Jan '10

#At lp:maria based on revid:monty@askmonty.org-20100112102853-jc9hnlfg5o853oo5 2796 Michael Widenius 2010-01-12 Merged patch from Percona to get proper fix for compilation issue of srv0srv.c on Solaris modified: sql/sql_cache.cc storage/xtradb/srv/srv0srv.c per-file messages: sql/sql_cache.cc Fixed wrong comment storage/xtradb/srv/srv0srv.c Merged patch from Percona to get proper fix for compilation issue of srv0srv.c on Solaris === modified file 'sql/sql_cache.cc' --- a/sql/sql_cache.cc 2009-12-03 11:19:05 +0000 +++ b/sql/sql_cache.cc 2010-01-12 17:31:11 +0000 @@ -1306,8 +1306,8 @@ end: to the user. RESULTS - 1 Query was not cached. - 0 The query was cached and user was sent the result. + 0 Query was not cached. + 1 The query was cached and user was sent the result. -1 The query was cached but we didn't have rights to use it. No error is sent to the client yet. === modified file 'storage/xtradb/srv/srv0srv.c' --- a/storage/xtradb/srv/srv0srv.c 2010-01-06 21:27:53 +0000 +++ b/storage/xtradb/srv/srv0srv.c 2010-01-12 17:31:11 +0000 @@ -81,7 +81,6 @@ Created 10/8/1995 Heikki Tuuri #include "ut0mem.h" #include "ut0ut.h" #include "os0proc.h" -#include "os0sync.h" #include "mem0mem.h" #include "mem0pool.h" #include "sync0sync.h" @@ -103,6 +102,10 @@ Created 10/8/1995 Heikki Tuuri #include "row0mysql.h" #include "ha_prototypes.h" #include "trx0i_s.h" +#include "os0sync.h" /* for HAVE_ATOMIC_BUILTINS */ + +/* prototypes for new functions added to ha_innodb.cc */ +ibool innobase_get_slow_log(); /* This is set to TRUE if the MySQL user has set it in MySQL; currently affects only FOREIGN KEY definition parsing */ @@ -162,6 +165,7 @@ UNIV_INTERN ulint* srv_data_file_sizes = UNIV_INTERN ibool srv_extra_undoslots = FALSE; UNIV_INTERN ibool srv_fast_recovery = FALSE; +UNIV_INTERN ibool srv_recovery_stats = FALSE; UNIV_INTERN ibool srv_use_purge_thread = FALSE; @@ -1077,7 +1081,7 @@ UNIV_INTERN ulong srv_max_purge_lag = 0 Puts an OS thread to wait if there are too many concurrent threads (>= srv_thread_concurrency) inside InnoDB. The threads wait in a FIFO queue. */ -#ifdef INNODB_RW_LOCKS_USE_ATOMICS +#ifdef HAVE_ATOMIC_BUILTINS static void enter_innodb_with_tickets(trx_t* trx) { @@ -1175,7 +1179,7 @@ srv_conc_enter_innodb( return; } -#ifdef INNODB_RW_LOCKS_USE_ATOMICS +#ifdef HAVE_ATOMIC_BUILTINS if (srv_thread_concurrency_timer_based) { srv_conc_enter_innodb_timer_based(trx); return; @@ -1325,7 +1329,7 @@ srv_conc_force_enter_innodb( } ut_ad(srv_conc_n_threads >= 0); -#ifdef INNODB_RW_LOCKS_USE_ATOMICS +#ifdef HAVE_ATOMIC_BUILTINS if (srv_thread_concurrency_timer_based) { os_atomic_increment_lint(&srv_conc_n_threads, 1); trx->declared_to_be_inside_innodb = TRUE; @@ -1366,7 +1370,7 @@ srv_conc_force_exit_innodb( return; } -#ifdef INNODB_RW_LOCKS_USE_ATOMICS +#ifdef HAVE_ATOMIC_BUILTINS if (srv_thread_concurrency_timer_based) { srv_conc_exit_innodb_timer_based(trx); return;

1 0

[Maria-developers] Updated (by Psergey): Subquery optimization: Avoid recalculating subquery if external fields values found in subquery cache (66)
by worklog-noreply＠askmonty.org 12 Jan '10

12 Jan '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Subquery optimization: Avoid recalculating subquery if external fields values found in subquery cache CREATION DATE..: Wed, 25 Nov 2009, 22:25 SUPERVISOR.....: Monty IMPLEMENTOR....: Sanja COPIES TO......: CATEGORY.......: Server-BackLog TASK ID........: 66 (http://askmonty.org/worklog/?tid=66) VERSION........: Server-5.2 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Psergey - Tue, 12 Jan 2010, 18:39)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.31666 2010-01-12 18:39:43.000000000 +0200 +++ /tmp/wklog.66.new.31666 2010-01-12 18:39:43.000000000 +0200 @@ -4,3 +4,99 @@ To check/discuss: To put subquery cache on all levels of subqueries or on highest level only. + + +<contents> +1. Scope of the task +2. Data structure used for the cache +3. Cache size +4. Interplay with other subquery optimizations +5. User interface +</contents> + +1. Scope of the task +-------------------- +This WL should handle all subquery predicates, i.e. it should handle these +cases: + + outer_expr IN (SELECT correlated_select) + outer_expr $CMP$ ALL/ANY (SELECT correlated_select) + EXISTS (SELECT correlated_select) + scalar-context subquery: (SELECT correlated_select) + +The cache will maintain + + (outer_expr, correlation_references)-> subquery_item_result + +mapping, where +- correlation_references is a list of tablename.column_name that are referred + from the correlated_select but tablename is a table that is ouside the + subquery. +- subquery_item_result is 'bool' for subquery predicates, and is of +some scalar or ROW(scalar1,...scalarN) type for scalar-context subquery. + +We dont support cases when outer_expr or correlation_references are blobs. + +2. Data structure used for the cache +------------------------------------ +There are two data structures available in the codebase that will allow fast +equality lookups: + +1. HASH (mysys/hash.c) tables +2. Temporary tables (the ones that are used for e.g. GROUP BY) + +None of them has any support for element eviction on overflow (using LRU or +some other policy). + +Query cache and MyISAM/Maria's key/page cache ought to support some eviction +mechanism, but code-wise it is not readily reusable, one will need to factor +it out (or copy it). + +We choose to use #2, and not to have any eviction policy. See subsequent +sections for details and reasoning behind the decision. + +3. Cache size +------------- +Typically, a cache has some maximum size and a policy which is used to +select a cache entry for removal when the cache becomes full (e.g. find +and remove the least [recently] used entry) + +For this WL entry we will use a cache of infinite size. The reasoning behind +this is that: +- is is easy to do: we have temporary tables that can grow to arbitrarily + large size while still providing the same insert/lookup interface. +- it suits us: unless the subquery is resolved with one index lookup, + hitting the cache would be many times cheaper than re-running the + subquery, so cache is worth having. + +4. Interplay with other subquery optimizations +---------------------------------------------- +* This WL entry should not care about IN->EXISTS transformation: caching for + IN subquery and result of its conversion to EXISTS would work in the same + way. + +* This optimization is orthogonal to <=>ANY -> MIN/MAX rewrite (it will + work/be useful irrespectively of whether the rewrite has been performed or + not) + +* TODO: compare this with materialization for uncorrelated IN-subqueries. Is + this basically the same? + A: no, it is not: + - IN-Materialization has to perform full materialization before it can + do the first subquery evaluation. This WL's code has almost no startup + costs. + - This optimization has temp.table of (corr_reference, predicate_value), + while IN-materialization will have (corr_reference) only. + +5. User interface +----------------- +* There will be an @@optimizer_switch flag to turn this optimization on and + off (TODO: name of the flag?) + +* TODO: how do we show this in EXPLAIN [EXTENDED]? The most easiest is to + print something in the warning text of EXPLAIN EXTEDED that would indicate + use of cache. + +* temporary table sizing (max size for heap table, whether to use MyISAM or + Maria) will be controlled with common temp.table control variables. + -=-=(Psergey - Mon, 11 Jan 2010, 13:25)=-=- As of today, there is code that - collects outside references - creates a temporary table with index that would allow for fast lookups. there is no code to - fill the temporary table - make lookups into it Reported zero hours worked. Estimate unchanged. -=-=(Sanja - Fri, 11 Dec 2009, 15:09)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.28164 2009-12-11 15:09:15.000000000 +0200 +++ /tmp/wklog.66.new.28164 2009-12-11 15:09:15.000000000 +0200 @@ -3,4 +3,4 @@ To check/discuss: -Are there sens to put subquery cache on all levels of subqueries of on highest. + To put subquery cache on all levels of subqueries or on highest level only. -=-=(Sanja - Fri, 11 Dec 2009, 15:08)=-=- Low Level Design modified. --- /tmp/wklog.66.old.28072 2009-12-11 15:08:16.000000000 +0200 +++ /tmp/wklog.66.new.28072 2009-12-11 15:08:16.000000000 +0200 @@ -1 +1,6 @@ +All items on which subquery depend could be collected in +st_select_lex::mark_as_dependent (direct of indirect reference?) + +Temporary table index should be created by all fields except result field +(TMP_TABLE_PARAM::keyinfo). -=-=(Sanja - Fri, 11 Dec 2009, 15:05)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.27795 2009-12-11 15:05:04.000000000 +0200 +++ /tmp/wklog.66.new.27795 2009-12-11 15:05:04.000000000 +0200 @@ -1 +1,6 @@ +Attach subquery cache to each Item_subquery. Interface should allow to use hash +or temporary table inside. + +To check/discuss: +Are there sens to put subquery cache on all levels of subqueries of on highest. DESCRIPTION: Collect all outer items/references (left part of the subquiery and outer references inside the subquery) in key string. Compare the string (which represents certain value set of the references) against values in hash table and return cached result of subquery if the reference values combination has already been used. For example in the following subquery: (L1, L2) IN (SELECT A, B FROM T WHERE T.F1>OTER_FIELD) set of references to look into the subquery cache is (L1, L2, OTER_FIELD). The subquery cache should be implemented as simple LRU connected to the subquery. Size of the subquery cache (in number of results (but maybe in used memory amount)) is limited by session variable (query parameter?). HIGH-LEVEL SPECIFICATION: Attach subquery cache to each Item_subquery. Interface should allow to use hash or temporary table inside. To check/discuss: To put subquery cache on all levels of subqueries or on highest level only. <contents> 1. Scope of the task 2. Data structure used for the cache 3. Cache size 4. Interplay with other subquery optimizations 5. User interface </contents> 1. Scope of the task -------------------- This WL should handle all subquery predicates, i.e. it should handle these cases: outer_expr IN (SELECT correlated_select) outer_expr $CMP$ ALL/ANY (SELECT correlated_select) EXISTS (SELECT correlated_select) scalar-context subquery: (SELECT correlated_select) The cache will maintain (outer_expr, correlation_references)-> subquery_item_result mapping, where - correlation_references is a list of tablename.column_name that are referred from the correlated_select but tablename is a table that is ouside the subquery. - subquery_item_result is 'bool' for subquery predicates, and is of some scalar or ROW(scalar1,...scalarN) type for scalar-context subquery. We dont support cases when outer_expr or correlation_references are blobs. 2. Data structure used for the cache ------------------------------------ There are two data structures available in the codebase that will allow fast equality lookups: 1. HASH (mysys/hash.c) tables 2. Temporary tables (the ones that are used for e.g. GROUP BY) None of them has any support for element eviction on overflow (using LRU or some other policy). Query cache and MyISAM/Maria's key/page cache ought to support some eviction mechanism, but code-wise it is not readily reusable, one will need to factor it out (or copy it). We choose to use #2, and not to have any eviction policy. See subsequent sections for details and reasoning behind the decision. 3. Cache size ------------- Typically, a cache has some maximum size and a policy which is used to select a cache entry for removal when the cache becomes full (e.g. find and remove the least [recently] used entry) For this WL entry we will use a cache of infinite size. The reasoning behind this is that: - is is easy to do: we have temporary tables that can grow to arbitrarily large size while still providing the same insert/lookup interface. - it suits us: unless the subquery is resolved with one index lookup, hitting the cache would be many times cheaper than re-running the subquery, so cache is worth having. 4. Interplay with other subquery optimizations ---------------------------------------------- * This WL entry should not care about IN->EXISTS transformation: caching for IN subquery and result of its conversion to EXISTS would work in the same way. * This optimization is orthogonal to <=>ANY -> MIN/MAX rewrite (it will work/be useful irrespectively of whether the rewrite has been performed or not) * TODO: compare this with materialization for uncorrelated IN-subqueries. Is this basically the same? A: no, it is not: - IN-Materialization has to perform full materialization before it can do the first subquery evaluation. This WL's code has almost no startup costs. - This optimization has temp.table of (corr_reference, predicate_value), while IN-materialization will have (corr_reference) only. 5. User interface ----------------- * There will be an @@optimizer_switch flag to turn this optimization on and off (TODO: name of the flag?) * TODO: how do we show this in EXPLAIN [EXTENDED]? The most easiest is to print something in the warning text of EXPLAIN EXTEDED that would indicate use of cache. * temporary table sizing (max size for heap table, whether to use MyISAM or Maria) will be controlled with common temp.table control variables. LOW-LEVEL DESIGN: All items on which subquery depend could be collected in st_select_lex::mark_as_dependent (direct of indirect reference?) Temporary table index should be created by all fields except result field (TMP_TABLE_PARAM::keyinfo). ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Psergey): Subquery optimization: Avoid recalculating subquery if external fields values found in subquery cache (66)
by worklog-noreply＠askmonty.org 12 Jan '10

12 Jan '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Subquery optimization: Avoid recalculating subquery if external fields values found in subquery cache CREATION DATE..: Wed, 25 Nov 2009, 22:25 SUPERVISOR.....: Monty IMPLEMENTOR....: Sanja COPIES TO......: CATEGORY.......: Server-BackLog TASK ID........: 66 (http://askmonty.org/worklog/?tid=66) VERSION........: Server-5.2 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Psergey - Tue, 12 Jan 2010, 18:39)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.31666 2010-01-12 18:39:43.000000000 +0200 +++ /tmp/wklog.66.new.31666 2010-01-12 18:39:43.000000000 +0200 @@ -4,3 +4,99 @@ To check/discuss: To put subquery cache on all levels of subqueries or on highest level only. + + +<contents> +1. Scope of the task +2. Data structure used for the cache +3. Cache size +4. Interplay with other subquery optimizations +5. User interface +</contents> + +1. Scope of the task +-------------------- +This WL should handle all subquery predicates, i.e. it should handle these +cases: + + outer_expr IN (SELECT correlated_select) + outer_expr $CMP$ ALL/ANY (SELECT correlated_select) + EXISTS (SELECT correlated_select) + scalar-context subquery: (SELECT correlated_select) + +The cache will maintain + + (outer_expr, correlation_references)-> subquery_item_result + +mapping, where +- correlation_references is a list of tablename.column_name that are referred + from the correlated_select but tablename is a table that is ouside the + subquery. +- subquery_item_result is 'bool' for subquery predicates, and is of +some scalar or ROW(scalar1,...scalarN) type for scalar-context subquery. + +We dont support cases when outer_expr or correlation_references are blobs. + +2. Data structure used for the cache +------------------------------------ +There are two data structures available in the codebase that will allow fast +equality lookups: + +1. HASH (mysys/hash.c) tables +2. Temporary tables (the ones that are used for e.g. GROUP BY) + +None of them has any support for element eviction on overflow (using LRU or +some other policy). + +Query cache and MyISAM/Maria's key/page cache ought to support some eviction +mechanism, but code-wise it is not readily reusable, one will need to factor +it out (or copy it). + +We choose to use #2, and not to have any eviction policy. See subsequent +sections for details and reasoning behind the decision. + +3. Cache size +------------- +Typically, a cache has some maximum size and a policy which is used to +select a cache entry for removal when the cache becomes full (e.g. find +and remove the least [recently] used entry) + +For this WL entry we will use a cache of infinite size. The reasoning behind +this is that: +- is is easy to do: we have temporary tables that can grow to arbitrarily + large size while still providing the same insert/lookup interface. +- it suits us: unless the subquery is resolved with one index lookup, + hitting the cache would be many times cheaper than re-running the + subquery, so cache is worth having. + +4. Interplay with other subquery optimizations +---------------------------------------------- +* This WL entry should not care about IN->EXISTS transformation: caching for + IN subquery and result of its conversion to EXISTS would work in the same + way. + +* This optimization is orthogonal to <=>ANY -> MIN/MAX rewrite (it will + work/be useful irrespectively of whether the rewrite has been performed or + not) + +* TODO: compare this with materialization for uncorrelated IN-subqueries. Is + this basically the same? + A: no, it is not: + - IN-Materialization has to perform full materialization before it can + do the first subquery evaluation. This WL's code has almost no startup + costs. + - This optimization has temp.table of (corr_reference, predicate_value), + while IN-materialization will have (corr_reference) only. + +5. User interface +----------------- +* There will be an @@optimizer_switch flag to turn this optimization on and + off (TODO: name of the flag?) + +* TODO: how do we show this in EXPLAIN [EXTENDED]? The most easiest is to + print something in the warning text of EXPLAIN EXTEDED that would indicate + use of cache. + +* temporary table sizing (max size for heap table, whether to use MyISAM or + Maria) will be controlled with common temp.table control variables. + -=-=(Psergey - Mon, 11 Jan 2010, 13:25)=-=- As of today, there is code that - collects outside references - creates a temporary table with index that would allow for fast lookups. there is no code to - fill the temporary table - make lookups into it Reported zero hours worked. Estimate unchanged. -=-=(Sanja - Fri, 11 Dec 2009, 15:09)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.28164 2009-12-11 15:09:15.000000000 +0200 +++ /tmp/wklog.66.new.28164 2009-12-11 15:09:15.000000000 +0200 @@ -3,4 +3,4 @@ To check/discuss: -Are there sens to put subquery cache on all levels of subqueries of on highest. + To put subquery cache on all levels of subqueries or on highest level only. -=-=(Sanja - Fri, 11 Dec 2009, 15:08)=-=- Low Level Design modified. --- /tmp/wklog.66.old.28072 2009-12-11 15:08:16.000000000 +0200 +++ /tmp/wklog.66.new.28072 2009-12-11 15:08:16.000000000 +0200 @@ -1 +1,6 @@ +All items on which subquery depend could be collected in +st_select_lex::mark_as_dependent (direct of indirect reference?) + +Temporary table index should be created by all fields except result field +(TMP_TABLE_PARAM::keyinfo). -=-=(Sanja - Fri, 11 Dec 2009, 15:05)=-=- High-Level Specification modified. --- /tmp/wklog.66.old.27795 2009-12-11 15:05:04.000000000 +0200 +++ /tmp/wklog.66.new.27795 2009-12-11 15:05:04.000000000 +0200 @@ -1 +1,6 @@ +Attach subquery cache to each Item_subquery. Interface should allow to use hash +or temporary table inside. + +To check/discuss: +Are there sens to put subquery cache on all levels of subqueries of on highest. DESCRIPTION: Collect all outer items/references (left part of the subquiery and outer references inside the subquery) in key string. Compare the string (which represents certain value set of the references) against values in hash table and return cached result of subquery if the reference values combination has already been used. For example in the following subquery: (L1, L2) IN (SELECT A, B FROM T WHERE T.F1>OTER_FIELD) set of references to look into the subquery cache is (L1, L2, OTER_FIELD). The subquery cache should be implemented as simple LRU connected to the subquery. Size of the subquery cache (in number of results (but maybe in used memory amount)) is limited by session variable (query parameter?). HIGH-LEVEL SPECIFICATION: Attach subquery cache to each Item_subquery. Interface should allow to use hash or temporary table inside. To check/discuss: To put subquery cache on all levels of subqueries or on highest level only. <contents> 1. Scope of the task 2. Data structure used for the cache 3. Cache size 4. Interplay with other subquery optimizations 5. User interface </contents> 1. Scope of the task -------------------- This WL should handle all subquery predicates, i.e. it should handle these cases: outer_expr IN (SELECT correlated_select) outer_expr $CMP$ ALL/ANY (SELECT correlated_select) EXISTS (SELECT correlated_select) scalar-context subquery: (SELECT correlated_select) The cache will maintain (outer_expr, correlation_references)-> subquery_item_result mapping, where - correlation_references is a list of tablename.column_name that are referred from the correlated_select but tablename is a table that is ouside the subquery. - subquery_item_result is 'bool' for subquery predicates, and is of some scalar or ROW(scalar1,...scalarN) type for scalar-context subquery. We dont support cases when outer_expr or correlation_references are blobs. 2. Data structure used for the cache ------------------------------------ There are two data structures available in the codebase that will allow fast equality lookups: 1. HASH (mysys/hash.c) tables 2. Temporary tables (the ones that are used for e.g. GROUP BY) None of them has any support for element eviction on overflow (using LRU or some other policy). Query cache and MyISAM/Maria's key/page cache ought to support some eviction mechanism, but code-wise it is not readily reusable, one will need to factor it out (or copy it). We choose to use #2, and not to have any eviction policy. See subsequent sections for details and reasoning behind the decision. 3. Cache size ------------- Typically, a cache has some maximum size and a policy which is used to select a cache entry for removal when the cache becomes full (e.g. find and remove the least [recently] used entry) For this WL entry we will use a cache of infinite size. The reasoning behind this is that: - is is easy to do: we have temporary tables that can grow to arbitrarily large size while still providing the same insert/lookup interface. - it suits us: unless the subquery is resolved with one index lookup, hitting the cache would be many times cheaper than re-running the subquery, so cache is worth having. 4. Interplay with other subquery optimizations ---------------------------------------------- * This WL entry should not care about IN->EXISTS transformation: caching for IN subquery and result of its conversion to EXISTS would work in the same way. * This optimization is orthogonal to <=>ANY -> MIN/MAX rewrite (it will work/be useful irrespectively of whether the rewrite has been performed or not) * TODO: compare this with materialization for uncorrelated IN-subqueries. Is this basically the same? A: no, it is not: - IN-Materialization has to perform full materialization before it can do the first subquery evaluation. This WL's code has almost no startup costs. - This optimization has temp.table of (corr_reference, predicate_value), while IN-materialization will have (corr_reference) only. 5. User interface ----------------- * There will be an @@optimizer_switch flag to turn this optimization on and off (TODO: name of the flag?) * TODO: how do we show this in EXPLAIN [EXTENDED]? The most easiest is to print something in the warning text of EXPLAIN EXTEDED that would indicate use of cache. * temporary table sizing (max size for heap table, whether to use MyISAM or Maria) will be controlled with common temp.table control variables. LOW-LEVEL DESIGN: All items on which subquery depend could be collected in st_select_lex::mark_as_dependent (direct of indirect reference?) Temporary table index should be created by all fields except result field (TMP_TABLE_PARAM::keyinfo). ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Monty): options for CREATE TABLE (43)
by worklog-noreply＠askmonty.org 12 Jan '10

12 Jan '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: options for CREATE TABLE CREATION DATE..: Tue, 11 Aug 2009, 17:02 SUPERVISOR.....: Bothorsen IMPLEMENTOR....: Sanja COPIES TO......: Monty CATEGORY.......: Server-BackLog TASK ID........: 43 (http://askmonty.org/worklog/?tid=43) VERSION........: Server-5.2 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 32 (hours remain) ORIG. ESTIMATE.: 32 PROGRESS NOTES: -=-=(Monty - Tue, 12 Jan 2010, 17:04)=-=- Version updated. --- /tmp/wklog.43.old.21802 2010-01-12 17:04:31.000000000 +0200 +++ /tmp/wklog.43.new.21802 2010-01-12 17:04:31.000000000 +0200 @@ -1 +1 @@ -Server-5.1 +Server-5.2 -=-=(Guest - Fri, 18 Sep 2009, 14:24)=-=- Low Level Design modified. --- /tmp/wklog.43.old.31654 2009-09-18 14:24:41.000000000 +0300 +++ /tmp/wklog.43.new.31654 2009-09-18 14:24:41.000000000 +0300 @@ -2,4 +2,6 @@ Handler have access to the options during creation and after opening frm. +It should has C interface. + -=-=(Sanja - Fri, 18 Sep 2009, 13:32)=-=- Low Level Design modified. --- /tmp/wklog.43.old.29567 2009-09-18 13:32:41.000000000 +0300 +++ /tmp/wklog.43.new.29567 2009-09-18 13:32:41.000000000 +0300 @@ -1 +1,5 @@ +Options stored in a list as pairs key, value which are LEX_STRING. + +Handler have access to the options during creation and after opening frm. + -=-=(Sanja - Fri, 18 Sep 2009, 13:21)=-=- High-Level Specification modified. --- /tmp/wklog.43.old.29107 2009-09-18 13:21:19.000000000 +0300 +++ /tmp/wklog.43.new.29107 2009-09-18 13:21:19.000000000 +0300 @@ -6,7 +6,8 @@ table_option1=tval1, table_option2=tval2; Exclusion should be made for old table and key options where -'=' was not obligatory. +'=' was not obligatory. Behaviour and way of storage for existing options will +be left as is. Old key options: KEY_BLOCK_SIZE <num> -> KEY_BLOCK_SIZE=num @@ -42,6 +43,6 @@ For fields options can go with field attributes (NOT NULL, UNIQUE and so on) can be separated from them by '=' sign. - - +Incorrect option during creation/altering table lead only to worning. During +opening unrecognised options should be ignored. -=-=(Guest - Tue, 11 Aug 2009, 19:57)=-=- High-Level Specification modified. --- /tmp/wklog.43.old.31856 2009-08-11 19:57:38.000000000 +0300 +++ /tmp/wklog.43.new.31856 2009-08-11 19:57:38.000000000 +0300 @@ -5,8 +5,43 @@ key key1(field) key_opt1=kval1 key_opt2=kval2) table_option1=tval1, table_option2=tval2; -Exclusion should be made for old table and key (KEY_BLOCK_SIZE) options where +Exclusion should be made for old table and key options where '=' was not obligatory. +Old key options: +KEY_BLOCK_SIZE <num> -> KEY_BLOCK_SIZE=num +WITH PARSER <name> -> PARSER=name + +Old table options: +ENGINE name -> ENGINE=name +TYPE name -> TYPE=name +MAX_ROWS num -> MAX_ROWS=num +MIX_ROWS num -> MIX_ROWS=num +AVG_ROW_LENGTH num -> AVG_ROW_LENGTH=num +PASSWORD string -> PASSWORD=string +COMMENT string -> COMMENT=string +AUTO_INCREMENT num -> AUTO_INCREMENT=num +PACK_KEYS num/default -> PACK_KEYS=num/default +CHECKSUM num -> CHECKSUM=num +TABLE_CHECKSUM num -> TABLE_CHECKSUM=num +PAGE_CHECKSUM num -> PAGE_CHECKSUM=num +DELAY_KEY_WRITE num -> DELAY_KEY_WRITE=num +ROW_FORMAT name -> ROW_FORMAT=name +INSERT_METHOD name -> INSERT_METHOD=name +KEY_BLOCK_SIZE num -> KEY_BLOCK_SIZE=num +TRANSACTIONAL num -> TRANSACTIONAL=num + +Table options which will be left hardcoded +UNION +default charset +default collation +DATA DIRECTORY +TABLESPACE +STORAGE + For fields options can go with field attributes (NOT NULL, UNIQUE and so on) can be separated from them by '=' sign. + + + + -=-=(Guest - Tue, 11 Aug 2009, 19:36)=-=- High-Level Specification modified. --- /tmp/wklog.43.old.30883 2009-08-11 19:36:45.000000000 +0300 +++ /tmp/wklog.43.new.30883 2009-08-11 19:36:45.000000000 +0300 @@ -1 +1,12 @@ +Table definition ca looks like following +CREATE TABLE table + (field int ... field_opt1=fval1 field_opt2=fval2, + key key1(field) key_opt1=kval1 key_opt2=kval2) + table_option1=tval1, table_option2=tval2; + +Exclusion should be made for old table and key (KEY_BLOCK_SIZE) options where +'=' was not obligatory. + +For fields options can go with field attributes (NOT NULL, UNIQUE and so on) can +be separated from them by '=' sign. DESCRIPTION: Add ability to create table with additional option which can be passed to engine. Also make current options such as TRANSACTIONAL working via this mechanism. HIGH-LEVEL SPECIFICATION: Table definition ca looks like following CREATE TABLE table (field int ... field_opt1=fval1 field_opt2=fval2, key key1(field) key_opt1=kval1 key_opt2=kval2) table_option1=tval1, table_option2=tval2; Exclusion should be made for old table and key options where '=' was not obligatory. Behaviour and way of storage for existing options will be left as is. Old key options: KEY_BLOCK_SIZE <num> -> KEY_BLOCK_SIZE=num WITH PARSER <name> -> PARSER=name Old table options: ENGINE name -> ENGINE=name TYPE name -> TYPE=name MAX_ROWS num -> MAX_ROWS=num MIX_ROWS num -> MIX_ROWS=num AVG_ROW_LENGTH num -> AVG_ROW_LENGTH=num PASSWORD string -> PASSWORD=string COMMENT string -> COMMENT=string AUTO_INCREMENT num -> AUTO_INCREMENT=num PACK_KEYS num/default -> PACK_KEYS=num/default CHECKSUM num -> CHECKSUM=num TABLE_CHECKSUM num -> TABLE_CHECKSUM=num PAGE_CHECKSUM num -> PAGE_CHECKSUM=num DELAY_KEY_WRITE num -> DELAY_KEY_WRITE=num ROW_FORMAT name -> ROW_FORMAT=name INSERT_METHOD name -> INSERT_METHOD=name KEY_BLOCK_SIZE num -> KEY_BLOCK_SIZE=num TRANSACTIONAL num -> TRANSACTIONAL=num Table options which will be left hardcoded UNION default charset default collation DATA DIRECTORY TABLESPACE STORAGE For fields options can go with field attributes (NOT NULL, UNIQUE and so on) can be separated from them by '=' sign. Incorrect option during creation/altering table lead only to worning. During opening unrecognised options should be ignored. LOW-LEVEL DESIGN: Options stored in a list as pairs key, value which are LEX_STRING. Handler have access to the options during creation and after opening frm. It should has C interface. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Monty): options for CREATE TABLE (43)
by worklog-noreply＠askmonty.org 12 Jan '10

12 Jan '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: options for CREATE TABLE CREATION DATE..: Tue, 11 Aug 2009, 17:02 SUPERVISOR.....: Bothorsen IMPLEMENTOR....: Sanja COPIES TO......: Monty CATEGORY.......: Server-BackLog TASK ID........: 43 (http://askmonty.org/worklog/?tid=43) VERSION........: Server-5.2 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 32 (hours remain) ORIG. ESTIMATE.: 32 PROGRESS NOTES: -=-=(Monty - Tue, 12 Jan 2010, 17:04)=-=- Version updated. --- /tmp/wklog.43.old.21802 2010-01-12 17:04:31.000000000 +0200 +++ /tmp/wklog.43.new.21802 2010-01-12 17:04:31.000000000 +0200 @@ -1 +1 @@ -Server-5.1 +Server-5.2 -=-=(Guest - Fri, 18 Sep 2009, 14:24)=-=- Low Level Design modified. --- /tmp/wklog.43.old.31654 2009-09-18 14:24:41.000000000 +0300 +++ /tmp/wklog.43.new.31654 2009-09-18 14:24:41.000000000 +0300 @@ -2,4 +2,6 @@ Handler have access to the options during creation and after opening frm. +It should has C interface. + -=-=(Sanja - Fri, 18 Sep 2009, 13:32)=-=- Low Level Design modified. --- /tmp/wklog.43.old.29567 2009-09-18 13:32:41.000000000 +0300 +++ /tmp/wklog.43.new.29567 2009-09-18 13:32:41.000000000 +0300 @@ -1 +1,5 @@ +Options stored in a list as pairs key, value which are LEX_STRING. + +Handler have access to the options during creation and after opening frm. + -=-=(Sanja - Fri, 18 Sep 2009, 13:21)=-=- High-Level Specification modified. --- /tmp/wklog.43.old.29107 2009-09-18 13:21:19.000000000 +0300 +++ /tmp/wklog.43.new.29107 2009-09-18 13:21:19.000000000 +0300 @@ -6,7 +6,8 @@ table_option1=tval1, table_option2=tval2; Exclusion should be made for old table and key options where -'=' was not obligatory. +'=' was not obligatory. Behaviour and way of storage for existing options will +be left as is. Old key options: KEY_BLOCK_SIZE <num> -> KEY_BLOCK_SIZE=num @@ -42,6 +43,6 @@ For fields options can go with field attributes (NOT NULL, UNIQUE and so on) can be separated from them by '=' sign. - - +Incorrect option during creation/altering table lead only to worning. During +opening unrecognised options should be ignored. -=-=(Guest - Tue, 11 Aug 2009, 19:57)=-=- High-Level Specification modified. --- /tmp/wklog.43.old.31856 2009-08-11 19:57:38.000000000 +0300 +++ /tmp/wklog.43.new.31856 2009-08-11 19:57:38.000000000 +0300 @@ -5,8 +5,43 @@ key key1(field) key_opt1=kval1 key_opt2=kval2) table_option1=tval1, table_option2=tval2; -Exclusion should be made for old table and key (KEY_BLOCK_SIZE) options where +Exclusion should be made for old table and key options where '=' was not obligatory. +Old key options: +KEY_BLOCK_SIZE <num> -> KEY_BLOCK_SIZE=num +WITH PARSER <name> -> PARSER=name + +Old table options: +ENGINE name -> ENGINE=name +TYPE name -> TYPE=name +MAX_ROWS num -> MAX_ROWS=num +MIX_ROWS num -> MIX_ROWS=num +AVG_ROW_LENGTH num -> AVG_ROW_LENGTH=num +PASSWORD string -> PASSWORD=string +COMMENT string -> COMMENT=string +AUTO_INCREMENT num -> AUTO_INCREMENT=num +PACK_KEYS num/default -> PACK_KEYS=num/default +CHECKSUM num -> CHECKSUM=num +TABLE_CHECKSUM num -> TABLE_CHECKSUM=num +PAGE_CHECKSUM num -> PAGE_CHECKSUM=num +DELAY_KEY_WRITE num -> DELAY_KEY_WRITE=num +ROW_FORMAT name -> ROW_FORMAT=name +INSERT_METHOD name -> INSERT_METHOD=name +KEY_BLOCK_SIZE num -> KEY_BLOCK_SIZE=num +TRANSACTIONAL num -> TRANSACTIONAL=num + +Table options which will be left hardcoded +UNION +default charset +default collation +DATA DIRECTORY +TABLESPACE +STORAGE + For fields options can go with field attributes (NOT NULL, UNIQUE and so on) can be separated from them by '=' sign. + + + + -=-=(Guest - Tue, 11 Aug 2009, 19:36)=-=- High-Level Specification modified. --- /tmp/wklog.43.old.30883 2009-08-11 19:36:45.000000000 +0300 +++ /tmp/wklog.43.new.30883 2009-08-11 19:36:45.000000000 +0300 @@ -1 +1,12 @@ +Table definition ca looks like following +CREATE TABLE table + (field int ... field_opt1=fval1 field_opt2=fval2, + key key1(field) key_opt1=kval1 key_opt2=kval2) + table_option1=tval1, table_option2=tval2; + +Exclusion should be made for old table and key (KEY_BLOCK_SIZE) options where +'=' was not obligatory. + +For fields options can go with field attributes (NOT NULL, UNIQUE and so on) can +be separated from them by '=' sign. DESCRIPTION: Add ability to create table with additional option which can be passed to engine. Also make current options such as TRANSACTIONAL working via this mechanism. HIGH-LEVEL SPECIFICATION: Table definition ca looks like following CREATE TABLE table (field int ... field_opt1=fval1 field_opt2=fval2, key key1(field) key_opt1=kval1 key_opt2=kval2) table_option1=tval1, table_option2=tval2; Exclusion should be made for old table and key options where '=' was not obligatory. Behaviour and way of storage for existing options will be left as is. Old key options: KEY_BLOCK_SIZE <num> -> KEY_BLOCK_SIZE=num WITH PARSER <name> -> PARSER=name Old table options: ENGINE name -> ENGINE=name TYPE name -> TYPE=name MAX_ROWS num -> MAX_ROWS=num MIX_ROWS num -> MIX_ROWS=num AVG_ROW_LENGTH num -> AVG_ROW_LENGTH=num PASSWORD string -> PASSWORD=string COMMENT string -> COMMENT=string AUTO_INCREMENT num -> AUTO_INCREMENT=num PACK_KEYS num/default -> PACK_KEYS=num/default CHECKSUM num -> CHECKSUM=num TABLE_CHECKSUM num -> TABLE_CHECKSUM=num PAGE_CHECKSUM num -> PAGE_CHECKSUM=num DELAY_KEY_WRITE num -> DELAY_KEY_WRITE=num ROW_FORMAT name -> ROW_FORMAT=name INSERT_METHOD name -> INSERT_METHOD=name KEY_BLOCK_SIZE num -> KEY_BLOCK_SIZE=num TRANSACTIONAL num -> TRANSACTIONAL=num Table options which will be left hardcoded UNION default charset default collation DATA DIRECTORY TABLESPACE STORAGE For fields options can go with field attributes (NOT NULL, UNIQUE and so on) can be separated from them by '=' sign. Incorrect option during creation/altering table lead only to worning. During opening unrecognised options should be ignored. LOW-LEVEL DESIGN: Options stored in a list as pairs key, value which are LEX_STRING. Handler have access to the options during creation and after opening frm. It should has C interface. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Monty): options for CREATE TABLE (43)
by worklog-noreply＠askmonty.org 12 Jan '10

12 Jan '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: options for CREATE TABLE CREATION DATE..: Tue, 11 Aug 2009, 17:02 SUPERVISOR.....: Bothorsen IMPLEMENTOR....: Sanja COPIES TO......: Monty CATEGORY.......: Server-BackLog TASK ID........: 43 (http://askmonty.org/worklog/?tid=43) VERSION........: Server-5.2 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 32 (hours remain) ORIG. ESTIMATE.: 32 PROGRESS NOTES: -=-=(Monty - Tue, 12 Jan 2010, 17:04)=-=- Version updated. --- /tmp/wklog.43.old.21802 2010-01-12 17:04:31.000000000 +0200 +++ /tmp/wklog.43.new.21802 2010-01-12 17:04:31.000000000 +0200 @@ -1 +1 @@ -Server-5.1 +Server-5.2 -=-=(Guest - Fri, 18 Sep 2009, 14:24)=-=- Low Level Design modified. --- /tmp/wklog.43.old.31654 2009-09-18 14:24:41.000000000 +0300 +++ /tmp/wklog.43.new.31654 2009-09-18 14:24:41.000000000 +0300 @@ -2,4 +2,6 @@ Handler have access to the options during creation and after opening frm. +It should has C interface. + -=-=(Sanja - Fri, 18 Sep 2009, 13:32)=-=- Low Level Design modified. --- /tmp/wklog.43.old.29567 2009-09-18 13:32:41.000000000 +0300 +++ /tmp/wklog.43.new.29567 2009-09-18 13:32:41.000000000 +0300 @@ -1 +1,5 @@ +Options stored in a list as pairs key, value which are LEX_STRING. + +Handler have access to the options during creation and after opening frm. + -=-=(Sanja - Fri, 18 Sep 2009, 13:21)=-=- High-Level Specification modified. --- /tmp/wklog.43.old.29107 2009-09-18 13:21:19.000000000 +0300 +++ /tmp/wklog.43.new.29107 2009-09-18 13:21:19.000000000 +0300 @@ -6,7 +6,8 @@ table_option1=tval1, table_option2=tval2; Exclusion should be made for old table and key options where -'=' was not obligatory. +'=' was not obligatory. Behaviour and way of storage for existing options will +be left as is. Old key options: KEY_BLOCK_SIZE <num> -> KEY_BLOCK_SIZE=num @@ -42,6 +43,6 @@ For fields options can go with field attributes (NOT NULL, UNIQUE and so on) can be separated from them by '=' sign. - - +Incorrect option during creation/altering table lead only to worning. During +opening unrecognised options should be ignored. -=-=(Guest - Tue, 11 Aug 2009, 19:57)=-=- High-Level Specification modified. --- /tmp/wklog.43.old.31856 2009-08-11 19:57:38.000000000 +0300 +++ /tmp/wklog.43.new.31856 2009-08-11 19:57:38.000000000 +0300 @@ -5,8 +5,43 @@ key key1(field) key_opt1=kval1 key_opt2=kval2) table_option1=tval1, table_option2=tval2; -Exclusion should be made for old table and key (KEY_BLOCK_SIZE) options where +Exclusion should be made for old table and key options where '=' was not obligatory. +Old key options: +KEY_BLOCK_SIZE <num> -> KEY_BLOCK_SIZE=num +WITH PARSER <name> -> PARSER=name + +Old table options: +ENGINE name -> ENGINE=name +TYPE name -> TYPE=name +MAX_ROWS num -> MAX_ROWS=num +MIX_ROWS num -> MIX_ROWS=num +AVG_ROW_LENGTH num -> AVG_ROW_LENGTH=num +PASSWORD string -> PASSWORD=string +COMMENT string -> COMMENT=string +AUTO_INCREMENT num -> AUTO_INCREMENT=num +PACK_KEYS num/default -> PACK_KEYS=num/default +CHECKSUM num -> CHECKSUM=num +TABLE_CHECKSUM num -> TABLE_CHECKSUM=num +PAGE_CHECKSUM num -> PAGE_CHECKSUM=num +DELAY_KEY_WRITE num -> DELAY_KEY_WRITE=num +ROW_FORMAT name -> ROW_FORMAT=name +INSERT_METHOD name -> INSERT_METHOD=name +KEY_BLOCK_SIZE num -> KEY_BLOCK_SIZE=num +TRANSACTIONAL num -> TRANSACTIONAL=num + +Table options which will be left hardcoded +UNION +default charset +default collation +DATA DIRECTORY +TABLESPACE +STORAGE + For fields options can go with field attributes (NOT NULL, UNIQUE and so on) can be separated from them by '=' sign. + + + + -=-=(Guest - Tue, 11 Aug 2009, 19:36)=-=- High-Level Specification modified. --- /tmp/wklog.43.old.30883 2009-08-11 19:36:45.000000000 +0300 +++ /tmp/wklog.43.new.30883 2009-08-11 19:36:45.000000000 +0300 @@ -1 +1,12 @@ +Table definition ca looks like following +CREATE TABLE table + (field int ... field_opt1=fval1 field_opt2=fval2, + key key1(field) key_opt1=kval1 key_opt2=kval2) + table_option1=tval1, table_option2=tval2; + +Exclusion should be made for old table and key (KEY_BLOCK_SIZE) options where +'=' was not obligatory. + +For fields options can go with field attributes (NOT NULL, UNIQUE and so on) can +be separated from them by '=' sign. DESCRIPTION: Add ability to create table with additional option which can be passed to engine. Also make current options such as TRANSACTIONAL working via this mechanism. HIGH-LEVEL SPECIFICATION: Table definition ca looks like following CREATE TABLE table (field int ... field_opt1=fval1 field_opt2=fval2, key key1(field) key_opt1=kval1 key_opt2=kval2) table_option1=tval1, table_option2=tval2; Exclusion should be made for old table and key options where '=' was not obligatory. Behaviour and way of storage for existing options will be left as is. Old key options: KEY_BLOCK_SIZE <num> -> KEY_BLOCK_SIZE=num WITH PARSER <name> -> PARSER=name Old table options: ENGINE name -> ENGINE=name TYPE name -> TYPE=name MAX_ROWS num -> MAX_ROWS=num MIX_ROWS num -> MIX_ROWS=num AVG_ROW_LENGTH num -> AVG_ROW_LENGTH=num PASSWORD string -> PASSWORD=string COMMENT string -> COMMENT=string AUTO_INCREMENT num -> AUTO_INCREMENT=num PACK_KEYS num/default -> PACK_KEYS=num/default CHECKSUM num -> CHECKSUM=num TABLE_CHECKSUM num -> TABLE_CHECKSUM=num PAGE_CHECKSUM num -> PAGE_CHECKSUM=num DELAY_KEY_WRITE num -> DELAY_KEY_WRITE=num ROW_FORMAT name -> ROW_FORMAT=name INSERT_METHOD name -> INSERT_METHOD=name KEY_BLOCK_SIZE num -> KEY_BLOCK_SIZE=num TRANSACTIONAL num -> TRANSACTIONAL=num Table options which will be left hardcoded UNION default charset default collation DATA DIRECTORY TABLESPACE STORAGE For fields options can go with field attributes (NOT NULL, UNIQUE and so on) can be separated from them by '=' sign. Incorrect option during creation/altering table lead only to worning. During opening unrecognised options should be ignored. LOW-LEVEL DESIGN: Options stored in a list as pairs key, value which are LEX_STRING. Handler have access to the options during creation and after opening frm. It should has C interface. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Guest): Enable the use of libstdc++ in MariaDB (63)
by worklog-noreply＠askmonty.org 12 Jan '10

12 Jan '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Enable the use of libstdc++ in MariaDB CREATION DATE..: Wed, 11 Nov 2009, 13:19 SUPERVISOR.....: Monty IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Server-BackLog TASK ID........: 63 (http://askmonty.org/worklog/?tid=63) VERSION........: Server-5.2 STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Guest - Tue, 12 Jan 2010, 16:26)=-=- Version updated. --- /tmp/wklog.63.old.19522 2010-01-12 16:26:23.000000000 +0200 +++ /tmp/wklog.63.new.19522 2010-01-12 16:26:23.000000000 +0200 @@ -1 +1 @@ -Connector/.NET-5.2 +Server-5.2 -=-=(Guest - Tue, 12 Jan 2010, 16:26)=-=- Category updated. --- /tmp/wklog.63.old.19506 2010-01-12 16:26:15.000000000 +0200 +++ /tmp/wklog.63.new.19506 2010-01-12 16:26:15.000000000 +0200 @@ -1 +1 @@ -Server-RawIdeaBin +Server-BackLog DESCRIPTION: Enable the use of libstdc++ in MariaDB. As time goes on, more and more plugins and external code/library will need linking to libstdc++ for stuff it uses. I have already seen this happen several times, with extra work needed to integrate things properly. It would be nice to have a general solution for this so that it is not necessary to spend time on individual solutions in each case. It also needs to be considered what the impact of this will be for the server in terms of binary compatibility, performance etc. I think it should be mostly ok, except that it might introduce a problem for bintar packages with an external dependency on libstdc++. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0