- developers - lists.mariadb.org

[Maria-developers] [Branch ~maria-captains/maria/5.1] Rev 2720: Fix `make dist` after solaris build file fixes.
by noreply＠launchpad.net 29 Aug '09

29 Aug '09

------------------------------------------------------------ revno: 2720 committer: knielsen(a)knielsen-hq.org branch nick: 5.1 timestamp: Sat 2009-08-29 21:04:46 +0200 message: Fix `make dist` after solaris build file fixes. Solaris build scripts under BUILD/ were changed, but Makefile.am was not updated to reflect the new set of file names. modified: BUILD/Makefile.am -- lp:maria https://code.launchpad.net/~maria-captains/maria/5.1 Your team Maria developers is subscribed to branch lp:maria. To unsubscribe from this branch go to https://code.launchpad.net/~maria-captains/maria/5.1/+edit-subscription.

1 0

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (knielsen:2720)
by knielsen＠knielsen-hq.org 29 Aug '09

29 Aug '09

#At lp:maria 2720 knielsen(a)knielsen-hq.org 2009-08-29 Fix `make dist` after solaris build file fixes. Solaris build scripts under BUILD/ were changed, but Makefile.am was not updated to reflect the new set of file names. modified: BUILD/Makefile.am === modified file 'BUILD/Makefile.am' --- a/BUILD/Makefile.am 2008-05-13 13:56:07 +0000 +++ b/BUILD/Makefile.am 2009-08-29 19:04:46 +0000 @@ -71,12 +71,16 @@ EXTRA_DIST = FINISH.sh \ compile-ppc-max \ compile-solaris-amd64 \ compile-solaris-amd64-debug \ + compile-solaris-amd64-debug-forte \ compile-solaris-amd64-forte \ - compile-solaris-amd64-forte-debug \ compile-solaris-sparc \ compile-solaris-sparc-debug \ compile-solaris-sparc-forte \ - compile-solaris-sparc-purify + compile-solaris-sparc-purify \ + compile-solaris-x86-32 \ + compile-solaris-x86-32-debug \ + compile-solaris-x86-32-debug-forte \ + compile-solaris-x86-forte-32 # Don't update the files from bitkeeper %::SCCS/s.%

1 0

[Maria-developers] Updated (by Guest): Backport key cache to 5.0 (52)
by worklog-noreply＠askmonty.org 28 Aug '09

28 Aug '09

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Backport key cache to 5.0 CREATION DATE..: Thu, 27 Aug 2009, 09:54 SUPERVISOR.....: Monty IMPLEMENTOR....: Monty COPIES TO......: Sanja CATEGORY.......: Server-Sprint TASK ID........: 52 (http://askmonty.org/worklog/?tid=52) VERSION........: Server-5.0 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 6 ESTIMATE.......: 2 (hours remain) ORIG. ESTIMATE.: 8 PROGRESS NOTES: -=-=(Guest - Fri, 28 Aug 2009, 10:55)=-=- Low Level Design modified. --- /tmp/wklog.52.old.16945 2009-08-28 10:55:40.000000000 +0300 +++ /tmp/wklog.52.new.16945 2009-08-28 10:55:40.000000000 +0300 @@ -1 +1,22 @@ -Just copy files (mf_keycache.c & key=cache.h) and fix what should be fixed. +Here is instructions of how to repeat this if needed: + +- Copy the following files from the latest MySQL 5.1 to MySQL 5.0: + +include/keycache.h +include/my_bit.h +mysys/mf_keycache.c +mysys/mf_keycaches.c +mysys/my_bit.c + +Apply the following change in mysys/mf_keycaches.c +81c81 +< static uchar *safe_hash_entry_get(SAFE_HASH_ENTRY *entry, size_t *length, +--- +> static uchar *safe_hash_entry_get(SAFE_HASH_ENTRY *entry, uint *length, + +recompile everything. + +Simple test to verify that this is working: +cd mysql-test +mysql-test-run --valgrind key_cache +(the --valgrind argument can be left out if you don't have valgrind installed) -=-=(Guest - Thu, 27 Aug 2009, 21:52)=-=- - Planning code (Email dicussions & discussions with Sanja): 2 hours. - Backport of code (2 hours). Notice that mysys/mf_keycaches.c::safe_hash_entry_get() is different from the code in MySQL 5.1 - Testing code with mysql-test-run (especially running the key_cache.test with valgrind) - Tested the server with sql-bench Worked 6 hours and estimate 2 hours remain (original estimate unchanged). -=-=(Sanja - Thu, 27 Aug 2009, 09:56)=-=- High-Level Specification modified. --- /tmp/wklog.52.old.20094 2009-08-27 09:56:05.000000000 +0300 +++ /tmp/wklog.52.new.20094 2009-08-27 09:56:05.000000000 +0300 @@ -1 +1 @@ - +Backport keycache to 5.0 -=-=(Sanja - Thu, 27 Aug 2009, 09:55)=-=- Low Level Design modified. --- /tmp/wklog.52.old.20035 2009-08-27 09:55:25.000000000 +0300 +++ /tmp/wklog.52.new.20035 2009-08-27 09:55:25.000000000 +0300 @@ -1 +1 @@ - +Just copy files (mf_keycache.c & key=cache.h) and fix what should be fixed. DESCRIPTION: Backport keycache to bring bugfixes made in 5.1. HIGH-LEVEL SPECIFICATION: Backport keycache to 5.0 LOW-LEVEL DESIGN: Here is instructions of how to repeat this if needed: - Copy the following files from the latest MySQL 5.1 to MySQL 5.0: include/keycache.h include/my_bit.h mysys/mf_keycache.c mysys/mf_keycaches.c mysys/my_bit.c Apply the following change in mysys/mf_keycaches.c 81c81 < static uchar *safe_hash_entry_get(SAFE_HASH_ENTRY *entry, size_t *length, --- > static uchar *safe_hash_entry_get(SAFE_HASH_ENTRY *entry, uint *length, recompile everything. Simple test to verify that this is working: cd mysql-test mysql-test-run --valgrind key_cache (the --valgrind argument can be left out if you don't have valgrind installed) ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Guest): Backport key cache to 5.0 (52)
by worklog-noreply＠askmonty.org 28 Aug '09

28 Aug '09

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Backport key cache to 5.0 CREATION DATE..: Thu, 27 Aug 2009, 09:54 SUPERVISOR.....: Monty IMPLEMENTOR....: Monty COPIES TO......: Sanja CATEGORY.......: Server-Sprint TASK ID........: 52 (http://askmonty.org/worklog/?tid=52) VERSION........: Server-5.0 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 6 ESTIMATE.......: 2 (hours remain) ORIG. ESTIMATE.: 8 PROGRESS NOTES: -=-=(Guest - Fri, 28 Aug 2009, 10:55)=-=- Low Level Design modified. --- /tmp/wklog.52.old.16945 2009-08-28 10:55:40.000000000 +0300 +++ /tmp/wklog.52.new.16945 2009-08-28 10:55:40.000000000 +0300 @@ -1 +1,22 @@ -Just copy files (mf_keycache.c & key=cache.h) and fix what should be fixed. +Here is instructions of how to repeat this if needed: + +- Copy the following files from the latest MySQL 5.1 to MySQL 5.0: + +include/keycache.h +include/my_bit.h +mysys/mf_keycache.c +mysys/mf_keycaches.c +mysys/my_bit.c + +Apply the following change in mysys/mf_keycaches.c +81c81 +< static uchar *safe_hash_entry_get(SAFE_HASH_ENTRY *entry, size_t *length, +--- +> static uchar *safe_hash_entry_get(SAFE_HASH_ENTRY *entry, uint *length, + +recompile everything. + +Simple test to verify that this is working: +cd mysql-test +mysql-test-run --valgrind key_cache +(the --valgrind argument can be left out if you don't have valgrind installed) -=-=(Guest - Thu, 27 Aug 2009, 21:52)=-=- - Planning code (Email dicussions & discussions with Sanja): 2 hours. - Backport of code (2 hours). Notice that mysys/mf_keycaches.c::safe_hash_entry_get() is different from the code in MySQL 5.1 - Testing code with mysql-test-run (especially running the key_cache.test with valgrind) - Tested the server with sql-bench Worked 6 hours and estimate 2 hours remain (original estimate unchanged). -=-=(Sanja - Thu, 27 Aug 2009, 09:56)=-=- High-Level Specification modified. --- /tmp/wklog.52.old.20094 2009-08-27 09:56:05.000000000 +0300 +++ /tmp/wklog.52.new.20094 2009-08-27 09:56:05.000000000 +0300 @@ -1 +1 @@ - +Backport keycache to 5.0 -=-=(Sanja - Thu, 27 Aug 2009, 09:55)=-=- Low Level Design modified. --- /tmp/wklog.52.old.20035 2009-08-27 09:55:25.000000000 +0300 +++ /tmp/wklog.52.new.20035 2009-08-27 09:55:25.000000000 +0300 @@ -1 +1 @@ - +Just copy files (mf_keycache.c & key=cache.h) and fix what should be fixed. DESCRIPTION: Backport keycache to bring bugfixes made in 5.1. HIGH-LEVEL SPECIFICATION: Backport keycache to 5.0 LOW-LEVEL DESIGN: Here is instructions of how to repeat this if needed: - Copy the following files from the latest MySQL 5.1 to MySQL 5.0: include/keycache.h include/my_bit.h mysys/mf_keycache.c mysys/mf_keycaches.c mysys/my_bit.c Apply the following change in mysys/mf_keycaches.c 81c81 < static uchar *safe_hash_entry_get(SAFE_HASH_ENTRY *entry, size_t *length, --- > static uchar *safe_hash_entry_get(SAFE_HASH_ENTRY *entry, uint *length, recompile everything. Simple test to verify that this is working: cd mysql-test mysql-test-run --valgrind key_cache (the --valgrind argument can be left out if you don't have valgrind installed) ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Simple compiler warning fixes for yassl-1.9.6
by Kristian Nielsen 28 Aug '09

28 Aug '09

The attached patch contains a few simple fixes for compiler warnings that I applied to our copy of YaSSL in the MariaDB source tree (http://askmonty.org/wiki/index.php/MariaDB) I am sending you these in case you want to include them in upstream YaSSL. I did not see instructions for submitting patches at http://www.yassl.com/, so please let me know if there is some other email address or procedure I should follow. Signed-off-by: Kristian Nielsen <knielsen(a)knielsen-hq.org>

2 1

Re: [Maria-developers] QC patch for review
by Michael Widenius 27 Aug '09

27 Aug '09

Hi! This is regarding a bug fix for query cache: - Bug#39249 Maria: query cache returns out-of-date results - Bug #41098 Query Cache returns wrong result with concurrent insert >>>>> "s" == Oleksandr Byelkin <Oleksandr> writes: s> Hi! s> Here it is QC patch for inserts (as we discussed) and also there is DBUG s> problem workaround in the test. s> +++ sql/handler.h 2009-06-11 11:25:58 +0000 s> @@ -247,6 +247,11 @@ s> #define HA_CACHE_TBL_NOCACHE 1 s> #define HA_CACHE_TBL_ASKTRANSACT 2 s> #define HA_CACHE_TBL_TRANSACT 4 s> +/** s> + Non transactional table but insert results visible for other threads s> + only on unlock s> +*/ s> +#define HA_CACHE_TBL_NTRNS_INS2LOCK 8 The above is true for all non transactional engines. To clarify, there are two types of non transactional engines when it comes to insert: 1) The table is locked at start of insert statement and can't be accessed by other threads until unlock. 2) The table is locked at start of insert statement, but other threads can read only the original rows until unlock, after which all rows are visible (myisam/maria with concurrent insert on) In both cases, the new rows are visible after unlock, in which case we shouldn't need the above flag. As we discussed in Majorca, I think the logic for this patch is wrong. The current patch works (as I can see) on the following idea: - mark non-transacitonal tables that are part of an insert - later in sql_base.cc::close_thread_table() call query_cache_invalidate4() to invalidate the table if there was a possible 'concurrent insert'. The reason the above doesn't work is that another thread may have done a lock on the table before the unlock happens in the concurrent-inserted thread and then call 'query_cache_store_query()' after the invalidate is done. In this case the other thread has and old view of the table and anything it stores in the cache will be 'old'. The new code only makes wrong things 'less unlikely' to happen; It doesn't remove the original bug. The reason for the problem with concurrent insert problem in the query cache are two: - The query cache should get invalidated at once when table changes state (ie, when the new table content is visible for another thread). This should be done under a mutex to ensure that there is no race condition for this table. - When validating of select with a table can be put into the query cache we should ensure that it's content is of the latest version. With the two above axioms in place, the query cache will be safe for concurrent inserts. There are some other benefits of changing to use this model: - We can safely use locked tables with the query cache. - We can put the query into the table cache even under lock tables, as long as all the tables are marked as 'cacheable'. - We can use the data from the query cache, as long as all locked tables are marked as 'cacheable'. - We could (later) merge the code of transactional and not transactional tables as they act the same. - We can provide caching for versioned tables (like Maria) more easily. Here is a last comment about the problem with this patch: s> === modified file 'sql/sql_base.cc' s> --- sql/sql_base.cc 2009-05-19 09:28:05 +0000 s> +++ sql/sql_base.cc 2009-06-11 11:25:58 +0000 s> @@ -1373,6 +1373,15 @@ bool close_thread_table(THD *thd, TABLE s> table->s->table_name.str, (long) table)); s> *table_ptr=table->next; s> + s> + /* Invalidate if it has mark about changing in insert s> + (not all tables has such marks */ Fix comment style and add missing brace + if (table->changed_in_insert) + { + table->changed_in_insert= FALSE; + query_cache_invalidate4(thd, table, FALSE, FALSE); + } I have some problems with the above code: - The real problem is that if someone accesses the table between the invalidate() and unlock, they see the old version of the table and not the real rows. The above change adds an extra invalidate in case of concurrent inserts. However, another thread may already have done the following before this call: - Do a query just after invalidate, in which case the table is seen as 'old' - Do another query after unlock, in which case table is seen as 'new'. - Another problem is that another thread that did a lock table before the invalidate, it will still see the old data from the table. If this thread does a query of the table and it goes into the query cache, then the query cache will be filled with wrong data. (Note that even if we don't put tables in the cache under lock tables, the problem is still the same for tables without lock tables. It's just easier to simulate the problem if we test things with lock tables) - A future problem is that for nontransactional tables that could do concurrent deletes/updates the above wouldn't work. So this code only makes the problem less likely, it doesn't fix it. ---------------------- Add to MyISAM a function: virtual my_bool register_query_cache_table(...) { /* check if this thread is seeing all rows in the file */ if (file->s->state.data_file_length != file->state->data_file_length) { /* Some thread (or even this thread) has inserted more rows into the table since it was locked; Don't use the query cache. */ return 0; } return 1; } We also need to invalidate tables when they change. This should be done by adding a callback in mi_locking.c:mi_update_status() to invalidate the query cache for any entries for this table. (This only needs to be done inside 'if (info->state == &info->save_state)' In Maira we need to to the same changes in _ma_update_status(). In Maria, we can also decide to handle transactional/versioned tables with the query cache: virtual my_bool register_query_cache_table(....) { .... if (share->now_transactional && share->have_versioning) return (file->trn != file->s->state.last_change_trn); ... } For the above to work, we need to set last_change_trn in _ma_trnman_end_trans_hook() when we store the state for a changed table (at same place where we update 'records' and 'checksum') Regards, Monty PS: Don't forget to also fix the MERGE tables and check that the Maria bug is truly fixed.

1 0

[Maria-developers] Progress (by Guest): Backport key cache to 5.0 (52)
by worklog-noreply＠askmonty.org 27 Aug '09

27 Aug '09

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Backport key cache to 5.0 CREATION DATE..: Thu, 27 Aug 2009, 09:54 SUPERVISOR.....: Monty IMPLEMENTOR....: Monty COPIES TO......: Sanja CATEGORY.......: Server-Sprint TASK ID........: 52 (http://askmonty.org/worklog/?tid=52) VERSION........: Server-5.0 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 6 ESTIMATE.......: 2 (hours remain) ORIG. ESTIMATE.: 8 PROGRESS NOTES: -=-=(Guest - Thu, 27 Aug 2009, 21:52)=-=- - Planning code (Email dicussions & discussions with Sanja): 2 hours. - Backport of code (2 hours). Notice that mysys/mf_keycaches.c::safe_hash_entry_get() is different from the code in MySQL 5.1 - Testing code with mysql-test-run (especially running the key_cache.test with valgrind) - Tested the server with sql-bench Worked 6 hours and estimate 2 hours remain (original estimate unchanged). -=-=(Sanja - Thu, 27 Aug 2009, 09:56)=-=- High-Level Specification modified. --- /tmp/wklog.52.old.20094 2009-08-27 09:56:05.000000000 +0300 +++ /tmp/wklog.52.new.20094 2009-08-27 09:56:05.000000000 +0300 @@ -1 +1 @@ - +Backport keycache to 5.0 -=-=(Sanja - Thu, 27 Aug 2009, 09:55)=-=- Low Level Design modified. --- /tmp/wklog.52.old.20035 2009-08-27 09:55:25.000000000 +0300 +++ /tmp/wklog.52.new.20035 2009-08-27 09:55:25.000000000 +0300 @@ -1 +1 @@ - +Just copy files (mf_keycache.c & key=cache.h) and fix what should be fixed. DESCRIPTION: Backport keycache to bring bugfixes made in 5.1. HIGH-LEVEL SPECIFICATION: Backport keycache to 5.0 LOW-LEVEL DESIGN: Just copy files (mf_keycache.c & key=cache.h) and fix what should be fixed. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Progress (by Guest): Backport key cache to 5.0 (52)
by worklog-noreply＠askmonty.org 27 Aug '09

27 Aug '09

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Backport key cache to 5.0 CREATION DATE..: Thu, 27 Aug 2009, 09:54 SUPERVISOR.....: Monty IMPLEMENTOR....: Monty COPIES TO......: Sanja CATEGORY.......: Server-Sprint TASK ID........: 52 (http://askmonty.org/worklog/?tid=52) VERSION........: Server-5.0 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 6 ESTIMATE.......: 2 (hours remain) ORIG. ESTIMATE.: 8 PROGRESS NOTES: -=-=(Guest - Thu, 27 Aug 2009, 21:52)=-=- - Planning code (Email dicussions & discussions with Sanja): 2 hours. - Backport of code (2 hours). Notice that mysys/mf_keycaches.c::safe_hash_entry_get() is different from the code in MySQL 5.1 - Testing code with mysql-test-run (especially running the key_cache.test with valgrind) - Tested the server with sql-bench Worked 6 hours and estimate 2 hours remain (original estimate unchanged). -=-=(Sanja - Thu, 27 Aug 2009, 09:56)=-=- High-Level Specification modified. --- /tmp/wklog.52.old.20094 2009-08-27 09:56:05.000000000 +0300 +++ /tmp/wklog.52.new.20094 2009-08-27 09:56:05.000000000 +0300 @@ -1 +1 @@ - +Backport keycache to 5.0 -=-=(Sanja - Thu, 27 Aug 2009, 09:55)=-=- Low Level Design modified. --- /tmp/wklog.52.old.20035 2009-08-27 09:55:25.000000000 +0300 +++ /tmp/wklog.52.new.20035 2009-08-27 09:55:25.000000000 +0300 @@ -1 +1 @@ - +Just copy files (mf_keycache.c & key=cache.h) and fix what should be fixed. DESCRIPTION: Backport keycache to bring bugfixes made in 5.1. HIGH-LEVEL SPECIFICATION: Backport keycache to 5.0 LOW-LEVEL DESIGN: Just copy files (mf_keycache.c & key=cache.h) and fix what should be fixed. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] MariaDB 5.1 release coordinator
by Henrik Ingo 27 Aug '09

27 Aug '09

Hi all Just finished a writeup of our meeting last week, on http://openlife.cc/blogs/2009/august/mariadb-release-plan-and-other-news-mp… (Btw, I've also uploaded slides to the MP internal wiki and updated the ProjectHandbook based on our discussion.) Then comes the final thing on my todo from discussions in Mallorca: Sergey P suggested we should appoint a release coordinator to chase all the tasks needed before releasing beta/RC/GA of MariaDB: http://askmonty.org/wiki/index.php/MariaDB_versus_MySQL#Things_to_add_to_Ma… http://askmonty.org/wiki/index.php/MariaDB51_release_plan he also volunteered, and doesn't have customer projects coming up, and Monty approved, so Sergey: you're it! This means - You should keep especially Sergey updated to whatever you contribute to release progress. - You should be able to ask Sergey any questions on status of tasks, should we possibly drop a feature etc... - Public channels (this mailing list, IRC) still preferred - Continue as usual, this doesn't mean that Sergey will go around asking people to do work. Sergey, please continue the list if you had something in mind that's not there. henrik -- email: henrik.ingo(a)avoinelama.fi tel: +358-40-5697354 www: www.avoinelama.fi/~hingo book: www.openlife.cc

1 0

Re: [Maria-developers] Fixing compiler warnings
by Kristian Nielsen 27 Aug '09

27 Aug '09

Paul McCullagh <paul.mccullagh(a)primebase.org> writes: > Hi Kristian, > > Thanks for the patch! > > I will definitely include these changes in the PBXT trunk. > > I am happy to accept any changes that allow the code to compile > without warnings (this is also a requirement for compiling with > Drizzle). The other warning I saw in PBXT was this: cache_xt.h:48: warning: ‘typedef’ was ignored in this declaration cache_xt.h:49: warning: ‘typedef’ was ignored in this declaration The code in question is this: typedef enum XTPageLockType { XT_LOCK_READ, XT_LOCK_WRITE, XT_XLOCK_LEAF }; typedef enum XTPageUnlockType { XT_UNLOCK_NONE, XT_UNLOCK_READ, XT_UNLOCK_WRITE, XT_UNLOCK_R_UPDATE, XT_UNLOCK_W_UPDATE }; When I first looked I thought this was due to mixing C and C++, and thought that the compiler was being stupid. But now I see that it is actually the code that is a bit silly, as it says typedef without giving a name of the type being defined! (So now I'm surprised that we don't get a syntax error, but I guess this is just one of those dark corners of C++ that are best left alone ...) Anyway, mentioning it in case you want to fix. - Kristian.

2 1