August 2010 - developers - lists.mariadb.org

[Maria-developers] really annoying bug in mysql 5.0/5.1 optimiser, opened for more than 2 years, and still present in MariaDB
by Jocelyn Fournier 15 Sep '10

15 Sep '10

Hi, There's a really annoying bug in the mysql optimiser existing for a really long time now, and which has a direct impact on performances : http://bugs.mysql.com/bug.php?id=36817 Since MySQL definitly doesn't seem to want to fix it, I hope MariaDB could do something to help the poor developpers who need to heavily hack there apps with USE INDEX ;) Note I've tested with mariadb 5.2.1-beta, and the bug is still present. Thanks a lot ! Jocelyn Fournier

2 3

[Maria-developers] WL#135 New (by Dreas): Expose marked as crashed in table status
by worklog-noreply＠askmonty.org 31 Aug '10

31 Aug '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Expose marked as crashed in table status CREATION DATE..: Tue, 31 Aug 2010, 14:59 SUPERVISOR.....: IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Server-RawIdeaBin TASK ID........: 135 (http://askmonty.org/worklog/?tid=135) VERSION........: WorkLog-4.0 STATUS.........: Un-Assigned PRIORITY.......: 10 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: DESCRIPTION: We regularly run into tables that are marked as crashed (shown in syslog). If MySQL is marking the table as crashed, why is there no way to get a list of tables marked as such? That would make it much faster to check/repair such tables automatically without the need to run a check on ALL tables. >From IRC: (04:04:58 PM) montywi: myisamchk -d */*.MYI ; search for 'marked' (04:05:09 PM) montywi: -d (--describe) is safe to do (04:05:59 PM) montywi: best thing would probably would be to add an option ' IF_CRASHED' to CHECK TABLE (04:06:48 PM) montywi: in that case one could trivially use mysqlcheck to find out crashed tables and then automaticly, if so desired, run repair on them ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v4.0.0)

1 0

[Maria-developers] WL#133 New (by Knielsen): Event applier API
by worklog-noreply＠askmonty.org 27 Aug '10

27 Aug '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Event applier API CREATION DATE..: Fri, 27 Aug 2010, 13:17 SUPERVISOR.....: Sergei IMPLEMENTOR....: Knielsen COPIES TO......: CATEGORY.......: Server-Sprint TASK ID........: 133 (http://askmonty.org/worklog/?tid=133) VERSION........: Server-9.x STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: DESCRIPTION: A part of the replication project, MWL#107. The event applier API does the opposite of the stacked event generator API, MWL#120. It allows a plugin to apply (=execute) abstract events on a slave. Just like generated events from MWL#120 are abstract without any fixed materialised format, applied events are also abstract with no fixed materialised format imposed by the API. All event data is supplied by the plugin by calling "provider" methods in applier objects obtained from the API. This allows the plugin to choose which information to supply. For example, one plugin may supply row event values identified by column name, while another may supply by column poisition. Using a provider interface, different plugins can supply different information, as long as the provided information is sufficient for the applier API to execute the event in _some_ way. The event applier API also need to provide a facility for initialising a thread (or multiple threads) to be able to execute events (setting up THD etc). Event appliers are stacked, just like event generators. For example, row-based events are stacked on statement-based events. So a row event applier can accept statement events in addition to row events (and recursively also accept transaction events, as statement-based is stacked on transaction-based). On top of the detailed interfaces for event generation and application, one can build simple materialised event formats, which are just byte strings containing all necessary information. On the generator side, this can be done as a filter that consumes row-based events (say) and encapsulates them in some way (say Google protobuf packets). On the applier side, it can be done by implementing a simple event applier class, that has a single provider method only, accepting a materialised event packet. The strength of the full abstraction below this is that it is possible to implement arbitrary such materialised event formats in plugins, and possible to implement event format plugins and event transport plugins independent of each other. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v4.0.0)

1 0

[Maria-developers] Rev 2897: Fix for LP bug#611379. in file:///home/bell/maria/bzr/work-maria-5.1-lb611379/
by sanja＠askmonty.org 27 Aug '10

27 Aug '10

At file:///home/bell/maria/bzr/work-maria-5.1-lb611379/ ------------------------------------------------------------ revno: 2897 revision-id: sanja(a)askmonty.org-20100827110519-b0gnykgc417kgi4q parent: monty(a)mysql.com-20100809170542-ewa2awm6pcoi1ipy committer: sanja(a)askmonty.org branch nick: work-maria-5.1-lb611379 timestamp: Fri 2010-08-27 14:05:19 +0300 message: Fix for LP bug#611379. maybe_null/null_value flag set to TRUE for Item_sum_distinct. === modified file 'mysql-test/r/func_group.result' --- a/mysql-test/r/func_group.result 2009-11-24 15:26:13 +0000 +++ b/mysql-test/r/func_group.result 2010-08-27 11:05:19 +0000 @@ -1713,4 +1713,16 @@ NULL NULL NULL NULL NULL drop table t1; # +#test for LP Bug#611379 +# +create table t1 (a int not null); +insert into t1 values (3), (1), (2); +select sum(distinct a) from t1 where a < 0; +sum(distinct a) +NULL +select * from (select sum(distinct a) from t1 where a < 0) as t; +sum(distinct a) +NULL +drop table t1; +# End of 5.1 tests === modified file 'mysql-test/t/func_group.test' --- a/mysql-test/t/func_group.test 2009-11-24 15:26:13 +0000 +++ b/mysql-test/t/func_group.test 2010-08-27 11:05:19 +0000 @@ -1082,6 +1082,16 @@ from t1 a, t1 b; select *, f1 = f2 from t1; drop table t1; + +--echo # +--echo #test for LP Bug#611379 +--echo # +create table t1 (a int not null); +insert into t1 values (3), (1), (2); +select sum(distinct a) from t1 where a < 0; +select * from (select sum(distinct a) from t1 where a < 0) as t; +drop table t1; + --echo # --echo End of 5.1 tests === modified file 'sql/item_sum.cc' --- a/sql/item_sum.cc 2010-08-02 09:01:24 +0000 +++ b/sql/item_sum.cc 2010-08-27 11:05:19 +0000 @@ -587,13 +587,11 @@ return TRUE; decimals=0; - maybe_null=0; for (uint i=0 ; i < arg_count ; i++) { if (args[i]->fix_fields(thd, args + i) || args[i]->check_cols(1)) return TRUE; set_if_bigger(decimals, args[i]->decimals); - maybe_null |= args[i]->maybe_null; } result_field=0; max_length=float_length(decimals); @@ -949,6 +947,7 @@ { DBUG_ASSERT(args[0]->fixed); + maybe_null= TRUE; table_field_type= args[0]->field_type(); /* Adjust tmp table type according to the chosen aggregation type */

1 0

[Maria-developers] [Fwd: DS-MRR improvements patch ready for review]
by Igor Babaev 25 Aug '10

25 Aug '10

Sergey, 1. I failed to find LLD for this patch anywhere. That's why I'll try to put together the description of the abstract data structures used by DsMrr_impl. 2. The main abstract data structure here could be called Flexible_LIFO_Buffer. I call it LIFO because it's filled in one direction while it's read always in the opposite direction starting from the end of the buffer. I call it Flexible because it can be extended when we write into it, and it can be shrunk when we read from it. 3. It makes sense to introduce two sub-classes of this data structure: Flexible_LIFO_Buffer_LR, that writes from left to right, and Flexible_LIFO_Buffer_RL, that writes from right to left. 4. You need two buffers: a buffer for keys [+ association] a buffer for rowids [+ association] If one is of the type Flexible_LIFO_Buffer_LR then the other must be of the type Flexible_LIFO_Buffer_RL. 5. It looks like the buffer for rowids should be of the type Flexible_FIFO_Buffer_LR/RL. Yet because we read from the buffer always after sorting we always can employ Flexible_LIFO_Buffer_LR/RL instead with a proper choice of the comparison function used by the sort procedure. 6. I don't mind if you use the name DsMrr_Buffer[_RL/RL] instead of Flexible_FIFO_Buffer_LR/RL. Maybe it will be even better. 7. The class DsMrr_Buffer must contain a method that sorts the elements in the buffer. Some other remarks: 1. Always use mnemonic constants instead of the constant like 1, -1 you used for direction values, or the constant from a enum type. 2. Each member of the class you use must me specified somehow. 3. Do not overload the semantics of the field members of you classes. E.g. key_tuple_length is a constant for any key buffer and should not everbe changed. 4. Your buffers are actually containers of elements. So it's more natural to have read/write methods for these elements. 5. It looks like it's enough to have only one cursor (let's call it 'pos') for your read/write operations. 6. I think that class DsMrr_impl does not have to know anything about the structure/representation of the elements in a buffer. 7. The companion iterator class should iterate over interesting association info. Maybe it make sense to have a counter of the remaining elements that still have to be iterated over. The init method of the iterator class should reset this counter. 8. DsMrr_impl should contain one method that shifts the boundary between the key buffer and the rowid buffer. It should employ the methods of the buffer classes to do this. 9. As you have only two buffers I don't mind if you have instead of two similar classes Flexible_LIFO_Buffer_LR and Flexible_LIFO_Buffer_RL two quite different classes DsMrr_Key_Buffer and DsMrr_Rowid_Buffer. The first one will be of the type Flexible_LIFO_Buffer with the iterator companion class in it. The second class may be of the Flexible_FIFO_Buffer type. However the generic functionality/interface still must be factored out into the base class DsMrr_Buffer. 10. I think it makes sense to skip unnecessary calls of handler functions. 11. Test cases for the new functionality are scarce. 12. Please take care of the bug I discovered earlier and report of which I resent you again (see [Fwd: [Fwd: [Fwd: Another serious problem with 5.3-dsmrr-cpk]]]) Regards, Igor. -------- Original Message -------- Subject: DS-MRR improvements patch ready for review Date: Fri, 20 Aug 2010 09:12:19 +0200 From: Sergey Petrunya <psergey(a)askmonty.org> To: igor(a)askmonty.org CC: maria-developers(a)lists.launchpad.net Hello Igor, Please find attached the combined patch of DS-MRR for clustered PKs and key sorting. The tree is in launchpad and buildbot also: https://code.launchpad.net/~maria-captains/maria/5.3-dsmrr-cpk and all observed buildbot failures in the tree are known to occur without the new code as well. BR Sergey -- Sergey Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] WL#132 New (by Knielsen): Transaction coordinator plugin
by worklog-noreply＠askmonty.org 24 Aug '10

24 Aug '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Transaction coordinator plugin CREATION DATE..: Tue, 24 Aug 2010, 14:06 SUPERVISOR.....: IMPLEMENTOR....: Knielsen COPIES TO......: CATEGORY.......: Server-Sprint TASK ID........: 132 (http://askmonty.org/worklog/?tid=132) VERSION........: Server-9.x STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: DESCRIPTION: This worklog describes an API that allows a replication plugin to hook deep into the commit mechanism of transactions in mysqld and implement its own transaction coordinator (TC). The existing binary log already implements a TC. This is used to make writes to the binary log happen in 2-phase commit with transactional engines. It is also used during crash recovery to get the list of prepared-but-not-committed transactions from the binary log and ensure that those transactions are committed (and any others rolled back), to avoid inconsistencies between the contents of binary log and engines. Thus the TC needs to be part of a replication API that allows the existing binary log implementation to be written as a plugin, or to allow the implementation of a replacement binary log. In addition, something like Galera synchronous replication needs more control over the commit process. In particular, it needs to be able to control the order in which transactions are committed. This worklog extends the TC interface to also allow TC to control commit order. This part of the worklog builds on the work in MWL#116, in particular on the prepare_ordered() and commit_ordered() extensions to the storage engine API. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v4.0.0)

1 0

[Maria-developers] DS-MRR improvements patch ready for review
by Sergey Petrunya 24 Aug '10

24 Aug '10

Hello Igor, Please find attached the combined patch of DS-MRR for clustered PKs and key sorting. The tree is in launchpad and buildbot also: https://code.launchpad.net/~maria-captains/maria/5.3-dsmrr-cpk and all observed buildbot failures in the tree are known to occur without the new code as well. BR Sergey -- Sergey Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

2 2

[Maria-developers] Ipv6
by Dolf Schimmel 24 Aug '10

24 Aug '10

Dear Folks, With 5 billion devices connected on the internet, and only a total of 4 billion ip(v4) addreses, available ipv4 addresses are becoming more and more scarcely. Latest estimates are that it will take less than a year before IANA has no more ipv4 addresses to distribute to LIR's (APNIC, RIPE, etc), and some time after that it will be really, really difficult for anybody to get an ipv4 address. Luckily there's a solution for that in the form of ipv6. Ipv6 is very common in certain parts of Asia (Japan) where students are only given a block of ipv6 addresses with a mere tunnel to access ipv4 hosts. In France one of the biggest ISP's ( http://free.fr ) is giving every residential dsl connection a block of ipv6 addresses, and in The Netherlands more and more shared webhosters assign each individual website its own ipv6 address (obviously together with a shared ipv4 address) since Direct Admin ( http://directadmin.com competitor of cpanel and plesk) fully supports ipv6. All in all, the world is calling for ipv6 and products that suppoort it. Whereas ipv4 is 32bit, resulting in a total number of 2^32 (=4billion) ip's, ipv6 is 128bit, giving 2^128 addresses. A number large enough for me not to spell it out for you :D Though an ipv6 address does have several hexadecimal notations, it still simply is a 128bit unsigned integer. And as such, I'm sending this email requesting to support it. My proposal would be to introduce two new datatypes, one for 128bit integers, and one extending the 128bit integer type for ipv6 addresses. Meaning that the latter accept ipv6 addresses only, but relies on the first one for storing it, and displaying it again as ipv6 address. With ipv6 one usually will do much more rangechecks than ipv4 because with ipv6 residential homes simply get a /64 block (2^(128-64)= a lot of addresses) instead of simply one address. Therefore a nice feature would be to allow queries like: `SELECT * FROM table WHERE columnName IN 2001:db8:85a3::8a2e:370:7334/62` which essentially would select all rows where columnName is in the same /64 block as 2001:db8:85a3::8a2e:370:7334 is. I'm a simple PHP developer so I can't supply a patch, but I would be really curious in what you think about the above, and hope someone is able and willing to pick it up. Regards, Dolf Schimmel -- Freeaqingme P.s. I'm sorry if the first paragraph is stating the obvious only. Some people know everything about ipv6, and some don't at all. Which is why I figured an introduction wouldn't hurt.

3 4

[Maria-developers] WL#131 New (by Sergei): new regular expression library
by worklog-noreply＠askmonty.org 23 Aug '10

23 Aug '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: new regular expression library CREATION DATE..: Mon, 23 Aug 2010, 19:57 SUPERVISOR.....: Sergei IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Server-BackLog TASK ID........: 131 (https://askmonty.org/worklog/?tid=131) VERSION........: Benchmarks-3.0 STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 48 (hours remain) ORIG. ESTIMATE.: 48 PROGRESS NOTES: DESCRIPTION: We need to replace old regex library that MariaDB uses at the moment with a new one, that works with multi-byte charsets and preferably has more features (pcre?). ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v4.0.0)

1 0

[Maria-developers] Get rid of all 32 bit Visual Studio compiler warnings
by Bo Thorsen 23 Aug '10

23 Aug '10

Hi all, I'm sending this patch again, since I didn't get any responses to the last two mails. The patch is almost identical to the last version except that I have removed some parts of it - those warnings were fixed by someone else after I sent the last version of the patch. Here is the patch description from the last mail: This is a new version of the patch, and it removes all compiler warnings on a 32 bit build using Visual Studio 2008. This patch sets a compiler flag on the bison and flex generated files that removes warnings on those. (That nothing uses the yyerrorlab label is fine.) The libmysqld.def file: The description line isn't supported in the more recent compilers. In fsp0fsp.c, there is an explicit cast to ullint. In this case, the code does what's intended, and the compiler warning is one of the "are you sure this is right" warnings. C4334 gives a warning if you make a 1u << 10 and store that in a 64 bit variable, because you could have meant 1i64 << 10. Can I push this to 5.1? Bo Thorsen. Monty Program AB. -- MariaDB: MySQL replacement Community developed. Feature enhanced. Backward compatible.

2 1