- developers - lists.mariadb.org

[Maria-developers] [Fwd: DS-MRR improvements patch ready for review]
by Igor Babaev 25 Aug '10

25 Aug '10

Sergey, 1. I failed to find LLD for this patch anywhere. That's why I'll try to put together the description of the abstract data structures used by DsMrr_impl. 2. The main abstract data structure here could be called Flexible_LIFO_Buffer. I call it LIFO because it's filled in one direction while it's read always in the opposite direction starting from the end of the buffer. I call it Flexible because it can be extended when we write into it, and it can be shrunk when we read from it. 3. It makes sense to introduce two sub-classes of this data structure: Flexible_LIFO_Buffer_LR, that writes from left to right, and Flexible_LIFO_Buffer_RL, that writes from right to left. 4. You need two buffers: a buffer for keys [+ association] a buffer for rowids [+ association] If one is of the type Flexible_LIFO_Buffer_LR then the other must be of the type Flexible_LIFO_Buffer_RL. 5. It looks like the buffer for rowids should be of the type Flexible_FIFO_Buffer_LR/RL. Yet because we read from the buffer always after sorting we always can employ Flexible_LIFO_Buffer_LR/RL instead with a proper choice of the comparison function used by the sort procedure. 6. I don't mind if you use the name DsMrr_Buffer[_RL/RL] instead of Flexible_FIFO_Buffer_LR/RL. Maybe it will be even better. 7. The class DsMrr_Buffer must contain a method that sorts the elements in the buffer. Some other remarks: 1. Always use mnemonic constants instead of the constant like 1, -1 you used for direction values, or the constant from a enum type. 2. Each member of the class you use must me specified somehow. 3. Do not overload the semantics of the field members of you classes. E.g. key_tuple_length is a constant for any key buffer and should not everbe changed. 4. Your buffers are actually containers of elements. So it's more natural to have read/write methods for these elements. 5. It looks like it's enough to have only one cursor (let's call it 'pos') for your read/write operations. 6. I think that class DsMrr_impl does not have to know anything about the structure/representation of the elements in a buffer. 7. The companion iterator class should iterate over interesting association info. Maybe it make sense to have a counter of the remaining elements that still have to be iterated over. The init method of the iterator class should reset this counter. 8. DsMrr_impl should contain one method that shifts the boundary between the key buffer and the rowid buffer. It should employ the methods of the buffer classes to do this. 9. As you have only two buffers I don't mind if you have instead of two similar classes Flexible_LIFO_Buffer_LR and Flexible_LIFO_Buffer_RL two quite different classes DsMrr_Key_Buffer and DsMrr_Rowid_Buffer. The first one will be of the type Flexible_LIFO_Buffer with the iterator companion class in it. The second class may be of the Flexible_FIFO_Buffer type. However the generic functionality/interface still must be factored out into the base class DsMrr_Buffer. 10. I think it makes sense to skip unnecessary calls of handler functions. 11. Test cases for the new functionality are scarce. 12. Please take care of the bug I discovered earlier and report of which I resent you again (see [Fwd: [Fwd: [Fwd: Another serious problem with 5.3-dsmrr-cpk]]]) Regards, Igor. -------- Original Message -------- Subject: DS-MRR improvements patch ready for review Date: Fri, 20 Aug 2010 09:12:19 +0200 From: Sergey Petrunya <psergey(a)askmonty.org> To: igor(a)askmonty.org CC: maria-developers(a)lists.launchpad.net Hello Igor, Please find attached the combined patch of DS-MRR for clustered PKs and key sorting. The tree is in launchpad and buildbot also: https://code.launchpad.net/~maria-captains/maria/5.3-dsmrr-cpk and all observed buildbot failures in the tree are known to occur without the new code as well. BR Sergey -- Sergey Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] WL#132 New (by Knielsen): Transaction coordinator plugin
by worklog-noreply＠askmonty.org 24 Aug '10

24 Aug '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Transaction coordinator plugin CREATION DATE..: Tue, 24 Aug 2010, 14:06 SUPERVISOR.....: IMPLEMENTOR....: Knielsen COPIES TO......: CATEGORY.......: Server-Sprint TASK ID........: 132 (http://askmonty.org/worklog/?tid=132) VERSION........: Server-9.x STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: DESCRIPTION: This worklog describes an API that allows a replication plugin to hook deep into the commit mechanism of transactions in mysqld and implement its own transaction coordinator (TC). The existing binary log already implements a TC. This is used to make writes to the binary log happen in 2-phase commit with transactional engines. It is also used during crash recovery to get the list of prepared-but-not-committed transactions from the binary log and ensure that those transactions are committed (and any others rolled back), to avoid inconsistencies between the contents of binary log and engines. Thus the TC needs to be part of a replication API that allows the existing binary log implementation to be written as a plugin, or to allow the implementation of a replacement binary log. In addition, something like Galera synchronous replication needs more control over the commit process. In particular, it needs to be able to control the order in which transactions are committed. This worklog extends the TC interface to also allow TC to control commit order. This part of the worklog builds on the work in MWL#116, in particular on the prepare_ordered() and commit_ordered() extensions to the storage engine API. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v4.0.0)

1 0

[Maria-developers] DS-MRR improvements patch ready for review
by Sergey Petrunya 24 Aug '10

24 Aug '10

Hello Igor, Please find attached the combined patch of DS-MRR for clustered PKs and key sorting. The tree is in launchpad and buildbot also: https://code.launchpad.net/~maria-captains/maria/5.3-dsmrr-cpk and all observed buildbot failures in the tree are known to occur without the new code as well. BR Sergey -- Sergey Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

2 2

[Maria-developers] Ipv6
by Dolf Schimmel 24 Aug '10

24 Aug '10

Dear Folks, With 5 billion devices connected on the internet, and only a total of 4 billion ip(v4) addreses, available ipv4 addresses are becoming more and more scarcely. Latest estimates are that it will take less than a year before IANA has no more ipv4 addresses to distribute to LIR's (APNIC, RIPE, etc), and some time after that it will be really, really difficult for anybody to get an ipv4 address. Luckily there's a solution for that in the form of ipv6. Ipv6 is very common in certain parts of Asia (Japan) where students are only given a block of ipv6 addresses with a mere tunnel to access ipv4 hosts. In France one of the biggest ISP's ( http://free.fr ) is giving every residential dsl connection a block of ipv6 addresses, and in The Netherlands more and more shared webhosters assign each individual website its own ipv6 address (obviously together with a shared ipv4 address) since Direct Admin ( http://directadmin.com competitor of cpanel and plesk) fully supports ipv6. All in all, the world is calling for ipv6 and products that suppoort it. Whereas ipv4 is 32bit, resulting in a total number of 2^32 (=4billion) ip's, ipv6 is 128bit, giving 2^128 addresses. A number large enough for me not to spell it out for you :D Though an ipv6 address does have several hexadecimal notations, it still simply is a 128bit unsigned integer. And as such, I'm sending this email requesting to support it. My proposal would be to introduce two new datatypes, one for 128bit integers, and one extending the 128bit integer type for ipv6 addresses. Meaning that the latter accept ipv6 addresses only, but relies on the first one for storing it, and displaying it again as ipv6 address. With ipv6 one usually will do much more rangechecks than ipv4 because with ipv6 residential homes simply get a /64 block (2^(128-64)= a lot of addresses) instead of simply one address. Therefore a nice feature would be to allow queries like: `SELECT * FROM table WHERE columnName IN 2001:db8:85a3::8a2e:370:7334/62` which essentially would select all rows where columnName is in the same /64 block as 2001:db8:85a3::8a2e:370:7334 is. I'm a simple PHP developer so I can't supply a patch, but I would be really curious in what you think about the above, and hope someone is able and willing to pick it up. Regards, Dolf Schimmel -- Freeaqingme P.s. I'm sorry if the first paragraph is stating the obvious only. Some people know everything about ipv6, and some don't at all. Which is why I figured an introduction wouldn't hurt.

3 4

[Maria-developers] WL#131 New (by Sergei): new regular expression library
by worklog-noreply＠askmonty.org 23 Aug '10

23 Aug '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: new regular expression library CREATION DATE..: Mon, 23 Aug 2010, 19:57 SUPERVISOR.....: Sergei IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Server-BackLog TASK ID........: 131 (https://askmonty.org/worklog/?tid=131) VERSION........: Benchmarks-3.0 STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 48 (hours remain) ORIG. ESTIMATE.: 48 PROGRESS NOTES: DESCRIPTION: We need to replace old regex library that MariaDB uses at the moment with a new one, that works with multi-byte charsets and preferably has more features (pcre?). ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v4.0.0)

1 0

[Maria-developers] Get rid of all 32 bit Visual Studio compiler warnings
by Bo Thorsen 23 Aug '10

23 Aug '10

Hi all, I'm sending this patch again, since I didn't get any responses to the last two mails. The patch is almost identical to the last version except that I have removed some parts of it - those warnings were fixed by someone else after I sent the last version of the patch. Here is the patch description from the last mail: This is a new version of the patch, and it removes all compiler warnings on a 32 bit build using Visual Studio 2008. This patch sets a compiler flag on the bison and flex generated files that removes warnings on those. (That nothing uses the yyerrorlab label is fine.) The libmysqld.def file: The description line isn't supported in the more recent compilers. In fsp0fsp.c, there is an explicit cast to ullint. In this case, the code does what's intended, and the compiler warning is one of the "are you sure this is right" warnings. C4334 gives a warning if you make a 1u << 10 and store that in a 64 bit variable, because you could have meant 1i64 << 10. Can I push this to 5.1? Bo Thorsen. Monty Program AB. -- MariaDB: MySQL replacement Community developed. Feature enhanced. Backward compatible.

2 1

[Maria-developers] Patch: Separate builddir from src dir with cmake
by Bo Thorsen 23 Aug '10

23 Aug '10

Hi everyone, I have updated the attached patch to apply to the 5.1 tree. Please see the comments in my original mail below. Any objections to apply this patch? -------- Original Meddelelse -------- Subject: Separate builddir from src dir with cmake Date: Mon, 12 Jul 2010 14:01:23 +0200 From: Bo Thorsen <bo(a)askmonty.org> Organization: Monty Program AB To: maria-developers(a)lists.launchpad.net Hi people, Our cmake scripts weren't able to build outside the source dir. This is bad for 3 reasons: - it's supposed to work - cmake recommends building outside the src dir - on Windows, cmake can't build 32 and 64 bit in the same solution, so patches has to be manually copied to test compilation on the other The attached patch fixes all the issues. I ran a find on the source dir before and after the the build, and they were identical. I didn't actually check that no existing files we're modified, but bzr di claim this isn't the case. I didn't port the zip creation script, but that shouldn't be a problem. src != blddir is more a development thing, and the buildbot slave producing the zip file will just continue to build in the same dir. The cpack installer builds just fine. OK to push to 5.1? Should I send this patch to the Oracle as well to minimize the drift between the cmake files? Bo Thorsen. Monty Program AB. -- MariaDB: MySQL replacement Community developed. Feature enhanced. Backward compatible.

2 1

[Maria-developers] Handling size_t -> int conversions
by Bo Thorsen 20 Aug '10

20 Aug '10

I'm working on getting rid of the compiler warnings on 64 bit Windows. There are literally thousands of places, where we store a size_t in an int, and the compiler warns about possible loss of data. I'm trying to figure out how to handle this in the best way, and I need a bit of advice on how to proceed. First, is it acceptable to change all/most of these vars from int to size_t? In places where there is a local variable that stores the difference between two pointers, it's fine. But what about places like struct z_stream_s in zlib.h? How do you guys prefer to see patches on this - one big patch, or several smaller patches? Bo Thorsen. Monty Program AB. -- MariaDB: MySQL replacement Community developed. Feature enhanced. Backward compatible.

4 3

[Maria-developers] Rev 2897: Fix for LP bug#611379. in file:///home/bell/maria/bzr/work-maria-5.1-lb611379/
by sanja＠askmonty.org 20 Aug '10

20 Aug '10

At file:///home/bell/maria/bzr/work-maria-5.1-lb611379/ ------------------------------------------------------------ revno: 2897 revision-id: sanja(a)askmonty.org-20100812075534-wc2dngmpiqm9grra parent: monty(a)mysql.com-20100809170542-ewa2awm6pcoi1ipy committer: sanja(a)askmonty.org branch nick: work-maria-5.1-lb611379 timestamp: Thu 2010-08-12 10:55:34 +0300 message: Fix for LP bug#611379. maybe_null/null_value flag set to TRUE for Item_sum_distinct. === modified file 'mysql-test/r/func_group.result' --- a/mysql-test/r/func_group.result 2009-11-24 15:26:13 +0000 +++ b/mysql-test/r/func_group.result 2010-08-12 07:55:34 +0000 @@ -1713,4 +1713,26 @@ NULL NULL NULL NULL NULL drop table t1; # +#test for LP Bug#611379 +# +create table t1 (a int not null); +insert into t1 values (1); +create table t2 (a int not null primary key); +insert into t2 values (10); +explain select sum(distinct t1.a) from t1,t2 where t1.a=t2.a; +id select_type table type possible_keys key key_len ref rows Extra +1 SIMPLE NULL NULL NULL NULL NULL NULL NULL Impossible WHERE noticed after reading const tables +explain select * from (select sum(distinct t1.a) from t1,t2 where t1.a=t2.a) +as t; +id select_type table type possible_keys key key_len ref rows Extra +1 PRIMARY <derived2> system NULL NULL NULL NULL 1 +2 DERIVED NULL NULL NULL NULL NULL NULL NULL Impossible WHERE noticed after reading const tables +select sum(distinct t1.a) from t1,t2 where t1.a=t2.a; +sum(distinct t1.a) +NULL +select * from (select sum(distinct t1.a) from t1,t2 where t1.a=t2.a) as t; +sum(distinct t1.a) +NULL +drop table t1,t2; +# End of 5.1 tests === modified file 'mysql-test/t/func_group.test' --- a/mysql-test/t/func_group.test 2009-11-24 15:26:13 +0000 +++ b/mysql-test/t/func_group.test 2010-08-12 07:55:34 +0000 @@ -1082,6 +1082,21 @@ from t1 a, t1 b; select *, f1 = f2 from t1; drop table t1; + +--echo # +--echo #test for LP Bug#611379 +--echo # +create table t1 (a int not null); +insert into t1 values (1); +create table t2 (a int not null primary key); +insert into t2 values (10); +explain select sum(distinct t1.a) from t1,t2 where t1.a=t2.a; +explain select * from (select sum(distinct t1.a) from t1,t2 where t1.a=t2.a) +as t; +select sum(distinct t1.a) from t1,t2 where t1.a=t2.a; +select * from (select sum(distinct t1.a) from t1,t2 where t1.a=t2.a) as t; +drop table t1,t2; + --echo # --echo End of 5.1 tests === modified file 'sql/item_sum.cc' --- a/sql/item_sum.cc 2010-08-02 09:01:24 +0000 +++ b/sql/item_sum.cc 2010-08-12 07:55:34 +0000 @@ -949,6 +949,7 @@ { DBUG_ASSERT(args[0]->fixed); + null_value= maybe_null= TRUE; table_field_type= args[0]->field_type(); /* Adjust tmp table type according to the chosen aggregation type */

2 1

[Maria-developers] WL#130 New (by Knielsen): Good defaults for ./configure (for end-user source builds)
by worklog-noreply＠askmonty.org 19 Aug '10

19 Aug '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Good defaults for ./configure (for end-user source builds) CREATION DATE..: Thu, 19 Aug 2010, 10:09 SUPERVISOR.....: IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Server-RawIdeaBin TASK ID........: 130 (http://askmonty.org/worklog/?tid=130) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: DESCRIPTION: I would really like it to be easy to build MariaDB from source. Basically, a plain ./configure should just work in the normal case, and produce something sensible that is close/identical to what a relase build looks like. Unfortunately, this is not currently so. There is a long tradition in MySQL (and now MariaDB) that release builds are made with a long carefully crafted list of ./configure options that are necessary to get everything included or enabled. One concrete example I found is that plain ./configure will build XtraDB/InnoDB as plugins by default. This means that upon starting the server, there will be no engine "innodb" available until one is installed with INSTALL PLUGIN! I think this is not desirable default behaviour. The task here is to go through the ourdelta build scripts [1] and check all the ./configure options we use in them and what the default behaviour of that option is. Defaults that is not sensible for a plain ./configure build should be changed. References: [1] http://bazaar.launchpad.net/~maria-captains/ourdelta/ourdelta-montyprogram-… ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v4.0.0)

1 0