October 2014 - developers - lists.mariadb.org

[Maria-developers] MariaDB 10.0 on POWER
by Colin Charles 14 Oct '14

14 Oct '14

Hi, I'm sure you have read: https://www.flamingspork.com/blog/2014/10/08/mariadb-10-0-on-power/ For XL: There’s obviously still some work in progress going on, but my initial results show performance within around 10% of MySQL, so with a bit of work we will hopefully see MariaDB reach performance parity. For Serg: One interesting find was the code to account for thread memory usage uses a single atomic variable: this does not scale and does end up showing up on profiles. It isn't clear 10% loss to which MySQL, but comparing to 5.6 or 5.7, this has to "just work better" Thanks -- Colin Charles, Chief Evangelist, MariaDB Corporation blog: http://bytebot.net/blog/| t: +6-012-204-3201 | Skype: colincharles

2 1

Re: [Maria-developers] [Commits] 47fea67: MDEV-12 OpenGIS: create required tables: GeometryColumns, related views.
by Sergei Golubchik 14 Oct '14

14 Oct '14

Hi, Alexey! On Sep 02, Alexey Botchkov wrote: > revision-id: 47fea6798f0f37b8a42ad047d7daf6ec9bb52813 > parent(s): e44751b65f4760067d15f8a526e8f97f84810c29 > committer: Alexey Botchkov > branch nick: > timestamp: 2014-09-02 11:31:26 +0500 > message: > > MDEV-12 OpenGIS: create required tables: GeometryColumns, related views. > GEOMETRY_COLUMNS and SPATIAL_REF_SYS tables added to the INFORMATION_SCHEMA. > > diff --git a/mysql-test/r/information_schema.result > b/mysql-test/r/information_schema.result > index 6df5b89..93c9e90 100644 > --- a/mysql-test/r/information_schema.result > +++ b/mysql-test/r/information_schema.result > @@ -43,7 +43,8 @@ create view v1 (c) as > SELECT table_name FROM information_schema.TABLES > WHERE table_schema IN ('mysql', 'INFORMATION_SCHEMA', 'test', > 'mysqltest') AND > table_name not like 'ndb_%' AND table_name not like 'innodb_%' AND > -table_name not like 'xtradb_%'; > +table_name not like 'xtradb_%' AND > +table_name not like 'geometry%' AND table_name not like 'spatial%'; Why did you filter these tables out? Normally, only plugin-created tables are filtered out, because these plugins might be disabled in the build. But we don't ever build without geometry, do we? I'd keep these tables in the result. > select * from v1; > c > ALL_PLUGINS > diff --git a/mysql-test/t/gis.test b/mysql-test/t/gis.test > index c387069..1a07a33 100644 > --- a/mysql-test/t/gis.test > +++ b/mysql-test/t/gis.test > @@ -1461,3 +1461,7 @@ insert into t1 values(Geomfromtext('POLYGON((1 1, > 2 2, 2 1, 1 1))')); > drop table t1; > > --echo End of 5.5 tests > + > +SHOW CREATE TABLE information_schema.geometry_columns; > +SHOW CREATE TABLE information_schema.spatial_ref_sys; Please, add tests for the content of these tables. Then ok to push (assuming these new tests will work fine). Regards, Sergei

1 0

[Maria-developers] Is it soon 'end of this year'?
by Peter Laursen 13 Oct '14

13 Oct '14

Around ½ year ago I discussed with some people here (Sergei G and/or Georg and/or Wlad) about support for Oracle-style plugins in the MariaDB C-API. The plugins include 1 Windows authentication plugin 2) Clear text plugin. At that time I was told that it could be considered 'by end end of this year'. 1) Windows authentication plugin. Not very important, but Sergei told that he had been playing wiht it and it could probably be done in ~1 day. 2) This is used by Oracle for setting up LDAP.authentication. I am very well aware that there is criticism of Oracle's implementation. But this does not change the fact hat people use it with servers from Oracle and expect clients to work with it. We get request(s) for it almost every week. So this is most important (for us). Any chance that those can soon be prioritized? (if you feel your self mis-quoted then I apologize).. -- Peter --Webyog

3 12

[Maria-developers] doubt about MDL
by Roberto Spadim 13 Oct '14

13 Oct '14

hi guys, i'm reading https://github.com/MariaDB/server/blob/10.1/sql/mdl.h i'm with a doubt... today we have the metadata lock plugin to check what mdl we have... my doubt is, how to expose how many threads are sharing the mdl key, and how many threads are waiting the mdl key, should it be possible or only one threrad could use one mdl? -- Roberto Spadim

2 3

Re: [Maria-developers] Page encryption exception handling.
by Sergei Golubchik 13 Oct '14

13 Oct '14

Hi, Clemens! On Oct 13, Clemens Dörrhöfer wrote: > Hi Sergei, > > I included an error, which was triggered by a missing key in the > keyfile. The server will see a wrong space ID, since it has not been > decrypted, which from my point of view is the reason for mariadb to > crash. > Basically we run into a problem when trying to decrypt a page using > fil_decrypt_page theoretically we can catch this exception much > sooner, since all we need to know is the key id we need for decryption > / encryption and throw an exception as soon as we know that we cannot > handle a key in best case before we trigger any aio. > > fil_decrypt_page() at fil0pageencryption.cc:391 0x555555feeb91 > os_file_read_func() at os0file.cc:3.156 0x555555e22648 > pfs_os_file_read_func() at os0file.ic:288 0x555555fdb4fa > fil_read_first_page() at fil0fil.cc:2.156 0x555555fe1eb0 > fil_validate_single_table_tablespace() at fil0fil.cc:4.303 0x555555fe5fd0 > fil_load_single_table_tablespace() at fil0fil.cc:4.443 0x555555fe64af > fil_load_single_table_tablespaces() at fil0fil.cc:4.806 0x555555fe6d1a > recv_init_crash_recovery() at log0recv.cc:3.046 0x555555e14447 > recv_recovery_from_checkpoint_start_func() at log0recv.cc:3.360 0x555555e14c69 > innobase_start_or_create_for_mysql() at srv0start.cc:2.471 0x555555eeea65 This still doesn't show where it crashes, only where you detect the missing key. But anyway, I agree, if you can detect a missing key sooner, you'd better do that, instead of returning invalid page to XtraDB, because it used to crash the server in case of severe errors like that. Regards, Sergei

1 0

[Maria-developers] Isn't thsi a bug?
by Peter Laursen 13 Oct '14

13 Oct '14

Refer: http://bugs.mysql.com/bug.php?id=74238 here I prefer the 'relaxed parsing' in Maria DB. But look at this: CREATE TABLE `d` ( `rec_id` INT(10) UNSIGNED NOT NULL AUTO_INCREMENT, `datetime` DATETIME(4) DEFAULT NULL, `timestamp` TIMESTAMP(4) NOT NULL DEFAULT CURRENT_TIMESTAMP(5) ON UPDATE CURRENT_TIMESTAMP(4), PRIMARY KEY (`rec_id`) ) ENGINE=INNODB; -- it actually creates the table SHOW CREATE TABLE d; /* returns CREATE TABLE `d` ( `rec_id` int(10) unsigned NOT NULL AUTO_INCREMENT, `datetime` datetime(4) DEFAULT NULL, `timestamp` timestamp(4) NOT NULL DEFAULT CURRENT_TIMESTAMP(4) ON UPDATE CURRENT_TIMESTAMP(4), PRIMARY KEY (`rec_id`) ) ENGINE=InnoDB DEFAULT CHARSET=latin1 */ I don't think DEFAULT CURRENT_TIMESTAMP(5) should be accepted for the datatype DATETIME(4). What say? -- Peter -- Webyog

2 2

[Maria-developers] Fwd: [JIRA] (MDEV-6566) Different INSERT behaviour on bad bytes with and without character set conversion
by Alexander Barkov 13 Oct '14

13 Oct '14

Hi Sergei, I suggest to fix this starting from 10.1. -------- Original Message -------- Subject: [JIRA] (MDEV-6566) Different INSERT behaviour on bad bytes with and without character set conversion Date: Thu, 9 Oct 2014 21:03:51 +0300 (EEST) From: Sergei Golubchik (JIRA) <jira(a)mariadb.atlassian.net> To: bar(a)mariadb.org [ https://mariadb.atlassian.net/browse/MDEV-6566?page=com.atlassian.jira.plug… ] Sergei Golubchik updated MDEV-6566: ----------------------------------- Fix Version/s: (was: 5.5.40) 5.5.41 > Different INSERT behaviour on bad bytes with and without character set conversion > --------------------------------------------------------------------------------- > > Key: MDEV-6566 > URL: https://mariadb.atlassian.net/browse/MDEV-6566 > Project: MariaDB Server > Issue Type: Bug > Affects Versions: 5.3.12, 5.5.39, 10.0.13 > Reporter: Alexander Barkov > Assignee: Alexander Barkov > Fix For: 10.0.15, 5.5.41 > > > If I run this SQL script in a utf8 client: > {noformat} > SET NAMES utf8; > DROP TABLE IF EXISTS t1; > CREATE TABLE t1 ( > a VARCHAR(10) CHARACTER SET ucs2, > b VARCHAR(10) CHARACTER SET utf8 > ); > INSERT INTO t1 SELECT 'a 😁 b', 'a 😁 b'; > SHOW WARNINGS; > SELECT * FROM t1; > {noformat} > It displays the following warnings: > {noformat} > +---------+------+----------------------------------------------------------------------+ > | Level | Code | Message | > +---------+------+----------------------------------------------------------------------+ > | Warning | 1300 | Invalid utf8 character string: '\xF0\x9F\x98\x81 b' | > | Warning | 1300 | Invalid utf8 character string: '\xF0\x9F\x98\x81 b' | > | Warning | 1366 | Incorrect string value: '\xF0\x9F\x98\x81 b' for column 'a' at row 1 | > | Warning | 1265 | Data truncated for column 'b' at row 1 | > +---------+------+----------------------------------------------------------------------+{noformat} > with the following results: > {noformat} > +----------+------+ > | a | b | > +----------+------+ > | a ???? b | a | > +----------+------+ > {noformat} > Notice, the character '😁' is "U+1F601 GRINNING FACE WITH SMILING EYES". It is outside of the BMP range supported by MariaDB character sets "utf8" and "ucs2". > The two columns worked differently: > - The ucs2 column correctly treated '😁' as an invalid sequence of four bytes, > converted it into four question marks, and appended the character 'b' after them. > - The utf8 column just stopped on this character and lost the trailing 'b'. > The warnings are also different for the two columns. > As "utf8" and "ucs2" are equitable character sets that support exactly the > same character range and repertoire (just with a different encoding), > it would be fair to expect the same behaviour of the two columns. > The column "a" exposes a better behaviour: it preserves as much > information as possible. > The column "b" should be fixed to work in the same way with "a". > The reason of the failure is that the conversion code that is activated > for the column "a" (with conversion) uses an mb_wc()..mb_wc() loop, while the > code activated for the column "b" (without conversion) copies only a well formed > prefix. > The code without conversion should not stop on a bad byte sequence > and try to copy as much data as possible, like the conversion code does. -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003)

2 1

[Maria-developers] Page encryption exception handling.
by Clemens Dörrhöfer 13 Oct '14

13 Oct '14

Hi community, I am working on implementing page encryption for XtraDB. Encryption itself works fine, now I am focusing on exception handling. My problem is that whenever page encryption is misconfigured it will crash the entire server, which is no acceptable behaviour. Example I encrypted a table with key id 42, I delete the key from the keypool thus I can no longer decrypt the table, because the appropriate key is missing. Ideally whenever a client tries to query data from an encrypted table, which cannot be decrypted, it will receive an error message informing the client that Table X could not be decrypted because key 42 is missing. Can you point me out to how to accomplish such a behaviour. Greetings Clemens Dörrhöfer

2 1

Re: [Maria-developers] [Commits] [svoj@mariadb.org: Rev 4437: Fixed mroonga build failure on Power8: define generic gcc version of in lp:maria/10.0]
by Kouhei Sutou 09 Oct '14

09 Oct '14

Hi Sergey, Thanks for the fix! I've merged your change into Groonga: https://github.com/groonga/groonga/commit/b3d38c5b9863cb7a9e93b9461c8b93fbf… Thanks, -- kou In <20141009083146.GA3231@june> "[Commits] [svoj(a)mariadb.org: Rev 4437: Fixed mroonga build failure on Power8: define generic gcc version of in lp:maria/10.0]" on Thu, 9 Oct 2014 12:31:47 +0400, Sergey Vojtovich <svoj(a)mariadb.org> wrote: > Hi Kentoku, > > FYI: I just pushed this fix to 10.0. There're also some test failures on Power8 > in mroonga/storage suite, mroonga/wrapper was 100% successful. > > Regards, > Sergey > > ----- Forwarded message from Sergey Vojtovich <svoj(a)mariadb.org> ----- > > Date: Thu, 9 Oct 2014 08:25:58 +0000 (UTC) > From: Sergey Vojtovich <svoj(a)mariadb.org> > To: commits(a)mariadb.org > Subject: [Commits] Rev 4437: Fixed mroonga build failure on Power8: define generic gcc version of in lp:maria/10.0 > > At lp:maria/10.0 > > ------------------------------------------------------------ > revno: 4437 > revision-id: svoj(a)mariadb.org-20141009082545-nt183b7y1gjihfjp > parent: psergey(a)askmonty.org-20141006112922-45pcytb993xcwh89 > committer: Sergey Vojtovich <svoj(a)mariadb.org> > branch nick: 10.0-mroonga > timestamp: Thu 2014-10-09 12:25:45 +0400 > message: > Fixed mroonga build failure on Power8: define generic gcc version of > GRN_SET_64BIT. > === modified file 'storage/mroonga/vendor/groonga/lib/groonga_in.h' > --- a/storage/mroonga/vendor/groonga/lib/groonga_in.h 2014-09-20 15:33:45 +0000 > +++ b/storage/mroonga/vendor/groonga/lib/groonga_in.h 2014-10-09 08:25:45 +0000 > @@ -505,6 +505,9 @@ typedef int grn_cond; > /* todo */ > # define GRN_SET_64BIT(p,v) \ > (void)atomic_swap_64(p, v) > +# elif defined(__ATOMIC_SEQ_CST) /* GCC atomic builtins */ > +# define GRN_SET_64BIT(p,v) \ > + __atomic_store_n(p, v, __ATOMIC_SEQ_CST) > # endif /* ATOMIC 64BIT SET */ > > # ifdef HAVE_MKOSTEMP > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits > > ----- End forwarded message ----- > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits

2 2

[Maria-developers] INET6, data type plugins
by Alexander Barkov 09 Oct '14

09 Oct '14

Hello all, I have mostly implemented the server side part of the fixed length 16-byte INET6 data type (only some clean-ups left). MDEV-274 The data type for IPv6/IPv4 addresses in MariaDB https://mariadb.atlassian.net/browse/MDEV-274 I made it in a very new style, which is ready to data type plugins, as described in: MDEV-4912 Add a plugin to field types (column types) https://mariadb.atlassian.net/browse/MDEV-4912 It is done by introducing a new abstract super-class Type_handler with a number of virtual public methods (around 50 public methods at this point). To implement a new data type one just needs to implement an instantiable class derived from Type_handler and register the new data type in a special global registry. Everything start to work automatically, including (but not limited to): - correct comparison - hybrid type functions like COALESCE, LEAST - aggregate functions MAX() and MIN() - CAST to the new type - ORDER BY, both for field and for items - conversion to the standard data types - indexing and index search The definition of the instantiable class that actually implements INET6 looks like this: class Type_handler_inet6: public Type_handler { private: class Field_inet6: public Field { ... }; class Arg_comparator_inet6: public Arg_comparator { ... }; class Item_typecast_inet6: public Item_func { ... }; public: // various access methods }; It looks very nice for me so far :) I have some questions about the remaining things: 1. In my current patch the new data type INET6 is not loaded dynamically from a shared library. It's implemented inside mysqld. Is it Okey? Or would we prefer to make it as a plugin from the beginning? (in this case there's some more work left: the dynamic loader code, client-server protocol, PS protocol). 2. In case of pluggable data types we'll need some changes in the client-server protocol, to transfer data type names (and probably some more metadata) to the client side in some textual form. Any ideas on that? 3. PS protocol obviously needs two ways to bind data: in text format and in binary format. For example, in case of INET6 it would be nice to be able to bind parameters and fetch values using: - text representation "ffff::ffff" - and binary representation 0xFFFF000000000000000000000000FFFF. I can see some ways to do this: a. Every pluggable data type uses its own type code in MYSQL_BIND: char ip6[16]= "\xff\xff...\xff"; MYSQL_BIND bind; bind.buffer_type= 128; // This is INET6 bind.buffer= ip6; bind.length= sizeof(ip6); but the code (128) may vary between servers, depending on how the plugin is installed. b. Or we could introduce a singe new type code MYSQL_TYPE_RAW: char ip6[16]= "\xff\xff...\xff"; MYSQL_BIND bind; bind.buffer_type= MYSQL_TYPE_RAW; bind.buffer= ip6; bind.length= sizeof(ip6); The server should be able to treat a single type code with different data types without any problems, because it knows the parameter context. But this will not allow to mix values of different types though, for example insert an INET4 value into an INET6 column (or vice verse) won't be possible. c. Or we could support both "native" type codes and MYSQL_TYPE_RAW. 4. Do we need the INET4 data type (fixed length, 4 bytes)? 5. Do we need a PostgreSQL-alike INET data type which is able to store both INET4 and INET6 values (I guess this one should be variable length: 4-bytes for INET4 and 16 bytes for INET6 values. Any feedback would be appreciated. The current patch (attached) will be polished. but the general idea should already be clear from the current version. Thanks.

3 4