- developers - lists.mariadb.org

[Maria-developers] MariaDB C-API and specific P_S tables
by Peter Laursen 24 Feb '14

24 Feb '14

THis is probably a question that Gerog Richter (or possibly Wlad) could reply to. I came across this bug report: http://bugs.mysql.com/bug.php?id=71801 .. files by former MySQL/Sun/Oracle supporter Veleriuy Kravtchuk We have a 64 bit C-application compiled with MariaDB C-API (on and for Windows 64 bit). I connect to MySQL 5.6.15 and execute SELECT * FROM `session_connect_attrs`; /* PROCESSLIST_ID ATTR_NAME ATTR_VALUE ORDINAL_POSITION -------------- --------------- ---------- ------------------ 2 _os Windows 0 2 _client_name libmariadb 1 2 _pid 5012 2 2 _thread 4640 3 2 _platform x86 4 2 _client_version 1.1.2 5 */ With the MySQL CLI 5.6.15, you will get /* PROCESSLIST_ID ATTR_NAME ATTR_VALUE ORDINAL_POSITION -------------- --------------- ---------- ------------------ 4 _os Win64 0 4 _client_name libmysql 1 4 _pid 4888 2 4 _thread 4320 3 4 _platform x86_64 4 4 program_name mysql 5 4 _client_version 5.6.15 6 */ The differences I notice are * Our application populates P_S with "Windows" - not "Win64" as the 5.6.15 CLI * Our application populates 'P_S with "x86" and not "x86_64". * Our application does not populate P_S with a "program_name". But it has a name of course! MySQL 5.6 docs at http://dev.mysql.com/doc/refman/5.6/en/performance-schema-connection-attrib… (as far as I can understand) that the client should use mysql_options() and mysql_options4() C API functions to populate P_S with all information. I have two questions: 1) Will the first 2 differences listed require a patch in our code or in libmariadb? 2) Obviosuly the lacking program name in the P_S tables is something that we will neeed to take care of ourselves. However I think that mysql_options4() was added in MySQL 5.6 and the MariaDB API does not supprot it currently (it is targeted to be compatible with Oracle 5.5 API, I think). And to populate the program name we may need to use mysql_options4(). Am I right?

2 4

[Maria-developers] Using MariaDB SQL for other storage engine
by Amith Nagarajappa 24 Feb '14

24 Feb '14

Hello, I am a database guy and want to plug mariaDB sql engine for other storage engine. If I want to plugin a storage engine with mariadb to use mariadb's SQL should i use handler interface in the storage/examples directory. I am not sure how to start with this interface implementation for MariaDB. Any help or docs or weblink on this would be helpful. Thanks and Regards, Amith

3 4

[Maria-developers] GROUP BY with non-grouped column returns NULL in some cases?
by Pavel Ivanov 24 Feb '14

24 Feb '14

Hi, I've noticed a difference in MariaDB's behavior compared to its documentation and compared to MySQL 5.1 (didn't check with MySQL 5.5 or 5.6 yet), and I wanted to know what's your opinion -- is it a bug or a conscious difference and documentation is just lagging. This doc https://mariadb.com/kb/en/select/#group-by says: "If you select a non-grouped column or a value computed from a non-grouped column, it is undefined which row the returned value is taken from". I believe this implies that selected value must be returned from a row group, not just something random. But I have a scenario when NULL is returned instead of real values. I understand that this is completely undefined behavior in SQL standard, but MySQL and MariaDB seem to have a documented support for this, that's why I'm asking. Here's the mysql session log: Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 5 Server version: 10.0.8-MariaDB-log Source distribution Copyright (c) 2000, 2013, Oracle, Monty Program Ab and others. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. MariaDB [(none)]> create database test; Query OK, 1 row affected (0.00 sec) MariaDB [(none)]> use test; Database changed MariaDB [test]> CREATE TABLE t (id bigint(20), name varchar(30), parent bigint(20), primary key (id)) ENGINE=InnoDB; Query OK, 0 rows affected (0.27 sec) MariaDB [test]> INSERT INTO t values (1, 'a', NULL), (4, 'b', 1), (5, 'c', 4), (10, 'd', 1); Query OK, 4 rows affected (0.04 sec) Records: 4 Duplicates: 0 Warnings: 0 MariaDB [test]> SELECT t1.id, t1.name, t2.id, t2.name FROM t t1 LEFT OUTER JOIN t t2 ON (t1.parent = t2.id and t2.name <> 'a') GROUP BY t1.id; +----+------+------+------+ | id | name | id | name | +----+------+------+------+ | 1 | a | NULL | NULL | | 4 | b | NULL | NULL | | 5 | c | NULL | NULL | | 10 | d | NULL | NULL | +----+------+------+------+ 4 rows in set (0.00 sec) MariaDB [test]> SELECT t1.id, t1.name, t2.id, t2.name FROM t t1 LEFT OUTER JOIN t t2 ON (t1.parent = t2.id and t2.name <> 'a'); +----+------+------+------+ | id | name | id | name | +----+------+------+------+ | 1 | a | NULL | NULL | | 4 | b | NULL | NULL | | 5 | c | 4 | b | | 10 | d | NULL | NULL | +----+------+------+------+ 4 rows in set (0.00 sec) MariaDB [test]> SELECT t1.id, t1.name, t2.id, t2.name FROM t t1 LEFT OUTER JOIN t t2 ON (t1.parent = t2.id and t2.name <> 'a') GROUP BY t1.id, t1.name; +----+------+------+------+ | id | name | id | name | +----+------+------+------+ | 1 | a | NULL | NULL | | 4 | b | NULL | NULL | | 5 | c | 4 | b | | 10 | d | NULL | NULL | +----+------+------+------+ 4 rows in set (0.00 sec) MariaDB [test]> SELECT t1.id, t1.name, t2.id, t2.name FROM t t1 LEFT OUTER JOIN t t2 ON (t1.parent = t2.id) GROUP BY t1.id; +----+------+------+------+ | id | name | id | name | +----+------+------+------+ | 1 | a | NULL | NULL | | 4 | b | 1 | a | | 5 | c | 4 | b | | 10 | d | 1 | a | +----+------+------+------+ 4 rows in set (0.00 sec) As you see the first query returns NULLs instead of real values which can be seen in second query without GROUP BY. And if I add another column to GROUP BY or remove one condition from JOIN I can see real values again. I've also tested with the table without PRIMARY KEY and it also returned real values in the first query. So is this a bug? Thank you, Pavel

2 2

[Maria-developers] Interested in GSOC
by Gopichand Paturi 24 Feb '14

24 Feb '14

Dear Sir/Madam, I am a student of Computer Science at BITS PILANI, India. I am interested in working as a developer for MariaDB in 2014. Can someone please let me know the ideas and proposed projects this year? Someone please guide me what to familiarize myself to be helpful and contribute for MariaDB. I am C++ developer....with good knowledge in Data Structures,Algorithms,Database Systems. Thanks& Regards, Gopichand.

2 1

[Maria-developers] Proposal for: MDEV-5244: Make extended_keys=ON by default in 10.0
by Sergey Petrunia 22 Feb '14

22 Feb '14

Hello, I was looking at how to make extended_keys=ON by default in 10.0. There haven't been any bugs in the feature for quite some time, but we still have a question of how to be with the testsuite (do we run it with extended_keys=OFF, the old way, or with extended_keys=ON, the new way? both approaches have their advantages) So, I've just changed the default, and ran the default set of test suites. A total 12 tests fail, of those only 3 tests fail in the way that indicates that extended keys feature had an impact on the testcase (e.g. it got a different execution plan and probably no longer tests what it used to test). For these, I intend to place set opimizer_switch='extended_keys=off' <testcase> set opimizer_switch='extended_keys=on' around the affected testcases. The cases are: range_vs_index_merge_innodb - switch @@optimizer_switch (~3 testcases) index_merge_innodb - switch @@optimizer_switch (1 testcase) innodb.innodb_mysql -switch @@optimizer_switch (1 testcase) Any comments/input on this? BR Sergei -- Sergei Petrunia, Software Developer MariaDB | Skype: sergefp | Blog: http://s.petrunia.net/blog

2 1

[Maria-developers] [svoj@mariadb.org: [Commits] Rev 4006: MDEV-5674 - Performance: my_hash_sort_bin is called too often in lp:maria/10.0]
by Sergey Vojtovich 20 Feb '14

20 Feb '14

Hi! JFYI: we discussed this during the Barcelona meeting. I will submit table cache part separately (not to overcomplicate this patch). Reviewers are very welcome. Regards, Sergey ----- Forwarded message from Sergey Vojtovich <svoj(a)mariadb.org> ----- Date: Fri, 14 Feb 2014 12:25:50 +0000 (UTC) From: Sergey Vojtovich <svoj(a)mariadb.org> To: commits(a)mariadb.org Subject: [Commits] Rev 4006: MDEV-5674 - Performance: my_hash_sort_bin is called too often in lp:maria/10.0 At lp:maria/10.0 ------------------------------------------------------------ revno: 4006 revision-id: svoj(a)mariadb.org-20140214122541-tenvcllk760deai8 parent: svoj(a)mariadb.org-20140213071355-psk0jpsarx2u7myc committer: Sergey Vojtovich <svoj(a)mariadb.org> branch nick: 10.0 timestamp: Fri 2014-02-14 16:25:41 +0400 message: MDEV-5674 - Performance: my_hash_sort_bin is called too often - reduced number of my_hash_sort_bin() calls from 4 to 2 per query. Let MDL subsystem use pre-calculated hash value for hash inserts and deletes. - reduced number of memory accesses done by my_hash_sort_bin() === modified file 'include/hash.h' --- a/include/hash.h 2013-09-14 01:09:36 +0000 +++ b/include/hash.h 2014-02-14 12:25:41 +0000 @@ -86,7 +86,13 @@ uchar *my_hash_first_from_hash_value(con uchar *my_hash_next(const HASH *info, const uchar *key, size_t length, HASH_SEARCH_STATE *state); my_bool my_hash_insert(HASH *info, const uchar *data); +my_bool my_hash_insert_using_hash_value(HASH *info, + my_hash_value_type hash_value, + const uchar *record); my_bool my_hash_delete(HASH *hash, uchar *record); +my_bool my_hash_delete_using_hash_value(HASH *hash, + my_hash_value_type hash_value, + uchar *record); my_bool my_hash_update(HASH *hash, uchar *record, uchar *old_key, size_t old_key_length); void my_hash_replace(HASH *hash, HASH_SEARCH_STATE *state, uchar *new_row); === modified file 'mysys/hash.c' --- a/mysys/hash.c 2013-09-14 01:09:36 +0000 +++ b/mysys/hash.c 2014-02-14 12:25:41 +0000 @@ -380,6 +380,15 @@ static int hashcmp(const HASH *hash, HAS my_bool my_hash_insert(HASH *info, const uchar *record) { + return my_hash_insert_using_hash_value(info, rec_hashnr(info, record), + record); +} + + +my_bool my_hash_insert_using_hash_value(HASH *info, + my_hash_value_type hash_value, + const uchar *record) +{ int flag; size_t idx,halfbuff,first_index; my_hash_value_type hash_nr; @@ -480,7 +489,7 @@ my_bool my_hash_insert(HASH *info, const } /* Check if we are at the empty position */ - idx= my_hash_mask(rec_hashnr(info, record), info->blength, info->records + 1); + idx= my_hash_mask(hash_value, info->blength, info->records + 1); pos=data+idx; if (pos == empty) { @@ -528,6 +537,15 @@ my_bool my_hash_insert(HASH *info, const my_bool my_hash_delete(HASH *hash, uchar *record) { + return my_hash_delete_using_hash_value(hash, rec_hashnr(hash, record), + record); +} + + +my_bool my_hash_delete_using_hash_value(HASH *hash, + my_hash_value_type hash_value, + uchar *record) +{ uint pos2,idx,empty_index; my_hash_value_type pos_hashnr, lastpos_hashnr; size_t blength; @@ -539,7 +557,7 @@ my_bool my_hash_delete(HASH *hash, uchar blength=hash->blength; data=dynamic_element(&hash->array,0,HASH_LINK*); /* Search after record with key */ - pos= data + my_hash_mask(rec_hashnr(hash, record), blength, hash->records); + pos= data + my_hash_mask(hash_value, blength, hash->records); gpos = 0; while (pos->data != record) === modified file 'sql/mdl.cc' --- a/sql/mdl.cc 2013-12-10 15:31:04 +0000 +++ b/sql/mdl.cc 2014-02-14 12:25:41 +0000 @@ -124,15 +124,9 @@ class MDL_map_partition public: MDL_map_partition(); ~MDL_map_partition(); - inline MDL_lock *find_or_insert(const MDL_key *mdl_key, - my_hash_value_type hash_value); - unsigned long get_lock_owner(const MDL_key *key, - my_hash_value_type hash_value); + inline MDL_lock *find_or_insert(const MDL_key *mdl_key); + unsigned long get_lock_owner(const MDL_key *key); inline void remove(MDL_lock *lock); - my_hash_value_type get_key_hash(const MDL_key *mdl_key) const - { - return my_calc_hash(&m_locks, mdl_key->ptr(), mdl_key->length()); - } private: bool move_from_hash_to_lock_mutex(MDL_lock *lock); /** A partition of all acquired locks in the server. */ @@ -846,11 +840,10 @@ MDL_lock* MDL_map::find_or_insert(const return lock; } - my_hash_value_type hash_value= m_partitions.at(0)->get_key_hash(mdl_key); - uint part_id= hash_value % mdl_locks_hash_partitions; + uint part_id= mdl_key->hash_value() % mdl_locks_hash_partitions; MDL_map_partition *part= m_partitions.at(part_id); - return part->find_or_insert(mdl_key, hash_value); + return part->find_or_insert(mdl_key); } @@ -863,15 +856,14 @@ MDL_lock* MDL_map::find_or_insert(const @retval NULL - Failure (OOM). */ -MDL_lock* MDL_map_partition::find_or_insert(const MDL_key *mdl_key, - my_hash_value_type hash_value) +MDL_lock* MDL_map_partition::find_or_insert(const MDL_key *mdl_key) { MDL_lock *lock; retry: mysql_mutex_lock(&m_mutex); if (!(lock= (MDL_lock*) my_hash_search_using_hash_value(&m_locks, - hash_value, + mdl_key->hash_value(), mdl_key->ptr(), mdl_key->length()))) { @@ -902,7 +894,9 @@ MDL_lock* MDL_map_partition::find_or_ins lock= MDL_lock::create(mdl_key, this); } - if (!lock || my_hash_insert(&m_locks, (uchar*)lock)) + if (!lock || my_hash_insert_using_hash_value(&m_locks, + mdl_key->hash_value(), + (uchar*) lock)) { if (unused_lock) { @@ -1023,10 +1017,9 @@ MDL_map::get_lock_owner(const MDL_key *m } else { - my_hash_value_type hash_value= m_partitions.at(0)->get_key_hash(mdl_key); - uint part_id= hash_value % mdl_locks_hash_partitions; + uint part_id= mdl_key->hash_value() % mdl_locks_hash_partitions; MDL_map_partition *part= m_partitions.at(part_id); - res= part->get_lock_owner(mdl_key, hash_value); + res= part->get_lock_owner(mdl_key); } return res; } @@ -1034,15 +1027,14 @@ MDL_map::get_lock_owner(const MDL_key *m unsigned long -MDL_map_partition::get_lock_owner(const MDL_key *mdl_key, - my_hash_value_type hash_value) +MDL_map_partition::get_lock_owner(const MDL_key *mdl_key) { MDL_lock *lock; unsigned long res= 0; mysql_mutex_lock(&m_mutex); lock= (MDL_lock*) my_hash_search_using_hash_value(&m_locks, - hash_value, + mdl_key->hash_value(), mdl_key->ptr(), mdl_key->length()); if (lock) @@ -1085,7 +1077,8 @@ void MDL_map::remove(MDL_lock *lock) void MDL_map_partition::remove(MDL_lock *lock) { mysql_mutex_lock(&m_mutex); - my_hash_delete(&m_locks, (uchar*) lock); + my_hash_delete_using_hash_value(&m_locks, lock->key.hash_value(), + (uchar*) lock); /* To let threads holding references to the MDL_lock object know that it was moved to the list of unused objects or destroyed, we increment the version === modified file 'sql/mdl.h' --- a/sql/mdl.h 2013-12-16 08:26:20 +0000 +++ b/sql/mdl.h 2014-02-14 12:25:41 +0000 @@ -347,12 +347,17 @@ class MDL_key m_ptr - 1); m_length= static_cast<uint16>(strmake(m_ptr + m_db_name_length + 2, name, NAME_LEN) - m_ptr + 1); + ulong nr2= 4; + m_hash_value= 1; + my_charset_bin.coll->hash_sort(&my_charset_bin, (uchar*) m_ptr, m_length, + &m_hash_value, &nr2); } void mdl_key_init(const MDL_key *rhs) { memcpy(m_ptr, rhs->m_ptr, rhs->m_length); m_length= rhs->m_length; m_db_name_length= rhs->m_db_name_length; + m_hash_value= rhs->m_hash_value; } bool is_equal(const MDL_key *rhs) const { @@ -392,10 +397,15 @@ class MDL_key { return & m_namespace_to_wait_state_name[(int)mdl_namespace()]; } + ulong hash_value() const + { + return m_hash_value; + } private: uint16 m_length; uint16 m_db_name_length; + ulong m_hash_value; char m_ptr[MAX_MDLKEY_LENGTH]; static PSI_stage_info m_namespace_to_wait_state_name[NAMESPACE_END]; private: === modified file 'strings/ctype-bin.c' --- a/strings/ctype-bin.c 2013-11-20 11:05:39 +0000 +++ b/strings/ctype-bin.c 2014-02-14 12:25:41 +0000 @@ -277,7 +277,9 @@ void my_hash_sort_8bit_bin(CHARSET_INFO ulong *nr1, ulong *nr2) { const uchar *pos = key; - + ulong tmp1= *nr1; + ulong tmp2= *nr2; + /* Remove trailing spaces. We have to do this to be able to compare 'A ' and 'A' as identical @@ -286,10 +288,13 @@ void my_hash_sort_8bit_bin(CHARSET_INFO for (; pos < (uchar*) key ; pos++) { - nr1[0]^=(ulong) ((((uint) nr1[0] & 63)+nr2[0]) * - ((uint)*pos)) + (nr1[0] << 8); - nr2[0]+=3; + tmp1^= (ulong) ((((uint) tmp1 & 63) + tmp2) * + ((uint) *pos)) + (tmp1 << 8); + tmp2+= 3; } + + *nr1= tmp1; + *nr2= tmp2; } @@ -297,15 +302,20 @@ void my_hash_sort_bin(CHARSET_INFO *cs _ const uchar *key, size_t len,ulong *nr1, ulong *nr2) { const uchar *pos = key; - + ulong tmp1= *nr1; + ulong tmp2= *nr2; + key+= len; - + for (; pos < (uchar*) key ; pos++) { - nr1[0]^=(ulong) ((((uint) nr1[0] & 63)+nr2[0]) * - ((uint)*pos)) + (nr1[0] << 8); - nr2[0]+=3; + tmp1^= (ulong) ((((uint) tmp1 & 63) + tmp2) * + ((uint) *pos)) + (tmp1 << 8); + tmp2+= 3; } + + *nr1= tmp1; + *nr2= tmp2; } _______________________________________________ commits mailing list commits(a)mariadb.org https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits ----- End forwarded message -----

1 1

[Maria-developers] Buildbot: 10.0-mroonga and 10.0-spider
by Alexander Barkov 18 Feb '14

18 Feb '14

Hello Daniel, Kristian, Sergei, Can you please add 10.0-mroonga and 10.0-spider into BuildBot? Thanks.

1 1

[Maria-developers] Fwd: [debian-mysql] mariadb-5.5_5.5.35-1_amd64.changes ACCEPTED into unstable
by Otto Kekäläinen 18 Feb '14

18 Feb '14

FYI, my 5.5.35 just got uploaded to Debian by James Page. Next time .36 is released I'll try to get the upload done in 1-2 days after release. This of course if you help me a little bit by setting up the discussed build bots so that Debian is part of your QA and if you accept to merge my debian/* upstream so that there will be less "delta" per release to look at. ---------- Forwarded message ---------- From: Debian FTP Masters <ftpmaster(a)ftp-master.debian.org> Date: 2014-02-18 11:21 GMT+02:00 Subject: [debian-mysql] mariadb-5.5_5.5.35-1_amd64.changes ACCEPTED into unstable To: Debian MySQL Maintainers <pkg-mysql-maint(a)lists.alioth.debian.org>, James Page <jamespage(a)debian.org> Accepted: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Format: 1.8 Date: Mon, 17 Feb 2014 16:51:52 +0000 Source: mariadb-5.5 Binary: libmariadbclient18 libmariadbd-dev libmariadbclient-dev mariadb-common mariadb-client-core-5.5 mariadb-client-5.5 mariadb-server-core-5.5 mariadb-test-5.5 mariadb-server-5.5 mariadb-server mariadb-client mariadb-test Architecture: source all amd64 Version: 5.5.35-1 Distribution: unstable Urgency: low Maintainer: Debian MySQL Maintainers <pkg-mysql-maint(a)lists.alioth.debian.org> Changed-By: James Page <jamespage(a)debian.org> Description: libmariadbclient-dev - MariaDB database development files libmariadbclient18 - MariaDB database client library libmariadbd-dev - MariaDB embedded database development files mariadb-client - MariaDB database client (metapackage depending on the latest vers mariadb-client-5.5 - MariaDB database client binaries mariadb-client-core-5.5 - MariaDB database core client binaries mariadb-common - MariaDB common metapackage mariadb-server - MariaDB database server (metapackage depending on the latest vers mariadb-server-5.5 - MariaDB database server binaries mariadb-server-core-5.5 - MariaDB database core server files mariadb-test - MariaDB database regression test suite (metapackage for the lates mariadb-test-5.5 - MariaDB database regression test suite Closes: 732967 733002 734426 735284 736480 737597 Changes: mariadb-5.5 (5.5.35-1) unstable; urgency=low . [ Otto Kekäläinen ] * New upstream release, fixing the following security issues: - Buffer overflow in client/mysql.cc (Closes: #737597). - CVE-2014-0001 - http://www.oracle.com/technetwork/topics/security/cpujan2014-1972949.html - CVE-2013-5891 - CVE-2013-5908 - CVE-2014-0386 - CVE-2014-0393 - CVE-2014-0401 - CVE-2014-0402 - CVE-2014-0412 - CVE-2014-0420 - CVE-2014-0437 * Upstream https://mariadb.atlassian.net/browse/MDEV-4902 fixes compatibility with Bison 3.0 (Closes: #733002) * Updated Russian debconf translation (Closes: #734426) * Updated Japanese debconf translation (Closes: #735284) * Updated French debconf translation (Closes: #736480) * Renamed SONAME properly (Closes: #732967) -- Otto Kekäläinen +358 44 566 2204 http://seravo.fi/

1 0

[Maria-developers] Plugin init()/deinit() call convention
by Sergey Vojtovich 17 Feb '14

17 Feb '14

Hi Sergei, IIRC during early days of plugins framework there was a strict convention when plugin deinit() function must be called. Specifically only after successful call to init() function. I'm now fixing a bug in related code (5.5) and noticed that: - deinit() may (or may not) be called without prior call to init() - deinit() may (or may not) be called with prior failed call to init() Do you think the above is correct? Thanks, Sergey

2 1

[Maria-developers] system variable value
by Rich Prohaska 17 Feb '14

17 Feb '14

I have a ULONGLONG system variable in my storage engine, and I set its value to '-1' on the mysqld command line. Its value gets set to all ones without any warnings or errors. The number conversion functions just cast between longlong and ulonglong types without regard to the sign of the number. Is this correct behavior?

2 1