September 2013 - developers - lists.mariadb.org

[Maria-developers] Extendet FederatedX storage engine to support updating Sphinx RT indexes
by Markus Lidel 11 Jun '14

11 Jun '14

Hello, the Sphinx Search engine has RT indexes, which could be updated with a subset of SQL commands, refered to as SphinxQL: http://sphinxsearch.com/docs/2.1.1/sphinxql-reference.html Because the FederatedX storage engine uses some commands, which Sphinx does not know, here is a patch, wich extends the FederatedX storage engine to support this subset of commands. Now its possible to update Sphinx RT indexes directly from MariaDB. To create a connection to the RT index in the Sphinx default RT-index: index rt { type = rt path = @CONFDIR@/data/rt rt_field = title rt_field = content rt_attr_string = title rt_attr_string = content rt_attr_uint = gid } you just have to call: CREATE TABLE `rt` ( `id` BIGINT UNSIGNED NOT NULL, `title` TEXT, `content` TEXT, `gid` INT UNSIGNED ) ENGINE=FEDERATED CONNECTION='sphinxql://root@localhost:9306/rt'; It would be great, if the attached patch could be integrated into MariaDB. If you have any suggestions, please feel free to contact me. Best regards, Markus Lidel

4 8

Re: [Maria-developers] MDEV-4956 - Reduce usage of LOCK_open: TABLE_SHARE::tdc.used_tables
by Sergei Golubchik 03 Dec '13

03 Dec '13

Hi, Sergey! On Aug 27, Sergey Vojtovich wrote: > At lp:maria/10.0 > > ------------------------------------------------------------ > revno: 3807 > revision-id: svoj(a)mariadb.org-20130827121233-xh1uyhgfwbhedqyf > parent: jplindst(a)mariadb.org-20130823060357-pww92qxla7o8iir7 > committer: Sergey Vojtovich <svoj(a)mariadb.org> > branch nick: 10.0 > timestamp: Tue 2013-08-27 16:12:33 +0400 > message: > MDEV-4956 - Reduce usage of LOCK_open: TABLE_SHARE::tdc.used_tables > > - tc_acquire_table and tc_release_table do not access > TABLE_SHARE::tdc.used_tables anymore > - in tc_acquire_table(): release LOCK_tdc after we relase LOCK_open > (saves a few CPU cycles in critical section) > - in tc_release_table(): if we reached table cache threshold, evict > to-be-released table without moving it to unused_tables. unused_tables > must be empty at this point (added assertion). I don't understand what you're doing here, could you elaborate? E.g. explain in a changeset comment what you've done, why you introduced a new list for tables (all_share), what is its semantics, etc. Regards, Sergei

3 12

[Maria-developers] Wrong SQL thread position reporting after IO thread restart
by Pavel Ivanov 11 Oct '13

11 Oct '13

Krisitan, I found what I think is a bug in IO and SQL thread accounting. How to reproduce: 1) Set up two servers S1 and S2. S1 is a master, S2 is slave with master_using_gtid = current_pos. 2) Execute some transactions on the master, e.g. create database d; create table d.t (n int); insert into d.t values (1); 3) Both servers are at 0-1-3 now, SHOW SLAVE STATUS on S2 shows Read_Master_Log_Pos equals to Exec_Master_Log_Pos. 4) Execute STOP SLAVE IO_THREAD on S2. 5) S2 reports in the logs: "Slave I/O thread exiting, ... GTID position 0-1-2". So IO thread didn't realize that it received full transaction for 0-1-3 even though it didn't receive next GTID event. 6) Execute START SLAVE IO_THREAD on S2. 7) At this point SHOW SLAVE STATUS on S2 shows Read_Master_Log_Pos the same as in step 3, but Exec_Master_Log_Pos is now less than in step 3 as if SQL thread didn't catch up with IO thread yet. But despite both threads running and no more transactions executed on master Exec_Master_Log_Pos doesn't change and doesn't become equal to Read_Master_Log_Pos. This apparently happens because IO thread restarts from one transaction behind, adds to relay log Rotate event that master sends with the position of that transaction, but then doesn't add any events for the transaction because it knows they already were added into relay log. I think both problems are bugs. And although after fixing the first it would be really hard (if possible) to reproduce second, I'd think the reporting of SQL thread's position still should be fixed. Thank you, Pavel

2 2

[Maria-developers] Questions re MDEV-4736 and MDEV-4739 (was Re: Spider's installation sql file)
by Sergey Vojtovich 07 Oct '13

07 Oct '13

Hi Kentoku, I'm adding MariaDB developers to CC. On Thu, Sep 19, 2013 at 01:19:13AM +0900, kentoku wrote: > Hi Sergey, > > > But what kind of errors are possible in your case? Other storage engines > doesn't > seem to suffer from this API violation. > > Spider support bulk updating and deleting for avoiding network roundtrip > between data node. Some times, last bulk updating is executed in rnd_end() > function. So rnd_end() has possibility getting errors from data node. I'm afraid fixing rnd_end() callers in the server may stall for a long time. Is it accaptable for spider to use bulk updates and deletes API instead (see handler.h: start_bulk_update/start_bulk_delete). > By the way, about MDEV-4739. I get the following stack trace. > Program received signal SIGSEGV, Segmentation fault. > 0x00000000005eabf6 in thd_wait_begin (thd=0x29da060, > wait_type=10) > at /ssd1/mariadb-10.0.4/sql/sql_class.cc:4277 > 4277 MYSQL_CALLBACK(thd->scheduler, thd_wait_begin, (thd, wait_type)); > (gdb) print thd > $1 = (THD *) 0x29da060 > (gdb) bt > #0 0x00000000005eabf6 in thd_wait_begin ( > thd=0x29da060, wait_type=10) > at /ssd1/mariadb-10.0.4/sql/sql_class.cc:4277 > #1 0x000000000072a114 in scheduler_wait_sync_begin () > at /ssd1/mariadb-10.0.4/sql/scheduler.cc:59 > #2 0x0000000000d6dc20 in my_sync (fd=23, my_flags=0) > at /ssd1/mariadb-10.0.4/mysys/my_sync.c:76 > #3 0x0000000000d6b54f in my_msync (fd=23, > addr=0x7ffff7ff4000, len=4096, flags=4) > at /ssd1/mariadb-10.0.4/mysys/my_mmap.c:27 > #4 0x00000000008bea03 in TC_LOG_MMAP::open ( > this=0x16e6a00, opt_name=0xe19c87 "tc.log") > at /ssd1/mariadb-10.0.4/sql/log.cc:7735 > #5 0x00000000005751cb in init_server_components () > at /ssd1/mariadb-10.0.4/sql/mysqld.cc:4797 > #6 0x0000000000575a07 in mysqld_main (argc=30, > argv=0x1f204d0) > at /ssd1/mariadb-10.0.4/sql/mysqld.cc:5208 > #7 0x000000000056d884 in main (argc=11, > argv=0x7fffffffe3a8) > at /ssd1/mariadb-10.0.4/sql/main.cc:25 > (gdb) print thd->scheduler > $2 = (scheduler_functions *) 0x8f8f8f8f8f8f8f8f > (gdb) print thd_wait_begin > $3 = {void (THD *, > int)} 0x5eaba4 <thd_wait_begin(THD*, int)> > (gdb) print wait_type > $4 = 10 > > It is looks that "thd->scheduler" is not initialized. What do you think? > Must storage engine set it? Nope, it looks like a bug in thread pool. MDEV-4739 has different trace, how did you get this one? Just executed given test? Regards, Sergey

3 16

[Maria-developers] Replicating same server_id problem
by Pavel Ivanov 01 Oct '13

01 Oct '13

Kristian, Currently MariaDB (as well as MySQL of all previous versions) has a very big problem related to replicating same server_id. There is --replicate-same-server-id flag which as I understand (when set to 0) controls two things: 1) It doesn't allow slave to connect to a master with the same server_id. 2) Slave ignores all binlog events in the replication stream that have the same server_id as slave. And this flag cannot be set to 1 when --log-slave-updates is used. And that is a big problem. Consider the following scenario: let's say we have two servers S1 (master) and S2 (slave). Let's say at some moment in time they are completely in sync and you bring down S2 to take cold backup (you can even include binlogs in it). Then you bring it back up, S1 is still master. Now you execute some transactions, then you do a failover, make S2 master and execute some more transactions. Then you bring down S1, restore it from the backup taken earlier and connect to replicate from S2 again. At this point S1 will have to reply some transaction that have the same server_id as S1 has, but S1 doesn't have those transactions yet, so it shouldn't skip them, but it will. What do you think about how this should be fixed? As I understand you explicitly wanted to support replication cycles, so you still want the skipping of transactions with the same server_id to exist. But the situation above is a valid production use case. Maybe in GTID world it can be solved better? E.g. if transaction has the same server_id, but the GTID wasn't applied yet then it shouldn't be skipped? Thank you, Pavel

4 8

[Maria-developers] regexp review
by Oleksandr Byelkin 30 Sep '13

30 Sep '13

Hi! Hi! Everything is OK. But there are some small issues: --- mysql-test/include/ctype_utf8mb4.inc 2010-03-05 08:17:19 +0000 +++ mysql-test/include/ctype_utf8mb4.inc 2013-09-26 14:02:17 +0000 @@ -234,15 +234,15 @@ set names utf8mb4; set names utf8mb4; # This should return TRUE -select 'вася' rlike '[[:<:]]вася[[:>:]]'; -select 'вася ' rlike '[[:<:]]вася[[:>:]]'; -select ' вася' rlike '[[:<:]]вася[[:>:]]'; -select ' вася ' rlike '[[:<:]]вася[[:>:]]'; +select 'вася' rlike '\\bвася\\b'; +select 'вася ' rlike '\\bвася\\b'; +select ' вася' rlike '\\bвася\\b'; +select ' вася ' rlike '\\bвася\\b'; Is above unsupported pattern? === modified file 'sql/mysqld.cc' --- sql/mysqld.cc 2013-09-18 11:07:31 +0000 +++ sql/mysqld.cc 2013-09-26 14:02:17 +0000 @@ -1898,7 +1898,7 @@ void clean_up(bool print_message) delete global_rpl_filter; end_ssl(); vio_end(); - my_regex_end(); + //my_regex_end(); #if defined(ENABLED_DEBUG_SYNC) /* End the debug sync facility. See debug_sync.cc. */ debug_sync_end(); @@ -3904,10 +3904,10 @@ static int init_common_variables() return 1; item_init(); #ifndef EMBEDDED_LIBRARY - my_regex_init(&my_charset_latin1, check_enough_stack_size); + //my_regex_init(&my_charset_latin1, check_enough_stack_size); my_string_stack_guard= check_enough_stack_size; #else - my_regex_init(&my_charset_latin1, NULL); + //my_regex_init(&my_charset_latin1, NULL); #endif /* Process a comma-separated character set list and choose Remove it please (I think it was just forgotten).

2 1

[Maria-developers] Empty directory in mariadb's datadir is not handled correctly
by Honza Horak 30 Sep '13

30 Sep '13

Hi guys, quite tricky problem came up after a directory has been created by something else than mysqld in the datadir. Suppose we use datadir=/var/lib/mysql. When we create any file in that directory by hand (or some program does it, which is not just imagination, it really happens), e.g. /var/lib/mysql/.local, MariaDB (Oracle's MySQL will behave the same probably) server thinks this directory is a database, which is wrong. Then we get some weird messages and outputs: # mysqlshow +--------------------+ | Databases | +--------------------+ | information_schema | | #mysql50#.local | | mysql | | performance_schema | | test | +--------------------+ # mysqlcheck -A mysqlcheck: Got error: 1102: Incorrect database name '#mysql50#.local' when selecting the database I believe this is not how the things should work. I'm not sure, but I'd expect there is still some information about existing databases in the information_schema table. So IMHO the server/tool should always fetch such info in addition to pure testing if the database directory exists. Or is such testing simply impossible? Regards, Honza

2 1

Re: [Maria-developers] [All] MariaDB's best (download) month ever
by Kristian Nielsen 29 Sep '13

29 Sep '13

Colin Charles <byte(a)mariadb.com> writes: > It's 17.09 today, and we've hit 50,160 total downloads (this number is > considered company-internal, we don't share this number externally). "company-internal"? What have you been smoking? Colin, you are supposed to be the community guy for crying out lout :-( This is *completely*, utterly unacceptable. MariaDB is a community project. We are about openness. We are about working together, all of us, to create a better world. We are *not* some individuals or companys playground. So no, the community's usage numbers of its own project is not "internal", thank you very much. So please cut out the crap already. - Kristian.

6 11

[Maria-developers] Prep for MariaDB Galera 5.5.33a
by Daniel Bartholomew 28 Sep '13

28 Sep '13

All, I've begun the preparations for the MariaDB Galera 5.5.33a release. Draft release notes and changelog are here: - https://mariadb.com/kb/en/mariadb-galera-5533a-release-notes/ - https://mariadb.com/kb/en/mariadb-galera-5533a-changelog/ Please make necessary changes/additions/etc... to the above pages (or let me know). Thanks. -- Daniel Bartholomew, Sr Tech Writer / Sys Admin MariaDB | t: +1 919-809-6069 | Skype: daniel-bartholomew

1 0

[Maria-developers] MySQL bug affects mariaDB
by Peter Laursen 27 Sep '13

27 Sep '13

I think the 'mysql' client with MariaDB has a similar problem as it has with MySQL - refer? http://bugs.mysql.com/bug.php?id=70443 MariaDB installer installs my.ini to /datadir - what with MariaDB is a subfolder to /basedir. The server will look for it correctly as the registry key defining the service has a --default-file specification. But the command line clients don't read the registry key. @Wlad? -- Peter -- Webyog

3 7