- developers - lists.mariadb.org

[Maria-developers] Bazaar-problem
by AugustQ 06 Sep '13

06 Sep '13

Hi, I'm struggling with Bazaar. Maybe you can help me. I've made some modifications to the source-code and committed these to the local repository. Next I tried to upload them via push. This is what I receive: Cannot lock LockDir(chroot-70341520:///% 2Bbranch/maria/.bzr/branch/lock): Transport operation not possible: readonly transport As I'm using Eclipse I found this within Eclipse on the console: Error while executing push Cannot lock LockDir(chroot-70341520:///% 2Bbranch/maria/.bzr/branch/lock): Transport operation not possible: readonly transport ------ XmlRpcClient Executing: push "bzr+ssh://bazaar.launchpad.net/+branch/maria/" Can you please give me a tip. Thanks AugustQ

5 9

[Maria-developers] Incorrect format description event skipping in queue_event()
by Pavel Ivanov 04 Sep '13

04 Sep '13

Kristian, You have the following comment in the queue_event() in sql/slave.cc: /* Do not queue any format description event that we receive after a reconnect where we are skipping over a partial event group received before the reconnect. (If we queued such an event, and it was the first format_description event after master restart, the slave SQL thread would think that the partial event group before it in the relay log was from a previous master crash and should be rolled back). */ I don't understand which failure scenario you are talking about here and I claim that this bypassing of queuing into relay log is incorrect. My reasoning is: relay log can always be rotated in the middle of event group. During rotation when new relay log file is created format description event is always written at the beginning of the file. So in this case SQL thread will see format description event in the middle of event group and according to my testing it doesn't rollback the active transaction because of that. So when do you think it will rollback? Now, you may say that there's no problem in skipping the format description event because everything works. But it's only for now. When IO thread is reconnecting it rotates relay log and as I said it writes format description event at the beginning of the new file. But it writes an event that it created itself, i.e. not the one that master have sent. And as format description event from master is not written into relay log SQL thread from this point on starts to use format description generated by slave which may be different from the one generated by master. It may lead to a broken replication and SQL thread may not recognize events in the relay log if difference between master's and slave's format descriptions is significant. Another somewhat related question: Gtid_log_event::peek() (as well as Gtid_log_event constructor from const char* buf) is implemented with assumption that Format_description_log_event::common_header_len is always equal to LOG_EVENT_HEADER_LEN. While currently it's true I believe common_header_len can be increased in future MariaDB versions. And when it happens old slave won't be able to replicate from a new master. I'd think this is a bug. What do you think? Pavel

2 10

Re: [Maria-developers] Rev 3876: MDEV-4472 Audit-plugin. Server-related part of the task
by Sergei Golubchik 04 Sep '13

04 Sep '13

Hi, Holyfoot! Mostly ok. There are few comments, see below: On Sep 03, holyfoot(a)askmonty.org wrote: > ------------------------------------------------------------ > revno: 3876 > revision-id: holyfoot(a)askmonty.org-20130903082521-xpia2wond6oggpsk > parent: igor(a)askmonty.org-20130831163309-vqtls4oubhvz9zcz > committer: Alexey Botchkov <holyfoot(a)askmonty.org> > branch nick: 55-4472 > timestamp: Tue 2013-09-03 13:25:21 +0500 > message: > MDEV-4472 Audit-plugin. Server-related part of the task. > file_logger became the service. > Data like query_id now are sent to the audit plugin. > Fix for MDEV-4770 ported from 10.0. > Fix added for the read_maria_plugin_info(). > === modified file 'include/mysql/plugin_audit.h' > --- a/include/mysql/plugin_audit.h 2013-04-19 10:50:16 +0000 > +++ b/include/mysql/plugin_audit.h 2013-09-03 08:25:21 +0000 > @@ -25,7 +25,7 @@ > > #define MYSQL_AUDIT_CLASS_MASK_SIZE 1 > > -#define MYSQL_AUDIT_INTERFACE_VERSION 0x0301 > +#define MYSQL_AUDIT_INTERFACE_VERSION 0x0302 > > > /************************************************************************* > @@ -59,6 +59,10 @@ struct mysql_event_general > struct charset_info_st *general_charset; > unsigned long long general_time; > unsigned long long general_rows; > + /* Added for 0x302 */ A bit confusing. Better "Added in version 0x0302" > + long long query_id; should it rather be unsigned? > + const char *database; > + unsigned int database_length; > }; > > > @@ -140,6 +144,8 @@ struct mysql_event_table > unsigned int new_database_length; > const char *new_table; > unsigned int new_table_length; > + /* Added for 0x302 */ > + long long query_id; > }; > > /************************************************************************* > > === renamed file 'plugin/sql_errlog/sql_logger.cc' => 'mysys/file_logger.c' > --- a/plugin/sql_errlog/sql_logger.cc 2013-06-22 12:02:03 +0000 > +++ b/mysys/file_logger.c 2013-09-03 08:25:21 +0000 > @@ -51,6 +53,12 @@ LOGGER_HANDLE *logger_open(const char *p > unsigned int rotations) > { > LOGGER_HANDLE new_log, *l_perm; > + char *local_mysql_data_home; > +#ifdef _WIN32 > + local_mysql_data_home= (char *)GetProcAddress(0, "mysql_data_home"); > +#else > + local_mysql_data_home= mysql_data_home; > +#endif /*_WIN32*/ You shouldn't need it, file_logger.c is part of the server, not a dynamically loaded plugin. > > /* > I don't think we ever need more rotations, > === modified file 'sql/sql_audit.h' > --- a/sql/sql_audit.h 2013-04-19 10:50:16 +0000 > +++ b/sql/sql_audit.h 2013-09-03 08:25:21 +0000 > @@ -93,7 +93,8 @@ void mysql_audit_general_log(THD *thd, t > > mysql_audit_notify(thd, MYSQL_AUDIT_GENERAL_CLASS, MYSQL_AUDIT_GENERAL_LOG, > 0, time, user, userlen, cmd, cmdlen, > - query, querylen, clientcs, 0); > + query, querylen, clientcs, (ha_rows) 0, > + thd->db, thd->db_length); no query_id? > } > } > > @@ -139,7 +140,8 @@ void mysql_audit_general(THD *thd, uint > > mysql_audit_notify(thd, MYSQL_AUDIT_GENERAL_CLASS, event_subtype, > error_code, time, user, userlen, msg, msglen, > - query.str(), query.length(), query.charset(), rows); > + query.str(), query.length(), query.charset(), rows, > + thd->db, thd->db_length); no query_id? > } > } > > === modified file 'sql/sql_plugin.cc' > --- a/sql/sql_plugin.cc 2013-06-13 18:19:11 +0000 > +++ b/sql/sql_plugin.cc 2013-09-03 08:25:21 +0000 > @@ -2645,13 +2647,16 @@ static void update_func_longlong(THD *th > static void update_func_str(THD *thd, struct st_mysql_sys_var *var, > void *tgt, const void *save) > { > - char *old= *(char **) tgt; > - *(char **)tgt= *(char **) save; > + char *value= *(char**) save; > if (var->flags & PLUGIN_VAR_MEMALLOC) > { > - *(char **)tgt= my_strdup(*(char **) save, MYF(0)); > + char *old= *(char**) tgt; > + if (value) > + *(char**) tgt= my_strdup(value, MYF(0)); add else *(char**) tgt= 0; > my_free(old); > } > + else > + *(char**) tgt= value; > } > Regards, Sergei

2 3

[Maria-developers] RFC - CREATE PROCEDURE clauses (question about docs from mariadb-discuss)
by Roberto Spadim 02 Sep '13

02 Sep '13

Hi mariadb-devel guys, could anyone help here in a documentation problem? (maybe not a problem...) *Please answer the BOLD questions* We are talking about CREATE PROCEDURE , about DETERMINISTIC / NO SQL http://dev.mysql.com/doc/refman/5.6/en/create-procedure.html *characteristic*: COMMENT '*string*' | LANGUAGE SQL | *[NOT] DETERMINISTIC* | { CONTAINS SQL | *NO SQL* | READS SQL DATA | MODIFIES SQL DATA } | SQL SECURITY { DEFINER | INVOKER } *--------* *from mariadb-discuss:* *--------* Hi Federico, sorry i read some points and start writing, i will read and write after... i will answer with my experience as an user, not as a mariadb/mysql developer... 2013/9/2 Federico Razzoli <federico_raz(a)yahoo.it> > Hi Roberto > I'll try to be more clear, please read carefully before answering > ok I know what a deterministic expression is - but *what kinds of output must > be deterministic for a procedure to be defined as DETERMINISTIC?* > hummm i don't know, but for procedure, there isn't a deterministic output right? just for funcitons... maybe it's a parser problem, allow deterministic keyword for procedures (must check) i'm thinking about, *why we use deterministic procs, and non deterministic procs?* *this change something important in mysql server? replication, query cache? other performace considerations?* > There are many types of output for a proc: > * OUT params > * SELECTs > * writes on tables > * SET @var > hum.... from what i remember procedure ~= function without return, something like a "void FUNCTION( any parameters )" function ~= a procedure that return values, in other words it must be different from void... that's why i think it's a problem at parser allowing deterministic keyword at procedure, but must check... i don't know if this keyword change something in mysql/mariadb code > For a deterministic proc, do all of these need to be deterministic? > well, for me, procs are always non deterministics since it don't change query cache invalidation and replication (must be done in binary i think) > * I guess the answer isn't easy. > * *yeah, i see, we need a simple answer not a simplified answer* > -- > For "NO SQL" a complete list of SQL statements would be needed. If we use > this as a reference: > http://dev.mysql.com/doc/refman/5.6/en/sql-syntax.html > , even RETURN seems to be a SQL statement, yes, maybe the point isn't the SQL statements list, but other functions that mysql/mariadb server do? i don't know... NO SQL maybe is different from NOSQL (memcache,mongo,etc...) since nosql is newer than mysql procedures the point is *what's NO SQL in CREATE PROCEDURE ?* but of course it can't be true, because it's impossible to write a func > without RETURN. > yeah, maybe we have a problem at docs... let's see if anyone else know about this... maybe we need help from mariadb-devel list, i will include in this email... > > Greetings > Federico > bye =) thanks guys -- Roberto Spadim

1 0

Re: [Maria-developers] FWD: Status on MDEV-4506, parallel replication
by Kristian Nielsen 02 Sep '13

02 Sep '13

Michael Widenius <monty(a)askmonty.org> writes: >> On the master, I implemented --binlog-commit-wait-count=N and >> --binlog-commit-wait-usec=T. A transaction will wait at most T microseconds >> for at least N transactions to queue up and be ready for group commit. This >> allows to deliberately delay transactions on the master in order to get bigger >> group commits and thus better opportunity for parallel execution (and again it >> makes testing easier). > > Do you think the above helps in any real world case, except testing? > > Assuming we are have N=4 and T=2 > > - We have 3 threads ready to commit > - We wait 2 milliseconds and get 2 more threads. > - We commit now 5 threads. During this time you get 3 other threads > waiting to commit. > - We wait again... > > In a scenario without waiting: > > - We have 3 threads ready to commit > - We commit 3 threads. During this time we get 5 more threads waiting > to commit. > - We commit 5 threads. > > In other words, as the group commit will anyway take up 50 > microseconds on a hard disk, and thus automaticly group things > together for the next commit, why do we need ever to wait more? > > I agree that with --binlog-commit-wait-count you may get less sync > calls, but at an expense that a lot of threads took up to T > microseconds longer to execute. > > The worst case scenario is when you have only one user doing a lot of > inserts with auto-commit. In this case using wait will slow down the > server with T microseconds for every query. > > Have you been able to run any kind of benchmark where using > binglog-commit-wait-count will give more performance? > > I agree that the options are good for testing. The main question I > have is if we want to have the variables in the production server and > how we should document when and how a user should use the variables. I agree that with respect to group commit, the --binlog-commit-wait-count in many cases will not improve performance (that is why I did not implement it earlier). However, for parallel replication things are a bit different. Suppose that the application is doing C transaction commits per second, and that the disk system is capable of F binlog fsyncs per second. Now, if C is significantly bigger than F, then things are as you describe. Generally several transactions will queue up while the previous group commit runs, and there will be sufficient parallelism without using --bicommit_wait-*. This will be typical for eg. a simple harddisk-based system (F=40 commits/second perhaps) where all data is cached in the InnoDB buffer pool (eg. C>500 transactions/second). On the other hand, if C is smaller than F (or of similar magnitide), then usually only few or no new transactions will have time to queue up while the previous transaction is committing. So there will not be much parallelism for parallel replication to exploit without using --binlog-commit-wait-*. This will be typical for eg. a good-quality server with battery-backup RAID controller (F > 1000 commits/second) where data is too big to fit in buffer pool and every update requires disk access to complete (C < 500 transactions/second for example). And in fact, it is the second case, where random I/O is the bottleneck and multiple disk spindels are needed to improve I/O throughput, that single-threaded slave hurts the most, and where increasing --binlog-commit-wait-usec can be used with least penalty. I agree we need to document clearly the risk that --binlog-commit-wait-* will decrease performance on the master. Basically, the user can look at the ratio of Binlog_commits and Binlog_group_commits to check if enough transactions are part of each group commit for parallel replication to be effective. >> 1. The existing code is not thread-safe for class Relay_log_info. This class >> contains a bunch of stuff that is specific to executed transactions, not >> related to relay-log at all. This needs to be moved to the new struct >> rpl_group_info I introduced, and all code updated to pass around a pointer to >> that struct instead. There may also be a need to add additional locking on >> Relay_log_info, existing code needs review for this. > > I could take a look working on the above tomorrow and next week. Ok, great. Ping me when you can so we can coordinate, I may have some partial patches for this lying around. - Kristian.

1 0

[Maria-developers] Fwd: Bazaar-problem
by Nitish Upreti 31 Aug '13

31 Aug '13

I cant even clone the repository. Bazaar crashes with a bug midway. Can we have an active git mirror of the project for people just starting out? On Sat, Aug 31, 2013 at 2:13 PM, Sergei Golubchik <serg(a)mariadb.org> wrote: > Hi, AugustQ! > > On Aug 31, AugustQ wrote: > > I'm struggling with Bazaar. Maybe you can help me. > > > > I've made some modifications to the source-code and committed these to > > the local repository. Next I tried to upload them via push. This is what > > I receive: > > > > Cannot lock LockDir(chroot-70341520:///% > > 2Bbranch/maria/.bzr/branch/lock): Transport operation not possible: > > readonly transport > > You don't have write access to the mariadb source repository. > That's why you can only pull, but you cannot push. > > Regards, > Sergei > > _______________________________________________ > Mailing list: https://launchpad.net/~maria-developers > Post to : maria-developers(a)lists.launchpad.net > Unsubscribe : https://launchpad.net/~maria-developers > More help : https://help.launchpad.net/ListHelp >

2 1

[Maria-developers] introduction
by AugustQ 29 Aug '13

29 Aug '13

Hi, i've joined this list some days ago because I want to participate in the discussions here. The following links give you an overview of what i've done in the past: without registration: https://www.xing.com/profiles/August_Quint with registration: http://www.linkedin.com/profile/view?id=9080630 unfortunately the texts are in German, so in short: i've worked as a software-developer mainly in the backend area. I've made some small modifications on the source code but currently I'm struggling with Bazaar. This should be solved soon and I will then upload some code to the repository. I hope I can help you a bit. AugustQ PS: i found this: in MySQL I call cmake with an argument: -DWITH_DEBUG=1 in MariaDB this is: -DCMAKE_BUILD_TYPE=Debug is this correct?

2 1

[Maria-developers] alter online table documentation
by Roberto Spadim 26 Aug '13

26 Aug '13

Hi guys, i was searching alter online docs, at mariadb https://kb.askmonty.org/en/alter-table-statement/ but didn't found anything, could we add ? thanks! -- Roberto Spadim

4 5

Re: [Maria-developers] Calll for testing: MariaDB 5.5.32 packages
by Otto Kekäläinen 23 Aug '13

23 Aug '13

2013/8/23 Сергей Войтович <svojtovich(a)gmail.com>: > thanks for packaging MariaDB. I did a few basic tests with your packages and I confirm that they work well. > The only minor note I have is that I was asked for root password 3x during installation, which is a bit too much. Thanks! I haven't noticed any extra password asking during installation. It should ask for MariaDB root password in two screens, and of course if you run sudo apt-get it will ask once for your user password in the beginning. Could you clarify a little what happened in you installation?

2 3

[Maria-developers] Additional documentation for GTID
by Kristian Nielsen 23 Aug '13

23 Aug '13

Hi Daniel, Ian, I have implemented a new system variable for global transaction ID and pushed it into 10.0-base (so it should appear in 10.0.5 I suppose). I have written some text for documenting the new variable, appended below. Can one of you help me get this text into the Knowledgebase with proper formatting and cross-referencing across relevant pages? - Kristian. ----------------------------------------------------------------------- Variable: gtid_binlog_state Scope: global Dynamic: Yes Type: String The variable gtid_binlog_state holds the internal state of the binlog. The state consists of the last GTID ever logged to the binary log for every combination of domain_id and server_id. This information is used by the master to determine whether a given GTID has been logged to the binlog in the past, even if it has later been deleted due to binlog purge. For each domain_id, the last entry in @@gtid_binlog_state is the last GTID logged into binlog, ie. this is the value that appears in @@gtid_binlog_pos. Normally this internal state is not needed by users, as @@gtid_binlog_pos is more useful in most cases. The main usage of @@gtid_binlog_state is to restore the state of the binlog after RESET MASTER (or equivalently if the binlog files are lost). If the value of @@gtid_binlog_state is saved before RESET MASTER and restored afterwards, the master will retain information about past history, same as if PURGE BINARY LOGS had been used (of course the actual events in the binary logs are still deleted). Note that to set the value of @@gtid_binlog_state, the binary log must be empty, that is it must not contain any GTID events and the previous value of @@gtid_binlog_state must be the empty string. If not, then RESET MASTER must be used first to erase the binary log first. For completeness, note that setting @@gtid_binlog_state internally executes a RESET MASTER. This is normally not noticable as it can only be changed when the binlog is empty of GTID events. However, if executed eg. immediately after upgrading to MariaDB 10, it is possible that the binlog is non-empty but without any GTID events, in which case all such events will be deleted, just as if RESET MASTER had been run.

2 1

[Maria-developers] Error in 10.0.4 changelog, "Partition lock pruning"?
by Sergei Petrunia 23 Aug '13

23 Aug '13

Hello, Igor has pointed out that the changelog for 10.0.4 https://kb.askmonty.org/en/mariadb-1004-release-notes/ has this line: Partition lock pruning (MySQL Bug #37252, Bug #11748732) while the feature doesn't seem to be there. I did some investigation: - The line was in the changelog from the very first edit, it was put there by Daniel (who currently is on vacation) - I was unable to find any trace of MySQL Bug #37252 or Bug #11748732 in current 10.0 tree. - It seems, the whole "partition lock pruning" feature so far has not been ported to MariaDB 10.0. Does anybody know anything else about this? Should we remove the line from the changelog? BR Sergei -- Sergei Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

3 3

Re: [Maria-developers] Fwd: [Commits] Rev 3674: MDEV-4645: Incorrect reads of frozen binlog events; FDE corrupted in relay log in file:///home/bell/maria/bzr/work-maria-10.0-base-MDEV-4645/
by Kristian Nielsen 23 Aug '13

23 Aug '13

Oleksandr Byelkin <sanja(a)montyprogram.com> writes: > Could you review this? > > (it is https://mariadb.atlassian.net/browse/MDEV-4645 ) Ok to push. - Kristian.

1 0

[Maria-developers] Buildbot failure in 10.0-base
by Sergei Petrunia 23 Aug '13

23 Aug '13

Hi Kristian, 10.0-base started to get compile failures on [nearly] all hosts after your yesterday push: > 3682. By Kristian Nielsen 22 hours ago > MDEV-4488: When master is on the list of ignore_server_ids, GTID position on > slave is not updated > > The ignored events are not written to the relay log, but instead a fake > Rotate event is generated to handle update of position. > > Extend this for Gtid so we similarly generate a fake Gtid_list event > to update the GTID position. > > Also fix an unrelated test issue that got triggered by the added test cases. Could you please look into this? BR Sergei -- Sergei Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

2 1

[Maria-developers] MariaDB mutex contention (was: Re: MariaDB 10.0 benchmarks on upcoming IVB-EP processors ...)
by Axel Schwenke 22 Aug '13

22 Aug '13

Hi, Michael Widenius wrote: > 10.0.4 should be very soon ready for benchmarking (Tuesday or > Wednesday). We are in the final stage of fixing the failing test cases > after our last merge with MySQL 5.6 GA. > > I would just prefer to see Sergei Vojtovich push his code that changes > how the table cache works. > > This solves the same problem as the split table cache in 5.6, > but is a much simpler and better implementation. > > After this, I would expect that MariaDB 10.0 performance would be > closer to 5.6 performance for this benchmark. > Axel, could you at some point run the same benchmark with MariaDB > 10.0.4 and MySQL 5.6 with gprof to see where time is spent? I have now beefed up my knowledge of performance_schema and used it to look at Mutex/RW-lock contention for readonly OLTP benchmark. ----- TL;DR: 10.0.4 spends very long time in LOCK_PLUGIN, LOCK_OPEN is indeed much better than 10.0.3 but not(?) nearly as good as MySQL-5.6. ----- Contestants: - MySQL-5.6.13 (release) - MariaDB-10.0.3 (release) - MariaDB-10.0 r3799 (bzr trunk) all 3 compiled from source on benchmark machine using the same build script Environment: - many core machine (32 cores, 64 threads) - sysbench-0.5 OLTP readonly, 10M rows, single table - mysqld bound to 48 cores (sysbench to the other 8) - tested with 64 threads (this is the sweet spot) - using tcmalloc (preloaded) - mutexes and rw locks fully instrumented via performance_schema, see my.cnf Benchmark results (transactions per second): P_S on P_S off MySQL: ~15K ~18K 10.0.3: ~7K ~7.5K 10.0.4: ~6.5K ~9K After stabilizing I took 3 probes from P_S, sorting by sum_timer_wait desc. Times shown by P_S are picoseconds. If this were seconds, the decimal point would be in the column between "m" and "_" in "sum_timer_wait". So i.e. 10.0.4 waited 242 seconds for LOCK_PLUGIN during the 10 second measurement interval. Since we have 48 cores, there are total 480 seconds and 10.0.4 wasted about 50% of that. Questions: - what is protected by LOCK_PLUGIN? - how is LOCK_OPEN split in MySQL-5.6?If some of that is now covered by THR_LOCK::mutex, then it would explain why this is the busiest mutex in MySQL-5.6. Let me know if you need more data from me or if you have ideas what else I should test. XL Top 20 Mutexes/RW-locks MySQL-5.6.13 +---------------------------------------------+------------+-----------------+ | event_name | count_star | sum_timer_wait | +---------------------------------------------+------------+-----------------+ | wait/synch/mutex/mysys/THR_LOCK::mutex | 3397732 | 123367802416236 | | wait/synch/rwlock/innodb/btr_search_latch | 1632176 | 22207357475142 | | wait/synch/rwlock/sql/LOCK_grant | 1682983 | 1526912505972 | | wait/synch/mutex/sql/LOCK_table_cache | 3397786 | 1247140492941 | | wait/synch/mutex/sql/THD::LOCK_thd_data | 9219327 | 1076510130276 | | wait/synch/rwlock/innodb/hash_table_locks | 4086413 | 1026755227155 | | wait/synch/mutex/innodb/trx_sys_mutex | 364146 | 639270297372 | | wait/synch/mutex/innodb/os_mutex | 149491 | 630758755641 | | wait/synch/rwlock/innodb/index_tree_rw_lock | 1016436 | 344264365074 | | wait/synch/rwlock/sql/MDL_lock::rwlock | 242778 | 183463650006 | | wait/synch/mutex/sql/MDL_map::mutex | 121342 | 69121460394 | | wait/synch/mutex/mysys/BITMAP::mutex | 242667 | 54426177477 | | wait/synch/mutex/innodb/fil_system_mutex | 179586 | 20229032103 | | wait/synch/mutex/sql/LOCK_plugin | 121352 | 15004585617 | | wait/synch/mutex/innodb/trx_mutex | 121475 | 13105527141 | | wait/synch/mutex/innodb/lock_mutex | 121374 | 12627692868 | | wait/synch/mutex/innodb/buf_pool_mutex | 2240 | 228234006 | | wait/synch/mutex/innodb/log_sys_mutex | 540 | 119752122 | | wait/synch/mutex/innodb/flush_list_mutex | 704 | 86802870 | | wait/synch/mutex/innodb/buf_dblwr_mutex | 640 | 30728817 | +---------------------------------------------+------------+-----------------+ MariaDB-10.0.3 +------------------------------------------------+------------+-----------------+ | event_name | count_star | sum_timer_wait | +------------------------------------------------+------------+-----------------+ | wait/synch/mutex/sql/LOCK_open | 3334966 | 251668176449112 | | wait/synch/mutex/mysys/THR_LOCK::mutex | 2206782 | 2761461343332 | | wait/synch/mutex/sql/LOCK_plugin | 2540899 | 1471913471196 | | wait/synch/mutex/sql/THD::LOCK_thd_data | 6485120 | 770989733856 | | wait/synch/rwlock/innodb/hash table locks | 3315788 | 674878255920 | | wait/synch/rwlock/innodb/btr_search_latch | 1358771 | 540950632524 | | wait/synch/mutex/innodb/trx_sys_mutex | 380210 | 245662486416 | | wait/synch/rwlock/sql/MDL_lock::rwlock | 191698 | 125257499748 | | wait/synch/mutex/sql/TABLE_SHARE::LOCK_ha_data | 1067704 | 95258676492 | | wait/synch/mutex/mysys/THR_LOCK_lock | 157621 | 52329482172 | | wait/synch/mutex/mysys/BITMAP::mutex | 154870 | 49415303736 | | wait/synch/mutex/sql/MDL_map::mutex | 79173 | 41936664096 | | wait/synch/mutex/innodb/fil_system_mutex | 129610 | 40258209024 | | wait/synch/mutex/innodb/lock_mutex | 93667 | 26848457928 | | wait/synch/mutex/innodb/trx_mutex | 94131 | 11398946760 | | wait/synch/mutex/innodb/buf_pool_mutex | 12672 | 571983480 | | wait/synch/mutex/innodb/buf_dblwr_mutex | 9152 | 312365052 | | wait/synch/mutex/innodb/log_sys_mutex | 456 | 138632868 | | wait/synch/mutex/innodb/flush_list_mutex | 480 | 109226268 | | wait/synch/mutex/innodb/os_mutex | 350 | 71117100 | +------------------------------------------------+------------+-----------------+ MariaDB-10.0-r3799 +---------------------------------------------+------------+-----------------+ | event_name | count_star | sum_timer_wait | +---------------------------------------------+------------+-----------------+ | wait/synch/mutex/sql/LOCK_plugin | 4194896 | 242595666533160 | | wait/synch/mutex/sql/LOCK_open | 1835243 | 33790056812460 | | wait/synch/mutex/mysys/THR_LOCK::mutex | 1835221 | 3136249420092 | | wait/synch/rwlock/sql/LOCK_tdc | 911928 | 1088553356028 | | wait/synch/mutex/sql/THD::LOCK_thd_data | 5766813 | 695664363024 | | wait/synch/rwlock/innodb/hash table locks | 2144715 | 513517462020 | | wait/synch/rwlock/innodb/btr_search_latch | 915043 | 399309801912 | | wait/synch/rwlock/sql/MDL_lock::rwlock | 131171 | 176610102480 | | wait/synch/mutex/innodb/trx_sys_mutex | 196751 | 154609451460 | | wait/synch/rwlock/innodb/index_tree_rw_lock | 527543 | 137903039712 | | wait/synch/mutex/sql/MDL_map::mutex | 158268 | 95133455712 | | wait/synch/mutex/innodb/fil_system_mutex | 97493 | 27285605172 | | wait/synch/mutex/mysys/BITMAP::mutex | 131099 | 26589125196 | | wait/synch/mutex/mysys/THR_LOCK_lock | 131105 | 23952666708 | | wait/synch/mutex/innodb/lock_mutex | 65579 | 10355822676 | | wait/synch/mutex/innodb/trx_mutex | 65812 | 9790097724 | | wait/synch/mutex/innodb/buf_pool_mutex | 11550 | 644569236 | | wait/synch/rwlock/innodb/checkpoint_lock | 2 | 236298492 | | wait/synch/mutex/innodb/buf_dblwr_mutex | 3880 | 156907368 | | wait/synch/mutex/innodb/flush_list_mutex | 692 | 124740900 | +---------------------------------------------+------------+-----------------+ Comparison ---------- MariaDB-10.0-r3799 MariaDB-10.0.3 MySQL-5.6.13 +---------------------------------------------+------------+-----------------+------------+-----------------+------------+-----------------+ | event_name | count_star | sum_timer_wait | count_star | sum_timer_wait | count_star | sum_timer_wait | +---------------------------------------------+------------+-----------------+------------+-----------------+------------+-----------------+ | wait/synch/mutex/sql/LOCK_plugin | 4194896 | 242595666533160 | 2540899 | 1471913471196 | 121352 | 15004585617 | | wait/synch/mutex/sql/LOCK_open | 1835243 | 33790056812460 | 3334966 | 251668176449112 | | wait/synch/mutex/mysys/THR_LOCK::mutex | 1835221 | 3136249420092 | 2206782 | 2761461343332 | 3397732 | 123367802416236 | | wait/synch/rwlock/sql/LOCK_tdc | 911928 | 1088553356028 | | wait/synch/mutex/sql/THD::LOCK_thd_data | 5766813 | 695664363024 | 6485120 | 770989733856 | 9219327 | 1076510130276 | | wait/synch/rwlock/innodb/hash table locks | 2144715 | 513517462020 | 3315788 | 674878255920 | 4086413 | 1026755227155 | | wait/synch/rwlock/innodb/btr_search_latch | 915043 | 399309801912 | 1358771 | 540950632524 | 1632176 | 22207357475142 | | wait/synch/rwlock/sql/MDL_lock::rwlock | 131171 | 176610102480 | 191698 | 125257499748 | 242778 | 183463650006 | | wait/synch/mutex/innodb/trx_sys_mutex | 196751 | 154609451460 | 380210 | 245662486416 | 364146 | 639270297372 | | wait/synch/rwlock/innodb/index_tree_rw_lock | 527543 | 137903039712 | 1016436 | 344264365074 | | wait/synch/mutex/sql/MDL_map::mutex | 158268 | 95133455712 | 79173 | 41936664096 | 121342 | 69121460394 | | wait/synch/mutex/innodb/fil_system_mutex | 97493 | 27285605172 | 129610 | 40258209024 | 179586 | 20229032103 | | wait/synch/mutex/mysys/BITMAP::mutex | 131099 | 26589125196 | 154870 | 49415303736 | 242667 | 54426177477 | | wait/synch/mutex/mysys/THR_LOCK_lock | 131105 | 23952666708 | 157621 | 52329482172 | | wait/synch/mutex/innodb/lock_mutex | 65579 | 10355822676 | 93667 | 26848457928 | 121374 | 12627692868 | | wait/synch/mutex/innodb/trx_mutex | 65812 | 9790097724 | 94131 | 11398946760 | 121475 | 13105527141 | | wait/synch/mutex/innodb/buf_pool_mutex | 11550 | 644569236 | 12672 | 571983480 | 2240 | 228234006 | | wait/synch/rwlock/innodb/checkpoint_lock | 2 | 236298492 | | wait/synch/mutex/innodb/buf_dblwr_mutex | 3880 | 156907368 | 9152 | 312365052 | 640 | 30728817 | | wait/synch/mutex/innodb/flush_list_mutex | 692 | 124740900 | 480 | 109226268 | 704 | 86802870 | +---------------------------------------------+------------+-----------------+------------+-----------------+------------+-----------------+

7 18

[Maria-developers] Flashback Feature patch review
by Lixun Peng 22 Aug '13

22 Aug '13

Hi Monty, I patched my flashback feature to 10.0-base, and this is the diff file, please help me to review it. How to use this feature? I added several new arguments for mysqlbinlog: --flashback (-B), it will let mysqlbinlog working on FLASHBACK mode. --table (-T), it likes -d, but this argument is for table. --review, it will le mysqlbinlog print the SQL for reviewing. Reviewing feature will create a new "review" table to record the data that will be changed by FLASHBACK feature. If you aren't special the --review-dbname/tablename, "review" table will created on current DB, and the table name will be "__"+original_table_name. And for getting the original table struct, mysqlbinlog need to connect mysql, so --user/--host/--password is needed if necessary. --review-dbname, the DB that you want to store the review table. --review-tablename the TABLE that you want to store the changed data. Only if you set -T, this argument is useful. For example: I have a table "t" in database "test", CREATE TABLE `t` ( `id` int(10) unsigned NOT NULL AUTO_INCREMENT, `info` varchar(32) DEFAULT NULL, PRIMARY KEY (`id`) ) ENGINE=InnoDB AUTO_INCREMENT=9043880 DEFAULT CHARSET=utf8; And I done this operation, root@localhost : (none) 14:52:50> reset master; root@localhost : (none) 14:53:02> use test root@localhost : test 14:53:08> delete from t limit 10; root@localhost : test 14:53:25> insert into t (info) select "你好"; root@localhost : test 14:54:01> update t set info ='hello你好' limit 5; We can compare the output with --flashback or not. #client/mysqlbinlog /data/mysqldata_10.0/binlog/mysql-bin.000001 -vv -d test -T t --start-datetime="2013-03-27 14:54:00" > /tmp/1.sql #client/mysqlbinlog /data/mysqldata_10.0/binlog/mysql-bin.000001 -vv -uroot -d test -T t --start-datetime="2013-03-27 14:54:00" -B --review > /tmp/2.sql How to implement Flashback: 1. mysqlbinlog tool can parse the binlog files and print them, so we can use the most of code in mysqlbinlog. 2. Changing the event type (INSERT -> DELETE, DELETE -> INSERT) 3. Exchanging the SET/WHERE part in UPDATE (UPDATE table SET x=1 WHERE y=2 -> UPDATE table SET y=2 WHERE x=1) 4. Invert the event output, from end to beginning (binlog: e1,e2,e3... -> flashback_log: en...e3,e2,e1) Then, importing the flashback output can flashback database/table to the special time. But binlog_format should be ROW. Thanks, Lixun -- Senior MySQL Developer @ Taobao.com Mobile Phone: +86 18658156856 (Hangzhou) Gtalk: penglixun(at)gmail.com Twitter: http://www.twitter.com/plinux Blog: http://www.penglixun.com

2 3

[Maria-developers] MariaDB config files order
by Honza Horak 20 Aug '13

20 Aug '13

Hi guys, maybe I just need some little kick or I should get a coffee, but I can't understand the following config file ordering difference. Reading [1] and code in mariadb-5.5.32/mysys/default.c:1226, it looks like expected order of config files in Linux should be: /etc/my.cnf /etc/mysql/my.cnf ~/.my.cnf However, when asking for that using --help --verbose, I get a bit different output: $ /usr/libexec/mysqld --help --verbose 2>/dev/null | grep --after=1 '^Default options' | tail -n 2 Default options are read from the following files in the given order: /etc/mysql/my.cnf /etc/my.cnf ~/.my.cnf I don't understand why /etc/my.cnf is not on the first place in that output. Can somebody give me a hint? Am I just blind? Just FTR I'm trying that on Fedora 19, but there are no changes in Fedora packages that would do anything related, so I guess the same result would be in other distros as well. [1] https://kb.askmonty.org/en/mysqld-startup-options/ Cheers, Honza

2 3

Re: [Maria-developers] [Commits] Rev 3677: Fix bug MDEV-4895 Valgrind warnings (Conditional jump or move depends on uninitialised value) in Field_datetime::get_date on GREATEST(..) IS NULL in file:///home/tsk/mprog/src/5.3/
by Alexander Barkov 20 Aug '13

20 Aug '13

Hi Timour, The patch looks Okey to push for me. Please find two small neat-peaking comments below though :) On 08/20/2013 04:36 PM, timour(a)askmonty.org wrote: > At file:///home/tsk/mprog/src/5.3/ > > ------------------------------------------------------------ > revno: 3677 > revision-id: timour(a)askmonty.org-20130820123610-cjtgvaybl1ip3128 > parent: igor(a)askmonty.org-20130815235920-io2h7tlypwlbunsp > fixes bug: https://mariadb.atlassian.net/browse/MDEV-4895 > committer: timour(a)askmonty.org > branch nick: 5.3 > timestamp: Tue 2013-08-20 15:36:10 +0300 > message: > Fix bug MDEV-4895 Valgrind warnings (Conditional jump or move depends on uninitialised value) in Field_datetime::get_date on GREATEST(..) IS NULL > > Analysis: > The cause of the valgrind warning was an attempt to evaluate a Field that was not yet read. > The reason was that on one hand Item_func_isnotnull was marked as constant by > Item_func_isnotnull::update_used_tables, and this allowed eval_const_cond() to be called. > On the other hand Item_func_isnotnull::val_int() evaluated its argument as if it was not > constant. > > Solution: > The fix make sure that Item_func_isnotnull::val_int() doesn't evaluate its argument when > it is constant and cannot be NULL, because the result is known in this case. > > > > === modified file 'mysql-test/r/null.result' > --- a/mysql-test/r/null.result 2011-11-21 13:16:16 +0000 > +++ b/mysql-test/r/null.result 2013-08-20 12:36:10 +0000 > @@ -343,3 +343,33 @@ Field Type Null Key Default Extra > IFNULL(NULL, b) decimal(1,0) YES NULL > DROP TABLE t1, t2; > # End of 5.0 tests > +# > +# MDEV-4895 Valgrind warnings (Conditional jump or move depends on uninitialised value) in Field_datetime::get_date on GREATEST(..) IS NULL > +# > +CREATE TABLE t1 (dt DATETIME NOT NULL); > +INSERT INTO t1 VALUES (NOW()),(NOW()); > +EXPLAIN > +SELECT * FROM t1 WHERE concat( dt, '2012-12-21 12:12:12' ) IS NULL; > +id select_type table type possible_keys key key_len ref rows Extra > +1 SIMPLE NULL NULL NULL NULL NULL NULL NULL Impossible WHERE > +SELECT * FROM t1 WHERE concat( dt, '2012-12-21 12:12:12' ) IS NULL; > +dt > +drop table t1; > +CREATE TABLE t1 (dt INT NOT NULL); > +INSERT INTO t1 VALUES (1),(2); > +EXPLAIN > +SELECT * FROM t1 WHERE concat( dt, '1' ) IS NULL; > +id select_type table type possible_keys key key_len ref rows Extra > +1 SIMPLE NULL NULL NULL NULL NULL NULL NULL Impossible WHERE > +SELECT * FROM t1 WHERE concat( dt, '1' ) IS NULL; > +dt > +drop table t1; > +CREATE TABLE t1 (dt INT NOT NULL); > +INSERT INTO t1 VALUES (1),(2); > +EXPLAIN > +SELECT * FROM t1 WHERE NOT (concat( dt, '1' ) IS NOT NULL); > +id select_type table type possible_keys key key_len ref rows Extra > +1 SIMPLE NULL NULL NULL NULL NULL NULL NULL Impossible WHERE > +SELECT * FROM t1 WHERE NOT (concat( dt, '1' ) IS NOT NULL); > +dt > +drop table t1; Please fix "drop table" to upper case here, and in the other places. > > === modified file 'mysql-test/t/null.test' > --- a/mysql-test/t/null.test 2009-02-10 03:00:11 +0000 > +++ b/mysql-test/t/null.test 2013-08-20 12:36:10 +0000 > @@ -255,3 +255,31 @@ DESCRIBE t2; > DROP TABLE t1, t2; > > --echo # End of 5.0 tests > + > +--echo # > +--echo # MDEV-4895 Valgrind warnings (Conditional jump or move depends on uninitialised value) in Field_datetime::get_date on GREATEST(..) IS NULL > +--echo # > + > +CREATE TABLE t1 (dt DATETIME NOT NULL); > +INSERT INTO t1 VALUES (NOW()),(NOW()); > + > +EXPLAIN > +SELECT * FROM t1 WHERE concat( dt, '2012-12-21 12:12:12' ) IS NULL; > +SELECT * FROM t1 WHERE concat( dt, '2012-12-21 12:12:12' ) IS NULL; > + > +drop table t1; > +CREATE TABLE t1 (dt INT NOT NULL); > +INSERT INTO t1 VALUES (1),(2); > +EXPLAIN > +SELECT * FROM t1 WHERE concat( dt, '1' ) IS NULL; > +SELECT * FROM t1 WHERE concat( dt, '1' ) IS NULL; > + > +drop table t1; > +CREATE TABLE t1 (dt INT NOT NULL); > +INSERT INTO t1 VALUES (1),(2); > + > +EXPLAIN > +SELECT * FROM t1 WHERE NOT (concat( dt, '1' ) IS NOT NULL); > +SELECT * FROM t1 WHERE NOT (concat( dt, '1' ) IS NOT NULL); > + > +drop table t1; > > === modified file 'sql/item_cmpfunc.cc' > --- a/sql/item_cmpfunc.cc 2013-08-15 23:59:20 +0000 > +++ b/sql/item_cmpfunc.cc 2013-08-20 12:36:10 +0000 > @@ -4621,6 +4621,8 @@ Item *and_expressions(Item *a, Item *b, > longlong Item_func_isnull::val_int() > { > DBUG_ASSERT(fixed == 1); > + if (const_item() && !args[0]->maybe_null) > + return false; "return 0" should be better in a longlong function. > return args[0]->is_null() ? 1: 0; > } > > @@ -4628,6 +4630,8 @@ longlong Item_is_not_null_test::val_int( > { > DBUG_ASSERT(fixed == 1); > DBUG_ENTER("Item_is_not_null_test::val_int"); > + if (const_item() && !args[0]->maybe_null) > + DBUG_RETURN(1); > if (args[0]->is_null()) > { > DBUG_PRINT("info", ("null")); >

1 0

Re: [Maria-developers] [Commits] MDEV-4923 file://home/jan/mysql/mariadb-5.5-mdev4923
by Sergei Golubchik 20 Aug '13

20 Aug '13

Hi, Jan! On Aug 20, Jan Lindström wrote: > Serg, > > Could you please review fix for MDEC-4923, where these changes are > pushed lp:maria/5.5 or lp:~maria-captains/maria/5.5 ? Looks correct. This seems to be XtraDB bug. lp:maria/5.5 is an alias for lp:~maria-captains/maria/5.5 > === modified file 'storage/xtradb/os/os0file.c' > --- storage/xtradb/os/os0file.c 2013-07-29 14:03:41 +0000 > +++ storage/xtradb/os/os0file.c 2013-08-20 05:35:22 +0000 > @@ -1658,8 +1658,8 @@ > } > } > > - if (srv_use_atomic_writes && type == OS_DATA_FILE && > - os_file_set_atomic_writes(file, name)) { > + if (srv_use_atomic_writes && type == OS_DATA_FILE && > + !os_file_set_atomic_writes(file, name)) { > CloseHandle(file); > *success = FALSE; > file = INVALID_HANDLE_VALUE; > @@ -1786,7 +1786,7 @@ > #endif /* USE_FILE_LOCK */ > > if (srv_use_atomic_writes && type == OS_DATA_FILE > - && os_file_set_atomic_writes(name, file)) { > + && !os_file_set_atomic_writes(name, file)) { > > *success = FALSE; > close(file); > Regards, Sergei

1 0

[Maria-developers] [Otto Kekäläinen] Fwd: initial feedback on mariadb
by Kristian Nielsen 20 Aug '13

20 Aug '13

Hi, Otto asked for help with the copyright file for the Debian MariaDB packages. As I understand it, the .deb packages have a file /usr/share/doc/mariadb-server-5.5/copyright and there are some scripts in Debian that check the file against what is in the headers of source files and possibly other checks (error output attached below). Otto did something, but there is apparently more needed. Can some of you who understand the finer points of licensing among Oracle, Monty Program, MariaDB Services, MariaDB foundation, Percona, ... please look into this and help Otto fix up things? Debian are usually very careful with licensing, so this could easily become a blocker otherwise (I assume). - Kristian.

4 4

[Maria-developers] MDEV-4911 - KILL command with query id
by Roberto Spadim 20 Aug '13

20 Aug '13

Hi guys, i sent some patchs, could anyone check it? it have some things to do: query syntax isn't in the MDEV-4911, i did a QUERY_ID instead of a more comlpex syntax (since i never used bison before :P ) the query_id works, but i need a new error message ( "Unknown query id: %") well i didn't make the gcc happy, there's some warnings, and i don't know if it's all right i send this patch after MDEV-4916 patch, in other words the %expect value of sql_yacc.yy is wrong if you don't apply the MDEV-4916 patch the 4916 is a IF_IDLE keyword to KILL command too it only kill if there's no query running well :) thanks guys! <https://mariadb.atlassian.net/browse/MDEV-4916> -- Roberto Spadim

2 3

Re: [Maria-developers] More Secure Kill Command
by Sergei Golubchik 20 Aug '13

20 Aug '13

Hi, Roberto! On Aug 17, Roberto Spadim wrote: > Hi sergei, > What about give two informations, thread id and query id > If thread id and query id is equal, kill it There's no need to. Query id identifies the query and the connection unambiguously. Specifying also a thread id is redundant. Regards, Sergei

2 9

[Maria-developers] linking an external library
by rg 19 Aug '13

19 Aug '13

Folks, I want to link an external library for a language extension. After several hours of trying to tell cmake (^%$@*&) to pick it up for linking the library would still not be built with. CMakeLists.txt in trunk/sql is in the meantime full of pointers to the library without avail. I did the normal FIND_LIBRARY() followed by TARGET_LINK_LIBRARIES({$LIBNAME}) on various targets plus as nothing worked: SET(CMAKE_LIBRARY_PATH) on it, explicity LINK_DIRECTORIES(), SET(CMAKE_PREFIX_PATH), SET(CMAKE_LIBRARY_PATH) ... How do I link mariadb with an external library?? Thanks Rob

3 4

[Maria-developers] Audit plugin and qc_info plugin - MDEV-4682
by Roberto Spadim 18 Aug '13

18 Aug '13

Hi guys, i put some fields in qc_info plugin https://mariadb.atlassian.net/browse/MDEV-4581 the second part was add some others informations (stats): https://mariadb.atlassian.net/browse/MDEV-4682 i was reading the "response_time_distribution" of percona, patch at https://mariadb.atlassian.net/browse/MDEV-4568 could i use the audit plugin to implement MDEV-4682? what i need is: 1)know if the query was executed with/without query cache 2)know the time expend (lock time, and total time) to execute the query (first execution) 3)know the time expend with query cache hit 4) both (3) and (2) don't include time of network packages, just server side times... is it possible? any idea how could i implement it? i didn't write a audit plugin yet =( well.. time to learn =D thanks guys! -- Roberto Spadim

2 2

[Maria-developers] ALTER ONLINE TABLE syntax
by Sergei Golubchik 18 Aug '13

18 Aug '13

Hi, You've added this ALTER ONLINE TABLE syntax, where ONLINE de facto means "without copying the data". Now, after 5.6 merge we also have ALGORITHM=COPY|INPLACE and LOCK=NONE|SHARED|EXCLUSIVE. I've mapped ONLINE to ALGORITHM=INPLACE, because that's what it means. But it's confusing, one can have an "inplace" operation under an exclusive lock, if the engine wants it that way. And it won't be "online" by any stretch of imagination. One can also technically have a "copy" operation with a very permissive write-allow-write lock, and it'll be as online as one can get it. I mean, logically ONLINE should mean LOCK=NONE, not ALGORITHM=INPLACE. But that would change the existing behavior and the test results too. For example, *any* ALTER on the temporary table would always be "online", because, well, it wouldn't take any lock on that table. Alternatively, we can keep ONLINE as ALGORITHM=INPLACE, and deprecate it, issue a warning, suggesting to use the new 5.6 syntax. Opinions? Regards, Sergei

5 7

[Maria-developers] MDEV-4916
by Roberto Spadim 18 Aug '13

18 Aug '13

Patch uploaded, and test work (but i didn't add a unit test yet :/ sorry i don't know yet how to write it) -- Roberto Spadim

1 0

[Maria-developers] Joining the team
by Esteban Orduno 17 Aug '13

17 Aug '13

Hi, all: I would like to contribute to the maria-db team. Who can I talk iniatially to get some type of orientation about it? Esteban Orduno

2 1

[Maria-developers] More Secure Kill Command
by Roberto Spadim 17 Aug '13

17 Aug '13

Hi guys i was mailing at maria-discuss, but i think it a new feature, and want some view point from developers... check this problem: connection 1) big select... connection 2) show processlist connection 1) other query connection 2) kill connection 1 id (i take +- 1 second between show processlist and this command) connection 1) killed the problem? the kill connection 1 (KILL thread_id) killed the wrong query could we implement something like: KILL [CONNECTION | QUERY] thread_id [QUERY "some relative unique query id in this database"] the relative unique query id, could be: substr(sha1 or md5 of (thread_id + query start time),0,6) (6 hex numbers is good?) it's like a git small hash commit id, maybe we could make it more unique... but since i will not use it very often, i think a less intensive work is nice here... a global query id could add more one lock for every query... example: KILL 1 QUERY "abcdef" the "abcdef" = substr(MD5( thread id + query start time ),0,6) we could add client ip or others unique information about this query, but something that don't increase (ok just a little) the load of a show processlist / select * from information_schema thanks -- Roberto Spadim

4 7

[Maria-developers] MariaDB 10.0.4 Release Prep
by Daniel Bartholomew 16 Aug '13

16 Aug '13

All, I've begun prep work for the MariaDB 10.0.4 release. Draft Release Notes and Changelogs are here: - https://kb.askmonty.org/en/mariadb-1004-release-notes/ - https://kb.askmonty.org/en/mariadb-1004-changelog/ Please let me know ASAP if anything needs to be added/changed/etc... Thanks. -- Daniel Bartholomew Google+ - http://gplus.to/dbart Twitter - http://twitter.com/daniel_bart MariaDB: An Enhanced Drop-in Replacement for MySQL Website - http://mariadb.org Twitter - http://twitter.com/mariadb Google+ - http://google.com/+mariadb Facebook - http://fb.com/MariaDB.dbms Knowledgebase - http://kb.askmonty.org

3 4

[Maria-developers] RFC - query cache insert
by Roberto Spadim 16 Aug '13

16 Aug '13

Hi guys! I was reading (again) the query cache source code of 10.0.3 ... please tell me if i understood right when we insert at query cache... thread put a lock in query cache (no one can result from query cache) thread read all data and write to network (caching the result to query cache) after end of thread real the query cache is unlocked.. my doubts are... what happen if the query is too slow? the query cache is disabled while executing the query? what happen if two equal queries (same SELECT) execute? the first one is locked until the second one end the query and get the result from query cache? or the first lock query cache and the second runs without query cache? the last one there's any query cache changes in mind? i was reading the sql_cache.cc of old mysql servers and there's a todo list (very old i think) that didn't changed from old mysql to today mariadb 10.0.3, this todo list is inside mysql bug tracker? or mariadb jira? thanks guys! -- Roberto Spadim

2 11

[Maria-developers] doubt about workaround on MDEV-4903
by Roberto Spadim 16 Aug '13

16 Aug '13

hi guys, could anyone help me with a workaround to this MDEV? https://mariadb.atlassian.net/browse/MDEV-4903 basically the problem is a (more than one column primary key) IN ( (a primary key const value), (a primary key const value), (a primary key const value), (a primary key const value) ) it runs a table scan instead of using the primary key index using OR = OR = OR =, use the index but i want a easier way to write this query thanks -- Roberto Spadim

1 0

[Maria-developers] RFC - query cache using storage engine
by Roberto Spadim 15 Aug '13

15 Aug '13

Hi guys, i was thinking about ideas for query cache... there's an mdev about partitioning query cache ( https://mariadb.atlassian.net/browse/MDEV-4454) instead of this, why not use a HEAP storage engine (with partitioning), or maybe a myisam or aria or innodb as "backend", and write a "frontend" (query cache) that execute commands over this storage engine? i'm talking about, changing the query/table/result blocks of query cache, to query/table/result tables when start mysqld truncate these tables when add new query "insert" to these tables when a table prune occur, "delete" from these tables when flush, optimize these tables when reset query cache, truncate this tables these tables could be at mysql schema, if they don't exists create as ENGINE=MEMORY for example since we must have a MAX SIZE for query cache, we could add a MAX SIZE for HEAP engine and a LOW MEMORY algorithm ( https://mariadb.atlassian.net/browse/MDEV-4675) at high level, what you think about this idea? -- Roberto Spadim

2 3

Re: [Maria-developers] Fwd: [Commits] Rev 3674: MDEV-4645: Incorrect reads of frozen binlog events; FDE corrupted in relay log in file:///home/bell/maria/bzr/work-maria-10.0-base-MDEV-4645/
by Kristian Nielsen 15 Aug '13

15 Aug '13

Oleksandr Byelkin <sanja(a)montyprogram.com> writes: > Could you review this? > > (it is https://mariadb.atlassian.net/browse/MDEV-4645 ) Ok, but it will be at least a couple more days before I will have time to start. If urgent, you can also ask Sergei Golubchik, IIRC this is the patch from Jeremy Cole that he asked me about, and I already looked at once and accepted the general idea? - Kristian.

2 1

Re: [Maria-developers] Review of patch for MDEV-4820
by Kristian Nielsen 15 Aug '13

15 Aug '13

Vilho Raatikka <vilho.raatikka(a)skysql.com> writes: > Is this documented profoundly already? Should be because probably we'll see > at least some users as ignorant as I am asking the same questions soon. I wrote up some comprehensive documentation in the knowledgebase, and also wrote a series of articles about in on my blog: https://kb.askmonty.org/en/global-transaction-id/ http://kristiannielsen.livejournal.com/ But this is complex stuff, I am sure it can be improved. Any help improving the documentation would be greatly appreciated. I fear that I have spent too much time on the details of the design to clearly see how to explain the high-level picture in a way suitable for new users. - Kristian.

1 0

[Maria-developers] Value returned by my_getcputime
by Weldon Whipple 14 Aug '13

14 Aug '13

Hello! (This is my first post to this list; long-time follower, however.) I've been looking at the block comment at the top of the my_getcputime() function at the bottom of the file mysys/my_getsystime.c The block comment reads: Return cpu time in milliseconds * 10 I interpret milliseconds * 10 to mean centiseconds (seconds * 10 ^ -2) I'm pretty sure I've convinced myself that it REALLY returns: seconds * 10 ^ -7 I think it would be more accurate to have the block comment read: Return cpu time in microseconds / 10 Am I "up in the night"? (I think the code itself is fine, and the callers of the function all do it correctly. I was just initially confused when I looked at the comment then the code.) Thanks in advance! Weldon Whipple weldon(a)whipple.org (P.S. I think I like Maria's implementation of userstat_v2 better than the original by Google/Percona ...)

2 1

Re: [Maria-developers] Review of patch for MDEV-4820
by Kristian Nielsen 14 Aug '13

14 Aug '13

>> SHOW VARIABLES or SELECT @@GLOBAL.gtid_strict_mode will show the current >> value. But there is no way to check if it was enabled eg. two weeks ago, if >> that is what you are asking (the value is not stored in the binlog). > That is really pity. Is it really so useful to give the user opportunity to > switch gtid mode on-the-fly? That option sounds really expensive. Isn't > binlog event ordering lost - or it can't be trusted anymore? Binlog event order is never lost, strict mode or no strict mode. That is the whole point: even if the user messes up, MariaDB GTID is still able to correctly find the right position in the binlog to start at. There is nothing expensive about changing gtid strict mode, nor any loss of trust. Gtid strict mode is something to help the _user_. The GTID _code_ does not care, it works correctly whether strict mode is enabled or not. The only thing strict mode does is give an error for some uses that are considered undesirable by people like you and me. All non-error cases work exactly the same whether strict mode is enabled or not. It seems the more I try to explain, the more I confuse the issue :-( Maybe it is better if I just go fix the actual bugs ... - Kristian.

1 0

Re: [Maria-developers] Rev 3804: MDEV-4712 : Fix "shutdown" test
by Sergei Golubchik 13 Aug '13

13 Aug '13

Hi, Vladislav! On Jul 31, Vladislav Vaintroub wrote: > ------------------------------------------------------------ > revno: 3804 > revision-id: wlad(a)montyprogram.com-20130731130414-ukqxbb0kxpexjnzp > parent: svoj(a)mariadb.org-20130731110211-09p66bwf4jrmi03i > committer: Vladislav Vaintroub <wlad(a)montyprogram.com> > branch nick: 10.0-serg > timestamp: Wed 2013-07-31 15:04:14 +0200 > message: > MDEV-4712 : Fix "shutdown" test. > > There is a race condition during shutdown, that happens when "show status" runs during shutdown. > Functions show_slave_running(), show_slave_received_heartbeats(), show_heartbeat_period() would dereference NULL pointer master_info_index, after this variable is set to NULL in slave_end() during shutdown. > Fix introduces check for master_info_index == NULL in the affected functions. Was it related to the new shutdown command? Or it could be also observed without it by, say, running show status in a loop, while stopping the server with mysqladmin? Regards, Sergei

2 1

[Maria-developers] Lack of linkage from MariaDB JIRA to commits
by Jeremy Cole 13 Aug '13

13 Aug '13

All, I have noticed that the MariaDB JIRA tickets are rarely linked to any specific commit which purportedly fixes them. Is there any standard at the moment for what should be done (and then it isn't followed) or is there no standard (and perhaps it should be made)? I would really like a consistent way to get to view the commits into various branches related to each MDEV issue, in a consistent way. Regards, Jeremy

5 9

[Maria-developers] 10.x compatibility and separate namespace
by Honza Horak 13 Aug '13

13 Aug '13

Hi Guys, since 10.x won't be compatible with mysql in every aspect any more, I'm wondering if there happen to be any plans to move all compatibility left-overs (client binary names, client library name, daemon binary name, etc.). Are there any such ideas? I understand that we would lose the compatibility for any later (keeping original names was mentioned in [1] as a MUST for compatibility), but when keeping the original names, it could become tricky to say which tools are compatible and which not any more in mariadb 10+. How this information will be provided actually? How to get users informed properly, which tools are not compatible any more? And a bit another POV -- is the incompatibility in 10.x strictly limited to the server only (so client side will stay compatible for anytime later)? [1] https://lists.launchpad.net/maria-discuss/msg00658.html Regards, Honza

3 2

Re: [Maria-developers] Review of patch for MDEV-4820
by Kristian Nielsen 13 Aug '13

13 Aug '13

Vilho Raatikka <vilho.raatikka(a)skysql.com> writes: > What exactly does it require for the user to enable strict or 'relaxed' > (=disasterous imho) mode? Technically, one needs to SET GLOBAL gtid_strict_mode=1 with root privileges (config file option works as well of course). That's it. The practical issue is that old-style MariaDB/MySQL replication allows a number of "disasterous" uses. One example is a multi-master ring A->B->C->A, writing to all masters simultaneously. In such setup, it is fundamentally impossible to automatically eg. move a slave S from A->S to B->S - there is just not enough information about what events have been applied to S from A, B, and C. GTID introduces a way to configure this correctly (replication domains), but if we were to give an error by default then we would break upgrades from 5.5. So instead we provide gtid strict mode, so that once users understand what the "disasterous" uses are and how to avoid them, they can enable strict mode and get saner semantics. BTW, for the normal non-advanced user, I think silly stuff like running random manual transactions on slaves that get into the binlog is a far more common mistake. Gtid strict mode will cause errors on this, so it needs to be explictly enabled by users to not break/confuse upgrades. > Is it possible at any point in server's life cycle to check which mode it > is running? SHOW VARIABLES or SELECT @@GLOBAL.gtid_strict_mode will show the current value. But there is no way to check if it was enabled eg. two weeks ago, if that is what you are asking (the value is not stored in the binlog). > Can that setting be modified during runtime or only prior startup? It can be changed at any time, though I think you need to STOP SLAVE/START SLAVE to get the change to take effect on a slave. - Kristian.

1 1

[Maria-developers] RFC: Compile-time checking for unmatched DBUG_ENTER/LEAVE via unused variable
by Jeremy Cole 11 Aug '13

11 Aug '13

Maria developers/captains, Prior to posing it as a final patch contribution or opening a JIRA ticket, I wanted to propose this idea. The attached patch adds what amounts to compile-time checking for unmatched DBUG_ENTER/DBUG_LEAVE (DBUG_RETURN, DBUG_VOID_RETURN) by introducing a variable in DBUG_ENTER which is only used in DBUG_LEAVE. This allows any compiler which can robustly detect unused variables to detect the mismatch at compile-time. There is already a run-time check for this case, but it is somewhat limited as it requires _db_return_ to be called in order to detect the mismatch, and this is in practice not always the case. Particularly three cases allow this to escape detection: 1. Some instrumented functions are called only from non-instrumented functions. 2. Some instrumented functions are called only from non-returning functions, such as abort or die functions. 3. Some instrumented functions are called from functions who call my_end or my_thread_end (making cs unavailable) before returning. In any case please see the attached DRAFT patch for a full implementation of the idea. Note one bit of ugliness is that non-returning functions, after this patch, must be marked explicitly as DBUG_ENTER_NO_RETURN -- this actually is not really a bad thing, though in and of itself. Regards, Jeremy

4 4

Re: [Maria-developers] Rev 3671: MDEV-4652 Wrong result for CONCAT(GREATEST(TIME('00:00:01'), TIME('00:00:00')))
by Sergei Golubchik 11 Aug '13

11 Aug '13

Hi, Alexander! On Aug 07, Alexander Barkov wrote: > At lp:~maria-captains/maria/5.3 > > ------------------------------------------------------------ > revno: 3671 > revision-id: bar(a)mariadb.org-20130807093420-khdah292yerbor8y > parent: sergii(a)pisem.net-20130715163225-6ch6x34lsufode3d > committer: Alexander Barkov <bar(a)mariadb.org> > branch nick: 5.3.b4652 > timestamp: Wed 2013-08-07 13:34:20 +0400 > message: > MDEV-4652 Wrong result for CONCAT(GREATEST(TIME('00:00:01'),TIME('00:00:00'))) Ok to push. > === modified file 'sql/item.cc' > --- a/sql/item.cc 2013-07-03 07:46:20 +0000 > +++ b/sql/item.cc 2013-08-07 09:34:20 +0000 > @@ -252,7 +252,8 @@ String *Item::val_string_from_decimal(St > String *Item::val_string_from_date(String *str) > { > MYSQL_TIME ltime; > - if (get_date(&ltime, 0) || > + if (get_date(&ltime, > + field_type() == MYSQL_TYPE_TIME ? TIME_TIME_ONLY : 0) || > str->alloc(MAX_DATE_STRING_REP_LENGTH)) > { > null_value= 1; > > === modified file 'sql/item_func.cc' > --- a/sql/item_func.cc 2013-07-03 07:46:20 +0000 > +++ b/sql/item_func.cc 2013-08-07 09:34:20 +0000 > @@ -2480,6 +2480,13 @@ bool Item_func_min_max::get_date(MYSQL_T > ltime->time_type= MYSQL_TIMESTAMP_DATE; > ltime->hour= ltime->minute= ltime->second= ltime->second_part= 0; > } > + else if (compare_as_dates->field_type() == MYSQL_TYPE_TIME) > + { > + ltime->time_type= MYSQL_TIMESTAMP_TIME; > + ltime->hour+= (ltime->month * 32 + ltime->day) * 24; > + ltime->month= ltime->day= 0; > + } > + > > return 0; > } Regards, Sergei

1 0

Re: [Maria-developers] [Commits] Rev 3552: Fix bug MDEV-4648 SQ pushdown: Wrong result (missing rows) with materialization+semijoin, IN and ALL subqueries, UNION in file:///home/tsk/mprog/src/10.0-md83/
by Oleksandr Byelkin 09 Aug '13

09 Aug '13

09.08.2013 14:30, timour(a)askmonty.org пишет: > At file:///home/tsk/mprog/src/10.0-md83/ > > ------------------------------------------------------------ > revno: 3552 > revision-id: timour(a)askmonty.org-20130809113013-ia2f7zzw3mv3in7u > parent: timour(a)askmonty.org-20130611081135-rge7p00h8e9nhypj > fixes bug: https://mariadb.atlassian.net/browse/MDEV-4648 > committer: timour(a)askmonty.org > branch nick: 10.0-md83 > timestamp: Fri 2013-08-09 14:30:13 +0300 > message: > Fix bug MDEV-4648 SQ pushdown: Wrong result (missing rows) with materialization+semijoin, IN and ALL subqueries, UNION > (part of MDEV-83) > > Analysis: > > The cause of the wrong result was that the ALL subquery was incorrectly > attached to the last table in the join plan <t1, sj-mat, t2>. The ALL > subquery was transformed into a MIN expression, however it still depended > on OUTER_REF_TABLE_BIT as if it was a subquery. The reason for this > dependence was that the transformed MIN expression referred to the > Item_cache object of the left subquery operand created for the initial > Item_in_optimizer. > > Solution: > > When transforming an ALL subquery into a MIN/MAX expression, use the > actual left argument expression, instead of the Item_cache created > for the subquery. > OK to push. Thank you!

1 0

[Maria-developers] Is it a bug?
by Alexey Botchkov 09 Aug '13

09 Aug '13

Hi, SerG. The code in sql/sql_plugin.cc function read_maria_plugin_info() sym= dlsym(plugin_dl->handle, maria_plugin_declarations_sym); ... if (plugin_dl->mariaversion != MARIA_PLUGIN_INTERFACE_VERSION) { ... if ((sym= dlsym(plugin_dl->handle, maria_sizeof_st_plugin_sym))) sizeof_st_plugin= *(int *)sym; ... if (sizeof_st_plugin != sizeof(st_mysql_plugin)) { ... sym= cur; } } plugin_dl->plugins= (struct st_maria_plugin *)sym; So that when we enter the first 'if' but not the second (sizeof_st_plugin == sizeof(st_mysql_plugin) the sym still points at the maria_sizeof_st_plugin_sym which makes a bad st_maria_plugin pointer. I think it's a bug, isn't it? Best regards. HF

2 1

Re: [Maria-developers] GTID and removing binary logs
by Kristian Nielsen 08 Aug '13

08 Aug '13

Elena Stepanova <elenst(a)montyprogram.com> writes: > Reading recent Pavel Ivanov's reports (the last one is MDEV-4820, but > I think there were some others which you've already closed as fixed), > I started wondering if I conceptually misunderstood something > before. From our previous discussions I made a conclusion that > removing *all* binary logs before trying to replicate from the server > is very wrong and against the whole idea of GTID. Is it not so? If > not, it's definitely an area which didn't undergo any testing from me, > except for my very first clumsy attempts to get familiar with GTID, so > I need to fill the gap. Well, we _do_ want to support running a slave with --log-slave-updates=0, and then making that slave into a master. Indeed, you tested that early and it was horribly broken. However, my intention is that this should now be supported (and thus can be tested). There are MTR testcases for this. The reason I mention this is that the case of --log-slave-updates=0 is actually quite similar (from the server's point of view) to manually deleting the logs. So I do also intend to support removing binlogs as Pavel wants to do. The main problem with doing this (deleting binlogs or using --log-slave-updates=0) is that it makes it harder to detect certain user errors, because the information to know that it is an error is missing. I suppose the limitations should be pretty obvious: If user deletes binlogs on a server, then that server, acting as a master, can not replicate to slaves starting from an event that was deleted. So as long as the user does not try to do that, things should be fine. I think the main problems come if the user makes a mistake in this respect, and the server fails to detect the error. For example, suppose that server A is a slave, replicates up to position 0-2-10. Then it becomes a master, generates events up to 0-1-20. Then user deletes the binlogs on A, and tries to have server B replicate from A starting at position 0-2-10. In this case, server A will allow B to connect, effectively loosing the deleted events 0-1-11 .. 0-1-20. I do not see any way to avoid this. Server A has no way to know that those events ever existed, as the user deliberately deleted all information about them. (In fact the user might be doing this deliberately, in case the events were wrongly created and should be removed from the history). A workaround in this example is that the user sets @@gtid_slave_pos explicitly to 0-1-20 after deleting the logs, thus preserving the needed information. So in summary: Yes, we try to support deleting binlogs, however there is some responsibility on the part of the user to not delete information that they actually need. I think that is rather reasonable. It is how DELETE and SELECT work, as well :-) - Kristian.

1 0

Re: [Maria-developers] [Commits] Rev 3772: MDEV-4645: Incorrect reads of frozen binlog events; FDE corrupted in relay log in file:///home/bell/maria/bzr/work-maria-10.0-MDEV-4548/
by Jeremy Cole 08 Aug '13

08 Aug '13

Sanja, Re: https://mariadb.atlassian.net/browse/MDEV-4645 Re: http://lists.askmonty.org/pipermail/commits/2013-July/005093.html I saw your commit on the commits list (sorry, wasn't on the list before the mail went out, so I can't reply directly to that message). A couple of points related to the commit: - You seem to have an error in the get_header_len, as it's implemented only in the base class, and you've left functions with the correct signature for it but the wrong names in the two frozen classes (they were left named has_fixed_minimal_header). This would mean the change is completely broken. :) - The change to get_header_len I think is in general fine, although it does alter the point of the whole thing -- there is again no way to exactly identify which events have frozen headers, programmatically (i.e. you can't differentiate them from events who just "happen" to have header_len = LOG_EVENT_MINIMAL_HEADER_LEN). I think that doesn't matter for this patch, but does defeat the point of what I was trying to add. - I don't really understand why you've made the hex printing code so complex and completely unreadable (IMHO). This is not performance-sensitive code and in my opinion should favor readability over performance. All you seem to have saved in this change is to avoid printing some static formatting characters ("#", "|", spaces, etc.) inside the loop. However, I can hardly even figure out what it's supposed to be doing now. - Additionally, you've changed the output slightly -- normally the ASCII character section prints its ending "|" immediately after the last character; it seems now to be fixed position (leaving extra spaces before "|". This differs from the output of hexdump, and it somewhat ambiguous/confusing. - You've mixed tabs and spaces. :( Regards, Jeremy

2 8

[Maria-developers] RFC - temporary path and SSD/HDD
by Roberto Spadim 07 Aug '13

07 Aug '13

Hi guys I have some server running with bcache and some in test running flash cache and others linux ssd cache solution I was thinking about how to optimize the temporary tables of mysql/mariadb... could we implement something like: Temporary Slow/Big Path = /tmp/hdd Temporary Fast/Small Path = /tmp/ssd Temporary Fast Max Size = 10MB When we start using temporary tables (file sort for example) instead of write to a hdd temporary disk, use a ssd temporary disk, if table get bigger than 10MB, copy the file to hdd drive, and remove from ssd drive, and continue writing to hdd I think with this i could optimize the temporary table use Any other idea about this kind of optimization? -- Roberto Spadim

1 0

Re: [Maria-developers] [Commits] Rev 3671: MDEV-4811 Assertion `offset < 0x1f' fails in type_and_offset_store on COLUMN_ADD in file:///home/bell/maria/bzr/work-maria-5.3-MDEV-4811/
by Sergei Petrunia 07 Aug '13

07 Aug '13

Hi Sanja, Ok to push. On Thu, Aug 01, 2013 at 11:46:14AM +0300, sanja(a)montyprogram.com wrote: > At file:///home/bell/maria/bzr/work-maria-5.3-MDEV-4811/ > > ------------------------------------------------------------ > revno: 3671 > revision-id: sanja(a)montyprogram.com-20130801084611-cu3wpoazd197g41w > parent: sergii(a)pisem.net-20130715163225-6ch6x34lsufode3d > committer: sanja(a)montyprogram.com > branch nick: work-maria-5.3-MDEV-4811 > timestamp: Thu 2013-08-01 11:46:11 +0300 > message: > MDEV-4811 Assertion `offset < 0x1f' fails in type_and_offset_store on COLUMN_ADD > MDEV-4812 Valgrind warnings (Invalid write) in dynamic_column_update_many on COLUMN_ADD > > Fixed problem of working on wrong data (do not allow offset to out of string length). > === modified file 'mysql-test/r/dyncol.result' > --- a/mysql-test/r/dyncol.result 2013-03-26 11:07:46 +0000 > +++ b/mysql-test/r/dyncol.result 2013-08-01 08:46:11 +0000 > @@ -1404,5 +1404,26 @@ v1 CREATE ALGORITHM=UNDEFINED DEFINER=`r > drop view v1; > drop table t1; > # > +# MDEV-4811: Assertion `offset < 0x1f' fails in type_and_offset_store > +# on COLUMN_ADD > +# > +CREATE TABLE t1 (dyn TINYBLOB) ENGINE=MyISAM; > +INSERT INTO t1 SET dyn = COLUMN_CREATE( 40, REPEAT('a', 233), 4, REPEAT('b', 322) ); > +Warnings: > +Warning 1265 Data truncated for column 'dyn' at row 1 > +SELECT COLUMN_ADD( dyn, 6, REPEAT('x',80), 4, REPEAT('y',215) AS INTEGER ) FROM t1; > +ERROR HY000: Encountered illegal format of dynamic column string > +DROP table t1; > +# > +# MDEV-4812: Valgrind warnings (Invalid write) in > +# dynamic_column_update_many on COLUMN_ADD > +# > +CREATE TABLE t1 (dyncol TINYBLOB) ENGINE=MyISAM; > +INSERT INTO t1 SET dyncol = COLUMN_CREATE( 7, REPEAT('k',487), 209, REPEAT('x',464) ); > +Warnings: > +Warning 1265 Data truncated for column 'dyncol' at row 1 > +SELECT COLUMN_ADD( dyncol, 7, '22:22:22', 8, REPEAT('x',270) AS CHAR ) FROM t1; > +DROP table t1; > +# > # end of 5.3 tests > # > > === modified file 'mysql-test/t/dyncol.test' > --- a/mysql-test/t/dyncol.test 2013-03-26 11:07:46 +0000 > +++ b/mysql-test/t/dyncol.test 2013-08-01 08:46:11 +0000 > @@ -600,6 +600,29 @@ drop view v1; > drop table t1; > > --echo # > +--echo # MDEV-4811: Assertion `offset < 0x1f' fails in type_and_offset_store > +--echo # on COLUMN_ADD > +--echo # > + > +CREATE TABLE t1 (dyn TINYBLOB) ENGINE=MyISAM; > +INSERT INTO t1 SET dyn = COLUMN_CREATE( 40, REPEAT('a', 233), 4, REPEAT('b', 322) ); > +--error ER_DYN_COL_WRONG_FORMAT > +SELECT COLUMN_ADD( dyn, 6, REPEAT('x',80), 4, REPEAT('y',215) AS INTEGER ) FROM t1; > + > +DROP table t1; > + > +--echo # > +--echo # MDEV-4812: Valgrind warnings (Invalid write) in > +--echo # dynamic_column_update_many on COLUMN_ADD > +--echo # > +CREATE TABLE t1 (dyncol TINYBLOB) ENGINE=MyISAM; > + > +INSERT INTO t1 SET dyncol = COLUMN_CREATE( 7, REPEAT('k',487), 209, REPEAT('x',464) ); > +--error 0,ER_DYN_COL_WRONG_FORMAT > +SELECT COLUMN_ADD( dyncol, 7, '22:22:22', 8, REPEAT('x',270) AS CHAR ) FROM t1; > +DROP table t1; > + > +--echo # > --echo # end of 5.3 tests > --echo # > > > === modified file 'mysys/ma_dyncol.c' > --- a/mysys/ma_dyncol.c 2011-10-18 10:44:12 +0000 > +++ b/mysys/ma_dyncol.c 2013-08-01 08:46:11 +0000 > @@ -1228,13 +1228,14 @@ dynamic_column_create(DYNAMIC_COLUMN *st > @param header_end Pointer to the header end > @param offset_size Size of offset field in bytes > @param last_offset Size of the data segment > + @param error Set in case of error > > @return number of bytes > */ > > static size_t get_length_interval(uchar *entry, uchar *entry_next, > uchar *header_end, size_t offset_size, > - size_t last_offset) > + size_t last_offset, my_bool *error) > { > size_t offset, offset_next; > DYNAMIC_COLUMN_TYPE type, type_next; > @@ -1242,8 +1243,12 @@ static size_t get_length_interval(uchar > > type_and_offset_read(&type, &offset, entry, offset_size); > if (entry_next >= header_end) > + { > + *error= 0; > return (last_offset - offset); > + } > type_and_offset_read(&type_next, &offset_next, entry_next, offset_size); > + *error= (offset_next > last_offset); > return (offset_next - offset); > } > > @@ -1255,17 +1260,18 @@ static size_t get_length_interval(uchar > @param header_end Pointer to the header end > @param offset_size Size of offset field in bytes > @param last_offset Size of the data segment > + @param error Set in case of error > > @return number of bytes > */ > > static size_t get_length(uchar *entry, uchar *header_end, > size_t offset_size, > - size_t last_offset) > + size_t last_offset, my_bool *error) > { > return get_length_interval(entry, > entry + offset_size + COLUMN_NUMBER_SIZE, > - header_end, offset_size, last_offset); > + header_end, offset_size, last_offset, error); > } > > > @@ -1304,6 +1310,7 @@ find_column(DYNAMIC_COLUMN_TYPE *type, u > uchar *entry; > size_t offset, total_data, header_size, entry_size; > uchar key[2+4]; > + my_bool error; > > if (!entry_pos) > entry_pos= &entry; > @@ -1329,12 +1336,12 @@ find_column(DYNAMIC_COLUMN_TYPE *type, u > return 1; > *data= header + header_size + offset; > *length= get_length(entry, header + header_size, offset_size, > - total_data); > + total_data, &error); > /* > Check that the found data is withing the ranges. This can happen if > we get data with wrong offsets. > */ > - if ((long) *length < 0 || offset + *length > total_data) > + if (error || (long) *length < 0 || offset + *length > total_data) > return 1; > > *entry_pos= entry; > @@ -1837,12 +1844,13 @@ dynamic_column_update_many(DYNAMIC_COLUM > entry_size, column_count, &entry)) > { > size_t entry_data_size; > + my_bool error; > > /* Data existed; We have to replace or delete it */ > > entry_data_size= get_length(entry, header_end, > - offset_size, max_offset); > - if ((long) entry_data_size < 0) > + offset_size, max_offset, &error); > + if (error || (long) entry_data_size < 0) > { > rc= ER_DYNCOL_FORMAT; > goto end; > @@ -2038,12 +2046,13 @@ dynamic_column_update_many(DYNAMIC_COLUM > /* copy first the data that was not replaced in original packed data */ > if (start < end) > { > + my_bool error; > /* Add old data last in 'tmp' */ > size_t data_size= > get_length_interval(header_base + start * entry_size, > header_base + end * entry_size, > - header_end, offset_size, max_offset); > - if ((long) data_size < 0 || > + header_end, offset_size, max_offset, &error); > + if (error || (long) data_size < 0 || > data_size > max_offset - first_offset) > { > dynamic_column_column_free(&tmp); > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits -- BR Sergei -- Sergei Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] [Commits] Rev 3797: MDEV-4816: rpl.rpl_trunc_temp fails in 10.0-serg in file:///data0/psergey/dev2/10.0-serg/
by Kristian Nielsen 07 Aug '13

07 Aug '13

Sergey Petrunya <psergey(a)askmonty.org> writes: > revision-id: psergey(a)askmonty.org-20130802141209-4dqfvx2db8acxwbl > message: > MDEV-4816: rpl.rpl_trunc_temp fails in 10.0-serg > Temorary fix for a number of replication tests (rpl.rpl_temp_table_mix_row > rpl.rpl_trunc_temp rpl.rpl_current_user rpl.rpl_gtid_master_promote): The patch is not ok to push. We should not introduce a special flag just for the table gtid_slave_pos. Instead of setting table->no_replicate in the first place in rpl_slave_state::truncate_state_table() and rpl_slave_state::record_gtid(), we can clear the OPTION_BIN_LOG option. I checked MySQL 5.6 code and this is what they do for their similar "crash-safe replication" tables. So remove the two statements "table->no_replicate= 1" in rpl_gtid.cc and replace with clear of OPTION_BIN_LOG in thd->variables.option_bits. Then table->no_replicate is not touched and the new logic in THD::decide_logging_format() does not trigger, so the problem should be solved. In rpl_slave_state::truncate_state_table() we can use tmp_disable_binlog() and reenable_binlog() I think. In record_gtid() it is probably better to combine the clearing of OPTION_BIN_LOG with the existing save/restore of variables.option_bits. > The question of what should be done when a user issues a statement that > explicitly modifies mysql.gtid_slave_pos table remains open. I do not think we need to do anything special for this case. User explicitly modifying the table should be treated no different than any other table. Hope this helps, - Kristian. > === modified file 'sql/rpl_gtid.cc' > --- a/sql/rpl_gtid.cc 2013-06-21 09:53:46 +0000 > +++ b/sql/rpl_gtid.cc 2013-08-02 14:12:09 +0000 > @@ -210,6 +210,7 @@ rpl_slave_state::truncate_state_table(TH > { > table= tlist.table; > table->no_replicate= 1; > + table->s->is_gtid_slave_pos= TRUE; // TEMPORARY CODE > err= table->file->ha_truncate(); > > if (err) > @@ -346,6 +347,7 @@ rpl_slave_state::record_gtid(THD *thd, c > goto end; > > table->no_replicate= 1; > + table->s->is_gtid_slave_pos= TRUE; // TEMPORARY CODE > if (!in_transaction) > thd->variables.option_bits&= > ~(ulonglong)(OPTION_NOT_AUTOCOMMIT|OPTION_BEGIN); > > === modified file 'sql/sql_class.cc' > --- a/sql/sql_class.cc 2013-07-24 10:43:57 +0000 > +++ b/sql/sql_class.cc 2013-08-02 14:12:09 +0000 > @@ -5001,7 +5001,7 @@ int THD::decide_logging_format(TABLE_LIS > DBUG_PRINT("info", ("table: %s; ha_table_flags: 0x%llx", > table->table_name, flags)); > > - if (table->table->no_replicate) > + if (table->table->no_replicate && !table->table->s->is_gtid_slave_pos) > { > /* > The statement uses a table that is not replicated. > > === modified file 'sql/table.cc' > --- a/sql/table.cc 2013-07-21 14:43:42 +0000 > +++ b/sql/table.cc 2013-08-02 14:12:09 +0000 > @@ -315,6 +315,8 @@ TABLE_SHARE *alloc_table_share(const cha > strmov(share->path.str, path); > share->normalized_path.str= share->path.str; > share->normalized_path.length= path_length; > + /* TEMPORARY FIX: if true, this means this is mysql.gtid_slave_pos table */ > + share->is_gtid_slave_pos= FALSE; > share->table_category= get_table_category(& share->db, & share->table_name); > share->set_refresh_version(); > share->open_errno= ENOENT; > > === modified file 'sql/table.h' > --- a/sql/table.h 2013-07-21 14:39:19 +0000 > +++ b/sql/table.h 2013-08-02 14:12:09 +0000 > @@ -644,6 +644,8 @@ struct TABLE_SHARE > LEX_STRING normalized_path; /* unpack_filename(path) */ > LEX_STRING connect_string; > > + bool is_gtid_slave_pos; > + > /* > Set of keys in use, implemented as a Bitmap. > Excludes keys disabled by ALTER TABLE ... DISABLE KEYS. > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits

1 0

[Maria-developers] COT() function
by Ian Gilfillan 03 Aug '13

03 Aug '13

Hi SELECT COT(0) returns: ERROR 1690 (22003): DOUBLE value is out of range in 'cot(0)' in both MariaDB 10 and 5.5, but in MySQL 5.1 it returns NULL. I can't find a changelog reference to this, but I see it's still documented as returning NULL in MySQL 5.5 (I don't have this to test on). Is this a MariaDB-specific change? When did this change in behaviour happen? thanks, ian

3 5

[Maria-developers] Please review: [Commits] Rev 3797: MDEV-4816: rpl.rpl_trunc_temp fails in 10.0-serg in file:///data0/psergey/dev2/10.0-serg/
by Sergei Petrunia 02 Aug '13

02 Aug '13

Hi Kristian, In order to get rid of test failures of 10.0-serg, I have pushed the below patch. It seems to have the right effect, but I have no confidence about this patch being achitecturally correct, etc. Could you please review this and/or make another fix for the problem? ----- Forwarded message from Sergey Petrunya <psergey(a)askmonty.org> ----- From: Sergey Petrunya <psergey(a)askmonty.org> To: commits(a)mariadb.org X-Mailer: mail (GNU Mailutils 2.2) Date: Fri, 2 Aug 2013 18:12:11 +0400 (MSK) Subject: [Commits] Rev 3797: MDEV-4816: rpl.rpl_trunc_temp fails in 10.0-serg in file:///data0/psergey/dev2/10.0-serg/ At file:///data0/psergey/dev2/10.0-serg/ ------------------------------------------------------------ revno: 3797 revision-id: psergey(a)askmonty.org-20130802141209-4dqfvx2db8acxwbl parent: svoj(a)mariadb.org-20130729140849-dciz5ffq25ciip6q committer: Sergey Petrunya <psergey(a)askmonty.org> branch nick: 10.0-serg timestamp: Fri 2013-08-02 18:12:09 +0400 message: MDEV-4816: rpl.rpl_trunc_temp fails in 10.0-serg Temorary fix for a number of replication tests (rpl.rpl_temp_table_mix_row rpl.rpl_trunc_temp rpl.rpl_current_user rpl.rpl_gtid_master_promote): - THD::decide_logging_format() should not assume that mysql.gtid_slave_pos is a non-replicated table. This used to cause unintended behavior for COMMIT statement: replication would switch to row-based, etc. The question of what should be done when a user issues a statement that explicitly modifies mysql.gtid_slave_pos table remains open. === modified file 'sql/rpl_gtid.cc' --- a/sql/rpl_gtid.cc 2013-06-21 09:53:46 +0000 +++ b/sql/rpl_gtid.cc 2013-08-02 14:12:09 +0000 @@ -210,6 +210,7 @@ rpl_slave_state::truncate_state_table(TH { table= tlist.table; table->no_replicate= 1; + table->s->is_gtid_slave_pos= TRUE; // TEMPORARY CODE err= table->file->ha_truncate(); if (err) @@ -346,6 +347,7 @@ rpl_slave_state::record_gtid(THD *thd, c goto end; table->no_replicate= 1; + table->s->is_gtid_slave_pos= TRUE; // TEMPORARY CODE if (!in_transaction) thd->variables.option_bits&= ~(ulonglong)(OPTION_NOT_AUTOCOMMIT|OPTION_BEGIN); === modified file 'sql/sql_class.cc' --- a/sql/sql_class.cc 2013-07-24 10:43:57 +0000 +++ b/sql/sql_class.cc 2013-08-02 14:12:09 +0000 @@ -5001,7 +5001,7 @@ int THD::decide_logging_format(TABLE_LIS DBUG_PRINT("info", ("table: %s; ha_table_flags: 0x%llx", table->table_name, flags)); - if (table->table->no_replicate) + if (table->table->no_replicate && !table->table->s->is_gtid_slave_pos) { /* The statement uses a table that is not replicated. === modified file 'sql/table.cc' --- a/sql/table.cc 2013-07-21 14:43:42 +0000 +++ b/sql/table.cc 2013-08-02 14:12:09 +0000 @@ -315,6 +315,8 @@ TABLE_SHARE *alloc_table_share(const cha strmov(share->path.str, path); share->normalized_path.str= share->path.str; share->normalized_path.length= path_length; + /* TEMPORARY FIX: if true, this means this is mysql.gtid_slave_pos table */ + share->is_gtid_slave_pos= FALSE; share->table_category= get_table_category(& share->db, & share->table_name); share->set_refresh_version(); share->open_errno= ENOENT; === modified file 'sql/table.h' --- a/sql/table.h 2013-07-21 14:39:19 +0000 +++ b/sql/table.h 2013-08-02 14:12:09 +0000 @@ -644,6 +644,8 @@ struct TABLE_SHARE LEX_STRING normalized_path; /* unpack_filename(path) */ LEX_STRING connect_string; + bool is_gtid_slave_pos; + /* Set of keys in use, implemented as a Bitmap. Excludes keys disabled by ALTER TABLE ... DISABLE KEYS. _______________________________________________ commits mailing list commits(a)mariadb.org https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits ----- End forwarded message ----- -- BR Sergei -- Sergei Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] [Commits] Rev 3215: MDEV-4823: Server crashes in Item_func_not::fix_fields on creating a table with a virtual column using NOT in file:///home/bell/maria/bzr/work-maria-5.2-MDEV-4823/
by Sergei Petrunia 31 Jul '13

31 Jul '13

Hi Sanja, On Tue, Jul 30, 2013 at 11:00:59AM +0300, sanja(a)montyprogram.com wrote: > At file:///home/bell/maria/bzr/work-maria-5.2-MDEV-4823/ > > ------------------------------------------------------------ > revno: 3215 > revision-id: sanja(a)montyprogram.com-20130730080057-ylgl6h11l2b3yflc > parent: wlad(a)montyprogram.com-20130709202457-6n9syq59jdd2sy3b > committer: sanja(a)montyprogram.com > branch nick: work-maria-5.2-MDEV-4823 > timestamp: Tue 2013-07-30 11:00:57 +0300 > message: > MDEV-4823: Server crashes in Item_func_not::fix_fields on creating a table with a virtual column using NOT > > fix_field() call protocol was brocken (zero pointer passed as link to item which is possible only if you are sure that there can not be Items which transforms). ... > === modified file 'sql/table.cc' > --- a/sql/table.cc 2013-01-16 00:46:27 +0000 > +++ b/sql/table.cc 2013-07-30 08:00:57 +0000 > @@ -1812,7 +1812,7 @@ bool fix_vcol_expr(THD *thd, > bool result= TRUE; > TABLE_LIST tables; > TABLE_LIST *save_table_list, *save_first_table, *save_last_table; > - int error; > + int error= 0; > Name_resolution_context *context; > const char *save_where; > char* db_name; > @@ -1860,7 +1860,13 @@ bool fix_vcol_expr(THD *thd, > save_use_only_table_context= thd->lex->use_only_table_context; > thd->lex->use_only_table_context= TRUE; > /* Fix fields referenced to by the virtual column function */ > - error= func_expr->fix_fields(thd, (Item**)0); > + if (!func_expr->fixed) > + error= func_expr->fix_fields(thd, (Item**)&vcol_info->expr_item); There is no need to use (Item**) cast, please remove it. Ok to push after that. > + > + /* fix_fields could change the expression */ > + func_expr= vcol_info->expr_item; > + /* Number of columns will be checked later */ > + > /* Restore the original context*/ > thd->lex->use_only_table_context= save_use_only_table_context; > context->table_list= save_table_list; > BR Sergei -- Sergei Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] Plugin security
by Leif Walsh 31 Jul '13

31 Jul '13

Hi, I'm working on adding a plugin API (not exactly a storage engine API, rather an extension one) to TokuMX[1] and I'm looking for advice on security. The basic idea is fairly simple: a plugin is a shared library (we're only targeting Linux right now, by the way) that defines a symbol to bootstrap the loading of additional functionality. We have an admin-only command that calls dlopen and dlsym to load the plugin. I haven't read any of the MariaDB plugin code yet, so pointers to where to start reading would be appreciated. Apart from that, I'm just looking for any advice from you all about what to watch out for. My concerns break down into two main areas: 1. When loading a single plugin, what should I watch out for? The command itself is admin-only so calling it is protected, but what about an attacker that drops a malicious plugin in a directory earlier on the load path? What should I check about the permissions/owner of the library file and the directory in which it lives? Is erroring out on world-writable libraries enough? If my process is setuid, should I be more careful? What does Maria do here? 2. About plugin persistence/autoloading: I understand MariaDB has a system table that tracks which plugins are installed and automatically loads those on startup. I believe I can do the same thing in TokuMX and I can restrict access to that collection to the admin user. What does MariaDB store in that table and how does it use it? I could imagine storing the checksum of the installed plugin in the system table and verifying it before loading the plugin, but this seems rather draconian and it might make plugin upgrades too difficult. Does MariaDB verify the permissions of the data files that store this system table? I could imagine an attack where you would replace this system table with one that includes instructions to load your malicious plugin, while the server is offline. Again, how does MariaDB solve these problems? Another option is to autoload all libraries in some directory, and to just be very careful about the permissions of that directory, but this seems very hard to get totally right so I'm leaning away from it. [1]: http://www.tokutek.com/products/tokumx-for-mongodb -- Cheers, Leif

3 5

[Maria-developers] Master slave
by Richard Couture 30 Jul '13

30 Jul '13

Is it possible to set up replication in such a manner as to permit writing records to either a master or a slave replica? We want to set up a fail safe replica that can easily take over when our connection to the main replica fails, which, unfortunately, is fairly often. I read a document, some time ago which mentioned several types of replication, i.e.: by record, by table ... and the pitfalls, like IDs not necessarily matching... which confused me more than anything else... Can someone point me to some good documentation to enable me to pick the correct type of replication that I need to accomplish my task?? Thanks Richard -- LinuxCabal Asociación Civil Ing. Richard Couture Novell CNE, ECNE, MCNE HP/Compaq ASE Tel.: (+52) (333) 145-2638 Cel.: (+52) (044) 333 377-7505 Cel.: (+52) (044) 333 377-7506 Web: http://www.LinuxCabal.org E-Mail: rrc(a)linuxcabal.org Hosted en la nube Cloud Sigma - www.CloudSigma.com AVISO DE CONFIDENCIALIDAD: Este correo electrónico, incluyendo en su caso, los archivos adjuntos al mismo, pueden contener información de carácter confidencial y/o privilegiada, y se envían a la atención única y exclusivamente de la persona y/o entidad a quien va dirigido. La copia, revisión, uso, revelación y/o distribución de dicha información confidencial sin la autorización por escrito de LinuxCabal está prohibida. Si usted no es el destinatario a quien se dirige el presente correo, favor de contactar al remitente respondiendo al presente correo y eliminar el correo original incluyendo sus archivos, así como cualesquiera copia del mismo. Mediante la recepción del presente correo usted reconoce y acepta que en caso de incumplimiento de su parte y/o de sus representantes a los términos antes mencionados, LinuxCabal tendrá derecho a los daños y perjuicios que esto le cause.

1 0

Re: [Maria-developers] Implementing implicit primary key in mysql server
by Lixun Peng 30 Jul '13

30 Jul '13

Hi Sergei, You are right, let users add primary key is best. But I can't let users who don't want to create primary key can't use our MySQL service. Amazon RDS also allow users to create the tables without primary key, just change binlog_format to MIXED to solve replication problem. I think this feature is very useful for any MySQL cloud service providers, all of them will face this problem in one day. I will try to do some analysis/research in this feature implement first, I will need your help :-) I will notice any new updates in this email. Thanks, Lixun On Wed, Jun 19, 2013 at 5:14 PM, Sergei Golubchik <serg(a)mariadb.org> wrote: > Hi, Lixun! > > On Jun 18, Lixun Peng wrote: > > Hi, > > > > As we know, InnoDB has implicit primary key if a table hasn't defined > > a primary key. However MySQL server doesn't know this primary key, so > > this primary key will not apear in binlog. > > > > When we are using ROW format binlog for replication, if a table has no > any > > indexes, that's a disaster. If a table without indexes do a DML > > (UPDATE/DELETE), of course it will run a long time in master, but in > slave, > > it still need a long time. It will cause serious slave replication delay. > ... > > I think mysql server can do the same thing as InnoDB do, if user doesn't > > define the primary key, mysql can add the primary key automatically. > > > > How do you think? > > Well, that's doable. A much easier solution would be to require a user > to create a primary key. It's a one-line change: > > - Table_flags ha_table_flags() const { return cached_table_flags; } > + Table_flags ha_table_flags() const { return cached_table_flags | > HA_REQUIRE_PRIMARY_KEY; } > > But what you suggest is possible too, I believe. > > Regards, > Sergei > -- Senior Database Engineer @ Taobao.com Mobile Phone: +86 18658156856 (Hangzhou) Gtalk: penglixun(at)gmail.com Twitter: http://www.twitter.com/plinux Blog: http://www.penglixun.com

4 8

Re: [Maria-developers] [Commits] Rev 3792: main.gis test fixed. in file:///home/hf/wmar/10.0-serg/
by Sergey Vojtovich 27 Jul '13

27 Jul '13

Hi Alexey, I just checked 5.6 and 10.0-base, this test is passing as is. I'm a bit worried that something functional was broken during the merge. Regards, Sergey 26.07.2013, в 16:13, <holyfoot(a)askmonty.org> написал(а): > At file:///home/hf/wmar/10.0-serg/ > > ------------------------------------------------------------ > revno: 3792 > revision-id: holyfoot(a)askmonty.org-20130726110356-9s17c1kw558ls14a > parent: svoj(a)mariadb.org-20130726091143-o2v0tt08n81n923m > committer: Alexey Botchkov <holyfoot(a)askmonty.org> > branch nick: 10.0-serg > timestamp: Fri 2013-07-26 16:03:56 +0500 > message: > main.gis test fixed. > === modified file 'mysql-test/r/gis.result' > --- a/mysql-test/r/gis.result 2013-03-27 09:03:28 +0000 > +++ b/mysql-test/r/gis.result 2013-07-26 11:03:56 +0000 > @@ -918,25 +918,25 @@ SELECT Overlaps(@horiz1, @point2) FROM D > Overlaps(@horiz1, @point2) > 0 > DROP TABLE t1; > -create table t1(f1 geometry, f2 point, f3 linestring); > +create table t1(f1 geometry, f2 linestring, f3 linestring); > select f1 from t1 union select f1 from t1; > f1 > -insert into t1 (f2,f3) values (GeomFromText('POINT(1 1)'), > +insert into t1 (f2,f3) values (GeomFromText('LINESTRING(1 1, 2 2)'), > GeomFromText('LINESTRING(0 0,1 1,2 2)')); > select AsText(f2),AsText(f3) from t1; > AsText(f2) AsText(f3) > -POINT(1 1) LINESTRING(0 0,1 1,2 2) > +LINESTRING(1 1,2 2) LINESTRING(0 0,1 1,2 2) > select AsText(a) from (select f2 as a from t1 union select f3 from t1) t; > AsText(a) > -POINT(1 1) > +LINESTRING(1 1,2 2) > LINESTRING(0 0,1 1,2 2) > create table t2 as select f2 as a from t1 union select f3 from t1; > desc t2; > Field Type Null Key Default Extra > -a point YES NULL > +a linestring YES NULL > select AsText(a) from t2; > AsText(a) > -POINT(1 1) > +LINESTRING(1 1,2 2) > LINESTRING(0 0,1 1,2 2) > drop table t1, t2; > SELECT 1; > > === modified file 'mysql-test/t/gis.test' > --- a/mysql-test/t/gis.test 2013-03-27 09:03:28 +0000 > +++ b/mysql-test/t/gis.test 2013-07-26 11:03:56 +0000 > @@ -609,9 +609,9 @@ DROP TABLE t1; > # > # Bug#28763: Selecting geometry fields in UNION caused server crash. > # > -create table t1(f1 geometry, f2 point, f3 linestring); > +create table t1(f1 geometry, f2 linestring, f3 linestring); > select f1 from t1 union select f1 from t1; > -insert into t1 (f2,f3) values (GeomFromText('POINT(1 1)'), > +insert into t1 (f2,f3) values (GeomFromText('LINESTRING(1 1, 2 2)'), > GeomFromText('LINESTRING(0 0,1 1,2 2)')); > select AsText(f2),AsText(f3) from t1; > select AsText(a) from (select f2 as a from t1 union select f3 from t1) t; > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits

1 0

[Maria-developers] we have seen sporadic crashes in the Protocol::net_store_data function
by Rich Prohaska 26 Jul '13

26 Jul '13

Crash occurs when using the TokuDB storage engine in MySQL 5.5.30. Any insight into what may be causing this problem would be really nice. Sorry, but this is all the info that i have. Here is a stack trace from the mysql error log: #1 0x00000000006b760f in handle_fatal_signal (sig=11) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/signal_handler.cc:247 #2 <signal handler called> #3 0x00007ff213a3b447 in __strncasecmp_l_sse2 () from /lib64/libc.so.6 #4 0x00000000005651a7 in Protocol::net_store_data (this=0x0, this@entry=0x7ff18f01e4b8, from=0x0, length=1936615494) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/protocol.cc:59 #5 0x00000000005662e4 in Protocol::store_string_aux (this=this@entry=0x7ff18f01e4b8, from=<optimized out>, length=<optimized out>, fromcs=<optimized out>, tocs=tocs@entry=0x1022920 <my_charset_latin1>) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/protocol.cc:960 #6 0x00000000005663a5 in Protocol_text::store (this=0x7ff18f01e4b8, field=<optimized out>) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/protocol.cc:1119 #7 0x0000000000565ea0 in Protocol::send_result_set_row (this=0x7ff18f01e4b8, row_items=<optimized out>) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/protocol.cc:847 #8 0x000000000059fc37 in select_send::send_data (this=0x7fe338243b60, items=...) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/sql_class.cc:2184 #9 0x00000000005f920c in end_send (join=0x7fe77f4a4010, join_tab=<optimized out>, end_of_records=<optimized out>) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/sql_select.cc:12762 #10 end_send (join=0x7fe77f4a4010, join_tab=<optimized out>, end_of_records=<optimized out>) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/sql_select.cc:12739 #11 0x00000000005e8a32 in evaluate_join_record (join=join@entry=0x7fe77f4a4010, join_tab=join_tab@entry=0x7fe77f470a60, error=<optimized out>) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/sql_select.cc:11925 #12 0x00000000005e8c36 in sub_select (join=0x7fe77f4a4010, join_tab=0x7fe77f470a60, end_of_records=<optimized out>) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/sql_select.cc:11778 #13 0x00000000005f681d in do_select (join=join@entry=0x7fe77f4a4010, fields=fields@entry=0x7ff18f020130, table=table@entry=0x0, procedure=0x0) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/sql_select.cc:11543 #14 0x000000000060655e in JOIN::exec (this=0x7fe77f4a4010) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/sql_select.cc:2385 #15 0x0000000000602095 in mysql_select (thd=thd@entry=0x7ff18f01e000, rref_pointer_array=rref_pointer_array@entry=0x7ff18f0201f8, tables=<optimized out>, wild_num=<optimized out>, fields=..., conds=<optimized out>, og_num=0, order=0x0, group=0x0, having=0x0, proc_param=0x0, select_options=2147748608, result=result@entry=0x7fe338243b60, unit=0x7ff18f01fa00, select_lex=select_lex@entry=0x7ff18f020020) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/sql_select.cc:2604 #16 0x0000000000608235 in handle_select (thd=0x7ff18f01e000, lex=0x7ff18f01f950, result=0x7fe338243b60, setup_tables_done_option=0) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/sql_select.cc:297 #17 0x00000000005c23d1 in execute_sqlcom_select (thd=thd@entry=0x7ff18f01e000, all_tables=0x7fe338243270) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/sql_parse.cc:4604 #18 0x00000000005ca700 in mysql_execute_command (thd=thd@entry =0x7ff18f01e000) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/sql_parse.cc:2154 #19 0x00000000005cd7c2 in mysql_parse (parser_state=0x7ff21213a7e0, thd=0x7ff18f01e000, rawbuf=<optimized out>, length=<optimized out>) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/sql_parse.cc:5641 #20 mysql_parse (thd=0x7ff18f01e000, rawbuf=<optimized out>, length=65, parser_state=0x7ff21213a7e0) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/sql_parse.cc:5565 #21 0x00000000005cebcc in dispatch_command (command=COM_QUERY, thd=0x7ff18f01e000, packet=<optimized out>, packet_length=65) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/sql_parse.cc:1041 #22 0x000000000066495f in do_handle_one_connection (thd_arg=thd_arg@entry =0x7ff18f01e000) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/sql_connect.cc:853 #23 0x0000000000664a11 in handle_one_connection (arg=0x7ff18f01e000) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/sql_connect.cc:772 #24 0x00007ff214711851 in start_thread () from /lib64/libpthread.so.0

1 0

[Maria-developers] help with mdev-4687 - aria engina problem with <= operator
by Roberto Spadim 26 Jul '13

26 Jul '13

hi guys i have a problem with aria https://mariadb.atlassian.net/browse/MDEV-4687 some queries that depend on <= operator don't return right rows (return different from myisam or innodb) i'm getting back to myisam, or innodb these are read intensive tables, and innodb runs slower, and myisam have a crash safe problem (but runs...) please any help is very wellcome thanks guys -- Roberto Spadim

2 4

[Maria-developers] innodb key comparisons
by Rich Prohaska 25 Jul '13

25 Jul '13

i am interested in the performance of in memory join queries on tokudb and innodb and, for innodb, i would like to know how many key comparisons were needed when running the query. i counted the number of calls to a function called "cmp_dtuple_rec_with_match". does this include ALL of the key comparisons? thanks rich prohaska

1 0

[Maria-developers] Documentation for old regex library calls
by Sudheera Palihakkara 25 Jul '13

25 Jul '13

Hi all, I'm replacing the current regular expression library with modern library(pcre) for mariadb. I need some information about the ways that code uses the old regex library. The relevant codes are in sql/item_cmpfunc.cc directory. For exapmle in the function : int Item_func_regex::regcomp(bool send_error) I can't understand most of the lines, and the code is not well-commented. In the new library(pcre) uses different way to compile a regEx so I have to understand this and code for pcre preserving the functionalists of the class. Is there any documentation for this file or the way they used to work with old regex library ? thanks. -- *Sudheera Palihakkara.* Undergraduate Department of *Computer Science and Engineering, *Faculty of Engineering, *University of Moratuwa*, Sri Lanka.

2 2

[Maria-developers] some bugs in dingqing parallel replication
by nanyi607rao 22 Jul '13

22 Jul '13

Hi,guys I have worked on this branch https://code.launchpad.net/~knielsen/maria/dingqi-parallel-replication for some days, and found bugs listed below.May this would be helpful to you. 1, when slave switch on table filter,this bug could lead server crash. how to reappear: on slave set replicate-wild-ignore-table = test.t5 in config file on master do these operations CREATE TABLE test.t3 (a INT AUTO_INCREMENT PRIMARY KEY, b DECIMAL(20,20), c INT); SET INSERT_ID=1; SET @c=2; SET @@rand_seed1=10000000, @@rand_seed2=1000000; INSERT INTO t3 VALUES (NULL, RAND(), @c); codes lead this bug: In execute_single_transaction() case RAND_EVENT: need_remove_from_trans= true; if(!rli->is_deferred_event(ev)) delete ev; break; reason: Rand Event object is deleted in execute_single_transaction(), but it's pointer would be used is slave_execute_deferred_events() later. 2, SQL thread could read and apply some log events repeated. how to reappear: it's a little hard to reappear. if you set max_relay_log_size=100M and keep SQL thread closed to IO thread, this bug may reappear. codes lead this bug: In reopen_relay_log() rli->event_relay_log_pos= max(rli->event_relay_log_pos, BIN_LOG_HEADER_SIZE); my_b_seek(cur_log,rli->event_relay_log_pos); reason: when SQL thread use a hot log,but the hot log was closed by IO thread just recently, SQL thread need to reopen this log and set read offset to rli->event_relay_log_pos, while rli->event_relay_log_pos could be set new value in other thread for there are many threads apply log events.so rli->event_relay_log_pos could be less then rli->future_event_relay_log_pos. 3, SQL thread do not report error information in result of "show slave status"and replication do not stop, when the slave insert duplicate record into a table with primary key. how to reappear: Just need to change master_log_pos to read duplicate records from master. codes lead this bug: In execute_single_transaction() retry_transaction: ev= trans->event_list_head; ... ... if (ret && rli->trans_retries < slave_trans_retries) { ... goto retry_transaction; } reason: as I have sayed in other email: Rows_log_event::do_apply_event() do twice but return different results for m_curr_row==m_rows_end in the second time. 4, when do oparetions such as "show slave status" and "stop slave", it could be blocked for a long time. how to reappear: just do "show slave status" again and again. codes lead this bug: In the queue_event() case FORMAT_DESCRIPTION_EVENT: ... wait_for_all_dml_done(&mi->rli, true); and in process_io_rotate() wait_for_all_dml_done(&mi->rli, true); reason: IO thread could wait in wait_for_all_dml_done() while holding the rpl_mi->data_lock, so operations like "show slave status" could be blocked for waiting rpl_mi->data_lock. 5, "START SLAVE UNTIL" make replication stop in different place. how to reappear: suppose log events in relay log like: BEGIN; ------->pos1 LOG_EVENT1; LOG_EVENT2; COMMIT; ------>pos2 BEGIN; ------>pos3 LOG_EVENT3; --->stop_pos LOG_EVENT4; COMMIT; ------>pos4 If we do START SLAVE UNTIL relay_log_pos=stop_pos; The replication should stop at pos4 but it stop pos2. 6, log_event->thd is wrong. suppose log_event was read is thread_1 so the log_event->thd==thread_1, but this log_event may be dispatch to other thread (suppose thread_2).the log_event is applyed in thread_2 but the log_event->thd==thread_1.this problem can make log event apply failed in MySQL, but in mariaDB it seems ok. 2013-07-19 nanyi607rao

2 1

[Maria-developers] RFC getting information from image (blob)
by Roberto Spadim 21 Jul '13

21 Jul '13

Hi guys, i will implement a gis system, but my problem isn't gis, my problem is raster (bitmap) images I will save images at mariadb using longblobs And i will create a table with, datetime and image I will need read each r,g,b value from each pixel, example.. Select rvalue(1,2,loadimage(blobfield)) >From table Order by datetime Rvalue= red value of x1 and y2 What's the faster or at least, how to use less cpu and memory I didn't search about images udf functions for mysql Any help is wellcome and ideas too :) Well i will use others sum, max, min functions too I don't want to read all images at client side, i want that server do this job I don't know the image size (it can be any pratical size, max near to 16mega pixels), image format could be png, jpg, or another one but i can use client side to rewrite image file to right format I'm considering implement some imagemagick functions inside mariadb, what you think? Thanks guys

2 2

[Maria-developers] Make error - cannot find -lregex
by Sudheera Palihakkara 21 Jul '13

21 Jul '13

Hi all, I have done some editing to sql/item_cmpfunc.cc file and CMakeList filesand then when I run the make command( after running cmake) it halts with this error in the middle. ([68%]) Linking CXX executable explain_filename-t /usr/bin/ld.bfd.real: cannot find -lregex collect2: error: ld returned 1 exit status make[2]: *** [unittest/mysys/explain_filename-t] Error 1 make[1]: *** [unittest/mysys/CMakeFiles/explain_filename-t.dir/all] Error 2 make: *** [all] Error 2 what might have gone wrong. :( -- *Sudheera Palihakkara.* Undergraduate Department of *Computer Science and Engineering, *Faculty of Engineering, *University of Moratuwa*, Sri Lanka.

2 3

[Maria-developers] Structure of MariaDB 10.0 binary logs
by Giuseppe Maxia 20 Jul '13

20 Jul '13

Hi, Where can I find a detailed structure of the binary logs supported by MariaDB 10.0.x? Specifically, I would like to know * what are the differences between binary logs in MariaDB 5.5 and 10.0 * what are the differences between binary logs in MariaDB 10.0 and MySQL 5.6. The purpose of this request id to parse the binary logs for usage in external applications. Thanks Giuseppe

2 2

[Maria-developers] Help about patchs in mariadb - jira and lauchpad url
by Roberto Spadim 18 Jul '13

18 Jul '13

Hi guys, where patchs are commited in mariadb? for example this one: https://mariadb.atlassian.net/browse/MDEV-617 it don't have a link to any patch, there's a link to any lauchpad code? -- Roberto Spadim

2 5

[Maria-developers] MariaDB 5.5.32 Release Prep
by Daniel Bartholomew 18 Jul '13

18 Jul '13

All, I've begun the prep work for the MariaDB 5.5.32 release. The packages are uploaded and the mirrors should start picking them up soon. The draft changelog and release notes are here: - https://kb.askmonty.org/en/mariadb-5532-changelog/ - https://kb.askmonty.org/en/mariadb-5532-release-notes/ Please let me know ASAP if there's anything that needs to be mentioned/highlighted/removed/etc... Assuming all goes well, the plan is to activate the release at around 12:00 US Eastern Time tomorrow (18 Jul 2013). Thanks. -- Daniel Bartholomew Google+ - http://gplus.to/dbart Twitter - http://twitter.com/daniel_bart MariaDB: An Enhanced Drop-in Replacement for MySQL Website - http://mariadb.org Twitter - http://twitter.com/mariadb Google+ - http://google.com/+mariadb Facebook - http://fb.com/MariaDB.dbms Knowledgebase - http://kb.askmonty.org

2 4

Re: [Maria-developers] [Commits] Rev 3839: Fix of using uninitialized variadle. in file:///home/bell/maria/bzr/work-maria-5.5-valgrind/
by Sergei Petrunia 18 Jul '13

18 Jul '13

Hi Sanja, On Thu, Jul 18, 2013 at 11:16:21AM +0300, sanja(a)montyprogram.com wrote: > At file:///home/bell/maria/bzr/work-maria-5.5-valgrind/ > > ------------------------------------------------------------ > revno: 3839 > revision-id: sanja(a)montyprogram.com-20130718081618-6ax63urznfnqmzgt > parent: sanja(a)montyprogram.com-20130717140359-123bw8dzelj6wddn > committer: sanja(a)montyprogram.com > branch nick: work-maria-5.5-valgrind > timestamp: Thu 2013-07-18 11:16:18 +0300 > message: > Fix of using uninitialized variadle. > === modified file 'sql/sql_select.cc' > --- a/sql/sql_select.cc 2013-07-17 13:42:13 +0000 > +++ b/sql/sql_select.cc 2013-07-18 08:16:18 +0000 > @@ -15674,6 +15674,8 @@ create_internal_tmp_table_from_heap2(THD > const char *save_proc_info; > int write_err= 0; > DBUG_ENTER("create_internal_tmp_table_from_heap2"); > + if (is_duplicate) > + *is_duplicate= FALSE; > > if (table->s->db_type() != heap_hton || > error != HA_ERR_RECORD_FILE_FULL) > > === modified file 'sql/sql_union.cc' > --- a/sql/sql_union.cc 2013-07-16 05:22:17 +0000 > +++ b/sql/sql_union.cc 2013-07-18 08:16:18 +0000 > @@ -83,7 +83,7 @@ int select_union::send_data(List<Item> & > */ > return -1; > } > - bool is_duplicate; > + bool is_duplicate= FALSE; If you do the part in create_internal_tmp_table_from_heap2(), is this change really necessary? > /* create_internal_tmp_table_from_heap will generate error if needed */ > if (table->file->is_fatal_error(write_err, HA_CHECK_DUP) && > create_internal_tmp_table_from_heap(thd, table, > Ok to push either way. BR Sergei -- Sergei Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] LDAP authentication plugin
by Sergei 18 Jul '13

18 Jul '13

Hi, Ilija! On Jul 11, Ilija Lazarevic wrote: > Hi Sergei, > > The reason I'm writing you is because I'm interested > in details about this gsoc2013 proposal, and I saw you were > supposed to be the mentor. > > Is it open for contributing apart from gsoc? Yes, it is. But please, write to maria-developers(a)lists.launchpad.net next time, not to me personally. Regards, Sergei

3 2

[Maria-developers] help creating error messages
by Roberto Spadim 16 Jul '13

16 Jul '13

hi guys, how could i create a new error / warning message in mariadb? 1) how to add the message text to error list (there's a list, right?) 2) how to return (send) a error to client 3) how to add a message to 'warning list' and after get it with SHOW WARNINGS maybe some example or point lines / files that i should read thanks guys :) -- Roberto Spadim

2 5

[Maria-developers] More enum values in enum fields
by Roberto Spadim 16 Jul '13

16 Jul '13

Hi guys, i have a problem, that i don't know how to easly solve (ok i know but i want the user easy way) i have a ENUM column with some values (today 248 distinct values) i see that in a near future i will have more than 65535 different values inside this enum column, the first solution is change it to varchar, but my problem will be data size the enum is very optimized in size, a char is 'converted' to a smallint value my second solution is a int column, and a external table with int/text values, but here i need to change apps, i don't want the second one, but i don't want a very big database (char column) how could i change the limit of enum size? maybe a new enum2 type? with a limit of bigint value (64bits), ok maybe less (32bits) today (reading code) i see that big enum is a problem for .frm file that don't allow this size of enums and i need to get back to a second table, but what should be done to allow a very big enum field? there's a "easy" way to change mariadb code to allow a bigger enum size? thanks guys -- Roberto Spadim

4 19

[Maria-developers] The funcation of retry_transaction in parallen replication
by nanyi607rao 16 Jul '13

16 Jul '13

hi, In order to test "retry_transaction" in function execute_single_transaction(), I change Relay_log_pos to make SQL thread insert Duplicate entry, but I find it successed and do not report error! so I debug the function execute_single_transaction() and find the reason : at first time, the insert Rows_log_event::do_apply_event() return 121, that means insert fails. But when retry the insert Rows_log_event::do_apply_event(),it return 0, that means insert success! it seems the Rows_log_event::do_apply_event() do twice, but return different results. how this happened? is it a bug of Rows_log_event::do_apply_event() ? By the way, I do this test in mysql-5.5.30, do mariaDB has the some problem? 2013-07-12 nanyi607rao

3 3

[Maria-developers] What issue tracker should we use?
by Roberto Spadim 16 Jul '13

16 Jul '13

hi guys, i see that mariadb have three issue tracker, i'm right? i saw the https://askmonty.org/worklog/Server-RawIdeaBin/?tid=145 (ask monty), jira, and lauchpad what issue tracker should be used? there's any idea porting the others issue to the main issue tracker? -- Roberto Spadim SPAEmpresarial

3 4

[Maria-developers] Reserved words
by Ian Gilfillan 14 Jul '13

14 Jul '13

How can one find a definitive list of reserved words for a particular version? The comments in sql/lex.h say that that's a list of all reserved words and functions, but it appears to contain some words that are neither (such as ACTION). SELECT * FROM mysql.help_keyword also lists ACTION ACTION is listed in the MYSQL docs as an example of one of the keywords that MySQL permits "because many people previously used them"

2 1

[Maria-developers] Pluggable authentication, custom error messages
by Vladislav Vaintroub 14 Jul '13

14 Jul '13

Hi Serg, is there any possibility to return own error messages and codes , if pluggable authentication failed on client on some reason ? Case in point here is Kerberos (GSSAPI or SSPI) - it may fail on client or on server, with some GSSAPI/SSPI specific code. We would like user to know what exactly happened , rather than return a generic CR_ERROR. Perhaps user can fix the situation - maybe he just needs to obtain Kerberos ticket with kinit, or login as Windows domain user, or something like that. If error happens on the server side, at least there it is possible to write to error log. But on the client side, there is no error log, and writing to stdout/err is not a good idea. Is there a trick that allows custom messages on the client? Wlad

2 2

[Maria-developers] MariaDB 10/trunk (re)install destructively overwrites existing my.cnf* @ spec'd SYSCONFDIR
by darx＠sent.com 14 Jul '13

14 Jul '13

Hi, I've a custom my.cnf ls -al /usr/local/etc/mariadb/my.cnf -rw-r--r--+ 1 root root 7.9K Jul 13 13:03 /usr/local/etc/mariadb/my.cnf If I (re)install mariadb 10/trunk from a src build with SYSCONFDIR spec'd for that dir, as done similarly to a MySQL build, cd mariadb/bld cmake .. \ ... -DINSTALL_SYSCONFDIR=/usr/local/etc/mariadb \ ... make make install The INSTALL process *destructively* overwrites the existing my.cnf ... -- Up-to-date: /usr/local/etc/mariadb/logrotate.d/mysql -- Up-to-date: /usr/local/etc/mariadb/init.d/mysql -- Installing: /usr/local/etc/mariadb/my.cnf <================================ !! -- Up-to-date: /usr/local/etc/mariadb/my.cnf.d/client.cnf -- Up-to-date: /usr/local/etc/mariadb/my.cnf.d/mysql-clients.cnf -- Up-to-date: /usr/local/etc/mariadb/my.cnf.d/server.cnf ... with an older default cnf, ls -al /usr/local/etc/mariadb/my.cnf -rw-r--r--+ 1 root root 202 Jul 12 11:31 /usr/local/etc/mariadb/my.cnf with no backup made of the existing cnf, find /usr/local/etc/mariadb -type f -name 'my.cnf*' /usr/local/etc/mariadb/my.cnf This is critically different behavior that MySQL. If that's intended, how are we to exec a mariadb install without overwriting our config files/dirs? If it's not, I suspect this should be fixed. Thanks.

2 2

[Maria-developers] building MariaDB 10/trunk with external SSL generates errors/warnings. Docs are inconsistent. What's the right procedure?
by darx＠sent.com 13 Jul '13

13 Jul '13

hi, building cd /usr/local/src/mariadb bzr log | head -n 5 ------------------------------------------------------------ revno: 3766 [merge] committer: Alexander Barkov <bar(a)mariadb.org> branch nick: maria-10.0.ms timestamp: Wed 2013-07-10 18:46:33 +0400 reading ./BUILD/SETUP.sh ... 120 # SSL library to use.--with-ssl will select our bundled yaSSL # implementation of SSL. To use OpenSSL you will need to specify # the location of OpenSSL headers and libs on your system. # Ex --with-ssl=/usr <================================= OK, do this SSL_LIBRARY=--with-ssl ... configuring @ `cmake` cd /usr/local/src/mariadb rm -rf bld mkdir bld cd bld cmake .. \ -DWITH_SSL=/usr/local/ssl \ -DOPENSSL_INCLUDE_DIR=/usr/local/ssl/include \ -DOPENSSL_SSL_LIBRARY=/usr/local/ssl/lib64/libssl.so \ -DOPENSSL_CRYPTO_LIBRARY=/usr/local/ssl/lib64/libcrypto.so fails, returning, ... CMake Error at cmake/ssl.cmake:90 (MESSAGE): Wrong option for WITH_SSL. Valid values are : yes, no, bundled Call Stack (most recent call first): CMakeLists.txt:260 (MYSQL_CHECK_SSL) ... -- Configuring incomplete, errors occurred! vi ../CmakeLists.txt ... # Add bundled or system zlib. MYSQL_CHECK_ZLIB_WITH_COMPRESS() # Optionally add bundled yassl/taocrypt or system openssl. 260 MYSQL_CHECK_SSL() # Add readline or libedit. MYSQL_CHECK_READLINE() checking vi cmake/ssl.cmake ... # MYSQL_CHECK_SSL # # Provides the following configure options: # WITH_SSL=[yes|no|bundled] MACRO (MYSQL_CHECK_SSL) IF(NOT WITH_SSL) IF(WIN32) CHANGE_SSL_SETTINGS("bundled") ELSE() CHANGE_SSL_SETTINGS("no") ENDIF() ENDIF() IF(WITH_SSL STREQUAL "bundled") MYSQL_USE_BUNDLED_SSL() ELSEIF(WITH_SSL STREQUAL "system" OR WITH_SSL STREQUAL "yes") # Check for system library SET(OPENSSL_FIND_QUIETLY TRUE) INCLUDE(FindOpenSSL) FIND_LIBRARY(CRYPTO_LIBRARY crypto) MARK_AS_ADVANCED(CRYPTO_LIBRARY) INCLUDE(CheckSymbolExists) SET(CMAKE_REQUIRED_INCLUDES ${OPENSSL_INCLUDE_DIR}) CHECK_SYMBOL_EXISTS(SHA512_DIGEST_LENGTH "openssl/sha.h" HAVE_SHA512_DIGEST_LENGTH) SET(CMAKE_REQUIRED_INCLUDES) IF(OPENSSL_FOUND AND CRYPTO_LIBRARY AND HAVE_SHA512_DIGEST_LENGTH) SET(SSL_SOURCES "") SET(SSL_LIBRARIES ${OPENSSL_LIBRARIES} ${CRYPTO_LIBRARY}) SET(SSL_INCLUDE_DIRS ${OPENSSL_INCLUDE_DIR}) SET(SSL_INTERNAL_INCLUDE_DIRS "") SET(SSL_DEFINES "-DHAVE_OPENSSL") CHANGE_SSL_SETTINGS("system") ELSE() IF(WITH_SSL STREQUAL "system") MESSAGE(SEND_ERROR "Cannot find appropriate system libraries for SSL. Use WITH_SSL=bundled to enable SSL su ENDIF() MYSQL_USE_BUNDLED_SSL() ENDIF() ELSEIF(NOT WITH_SSL STREQUAL "no") MESSAGE(SEND_ERROR "Wrong option for WITH_SSL. Valid values are : yes, no, bundled") ENDIF() ENDMACRO() there's no accommodation for "--with-ssl=/path/to/ssl/install". If I instead cmake .. \ -DWITH_SSL=yes \ -DOPENSSL_INCLUDE_DIR=/usr/local/ssl/include \ -DOPENSSL_SSL_LIBRARY=/usr/local/ssl/lib64/libssl.so \ -DOPENSSL_CRYPTO_LIBRARY=/usr/local/ssl/lib64/libcrypto.so it returns, ... -- Running cmake version 2.8.11.1 -- MariaDB 10.0.3 -- Packaging as: mariadb-10.0.3-Linux-x86_64 -- Found OpenSSL: /usr/local/ssl/lib64/libssl.so;/usr/local/ssl/lib64/libcrypto.so (found version "1.0.1e") ... -- Configuring done then a bunch of these messages: ... CMake Warning at cmake/plugin.cmake:184 (ADD_LIBRARY): Cannot generate a safe runtime search path for target feedback because files in some directories may conflict with libraries in implicit directories: runtime library [libcrypto.so.1.0.0] in /usr/lib64 may be hidden by files in: /usr/local/ssl/lib64 Some of these libraries may not be found correctly. Call Stack (most recent call first): plugin/feedback/CMakeLists.txt:20 (MYSQL_ADD_PLUGIN) CMake Warning at cmake/libutils.cmake:259 (ADD_LIBRARY): Cannot generate a safe runtime search path for target libmysql because files in some directories may conflict with libraries in implicit directories: runtime library [libcrypto.so.1.0.0] in /usr/lib64 may be hidden by files in: /usr/local/ssl/lib64 Some of these libraries may not be found correctly. Call Stack (most recent call first): libmysql/CMakeLists.txt:374 (MERGE_LIBRARIES) CMake Warning at client/CMakeLists.txt:78 (ADD_EXECUTABLE): Cannot generate a safe runtime search path for target async_example because files in some directories may conflict with libraries in implicit directories: runtime library [libcrypto.so.1.0.0] in /usr/lib64 may be hidden by files in: /usr/local/ssl/lib64 Some of these libraries may not be found correctly. ... What's the right method to find/use external SSL libs/headers in a MariaDB build without errors or warnings? --darx

2 1

[Maria-developers] Doubts about index and optimization
by Roberto Spadim 13 Jul '13

13 Jul '13

Hi guys i have two doubts... First the r-tree index Could i index two integer or float or decimal columns using rtree And queries like X>=1 and x<=2 and y>=1 and y<=2 Be optimized using rtree? Another doubt... i dont have a mariadb now to test, i'm in cellphone... this kind of query is optimized if i have a x,y btree index? Btree will use x only, or x and y? Second doubt I'm thinking about enum rewrite optimzation and i didn't write anything yet, just studying mariadb code... When using queries like this Where x<=5 Or where x>=5 Or without = part Does mysql know what's the max and min value of x, and rewrite it to something X<=5. To. X in (0,1,2,3,4,5) when x is a integer for example and min(x)=0 I'm thinking about this, in conditions like the rtree optimization, example... we could optimize x and y with something like X>1 and x<5 and y>1 and y<5 to X in (2,3,4) and y in (2,3,4) It's something like enum optimization but using integers and max min informations Well i don't know the full code but i didn't found this information (i don't remember at least) Anyone know something about this in source? If not i will consider this optimization when doing the enum optimization (i don't remember the mdev number now, but anyone interested in this optimization can search in jira and find it, i reported and done any job yet just the raw and high implementation idea) Thanks guys

1 0

[Maria-developers] OLAP
by Roberto Spadim 12 Jul '13

12 Jul '13

Hi guys, there's any good OLAP tool for mariadb? maybe one that we could package with mariadb as default? -- Roberto Spadim

2 5

[Maria-developers] Status on MDEV-4506, parallel replication
by Kristian Nielsen 12 Jul '13

12 Jul '13

So I have been working for some weeks now on implementing MDEV-4506. This task is about making the slave apply (some) events in parallel threads, to speed up replication and reduce the risk of slave not being able to keep up with a busy master. Events are applied in parallel on the slave if they were group-committed together on the master. This is an easy way to detect transactions that are independent. Note that this is transparent to applications; while transactions are executed in parallel on the slave, they are still committed in the same order as on the master. I also added parallel execution of events with different GTID domain id. This makes testing a lot easier (no need to carefully arrange timing to get a specific group commit on the master), and also really is the whole point of much of my hard work on GTID. So if we have multi-source M1->S1, M2->S1, and S1->S2, then S2 will be able to execute events from M1 in parallel with those from M2, just like S1 can. And the user can explicitly set the domain_id different for eg. a long-running ALTER or UPDATE, and this way get it to run in parallel and not cause a huge replication delay. ---- On the master, I added to each GTID event a commit_id. If two transactions group-commit together, they are binlogged with the same commit_id; if not, they get different commit_ids. Thus, the slave can detect the possibility of executing two transactions in parallel by checking if the commit_ids are equal. On the master, I implemented --binlog-commit-wait-count=N and --binlog-commit-wait-usec=T. A transaction will wait at most T microseconds for at least N transactions to queue up and be ready for group commit. This allows to deliberately delay transactions on the master in order to get bigger group commits and thus better opportunity for parallel execution (and again it makes testing easier). On the slave, I implemented --slave-parallel-threads=N. If N>0, that many threads will be spawned, and events will be executed in parallel (if possible) by those threads. ---- The current code is pushed here (it is based on 10.0-base): lp:~maria-captains/maria/10.0-knielsen It is still far from finished, but it now works sufficiently that I could do some basic benchmarking (just on my laptop, completely unscientifically). First, I prepared a binlog on the master with plenty of opportunity for parallel replication. I started the master with --binlog-commit-wait-count=20 --binlog-commit-wait-usec=1000000. I then ran this Gypsy script: ----------------------------------------------------------------------- i|1|DROP TABLE IF EXISTS t1 i|1|CREATE TABLE t1 (a INT PRIMARY KEY, b VARCHAR(100)) ENGINE=InnoDB p|1|REPLACE INTO t1 (a,b) VALUES (? MOD 10000, ?)|int,varchar /home/knielsen/my/gypsy/words ----------------------------------------------------------------------- gypsy --queryfile=simple_replace_load.gypsy --duration=20 --threads=40 This results in a binlog with about 65k updates to the table, group-committed in batches of 20 transactions. I then started a fresh slave and let it replicate everything with START SLAVE UNTIL; the time to replicate all the events is then easy to see in the slave error log. The time to replicate everything with unmodified 10.0-base was 99 seconds. With --slave-parallel-threads=25, it was just 22 seconds. So that is a 4.5 times speedup, which is quite promising. Also note that at 22 seconds, the slave is within 10% of the speed of the master. ---- But as I said, there is still significant work left to do. I put a ToDo list at the top of sql/rpl_parallel.cc. Some of the big remaining issues: 1. The existing code is not thread-safe for class Relay_log_info. This class contains a bunch of stuff that is specific to executed transactions, not related to relay-log at all. This needs to be moved to the new struct rpl_group_info I introduced, and all code updated to pass around a pointer to that struct instead. There may also be a need to add additional locking on Relay_log_info, existing code needs review for this. 2. Error handling needs to be implemented, it is rather more complex in the parallel case. If one transaction fails (and retry also fails), then we need to somehow get hold of all later transactions that are in the process of parallel execution, and abort them + roll them back. Otherwise we get inconsistent binlog position for the next slave restart. 3. In the old code, when the SQL thread is stopped, it has logic to let the current event group (=transaction) replicate to completeness first, with a timeout to force a stop in the middle of the event group if eg. the master has disappeared. This logic needs to be re-implemented to work when having any number of event groups executing in parallel. (It is important to let the groups complete execution when doing non-transactional stuff that cannot be rolled back, otherwise again the slave position becomes inconsistent for next slave restart). So as you see, there is quite a bit of work left on this (as well as on GTID). So I would very much welcome any help on this to avoid causing delays for 10.0-GA ... - Kristian.

1 0

[Maria-developers] a bug of ding-qing parallel replication
by nanyi607rao 11 Jul '13

11 Jul '13

hi guys, I find a bug of ding-qing parallel replication. let's consider a table like this: create table t1( a int(11) NOT NULL DEFAULT '0', b varchar(10), PRIMARY KEY (a) )ENGINE=InnoDB if we do transactions on master(binlog_format=ROW) like this: trans 1: begin; insert into t1 values(1, "tt"); update t1 set a=2 where a=1; commit; trans 2; begin; update t1 set b="kk" where a=2; commit; then this two transactions binlog transmit to the slave, because trans 1 and trnas 2 has different primary keys, it would disaptch in different threads like this: thread 1 do trans 1 thread 2 do trans 2 thread 2 may update t1 before thread 1, then it will fail. maybe thread 2 can retry sever times ,but it don't solve this problem compeletly.. 2013-07-11 nanyi607rao

3 2

[Maria-developers] deferred_event
by nanyi607rao 10 Jul '13

10 Jul '13

Hi guys, as I know, when replication filter used, the defered events is a container hold on events(such as Intvar-, Rand-, User_var- log_event)that be applied delay. why those events need be applied delay? and what is parent Query mentioned in codes? Thanks! 2013-07-10 nanyi607rao

2 1

[Maria-developers] buildbot config
by Leif Walsh 09 Jul '13

09 Jul '13

Hi, I'm learning buildbot and I was wondering if I could take a look at the master.cfg for buildbot.askmonty.org as a reference for learning common practices and things like that. Is it available somewhere? -- Cheers, Leif

2 2

[Maria-developers] merge for MySQL56 FSP data types
by Alexander Barkov 09 Jul '13

09 Jul '13

Hi Serg, Please review the patch merging MySQL-5.6 fractional second precision enabled data types. Thanks.

3 7

[Maria-developers] Connect: compilation warnings
by Alexander Barkov 08 Jul '13

08 Jul '13

Hi Oliver, I fixed almost all compilation warnings that happened in Connect on my Fedora 17 laptop with gcc-4.7.2. Most of them were "variable is initialized but not used" and "no previous declaration". There were not serious, but very unpleasant for developers. (Note, we try to keep the tree in a shape that it compiles without warnings. Connect must follow this rule as well!!!) I think there was a bug in one place: how SQLSetStmtAttr() was called. See "bzr diff -c 3827" for details. The value of m_RowsetSize was erroneously passed, but SQLSetStmtAttr() expects the address of a SQLUINT variable (not a value!). See the ODBC docs for the details: http://msdn.microsoft.com/en-us/library/windows/desktop/ms712631%28v=vs.85%… One compilation warning still remains: maria-10.0-connect/storage/connect/tabsys.cpp:279:26: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast] It looks like a bug. Cast from 4-byte integer to 8-byte pointer is obviously wrong. Please address the problem. Thanks.

1 0

[Maria-developers] Warning in dynamic columns code
by Sergei Petrunia 08 Jul '13

08 Jul '13

Hi Sanja, I have been seeing this warning on a number of platforms: 10.0-monty/sql/item_strfunc.cc: In member function ‘virtual bool Item_func_dyncol_create::fix_fields(THD*, Item**)’: 10.0-monty/sql/item_strfunc.cc:3887: warning: suggest a space before ‘;’ or explicit braces around empty body in ‘for’ statement I assume it is harmless, but just in case, passing it over to you. Could you please check it and get rid of it? Thanks Sergei -- Sergei Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

2 1

[Maria-developers] SO_REUSEPORT and mariadb
by Roberto Spadim 08 Jul '13

08 Jul '13

hi guys, i was reading the SO_REUSEPORT of kernel 3.9, and i didn't found it in mariadb maybe we could use it? here are the SO_REUSEPORT topics: https://lwn.net/Articles/542629/ http://linux.dell.com/files/presentations/Linux_Plumbers_Conf_2010/Scaling_… here the important parts: ------------------ 1)The first of the traditional approaches is to have a single listener thread that accepts all incoming connections and then passes these off to other threads for processing. The problem with this approach is that the listening thread can become a bottleneck in extreme cases. In early discussions<http://thread.gmane.org/gmane.linux.network/102140/focus=102150> on SO_REUSEPORT, Tom noted that he was dealing with applications that accepted 40,000 connections per second. Given that sort of number, it's unsurprising to learn that Tom works at Google. 2)The second of the traditional approaches used by multithreaded servers operating on a single port is to have all of the threads (or processes) perform an accept() call on a single listening socket in a simple event loop of the form: while (1) { new_fd = accept(...); process_connection(new_fd); } ------------------ i didn -- Roberto Spadim

1 1

[Maria-developers] Replication
by Roberto Spadim 05 Jul '13

05 Jul '13

hi guys, i don't know if it's the better place to ask... maybe in user list is the right one, please tell me what's better... i'm using replication, and i need to mark all tables that don't replicate, it's very sad boring time consuming, etc etc... there's a easier way to mark tables? maybe alter table xxxx REPLICATION=ON/OFF if not, could we add this feature? thanks guys -- Roberto Spadim

1 0

[Maria-developers] Developer course
by Roberto Spadim 05 Jul '13

05 Jul '13

hi guys, there's a internal mysql/mariadb developer course? i'm studying source, but it's a bit difficult understand what's the best class/methods to use, maybe a course could help me developing and understanding better the source any one know about a course like it? i will focus on query rewrite (select.cc), optimizer (opt_range.cc/opt_*) and partition engine, others engine (myisam,heap,innodb,aria) is not my priority now -- Roberto Spadim

2 3

[Maria-developers] Review request : MDEV-573 (mysqlreport update)
by Jean Weisbuch 04 Jul '13

04 Jul '13

Hi, I have been doing modifications to the MariaDB mysqlreport version (which is shipped on the Debian/Ubuntu packages and can be found on the debian/additions/ directory), especially to add stats/informations about MariaDB specific functions but also to fix some bugs and limitations of the tool. As of today, the script works without throwing a warning and seems to output correct informations on MariaDB 5.5 (either with or without threadpool and binary log activated), MariaDB 5.5 + TokuDB, MySQL 5.1, MySQL 5.5 and it solves some small bugs and issues that could lead warning messages, invalid values returned or truncated values. The main additions are the AriaDB pagecache stats, binlog cache stats, subquery cache stats, TokuDB stats and Threadpool stats. There are also some "cosmetic" changes/fixes : the addition of values such as the % of disk tables used or the InnoDB read "From file" modified to "From disk" as of the documentation of the status variables states ; values such as the threads cache size or the table cache size that were limited to 3 chars are now limited to 5 chars ; the per seconds stats was truncated to 5 chars max on Questions and Query Cache sections (for example 153.6k/s was truncated to 153.6/s). The JIRA task : https://mariadb.atlassian.net/browse/MDEV-573 The GIT repository where i put the latest revisions (i only have been using it lately but i commented all the changes and bugfixes along the time on the JIRA task and could provide older revisions/diff if required) : https://github.com/jb-boin/mariadb/blob/5.5.30/debian/additions/mysqlreport I would like to be sure that the values/stats i added are really useful to monitor (and the calculations of these correct) and that the way informations are shown, named and ordered is clear enough ; for example, the way the test that determines if its a MariaDB or a MySQL server might not be the best possible (it checks if the Aria engine is available). There is still an upstream bug on the "Questions" and the "SELECT and Sort" section, the %Total values are not correctly calculated and lead to values > 100%, i havent got time to give it a look yet and i included in the example output below values hitting that bug. Here is an example of the output of the latest revision on a server having about all the possible functions enabled (values are mixed from different servers) : MariaDB 5.5.30-tokudb-7.0.1 uptime 70 9:4:47 Thu Jul 4 01:48:24 2013 __ Key _________________________________________________________________ Buffer used 651.41M of 1.00G %Used: 63.61 Current 675.43M %Usage: 65.96 Write hit 79.79% Read hit 99.99% __ Questions ___________________________________________________________ Total 2.57M 0.4/s -Unknown 39.92M 6.6/s %Total: 1550.4 DMS 39.78M 6.5/s 1545.3 Com_ 2.71M 0.4/s 105.09 COM_QUIT 444 0.0/s 0.02 Slow 4 s 285 0.0/s 0.01 %DMS: 0.00 Log: OFF DMS 39.78M 6.5/s 1545.3 UPDATE 39.76M 6.5/s 1544.5 99.95 SELECT 21.11k 0.0/s 0.82 0.05 REPLACE 0 0/s 0.00 0.00 DELETE 0 0/s 0.00 0.00 INSERT 0 0/s 0.00 0.00 Com_ 2.71M 0.4/s 105.09 show_proces 2.51M 0.4/s 97.33 admin_comma 152.51k 0.0/s 5.92 show_status 9.23k 0.0/s 0.36 __ Rows ________________________________________________________________ Rows 186.00G 240.3k/s Using idx 168.06G 217.1k/s %Index: 90.35 Rows/question 158.90 __ SELECT and Sort _____________________________________________________ Scan 42.33k 0.0/s %SELECT: 200.53 Range 0 0/s 0.00 Full join 19.27k 0.0/s 91.29 Range check 0 0/s 0.00 Full rng join 0 0/s 0.00 Sort scan 0 0/s Sort range 0 0/s Sort mrg pass 0 0/s __ Table Locks _________________________________________________________ Waited 8.86M 11.4/s %Total: 0.46 Immediate 1.93G 2.5k/s __ Tables ______________________________________________________________ Open 6843 of 8192 %Cache: 83.53 Opened 14.14k 0.0/s __ Connections _________________________________________________________ Max used 5403 of 22000 %Max: 24.56 Total 176.48M 228.0/s __ Created Temp ________________________________________________________ Disk table 7.76k 0.0/s %Disk: 14.55 Table 53.38k 0.0/s Size: 64.0M File 12 0.0/s __ Threads _____________________________________________________________ Running 274 of 2788 Created 181.68k 0.2/s Slow 0 0/s Threadpool 892 of 1200 %Used: 74.33 Running 458 of 1200 %Running: 38.17 Idle 434 of 1200 %Idle: 36.17 __ Aborted _____________________________________________________________ Clients 5.03k 0.0/s Connects 3.65k 0.0/s __ Bytes _______________________________________________________________ Sent 1.51T 1.9M/s Received 445.91G 576.0k/s __ InnoDB Buffer Pool __________________________________________________ Usage 25.97G of 26.00G %Usage: 99.89 Read hit 100.00% Pages Free 1.83k %Total: 0.11 Data 1.67M 98.23 %Drty: 0.07 Misc 28381 1.67 Latched 0 0.00 Reads 527.54G 681.5k/s From disk 100.00k 0.1/s %Disk: 0.00 Ahead Rnd 0 0/s Writes 5.21G 6.7k/s Flushes 17.35M 22.4/s Wait Free 9 0.0/s %Wait: 0.00 __ InnoDB Lock _________________________________________________________ Waits 36660726 47.4/s Current 0 Time acquiring Total 627441780 ms Average 171 ms Max 1537520 ms __ InnoDB Data, Pages, Rows ____________________________________________ Data Reads 512.26k 0.7/s Writes 181.92M 235.0/s fsync 2.07M 2.7/s Pending Reads 0 Writes 0 fsync 0 Pages Created 3.72M 4.8/s Read 512.18k 0.7/s Written 17.35M 22.4/s Rows Deleted 175.85M 227.2/s Inserted 200.46M 259.0/s Read 174.52G 225.4k/s Updated 224.40M 289.9/s __ Aria Pagecache ______________________________________________________ Buffer used 3.43M of 512.00M %Used: 0.67 Current 20.26M %Usage: 3.96 Write hit 47.09% Read hit 99.96% __ Subquery Cache ______________________________________________________ Hit ratio 99.91% Hits 38.63G 49.9k/s Miss 35.97M 46.5/s __ Binary Log Cache _____________________________________________________ Disk use Transactional 0.43% Non transactional 0.50% __ TokuDB ______________________________________________________________ Cachetable 4.00G of 4.00G %Usage: 99.99 Miss 1.83M 0.3/s Evictions 1.95M 0.3/s Thanks in advance, Jean Weisbuch

1 0

Re: [Maria-developers] Rev 3824: MDEV-4472 Audit plugin
by Sergei Golubchik 03 Jul '13

03 Jul '13

Hi, Holyfoot! Generally, it looks pretty good. One comment, see below about MARIADB_ONLY. On Jul 02, holyfoot(a)askmonty.org wrote: > revno: 3824 > revision-id: holyfoot(a)askmonty.org-20130702213841-xwmzadaotzxqhz2k > parent: sergii(a)pisem.net-20130701155424-lx0dvaocejx0cn8l > committer: Alexey Botchkov <holyfoot(a)askmonty.org> > branch nick: mdev-4472 > timestamp: Wed 2013-07-03 02:38:41 +0500 > message: > MDEV-4472 Audit plugin. > Plugin implemented in plugin/server_audit/server_audit.c. > Rotating log file support modified to be mysql-service. > === added directory 'plugin/server_audit' > === added file 'plugin/server_audit/CMakeLists.txt' > --- a/plugin/server_audit/CMakeLists.txt 1970-01-01 00:00:00 +0000 > +++ b/plugin/server_audit/CMakeLists.txt 2013-07-02 21:38:41 +0000 > @@ -0,0 +1,16 @@ > +# Copyright (C) 2012 Monty Program Ab. > +# > +# This program is free software; you can redistribute it and/or modify > +# it under the terms of the GNU General Public License as published by > +# the Free Software Foundation; version 2 of the License. > +# > +# This program is distributed in the hope that it will be useful, > +# but WITHOUT ANY WARRANTY; without even the implied warranty of > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > +# GNU General Public License for more details. > +# > +# You should have received a copy of the GNU General Public License > +# along with this program; if not, write to the Free Software > +# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA > + > +MYSQL_ADD_PLUGIN(server_audit server_audit.c MODULE_ONLY) I'd remove MODULE_ONLY, if the plugin can be complied as a static plugin. No need to add artificial restrictions. > === added file 'plugin/server_audit/server_audit.c' > --- a/plugin/server_audit/server_audit.c 1970-01-01 00:00:00 +0000 > +++ b/plugin/server_audit/server_audit.c 2013-07-02 21:38:41 +0000 ... > +#ifndef MARIADB_ONLY > +#define MYSQL_SERVICE_LOGGER_INCLUDED > +#endif /*MARIADB_ONLY*/ Is MARIADB_ONLY normally defined? As far as I can see, it's not. So in the packages this plugin won't have MARIADB_ONLY. That's ok in your tree, but let's think about how to solve this before pushing into the main tree. Regards, Sergei

2 1

[Maria-developers] merge for MySQL56 temporal literals
by Alexander Barkov 03 Jul '13

03 Jul '13

Hi Serg, Please review the patch merging MySQL-5.6 temporal literals. Thanks.

3 4

[Maria-developers] MDEV-4667 DATE('string') incompability between mysql and mariadb
by Alexander Barkov 03 Jul '13

03 Jul '13

Hi Serg, my review comments for http://bazaar.launchpad.net/~maria-captains/maria/5.3-serg/revision/3665 The patch looks ok. Just a couple of small comments: > === modified file 'include/my_time.h' > --- include/my_time.h 2011-12-11 09:34:44 +0000 > +++ include/my_time.h 2013-07-03 07:46:20 +0000 > @@ -68,14 +68,17 @@ > #endif > > /* Flags to str_to_datetime */ > -#define TIME_FUZZY_DATE 1 > + > +/* > + TIME_FUZZY_DATES is used for the result will only be used for comparison > + purposes. Conversion is as relaxed as possible. > +*/ > +#define TIME_FUZZY_DATES 1 > #define TIME_DATETIME_ONLY 2 > #define TIME_TIME_ONLY 4 > -/* Must be same as MODE_NO_ZERO_IN_DATE */ > -#define TIME_NO_ZERO_IN_DATE (65536L*2*2*2*2*2*2*2) > -/* Must be same as MODE_NO_ZERO_DATE */ > -#define TIME_NO_ZERO_DATE (TIME_NO_ZERO_IN_DATE*2) > -#define TIME_INVALID_DATES (TIME_NO_ZERO_DATE*2) > +#define TIME_NO_ZERO_IN_DATE (1UL << 23) /* == MODE_NO_ZERO_IN_DATE */ > +#define TIME_NO_ZERO_DATE (1UL << 24) /* == MODE_NO_ZERO_DATE */ > +#define TIME_INVALID_DATES (1UL << 25) /* == MODE_INVALID_DATES */ > > #define MYSQL_TIME_WARN_TRUNCATED 1 > #define MYSQL_TIME_WARN_OUT_OF_RANGE 2 > > === modified file 'libmysql/libmysql.c' > --- libmysql/libmysql.c 2013-01-10 14:40:21 +0000 > +++ libmysql/libmysql.c 2013-07-03 07:46:20 +0000 > @@ -3549,7 +3549,7 @@ > case MYSQL_TYPE_TIME: > { > MYSQL_TIME *tm= (MYSQL_TIME *)buffer; > - str_to_time(value, length, tm, TIME_FUZZY_DATE, &err); > + str_to_time(value, length, tm, 0, &err); > *param->error= test(err); > break; > } > @@ -3558,7 +3558,7 @@ > case MYSQL_TYPE_TIMESTAMP: > { > MYSQL_TIME *tm= (MYSQL_TIME *)buffer; > - (void) str_to_datetime(value, length, tm, TIME_FUZZY_DATE, &err); > + (void) str_to_datetime(value, length, tm, 0, &err); > *param->error= test(err) && (param->buffer_type == MYSQL_TYPE_DATE && > tm->time_type != MYSQL_TIMESTAMP_DATE); > break; > @@ -3681,9 +3681,7 @@ > case MYSQL_TYPE_DATETIME: > { > int error; > - value= number_to_datetime(value, 0, > - (MYSQL_TIME *) buffer, TIME_FUZZY_DATE, > - &error); > + value= number_to_datetime(value, 0, (MYSQL_TIME *) buffer, 0, &error); > *param->error= test(error); > break; > } > > === modified file 'mysql-test/r/adddate_454.result' > --- mysql-test/r/adddate_454.result 2012-08-29 15:55:59 +0000 > +++ mysql-test/r/adddate_454.result 2013-07-03 07:46:20 +0000 > @@ -4,6 +4,8 @@ > d > 2012-00-00 > update t1 set d = adddate(d, interval 1 day); > +Warnings: > +Warning 1292 Incorrect datetime value: '2012-00-00' > select * from t1; > d > NULL > > === modified file 'mysql-test/r/cast.result' > --- mysql-test/r/cast.result 2013-03-17 06:41:22 +0000 > +++ mysql-test/r/cast.result 2013-07-03 07:46:20 +0000 > @@ -268,37 +268,37 @@ > 0001-02-03 10:11:12 > select cast(120010203101112.121314 as datetime); > cast(120010203101112.121314 as datetime) > -0000-00-00 00:00:00 > +NULL > Warnings: > Warning 1292 Incorrect datetime value: '120010203101112.121314' > select cast(cast(1.1 as decimal) as datetime); > cast(cast(1.1 as decimal) as datetime) > -0000-00-00 00:00:00 > +NULL > Warnings: > Warning 1292 Incorrect datetime value: '1' > select cast(cast(-1.1 as decimal) as datetime); > cast(cast(-1.1 as decimal) as datetime) > -0000-00-00 00:00:00 > +NULL > Warnings: > Warning 1292 Incorrect datetime value: '-1' > select cast('0' as date); > cast('0' as date) > -0000-00-00 > +NULL > Warnings: > Warning 1292 Incorrect datetime value: '0' > select cast('' as date); > cast('' as date) > -0000-00-00 > +NULL > Warnings: > Warning 1292 Incorrect datetime value: '' > select cast('0' as datetime); > cast('0' as datetime) > -0000-00-00 00:00:00 > +NULL > Warnings: > Warning 1292 Incorrect datetime value: '0' > select cast('' as datetime); > cast('' as datetime) > -0000-00-00 00:00:00 > +NULL > Warnings: > Warning 1292 Incorrect datetime value: '' > select cast('0' as time); > @@ -306,7 +306,7 @@ > 00:00:00 > select cast('' as time); > cast('' as time) > -00:00:00 > +NULL > Warnings: > Warning 1292 Truncated incorrect time value: '' > select cast(NULL as DATE); > @@ -323,13 +323,13 @@ > NULL > select cast(cast(120010203101112.121314 as double) as datetime); > cast(cast(120010203101112.121314 as double) as datetime) > -0000-00-00 00:00:00 > +NULL > select cast(cast(1.1 as double) as datetime); > cast(cast(1.1 as double) as datetime) > 0000-00-00 00:00:01 > select cast(cast(-1.1 as double) as datetime); > cast(cast(-1.1 as double) as datetime) > -0000-00-00 00:00:00 > +NULL > explain extended select cast(10 as double(5,2)); > id select_type table type possible_keys key key_len ref rows filtered Extra > 1 SIMPLE NULL NULL NULL NULL NULL NULL NULL NULL No tables used > @@ -764,4 +764,10 @@ > CAST(TIME('10:20:30') AS DATE) + INTERVAL 1 DAY > NULL > Warnings: > -Warning 1292 Truncated incorrect date value: '0000-00-00' > +Warning 1292 Incorrect datetime value: '0000-00-00' > +SET SQL_MODE=ALLOW_INVALID_DATES; > +SELECT DATE("foo"); > +DATE("foo") > +NULL > +Warnings: > +Warning 1292 Incorrect datetime value: 'foo' > > === modified file 'mysql-test/r/date_formats.result' > --- mysql-test/r/date_formats.result 2013-03-17 06:41:22 +0000 > +++ mysql-test/r/date_formats.result 2013-07-03 07:46:20 +0000 > @@ -586,7 +586,7 @@ > 1 AM > SELECT DATE_FORMAT('%Y-%m-%d %H:%i:%s', 1151414896); > DATE_FORMAT('%Y-%m-%d %H:%i:%s', 1151414896) > -1151414896 > +NULL > Warnings: > Warning 1292 Incorrect datetime value: '%Y-%m-%d %H:%i:%s' > select str_to_date('04 /30/2004', '%m /%d/%Y'); > > === modified file 'mysql-test/r/func_sapdb.result' > --- mysql-test/r/func_sapdb.result 2013-03-17 06:41:22 +0000 > +++ mysql-test/r/func_sapdb.result 2013-07-03 07:46:20 +0000 > @@ -168,7 +168,7 @@ > 1997-12-31 > select date("1997-13-31 23:59:59.000001"); > date("1997-13-31 23:59:59.000001") > -0000-00-00 > +NULL > Warnings: > Warning 1292 Incorrect datetime value: '1997-13-31 23:59:59.000001' > select time("1997-12-31 23:59:59.000001"); > @@ -176,7 +176,7 @@ > 23:59:59.000001 > select time("1997-12-31 25:59:59.000001"); > time("1997-12-31 25:59:59.000001") > -00:00:00 > +NULL > Warnings: > Warning 1292 Truncated incorrect time value: '1997-12-31 25:59:59.000001' > select microsecond("1997-12-31 23:59:59.000001"); > @@ -250,8 +250,6 @@ > select microsecond(19971231235959.01) as a; > a > 10000 > -Warnings: > -Warning 1292 Truncated incorrect time value: '19971231235959.01' > select date_add("1997-12-31",INTERVAL "10.09" SECOND_MICROSECOND) as a; > a > 1997-12-31 00:00:10.090000 > > === modified file 'mysql-test/r/func_time.result' > --- mysql-test/r/func_time.result 2013-06-17 15:25:55 +0000 > +++ mysql-test/r/func_time.result 2013-07-03 07:46:20 +0000 > @@ -1239,14 +1239,13 @@ > set time_zone= @@global.time_zone; > select str_to_date('10:00 PM', '%h:%i %p') + INTERVAL 10 MINUTE; > str_to_date('10:00 PM', '%h:%i %p') + INTERVAL 10 MINUTE > -NULL > -Warnings: > -Error 1411 Incorrect datetime value: '10:00 PM' for function str_to_date > +22:10:00 > select str_to_date("1997-00-04 22:23:00","%Y-%m-%D") + interval 10 minute; > str_to_date("1997-00-04 22:23:00","%Y-%m-%D") + interval 10 minute > NULL > Warnings: > -Error 1411 Incorrect datetime value: '1997-00-04 22:23:00' for function str_to_date > +Warning 1292 Truncated incorrect date value: '1997-00-04 22:23:00' > +Warning 1292 Incorrect datetime value: '1997-00-04' > create table t1 (field DATE); > insert into t1 values ('2006-11-06'); > select * from t1 where field < '2006-11-06 04:08:36.0'; > @@ -1452,13 +1451,15 @@ > NULL > SELECT WEEK(DATE_ADD(FROM_DAYS(1),INTERVAL 1 MONTH), 1); > WEEK(DATE_ADD(FROM_DAYS(1),INTERVAL 1 MONTH), 1) > -0 > +NULL > +Warnings: > +Warning 1292 Incorrect datetime value: '0000-00-00' > # > # Bug#12584302 AFTER FIX FOR #12403504: ASSERTION FAILED: DELSUM+(INT) Y/4-TEMP > 0, > # > DO WEEK((DATE_ADD((CAST(0 AS DATE)), INTERVAL 1 YEAR_MONTH)), 5); > Warnings: > -Warning 1292 Incorrect datetime value: '0' > +Warning 1292 Incorrect datetime value: '0000-00-00' > # > # BUG#13458237 INCONSISTENT HANDLING OF INVALIDE DATES WITH ZERO DAY > # SIMILAR TO '2009-10-00' > @@ -1755,7 +1756,7 @@ > Warning 1441 Datetime function: time field overflow > select cast('131415.123e0' as time); > cast('131415.123e0' as time) > -00:00:00 > +NULL > Warnings: > Warning 1292 Truncated incorrect time value: '131415.123e0' > select cast('2010-01-02 03:04:05' as datetime) between null and '2010-01-02 03:04:04'; > @@ -1775,12 +1776,12 @@ > NULL > select truncate(date('2010-40-10'), 6); > truncate(date('2010-40-10'), 6) > -0.000000 > +NULL > Warnings: > Warning 1292 Incorrect datetime value: '2010-40-10' > select extract(month from '2010-40-50'); > extract(month from '2010-40-50') > -0 > +NULL > Warnings: > Warning 1292 Incorrect datetime value: '2010-40-50' > select subtime('0000-00-10 10:10:10', '30 10:00:00'); > @@ -1858,6 +1859,8 @@ > select timestampadd(week, 1, f1) from t1; > timestampadd(week, 1, f1) > NULL > +Warnings: > +Warning 1292 Incorrect datetime value: '0000-00-00' > select timestampadd(week, 1, date("0000-00-00")); > timestampadd(week, 1, date("0000-00-00")) > NULL > > === modified file 'mysql-test/r/parser.result' > --- mysql-test/r/parser.result 2011-03-01 12:24:36 +0000 > +++ mysql-test/r/parser.result 2013-07-03 07:46:20 +0000 > @@ -555,14 +555,10 @@ > DROP TABLE IF EXISTS t1; > SELECT STR_TO_DATE('10:00 PM', '%h:%i %p') + INTERVAL 10 MINUTE; > STR_TO_DATE('10:00 PM', '%h:%i %p') + INTERVAL 10 MINUTE > -NULL > -Warnings: > -Error 1411 Incorrect datetime value: '10:00 PM' for function str_to_date > +22:10:00 > SELECT STR_TO_DATE('10:00 PM', '%h:%i %p') + INTERVAL (INTERVAL(1,2,3) + 1) MINUTE; > STR_TO_DATE('10:00 PM', '%h:%i %p') + INTERVAL (INTERVAL(1,2,3) + 1) MINUTE > -NULL > -Warnings: > -Error 1411 Incorrect datetime value: '10:00 PM' for function str_to_date > +22:01:00 > SELECT "1997-12-31 23:59:59" + INTERVAL 1 SECOND; > "1997-12-31 23:59:59" + INTERVAL 1 SECOND > 1998-01-01 00:00:00 > > === modified file 'mysql-test/r/partition_pruning.result' > --- mysql-test/r/partition_pruning.result 2013-03-17 06:41:22 +0000 > +++ mysql-test/r/partition_pruning.result 2013-07-03 07:46:20 +0000 > @@ -1906,10 +1906,9 @@ > # test with an invalid date, which lead to item->null_value is set. > EXPLAIN PARTITIONS SELECT * FROM t1 WHERE b < CAST('2009-04-99' AS DATETIME); > id select_type table partitions type possible_keys key key_len ref rows Extra > -1 SIMPLE t1 p20090401 ALL NULL NULL NULL NULL 2 Using where > +1 SIMPLE NULL NULL NULL NULL NULL NULL NULL NULL Impossible WHERE noticed after reading const tables > Warnings: > Warning 1292 Incorrect datetime value: '2009-04-99' > -Warning 1292 Incorrect datetime value: '2009-04-99' > DROP TABLE t1; > CREATE TABLE t1 > (a INT NOT NULL AUTO_INCREMENT, > > === modified file 'mysql-test/r/type_date.result' > --- mysql-test/r/type_date.result 2013-06-28 08:00:25 +0000 > +++ mysql-test/r/type_date.result 2013-07-03 07:46:20 +0000 > @@ -136,7 +136,7 @@ > 1311 > select year(@d), month(@d), day(@d), cast(@d as date); > year(@d) month(@d) day(@d) cast(@d as date) > -0 0 0 0000-00-00 > +NULL NULL NULL NULL > Warnings: > Warning 1292 Incorrect datetime value: '1311' > Warning 1292 Incorrect datetime value: '1311' > > === modified file 'mysql-test/r/type_datetime.result' > --- mysql-test/r/type_datetime.result 2013-06-28 08:00:25 +0000 > +++ mysql-test/r/type_datetime.result 2013-07-03 07:46:20 +0000 > @@ -657,8 +657,8 @@ > insert into t1 values ('2000-12-03','22:55:23'),('2008-05-03','10:19:31'); > select case when d = '2012-12-12' then d else t end as cond, group_concat( d ) from t1 group by cond; > cond group_concat( d ) > -0000-00-00 00:00:00 2000-12-03 > -0000-00-00 00:00:00 2008-05-03 > +NULL 2000-12-03 > +NULL 2008-05-03 > Warnings: > Warning 1292 Incorrect datetime value: '22:55:23' > Warning 1292 Incorrect datetime value: '10:19:31' > > === modified file 'mysql-test/suite/vcol/r/vcol_misc.result' > --- mysql-test/suite/vcol/r/vcol_misc.result 2013-01-22 05:29:19 +0000 > +++ mysql-test/suite/vcol/r/vcol_misc.result 2013-07-03 07:46:20 +0000 > @@ -187,7 +187,11 @@ > tsv TIMESTAMP AS (ADDDATE(ts, INTERVAL 1 DAY)) VIRTUAL > ) ENGINE=MyISAM; > INSERT INTO t1 (tsv) VALUES (DEFAULT); > +Warnings: > +Warning 1292 Incorrect datetime value: '0000-00-00' > INSERT DELAYED INTO t1 (tsv) VALUES (DEFAULT); > +Warnings: > +Warning 1292 Incorrect datetime value: '0000-00-00' > FLUSH TABLES; > SELECT COUNT(*) FROM t1; > COUNT(*) > > === modified file 'mysql-test/t/cast.test' > --- mysql-test/t/cast.test 2012-05-20 12:57:29 +0000 > +++ mysql-test/t/cast.test 2013-07-03 07:46:20 +0000 > @@ -439,3 +439,6 @@ > # > SELECT CAST(TIME('10:20:30') AS DATE) + INTERVAL 1 DAY; > > +SET SQL_MODE=ALLOW_INVALID_DATES; > +SELECT DATE("foo"); > + > > === modified file 'sql-common/my_time.c' > --- sql-common/my_time.c 2012-06-08 17:15:01 +0000 > +++ sql-common/my_time.c 2013-07-03 07:46:20 +0000 > @@ -83,7 +83,7 @@ > { > if (not_zero_date) > { > - if ((((flags & TIME_NO_ZERO_IN_DATE) || !(flags & TIME_FUZZY_DATE)) && > + if (((flags & TIME_NO_ZERO_IN_DATE) && > (ltime->month == 0 || ltime->day == 0)) || ltime->neg || > (!(flags & TIME_INVALID_DATES) && > ltime->month && ltime->day > days_in_month[ltime->month-1] && > @@ -115,7 +115,7 @@ > length Length of string > l_time Date is stored here > flags Bitmap of following items > - TIME_FUZZY_DATE Set if we should allow partial dates > + TIME_FUZZY_DATE > TIME_DATETIME_ONLY Set if we only allow full datetimes. > TIME_NO_ZERO_IN_DATE Don't allow partial dates > TIME_NO_ZERO_DATE Don't allow 0000-00-00 date > @@ -1324,7 +1324,7 @@ > if (nr > 9999999 && neg == 0) > { > if (number_to_datetime(nr, sec_part, ltime, > - TIME_FUZZY_DATE | TIME_INVALID_DATES, was_cut) < 0) > + TIME_INVALID_DATES, was_cut) < 0) > return -1; > > ltime->year= ltime->month= ltime->day= 0; > > === modified file 'sql/field.cc' > --- sql/field.cc 2013-02-28 20:48:47 +0000 > +++ sql/field.cc 2013-07-03 07:46:20 +0000 > @@ -5127,10 +5127,9 @@ > Lazy_string_str str(from, len); > > func_res= str_to_datetime(from, len, &ltime, > - (TIME_FUZZY_DATE | > - (thd->variables.sql_mode & > + (thd->variables.sql_mode & > (MODE_NO_ZERO_IN_DATE | MODE_NO_ZERO_DATE | > - MODE_INVALID_DATES))), > + MODE_INVALID_DATES)), Why not to add a new method in THD to return these sql_mode datetime flags? These flags repeat many times. Something similar to sql_mode_for_dates(), but always returning the "strict_date_checking" version that ORs all three flags: return thd->variables.sql_mode & (MODE_NO_ZERO_IN_DATE | MODE_NO_ZERO_DATE | MODE_INVALID_DATES); By the way, sql_mode_for_dates() is used in str_to_datetime_with_warn() and Item::send(). Do you know why? Perhaps they also should follow all these three sql_mode date flags... > &error); > return store_TIME_with_warning(&ltime, &str, error, func_res > MYSQL_TIMESTAMP_ERROR); > } > @@ -5144,11 +5143,10 @@ > Lazy_string_double str(nr); > > longlong tmp= double_to_datetime(nr, &ltime, > - (TIME_FUZZY_DATE | > - (thd->variables.sql_mode & > + (thd->variables.sql_mode & > (MODE_NO_ZERO_IN_DATE | > MODE_NO_ZERO_DATE | > - MODE_INVALID_DATES))), &error); > + MODE_INVALID_DATES)), &error); > return store_TIME_with_warning(&ltime, &str, error, tmp != -1); > } > > @@ -5161,11 +5159,10 @@ > THD *thd= table->in_use; > Lazy_string_num str(nr); > > - tmp= number_to_datetime(nr, 0, &ltime, (TIME_FUZZY_DATE | > - (thd->variables.sql_mode & > + tmp= number_to_datetime(nr, 0, &ltime, (thd->variables.sql_mode & > (MODE_NO_ZERO_IN_DATE | > MODE_NO_ZERO_DATE | > - MODE_INVALID_DATES))), &error); > + MODE_INVALID_DATES)), &error); > > return store_TIME_with_warning(&ltime, &str, error, tmp != -1); > } > @@ -5181,17 +5178,16 @@ > structure always fit into DATETIME range. > */ > have_smth_to_conv= !check_date(&l_time, pack_time(&l_time) != 0, > - (TIME_FUZZY_DATE | > - (current_thd->variables.sql_mode & > + (current_thd->variables.sql_mode & > (MODE_NO_ZERO_IN_DATE | MODE_NO_ZERO_DATE | > - MODE_INVALID_DATES))), &error); > + MODE_INVALID_DATES)), &error); > return store_TIME_with_warning(&l_time, &str, error, have_smth_to_conv); > } > > my_decimal *Field_temporal::val_decimal(my_decimal *d) > { > MYSQL_TIME ltime; > - if (get_date(&ltime, TIME_FUZZY_DATE)) > + if (get_date(&ltime, 0)) Now get_date() is called in a lot of places with 0 flags. Why not to add a method: bool get_date(MYSQL_TIME *ltime) { return get_date(ltime, 0); } And a similar wrapper for get_arg0_date(), with flags==0... ? > { > bzero(&ltime, sizeof(ltime)); > ltime.time_type= mysql_type_to_time_type(type()); > @@ -5330,7 +5326,8 @@ > bool Field_time::get_date(MYSQL_TIME *ltime, uint fuzzydate) > { > THD *thd= table->in_use; > - if (!(fuzzydate & (TIME_FUZZY_DATE|TIME_TIME_ONLY))) > + if (!(fuzzydate & TIME_TIME_ONLY) && > + (fuzzydate & TIME_NO_ZERO_IN_DATE)) > { > push_warning_printf(thd, MYSQL_ERROR::WARN_LEVEL_WARN, > ER_WARN_DATA_OUT_OF_RANGE, > @@ -5459,7 +5456,7 @@ > ltime->time_type= MYSQL_TIMESTAMP_TIME; > ltime->hour+= (ltime->month*32+ltime->day)*24; > ltime->month= ltime->day= 0; > - return fuzzydate & (TIME_FUZZY_DATE | TIME_TIME_ONLY) ? 0 : 1; > + return !(fuzzydate & TIME_TIME_ONLY) && (fuzzydate & TIME_NO_ZERO_IN_DATE); > } > > > @@ -5848,7 +5845,7 @@ > bool Field_datetime::send_binary(Protocol *protocol) > { > MYSQL_TIME tm; > - Field_datetime::get_date(&tm, TIME_FUZZY_DATE); > + Field_datetime::get_date(&tm, 0); > return protocol->store(&tm, 0); > } > > @@ -5931,7 +5928,7 @@ > if (!tmp) > return (fuzzydate & TIME_NO_ZERO_DATE) != 0; > if (!ltime->month || !ltime->day) > - return !(fuzzydate & TIME_FUZZY_DATE); > + return fuzzydate & TIME_NO_ZERO_IN_DATE; > return 0; > } > > @@ -5984,11 +5981,10 @@ > error= 2; > } > else > - tmp= number_to_datetime(nr, sec_part, &ltime, (TIME_FUZZY_DATE | > - (thd->variables.sql_mode & > + tmp= number_to_datetime(nr, sec_part, &ltime, (thd->variables.sql_mode & > (MODE_NO_ZERO_IN_DATE | > MODE_NO_ZERO_DATE | > - MODE_INVALID_DATES))), &error); > + MODE_INVALID_DATES)), &error); > > return store_TIME_with_warning(&ltime, &str, error, tmp != -1); > } > @@ -5996,7 +5992,7 @@ > bool Field_datetime_hires::send_binary(Protocol *protocol) > { > MYSQL_TIME ltime; > - Field_datetime_hires::get_date(&ltime, TIME_FUZZY_DATE); > + Field_datetime_hires::get_date(&ltime, 0); > return protocol->store(&ltime, dec); > } > > @@ -6004,14 +6000,14 @@ > double Field_datetime_hires::val_real(void) > { > MYSQL_TIME ltime; > - Field_datetime_hires::get_date(&ltime, TIME_FUZZY_DATE); > + Field_datetime_hires::get_date(&ltime, 0); > return TIME_to_double(&ltime); > } > > longlong Field_datetime_hires::val_int(void) > { > MYSQL_TIME ltime; > - Field_datetime_hires::get_date(&ltime, TIME_FUZZY_DATE); > + Field_datetime_hires::get_date(&ltime, 0); > return TIME_to_ulonglong_datetime(&ltime); > } > > @@ -6020,7 +6016,7 @@ > String *unused __attribute__((unused))) > { > MYSQL_TIME ltime; > - Field_datetime_hires::get_date(&ltime, TIME_FUZZY_DATE); > + Field_datetime_hires::get_date(&ltime, 0); > str->alloc(field_length+1); > str->length(field_length); > my_datetime_to_str(&ltime, (char*) str->ptr(), dec); > @@ -6035,7 +6031,7 @@ > if (!packed) > return fuzzydate & TIME_NO_ZERO_DATE; > if (!ltime->month || !ltime->day) > - return !(fuzzydate & TIME_FUZZY_DATE); > + return fuzzydate & TIME_NO_ZERO_IN_DATE; > return 0; > } > > > === modified file 'sql/field_conv.cc' > --- sql/field_conv.cc 2012-04-05 21:07:18 +0000 > +++ sql/field_conv.cc 2013-07-03 07:46:20 +0000 > @@ -413,7 +413,7 @@ > static void do_field_temporal(Copy_field *copy) > { > MYSQL_TIME ltime; > - copy->from_field->get_date(&ltime, TIME_FUZZY_DATE); > + copy->from_field->get_date(&ltime, 0); > copy->to_field->store_time_dec(&ltime, copy->from_field->decimals()); > } > > @@ -884,7 +884,7 @@ > if (from->cmp_type() == TIME_RESULT) > { > MYSQL_TIME ltime; > - if (from->get_date(&ltime, TIME_FUZZY_DATE)) > + if (from->get_date(&ltime, 0)) > return to->reset(); > else > return to->store_time_dec(&ltime, from->decimals()); > > === modified file 'sql/filesort.cc' > --- sql/filesort.cc 2013-03-17 06:41:22 +0000 > +++ sql/filesort.cc 2013-07-03 07:46:20 +0000 > @@ -868,7 +868,7 @@ > else > { > MYSQL_TIME buf; > - if (item->get_date_result(&buf, TIME_FUZZY_DATE | TIME_INVALID_DATES)) > + if (item->get_date_result(&buf, TIME_INVALID_DATES)) > { > DBUG_ASSERT(maybe_null); > DBUG_ASSERT(item->null_value); > > === modified file 'sql/item.cc' > --- sql/item.cc 2013-03-17 06:41:22 +0000 > +++ sql/item.cc 2013-07-03 07:46:20 +0000 > @@ -252,7 +252,7 @@ > String *Item::val_string_from_date(String *str) > { > MYSQL_TIME ltime; > - if (get_date(&ltime, TIME_FUZZY_DATE) || > + if (get_date(&ltime, 0) || > str->alloc(MAX_DATE_STRING_REP_LENGTH)) > { > null_value= 1; > @@ -308,7 +308,7 @@ > { > DBUG_ASSERT(fixed == 1); > MYSQL_TIME ltime; > - if (get_date(&ltime, TIME_FUZZY_DATE)) > + if (get_date(&ltime, 0)) > { > my_decimal_set_zero(decimal_value); > null_value= 1; // set NULL, stop processing > @@ -367,7 +367,7 @@ > int Item::save_date_in_field(Field *field) > { > MYSQL_TIME ltime; > - if (get_date(&ltime, TIME_FUZZY_DATE)) > + if (get_date(&ltime, 0)) > return set_field_to_null_with_conversions(field, 0); > field->set_notnull(); > return field->store_time_dec(&ltime, decimals); > @@ -1205,7 +1205,7 @@ > if allowed, otherwise - null. > */ > bzero((char*) ltime,sizeof(*ltime)); > - return null_value|= (fuzzydate & (TIME_NO_ZERO_DATE|TIME_NO_ZERO_IN_DATE)); > + return null_value|= !(fuzzydate & TIME_FUZZY_DATES); > } > > bool Item::get_seconds(ulonglong *sec, ulong *sec_part) > @@ -6032,7 +6032,7 @@ > case MYSQL_TYPE_TIMESTAMP: > { > MYSQL_TIME tm; > - get_date(&tm, TIME_FUZZY_DATE | sql_mode_for_dates()); > + get_date(&tm, sql_mode_for_dates()); > if (!null_value) > { > if (f_type == MYSQL_TYPE_DATE) > @@ -8161,8 +8161,8 @@ > } > else > { > - field->get_date(&field_time, TIME_FUZZY_DATE | TIME_INVALID_DATES); > - item->get_date(&item_time, TIME_FUZZY_DATE | TIME_INVALID_DATES); > + field->get_date(&field_time, TIME_INVALID_DATES); > + item->get_date(&item_time, TIME_INVALID_DATES); > } > return my_time_compare(&field_time, &item_time); > } > @@ -8343,7 +8343,7 @@ > value_cached= true; > > MYSQL_TIME ltime; > - if (example->get_date_result(&ltime, TIME_FUZZY_DATE)) > + if (example->get_date_result(&ltime, 0)) > value=0; > else > value= pack_time(&ltime); > > === modified file 'sql/item.h' > --- sql/item.h 2013-05-03 13:07:13 +0000 > +++ sql/item.h 2013-07-03 07:46:20 +0000 > @@ -936,7 +936,7 @@ > Item **ref, bool skip_registered); > virtual bool get_date(MYSQL_TIME *ltime,uint fuzzydate); > bool get_time(MYSQL_TIME *ltime) > - { return get_date(ltime, TIME_TIME_ONLY | TIME_FUZZY_DATE); } > + { return get_date(ltime, TIME_TIME_ONLY); } > bool get_seconds(ulonglong *sec, ulong *sec_part); > virtual bool get_date_result(MYSQL_TIME *ltime,uint fuzzydate) > { return get_date(ltime,fuzzydate); } > > === modified file 'sql/item_cmpfunc.cc' > --- sql/item_cmpfunc.cc 2013-02-25 03:16:11 +0000 > +++ sql/item_cmpfunc.cc 2013-07-03 07:46:20 +0000 > @@ -713,7 +713,7 @@ > bool value; > int error; > enum_mysql_timestamp_type timestamp_type; > - int flags= TIME_FUZZY_DATE | MODE_INVALID_DATES; > + int flags= TIME_FUZZY_DATES | MODE_INVALID_DATES; > > if (warn_type == MYSQL_TIMESTAMP_TIME) > flags|= TIME_TIME_ONLY; > @@ -885,7 +885,7 @@ > else > { > MYSQL_TIME ltime; > - uint fuzzydate= TIME_FUZZY_DATE | TIME_INVALID_DATES; > + uint fuzzydate= TIME_FUZZY_DATES | TIME_INVALID_DATES; > if (f_type == MYSQL_TYPE_TIME) > fuzzydate|= TIME_TIME_ONLY; > if (item->get_date(&ltime, fuzzydate)) > > === modified file 'sql/item_func.cc' > --- sql/item_func.cc 2013-06-28 08:00:25 +0000 > +++ sql/item_func.cc 2013-07-03 07:46:20 +0000 > @@ -2538,7 +2538,7 @@ > if (compare_as_dates) > { > MYSQL_TIME ltime; > - if (get_date(&ltime, TIME_FUZZY_DATE)) > + if (get_date(&ltime, 0)) > return 0; > > return TIME_to_double(&ltime); > @@ -2567,7 +2567,7 @@ > if (compare_as_dates) > { > MYSQL_TIME ltime; > - if (get_date(&ltime, TIME_FUZZY_DATE)) > + if (get_date(&ltime, 0)) > return 0; > > return TIME_to_ulonglong(&ltime); > @@ -2597,7 +2597,7 @@ > if (compare_as_dates) > { > MYSQL_TIME ltime; > - if (get_date(&ltime, TIME_FUZZY_DATE)) > + if (get_date(&ltime, 0)) > return 0; > > return date2my_decimal(&ltime, dec); > > === modified file 'sql/item_strfunc.cc' > --- sql/item_strfunc.cc 2013-03-26 18:09:47 +0000 > +++ sql/item_strfunc.cc 2013-07-03 07:46:20 +0000 > @@ -3618,10 +3618,10 @@ > } > break; > case DYN_COL_DATETIME: > - args[valpos]->get_date(&vals[i].x.time_value, TIME_FUZZY_DATE); > + args[valpos]->get_date(&vals[i].x.time_value, 0); > break; > case DYN_COL_DATE: > - args[valpos]->get_date(&vals[i].x.time_value, TIME_FUZZY_DATE); > + args[valpos]->get_date(&vals[i].x.time_value, 0); > break; > case DYN_COL_TIME: > args[valpos]->get_time(&vals[i].x.time_value); > > === modified file 'sql/item_timefunc.cc' > --- sql/item_timefunc.cc 2013-06-28 08:00:25 +0000 > +++ sql/item_timefunc.cc 2013-07-03 07:46:20 +0000 > @@ -791,7 +791,7 @@ > MYSQL_TIME ltime; > longlong res; > int dummy; /* unused */ > - if (get_arg0_date(&ltime, TIME_FUZZY_DATE)) > + if (get_arg0_date(&ltime, 0)) > { > /* got NULL, leave the incl_endp intact */ > return LONGLONG_MIN; > @@ -855,14 +855,14 @@ > { > DBUG_ASSERT(fixed == 1); > MYSQL_TIME ltime; > - return get_arg0_date(&ltime, TIME_FUZZY_DATE) ? 0 : (longlong) ltime.day; > + return get_arg0_date(&ltime, 0) ? 0 : (longlong) ltime.day; > } > > longlong Item_func_month::val_int() > { > DBUG_ASSERT(fixed == 1); > MYSQL_TIME ltime; > - return get_arg0_date(&ltime, TIME_FUZZY_DATE) ? 0 : (longlong) ltime.month; > + return get_arg0_date(&ltime, 0) ? 0 : (longlong) ltime.month; > } > > > @@ -907,7 +907,7 @@ > { > DBUG_ASSERT(fixed == 1); > MYSQL_TIME ltime; > - if (get_arg0_date(&ltime, TIME_FUZZY_DATE)) > + if (get_arg0_date(&ltime, 0)) > return 0; > return (longlong) ((ltime.month+2)/3); > } > @@ -981,7 +981,7 @@ > DBUG_ASSERT(fixed == 1); > uint year; > MYSQL_TIME ltime; > - if (get_arg0_date(&ltime, TIME_NO_ZERO_DATE)) > + if (get_arg0_date(&ltime, TIME_NO_ZERO_DATE | TIME_NO_ZERO_IN_DATE)) > return 0; > return (longlong) calc_week(&ltime, > week_mode((uint) args[1]->val_int()), > @@ -994,7 +994,7 @@ > DBUG_ASSERT(fixed == 1); > uint year,week; > MYSQL_TIME ltime; > - if (get_arg0_date(&ltime, TIME_NO_ZERO_DATE)) > + if (get_arg0_date(&ltime, TIME_NO_ZERO_DATE | TIME_NO_ZERO_IN_DATE)) > return 0; > week= calc_week(&ltime, > (week_mode((uint) args[1]->val_int()) | WEEK_YEAR), > @@ -1008,7 +1008,7 @@ > DBUG_ASSERT(fixed == 1); > MYSQL_TIME ltime; > > - if (get_arg0_date(&ltime, TIME_NO_ZERO_DATE)) > + if (get_arg0_date(&ltime, TIME_NO_ZERO_DATE | TIME_NO_ZERO_IN_DATE)) > return 0; > > return (longlong) calc_weekday(calc_daynr(ltime.year, ltime.month, > @@ -1050,7 +1050,7 @@ > { > DBUG_ASSERT(fixed == 1); > MYSQL_TIME ltime; > - return get_arg0_date(&ltime, TIME_FUZZY_DATE) ? 0 : (longlong) ltime.year; > + return get_arg0_date(&ltime, 0) ? 0 : (longlong) ltime.year; > } > > > @@ -1082,7 +1082,7 @@ > { > DBUG_ASSERT(fixed == 1); > MYSQL_TIME ltime; > - if (get_arg0_date(&ltime, TIME_FUZZY_DATE)) > + if (get_arg0_date(&ltime, 0)) > { > /* got NULL, leave the incl_endp intact */ > return LONGLONG_MIN; > @@ -1354,7 +1354,7 @@ > { > DBUG_ASSERT(fixed == 1); > MYSQL_TIME ltime; > - if (get_date(&ltime, TIME_FUZZY_DATE)) > + if (get_date(&ltime, 0)) > return 0; > longlong v= TIME_to_ulonglong(&ltime); > return ltime.neg ? -v : v; > @@ -1365,7 +1365,7 @@ > { > DBUG_ASSERT(fixed == 1); > MYSQL_TIME ltime; > - if (get_date(&ltime, TIME_FUZZY_DATE)) > + if (get_date(&ltime, 0)) > return 0; > return TIME_to_double(&ltime); > } > @@ -1744,7 +1744,7 @@ > int is_time_flag = is_time_format ? TIME_TIME_ONLY : 0; > DBUG_ASSERT(fixed == 1); > > - if (get_arg0_date(&l_time, TIME_FUZZY_DATE | is_time_flag)) > + if (get_arg0_date(&l_time, is_time_flag)) > return 0; Perhaps now it's easier to put the expression directly to get_arg0_date(), instead of having a variable. > > if (!(format = args[1]->val_str(str)) || !format->length()) > @@ -1923,10 +1923,15 @@ > { > INTERVAL interval; > > - if (args[0]->get_date(ltime, TIME_NO_ZERO_DATE | TIME_FUZZY_DATE | TIME_NO_ZERO_IN_DATE) || > + if (args[0]->get_date(ltime, 0) || > get_interval_value(args[1], int_type, &value, &interval)) > return (null_value=1); > > + if (ltime->time_type != MYSQL_TIMESTAMP_TIME && > + check_date_with_warn(ltime, TIME_NO_ZERO_DATE | TIME_NO_ZERO_IN_DATE, > + MYSQL_TIMESTAMP_ERROR)) > + return (null_value=1); > + > if (date_sub_interval) > interval.neg = !interval.neg; > > @@ -2019,7 +2024,7 @@ > long neg; > int is_time_flag = date_value ? 0 : TIME_TIME_ONLY; One more time (see the previous comment) > - if (get_arg0_date(&ltime, TIME_FUZZY_DATE | is_time_flag)) > + if (get_arg0_date(&ltime, is_time_flag)) > return 0; > neg= ltime.neg ? -1 : 1; > > @@ -2430,7 +2435,7 @@ > > if (is_date) // TIMESTAMP function > { > - if (get_arg0_date(&l_time1, TIME_FUZZY_DATE) || > + if (get_arg0_date(&l_time1, 0) || > args[1]->get_time(&l_time2) || > l_time1.time_type == MYSQL_TIMESTAMP_TIME || > l_time2.time_type != MYSQL_TIMESTAMP_TIME) > > === modified file 'sql/time.cc' > --- sql/time.cc 2013-06-28 12:25:06 +0000 > +++ sql/time.cc 2013-07-03 07:46:20 +0000 > @@ -263,7 +263,7 @@ > @param nr integer part of the number to convert > @param sec_part microsecond part of the number > @param ltime converted value will be written here > - @param fuzzydate conversion flags (TIME_FUZZY_DATE, etc) > + @param fuzzydate conversion flags (TIME_INVALID_DATE, etc) > @param str original number, as a Lazy_string. For the warning > @param field_name field name or NULL if not a field. For the warning > > @@ -280,6 +280,7 @@ > > if (fuzzydate & TIME_TIME_ONLY) > { > + fuzzydate= TIME_TIME_ONLY; // clear other flags > f_type= MYSQL_TYPE_TIME; > res= number_to_time(neg, nr, sec_part, ltime, &was_cut); > } > @@ -289,7 +290,7 @@ > res= neg ? -1 : number_to_datetime(nr, sec_part, ltime, fuzzydate, &was_cut); > } > > - if (res < 0 || (was_cut && !(fuzzydate & TIME_FUZZY_DATE))) > + if (res < 0 || (was_cut && (fuzzydate & TIME_NO_ZERO_IN_DATE))) > { > make_truncated_value_warning(current_thd, > MYSQL_ERROR::WARN_LEVEL_WARN, str,

2 1

[Maria-developers] MDEV-4472 (audit plugin)
by Alexey Botchkov 03 Jul '13

03 Jul '13

Hi, Sergei. Here is the plugin code for you to check out. It constists of a single new file, so i send you the link to it. http://myoffice.izhnet.ru/~alex/server_audit/server_audit.c What's missing in it as i'm not sure how it can be done: - No server host/ip in the log. How get that information? - SYSLOG logging. The SysLog i knew was declared in the syslog.h. That's the interface talking to the local syslogd instance. If it's what we need to use, then i have no idea how to specify the Syslog server IP for that daemon (which is required by the spec). I see nothing in the API about it and i thought it should be configured with the daemon's cfg file or something like that. Best regards. HF

2 6

[Maria-developers] On the question of whether select_id allow to reconstruct the query tree
by Sergei Petrunia 02 Jul '13

02 Jul '13

1 0

[Maria-developers] Dynamic Columns Test Error
by River wubx 02 Jul '13

02 Jul '13

MariaDB [test]> show create table assets ; +--------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Table | Create Table | +--------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | assets | CREATE TABLE `assets` ( `item_name` varchar(32) NOT NULL, `dynamic_cols` blob, PRIMARY KEY (`item_name`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8 PAGE_CHECKSUM=1 | +--------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 1 row in set (0.00 sec) MariaDB [test]> insert into assets values ('MariaDB T-shirt', COLUMN_CREATE('color', 'blue', 'size', 'XL')); ERROR 1921 (22007): Illegal value used as argument of dynamic column function MariaDB [test]> select version(); +--------------------+ | version() | +--------------------+ | 5.5.31-MariaDB-log | +--------------------+ 1 row in set (0.00 sec) Why? -- 姓名： river/吴炳锡手机：13501245755 http://www.mysqlsupport.cn 创业公司MySQL 支持，优化，架构设计

2 1

[Maria-developers] Doubt at query cache resize
by Roberto Spadim 28 Jun '13

28 Jun '13

Hi guys I was trying to compile in windows and warning was displayed at mariadb 10:0:3 in sysvars.cc at line 2235 Ulong new cache size = ulonglong function Could we lost the query cache size? Like a overflow

2 2

Re: [Maria-developers] Please review: EXPLAIN UPDATE patch + refactoring
by Sergei Petrunia 28 Jun '13

28 Jun '13

Now with attachment. On Fri, Jun 28, 2013 at 09:18:56PM +0400, Sergei Petrunia wrote: > Hello Igor, > > Please review the attached patch against current 10.0-base which contains > - EXPLAIN UPDATE/DELETE support > - "Query Plan Footprint" refactoring. > > The tree is at lp:~maria-captains/maria/10.0-base-explain-update-r2 > > BR Sergei -- Sergei Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] Rev 3768: Changed acl_roles to be stored into a HASH
by Sergei 28 Jun '13

28 Jun '13

Hi, Vicențiu! A couple of small notes: On Jun 27, noreply(a)launchpad.net wrote: > ------------------------------------------------------------ > revno: 3768 > committer: Vicențiu Ciorbaru <cvicentiu(a)gmail.com> > branch nick: trunk > timestamp: Fri 2013-06-28 01:20:45 +0300 > message: > Changed acl_roles to be stored into a HASH. > > === modified file 'sql/sql_acl.cc' > --- sql/sql_acl.cc 2013-06-25 21:49:50 +0000 > +++ sql/sql_acl.cc 2013-06-27 22:20:45 +0000 > @@ -51,6 +51,7 @@ > #include "hostname.h" > #include "sql_db.h" > #include "sql_array.h" > +#include "sql_hset.h" You should probably include hash.h not sql_hset.h because you don't use Hash_set<> template. > @@ -520,6 +521,13 @@ > return (uchar*) entry->key; > } > > +uchar* acl_role_get_key(ACL_USER *entry, size_t *length, > + my_bool not_used __attribute__((unused))) > +{ > + *length=(uint) (entry->user ? strlen(entry->user) : 0); You know, we generally try to avoid strlens, especially in a code that's used often, like here. Please, replace char *user; in the ACL_USER with LEX_STRING user; > + return (uchar*) entry->user; > +} > + > #define IP_ADDR_STRLEN (3 + 1 + 3 + 1 + 3 + 1 + 3) > #define ACL_KEY_LENGTH (IP_ADDR_STRLEN + 1 + NAME_LEN + \ > 1 + USERNAME_LENGTH + 1) Regards, Sergei

1 0

Re: [Maria-developers] Review request: mdev-4634
by Alexander Barkov 27 Jun '13

27 Jun '13

Hi Sergei, Please review a small patch that fixes the exact problem reported in 4634 (and nothing else). Thanks. (the other problems from the original big patch are now reported separately, with some of them already fixed). On 06/13/2013 12:08 PM, Sergei Golubchik wrote: > Hi, Alexander! > > On Jun 13, Alexander Barkov wrote: >>> Hi, Alexander! >>> >>> This looks like a lot of changes for what you explained the reason of >>> the bug was. There will be lots of "whys" below... >> >> The fix itself was quite small. However, an attempt >> to write a reasonable test case revealed a few other bugs. >> I just tried to fixing everything in a single patch. > > I understood this and it was okay. > > But I still wanted to know what change fixes what bug and what test > case. > >> Okey. Let's do everything separately. >> These are separate reports for the problems >> found during writing a test case for 4634: >> https://mariadb.atlassian.net/browse/MDEV-4652 >> https://mariadb.atlassian.net/browse/MDEV-4653 >> https://mariadb.atlassian.net/browse/MDEV-4654 >> >> Here's a new one, found today: >> https://mariadb.atlassian.net/browse/MDEV-4651 > > Okay, as you prefer :) > > Regards, > Sergei >

2 1

[Maria-developers] query cache issue
by Roberto Spadim 27 Jun '13

27 Jun '13

guys, i sent some (many) MDEV issues to have a better control over query cache i want to comment here to don't stay with this idea in my mind only... i created some labels in JIRA: querycache - relative to query cache qc_info - relative to qc_info plugin querycache_client - relative to implement query cache at client side (not in server) querycache_control - relative to DELETE queries in cache, DELETE tables in cache, LOCK/UNLOCK QUERY CACHE querycache_insert - relative to prune of queries that should be or shouldn't be cacheable querycache_lowmen - realtive to what we could do if we get low men in query cache querycache_result - features relative to when should or shouldn't hit query cache when query is cached if they will be or not implemented is another history is labels ok? should i remove it? -- Roberto Spadim

2 5

Re: [Maria-developers] Rev 3766: Added separation between roles and users in the mysql.user table
by Sergei 26 Jun '13

26 Jun '13

Hi, Vicențiu! On Jun 25, noreply(a)launchpad.net wrote: > ------------------------------------------------------------ > revno: 3766 > committer: Vicențiu Ciorbaru <cvicentiu(a)gmail.com> > branch nick: trunk > timestamp: Tue 2013-06-25 23:52:30 +0300 > message: > Added separation between roles and users in the mysql.user table > modified: > sql/sql_acl.cc > > === modified file 'sql/sql_acl.cc' > --- sql/sql_acl.cc 2013-06-23 12:25:04 +0000 > +++ sql/sql_acl.cc 2013-06-25 20:52:30 +0000 > @@ -542,7 +542,16 @@ > #endif /* HAVE_OPENSSL && !EMBEDDED_LIBRARY */ > #define NORMAL_HANDSHAKE_SIZE 6 > > -static DYNAMIC_ARRAY acl_hosts, acl_users, acl_dbs, acl_proxy_users; > +static DYNAMIC_ARRAY acl_hosts, acl_users, acl_dbs, acl_proxy_users, acl_roles; > +/* XXX > + ***** Potential optimization ***** > + role_grants could potentially be a HASH with keys as usernames and values > + as a DYNAMIC_ARRAY of pointers to granted roles > + XXX > + ***** Implementation choice for now ***** > + An array representing mappings between acl_users and acl_roles; > +*/ Agree about a hash. Hosts, users and dbs are stored in an array, because they support wildcards in MariaDB, not only exact matches. And very specific sort order, from most specific to least specific. But we only need exact matches for role names, so you can put all roles in a hash (see HASH or Hash_set). > +static DYNAMIC_ARRAY role_grants; Same. Do you need an array here? Is the order of entries important? I don't think it is. In fact, you probably may store role<->user relationships directly in the ACL_USER structure. For a user - list of granted roles. For a role - list of users it is granted to. Then it should be easy to check if a given user can SET a given role. > static MEM_ROOT mem, memex; > static bool initialized=0; > static bool allow_all_hosts=1; > @@ -876,11 +891,20 @@ > while (!(read_record_info.read_record(&read_record_info))) > { > ACL_USER user; > + bool is_role= FALSE; > bzero(&user, sizeof(user)); > update_hostname(&user.host, get_field(&mem, table->field[0])); > user.user= get_field(&mem, table->field[1]); > - DBUG_PRINT("info", ("user_entry %s", user.user)); > - if (check_no_resolve && hostname_requires_resolving(user.host.hostname)) > + > + /* If the user entry is a role, skip password and hostname checks > + A user can not log in with a role so some checks are not necessary > + */ > + if (!user.host.hostname) { > + is_role= TRUE; I'm sorry, but it wouldn't do. Apparently it is possible to create a user with the empty host name: MariaDB [test]> create user foo@''; Query OK, 0 rows affected (0.00 sec) MariaDB [test]> select user,host from mysql.user; +------+-----------+ | user | host | +------+-----------+ | foo | | Looking at compare_hostname() function, one can see that empty hostname means "any hostname is ok", it's a wildcard. It means, you cannot use empty hostnames to distinguish between roles and users. Which is a pity, I'd also prefer to use an empty hostname for that. You can use a NULL hostname (you'd need ALTER TABLE to make the column nullable) or a separate column, like, Is_role ENUM('N','Y'). > + } > + > + if (!is_role && check_no_resolve && > + hostname_requires_resolving(user.host.hostname)) Regards, Sergei

1 0

Re: [Maria-developers] Rev 3517: MySQL 5.6.10 performance schema: merge of host_cache table
by Sergei Golubchik 25 Jun '13

25 Jun '13

Hi, Timour! Just few comments On Jun 25, timour(a)askmonty.org wrote: > ------------------------------------------------------------ > revno: 3517 > revision-id: timour(a)askmonty.org-20130621115451-tyxgeeroqyxwz7gn > parent: monty(a)askmonty.org-20130623091543-zqu5dvhry6okna6g > committer: timour(a)askmonty.org > branch nick: 10.0-perf-schema-merge > timestamp: Fri 2013-06-21 14:54:51 +0300 > message: > MySQL 5.6.10 performance schema: merge of host_cache table > === modified file 'sql/hostname.cc' > --- a/sql/hostname.cc 2013-01-15 18:13:32 +0000 > +++ b/sql/hostname.cc 2013-06-21 11:54:51 +0000 > @@ -24,7 +24,7 @@ > Hostnames are checked with reverse name lookup and checked that they > doesn't resemble an IP address. > */ > - > +#include "my_global.h" > #include "sql_priv.h" > #include "hostname.h" > #include "my_global.h" you don't need to include my_global.h twice here > === modified file 'sql/sql_acl.cc' > --- a/sql/sql_acl.cc 2013-06-17 23:01:34 +0000 > +++ b/sql/sql_acl.cc 2013-06-21 11:54:51 +0000 > @@ -8730,7 +8737,6 @@ static int server_mpvio_read_packet(MYSQ > err: > if (mpvio->status == MPVIO_EXT::FAILURE) > { > - inc_host_errors(mpvio->thd->security_ctx->ip); > if (!mpvio->thd->is_error()) > { > if (mpvio->make_it_fail) > @@ -8890,6 +8896,10 @@ static int do_auth_once(THD *thd, const > else > { > /* Server cannot load the required plugin. */ > + Host_errors errors; > + errors.m_no_auth_plugin= 1; > + // merge-todo: struct MPVIO_EXT’ has no member named ‘ip’ > + // inc_host_errors(mpvio->ip, &errors); see above, the line you've removed in a previous chunk, used inc_host_errors(mpvio->thd->security_ctx->ip); > my_error(ER_PLUGIN_IS_NOT_LOADED, MYF(0), auth_plugin_name->str); > res= CR_ERROR; > } > @@ -8913,8 +8923,6 @@ static int do_auth_once(THD *thd, const > Perform the handshake, authorize the client and update thd sctx variables. > > @param thd thread handle > - @param connect_errors number of previous failed connect attemps > - from this host really? you've removed it from the comment, but not from the function, so now acl_authenticate() has one undocumented argument. > @param com_change_user_pkt_len size of the COM_CHANGE_USER packet > (without the first, command, byte) or 0 > if it's not a COM_CHANGE_USER (that is, if > @@ -9017,8 +9025,27 @@ bool acl_authenticate(THD *thd, uint con > > if (res > CR_OK && mpvio.status != MPVIO_EXT::SUCCESS) > { > + Host_errors errors; > DBUG_ASSERT(mpvio.status == MPVIO_EXT::FAILURE); > - > + switch (res) > + { > + case CR_AUTH_PLUGIN_ERROR: > + errors.m_auth_plugin= 1; > + break; > + case CR_AUTH_HANDSHAKE: > + errors.m_handshake= 1; > + break; > + case CR_AUTH_USER_CREDENTIALS: > + errors.m_authentication= 1; > + break; > + case CR_ERROR: > + default: > + /* Unknown of unspecified auth plugin error. */ > + errors.m_auth_plugin= 1; > + break; > + } > + // merge-todo: struct MPVIO_EXT’ has no member named ‘ip’ > + // inc_host_errors(mpvio.ip, &errors); same as above. And personally, I'd rather make a method errors.set_error(res); and move the switch inside. But you may decide to stay closer to 5.6 code. > if (!thd->is_error()) > login_failed_error(thd); > DBUG_RETURN(1); > @@ -9256,23 +9283,20 @@ static int native_password_authenticate( > #endif > > if (pkt_len == 0) /* no password */ > - DBUG_RETURN(info->auth_string[0] ? CR_ERROR : CR_OK); > + DBUG_RETURN(mpvio->acl_user->salt_len != 0 ? CR_AUTH_USER_CREDENTIALS : CR_OK); Why did you change the condition? Our pluggable auth code is not identical to MySQL's, so not all changes can be copied verbatim from there. I think the old condition was ok. > info->password_used= PASSWORD_USED_YES; > if (pkt_len == SCRAMBLE_LENGTH) > { > if (!mpvio->acl_user->salt_len) > - DBUG_RETURN(CR_ERROR); > + DBUG_RETURN(CR_AUTH_USER_CREDENTIALS); > > - if (check_scramble(pkt, thd->scramble, mpvio->acl_user->salt)) > - DBUG_RETURN(CR_ERROR); > - else > - DBUG_RETURN(CR_OK); > + DBUG_RETURN(check_scramble(pkt, thd->scramble, mpvio->acl_user->salt) ? > + CR_AUTH_USER_CREDENTIALS : CR_OK); could you change it back please and only replace the error code? if() is easier to debug, than a function call inside DBUG_RETURN(). I suppose that's why we have it that way. > } > > - inc_host_errors(mpvio->thd->security_ctx->ip); > my_error(ER_HANDSHAKE_ERROR, MYF(0)); > - DBUG_RETURN(CR_ERROR); > + DBUG_RETURN(CR_AUTH_HANDSHAKE); > } Regards, Sergei

1 0

[Maria-developers] Debug interface
by Roberto Spadim 25 Jun '13

25 Jun '13

hi guys, what ide could i use to debug easily in linux mariadb version? i'm using gdb via command line, but it's very complicate, i'm a visual studio user, but a console interface is nice, if i could put some break points and F5 (run),F8 (run one step) there's another place to ask about it? -- Roberto Spadim

3 2

[Maria-developers] RFC: Pausing a query thread
by Marian Marinov 25 Jun '13

25 Jun '13

Hi guys, I work with hosting companies and all of my clients have the same irritating problem, one query taking a lot of time to finish and consuming a lot of resources. So I have solved the same issue but with processes on the machine by simply pausing(kill -STOP) these processes when the load is high and continuing(kill -CONT) them when the server load goes down. I want to implement the same logic within MariaDB, so I can STOP/CONT the thread that is executing this query. My idea is to implement this in the 'KILL query_id' function, so it will accept one more argument, which will be either STOP or CONT. If the parameter is not present, the default behavior should remain. What do you think about my approach? Marian

6 18

Re: [Maria-developers] Help with MDEV-4419, ENUM optimization
by Roberto Spadim 24 Jun '13

24 Jun '13

hummm :) nice, i will study it =) it give me a lot of speed, since some system are closed source and i can't rewrite the query, just change the mysql server or put a mysql proxy

1 5

[Maria-developers] users table
by Roberto Spadim 24 Jun '13

24 Jun '13

hi guys, what's the better way to have a shared mysql schema? i'm talking about creating a user in one database and this user be created in many databases (in the same server) i'm trying with symbolic links in data directory, but sure i have some problems that i need same server version (ok no problem) and flush tables in all servers after add an user i don't know if i could do a two replications since it's like a master-master storage when one database add a user all databases need that information any idea? -- Roberto Spadim

4 4

[Maria-developers] Debug on windows
by Roberto Spadim 24 Jun '13

24 Jun '13

Hi guys i want to learn about optrange.cc, but i dont know how to debug I'm using visual studio 2008 on windows, could anyone help me I will place a break point at get_mm_leaf function and watch all variables Anyone could help me?

1 0

[Maria-developers] Build and/or support of MariaDB on AIX
by Frank Fegert 23 Jun '13

23 Jun '13

Hello all, i'm currently running MySQL 5.1.62 on IBM Power LPARs with AIX 6.1. I'll spare you the discussion of why AIX and not Linux and will just say "it is what it is ATM" ;-) Anyhow, with things proceeding as they are, i guess chances are that Oracle and IBM are not going to get any more friendly with each other, so i'm looking for an alternative way to move forward. That being said, i'd like to get an idea of the general feeling around the MariaDB development team on the support of non-(Linux|*BSD|Windows) platforms. Are there currently any plans? Would you be willing to invest some time in helping to get MariaDB running on AIX? Right now, i can successfully build stock MySQL 5.1.70 with some minor patches and stock MySQL 5.5.32 with some minor patches and the option "DWITHOUT_PERFSCHEMA_STORAGE_ENGINE=ON". With performance schema enabled the build fails at linking the unittests in "storage/perfschema/unittest/", see below for a full error output. The MariaDB 5.5.31 build does currently not even go that far, if fails earlier on the threadpool code, see below. Unfortunately my C/C++-fu is not strong enough to determine what might be considered a sound alternative to the epoll/kevent/port_event calls on AIX to do an alteration like the one in MDEV-4174. Can someone please help me out here? My build environment: Compiler: IBM XL C/C++ for AIX, V11.1 (5724-X13) Version: 11.01.0000.0009 AIX oslevel: 6100-08-01-1245 Thanks & best regards, Frank Fegert ## Begin: MariaDB 5.5.31 ############################################## ... [ 37%] Building CXX object sql/CMakeFiles/sql.dir/threadpool_unix.cc.o cd /home/user/rpmbuild/tmp/mariadb-5.5.31/build-mysql/sql && /usr/vacpp/bin/xlC_r -+ -DHAVE_CONFIG_H -DHAVE_EVENT_SCHEDULER -DHAVE_OPENSSL -DHAVE_POOL_OF_THREADS -DMYSQL_SERVER -ma -O2 -qlanglvl=extc99 -qstrict -qoptimize=2 -qmaxmem=16384 -DSYSV -D_AIX -D_AIX32 -D_AIX41 -D_AIX43 -D_AIX51 -D_AIX52 -D_AIX53 -D_AIX61 -D_ALL_SOURCE -DFUNCPROTO=15 -I/opt/freeware/include -qthreaded -qhalt=e -qstaticinline -g -DDBUG_OFF -I/home/user/rpmbuild/tmp/mariadb-5.5.31/build-mysql/include -I/home/user/rpmbuild/tmp/mariadb-5.5.31/include -I/home/user/rpmbuild/tmp/mariadb-5.5.31/sql -I/home/user/rpmbuild/tmp/mariadb-5.5.31/regex -I/home/user/rpmbuild/tmp/mariadb-5.5.31/build-mysql/sql -o CMakeFiles/sql.dir/threadpool_unix.cc.o -c /home/user/rpmbuild/tmp/mariadb-5.5.31/sql/threadpool_unix.cc "/home/user/rpmbuild/tmp/mariadb-5.5.31/sql/threadpool_unix.cc", line 39.2: 1540-0859 (S) #error directive: threadpool is not available on this platform. gmake[2]: *** [sql/CMakeFiles/sql.dir/threadpool_unix.cc.o] Error 1 gmake[2]: Leaving directory `/home/user/rpmbuild/tmp/mariadb-5.5.31/build-mysql' gmake[1]: *** [sql/CMakeFiles/sql.dir/all] Error 2 gmake[1]: Leaving directory `/home/user/rpmbuild/tmp/mariadb-5.5.31/build-mysql' gmake: *** [all] Error 2 ## End: MariaDB 5.5.31 ################################################ ## Begin: MySQL 5.5.32 ################################################ ... Linking CXX executable pfs-t cd /home/user/rpmbuild/tmp/mysql-5.5.32/build-mysql/storage/perfschema/unittest && /opt/freeware/bin/cmake -E cmake_link_script CMakeFiles/pfs-t.dir/lin k.txt --verbose=1 /usr/vacpp/bin/xlC_r -ma -O2 -qlanglvl=extc99 -qstrict -qoptimize=2 -qmaxmem=16384 -DSYSV -D_AIX -D_AIX32 -D_AIX41 -D_AIX43 -D_AIX51 -D_AIX52 -D_AIX53 -D_A IX61 -D_ALL_SOURCE -DFUNCPROTO=15 -I/opt/freeware/include -qthreaded -qhalt=e -qstaticinline -L/opt/freeware/lib/mysql -L/opt/freeware/lib -Wl,-bmaxdata:0x80000000 CMakeFiles/pfs-t.dir/pfs-t.cc.o -o pfs-t -Wl,-brtl,-bnoipath,-bexpfull ../../../unittest/mytap/libmytap.a ../libperfschema.a ../../../mysys/libmysys.a ../../../dbug/libdbug.a ../../../mysys/libmysys.a ../../../dbug/libdbug.a ../../../strings/libstrings.a -lz -lm -Wl,-blibpath:/opt/freeware/lib:/usr/lib:/lib ld: 0711-317 ERROR: Undefined symbol: .MDL_request::init(MDL_key::enum_mdl_namespace,const char*,const char*,enum_mdl_type,enum_mdl_duration) ld: 0711-317 ERROR: Undefined symbol: .LEX::LEX() ld: 0711-317 ERROR: Undefined symbol: .lex_start(THD*) ld: 0711-317 ERROR: Undefined symbol: Virtual table for class "DML_prelocking_strategy": Some possible causes are: first non-inline virtual function in "DML_prelocking_strategy" is not defined; or the class is a template instantiation and an explicit instantiation definition of the class is missing. ld: 0711-317 ERROR: Undefined symbol: .open_and_lock_tables(THD*,TABLE_LIST*,bool,unsigned int,Prelocking_strategy*) ld: 0711-317 ERROR: Undefined symbol: .Table_check_intact::check(TABLE*,const st_table_field_def*) ld: 0711-317 ERROR: Undefined symbol: .close_thread_tables(THD*) ld: 0711-317 ERROR: Undefined symbol: .lex_end(LEX*) ld: 0711-317 ERROR: Undefined symbol: .Query_tables_list::destroy_query_tables_list() ld: 0711-317 ERROR: Undefined symbol: .plugin_unlock_list(THD*,st_plugin_int**,unsigned int) ld: 0711-317 ERROR: Undefined symbol: THR_THD ld: 0711-317 ERROR: Undefined symbol: .sql_print_error(const char*,...) ld: 0711-317 ERROR: Undefined symbol: Virtual table for class "st_select_lex_node": Some possible causes are: first non-inline virtual function in "st_select_lex_node" is not defined; or the class is a template instantiation and an explicit instantiation definition of the class is missing. ld: 0711-317 ERROR: Undefined symbol: lower_case_table_names ld: 0711-317 ERROR: Undefined symbol: .Field_enum::store_type(unsigned long long) ld: 0711-317 ERROR: Undefined symbol: .ACL_internal_schema_registry::register_schema(const st_mysql_lex_string*,const ACL_internal_schema_access*) ld: 0711-317 ERROR: Undefined symbol: handler::clone(const char*,st_mem_root*) ld: 0711-317 ERROR: Undefined symbol: handler::get_error_message(int,String*) ld: 0711-317 ERROR: Undefined symbol: handler::column_bitmaps_signal() ld: 0711-317 ERROR: Undefined symbol: handler::index_read_idx_map(unsigned char*,unsigned int,const unsigned char*,unsigned long,ha_rkey_function) ld: 0711-317 ERROR: Undefined symbol: handler::index_next_same(unsigned char*,const unsigned char*,unsigned int) ld: 0711-317 ERROR: Undefined symbol: handler::read_multi_range_first(st_key_multi_range**,st_key_multi_range*,unsigned int,bool,st_handler_buffer*) ld: 0711-317 ERROR: Undefined symbol: handler::read_multi_range_next(st_key_multi_range**) ld: 0711-317 ERROR: Undefined symbol: handler::read_range_first(const st_key_range*,const st_key_range*,bool,bool) ld: 0711-317 ERROR: Undefined symbol: handler::read_range_next() ld: 0711-317 ERROR: Undefined symbol: handler::read_first_row(unsigned char*,unsigned int) ld: 0711-317 ERROR: Undefined symbol: handler::get_dynamic_partition_info(PARTITION_STATS*,unsigned int) ld: 0711-317 ERROR: Undefined symbol: handler::get_auto_increment(unsigned long long,unsigned long long,unsigned long long,unsigned long long*,unsigned long long*) ld: 0711-317 ERROR: Undefined symbol: handler::drop_table(const char*) ld: 0711-317 ERROR: Undefined symbol: Virtual table for class "handler": Some possible causes are: first non-inline virtual function in "handler" is not defined; or the class is a template instantiation and an explicit instantiation definition of the class is missing. ld: 0711-317 ERROR: Undefined symbol: .handler::ha_table_share_psi(const TABLE_SHARE*) const ld: 0711-317 ERROR: Undefined symbol: .handler::ha_statistic_increment(unsigned long system_status_var::*) const ld: 0711-317 ERROR: Undefined symbol: .handler::print_error(int,int) ld: 0711-317 ERROR: Undefined symbol: key_map_empty ld: 0711-317 ERROR: Undefined symbol: .calculate_key_len(TABLE*,unsigned int,const unsigned char*,unsigned long) ld: 0711-345 Use the -bloadmap or -bnoquiet option to obtain more information. gmake[2]: *** [storage/perfschema/unittest/pfs-t] Error 8 gmake[2]: Leaving directory `/home/user/rpmbuild/tmp/mysql-5.5.32/build-mysql' gmake[1]: *** [storage/perfschema/unittest/CMakeFiles/pfs-t.dir/all] Error 2 gmake[1]: Leaving directory `/home/user/rpmbuild/tmp/mysql-5.5.32/build-mysql' gmake: *** [all] Error 2 ###End: MySQL 5.5.32 ##################################################

3 3

Re: [Maria-developers] Build and/or support of MariaDB on AIX
by Frank Fegert 23 Jun '13

23 Jun '13

Hello Harald, On Sun, Jun 23, 2013 at 12:37:24PM +0200, Reindl Harald wrote: > your outoput contains "#error directive: threadpool is not available > on this platform" which clearly states you must not configure > "-DHAVE_POOL_OF_THREADS" on your platform thanks for your fast reply! Yes, to disable threadpool for AIX would be an option, but unfortunately the /dev/human attached to my computer seems to be defective ;-) All fun aside, i went through the configure option list, but nothing related to threadpools caught my eye. Manually patching the above option out of "sql/CMakeLists.txt" didn't help either. I'm missing something here, aren't i? Thanks & best regards, Frank

1 0

[Maria-developers] Uninstalling MariaDB 5.2 does not remove everything.
by Peter Laursen 22 Jun '13

22 Jun '13

For @wlad I guess: After uninstalling MariaDB 5.2.on Win7/64 some (don't remember which minor version), it remains in the list of installed programs in Control Panel .. Programs. Trying to uninstall displays this dialog (translates to "This action is only valid for programs currently installed"). I did not try if rebooting system would remove the item from the list. It may also have been fixed in the installer since the 5.2 release I installed. -- Peter

2 5

Re: [Maria-developers] [GSoC] Kerberize MariaDB -- some unclear point about the project
by QIU Shuang 22 Jun '13

22 Jun '13

Hi wlad, Thank you for your concern. > Create user > 'foo@bar'@localhost creates user foo@bar, on localhost. > create user that is identified with name and domain and can connect > from any computer I admit that I mis-understood the usage of User@Host in MariaDB. I thought the User and Host fields in MariaDB are in the same place as those in a Kerberos principal. i.e. if my MariaDB login name is qiush(a)sjtu.edu.cn, then my Kerberos principal will be qiush(a)sjtu.edu.cn/CHINA, where MariaDB login name is part of Kerberos principal. (if that case, the realm part is omitted in MariaDB, and we should find another way to figure it out. That's what I argued in my previous email.) From your reply, it seems qiush@sjtu.edu.cn/CHINA@sjtu.edu.cn, the bold part is MariaDB User and italic part Host, can be a valid login name in our project. Suddenly realise the Host in MariaDB login name will constraint the user login place. It's much clear now. > Re realm, I do not know this much but 'shuang(a)uni.shanghai.edu/REALM' also > does not look too weird to me. To me, either :). > Or perhaps I miss something still? Can you elaborate? No, you're right. I confused these two names. Thank you for you hints! Sincerely, Shuang On Jun 20, 2013, at 2:22 AM, Vladislav Vaintroub <wlad(a)montyprogram.com> wrote: > > > From: QIU Shuang [mailto:qiush.summer@gmail.com] > Sent: Mittwoch, 19. Juni 2013 19:52 > To: Vladislav Vaintroub > Subject: Re: [Maria-developers] [GSoC] Kerberize MariaDB -- some unclear > point about the project > > > Hi Shuang, > >>> Trying to make a nicer name, for example by removing domain part could > introduce some ambiguity here and different Kerberos users to login as the > same. >> I think so. >> But per my knowledge, the fully qualified name in MariaDB is > username@hostname. >> What about the realm/domain part? >> I think this may be a gap between MariaDB and Kerberos. > > Maybe I oversee something, but I do not really see any contradiction here. > Do you mean that @ is special character should not be used in usernames? It > actually can, it just must be properly escaped. Create user > 'foo@bar'@localhost creates user foo@bar, on localhost. > Hypothetical CREATE USER 'shuang(a)uni.shanghai.edu' @'%' IDENTIFIED WITH > 'Kerberos' > > will create user that is identified with name and domain and can connect > from any computer (due to use of wildcard for computername part, this > wildcard can be omitted). > Re realm, I do not know this much but 'shuang(a)uni.shanghai.edu/REALM' also > does not look too weird to me. > > Or perhaps I miss something still? Can you elaborate? > > Wlad >

3 4

[Maria-developers] help with parser
by Roberto Spadim 20 Jun '13

20 Jun '13

hi guys i'm trying to implement MDEV-4579<https://mariadb.atlassian.net/browse/MDEV-4579>, could anyone help me with parser + warning generation? MDEV-4579 title: SQL PARSER - Add Ignore to SET SQL command example: SET @@some_global_variable_that_dont_exists=1 *this cause an error* SET @@some_global_variable_that_dont_exists=1,@@some_variable_that_exists_and_should_be_changed=3 *this cause an error and don't change the second variable* *could we implement:* SET *IGNORE * @@some_global_variable_that_dont_exists=1,@@some_variable_that_exists_and_should_be_changed=3 *this cause an warning (first variable don't exists) and change the second variable* thanks -- Roberto Spadim

2 1

[Maria-developers] [GSoC] Kerberize MariaDB -- some unclear point about the project
by QIU Shuang 19 Jun '13

19 Jun '13

Hi Wlad and the community, I've developed a Kerberos plugin prototype with both raw API and GSSAPI supported on my own Linux box. The codes are pushed to my private repo already. It is just a prototype, which can handle only the normal case, without any tests in deployed environment. And I know there is a long way to go before accepted I now have a clear knowledge about the control flow of authentication plugin and Kerberos protocol. The working use case is like this: 1. login my computer as, for example, qiush; 2. create a user in MariaDB named qiush; 3. get the initial credential from Kerberos Key Distribution Center (KDC), i.e. tgt; 4. connect mysqld as qiush and now I can login DBMS passwordlessly before tgt expires. Correctly me if I miss something in this case. The exception case is how to get/renew tgt when the original expired. I have created a private repo based on 5.5 release version, which is available at https://launchpad.net/~qiush-summer. For those that concern about our project, you can track my progress. (The reason why I choose 5.5 release version instead of the latest development branch 10.x is that I cannot clone the repo lp:~maria-captains/maria/10.0-base from launchpad and I don't know why. I'll figure it out later if necessary). Some problems (three, exactly) when implementing the toy plugin. The first one is about the private repo and bazaar (I'm a git guy and have little experience working with bazaar). In order to make some demonstrated progress, I push my local update to the private repo each time I create a new commit. It seems the commit number is global (see the screen shot of my private repo history below). Recent revisions 3796. By Shuang Qiu 20 hours ago prototype with GSSAPI enabled 3795. By Shuang Qiu on 2013-06-15 Kerberos plugin prototype with raw krb5 API 3794. By Shuang Qiu on 2013-06-13 doodle 3793. By Michael Widenius on 2013-06-11 Fixed tests that failed on 32 bit because of my earlier fixes of 32 bit limits. 3792. By Sergei on 2013-06-07 MDEV-4468 Assertion `error != 0' fails or timeout occurs on select from a FEDERATED table which points at a non-existent table I'm afraid that my commits may pollute the code base. So the question is that will others (e.g. Sergei since commit 3792) see my changes next time he gets local branch updated. Is it the best practice to use bazaar like this (push after each commit)? If not, could you please give me some suggestion to use bazaar and launchpad correctly in our community? The other two are about the exception case. When get the initial credential from KDC, clients/services are asked for a password or asked to provide a keytab file to derive a long-term key that can be used encrypting messages between clients and KDC. On the server/service side, I think the keytab approach is better. Otherwise, each time we restart the service, a password will be asked. It may be annoying, I think :p. Then how to figure out the name of keytab file? I can come up with three approaches: 1. hard coded the filename and document the keytab file name clearly in the manual for users to follow (not the best choice, I think); 2. use the default Kerberos setting, which is /etc/krb5.keytab when properly configured; 3. make the filename an option in configuration file, e.g. ~/.my.cnf, with a default value /etc/krb5.keytab, since it is the default Kerberos setting. I prefer the third solution. As to the client side, I think both are OK. The problem of keytab choice is the same. Then I try to get some hints from similar password requesting authentication plugin. I read code in plugin/auth_dialog and plugin/auth_examples where server plugin and client plugin are implemented separately. In the client plugin, a external function of type mysql_authenticaton_dialog_ask_t static mysql_authentication_dialog_ask_t ask; can be provided as an alternative to command line input. Should I also implement them in two parts? And also provide the mysql_authentication_dialog_ask_t hook? The last problem. Kerberos requires user to provide a service name to resolve before requesting service ticket. How to name our service? Hard coded it as MYSQL or set in configuration file (I prefer the latter one)? Let me know your thoughts :). Thank you very much and best regards! Sincerely, Shuang

2 2

Re: [Maria-developers] [GSoC] Kerberize MariaDB -- some unclear point about the project
by QIU Shuang 19 Jun '13

19 Jun '13

Hi serg, Thank you for your comment. I will clean my repo at some later time follow your advice. :) Sincerely, Shuang On Jun 18, 2013, at 4:51 PM, Sergei Golubchik <serg(a)mariadb.org> wrote: > Hi, QIU Shuang! > > Just to answer the bzr part: > > On Jun 16, QIU Shuang wrote: >> The first one is about the private repo and bazaar (I'm a git guy and >> have little experience working with bazaar). In order to make some >> demonstrated progress, I push my local update to the private repo each >> time I create a new commit. It seems the commit number is global (see >> the screen shot of my private repo history below). >> >> Recent revisions >> 3796. By Shuang Qiu 20 hours ago >> prototype with GSSAPI enabled >> 3795. By Shuang Qiu on 2013-06-15 >> Kerberos plugin prototype with raw krb5 API >> 3794. By Shuang Qiu on 2013-06-13 >> doodle >> 3793. By Michael Widenius on 2013-06-11 >> Fixed tests that failed on 32 bit because of my earlier fixes of 32 bit limits. >> 3792. By Sergei on 2013-06-07 >> MDEV-4468 Assertion `error != 0' fails or timeout occurs on select from a FEDERATED table which points at a non-existent table >> >> I'm afraid that my commits may pollute the code base. So the question >> is that will others (e.g. Sergei since commit 3792) see my changes >> next time he gets local branch updated. Is it the best practice to >> use bazaar like this (push after each commit)? > > Don't worry about it. Revision numbers in bzr aren't stable, during a > merge they might be renumbered (*). Revision ids are stable and unique, > but they look like > > sergii(a)pisem.net-20130607133513-p27s1bg8gt0amkiw > > (this is for my commit with revno 3792 above). > Think of revision numbers as of convenient shortcuts that help to avoid > typing and that are easy to remember. But don't use them for long-term > references or something. > > And no, I won't see your changes when I pull from 5.5. > Not unless I pull from your repository. > > Regards, > Sergei > > [1]. Details. Example. Say you have a repo with only one changeset. It > has revno 1. You've branched. And you've made changes and committed > something in both branches, independently. Every branch has now two > changesets. Changeset 1 (it's the same in both, save revid), and > Changeset 2 (revno is the same, revid is different). Now if you merge > the second branch into the first, the changeset 2 of the second branch > will be renumbered, it'll get revno 1.1 (but its revid won't change). > The history will look like (Changeset 3 is the merge changeset): > > 1 > | \ > 2 1.1 > | / > 3 > >

1 0

[Maria-developers] mysql 5.6, 5.7
by Roberto Spadim 18 Jun '13

18 Jun '13

hi guys, i was reading mysql 5.6 and 5.7 manual i know that mariadb is very different (a bit) than mysql internally there's a mysql merge issue or something like that i can see what will be merged and what not? i like the partition lock prune, memcache and optimizer tracer of mysql 5.6, but today i'm not using it in mariadb (not a problem, but it's nice features) -- Roberto Spadim

3 6

Re: [Maria-developers] review / close MDEV-4581
by Roberto Spadim 18 Jun '13

18 Jun '13

nice! thanks sergei! i think it's now rich in features, maybe should be used by community to check if it's usefull (there's many columns) thanks again! 2013/6/18 Sergei Golubchik <serg(a)mariadb.org> > Hi, Roberto! > > On Jun 18, Roberto Spadim wrote: > > hi guys, could review / close > > MDEV-4581<https://mariadb.atlassian.net/browse/MDEV-4581> ? > > Roberto, as you can see in Jira, > MDEV-4581 is assigned to me, it's targeted for the 10.0.4 release. > So yes, I could and I will. > > Regards, > Sergei > -- Roberto Spadim SPAEmpresarial

1 0

Re: [Maria-developers] close some mdev
by Roberto Spadim 18 Jun '13

18 Jun '13

hi sergei! thanks about others mdev, and yes mdev 4583 shouldn't be closed, i will study a bit more to see what can be done to write a patch about it and test the speed of a procedure removing query cache queries 2013/6/18 Sergei Golubchik <serg(a)mariadb.org> > Hi, Roberto! > > On Jun 17, Roberto Spadim wrote: > > hi guys could anyone close some opened and fixed (won't fix) mdev's ? the > > comment explain why to close it > > > > MDEV-4571 <https://mariadb.atlassian.net/browse/MDEV-4571> > > MDEV-4589 <https://mariadb.atlassian.net/browse/MDEV-4589> > > MDEV-575 <https://mariadb.atlassian.net/browse/MDEV-575> > > Closed. > > > MDEV-4583 <https://mariadb.atlassian.net/browse/MDEV-4583> > > Not closed, it doesn't have a comment like the three above? > Should I still close it? > > Regards, > Sergei > -- Roberto Spadim SPAEmpresarial

1 0

[Maria-developers] a problem of Parallel replication
by nanyi607rao 18 Jun '13

18 Jun '13

hi, when i read patchs from http://bazaar.launchpad.net/~knielsen/maria/dingqi-parallel-replication/cha…, i found a problem (maybe would cause bug) :main SQL thread read a event from relay log, it make the log_event->thd point to the thd of main SQL thread. But when a log_event is disaptched to a sub SQL thread, the log_event->thd still point to the old value instead of the thd of sub SQL thread. this problem will lead check_stack_overrun() fails, then lead open_table() fails... maybe this will be helpful to you . 2013-06-18 nanyi607rao

1 0

[Maria-developers] review / close MDEV-4581
by Roberto Spadim 18 Jun '13

18 Jun '13

hi guys, could review / close MDEV-4581<https://mariadb.atlassian.net/browse/MDEV-4581> ? -- Roberto Spadim

1 0

[Maria-developers] Implementing implicit primary key in mysql server
by Lixun Peng 18 Jun '13

18 Jun '13

Hi, As we know, InnoDB has implicit primary key if a table hasn't defined a primary key. However MySQL server doesn't know this primary key, so this primary key will not apear in binlog. When we are using ROW format binlog for replication, if a table has no any indexes, that's a disaster. If a table without indexes do a DML (UPDATE/DELETE), of course it will run a long time in master, but in slave, it still need a long time. It will cause serious slave replication delay. For example: binlog_format=ROW; CREATE TABLE table1 ( col1 varchar(32), col2 int ); table1 hasn't any indexes, if table1 has 100 million rows, I do a query "UPDATE table1 SET col1='xxx' WHERE col2=100", and this query will effect 1 million rows. Then it has 1 million UPDATE events in binlog. After slave received these events, each events should scan all rows!!! It will cause serious replication delay! I saw:https://kb.askmonty.org/en/row-based-replication-with-no-primary-key/ But if no any indexes, it still a problem. I think mysql server can do the same thing as InnoDB do, if user doesn't define the primary key, mysql can add the primary key automatically. For example: If user create a table like "table1", we can do "ALTER TABLE table1 ADD __id bigint auto_increment, ADD PRIMARY KEY(__id)", so in fact, the "table1" structure is CREATE TABLE table1 ( col1 varchar(32), col2 int, __id bigint auto_increment primary key ); But when user do "SELECT * FROM table xxx" or "INSERT INTO table VALUES(xx)", we should ignore implicit primary key, "__id" will not apear in the result set. How do you think? Thanks, Lixun -- Senior Database Engineer @ Taobao.com Mobile Phone: +86 18658156856 (Hangzhou) Gtalk: penglixun(at)gmail.com Twitter: http://www.twitter.com/plinux Blog: http://www.penglixun.com

1 0

[Maria-developers] close some mdev
by Roberto Spadim 18 Jun '13

18 Jun '13

hi guys could anyone close some opened and fixed (won't fix) mdev's ? the comment explain why to close it 1. MDEV-4571 <https://mariadb.atlassian.net/browse/MDEV-4571> 1. MDEV-4583 <https://mariadb.atlassian.net/browse/MDEV-4583> 1. MDEV-4589 <https://mariadb.atlassian.net/browse/MDEV-4589> 1. MDEV-575 <https://mariadb.atlassian.net/browse/MDEV-575> -- Roberto Spadim SPAEmpresarial

1 0

Re: [Maria-developers] Jira new field
by Roberto Spadim 17 Jun '13

17 Jun '13

ok =) thanks 2013/6/17 Sergei Golubchik <serg(a)mariadb.org> > Hi, Roberto! > > On Jun 17, Roberto Spadim wrote: > > > > today i'm 'working around ' this, and putting the 'group information' in > > MDEV issue, example.. query cache things i put > > QUERY CACHE - some issue title > > processlist things i put > > PROCESSLIST - some issue title > > > > this works too, but well. mariadb team can change and when i lost my > memory > > (some times =) ) i need to read all issues again and find where i put the > > information > > You can certainly add labels like QUERY CACHE and PROCESSLIST. This > shouldn't be a problem. > > Regards, > Sergei > -- Roberto Spadim SPAEmpresarial

1 0

Re: [Maria-developers] Jira new field
by Roberto Spadim 17 Jun '13

17 Jun '13

heeh sometimes i just add a label that don't have interest to mariadb team, it's just to 'group' my issues if there's a interesting and nice way to do this could be nice it's just to organize my issues and when i have time see what changed and where, not changing the work of mariadb placing the right label in the right issue 2013/6/17 Sergei Golubchik <serg(a)mariadb.org> > Hi, Roberto! > > On Jun 17, Roberto Spadim wrote: > > hi guys, could we add a Reporter Label inside jira issue? instead of only > > Label (used by mariadb team) > > it's easier to filter issues, thinking about the reporter filter not for > > mariadb team filters > > But you can add your labels to the Label field, why would one need > another list of Labels? > > Regards, > Sergei > -- Roberto Spadim SPAEmpresarial

1 3

[Maria-developers] Jira new field
by Roberto Spadim 17 Jun '13

17 Jun '13

hi guys, could we add a Reporter Label inside jira issue? instead of only Label (used by mariadb team) it's easier to filter issues, thinking about the reporter filter not for mariadb team filters -- Roberto Spadim

1 0

[Maria-developers] Is it possible to export last seen GTID from IO thread?
by Pavel Ivanov 15 Jun '13

15 Jun '13

Kristian, We had an idea that can make master failovers much easier, please could you tell us how feasible to do that in MariaDB. Currently when master dies and failover is necessary we have to wait while SQL thread catches up with IO thread on all replicas before we'll be able to decide which replica should be new master. Otherwise we can loose some transactions. But if IO thread could export some variable indicating which GTID it's seen last then we'll be able to understand which replica is furthest in replication stream faster, without waiting. Of course for multi-domain replication it will be more complicated and IO thread will have to synthesize gtid_slave_pos which server would have after applying all relay logs. But exporting this synthesized position will still help a lot. So what do you think? Is it possible to do this in MariaDB? Thank you, Pavel

1 0

Re: [Maria-developers] Process suggestion for minor issues
by Elena Stepanova 14 Jun '13

14 Jun '13

(warning: it's a long one) Hi Sergei, On 6/14/2013 6:26 PM, Sergei Golubchik wrote: > Hi, Elena! > > On Jun 14, Elena Stepanova wrote: >> On 6/14/2013 2:27 PM, Rasmus Johansson wrote: >>> Here is a suggested process to handle the ever growing amount of >>> minors. If a minor issue isn't fixed after two consecutive releases >>> it's re-reviewed and either priority changed to major (or higher), >>> or the minor issue is closed with resolution Won't Fix and a >>> comment. Example of comment can be that "this is an edge case where >>> effort and benefits doesn't correlate". >> >> I don't think the algorithm is quite fair. It means that if a really >> minor bug happens to be found during a relatively slow period, it gets >> fixed, but another bug, even considerably more important (but still >> falling into "not major => minor" category), can be by this time closed >> as "not worth fixing" just because people had been busy with some other >> stuff and couldn't fix it over two previous releases; while logically, >> if somebody suddenly has time to fix a minor bug, the latter should be >> fixed first. > > This algorithm has a "re-reviewed and either priority changed to > major..." step, so it's not just the timing. Not like automatic closing > of old bugs. Yes, I got that part; but it doesn't change the potential faulty result. Lets say we have imaginary priorities 1-5 which are all "minor", 1 being the lowest. For a while you had had a bug of "priority 4". After 2 releases the reviewing force reviewed it, decided it was minor (rightly so), and closed it. A few days later you got a bug of "priority 1". A few more days later you suddenly have time to fix a bug -- you are stuck at your main task and need a distraction, or you've got a headache and can't work on anything serious, or you only have half an hour before you need to go pick up kids... whatever. Naturally, you go to the list of your open bugs, take a quick look and pick whatever fits. If you still had the bug of priority 4, you'd fix it now, but instead you fix the bug of priority 1. I don't know about others, but I do keep a bunch of open low-priority tasks, and while rarely, sometimes it happens that I can do something about a minor one. And then I look at them and pick by importance and "suitability for the moment", not by the creation time. > >> I'm fine with closing bugs as "Won't Fix", but imho it should be based >> on the merits of the bug itself, not on the bad timing of its >> creation. Since all our developers are very senior and can totally >> make such a decision on their own, I suppose anyone who a bug is >> assigned to can take a look at it and either close it as "Won't fix" >> with a proper explanation, regardless the timing (even right after >> receiving it), or keep it open. > > The main issue I didn't like - what that algorithm is making an attempt > of solving - is a growing pile of minor bugs that we aren't going to > fix. I agree, there is no point at keeping those that we are really not going to fix. But I don't see anything critically bad at preserving those that we *want* to fix, just currently don't have time to -- as long as we don't schedule them for the closest release, thus lying to users and to ourselves (more on that later). I also think it's two essentially different categories. Later here you are saying that "a closed bug can always be reopened later and fixed". If we put the "don't have resources to fix" bugs into the same trash bin as we currently use for "really won't fix" bugs, re-opening will never happen. Who will go through the whole bunch of "Won't fix" bugs, review them *again* and see if they are really not worth fixing, or if we had to close them because we didn't have resources? Certainly you can't do it when you just have little time to fix one-two small bugs, you'll spend more time searching. > > A bug is real, so it's difficult to say "won't fix", it's much easier, > psychologically, to say, "okay, I'll fix it, *when I have time*". But > really, for minor bugs this might never happen. But sooner or later somebody who's reviewing them will dare close it anyway?.. Okay, I agree that psychologically it is easier to reject a bug *together with somebody* than make such a decision on your own, group mentality and all that, I get it and have it myself; so if it's all about having some kind of committee for closing such bugs, let it be. (I don't really know who could this committee consist of. Generally, I'm a big supporter of a triage team, it makes the process less personal. But in a company where everyone is pretty much unique in his area, it's a bit strange, because at the end only the expert can bring in an educated verdict -- and this would normally be the person the bug is already assigned to anyway; others will just be there to hold his hand.) > And de-facto this leads to wrong expectations (users believe the bug > will be fixed in a specific version, but it isn't) and us lying to our > users. True, but I think the new approach will mess up more than it will fix. Now we have Fix version evolution like that: 5.5.30 => 5.5.31 => 5.5.32 => 5.5.33 => .... It's bad. I still think that we shouldn't schedule minor bugs for any particular version. We've discussed it before, and yes, I know that we do it to keep them on somebody's radar, but I would think that having it assigned to somebody should be enough, since it's already on a radar of the person who it's assigned to. But maybe I'm wrong and nobody looks at lists of bugs assigned to them -- shouldn't we try to change it instead, then? But with the new system, we will have 5.5.30 => 5.5.31 => "Won't fix, it isn't worth it". If I were a user, now I'd be mad. If it isn't worth fixing, why didn't you say so right away? Why schedule it and even move it to the next release then? That's even more of lying to users than we currently do, and in a more irritating way. > My idea is to be honest about it, and if we don't have resources to fix > a specific bug, we won't drag it forever from one release to another, > but close it eventually. I agree. But I think we shouldn't even start dragging it then. If you are going to have a committee which would regularly review bugs (those that have waited for 2 releases) and decide if they're worth fixing or not, why not do it right away -- let the same committee decide it for *new* bugs instead, so that unimportant ones don't even get scheduled and go to the trash bin immediately. It will spare users wrong expectations. > And yes, I hate closing non-fixed bugs too. > > If you can suggest another solution, please do. But "keep it open" > doesn't work, as we've seen already. Well, I'd suggest to consider adjusting yours a little, in the following way: - define who will be reviewing the bugs; - those person(s) will screen new bugs instead of those that have waited for 2 releases; - , set "Won't fix" when we really mean it (example: PBXT bugs, etc.), and create a new resolution "Don't have time to fix" or "Wishlist" or whatever, and put corresponding bugs there, while keeping them assigned. The last point is to preserve at least hypothetical possibility for the bugs to be re-opened later. So, if somebody has time to fix a minor bug, first he looks at the list of his open bugs (we already know that the committee kept only most worthy ones there). If nothing fits the moment, or (a miracle!) the list is empty, the somebody goes to the "wishlist" bugs assigned to him, and picks from there. At the same time, those bugs aren't shown as open, aren't scheduled for any particular release, the users aren't misled, etc. A couple more notes: - I think external bugs should generally have higher priority than otherwise equal ones found internally; - a different approach might be needed for bugs in new feature trees, it should be decided separately (the basic question is -- when we consider the feature good enough to be pushed to the main tree, whether it requires fixing minor bugs or not). /E > > Besides, a closed bug can always be reopened later and fixed. > > Regards, > Sergei >

2 1

[Maria-developers] Process suggestion for minor issues
by Rasmus Johansson 14 Jun '13

14 Jun '13

Hi MariaDB Developers, This is a suggestion on how to deal with minor or lower priority bugs and tasks in JIRA, http://mariadb.org/jira . Currently, the process we use makes a clear differentiation between major or higher prioirty issues (called majors altogether after this) and minor or lower priority issues (called minors altogether after this). The majors are the ones that should be fixed for a release and the minors are rolling, meaning that they are done when the corresponding developer has finished his/her majors. This of course sounds logical, but it also leads to a growing number of minors for every consecutive release, many of them which never will be fixed but in the current process will hang around. Here is a suggested process to handle the ever growing amount of minors. If a minor issue isn't fixed after two consecutive releases it's re-reviewed and either priority changed to major (or higher), or the minor issue is closed with resolution Won't Fix and a comment. Example of comment can be that "this is an edge case where effort and benefits doesn't correlate". It should also be noted that after closing an issue in JIRA, it doesn't mean that it disappears. It will still show up in free text searches and among the issues for the version where it was closed. Please provide your thoughts. Do you think it's an OK way to deal with the minors or do you prefer some other way? Best regards, Rasmus

6 8

Re: [Maria-developers] Review request: mdev-4635
by Alexander Barkov 14 Jun '13

14 Jun '13

Hi Sergei, On 06/14/2013 04:30 PM, Sergei Golubchik wrote: > Hi, Alexander! > > On Jun 13, Alexander Barkov wrote: >> On 06/13/2013 08:16 PM, Sergei Golubchik wrote: >>> >>>> === modified file 'mysql-test/r/strict.result' >>>> --- mysql-test/r/strict.result 2011-06-05 02:56:06 +0000 >>>> +++ mysql-test/r/strict.result 2013-06-13 12:15:43 +0000 >>>> @@ -213,11 +213,11 @@ ERROR 22007: Incorrect date value: '2004 >>>> INSERT INTO t1 (col1) VALUES(STR_TO_DATE('0.10.2004 15.30','%d.%m.%Y %H.%i')); >>>> ERROR 22007: Incorrect date value: '2004-10-00 15:30:00' for column 'col1' at row 1 >>>> INSERT INTO t1 (col1) VALUES(STR_TO_DATE('31.9.2004 15.30','%d.%m.%Y %H.%i')); >>>> -ERROR 22007: Incorrect date value: '2004-09-31 15:30:00' for column 'col1' at row 1 >>>> +ERROR HY000: Incorrect datetime value: '31.9.2004 15.30' for function str_to_date >>> >>> Hmm, I don't know. I liked the old error message more. >>> Indeed, datetime value '31.9.2004 15.30' is just fine for a function str_to_date(). >>> But '2004-09-31 15:30:00' is not fine for column 'col1' in the strict mode. >>> >>> Why did that change? >> >> Before the change the warning was generated in >> Field_timestamp::store_TIME_with_warning. >> >> Now this warning is generated much earlier, >> in Item_func_str_to_date::get_date(), >> after check_date() is called. > > Which, I think, it wrong. The date is perfectly ok, as far as > STR_TO_DATE is concerted, it's wrong for a field. > >> The problem was that Item_func_str_to_date: >> a. checked TIME_NO_ZERO_DATE >> b. did not check TIME_NO_ZERO_IN_DATE (the bug) >> c. did not check invalid dates >> >> After the patch it: >> >> a. checks TIME_NO_ZERO_DATE >> b. checks TIME_NO_ZERO_IN_DATE >> c. checks invalid dates >> >> I thought it would be good to check everything, >> for consistency. But a side effect is a different warning. > > I agree, it should check everything. But, perhaps, the field should not > pass TIME_NO_ZERO_DATE|TIME_NO_ZERO_IN_DATE down to the arg->get_date() ? It does not. The above warnings are not about zeros. They are about wrong dates: for example, there is no September 31, like in '31.9.2004 15.30'. The last version of the patch does not change the warnings. It works as follows: - str_to_date checks TIME_NO_ZERO_DATE and TIME_NO_ZERO_IN_DATE with help of check_date(), but only if the caller wants it (CONVERT_TZ wants, field does not). - str_to_date suppresses checking of invalid dates inside check_date() and leaves this responsibility to the caller (like it was before the patch). If we want check invalid dates inside check_date() in str_to_date, then this warning "Incorrect date value: '2004-09-31 15:30:00' for column 'col1' at row 1" won't be possible, because str_to_date does not know the name of the column it's being inserted into. But we could try to produce something like this: "Incorrect datetime value: '2004-09-31 15:30:00'" I.e. a better formatted value, but without the column name. > > Regards, > Sergei >

1 0

[Maria-developers] Review request: new SHUTDOWN server command
by James Briggs 14 Jun '13

14 Jun '13

Hi Monty. Here's the shutdown command (WL#232) I wrote for review. I think there's 2 steps to improve server shutdown: 1) apply this patch now 2) later on, fix the old shutdown thread race problems, clean up the old kill_mysql() design and code, then add a few options to the shutdown command like WAIT|IMMEDIATE|ABORT. This code is released under the Open Source BSD-new License, according to the MariaDB Contributor Agreement. Thanks, James Briggs. james.briggs(a)yahoo.com http://www.jebriggs.com/blog/2013/05/patch-to-add-shutdown-statement-to-mys… --- sql_parse.cc 2013-03-11 03:29:13.000000000 -0700 +++ /home/james/mariadb-5.5.30-new/sql/sql_parse.cc 2013-05-15 13:17:05.000000000 -0700 @@ -1305,7 +1305,6 @@ my_ok(thd); break; } -#ifndef EMBEDDED_LIBRARY case COM_SHUTDOWN: { status_var_increment(thd->status_var.com_other); @@ -1333,7 +1332,6 @@ error=TRUE; break; } -#endif case COM_STATISTICS: { STATUS_VAR *current_global_status_var; // Big; Don't allocate on stack @@ -3736,6 +3734,31 @@ lex->kill_signal); break; } + case SQLCOM_SHUTDOWN: + { + // jeb - This code block is copied from COM_SHUTDOWN above. Since kill_mysql(void) {} doesn't take a level argument, the level code is pointless. + // jeb - In fact, the level code should be removed and Oracle Database statements implemented: SHUTDOWN, SHUTDOWN IMMEDIATE and SHUTDOWN ABORT. See WL#232. + + status_var_increment(thd->status_var.com_other); + if (check_global_access(thd,SHUTDOWN_ACL)) + break; /* purecov: inspected */ + + enum mysql_enum_shutdown_level level; + level= SHUTDOWN_DEFAULT; + if (level == SHUTDOWN_DEFAULT) + level= SHUTDOWN_WAIT_ALL_BUFFERS; // soon default will be configurable + else if (level != SHUTDOWN_WAIT_ALL_BUFFERS) + { + my_error(ER_NOT_SUPPORTED_YET, MYF(0), "this shutdown level"); + break; + } + DBUG_PRINT("SQLCOM_SHUTDOWN",("Got shutdown command for level %u", level)); + my_eof(thd); + kill_mysql(); + res=TRUE; + break; + } + #ifndef NO_EMBEDDED_ACCESS_CHECKS case SQLCOM_SHOW_GRANTS: { --- sql_yacc.yy 2013-03-11 03:29:19.000000000 -0700 +++ /home/james/mariadb-5.5.30-new/sql/sql_yacc.yy 2013-05-15 11:12:03.000000000 -0700 @@ -791,7 +791,7 @@ Currently there are 174 shift/reduce conflicts. We should not introduce new conflicts any more. */ -%expect 174 +%expect 196 /* Comments for TOKENS. @@ -1645,6 +1645,7 @@ definer_opt no_definer definer parse_vcol_expr vcol_opt_specifier vcol_opt_attribute vcol_opt_attribute_list vcol_attribute + shutdown END_OF_INPUT %type call sp_proc_stmts sp_proc_stmts1 sp_proc_stmt @@ -1796,6 +1797,7 @@ | savepoint | select | set + | shutdown | signal_stmt | show | slave @@ -13715,6 +13717,17 @@ ; +shutdown: + SHUTDOWN + { + LEX *lex=Lex; + lex->value_list.empty(); + lex->users_list.empty(); + lex->sql_command= SQLCOM_SHUTDOWN; + } + ; + + set_expr_or_default: expr { $$=$1; } | DEFAULT { $$=0; } --- sql_prepare.cc 2013-03-11 03:29:11.000000000 -0700 +++ /home/james/mariadb-5.5.30-new/sql/sql_prepare.cc 2013-05-15 03:07:00.000000000 -0700 @@ -2173,6 +2173,7 @@ case SQLCOM_GRANT: case SQLCOM_REVOKE: case SQLCOM_KILL: + case SQLCOM_SHUTDOWN: break; case SQLCOM_PREPARE: --- mysqld.cc 2013-03-11 03:29:14.000000000 -0700 +++ /home/james/mariadb-5.5.30-new/sql/mysqld.cc 2013-05-15 01:20:11.000000000 -0700 @@ -3333,6 +3333,7 @@ {"savepoint", (char*) offsetof(STATUS_VAR, com_stat[(uint) SQLCOM_SAVEPOINT]), SHOW_LONG_STATUS}, {"select", (char*) offsetof(STATUS_VAR, com_stat[(uint) SQLCOM_SELECT]), SHOW_LONG_STATUS}, {"set_option", (char*) offsetof(STATUS_VAR, com_stat[(uint) SQLCOM_SET_OPTION]), SHOW_LONG_STATUS}, + {"shutdown", (char*) offsetof(STATUS_VAR, com_stat[(uint) SQLCOM_SHUTDOWN]), SHOW_LONG_STATUS}, {"signal", (char*) offsetof(STATUS_VAR, com_stat[(uint) SQLCOM_SIGNAL]), SHOW_LONG_STATUS}, {"show_authors", (char*) offsetof(STATUS_VAR, com_stat[(uint) SQLCOM_SHOW_AUTHORS]), SHOW_LONG_STATUS}, {"show_binlog_events", (char*) offsetof(STATUS_VAR, com_stat[(uint) SQLCOM_SHOW_BINLOG_EVENTS]), SHOW_LONG_STATUS}, --- sql_lex.h 2013-03-11 03:29:13.000000000 -0700 +++ /home/james/mariadb-5.5.30-new/sql/sql_lex.h 2013-05-15 01:19:17.000000000 -0700 @@ -193,6 +193,7 @@ SQLCOM_SHOW_RELAYLOG_EVENTS, SQLCOM_SHOW_USER_STATS, SQLCOM_SHOW_TABLE_STATS, SQLCOM_SHOW_INDEX_STATS, SQLCOM_SHOW_CLIENT_STATS, + SQLCOM_SHUTDOWN, /*

1 0

Re: [Maria-developers] Review request: mdev-4635
by Alexander Barkov 13 Jun '13

13 Jun '13

Hi, Serg, On 06/13/2013 08:16 PM, Sergei Golubchik wrote: > Hi, Alexander! > > On Jun 13, Alexander Barkov wrote: >> >> please review a fix for MDEV-4635 > > Is it for 5.1 or 5.5? For 5.3. > >> === modified file 'mysql-test/r/func_time.result' >> --- mysql-test/r/func_time.result 2013-03-20 15:13:00 +0000 >> +++ mysql-test/r/func_time.result 2013-06-13 08:39:15 +0000 >> @@ -1910,3 +1910,13 @@ SELECT 1 FROM DUAL WHERE MINUTE(TIMEDIFF >> 1 >> SELECT 1 FROM DUAL WHERE SECOND(TIMEDIFF(NULL, '12:12:12')); >> 1 >> +# >> +# MDEV-4635 Crash in UNIX_TIMESTAMP(STR_TO_DATE('2020','%Y')) >> +# >> +SET TIME_ZONE='+02:00'; >> +SELECT UNIX_TIMESTAMP(STR_TO_DATE('2020','%Y')); >> +UNIX_TIMESTAMP(STR_TO_DATE('2020','%Y')) >> +NULL >> +Warnings: >> +Error 1411 Incorrect datetime value: '2020' for function str_to_date >> +SET TIME_ZONE=DEFAULT; >> >> === modified file 'mysql-test/r/strict.result' >> --- mysql-test/r/strict.result 2011-06-05 02:56:06 +0000 >> +++ mysql-test/r/strict.result 2013-06-13 12:15:43 +0000 >> @@ -213,11 +213,11 @@ ERROR 22007: Incorrect date value: '2004 >> INSERT INTO t1 (col1) VALUES(STR_TO_DATE('0.10.2004 15.30','%d.%m.%Y %H.%i')); >> ERROR 22007: Incorrect date value: '2004-10-00 15:30:00' for column 'col1' at row 1 >> INSERT INTO t1 (col1) VALUES(STR_TO_DATE('31.9.2004 15.30','%d.%m.%Y %H.%i')); >> -ERROR 22007: Incorrect date value: '2004-09-31 15:30:00' for column 'col1' at row 1 >> +ERROR HY000: Incorrect datetime value: '31.9.2004 15.30' for function str_to_date > > Hmm, I don't know. I liked the old error message more. > Indeed, datetime value '31.9.2004 15.30' is just fine for a function str_to_date(). > But '2004-09-31 15:30:00' is not fine for column 'col1' in the strict mode. > > Why did that change? Before the change the warning was generated in Field_timestamp::store_TIME_with_warning. Now this warning is generated much earlier, in Item_func_str_to_date::get_date(), after check_date() is called. The problem was that Item_func_str_to_date: a. checked TIME_NO_ZERO_DATE b. did not check TIME_NO_ZERO_IN_DATE (the bug) c. did not check invalid dates After the patch it: a. checks TIME_NO_ZERO_DATE b. checks TIME_NO_ZERO_IN_DATE c. checks invalid dates I thought it would be good to check everything, for consistency. But a side effect is a different warning. But we can suppress invalid dates detection, so it: a. checks TIME_NO_ZERO_DATE b. checks TIME_NO_ZERO_IN_DATE c. does not check invalid dates The test for invalid dates can be suppressed by passing "fuzzy_date | TIME_INVALID_DATES" to check_date(). See this version of the patch attached. I agree that the old warning is nicer. But I'm not sure what's better. Suppressing the test rather slightly looks like hiding more potential bugs... > >> === modified file 'sql/item_timefunc.cc' >> --- sql/item_timefunc.cc 2013-03-20 15:13:00 +0000 >> +++ sql/item_timefunc.cc 2013-06-13 08:20:13 +0000 >> @@ -1126,7 +1125,7 @@ bool Item_func_unix_timestamp::get_times >> } >> >> MYSQL_TIME ltime; >> - if (get_arg0_date(&ltime, 0)) >> + if (get_arg0_date(&ltime, TIME_NO_ZERO_IN_DATE)) >> return 1; > > This is correct, of course. > > Regards, > Sergei >

1 0

[Maria-developers] Review request: mdev-4651
by Alexander Barkov 13 Jun '13

13 Jun '13

Hi Serg, Please review a patch for "mdev-4651 Crash in my_decimal2decimal in a ORDER BY query". Thanks.

1 0

[Maria-developers] Review request: mdev-4635
by Alexander Barkov 13 Jun '13

13 Jun '13

Hi Serg, please review a fix for MDEV-4635 Thanks.

1 0

Re: [Maria-developers] Review request: mdev-4634
by Alexander Barkov 13 Jun '13

13 Jun '13

Hi Sergei, Thanks for review! On 06/11/2013 09:33 PM, Sergei Golubchik wrote: > Hi, Alexander! > > This looks like a lot of changes for what you explained the reason of > the bug was. There will be lots of "whys" below... The fix itself was quite small. However, an attempt to write a reasonable test case revealed a few other bugs. I just tried to fixing everything in a single patch. Okey. Let's do everything separately. These are separate reports for the problems found during writing a test case for 4634: https://mariadb.atlassian.net/browse/MDEV-4652 https://mariadb.atlassian.net/browse/MDEV-4653 https://mariadb.atlassian.net/browse/MDEV-4654 Here's a new one, found today: https://mariadb.atlassian.net/browse/MDEV-4651 > > On Jun 11, Alexander Barkov wrote: >> === modified file 'sql/item.cc' >> --- sql/item.cc 2013-03-17 06:41:22 +0000 >> +++ sql/item.cc 2013-06-11 11:52:43 +0000 >> @@ -252,7 +252,8 @@ String *Item::val_string_from_decimal(St >> String *Item::val_string_from_date(String *str) >> { >> MYSQL_TIME ltime; >> - if (get_date(&ltime, TIME_FUZZY_DATE) || >> + uint time_flag= field_type() == MYSQL_TYPE_TIME ? TIME_TIME_ONLY : 0; >> + if (get_date(&ltime, TIME_FUZZY_DATE | time_flag) || > > Why? > >> str->alloc(MAX_DATE_STRING_REP_LENGTH)) >> { >> null_value= 1; >> >> === modified file 'sql/item_func.cc' >> --- sql/item_func.cc 2013-04-06 13:14:46 +0000 >> +++ sql/item_func.cc 2013-06-11 10:54:50 +0000 >> @@ -2456,7 +2456,8 @@ bool Item_func_min_max::get_date(MYSQL_T >> { >> Item **arg= args + i; >> bool is_null; >> - longlong res= get_datetime_value(thd, &arg, 0, compare_as_dates, &is_null); >> + longlong res= get_datetime_value(thd, &arg, 0, compare_as_dates, >> + &is_null, fuzzy_date); > > why is that? to handle CONVERT_TZ(GREATEST(...)) ? > That is to pass NO_ZERO_DATE down? > > I'm not sure it's a good idea. Strictly speaking, only the greatest > value should be NO_ZERO_DATE, not every single one. > > I'd suggest to add check_date in Item_func_min_max::get_date() instead. > >> /* Check if we need to stop (because of error or KILL) and stop the loop */ >> if (thd->is_error() || args[i]->null_value) >> @@ -2469,6 +2470,11 @@ bool Item_func_min_max::get_date(MYSQL_T >> min_max= res; >> } >> unpack_time(min_max, ltime); >> + >> + if (!(fuzzy_date & TIME_TIME_ONLY) && >> + ((null_value= check_date_with_warn(ltime, fuzzy_date)))) >> + return true; > > Yes, that's what I mean. If you check the return value anyway, there's > no need to check it inside get_datetime_value(). > >> if (compare_as_dates->field_type() == MYSQL_TYPE_DATE) >> { >> ltime->time_type= MYSQL_TIMESTAMP_DATE; >> >> === modified file 'sql/item_timefunc.cc' >> --- sql/item_timefunc.cc 2013-03-20 15:13:00 +0000 >> +++ sql/item_timefunc.cc 2013-06-11 10:57:09 +0000 >> @@ -1923,8 +1921,10 @@ void Item_date_add_interval::fix_length_ >> bool Item_date_add_interval::get_date(MYSQL_TIME *ltime, uint fuzzy_date) >> { >> INTERVAL interval; >> - >> - if (args[0]->get_date(ltime, TIME_NO_ZERO_DATE | TIME_FUZZY_DATE | TIME_NO_ZERO_IN_DATE) || >> + if (args[0]->get_date(ltime, >> + TIME_NO_ZERO_DATE | TIME_FUZZY_DATE | TIME_NO_ZERO_IN_DATE | >> + ((args[0]->field_type() == MYSQL_TYPE_TIME) ? >> + TIME_TIME_ONLY : 0)) || > > why is that? > >> get_interval_value(args[1], int_type, &value, &interval)) >> return (null_value=1); >> >> @@ -2290,6 +2290,12 @@ bool Item_time_typecast::get_date(MYSQL_ >> return 1; >> if (decimals < TIME_SECOND_PART_DIGITS) >> ltime->second_part= sec_part_truncate(ltime->second_part, decimals); >> + if (!(fuzzy_date & TIME_TIME_ONLY)) >> + { >> + if (ltime->time_type == MYSQL_TIMESTAMP_TIME) >> + ltime->time_type= MYSQL_TIMESTAMP_DATETIME; > > why? > >> + return ((null_value= check_date_with_warn(ltime, fuzzy_date))); >> + } >> /* >> MYSQL_TIMESTAMP_TIME value can have non-zero day part, >> which we should not lose. >> === modified file 'sql/item_timefunc.h' >> --- sql/item_timefunc.h 2012-12-28 12:41:46 +0000 >> +++ sql/item_timefunc.h 2013-06-11 11:19:03 +0000 >> @@ -492,6 +492,8 @@ class Item_timefunc :public Item_tempora >> Item_timefunc(Item *a,Item *b) :Item_temporal_func(a,b) {} >> Item_timefunc(Item *a, Item *b, Item *c) :Item_temporal_func(a, b ,c) {} >> enum_field_types field_type() const { return MYSQL_TYPE_TIME; } >> + int save_in_field(Field *field, bool no_conversions) >> + { return save_time_in_field(field); } > > why? > >> }; >> >> === modified file 'sql/time.cc' >> --- sql/time.cc 2013-01-08 20:23:03 +0000 >> +++ sql/time.cc 2013-06-11 10:51:31 +0000 >> @@ -213,6 +213,20 @@ ulong convert_month_to_period(ulong mont >> } >> >> >> +bool >> +check_date_with_warn(const MYSQL_TIME *ltime, uint fuzzy_date) >> +{ >> + int dummy_warnings; >> + if (check_date(ltime, fuzzy_date, &dummy_warnings)) >> + { >> + Lazy_string_time str(ltime); >> + make_truncated_value_warning(current_thd, MYSQL_ERROR::WARN_LEVEL_WARN, >> + &str, MYSQL_TIMESTAMP_ERROR, 0); > > please, use this function also in Item_date_typecast::get_date() and > may be in other places. you might need to pass MYSQL_TIMESTAMP_xxx > value as an argument, though. > >> + return true; >> + } >> + return false; >> +} >> + >> /* >> Convert a timestamp string to a MYSQL_TIME value and produce a warning >> if string was truncated during conversion. >> > > Regards, > Sergei >

1 0

[Maria-developers] I_S plugin doubts
by Roberto Spadim 13 Jun '13

13 Jun '13

Hi i was reading locale_info and qc_info source that i sent in jira changing the last parameter from 0 to SKIP_OPEN_TABLE, change the table from aria engine to memory engine, why? this change something internally in I_S table ? example: from static ST_FIELD_INFO qc_info_fields_queries[]= { {"ENTRY_POSITION_IN_CACHE", MY_INT32_NUM_DECIMAL_DIGITS, MYSQL_TYPE_LONG, 0, 0, 0, 0}, ... to static ST_FIELD_INFO qc_info_fields_queries[]= { {"ENTRY_POSITION_IN_CACHE", MY_INT32_NUM_DECIMAL_DIGITS, MYSQL_TYPE_LONG, 0, 0, 0, SKIP_OPEN_TABLE}, ... -- Roberto Spadim

2 5

Re: [Maria-developers] MDEV-4581 , more query cache information fields
by Roberto Spadim 12 Jun '13

12 Jun '13

hi sergei, i sent the patch, could you check it? i think that's easy to read and understand maybe some code standard should be done since i don't know how code in mariadb cs https://mariadb.atlassian.net/browse/MDEV-4581

1 0

[Maria-developers] MDEV-4581 , more query cache information fields
by Roberto Spadim 12 Jun '13

12 Jun '13

Hi guys, i sent a patch for qc_info plugin in MDEV-4581 Could mariadb team review and tell me what i should do, to put it in the next release? i will rewrite it, at 10.0.3 release, and change some things: the add to query to cache, i will change to only a function call at sql_cache.cc, and put the function at sql_cache.h something like: qc->qc_info_set_informations(insert time, rows_sent, rows_read, etc etc etc) rewrite the part of query hit. to execute only a command too qc->qc_info_hit(current_time, expend_qc_time, etc etc etc) this will make sql_cache.cc code cleaner and sql_cache.h "dirty" but will be more easly to read and patch future versions. only add 2 lines to sql_cache.cc , and many lines to sql_cache.h since sql_cache.h have the structure it will be easly to write function call at struct than sql_cache.cc file bye guys -- Roberto Spadim

1 0

Re: [Maria-developers] audit plugin help with MDEV-575 (fail2ban like)
by Roberto Spadim 12 Jun '13

12 Jun '13

well i'm thinking about again, maybe just a syslog could do the job with fail2ban but in windows... well it will not work maybe the second option of a internal table could do the job too Host | bad_login_datetime if host have more than 'bad_login_tries' bad logins in 'bad_login_seconds' seconds before now it will be disconnect after xxx seconds or after each minute, or after each disconnect clean the internal table that have bad_login_datetime < now() - 'bad_login_seconds' the clean should be optimized in a not paintfull way (maybe like a garbage colector that runs after X disconnects, or a event that happens each x seconds, or both of them, i prefer the disconnect method) any other idea?

1 2

Re: [Maria-developers] [Commits] Rev 3551: MDEV-4612 SQ pushdown: Server crashes in make_join_statistics with materialization+semijoin, IN subqueries, constant table, impossible condition in file:///home/tsk/mprog/src/10.0-md83/
by Oleksandr Byelkin 11 Jun '13

11 Jun '13

Hi! 11.06.2013 11:11, timour(a)askmonty.org пишет: > At file:///home/tsk/mprog/src/10.0-md83/ > > ------------------------------------------------------------ > revno: 3551 > revision-id: timour(a)askmonty.org-20130611081135-rge7p00h8e9nhypj > parent: timour(a)askmonty.org-20130528122232-kisslvdw5vhrna0q > fixes bug: https://mariadb.atlassian.net/browse/MDEV-4612 > committer: timour(a)askmonty.org > branch nick: 10.0-md83 > timestamp: Tue 2013-06-11 11:11:35 +0300 > message: > MDEV-4612 SQ pushdown: Server crashes in make_join_statistics with materialization+semijoin, IN subqueries, constant table, impossible condition > > Analysis: > The subquery materialization execution method subselect_hash_sj_engine::exec() > didn't restore thd->lex->current_select through all return paths. This was > not a problem when the method is called during execution. When a subquery > was executed during the optimization phase of the outer query, current_select > remained to point to the wrong subquery. This resulted in update_ref_and_keys() > getting the wrong number of conditions via thd->lex->current_select->cond_count, > thus allocating a smaller buffer for sargable conditions. This later resulted > in memory overwrite. > > Solution: > Make sure that subselect_hash_sj_engine::exec() always restores current_select. The patch looks OK to push. [skip]

1 0

[Maria-developers] Group commit comparison between mysql and mariadb
by Sergei Petrunia 11 Jun '13

11 Jun '13

Hi Kristian, I'm wondering, do we have any comparison of MariaDB's binlog group commit implementation vs MySQL's binlog's group commit? I recall Mats writing a blog about the differences last year, but he didn't have the code back then. I'm wondering if Oracle had implemented what he described, or did something else... I understand that in order to do benchmark we need to finish merging InnoDB from MySQL 5.6 into MariaDB 10.0. One can't benchmark group commit per se, it is tied to innodb. Do we still have any clue, before the merge happens? Btw, what should the merge do - take group commit from MariaDB or from MySQL? BR Sergei -- Sergei Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

4 3

Re: [Maria-developers] Buildbot question
by Kristian Nielsen 11 Jun '13

11 Jun '13

Sergey Vojtovich <svoj(a)mariadb.org> writes: > I'm observing pretty odd buildbot behavior with some of my pushes (probably > merges). Bar and Daniel were unable to answer my questions and suggested to > ask you instead. The problem seems to be that Buildbot has become confused about revisions. Look at the top of the page you linked, at the revisions for each column. You will notice that they are not in order. The 3rd column is the highest revision, which is probably why spider suite is enabled there (I assume you pushed something to add the suite). I see your current top revision in the tree is 3760, while the Buildbot page has a newer revision 3767. It also looks like the Buildbot history contains _another_ earlier revision that was also number 3760, maybe that is the cause of the confusion (Buildbot perhaps thinks it is rebuilding the old 3760). You can see your revision sorted as the first columns in this link: http://buildbot.askmonty.org/buildbot/grid?branch=10.0-spider&width=20 This confusion usually happens because the tree was overwritten and revisions became renumbered. This can happen due to bzr push --overwrite, but in your case it is probably because you merged in the opposite direction. When merging a tree A with a tree B and pushing to B, it is generally best to merge A into B (cd to B and bzr merge A). This keeps the existing revision numbers of B intact. In the main trees we have protection that enforces this, but not in feature trees. (I would be nice if we could fix Buildbot to prevent this confusion, but so far noone has taken it upon themselves. At one point I asked the Launchpad/bzr developers to make the revision id (as opposed to revision numbers) available so that Buildbot could avoid using revision numbers completely, but the answer was that "hiding revision ids was a feature" ...). Usually the confusion is sorted out by itself on a following push, or you can use the trick above adding width=N to the link to see the column of the revision you want. Hope this helps, - Kristian.

2 1

[Maria-developers] MariaDB 10.0.3 release prep
by Daniel Bartholomew 11 Jun '13

11 Jun '13

All, I've begun the prep work for the MariaDB 10.0.3 alpha release. The packages are uploaded and the mirrors should pick them up soon (if they haven't already). The draft changelog and release notes are here: - https://kb.askmonty.org/en/mariadb-1003-release-notes/ - https://kb.askmonty.org/en/mariadb-1003-changelog/ Please let me know ASAP if there's anything that needs to be mentioned/highlighted/removed/etc... Assuming all goes well, the plan is to activate the release tomorrow (11 Jun 2013) at around 10am US Eastern Time. Thanks. -- Daniel Bartholomew Google+ - http://gplus.to/dbart Twitter - http://twitter.com/daniel_bart MariaDB: An Enhanced Drop-in Replacement for MySQL Website - http://mariadb.org Twitter - http://twitter.com/mariadb Google+ - http://google.com/+mariadb Facebook - http://fb.com/MariaDB.dbms Knowledgebase - http://kb.askmonty.org

1 0

[Maria-developers] audit plugin help with MDEV-575 (fail2ban like)
by Roberto Spadim 09 Jun '13

09 Jun '13

hi guys i want get users / ip that try to connect to server with bad password/user and retry very frequently (normally a brute force attack) some things i need: 1)get information about connection when it fail or not (maybe an audit plugin could do this?), information that i need: 1.1)protocol (tcp/unix/others) 1.2)user 1.3)remote ip (when tcp only) it must be executed in every login 2) log this information maybe we could use CSV,aria, myisam, to log bad logins and put it in a log table: PROTOCOL, USER, REMOTE IP, DATATIME 3) add columns to user BAD_LOGIN_MAX_TRIES, BAD_LOGIN_LOG_TIME, BAD_LOGIN_MIN_RETRY_TIME 4) add server global bad_login_tcp_log_time, bad_login_tcp_max_errors 5) implement checks before connection: when openned a connection to server via TCP execute something like: SELECT COUNT(*) FROM bad_login_log WHERE remote_ip="remote_ip" AND datetime>NOW() - @@bad_login_tcp_log_time if COUNT(*) > @@bad_login_tcp_max_erros disconnect 6) implement user checks: SELECT COUNT(*),(SELECT BAD_LOGIN_MAX_TRIES FROM users WHERE user="user") FROM bad_login_log WHERE user="user" AND datetime>NOW() - (SELECT BAD_LOGIN_LOG_TIME FROM users WHERE user="user") if count(*) > bad_login_max_tries form that user disconnect that's all -- Roberto Spadim SPAEmpresarial

1 0

[Maria-developers] Help with MDEV-4419, ENUM optimization
by Roberto Spadim 09 Jun '13

09 Jun '13

Hi guys, i want to optimize fields with ENUM() in item comparation the point is: SELECT COUNT(*) FROM table WHERE enum_field!='x' column 'enum_field' is ENUM('x,'y','z') NOT NULL DEFAULT 'x' i want to rewrite the query in a way that we can execute it with index: SELECT COUNT(*)FROM table WHERE enum_field IN ('y','z','') the "" is not allowed values ('a' for example) the feature is 'easy' to understand but i don't know where to implement it will read possible values of enum fields, and rewrite "!=","<","<=",">=",">" operators to IN operator with fields that match the condition of the primary operator some think like: column "operator" const value if(operator in ('!=','<','<=','>=','>' AND column is ENUM){ get all possible values from column copy all values to a return variable execute the "operator" in all values with const values if operator return false remove the value from the return variable change the "operator" and "const value" to : "IN" and return variable comma separated }else{ no optimization leave operation as it is } example: column = enum('x','y','z'), operator = ">" const value = "3" if(column is ENUM and operator = ">"){ possible values => "x","y","z","" (check that "" is used when insert into, value = "a" for example) return variable = "x","y","z", "" for each value.... "x" > "3" ? yes, ASCII 120 > ASCII 51 -> do nothing "y" > "3" ? yes, ASCII 121 > ASCII 51 -> do nothing "z" > "3" ? yes, ASCII 122 > ASCII 51 -> do nothing "" > "3" ? no, "" < ASCII 51 -> remove the value "" from return variable end for now the return variable is : "x","y","z" the "" was removed rewrite the operator to: "column IN ('x','y','z')" } ----------------------- this will optimize a lot: using != in a big table (446577 rows) result = Using where, 446577 rows, type= ALL using IN() in a big table: result = Using where, 28 rows, key = index, type = range in other words, a lot of optimization with ENUM fields, since they are a pseudo "index" of what we can found in the column =) thank guys any help is wellcome to make a patch -- Roberto Spadim SPAEmpresarial

1 0

[Maria-developers] GSoC MariaDB Kerberos Authentication Plugin Proposal
by QIU Shuang 06 Jun '13

06 Jun '13

Hi Sergei, Thank you and the community for accepting my proposal of this year's GSoC: Kerberize MariaDB -- add kerberos authentication support to MariaDB. I'm Shuang Qiu, a second year graduate student from Shanghai Jiao Tong University, Shanghai, China. My field of research is distributed computing, especially distributed storage. I just get in touch with my project mentor Mr. Vaintroub and join the #maria IRC channel at freenode.net, where I can be found as shuang. Thank you again and hope we can have a happy and productive summer! Cheers! Sincerely, Shuang

1 0

Re: [Maria-developers] [Maria-discuss] Clear-text plugin
by Peter Laursen 06 Jun '13

06 Jun '13

"no support for dlopen()" . How does this apply to Windows clients? We always build first on Windows and once we have no errors of any kind we build on Linux if the application exists for that platform (what MONyog does and SQLyog does not) and fix what issues may occur there. This issue has occurred with SQLyog (and as such for a Windows client). I am not coder/developer myself, but on Windows I think we have no other option than statically linking? We could of could compile the API as a .dll, but we don't (because we had a lot of problems with it earlier). -- Peter On Thu, Jun 6, 2013 at 3:25 PM, Sergei Golubchik <serg(a)mariadb.org> wrote: > Hi, Peter! > > On Jun 04, Peter Laursen wrote: > > We have the same problem. We cannot test either. But we have a few user > > reports. We will discuss what we can do. > > > > Is there any parameter or compile-time option we should be aware of? Any > > specific documentation somewhere? Anything? > > Not really. If the cleartext plugin is built and installed, it'll be > used when the server requests it. > > I've just checked that this plugin is present both in rpm and deb > packages. I didn't check binary tarballs, but it should be there too. > > So, there's nothing to configure at run-time or compile-time, everything > should work automatically and transparently. > > Hmm, unless you build a completely static client library with no support > for dlopen(). > > Regards, > Sergei >

2 1

Re: [Maria-developers] [Maria-discuss] Clear-text plugin
by Peter Laursen 04 Jun '13

04 Jun '13

We have the same problem. We cannot test either. But we have a few user reports. We will discuss what we can do. Is there any parameter or compile-time option we should be aware of? Any specific documentation somewhere? Anything? -- Peter On Tue, Jun 4, 2013 at 2:19 PM, Sergei Golubchik <serg(a)mariadb.org> wrote: > Hi, Peter! > > On Jun 04, Peter Laursen wrote: > > For clarity: a 'dummy' parameter doing nothing does not solve our > > problem. We actually want to build clients that can (also) connect to > > Oracle MySQL Enterprise servers configured to make use of Oracle's PAM > > implementation. > > We build and distribute a cleartext plugin. And it can be used if the > server requests. In a sense, in MariaDB a cleartext plugin is always > enabled. So a dummy option is needed only to restore the binary protocol > compatibility. That's my understanding, at least, but I don't have > Oracle's PAM plugin, so I cannot test it. > > But if you have verified that your client with our C connector can not > connect to Oracle MySQL Enterprise server with the Oracle's PAM > implementation - please submit a bug report, and we will fix it. > > Regards, > Sergei > >

1 0

Re: [Maria-developers] [Maria-discuss] Clear-text plugin
by Peter Laursen 04 Jun '13

04 Jun '13

For clarity: a 'dummy' parameter doing nothing does not solve our problem. We actually want to build clients that can (also) connect to Oracle MySQL Enterprise servers configured to make use of Oracle's PAM implementation. -- Peter On Tue, Jun 4, 2013 at 11:05 AM, Peter Laursen <peter_laursen(a)webyog.com>wrote: > yes .. in the Connector/C. We use C/C++ for both SQLyog and MONyog. > > -- Peter > > > On Tue, Jun 4, 2013 at 10:17 AM, Sergei Golubchik <serg(a)mariadb.org>wrote: > >> Hi, Peter! >> >> On Jun 04, Peter Laursen wrote: >> > This is a continuation of >> > https://mariadb.atlassian.net/browse/CONC-25. We requested support for >> > the clear-text plugin (or a workaround of any kind) that would allow a >> > client compiled with the MariaDB connector to use PAM authentication >> > with MySQL Enterprise servers from Oracle. >> >> Another related issue is https://mariadb.atlassian.net/browse/MDEV-4574 >> which should be fixed in 5.5.32, next 5.5 release. >> >> But you, probably, want it in Connector/C, don't you? >> >> Regards, >> Sergei >> >> >

1 0

[Maria-developers] Clear-text plugin
by Peter Laursen 04 Jun '13

04 Jun '13

This is a continuation of https://mariadb.atlassian.net/browse/CONC-25. We requested support for the clear-text plugin (or a workaround of any kind) that would allow a client compiled with the MariaDB connector to use PAM authentication with MySQL Enterprise servers from Oracle. I will not dispute Sergei's opinions about Oracle's implementation. But I believe it misses the point. The question is if the MariaDB connector should be a replacement for the C-IPA from Oracle. And the question is nothing else than that. Not the quality of solutions bundled with Enterprise servers from Oracle. I would like to hear opinions not only from 'core developers' like Sergei, but also from wider circles in MariaDB organization (management, marketing) as well as the user community. For us the problem is like this: We introduced support for the clear-text plugin in SQLyog 11.1. In 11.12 beta2 we replaced Oracles connector with MariaDB (and had to removed it again). We had a few crashes and memory leaks with mariaDB Connector but that is getting in control now (ther were bugs in both the API and in oru code). We probably could patch the MariaDB code without much problems to support the full-text plugin - but we don't have a MySQL Enterprise server to test with and we cannot release without proper testing. We have already started getting complaints from users/customers about that what we introduced in 11.1 has been removed in current development tree. It seems that we have quite a lot of customers using Oracle's Enterprise servers wiht Oracle's PAM authentication. A solution/workaround would very much be appreciated. Regards Peter Laursen Webyog

1 0

Re: [Maria-developers] MDEV-4568 - Port Percona response time distribution as audit plugin
by Sergey Vojtovich 03 Jun '13

03 Jun '13

Hi Sergei, 03.06.2013, в 16:36, Sergei Golubchik <serg(a)mariadb.org> написал(а): > Hi, Sergey! > > On May 31, Sergey Vojtovich wrote: >> revno: 3753 >> revision-id: svoj(a)mariadb.org-20130531131833-zs83x5wntvl1i15f >> parent: wlad(a)montyprogram.com-20130523125248-22m56m01r24t0dmb >> committer: Sergey Vojtovich <svoj(a)mariadb.org> >> branch nick: maria-qrtp >> timestamp: Fri 2013-05-31 17:18:33 +0400 >> message: >> MDEV-4568 - Port Percona response time distribution as audit plugin >> >> sql/sp_head.cc: >> Trigger MYSQL_AUDIT_GENERAL_STATUS event for each statement in stored >> procedures. Not strictly needed for query_response_time plugin, but >> makes it behave more like Percona patch. > > I'm sorry, I couldn't quite understand what you have changed, as > compared to the first patch (and the first was not in emails, so I > couldn't compare them). The difference is rather minor: - renamed query_response_time_exec_time to query_response_time_exec_time_debug - updated plugin author and added GPL header to plugin.cc - fixed I_S plugin description What I didn't touch were query_response_time.cc and .h, but below you agree that we shouldn't touch them. > Looks pretty much the same as the previous one, although you need to run > the tests and to see that they pass (you've changed plugin description > in the code, but the test result wasn't changed accordingly). Hmm… you're right, somehow I forgot to update basic test. Other tests are up to date. > Otherwise, looks ok to push, I suppose we can let it be as close to > Percona's version as possible. I don't think you need to rewrite it, > it's in a plugin now anyway. > > Regards, > Sergei Thanks, Sergey

1 0

[Maria-developers] Partition prune and lock
by Roberto Spadim 02 Jun '13

02 Jun '13

Hi guys i'm studing mariadb code and reading user manuals to know more about features. My last study was about query cache with many ideas in some MDEV and some patchs sent Now i'm reading about partition My questions are 1)when a update or a delete is done on a expecific partition like: Update table set field=2 where partition=1 How lock is done? Lock is done at table level or partition level? 2)Today we have vertical partition, could we implement 'virtual' horizontal partition update and delete in some group of fields (horizontal partition) only lock the horizontal partition being changed and used in where clausule? Any idea and problems on how to develop it? 3) can query cache be partition pruned? instead of removing entire table, just remove the partition that was changed It will consume more memory but will do a better work in cache Maybe a new key like SQL_CACHE_PARTITON or a new variable, query_cache_min_unit=PARTITION or TABLE could change how the query will be cached and pruned Thanks guys

1 1

Re: [Maria-developers] [codership-team] Hanging Galera Cluster, looking for the cause
by Jan Kirchhoff 01 Jun '13

01 Jun '13

I have seen exactly the same issue again, second day in a row. This time at 7:30 in the morning and again, no logs or anything that would give me an idea of what could have caused this. "killall -9 mysqld" and "service mysql start" on the hanging node is the only solution I have so far... I am crossposting this to maria-developers, maybe somebody on that list has an idea where to start? Jan Am 31.05.2013 10:29, schrieb Jan Kirchhoff: > Hi, > > we have been testing MariaDB 5.5.29 with Galera > (wsrep_provider_version 23.2.4(r147)) for a few months now and have had > a freeze of the cluster for the second time now. I didn't report the > first hang about 6 weeks ago as I wanted to be sure it wasn't our fault, > but I can't find any type of network issue or such in our monitoring > system and nothing in the logs that could be related to this problem: > > We have 2 database servers running Ubuntu 12.10 with Kernel 3.5.0 > (names: db8, db9), with a bunch of 15k-disks, some SSDs, 128GB RAM and a > test database of about 600-700GB. The amount of binlogs written every > day is around 500GB, so you get an idea of the workload. > db8 is a slave (traditional replication) of our main database. We have a > garbd running on another server as a third galera instance. > > Everything runs smooth for some weeks, but since this morning at 3:50am > db9 seems to be frozen. The server itself looks good, mysqld is taking > about 50% of one CPU core, everything else looks normal. Nothing in the > logs. Connecting to mysql works, but any query (like "show status" or > normal selects) hang forever. Only "show engine innodb status" returns. > There are almost 3000 processes that piled up like this: > > ---TRANSACTION 2F89D7CB, not started > MySQL thread id 113575, OS thread handle 0x7fab31821700, query id > 45249946 cl4.logical-line.de 192.168.0.16 stats_pflege sleeping > > but none of them ever gets any further. I can issue a "kill 113575" and > the prompt returns, but nothing happens. > > This is what db8's global status says: > wsrep_local_state_uuid 41d7259d-7200-11e2-0800-943dd0e2f659 > wsrep_protocol_version 4 > wsrep_last_committed 179518511 > wsrep_replicated 53590833 > wsrep_replicated_bytes 14768686288759 > wsrep_received 894995 > wsrep_received_bytes 7161712 > wsrep_local_commits 53583808 > wsrep_local_cert_failures 60 > wsrep_local_bf_aborts 0 > wsrep_local_replays 0 > wsrep_local_send_queue 8 > wsrep_local_send_queue_avg 0.000000 > wsrep_local_recv_queue 0 > wsrep_local_recv_queue_avg 0.000000 > wsrep_flow_control_paused 1.000000 > wsrep_flow_control_sent 0 > wsrep_flow_control_recv 0 > wsrep_cert_deps_distance 73.229730 > wsrep_apply_oooe 0.000000 > wsrep_apply_oool 0.000000 > wsrep_apply_window 0.000000 > wsrep_commit_oooe 0.000000 > wsrep_commit_oool 0.000000 > wsrep_commit_window 0.000000 > wsrep_local_state 4 > wsrep_local_state_comment Synced > wsrep_cert_index_size 25606 > wsrep_causal_reads 0 > wsrep_incoming_addresses 192.168.10.30:3306,192.168.10.31:3306, > wsrep_cluster_conf_id 15 > wsrep_cluster_size 3 > wsrep_cluster_state_uuid 41d7259d-7200-11e2-0800-943dd0e2f659 > wsrep_cluster_status Primary > wsrep_connected ON > wsrep_local_index 0 > wsrep_provider_name Galera > wsrep_provider_vendor Codership Oy <info(a)codership.com> > wsrep_provider_version 23.2.4(r147) > wsrep_ready ON > > > > > Here is the innodb status from db9 (the one that hangs). This is the > only output I could get out of db9 at all: > > Type Name Status > InnoDB > ===================================== > 130531 10:02:08 INNODB MONITOR OUTPUT > ===================================== > Per second averages calculated from the last 18 seconds > ----------------- > BACKGROUND THREAD > ----------------- > srv_master_thread loops: 907033 1_second, 906789 sleeps, 89831 > 10_second, 22491 background, 22489 flush > srv_master_thread log flush and writes: 478222 > ---------- > SEMAPHORES > ---------- > OS WAIT ARRAY INFO: reservation count 17017782, signal count 287266881 > Mutex spin waits 98936299, rounds 91970366, OS waits 868715 > RW-shared spins 139527705, rounds 455067309, OS waits 6488041 > RW-excl spins 58122849, rounds 842712424, OS waits 8565312 > Spin rounds per wait: 0.93 mutex, 3.26 RW-shared, 14.50 RW-excl > -------- > FILE I/O > -------- > I/O thread 0 state: waiting for completed aio requests (insert buffer > thread) > I/O thread 1 state: waiting for completed aio requests (log thread) > I/O thread 2 state: waiting for completed aio requests (read thread) > I/O thread 3 state: waiting for completed aio requests (read thread) > I/O thread 4 state: waiting for completed aio requests (read thread) > I/O thread 5 state: waiting for completed aio requests (read thread) > I/O thread 6 state: waiting for completed aio requests (write thread) > I/O thread 7 state: waiting for completed aio requests (write thread) > I/O thread 8 state: waiting for completed aio requests (write thread) > I/O thread 9 state: waiting for completed aio requests (write thread) > Pending normal aio reads: 0 [0, 0, 0, 0] , aio writes: 0 [0, 0, 0, 0] , > ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0 > Pending flushes (fsync) log: 0; buffer pool: 0 > 49405970 OS file reads, 277420410 OS file writes, 6653801 OS fsyncs > 1 pending preads, 0 pending pwrites > 49.44 reads/s, 16384 avg bytes/read, 0.61 writes/s, 0.61 fsyncs/s > ------------------------------------- > INSERT BUFFER AND ADAPTIVE HASH INDEX > ------------------------------------- > Ibuf: size 1, free list len 698, seg size 700, 300226 merges > merged operations: > insert 410879, delete mark 8032461, delete 82075 > discarded operations: > insert 0, delete mark 0, delete 0 > Hash table size 212497717, node heap has 450852 buffer(s) > 16846.79 hash searches/s, 77045.28 non-hash searches/s > --- > LOG > --- > Log sequence number 18522888544071 > Log flushed up to 18522888544071 > Last checkpoint at 18522888544051 > Max checkpoint age 2035833201 > Checkpoint age target 1972213414 > Modified age 20 > Checkpoint age 20 > 0 pending log writes, 0 pending chkp writes > 6895012 log i/o's done, 0.39 log i/o's/second > ---------------------- > BUFFER POOL AND MEMORY > ---------------------- > Total memory allocated 110310195200; in additional pool allocated 0 > Internal hash tables (constant factor + variable factor) > Adaptive hash index 9086745120 \t(1699981736 + 7386763384) > Page hash 106249768 (buffer pool 0 only) > Dictionary cache 426512815 \t(424997488 + 1515327) > File system 429392 \t(82672 + 346720) > Lock system 296805680 \t(295674296 + 1131384) > Recovery system 0 \t(0 + 0) > Dictionary memory allocated 1515327 > Buffer pool size 6553599 > Buffer pool size, bytes 107374166016 > Free buffers 0 > Database pages 6102747 > Old database pages 2252752 > Modified db pages 1 > Pending reads 1 > Pending writes: LRU 0, flush list 0, single page 0 > Pages made young 48135625, not young 0 > 64.83 youngs/s, 0.00 non-youngs/s > Pages read 49405737, created 7213276, written 266046626 > 49.44 reads/s, 0.00 creates/s, 0.11 writes/s > Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000 > Pages read ahead 0.00/s, evicted without access 40.11/s, Random read > ahead 0.00/s > LRU len: 6102747, unzip_LRU len: 0 > I/O sum[2662]:cur[22], unzip sum[0]:cur[0] > -------------- > ROW OPERATIONS > -------------- > 1 queries inside InnoDB, 0 queries in queue > 2 read views open inside InnoDB > ---OLDEST VIEW--- > Normal read view > Read view low limit trx n:o 2F754555 > Read view up limit trx id 2F754555 > Read view low limit trx id 2F754555 > Read view individually stored trx ids: > ----------------- > Main thread process no. 15888, id 140381271201536, state: sleeping > Number of rows inserted 46059494, updated 2501543763, deleted 28801859, > read 17046037029 > 0.00 inserts/s, 0.00 updates/s, 0.00 deletes/s, 48965.50 reads/s > ------------ > TRANSACTIONS > ------------ > Trx id counter 2F89DD34 > Purge done for trx's n:o < 2F754554 undo n:o < 0 > History list length 140 > LIST OF TRANSACTIONS FOR EACH SESSION: > ---TRANSACTION 0, not started > MySQL thread id 113586, OS thread handle 0x7fab314fe700, query id > 45250432 localhost root sleeping > SHOW engine INNODB STATUS > ---TRANSACTION 2F89DC7B, not started > MySQL thread id 113583, OS thread handle 0x7fab315d9700, query id > 45250367 cl4.logical-line.de 192.168.0.16 stats_pflege sleeping > ---TRANSACTION 2F89DB4F, not started > MySQL thread id 113582, OS thread handle 0x7fab31622700, query id > 45250261 cl4.logical-line.de 192.168.0.16 stats_pflege sleeping > ---TRANSACTION 2F89DA23, not started > MySQL thread id 113578, OS thread handle 0x7fab31746700, query id > 45250159 cl4.logical-line.de 192.168.0.16 stats_pflege sleeping > ---TRANSACTION 0, not started > <many many rows like these, around 3000 although max_connections is 300?! > > > As soon as I trigger the restart of mysql on db9 (service mysql > restart), I see normal log entries like this: > May 31 10:16:59 db9 mysqld: 130531 10:16:59 [Note] WSREP: Stop replication > May 31 10:16:59 db9 mysqld: 130531 10:16:59 [Note] WSREP: Closing send > monitor... > May 31 10:16:59 db9 mysqld: 130531 10:16:59 [Note] WSREP: Closed send > monitor. > May 31 10:16:59 db9 mysqld: 130531 10:16:59 [Note] WSREP: gcomm: > terminating thread > > and db8 is accepting inserts again and catches up with replication. > > meanwhile, db9 says: > May 31 10:17:02 db9 mysqld: 130531 10:17:02 [Note] WSREP: killing local > connection: 113529 > May 31 10:17:02 db9 mysqld: 130531 10:17:02 [Note] WSREP: killing local > connection: 113199 > May 31 10:17:02 db9 mysqld: 130531 10:17:02 [Note] WSREP: killing local > connection: 110221 > > and nothing happens any more. No CPU and no IO load coming from the > mysqld processes. > After 5 minutes I killed the mysqld with "killall -9 mysqld" and then > started it again the normal way. > db9 syncs via IST and everything is fine again. But I have no idea what > caused all this. According to a colleague killing mysql on db9 was the > thing that helped 6 weeks ago, too. > > Thanks for any pointers where we should look further. > > Jan Kirchhoff >

1 0

[Maria-developers] Open Questions in the knowledgebase
by Ian Gilfillan 31 May '13

31 May '13

I've been cleaning up the questions on the knowledgebase, but there are still a few open ones that need a response. If anyone can help, please pay a visit. Here are the ones still needing feedback: Multiple error logs: https://kb.askmonty.org/en/multiple-err-logs/ Query optimizer: https://kb.askmonty.org/en/does-the-query-optimize-a-low-cardinality-index-… Strange SSL messages: https://kb.askmonty.org/en/strange-ssl-messages-in-error-log/ Multi-source replication and time zones: https://kb.askmonty.org/en/time-zones-multi-source-replication/ And of course, the old faithful, when will it be released (10.x as stable) - https://kb.askmonty.org/en/release-date-for-10x-stable-version/ Also, a comment asking for specifics on JSON implementation going forward: https://kb.askmonty.org/en/column_json/ thanks, ian

1 0

[Maria-developers] a bug also in mariadb
by xiaobing jiang 30 May '13

30 May '13

hi all: I submit a bug: http://bugs.mysql.com/bug.php?id=69352 . I review the mariadb code, I think it is also in mariadb. I have read the page. https://kb.askmonty.org/en/reporting-bugs/ should I report this bug to mariadb? Thanks!

1 0

[Maria-developers] GSoC 2013 - Interested Student - Roles Project
by Vicentiu Ciorbaru 28 May '13

28 May '13

Hello, My name is Vicențiu Ciorbaru. I am 3'rd year undergraduate student at the Polytechnic University of Bucharest, Romania and I am very interested in contributing to MariaDB as part of Google Summer of Code. I have not used MariaDB previously but I have taken it for a spin just now. I have participated in GSoC once before and would like to do so again. I have contributed to other open source organisations (Mozilla, Samba, ROSEdu) and I submit patches when time permits. The project that interests me the most is MDEV-4397: Roles. I want to ask you what would be a good place to start in the codebase to help me get an idea what I need to do for this project? Looking forward to hearing from you, Vicențiu

2 7

Re: [Maria-developers] GTID work priorities
by Kristian Nielsen 28 May '13

28 May '13

Pavel Ivanov <pivanof(a)google.com> writes: > we would really love to see "strict gtid mode" and related stuff next. I've now pushed strict mode to 10.0-base, and documented it in the knowledgebase. However, one thing was not clear from your description in MDEV-4478: > 4) Probably with a separate flag we want to prohibit any writes to binlog on > the slave except from the slave thread(s), i.e. whenever MASTER_HOST is > configured server works as if sql_log_bin = 0 globally except for the slave > thread(s) which work as if sql_log_bin = 1 and SELECT @@global.sql_log_bin > still returns 1. Can you elaborate exactly what semantics you have in mind for this new option? And why a new option is needed/how it differs from the already existing read-only and sql-log-bin options? It sounds to me what you want is simply to be able to start the server with --sql-log-bin=0 and still have the slave sql thread log its events to the binlog (if --log-slave-updates=1)? I would expect that to work already, but maybe there is a bug (that we could fix), or maybe you had something else in mind? - Kristian.

2 1

[Maria-developers] Query cache information plugin - MDEV-4516
by Roberto Spadim 28 May '13

28 May '13

Hi guys i'm reading query cache code, and i have some doubts... mariadb-10.0.1 source: 1)PROTOCOL_LOCAL: at line: 1412 (Query_cache::store_query) we have this comment: /* PROTOCOL_LOCAL results are not cached. */ what's PROTOCOL_LOCAL ? it's the unix socket? windows pipe? embedded? memcache? mysql? another thing like this? or it's the internal mariadb source running queries? 2)the query_size=0 (MDEV-4516) i was reading at line 1680 if (is_disabled() || thd->locked_tables_mode || thd->variables.query_cache_type == 0) goto err; and line : 1694 DBUG_ASSERT(query_cache_size != 0); // otherwise cache would be disabled why it's isn't used in query_cache_information plugin? *check that in line 1680 it test the "query_cache_type" too, * *in MDEV-4516 report about size=0 we didn't tested this case!!!* 3) i was at line 1886: DBUG_PRINT("qcache", ("\ long %d, 4.1: %d, bin_proto: %d, more results %d, pkt_nr: %d, \ CS client: %u, CS result: %u, CS conn: %u, limit: %lu, TZ: 0x%lx, \ sql mode: 0x%llx, sort len: %lu, conncat len: %lu, div_precision: %lu, \ def_week_frmt: %lu, in_trans: %d, autocommit: %d", is this the "primary key" of the query cache!?!?!?!?!?!?! could we use a sha1 or another "primary key" method (index [] of the query in cache + '';'+ 5 bytes of a hash key to compare if the index still with the same query) to easly show query entry "number", and allow a "delete from query_cache_information where key=xxx" function? at line 1938 we have this maybe the query position on cache ("primary key of query cache")?: DBUG_PRINT("qcache", ("Query have result 0x%lx", (ulong) query)); 4) there's some more ideas about query cache information plugin at MDEV-4571, if anyone could read it =) well that's all =] -- Roberto Spadim

2 19

[Maria-developers] GTID replication and relay_log.info
by Pavel Ivanov 28 May '13

28 May '13

Kristian, There's a well-known problem in MySQL that relay_log.info is not always in sync with the database state, and killing and restarting server at an unfortunate time will cause it to re-execute last statement and potentially break replication. Did you fix this situation with GTIDs and rpl_slave_state table? It looks like with GTIDs it's really easy to solve the problem -- rpl_slave_state has GTID of the last executed transaction, so on restart one needs just to ignore all relay logs and re-connect to master with the last GTID. Did you do that? If not are you planning to do that? Thank you, Pavel

2 11

Re: [Maria-developers] MariaDB 5.5 EOL plans
by Honza Horak 27 May '13

27 May '13

On 05/24/2013 11:04 PM, Rasmus Johansson wrote: > Hi, > > On Tue, May 14, 2013 at 5:57 PM, Honza Horak <hhorak(a)redhat.com > <mailto:hhorak@redhat.com>> wrote: > > Hi guys, > > I'm not able to find any info about life cycle of MariaDB 5.5 or > even other versions. Particularly if the MariaDB 5.5 life cycle will > somehow follow the MySQL's 5.5 life cycle or if there is some > another vision/rough plans when supporting of 5.5 will end? > > > Thanks for bringing this up! It has been on the table quite some times, > but now finally I think we got it documented. So here it is: > Thank you very much, that looks really great. Honza

1 0

[Maria-developers] Syncs server status variable
by Ian Gilfillan 26 May '13

26 May '13

What does the Syncs server status variable record, and when was it introduced? I can't seem to find this documented anywhere. thanks, ian gilfillan

2 1

[Maria-developers] MariaDB 5.5 EOL plans
by Honza Horak 25 May '13

25 May '13

Hi guys, I'm not able to find any info about life cycle of MariaDB 5.5 or even other versions. Particularly if the MariaDB 5.5 life cycle will somehow follow the MySQL's 5.5 life cycle or if there is some another vision/rough plans when supporting of 5.5 will end? Cheers, Honza

3 3

[Maria-developers] query cache restrictions
by Roberto Spadim 25 May '13

25 May '13

hi guys, why we can't cache "union" queries with query cache? http://www.mysqlperformanceblog.com/2006/07/27/mysql-query-cache/ *Caching full queries only *– Meaning it does not work for subselects, inline views, parts of the UNION. This is also common missunderstanding. -- Roberto Spadim

1 1

Re: [Maria-developers] To Sergei:
by Sergei Golubchik 24 May '13

24 May '13

Hi, Samarth! On May 24, Samarth Mathur wrote: > I put the <plugin>.c file and the CmakeLists.txt in a folder on my desktop > and ran > *cmake .* > > CMake Warning (dev) in CMakeLists.txt: > No cmake_minimum_required command is present. A line of code such as > > cmake_minimum_required(VERSION 2.8) > > should be added at the top of the file. The version specified may be > lower > if you wish to support older CMake versions for this project. For more > information run "cmake --help-policy CMP0000". > This warning is for project developers. Use -Wno-dev to suppress it. > > then i ran > *cmake . -Wno-dev* > > CMake Error at CMakeLists.txt:1 (MYSQL_ADD_PLUGIN): > Unknown CMake command "MYSQL_ADD_PLUGIN". You need to do that from the MariaDB source tree. Put your plugin in the plugin/ directory, where other plugins are. Regards, Sergei

1 0

[Maria-developers] Fwd: To Sergei:
by Samarth Mathur 24 May '13

24 May '13

I put the <plugin>.c file and the CmakeLists.txt in a folder on my desktop and ran *cmake .* CMake Warning (dev) in CMakeLists.txt: No cmake_minimum_required command is present. A line of code such as cmake_minimum_required(VERSION 2.8) should be added at the top of the file. The version specified may be lower if you wish to support older CMake versions for this project. For more information run "cmake --help-policy CMP0000". This warning is for project developers. Use -Wno-dev to suppress it. then i ran *cmake . -Wno-dev* CMake Error at CMakeLists.txt:1 (MYSQL_ADD_PLUGIN): Unknown CMake command "MYSQL_ADD_PLUGIN". -- Configuring incomplete, errors occurred! Thanks On Fri, May 24, 2013 at 10:40 PM, Sergei Golubchik <serg(a)askmonty.org>wrote: > Hi, Samarth! > > On May 24, Samarth Mathur wrote: > > Hi, > > > > How do I go about compiling the plugin in mariadb? I searched online, but > > only found the mysql method : > > https://dev.mysql.com/doc/refman/5.5/en/compiling-plugin-libraries.html > > I'm getting some errors there, but, I wanna know, if it's done > differently > > or like this? > > Like in the manual, at the link above. > What errors are you getting? > > Regards, > Sergei >

1 0

Re: [Maria-developers] To Sergei:
by Samarth Mathur 24 May '13

24 May '13

Hi, How do I go about compiling the plugin in mariadb? I searched online, but only found the mysql method : https://dev.mysql.com/doc/refman/5.5/en/compiling-plugin-libraries.html I'm getting some errors there, but, I wanna know, if it's done differently or like this? Thanks On Thu, May 23, 2013 at 6:28 PM, Sergei Golubchik <serg(a)askmonty.org> wrote: > Hi, Samarth! > > On May 23, Samarth Mathur wrote: > > 2 questions: > > > > to make .so file from .c, is it necessary for me to compile mariadb > > after each change? Or is there a shorter way as far as plugins are > > concerned? > > You only need to compile the plugin, you don't need to compile MariaDB > at all, you can use a binary from our site or as provided by your OS. > > > Is there any specific tip you would have as far as coding is > > concerned? Or does the normal convention of good code, good comments > > and indentation work alright? > > Just normal convention. The usual - see how it's done elsewhere in > MariaDB and do the same. > > Regards, > Sergei > >

2 1

[Maria-developers] Fusion-io atomic writes patch comments
by Laurynas Biveinis 24 May '13

24 May '13

Hi - We ported your patch to Percona Server, meanwhile we collected the following comments which perhaps are useful for you. These are addressed in our XtraDB port, so you will pick them up, but since you patched InnoDB as well, you might consider addressing them there as well. I tried to order them roughly in the order of importance. The comments below refer to rev 3720 of 5.5 tree (and one comment to MariaDB 10 trunk). - Where are the testcases? sys_vars test doesn't count :) - fil_extend_space_to_desired_ size() fails to consider that it might be extending a single node on a multi-node tablespace. It will calculate the desired size as if the last node was the only one, overextending the tablespace by the size of all the previous nodes. Fix is replacing offset_high = size_after_extend * page_size / (4ULL*1024*1024*1024); offset_low = size_after_extend * page_size % (4ULL*1024*1024*1024); with offset_high = (size_after_extend - file_start_page_no) * page_size / (4ULL * 1024 * 1024 * 1024); offset_low = (size_after_extend - file_start_page_no) * page_size % (4ULL * 1024 * 1024 * 1024); - In the same function in InnoDB 5.6 / MariaDB 10.0 it is necessary to set node->being_extended = FALSE; before jumping to complete_io. - In the same function, complete_io: label may skip over the fil_node_complete_io() call, which is required for os_aio() tablespace-extending call only. - os_file_set_atomic_writes() should return not an int, but bool or ibool instead, also its return value currently is raw ioctl() result, which is used as a bool at callers anyway and reversed from the InnoDB convention of true/non-zero meaning success. While at that, it's also nice to add __attribute__((warn_unused_result)) to it. - If ioctl() failed, then it's customary for the failed I/O call handling to call one of the os_file_handle_error... functions, here os_file_handle_error_no_exit() looks like a good fit. - If atomic writes are unsupported, then I'm not sure why set errno / LastError. No syscall has failed, the atomic writes unsupported-ness is communicated by the os_file_set_atomic_writes() return value. - os_file_set_atomic_writes() declaration should not depend on __linux__ etc, thus we pushed the #ifdef to function body. - InnoDB uses UNIV_LINUX instead of __linux__, the build sys checks for the presence of sys/ioctl.h and defines corresponding define, thus we replaced #ifdef __linux__ #include <sys/ioctl.h> #ifndef DFS_IOCTL_ATOMIC_WRITE_SET #define DFS_IOCTL_ATOMIC_WRITE_SET _IOW(0x95, 2, uint) #endif with #if defined(UNIV_LINUX) && defined(HAVE_SYS_IOCTL_H) # include <sys/ioctl.h> # ifndef DFS_IOCTL_ATOMIC_WRITE_SET # define DFS_IOCTL_ATOMIC_WRITE_SET _IOW(0x95, 2, uint) # endif #endif at the top of the file, and the #ifdef inside the function is #ifdef DFS_IOCTL... - Is this bit inside __WIN__ in os_file_create_func() actually needed? if (srv_use_atomic_writes && type == OS_DATA_FILE && os_file_set_atomic_writes(file, name)) { CloseHandle(file); *success = FALSE; file = INVALID_HANDLE_VALUE; } - os_file_set_size() could call os_file_handle_error_no_exit() for failed posix_fallocate() instead of printing the errno itself. - Comments don't follow InnoDB style, missing header function comments, too long lines, spurious whitespace, etc. Our code is at https://bazaar.launchpad.net/~percona-core/percona-server/5.5/revision/518 https://bazaar.launchpad.net/~percona-core/percona-server/5.6/revision/353 We haven't addressed the testcase issue yet neither. Regards, -- Laurynas

2 2

Re: [Maria-developers] [Commits] Rev 3770: MDEV-4042 & MDEV-4536 fix. in file:///home/bell/maria/bzr/work-maria-5.5-MDEV-4536/
by Sergei Petrunia 24 May '13

24 May '13

Hi Sanja, The patch doesn't take one thing into account. Consider a query: explain select * from t12 where b>3 or exists (select max(t10.b) Z from t10 where t10.a in ( select t11.a from t11) group by t10.c having Z<100); +--+------------+-----------+------+-------------+------------+-------+----+----+-------------------------------+ |id|select_type |table |type |possible_keys|key |key_len|ref |rows|Extra | +--+------------+-----------+------+-------------+------------+-------+----+----+-------------------------------+ |1 |PRIMARY |t12 |ALL |NULL |NULL |NULL |NULL|10 | | |2 |SUBQUERY |t10 |ALL |NULL |NULL |NULL |NULL|6 |Using temporary; Using filesort| |2 |SUBQUERY |<subquery3>|eq_ref|distinct_key |distinct_key|4 |func|1 | | |3 |MATERIALIZED|t11 |ALL |NULL |NULL |NULL |NULL|6 | | +--+------------+-----------+------+-------------+------------+-------+----+----+-------------------------------+ Here, the EXISTS subquery is the one that will have the problem. I have added "in ( select t11.a from t11)" so that EXISTS subquery's join_tab array is not linear. The problem is not visible to the user, but it can be seen in debugger. Put a breakpoint in JOIN::cleanup(), with these commands: p table->alias->Ptr p table->file Run the query and you'll see: Breakpoint 5, st_join_table::cleanup (this=0x7fff280dbea0) at /home/psergey/dev2/5.5/sql/sql_select.cc:10248 $30 = 0x7fff28000af0 "t12" $31 = (ha_innobase *) 0x7fff28047198 ... Breakpoint 5, st_join_table::cleanup (this=0x7fff28053078) at /home/psergey/dev2/5.5/sql/sql_select.cc:10248 $32 = 0xd05418 "" $33 = (ha_heap *) 0x7fff2805ab98 ... Breakpoint 5, st_join_table::cleanup (this=0x7fff28051ef8) at /home/psergey/dev2/5.5/sql/sql_select.cc:10248 $36 = 0x7fff28025720 "t10" $37 = (ha_innobase *) 0x7fff28009558 ... Breakpoint 5, st_join_table::cleanup (this=0x7fff28052218) at /home/psergey/dev2/5.5/sql/sql_select.cc:10248 $38 = 0xd30a69 "sj-materialize" $39 = (ha_heap *) 0x7fff28054398 ... Note that in subquery join_tab->cleanup() was called for t10, and for sj-materialize. For t11 (which is inside the sj-materialize) it was not called. This caused by this loop being flat loop: > @@ -10646,6 +10646,12 @@ void JOIN::cleanup(bool full) > { > tab->cleanup(); > } > + if (join_tab != table_access_tabs) > + { > + for (uint i= 0; i < top_table_access_tabs_count; i++)a ^^^^^^^^^^^ here. > + table_access_tabs[i].cleanup(); > + table_access_tabs= join_tab; // avoid second cleanup() > + } > cleaned= true; > } > else It turns out, I was incorrect with the statement that "table_access_tabs array is always flat". On Tue, May 21, 2013 at 08:30:35AM +0300, sanja(a)montyprogram.com wrote: > At file:///home/bell/maria/bzr/work-maria-5.5-MDEV-4536/ > > ------------------------------------------------------------ > revno: 3770 > revision-id: sanja(a)montyprogram.com-20130521053031-xe34093yhvey0l5o > parent: monty(a)askmonty.org-20130515132812-pnjhza9iugz89356 > committer: sanja(a)montyprogram.com > branch nick: work-maria-5.5-MDEV-4536 > timestamp: Tue 2013-05-21 08:30:31 +0300 > message: > MDEV-4042 & MDEV-4536 fix. > > Optimization JOIN tabs also should be cleaned up (in case of EXPLAIN when we cleaning up everythibng at the end of execution). > === added file 'mysql-test/r/innodb_explain.result' > --- a/mysql-test/r/innodb_explain.result 1970-01-01 00:00:00 +0000 > +++ b/mysql-test/r/innodb_explain.result 2013-05-21 05:30:31 +0000 > @@ -0,0 +1,34 @@ > +drop table if exists t1,t2,t3; > +# > +# MDEV-4536: sql_base.cc:1598: bool close_thread_table(THD*, TABLE**): > +# Assertion `table->key_read == 0' failed. > +# > +set optimizer_switch="index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=off"; > +CREATE TABLE t1 (pk int auto_increment primary key, `col_int_key` int(11), > +key `col_int_key` (`col_int_key`),`col_varchar_key` varchar(128), key > +(col_varchar_key)) engine=innodb; > +EXPLAIN SELECT 1 FROM t1 AS alias1 WHERE EXISTS ( SELECT SQ2_alias1 . col_int_key AS SQ2_field1 FROM ( t1 AS SQ2_alias1 RIGHT OUTER JOIN t1 AS SQ2_alias2 ON (SQ2_alias2 . col_int_key = SQ2_alias1 . col_int_key ) ) GROUP BY SQ2_field1 HAVING SQ2_alias1 . col_int_key >= 7 ); > +id select_type table type possible_keys key key_len ref rows Extra > +1 PRIMARY NULL NULL NULL NULL NULL NULL NULL Impossible WHERE > +2 SUBQUERY SQ2_alias2 index NULL col_int_key 5 NULL 1 Using index; Using temporary; Using filesort > +2 SUBQUERY SQ2_alias1 ref col_int_key col_int_key 5 test.SQ2_alias2.col_int_key 1 Using where; Using index > +set optimizer_switch=default; > +drop table t1; > +# > +# MDEV-4042: Assertion `table->key_read == 0' fails in > +# close_thread_table on EXPLAIN with GROUP BY and HAVING in > +# EXISTS SQ, InnoDB > +# > +CREATE TABLE t1 (a INT) ENGINE=InnoDB; > +CREATE TABLE t2 (b INT PRIMARY KEY, c INT) ENGINE=InnoDB; > +CREATE TABLE t3 (d INT) ENGINE=InnoDB; > +EXPLAIN > +SELECT * FROM t1 > +WHERE EXISTS ( SELECT b FROM t2, t3 > +GROUP BY b > +HAVING b != 3 ); > +id select_type table type possible_keys key key_len ref rows Extra > +1 PRIMARY NULL NULL NULL NULL NULL NULL NULL Impossible WHERE > +2 SUBQUERY t2 index NULL PRIMARY 4 NULL 1 Using index; Using temporary; Using filesort > +2 SUBQUERY t3 ALL NULL NULL NULL NULL 1 Using join buffer (flat, BNL join) > +drop table t1,t2,t3; > > === added file 'mysql-test/t/innodb_explain.test' > --- a/mysql-test/t/innodb_explain.test 1970-01-01 00:00:00 +0000 > +++ b/mysql-test/t/innodb_explain.test 2013-05-21 05:30:31 +0000 > @@ -0,0 +1,41 @@ > +# > +# Innodb specific explains > + > +--disable_warnings > +drop table if exists t1,t2,t3; > +--enable_warnings > + > +--source include/have_innodb.inc > + > +--echo # > +--echo # MDEV-4536: sql_base.cc:1598: bool close_thread_table(THD*, TABLE**): > +--echo # Assertion `table->key_read == 0' failed. > +--echo # > +set optimizer_switch="index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=off"; > + > +CREATE TABLE t1 (pk int auto_increment primary key, `col_int_key` int(11), > +key `col_int_key` (`col_int_key`),`col_varchar_key` varchar(128), key > +(col_varchar_key)) engine=innodb; > + > +EXPLAIN SELECT 1 FROM t1 AS alias1 WHERE EXISTS ( SELECT SQ2_alias1 . col_int_key AS SQ2_field1 FROM ( t1 AS SQ2_alias1 RIGHT OUTER JOIN t1 AS SQ2_alias2 ON (SQ2_alias2 . col_int_key = SQ2_alias1 . col_int_key ) ) GROUP BY SQ2_field1 HAVING SQ2_alias1 . col_int_key >= 7 ); > + > + > +set optimizer_switch=default; > +drop table t1; > + > +--echo # > +--echo # MDEV-4042: Assertion `table->key_read == 0' fails in > +--echo # close_thread_table on EXPLAIN with GROUP BY and HAVING in > +--echo # EXISTS SQ, InnoDB > +--echo # > +CREATE TABLE t1 (a INT) ENGINE=InnoDB; > +CREATE TABLE t2 (b INT PRIMARY KEY, c INT) ENGINE=InnoDB; > +CREATE TABLE t3 (d INT) ENGINE=InnoDB; > + > +EXPLAIN > +SELECT * FROM t1 > +WHERE EXISTS ( SELECT b FROM t2, t3 > +GROUP BY b > +HAVING b != 3 ); > + > +drop table t1,t2,t3; > > === modified file 'sql/sql_select.cc' > --- a/sql/sql_select.cc 2013-05-07 11:05:09 +0000 > +++ b/sql/sql_select.cc 2013-05-21 05:30:31 +0000 > @@ -3680,8 +3680,8 @@ make_join_statistics(JOIN *join, List<TA > if (pull_out_semijoin_tables(join)) > DBUG_RETURN(TRUE); > > - join->join_tab=stat; > - join->top_join_tab_count= table_count; > + join->table_access_tabs= join->join_tab=stat; > + join->top_table_access_tabs_count= join->top_join_tab_count= table_count; > join->map2table=stat_ref; > join->table= table_vector; > join->const_tables=const_count; > @@ -10646,6 +10646,12 @@ void JOIN::cleanup(bool full) > { > tab->cleanup(); > } > + if (join_tab != table_access_tabs) > + { > + for (uint i= 0; i < top_table_access_tabs_count; i++) > + table_access_tabs[i].cleanup(); > + table_access_tabs= join_tab; // avoid second cleanup() > + } > cleaned= true; > } > else > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits -- BR Sergei -- Sergei Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] Fwd: [Commits] Rev 3549: Fix bug MDEV-4407 SQ pushdown: Wrong result (extra rows) with IN and EXISTS subqueries, optimizer_use_condition_selectivity>1 in file:///home/tsk/mprog/src/10.0-md83/
by Sergei Petrunia 24 May '13

24 May '13

Hi Timour, Ok to push. On Thu, May 23, 2013 at 11:19:36PM +0300, Timour Katchaounov wrote: > Sergey, > > Please approve the following patch we already discussed. > > Timour > > > ------------------------------------------------------------ > revno: 3549 > revision-id: timour(a)askmonty.org-20130523201638-4vj8cx0yjuww29nt > parent: timour(a)askmonty.org-20130520190622-rdrru0c2gbdgtjy5 > fixes bug: https://mariadb.atlassian.net/browse/MDEV-4407 > committer: timour(a)askmonty.org > branch nick: 10.0-md83 > timestamp: Thu 2013-05-23 23:16:38 +0300 > message: > Fix bug MDEV-4407 SQ pushdown: Wrong result (extra rows) with IN and EXISTS subqueries, optimizer_use_condition_selectivity>1 > (Part of MDEV-83) > > Analysis: > MDEV-83 takes into account the cost of subqueries, which results > in a materialized semi-join where the materialized table is on > the right side of the join. At the same time, the WHERE clause > has an EXISTS predicate which is moved to the second table in > the join because of smaller join cardinality. > > The semi-join startup function setup_sj_materialization_part2 > didn't take into account that in addition to the injected > IN-EXISTS conditions, there may be other conditions, such as > ones moved by pushdown of subquery predicates in mdev-83. So > setup_sj_materialization_part2 just set the condition of the > semi-join to be the IN-EXISTS equality, this removing the > subquery moved by the subquery pushdown logic. > > Solution: > And the conditions instead of just setting them. > > > > === modified file 'mysql-test/r/subselect4.result' > --- a/mysql-test/r/subselect4.result 2013-05-20 19:06:22 +0000 > +++ b/mysql-test/r/subselect4.result 2013-05-23 20:16:38 +0000 > @@ -2393,5 +2393,31 @@ COUNT(b) > 1 > 2 > drop table t1, t2, t3; > +# > +# MDEV-4407 SQ pushdown: Wrong result (extra rows) with IN and EXISTS subqueries, optimizer_use_condition_selectivity>1 > +# > +SET optimizer_switch='expensive_pred_static_pushdown=on'; > +SET optimizer_use_condition_selectivity=3; > +SET use_stat_tables=PREFERABLY; > +CREATE TABLE t1 (a INT, b INT, KEY(b)) ENGINE=MyISAM; > +INSERT INTO t1 VALUES (1,1),(2,2),(1,1),(0,0); > +ANALYZE TABLE t1; > +Table Op Msg_type Msg_text > +test.t1 analyze status OK > +FLUSH TABLES; > +EXPLAIN > +SELECT * FROM t1 AS outer_t1 WHERE b IN ( SELECT a FROM t1 WHERE b != 1 ) AND EXISTS ( SELECT 1 FROM t1 WHERE b < outer_t1.a ); > +id select_type table type possible_keys key key_len ref rows Extra > +1 PRIMARY outer_t1 ALL b NULL NULL NULL 4 > +1 PRIMARY <subquery2> eq_ref distinct_key distinct_key 4 func 1 Using where > +2 MATERIALIZED t1 ALL b NULL NULL NULL 4 Using where > +3 DEPENDENT SUBQUERY t1 index b b 5 NULL 4 Using where; Using index > +SELECT * FROM t1 AS outer_t1 WHERE b IN ( SELECT a FROM t1 WHERE b != 1 ) AND EXISTS ( SELECT 1 FROM t1 WHERE b < outer_t1.a ); > +a b > +2 2 > +drop table t1; > +SET optimizer_switch=@@global.optimizer_switch; > +SET optimizer_use_condition_selectivity=default; > +SET use_stat_tables=default; > SET optimizer_switch= @@global.optimizer_switch; > set @@tmp_table_size= @@global.tmp_table_size; > > === modified file 'mysql-test/t/subselect4.test' > --- a/mysql-test/t/subselect4.test 2013-05-20 19:06:22 +0000 > +++ b/mysql-test/t/subselect4.test 2013-05-23 20:16:38 +0000 > @@ -1917,5 +1917,29 @@ GROUP BY a; > > drop table t1, t2, t3; > > +--echo # > +--echo # MDEV-4407 SQ pushdown: Wrong result (extra rows) with IN and EXISTS subqueries, optimizer_use_condition_selectivity>1 > +--echo # > + > +SET optimizer_switch='expensive_pred_static_pushdown=on'; > +SET optimizer_use_condition_selectivity=3; > +SET use_stat_tables=PREFERABLY; > + > +CREATE TABLE t1 (a INT, b INT, KEY(b)) ENGINE=MyISAM; > +INSERT INTO t1 VALUES (1,1),(2,2),(1,1),(0,0); > +ANALYZE TABLE t1; > +FLUSH TABLES; > + > +EXPLAIN > +SELECT * FROM t1 AS outer_t1 WHERE b IN ( SELECT a FROM t1 WHERE b != 1 ) AND EXISTS ( SELECT 1 FROM t1 WHERE b < outer_t1.a ); > +SELECT * FROM t1 AS outer_t1 WHERE b IN ( SELECT a FROM t1 WHERE b != 1 ) AND EXISTS ( SELECT 1 FROM t1 WHERE b < outer_t1.a ); > + > +drop table t1; > + > +SET optimizer_switch=@@global.optimizer_switch; > +SET optimizer_use_condition_selectivity=default; > +SET use_stat_tables=default; > + > + > SET optimizer_switch= @@global.optimizer_switch; > set @@tmp_table_size= @@global.tmp_table_size; > > === modified file 'sql/opt_subselect.cc' > --- a/sql/opt_subselect.cc 2013-04-01 10:36:05 +0000 > +++ b/sql/opt_subselect.cc 2013-05-23 20:16:38 +0000 > @@ -3620,7 +3620,10 @@ bool setup_sj_materialization_part2(JOIN > emb_sj_nest->sj_subq_pred))) > DBUG_RETURN(TRUE); /* purecov: inspected */ > sjm_tab->type= JT_EQ_REF; > - sjm_tab->select_cond= sjm->in_equality; > + remove_sj_conds(&sjm_tab->select_cond); > + sjm_tab->select_cond= and_items(sjm_tab->select_cond, sjm->in_equality); > + if (!sjm_tab->select_cond->fixed) > + sjm_tab->select_cond->fix_fields(thd, &sjm_tab->select_cond); > } > else > { > > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits -- BR Sergei -- Sergei Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] [Commits] Rev 3769: MDEV-4520 fix
by Sergei Golubchik 23 May '13

23 May '13

Hi, Oleksandr! On May 23, Oleksandr Byelkin wrote: > 22.05.2013 20:27, Sergei Golubchik пишет: > > On May 22, sanja(a)montyprogram.com wrote: > >> ------------------------------------------------------------ > >> revno: 3769 > >> revision-id: sanja(a)montyprogram.com-20130522125526-aw778iepj1yceskd > >> parent: holyfoot(a)askmonty.org-20130514213637-3uvcda98gemgiquf > >> committer: sanja(a)montyprogram.com > >> branch nick: work-maria-5.5-MDEV-4520 > >> timestamp: Wed 2013-05-22 15:55:26 +0300 > >> message: > >> MDEV-4520 fix. > > First, please, fix the changeset comment. The first line should be > > MDEV-<number><space><complete issue summary> > > > > For example: > > > > MDEV-4520 Assertion `0' fails in Query_cache::end_of_result on concurrent drop event and event execution > > > > Then, after an empty line, you add whatever explanations you think are > > necessary. Like the one below: > The only question is how happened that I did not know standard of commit > comment (where that standard is/was described)? This is a de facto standard. It wasn't formally stated as a rule to follow, but everyone is doing it anyway, and it's really convenient to have changeset comments structured this way. But now I don't mind making it a written rule - because it practically won't change anything :) Where do you suggest it should be written? > >> If there is no net.vio then query cache cant't get data via > >> net_real_write() so it is better just do not try to cache such > >> query. > > Also, it'd be good to record the fixed bug in the changeset metadata. > > This can be done either with "bzr gcommit" and my patches to it (see the > > article in KB), or from the command line with "bzr commit --fixes". > > Again previous question... It's not at all a standard. It's nice to have, but very few are doing it. Feel free not to. > >> === modified file 'sql/sql_cache.cc' > >> --- a/sql/sql_cache.cc 2013-05-07 11:05:09 +0000 > >> +++ b/sql/sql_cache.cc 2013-05-22 12:55:26 +0000 > >> @@ -3977,7 +3977,8 @@ Query_cache::is_cacheable(THD *thd, LEX > >> if (thd->lex->safe_to_cache_query && > >> (thd->variables.query_cache_type == 1 || > >> (thd->variables.query_cache_type == 2 && (lex->select_lex.options & > >> - OPTION_TO_QUERY_CACHE)))) > >> + OPTION_TO_QUERY_CACHE))) && > >> + thd->net.vio) > >> { > >> DBUG_PRINT("qcache", ("options: %lx %lx type: %u", > >> (long) OPTION_TO_QUERY_CACHE, > > The fix itself looks ok. But please add a test case. > > > > And think if there can be more cases (besides events) where SELECT is > > executed with thd->net.vio == 0. E.g. init-file? or init-connect? > > Or INSERT (in the replication thread) that invokes a stored function or > > a trigger? Anything you can think of. > I spent more then half a day with this and here is result: > > - vio is definitely unset for replication and/or events (I have no idea > how that tests work, probably it will require several days to figure out) Right, that's why I suggested a test with events. And a test with a SELECT in the replication thread. > - one bug found in mysql-test-run.pl and options files (MDEV-4567) (the > bug really slows down any investigation about any test with options). Sanja, if you want your submitted bug to have any chance of ever being fixed, please at least specify the affected version and the fix version. > - putting SELECT statement in --init_connect does not work. What does it mean? How exactly it "does not work" ? > I was trying to catch it with putting assertion in my_net_write. there > is no great success so far. > > Should I continue or left it as is taking into account that bug was > reported against our standard test suite? But that's the point - it's not against our standard test suite. It's a test from the test suite run with the non-default command line options. Nobody runs the test suite this way, so if this will ever be broken again, it won't be noticed, no test will fail. That's why we need a test for this bug. Regards, Sergei

1 0

[Maria-developers] Proposed changes to GTID syntax
by Kristian Nielsen 23 May '13

23 May '13

I am planning some changes to the user interface of Global Transaction ID. Testing by Elena has shown usability issues that I want to address with these changes, but I wanted to give everyone a change to voice their opinion on the changes. The problem: The issue concerns when a slave server initially connects to the master using GTID. The slave needs to send to the master the GTID position it wants to start replicating from (the GTID of the last transaction executed within each replication domain). If the connecting server was already a slave to another master before, then the position is the last replicated event(s), stored in table mysql.rpl_slave_state. But if we server is the prior master that is now to become a slave, we need to connect at the position of the last event(s) logged by the prior master in the binlog. The problem occurs when users do manual updates on the slave and those updates get into the binlog of the slave. Of course, in principle one should not do that, as it creates differences between the master and the slave. But it is a very common thing to do among users no doubt. Maybe they need to fix some replication problem, maybe add an account only on the slave, whatever, and they forget to set @@sql_log_bin=0 while doing the change, so it gets into the slave binlog. With current code, if the slave connects to a new master right after doing the manual changes to the binlog on the slave, it will connect at the GTID position of those changes, not at the last replicated transaction. Because doing changes directly on the slave server in effect makes that server temporarily a "master". And besides, this is user error anyway, creating divergence betheen the binlogs on slave and master server. But I agree with Elena that this will be a nasty surprise to many users trying out GTID for the first time. It is inconsistent with old-style replication, where direct updates on the slave never change the slave replication position. And many users will not read the complete documentation on GTID before trying to use it, and will not understand the issues with manual changes on a slave in effect turning it into a master temporarily. Note that this affects usability when users abuse the replication, using it against the recommended way, not taking care to keep master and slaves in sync. For disciplined usage, eg. running with the "strict GTID" mode that we plan to implement, these kind of problems will never occur in the first place. But I want to do as much as possible to also make things work well for "undisciplined" users. The proposed solution: There will be two ways to configure a slave to use GTID. One will be for "disciplined" usage, same as the current code, where slave connects with whatever is most recent in mysql.rpl_slave_state or binlog. The other will be for "sloppy" usage, where the user will need to explicitly state whether to use the position from the last replicated transaction, or to use what is in the binlog. I hope this will solve the problem. Since new users need to learn new syntax to enable GTID anyway (CHANGE MASTER ... master_use_gtid=...), they will at least have made a conscious choice about whether to use one syntax or the other. And for "disciplined" users there is not really any change, they will just use the syntax for the current behaviour. I also think this improves things on the conceptual level. Currently, there is too much magic about the value of the variable @@GLOBAL.gtid_pos. You can set this manually (and that updates mysql.rpl_slave_state), but it also changes value whenever something is added to the binlog, and you cannot manually set it to something that conflicts with the binlog. This magic all becomes unnecessary with the proposed changes. So there will be some changes to the syntax for CHANGE MASTER, and some stuff will be renamed to make things clearer. Details: CHANGE MASTER syntax will be modified to this: 1. CHANGE MASTER TO master_use_gtid = current_gtid_pos This is the "disciplined" mode, the same as master_use_gtid=1 in current code. It will use as replication start position whatever is most recent, last replicated GTID or last binlogged GTID. 2. CHANGE MASTER TO master_use_gtid = slave_gtid_pos This is "sloppy" mode, for pointing an existing slave to a new master. It will only look at the last replicated GTID for the start position, not at any transactions logged directly to the slave binlog. 3. CHANGE MASTER TO master_use_gtid = off For completeness, to turn of using GTID to connect, go back to using old-style replication. There will be three new system variables introduced, replacing the current @@GLOBAL.gtid_pos: 1. @@GLOBAL.slave_gtid_pos. Read-write. This is the set of last replicated GTID, per replication domain. 2. @@GLOBAL.binlog_gtid_pos. Read only. This is the set of last binlogged GTID, per domain. 3. @@GLOBAL.current_gtid_pos. Read only. This is a combination of @@GLOBAL.slave_gtid_pos and @@GLOBAL.binlog_gtid_pos, per-domain it holds the most recent GTID either replicated or binlogged. The current mysql.rpl_slave_state table will be renamed to mysql.slave_gtid_pos. Note how the naming is now more consistent. slave_gtid_pos means the most recently replicated GTIDs, both for table, variable, and CHANGE MASTER syntax. Similarly, current_gtid_pos is used for both the variable and CHANGE MASTER to mean the most recent GTID either replicated or binlogged. The user can see the GTID position the slave will start replicating at by inspecting the value of the corresponding variable. When using master_use_gtid=slave_gtid_pos, the following statement can be used on a server that used to be the master but should now become a slave: SET GLOBAL slave_gtid_pos = @@binlog_gtid_pos; This will load whatever was logged to the binlog into the slave position. When using master_use_gtid=current_gtid_position, there is no need to do anything special, the binlog will have the most recent GTID on a prior master so it will be used automatically. I think these changes will be a good improvement, justifying doing the change this late during 10.0 release state. But I very much welcome input and suggestions for doing things differently. - Kristian.

1 2

[Maria-developers] Ubuntu Hardy and Oneiric EOL
by Daniel Bartholomew 23 May '13

23 May '13

FYI: As of the end of April, Ubuntu 8.04 "Hardy" and 11.10 "Oneiric" have reached EOL and are no longer supported by Canonical. Sometime in the next week or so I plan to turn off these builders in buildbot. The VMs will be archived, in case we need them, but no regular builds will be made. This includes Debian packages for Hardy and Oneiric and the binary tarballs which are built on Hardy (binary tarballs are also built on Ubuntu Precise, and they are not going away). The MariaDB 5.5.31 release will be the last MariaDB release to feature packages for these two distros. The repos for them will remain online until the 5.5.32 release. At that time they will go away. This also affects the Mint 9 and Mint 12 as they use the hardy and oneiric repos, respectively. Let me know if you have any questions or concerns. Thanks! -- Daniel Bartholomew Google+ - http://gplus.to/dbart Twitter - http://twitter.com/daniel_bart MariaDB: An Enhanced Drop-in Replacement for MySQL Website - http://mariadb.org Twitter - http://twitter.com/mariadb Google+ - http://google.com/+mariadb Facebook - http://fb.com/MariaDB.dbms Knowledgebase - http://kb.askmonty.org

1 0

[Maria-developers] MariaDB 5.5.31 Release Prep
by Daniel Bartholomew 22 May '13

22 May '13

All, I've uploaded almost all of the packages for the release (working on a fix for an error with the sol-10 package) and the mirrors will be picking them up soon. Draft Changelog and Release Notes are here: - https://kb.askmonty.org/en/mariadb-5531-release-notes/ - https://kb.askmonty.org/en/mariadb-5531-changelog/ One of the big items in this release is TokuDB. We didn't have a proper category for TokuDB yet, so I created a stub TokuDB category with some links to the Tokutek website: - https://kb.askmonty.org/en/tokudb/ If there is anything I should be sure to mention in the release notes, please let me know ASAP. My current plan is to activate the release tomorrow, 23 May 2013. Thanks! -- Daniel Bartholomew Google+ - http://gplus.to/dbart Twitter - http://twitter.com/daniel_bart MariaDB: An Enhanced Drop-in Replacement for MySQL Website - http://mariadb.org Twitter - http://twitter.com/mariadb Google+ - http://google.com/+mariadb Facebook - http://fb.com/MariaDB.dbms Knowledgebase - http://kb.askmonty.org

3 4

Re: [Maria-developers] [Commits] Rev 3769: MDEV-4520 fix
by Sergei Golubchik 22 May '13

22 May '13

Hi, Sanja! On May 22, sanja(a)montyprogram.com wrote: > ------------------------------------------------------------ > revno: 3769 > revision-id: sanja(a)montyprogram.com-20130522125526-aw778iepj1yceskd > parent: holyfoot(a)askmonty.org-20130514213637-3uvcda98gemgiquf > committer: sanja(a)montyprogram.com > branch nick: work-maria-5.5-MDEV-4520 > timestamp: Wed 2013-05-22 15:55:26 +0300 > message: > MDEV-4520 fix. First, please, fix the changeset comment. The first line should be MDEV-<number><space><complete issue summary> For example: MDEV-4520 Assertion `0' fails in Query_cache::end_of_result on concurrent drop event and event execution Then, after an empty line, you add whatever explanations you think are necessary. Like the one below: > If there is no net.vio then query cache cant't get data via > net_real_write() so it is better just do not try to cache such > query. Also, it'd be good to record the fixed bug in the changeset metadata. This can be done either with "bzr gcommit" and my patches to it (see the article in KB), or from the command line with "bzr commit --fixes". > === modified file 'sql/sql_cache.cc' > --- a/sql/sql_cache.cc 2013-05-07 11:05:09 +0000 > +++ b/sql/sql_cache.cc 2013-05-22 12:55:26 +0000 > @@ -3977,7 +3977,8 @@ Query_cache::is_cacheable(THD *thd, LEX > if (thd->lex->safe_to_cache_query && > (thd->variables.query_cache_type == 1 || > (thd->variables.query_cache_type == 2 && (lex->select_lex.options & > - OPTION_TO_QUERY_CACHE)))) > + OPTION_TO_QUERY_CACHE))) && > + thd->net.vio) > { > DBUG_PRINT("qcache", ("options: %lx %lx type: %u", > (long) OPTION_TO_QUERY_CACHE, The fix itself looks ok. But please add a test case. And think if there can be more cases (besides events) where SELECT is executed with thd->net.vio == 0. E.g. init-file? or init-connect? Or INSERT (in the replication thread) that invokes a stored function or a trigger? Anything you can think of. Regards, Sergei

1 0

[Maria-developers] History of table changes
by Roberto Spadim 22 May '13

22 May '13

Hi guys, i was looking git and i don't know if there's something similar to mariadb alter table command could we develop something like git for create database,alter tables, create table, drop table, drop database? a log that could be useful to update databases table definition and read changes done and 'commits' done by dba or developer or a tool to read my table and read remote table and show the create command to make they in sync (ok here history is lost) thanks guys -- Roberto Spadim

1 0

[Maria-developers] unexpected "Duplicate entry" error
by Kazuhiko Shiozaki 21 May '13

21 May '13

Hello, We sometimes (not always, not often but sometimes) have unexpected "Duplicate entry" error on the following table and with the following query : * table CREATE TABLE `category` ( `uid` bigint(20) unsigned NOT NULL, `category_uid` bigint(20) unsigned NOT NULL DEFAULT '0', `base_category_uid` bigint(20) unsigned NOT NULL DEFAULT '0', `category_strict_membership` tinyint(1) NOT NULL DEFAULT '0', PRIMARY KEY (`uid`,`category_uid`,`base_category_uid`,`category_strict_membership`), KEY `Membership` (`category_uid`,`base_category_uid`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci * query BEGIN; (some queries for other tables) DELETE FROM category WHERE uid=14026268; INSERT INTO category VALUES (14026268, 3131502, 2074, 1),(14026268, 2992, 2992, 0),(14026268, 3133077, 191, 1),(14026268, 1212264, 2992, 1); (some queries for other tables) COMMIT; * error Duplicate entry '14026268-3131502-2074-1' for key 'PRIMARY' But this duplicate entry error should not happen because DELETE is called in the same transaction, even if similar queries are sent from another connection at the same time. We confirmed that this kind of issue happens on the following environments using MySQL_python-1.2.3 : * MariaDB 5.5.25, XtraDB * MariaDB 5.5.29, XtraDB * MariaDB 5.5.29, InnoDB plugin Since it happens SOMETIMES, I cannot reproduce this issue by small query. Do you have any idea so that we can investigate this issue further ? Thanks in advance ! -- Kazuhiko Shiozaki Nexedi: Consulting and Development of Free / Open Source Software http://www.nexedi.com/ ERP5: Full Featured High End Open Source ERP http://www.erp5.com/

4 12

Re: [Maria-developers] MariaDB and MySQL replication
by Kristian Nielsen 21 May '13

21 May '13

Jan Lindström <jplindst(a)mariadb.org> writes: > My question is how compatible replication between > > 1) MySQL (master) -> MariaDB (slave) > 2) MariaDB (master) -> MySQL (slave) In general, it should work, though there are some limitations. It also depends on versions. I always try to make MariaDB (master) -> MySQL (slave) work. For example, when new binlog events are introduced to MariaDB, the MariaDB master will not send them to slaves that do not understand the new events (whether MySQL slave, or older version of MariaDB), it will replace them with something compatible. This is not the best tested part of the code though, so there may be bugs. MySQL (master) -> MariaDB (slave) generally has limitations. This is the same as MySQL (high version master) -> MySQL (low version master). This is due to the MySQL replication teams approach, where you need to disable new features on the master for things to work with old slaves. For example, MySQL 5.6 (master) -> MySQL 5.5 (slave) does not work out-of-the box, so MariaDB 5.5 slave is the same. Generally, when incompatibilities are introduced in new MySQL versions used as a master, we will backport to MariaDB at least to the extent that new MariaDB versions will be able to replicate as slaves from a MySQL master without needing any special configuration. Though it depends on time available and priority against other work, for example there are known incompatibilities that have not yet been fixed in 10.0 alpha but will hopefully be fixed by 10.0 GA. Officially, replicating from new master to old slave is not supported (you should upgrade slaves first). This applies mostly the same whether MySQL or MariaDB or both are involved. Unofficially, as explained above, we do try to make it work as well as reasonably possible on the MariaDB side. > Before you say they are fully compatible, I point out at least > following types at mysql_com.h that do not exist on MariaDB mysql_com.h: > > MYSQL_TYPE_TIMESTAMP2, > MYSQL_TYPE_DATETIME2, > MYSQL_TYPE_TIME2, I do not know the details of this particular point (also you did not mention any version info). Best is probably to test it and see, and then report your findings and if needed we can file a bug for fixing things. - Kristian.

2 1

Re: [Maria-developers] Getting community to write missing test cases
by Kristian Nielsen 21 May '13

21 May '13

Elena Stepanova <elenst(a)montyprogram.com> writes: > If we start doing it, Oracle will soon realize that this activity is > based at least as much on commit comments as on the code. As a result, > they will probably start making much vaguer comments than they do now, > which in turn might make merge and related activities even more > complicated than they are now. I wouldn't worry about that. First, I do not believe day-to-day developers spend any effort to actively conceal things for outside developers. The ban on bug test cases seems to have come from a generic security team mandate, which has roots some years back where Oracle were hit quite hard with countless vulnerabilities after their "can't break it - can't break in" campain. Second, even if they did, anything that makes Oracle close down more benefits the adoption of the forks. But most importantly, we must not get into the mentality that we let what Oracle might or might not do influence how we work. We need to go in whatever direction we believe is best for the project. Oracle has made it plain that they do not want to participate in the community, and we should just ignore them until such time as they might decide otherwise. - Kristian.

1 0

[Maria-developers] Feature Request - Working auto_increment on BlackHole Engine
by Gordan Bobic 20 May '13

20 May '13

How difficult would be be to implement working auto-increment functionality on the BlackHole engine? This would be really useful for what I'm trying to do at the moment. Since it is not available and there are no sequences in MariaDB and MySQL, I have to resort to a hack such as a single-int-field table doing the auto-increment, and an insert trigger on the blackhole table to insert a null into the auto-increment tracking table, get LAST_INSERT_ID() to set the value in the auto-increment table, and then delete the record. Even with InnoDB log files and tablespace on tmpfs this makes the performance nosedive by a factor of 4 if the trigger doesn't bother cleaning up the auto-increment table, and a factor of 13 if it deletes the record immediately after inserting it. The actual times I tested with 24 inserter threads on a 24-thread server are 8m vs. 30m vs. 105m, i.e. a massive drop. Is there an better place than this list for a feature request? Gordan

3 6

[Maria-developers] Ogg vorbis based table
by Roberto Spadim 18 May '13

18 May '13

Hi guys, i was listening some musics, and i had a idea... let's me explain Some systems are based in time and value, an example of scada systems. In this kind of system we have an 'channel' (like right or left sound channel) a timestamp and a value, normally a table like CREATE TABLE history( Channel int not null default 0, Datetimestamp decimal(22,7) not null default 0, Value decimal(22,5) default NULL, Primary key (channel,datetimestamp) ) Some explanations and features that scada systems use... Value can be NULL, in this case we could think like MUTE function on audio files Datetime normally are gmt, or unixtimestamp values with microsecond precision, they could be fixed interval or not, example... 1 2 3 4 5... (q second interval), in audio it's fixed sample rate, or not fixed interval (variable sample rate), like 1 3 4 9... Well what i'm thinking... today mariadb 10.0.2 started the CONNECT table type (please correct me if i'm wrong), that can 'connect' to a xml,txt,csv and others kinds of files to read and write it's values... Could we implement mp3, wave, ogg vorbis, file type type? This is a really nice 'table compression' for scada systems, and allow a new era of table type, think about... Select datetimestamp from someaudiofile where someniceaudiofunction()>1 In this example we could check date time where some scada value get over a fixed level, or where a frequency get higher and start a alarm condition, it could be used to get more information using statistics functions... i'm talking now about a new table type and new functions... but thats the idea... Anyone intested in this? About table compression think about compressing tax values in nasdaq for example, and execute a statistic analyse with this information... Well just some uses... Thanks for the space to show my idea and your time to read it Maybe a video table in future, or image based... just to use jpeg compreesion (x,y,color values) and others kinds of files... Bye

1 0

[Maria-developers] Empty_queries vs Com_empty_queries
by Ian Gilfillan 18 May '13

18 May '13

I'm documenting the status variables at the moment, and have a question about Empty_queries vs Com_empty_queries. Com_empty_queries is not incremented when an empty result is returned, and appears to be incremented only when an empty query is sent to the server, eg: mysql --comments -e '-- sql comment' Empty_queries appears to match the empty queries returned with SHOW USER_STATISTICS. It increments each time a query is performed that returns an empty result. Currently the documentation on the user statistics page at https://kb.askmonty.org/en/user-statistics/ says that EMPTY_QUERIES (and therefore the Empty_queries status variable) is "The number of times this user's connections sent empty queries to the server", which would describe Com_empty_queries rather than Empty_queries, and would need to be corrected. The documentation on https://kb.askmonty.org/en/server-status-variables/ is what I've just written based on the above. Have I understood this correctly, or is there something else to it? thanks, ian

3 2

[Maria-developers] To Sergei:
by Samarth Mathur 16 May '13

16 May '13

Hi Sergei.. just a few questions. Hope not to bother too much! does the .so file of two_questions exist in the mariadb 10.0.2 source code or was that just an example? Also, if I'm looking to implement pluggable authentication for LDAP, does the version of mariadb, i'm doing this on matter? if it does, which version would you recommend? I'm currently on v10.0.2.. What is the advantage of going for a pluggable authentication over say a module made for the PAM plugin in mariadb? Regards Sam

3 6

Re: [Maria-developers] Feature Request - Working auto_inc rement on BlackHole Engine
by Gordan Bobic 16 May '13

16 May '13

On 05/15/2013 11:27 PM, Jocelyn Fournier wrote: > Hi Gordan, > > > Le 15/05/13 22:18, Gordan Bobic a écrit : >> On 05/15/2013 09:06 PM, Michael Widenius wrote: >>> >>> Hi! >>> >>>>>>>> "Gordan" == Gordan Bobic <gordan(a)bobich.net> writes: >>> >>> Gordan> How difficult would be be to implement working >>> Gordan> auto-increment functionality on the BlackHole engine? >>> >>> Gordan> This would be really useful for what I'm trying to do at >>> Gordan> the moment. Since it is not available and there are no >>> Gordan> sequences in MariaDB and MySQL, I have to resort to a >>> Gordan> hack such as a single-int-field table doing the >>> Gordan> auto-increment, and an insert trigger on the blackhole >>> Gordan> table to insert a null into the auto-increment >>> Gordan> tracking table, get LAST_INSERT_ID() to set the >>> Gordan> value in the auto-increment table, and then delete >>> Gordan> the record. >>> >>> What problem is that you wanted to solve? >>> Normally blackhole is used just for creating a slave of a slave that >>> doesn't have to hold any data (only the logs). >> >> The problem is a huge volume logging application that needs to have >> uniquely enumerated records for auditing purposes. > > Have you tried TokuDB engine ? It isn't suitable for what I am doing in this case - note that the blackhole engine in this case suits all of my requirements except for the working auto_increment functionality. I don't need to store the data on these nodes, merely binlog it up for further processing downstream. Gordan

1 0

[Maria-developers] MariaDB GTID first impression
by Giuseppe Maxia 15 May '13

15 May '13

Kristian, Thanks for the docs. It's a very good start. Compared to MySQL 5.6, GTID in MariaDB has some advantages: it is enabled by default and it is easy to read. There are a few items, though, that should need some more thought: 1) rpl_slave_state is probably ill-named. In this architecture, it is only written by the slaves, and thus the name matches, but I wonder if the master should write this table (or a similar one) to keep track of its progress in the binary log. Rationale: all the info related to replication should be in tables and easy to query. 2) rpl_slave_state does not have enough metadata. What is missing here, and would be extremely useful, is the corresponding binlog+position for each GTID, a timestamp of when the transaction was saved to the binary log, and the timestamp of when it was applied. (The host name of the origin would also be useful). Also very important (when you will implement parallel replication) is a shard identifier. 3) Multiple domains do not integrate well with GTID. In multi-source replication, we use "connection names", but for GTID we use "domain numbers". They do not play well together : there is no correspondence between the gtid_domain_id and the connection name, and a "SHOW ALL SLAVES STATUS" lists all the GTID for every connection. For example: mysql -e 'show all slaves status\G' |grep 'Connection_name\|Running\|Master_Host\|Master_Port\|Gtid_Pos\|Master_Log' Connection_name: Bear Master_Host: 127.0.0.1 Master_Port: 8203 Master_Log_File: mysql-bin.000001 Read_Master_Log_Pos: 1020 Relay_Master_Log_File: mysql-bin.000001 Slave_IO_Running: Yes Slave_SQL_Running: Yes Exec_Master_Log_Pos: 1020 Gtid_Pos: 100-101-21,400-104-23,200-102-8,300-103-12,500-105-20 Connection_name: Wolf Master_Host: 127.0.0.1 Master_Port: 8204 Master_Log_File: mysql-bin.000001 Read_Master_Log_Pos: 896 Relay_Master_Log_File: mysql-bin.000001 Slave_IO_Running: Yes Slave_SQL_Running: Yes Exec_Master_Log_Pos: 896 Gtid_Pos: 100-101-21,400-104-23,200-102-8,300-103-12,500-105-20 Connection_name: Fox Master_Host: 127.0.0.1 Master_Port: 8205 Master_Log_File: mysql-bin.000002 Read_Master_Log_Pos: 365 Relay_Master_Log_File: mysql-bin.000002 Slave_IO_Running: Yes Slave_SQL_Running: Yes Exec_Master_Log_Pos: 365 Gtid_Pos: 100-101-21,400-104-23,200-102-8,300-103-12,500-105-20 Connection_name: Deer Master_Host: 127.0.0.1 Master_Port: 8206 Master_Log_File: mysql-bin.000001 Read_Master_Log_Pos: 1144 Relay_Master_Log_File: mysql-bin.000001 Slave_IO_Running: Yes Slave_SQL_Running: Yes Exec_Master_Log_Pos: 1144 Gtid_Pos: 100-101-21,400-104-23,200-102-8,300-103-12,500-105-20 Which connection is running which gtid number? Not easy to tell, as the correspondence between domain and connections is only in the DBA's head. 4) gtid_pos is text that includes all sequences from all domains. The docs say that I can modify it if needed, but in a case like the one above, modifying this string can be very error prone. For example, if server_ids are assigned using inet_aton(@IP), you could have a gtid_pos like this '1001-170062692-230000,1002-170062593-12893445,1003-170062700-672009,1003-170062699-700002' Instead of editing a string, and risk breaking replication in all domains, I would expect the availability of commands like: CHANGE GTID for domain 'Wolf' set gtid_pos=1900 That's it for now. Hope there is still room for improvements. Cheers Giuseppe

3 16

Re: [Maria-developers] MariaDB build logs
by Kristian Nielsen 14 May '13

14 May '13

Marcin Kulisz <debian(a)kulisz.net> writes: > as promised I'm attaching build log for maria DB, it's much smaller then > anticipated. Thanks! > From what I can tell it was just one test which failed and rendered whole build as a > failure. > My build env is based on cowbuilder on fully patched SID chroot. Ah, I see what is going on: > plugins.unix_socket [ fail ] > Test ended at 2013-05-11 00:12:52 > --- /tmp/buildd/mariadb-5.5-5.5.29/builddir/mysql-test/suite/plugins/r/unix_socket.result 2013-05-10 21:49:20.510929790 +0000 > +++ /tmp/buildd/mariadb-5.5-5.5.29/builddir/mysql-test/suite/plugins/r/unix_socket.reject 2013-05-11 00:12:52.562350485 +0000 > @@ -8,7 +8,7 @@ > # > select user(), current_user(), database(); > user() current_user() database() > -USER@localhost USER@% test > +USER@localhost USER@localhost test > # > # name does not match = failure > # Right, so the problem is that this test is poorly written. Its output depends on the name of unix username of the account running the test! I am guessing that in your case, you build as user root. Since the MariaDB server running in the test happens to also have a (database) account named root, we get an authenticated user USER@localhost instead of an anonymous user. I have now pushed a fix for this problem to the MariaDB tree: http://lists.askmonty.org/pipermail/commits/2013-May/004715.html So this fix will appear in MariaDB 5.5.31 or possibly 5.5.32 (not sure if it made it in 5.5.31, which I think is due for release). Until that fix propagates to the MariaDB packaging, some options for a work-around are: - Run the build as a non-root user, with fakeroot. - Skip the test suite during build (DEB_BUILD_OPTIONS=nocheck) (this will also greatly speed up the build). - Apply the linked patch manually Hope this helps, - Kristian.

1 0

Re: [Maria-developers] Forked mytop - conflicts, issues | fullqueries overflow bug patch
by Jean Weisbuch 14 May '13

14 May '13

Hi, I attached to this message a patch to fix the overflow when fullqueries is activated and full queries are taking more than a line that might be resulting in a page overflow (the informations shown on mytop will exceed the terminal height). As for the latest commit upstream (21, mar 2013 https://github.com/jzawodn/mytop/commit/437f2ef8d3fce02eafe935ddbf860d1dfbc…) its only a small fix that seems to already have been fixed on the MariaDB version. Regards.

1 0

Re: [Maria-developers] [Commits] Rev 3654: MDEV-4290: in file:///home/bell/maria/bzr/work-maria-5.3-MDEV-4290/
by Sergei Golubchik 13 May '13

13 May '13

Hi, Sanja! On May 03, sanja(a)askmonty.org wrote: > revno: 3654 > revision-id: sanja(a)askmonty.org-20130503130713-0n5lgrniay7n3jua > parent: igor(a)askmonty.org-20130430033140-qhx9wsgc5w2dx14r > committer: sanja(a)askmonty.org > branch nick: work-maria-5.3-MDEV-4290 > timestamp: Fri 2013-05-03 16:07:13 +0300 > message: > MDEV-4290: > Fix agregate function resolution in derived tables (no name > resolution over a derived table border) ok to push Regards, Sergei

1 0

[Maria-developers] An invalid memory access bug in HandlerSocket
by ahiguti100＠gmail.com 10 May '13

10 May '13

Hi MariaDB developers, I found an invalid memory access bug of HandlerSocket is not yet fixed in MariaDB. https://github.com/DeNA/HandlerSocket-Plugin-for-MySQL/issues/78 This bug can be reproduced by running the following script: https://github.com/DeNA/HandlerSocket-Plugin-for-MySQL/blob/master/regtest/… You may need to run mysqld with valgrind to observe the invalid memory access, because hitting this bug may not cause mysqld to crash. I've filed a merge request. https://code.launchpad.net/~ahiguti100/maria/handlersocket-fix-78/+merge/16… Rev #3750 fixes the bug, and #3751 updates documents and tests for HandlerSocket. I want docs to be updated because my github url for HandlerSocket has been changed. All the changes are copied from my github repo, and are provided under the BSD license. Please let me know if something is wrong with the merge request. Thanks, Akira

3 3

[Maria-developers] Updated myisamchk to handle LARGE files
by Michael Widenius 10 May '13

10 May '13

Hi! Here is a patch that should fix your myisamchk issues. This is to be applied to MariaDB 5.5.30 source. Can you verify that this works so I can add it to MariaDB 10.0 ? Some notes: - If you run repair with two --force options, like this: myisamchk --repair --force --force ... Then myisamchk will use the full myisam_sort_buffer_size for keys. Other buffers will be allocated as needed. This means that with this option one will never get the "myisam_sort_buffer_size is too small" error. I also added the option --create-missing-keys, which will create keys just for any disabled keys. Other things: - Better error message for 'myisam_sort_buffer_size is too small' errors. - Somewhat larger buffers as default. - Changed a lot of variables from uint -> ulonglong to try to ensure that big files will not cause any problems. - Removed some checks that limited the sort_buffer_size to 4G. - Fixed code that would cause problems if sort_buffer_size would be > 8G - Fixed printing of longlong variables. - Fixed possible core dump if 'alloc_dynamic()' would fail. - Fixed some possible problems on system where uint would be <= ulong. Hope this helps! Regards, Monty

2 2

Re: [Maria-developers] GSoC projects related on data replication.
by Kristian Nielsen 09 May '13

09 May '13

QIU Shuang <qiush.summer(a)gmail.com> writes: > A quick question: are the codes related to replication in the path `sql/rpl_*' and `sql/repl_*'? Much of the replication code is in: sql/log_event.* sql/slave.* sql/log.* sql/sql_repl.* sql/rpl_* sql/repl* Hope this helps, - Kristian.

1 0

[Maria-developers] how to use TC_LOG_MMAP in MySQL
by nanyi607rao 09 May '13

09 May '13

hi , In http://bazaar.launchpad.net/~maria-captains/maria/10.0-mdev452/revision/3434 you make some change to the class TC_LOG_MMAP,and I want to debug those codes to see how it works.but i don't understand in what conditions will MySQL use the TC_LOG_MMAP.it only use TC_LOG_DUMMY or MYSQL_BIN_LOG depends on the value of opt_bin_log.so how will it use the TC_LOG_MMAP in MySQL. 2013-05-09 nanyi607rao

3 2

[Maria-developers] GTID and failovers with multi-domain replication
by Pavel Ivanov 08 May '13

08 May '13

Kristian, You've mentioned that in future implementation of parallel replication MariaDB will use multiple domains to replicate from one master to a slave. It means for me that I need to understand how multiple domains will work. And I realized that I don't know how the following situation can be handled. I don't know how parallel replication will work, so I'll explain situation in terms of several masters. Let's assume we have server S1 that is master working with domain_id=0, server S2 is master working with domain_id=1, servers S3 and S4 are slaves and replicate from both of these masters, i.e. they have both domains in their databases. Now let's say S1 has last GTID 0-1-100, S2 has last GTID 1-2-100. Before S3 and S4 were able to fully catch up with S1 and S2 power got cut out from S1 and S2. As replication from two masters goes independently it's possible that S3 will have last transactions 0-1-100, 1-2-99 while S4 will have last transactions 0-1-99, 1-2-100. As my masters are out I want either S3 or S4 to temporarily become master. But it looks like I won't be able to do so: S3 won't connect to S4 because S4 doesn't have 0-1-100 and S4 won't connect to S3 because S3 doesn't have 1-2-100. Ideally I'd want for S3 to replicate from S4 in domain 1 and S4 to replicate from S3 in domain 0, and when they are equal in their position I can declare one of them master for both domains. But it looks like there are no tools to do such operation. How would you suggest to resolve such situation? Thank you, Pavel

2 3

[Maria-developers] Documentation about GTID
by Giuseppe Maxia 08 May '13

08 May '13

Hi! I am testing MariaDB 10.0.2, and I am interested in GTID. The only documentation I found is the worklog. Without docs at hand, I tried something: slave1 [localhost] {msandbox} ((none)) > CHANGE MASTER TO master_gtid_pos=''; ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'master_gtid_pos=''' at line 1 slave1 [localhost] {msandbox} ((none)) > CHANGE MASTER TO master_gtid_pos=auto; ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'master_gtid_pos=auto' at line 1 This is supposed to work, according to a blog post and some worklogs, but it doesn't. As for GTID in practice, it seems to be active. slave1 [localhost] {msandbox} ((none)) > show variables like '%gtid%'; +----------------+--------+ | Variable_name | Value | +----------------+--------+ | gtid_domain_id | 0 | | gtid_pos | 0-1-11 | | gtid_seq_no | 0 | +----------------+--------+ 3 rows in set (0.00 sec) slave1 [localhost] {msandbox} ((none)) > use mysql Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Database changed slave1 [localhost] {msandbox} (mysql) > select * from rpl_slave_state; +-----------+--------+-----------+--------+ | domain_id | sub_id | server_id | seq_no | +-----------+--------+-----------+--------+ | 0 | 12 | 1 | 11 | +-----------+--------+-----------+--------+ However, when I run "show slave status", I get: Using_Gtid: 0 Could you provide a pointer to the docs of the expected behavior at this stage? Thanks Giuseppe

9 35

[Maria-developers] MYSQL_TYPE_DECIMAL type
by Zardosht Kasheff 08 May '13

08 May '13

Hello all, Under what circumstances can a table be created with MYSQL_TYPE_DECIMAL as being one of the columns? I thought MYSQL_TYPE_DECIMAL was deprecated pre 5.1. I thought all new tables use MYSQL_TYPE_NEWDECIMAL. We have a user reporting a table he wants to alter has that type. However, when I create that table, I don't get that type. As an aside, we don't support that type, mainly because I thought it was impossible to have a table created with it (I am apparently wrong). Is there a way to ensure that tables undergoing an alter convert that column to MYSQL_TYPE_NEWDECIMAL? Thanks -Zardosht

3 3

[Maria-developers] is this a bug?
by jhx1008 08 May '13

08 May '13

hi all: I have a problem about the type cast when I have read the source of Maria & MySQL I notice that when cast from string into int, Maria & MySQL do it like the C function atoi code: for (ul= 0 ; str < end9 && (ch= (uchar) (*str - '0')) < 10; str++) { ul= ul * 10 + ch; } but cast from hex into int, the code is: code: char *end=(char*) str_value.ptr()+str_value.length(), *ptr=end-min(str_value.length(),sizeof(longlong)); ulonglong value=0; for (; ptr != end ; ptr++) value=(value << 8)+ (ulonglong) (uchar) *ptr; so I do the test blew: create table t(id int auto_increment primary key, a int)engine=innodb; insert into t(a) values('1'); insert into t(a) values(0x31); everything is ok, I get the result: mysql> select * from t; +----+------+ | id | a | +----+------+ | 1 | 1 | | 2 | 49 | +----+------+ the value 0x31 is the ascii of the string '1', but we got the different result after inserting maybe it's not a problem, but in the replication environment, it maybe cause the data of slave not the same as master In the master, when we set the connection characterset into multi-bytes characterset and this characterset may escape with backslash is dangerous (like gbk, cp932) and then we use the prepared statement to do the insert, the string value in the insert must be cast into hex before write into the binlog, so we do the insert(insert into t(a) values('1')) on the master, but in the slave it does the another(insert into t(a) values(0x31)) , the data is not the same. how to reappear: master(binlog format=statement): create table t(id int auto_increment primay key, a int) engine=innodb; set names gbk; prepare stmt from 'insert into t(a) values(?)'; set @a='1'; execute stmt using @a; the result in master is: mysql> select * from t; +----+------+ | id | a | +----+------+ | 1 | 1 | +----+------+ and result in slave is: mysql> select * from t; +----+------+ | id | a | +----+------+ | 1 | 49 | +----+------+ I test the newest release version mariadb-10.0.2 and MySQL-5.6.10, the problem is still exist, is this a bug? 2013-04-28 jhx1008 @netease

4 7

[Maria-developers] MariaDB allows for slave to connect with non-existent GTID
by Pavel Ivanov 07 May '13

07 May '13

Kristian, I've realized that the way slaves are processed now on the master allows them to connect even if they request non-existent GTID. Is it "works as intended" and will be different in the "strict mode" or you didn't want for such things to happen even in non-strict mode? I've attached a test reproducing the problem. It sets up replication 1->2->3, executes transaction on 1, executes transaction on 2, disconnects 3, executes one more transaction on 1, fails over to 2, executes one more transaction on 2. Then when it tries to connect 3 to 1 it should fail because 1 doesn't have the last transaction that 3 has. I agree that this scenario is artificial, but in reality scenario can be e.g. like this: 2 is slave of 1, then it gets disconnected and executes one extra transaction, then backup is taken from 2, then 2 gets restarted and restored to the state of 1, then failover happens and 2 becomes master, then 3 restores from the backup taken earlier from 2 and if all this time 1 and then 2 as masters executed some transactions 3 will be able to connect to 2 even though its last transaction doesn't exist on 2... Pavel

2 3

[Maria-developers] Missing locking around THD::set_db() ?
by Kristian Nielsen 07 May '13

07 May '13

Hi, I was looking at a Valgrind warning in Buildbot (appended below). What happens seems to be this: Thread 1 is running SHOW PROCESSLIST, it grabs the pointer THD::db to the current database of thread 2. Thread 2 then does THD::set_db(), freeing the old THD::db pointer and allocating a new one with the new data. Thread 1 then resumes, doing strdup() of the _old_, now invalid, THD::db pointer, which reads garbage data (or could even segfault if we get really unlucky). This seems like a genuine bug. I see absolutely no locking protecting against this race :-( Any suggestions for how to deal with this? The only locking I see in mysqld_list_processes() is holding LOCK_thread_count, but taking that for every THD::set_db() sounds quite bad for scalability (for example, set_db() is run by every query replicated by a slave thread). Should we use LOCK_thd_data also for THD::db ? Or maybe we could have a fixed buffer in THD for the db? Seems we would need 193 bytes. Then SHOW PROCESSLIST could sometimes show a random mix of old and new db name if we race, but at least we would not read invalid memory or crash due to mummap() ... Any better ideas? - Kristian. ----------------------------------------------------------------------- Original valgrind warning, for reference: rpl.rpl_stop_slave 'mix,xtradb' w5 [ fail ] Found warnings/errors in server log file! Test ended at 2013-04-22 22:05:12 line ==9897== Thread 20: ==9897== Invalid read of size 1 ==9897== at 0x4C2826B: memcpy (mc_replace_strmem.c:878) ==9897== by 0xBAF9E8: strmake_root (my_alloc.c:424) ==9897== by 0x63C74E: mysqld_list_processes(THD*, char const*, bool) (sql_class.h:771) ==9897== by 0x5C8574: mysql_execute_command(THD*) (sql_parse.cc:3149) ==9897== by 0x5CC844: mysql_parse(THD*, char*, unsigned int, Parser_state*) (sql_parse.cc:5759) ==9897== by 0x5CDDC9: dispatch_command(enum_server_command, THD*, char*, unsigned int) (sql_parse.cc:1068) ==9897== by 0x5CE4BD: do_command(THD*) (sql_parse.cc:794) ==9897== by 0x6A4E23: do_handle_one_connection(THD*) (sql_connect.cc:1266) ==9897== by 0x6A4F23: handle_one_connection (sql_connect.cc:1181) ==9897== by 0x8F0027: pfs_spawn_thread (pfs.cc:1015) ==9897== by 0x4E3506F: start_thread (in /lib64/libpthread-2.9.so) ==9897== by 0x62F713C: clone (in /lib64/libc-2.9.so) ==9897== Address 0x1a06a973 is 3 bytes inside a block of size 5 free'd ==9897== at 0x4C257D8: free (vg_replace_malloc.c:446) ==9897== by 0xBB96BB: my_free (my_malloc.c:115) ==9897== by 0x838C86: Query_log_event::do_apply_event(Relay_log_info const*, char const*, unsigned int) (sql_class.h:2846) ==9897== by 0x53904C: apply_event_and_update_pos(Log_event*, THD*, Relay_log_info*) (log_event.h:1230) ==9897== by 0x5446DD: exec_relay_log_event(THD*, Relay_log_info*) (slave.cc:2766) ==9897== by 0x545BD9: handle_slave_sql (slave.cc:3627) ==9897== by 0x8F0027: pfs_spawn_thread (pfs.cc:1015) ==9897== by 0x4E3506F: start_thread (in /lib64/libpthread-2.9.so) ==9897== by 0x62F713C: clone (in /lib64/libc-2.9.so)

4 4

Re: [Maria-developers] [Gsoc] Regex enhancements Project
by Sergei Golubchik 04 May '13

04 May '13

Hi, Sudheera! On Apr 19, Sudheera Palihakkara wrote: > Hi, > I went through other threads on this topic. In one thread you mentioned to > choose a suitable regex library. > > *( Preliminary research - only about chosing a regex library to use in > MariaDB. You should be able to explain why we should use this library and > not some other one.) > > * > What do you mean by "choosing"? don't we have to enhance the exiting regex > library? Or choose from exiting already implemented libraries which are > free to use? sorry if it's a stupid question, but I'm confused. :O Enhancing our old regex library to support all modern features and multiple charsets is complex and bug-prone work. I don't see why we should bother doing it, when there are plenty of regex libraries available. There's PHP's mb_regex, there's prce, and many others too. We'd better just pick one that works better for MariaDB, and put it instead of Henry Spencer's library. Regards, Sergei P.S. Please, don't reply to me only, use reply-to-all, so that your mails appear on the mailing list.

3 9

[Maria-developers] Fwd: MariaDB release numbering
by Jorge Manuel B. S. Vicetto 03 May '13

03 May '13

Hi. I've emailed Kristian about this subject and he suggested I sent this to your mailing list to get a more complete picture. On our work packaging MariaDB for Gentoo, me and Robin need to decide how to deal with the packaging of MySQL vs MariaDB and to that it would be really helpful to have an idea of how you plan to number your future releases. AFAIK, currently we can compare MySQL vs MariaDB as: MySQL MariaDB 5.1 5.1 / 5.2 / 5.3 5.5 5.5 (including MariaDB features from 5.2/5.3) 5.6 10.0 (based on MySQL 5.5 with backports from MySQL 5.6 and features from previous MariaDB versions) 5.7 ??? The first question for us is what version number will you use for a newer version based on MySQL-5.7. Another question is if you plan to port all your work to the 5.7 codebase or if you'll keep working on the current 10.0 codebase and do backports from 5.7 (like you did for MySQL 5.6). A second question is whether you still claim binary compatibility with MySQL[1] and how would you compare the latest versions of MariaDB and MySQL (10.0 vs 5.6, 5.7). [1] - https://kb.askmonty.org/en/mariadb-versus-mysql-compatibility/ Any info you can share on the release versioning plans would be really helpful. If this is still too early for any definitions or comments, is there any way we can follow or learn about such decisions? Thank you and regards, Jorge Manuel B. S. Vicetto Gentoo Developer

2 1

[Maria-developers] Forked mytop - conflicts, issues
by Honza Horak 03 May '13

03 May '13

Hi guys, MariaDB forked mytop utility source some time back, but it causes some issues for us in Fedora for example (and similarly in other distributions I guess) -- particularly the MariaDB's binary mytop conflicts with the same utility from separate package mytop. So, I'd like to take a look at possible solutions. Since there was a small commit in mytop upstream few weeks back, it doesn't really look like totally dead upstream. So the ideal solution from my POV would be including enhancements that MariaDB did to mytop back to upstream and not including mytop utility in MariaDB any more. Would something like that be possible from the perspective of MariaDB? Cheers, Honza

4 4

[Maria-developers] xa transaction cause replication broken
by jhx1008 02 May '13

02 May '13

hello I have another problem witch seems like a bug(Server version: 5.5.5-10.0.2-MariaDB-log MariaDB Server). the problem is about the replication for the xa transactions when the master crashed and then restarted. I know when we do a transaction, the queries will be stored in the binlog cache, after commit, flush the binlog cache into the binlog file, any normal un-committed transactions will be rollbacked when the server crashed then restarted, but the prepared xa transactions are not. Prepared xa transactions can be recovered use the command 'xa recover', then committed use 'xa commit xid', but the binlog cache is empty when the master restart, so if we do the 'xa commit xid', nothing to be recorded into the binlog files, the replication will be broken. test: master mysql> create table t(id int auto_increment primary key, a int) engine=innodb; mysql> xa start '111'; mysql> insert into t(a) values(1); mysql> xa end '111'; mysql> xa prepare '111'; kill -9 master_pid restart the master mysql> xa recover; +----------+--------------+--------------+------+ | formatID | gtrid_length | bqual_length | data | +----------+--------------+--------------+------+ | 1 | 3 | 0 | 111 | +----------+--------------+--------------+------+ mysql> xa commit '111'; mysql> select * from t; Empty set (0.00 sec) why? I did the xa commit '111', but no result return, maybe another bug? do the test continue mysql> xa start '222'; mysql> insert into t(a) values(2); mysql> xa end '222'; mysql> prepare '222'; mysql> xa commit '111'; mysql> select * from t; +----+------+ | id | a | +----+------+ | 1 | 1 | | 2 | 2 | +----+------+ value(1,1) return at this time, why? what's hanppen on the slave? mysql> select * from t; +----+------+ | id | a | +----+------+ | 2 | 2 | +----+------+ the replication is broken now!! by the way, when the xa transactions preapred, if the client exit, the transactions will be rollbacked in the THD::cleanup function, maybe the server rollback all the xa prepared transcations when restart is a good method? 2013-05-02 jhx1008 @netease

2 2

[Maria-developers] GTIDs, events sequence and alternate futures
by Pavel Ivanov 01 May '13

01 May '13

Kristian, I can't find the answer to my questions in the code and don't know if there's some documentation I can use (is there such thing?), so I'm writing my questions here. As I understand GTID is constructed as <domain id>-<server id>-<sequence number>. Is it guaranteed that sequence numbers are always increasing inside the domain id? So let's say two servers have GTID 0-1-100 and if they have alternate futures then one server will have GTIDs 0-1-101, 0-1-102 and so on, while second server will have 0-2-101, 0-2-102 etc. Is that correct? BTW, where in the code is sequence number increased exactly? Thank you, Pavel

2 4

Re: [Maria-developers] Installing jemalloc on build VMs
by Kristian Nielsen 26 Apr '13

26 Apr '13

Daniel Bartholomew <dbart(a)mariadb.org> writes: > I'm getting ready to install jemalloc on the buildslaves for TODO-459 - > https://mariadb.atlassian.net/browse/TODO-459 , and wanted to know if > there was a preferred way to do it. Since we want to use a specific > version of jemalloc (3.1.0), I figured I would build from source on all > of the build machines. Has this been thought through properly with respect to packaging? My guess is it has not. For .debs, which are what I know about, it is generally expected that one can rebuild from `apt-get source mariadb-server`. This requires proper build dependencies. It doesn't work to install random stuff on random machines. It does not appear that jemalloc 3.1 is available on most .deb distros we support. Wheezy seems to have 3.0.0. It is said that tokutek "requires" jemalloc - maybe it would be better to fix tokutek to work with standard libraries? The proper way to produce .debs with custom jemalloc requires creating new custom packages for libjemalloc and including them in our repos. - Kristian.

8 11