- developers - lists.mariadb.org

[Maria-developers] Review of MDEV-4506, parallel replication, part 3
by Michael Widenius 22 Sep '13

22 Sep '13

Hi! This is a review of the latest code pushed to your tree: bzr diff -r 3662.. Note that I will apply some of the trivial changes myself to the tree to speed up things (this is mostly about comments). === added file 'mysql-test/suite/rpl/r/rpl_parallel.result' --- mysql-test/suite/rpl/r/rpl_parallel.result 1970-01-01 00:00:00 +0000 +++ mysql-test/suite/rpl/r/rpl_parallel.result 2013-09-22 20:25:20 +0000 Great that you got these done! Will make my life much easier when I work the parallel replication code! <cut> === modified file 'sql/log.cc' --- sql/log.cc 2013-07-04 22:26:15 +0000 +++ sql/log.cc 2013-09-22 20:25:20 +0000 @@ -6542,27 +6542,93 @@ MYSQL_BIN_LOG::write_transaction_to_binl } } + +/* + Put a transaction that is ready to commit in the group commit queue. + The transaction is identified by the ENTRY object passed into this function. + + To facilitate group commit for the binlog, we first queue up ourselves in + this function. Then later the first thread to enter the queue waits for + the LOCK_log mutex, and commits for everyone in the queue once it gets the + lock. Any other threads in the queue just wait for the first one to finish + the commit and wake them up. This way, all transactions in the queue get + committed in a single disk operation. + + The return value of this function is TRUE if queued as the first entry in + the queue (meaning this is the leader), FALSE otherwise. I moved this last in the comment as: @retval TRUE If queued as the first entry in the queue (meaning this is the leader) @retval FALSE Otherwise + + The main work in this function is when the commit in one transaction has + been marked to wait for the commit of another transaction to happen + first. This is used to support in-order parallel replication, where + transactions can execute out-of-order but need to be committed in-order with + how they happened on the master. The waiting of one commit on another needs + to be integrated with the group commit queue, to ensure that the waiting + transaction can participate in the same group commit as the waited-for + transaction. + + So when we put a transaction in the queue, we check if there were other + transactions already prepared to commit but just waiting for the first one + to commit. If so, we add those to the queue as well, transitively for all + waiters. +*/ + <cut> @@ -6574,10 +6640,28 @@ MYSQL_BIN_LOG::queue_for_group_commit(gr This would be natural to do with recursion, but we want to avoid potentially unbounded recursion blowing the C stack, so we use the list approach instead. + + We keep a list of all the waiters that need to be processed in `list', + linked through the next_subsequent_commit pointer. Initially this list + contains only the entry passed into this function. + + We process entries in the list one by one. The element currently being + processed is pointed to by `cur`, and the element at the end of the list + is pointed to by `last` (we do not use NULL to terminate the list). + + As we process an element, it is first added to the group_commit_queue. + Then any waiters for that element are added at the end of the list, to + be processed in subsequent iterations. This continues until the list + is exhausted, with all elements ever added eventually processed. + + The end result is a breath-first traversal of the tree of waiters, + re-using the next_subsequent_commit pointers in place of extra stack + space in a recursive traversal. I added: The temporary list created in next_subsequent_commit is not used by the caller or any other function. @@ -6670,6 +6763,18 @@ MYSQL_BIN_LOG::queue_for_group_commit(gr { if (list->wakeup_subsequent_commits_running) { + /* + ToDo: We should not need a full lock/unlock of LOCK_wait_commit + here. All we need is a single (full) memory barrier, to ensure that + the reads of the list above are not reordered with the write of + wakeup_subsequent_commits_running, or with the writes to the list + from other threads that is allowed to happen after + wakeup_subsequent_commits_running has been set to false. + + We do not currently have explicit memory barrier primitives in the + source tree, but if we get them the below mysql_mutex_lock() could + be replaced with a full memory barrier just before the loop. + */ ok (for now). Another way would of course to only do a mutex lock for the first entry we find... No more comments. Everything looked fine! I will start coordinate with you tomorrow about things to do. Regards, Monty

1 0

[Maria-developers] Status and future of spatial extensions
by Mateusz Loskot 22 Sep '13

22 Sep '13

Hi, I haven't monitored MariaDB development closely, neither I tracked what's happening in the MySQL sandbox, so I'm hoping to catch up by asking questions here. I'm interested in the Spatial Extensions. So, I looked around the KB and the documentation and I found a few pages: general overview [1], 5.3 features [2] and page about plans [3]. I've looked into JIRA issues and recent commits too. Still, I don't see a clear big picture about the current status, roadmap and developers of the Spatial Extensions. Could anyone shed more light on the Spatial part of the MariaDB project, where it is now, where it is going, who is behind the wheel, etc.? What is (if any) the relation with geo-spatial development in MySQL? Is that correct understanding that MySQL and MariaDB follow separate paths, thus the GIS implementations go separately and in future they may seriously diverge and become no longer equivalent drop-in replacements? [1] https://mariadb.com/kb/en/gis-functionality/ [2] https://mariadb.com/kb/en/the-maria53-gis-tree-on-launchpad/ [3] https://mariadb.com/kb/en/mariadb-plans-gis/ Best regards, -- Mateusz Loskot, http://mateusz.loskot.net "Participation in this whole process is a form of torture" ~~ Szalony

1 0

Re: [Maria-developers] Rev 3818: Check for duplicate index (port from mysql)
by Sergei Golubchik 22 Sep '13

22 Sep '13

Hi, Sanja! On Sep 20, sanja(a)askmonty.org wrote: > ------------------------------------------------------------ > revno: 3818 > revision-id: sanja(a)askmonty.org-20130920193019-vr2q24c8g255aofu > parent: sanja(a)askmonty.org-20130920141533-nwk5jt07ojwmmctr > committer: sanja(a)askmonty.org > branch nick: work-maria-10.0-fts2 > timestamp: Fri 2013-09-20 22:30:19 +0300 > message: > Check for duplicate index (port from mysql) > (pre fts) ok to push Regards, Sergei

1 0

[Maria-developers] Review of MDEV-4506, parallel replication, part 2
by Michael Widenius 22 Sep '13

22 Sep '13

Hi! Part 2 of review of parallel replication === added file 'sql/rpl_parallel.cc' <cut> +pthread_handler_t +handle_rpl_parallel_thread(void *arg) +{ <cut> + if (end_of_group) + { + in_event_group= false; + Add comment: /* All events for this group has now been executed and logged (but not necessaerly synced). Inform the other event groups that are waiting for this thread that we are done. */ + rgi->commit_orderer.unregister_wait_for_prior_commit(); + thd->wait_for_commit_ptr= NULL; + + /* + Record that we have finished, so other event groups will no + longer attempt to wait for us to commit. + + We can race here with the next transactions, but that is fine, as + long as we check that we do not decrease last_committed_sub_id. If + this commit is done, then any prior commits will also have been + done and also no longer need waiting for. + */ + mysql_mutex_lock(&entry->LOCK_parallel_entry); + if (entry->last_committed_sub_id < event_gtid_sub_id) + { + entry->last_committed_sub_id= event_gtid_sub_id; + mysql_cond_broadcast(&entry->COND_parallel_entry); + } + mysql_mutex_unlock(&entry->LOCK_parallel_entry); + Add: /* Tell storage engines that they can stop waiting. See comments in sql_class.cc::wakeup_subsequent_commits2() for why we can't do this part of the wakeup in unregister_wait_for_prior_commit(). */ + rgi->commit_orderer.wakeup_subsequent_commits(); + delete rgi; + } + + events= next; + } + + mysql_mutex_lock(&rpt->LOCK_rpl_thread); + if ((events= rpt->event_queue) != NULL) + { + rpt->event_queue= rpt->last_in_queue= NULL; + mysql_mutex_unlock(&rpt->LOCK_rpl_thread); + goto more_events; + } + + if (!in_event_group) + { + rpt->current_entry= NULL; As far as I can see, rpt->free is always 0 here. - It's set to 0 at the top of this loop and only set to 1 here. Can we remove the rpt->free variable? + if (!rpt->stop && !rpt->free) + { Add comment. /* Take next group of events from the replication pool. This is faster than having to wakeup the pool manager thread to give us a new event. */ + mysql_mutex_lock(&rpt->pool->LOCK_rpl_thread_pool); + list= rpt->pool->free_list; + rpt->next= list; + rpt->pool->free_list= rpt; + if (!list) + mysql_cond_broadcast(&rpt->pool->COND_rpl_thread_pool); + mysql_mutex_unlock(&rpt->pool->LOCK_rpl_thread_pool); + rpt->free= true; + } + } + } <cut> +int +rpl_parallel_change_thread_count(rpl_parallel_thread_pool *pool, + uint32 new_count, bool skip_check) +{ + uint32 i; + rpl_parallel_thread **new_list= NULL; + rpl_parallel_thread *new_free_list= NULL; + + /* + Allocate the new list of threads up-front. + That way, if we fail half-way, we only need to free whatever we managed + to allocate, and will not be left with a half-functional thread pool. + */ + if (new_count && + !(new_list= (rpl_parallel_thread **)my_malloc(new_count*sizeof(*new_list), + MYF(MY_WME)))) + { + my_error(ER_OUTOFMEMORY, MYF(0), (int(new_count*sizeof(*new_list)))); + goto err;; + } Why not allocate new_list[] elements at the same time? Faster, less memory not allocation to check and freeing is also esier. if (!my_multi_malloc(MYF(MY_WME), &new_list, (uint) new_count*sizeof(*new_list), &list_part, new_count*sizeof(**new_list), NULL)) ... + + for (i= 0; i < new_count; ++i) + { + pthread_t th; + + if (!(new_list[i]= (rpl_parallel_thread *)my_malloc(sizeof(*(new_list[i])), + MYF(MY_WME)))) + { + my_error(ER_OUTOFMEMORY, MYF(0), sizeof(*(new_list[i]))); + goto err; + } The above would then simple be: new_list[i]= list_part++; Instead of setting everything to false and NULL, why not simple bzero() the structure? That way we can be more sure that we don't forget to initialize any new element in the future. + new_list[i]->delay_start= true; + new_list[i]->running= false; + new_list[i]->stop= false; + new_list[i]->free= false; + mysql_mutex_init(key_LOCK_rpl_thread, &new_list[i]->LOCK_rpl_thread, + MY_MUTEX_INIT_SLOW); + mysql_cond_init(key_COND_rpl_thread, &new_list[i]->COND_rpl_thread, NULL); + new_list[i]->pool= pool; + new_list[i]->current_entry= NULL; + new_list[i]->event_queue= NULL; + new_list[i]->last_in_queue= NULL; <cut> Add: /* The following loops wait until each thread is done it's work for each event group and then stops it when the thread has nothing to do. Note that this may take some time as rpl_paralell:do_event() may request threads at the same time. */ To make this faster and ensuring we don't start new request, wouldn't it be better to go through pool->threads and signal each of them? As long as rpt->stop is defined as 'stop when you are done processing one group' then this is fine (which is how it looks it's defined now). + for (i= 0; i < pool->count; ++i) + { + rpl_parallel_thread *rpt= pool->get_thread(NULL); + rpt->stop= true; + mysql_cond_signal(&rpt->COND_rpl_thread); + mysql_mutex_unlock(&rpt->LOCK_rpl_thread); + } + + for (i= 0; i < pool->count; ++i) + { + rpl_parallel_thread *rpt= pool->threads[i]; + mysql_mutex_lock(&rpt->LOCK_rpl_thread); + while (rpt->running) + mysql_cond_wait(&rpt->COND_rpl_thread, &rpt->LOCK_rpl_thread); + mysql_mutex_unlock(&rpt->LOCK_rpl_thread); + mysql_mutex_destroy(&rpt->LOCK_rpl_thread); + mysql_cond_destroy(&rpt->COND_rpl_thread); + my_free(rpt); By using my_multi_malloc() you can remove the above free + } + + my_free(pool->threads); + pool->threads= new_list; + pool->free_list= new_free_list; + pool->count= new_count; + for (i= 0; i < pool->count; ++i) + { + mysql_mutex_lock(&pool->threads[i]->LOCK_rpl_thread); + pool->threads[i]->delay_start= false; + mysql_cond_signal(&pool->threads[i]->COND_rpl_thread); + mysql_mutex_unlock(&pool->threads[i]->LOCK_rpl_thread); + } + + if (!skip_check) + { + mysql_mutex_lock(&LOCK_active_mi); + pool->changing= false; + mysql_mutex_unlock(&LOCK_active_mi); + } + return 0; + +err: + if (new_list) + { + while (new_free_list) + { + rpl_parallel_thread *next= new_free_list->next; + mysql_mutex_lock(&new_free_list->LOCK_rpl_thread); + new_free_list->delay_start= false; + new_free_list->stop= true; + mysql_cond_signal(&new_free_list->COND_rpl_thread); + while (!new_free_list->running) + mysql_cond_wait(&new_free_list->COND_rpl_thread, + &new_free_list->LOCK_rpl_thread); + while (new_free_list->running) + mysql_cond_wait(&new_free_list->COND_rpl_thread, + &new_free_list->LOCK_rpl_thread); + mysql_mutex_unlock(&new_free_list->LOCK_rpl_thread); + my_free(new_free_list); By using my_multi_malloc() you can remove the above free and even the 'next' variable + new_free_list= next; + } + my_free(new_list); + } <cut> <cut> +void +rpl_parallel::wait_for_done() +{ + struct rpl_parallel_entry *e; + uint32 i; + + for (i= 0; i < domain_hash.records; ++i) + { + e= (struct rpl_parallel_entry *)my_hash_element(&domain_hash, i); + mysql_mutex_lock(&e->LOCK_parallel_entry); + while (e->current_sub_id > e->last_committed_sub_id) + mysql_cond_wait(&e->COND_parallel_entry, &e->LOCK_parallel_entry); + mysql_mutex_unlock(&e->LOCK_parallel_entry); + } +} How to handle errors in the above case? For example when we get an error and we had to abort things? I assume we should also abort for: e->rpl_thread->thd->killed() but this is hard as thd and even rpl_thread may be deleted. One way to fix this would be to add an entry 'killed' to rpl_parell_entry that we can set. However it will be hard to keep this up to date. The other way would be to, on error, set last_committed_sub_id back to it's last value. We could also have some kind of shutdown flag that we check to avoid hang on shutdown. What do you suggest? Assuming setting the value back would be the easiest solution.. + for (i= 0; i < domain_hash.records; ++i) + { + e= (struct rpl_parallel_entry *)my_hash_element(&domain_hash, i); + mysql_mutex_lock(&e->LOCK_parallel_entry); + while (e->current_sub_id > e->last_committed_sub_id) + mysql_cond_wait(&e->COND_parallel_entry, &e->LOCK_parallel_entry); + mysql_mutex_unlock(&e->LOCK_parallel_entry); + } +bool +rpl_parallel::do_event(struct rpl_group_info *serial_rgi, Log_event *ev) +{ + rpl_parallel_entry *e; + rpl_parallel_thread *cur_thread; + rpl_parallel_thread::queued_event *qev; + struct rpl_group_info *rgi; + Relay_log_info *rli= serial_rgi->rli; + enum Log_event_type typ; + + /* ToDo: what to do with this lock?!? */ + mysql_mutex_unlock(&rli->data_lock); + + if (!(qev= (rpl_parallel_thread::queued_event *)my_malloc(sizeof(*qev), + MYF(0)))) + { + my_error(ER_OUT_OF_RESOURCES, MYF(0)); + return true; + } + qev->ev= ev; + qev->next= NULL; + + if ((typ= ev->get_type_code()) == GTID_EVENT) + { + Gtid_log_event *gtid_ev= static_cast<Gtid_log_event *>(ev); + + if (!(e= find(gtid_ev->domain_id)) || + !(rgi= new rpl_group_info(rli)) || + event_group_new_gtid(rgi, gtid_ev)) + { Here add delete of rgi. Also set rgi to 0 before. + my_error(ER_OUT_OF_RESOURCES, MYF(MY_WME)); + return true; + } + + if ((gtid_ev->flags2 & Gtid_log_event::FL_GROUP_COMMIT_ID) && + e->last_commit_id == gtid_ev->commit_id) + { + /* + We are already executing something else in this domain. But the two + event groups were committed together in the same group commit on the + master, so we can still do them in parallel here on the slave. + + However, the commit of this event must wait for the commit of the prior + event, to preserve binlog commit order and visibility across all + servers in the replication hierarchy. + */ + rpl_parallel_thread *rpt= global_rpl_thread_pool.get_thread(e); Here we could add a test if get_thread() is 0, which could happen on shutdown or when killing a replication thread. I think it's relatively safe to use get_thread() as a end-synchronize point as we should never have anything half-done when we call get_thread()). As get_thread() can take a very long time and is not called in many places, it's also sounds like a resonable thing to do. + rgi->wait_commit_sub_id= e->current_sub_id; + rgi->wait_commit_group_info= e->current_group_info; + rgi->wait_start_sub_id= e->prev_groupcommit_sub_id; + e->rpl_thread= cur_thread= rpt; + /* get_thread() returns with the LOCK_rpl_thread locked. */ Add the abvove comments also where get_thread() is defined. + } + else + { + /* Check if we already have a worker thread for this entry. */ I had a hard time to understand what the above meant. Is this new suggested comments correct? /* Get the last or current thread for the domain */ + cur_thread= e->rpl_thread; + if (cur_thread) + { /* Active thread exists. The thread may still be executing queries for this domain or it may have been rescheduled to work for another domain. */ + mysql_mutex_lock(&cur_thread->LOCK_rpl_thread); + if (cur_thread->current_entry != e) + { + /* Not ours anymore, we need to grab a new one. */ Suggested new comment: /* The thread has been reused for another domain. (e->rpl_thread was not reset or was changed during the mutex_lock) As there is no threads working on this domain, it's safe to assign a new thread to work on it */ + mysql_mutex_unlock(&cur_thread->LOCK_rpl_thread); + e->rpl_thread= cur_thread= NULL; + } + } <cut> Add here: /* Current is not set for the case where the master did not have GTID, like a MariaDB 5.5 or MySQL master. */ + else if (!Log_event::is_group_event(typ) || !current) + { + /* + Events like ROTATE and FORMAT_DESCRIPTION. Do not run in worker thread. + Same for events not preceeded by GTID (we should not see those normally, + but they might be from an old master). + */ + qev->rgi= serial_rgi; + rpt_handle_event(qev, NULL); + delete_or_keep_event_post_apply(rli, typ, qev->ev); Why is delete_or_keep_event_post_apply() not part of rpt_handle_event()? You always call these two together... + + return false; + } + else + { The following code is identical (except one line) to the code earlier in this function. Better to create a function for this to reuse code and to clarify the differences. + cur_thread= current->rpl_thread; + if (cur_thread) + { + mysql_mutex_lock(&cur_thread->LOCK_rpl_thread); + if (cur_thread->current_entry != current) + { + /* Not ours anymore, we need to grab a new one. */ + mysql_mutex_unlock(&cur_thread->LOCK_rpl_thread); + cur_thread= NULL; + } + } + if (!cur_thread) + { + cur_thread= current->rpl_thread= + global_rpl_thread_pool.get_thread(current); + } + } + qev->rgi= current->current_group_info; + + /* + Queue the event for processing. + */ + if (cur_thread->last_in_queue) + cur_thread->last_in_queue->next= qev; + else + cur_thread->event_queue= qev; + cur_thread->last_in_queue= qev; + mysql_cond_signal(&cur_thread->COND_rpl_thread); + mysql_mutex_unlock(&cur_thread->LOCK_rpl_thread); In this case it's safe to move the cond_signal outside of the mutex_unlock. (Should make things notable faster as we win a sleep-wakup in other thread). <cut> === added file 'sql/rpl_parallel.h' --- sql/rpl_parallel.h 1970-01-01 00:00:00 +0000 +++ sql/rpl_parallel.h 2013-08-29 08:18:22 +0000 @@ -0,0 +1,92 @@ +#ifndef RPL_PARALLEL_H +#define RPL_PARALLEL_H + +#include "log_event.h" + + +struct rpl_parallel; +struct rpl_parallel_entry; +struct rpl_parallel_thread_pool; + +class Relay_log_info; +struct rpl_parallel_thread { + bool delay_start; I think we can remove deley_start. Se comments from previous review. + bool running; + bool stop; We need to define 'stop' properly. Like 'stop after we have executed the current event group' (So that we don't have to do rollback). + bool free; I think we can remove free. Se comments above. Otherwise it would be good to have a good descripition for each struct in this file. Will now start reviewing your new commits to the parallel replication tree. Regards, Monty

1 0

[Maria-developers] MDEV-5019 - THREADPOOL - Create Information Schema Table for Threadpool
by Roberto Spadim 22 Sep '13

22 Sep '13

Hi guys! I openned a new MDEV https://mariadb.atlassian.net/browse/MDEV-5019 the idea is expose the threadpool group information like oracle thread pool (more information about oracle thread pool here: http://mikaelronstrom.blogspot.com.br/2011_10_01_archive.html) I can help, but i don't know how access the threadpool at scheduler.h/scheduler.c from a information schema plugin 1) lock important mutex and others mdl 2) acess variables 3) write information schema based on variables 1 and 2 i don't know how to do :( can anyone help ? :) thanks guys :) -- Roberto Spadim

2 9

[Maria-developers] Perf tuning for hot key updating with high concurrency
by Hui Liu 22 Sep '13

22 Sep '13

It's an old issue which discussed in Percona-google-group: https://groups.google.com/forum/?fromgroups#!topic/percona-discussion/z3r4-… The patch provided in that discuss group is a rough idea, based on Percona-Server, and later I almost re-wrote the patch for real-case testing. I think it's more simple to implement in the InnoDB layer, compared to the the idea post by Monty, that introduce 'Incremental lock' or 'Relative update lock' to solve this problem. I backported them to Mariadb-10.0.4, and use simple simulate test with oltp_cc.lua, which is provided as an attachment. These notes migth be helpful to understand of this patch. 1) Why InnoDB's concurrency control is BAD for high concurrency hot key updating case? It's described in the percona-disscuss-group, please refer that. 2) What's the basic logic for high concurrency hot key updating? The idea is still the same as before, but the implement is different now. With a simple case that 1000 threads updating the same record id=1 and threshold=2 to describe the basic idea for this patch. conn (1000) ... (6) -> (5) -> (4) (1) (running) (2) (suspended) (3) (suspended) That is, thread 1 is running, and thread 2 and 3 are suspended, waiting for thread 1 relase the record lock for id=1. For thread 4, it detected they are already two threads suspended, and it would be rescheduled in hash queue. conn (1000) .. (6) -> (5) (1) (running) (2) (suspended) (3) (suspended) bucket1: (4) bucket2: And for 5 and 6 ... 1000, they are also rescheduled in that queue (1) (running) (2) (suspended) (3) (suspended) bucket1: (1000)...(6) -> (5) -> (4) bucket2: After 1 is committed, 2 is waked up and running, then 4 detects it would entry now, so it enters. (1) (commit/exit) (2) (running) (3) (suspended) (4) (suspended) bucket1: (1000) ... (6) -> (5) bucket2: The same logic for 5 and 6...1000. While for the old logic, a snapshot might be: (1) (commit/exit) (2) (running) (3) (suspended) (4) (suspended) (5) (suspended) .... (1000) (suspended) Hope the demo is not too naive. You can see, in such case, there are always threshold threads running inside innodb, while not all are suspended inside. 3) Why not use the new concurrency control with AOTMIC add/sub for active conc? The result is bad for new concurrency control in 5.6/10.0, that no FIFO any more. So I changed to default (5.1/5.5) control strategy again. Note the changes in CMakeLists.txt that make -DUSE_DEFAULT_CONC=ON as default. 4) How about the concurrency counter for rescheduled thread in bucket queue? If thread number waiting for the record lock is over the configured threshold, it would be rescheduled into queue in one hash bucket, hashed by (space,space_no, heap_no) a) for the first one in the queue, that is the queue header, it would timed-wait retrying whether it would enter into innodb layer, by checking the n_ref_count for that record stored by HASH. b) for the non-head threads, they would be FAKE released the concurrent slot, that means threads in FIFO would be waked up, enter/exit innodb with faked inside and tickets. Until they are force to exit innodb, the active slot number is equal to real case without rescheduling. 5) What about the basic test result? $ ./sysbench/sysbench --test=sysbench/tests/db/oltp_cc.lua --num-threads=1000 --mysql-socket=/u01/md1004/run/mysql.sock --mysql-db=bmw --oltp-tables-count=4 --mysql-user=root --oltp-table-size=4 --max-requests=0 --max-time=120000 --oltp-read-only=off run 1000conn, oltp-cc.lua, 4x4(16 hotkey): patched (innodb_lock_wait_threshold=12 as default) : 3.1K QPS, 9.1K TPS unpatched (innodb_lock_wait_threshold=0 as disabled) : 1.2K QPS, 3.5K TPS $ ./sysbench/sysbench --test=sysbench/tests/db/oltp_cc.lua --num-threads=1000 --mysql-socket=/u01/md1004/run/mysql.sock --mysql-db=bmw --oltp-tables-count=1 --mysql-user=root --oltp-table-size=4 --max-requests=0 --max-time=120000 --oltp-read-only=off run 1000conn, oltp-cc.lua, 1x4(4 hotkey): patched (innodb_lock_wait_threshold=12 as default) : 1.3K QPS, 3.5K TPS unpatched (innodb_lock_wait_threshold=0 as disabled) : 200 QPS, 720 TPS 6) Any rest issues? The rescheduling of the waiting thread breaks the normal mechanism: each lock request must be registered in lock-table, then deadlock could be found if exists. As I am trying to reduce n in lock relaetd O(n) and O(n^2) operations, for the deadlock case, it does not work very well. For example, the update sequance: t1 updates 1, and t2 updates 1, then t3/t4/t5/t6 update 1, and then t1 updates 2, after that t2 updates 1: t1 1 2 t2 2 1 t3 1 t4 1 t5 1 t6 1 As we can see, in the new schedule model, t3/t4/t5 are suspended, and might be rescheduled in queue. Then t1 is suspended as it updates 2 which record lock is hold by t2. t2 is rescheduled as it detected too many lock waiting for rec 1. The snapshot might be: (t1[2]) (suspended) (t3) (suspended) (t4) (suspended) bucket1: (t2[1]) -> (t6) -> (t5) bucket2: The deadlock can be break until t2 enters with 1. Then for t5/t6, they would wait until timeout, and then force enter into innodb side, suspended as it should be. It's same for t2. Until t2 enters, the deadlock is found. So, the cost for deadlock detecting might be much large than it should be, depends on the timeout and retry counts. Hope Jan or Monty would have a look for this patch, or other ideas for solve this problem, if you are going.

1 0

[Maria-developers] Prep for MariaDB 5.5.33a
by Daniel Bartholomew 20 Sep '13

20 Sep '13

I've begun the prep for the special 5.5.33a release. Draft changelog and release notes are here: https://mariadb.com/kb/en/mariadb-5533a-changelog/ https://mariadb.com/kb/en/mariadb-5533a-release-notes/ As always, fixes, additions, and such are welcome. I'm currently just waiting for buildbot to finish building and then I'll package up the release so that it can get mirrored over the next few hours and we can release tomorrow after some final testing of the finished repos. Assuming things go well, my plan is to release around noon US Eastern time. Thanks. -- Daniel Bartholomew, Sr Tech Writer / Sys Admin MariaDB

2 1

Re: [Maria-developers] [Commits] Rev 3816: MDEV-4883 - Not all host_cache tests have been merged in lp:maria/10.0
by Roberto Spadim 20 Sep '13

20 Sep '13

Hi guys, what's the "MySQL variables auto-tuning code" means? i didn't saw that in mysql source and at mysql docs, any idea what this means? 2013/9/18 Sergey Vojtovich <svoj(a)mariadb.org> > At lp:maria/10.0 > > ------------------------------------------------------------ > revno: 3816 > revision-id: svoj(a)mariadb.org-20130918071716-yu2a5489uldzfuy9 > parent: sanja(a)askmonty.org-20130917190515-g3xncb1slqwy4cix > committer: Sergey Vojtovich <svoj(a)mariadb.org> > branch nick: 10.0-mdev4883 > timestamp: Wed 2013-09-18 11:17:16 +0400 > message: > MDEV-4883 - Not all host_cache tests have been merged > > Merged main.host_cache_size_functionality test. > > Test had to be adjusted because MariaDB doesn't have > MySQL variables auto-tuning code. > === added file 'mysql-test/r/host_cache_size_functionality.result' > --- a/mysql-test/r/host_cache_size_functionality.result 1970-01-01 > 00:00:00 +0000 > +++ b/mysql-test/r/host_cache_size_functionality.result 2013-09-18 > 07:17:16 +0000 > @@ -0,0 +1,78 @@ > +'#________________________VAR_06_Host_Cache_Size__________________#' > +echo '##' > +--echo '#---------------------WL6372_VAR_6_01----------------------#' > +#################################################################### > +# Checking default value # > +#################################################################### > +SELECT COUNT(@@GLOBAL.Host_Cache_Size) > +1 Expected > +set @Default_host_cache_size=128; > +select @@global.Host_Cache_Size=@Default_host_cache_size; > +@@global.Host_Cache_Size=@Default_host_cache_size > +1 > +1 Expected > +'#---------------------WL6372_VAR_6_02----------------------#' > +# Restart server with Host_Cache_Size 1 > +SELECT @@GLOBAL.Host_Cache_Size; > +@@GLOBAL.Host_Cache_Size > +1 > +1 Expected > +set @Default_host_cache_size=128; > +SET @@GLOBAL.Host_Cache_Size=DEFAULT; > +select @@global.Host_Cache_Size=@Default_host_cache_size; > +@@global.Host_Cache_Size=@Default_host_cache_size > +1 > +1 Expected > +'#---------------------WL6372_VAR_6_03----------------------#' > +SET @@local.Host_Cache_Size=1; > +ERROR HY000: Variable 'host_cache_size' is a GLOBAL variable and should > be set with SET GLOBAL > +Expected error 'Global variable' > +SET @@session.Host_Cache_Size=1; > +ERROR HY000: Variable 'host_cache_size' is a GLOBAL variable and should > be set with SET GLOBAL > +Expected error 'Global variable' > +SET @@GLOBAL.Host_Cache_Size=1; > +SET @@GLOBAL.Host_Cache_Size=DEFAULT; > +SELECT COUNT(@@GLOBAL.Host_Cache_Size); > +COUNT(@@GLOBAL.Host_Cache_Size) > +1 > +1 Expected > +select @@global.Host_Cache_Size=@Default_host_cache_size; > +@@global.Host_Cache_Size=@Default_host_cache_size > +1 > +1 Expected > +'#---------------------WL6372_VAR_6_04----------------------#' > +SELECT @@GLOBAL.Host_Cache_Size = VARIABLE_VALUE > +FROM INFORMATION_SCHEMA.GLOBAL_VARIABLES > +WHERE VARIABLE_NAME='Host_Cache_Size'; > +@@GLOBAL.Host_Cache_Size = VARIABLE_VALUE > +1 > +1 Expected > +SELECT COUNT(@@GLOBAL.Host_Cache_Size); > +COUNT(@@GLOBAL.Host_Cache_Size) > +1 > +1 Expected > +SELECT COUNT(VARIABLE_VALUE) > +FROM INFORMATION_SCHEMA.GLOBAL_VARIABLES > +WHERE VARIABLE_NAME='Host_Cache_Size'; > +COUNT(VARIABLE_VALUE) > +1 > +1 Expected > +'#---------------------WL6372_VAR_6_05----------------------#' > +SELECT @@Host_Cache_Size = @@GLOBAL.Host_Cache_Size; > +@@Host_Cache_Size = @@GLOBAL.Host_Cache_Size > +1 > +1 Expected > +SELECT COUNT(@@local.Host_Cache_Size); > +ERROR HY000: Variable 'host_cache_size' is a GLOBAL variable > +Expected error 'Variable is a GLOBAL variable' > +SELECT COUNT(@@SESSION.Host_Cache_Size); > +ERROR HY000: Variable 'host_cache_size' is a GLOBAL variable > +Expected error 'Variable is a GLOBAL variable' > +SELECT COUNT(@@GLOBAL.Host_Cache_Size); > +COUNT(@@GLOBAL.Host_Cache_Size) > +1 > +1 Expected > +SELECT Host_Cache_Size = @@SESSION.Host_Cache_Size; > +ERROR 42S22: Unknown column 'Host_Cache_Size' in 'field list' > +Expected error 'Unknown column Host_Cache_Size in field list' > +SET @@GLOBAL.Host_Cache_Size=DEFAULT; > > === added file 'mysql-test/t/host_cache_size_functionality.test' > --- a/mysql-test/t/host_cache_size_functionality.test 1970-01-01 > 00:00:00 +0000 > +++ b/mysql-test/t/host_cache_size_functionality.test 2013-09-18 > 07:17:16 +0000 > @@ -0,0 +1,185 @@ > > +############################################################################### > +# > # > +# Variable Name: Host_Cache_Size > # > +# Scope: Global > # > +# Access Type: Dynamic > # > +# Data Type: numeric > # > +# > # > +# > # > +# Creation Date: 2012-08-31 > # > +# Author : Tanjot Singh Uppal > # > +# > # > +# > # > +# Description:Test Cases of Dynamic System Variable Host_Cache_Size > # > +# that checks the behavior of this variable in the following > ways # > +# * Value Check > # > +# * Scope Check > # > +# * Functionality Check > # > +# * Accessability Check > # > +# > # > +# This test does not perform the crash recovery on this variable > # > +# For crash recovery test on default change please run the ibtest > # > > +############################################################################### > + > +-- source include/have_innodb.inc > +-- source include/not_embedded.inc > +-- source include/have_innodb_16k.inc > +--disable_warnings > +echo '#________________________VAR_06_Host_Cache_Size__________________#' > +echo '##' > +--echo '#---------------------WL6372_VAR_6_01----------------------#' > +#################################################################### > +# Checking default value # > +#################################################################### > +SELECT COUNT(@@GLOBAL.Host_Cache_Size); > +--echo 1 Expected > + > +#set @Default_host_cache_size=(select > if(if(@@global.max_connections<500,128+@@global.max_connections,128+@ > @global.max_connections+floor((@@global.max_connections-500)/20))>2000,2000,if(@@global.max_connections<500,128+@ > @global.max_connections,128+@ > @global.max_connections+floor((@@global.max_connections-500)/20)))); > +set @Default_host_cache_size=128; > + > +select @@global.Host_Cache_Size=@Default_host_cache_size; > +--echo 1 Expected > + > + > +--echo '#---------------------WL6372_VAR_6_02----------------------#' > > +################################################################################# > +# Checking the Default value post starting the server with other value > # > > +################################################################################# > +--echo # Restart server with Host_Cache_Size 1 > + > +let $restart_file= $MYSQLTEST_VARDIR/tmp/mysqld.1.expect; > +--exec echo "wait" > $restart_file > +--shutdown_server 10 > +--source include/wait_until_disconnected.inc > +-- exec echo "restart:--host_cache_size=1 " > > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect > +-- enable_reconnect > +-- source include/wait_until_connected_again.inc > + > +--disable_warnings > + > +SELECT @@GLOBAL.Host_Cache_Size; > +--echo 1 Expected > + > +#set @Default_host_cache_size=(select > if(if(@@global.max_connections<500,128+@@global.max_connections,128+@ > @global.max_connections+floor((@@global.max_connections-500)/20))>2000,2000,if(@@global.max_connections<500,128+@ > @global.max_connections,128+@ > @global.max_connections+floor((@@global.max_connections-500)/20)))); > +set @Default_host_cache_size=128; > +SET @@GLOBAL.Host_Cache_Size=DEFAULT; > +select @@global.Host_Cache_Size=@Default_host_cache_size; > +--echo 1 Expected > + > + > +--echo '#---------------------WL6372_VAR_6_03----------------------#' > +#################################################################### > +# Checking Value can be set - Dynamic # > +#################################################################### > +--error ER_GLOBAL_VARIABLE > +SET @@local.Host_Cache_Size=1; > +--echo Expected error 'Global variable' > + > +--error ER_GLOBAL_VARIABLE > +SET @@session.Host_Cache_Size=1; > +--echo Expected error 'Global variable' > + > +SET @@GLOBAL.Host_Cache_Size=1; > +SET @@GLOBAL.Host_Cache_Size=DEFAULT; > + > +SELECT COUNT(@@GLOBAL.Host_Cache_Size); > +--echo 1 Expected > + > +select @@global.Host_Cache_Size=@Default_host_cache_size; > +--echo 1 Expected > + > +--echo '#---------------------WL6372_VAR_6_04----------------------#' > +################################################################# > +# Check if the value in GLOBAL Table matches value in variable # > +################################################################# > +SELECT @@GLOBAL.Host_Cache_Size = VARIABLE_VALUE > +FROM INFORMATION_SCHEMA.GLOBAL_VARIABLES > +WHERE VARIABLE_NAME='Host_Cache_Size'; > +--echo 1 Expected > + > +SELECT COUNT(@@GLOBAL.Host_Cache_Size); > +--echo 1 Expected > + > +SELECT COUNT(VARIABLE_VALUE) > +FROM INFORMATION_SCHEMA.GLOBAL_VARIABLES > +WHERE VARIABLE_NAME='Host_Cache_Size'; > +--echo 1 Expected > + > +--echo '#---------------------WL6372_VAR_6_05----------------------#' > > +################################################################################ > +# Checking Variable Scope > # > > +################################################################################ > +SELECT @@Host_Cache_Size = @@GLOBAL.Host_Cache_Size; > +--echo 1 Expected > + > +--Error ER_INCORRECT_GLOBAL_LOCAL_VAR > +SELECT COUNT(@@local.Host_Cache_Size); > +--echo Expected error 'Variable is a GLOBAL variable' > + > +--Error ER_INCORRECT_GLOBAL_LOCAL_VAR > +SELECT COUNT(@@SESSION.Host_Cache_Size); > +--echo Expected error 'Variable is a GLOBAL variable' > + > +SELECT COUNT(@@GLOBAL.Host_Cache_Size); > +--echo 1 Expected > + > +--Error ER_BAD_FIELD_ERROR > +SELECT Host_Cache_Size = @@SESSION.Host_Cache_Size; > +--echo Expected error 'Unknown column Host_Cache_Size in field list' > + > +#The below check has been commented out as the IP fetch is different in a > P2P connection than BroadBand connection > +#--echo '#---------------------WL6372_VAR_6_06----------------------#' > > +############################################################################### > +# Checking the Host cahce functionality > # > > +############################################################################### > + > +#SET @@GLOBAL.Host_Cache_Size=2; > +#--disable_warnings > + > +#--perl > +#my $ip=`ifconfig | egrep "inet addr|inet" | sed -e 's/^.*inet addr://' > -e 's/^.*inet//'| sed 's/ .*\$//'|egrep -i "broadcast|bcast"|head -1|awk > '{print $1}'`; > +#open (LOGFH, ">" . $ENV{'MYSQL_TMP_DIR'} . "/bind_ip"); > +#print LOGFH "let \$bind_ip = $ip;\n"; > +#close LOGFH; > +#EOF > + > +#--source $MYSQL_TMP_DIR/bind_ip > +#--remove_file $MYSQL_TMP_DIR/bind_ip > + > +#let $restart_file= $MYSQLTEST_VARDIR/tmp/mysqld.1.expect; > +#--exec echo "wait" > $restart_file > +#--shutdown_server 10 > +#--source include/wait_until_disconnected.inc > +#-- exec echo "restart:--bind-address=$bind_ip " > > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect > +#-- enable_reconnect > +#-- source include/wait_until_connected_again.inc > + > +#connection default; > +#--disable_warnings > + > +#create user binduser; > +#grant all on *.* to binduser; > + > +#select count(IP) from performance_schema.host_cache; > +#--echo 0 Expected > + > +#connect (con1,$bind_ip,binduser,,); > +#select count(IP) from performance_schema.host_cache; > +#--echo 1 Expected > + > +#disconnect con1; > +#connection default; > + > +#--disable_warnings > + > +# The below check is hashed until the BUG14689561# is fixed > +# > +#flush hosts; > + > +#connect (con2,$bind_ip,binduser,,); > +#select count(IP) from performance_schema.host_cache; > +#--echo 1 Expected > + > +#disconnect con2; > + > +SET @@GLOBAL.Host_Cache_Size=DEFAULT; > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits > -- Roberto Spadim SPAEmpresarial

2 1

Re: [Maria-developers] Rev 3814: Client attributes
by Sergei Golubchik 19 Sep '13

19 Sep '13

Hi, Sanja! On Sep 19, sanja(a)askmonty.org wrote: > === modified file 'sql/sql_acl.cc' > --- a/sql/sql_acl.cc 2013-07-21 14:39:19 +0000 > +++ b/sql/sql_acl.cc 2013-09-19 10:19:36 +0000 > @@ -4615,7 +4615,8 @@ bool check_grant(THD *thd, ulong want_ac > case ACL_INTERNAL_ACCESS_GRANTED: > /* > Currently, > - - the information_schema does not subclass ACL_internal_table_access, > + - the information_schema does not subclass ACL_internal_table_access > +, Eh? What was that? > there are no per table privilege checks for I_S, > - the performance schema does use per tables checks, but at most > returns 'CHECK_GRANT', and never 'ACCESS_GRANTED'. > @@ -8222,6 +8223,43 @@ static bool find_mpvio_user(MPVIO_EXT *m > mpvio->acl_user->plugin.str)); > DBUG_RETURN(0); > } > + > +static bool > +read_client_connect_attrs(char **ptr, size_t *max_bytes_available, > + const CHARSET_INFO *from_cs) > +{ > + size_t length, length_length; > + char *ptr_save; > + /* not enough bytes to hold the length */ > + if (*max_bytes_available < 1) > + return true; > + > + /* read the length */ > + ptr_save= *ptr; > + length= net_field_length_ll((uchar **) ptr); > + length_length= *ptr - ptr_save; > + if (*max_bytes_available < length_length) > + return true; This is incorrect, it might read past the end of the buffer. it's too late to check the length *after* you've read the data. Correct code might look like if (*max_bytes_available >= 9) { ptr_save= *ptr; length= net_field_length_ll((uchar **) ptr); length_length= *ptr - ptr_save; DBUG_ASSERT(length_length <= 9); *max_bytes_available-= length_length; } else { char buf[10], *len_ptr= buf; strncpy(buf, *ptr, 9) length= net_field_length_ll((uchar **) &len_ptr); length_length= len_ptr - buf; *ptr+= length_length; if (*max_bytes_available < length_length) return true; *max_bytes_available-= length_length; } > + > + *max_bytes_available-= length_length; > + > + /* length says there're more data than can fit into the packet */ > + if (length > *max_bytes_available) > + return true; > + > + /* impose an artificial length limit of 64k */ > + if (length > 65535) > + return true; > + > +#ifdef HAVE_PSI_THREAD_INTERFACE > + if (PSI_THREAD_CALL(set_thread_connect_attrs)(*ptr, length, from_cs) && > + current_thd->variables.log_warnings) > + sql_print_warning("Connection attributes of length %lu were truncated", > + (unsigned long) length); > +#endif > + return false; > +} > + > #endif > > /* the packet format is described in send_change_user_packet() */ > @@ -8349,6 +8387,13 @@ static bool parse_com_change_user_packet > } > } > > + size_t bytes_remaining_in_packet= end - ptr; 1. This is very wrong, did you ever test it? You need a pointer to *after* the plugin name. But 'ptr' is two bytes *before* the plugin. 2. remove bytes_remaining_in_packet, pass 'end' instead to read_client_connect_attrs(). > + > + if ((mpvio->thd->client_capabilities & CLIENT_CONNECT_ATTRS) && > + read_client_connect_attrs(&ptr, &bytes_remaining_in_packet, > + mpvio->thd->charset())) > + return packet_error; > + > DBUG_PRINT("info", ("client_plugin=%s, restart", client_plugin)); > /* > Remember the data part of the packet, to present it to plugin in > @@ -8487,7 +8532,21 @@ static ulong parse_client_handshake_pack > /* strlen() can't be easily deleted without changing protocol */ > db_len= db ? strlen(db) : 0; > > - char *client_plugin= passwd + passwd_len + (db ? db_len + 1 : 0); > + char *client_plugin= end= passwd + passwd_len + (db ? db_len + 1 : 0); > + > + if ((thd->client_capabilities & CLIENT_PLUGIN_AUTH) && > + (client_plugin < (char *)net->read_pos + pkt_len)) > + { > + end+= strlen(end) + 1; > + } > + > + size_t bytes_remaining_in_packet= (((char *)net->read_pos) + pkt_len) - end; This is very-very wrong. if there're no connection attributes, then your 'end' will be after the end of the packet, it'll be after net->read_pos + pkt_len. But then you subtract, get -1 and cast it to unsigned size_t ! Don't worry, when you remove bytes_remaining_in_packet as I wrote above, this bug will go away. > + > + if (db && > + (thd->client_capabilities & CLIENT_CONNECT_ATTRS) && > + read_client_connect_attrs(&end, &bytes_remaining_in_packet, > + mpvio->thd->charset())) > + return packet_error; Put it in the same place as in parse_com_change_user_packet, that is after reading the plugin name (and fix_plugin_ptr). And rename 'end' to 'connection_attributes' > > /* Since 4.1 all database names are stored in utf8 */ > if (db) > > === modified file 'sql/sys_vars.cc' > --- a/sql/sys_vars.cc 2013-08-14 08:48:50 +0000 > +++ b/sql/sys_vars.cc 2013-09-19 10:19:36 +0000 > @@ -71,6 +71,10 @@ > #define export /* not static */ > > #ifdef WITH_PERFSCHEMA_STORAGE_ENGINE > +#ifndef EMBEDDED_LIBRARY > +#define PFS_TRAILING_PROPERTIES \ > + NO_MUTEX_GUARD, NOT_IN_BINLOG, ON_CHECK(NULL), ON_UPDATE(NULL), \ > + NULL, sys_var::PARSE_EARLY Remove PFS_TRAILING_PROPERTIES define (not used in MariaDB) > > static Sys_var_mybool Sys_pfs_enabled( > "performance_schema", > @@ -331,8 +335,10 @@ static Sys_var_long Sys_pfs_connect_attr > GLOBAL_VAR(pfs_param.m_session_connect_attrs_sizing), > CMD_LINE(REQUIRED_ARG), VALID_RANGE(-1, 1024 * 1024), > DEFAULT(-1), > - BLOCK_SIZE(1)); > + BLOCK_SIZE(1), > + NO_MUTEX_GUARD, NOT_IN_BINLOG); Remove this change too (sysvars are NO_MUTEX_GUARD and NOT_IN_BINLOG by default) > > +#endif /* EMBEDDED_LIBRARY */ > #endif /* WITH_PERFSCHEMA_STORAGE_ENGINE */ > > static Sys_var_ulong Sys_auto_increment_increment( > > === modified file 'libmysqld/lib_sql.cc' > --- a/libmysqld/lib_sql.cc 2013-08-01 12:09:26 +0000 > +++ b/libmysqld/lib_sql.cc 2013-09-19 10:19:36 +0000 > @@ -698,12 +698,38 @@ err: > } > > > +static void > +emb_transfer_connect_attrs(MYSQL *mysql) > +{ > +#ifdef HAVE_PSI_THREAD_INTERFACE > + if (mysql->options.extension && > + mysql->options.extension->connection_attributes_length) > + { > + uchar *buf, *ptr; > + THD *thd= (THD*)mysql->thd; > + size_t length= mysql->options.extension->connection_attributes_length; > + > + /* 9 = max length of the serialized length */ > + ptr= buf= (uchar *) my_alloca(length + 9); > + send_client_connect_attrs(mysql, buf); > + net_field_length_ll(&ptr); > + PSI_THREAD_CALL(set_thread_connect_attrs)((char *) ptr, length, thd->charset()); > + my_afree(buf); > + } > +#endif > +} > + > + > #ifdef NO_EMBEDDED_ACCESS_CHECKS > int check_embedded_connection(MYSQL *mysql, const char *db) > { > int result; > LEX_STRING db_str = { (char*)db, db ? strlen(db) : 0 }; > THD *thd= (THD*)mysql->thd; > + > + /* the server does the same as the client */ > + mysql->server_capabilities= mysql->client_flag; > + Remove the similar line in init_embedded_mysql() > thd_init_client_charset(thd, mysql->charset->number); > thd->update_charset(); > Security_context *sctx= thd->security_ctx; > @@ -728,11 +755,17 @@ int check_embedded_connection(MYSQL *mys > we emulate a COM_CHANGE_USER user here, > it's easier than to emulate the complete 3-way handshake > */ > - char buf[USERNAME_LENGTH + SCRAMBLE_LENGTH + 1 + 2*NAME_LEN + 2], *end; > + char *buf, *end; > NET *net= &mysql->net; > THD *thd= (THD*)mysql->thd; > Security_context *sctx= thd->security_ctx; > + size_t connect_attrs_len= > + (mysql->server_capabilities & CLIENT_CONNECT_ATTRS && This doesn't seem to make any sense. Why does it check for CLIENT_CONNECT_ATTRS if it's part of the server (embedded). If a client connects to the server, then the client needs to check whether the server supports CLIENT_CONNECT_ATTRS, and the server needs to check whether the client supports it. But a server doesn't need to check whether it itself support it, it knows it already :) > + mysql->options.extension) ? > + mysql->options.extension->connection_attributes_length : 0; > > + buf= my_alloca(USERNAME_LENGTH + SCRAMBLE_LENGTH + 1 + 2*NAME_LEN + 2 + > + connect_attrs_len + 2); > if (mysql->options.client_ip) > { > sctx->host= my_strdup(mysql->options.client_ip, MYF(0)); > @@ -766,6 +799,13 @@ int check_embedded_connection(MYSQL *mys > int2store(end, (ushort) mysql->charset->number); > end+= 2; > > + //end= strmake(end, "mysql_native_password", NAME_LEN) + 1; Why is it commented out? > + > + /* the server does the same as the client */ > + mysql->server_capabilities= mysql->client_flag; > + > + end= (char *) send_client_connect_attrs(mysql, (uchar *) end); > + > /* acl_authenticate() takes the data from thd->net->read_pos */ > thd->net.read_pos= (uchar*)buf; Regards, Sergei

1 1

[Maria-developers] Please review: EXPLAIN UPDATE/DELETE, post-review changes.
by Sergei Petrunia 17 Sep '13

17 Sep '13

Hello Igor, I think I'm ready to submit the EXPLAIN UPDATE/DELETE code refactoring patch for another round of review. The code is pushed to lp:~~maria-captains/maria/10.0-base-explain-update-r2/ . I am also attaching the combined patch (against 10.0-base). BR Sergei -- Sergei Petrunia, Software Developer MariaDB | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] BUG? at windows versino of mariadb 10.0.4
by Roberto Spadim 16 Sep '13

16 Sep '13

Hi guys, i was trying the mariadb with lower value of thread_pool_max_threads to 3 and after my fourth connection i got a error and server crashed, it's a new server, i downloaded the 32bits version of mariadb, unziped and run this command line: C:\Program Files\mariadb-10.0.4-win32\bin>mysqld --datadir=..\data\ --log_error=..\log --port=3306 --thread_pool_max_threads=3 the error output at log was: 130915 21:43:37 [ERROR] mysqld got exception 0x80000003 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. To report this bug, see http://kb.askmonty.org/en/reporting-bugs We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. Server version: 10.0.4-MariaDB key_buffer_size=134217728 read_buffer_size=131072 max_used_connections=0 max_threads=4 thread_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 141750 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x0x0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... mysqld.exe!os_aio_windows_handle()[os0file.cc:4786] mysqld.exe!fil_aio_wait()[fil0fil.cc:5396] mysqld.exe!io_handler_thread()[srv0start.cc:477] kernel32.dll!BaseThreadInitThunk() ntdll.dll!RtlInitializeExceptionChain() ntdll.dll!RtlInitializeExceptionChain() The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. any help is wellcome, maybe the max_threads is bigger than thread_pool_max_threads? that's could be the problem? i didn't checked the source yet -- Roberto Spadim SPAEmpresarial

2 2

Re: [Maria-developers] review of mdev-4911
by Roberto Spadim 16 Sep '13

16 Sep '13

Hi! MDEV Created 1. MDEV-5018 <https://mariadb.atlassian.net/browse/MDEV-5018> 2013/9/14 Roberto Spadim <roberto(a)spadim.com.br> > Hi sergey :) well 10.1 with kill and subquery :) nice feature :) maybe too > much, but very nice > > > Em 14/09/2013 13:58, "Sergei Golubchik" <serg(a)mariadb.org> escreveu: > > Hi, Roberto! >> >> On Sep 13, Roberto Spadim wrote: >> > >> > > My preference is to kill connection by thread_id and query by >> > > query_id, because I normally either want to stop a particular query, >> > > or stop all activity in particular connection. But it is >> > > incompatible change. >> > > >> > there's mysql work with this kind of syntax? i didn't found it at >> bug.mysql >> > at mail list, Justin at Percona, talked about patchs in others forks >> maybe >> > a single unique syntax is better than maridb only syntax, check message >> > from MDEV description (copied from mail list): >> > >> > Justin Swanhartgreenlion(a)gmail.com Percona, Inc >> > >> > KILL THREAD THREAD_ID WITH QUERY_ID QUERY_ID >> > and >> > KILL QUERY THREAD_ID WITH QUERY_ID QUERY_ID >> > and possibly >> > KILL QUERY WITH QUERY_ID >> > >> > should be supported. This is a very important and missing feature >> > which is included in other forks. >> >> It's not in MySQL 5.6 and not in Percona Server 5.5 or 5.6. >> I did not check Google patches, Facebook patches and other sets of MySQL >> patches, though. >> > i didn`t found too :/ > >> >> > my problem isn't the program allowing a kill command since it can >> restart >> > the work or stop, it's not a problem >> > the "problem" is the boring time lost at a wrong kill command, since i >> use >> > persistent connections at php, and a thread running a script can be >> used in >> > another script without changing it thread_id (can be confirmed at show >> > processlist) >> > my problem is sending a kill command to the wrong thread since i'm using >> > the thread_id to kill the connection and not the query_id, check i use >> > "kill connection xxx" not "kill query xxx" >> >> Okay... This makes sense. If you use a connection pool that, indeed, >> connection id does not correspond to a logical connection. >> >> Still, while KILL CONNECTION QUERY_ID is kind of ok, KILL QUERY QUERY_ID >> is very silly. I'd rather allow subqueries in KILL, to support >> > kill query query_id is very ugly hehehe, kill query id is ok, and kill > connection query_id or query id with space is ok too > > >> >> KILL CONNECTION (select thread_id from information_schema.processlist >> where ...) >> >> then you won't need to kill by query id or state or if_idle - you can >> have everything in the where clause. >> >> but this is a larger change that we cannot do 10.0, we simply don't have >> time for it. We could try to do it in 10.1 though, properly and >> generally, so that you can kill using as complex conditions as you like. >> Instead of creating many limited shortcut syntax variants for special >> cases. >> > yeah =) in future we can go back and do some job here > > >> >> > i'm not using the threadpool yet and i don't know how processlist is >> > reported with thread pool, is the id isn't unique in this case (using >> > threadpool)? >> >> Unique. Every connection has its own connection id, with or without >> thread pool. Internal scheduling implementation doesn't affect >> user-visible connection ids. >> > nice i will test thread pool and check how it works > > >> >> > i don't know if it's what mariadb/mysql should do inside code when using >> > threadpool, i'm using only one process per connection, and don't have >> this >> > kind of problem >> >> threadpool doesn't have this problem either. >> >> > other doubt now... when we have a daemon process (plugin) there's a >> > query id for it? >> >> No. >> >> > query_id=0? in this case we only have an "unique query id" with >> > thread_id+query_id? maybe we should avoid the KILL QUERY_ID = 0 >> >> Right, thanks. >> >> > again about sintax... maybe a WHERE could be added to KILL instead of a >> > DELETE FROM INFORMATION_SCHEMA... >> > >> > KILL [CONNECTION | QUERY] [WHERE some_fields some_operators some_values >> > and_no_subquery | <thread_id> | QUERY_ID <query_id>] >> > >> > about WHERE, we could use the same fields of show processlist: >> > >> > ID, USER, HOST, DB, COMMAND, TIME, STATE, INFO, TIME_MS, STAGE, >> > MAX_STAGE, PROGRESS, MEMORY_USED, EXAMINED_ROWS, QUERY_ID >> >> I'd rather allow subqueries there, it'll be much more natural. >> > yes maybe information schema engine, or kill subquery? > > >> >> Regards, >> Sergei >> > > bye :) > -- Roberto Spadim SPAEmpresarial

1 0

[Maria-developers] Review of patch for MDEV-4820
by Kristian Nielsen 16 Sep '13

16 Sep '13

I took a close look at your patch for MDEV-4820. I think there is a fundamental disconnect. In MariaDB GTID, I do not require or rely on monotonically increasing seqeunce numbers (monoticity is requred per-server-id, but not between different servers). Nor do I enforce or rely on absence of holes in the sequence numbers. This decision was a hard one to make and I spent considerable thought on this point quite early. It is true that this design reduces possibilities to detect some kinds of errors, like missing events and alternate futures. I can understand if this design is not optimal for what you are trying to do. However, implementing two different designs (eg. based on value of gtid_strict_mode) is not workable. I believe at least for the first version, the current design is what we have to work with. The gtid_strict_mode is about helping the user by giving an error instead of logging certain undeirable event sequences to the binlog. It should not cause different non-error behaviour ever, nor should it prevent working with binlogs that were logged when strict mode was not enabled. > diff --git a/sql/rpl_gtid.cc b/sql/rpl_gtid.cc > --- a/sql/rpl_gtid.cc > +++ b/sql/rpl_gtid.cc > @@ -993,7 +993,7 @@ rpl_binlog_state::check_strict_sequence(uint32 domain_id, uint32 server_id, > > if ((elem= (element *)my_hash_search(&hash, > (const uchar *)(&domain_id), 0)) && > - elem->last_gtid && elem->last_gtid->seq_no >= seq_no) > + elem->last_gtid && elem->last_gtid->seq_no + 1 != seq_no) If I understand this part correctly, this is about enforcing that there are no holes in sequence numbers. This goes against the GTID design. > @@ -854,8 +857,16 @@ contains_all_slave_gtid(slave_connection_state *st, Gtid_list_log_event *glev) > */ > return false; > } > - if (gtid->server_id == glev->list[i].server_id && > - gtid->seq_no <= glev->list[i].seq_no) > + if (slave_gtid_strict_mode) > + { > + if (gtid->seq_no < glev->list[i].seq_no) > + { > + has_greater_seq_no= true; > + break; > + } > + } I think here you want to depend on sequence numbers being monotonic. This does not work, as you cannot be sure that strict mode was enabled when this binlog file was written. > - if (!rpl_global_gtid_slave_state.domain_to_gtid(slave_gtid->domain_id, > - &master_replication_gtid) || > + bool master_repl_gtid_found = > + rpl_global_gtid_slave_state.domain_to_gtid(slave_gtid->domain_id, > + &master_replication_gtid); > + if (!master_repl_gtid_found || > slave_gtid->server_id != master_replication_gtid.server_id || > slave_gtid->seq_no != master_replication_gtid.seq_no) > { Right, so slave requests something in a domain that we do not have anything about in the binlog, but we do have something in the slave state (but not the correct one). So this is a bug in the current code. If slave requests to start at a position in a domain we have nothing in, either that domain is served by a different master, or slave wants to start from the very first event in that domain. Currently, we fail to check in the second case that we actually have the correct GTID as the first event. But this is the wrong way to fix it. A master should not be prevented from serving some domain D1 just because it has some old junk for another domain D2 in the slave state. Instead, when we allow to start in a domain that is missing, and then later actually encounter an event in this domain, we should check that it matches what the slave originally requested. If not, we should give an error. I'll try to find time to do this. > @@ -1191,6 +1239,16 @@ gtid_find_binlog_file(slave_connection_state *state, char *out_name, > */ > state->remove(gtid); > } > + else if (slave_gtid_strict_mode && > + gtid->seq_no == glev->list[i].seq_no) > + { > + *errormsg= "Requested slave GTID state have different server id " > + "than the one in binlog"; > + *error_gtid= *gtid; > + *good_gtid= glev->list[i]; > + error= ER_SLAVE_IS_FROM_ALTERNATE_FUTURE; > + goto end; > + } This can fail if the binlog file was written when strict mode was not enabled. > > @@ -1559,7 +1620,39 @@ send_event_to_slave(THD *thd, NET *net, String* const packet, ushort flags, > event_gtid.seq_no <= gtid->seq_no) > *gtid_skip_group = (flags2 & Gtid_log_event::FL_STANDALONE ? > GTID_SKIP_STANDALONE : GTID_SKIP_TRANSACTION); > - if (event_gtid.server_id == gtid->server_id && > + if (slave_gtid_strict_mode) > + { > + if (event_gtid.seq_no == gtid->seq_no && > + event_gtid.server_id != gtid->server_id) > + { > + my_errno= ER_SLAVE_IS_FROM_ALTERNATE_FUTURE; > + *error_gtid= *gtid; > + *good_gtid= event_gtid; > + return "Requested slave GTID state have different server id " > + "than the one in binlog"; > + } Again, what if this is an old part of the binlog where strict mode was disabled? > + else if (*first_event && event_gtid.seq_no > gtid->seq_no) > + { > + /* > + Earlier we decided that this binlog file has event that we need, > + but the first event in the file has seq_no beyond what we need. > + This can happen in two cases: > + - this file is the first binlog we have and Gtid_list event > + in it is empty; > + - there is a gap between the last event mentioned in Gtid_list > + and the first event in the binlog file. > + As the first reason is the most probable one let's issue the > + appropriate error message. > + */ > + my_errno= ER_MASTER_FATAL_ERROR_READING_BINLOG; > + *error_gtid= *gtid; > + return "Could not find GTID state requested by slave in any " > + "binlog files. Probably the slave state is too old and " > + "required binlog files have been purged"; > + } > + } This "first_event" (which I admit I do not fully understand) looks wrong - all such things usually need to be per-domain. However, I think this is the place in the code I mention above, where we get an event in a domain that the slave requested, but we did not have anything at connect time. So here I want an even stricter check, and also in non-strict mode: The first GTID in the domain must be _equal_ to what the slave requested, or we should fail. ---- Next I will take a closer look at your test cases and see what can be done to fix it in a way compatible with the basic design. I also want to re-iterate the suggestion to not delete binlogs when cloning a master to a new master. The most recent binlog file has real information (in the initial Gtid_list event) that is needed for proper operation as a master, and it really doesn't make sense to first purposely corrupt the server state and subsequently try to hack the code to fix the damage. If one really wanted not to copy any binlog files, then a better idea would be a way to restore the state, eg. a RESET MASTER TO gtid_list='xxx,yyy,...' command or something like that. Hope this helps, - Kristian.

3 23

[Maria-developers] review of mdev-4911
by Sergei Golubchik 14 Sep '13

14 Sep '13

Hi. Good work. I only have one small comment about the code, see below. And one about the syntax. > === modified file 'sql/sql_parse.cc' > --- sql/sql_parse.cc 2013-08-14 08:48:50 +0000 > +++ sql/sql_parse.cc 2013-09-11 12:29:32 +0000 > @@ -7139,24 +7139,60 @@ THD *find_thread_by_id(ulong id) > > > /** > - kill on thread. > + Find a thread by query id and return it, locking it LOCK_thd_data > + > + @param id Identifier of the query we're looking for > + > + @return NULL - not found > + pointer - thread found, and its LOCK_thd_data is locked. > +*/ > + > +static THD *find_thread_by_query_id(longlong id) why didn't you reuse find_thread_by_id()? this function is almost identical to it. > +{ > + THD *tmp; > + mysql_mutex_lock(&LOCK_thread_count); // For unlink from list > + I_List_iterator<THD> it(threads); > + while ((tmp=it++)) > + { > + if (tmp->get_command() == COM_DAEMON) > + continue; > + if (tmp->query_id == id) > + { > + mysql_mutex_lock(&tmp->LOCK_thd_data); // Lock from delete > + break; > + } > + } > + mysql_mutex_unlock(&LOCK_thread_count); > + return tmp; > +} > + > + > === modified file 'sql/sql_yacc.yy' > --- sql/sql_yacc.yy 2013-08-13 11:35:36 +0000 > +++ sql/sql_yacc.yy 2013-09-11 12:29:32 +0000 > @@ -12890,6 +12891,11 @@ kill_expr: > Lex->users_list.push_back($2); > Lex->kill_type= KILL_TYPE_USER; > } > + | ID_SYM expr > + { > + Lex->value_list.push_front($2); > + Lex->kill_type= KILL_TYPE_QUERY; > + } So, you implemented KILL [ CONNECTION | QUERY ] [ ID ] expr It allows, in particular KILL CONNECTION ID 10 which looks strange, why would that mean that 10 is a query id? I originally suggested KILL [ CONNECTION | QUERY [ ID ] ] expr to allow only KILL CONNECTION expr KILL QUERY expr KILL QUERY ID expr because in this case it's quite clear "QUERY ID" and because I thought it's a bit strange to kill a connection by query_id. If you want to allow that (as I saw in the comments, Roberto didn't thought it's strange to kill a connection by query id), may be you'd better use KILL [ CONNECTION | QUERY ] [ QUERY_ID ] expr ? Regards, Sergei

3 5

[Maria-developers] Can't sign up to JIRA
by Kouhei Sutou 13 Sep '13

13 Sep '13

Hi, I tried to sign up to JIRA(*) for submitting a bug report. (*) https://mariadb.atlassian.net/secure/Signup!default.jspa But I couldn't sign up because I couldn't get captcha image. I used "Chromium 28.0.1500.95 Debian jessie/sid (213514)" and "Iceweasel 7.0.8" on Debian GNU/Linux sid as Web browsers. I got the same result with these Web browsers. I checked captcha image response by Chromium's developer tool. https://mariadb.atlassian.net/captcha returns "500 Internal Server Error" with the following body. Could you confirm it? ---- We had some technical problems during your last operation. We deeply apologize for that :( If you need further assistance, please: Expand the section "Technical details" below Copy the content from it Send it to your JIRA Administrator Technical details Cause Referer URL: https://mariadb.atlassian.net/secure/Signup!default.jspa java.lang.NoClassDefFoundError: Could not initialize class com.atlassian.jira.servlet.NoOpImageCaptchaService java.lang.NoClassDefFoundError: Could not initialize class com.atlassian.jira.servlet.NoOpImageCaptchaService at com.atlassian.jira.servlet.NoOpCaptchaServiceImpl.getImageCaptchaService(NoOpCaptchaServiceImpl.java:14) at com.atlassian.jira.servlet.JiraCaptchaServlet.doGet(JiraCaptchaServlet.java:50) at javax.servlet.http.HttpServlet.service(HttpServlet.java:617) at javax.servlet.http.HttpServlet.service(HttpServlet.java:717) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at com.atlassian.jira.web.filters.steps.ChainedFilterStepRunner.doFilter(ChainedFilterStepRunner.java:87) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at com.atlassian.jira.web.filters.XContentTypeOptionsNoSniffFilter.doFilter(XContentTypeOptionsNoSniffFilter.java:22) at com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at com.atlassian.core.filters.HeaderSanitisingFilter.doFilter(HeaderSanitisingFilter.java:44) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:46) at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:66) at com.atlassian.jira.tzdetect.IncludeResourcesFilter.doFilter(IncludeResourcesFilter.java:39) at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:74) at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42) at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:66) at com.atlassian.jira.baseurl.IncludeResourcesFilter.doFilter(IncludeResourcesFilter.java:38) at com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31) at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:74) at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42) at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:66) at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:25) at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:74) at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42) at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:66) at com.atlassian.mywork.client.filter.ServingRequestsFilter.doFilter(ServingRequestsFilter.java:37) at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:74) at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42) at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:66) at com.atlassian.studio.core.servlet.filter.AlacarteLicenseEnforcer.doFilter(AlacarteLicenseEnforcer.java:71) at com.atlassian.studio.jira.servlet.filter.AlacarteLicenseJiraEnforcer.doFilter(AlacarteLicenseJiraEnforcer.java:121) at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:74) at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42) at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:66) at com.atlassian.prettyurls.filter.PrettyUrlsDispatcherFilter.doFilter(PrettyUrlsDispatcherFilter.java:60) at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:74) at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42) at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:66) at com.atlassian.prettyurls.filter.PrettyUrlsSiteMeshFilter.doFilter(PrettyUrlsSiteMeshFilter.java:92) at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:74) at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42) at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:66) at com.atlassian.prettyurls.filter.PrettyUrlsMatcherFilter.doFilter(PrettyUrlsMatcherFilter.java:56) at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:74) at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42) at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:66) at com.atlassian.labs.botkiller.BotKillerFilter.doFilter(BotKillerFilter.java:36) at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:74) at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42) at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:77) at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:63) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at com.atlassian.jira.web.filters.accesslog.AccessLogFilter.executeRequest(AccessLogFilter.java:103) at com.atlassian.jira.web.filters.accesslog.AccessLogFilter.doFilter(AccessLogFilter.java:87) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at com.atlassian.jira.security.xsrf.XsrfTokenAdditionRequestFilter.doFilter(XsrfTokenAdditionRequestFilter.java:54) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at com.atlassian.jira.web.filters.steps.ChainedFilterStepRunner.doFilter(ChainedFilterStepRunner.java:87) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:46) at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:66) at com.atlassian.plugin.remotable.plugin.module.permission.ApiScopingFilter.doFilter(ApiScopingFilter.java:63) at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:74) at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42) at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:66) at com.atlassian.prettyurls.filter.PrettyUrlsCombinedMatchDispatcherFilter.doFilter(PrettyUrlsCombinedMatchDispatcherFilter.java:61) at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:74) at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42) at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:77) at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:63) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at com.atlassian.seraph.filter.SecurityFilter.doFilter(SecurityFilter.java:234) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at com.atlassian.security.auth.trustedapps.filter.TrustedApplicationsFilter.doFilter(TrustedApplicationsFilter.java:98) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at com.atlassian.seraph.filter.BaseLoginFilter.doFilter(BaseLoginFilter.java:169) at com.atlassian.jira.web.filters.JiraLoginFilter.doFilter(JiraLoginFilter.java:70) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:46) at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:66) at com.atlassian.oauth.serviceprovider.internal.servlet.OAuthFilter.doFilter(OAuthFilter.java:55) at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:74) at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42) at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:66) at com.atlassian.prettyurls.filter.PrettyUrlsCombinedMatchDispatcherFilter.doFilter(PrettyUrlsCombinedMatchDispatcherFilter.java:61) at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:74) at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42) at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:77) at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:63) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.tuckey.web.filters.urlrewrite.RuleChain.handleRewrite(RuleChain.java:176) at org.tuckey.web.filters.urlrewrite.RuleChain.doRules(RuleChain.java:145) at org.tuckey.web.filters.urlrewrite.UrlRewriter.processRequest(UrlRewriter.java:92) at org.tuckey.web.filters.urlrewrite.UrlRewriteFilter.doFilter(UrlRewriteFilter.java:394) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at com.atlassian.gzipfilter.GzipFilter.doFilterInternal(GzipFilter.java:80) at com.atlassian.gzipfilter.GzipFilter.doFilter(GzipFilter.java:51) at com.atlassian.jira.web.filters.gzip.JiraGzipFilter.doFilter(JiraGzipFilter.java:55) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:46) at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:66) at com.atlassian.plugin.remotable.plugin.module.oauth.OAuth2LOFilter.doFilter(OAuth2LOFilter.java:81) at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:74) at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42) at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:66) at com.atlassian.plugin.remotable.host.common.service.http.bigpipe.BigPipeRequestIdFilter.doFilter(BigPipeRequestIdFilter.java:33) at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:74) at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42) at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:66) at com.atlassian.analytics.client.filter.AnalyticsFilter.doFilter(AnalyticsFilter.java:28) at com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31) at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:74) at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42) at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:66) at com.atlassian.prettyurls.filter.PrettyUrlsCombinedMatchDispatcherFilter.doFilter(PrettyUrlsCombinedMatchDispatcherFilter.java:61) at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:74) at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42) at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:66) at com.sysbliss.jira.plugins.workflow.servlet.JWDSendRedirectFilter.doFilter(JWDSendRedirectFilter.java:25) at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:74) at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42) at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:77) at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:63) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at com.atlassian.jira.web.filters.steps.ChainedFilterStepRunner.doFilter(ChainedFilterStepRunner.java:87) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at com.atlassian.core.filters.cache.AbstractCachingFilter.doFilter(AbstractCachingFilter.java:33) at com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at com.atlassian.core.filters.encoding.AbstractEncodingFilter.doFilter(AbstractEncodingFilter.java:41) at com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31) at com.atlassian.jira.web.filters.PathMatchingEncodingFilter.doFilter(PathMatchingEncodingFilter.java:49) at com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at com.atlassian.jira.startup.JiraStartupChecklistFilter.doFilter(JiraStartupChecklistFilter.java:78) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at com.atlassian.jira.web.filters.steps.ChainedFilterStepRunner.doFilter(ChainedFilterStepRunner.java:87) at com.atlassian.jira.web.filters.JiraFirstFilter.doFilter(JiraFirstFilter.java:57) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:190) at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:291) at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:776) at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:705) at org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:898) at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:690) at java.lang.Thread.run(Thread.java:662) Go back to the previous page ---- Thanks, -- kou

4 8

Re: [Maria-developers] [Commits] Rev 3810: MDEV-4994: Crash in dynamic column API in file:///home/bell/maria/bzr/work-maria-10.0-MDEV-4994/
by Kristian Nielsen 13 Sep '13

13 Sep '13

sanja(a)askmonty.org writes: > At file:///home/bell/maria/bzr/work-maria-10.0-MDEV-4994/ > > ------------------------------------------------------------ > revno: 3810 > revision-id: sanja(a)askmonty.org-20130911151436-rjz6pw6fh6rtj83e > parent: svoj(a)mariadb.org-20130904072837-u3zj89fq4xvb4cnp > committer: sanja(a)askmonty.org > branch nick: work-maria-10.0-MDEV-4994 > timestamp: Wed 2013-09-11 18:14:36 +0300 > message: > MDEV-4994: Crash in dynamic column API > Dynamic columns unittest fix. > > Correct processing of empty string added to creation dynamic string procedure > List function of numeric format fixed when empty string passed. > Unit test calls changed to new ones. > Unit test added to cmake file. > === modified file 'mysys/ma_dyncol.c' > --- a/mysys/ma_dyncol.c 2013-08-07 15:40:25 +0000 > +++ b/mysys/ma_dyncol.c 2013-09-11 15:14:36 +0000 > @@ -1618,15 +1618,22 @@ dynamic_new_column_store(DYNAMIC_COLUMN > > if (!(columns_order= malloc(sizeof(void*)*column_count))) > return ER_DYNCOL_RESOURCE; > - if (new_str) > + if (new_str || str->str == 0) > { > - if (dynamic_column_init_named(str, > - fmt->fixed_hdr + > - hdr->header_size + > - hdr->nmpool_size + > - hdr->data_size + > - DYNCOL_SYZERESERVE)) > - goto err; > + if (column_count) > + { > + if (dynamic_column_init_named(str, > + fmt->fixed_hdr + > + hdr->header_size + > + hdr->nmpool_size + > + hdr->data_size + > + DYNCOL_SYZERESERVE)) > + goto err; > + } > + else > + { > + dynamic_column_initialize(str); > + } > } > else > { > @@ -2438,7 +2445,8 @@ mariadb_dyncol_list(DYNAMIC_COLUMN *str, > uint i; > enum enum_dyncol_func_result rc; > > - (*nums)= 0; /* In case of errors */ > + (*nums)= 0; (*count)= 0; /* In case of errors */ > + > if (str->length == 0) > return ER_DYNCOL_OK; /* no columns */ > > > === modified file 'unittest/mysys/CMakeLists.txt' > --- a/unittest/mysys/CMakeLists.txt 2013-07-16 17:09:54 +0000 > +++ b/unittest/mysys/CMakeLists.txt 2013-09-11 15:14:36 +0000 > @@ -20,6 +20,9 @@ INCLUDE_DIRECTORIES(${CMAKE_SOURCE_DIR}/ > MY_ADD_TESTS(bitmap base64 my_vsnprintf my_atomic my_rdtsc lf my_malloc > LINK_LIBRARIES mysys) > > +MY_ADD_TESTS(ma_dyncol > + LINK_LIBRARIES mysqlclient) > + > IF(WIN32) > MY_ADD_TESTS(my_delete LINK_LIBRARIES mysys) > ENDIF() > > === modified file 'unittest/mysys/ma_dyncol-t.c' > --- a/unittest/mysys/ma_dyncol-t.c 2012-07-31 17:29:07 +0000 > +++ b/unittest/mysys/ma_dyncol-t.c 2013-09-11 15:14:36 +0000 > @@ -35,17 +35,18 @@ > void test_value_single_null() > { > int rc= FALSE; > + uint ids[1]= {1}; > DYNAMIC_COLUMN_VALUE val, res; > DYNAMIC_COLUMN str; > /* init values */ > val.type= DYN_COL_NULL; > - dynamic_column_value_init(&res); > + mariadb_dyncol_value_init(&res); > /* create column */ > - if (dynamic_column_create(&str, 1, &val)) > + if (mariadb_dyncol_create_many(&str, 1, ids, &val, 1)) > goto err; > dynstr_append(&str, "\1"); str.length--; //check for overflow > /* read column */ > - if (dynamic_column_get(&str, 1, &res)) > + if (mariadb_dyncol_get(&str, 1, &res)) > goto err; > rc= (res.type == DYN_COL_NULL); > err: > @@ -57,18 +58,19 @@ err: > void test_value_single_uint(ulonglong num, const char *name) > { > int rc= FALSE; > + uint ids[1]= {1}; > DYNAMIC_COLUMN_VALUE val, res; > DYNAMIC_COLUMN str; > /* init values */ > val.type= DYN_COL_UINT; > val.x.ulong_value= num; > - dynamic_column_value_init(&res); > + mariadb_dyncol_value_init(&res); > /* create column */ > - if (dynamic_column_create(&str, 1, &val)) > + if (mariadb_dyncol_create_many(&str, 1, ids, &val, 1)) > goto err; > dynstr_append(&str, "\1"); str.length--; //check for overflow > /* read column */ > - if (dynamic_column_get(&str, 1, &res)) > + if (mariadb_dyncol_get(&str, 1, &res)) > goto err; > rc= (res.type == DYN_COL_UINT) && (res.x.ulong_value == num); > num= res.x.ulong_value; > @@ -81,18 +83,19 @@ err: > void test_value_single_sint(longlong num, const char *name) > { > int rc= FALSE; > + uint ids[1]= {1}; > DYNAMIC_COLUMN_VALUE val, res; > DYNAMIC_COLUMN str; > /* init values */ > val.type= DYN_COL_INT; > val.x.long_value= num; > - dynamic_column_value_init(&res); > + mariadb_dyncol_value_init(&res); > /* create column */ > - if (dynamic_column_create(&str, 1, &val)) > + if (mariadb_dyncol_create_many(&str, 1, ids, &val, 1)) > goto err; > dynstr_append(&str, "\1"); str.length--; //check for overflow > /* read column */ > - if (dynamic_column_get(&str, 1, &res)) > + if (mariadb_dyncol_get(&str, 1, &res)) > goto err; > rc= (res.type == DYN_COL_INT) && (res.x.long_value == num); > num= res.x.ulong_value; > @@ -106,18 +109,19 @@ err: > void test_value_single_double(double num, const char *name) > { > int rc= FALSE; > + uint ids[1]= {1}; > DYNAMIC_COLUMN_VALUE val, res; > DYNAMIC_COLUMN str; > /* init values */ > val.type= DYN_COL_DOUBLE; > val.x.double_value= num; > - dynamic_column_value_init(&res); > + mariadb_dyncol_value_init(&res); > /* create column */ > - if (dynamic_column_create(&str, 1, &val)) > + if (mariadb_dyncol_create_many(&str, 1, ids, &val, 1)) > goto err; > dynstr_append(&str, "\1"); str.length--; //check for overflow > /* read column */ > - if (dynamic_column_get(&str, 1, &res)) > + if (mariadb_dyncol_get(&str, 1, &res)) > goto err; > rc= (res.type == DYN_COL_DOUBLE) && (res.x.double_value == num); > num= res.x.ulong_value; > @@ -133,21 +137,22 @@ void test_value_single_decimal(const cha > char buff[80]; > int rc= FALSE; > int length= 80; > + uint ids[1]= {1}; > DYNAMIC_COLUMN_VALUE val, res; > DYNAMIC_COLUMN str; > > /* init values */ > - dynamic_column_prepare_decimal(&val); // special procedure for decimal!!! > + mariadb_dyncol_prepare_decimal(&val); // special procedure for decimal!!! > if (string2decimal(num, &val.x.decimal.value, &end) != E_DEC_OK) > goto err; > - dynamic_column_value_init(&res); > + mariadb_dyncol_value_init(&res); > > /* create column */ > - if (dynamic_column_create(&str, 1, &val)) > + if (mariadb_dyncol_create_many(&str, 1, ids, &val, 1)) > goto err; > dynstr_append(&str, "\1"); str.length--; //check for overflow > /* read column */ > - if (dynamic_column_get(&str, 1, &res)) > + if (mariadb_dyncol_get(&str, 1, &res)) > goto err; > rc= ((res.type == DYN_COL_DECIMAL) && > (decimal_cmp(&res.x.decimal.value, &val.x.decimal.value) == 0)); > @@ -206,6 +211,7 @@ void test_value_single_string(const char > CHARSET_INFO *cs) > { > int rc= FALSE; > + uint ids[1]= {1}; > DYNAMIC_COLUMN_VALUE val, res; > DYNAMIC_COLUMN str; > > @@ -214,14 +220,14 @@ void test_value_single_string(const char > val.x.string.value.str= (char*)string; > val.x.string.value.length= len; > val.x.string.charset= cs; > - dynamic_column_value_init(&res); > + mariadb_dyncol_value_init(&res); > > /* create column */ > - if (dynamic_column_create(&str, 1, &val)) > + if (mariadb_dyncol_create_many(&str, 1, ids, &val, 1)) > goto err; > dynstr_append(&str, "\1"); str.length--; //check for overflow > /* read column */ > - if (dynamic_column_get(&str, 1, &res)) > + if (mariadb_dyncol_get(&str, 1, &res)) > goto err; > rc= ((res.type == DYN_COL_STRING) && > (res.x.string.value.length == len) && > @@ -239,6 +245,7 @@ err: > void test_value_single_date(uint year, uint month, uint day, const char *name) > { > int rc= FALSE; > + uint ids[1]= {1}; > DYNAMIC_COLUMN_VALUE val, res; > DYNAMIC_COLUMN str; > /* init values */ > @@ -247,13 +254,13 @@ void test_value_single_date(uint year, u > val.x.time_value.year= year; > val.x.time_value.month= month; > val.x.time_value.day= day; > - dynamic_column_value_init(&res); > + mariadb_dyncol_value_init(&res); > /* create column */ > - if (dynamic_column_create(&str, 1, &val)) > + if (mariadb_dyncol_create_many(&str, 1, ids, &val, 1)) > goto err; > dynstr_append(&str, "\1"); str.length--; //check for overflow > /* read column */ > - if (dynamic_column_get(&str, 1, &res)) > + if (mariadb_dyncol_get(&str, 1, &res)) > goto err; > rc= ((res.type == DYN_COL_DATE) && > (res.x.time_value.time_type == MYSQL_TIMESTAMP_DATE) && > @@ -270,6 +277,7 @@ void test_value_single_time(uint neg, ui > uint mic, const char *name) > { > int rc= FALSE; > + uint ids[1]= {1}; > DYNAMIC_COLUMN_VALUE val, res; > DYNAMIC_COLUMN str; > /* init values */ > @@ -280,13 +288,13 @@ void test_value_single_time(uint neg, ui > val.x.time_value.minute= minute; > val.x.time_value.second= second; > val.x.time_value.second_part= mic; > - dynamic_column_value_init(&res); > + mariadb_dyncol_value_init(&res); > /* create column */ > - if (dynamic_column_create(&str, 1, &val)) > + if (mariadb_dyncol_create_many(&str, 1, ids, &val, 1)) > goto err; > dynstr_append(&str, "\1"); str.length--; //check for overflow > /* read column */ > - if (dynamic_column_get(&str, 1, &res)) > + if (mariadb_dyncol_get(&str, 1, &res)) > goto err; > rc= ((res.type == DYN_COL_TIME) && > (res.x.time_value.time_type == MYSQL_TIMESTAMP_TIME) && > @@ -308,6 +316,7 @@ void test_value_single_datetime(uint neg > uint mic, const char *name) > { > int rc= FALSE; > + uint ids[1]= {1}; > DYNAMIC_COLUMN_VALUE val, res; > DYNAMIC_COLUMN str; > /* init values */ > @@ -321,13 +330,13 @@ void test_value_single_datetime(uint neg > val.x.time_value.minute= minute; > val.x.time_value.second= second; > val.x.time_value.second_part= mic; > - dynamic_column_value_init(&res); > + mariadb_dyncol_value_init(&res); > /* create column */ > - if (dynamic_column_create(&str, 1, &val)) > + if (mariadb_dyncol_create_many(&str, 1, ids, &val, 1)) > goto err; > dynstr_append(&str, "\1"); str.length--; //check for overflow > /* read column */ > - if (dynamic_column_get(&str, 1, &res)) > + if (mariadb_dyncol_get(&str, 1, &res)) > goto err; > rc= ((res.type == DYN_COL_DATETIME) && > (res.x.time_value.time_type == MYSQL_TIMESTAMP_DATETIME) && > @@ -373,7 +382,7 @@ void test_value_multi(ulonglong num0, > val[1].x.long_value= num1; > val[2].type= DYN_COL_DOUBLE; > val[2].x.double_value= num2; > - dynamic_column_prepare_decimal(val + 3); // special procedure for decimal!!! > + mariadb_dyncol_prepare_decimal(val + 3); // special procedure for decimal!!! > if (string2decimal(num3, &val[3].x.decimal.value, &end3) != E_DEC_OK) > goto err; > val[4].type= DYN_COL_STRING; > @@ -404,14 +413,14 @@ void test_value_multi(ulonglong num0, > val[7].x.time_value.second_part= mic7; > val[8].type= DYN_COL_NULL; > for (i= 0; i < 9; i++) > - dynamic_column_value_init(res + i); > + mariadb_dyncol_value_init(res + i); > /* create column */ > - if (dynamic_column_create_many(&str, 9, column_numbers, val)) > + if (mariadb_dyncol_create_many(&str, 9, column_numbers, val, 1)) > goto err; > dynstr_append(&str, "\1"); str.length--; //check for overflow > /* read column */ > for (i= 0; i < 9; i++) > - if (dynamic_column_get(&str, column_numbers[i], res + i)) > + if (mariadb_dyncol_get(&str, column_numbers[i], res + i)) > goto err; > rc= ((res[0].type == DYN_COL_UINT) && > (res[0].x.ulong_value == num0) && > @@ -467,7 +476,7 @@ void test_value_multi_same_num() > for (i= 0; i < 5; i++) > val[i].type= DYN_COL_NULL; > /* create column */ > - if (!dynamic_column_create_many(&str, 5, column_numbers, val)) > + if (!mariadb_dyncol_create_many(&str, 5, column_numbers, val, 1)) > goto err; > rc= TRUE; > err: > @@ -487,68 +496,69 @@ void test_update_multi(uint *column_numb > > val.type= DYN_COL_UINT; > val.x.ulong_value= column_values[0]; > - if (dynamic_column_create(&str, column_numbers[0], &val)) > + if (mariadb_dyncol_create_many(&str, 1, column_numbers, &val, 1)) > goto err; > for (i= 1; i < all; i++) > { > val.type= (null_values[i] ? DYN_COL_NULL : DYN_COL_UINT); > val.x.ulong_value= column_values[i]; > - if (dynamic_column_update(&str, column_numbers[i], &val)) > + if (mariadb_dyncol_update_many(&str, 1, column_numbers +i, &val)) > goto err; > > /* check value(s) */ > for (j= i; j >= (i < only_add ? 0 : i); j--) > { > - if (dynamic_column_get(&str, column_numbers[j], &val)) > + if (mariadb_dyncol_get(&str, column_numbers[j], &val)) > goto err; > if (null_values[j]) > { > if (val.type != DYN_COL_NULL || > - dynamic_column_exists(&str, column_numbers[j]) == ER_DYNCOL_YES) > + mariadb_dyncol_exists(&str, column_numbers[j]) == ER_DYNCOL_YES) > goto err; > } > else > { > if (val.type != DYN_COL_UINT || > val.x.ulong_value != column_values[j] || > - dynamic_column_exists(&str, column_numbers[j]) == ER_DYNCOL_NO) > + mariadb_dyncol_exists(&str, column_numbers[j]) == ER_DYNCOL_NO) > goto err; > } > } > if (i < only_add) > { > - DYNAMIC_ARRAY num; > - if (dynamic_column_list(&str, &num)) > + uint elements, *num; > + if (mariadb_dyncol_list(&str, &elements, &num)) > + { > + my_free(num); > goto err; > + } > /* cross check arrays */ > - if ((int)num.elements != i + 1) > + if ((int)elements != i + 1) > { > - delete_dynamic(&num); > + my_free(num); > goto err; > } > for(j= 0; j { > int k; > for(k= 0; > - k - *dynamic_element(&num, k, uint*); > + k k++); > if (k >= i + 1) > { > - delete_dynamic(&num); > + my_free(num); > goto err; > } > for(k= 0; > - k - *dynamic_element(&num, j, uint*); > + k k++); > if (k >= i + 1) > { > - delete_dynamic(&num); > + my_free(num); > goto err; > } > } > - delete_dynamic(&num); > + my_free(num); > } > } > > @@ -563,32 +573,52 @@ void test_empty_string() > { > DYNAMIC_COLUMN_VALUE val, res; > DYNAMIC_COLUMN str; > - DYNAMIC_ARRAY array_of_uint; > + uint *array_of_uint; > + uint number_of_uint; > int rc; > + uint ids[1]= {1}; > + DYNAMIC_COLUMN_VALUE vals[0]; > /* empty string */ > bzero(&str, sizeof(str)); > > - rc= dynamic_column_get(&str, 1, &res); > + rc= mariadb_dyncol_get(&str, 1, &res); > ok( (rc == ER_DYNCOL_OK) && (res.type == DYN_COL_NULL), "%s", "empty get"); > > - rc= dynamic_column_delete(&str, 1); > - ok( (rc == ER_DYNCOL_OK), "%s", "empty delete"); > + vals[0].type= DYN_COL_NULL; > + rc= mariadb_dyncol_update_many(&str, 1, ids, vals); > + ok( (rc == ER_DYNCOL_OK) && (str.str == 0), "%s", "empty delete"); > > - rc= dynamic_column_exists(&str, 1); > + rc= mariadb_dyncol_exists(&str, 1); > ok( (rc == ER_DYNCOL_NO), "%s", "empty exists"); > > - rc= dynamic_column_list(&str, &array_of_uint); > - ok( (rc == ER_DYNCOL_OK) && (array_of_uint.elements == 0), > + rc= mariadb_dyncol_list(&str, &number_of_uint, &array_of_uint); > + ok( (rc == ER_DYNCOL_OK) && (number_of_uint == 0) && (str.str == 0), > "%s", "empty list"); > > val.type= DYN_COL_UINT; > val.x.ulong_value= 1212; > - rc= dynamic_column_update(&str, 1, &val); > + rc= mariadb_dyncol_update_many(&str, 1, ids, &val); > if (rc == ER_DYNCOL_OK) > - rc= dynamic_column_get(&str, 1, &res); > - ok( (rc == ER_DYNCOL_OK) && > + rc= mariadb_dyncol_get(&str, 1, &res); > + ok( (rc == ER_DYNCOL_OK) && (str.str != 0) && > (res.type == DYN_COL_UINT) && (res.x.ulong_value == val.x.ulong_value), > "%s", "empty update"); > + dynamic_column_column_free(&str); > +} > + > +void test_mdev_4994() > +{ > + DYNAMIC_COLUMN dyncol; > + LEX_STRING key= {0,0}; > + DYNAMIC_COLUMN_VALUE val; > + int rc; > + > + val.type= DYN_COL_NULL; > + > + dynamic_column_initialize(&dyncol); > + rc= mariadb_dyncol_create_many_named(&dyncol, 1, &key, &val, 0); /* crash */ > + ok( (rc == ER_DYNCOL_OK), "%s", "test_mdev_4994"); > + dynamic_column_column_free(&dyncol); > } > > > @@ -633,11 +663,11 @@ void test_update_many(uint *column_numbe > res[i].type= DYN_COL_UINT; > res[i].x.ulong_value= result_values[i]; > } > - if (dynamic_column_create_many(&str1, column_count, column_numbers, val)) > + if (mariadb_dyncol_create_many(&str1, column_count, column_numbers, val, 1)) > goto err; > - if (dynamic_column_update_many(&str1, update_count, update_numbers, upd)) > + if (mariadb_dyncol_update_many(&str1, update_count, update_numbers, upd)) > goto err; > - if (dynamic_column_create_many(&str2, result_count, result_numbers, res)) > + if (mariadb_dyncol_create_many(&str2, result_count, result_numbers, res, 1)) > goto err; > if (str1.length == str2.length && > memcmp(str1.str, str2.str, str1.length) ==0) > @@ -656,7 +686,7 @@ int main(int argc __attribute__((unused) > char *big_string= (char *)malloc(1024*1024); > > MY_INIT(argv[0]); > - plan(60); > + plan(61); > > if (!big_string) > exit(1); > @@ -664,21 +694,17 @@ int main(int argc __attribute__((unused) > big_string[i]= ('0' + (i % 10)); > test_value_single_null(); > test_value_single_uint(0, "0"); > - test_value_single_uint(ULL(0xffffffffffffffff), "0xffffffffffffffff"); > - test_value_single_uint(ULL(0xaaaaaaaaaaaaaaaa), "0xaaaaaaaaaaaaaaaa"); > - test_value_single_uint(ULL(0x5555555555555555), "0x5555555555555555"); > + test_value_single_uint(0xffffffffffffffffULL, "0xffffffffffffffff"); > + test_value_single_uint(0xaaaaaaaaaaaaaaaaULL, "0xaaaaaaaaaaaaaaaa"); > + test_value_single_uint(0x5555555555555555ULL, "0x5555555555555555"); > test_value_single_uint(27652, "27652"); > test_value_single_sint(0, "0"); > test_value_single_sint(1, "1"); > test_value_single_sint(-1, "-1"); > - test_value_single_sint(LL(0x7fffffffffffffff), > - "0x7fffffffffffffff"); > - test_value_single_sint(LL(0xaaaaaaaaaaaaaaaa), > - "0xaaaaaaaaaaaaaaaa"); > - test_value_single_sint(LL(0x5555555555555555), > - "0x5555555555555555"); > - test_value_single_sint(LL(0x8000000000000000), > - "0x8000000000000000"); > + test_value_single_sint(0x7fffffffffffffffLL, "0x7fffffffffffffff"); > + test_value_single_sint(0xaaaaaaaaaaaaaaaaLL, "0xaaaaaaaaaaaaaaaa"); > + test_value_single_sint(0x5555555555555555LL, "0x5555555555555555"); > + test_value_single_sint(0x8000000000000000LL, "0x8000000000000000"); > test_value_single_double(0.0, "0.0"); > test_value_single_double(1.0, "1.0"); > test_value_single_double(-1.0, "-1.0"); > @@ -735,7 +761,7 @@ int main(int argc __attribute__((unused) > } > { > uint column_numbers[]= {10,1,12,37,4,57,6,76,87}; > - test_value_multi(ULL(0xffffffffffffffff), LL(0x7fffffffffffffff), > + test_value_multi(0xffffffffffffffffULL, 0x7fffffffffffffffLL, > 99999999.999e120, "9999999999999999999999999999999", > big_string, 1024*1024, charset_list[0], > 9999, 12, 31, > @@ -791,5 +817,8 @@ int main(int argc __attribute__((unused) > update_numbers, update_values, update_nulls, 4, > result_numbers, result_values, 3); > } > + test_mdev_4994(); > + > + my_end(0); > return exit_status(); > } > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits

1 1

[Maria-developers] Can there be a better storage engine API?
by MARK CALLAGHAN 13 Sep '13

13 Sep '13

The storage engine API has never been my favorite part of MySQL. I haven't written a storage engine but I used to maintain a custom storage engine and migrated that from 4.0 to 5.0 so I had some experience with the API. Is there a different API that would make it easier for innovation to come to MySQL/MariaDB? For example there is a lot going on in the write-optimized database space. Until recently we had nothing for MySQL open-source. Now we have open source TokuDB (great news). I haven't read their source but I have been told they use something that was derived from the BerkeleyDB API. Would the BerkeleyDB API make it easier to get new storage engines into MySQL? For example, we don't have an engine optimized for in-memory workloads. And while InnoDB will have great cache hit rates when all data is in-memory. It might write much more data to disk than required. Do we need a better API or is the current code good enough? I hope to hear from people who have done serious storage engine work. -- Mark Callaghan mdcallag(a)gmail.com

7 10

[Maria-developers] updating a string session variable from a storage engine
by Rich Prohaska 12 Sep '13

12 Sep '13

i have a MYSQL_THDVAR_STR session variable in my storage engine. i want to update its value from the storage engine. it was suggested that i use "THDVAR(x)=new_value" to do it. i noticed that update_func_str is NOT called, so i wonder about the contract. is the caller responsible for memory management? i used PLUGIN_VAR_MEMALLOC for the var def. thanks

3 2

[Maria-developers] Problems building debug version of MariaDB 5.5.32
by Weldon Whipple 11 Sep '13

11 Sep '13

Hello, I'm having problems building a Debug version of MariaDB 5.5.32. (It's a new problem, I think; I'm almost positive it was working about two weeks ago.) Here is what I'm doing to build MariaDB: $ cmake -DCMAKE_BUILD_TYPE=Debug . $ make package At 69% complete, I get about 14 lines like this: [ 69%] Building CXX object unittest/mysys/CMakeFiles/explain_filename-t.dir/explain_filename-t.cc.o Linking CXX executable explain_filename-t `.text._ZN7handler17ha_index_read_mapEPhPKhm16ha_rkey_function' referenced in section `.note.stapsdt' of ../../sql/libpartition.a(ha_partition.cc.o): defined in discarded section `.text._ZN7handler17ha_index_read_mapEPhPKhm16ha_rkey_function[handler::ha_index_read_map(unsigned char*, unsigned char const*, unsigned long, ha_rkey_function)]' of ../../sql/libpartition.a(ha_partition.cc.o) followed by: collect2: ld returned 1 exit status make[2]: *** [unittest/mysys/explain_filename-t] Error 1 make[1]: *** [unittest/mysys/CMakeFiles/explain_filename-t.dir/all] Error 2 make: *** [all] Error 2 Some comments: 1. I'm running on CentOS 6.4: $ uname -a Linux weldon 2.6.32-358.18.1.el6.x86_64 #1 SMP Wed Aug 28 17:19:38 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux 2. I've tried "cmake -DRPM=centos6 . " (without Debug specified) and it works just fine. 3. I found the line in (282:) in ./CMakeLists.txt that reads "ADD_SUBDIRECTORY(unittest/mysys)" and commented it out. Then the build continued until it was trying to link mysqld ... and failed with the similar errors at 99% completion. (I actually WANT mysqld, so I couldn't comment it out too :-)) 4. Something I read indicates that MariaDB requires a GCC build against recent binutils. yum seems to indicate that both GCC and binutils are up-to-date. 5. The cmake option I'm using is the one for 5.5+ as instructed at https://mariadb.com/kb/en/compiling-mariadb-for-debugging/ 6. Yes, I also tried the old ./BUILD/compile-pentium64-debug-max (but it failed similarly) I THINK it's probably an incompatible version of some build tool on my development box. Thanks in advance for any advice. Weldon Whipple

2 1

[Maria-developers] MDEV 5007 - help me! it's a old mdev closed as not a bug but don't solve the problem
by Roberto Spadim 10 Sep '13

10 Sep '13

Hi guys! I posted the MDEV-5007 and it's a old problem from MDEV-3795 (the same problem) It (MDEV-3795) was closed as a not a bug, but i think it's a bug, but i didn't explained well at the first time I changed from a mysql server to a mariadb server and the result order by isn't executed, i tried myisam,aria,innodb,memory engines and all of them return the same problem (order by not executed) On the wrong query i don't see the "using filesort" at explain, and at the right query (with right order by) it show "using filesort" Could anyone help me? or at least EXPLAIN why the order by shouldn't be executed on the first query, and maybe a workaround using sql_optimizer variable or something like it? it's a closed software and i need to contact developer to change the query, that's why i'm talking about the sql_optimizer variable instead of rewrite the query Thanks -- Roberto Spadim

2 3

[Maria-developers] Rollback causes assertion with Galera 10.0
by Jan Lindström 10 Sep '13

10 Sep '13

Hi, Background: I merged 10.0 to lp:~maria-captains/maria/maria-10.0-galera/, merge result is pushed to the tree. In testing I hit repeatable to following assertion #0 0x00007f1ff987f037 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56 #1 0x00007f1ff9882698 in __GI_abort () at abort.c:90 #2 0x00007f1ff9877e03 in __assert_fail_base (fmt=0x7f1ff99ce578 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=assertion@entry=0x109b4ff "empty()", file=file@entry=0x109b4d8 "/home/jan/mysql/galera-10.0/sql/log.cc", line=line@entry=294, function=function@entry=0x10a01e0 <binlog_cache_data::reset()::__PRETTY_FUNCTION__> "void binlog_cache_data::reset()") at assert.c:92 #3 0x00007f1ff9877eb2 in __GI___assert_fail (assertion=0x109b4ff "empty()", file=0x109b4d8 "/home/jan/mysql/galera-10.0/sql/log.cc", line=294, function=0x10a01e0 <binlog_cache_data::reset()::__PRETTY_FUNCTION__> "void binlog_cache_data::reset()") at assert.c:101 #4 0x00000000009497d4 in binlog_cache_data::reset (this=0x7f1f7403ad60) at /home/jan/mysql/galera-10.0/sql/log.cc:294 #5 0x0000000000949b6a in binlog_cache_mngr::reset (this=0x7f1f7403ab90, do_stmt=false, do_trx=true) at /home/jan/mysql/galera-10.0/sql/log.cc:462 #6 0x0000000000935a4c in binlog_truncate_trx_cache (thd=0x45b7d40, cache_mngr=0x7f1f7403ab90, all=true) at /home/jan/mysql/galera-10.0/sql/log.cc:1991 #7 0x0000000000936123 in binlog_rollback (hton=0x3891fb0, thd=0x45b7d40, all=true) at /home/jan/mysql/galera-10.0/sql/log.cc:2180 #8 0x000000000086b750 in ha_rollback_trans (thd=0x45b7d40, all=true) at /home/jan/mysql/galera-10.0/sql/handler.cc:1616 #9 0x00000000007a0c5f in trans_rollback (thd=0x45b7d40) at /home/jan/mysql/galera-10.0/sql/transaction.cc:321 #10 0x00000000006718b2 in mysql_execute_command (thd=0x45b7d40) at /home/jan/mysql/galera-10.0/sql/sql_parse.cc:4634 #11 0x0000000000677d52 in mysql_parse (thd=0x45b7d40, rawbuf=0x7f1f74005288 "ROLLBACK", length=8, parser_state=0x7f1fe83bb610) at /home/jan/mysql/galera-10.0/sql/sql_parse.cc:6816 #12 0x00000000006774d3 in wsrep_mysql_parse (thd=0x45b7d40, rawbuf=0x7f1f74005288 "ROLLBACK", length=8, parser_state=0x7f1fe83bb610) at /home/jan/mysql/galera-10.0/sql/sql_parse.cc:6647 #13 0x00000000006688cd in dispatch_command (command=COM_QUERY, thd=0x45b7d40, packet=0x44807c1 "ROLLBACK ", packet_length=9) at /home/jan/mysql/galera-10.0/sql/sql_parse.cc:1451 #14 0x00000000006678a3 in do_command (thd=0x45b7d40) at /home/jan/mysql/galera-10.0/sql/sql_parse.cc:1077 #15 0x000000000078ba4d in do_handle_one_connection (thd_arg=0x45b7d40) at /home/jan/mysql/galera-10.0/sql/sql_connect.cc:1404 #16 0x000000000078b78d in handle_one_connection (arg=0x45b7d40) at /home/jan/mysql/galera-10.0/sql/sql_connect.cc:1311 #17 0x0000000000b0c20a in pfs_spawn_thread (arg=0x45713a0) at /home/jan/mysql/galera-10.0/storage/perfschema/pfs.cc:1853 #18 0x00007f1ffa423f8e in start_thread (arg=0x7f1fe83bc700) at pthread_create.c:311 #19 0x00007f1ff9941e1d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113 (gdb) We go from log.cc::binlog_rollback : else if (ending_trans(thd, all) || (!(thd->variables.option_bits & OPTION_KEEP_LOG) && (!stmt_has_updated_non_trans_table(thd) || WSREP_FORMAT(thd->variables.binlog_format) != BINLOG_FORMAT_STMT) && (!cache_mngr->trx_cache.changes_to_non_trans_temp_table() || WSREP_FORMAT(thd->variables.binlog_format) != BINLOG_FORMAT_MIXED))) error= binlog_truncate_trx_cache(thd, cache_mngr, all); end there to cache_mngr->reset(false, true); and there is DBUG_ASSERT(empty()); bool empty() const { return pending() == NULL && my_b_tell(&cache_log) == 0; } At empty() this is what does not hold : my_b_tell(&cache_log) == 0; // not sure what this means At test output there is at the end: # 2013-09-09T10:03:21 [6620] Query: UPDATE LOW_PRIORITY IGNORE `table100_int_autoinc` AS X SET `col_char_12` = 220 WHERE X . `col_char_12_key` IN ( 2427781120 , 1 , 'i' , 'm' , 'o' ) ORDER BY `col_char_12`,`col_char_12_key`,`col_int`,`col_int_key`,`pk` LIMIT 8 failed: 1213 Deadlock found when trying to get lock; try restarting transaction. Further errors of this kind will be suppressed. # 2013-09-09T10:04:14 [6620] Query: DELETE LOW_PRIORITY IGNORE FROM `table10_key_pk_parts_2_int_autoinc` WHERE `col_char_12_key` IN ( 4200660992 , 1646198784 , 3888709632 , 32 , 'c' ) ORDER BY `col_char_12`,`col_char_12_key`,`col_int`,`col_int_key`,`pk` LIMIT 1 failed: 1205 Lock wait timeout exceeded; try restarting transaction. Further errors of this kind will be suppressed. Unfortunately, general log contains so many statements from different connections that they are not really helpful for getting repeatable test case. R: Jan -- Jan Lindström Principal Engineer MariaDB | MaxScale | skype: jan_p_lindstrom www.skysql.com <http://www.skysql.com/> Twitter <http://twitter.com/skysql> Blog <http://www.skysql.com/blog/> Facebook <http://www.facebook.com/skysql> LinkedIn <http://www.linkedin.com/company/1214250> Google+ <https://plus.google.com/117544963211695643458/posts>

2 4

[Maria-developers] AIX "make fails" - related to: MDEV-4941
by Michael Felt 09 Sep '13

09 Sep '13

AIX make fails with syntax errors. One has been fixed, per https://mariadb.atlassian.net/browse/MDEV-4941. Rather than submit error by error I thought it would be more productive to provide the output from make -i. After trying, but not seeing it 'attach', I am going to try attaching the output of make -i here. regards, Michael

2 2

[Maria-developers] Some clarifications requested for C client
by Peter Laursen 09 Sep '13

09 Sep '13

This is probably for Georg Richter (and/or @Wlad). I have a few qeustion! 1) https://bazaar.launchpad.net/~maria-captains/mariadb-native-client/trunk/re… "Fix for CONC-44: LOAD DATA INFILE can't open utf16le encoded filenames" But we we can stil not specify UTF16LE as the CHARSET, I think? Say I have a file *named* "ÄÜÑБГД.csv". (both Western special characters and cyrillic characters in the file name). This file I will be able to read now (using revision 80), right? But if file *content* is this single line (one row, two columns) ÄÜÑ,БГД and if it is encoded as 'Windows Unicode'/utf16le I still will need to recode the file to utf8 before executing it with LOAD DATA (as UTF16LE still cannot be specified for CHARSET in LOAD DATA), right? Please confirm that I understand this! 2) https://bazaar.launchpad.net/~maria-captains/mariadb-native-client/trunk/re… "Added support for embedded (sqlite)". This happened by beginning of this year but I just noticed now. Now what can we actually do with SQLite and the MariaDB C-API and what we the purpose of this revision? Is it to make it easier to write client applications using SQLite for internal storage? Or to make the C client code 'a generic API' for connecting to both MySQL (of all flavors) and SQLite? Is the the code/files that that were added different from offical SQLite downloads from http://www.sqlite.org/? I am not able to find any documentation anywhere. Also not a blog or whatever either. There must be some reason to add sqlite3 code?! 3) This seems to be the latest revision: https://bazaar.launchpad.net/~maria-captains/mariadb-native-client/trunk/re… "Added dyamic column api; Added character set conversion" ('dyamic' is misspelled BTW!) Won't we need some documentation here? The claim that MariaDB's connector "has exactly the same API as the MySQL Connector/C for MySQL 5.5" ( https://kb.askmonty.org/en/mariadb-client-library-for-c/) cannot be true now as it adds funtionalities for managing MariaDB ' dymamic column waht oracle's API has not? The API from Oracle's library is now a *subset* or MariaDB API (or MariaDB's is a *superset* or Oracle). So much I can figure out, but should I need to code with it I would need description of functions' syntax, expected results and error listings. Also I am not able to figure out what the "character set conversion" referred to is doing and why it was added. 4) Why does the changelog here https://kb.askmonty.org/en/mariadb-client-library-for-c-100-changelog/ stop at revision 77? -- Peter Laursen -- Webyog

2 3

Re: [Maria-developers] Rev 3883: MDEV-4472 Audit-plugin. Server-related part of the task
by Sergei Golubchik 08 Sep '13

08 Sep '13

Hi, Holyfoot! Just one question, see below. Otherwise ok. On Sep 08, holyfoot(a)askmonty.org wrote: > revno: 3883 > revision-id: holyfoot(a)askmonty.org-20130908092209-xar471jadl7lmwzo > parent: sergii(a)pisem.net-20130907203634-htu89dqmwe5gqwy5 > committer: Alexey Botchkov <holyfoot(a)askmonty.org> > branch nick: 55-4472 > timestamp: Sun 2013-09-08 14:22:09 +0500 > message: > MDEV-4472 Audit-plugin. Server-related part of the task. > file_logger became the service. > Data like query_id now are sent to the audit plugin. > Fix for MDEV-4770 ported from 10.0. > Fix added for the read_maria_plugin_info(). > Log rotation can be disabled with 'set rotations=0'. > === renamed file 'plugin/sql_errlog/sql_logger.cc' => 'mysys/file_logger.c' > --- a/plugin/sql_errlog/sql_logger.cc 2013-06-22 12:02:03 +0000 > +++ b/mysys/file_logger.c 2013-09-08 09:22:09 +0000 > @@ -16,7 +16,7 @@ > > #include "my_global.h" > #include <my_sys.h> > -#include "service_logger.h" > +#include <mysql/service_logger.h> > #include <my_pthread.h> > > extern MYSQL_PLUGIN_IMPORT char *mysql_data_home; I believe we agreed to keep file_logger.c completely clean from dynamic-plugin-related code. Did you forget to remove the line above? Regards, Sergei

1 0

Re: [Maria-developers] MDEV-4472 Audit-plugin. Server-related part of the task
by Sergei Golubchik 08 Sep '13

08 Sep '13

Hi, Holyfoot! On Sep 05, holyfoot(a)askmonty.org wrote: > At file:///home/hf/wmar/55-4472/ > > === renamed file 'plugin/sql_errlog/sql_logger.cc' => 'mysys/file_logger.c' > --- a/plugin/sql_errlog/sql_logger.cc 2013-06-22 12:02:03 +0000 > +++ b/mysys/file_logger.c 2013-09-05 12:47:00 +0000 > @@ -16,10 +16,12 @@ > > #include "my_global.h" > #include <my_sys.h> > -#include "service_logger.h" > +#include <mysql/service_logger.h> > #include <my_pthread.h> > > +#ifndef _WIN32 > extern MYSQL_PLUGIN_IMPORT char *mysql_data_home; > +#endif /*!_WIN32*/ > > #ifdef HAVE_PSI_INTERFACE > /* These belong to the service initialization */ > @@ -51,6 +53,12 @@ LOGGER_HANDLE *logger_open(const char *p > unsigned int rotations) > { > LOGGER_HANDLE new_log, *l_perm; > + //char *local_mysql_data_home; > +//#ifdef _WIN32 > + //local_mysql_data_home= (char *)GetProcAddress(0, "mysql_data_home"); > +//#else > + //local_mysql_data_home= mysql_data_home; > +//#endif /*_WIN32*/ I think you can completely remove this and the #ifndef above. For your MySQL plugin you can simply #include this in the plugin body like extern char *mysql_data_home; #include "file_logger.c" and later in your plugin_init function: #ifdef _WIN32 mysql_data_home= (char *)GetProcAddress(0, "mysql_data_home"); #endif This way you will keep file_logger.c completely free of the dynamic plugin hacks. > /* > I don't think we ever need more rotations, > @@ -168,6 +176,29 @@ int logger_vprintf(LOGGER_HANDLE *log, c > int logger_rotate(LOGGER_HANDLE *log) > { > int result; I believe you've added recently a test to disable rotation if number of rotations is 0? Regards, Sergei

2 1

[Maria-developers] first contribution
by AugustQ 06 Sep '13

06 Sep '13

Hi, after some struggling I got this message from Bazaar: Created new stacked branch referring to /+branch-id/633549. I hope everything worked fine. If not let me know so that I can correct it. I created a branch to 10.0.4 of MariaDB and named it: contrib_AQ_01. I've uploaded this some minutes before. For some time I played with the source of MySQL 5.5.8. I've taken some of my modifications to MariaDB 10.0.4 and want to give them to you for further discussions. To say it first: this is not production-ready-code, it's a proof of a concept. My modifications are made for the MyIsam-engine in the handling of a full table-scan with a table of dynamic sized records. The reason for uploading this is: during my tests I executed statements like SELECT * FROM table INTO OUTFILE 'x.x'; With my test-data this took about 20 seconds with the original code and 12 seconds with my modifications. I've tested my code with a single table which contains 10 Mio records, a total size of 1GB. As i've written before: it's a proof of concept. I've not implemented all the possible cases. I wanted to know if a full implementation is a complete waste of time or if it is a bit useful (in this case I will do it). Some more details: in mi_dynrec.c there is a funtion _mi_get_block_info() and i've added a funtion AQ_mi_get_block_info(), these 2 functions do the same thing but in a different way. In my version of this function I've implemented only those case that I needed for my tests. Because I started with examining the source of MySQL I modified the code in the MyIsam-engine and applied these to the MyIsam-engine in MariaDB. For staying compatible with MySQL I think it's better to use these modifications in the Maria-engine of MariaDB, but only if you agree with my approach. This is what I modified: in myisamdef.h I added: #define USE_CODE_OF_AQ converting this line in a comment allowed me to use the old code, a define with this identifier activates my code after a recompile. This allowed me to do some comparisons. You will find my modifications in myisamdef.h, mi_dynrec.c and mi_extra.c if you look for this identifier. Please look at my approach. If you agree I will continue with this and check all the cases which i didn't touch for now. thank you AugustQ

1 0

Re: [Maria-developers] MDEV-4978 - Server cursor is broken with blobs in the select list
by Sergei Golubchik 06 Sep '13

06 Sep '13

Hi, Sergey! On Sep 06, Sergey Vojtovich wrote: > revno: 3878 > revision-id: svoj(a)mariadb.org-20130906111656-oy2c3pwy2o1q0zqn > parent: wlad(a)montyprogram.com-20130903204512-hgwu77prle0lw7a3 > committer: Sergey Vojtovich <svoj(a)mariadb.org> > branch nick: 5.5-mdev4978 > timestamp: Fri 2013-09-06 15:16:56 +0400 > message: > MDEV-4978 - Server cursor is broken with blobs in the select list, > ORDER BY does not work > > Use "dynamic" row format (instead of "block") for MARIA internal > temporary tables created for cursors. > With "block" row format MARIA may shuffle rows, with "dynamic" row > format records are inserted sequentially (there are no gaps in data > file while we fill temporary tables). > This is needed to preserve row order when scanning materialized cursors. > === modified file 'sql/sql_select.cc' > --- a/sql/sql_select.cc 2013-09-03 14:41:07 +0000 > +++ b/sql/sql_select.cc 2013-09-06 11:16:56 +0000 > @@ -15373,7 +15374,8 @@ create_tmp_table(THD *thd, TMP_TABLE_PAR > if (thd->is_fatal_error) // If end of memory > goto err; /* purecov: inspected */ > share->db_record_offset= 1; > - table->used_for_duplicate_elimination= (param->sum_func_count == 0 && > + table->used_for_duplicate_elimination= keep_row_order || > + (param->sum_func_count == 0 && > (table->group || table->distinct)); That's very confusing. Why keep_row_order means that the table is "used for duplicate elimination"? I had to check the code to see that used_for_duplicate_elimination affects the row format in Aria. This is not obvious too. And it forces the row format to be DYNAMIC for a very different reason, not for preserving row order. Perhaps it'd be better to rename used_for_duplicate_elimination to "force_dynamic_Aria_row_format" ? A prettier solution would be to introduce another flag: table->keep_row_order. Regards, Sergei

2 2

[Maria-developers] Bazaar-problem
by AugustQ 06 Sep '13

06 Sep '13

Hi, I'm struggling with Bazaar. Maybe you can help me. I've made some modifications to the source-code and committed these to the local repository. Next I tried to upload them via push. This is what I receive: Cannot lock LockDir(chroot-70341520:///% 2Bbranch/maria/.bzr/branch/lock): Transport operation not possible: readonly transport As I'm using Eclipse I found this within Eclipse on the console: Error while executing push Cannot lock LockDir(chroot-70341520:///% 2Bbranch/maria/.bzr/branch/lock): Transport operation not possible: readonly transport ------ XmlRpcClient Executing: push "bzr+ssh://bazaar.launchpad.net/+branch/maria/" Can you please give me a tip. Thanks AugustQ

5 9

[Maria-developers] Incorrect format description event skipping in queue_event()
by Pavel Ivanov 04 Sep '13

04 Sep '13

Kristian, You have the following comment in the queue_event() in sql/slave.cc: /* Do not queue any format description event that we receive after a reconnect where we are skipping over a partial event group received before the reconnect. (If we queued such an event, and it was the first format_description event after master restart, the slave SQL thread would think that the partial event group before it in the relay log was from a previous master crash and should be rolled back). */ I don't understand which failure scenario you are talking about here and I claim that this bypassing of queuing into relay log is incorrect. My reasoning is: relay log can always be rotated in the middle of event group. During rotation when new relay log file is created format description event is always written at the beginning of the file. So in this case SQL thread will see format description event in the middle of event group and according to my testing it doesn't rollback the active transaction because of that. So when do you think it will rollback? Now, you may say that there's no problem in skipping the format description event because everything works. But it's only for now. When IO thread is reconnecting it rotates relay log and as I said it writes format description event at the beginning of the new file. But it writes an event that it created itself, i.e. not the one that master have sent. And as format description event from master is not written into relay log SQL thread from this point on starts to use format description generated by slave which may be different from the one generated by master. It may lead to a broken replication and SQL thread may not recognize events in the relay log if difference between master's and slave's format descriptions is significant. Another somewhat related question: Gtid_log_event::peek() (as well as Gtid_log_event constructor from const char* buf) is implemented with assumption that Format_description_log_event::common_header_len is always equal to LOG_EVENT_HEADER_LEN. While currently it's true I believe common_header_len can be increased in future MariaDB versions. And when it happens old slave won't be able to replicate from a new master. I'd think this is a bug. What do you think? Pavel

2 10

Re: [Maria-developers] Rev 3876: MDEV-4472 Audit-plugin. Server-related part of the task
by Sergei Golubchik 04 Sep '13

04 Sep '13

Hi, Holyfoot! Mostly ok. There are few comments, see below: On Sep 03, holyfoot(a)askmonty.org wrote: > ------------------------------------------------------------ > revno: 3876 > revision-id: holyfoot(a)askmonty.org-20130903082521-xpia2wond6oggpsk > parent: igor(a)askmonty.org-20130831163309-vqtls4oubhvz9zcz > committer: Alexey Botchkov <holyfoot(a)askmonty.org> > branch nick: 55-4472 > timestamp: Tue 2013-09-03 13:25:21 +0500 > message: > MDEV-4472 Audit-plugin. Server-related part of the task. > file_logger became the service. > Data like query_id now are sent to the audit plugin. > Fix for MDEV-4770 ported from 10.0. > Fix added for the read_maria_plugin_info(). > === modified file 'include/mysql/plugin_audit.h' > --- a/include/mysql/plugin_audit.h 2013-04-19 10:50:16 +0000 > +++ b/include/mysql/plugin_audit.h 2013-09-03 08:25:21 +0000 > @@ -25,7 +25,7 @@ > > #define MYSQL_AUDIT_CLASS_MASK_SIZE 1 > > -#define MYSQL_AUDIT_INTERFACE_VERSION 0x0301 > +#define MYSQL_AUDIT_INTERFACE_VERSION 0x0302 > > > /************************************************************************* > @@ -59,6 +59,10 @@ struct mysql_event_general > struct charset_info_st *general_charset; > unsigned long long general_time; > unsigned long long general_rows; > + /* Added for 0x302 */ A bit confusing. Better "Added in version 0x0302" > + long long query_id; should it rather be unsigned? > + const char *database; > + unsigned int database_length; > }; > > > @@ -140,6 +144,8 @@ struct mysql_event_table > unsigned int new_database_length; > const char *new_table; > unsigned int new_table_length; > + /* Added for 0x302 */ > + long long query_id; > }; > > /************************************************************************* > > === renamed file 'plugin/sql_errlog/sql_logger.cc' => 'mysys/file_logger.c' > --- a/plugin/sql_errlog/sql_logger.cc 2013-06-22 12:02:03 +0000 > +++ b/mysys/file_logger.c 2013-09-03 08:25:21 +0000 > @@ -51,6 +53,12 @@ LOGGER_HANDLE *logger_open(const char *p > unsigned int rotations) > { > LOGGER_HANDLE new_log, *l_perm; > + char *local_mysql_data_home; > +#ifdef _WIN32 > + local_mysql_data_home= (char *)GetProcAddress(0, "mysql_data_home"); > +#else > + local_mysql_data_home= mysql_data_home; > +#endif /*_WIN32*/ You shouldn't need it, file_logger.c is part of the server, not a dynamically loaded plugin. > > /* > I don't think we ever need more rotations, > === modified file 'sql/sql_audit.h' > --- a/sql/sql_audit.h 2013-04-19 10:50:16 +0000 > +++ b/sql/sql_audit.h 2013-09-03 08:25:21 +0000 > @@ -93,7 +93,8 @@ void mysql_audit_general_log(THD *thd, t > > mysql_audit_notify(thd, MYSQL_AUDIT_GENERAL_CLASS, MYSQL_AUDIT_GENERAL_LOG, > 0, time, user, userlen, cmd, cmdlen, > - query, querylen, clientcs, 0); > + query, querylen, clientcs, (ha_rows) 0, > + thd->db, thd->db_length); no query_id? > } > } > > @@ -139,7 +140,8 @@ void mysql_audit_general(THD *thd, uint > > mysql_audit_notify(thd, MYSQL_AUDIT_GENERAL_CLASS, event_subtype, > error_code, time, user, userlen, msg, msglen, > - query.str(), query.length(), query.charset(), rows); > + query.str(), query.length(), query.charset(), rows, > + thd->db, thd->db_length); no query_id? > } > } > > === modified file 'sql/sql_plugin.cc' > --- a/sql/sql_plugin.cc 2013-06-13 18:19:11 +0000 > +++ b/sql/sql_plugin.cc 2013-09-03 08:25:21 +0000 > @@ -2645,13 +2647,16 @@ static void update_func_longlong(THD *th > static void update_func_str(THD *thd, struct st_mysql_sys_var *var, > void *tgt, const void *save) > { > - char *old= *(char **) tgt; > - *(char **)tgt= *(char **) save; > + char *value= *(char**) save; > if (var->flags & PLUGIN_VAR_MEMALLOC) > { > - *(char **)tgt= my_strdup(*(char **) save, MYF(0)); > + char *old= *(char**) tgt; > + if (value) > + *(char**) tgt= my_strdup(value, MYF(0)); add else *(char**) tgt= 0; > my_free(old); > } > + else > + *(char**) tgt= value; > } > Regards, Sergei

2 3

[Maria-developers] RFC - CREATE PROCEDURE clauses (question about docs from mariadb-discuss)
by Roberto Spadim 02 Sep '13

02 Sep '13

Hi mariadb-devel guys, could anyone help here in a documentation problem? (maybe not a problem...) *Please answer the BOLD questions* We are talking about CREATE PROCEDURE , about DETERMINISTIC / NO SQL http://dev.mysql.com/doc/refman/5.6/en/create-procedure.html *characteristic*: COMMENT '*string*' | LANGUAGE SQL | *[NOT] DETERMINISTIC* | { CONTAINS SQL | *NO SQL* | READS SQL DATA | MODIFIES SQL DATA } | SQL SECURITY { DEFINER | INVOKER } *--------* *from mariadb-discuss:* *--------* Hi Federico, sorry i read some points and start writing, i will read and write after... i will answer with my experience as an user, not as a mariadb/mysql developer... 2013/9/2 Federico Razzoli <federico_raz(a)yahoo.it> > Hi Roberto > I'll try to be more clear, please read carefully before answering > ok I know what a deterministic expression is - but *what kinds of output must > be deterministic for a procedure to be defined as DETERMINISTIC?* > hummm i don't know, but for procedure, there isn't a deterministic output right? just for funcitons... maybe it's a parser problem, allow deterministic keyword for procedures (must check) i'm thinking about, *why we use deterministic procs, and non deterministic procs?* *this change something important in mysql server? replication, query cache? other performace considerations?* > There are many types of output for a proc: > * OUT params > * SELECTs > * writes on tables > * SET @var > hum.... from what i remember procedure ~= function without return, something like a "void FUNCTION( any parameters )" function ~= a procedure that return values, in other words it must be different from void... that's why i think it's a problem at parser allowing deterministic keyword at procedure, but must check... i don't know if this keyword change something in mysql/mariadb code > For a deterministic proc, do all of these need to be deterministic? > well, for me, procs are always non deterministics since it don't change query cache invalidation and replication (must be done in binary i think) > * I guess the answer isn't easy. > * *yeah, i see, we need a simple answer not a simplified answer* > -- > For "NO SQL" a complete list of SQL statements would be needed. If we use > this as a reference: > http://dev.mysql.com/doc/refman/5.6/en/sql-syntax.html > , even RETURN seems to be a SQL statement, yes, maybe the point isn't the SQL statements list, but other functions that mysql/mariadb server do? i don't know... NO SQL maybe is different from NOSQL (memcache,mongo,etc...) since nosql is newer than mysql procedures the point is *what's NO SQL in CREATE PROCEDURE ?* but of course it can't be true, because it's impossible to write a func > without RETURN. > yeah, maybe we have a problem at docs... let's see if anyone else know about this... maybe we need help from mariadb-devel list, i will include in this email... > > Greetings > Federico > bye =) thanks guys -- Roberto Spadim

1 0

Re: [Maria-developers] FWD: Status on MDEV-4506, parallel replication
by Kristian Nielsen 02 Sep '13

02 Sep '13

Michael Widenius <monty(a)askmonty.org> writes: >> On the master, I implemented --binlog-commit-wait-count=N and >> --binlog-commit-wait-usec=T. A transaction will wait at most T microseconds >> for at least N transactions to queue up and be ready for group commit. This >> allows to deliberately delay transactions on the master in order to get bigger >> group commits and thus better opportunity for parallel execution (and again it >> makes testing easier). > > Do you think the above helps in any real world case, except testing? > > Assuming we are have N=4 and T=2 > > - We have 3 threads ready to commit > - We wait 2 milliseconds and get 2 more threads. > - We commit now 5 threads. During this time you get 3 other threads > waiting to commit. > - We wait again... > > In a scenario without waiting: > > - We have 3 threads ready to commit > - We commit 3 threads. During this time we get 5 more threads waiting > to commit. > - We commit 5 threads. > > In other words, as the group commit will anyway take up 50 > microseconds on a hard disk, and thus automaticly group things > together for the next commit, why do we need ever to wait more? > > I agree that with --binlog-commit-wait-count you may get less sync > calls, but at an expense that a lot of threads took up to T > microseconds longer to execute. > > The worst case scenario is when you have only one user doing a lot of > inserts with auto-commit. In this case using wait will slow down the > server with T microseconds for every query. > > Have you been able to run any kind of benchmark where using > binglog-commit-wait-count will give more performance? > > I agree that the options are good for testing. The main question I > have is if we want to have the variables in the production server and > how we should document when and how a user should use the variables. I agree that with respect to group commit, the --binlog-commit-wait-count in many cases will not improve performance (that is why I did not implement it earlier). However, for parallel replication things are a bit different. Suppose that the application is doing C transaction commits per second, and that the disk system is capable of F binlog fsyncs per second. Now, if C is significantly bigger than F, then things are as you describe. Generally several transactions will queue up while the previous group commit runs, and there will be sufficient parallelism without using --bicommit_wait-*. This will be typical for eg. a simple harddisk-based system (F=40 commits/second perhaps) where all data is cached in the InnoDB buffer pool (eg. C>500 transactions/second). On the other hand, if C is smaller than F (or of similar magnitide), then usually only few or no new transactions will have time to queue up while the previous transaction is committing. So there will not be much parallelism for parallel replication to exploit without using --binlog-commit-wait-*. This will be typical for eg. a good-quality server with battery-backup RAID controller (F > 1000 commits/second) where data is too big to fit in buffer pool and every update requires disk access to complete (C < 500 transactions/second for example). And in fact, it is the second case, where random I/O is the bottleneck and multiple disk spindels are needed to improve I/O throughput, that single-threaded slave hurts the most, and where increasing --binlog-commit-wait-usec can be used with least penalty. I agree we need to document clearly the risk that --binlog-commit-wait-* will decrease performance on the master. Basically, the user can look at the ratio of Binlog_commits and Binlog_group_commits to check if enough transactions are part of each group commit for parallel replication to be effective. >> 1. The existing code is not thread-safe for class Relay_log_info. This class >> contains a bunch of stuff that is specific to executed transactions, not >> related to relay-log at all. This needs to be moved to the new struct >> rpl_group_info I introduced, and all code updated to pass around a pointer to >> that struct instead. There may also be a need to add additional locking on >> Relay_log_info, existing code needs review for this. > > I could take a look working on the above tomorrow and next week. Ok, great. Ping me when you can so we can coordinate, I may have some partial patches for this lying around. - Kristian.

1 0

[Maria-developers] Fwd: Bazaar-problem
by Nitish Upreti 31 Aug '13

31 Aug '13

I cant even clone the repository. Bazaar crashes with a bug midway. Can we have an active git mirror of the project for people just starting out? On Sat, Aug 31, 2013 at 2:13 PM, Sergei Golubchik <serg(a)mariadb.org> wrote: > Hi, AugustQ! > > On Aug 31, AugustQ wrote: > > I'm struggling with Bazaar. Maybe you can help me. > > > > I've made some modifications to the source-code and committed these to > > the local repository. Next I tried to upload them via push. This is what > > I receive: > > > > Cannot lock LockDir(chroot-70341520:///% > > 2Bbranch/maria/.bzr/branch/lock): Transport operation not possible: > > readonly transport > > You don't have write access to the mariadb source repository. > That's why you can only pull, but you cannot push. > > Regards, > Sergei > > _______________________________________________ > Mailing list: https://launchpad.net/~maria-developers > Post to : maria-developers(a)lists.launchpad.net > Unsubscribe : https://launchpad.net/~maria-developers > More help : https://help.launchpad.net/ListHelp >

2 1

[Maria-developers] introduction
by AugustQ 29 Aug '13

29 Aug '13

Hi, i've joined this list some days ago because I want to participate in the discussions here. The following links give you an overview of what i've done in the past: without registration: https://www.xing.com/profiles/August_Quint with registration: http://www.linkedin.com/profile/view?id=9080630 unfortunately the texts are in German, so in short: i've worked as a software-developer mainly in the backend area. I've made some small modifications on the source code but currently I'm struggling with Bazaar. This should be solved soon and I will then upload some code to the repository. I hope I can help you a bit. AugustQ PS: i found this: in MySQL I call cmake with an argument: -DWITH_DEBUG=1 in MariaDB this is: -DCMAKE_BUILD_TYPE=Debug is this correct?

2 1

[Maria-developers] alter online table documentation
by Roberto Spadim 26 Aug '13

26 Aug '13

Hi guys, i was searching alter online docs, at mariadb https://kb.askmonty.org/en/alter-table-statement/ but didn't found anything, could we add ? thanks! -- Roberto Spadim

4 5

Re: [Maria-developers] Calll for testing: MariaDB 5.5.32 packages
by Otto Kekäläinen 23 Aug '13

23 Aug '13

2013/8/23 Сергей Войтович <svojtovich(a)gmail.com>: > thanks for packaging MariaDB. I did a few basic tests with your packages and I confirm that they work well. > The only minor note I have is that I was asked for root password 3x during installation, which is a bit too much. Thanks! I haven't noticed any extra password asking during installation. It should ask for MariaDB root password in two screens, and of course if you run sudo apt-get it will ask once for your user password in the beginning. Could you clarify a little what happened in you installation?

2 3

[Maria-developers] Additional documentation for GTID
by Kristian Nielsen 23 Aug '13

23 Aug '13

Hi Daniel, Ian, I have implemented a new system variable for global transaction ID and pushed it into 10.0-base (so it should appear in 10.0.5 I suppose). I have written some text for documenting the new variable, appended below. Can one of you help me get this text into the Knowledgebase with proper formatting and cross-referencing across relevant pages? - Kristian. ----------------------------------------------------------------------- Variable: gtid_binlog_state Scope: global Dynamic: Yes Type: String The variable gtid_binlog_state holds the internal state of the binlog. The state consists of the last GTID ever logged to the binary log for every combination of domain_id and server_id. This information is used by the master to determine whether a given GTID has been logged to the binlog in the past, even if it has later been deleted due to binlog purge. For each domain_id, the last entry in @@gtid_binlog_state is the last GTID logged into binlog, ie. this is the value that appears in @@gtid_binlog_pos. Normally this internal state is not needed by users, as @@gtid_binlog_pos is more useful in most cases. The main usage of @@gtid_binlog_state is to restore the state of the binlog after RESET MASTER (or equivalently if the binlog files are lost). If the value of @@gtid_binlog_state is saved before RESET MASTER and restored afterwards, the master will retain information about past history, same as if PURGE BINARY LOGS had been used (of course the actual events in the binary logs are still deleted). Note that to set the value of @@gtid_binlog_state, the binary log must be empty, that is it must not contain any GTID events and the previous value of @@gtid_binlog_state must be the empty string. If not, then RESET MASTER must be used first to erase the binary log first. For completeness, note that setting @@gtid_binlog_state internally executes a RESET MASTER. This is normally not noticable as it can only be changed when the binlog is empty of GTID events. However, if executed eg. immediately after upgrading to MariaDB 10, it is possible that the binlog is non-empty but without any GTID events, in which case all such events will be deleted, just as if RESET MASTER had been run.

2 1

[Maria-developers] Error in 10.0.4 changelog, "Partition lock pruning"?
by Sergei Petrunia 23 Aug '13

23 Aug '13

Hello, Igor has pointed out that the changelog for 10.0.4 https://kb.askmonty.org/en/mariadb-1004-release-notes/ has this line: Partition lock pruning (MySQL Bug #37252, Bug #11748732) while the feature doesn't seem to be there. I did some investigation: - The line was in the changelog from the very first edit, it was put there by Daniel (who currently is on vacation) - I was unable to find any trace of MySQL Bug #37252 or Bug #11748732 in current 10.0 tree. - It seems, the whole "partition lock pruning" feature so far has not been ported to MariaDB 10.0. Does anybody know anything else about this? Should we remove the line from the changelog? BR Sergei -- Sergei Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

3 3

Re: [Maria-developers] Fwd: [Commits] Rev 3674: MDEV-4645: Incorrect reads of frozen binlog events; FDE corrupted in relay log in file:///home/bell/maria/bzr/work-maria-10.0-base-MDEV-4645/
by Kristian Nielsen 23 Aug '13

23 Aug '13

Oleksandr Byelkin <sanja(a)montyprogram.com> writes: > Could you review this? > > (it is https://mariadb.atlassian.net/browse/MDEV-4645 ) Ok to push. - Kristian.

1 0

[Maria-developers] Buildbot failure in 10.0-base
by Sergei Petrunia 23 Aug '13

23 Aug '13

Hi Kristian, 10.0-base started to get compile failures on [nearly] all hosts after your yesterday push: > 3682. By Kristian Nielsen 22 hours ago > MDEV-4488: When master is on the list of ignore_server_ids, GTID position on > slave is not updated > > The ignored events are not written to the relay log, but instead a fake > Rotate event is generated to handle update of position. > > Extend this for Gtid so we similarly generate a fake Gtid_list event > to update the GTID position. > > Also fix an unrelated test issue that got triggered by the added test cases. Could you please look into this? BR Sergei -- Sergei Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

2 1

[Maria-developers] MariaDB mutex contention (was: Re: MariaDB 10.0 benchmarks on upcoming IVB-EP processors ...)
by Axel Schwenke 22 Aug '13

22 Aug '13

Hi, Michael Widenius wrote: > 10.0.4 should be very soon ready for benchmarking (Tuesday or > Wednesday). We are in the final stage of fixing the failing test cases > after our last merge with MySQL 5.6 GA. > > I would just prefer to see Sergei Vojtovich push his code that changes > how the table cache works. > > This solves the same problem as the split table cache in 5.6, > but is a much simpler and better implementation. > > After this, I would expect that MariaDB 10.0 performance would be > closer to 5.6 performance for this benchmark. > Axel, could you at some point run the same benchmark with MariaDB > 10.0.4 and MySQL 5.6 with gprof to see where time is spent? I have now beefed up my knowledge of performance_schema and used it to look at Mutex/RW-lock contention for readonly OLTP benchmark. ----- TL;DR: 10.0.4 spends very long time in LOCK_PLUGIN, LOCK_OPEN is indeed much better than 10.0.3 but not(?) nearly as good as MySQL-5.6. ----- Contestants: - MySQL-5.6.13 (release) - MariaDB-10.0.3 (release) - MariaDB-10.0 r3799 (bzr trunk) all 3 compiled from source on benchmark machine using the same build script Environment: - many core machine (32 cores, 64 threads) - sysbench-0.5 OLTP readonly, 10M rows, single table - mysqld bound to 48 cores (sysbench to the other 8) - tested with 64 threads (this is the sweet spot) - using tcmalloc (preloaded) - mutexes and rw locks fully instrumented via performance_schema, see my.cnf Benchmark results (transactions per second): P_S on P_S off MySQL: ~15K ~18K 10.0.3: ~7K ~7.5K 10.0.4: ~6.5K ~9K After stabilizing I took 3 probes from P_S, sorting by sum_timer_wait desc. Times shown by P_S are picoseconds. If this were seconds, the decimal point would be in the column between "m" and "_" in "sum_timer_wait". So i.e. 10.0.4 waited 242 seconds for LOCK_PLUGIN during the 10 second measurement interval. Since we have 48 cores, there are total 480 seconds and 10.0.4 wasted about 50% of that. Questions: - what is protected by LOCK_PLUGIN? - how is LOCK_OPEN split in MySQL-5.6?If some of that is now covered by THR_LOCK::mutex, then it would explain why this is the busiest mutex in MySQL-5.6. Let me know if you need more data from me or if you have ideas what else I should test. XL Top 20 Mutexes/RW-locks MySQL-5.6.13 +---------------------------------------------+------------+-----------------+ | event_name | count_star | sum_timer_wait | +---------------------------------------------+------------+-----------------+ | wait/synch/mutex/mysys/THR_LOCK::mutex | 3397732 | 123367802416236 | | wait/synch/rwlock/innodb/btr_search_latch | 1632176 | 22207357475142 | | wait/synch/rwlock/sql/LOCK_grant | 1682983 | 1526912505972 | | wait/synch/mutex/sql/LOCK_table_cache | 3397786 | 1247140492941 | | wait/synch/mutex/sql/THD::LOCK_thd_data | 9219327 | 1076510130276 | | wait/synch/rwlock/innodb/hash_table_locks | 4086413 | 1026755227155 | | wait/synch/mutex/innodb/trx_sys_mutex | 364146 | 639270297372 | | wait/synch/mutex/innodb/os_mutex | 149491 | 630758755641 | | wait/synch/rwlock/innodb/index_tree_rw_lock | 1016436 | 344264365074 | | wait/synch/rwlock/sql/MDL_lock::rwlock | 242778 | 183463650006 | | wait/synch/mutex/sql/MDL_map::mutex | 121342 | 69121460394 | | wait/synch/mutex/mysys/BITMAP::mutex | 242667 | 54426177477 | | wait/synch/mutex/innodb/fil_system_mutex | 179586 | 20229032103 | | wait/synch/mutex/sql/LOCK_plugin | 121352 | 15004585617 | | wait/synch/mutex/innodb/trx_mutex | 121475 | 13105527141 | | wait/synch/mutex/innodb/lock_mutex | 121374 | 12627692868 | | wait/synch/mutex/innodb/buf_pool_mutex | 2240 | 228234006 | | wait/synch/mutex/innodb/log_sys_mutex | 540 | 119752122 | | wait/synch/mutex/innodb/flush_list_mutex | 704 | 86802870 | | wait/synch/mutex/innodb/buf_dblwr_mutex | 640 | 30728817 | +---------------------------------------------+------------+-----------------+ MariaDB-10.0.3 +------------------------------------------------+------------+-----------------+ | event_name | count_star | sum_timer_wait | +------------------------------------------------+------------+-----------------+ | wait/synch/mutex/sql/LOCK_open | 3334966 | 251668176449112 | | wait/synch/mutex/mysys/THR_LOCK::mutex | 2206782 | 2761461343332 | | wait/synch/mutex/sql/LOCK_plugin | 2540899 | 1471913471196 | | wait/synch/mutex/sql/THD::LOCK_thd_data | 6485120 | 770989733856 | | wait/synch/rwlock/innodb/hash table locks | 3315788 | 674878255920 | | wait/synch/rwlock/innodb/btr_search_latch | 1358771 | 540950632524 | | wait/synch/mutex/innodb/trx_sys_mutex | 380210 | 245662486416 | | wait/synch/rwlock/sql/MDL_lock::rwlock | 191698 | 125257499748 | | wait/synch/mutex/sql/TABLE_SHARE::LOCK_ha_data | 1067704 | 95258676492 | | wait/synch/mutex/mysys/THR_LOCK_lock | 157621 | 52329482172 | | wait/synch/mutex/mysys/BITMAP::mutex | 154870 | 49415303736 | | wait/synch/mutex/sql/MDL_map::mutex | 79173 | 41936664096 | | wait/synch/mutex/innodb/fil_system_mutex | 129610 | 40258209024 | | wait/synch/mutex/innodb/lock_mutex | 93667 | 26848457928 | | wait/synch/mutex/innodb/trx_mutex | 94131 | 11398946760 | | wait/synch/mutex/innodb/buf_pool_mutex | 12672 | 571983480 | | wait/synch/mutex/innodb/buf_dblwr_mutex | 9152 | 312365052 | | wait/synch/mutex/innodb/log_sys_mutex | 456 | 138632868 | | wait/synch/mutex/innodb/flush_list_mutex | 480 | 109226268 | | wait/synch/mutex/innodb/os_mutex | 350 | 71117100 | +------------------------------------------------+------------+-----------------+ MariaDB-10.0-r3799 +---------------------------------------------+------------+-----------------+ | event_name | count_star | sum_timer_wait | +---------------------------------------------+------------+-----------------+ | wait/synch/mutex/sql/LOCK_plugin | 4194896 | 242595666533160 | | wait/synch/mutex/sql/LOCK_open | 1835243 | 33790056812460 | | wait/synch/mutex/mysys/THR_LOCK::mutex | 1835221 | 3136249420092 | | wait/synch/rwlock/sql/LOCK_tdc | 911928 | 1088553356028 | | wait/synch/mutex/sql/THD::LOCK_thd_data | 5766813 | 695664363024 | | wait/synch/rwlock/innodb/hash table locks | 2144715 | 513517462020 | | wait/synch/rwlock/innodb/btr_search_latch | 915043 | 399309801912 | | wait/synch/rwlock/sql/MDL_lock::rwlock | 131171 | 176610102480 | | wait/synch/mutex/innodb/trx_sys_mutex | 196751 | 154609451460 | | wait/synch/rwlock/innodb/index_tree_rw_lock | 527543 | 137903039712 | | wait/synch/mutex/sql/MDL_map::mutex | 158268 | 95133455712 | | wait/synch/mutex/innodb/fil_system_mutex | 97493 | 27285605172 | | wait/synch/mutex/mysys/BITMAP::mutex | 131099 | 26589125196 | | wait/synch/mutex/mysys/THR_LOCK_lock | 131105 | 23952666708 | | wait/synch/mutex/innodb/lock_mutex | 65579 | 10355822676 | | wait/synch/mutex/innodb/trx_mutex | 65812 | 9790097724 | | wait/synch/mutex/innodb/buf_pool_mutex | 11550 | 644569236 | | wait/synch/rwlock/innodb/checkpoint_lock | 2 | 236298492 | | wait/synch/mutex/innodb/buf_dblwr_mutex | 3880 | 156907368 | | wait/synch/mutex/innodb/flush_list_mutex | 692 | 124740900 | +---------------------------------------------+------------+-----------------+ Comparison ---------- MariaDB-10.0-r3799 MariaDB-10.0.3 MySQL-5.6.13 +---------------------------------------------+------------+-----------------+------------+-----------------+------------+-----------------+ | event_name | count_star | sum_timer_wait | count_star | sum_timer_wait | count_star | sum_timer_wait | +---------------------------------------------+------------+-----------------+------------+-----------------+------------+-----------------+ | wait/synch/mutex/sql/LOCK_plugin | 4194896 | 242595666533160 | 2540899 | 1471913471196 | 121352 | 15004585617 | | wait/synch/mutex/sql/LOCK_open | 1835243 | 33790056812460 | 3334966 | 251668176449112 | | wait/synch/mutex/mysys/THR_LOCK::mutex | 1835221 | 3136249420092 | 2206782 | 2761461343332 | 3397732 | 123367802416236 | | wait/synch/rwlock/sql/LOCK_tdc | 911928 | 1088553356028 | | wait/synch/mutex/sql/THD::LOCK_thd_data | 5766813 | 695664363024 | 6485120 | 770989733856 | 9219327 | 1076510130276 | | wait/synch/rwlock/innodb/hash table locks | 2144715 | 513517462020 | 3315788 | 674878255920 | 4086413 | 1026755227155 | | wait/synch/rwlock/innodb/btr_search_latch | 915043 | 399309801912 | 1358771 | 540950632524 | 1632176 | 22207357475142 | | wait/synch/rwlock/sql/MDL_lock::rwlock | 131171 | 176610102480 | 191698 | 125257499748 | 242778 | 183463650006 | | wait/synch/mutex/innodb/trx_sys_mutex | 196751 | 154609451460 | 380210 | 245662486416 | 364146 | 639270297372 | | wait/synch/rwlock/innodb/index_tree_rw_lock | 527543 | 137903039712 | 1016436 | 344264365074 | | wait/synch/mutex/sql/MDL_map::mutex | 158268 | 95133455712 | 79173 | 41936664096 | 121342 | 69121460394 | | wait/synch/mutex/innodb/fil_system_mutex | 97493 | 27285605172 | 129610 | 40258209024 | 179586 | 20229032103 | | wait/synch/mutex/mysys/BITMAP::mutex | 131099 | 26589125196 | 154870 | 49415303736 | 242667 | 54426177477 | | wait/synch/mutex/mysys/THR_LOCK_lock | 131105 | 23952666708 | 157621 | 52329482172 | | wait/synch/mutex/innodb/lock_mutex | 65579 | 10355822676 | 93667 | 26848457928 | 121374 | 12627692868 | | wait/synch/mutex/innodb/trx_mutex | 65812 | 9790097724 | 94131 | 11398946760 | 121475 | 13105527141 | | wait/synch/mutex/innodb/buf_pool_mutex | 11550 | 644569236 | 12672 | 571983480 | 2240 | 228234006 | | wait/synch/rwlock/innodb/checkpoint_lock | 2 | 236298492 | | wait/synch/mutex/innodb/buf_dblwr_mutex | 3880 | 156907368 | 9152 | 312365052 | 640 | 30728817 | | wait/synch/mutex/innodb/flush_list_mutex | 692 | 124740900 | 480 | 109226268 | 704 | 86802870 | +---------------------------------------------+------------+-----------------+------------+-----------------+------------+-----------------+

7 18

[Maria-developers] Flashback Feature patch review
by Lixun Peng 22 Aug '13

22 Aug '13

Hi Monty, I patched my flashback feature to 10.0-base, and this is the diff file, please help me to review it. How to use this feature? I added several new arguments for mysqlbinlog: --flashback (-B), it will let mysqlbinlog working on FLASHBACK mode. --table (-T), it likes -d, but this argument is for table. --review, it will le mysqlbinlog print the SQL for reviewing. Reviewing feature will create a new "review" table to record the data that will be changed by FLASHBACK feature. If you aren't special the --review-dbname/tablename, "review" table will created on current DB, and the table name will be "__"+original_table_name. And for getting the original table struct, mysqlbinlog need to connect mysql, so --user/--host/--password is needed if necessary. --review-dbname, the DB that you want to store the review table. --review-tablename the TABLE that you want to store the changed data. Only if you set -T, this argument is useful. For example: I have a table "t" in database "test", CREATE TABLE `t` ( `id` int(10) unsigned NOT NULL AUTO_INCREMENT, `info` varchar(32) DEFAULT NULL, PRIMARY KEY (`id`) ) ENGINE=InnoDB AUTO_INCREMENT=9043880 DEFAULT CHARSET=utf8; And I done this operation, root@localhost : (none) 14:52:50> reset master; root@localhost : (none) 14:53:02> use test root@localhost : test 14:53:08> delete from t limit 10; root@localhost : test 14:53:25> insert into t (info) select "你好"; root@localhost : test 14:54:01> update t set info ='hello你好' limit 5; We can compare the output with --flashback or not. #client/mysqlbinlog /data/mysqldata_10.0/binlog/mysql-bin.000001 -vv -d test -T t --start-datetime="2013-03-27 14:54:00" > /tmp/1.sql #client/mysqlbinlog /data/mysqldata_10.0/binlog/mysql-bin.000001 -vv -uroot -d test -T t --start-datetime="2013-03-27 14:54:00" -B --review > /tmp/2.sql How to implement Flashback: 1. mysqlbinlog tool can parse the binlog files and print them, so we can use the most of code in mysqlbinlog. 2. Changing the event type (INSERT -> DELETE, DELETE -> INSERT) 3. Exchanging the SET/WHERE part in UPDATE (UPDATE table SET x=1 WHERE y=2 -> UPDATE table SET y=2 WHERE x=1) 4. Invert the event output, from end to beginning (binlog: e1,e2,e3... -> flashback_log: en...e3,e2,e1) Then, importing the flashback output can flashback database/table to the special time. But binlog_format should be ROW. Thanks, Lixun -- Senior MySQL Developer @ Taobao.com Mobile Phone: +86 18658156856 (Hangzhou) Gtalk: penglixun(at)gmail.com Twitter: http://www.twitter.com/plinux Blog: http://www.penglixun.com

2 3

[Maria-developers] MariaDB config files order
by Honza Horak 20 Aug '13

20 Aug '13

Hi guys, maybe I just need some little kick or I should get a coffee, but I can't understand the following config file ordering difference. Reading [1] and code in mariadb-5.5.32/mysys/default.c:1226, it looks like expected order of config files in Linux should be: /etc/my.cnf /etc/mysql/my.cnf ~/.my.cnf However, when asking for that using --help --verbose, I get a bit different output: $ /usr/libexec/mysqld --help --verbose 2>/dev/null | grep --after=1 '^Default options' | tail -n 2 Default options are read from the following files in the given order: /etc/mysql/my.cnf /etc/my.cnf ~/.my.cnf I don't understand why /etc/my.cnf is not on the first place in that output. Can somebody give me a hint? Am I just blind? Just FTR I'm trying that on Fedora 19, but there are no changes in Fedora packages that would do anything related, so I guess the same result would be in other distros as well. [1] https://kb.askmonty.org/en/mysqld-startup-options/ Cheers, Honza

2 3

Re: [Maria-developers] [Commits] Rev 3677: Fix bug MDEV-4895 Valgrind warnings (Conditional jump or move depends on uninitialised value) in Field_datetime::get_date on GREATEST(..) IS NULL in file:///home/tsk/mprog/src/5.3/
by Alexander Barkov 20 Aug '13

20 Aug '13

Hi Timour, The patch looks Okey to push for me. Please find two small neat-peaking comments below though :) On 08/20/2013 04:36 PM, timour(a)askmonty.org wrote: > At file:///home/tsk/mprog/src/5.3/ > > ------------------------------------------------------------ > revno: 3677 > revision-id: timour(a)askmonty.org-20130820123610-cjtgvaybl1ip3128 > parent: igor(a)askmonty.org-20130815235920-io2h7tlypwlbunsp > fixes bug: https://mariadb.atlassian.net/browse/MDEV-4895 > committer: timour(a)askmonty.org > branch nick: 5.3 > timestamp: Tue 2013-08-20 15:36:10 +0300 > message: > Fix bug MDEV-4895 Valgrind warnings (Conditional jump or move depends on uninitialised value) in Field_datetime::get_date on GREATEST(..) IS NULL > > Analysis: > The cause of the valgrind warning was an attempt to evaluate a Field that was not yet read. > The reason was that on one hand Item_func_isnotnull was marked as constant by > Item_func_isnotnull::update_used_tables, and this allowed eval_const_cond() to be called. > On the other hand Item_func_isnotnull::val_int() evaluated its argument as if it was not > constant. > > Solution: > The fix make sure that Item_func_isnotnull::val_int() doesn't evaluate its argument when > it is constant and cannot be NULL, because the result is known in this case. > > > > === modified file 'mysql-test/r/null.result' > --- a/mysql-test/r/null.result 2011-11-21 13:16:16 +0000 > +++ b/mysql-test/r/null.result 2013-08-20 12:36:10 +0000 > @@ -343,3 +343,33 @@ Field Type Null Key Default Extra > IFNULL(NULL, b) decimal(1,0) YES NULL > DROP TABLE t1, t2; > # End of 5.0 tests > +# > +# MDEV-4895 Valgrind warnings (Conditional jump or move depends on uninitialised value) in Field_datetime::get_date on GREATEST(..) IS NULL > +# > +CREATE TABLE t1 (dt DATETIME NOT NULL); > +INSERT INTO t1 VALUES (NOW()),(NOW()); > +EXPLAIN > +SELECT * FROM t1 WHERE concat( dt, '2012-12-21 12:12:12' ) IS NULL; > +id select_type table type possible_keys key key_len ref rows Extra > +1 SIMPLE NULL NULL NULL NULL NULL NULL NULL Impossible WHERE > +SELECT * FROM t1 WHERE concat( dt, '2012-12-21 12:12:12' ) IS NULL; > +dt > +drop table t1; > +CREATE TABLE t1 (dt INT NOT NULL); > +INSERT INTO t1 VALUES (1),(2); > +EXPLAIN > +SELECT * FROM t1 WHERE concat( dt, '1' ) IS NULL; > +id select_type table type possible_keys key key_len ref rows Extra > +1 SIMPLE NULL NULL NULL NULL NULL NULL NULL Impossible WHERE > +SELECT * FROM t1 WHERE concat( dt, '1' ) IS NULL; > +dt > +drop table t1; > +CREATE TABLE t1 (dt INT NOT NULL); > +INSERT INTO t1 VALUES (1),(2); > +EXPLAIN > +SELECT * FROM t1 WHERE NOT (concat( dt, '1' ) IS NOT NULL); > +id select_type table type possible_keys key key_len ref rows Extra > +1 SIMPLE NULL NULL NULL NULL NULL NULL NULL Impossible WHERE > +SELECT * FROM t1 WHERE NOT (concat( dt, '1' ) IS NOT NULL); > +dt > +drop table t1; Please fix "drop table" to upper case here, and in the other places. > > === modified file 'mysql-test/t/null.test' > --- a/mysql-test/t/null.test 2009-02-10 03:00:11 +0000 > +++ b/mysql-test/t/null.test 2013-08-20 12:36:10 +0000 > @@ -255,3 +255,31 @@ DESCRIBE t2; > DROP TABLE t1, t2; > > --echo # End of 5.0 tests > + > +--echo # > +--echo # MDEV-4895 Valgrind warnings (Conditional jump or move depends on uninitialised value) in Field_datetime::get_date on GREATEST(..) IS NULL > +--echo # > + > +CREATE TABLE t1 (dt DATETIME NOT NULL); > +INSERT INTO t1 VALUES (NOW()),(NOW()); > + > +EXPLAIN > +SELECT * FROM t1 WHERE concat( dt, '2012-12-21 12:12:12' ) IS NULL; > +SELECT * FROM t1 WHERE concat( dt, '2012-12-21 12:12:12' ) IS NULL; > + > +drop table t1; > +CREATE TABLE t1 (dt INT NOT NULL); > +INSERT INTO t1 VALUES (1),(2); > +EXPLAIN > +SELECT * FROM t1 WHERE concat( dt, '1' ) IS NULL; > +SELECT * FROM t1 WHERE concat( dt, '1' ) IS NULL; > + > +drop table t1; > +CREATE TABLE t1 (dt INT NOT NULL); > +INSERT INTO t1 VALUES (1),(2); > + > +EXPLAIN > +SELECT * FROM t1 WHERE NOT (concat( dt, '1' ) IS NOT NULL); > +SELECT * FROM t1 WHERE NOT (concat( dt, '1' ) IS NOT NULL); > + > +drop table t1; > > === modified file 'sql/item_cmpfunc.cc' > --- a/sql/item_cmpfunc.cc 2013-08-15 23:59:20 +0000 > +++ b/sql/item_cmpfunc.cc 2013-08-20 12:36:10 +0000 > @@ -4621,6 +4621,8 @@ Item *and_expressions(Item *a, Item *b, > longlong Item_func_isnull::val_int() > { > DBUG_ASSERT(fixed == 1); > + if (const_item() && !args[0]->maybe_null) > + return false; "return 0" should be better in a longlong function. > return args[0]->is_null() ? 1: 0; > } > > @@ -4628,6 +4630,8 @@ longlong Item_is_not_null_test::val_int( > { > DBUG_ASSERT(fixed == 1); > DBUG_ENTER("Item_is_not_null_test::val_int"); > + if (const_item() && !args[0]->maybe_null) > + DBUG_RETURN(1); > if (args[0]->is_null()) > { > DBUG_PRINT("info", ("null")); >

1 0

Re: [Maria-developers] [Commits] MDEV-4923 file://home/jan/mysql/mariadb-5.5-mdev4923
by Sergei Golubchik 20 Aug '13

20 Aug '13

Hi, Jan! On Aug 20, Jan Lindström wrote: > Serg, > > Could you please review fix for MDEC-4923, where these changes are > pushed lp:maria/5.5 or lp:~maria-captains/maria/5.5 ? Looks correct. This seems to be XtraDB bug. lp:maria/5.5 is an alias for lp:~maria-captains/maria/5.5 > === modified file 'storage/xtradb/os/os0file.c' > --- storage/xtradb/os/os0file.c 2013-07-29 14:03:41 +0000 > +++ storage/xtradb/os/os0file.c 2013-08-20 05:35:22 +0000 > @@ -1658,8 +1658,8 @@ > } > } > > - if (srv_use_atomic_writes && type == OS_DATA_FILE && > - os_file_set_atomic_writes(file, name)) { > + if (srv_use_atomic_writes && type == OS_DATA_FILE && > + !os_file_set_atomic_writes(file, name)) { > CloseHandle(file); > *success = FALSE; > file = INVALID_HANDLE_VALUE; > @@ -1786,7 +1786,7 @@ > #endif /* USE_FILE_LOCK */ > > if (srv_use_atomic_writes && type == OS_DATA_FILE > - && os_file_set_atomic_writes(name, file)) { > + && !os_file_set_atomic_writes(name, file)) { > > *success = FALSE; > close(file); > Regards, Sergei

1 0

[Maria-developers] [Otto Kekäläinen] Fwd: initial feedback on mariadb
by Kristian Nielsen 20 Aug '13

20 Aug '13

Hi, Otto asked for help with the copyright file for the Debian MariaDB packages. As I understand it, the .deb packages have a file /usr/share/doc/mariadb-server-5.5/copyright and there are some scripts in Debian that check the file against what is in the headers of source files and possibly other checks (error output attached below). Otto did something, but there is apparently more needed. Can some of you who understand the finer points of licensing among Oracle, Monty Program, MariaDB Services, MariaDB foundation, Percona, ... please look into this and help Otto fix up things? Debian are usually very careful with licensing, so this could easily become a blocker otherwise (I assume). - Kristian.

4 4

[Maria-developers] MDEV-4911 - KILL command with query id
by Roberto Spadim 20 Aug '13

20 Aug '13

Hi guys, i sent some patchs, could anyone check it? it have some things to do: query syntax isn't in the MDEV-4911, i did a QUERY_ID instead of a more comlpex syntax (since i never used bison before :P ) the query_id works, but i need a new error message ( "Unknown query id: %") well i didn't make the gcc happy, there's some warnings, and i don't know if it's all right i send this patch after MDEV-4916 patch, in other words the %expect value of sql_yacc.yy is wrong if you don't apply the MDEV-4916 patch the 4916 is a IF_IDLE keyword to KILL command too it only kill if there's no query running well :) thanks guys! <https://mariadb.atlassian.net/browse/MDEV-4916> -- Roberto Spadim

2 3

Re: [Maria-developers] More Secure Kill Command
by Sergei Golubchik 20 Aug '13

20 Aug '13

Hi, Roberto! On Aug 17, Roberto Spadim wrote: > Hi sergei, > What about give two informations, thread id and query id > If thread id and query id is equal, kill it There's no need to. Query id identifies the query and the connection unambiguously. Specifying also a thread id is redundant. Regards, Sergei

2 9

[Maria-developers] linking an external library
by rg 19 Aug '13

19 Aug '13

Folks, I want to link an external library for a language extension. After several hours of trying to tell cmake (^%$@*&) to pick it up for linking the library would still not be built with. CMakeLists.txt in trunk/sql is in the meantime full of pointers to the library without avail. I did the normal FIND_LIBRARY() followed by TARGET_LINK_LIBRARIES({$LIBNAME}) on various targets plus as nothing worked: SET(CMAKE_LIBRARY_PATH) on it, explicity LINK_DIRECTORIES(), SET(CMAKE_PREFIX_PATH), SET(CMAKE_LIBRARY_PATH) ... How do I link mariadb with an external library?? Thanks Rob

3 4

[Maria-developers] Audit plugin and qc_info plugin - MDEV-4682
by Roberto Spadim 18 Aug '13

18 Aug '13

Hi guys, i put some fields in qc_info plugin https://mariadb.atlassian.net/browse/MDEV-4581 the second part was add some others informations (stats): https://mariadb.atlassian.net/browse/MDEV-4682 i was reading the "response_time_distribution" of percona, patch at https://mariadb.atlassian.net/browse/MDEV-4568 could i use the audit plugin to implement MDEV-4682? what i need is: 1)know if the query was executed with/without query cache 2)know the time expend (lock time, and total time) to execute the query (first execution) 3)know the time expend with query cache hit 4) both (3) and (2) don't include time of network packages, just server side times... is it possible? any idea how could i implement it? i didn't write a audit plugin yet =( well.. time to learn =D thanks guys! -- Roberto Spadim

2 2

[Maria-developers] ALTER ONLINE TABLE syntax
by Sergei Golubchik 18 Aug '13

18 Aug '13

Hi, You've added this ALTER ONLINE TABLE syntax, where ONLINE de facto means "without copying the data". Now, after 5.6 merge we also have ALGORITHM=COPY|INPLACE and LOCK=NONE|SHARED|EXCLUSIVE. I've mapped ONLINE to ALGORITHM=INPLACE, because that's what it means. But it's confusing, one can have an "inplace" operation under an exclusive lock, if the engine wants it that way. And it won't be "online" by any stretch of imagination. One can also technically have a "copy" operation with a very permissive write-allow-write lock, and it'll be as online as one can get it. I mean, logically ONLINE should mean LOCK=NONE, not ALGORITHM=INPLACE. But that would change the existing behavior and the test results too. For example, *any* ALTER on the temporary table would always be "online", because, well, it wouldn't take any lock on that table. Alternatively, we can keep ONLINE as ALGORITHM=INPLACE, and deprecate it, issue a warning, suggesting to use the new 5.6 syntax. Opinions? Regards, Sergei

5 7

[Maria-developers] MDEV-4916
by Roberto Spadim 18 Aug '13

18 Aug '13

Patch uploaded, and test work (but i didn't add a unit test yet :/ sorry i don't know yet how to write it) -- Roberto Spadim

1 0

[Maria-developers] Joining the team
by Esteban Orduno 17 Aug '13

17 Aug '13

Hi, all: I would like to contribute to the maria-db team. Who can I talk iniatially to get some type of orientation about it? Esteban Orduno

2 1

[Maria-developers] More Secure Kill Command
by Roberto Spadim 17 Aug '13

17 Aug '13

Hi guys i was mailing at maria-discuss, but i think it a new feature, and want some view point from developers... check this problem: connection 1) big select... connection 2) show processlist connection 1) other query connection 2) kill connection 1 id (i take +- 1 second between show processlist and this command) connection 1) killed the problem? the kill connection 1 (KILL thread_id) killed the wrong query could we implement something like: KILL [CONNECTION | QUERY] thread_id [QUERY "some relative unique query id in this database"] the relative unique query id, could be: substr(sha1 or md5 of (thread_id + query start time),0,6) (6 hex numbers is good?) it's like a git small hash commit id, maybe we could make it more unique... but since i will not use it very often, i think a less intensive work is nice here... a global query id could add more one lock for every query... example: KILL 1 QUERY "abcdef" the "abcdef" = substr(MD5( thread id + query start time ),0,6) we could add client ip or others unique information about this query, but something that don't increase (ok just a little) the load of a show processlist / select * from information_schema thanks -- Roberto Spadim

4 7

[Maria-developers] MariaDB 10.0.4 Release Prep
by Daniel Bartholomew 16 Aug '13

16 Aug '13

All, I've begun prep work for the MariaDB 10.0.4 release. Draft Release Notes and Changelogs are here: - https://kb.askmonty.org/en/mariadb-1004-release-notes/ - https://kb.askmonty.org/en/mariadb-1004-changelog/ Please let me know ASAP if anything needs to be added/changed/etc... Thanks. -- Daniel Bartholomew Google+ - http://gplus.to/dbart Twitter - http://twitter.com/daniel_bart MariaDB: An Enhanced Drop-in Replacement for MySQL Website - http://mariadb.org Twitter - http://twitter.com/mariadb Google+ - http://google.com/+mariadb Facebook - http://fb.com/MariaDB.dbms Knowledgebase - http://kb.askmonty.org

3 4

[Maria-developers] RFC - query cache insert
by Roberto Spadim 16 Aug '13

16 Aug '13

Hi guys! I was reading (again) the query cache source code of 10.0.3 ... please tell me if i understood right when we insert at query cache... thread put a lock in query cache (no one can result from query cache) thread read all data and write to network (caching the result to query cache) after end of thread real the query cache is unlocked.. my doubts are... what happen if the query is too slow? the query cache is disabled while executing the query? what happen if two equal queries (same SELECT) execute? the first one is locked until the second one end the query and get the result from query cache? or the first lock query cache and the second runs without query cache? the last one there's any query cache changes in mind? i was reading the sql_cache.cc of old mysql servers and there's a todo list (very old i think) that didn't changed from old mysql to today mariadb 10.0.3, this todo list is inside mysql bug tracker? or mariadb jira? thanks guys! -- Roberto Spadim

2 11

[Maria-developers] doubt about workaround on MDEV-4903
by Roberto Spadim 16 Aug '13

16 Aug '13

hi guys, could anyone help me with a workaround to this MDEV? https://mariadb.atlassian.net/browse/MDEV-4903 basically the problem is a (more than one column primary key) IN ( (a primary key const value), (a primary key const value), (a primary key const value), (a primary key const value) ) it runs a table scan instead of using the primary key index using OR = OR = OR =, use the index but i want a easier way to write this query thanks -- Roberto Spadim

1 0

[Maria-developers] RFC - query cache using storage engine
by Roberto Spadim 15 Aug '13

15 Aug '13

Hi guys, i was thinking about ideas for query cache... there's an mdev about partitioning query cache ( https://mariadb.atlassian.net/browse/MDEV-4454) instead of this, why not use a HEAP storage engine (with partitioning), or maybe a myisam or aria or innodb as "backend", and write a "frontend" (query cache) that execute commands over this storage engine? i'm talking about, changing the query/table/result blocks of query cache, to query/table/result tables when start mysqld truncate these tables when add new query "insert" to these tables when a table prune occur, "delete" from these tables when flush, optimize these tables when reset query cache, truncate this tables these tables could be at mysql schema, if they don't exists create as ENGINE=MEMORY for example since we must have a MAX SIZE for query cache, we could add a MAX SIZE for HEAP engine and a LOW MEMORY algorithm ( https://mariadb.atlassian.net/browse/MDEV-4675) at high level, what you think about this idea? -- Roberto Spadim

2 3

Re: [Maria-developers] Fwd: [Commits] Rev 3674: MDEV-4645: Incorrect reads of frozen binlog events; FDE corrupted in relay log in file:///home/bell/maria/bzr/work-maria-10.0-base-MDEV-4645/
by Kristian Nielsen 15 Aug '13

15 Aug '13

Oleksandr Byelkin <sanja(a)montyprogram.com> writes: > Could you review this? > > (it is https://mariadb.atlassian.net/browse/MDEV-4645 ) Ok, but it will be at least a couple more days before I will have time to start. If urgent, you can also ask Sergei Golubchik, IIRC this is the patch from Jeremy Cole that he asked me about, and I already looked at once and accepted the general idea? - Kristian.

2 1

Re: [Maria-developers] Review of patch for MDEV-4820
by Kristian Nielsen 15 Aug '13

15 Aug '13

Vilho Raatikka <vilho.raatikka(a)skysql.com> writes: > Is this documented profoundly already? Should be because probably we'll see > at least some users as ignorant as I am asking the same questions soon. I wrote up some comprehensive documentation in the knowledgebase, and also wrote a series of articles about in on my blog: https://kb.askmonty.org/en/global-transaction-id/ http://kristiannielsen.livejournal.com/ But this is complex stuff, I am sure it can be improved. Any help improving the documentation would be greatly appreciated. I fear that I have spent too much time on the details of the design to clearly see how to explain the high-level picture in a way suitable for new users. - Kristian.

1 0

[Maria-developers] Value returned by my_getcputime
by Weldon Whipple 14 Aug '13

14 Aug '13

Hello! (This is my first post to this list; long-time follower, however.) I've been looking at the block comment at the top of the my_getcputime() function at the bottom of the file mysys/my_getsystime.c The block comment reads: Return cpu time in milliseconds * 10 I interpret milliseconds * 10 to mean centiseconds (seconds * 10 ^ -2) I'm pretty sure I've convinced myself that it REALLY returns: seconds * 10 ^ -7 I think it would be more accurate to have the block comment read: Return cpu time in microseconds / 10 Am I "up in the night"? (I think the code itself is fine, and the callers of the function all do it correctly. I was just initially confused when I looked at the comment then the code.) Thanks in advance! Weldon Whipple weldon(a)whipple.org (P.S. I think I like Maria's implementation of userstat_v2 better than the original by Google/Percona ...)

2 1

Re: [Maria-developers] Review of patch for MDEV-4820
by Kristian Nielsen 14 Aug '13

14 Aug '13

>> SHOW VARIABLES or SELECT @@GLOBAL.gtid_strict_mode will show the current >> value. But there is no way to check if it was enabled eg. two weeks ago, if >> that is what you are asking (the value is not stored in the binlog). > That is really pity. Is it really so useful to give the user opportunity to > switch gtid mode on-the-fly? That option sounds really expensive. Isn't > binlog event ordering lost - or it can't be trusted anymore? Binlog event order is never lost, strict mode or no strict mode. That is the whole point: even if the user messes up, MariaDB GTID is still able to correctly find the right position in the binlog to start at. There is nothing expensive about changing gtid strict mode, nor any loss of trust. Gtid strict mode is something to help the _user_. The GTID _code_ does not care, it works correctly whether strict mode is enabled or not. The only thing strict mode does is give an error for some uses that are considered undesirable by people like you and me. All non-error cases work exactly the same whether strict mode is enabled or not. It seems the more I try to explain, the more I confuse the issue :-( Maybe it is better if I just go fix the actual bugs ... - Kristian.

1 0

Re: [Maria-developers] Rev 3804: MDEV-4712 : Fix "shutdown" test
by Sergei Golubchik 13 Aug '13

13 Aug '13

Hi, Vladislav! On Jul 31, Vladislav Vaintroub wrote: > ------------------------------------------------------------ > revno: 3804 > revision-id: wlad(a)montyprogram.com-20130731130414-ukqxbb0kxpexjnzp > parent: svoj(a)mariadb.org-20130731110211-09p66bwf4jrmi03i > committer: Vladislav Vaintroub <wlad(a)montyprogram.com> > branch nick: 10.0-serg > timestamp: Wed 2013-07-31 15:04:14 +0200 > message: > MDEV-4712 : Fix "shutdown" test. > > There is a race condition during shutdown, that happens when "show status" runs during shutdown. > Functions show_slave_running(), show_slave_received_heartbeats(), show_heartbeat_period() would dereference NULL pointer master_info_index, after this variable is set to NULL in slave_end() during shutdown. > Fix introduces check for master_info_index == NULL in the affected functions. Was it related to the new shutdown command? Or it could be also observed without it by, say, running show status in a loop, while stopping the server with mysqladmin? Regards, Sergei

2 1

[Maria-developers] Lack of linkage from MariaDB JIRA to commits
by Jeremy Cole 13 Aug '13

13 Aug '13

All, I have noticed that the MariaDB JIRA tickets are rarely linked to any specific commit which purportedly fixes them. Is there any standard at the moment for what should be done (and then it isn't followed) or is there no standard (and perhaps it should be made)? I would really like a consistent way to get to view the commits into various branches related to each MDEV issue, in a consistent way. Regards, Jeremy

5 9

[Maria-developers] 10.x compatibility and separate namespace
by Honza Horak 13 Aug '13

13 Aug '13

Hi Guys, since 10.x won't be compatible with mysql in every aspect any more, I'm wondering if there happen to be any plans to move all compatibility left-overs (client binary names, client library name, daemon binary name, etc.). Are there any such ideas? I understand that we would lose the compatibility for any later (keeping original names was mentioned in [1] as a MUST for compatibility), but when keeping the original names, it could become tricky to say which tools are compatible and which not any more in mariadb 10+. How this information will be provided actually? How to get users informed properly, which tools are not compatible any more? And a bit another POV -- is the incompatibility in 10.x strictly limited to the server only (so client side will stay compatible for anytime later)? [1] https://lists.launchpad.net/maria-discuss/msg00658.html Regards, Honza

3 2

Re: [Maria-developers] Review of patch for MDEV-4820
by Kristian Nielsen 13 Aug '13

13 Aug '13

Vilho Raatikka <vilho.raatikka(a)skysql.com> writes: > What exactly does it require for the user to enable strict or 'relaxed' > (=disasterous imho) mode? Technically, one needs to SET GLOBAL gtid_strict_mode=1 with root privileges (config file option works as well of course). That's it. The practical issue is that old-style MariaDB/MySQL replication allows a number of "disasterous" uses. One example is a multi-master ring A->B->C->A, writing to all masters simultaneously. In such setup, it is fundamentally impossible to automatically eg. move a slave S from A->S to B->S - there is just not enough information about what events have been applied to S from A, B, and C. GTID introduces a way to configure this correctly (replication domains), but if we were to give an error by default then we would break upgrades from 5.5. So instead we provide gtid strict mode, so that once users understand what the "disasterous" uses are and how to avoid them, they can enable strict mode and get saner semantics. BTW, for the normal non-advanced user, I think silly stuff like running random manual transactions on slaves that get into the binlog is a far more common mistake. Gtid strict mode will cause errors on this, so it needs to be explictly enabled by users to not break/confuse upgrades. > Is it possible at any point in server's life cycle to check which mode it > is running? SHOW VARIABLES or SELECT @@GLOBAL.gtid_strict_mode will show the current value. But there is no way to check if it was enabled eg. two weeks ago, if that is what you are asking (the value is not stored in the binlog). > Can that setting be modified during runtime or only prior startup? It can be changed at any time, though I think you need to STOP SLAVE/START SLAVE to get the change to take effect on a slave. - Kristian.

1 1

[Maria-developers] RFC: Compile-time checking for unmatched DBUG_ENTER/LEAVE via unused variable
by Jeremy Cole 11 Aug '13

11 Aug '13

Maria developers/captains, Prior to posing it as a final patch contribution or opening a JIRA ticket, I wanted to propose this idea. The attached patch adds what amounts to compile-time checking for unmatched DBUG_ENTER/DBUG_LEAVE (DBUG_RETURN, DBUG_VOID_RETURN) by introducing a variable in DBUG_ENTER which is only used in DBUG_LEAVE. This allows any compiler which can robustly detect unused variables to detect the mismatch at compile-time. There is already a run-time check for this case, but it is somewhat limited as it requires _db_return_ to be called in order to detect the mismatch, and this is in practice not always the case. Particularly three cases allow this to escape detection: 1. Some instrumented functions are called only from non-instrumented functions. 2. Some instrumented functions are called only from non-returning functions, such as abort or die functions. 3. Some instrumented functions are called from functions who call my_end or my_thread_end (making cs unavailable) before returning. In any case please see the attached DRAFT patch for a full implementation of the idea. Note one bit of ugliness is that non-returning functions, after this patch, must be marked explicitly as DBUG_ENTER_NO_RETURN -- this actually is not really a bad thing, though in and of itself. Regards, Jeremy

4 4

Re: [Maria-developers] Rev 3671: MDEV-4652 Wrong result for CONCAT(GREATEST(TIME('00:00:01'), TIME('00:00:00')))
by Sergei Golubchik 11 Aug '13

11 Aug '13

Hi, Alexander! On Aug 07, Alexander Barkov wrote: > At lp:~maria-captains/maria/5.3 > > ------------------------------------------------------------ > revno: 3671 > revision-id: bar(a)mariadb.org-20130807093420-khdah292yerbor8y > parent: sergii(a)pisem.net-20130715163225-6ch6x34lsufode3d > committer: Alexander Barkov <bar(a)mariadb.org> > branch nick: 5.3.b4652 > timestamp: Wed 2013-08-07 13:34:20 +0400 > message: > MDEV-4652 Wrong result for CONCAT(GREATEST(TIME('00:00:01'),TIME('00:00:00'))) Ok to push. > === modified file 'sql/item.cc' > --- a/sql/item.cc 2013-07-03 07:46:20 +0000 > +++ b/sql/item.cc 2013-08-07 09:34:20 +0000 > @@ -252,7 +252,8 @@ String *Item::val_string_from_decimal(St > String *Item::val_string_from_date(String *str) > { > MYSQL_TIME ltime; > - if (get_date(&ltime, 0) || > + if (get_date(&ltime, > + field_type() == MYSQL_TYPE_TIME ? TIME_TIME_ONLY : 0) || > str->alloc(MAX_DATE_STRING_REP_LENGTH)) > { > null_value= 1; > > === modified file 'sql/item_func.cc' > --- a/sql/item_func.cc 2013-07-03 07:46:20 +0000 > +++ b/sql/item_func.cc 2013-08-07 09:34:20 +0000 > @@ -2480,6 +2480,13 @@ bool Item_func_min_max::get_date(MYSQL_T > ltime->time_type= MYSQL_TIMESTAMP_DATE; > ltime->hour= ltime->minute= ltime->second= ltime->second_part= 0; > } > + else if (compare_as_dates->field_type() == MYSQL_TYPE_TIME) > + { > + ltime->time_type= MYSQL_TIMESTAMP_TIME; > + ltime->hour+= (ltime->month * 32 + ltime->day) * 24; > + ltime->month= ltime->day= 0; > + } > + > > return 0; > } Regards, Sergei

1 0

Re: [Maria-developers] [Commits] Rev 3552: Fix bug MDEV-4648 SQ pushdown: Wrong result (missing rows) with materialization+semijoin, IN and ALL subqueries, UNION in file:///home/tsk/mprog/src/10.0-md83/
by Oleksandr Byelkin 09 Aug '13

09 Aug '13

09.08.2013 14:30, timour(a)askmonty.org пишет: > At file:///home/tsk/mprog/src/10.0-md83/ > > ------------------------------------------------------------ > revno: 3552 > revision-id: timour(a)askmonty.org-20130809113013-ia2f7zzw3mv3in7u > parent: timour(a)askmonty.org-20130611081135-rge7p00h8e9nhypj > fixes bug: https://mariadb.atlassian.net/browse/MDEV-4648 > committer: timour(a)askmonty.org > branch nick: 10.0-md83 > timestamp: Fri 2013-08-09 14:30:13 +0300 > message: > Fix bug MDEV-4648 SQ pushdown: Wrong result (missing rows) with materialization+semijoin, IN and ALL subqueries, UNION > (part of MDEV-83) > > Analysis: > > The cause of the wrong result was that the ALL subquery was incorrectly > attached to the last table in the join plan <t1, sj-mat, t2>. The ALL > subquery was transformed into a MIN expression, however it still depended > on OUTER_REF_TABLE_BIT as if it was a subquery. The reason for this > dependence was that the transformed MIN expression referred to the > Item_cache object of the left subquery operand created for the initial > Item_in_optimizer. > > Solution: > > When transforming an ALL subquery into a MIN/MAX expression, use the > actual left argument expression, instead of the Item_cache created > for the subquery. > OK to push. Thank you!

1 0

[Maria-developers] Is it a bug?
by Alexey Botchkov 09 Aug '13

09 Aug '13

Hi, SerG. The code in sql/sql_plugin.cc function read_maria_plugin_info() sym= dlsym(plugin_dl->handle, maria_plugin_declarations_sym); ... if (plugin_dl->mariaversion != MARIA_PLUGIN_INTERFACE_VERSION) { ... if ((sym= dlsym(plugin_dl->handle, maria_sizeof_st_plugin_sym))) sizeof_st_plugin= *(int *)sym; ... if (sizeof_st_plugin != sizeof(st_mysql_plugin)) { ... sym= cur; } } plugin_dl->plugins= (struct st_maria_plugin *)sym; So that when we enter the first 'if' but not the second (sizeof_st_plugin == sizeof(st_mysql_plugin) the sym still points at the maria_sizeof_st_plugin_sym which makes a bad st_maria_plugin pointer. I think it's a bug, isn't it? Best regards. HF

2 1

Re: [Maria-developers] GTID and removing binary logs
by Kristian Nielsen 08 Aug '13

08 Aug '13

Elena Stepanova <elenst(a)montyprogram.com> writes: > Reading recent Pavel Ivanov's reports (the last one is MDEV-4820, but > I think there were some others which you've already closed as fixed), > I started wondering if I conceptually misunderstood something > before. From our previous discussions I made a conclusion that > removing *all* binary logs before trying to replicate from the server > is very wrong and against the whole idea of GTID. Is it not so? If > not, it's definitely an area which didn't undergo any testing from me, > except for my very first clumsy attempts to get familiar with GTID, so > I need to fill the gap. Well, we _do_ want to support running a slave with --log-slave-updates=0, and then making that slave into a master. Indeed, you tested that early and it was horribly broken. However, my intention is that this should now be supported (and thus can be tested). There are MTR testcases for this. The reason I mention this is that the case of --log-slave-updates=0 is actually quite similar (from the server's point of view) to manually deleting the logs. So I do also intend to support removing binlogs as Pavel wants to do. The main problem with doing this (deleting binlogs or using --log-slave-updates=0) is that it makes it harder to detect certain user errors, because the information to know that it is an error is missing. I suppose the limitations should be pretty obvious: If user deletes binlogs on a server, then that server, acting as a master, can not replicate to slaves starting from an event that was deleted. So as long as the user does not try to do that, things should be fine. I think the main problems come if the user makes a mistake in this respect, and the server fails to detect the error. For example, suppose that server A is a slave, replicates up to position 0-2-10. Then it becomes a master, generates events up to 0-1-20. Then user deletes the binlogs on A, and tries to have server B replicate from A starting at position 0-2-10. In this case, server A will allow B to connect, effectively loosing the deleted events 0-1-11 .. 0-1-20. I do not see any way to avoid this. Server A has no way to know that those events ever existed, as the user deliberately deleted all information about them. (In fact the user might be doing this deliberately, in case the events were wrongly created and should be removed from the history). A workaround in this example is that the user sets @@gtid_slave_pos explicitly to 0-1-20 after deleting the logs, thus preserving the needed information. So in summary: Yes, we try to support deleting binlogs, however there is some responsibility on the part of the user to not delete information that they actually need. I think that is rather reasonable. It is how DELETE and SELECT work, as well :-) - Kristian.

1 0

Re: [Maria-developers] [Commits] Rev 3772: MDEV-4645: Incorrect reads of frozen binlog events; FDE corrupted in relay log in file:///home/bell/maria/bzr/work-maria-10.0-MDEV-4548/
by Jeremy Cole 08 Aug '13

08 Aug '13

Sanja, Re: https://mariadb.atlassian.net/browse/MDEV-4645 Re: http://lists.askmonty.org/pipermail/commits/2013-July/005093.html I saw your commit on the commits list (sorry, wasn't on the list before the mail went out, so I can't reply directly to that message). A couple of points related to the commit: - You seem to have an error in the get_header_len, as it's implemented only in the base class, and you've left functions with the correct signature for it but the wrong names in the two frozen classes (they were left named has_fixed_minimal_header). This would mean the change is completely broken. :) - The change to get_header_len I think is in general fine, although it does alter the point of the whole thing -- there is again no way to exactly identify which events have frozen headers, programmatically (i.e. you can't differentiate them from events who just "happen" to have header_len = LOG_EVENT_MINIMAL_HEADER_LEN). I think that doesn't matter for this patch, but does defeat the point of what I was trying to add. - I don't really understand why you've made the hex printing code so complex and completely unreadable (IMHO). This is not performance-sensitive code and in my opinion should favor readability over performance. All you seem to have saved in this change is to avoid printing some static formatting characters ("#", "|", spaces, etc.) inside the loop. However, I can hardly even figure out what it's supposed to be doing now. - Additionally, you've changed the output slightly -- normally the ASCII character section prints its ending "|" immediately after the last character; it seems now to be fixed position (leaving extra spaces before "|". This differs from the output of hexdump, and it somewhat ambiguous/confusing. - You've mixed tabs and spaces. :( Regards, Jeremy

2 8

[Maria-developers] RFC - temporary path and SSD/HDD
by Roberto Spadim 07 Aug '13

07 Aug '13

Hi guys I have some server running with bcache and some in test running flash cache and others linux ssd cache solution I was thinking about how to optimize the temporary tables of mysql/mariadb... could we implement something like: Temporary Slow/Big Path = /tmp/hdd Temporary Fast/Small Path = /tmp/ssd Temporary Fast Max Size = 10MB When we start using temporary tables (file sort for example) instead of write to a hdd temporary disk, use a ssd temporary disk, if table get bigger than 10MB, copy the file to hdd drive, and remove from ssd drive, and continue writing to hdd I think with this i could optimize the temporary table use Any other idea about this kind of optimization? -- Roberto Spadim

1 0

Re: [Maria-developers] [Commits] Rev 3671: MDEV-4811 Assertion `offset < 0x1f' fails in type_and_offset_store on COLUMN_ADD in file:///home/bell/maria/bzr/work-maria-5.3-MDEV-4811/
by Sergei Petrunia 07 Aug '13

07 Aug '13

Hi Sanja, Ok to push. On Thu, Aug 01, 2013 at 11:46:14AM +0300, sanja(a)montyprogram.com wrote: > At file:///home/bell/maria/bzr/work-maria-5.3-MDEV-4811/ > > ------------------------------------------------------------ > revno: 3671 > revision-id: sanja(a)montyprogram.com-20130801084611-cu3wpoazd197g41w > parent: sergii(a)pisem.net-20130715163225-6ch6x34lsufode3d > committer: sanja(a)montyprogram.com > branch nick: work-maria-5.3-MDEV-4811 > timestamp: Thu 2013-08-01 11:46:11 +0300 > message: > MDEV-4811 Assertion `offset < 0x1f' fails in type_and_offset_store on COLUMN_ADD > MDEV-4812 Valgrind warnings (Invalid write) in dynamic_column_update_many on COLUMN_ADD > > Fixed problem of working on wrong data (do not allow offset to out of string length). > === modified file 'mysql-test/r/dyncol.result' > --- a/mysql-test/r/dyncol.result 2013-03-26 11:07:46 +0000 > +++ b/mysql-test/r/dyncol.result 2013-08-01 08:46:11 +0000 > @@ -1404,5 +1404,26 @@ v1 CREATE ALGORITHM=UNDEFINED DEFINER=`r > drop view v1; > drop table t1; > # > +# MDEV-4811: Assertion `offset < 0x1f' fails in type_and_offset_store > +# on COLUMN_ADD > +# > +CREATE TABLE t1 (dyn TINYBLOB) ENGINE=MyISAM; > +INSERT INTO t1 SET dyn = COLUMN_CREATE( 40, REPEAT('a', 233), 4, REPEAT('b', 322) ); > +Warnings: > +Warning 1265 Data truncated for column 'dyn' at row 1 > +SELECT COLUMN_ADD( dyn, 6, REPEAT('x',80), 4, REPEAT('y',215) AS INTEGER ) FROM t1; > +ERROR HY000: Encountered illegal format of dynamic column string > +DROP table t1; > +# > +# MDEV-4812: Valgrind warnings (Invalid write) in > +# dynamic_column_update_many on COLUMN_ADD > +# > +CREATE TABLE t1 (dyncol TINYBLOB) ENGINE=MyISAM; > +INSERT INTO t1 SET dyncol = COLUMN_CREATE( 7, REPEAT('k',487), 209, REPEAT('x',464) ); > +Warnings: > +Warning 1265 Data truncated for column 'dyncol' at row 1 > +SELECT COLUMN_ADD( dyncol, 7, '22:22:22', 8, REPEAT('x',270) AS CHAR ) FROM t1; > +DROP table t1; > +# > # end of 5.3 tests > # > > === modified file 'mysql-test/t/dyncol.test' > --- a/mysql-test/t/dyncol.test 2013-03-26 11:07:46 +0000 > +++ b/mysql-test/t/dyncol.test 2013-08-01 08:46:11 +0000 > @@ -600,6 +600,29 @@ drop view v1; > drop table t1; > > --echo # > +--echo # MDEV-4811: Assertion `offset < 0x1f' fails in type_and_offset_store > +--echo # on COLUMN_ADD > +--echo # > + > +CREATE TABLE t1 (dyn TINYBLOB) ENGINE=MyISAM; > +INSERT INTO t1 SET dyn = COLUMN_CREATE( 40, REPEAT('a', 233), 4, REPEAT('b', 322) ); > +--error ER_DYN_COL_WRONG_FORMAT > +SELECT COLUMN_ADD( dyn, 6, REPEAT('x',80), 4, REPEAT('y',215) AS INTEGER ) FROM t1; > + > +DROP table t1; > + > +--echo # > +--echo # MDEV-4812: Valgrind warnings (Invalid write) in > +--echo # dynamic_column_update_many on COLUMN_ADD > +--echo # > +CREATE TABLE t1 (dyncol TINYBLOB) ENGINE=MyISAM; > + > +INSERT INTO t1 SET dyncol = COLUMN_CREATE( 7, REPEAT('k',487), 209, REPEAT('x',464) ); > +--error 0,ER_DYN_COL_WRONG_FORMAT > +SELECT COLUMN_ADD( dyncol, 7, '22:22:22', 8, REPEAT('x',270) AS CHAR ) FROM t1; > +DROP table t1; > + > +--echo # > --echo # end of 5.3 tests > --echo # > > > === modified file 'mysys/ma_dyncol.c' > --- a/mysys/ma_dyncol.c 2011-10-18 10:44:12 +0000 > +++ b/mysys/ma_dyncol.c 2013-08-01 08:46:11 +0000 > @@ -1228,13 +1228,14 @@ dynamic_column_create(DYNAMIC_COLUMN *st > @param header_end Pointer to the header end > @param offset_size Size of offset field in bytes > @param last_offset Size of the data segment > + @param error Set in case of error > > @return number of bytes > */ > > static size_t get_length_interval(uchar *entry, uchar *entry_next, > uchar *header_end, size_t offset_size, > - size_t last_offset) > + size_t last_offset, my_bool *error) > { > size_t offset, offset_next; > DYNAMIC_COLUMN_TYPE type, type_next; > @@ -1242,8 +1243,12 @@ static size_t get_length_interval(uchar > > type_and_offset_read(&type, &offset, entry, offset_size); > if (entry_next >= header_end) > + { > + *error= 0; > return (last_offset - offset); > + } > type_and_offset_read(&type_next, &offset_next, entry_next, offset_size); > + *error= (offset_next > last_offset); > return (offset_next - offset); > } > > @@ -1255,17 +1260,18 @@ static size_t get_length_interval(uchar > @param header_end Pointer to the header end > @param offset_size Size of offset field in bytes > @param last_offset Size of the data segment > + @param error Set in case of error > > @return number of bytes > */ > > static size_t get_length(uchar *entry, uchar *header_end, > size_t offset_size, > - size_t last_offset) > + size_t last_offset, my_bool *error) > { > return get_length_interval(entry, > entry + offset_size + COLUMN_NUMBER_SIZE, > - header_end, offset_size, last_offset); > + header_end, offset_size, last_offset, error); > } > > > @@ -1304,6 +1310,7 @@ find_column(DYNAMIC_COLUMN_TYPE *type, u > uchar *entry; > size_t offset, total_data, header_size, entry_size; > uchar key[2+4]; > + my_bool error; > > if (!entry_pos) > entry_pos= &entry; > @@ -1329,12 +1336,12 @@ find_column(DYNAMIC_COLUMN_TYPE *type, u > return 1; > *data= header + header_size + offset; > *length= get_length(entry, header + header_size, offset_size, > - total_data); > + total_data, &error); > /* > Check that the found data is withing the ranges. This can happen if > we get data with wrong offsets. > */ > - if ((long) *length < 0 || offset + *length > total_data) > + if (error || (long) *length < 0 || offset + *length > total_data) > return 1; > > *entry_pos= entry; > @@ -1837,12 +1844,13 @@ dynamic_column_update_many(DYNAMIC_COLUM > entry_size, column_count, &entry)) > { > size_t entry_data_size; > + my_bool error; > > /* Data existed; We have to replace or delete it */ > > entry_data_size= get_length(entry, header_end, > - offset_size, max_offset); > - if ((long) entry_data_size < 0) > + offset_size, max_offset, &error); > + if (error || (long) entry_data_size < 0) > { > rc= ER_DYNCOL_FORMAT; > goto end; > @@ -2038,12 +2046,13 @@ dynamic_column_update_many(DYNAMIC_COLUM > /* copy first the data that was not replaced in original packed data */ > if (start < end) > { > + my_bool error; > /* Add old data last in 'tmp' */ > size_t data_size= > get_length_interval(header_base + start * entry_size, > header_base + end * entry_size, > - header_end, offset_size, max_offset); > - if ((long) data_size < 0 || > + header_end, offset_size, max_offset, &error); > + if (error || (long) data_size < 0 || > data_size > max_offset - first_offset) > { > dynamic_column_column_free(&tmp); > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits -- BR Sergei -- Sergei Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] [Commits] Rev 3797: MDEV-4816: rpl.rpl_trunc_temp fails in 10.0-serg in file:///data0/psergey/dev2/10.0-serg/
by Kristian Nielsen 07 Aug '13

07 Aug '13

Sergey Petrunya <psergey(a)askmonty.org> writes: > revision-id: psergey(a)askmonty.org-20130802141209-4dqfvx2db8acxwbl > message: > MDEV-4816: rpl.rpl_trunc_temp fails in 10.0-serg > Temorary fix for a number of replication tests (rpl.rpl_temp_table_mix_row > rpl.rpl_trunc_temp rpl.rpl_current_user rpl.rpl_gtid_master_promote): The patch is not ok to push. We should not introduce a special flag just for the table gtid_slave_pos. Instead of setting table->no_replicate in the first place in rpl_slave_state::truncate_state_table() and rpl_slave_state::record_gtid(), we can clear the OPTION_BIN_LOG option. I checked MySQL 5.6 code and this is what they do for their similar "crash-safe replication" tables. So remove the two statements "table->no_replicate= 1" in rpl_gtid.cc and replace with clear of OPTION_BIN_LOG in thd->variables.option_bits. Then table->no_replicate is not touched and the new logic in THD::decide_logging_format() does not trigger, so the problem should be solved. In rpl_slave_state::truncate_state_table() we can use tmp_disable_binlog() and reenable_binlog() I think. In record_gtid() it is probably better to combine the clearing of OPTION_BIN_LOG with the existing save/restore of variables.option_bits. > The question of what should be done when a user issues a statement that > explicitly modifies mysql.gtid_slave_pos table remains open. I do not think we need to do anything special for this case. User explicitly modifying the table should be treated no different than any other table. Hope this helps, - Kristian. > === modified file 'sql/rpl_gtid.cc' > --- a/sql/rpl_gtid.cc 2013-06-21 09:53:46 +0000 > +++ b/sql/rpl_gtid.cc 2013-08-02 14:12:09 +0000 > @@ -210,6 +210,7 @@ rpl_slave_state::truncate_state_table(TH > { > table= tlist.table; > table->no_replicate= 1; > + table->s->is_gtid_slave_pos= TRUE; // TEMPORARY CODE > err= table->file->ha_truncate(); > > if (err) > @@ -346,6 +347,7 @@ rpl_slave_state::record_gtid(THD *thd, c > goto end; > > table->no_replicate= 1; > + table->s->is_gtid_slave_pos= TRUE; // TEMPORARY CODE > if (!in_transaction) > thd->variables.option_bits&= > ~(ulonglong)(OPTION_NOT_AUTOCOMMIT|OPTION_BEGIN); > > === modified file 'sql/sql_class.cc' > --- a/sql/sql_class.cc 2013-07-24 10:43:57 +0000 > +++ b/sql/sql_class.cc 2013-08-02 14:12:09 +0000 > @@ -5001,7 +5001,7 @@ int THD::decide_logging_format(TABLE_LIS > DBUG_PRINT("info", ("table: %s; ha_table_flags: 0x%llx", > table->table_name, flags)); > > - if (table->table->no_replicate) > + if (table->table->no_replicate && !table->table->s->is_gtid_slave_pos) > { > /* > The statement uses a table that is not replicated. > > === modified file 'sql/table.cc' > --- a/sql/table.cc 2013-07-21 14:43:42 +0000 > +++ b/sql/table.cc 2013-08-02 14:12:09 +0000 > @@ -315,6 +315,8 @@ TABLE_SHARE *alloc_table_share(const cha > strmov(share->path.str, path); > share->normalized_path.str= share->path.str; > share->normalized_path.length= path_length; > + /* TEMPORARY FIX: if true, this means this is mysql.gtid_slave_pos table */ > + share->is_gtid_slave_pos= FALSE; > share->table_category= get_table_category(& share->db, & share->table_name); > share->set_refresh_version(); > share->open_errno= ENOENT; > > === modified file 'sql/table.h' > --- a/sql/table.h 2013-07-21 14:39:19 +0000 > +++ b/sql/table.h 2013-08-02 14:12:09 +0000 > @@ -644,6 +644,8 @@ struct TABLE_SHARE > LEX_STRING normalized_path; /* unpack_filename(path) */ > LEX_STRING connect_string; > > + bool is_gtid_slave_pos; > + > /* > Set of keys in use, implemented as a Bitmap. > Excludes keys disabled by ALTER TABLE ... DISABLE KEYS. > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits

1 0

[Maria-developers] COT() function
by Ian Gilfillan 03 Aug '13

03 Aug '13

Hi SELECT COT(0) returns: ERROR 1690 (22003): DOUBLE value is out of range in 'cot(0)' in both MariaDB 10 and 5.5, but in MySQL 5.1 it returns NULL. I can't find a changelog reference to this, but I see it's still documented as returning NULL in MySQL 5.5 (I don't have this to test on). Is this a MariaDB-specific change? When did this change in behaviour happen? thanks, ian

3 5

[Maria-developers] Please review: [Commits] Rev 3797: MDEV-4816: rpl.rpl_trunc_temp fails in 10.0-serg in file:///data0/psergey/dev2/10.0-serg/
by Sergei Petrunia 02 Aug '13

02 Aug '13

Hi Kristian, In order to get rid of test failures of 10.0-serg, I have pushed the below patch. It seems to have the right effect, but I have no confidence about this patch being achitecturally correct, etc. Could you please review this and/or make another fix for the problem? ----- Forwarded message from Sergey Petrunya <psergey(a)askmonty.org> ----- From: Sergey Petrunya <psergey(a)askmonty.org> To: commits(a)mariadb.org X-Mailer: mail (GNU Mailutils 2.2) Date: Fri, 2 Aug 2013 18:12:11 +0400 (MSK) Subject: [Commits] Rev 3797: MDEV-4816: rpl.rpl_trunc_temp fails in 10.0-serg in file:///data0/psergey/dev2/10.0-serg/ At file:///data0/psergey/dev2/10.0-serg/ ------------------------------------------------------------ revno: 3797 revision-id: psergey(a)askmonty.org-20130802141209-4dqfvx2db8acxwbl parent: svoj(a)mariadb.org-20130729140849-dciz5ffq25ciip6q committer: Sergey Petrunya <psergey(a)askmonty.org> branch nick: 10.0-serg timestamp: Fri 2013-08-02 18:12:09 +0400 message: MDEV-4816: rpl.rpl_trunc_temp fails in 10.0-serg Temorary fix for a number of replication tests (rpl.rpl_temp_table_mix_row rpl.rpl_trunc_temp rpl.rpl_current_user rpl.rpl_gtid_master_promote): - THD::decide_logging_format() should not assume that mysql.gtid_slave_pos is a non-replicated table. This used to cause unintended behavior for COMMIT statement: replication would switch to row-based, etc. The question of what should be done when a user issues a statement that explicitly modifies mysql.gtid_slave_pos table remains open. === modified file 'sql/rpl_gtid.cc' --- a/sql/rpl_gtid.cc 2013-06-21 09:53:46 +0000 +++ b/sql/rpl_gtid.cc 2013-08-02 14:12:09 +0000 @@ -210,6 +210,7 @@ rpl_slave_state::truncate_state_table(TH { table= tlist.table; table->no_replicate= 1; + table->s->is_gtid_slave_pos= TRUE; // TEMPORARY CODE err= table->file->ha_truncate(); if (err) @@ -346,6 +347,7 @@ rpl_slave_state::record_gtid(THD *thd, c goto end; table->no_replicate= 1; + table->s->is_gtid_slave_pos= TRUE; // TEMPORARY CODE if (!in_transaction) thd->variables.option_bits&= ~(ulonglong)(OPTION_NOT_AUTOCOMMIT|OPTION_BEGIN); === modified file 'sql/sql_class.cc' --- a/sql/sql_class.cc 2013-07-24 10:43:57 +0000 +++ b/sql/sql_class.cc 2013-08-02 14:12:09 +0000 @@ -5001,7 +5001,7 @@ int THD::decide_logging_format(TABLE_LIS DBUG_PRINT("info", ("table: %s; ha_table_flags: 0x%llx", table->table_name, flags)); - if (table->table->no_replicate) + if (table->table->no_replicate && !table->table->s->is_gtid_slave_pos) { /* The statement uses a table that is not replicated. === modified file 'sql/table.cc' --- a/sql/table.cc 2013-07-21 14:43:42 +0000 +++ b/sql/table.cc 2013-08-02 14:12:09 +0000 @@ -315,6 +315,8 @@ TABLE_SHARE *alloc_table_share(const cha strmov(share->path.str, path); share->normalized_path.str= share->path.str; share->normalized_path.length= path_length; + /* TEMPORARY FIX: if true, this means this is mysql.gtid_slave_pos table */ + share->is_gtid_slave_pos= FALSE; share->table_category= get_table_category(& share->db, & share->table_name); share->set_refresh_version(); share->open_errno= ENOENT; === modified file 'sql/table.h' --- a/sql/table.h 2013-07-21 14:39:19 +0000 +++ b/sql/table.h 2013-08-02 14:12:09 +0000 @@ -644,6 +644,8 @@ struct TABLE_SHARE LEX_STRING normalized_path; /* unpack_filename(path) */ LEX_STRING connect_string; + bool is_gtid_slave_pos; + /* Set of keys in use, implemented as a Bitmap. Excludes keys disabled by ALTER TABLE ... DISABLE KEYS. _______________________________________________ commits mailing list commits(a)mariadb.org https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits ----- End forwarded message ----- -- BR Sergei -- Sergei Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] [Commits] Rev 3215: MDEV-4823: Server crashes in Item_func_not::fix_fields on creating a table with a virtual column using NOT in file:///home/bell/maria/bzr/work-maria-5.2-MDEV-4823/
by Sergei Petrunia 31 Jul '13

31 Jul '13

Hi Sanja, On Tue, Jul 30, 2013 at 11:00:59AM +0300, sanja(a)montyprogram.com wrote: > At file:///home/bell/maria/bzr/work-maria-5.2-MDEV-4823/ > > ------------------------------------------------------------ > revno: 3215 > revision-id: sanja(a)montyprogram.com-20130730080057-ylgl6h11l2b3yflc > parent: wlad(a)montyprogram.com-20130709202457-6n9syq59jdd2sy3b > committer: sanja(a)montyprogram.com > branch nick: work-maria-5.2-MDEV-4823 > timestamp: Tue 2013-07-30 11:00:57 +0300 > message: > MDEV-4823: Server crashes in Item_func_not::fix_fields on creating a table with a virtual column using NOT > > fix_field() call protocol was brocken (zero pointer passed as link to item which is possible only if you are sure that there can not be Items which transforms). ... > === modified file 'sql/table.cc' > --- a/sql/table.cc 2013-01-16 00:46:27 +0000 > +++ b/sql/table.cc 2013-07-30 08:00:57 +0000 > @@ -1812,7 +1812,7 @@ bool fix_vcol_expr(THD *thd, > bool result= TRUE; > TABLE_LIST tables; > TABLE_LIST *save_table_list, *save_first_table, *save_last_table; > - int error; > + int error= 0; > Name_resolution_context *context; > const char *save_where; > char* db_name; > @@ -1860,7 +1860,13 @@ bool fix_vcol_expr(THD *thd, > save_use_only_table_context= thd->lex->use_only_table_context; > thd->lex->use_only_table_context= TRUE; > /* Fix fields referenced to by the virtual column function */ > - error= func_expr->fix_fields(thd, (Item**)0); > + if (!func_expr->fixed) > + error= func_expr->fix_fields(thd, (Item**)&vcol_info->expr_item); There is no need to use (Item**) cast, please remove it. Ok to push after that. > + > + /* fix_fields could change the expression */ > + func_expr= vcol_info->expr_item; > + /* Number of columns will be checked later */ > + > /* Restore the original context*/ > thd->lex->use_only_table_context= save_use_only_table_context; > context->table_list= save_table_list; > BR Sergei -- Sergei Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] Plugin security
by Leif Walsh 30 Jul '13

30 Jul '13

Hi, I'm working on adding a plugin API (not exactly a storage engine API, rather an extension one) to TokuMX[1] and I'm looking for advice on security. The basic idea is fairly simple: a plugin is a shared library (we're only targeting Linux right now, by the way) that defines a symbol to bootstrap the loading of additional functionality. We have an admin-only command that calls dlopen and dlsym to load the plugin. I haven't read any of the MariaDB plugin code yet, so pointers to where to start reading would be appreciated. Apart from that, I'm just looking for any advice from you all about what to watch out for. My concerns break down into two main areas: 1. When loading a single plugin, what should I watch out for? The command itself is admin-only so calling it is protected, but what about an attacker that drops a malicious plugin in a directory earlier on the load path? What should I check about the permissions/owner of the library file and the directory in which it lives? Is erroring out on world-writable libraries enough? If my process is setuid, should I be more careful? What does Maria do here? 2. About plugin persistence/autoloading: I understand MariaDB has a system table that tracks which plugins are installed and automatically loads those on startup. I believe I can do the same thing in TokuMX and I can restrict access to that collection to the admin user. What does MariaDB store in that table and how does it use it? I could imagine storing the checksum of the installed plugin in the system table and verifying it before loading the plugin, but this seems rather draconian and it might make plugin upgrades too difficult. Does MariaDB verify the permissions of the data files that store this system table? I could imagine an attack where you would replace this system table with one that includes instructions to load your malicious plugin, while the server is offline. Again, how does MariaDB solve these problems? Another option is to autoload all libraries in some directory, and to just be very careful about the permissions of that directory, but this seems very hard to get totally right so I'm leaning away from it. [1]: http://www.tokutek.com/products/tokumx-for-mongodb -- Cheers, Leif

3 5

[Maria-developers] Master slave
by Richard Couture 30 Jul '13

30 Jul '13

Is it possible to set up replication in such a manner as to permit writing records to either a master or a slave replica? We want to set up a fail safe replica that can easily take over when our connection to the main replica fails, which, unfortunately, is fairly often. I read a document, some time ago which mentioned several types of replication, i.e.: by record, by table ... and the pitfalls, like IDs not necessarily matching... which confused me more than anything else... Can someone point me to some good documentation to enable me to pick the correct type of replication that I need to accomplish my task?? Thanks Richard -- LinuxCabal Asociación Civil Ing. Richard Couture Novell CNE, ECNE, MCNE HP/Compaq ASE Tel.: (+52) (333) 145-2638 Cel.: (+52) (044) 333 377-7505 Cel.: (+52) (044) 333 377-7506 Web: http://www.LinuxCabal.org E-Mail: rrc(a)linuxcabal.org Hosted en la nube Cloud Sigma - www.CloudSigma.com AVISO DE CONFIDENCIALIDAD: Este correo electrónico, incluyendo en su caso, los archivos adjuntos al mismo, pueden contener información de carácter confidencial y/o privilegiada, y se envían a la atención única y exclusivamente de la persona y/o entidad a quien va dirigido. La copia, revisión, uso, revelación y/o distribución de dicha información confidencial sin la autorización por escrito de LinuxCabal está prohibida. Si usted no es el destinatario a quien se dirige el presente correo, favor de contactar al remitente respondiendo al presente correo y eliminar el correo original incluyendo sus archivos, así como cualesquiera copia del mismo. Mediante la recepción del presente correo usted reconoce y acepta que en caso de incumplimiento de su parte y/o de sus representantes a los términos antes mencionados, LinuxCabal tendrá derecho a los daños y perjuicios que esto le cause.

1 0

Re: [Maria-developers] Implementing implicit primary key in mysql server
by Lixun Peng 30 Jul '13

30 Jul '13

Hi Sergei, You are right, let users add primary key is best. But I can't let users who don't want to create primary key can't use our MySQL service. Amazon RDS also allow users to create the tables without primary key, just change binlog_format to MIXED to solve replication problem. I think this feature is very useful for any MySQL cloud service providers, all of them will face this problem in one day. I will try to do some analysis/research in this feature implement first, I will need your help :-) I will notice any new updates in this email. Thanks, Lixun On Wed, Jun 19, 2013 at 5:14 PM, Sergei Golubchik <serg(a)mariadb.org> wrote: > Hi, Lixun! > > On Jun 18, Lixun Peng wrote: > > Hi, > > > > As we know, InnoDB has implicit primary key if a table hasn't defined > > a primary key. However MySQL server doesn't know this primary key, so > > this primary key will not apear in binlog. > > > > When we are using ROW format binlog for replication, if a table has no > any > > indexes, that's a disaster. If a table without indexes do a DML > > (UPDATE/DELETE), of course it will run a long time in master, but in > slave, > > it still need a long time. It will cause serious slave replication delay. > ... > > I think mysql server can do the same thing as InnoDB do, if user doesn't > > define the primary key, mysql can add the primary key automatically. > > > > How do you think? > > Well, that's doable. A much easier solution would be to require a user > to create a primary key. It's a one-line change: > > - Table_flags ha_table_flags() const { return cached_table_flags; } > + Table_flags ha_table_flags() const { return cached_table_flags | > HA_REQUIRE_PRIMARY_KEY; } > > But what you suggest is possible too, I believe. > > Regards, > Sergei > -- Senior Database Engineer @ Taobao.com Mobile Phone: +86 18658156856 (Hangzhou) Gtalk: penglixun(at)gmail.com Twitter: http://www.twitter.com/plinux Blog: http://www.penglixun.com

4 8

Re: [Maria-developers] [Commits] Rev 3792: main.gis test fixed. in file:///home/hf/wmar/10.0-serg/
by Sergey Vojtovich 27 Jul '13

27 Jul '13

Hi Alexey, I just checked 5.6 and 10.0-base, this test is passing as is. I'm a bit worried that something functional was broken during the merge. Regards, Sergey 26.07.2013, в 16:13, <holyfoot(a)askmonty.org> написал(а): > At file:///home/hf/wmar/10.0-serg/ > > ------------------------------------------------------------ > revno: 3792 > revision-id: holyfoot(a)askmonty.org-20130726110356-9s17c1kw558ls14a > parent: svoj(a)mariadb.org-20130726091143-o2v0tt08n81n923m > committer: Alexey Botchkov <holyfoot(a)askmonty.org> > branch nick: 10.0-serg > timestamp: Fri 2013-07-26 16:03:56 +0500 > message: > main.gis test fixed. > === modified file 'mysql-test/r/gis.result' > --- a/mysql-test/r/gis.result 2013-03-27 09:03:28 +0000 > +++ b/mysql-test/r/gis.result 2013-07-26 11:03:56 +0000 > @@ -918,25 +918,25 @@ SELECT Overlaps(@horiz1, @point2) FROM D > Overlaps(@horiz1, @point2) > 0 > DROP TABLE t1; > -create table t1(f1 geometry, f2 point, f3 linestring); > +create table t1(f1 geometry, f2 linestring, f3 linestring); > select f1 from t1 union select f1 from t1; > f1 > -insert into t1 (f2,f3) values (GeomFromText('POINT(1 1)'), > +insert into t1 (f2,f3) values (GeomFromText('LINESTRING(1 1, 2 2)'), > GeomFromText('LINESTRING(0 0,1 1,2 2)')); > select AsText(f2),AsText(f3) from t1; > AsText(f2) AsText(f3) > -POINT(1 1) LINESTRING(0 0,1 1,2 2) > +LINESTRING(1 1,2 2) LINESTRING(0 0,1 1,2 2) > select AsText(a) from (select f2 as a from t1 union select f3 from t1) t; > AsText(a) > -POINT(1 1) > +LINESTRING(1 1,2 2) > LINESTRING(0 0,1 1,2 2) > create table t2 as select f2 as a from t1 union select f3 from t1; > desc t2; > Field Type Null Key Default Extra > -a point YES NULL > +a linestring YES NULL > select AsText(a) from t2; > AsText(a) > -POINT(1 1) > +LINESTRING(1 1,2 2) > LINESTRING(0 0,1 1,2 2) > drop table t1, t2; > SELECT 1; > > === modified file 'mysql-test/t/gis.test' > --- a/mysql-test/t/gis.test 2013-03-27 09:03:28 +0000 > +++ b/mysql-test/t/gis.test 2013-07-26 11:03:56 +0000 > @@ -609,9 +609,9 @@ DROP TABLE t1; > # > # Bug#28763: Selecting geometry fields in UNION caused server crash. > # > -create table t1(f1 geometry, f2 point, f3 linestring); > +create table t1(f1 geometry, f2 linestring, f3 linestring); > select f1 from t1 union select f1 from t1; > -insert into t1 (f2,f3) values (GeomFromText('POINT(1 1)'), > +insert into t1 (f2,f3) values (GeomFromText('LINESTRING(1 1, 2 2)'), > GeomFromText('LINESTRING(0 0,1 1,2 2)')); > select AsText(f2),AsText(f3) from t1; > select AsText(a) from (select f2 as a from t1 union select f3 from t1) t; > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits

1 0

[Maria-developers] we have seen sporadic crashes in the Protocol::net_store_data function
by Rich Prohaska 26 Jul '13

26 Jul '13

Crash occurs when using the TokuDB storage engine in MySQL 5.5.30. Any insight into what may be causing this problem would be really nice. Sorry, but this is all the info that i have. Here is a stack trace from the mysql error log: #1 0x00000000006b760f in handle_fatal_signal (sig=11) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/signal_handler.cc:247 #2 <signal handler called> #3 0x00007ff213a3b447 in __strncasecmp_l_sse2 () from /lib64/libc.so.6 #4 0x00000000005651a7 in Protocol::net_store_data (this=0x0, this@entry=0x7ff18f01e4b8, from=0x0, length=1936615494) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/protocol.cc:59 #5 0x00000000005662e4 in Protocol::store_string_aux (this=this@entry=0x7ff18f01e4b8, from=<optimized out>, length=<optimized out>, fromcs=<optimized out>, tocs=tocs@entry=0x1022920 <my_charset_latin1>) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/protocol.cc:960 #6 0x00000000005663a5 in Protocol_text::store (this=0x7ff18f01e4b8, field=<optimized out>) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/protocol.cc:1119 #7 0x0000000000565ea0 in Protocol::send_result_set_row (this=0x7ff18f01e4b8, row_items=<optimized out>) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/protocol.cc:847 #8 0x000000000059fc37 in select_send::send_data (this=0x7fe338243b60, items=...) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/sql_class.cc:2184 #9 0x00000000005f920c in end_send (join=0x7fe77f4a4010, join_tab=<optimized out>, end_of_records=<optimized out>) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/sql_select.cc:12762 #10 end_send (join=0x7fe77f4a4010, join_tab=<optimized out>, end_of_records=<optimized out>) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/sql_select.cc:12739 #11 0x00000000005e8a32 in evaluate_join_record (join=join@entry=0x7fe77f4a4010, join_tab=join_tab@entry=0x7fe77f470a60, error=<optimized out>) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/sql_select.cc:11925 #12 0x00000000005e8c36 in sub_select (join=0x7fe77f4a4010, join_tab=0x7fe77f470a60, end_of_records=<optimized out>) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/sql_select.cc:11778 #13 0x00000000005f681d in do_select (join=join@entry=0x7fe77f4a4010, fields=fields@entry=0x7ff18f020130, table=table@entry=0x0, procedure=0x0) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/sql_select.cc:11543 #14 0x000000000060655e in JOIN::exec (this=0x7fe77f4a4010) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/sql_select.cc:2385 #15 0x0000000000602095 in mysql_select (thd=thd@entry=0x7ff18f01e000, rref_pointer_array=rref_pointer_array@entry=0x7ff18f0201f8, tables=<optimized out>, wild_num=<optimized out>, fields=..., conds=<optimized out>, og_num=0, order=0x0, group=0x0, having=0x0, proc_param=0x0, select_options=2147748608, result=result@entry=0x7fe338243b60, unit=0x7ff18f01fa00, select_lex=select_lex@entry=0x7ff18f020020) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/sql_select.cc:2604 #16 0x0000000000608235 in handle_select (thd=0x7ff18f01e000, lex=0x7ff18f01f950, result=0x7fe338243b60, setup_tables_done_option=0) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/sql_select.cc:297 #17 0x00000000005c23d1 in execute_sqlcom_select (thd=thd@entry=0x7ff18f01e000, all_tables=0x7fe338243270) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/sql_parse.cc:4604 #18 0x00000000005ca700 in mysql_execute_command (thd=thd@entry =0x7ff18f01e000) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/sql_parse.cc:2154 #19 0x00000000005cd7c2 in mysql_parse (parser_state=0x7ff21213a7e0, thd=0x7ff18f01e000, rawbuf=<optimized out>, length=<optimized out>) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/sql_parse.cc:5641 #20 mysql_parse (thd=0x7ff18f01e000, rawbuf=<optimized out>, length=65, parser_state=0x7ff21213a7e0) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/sql_parse.cc:5565 #21 0x00000000005cebcc in dispatch_command (command=COM_QUERY, thd=0x7ff18f01e000, packet=<optimized out>, packet_length=65) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/sql_parse.cc:1041 #22 0x000000000066495f in do_handle_one_connection (thd_arg=thd_arg@entry =0x7ff18f01e000) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/sql_connect.cc:853 #23 0x0000000000664a11 in handle_one_connection (arg=0x7ff18f01e000) at mysql-5.5.30-tokudb-7.0.1-linux-x86_64-build/mysql-5.5.30-tokudb-7.0.1/sql/sql_connect.cc:772 #24 0x00007ff214711851 in start_thread () from /lib64/libpthread.so.0

1 0

[Maria-developers] help with mdev-4687 - aria engina problem with <= operator
by Roberto Spadim 26 Jul '13

26 Jul '13

hi guys i have a problem with aria https://mariadb.atlassian.net/browse/MDEV-4687 some queries that depend on <= operator don't return right rows (return different from myisam or innodb) i'm getting back to myisam, or innodb these are read intensive tables, and innodb runs slower, and myisam have a crash safe problem (but runs...) please any help is very wellcome thanks guys -- Roberto Spadim

2 4

[Maria-developers] innodb key comparisons
by Rich Prohaska 25 Jul '13

25 Jul '13

i am interested in the performance of in memory join queries on tokudb and innodb and, for innodb, i would like to know how many key comparisons were needed when running the query. i counted the number of calls to a function called "cmp_dtuple_rec_with_match". does this include ALL of the key comparisons? thanks rich prohaska

1 0

[Maria-developers] Documentation for old regex library calls
by Sudheera Palihakkara 25 Jul '13

25 Jul '13

Hi all, I'm replacing the current regular expression library with modern library(pcre) for mariadb. I need some information about the ways that code uses the old regex library. The relevant codes are in sql/item_cmpfunc.cc directory. For exapmle in the function : int Item_func_regex::regcomp(bool send_error) I can't understand most of the lines, and the code is not well-commented. In the new library(pcre) uses different way to compile a regEx so I have to understand this and code for pcre preserving the functionalists of the class. Is there any documentation for this file or the way they used to work with old regex library ? thanks. -- *Sudheera Palihakkara.* Undergraduate Department of *Computer Science and Engineering, *Faculty of Engineering, *University of Moratuwa*, Sri Lanka.

2 2

[Maria-developers] some bugs in dingqing parallel replication
by nanyi607rao 22 Jul '13

22 Jul '13

Hi,guys I have worked on this branch https://code.launchpad.net/~knielsen/maria/dingqi-parallel-replication for some days, and found bugs listed below.May this would be helpful to you. 1, when slave switch on table filter,this bug could lead server crash. how to reappear: on slave set replicate-wild-ignore-table = test.t5 in config file on master do these operations CREATE TABLE test.t3 (a INT AUTO_INCREMENT PRIMARY KEY, b DECIMAL(20,20), c INT); SET INSERT_ID=1; SET @c=2; SET @@rand_seed1=10000000, @@rand_seed2=1000000; INSERT INTO t3 VALUES (NULL, RAND(), @c); codes lead this bug: In execute_single_transaction() case RAND_EVENT: need_remove_from_trans= true; if(!rli->is_deferred_event(ev)) delete ev; break; reason: Rand Event object is deleted in execute_single_transaction(), but it's pointer would be used is slave_execute_deferred_events() later. 2, SQL thread could read and apply some log events repeated. how to reappear: it's a little hard to reappear. if you set max_relay_log_size=100M and keep SQL thread closed to IO thread, this bug may reappear. codes lead this bug: In reopen_relay_log() rli->event_relay_log_pos= max(rli->event_relay_log_pos, BIN_LOG_HEADER_SIZE); my_b_seek(cur_log,rli->event_relay_log_pos); reason: when SQL thread use a hot log,but the hot log was closed by IO thread just recently, SQL thread need to reopen this log and set read offset to rli->event_relay_log_pos, while rli->event_relay_log_pos could be set new value in other thread for there are many threads apply log events.so rli->event_relay_log_pos could be less then rli->future_event_relay_log_pos. 3, SQL thread do not report error information in result of "show slave status"and replication do not stop, when the slave insert duplicate record into a table with primary key. how to reappear: Just need to change master_log_pos to read duplicate records from master. codes lead this bug: In execute_single_transaction() retry_transaction: ev= trans->event_list_head; ... ... if (ret && rli->trans_retries < slave_trans_retries) { ... goto retry_transaction; } reason: as I have sayed in other email: Rows_log_event::do_apply_event() do twice but return different results for m_curr_row==m_rows_end in the second time. 4, when do oparetions such as "show slave status" and "stop slave", it could be blocked for a long time. how to reappear: just do "show slave status" again and again. codes lead this bug: In the queue_event() case FORMAT_DESCRIPTION_EVENT: ... wait_for_all_dml_done(&mi->rli, true); and in process_io_rotate() wait_for_all_dml_done(&mi->rli, true); reason: IO thread could wait in wait_for_all_dml_done() while holding the rpl_mi->data_lock, so operations like "show slave status" could be blocked for waiting rpl_mi->data_lock. 5, "START SLAVE UNTIL" make replication stop in different place. how to reappear: suppose log events in relay log like: BEGIN; ------->pos1 LOG_EVENT1; LOG_EVENT2; COMMIT; ------>pos2 BEGIN; ------>pos3 LOG_EVENT3; --->stop_pos LOG_EVENT4; COMMIT; ------>pos4 If we do START SLAVE UNTIL relay_log_pos=stop_pos; The replication should stop at pos4 but it stop pos2. 6, log_event->thd is wrong. suppose log_event was read is thread_1 so the log_event->thd==thread_1, but this log_event may be dispatch to other thread (suppose thread_2).the log_event is applyed in thread_2 but the log_event->thd==thread_1.this problem can make log event apply failed in MySQL, but in mariaDB it seems ok. 2013-07-19 nanyi607rao

2 1

[Maria-developers] RFC getting information from image (blob)
by Roberto Spadim 21 Jul '13

21 Jul '13

Hi guys, i will implement a gis system, but my problem isn't gis, my problem is raster (bitmap) images I will save images at mariadb using longblobs And i will create a table with, datetime and image I will need read each r,g,b value from each pixel, example.. Select rvalue(1,2,loadimage(blobfield)) >From table Order by datetime Rvalue= red value of x1 and y2 What's the faster or at least, how to use less cpu and memory I didn't search about images udf functions for mysql Any help is wellcome and ideas too :) Well i will use others sum, max, min functions too I don't want to read all images at client side, i want that server do this job I don't know the image size (it can be any pratical size, max near to 16mega pixels), image format could be png, jpg, or another one but i can use client side to rewrite image file to right format I'm considering implement some imagemagick functions inside mariadb, what you think? Thanks guys

2 2

[Maria-developers] Make error - cannot find -lregex
by Sudheera Palihakkara 21 Jul '13

21 Jul '13

Hi all, I have done some editing to sql/item_cmpfunc.cc file and CMakeList filesand then when I run the make command( after running cmake) it halts with this error in the middle. ([68%]) Linking CXX executable explain_filename-t /usr/bin/ld.bfd.real: cannot find -lregex collect2: error: ld returned 1 exit status make[2]: *** [unittest/mysys/explain_filename-t] Error 1 make[1]: *** [unittest/mysys/CMakeFiles/explain_filename-t.dir/all] Error 2 make: *** [all] Error 2 what might have gone wrong. :( -- *Sudheera Palihakkara.* Undergraduate Department of *Computer Science and Engineering, *Faculty of Engineering, *University of Moratuwa*, Sri Lanka.

2 3

[Maria-developers] Structure of MariaDB 10.0 binary logs
by Giuseppe Maxia 20 Jul '13

20 Jul '13

Hi, Where can I find a detailed structure of the binary logs supported by MariaDB 10.0.x? Specifically, I would like to know * what are the differences between binary logs in MariaDB 5.5 and 10.0 * what are the differences between binary logs in MariaDB 10.0 and MySQL 5.6. The purpose of this request id to parse the binary logs for usage in external applications. Thanks Giuseppe

2 2

[Maria-developers] Help about patchs in mariadb - jira and lauchpad url
by Roberto Spadim 18 Jul '13

18 Jul '13

Hi guys, where patchs are commited in mariadb? for example this one: https://mariadb.atlassian.net/browse/MDEV-617 it don't have a link to any patch, there's a link to any lauchpad code? -- Roberto Spadim

2 5

[Maria-developers] MariaDB 5.5.32 Release Prep
by Daniel Bartholomew 18 Jul '13

18 Jul '13

All, I've begun the prep work for the MariaDB 5.5.32 release. The packages are uploaded and the mirrors should start picking them up soon. The draft changelog and release notes are here: - https://kb.askmonty.org/en/mariadb-5532-changelog/ - https://kb.askmonty.org/en/mariadb-5532-release-notes/ Please let me know ASAP if there's anything that needs to be mentioned/highlighted/removed/etc... Assuming all goes well, the plan is to activate the release at around 12:00 US Eastern Time tomorrow (18 Jul 2013). Thanks. -- Daniel Bartholomew Google+ - http://gplus.to/dbart Twitter - http://twitter.com/daniel_bart MariaDB: An Enhanced Drop-in Replacement for MySQL Website - http://mariadb.org Twitter - http://twitter.com/mariadb Google+ - http://google.com/+mariadb Facebook - http://fb.com/MariaDB.dbms Knowledgebase - http://kb.askmonty.org

2 4

Re: [Maria-developers] [Commits] Rev 3839: Fix of using uninitialized variadle. in file:///home/bell/maria/bzr/work-maria-5.5-valgrind/
by Sergei Petrunia 18 Jul '13

18 Jul '13

Hi Sanja, On Thu, Jul 18, 2013 at 11:16:21AM +0300, sanja(a)montyprogram.com wrote: > At file:///home/bell/maria/bzr/work-maria-5.5-valgrind/ > > ------------------------------------------------------------ > revno: 3839 > revision-id: sanja(a)montyprogram.com-20130718081618-6ax63urznfnqmzgt > parent: sanja(a)montyprogram.com-20130717140359-123bw8dzelj6wddn > committer: sanja(a)montyprogram.com > branch nick: work-maria-5.5-valgrind > timestamp: Thu 2013-07-18 11:16:18 +0300 > message: > Fix of using uninitialized variadle. > === modified file 'sql/sql_select.cc' > --- a/sql/sql_select.cc 2013-07-17 13:42:13 +0000 > +++ b/sql/sql_select.cc 2013-07-18 08:16:18 +0000 > @@ -15674,6 +15674,8 @@ create_internal_tmp_table_from_heap2(THD > const char *save_proc_info; > int write_err= 0; > DBUG_ENTER("create_internal_tmp_table_from_heap2"); > + if (is_duplicate) > + *is_duplicate= FALSE; > > if (table->s->db_type() != heap_hton || > error != HA_ERR_RECORD_FILE_FULL) > > === modified file 'sql/sql_union.cc' > --- a/sql/sql_union.cc 2013-07-16 05:22:17 +0000 > +++ b/sql/sql_union.cc 2013-07-18 08:16:18 +0000 > @@ -83,7 +83,7 @@ int select_union::send_data(List<Item> & > */ > return -1; > } > - bool is_duplicate; > + bool is_duplicate= FALSE; If you do the part in create_internal_tmp_table_from_heap2(), is this change really necessary? > /* create_internal_tmp_table_from_heap will generate error if needed */ > if (table->file->is_fatal_error(write_err, HA_CHECK_DUP) && > create_internal_tmp_table_from_heap(thd, table, > Ok to push either way. BR Sergei -- Sergei Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] LDAP authentication plugin
by Sergei 18 Jul '13

18 Jul '13

Hi, Ilija! On Jul 11, Ilija Lazarevic wrote: > Hi Sergei, > > The reason I'm writing you is because I'm interested > in details about this gsoc2013 proposal, and I saw you were > supposed to be the mentor. > > Is it open for contributing apart from gsoc? Yes, it is. But please, write to maria-developers(a)lists.launchpad.net next time, not to me personally. Regards, Sergei

3 2

[Maria-developers] help creating error messages
by Roberto Spadim 16 Jul '13

16 Jul '13

hi guys, how could i create a new error / warning message in mariadb? 1) how to add the message text to error list (there's a list, right?) 2) how to return (send) a error to client 3) how to add a message to 'warning list' and after get it with SHOW WARNINGS maybe some example or point lines / files that i should read thanks guys :) -- Roberto Spadim

2 5

[Maria-developers] More enum values in enum fields
by Roberto Spadim 16 Jul '13

16 Jul '13

Hi guys, i have a problem, that i don't know how to easly solve (ok i know but i want the user easy way) i have a ENUM column with some values (today 248 distinct values) i see that in a near future i will have more than 65535 different values inside this enum column, the first solution is change it to varchar, but my problem will be data size the enum is very optimized in size, a char is 'converted' to a smallint value my second solution is a int column, and a external table with int/text values, but here i need to change apps, i don't want the second one, but i don't want a very big database (char column) how could i change the limit of enum size? maybe a new enum2 type? with a limit of bigint value (64bits), ok maybe less (32bits) today (reading code) i see that big enum is a problem for .frm file that don't allow this size of enums and i need to get back to a second table, but what should be done to allow a very big enum field? there's a "easy" way to change mariadb code to allow a bigger enum size? thanks guys -- Roberto Spadim

4 19

[Maria-developers] The funcation of retry_transaction in parallen replication
by nanyi607rao 16 Jul '13

16 Jul '13

hi, In order to test "retry_transaction" in function execute_single_transaction(), I change Relay_log_pos to make SQL thread insert Duplicate entry, but I find it successed and do not report error! so I debug the function execute_single_transaction() and find the reason : at first time, the insert Rows_log_event::do_apply_event() return 121, that means insert fails. But when retry the insert Rows_log_event::do_apply_event(),it return 0, that means insert success! it seems the Rows_log_event::do_apply_event() do twice, but return different results. how this happened? is it a bug of Rows_log_event::do_apply_event() ? By the way, I do this test in mysql-5.5.30, do mariaDB has the some problem? 2013-07-12 nanyi607rao

3 3

[Maria-developers] What issue tracker should we use?
by Roberto Spadim 16 Jul '13

16 Jul '13

hi guys, i see that mariadb have three issue tracker, i'm right? i saw the https://askmonty.org/worklog/Server-RawIdeaBin/?tid=145 (ask monty), jira, and lauchpad what issue tracker should be used? there's any idea porting the others issue to the main issue tracker? -- Roberto Spadim SPAEmpresarial

3 4

[Maria-developers] Reserved words
by Ian Gilfillan 14 Jul '13

14 Jul '13

How can one find a definitive list of reserved words for a particular version? The comments in sql/lex.h say that that's a list of all reserved words and functions, but it appears to contain some words that are neither (such as ACTION). SELECT * FROM mysql.help_keyword also lists ACTION ACTION is listed in the MYSQL docs as an example of one of the keywords that MySQL permits "because many people previously used them"

2 1

[Maria-developers] Pluggable authentication, custom error messages
by Vladislav Vaintroub 14 Jul '13

14 Jul '13

Hi Serg, is there any possibility to return own error messages and codes , if pluggable authentication failed on client on some reason ? Case in point here is Kerberos (GSSAPI or SSPI) - it may fail on client or on server, with some GSSAPI/SSPI specific code. We would like user to know what exactly happened , rather than return a generic CR_ERROR. Perhaps user can fix the situation - maybe he just needs to obtain Kerberos ticket with kinit, or login as Windows domain user, or something like that. If error happens on the server side, at least there it is possible to write to error log. But on the client side, there is no error log, and writing to stdout/err is not a good idea. Is there a trick that allows custom messages on the client? Wlad

2 2