- developers - lists.mariadb.org

[Maria-developers] 10.0 sprint: Please advise re MDEV-8109 unexpected CAST result
by Alexander Barkov 11 Jun '15

11 Jun '15

Hi Sergei, I'm thinking about what to do with MDEV-8109 unexpected CAST result. Observations: 1. An empty string in VALUES SET sql_mode='STRICT_ALL_TABLES'; DROP TABLE IF EXISTS t1; CREATE TABLE t1 (a DECIMAL); INSERT INTO t1 VALUES (''); ERROR 1292 (22007): Truncated incorrect DECIMAL value: '' This is OK. In strict mode INSERT with a bad value correctly returns the error. 2. CAST('' AS DECIMAL) SET sql_mode='STRICT_ALL_TABLES'; SELECT CAST('' AS DECIMAL); +---------------------+ | CAST('' AS DECIMAL) | +---------------------+ | 0 | +---------------------+ 1 row in set, 1 warning (0.00 sec) This is also OK. There are no tables involved, and CAST itself does not convert warnings to errors. 3. CAST('' AS DECIMAL) in VALUES: INSERT INTO t1 VALUES(CAST('' AS DECIMAL)); ERROR 1292 (22007): Truncated incorrect DECIMAL value: '' I think this is not OK. I wrote an explicit CAST, so I expect: - CAST to return 0 with a warning, like in #2 - INSERT to write 0 into the table normally, as I'm actually inserting the CAST result, which is 0. 4. COLUMN_GET(AS DECIMAL) in VALUES: SET @aaa = COLUMN_CREATE('price', ''); INSERT INTO t1 VALUES(COLUMN_GET(@aaa, 'price' AS DECIMAL)); ERROR 1918 (22007): Encountered illegal value '' when converting to DECIMAL This is also not OK. I wrote explicit column conversion to DECIMAL. I expect: - COLUMN_GET() to return 0 with a warning, like in #2 - INSERT to write 0 into the table normally, as I'm actually inserting the COLUMN_GET result, which is 0. Do you agree? Thanks.

2 4

Re: [Maria-developers] MariaDB replication event generators?
by Kristian Nielsen 11 Jun '15

11 Jun '15

James Cheng <jcheng(a)tivo.com> writes: > The binlog stream has all the row data, but doesn't have > everything. It doesn't have column names, for example, or primary > keys, or character sets. Using the binlog for replication seems to > assume that you have a copy of the schema beforehand on the receiving > slave. It doesn't seem possible to replicate to a slave using only the > information in the binlogs. It is a common request to be able to do 3rd-party replication from mysqld. However, the replication protocol between master and slave servers was never designed as a public API, it is based on internal data formats and so on. So unfortunately, there does not exist currently a public API for doing external replication, at least not that I am aware of. The binlog stream _does_ have everything of course, eg. in the form of CREATE TABLE statements and so on. However, being internal to the server, they are not very easy to use by an external replication mechanism. - Kristian.

1 0

Re: [Maria-developers] [Commits] 6ff2c0a: MDEV-5309 - RENAME TABLE does not check for existence of the table's engine
by Sergei Golubchik 10 Jun '15

10 Jun '15

Hi, Sergey! On Feb 09, svoj(a)mariadb.org wrote: > revision-id: 6ff2c0a05155f985484e23bd8611eb95f045907f > parent(s): f13939061d468a47985dee0268652a6fe2db7862 > committer: Sergey Vojtovich > branch nick: 10.1 > timestamp: 2015-02-09 16:12:00 +0400 > message: > > MDEV-5309 - RENAME TABLE does not check for existence of the table's engine > > When RENAME TABLE is executed, it apparently does not check whether the engine > is available (unlike ALTER TABLE .. RENAME, which does). It means that if the > engine in question was not loaded on some reason, the table might become > unusable, since the engine won't know about the change. > > With this patch RENAME TABLE fails if storage engine is not available. .... > diff --git a/sql/sql_table.cc b/sql/sql_table.cc > index e9a1ae9..f1096da 100644 > --- a/sql/sql_table.cc > +++ b/sql/sql_table.cc > @@ -5172,18 +5172,19 @@ bool mysql_create_table(THD *thd, TABLE_LIST *create_table, > error= my_errno; > (void) file->ha_create_partitioning_metadata(to, from, CHF_RENAME_FLAG); > } > - else if (!file || !(error=file->ha_rename_table(from_base, to_base))) > + else if (!file) > { > + error= ER_UNKNOWN_STORAGE_ENGINE; > + } > + else if (!(error= file->ha_rename_table(from_base, to_base))) > + { > if (!(flags & NO_FRM_RENAME) && rename_file_ext(from,to,reg_ext)) > { > error= my_errno; > - if (file) > - { > if (error == ENOENT) > error= 0; // this is ok if file->ha_rename_table() succeeded > else > file->ha_rename_table(to_base, from_base); // Restore old file name > - } > } > } > delete file; May be you'd rather fix it in the caller? Let's state that the first argument of mysql_rename_table() can never be NULL. While mysql_rename_table() is called in many places, the only place where it can actually get a NULL hton is sql_rename.cc, if ha_table_exists() returns hton=NULL. I'd add the check here and simplified mysql_rename_table(). Regards, Sergei

1 0

[Maria-developers] Progress Report on [MDEV5199]: UDF Table Functions.
by Blessing Atie 10 Jun '15

10 Jun '15

Hi Serg, Colin! The following is a rundown on my progress with the above task. I started this project by getting the source of 10.1 from Github, setting my working environment by building and running the tests on ubuntu 14 then proceeded with an analysis by studying plugins and how they interact with the server.(Especially storage engines plugins). With analysis and sources investigations being done, I realized with help from assigned mentor(Serg) that our task is about implementing MYSQL_UDF_PLUGIN with the functionalities of the CONNECT engine which is similar in functionality to table functions in a way. This past week, I worked on the interface for our plugin by declaring a structure with version and some useful init/deinit function pointers to work with our UDFs(table UDF only for now). I am currently studying the Connect engine to see how I can implement its functionalities as a UDF plugin. For this week, I intend to look at other databases for example table functions that will be good candidates that we can implement with our plugin(This was not mentioned in my proposal but I intend to implement a couple of them using our plugin even if we are short of time after the final evaluation). So far, I have had difficulties in communication due to unscheduled powerouts in my city. This has really been an issue to me because my mentor was beginning to loose hope and on the other side, I don't have the chance to ask questions when im faced with a powerout. I really hope my mentors understand with me how difficult it is to code from Cameroon. Please feel free to request for any important details i might have not mentioned which you will want to know. Best Wishes, Blessing Atie

2 1

Re: [Maria-developers] MariaDB replication event generators?
by Kristian Nielsen 10 Jun '15

10 Jun '15

James Cheng <jcheng(a)tivo.com> writes: > On Jun 7, 2015, at 12:05 AM, Kristian Nielsen <knielsen(a)knielsen-hq.org> wrote: > >> James Cheng <jcheng(a)tivo.com> writes: >> >>> I recently read your old blog post from 2010 >>> (http://kristiannielsen.livejournal.com/14305.html) about some ideas >>> for MariaDB replication. In particular, your ideas for replication >>> event generators that would allow plugins to be written to receive >>> replication events. >>> >>> Did this work ever happen in MariaDB? Or anything close/equivalent? If >>> so, could you point me to something so that I can learn more about it? >> >> Unfortunately, nothing like this ever happened. > That's too bad. Are there any interesting details about why it never > happened? Was it simply due to not enough time, not high enough > priority? Was there any technical or organizational resistance to the > actual ideas or functionality? If someone were to build this today and > contribute it back to the community, do you think it would be > accepted? I think there was not enough interest and/or time/priority. For example, Galera cluster would be a prime example for something using this kind of API. But the developers did not prioritise spending time working on such API, rather they hacked in their functionality all over the server. It is the same with other parties in the community, everyone is focused on solving their own narrow problem, not in working together. And MySQL@Oracle, of course, is not interested in anything that involves working with anyone else on the code. So I don't see much chance in the current state of the community of any significant effort being spent neither on developing such an API, nor on using it. - Kristian.

1 0

Re: [Maria-developers] [Commits] 865bbbf: MDEV-3870 - Valgrind warnings on OPTIMIZE MyISAM or Aria TABLE with disabled
by Sergei Golubchik 09 Jun '15

09 Jun '15

Hi, Sergey! On Jun 09, svoj(a)mariadb.org wrote: > revision-id: 865bbbf14d1a8ed500eeeadabe0a7fdfa90420df > parent(s): b8e26b04d4ac22e29a78bca68e101176b6ef507a > committer: Sergey Vojtovich > branch nick: mariadb > timestamp: 2015-06-09 16:08:09 +0400 > message: > > MDEV-3870 - Valgrind warnings on OPTIMIZE MyISAM or Aria TABLE with disabled > keys > > Fixed that OPTIMIZE TABLE against MyISAM/Aria table may write uninitialized key > root position for disabled keys. ok to push, thanks! Regards, Sergei

1 0

Re: [Maria-developers] [Commits] 7948cc3: MDEV-8260 : Issues related to concurrent CTAS
by Sergei Golubchik 09 Jun '15

09 Jun '15

Hi, Nirbhay! On Jun 08, Nirbhay Choubey wrote: > revision-id: 7948cc32b6c12c9696e8b497ae90f123cafcd234 > parent(s): 6d5b723bdc3e04978619b9673fca266e0426916f > committer: Nirbhay Choubey > branch nick: 10.0-galera.ctas > timestamp: 2015-06-08 22:50:26 -0400 > message: > > MDEV-8260 : Issues related to concurrent CTAS > > * Wait for aborted thd (victim) to release MDL locks > * Skip aborting an already aborted thd > * Defer setting OK status in case of CTAS > * Minor cosmetic changes > * Added a test case ... > +bool select_insert::send_eof() > +{ > + DBUG_ENTER("select_insert::send_eof"); > + DBUG_RETURN(prepare_eof() || send_ok_packet()); > } Please, don't do that, don't put function calls in DBUG_RETURN. This will result in dbug traces like > select_insert::send_eof < select_insert::send_eof > select_insert::prepare_eof < select_insert::prepare_eof > select_insert::send_ok_packet < select_insert::send_ok_packet while the correct trace should've be > select_insert::send_eof | > select_insert::prepare_eof | < select_insert::prepare_eof | > select_insert::send_ok_packet | < select_insert::send_ok_packet < select_insert::send_eof So, either write like bool res= prepare_eof() || send_ok_packet(); DBUG_RETURN(res); or fix DBUG_RETURN() macro. Whatever you prefer :) Otherwise the patch is ok. Regards, Sergei

2 1

[Maria-developers] Review requested for MDEV-7818
by Kristian Nielsen 09 Jun '15

09 Jun '15

I need someone to review my below fix for MDEV-7818. This involves the FLUSH TABLES WITH READ LOCK command and associated metadata locking, something that I know little about. Monty suggested the basic semantics - that all currently running parallel replication transactions would be allowed to complete before granting the global read lock. But it really needs a review of the actual implementation, by someone who has an understanding of the FTWRL semantics and implementation. Thanks, - Kristian. Kristian Nielsen <knielsen(a)knielsen-hq.org> writes: > revision-id: 0e609daea4796fafca94f7adf796addfc506168b > parent(s): 22b6c297dc0de6b176238ae820bddb5e70f2c8ab > committer: Kristian Nielsen > branch nick: mariadb > timestamp: 2015-06-09 16:01:53 +0200 > message: > > MDEV-7818: Deadlock occurring with parallel replication and FTWRL > > Problem is that FLUSH TABLES WITH READ LOCK first blocks threads from > starting new commits, then waits for running commits to complete. But > in-order parallel replication needs commits to happen in a particular > order, so this can easily deadlock. > > To fix this problem, this patch introduces a way to temporarily pause > the parallel replication worker threads. Before starting FTWRL, we let > all worker threads complete in-progress transactions, and then > wait. Then we proceed to take the global read lock. Once the lock is > obtained, we unpause the worker threads. Now commits are blocked from > starting by the global read lock, so the deadlock will no longer occur. > > --- > sql/mysqld.cc | 3 + > sql/mysqld.h | 3 + > sql/rpl_parallel.cc | 230 ++++++++++++++++++++++++++++++++++++++++++++++++++-- > sql/rpl_parallel.h | 27 ++++-- > sql/sql_parse.cc | 13 +++ > 5 files changed, 262 insertions(+), 14 deletions(-) > > diff --git a/sql/mysqld.cc b/sql/mysqld.cc > index e05c0b6..af8ae2c 100644 > --- a/sql/mysqld.cc > +++ b/sql/mysqld.cc > @@ -9514,6 +9514,9 @@ PSI_stage_info stage_waiting_for_prior_transaction_to_commit= { 0, "Waiting for > PSI_stage_info stage_waiting_for_prior_transaction_to_start_commit= { 0, "Waiting for prior transaction to start commit before starting next transaction", 0}; > PSI_stage_info stage_waiting_for_room_in_worker_thread= { 0, "Waiting for room in worker thread event queue", 0}; > PSI_stage_info stage_waiting_for_workers_idle= { 0, "Waiting for worker threads to be idle", 0}; > +PSI_stage_info stage_waiting_for_ftwrl= { 0, "Waiting due to global read lock", 0}; > +PSI_stage_info stage_waiting_for_ftwrl_threads_to_pause= { 0, "Waiting for worker threads to pause for global read lock", 0}; > +PSI_stage_info stage_waiting_for_rpl_thread_pool= { 0, "Waiting while replication worker thread pool is busy", 0}; > PSI_stage_info stage_master_gtid_wait_primary= { 0, "Waiting in MASTER_GTID_WAIT() (primary waiter)", 0}; > PSI_stage_info stage_master_gtid_wait= { 0, "Waiting in MASTER_GTID_WAIT()", 0}; > PSI_stage_info stage_gtid_wait_other_connection= { 0, "Waiting for other master connection to process GTID received on multiple master connections", 0}; > diff --git a/sql/mysqld.h b/sql/mysqld.h > index 156e7f9..8c953c1 100644 > --- a/sql/mysqld.h > +++ b/sql/mysqld.h > @@ -454,6 +454,9 @@ extern PSI_stage_info stage_waiting_for_prior_transaction_to_commit; > extern PSI_stage_info stage_waiting_for_prior_transaction_to_start_commit; > extern PSI_stage_info stage_waiting_for_room_in_worker_thread; > extern PSI_stage_info stage_waiting_for_workers_idle; > +extern PSI_stage_info stage_waiting_for_ftwrl; > +extern PSI_stage_info stage_waiting_for_ftwrl_threads_to_pause; > +extern PSI_stage_info stage_waiting_for_rpl_thread_pool; > extern PSI_stage_info stage_master_gtid_wait_primary; > extern PSI_stage_info stage_master_gtid_wait; > extern PSI_stage_info stage_gtid_wait_other_connection; > diff --git a/sql/rpl_parallel.cc b/sql/rpl_parallel.cc > index 738d0e5..d90507a 100644 > --- a/sql/rpl_parallel.cc > +++ b/sql/rpl_parallel.cc > @@ -280,6 +280,8 @@ do_gco_wait(rpl_group_info *rgi, group_commit_orderer *gco, > rpl_parallel_entry *entry= rgi->parallel_entry; > uint64 wait_count; > > + mysql_mutex_assert_owner(&entry->LOCK_parallel_entry); > + > if (!gco->installed) > { > group_commit_orderer *prev_gco= gco->prev_gco; > @@ -336,6 +338,159 @@ do_gco_wait(rpl_group_info *rgi, group_commit_orderer *gco, > } > > > +static void > +do_ftwrl_wait(rpl_group_info *rgi, > + bool *did_enter_cond, PSI_stage_info *old_stage) > +{ > + THD *thd= rgi->thd; > + rpl_parallel_entry *entry= rgi->parallel_entry; > + uint64 sub_id= rgi->gtid_sub_id; > + > + mysql_mutex_assert_owner(&entry->LOCK_parallel_entry); > + > + if (unlikely(entry->pause_sub_id > 0) && sub_id > entry->pause_sub_id) > + { > + thd->ENTER_COND(&entry->COND_parallel_entry, &entry->LOCK_parallel_entry, > + &stage_waiting_for_ftwrl, old_stage); > + *did_enter_cond= true; > + do > + { > + if (entry->force_abort || rgi->worker_error) > + break; > + if (thd->check_killed()) > + { > + thd->send_kill_message(); > + slave_output_error_info(rgi, thd); > + signal_error_to_sql_driver_thread(thd, rgi, 1); > + break; > + } > + mysql_cond_wait(&entry->COND_parallel_entry, &entry->LOCK_parallel_entry); > + } while (entry->pause_sub_id > 0 && sub_id > entry->pause_sub_id); > + } > + > + if (sub_id > entry->largest_started_sub_id) > + entry->largest_started_sub_id= sub_id; > +} > + > + > +static void > +pool_mark_busy(rpl_parallel_thread_pool *pool) > +{ > + mysql_mutex_assert_owner(&pool->LOCK_rpl_thread_pool); > + DBUG_ASSERT(!pool->busy); > + pool->busy= true; > +} > + > + > +static void > +pool_mark_not_busy(rpl_parallel_thread_pool *pool) > +{ > + mysql_mutex_assert_owner(&pool->LOCK_rpl_thread_pool); > + DBUG_ASSERT(pool->busy); > + pool->busy= false; > + mysql_cond_broadcast(&pool->COND_rpl_thread_pool); > +} > + > + > +void > +rpl_unpause_for_ftwrl(THD *thd) > +{ > + uint32 i; > + rpl_parallel_thread_pool *pool= &global_rpl_thread_pool; > + > + DBUG_ASSERT(pool->busy); > + > + for (i= 0; i < pool->count; ++i) > + { > + rpl_parallel_entry *e; > + rpl_parallel_thread *rpt= pool->threads[i]; > + > + mysql_mutex_lock(&rpt->LOCK_rpl_thread); > + if (!rpt->current_owner) > + { > + mysql_mutex_unlock(&rpt->LOCK_rpl_thread); > + continue; > + } > + e= rpt->current_entry; > + mysql_mutex_lock(&e->LOCK_parallel_entry); > + mysql_mutex_unlock(&rpt->LOCK_rpl_thread); > + e->pause_sub_id= 0; > + mysql_cond_broadcast(&e->COND_parallel_entry); > + } > + > + mysql_mutex_lock(&pool->LOCK_rpl_thread_pool); > + pool_mark_not_busy(pool); > + mysql_cond_broadcast(&pool->COND_rpl_thread_pool); > + mysql_mutex_unlock(&pool->LOCK_rpl_thread_pool); > +} > + > + > +/* > + . > + > + Note: in case of error return, unpause_for_ftwrl() must _not_ be called. > +*/ > +int > +rpl_pause_for_ftwrl(THD *thd) > +{ > + uint32 i; > + rpl_parallel_thread_pool *pool= &global_rpl_thread_pool; > + int err= 0; > + > + /* > + While the count_pending_pause_for_ftwrl counter is non-zero, the pool > + cannot be shutdown/resized, so threads are guaranteed to not disappear. > + > + This is required to safely be able to access the individual threads below. > + (We cannot lock an individual thread while holding LOCK_rpl_thread_pool, > + as this can deadlock against release_thread()). > + */ > + mysql_mutex_lock(&pool->LOCK_rpl_thread_pool); > + pool_mark_busy(pool); > + mysql_mutex_unlock(&pool->LOCK_rpl_thread_pool); > + > + for (i= 0; i < pool->count; ++i) > + { > + PSI_stage_info old_stage; > + rpl_parallel_entry *e; > + rpl_parallel_thread *rpt= pool->threads[i]; > + > + mysql_mutex_lock(&rpt->LOCK_rpl_thread); > + if (!rpt->current_owner) > + { > + mysql_mutex_unlock(&rpt->LOCK_rpl_thread); > + continue; > + } > + e= rpt->current_entry; > + mysql_mutex_lock(&e->LOCK_parallel_entry); > + mysql_mutex_unlock(&rpt->LOCK_rpl_thread); > + ++e->need_sub_id_signal; > + if (!e->pause_sub_id) > + e->pause_sub_id= e->largest_started_sub_id; > + thd->ENTER_COND(&e->COND_parallel_entry, &e->LOCK_parallel_entry, > + &stage_waiting_for_ftwrl_threads_to_pause, &old_stage); > + while (e->last_committed_sub_id < e->pause_sub_id && !err) > + { > + if (thd->check_killed()) > + { > + thd->send_kill_message(); > + err= 1; > + break; > + } > + mysql_cond_wait(&e->COND_parallel_entry, &e->LOCK_parallel_entry); > + }; > + --e->need_sub_id_signal; > + thd->EXIT_COND(&old_stage); > + if (err) > + break; > + } > + > + if (err) > + rpl_unpause_for_ftwrl(thd); > + return err; > +} > + > + > #ifndef DBUG_OFF > static int > dbug_simulate_tmp_error(rpl_group_info *rgi, THD *thd) > @@ -856,6 +1011,9 @@ handle_rpl_parallel_thread(void *arg) > > if (unlikely(entry->stop_on_error_sub_id <= rgi->wait_commit_sub_id)) > skip_event_group= true; > + if (likely(!skip_event_group)) > + do_ftwrl_wait(rgi, &did_enter_cond, &old_stage); > + > register_wait_for_prior_event_group_commit(rgi, entry); > > unlock_or_exit_cond(thd, &entry->LOCK_parallel_entry, > @@ -1058,6 +1216,47 @@ rpl_parallel_change_thread_count(rpl_parallel_thread_pool *pool, > rpl_parallel_thread **new_list= NULL; > rpl_parallel_thread *new_free_list= NULL; > rpl_parallel_thread *rpt_array= NULL; > + THD *thd; > + PSI_stage_info old_stage; > + int res; > + > + /* > + Wait here while the queue is busy. This is done to make FLUSH TABLES WITH > + READ LOCK work correctly, without incuring extra locking penalties in > + normal operation. FLUSH TABLES WITH READ LOCK needs to lock threads in the > + thread pool, and for this we need to make sure the pool will not go away > + during the operation. The LOCK_rpl_thread_pool is not suitable for > + this. It is taken by release_thread() while holding LOCK_rpl_thread; so it > + must be released before locking any LOCK_rpl_thread lock, or a deadlock > + can occur. > + > + So we protect the infrequent operations of FLUSH TABLES WITH READ LOCK and > + pool size changes with this condition wait. > + */ > + thd= current_thd; > + res= 0; > + mysql_mutex_lock(&pool->LOCK_rpl_thread_pool); > + if (thd) > + thd->ENTER_COND(&pool->COND_rpl_thread_pool, &pool->LOCK_rpl_thread_pool, > + &stage_waiting_for_rpl_thread_pool, &old_stage); > + while (pool->busy) > + { > + if (thd->check_killed()) > + { > + thd->send_kill_message(); > + res= 1; > + break; > + } > + mysql_cond_wait(&pool->COND_rpl_thread_pool, &pool->LOCK_rpl_thread_pool); > + } > + if (!res) > + pool_mark_busy(pool); > + if (thd) > + thd->EXIT_COND(&old_stage); > + else > + mysql_mutex_unlock(&pool->LOCK_rpl_thread_pool); > + if (res) > + return res; > > /* > Allocate the new list of threads up-front. > @@ -1106,7 +1305,14 @@ rpl_parallel_change_thread_count(rpl_parallel_thread_pool *pool, > */ > for (i= 0; i < pool->count; ++i) > { > - rpl_parallel_thread *rpt= pool->get_thread(NULL, NULL); > + rpl_parallel_thread *rpt; > + > + mysql_mutex_lock(&pool->LOCK_rpl_thread_pool); > + while ((rpt= pool->free_list) == NULL) > + mysql_cond_wait(&pool->COND_rpl_thread_pool, &pool->LOCK_rpl_thread_pool); > + pool->free_list= rpt->next; > + mysql_mutex_unlock(&pool->LOCK_rpl_thread_pool); > + mysql_mutex_lock(&rpt->LOCK_rpl_thread); > rpt->stop= true; > mysql_cond_signal(&rpt->COND_rpl_thread); > mysql_mutex_unlock(&rpt->LOCK_rpl_thread); > @@ -1157,7 +1363,7 @@ rpl_parallel_change_thread_count(rpl_parallel_thread_pool *pool, > } > > mysql_mutex_lock(&pool->LOCK_rpl_thread_pool); > - mysql_cond_broadcast(&pool->COND_rpl_thread_pool); > + pool_mark_not_busy(pool); > mysql_mutex_unlock(&pool->LOCK_rpl_thread_pool); > > return 0; > @@ -1182,6 +1388,9 @@ rpl_parallel_change_thread_count(rpl_parallel_thread_pool *pool, > } > my_free(new_list); > } > + mysql_mutex_lock(&pool->LOCK_rpl_thread_pool); > + pool_mark_not_busy(pool); > + mysql_mutex_unlock(&pool->LOCK_rpl_thread_pool); > return 1; > } > > @@ -1445,7 +1654,7 @@ rpl_parallel_thread::loc_free_gco(group_commit_orderer *gco) > > > rpl_parallel_thread_pool::rpl_parallel_thread_pool() > - : count(0), threads(0), free_list(0), inited(false) > + : threads(0), free_list(0), count(0), inited(false), busy(false) > { > } > > @@ -1453,9 +1662,10 @@ rpl_parallel_thread_pool::rpl_parallel_thread_pool() > int > rpl_parallel_thread_pool::init(uint32 size) > { > - count= 0; > threads= NULL; > free_list= NULL; > + count= 0; > + busy= false; > > mysql_mutex_init(key_LOCK_rpl_thread_pool, &LOCK_rpl_thread_pool, > MY_MUTEX_INIT_SLOW); > @@ -1496,8 +1706,14 @@ rpl_parallel_thread_pool::get_thread(rpl_parallel_thread **owner, > rpl_parallel_thread *rpt; > > mysql_mutex_lock(&LOCK_rpl_thread_pool); > - while ((rpt= free_list) == NULL) > + for (;;) > + { > + while (unlikely(busy)) > + mysql_cond_wait(&COND_rpl_thread_pool, &LOCK_rpl_thread_pool); > + if ((rpt= free_list) != NULL) > + break; > mysql_cond_wait(&COND_rpl_thread_pool, &LOCK_rpl_thread_pool); > + } > free_list= rpt->next; > mysql_mutex_unlock(&LOCK_rpl_thread_pool); > mysql_mutex_lock(&rpt->LOCK_rpl_thread); > @@ -1908,7 +2124,7 @@ rpl_parallel::wait_for_workers_idle(THD *thd) > > e= (struct rpl_parallel_entry *)my_hash_element(&domain_hash, i); > mysql_mutex_lock(&e->LOCK_parallel_entry); > - e->need_sub_id_signal= true; > + ++e->need_sub_id_signal; > thd->ENTER_COND(&e->COND_parallel_entry, &e->LOCK_parallel_entry, > &stage_waiting_for_workers_idle, &old_stage); > while (e->current_sub_id > e->last_committed_sub_id) > @@ -1921,7 +2137,7 @@ rpl_parallel::wait_for_workers_idle(THD *thd) > } > mysql_cond_wait(&e->COND_parallel_entry, &e->LOCK_parallel_entry); > } > - e->need_sub_id_signal= false; > + --e->need_sub_id_signal; > thd->EXIT_COND(&old_stage); > if (err) > return err; > diff --git a/sql/rpl_parallel.h b/sql/rpl_parallel.h > index 04f8706..77a4005 100644 > --- a/sql/rpl_parallel.h > +++ b/sql/rpl_parallel.h > @@ -199,12 +199,18 @@ struct rpl_parallel_thread { > > > struct rpl_parallel_thread_pool { > - uint32 count; > struct rpl_parallel_thread **threads; > struct rpl_parallel_thread *free_list; > mysql_mutex_t LOCK_rpl_thread_pool; > mysql_cond_t COND_rpl_thread_pool; > + uint32 count; > bool inited; > + /* > + While FTWRL runs, this counter is incremented to make SQL thread or > + STOP/START slave not try to start new activity while that operation > + is in progress. > + */ > + bool busy; > > rpl_parallel_thread_pool(); > int init(uint32 size); > @@ -219,6 +225,12 @@ struct rpl_parallel_entry { > mysql_mutex_t LOCK_parallel_entry; > mysql_cond_t COND_parallel_entry; > uint32 domain_id; > + /* > + Incremented by wait_for_workers_idle() and rpl_pause_for_ftwrl() to show > + that they are waiting, so that finish_event_group knows to signal them > + when last_committed_sub_id is increased. > + */ > + uint32 need_sub_id_signal; > uint64 last_commit_id; > bool active; > /* > @@ -228,12 +240,6 @@ struct rpl_parallel_entry { > */ > bool force_abort; > /* > - Set in wait_for_workers_idle() to show that it is waiting, so that > - finish_event_group knows to signal it when last_committed_sub_id is > - increased. > - */ > - bool need_sub_id_signal; > - /* > At STOP SLAVE (force_abort=true), we do not want to process all events in > the queue (which could unnecessarily delay stop, if a lot of events happen > to be queued). The stop_count provides a safe point at which to stop, so > @@ -291,6 +297,11 @@ struct rpl_parallel_entry { > The value is ULONGLONG_MAX when no error occured. > */ > uint64 stop_on_error_sub_id; > + /* > + During FLUSH TABLES WITH READ LOCK, transactions with sub_id larger than > + this value must not start, but wait until the global read lock is released. > + */ > + uint64 pause_sub_id; > /* Total count of event groups queued so far. */ > uint64 count_queued_event_groups; > /* > @@ -331,5 +342,7 @@ extern struct rpl_parallel_thread_pool global_rpl_thread_pool; > extern int rpl_parallel_activate_pool(rpl_parallel_thread_pool *pool); > extern int rpl_parallel_inactivate_pool(rpl_parallel_thread_pool *pool); > extern bool process_gtid_for_restart_pos(Relay_log_info *rli, rpl_gtid *gtid); > +extern int rpl_pause_for_ftwrl(THD *thd); > +extern void rpl_unpause_for_ftwrl(THD *thd); > > #endif /* RPL_PARALLEL_H */ > diff --git a/sql/sql_parse.cc b/sql/sql_parse.cc > index 5635e9a..ca6b11e 100644 > --- a/sql/sql_parse.cc > +++ b/sql/sql_parse.cc > @@ -4271,6 +4271,17 @@ case SQLCOM_PREPARE: > break; > } > > + if (lex->type & REFRESH_READ_LOCK) > + { > + /* > + We need to pause any parallel replication slave workers during FLUSH > + TABLES WITH READ LOCK. Otherwise we might cause a deadlock, as > + worker threads eun run in arbitrary order but need to commit in a > + specific given order. > + */ > + if (rpl_pause_for_ftwrl(thd)) > + goto error; > + } > /* > reload_acl_and_cache() will tell us if we are allowed to write to the > binlog or not. > @@ -4301,6 +4312,8 @@ case SQLCOM_PREPARE: > if (!res) > my_ok(thd); > } > + if (lex->type & REFRESH_READ_LOCK) > + rpl_unpause_for_ftwrl(thd); > > break; > } > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits

1 0

Re: [Maria-developers] b8e26b0: MDEV-363 - Server crashes in intern_plugin_lock on concurrent installing
by Sergei Golubchik 09 Jun '15

09 Jun '15

Hi, Sergey! On Jun 09, svoj(a)mariadb.org wrote: > revision-id: b8e26b04d4ac22e29a78bca68e101176b6ef507a > parent(s): 08b2e1ae73f6fe055391bf8437b6dff1a889c49c > committer: Sergey Vojtovich > branch nick: mariadb > timestamp: 2015-06-09 13:50:43 +0400 > message: > > MDEV-363 - Server crashes in intern_plugin_lock on concurrent installing > semisync plugin and setting rpl_semi_sync_master_enabled > > There was race condition between INSTALL PLUGIN and SET. It was caused by a > gap in INSTALL PLUGIN when plugin variables were registered but not fully > initialized. Accessing such variables concurrently may reference uninitialized > memory, specifically sys_var_pluginvar::plugin. > > Fixed by initializing sys_var_pluginvar::plugin early, before variable is > registered. ok to push! Regards, Sergei

1 0

[Maria-developers] MDEV-7268 Column of table cannot be converted from type 'decimal(0, ?)' to type ' 'decimal(10, 7)'
by Alexander Barkov 09 Jun '15

09 Jun '15

Hi Sergei, Please review a patch for mdev-7268. Thanks.

2 1