[Maria-developers] Preparing MDEV-4506, Parallel replication, for 10.0.5 release
Hi Monty, I was asked to prepare MDEV-4506, parallel replication, for release of 10.0.5 on Monday. For this, I did merge to 10.0-base and then to 10.0. I have pushed the results here: lp:~maria-captains/maria/mdev-4506-base # for 10.0-base merge lp:~maria-captains/maria/mdev-4506 # for 10.0 merge However, I had problems with a couple of your changes, to temporary tables and innodb log file size. The code that you modified in 10.0-base does not seem to exist in 10.0 (or at least I couldn't find it easily). Can you take a look at this and see if you can complete the merge? The missing bits are just the few hunks appended at the end of this mail, so I assume it should be easy for someone who knows what the patches are supposed to do. The tree lp:~maria-captains/maria/mdev-4506 currently has a lot of test failures, but it looks like most of them will be fixed by completing the last bits of the merge. - Kristian. ----------------------------------------------------------------------- sql/sql_base.cc: <<<<<<< TREE if (table_list->mdl_request.type >= MDL_SHARED_WRITE && thd->tx_read_only && !(flags & (MYSQL_LOCK_LOG_TABLE | MYSQL_OPEN_HAS_MDL_LOCK))) ||||||| BASE-REVISION if (table_list->open_type != OT_BASE_ONLY && ! (flags & MYSQL_OPEN_SKIP_TEMPORARY)) ======= if (table_list->open_type != OT_BASE_ONLY && ! (flags & MYSQL_OPEN_SKIP_TEMPORARY) && thd->have_temporary_tables())
> MERGE-SOURCE { <<<<<<< TREE my_error(ER_CANT_EXECUTE_IN_READ_ONLY_TRANSACTION, MYF(0)); DBUG_RETURN(true); ||||||| BASE-REVISION for (table= thd->temporary_tables; table ; table=table->next) { if (table->s->table_cache_key.length == key_length + TMP_TABLE_KEY_EXTRA && !memcmp(table->s->table_cache_key.str, key, key_length + TMP_TABLE_KEY_EXTRA)) { /* We're trying to use the same temporary table twice in a query. Right now we don't support this because a temporary table is always represented by only one TABLE object in THD, and it can not be cloned. Emit an error for an unsupported behaviour. */ if (table->query_id) { DBUG_PRINT("error", ("query_id: %lu server_id: %u pseudo_thread_id: %lu", (ulong) table->query_id, (uint) thd->variables.server_id, (ulong) thd->variables.pseudo_thread_id)); my_error(ER_CANT_REOPEN_TABLE, MYF(0), table->alias.c_ptr()); DBUG_RETURN(TRUE); } table->query_id= thd->query_id; thd->thread_specific_used= TRUE; DBUG_PRINT("info",("Using temporary table")); goto reset; } } ======= if ((table= find_temporary_table(thd, key, key_length + TMP_TABLE_KEY_EXTRA))) { /* Check if we're trying to use the same temporary table twice in a query. Right now we don't support this because a temporary table is always represented by only one TABLE object in THD, and it can not be cloned. Emit an error for an unsupported behaviour. */ if (table->query_id) { DBUG_PRINT("error", ("query_id: %lu server_id: %u pseudo_thread_id: %lu", (ulong) table->query_id, (uint) thd->variables.server_id, (ulong) thd->variables.pseudo_thread_id)); my_error(ER_CANT_REOPEN_TABLE, MYF(0), table->alias.c_ptr()); DBUG_RETURN(TRUE); } table->query_id= thd->query_id; thd->thread_specific_used= TRUE; DBUG_PRINT("info",("Using temporary table")); goto reset; } > MERGE-SOURCE
sql/slave.cc: <<<<<<< TREE static void set_thd_in_use_temporary_tables(Relay_log_info *rli) { TABLE *table; for (table= rli->save_temporary_tables ; table ; table= table->next) { table->in_use= rli->sql_thd; if (table->file != NULL) { /* Since we are stealing opened temporary tables from one thread to another, we need to let the performance schema know that, for aggregates per thread to work properly. */ table->file->unbind_psi(); table->file->rebind_psi(); } } } ||||||| BASE-REVISION static void set_thd_in_use_temporary_tables(Relay_log_info *rli) { TABLE *table; for (table= rli->save_temporary_tables ; table ; table= table->next) table->in_use= rli->sql_thd; } =======
> MERGE-SOURCE
=== modified file 'storage/innobase/srv/srv0start.c' --- storage/innobase/srv/srv0start.c 2013-09-06 20:31:30 +0000 +++ storage/innobase/srv/srv0start.c 2013-11-01 08:17:06 +0000 @@ -621,7 +621,7 @@ open_or_create_log_file( || size_high != srv_calc_high32(srv_log_file_size)) { fprintf(stderr, - "InnoDB: Error: log file %s is" + "InnoDB: Warning: log file %s is" " of different size %lu %lu bytes\n" "InnoDB: than specified in the .cnf" " file %lu %lu bytes!\n", @@ -629,7 +629,8 @@ open_or_create_log_file( (ulong) srv_calc_high32(srv_log_file_size), (ulong) srv_calc_low32(srv_log_file_size)); - return(DB_ERROR); + srv_log_file_size= size + + (((longlong) size_high) << 32); } } else { *log_file_created = TRUE;
Hi!
"Kristian" == Kristian Nielsen <knielsen@knielsen-hq.org> writes:
Kristian> Hi Monty, Kristian> I was asked to prepare MDEV-4506, parallel replication, for release of 10.0.5 Kristian> on Monday. Kristian> For this, I did merge to 10.0-base and then to 10.0. I have pushed the results Kristian> here: Kristian> lp:~maria-captains/maria/mdev-4506-base # for 10.0-base merge Kristian> lp:~maria-captains/maria/mdev-4506 # for 10.0 merge Kristian> However, I had problems with a couple of your changes, to temporary tables and Kristian> innodb log file size. The code that you modified in 10.0-base does not seem to Kristian> exist in 10.0 (or at least I couldn't find it easily). Kristian> Can you take a look at this and see if you can complete the merge? The missing Kristian> bits are just the few hunks appended at the end of this mail, so I assume it Kristian> should be easy for someone who knows what the patches are supposed to do. IKristian> The tree lp:~maria-captains/maria/mdev-4506 currently has a lot of test Kristian> failures, but it looks like most of them will be fixed by completing the last Kristian> bits of the merge. ok. What kind of errors ? (I assume I will notice these when I start testing this). I will fix mdev-4506 and push this into mdev-4506 and in 10.0 if I can get things to run without errors. I assume we can then push mdev-4506-base to 10.0-base. Kristian> sql/sql_base.cc: Kristian> <<<<<<< TREE Kristian> if (table_list->mdl_request.type >= MDL_SHARED_WRITE && Kristian> thd->tx_read_only && Kristian> !(flags & (MYSQL_LOCK_LOG_TABLE | MYSQL_OPEN_HAS_MDL_LOCK))) Kristian> ||||||| BASE-REVISION Kristian> if (table_list->open_type != OT_BASE_ONLY && Kristian> ! (flags & MYSQL_OPEN_SKIP_TEMPORARY)) Kristian> ======= Kristian> if (table_list->open_type != OT_BASE_ONLY && Kristian> ! (flags & MYSQL_OPEN_SKIP_TEMPORARY) && thd->have_temporary_tables()) The reason for the merge failure is that in 5.6 they now open temporary tables in a total manner and in a different place than before. Fixed. (This code was now in open_temporary_table()). Kristian> sql/slave.cc: Kristian> <<<<<<< TREE Kristian> static void set_thd_in_use_temporary_tables(Relay_log_info *rli) Kristian> { Kristian> TABLE *table; Kristian> for (table= rli->save_temporary_tables ; table ; table= table->next) Kristian> { Kristian> table->in_use= rli->sql_thd; Kristian> if (table->file != NULL) Kristian> { Kristian> /* Kristian> Since we are stealing opened temporary tables from one thread to another, Kristian> we need to let the performance schema know that, Kristian> for aggregates per thread to work properly. Kristian> */ Kristian> table->file->unbind_psi(); Kristian> table->file->rebind_psi(); Kristian> } Kristian> } Kristian> } Kristian> ||||||| BASE-REVISION Kristian> static void set_thd_in_use_temporary_tables(Relay_log_info *rli) Kristian> { Kristian> TABLE *table; Kristian> for (table= rli->save_temporary_tables ; table ; table= table->next) Kristian> table->in_use= rli->sql_thd; Kristian> } Kristian> ======= We don't need the function set_thd_in_use_for_temporary_tables() anymore, so there was nothing to merge. Kristian> === modified file 'storage/innobase/srv/srv0start.c' Kristian> --- storage/innobase/srv/srv0start.c 2013-09-06 20:31:30 +0000 Kristian> +++ storage/innobase/srv/srv0start.c 2013-11-01 08:17:06 +0000 Kristian> @@ -621,7 +621,7 @@ open_or_create_log_file( Kristian> || size_high != srv_calc_high32(srv_log_file_size)) { Kristian> fprintf(stderr, Kristian> - "InnoDB: Error: log file %s is" Kristian> + "InnoDB: Warning: log file %s is" Kristian> " of different size %lu %lu bytes\n" Kristian> "InnoDB: than specified in the .cnf" Kristian> " file %lu %lu bytes!\n", Kristian> @@ -629,7 +629,8 @@ open_or_create_log_file( Kristian> (ulong) srv_calc_high32(srv_log_file_size), Kristian> (ulong) srv_calc_low32(srv_log_file_size)); Kristian> - return(DB_ERROR); Kristian> + srv_log_file_size= size + Kristian> + (((longlong) size_high) << 32); The last patch was not needed anymore. Will now start building and testing my changes. Will push to mdev-4506 before going to bed. The main thing to fix was to change sql_base.cc::find_temporary_table(THD *thd, const TABLE_LIST *tl) to use the same find_temporary_table() function as the rest of the code. Regards, Monty
Hi!
"Kristian" == Kristian Nielsen <knielsen@knielsen-hq.org> writes:
Kristian> Hi Monty, Kristian> I was asked to prepare MDEV-4506, parallel replication, for release of 10.0.5 Kristian> on Monday. Kristian> For this, I did merge to 10.0-base and then to 10.0. I have pushed the results Kristian> here: Kristian> lp:~maria-captains/maria/mdev-4506-base # for 10.0-base merge I had planned to merge this into the normal 10.0-base tree, but run into a problem:
cd /my/maria-10.0-base/ bzr merge /my/mdev-4506-base Nothing to do.
I checked in the mdev-4506-base sql directory, but there was no rpl_parallel.cc file. Did you forget to push the latest changes into mdev-4506-base ? If all tests passes in your 10.0-base, please push the changes into 10.0-base tree. I have now one failing test in suite/rpl in mdev-4506 that I am trying to fix before I will push. (rpl.rpl_deadlock_innodb) Regards, Monty
Michael Widenius <monty@askmonty.org> writes:
Did you forget to push the latest changes into mdev-4506-base ?
Yes, sorry about that.
If all tests passes in your 10.0-base, please push the changes into 10.0-base tree.
Done.
I have now one failing test in suite/rpl in mdev-4506 that I am trying to fix before I will push. (rpl.rpl_deadlock_innodb)
Ok, let me know if you need any help. For now, I will see if I can fix MDEV-5217. - Kristian.
participants (2)
-
Kristian Nielsen -
Michael Widenius