- developers - lists.mariadb.org

Re: [Maria-developers] [Commits] b60fb6d: MDEV-11636 Extra persistent columns on slave always gets NULL in RBR
by Kristian Nielsen 22 Dec '16

22 Dec '16

sachin.setiya(a)mariadb.com writes: > revision-id: b60fb6daee2fcc2f433a50b5a5639065f5a46fe8 (mariadb-galera-10.0.28-8-gb60fb6d) > parent(s): be430b80df0cdd4eba32df1570195721dbfd1b39 > author: Sachin Setiya > committer: Sachin Setiya > timestamp: 2016-12-22 19:22:36 +0530 > message: > > MDEV-11636 Extra persistent columns on slave always gets NULL in RBR > > Problem:- In replication if slave has extra persistent column then these > column are not computed while applying write-set from master. Generally, when slave and master differs there will be situations where things will not fully work. However, in this case, the result is really corrupt data (a persistent column with wrong values), and I agree this should be fixed (and the patch looks simple). I am not familiar with the code for virtual columns. I assume you looked at the normal code path for virtual columns and made similar functionality for row-based replication applier. Your code looks reasonable to me. You add code in Rows_log_event::write_row(). What about update_row? It is not tested in the testcase, you should add test of the UPDATE case (and fix the code, if needed). The testcase needs more work, see comments inline. Getting replication tests to work reliably (not generate occasional spurious errors on buildbot etc.) is quite tricky (spoken from painful experience :-). Please push this to a feature tree and make sure the test passes on all builders before pushing to main. (A useful trick also is to run on your own machine something like: ./mysql-test-run.pl rpl.rpl_alter_extra_persistent --repeat=1000 to catch at least some timing-dependent errors). > diff --git a/mysql-test/suite/rpl/t/rpl_alter_extra_persistent.test b/mysql-test/suite/rpl/t/rpl_alter_extra_persistent.test > +--sync_slave_with_master > + > +connection slave; > +select * from t1; Use ORDER BY a (here and below), to avoid any risk of getting different output order. > +--connection master > +insert into t1 values(5); > +insert into t1 values(6); > + > +--connection slave > +sleep 2; Never use sleep in test cases. Use an appropriate mechanism to sync the slave to the master. > +select * from t1; > + > +--echo #Break Unique Constraint > +alter table t1 add column z4 int as (a % 6) persistent unique; > + > +--connection master > + > +--echo #entering duplicate value for slave persistent column > +insert into t1 values(7); > +select * from t1; > + > +--connection slave > +select * from t1; > +--query_vertical show slave status; This will not work reliably. You cannot put SHOW SLAVE STATUS output in .result, it will not be stable between test runs. And there is a race on whether the slave has time to hit the duplicate error. For this, use the existing include/wait_for_slave_sql_error.inc file, which can reliably detect a slave failure. See comments in the file and example usage in other test files. > + > +--connection master > +drop table t1; > + > +--connection slave > +alter table t1 drop column z4 ; > +start slave; > +drop table t1; This drop table should not be there (the drop table from the master will be replicated). > diff --git a/sql/log_event.cc b/sql/log_event.cc > index 1cc58cd..a8f035b 100644 > --- a/sql/log_event.cc > +++ b/sql/log_event.cc > @@ -11541,6 +11541,9 @@ Rows_log_event::write_row(rpl_group_info *rgi, > DBUG_RETURN(error); > } > > + if ((error= fill_extra_persistent_columns(table, &m_cols)) > + DBUG_RETURN(error); > + Is similar code needed for update_row? At least, this should be tested in the test case. - Kristian.

1 0

[Maria-developers] aria engine crash at recovery
by Roberto Spadim 22 Dec '16

22 Dec '16

hi guys i got a crash and aria engine don't startup, any help? 161222 03:41:13 mysqld_safe Starting mysqld daemon with databases from /home/mysql/data/local/ 2016-12-22 3:41:13 139878188566400 [Warning] 'THREAD_CONCURRENCY' is deprecated and will be removed in a future release. 2016-12-22 3:41:13 139878188566400 [Note] /usr/mariadb/mariadb/bin//mysqld (mysqld 10.1.5-MariaDB-log) starting as process 19217 ... 2016-12-22 3:41:13 139878188566400 [Note] mysqld: Aria engine: starting recovery recovered pages: 0% 3082852397% 0%161222 3:41:13 [ERROR] mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. To report this bug, see http://kb.askmonty.org/en/reporting-bugs We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. Server version: 10.1.5-MariaDB-log key_buffer_size=629145600 read_buffer_size=4194304 max_used_connections=0 max_threads=52 thread_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1041422 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x0x0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x0 thread_stack 0x48000 mysys/stacktrace.c:247(my_print_stacktrace)[0x7f37ee645b0b] sql/signal_handler.cc:153(handle_fatal_signal)[0x7f37ee1dfdb5] /lib64/libpthread.so.0(+0x10d00)[0x7f37ecd72d00] maria/ma_recovery.c:2142(exec_REDO_LOGREC_CLR_END)[0x7f37ee5683b7] maria/ma_recovery.c:588(display_and_apply_record)[0x7f37ee56985d] maria/ma_recovery.c:350(maria_apply_log)[0x7f37ee56d45c] maria/ma_recovery.c:242(maria_recovery_from_log)[0x7f37ee56df1e] maria/ha_maria.cc:3594(ha_maria_init)[0x7f37ee546c2d] sql/handler.cc:512(ha_initialize_handlerton(st_plugin_int*))[0x7f37ee1e2518] sql/sql_plugin.cc:1403(plugin_initialize)[0x7f37ee089e65] sql/sql_plugin.cc:1675(plugin_init(int*, char**, int))[0x7f37ee08add2] sql/mysqld.cc:5064(init_server_components)[0x7f37edfc37e8] sql/mysqld.cc:5659(mysqld_main(int, char**))[0x7f37edfc61ca] /lib64/libc.so.6(__libc_start_main+0xf0)[0x7f37ec3a79f0] /usr/mariadb/mariadb/bin//mysqld(+0x39dbad)[0x7f37edfbabad]

1 4

[Maria-developers] How correctly allocate memory to avoid problems with 2nd execution (instruction)
by Oleksandr Byelkin 22 Dec '16

22 Dec '16

Hi! It is for all who decide to write or already is writing something for server. There is 2 ways to execute query/statement: 1) conventional (one time) 2) with storing the statement and executing it many times (Prepared statements (PS)/ Stored Procedures(SP)) Also there is so called Query_arena, which is a MEM_ROOT and list of allocated Item-s in it. (we need list of items because they could require not trivial destruction/cleanup procedure before freeing memory, see cleanup() methods of Item-s) For conventional execution there is only one (run-time) query arena (part of THD in this case) where all memory allocated, and all Item-s linked. After execution Item-s cleaned up and memory freed. For case with stored statements (PS/SP) there is one Query_arena where Parsing result allocated (statement arena), and the other one so called run-time arena (which freed after execution as for conventional execution). During parsing statement arena is active (default), during all execution steps the run-time arena is active (if nothing changed it specially). So if some one decided allocate some memory during execution one should think if it is: I) only for execution II) should live with the statement All references on the memory allocated as case I) should be returned in the state as it was before execution or taken other measures to prevent using them during the next execution. All object which allocated as persistent (for statement of course) should be a) allocated only once (can be first execution or prepare phase, it does not matter, what does matter: do not make memory leak by allocating on each execution) b) reference on them should not be lost (because it supposed to be reused during next execution) c) procedure of allocation during run-time should looks like this: Query_arena *arena, backup; // backup run-time arena and set as default statement arena if needed // (if it is not conventional execution) arena= thd->activate_stmt_arena_if_needed(&backup);\ <MAKE ALLOOCATION HERE USING thd->mem_root> // return old default arena if it was changed before if (arena) thd->restore_active_arena(arena, &backup);

1 0

[Maria-developers] Fwd: Update of RPM layout
by Michal Schorm 21 Dec '16

21 Dec '16

Hello all, I went through an idea [1] to lighten the server package. I tested it, and I found out, I can save about 40MB of space, if I'd move all Perl utilites to a standalone subpackage. It would make sense to move them, so the users don't need to install Perl with the server package. Especially in technologies like containers (or cloud images) size matters. It all depends on how often are those utilities [2] used together with the server, sice we don't want to displace importatnt scripts. In case, it would look like a good idea, there is also option to move to this subpackage other - infrequently used - utilities. -> Q: Hey, Users and Admins, which utilities do you think are essential for the server? Note: It can be solved by soft dependencies, so those utils would be installed with the server by default. [1] https://bugzilla.redhat.com/show_bug.cgi?id=1205082 [2] mysql_find_rows mysqlaccess mysql_convert_table_format mysql_fix_extensions mysql_setpermission mysql_zap mysqldumpslow mysqld_multi mysqlhotcopy -- Michal Schorm Core Services - Databases Team mail: mschorm(a)redhat.com Brno-IRC: mschorm

1 0

[Maria-developers] Please review MDEV-10386 Assertion `fixed == 1' failed in virtual String* Item_func_conv_charset::val_str(String*)
by Alexander Barkov 21 Dec '16

21 Dec '16

Hello Sergei, Please review a patch for MDEV-10386. Thanks!

2 3

[Maria-developers] mysql_secure_installation documentation
by Michal Schorm 20 Dec '16

20 Dec '16

Hello all, I took care about bug against "mysql_secure_installation" script. [1] The problem is pretty simple. When you use "mysql_secure_installation" script, it will ingore all of yours current configuration. However, there are some options, that can be used. One of them is used to add your custom config file. It is documented, for example, here: [2] Unfortunatelly, the "man mysql_secure_installation" is quite short of infromation. (same as [3]) I believe, the information about options really should be added to both manpage and [3]. That should also solve the bug, without any [1] https://bugzilla.redhat.com/show_bug.cgi?id=1260821 [2] https://dev.mysql.com/doc/refman/5.7/en/mysql-secure-installation.html [3] https://mariadb.com/kb/en/mariadb/mysql_secure_installation/ -- I would like your general opinion about this. Based on it, I can file issue on JIRA. -- Michal Schorm Core Services - Databases Team mail: mschorm(a)redhat.com Brno-IRC: mschorm

1 0

[Maria-developers] Please review MDEV-11615 Split Item_hybrid_func::fix_attributes into virtual methods in Type_handler
by Alexander Barkov 20 Dec '16

20 Dec '16

Hello Sanja, Please review a patch for MDEV-11615. Thanks!

1 0

[Maria-developers] 10.3 branches
by Alexander Barkov 20 Dec '16

20 Dec '16

Hello Serg, all, A few weeks ago I created a new branch "10.3" and pushed essential patches needed for the compatibility project. It worked as follows: - In 10.3: git rebase origin/10.2 git push --force - In bb-10.2-compatibility: git rebase origin/10.3 git push --force Now as other developers start to push into 10.3, this won't work anymore. We briefly discussed with Monty that, as the compatibility project should be as stable as possible and merging all 10.3 patches into bb-10.2-compatibility is not desirable, we could try something like this: Add a new branch (Monty proposed "10.3-base" as the name) and propagate changes as follows: - In 10.3-base: git rebase origin/10.2 git push --force - In bb-10.2-compatibility git rebase origin/10.3-base git push --force - In 10.3: git merge origin/10.3-base git push (no --force) 10.2 ---(rebase)--->10.3-base--(rebase)-->bb-10.2-compatibility | | (merge) (merge) | | V | 10.3<-----------------/ Any objections about this scheme? I'd only propose to use "10.2-ext" instead of "10.3-base", as this intermediate branch is going to be more 10.2 than 10.3. Thanks.

3 6

Re: [Maria-developers] [Commits] 9f863a15b03: MDEV-11602 InnoDB leaks foreign key metadata on DDL operations
by Jan Lindström 19 Dec '16

19 Dec '16

ok to push. Most likely the double free was result of some other bug fixed later. R: Jan On Mon, Dec 19, 2016 at 4:00 PM, <marko.makela(a)mariadb.com> wrote: > revision-id: 9f863a15b037c0ddffc2d12c1f7d841466f9f345 (mariadb-10.0.28-27- > g9f863a15b03) > parent(s): eb4f2e063c341d9f3644339c68cb01679e782001 > author: Marko Mäkelä > committer: Marko Mäkelä > timestamp: 2016-12-19 15:57:41 +0200 > message: > > MDEV-11602 InnoDB leaks foreign key metadata on DDL operations > > Essentially revert MDEV-6759, which addressed a double free of memory > by removing the freeing altogether, introducing the memory leaks. > No double free was observed when running the test suite -DWITH_ASAN. > > Replace some mem_heap_free(foreign->heap) with dict_foreign_free(foreign) > so that the calls can be located and instrumented more easily when needed. > > --- > storage/innobase/dict/dict0dict.cc | 8 ++++---- > storage/innobase/dict/dict0load.cc | 2 +- > storage/xtradb/dict/dict0dict.cc | 8 ++++---- > storage/xtradb/dict/dict0load.cc | 2 +- > 4 files changed, 10 insertions(+), 10 deletions(-) > > diff --git a/storage/innobase/dict/dict0dict.cc b/storage/innobase/dict/ > dict0dict.cc > index f75d584e70d..2e40946224e 100644 > --- a/storage/innobase/dict/dict0dict.cc > +++ b/storage/innobase/dict/dict0dict.cc > @@ -1614,6 +1614,7 @@ struct dict_foreign_remove_partial > if (table != NULL) { > table->referenced_set.erase(foreign); > } > + dict_foreign_free(foreign); > } > }; > > @@ -3535,8 +3536,7 @@ dict_foreign_add_to_cache( > } > > if (for_in_cache) { > - /* Free the foreign object */ > - mem_heap_free(foreign->heap); > + dict_foreign_free(foreign); > } else { > for_in_cache = foreign; > } > @@ -3564,7 +3564,7 @@ dict_foreign_add_to_cache( > " the ones in table."); > > if (for_in_cache == foreign) { > - mem_heap_free(foreign->heap); > + dict_foreign_free(foreign); > } > > return(DB_CANNOT_ADD_CONSTRAINT); > @@ -3620,7 +3620,7 @@ dict_foreign_add_to_cache( > be one */ > } > > - mem_heap_free(foreign->heap); > + dict_foreign_free(foreign); > } > > return(DB_CANNOT_ADD_CONSTRAINT); > diff --git a/storage/innobase/dict/dict0load.cc b/storage/innobase/dict/ > dict0load.cc > index e68bbe7d10a..77a31e5de63 100644 > --- a/storage/innobase/dict/dict0load.cc > +++ b/storage/innobase/dict/dict0load.cc > @@ -489,7 +489,7 @@ dict_process_sys_foreign_rec( > } > > /* This recieves a dict_foreign_t* that points to a stack variable. > - So mem_heap_free(foreign->heap) is not used as elsewhere. > + So dict_foreign_free(foreign) is not used as elsewhere. > Since the heap used here is freed elsewhere, foreign->heap > is not assigned. */ > foreign->id = mem_heap_strdupl(heap, (const char*) field, len); > diff --git a/storage/xtradb/dict/dict0dict.cc b/storage/xtradb/dict/ > dict0dict.cc > index 5339ecfaed6..3845138c8bf 100644 > --- a/storage/xtradb/dict/dict0dict.cc > +++ b/storage/xtradb/dict/dict0dict.cc > @@ -1614,6 +1614,7 @@ struct dict_foreign_remove_partial > if (table != NULL) { > table->referenced_set.erase(foreign); > } > + dict_foreign_free(foreign); > } > }; > > @@ -3539,8 +3540,7 @@ dict_foreign_add_to_cache( > } > > if (for_in_cache) { > - /* Free the foreign object */ > - mem_heap_free(foreign->heap); > + dict_foreign_free(foreign); > } else { > for_in_cache = foreign; > } > @@ -3564,7 +3564,7 @@ dict_foreign_add_to_cache( > " the ones in table."); > > if (for_in_cache == foreign) { > - mem_heap_free(foreign->heap); > + dict_foreign_free(foreign); > } > > return(DB_CANNOT_ADD_CONSTRAINT); > @@ -3620,7 +3620,7 @@ dict_foreign_add_to_cache( > be one */ > } > > - mem_heap_free(foreign->heap); > + dict_foreign_free(foreign); > } > > return(DB_CANNOT_ADD_CONSTRAINT); > diff --git a/storage/xtradb/dict/dict0load.cc b/storage/xtradb/dict/ > dict0load.cc > index ca7de72c9b9..9ea4f4d873a 100644 > --- a/storage/xtradb/dict/dict0load.cc > +++ b/storage/xtradb/dict/dict0load.cc > @@ -491,7 +491,7 @@ dict_process_sys_foreign_rec( > } > > /* This recieves a dict_foreign_t* that points to a stack variable. > - So mem_heap_free(foreign->heap) is not used as elsewhere. > + So dict_foreign_free(foreign) is not used as elsewhere. > Since the heap used here is freed elsewhere, foreign->heap > is not assigned. */ > foreign->id = mem_heap_strdupl(heap, (const char*) field, len); > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits

1 0

Re: [Maria-developers] [Commits] 8bfc350: MDEV-10148: Database crashes in the query to the View
by Oleksandr Byelkin 19 Dec '16

19 Dec '16

Hi, Sergei! On 12/16/2016 03:16 PM, Sergei Petrunia wrote: > revision-id: 8bfc35041195b46c8f4666dd02f600e878f45643 > parent(s): 96d2e0480ffbc36c803db5a7d5bcefd3470c5f72 > committer: Sergei Petrunia > branch nick: 5.5 > timestamp: 2016-12-16 17:16:02 +0300 > message: > > MDEV-10148: Database crashes in the query to the View > > Fix st_select_lex::is_merged_child_of to work across merged views or > derived tables. > OK to push! [skip]

1 0