Hi, I repeated the exact problem. Firstly, you need to have this branch: jan@jan-GE70-0NC-0ND ~/mysql/mariadb-patches $ bzr info Standalone tree (format: 2a) Location: branch root: . Related branches: parent branch: bzr+ssh://bazaar.launchpad.net/~elenst/randgen/mariadb-patches/ Inside that directory I run following command : perl ./runall-new.pl --grammar=conf/engines/engine_stress.yy --gendata=conf/engines/engine_stress.zz --duration=600 --queries=100M --threads=4 --galera=mmm --basedir=/home/jan/mysql/galera-10.0 --vardir=/home/jan/mysql/galera-test --mysqld=--wsrep-provider=/usr/lib/libgalera_smm.so --debug --mysqld=--general-log=1 --mysqld=--general_log_file=/home/jan/mysql/galera-test/stmt.log --logfile=/home/jan/mysql/galera-test/rpg.log --valgrind Correct log file attached.
Yes, I did run using valgrind, while it reported some uninitialized variables, not really any memory corruptions. Attached, the log from node0. Similarly, Well, uninitialised variables are exactly what I would expect could lead to things looking like you have shown so far. So if you have Valgrind errors, you should definitely start by fixing them.
However, I see no messages from Valgrind in the log you attached?!?
So please, repeat the exact problem you get (assertion during rollback) while running with Valgrind. Provide the exact command you used to get the error in this particular run, and the correct log that shows the Valgrind errors about uninitialised variables, and I can give more suggestions for how to fix.
debug=+d did not help. Where I should add some output to narrow down the place where the corruption happens.
It did produce the output, not yet found the reason, some of the output just before corruption MYSQL_BIN_LOG::flush_and_set_pending_rows_event(event): enter: event: 0x7f2a200bc800 MYSQL_BIN_LOG::flush_and_set_pending_rows_event(event): info: cache_mngr->pending(): 0x0 Rows_log_event::do_add_row_data: enter: row_data: 0x7f2a200cb248 length: 10 Rows_log_event::do_add_row_data: row_data: Memory: 0x7f2a200cb248 Bytes: (10) F8 01 00 00 00 01 74 02 77 65 my_realloc: my: ptr: 0x0 size: 1024 my_flags: 80 my_malloc: my: size: 1024 my_flags: 80 my_malloc: exit: ptr: 0x7f2a200895e0 THD::binlog_query: enter: qtype: ROW query: 'INSERT /*! IGNORE */ INTO table1_int_autoinc VALUES (NULL, 't', 'we', NULL, NULL)' MYSQL_BIN_LOG::flush_and_set_pending_rows_event(event): enter: event: 0x0 MYSQL_BIN_LOG::flush_and_set_pending_rows_event(event): info: cache_mngr->pending(): 0x7f2a200bc800 Log_event::write_header: enter: filepos: 65 length: 20 type: 23 MYSQL_BIN_LOG::flush_and_set_pending_rows_event(event): m_width: Memory: 0x7f2a882e0190 Bytes: (1) 05 MYSQL_BIN_LOG::flush_and_set_pending_rows_event(event): m_cols: Memory: 0x7f2a200bc8c8 Bytes: (1) FF MYSQL_BIN_LOG::flush_and_set_pending_rows_event(event): rows: Memory: 0x7f2a200895e0 Bytes: (10) F8 01 00 00 00 01 74 02 77 65 my_free: my: ptr: 0x7f2a200895e0 sf_malloc_usable_size: exit: size: 1024 flags: 80 my_free: my: ptr: 0x7f2a200bc800 sf_malloc_usable_size: exit: size: 304 flags: 24 THD::binlog_query: debug: is_current_stmt_binlog_format_row: 1 mysql_insert: THD::enter_stage: /home/jan/mysql/galera-10.0/sql/sql_insert.cc:1078 alloc_root: enter: root: 0x39a4640 alloc_root: exit: ptr: 0x7f2a20005bf8 mysql_execute_command: THD::enter_stage: /home/jan/mysql/galera-10.0/sql/sql_parse.cc:5323 trans_register_ha: enter: stmt ha_commit_trans: info: all: 0 thd->in_sub_stmt: 0 ha_info: 0x39a14c8 is_real_trans: 1 wsrep_run_wsrep_commit: wsrep: replicating commit wsrep_run_wsrep_commit: mutex: LOCK_wsrep_thd (0x39a2fe0) locking wsrep_run_wsrep_commit: mutex: LOCK_wsrep_thd (0x39a2fe0) locked wsrep_run_wsrep_commit: mutex: LOCK_wsrep_replaying (0x18ed380) locking my_malloc: my: size: 192 my_flags: 24 my_malloc: exit: ptr: 0x7f2a200bc800 my_hash_init: enter: hash: 0x7f2a200bc800 size: 128 my_malloc: my: size: 2048 my_flags: 0 my_malloc: exit: ptr: 0x7f2a20082e60 my_hash_init: enter: hash: 0x7f2a200bc860 size: 128 my_malloc: my: size: 2048 my_flags: 0 my_malloc: exit: ptr: 0x7f2a200b7f70 my_malloc: my: size: 56 my_flags: 56 my_malloc: exit: ptr: 0x7f2a200cedd0 add_to_locked_mutex: info: inserting 0x7f2a200cedd0 into 0x39a2fe0 (id: 284 -> 254) wsrep_run_wsrep_commit: mutex: LOCK_wsrep_replaying (0x18ed380) locked wsrep_run_wsrep_commit: mutex: LOCK_wsrep_replaying (0x18ed380) unlocking wsrep_run_wsrep_commit: mutex: LOCK_wsrep_thd (0x39a2fe0) unlocking reinit_io_cache: enter: cache: 0x7f2a200cf5e0 type: 1 seek_offset: 0 clear_cache: 0 my_malloc: my: size: 104 my_flags: 0 my_malloc: exit: ptr: 0x7f2a200bc940 my_free: my: ptr: 0x7f2a200bc940 sf_malloc_usable_size: exit: size: 104 flags: 0 wsrep_run_wsrep_commit: mutex: LOCK_wsrep_thd (0x39a2fe0) locking wsrep_run_wsrep_commit: mutex: LOCK_wsrep_thd (0x39a2fe0) locked wsrep_run_wsrep_commit: wsrep: replicating commit success wsrep_run_wsrep_commit: mutex: LOCK_wsrep_thd (0x39a2fe0) unlocking print_buffer_to_file: enter: buffer: WSREP: ::jan_BINLOG_COMMIT:: TRX: Pending (nil) pos 0, pos_in_file 139818903074288 current_pos 139818903139504 request_pos 139818903139608 write_pos 49062320 R: -- Jan Lindström Principal Engineer MariaDB | MaxScale | skype: jan_p_lindstrom www.skysql.com <http://www.skysql.com/> Twitter <http://twitter.com/skysql> Blog <http://www.skysql.com/blog/> Facebook <http://www.facebook.com/skysql> LinkedIn <http://www.linkedin.com/company/1214250> Google+ <https://plus.google.com/117544963211695643458/posts>
