[Maria-discuss] MariaDB 10.1.15 Released and Crash
Hi, Last night one of my servers updated MariaDB to 10.1.15 and it started to crash every few minutes since the update ( from 10.1.14 -> 10.1.15 ). ------------ 160702 10:22:13 [ERROR] mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. To report this bug, see https://mariadb.com/kb/en/reporting-bugs We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. Server version: 10.1.15-MariaDB key_buffer_size=268435456 read_buffer_size=524288 max_used_connections=8 max_threads=1001 thread_count=4 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1804821 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x0x7f5a89876008 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x7f5ab5dd9058 thread_stack 0x48400 /usr/sbin/mysqld(my_print_stacktrace+0x2b)[0x7f5c9736641b] /usr/sbin/mysqld(handle_fatal_signal+0x475)[0x7f5c96ec29f5] /lib64/libpthread.so.0(+0xf7e0)[0x7f5c964c77e0] /usr/sbin/mysqld(_ZN10SQL_SELECT17test_quick_selectEP3THD6BitmapILj64EEyybbb+0x1fe5)[0x7f5c96fae535] /usr/sbin/mysqld(+0x49bb90)[0x7f5c96d92b90] /usr/sbin/mysqld(_ZN4JOIN14optimize_innerEv+0x1e2e)[0x7f5c96da1e9e] /usr/sbin/mysqld(_ZN4JOIN8optimizeEv+0x20)[0x7f5c96da2fe0] /usr/sbin/mysqld(_Z12mysql_selectP3THDPPP4ItemP10TABLE_LISTjR4ListIS1_ES2_jP8st_orderSB_S2_SB_yP13select_resultP18st_select_lex_unitP13st_select_lex+0x9d)[0x7f5c96da313d] /usr/sbin/mysqld(_Z13handle_selectP3THDP3LEXP13select_resultm+0x25d)[0x7f5c96da6add] /usr/sbin/mysqld(+0x451d52)[0x7f5c96d48d52] /usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x5d7f)[0x7f5c96d5500f] /usr/sbin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x28d)[0x7f5c96d5854d] /usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x243b)[0x7f5c96d5b1ab] /usr/sbin/mysqld(_Z10do_commandP3THD+0x16b)[0x7f5c96d5b75b] /usr/sbin/mysqld(_Z26threadpool_process_requestP3THD+0xa7)[0x7f5c96e51777] mysys/stacktrace.c:247(my_print_stacktrace)[0x7f5c96e9c425] sql/signal_handler.cc:160(handle_fatal_signal)[0x7f5c964bfaa1] sql/opt_range.cc:12843(get_sel_arg_for_keypart)[0x7f5c949a4a8d] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query (0x7f5aaf1b5020): is an invalid pointer Connection ID (thread ID): 456 Status: NOT_KILLED Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=off ------------ Anyone else encountered crashes on 10.1.15 ? Regards
Hi Alex, On 07/02/2016 11:18 AM, Alex wrote:
Hi,
Last night one of my servers updated MariaDB to 10.1.15 and it started to crash every few minutes since the update ( from 10.1.14 -> 10.1.15 ).
Sorry about that. I assume you'll want to downgrade, but before you do that, could you please enable the general log and check which query it crashes on? It will be the last query from the connection ID which is mentioned in the crash log (xxx): Some pointers may be invalid and cause the dump to abort. Query (0x7f5aaf1b5020): is an invalid pointer Connection ID (thread ID): xxx Status: NOT_KILLED We have an open report with a stack trace similar to the visible part of yours, https://jira.mariadb.org/browse/MDEV-7865. As the description says, on some reason the scenario in the bug report didn't cause the crash on 10.x (I just re-checked, still does not, and no valgrind warnings either); but it's quite possible that the problem is there anyway. Maybe with your query we'll be able to create a test case for 10.x -- or maybe it's an entirely different bug, in which case a JIRA report will be needed. Thanks, Elena
------------
160702 10:22:13 [ERROR] mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware.
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail.
Server version: 10.1.15-MariaDB key_buffer_size=268435456 read_buffer_size=524288 max_used_connections=8 max_threads=1001 thread_count=4 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1804821 K bytes of memory Hope that's ok; if not, decrease some variables in the equation.
Thread pointer: 0x0x7f5a89876008 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x7f5ab5dd9058 thread_stack 0x48400 /usr/sbin/mysqld(my_print_stacktrace+0x2b)[0x7f5c9736641b] /usr/sbin/mysqld(handle_fatal_signal+0x475)[0x7f5c96ec29f5] /lib64/libpthread.so.0(+0xf7e0)[0x7f5c964c77e0] /usr/sbin/mysqld(_ZN10SQL_SELECT17test_quick_selectEP3THD6BitmapILj64EEyybbb+0x1fe5)[0x7f5c96fae535]
/usr/sbin/mysqld(+0x49bb90)[0x7f5c96d92b90] /usr/sbin/mysqld(_ZN4JOIN14optimize_innerEv+0x1e2e)[0x7f5c96da1e9e] /usr/sbin/mysqld(_ZN4JOIN8optimizeEv+0x20)[0x7f5c96da2fe0] /usr/sbin/mysqld(_Z12mysql_selectP3THDPPP4ItemP10TABLE_LISTjR4ListIS1_ES2_jP8st_orderSB_S2_SB_yP13select_resultP18st_select_lex_unitP13st_select_lex+0x9d)[0x7f5c96da313d]
/usr/sbin/mysqld(_Z13handle_selectP3THDP3LEXP13select_resultm+0x25d)[0x7f5c96da6add]
/usr/sbin/mysqld(+0x451d52)[0x7f5c96d48d52] /usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x5d7f)[0x7f5c96d5500f] /usr/sbin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x28d)[0x7f5c96d5854d]
/usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x243b)[0x7f5c96d5b1ab]
/usr/sbin/mysqld(_Z10do_commandP3THD+0x16b)[0x7f5c96d5b75b] /usr/sbin/mysqld(_Z26threadpool_process_requestP3THD+0xa7)[0x7f5c96e51777] mysys/stacktrace.c:247(my_print_stacktrace)[0x7f5c96e9c425] sql/signal_handler.cc:160(handle_fatal_signal)[0x7f5c964bfaa1] sql/opt_range.cc:12843(get_sel_arg_for_keypart)[0x7f5c949a4a8d]
Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query (0x7f5aaf1b5020): is an invalid pointer Connection ID (thread ID): 456 Status: NOT_KILLED
Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=off
------------
Anyone else encountered crashes on 10.1.15 ? Regards
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
Hi Elena, I had to downgrade and went back to 10.1.14 which doesn't cause any problems. I may try to upgrade again tonight and turn on the general log. Regards On 7/2/2016 1:24 PM, Elena Stepanova wrote:
Hi Alex,
On 07/02/2016 11:18 AM, Alex wrote:
Hi,
Last night one of my servers updated MariaDB to 10.1.15 and it started to crash every few minutes since the update ( from 10.1.14 -> 10.1.15 ).
Sorry about that. I assume you'll want to downgrade, but before you do that, could you please enable the general log and check which query it crashes on? It will be the last query from the connection ID which is mentioned in the crash log (xxx):
Some pointers may be invalid and cause the dump to abort. Query (0x7f5aaf1b5020): is an invalid pointer Connection ID (thread ID): xxx Status: NOT_KILLED
We have an open report with a stack trace similar to the visible part of yours, https://jira.mariadb.org/browse/MDEV-7865. As the description says, on some reason the scenario in the bug report didn't cause the crash on 10.x (I just re-checked, still does not, and no valgrind warnings either); but it's quite possible that the problem is there anyway. Maybe with your query we'll be able to create a test case for 10.x -- or maybe it's an entirely different bug, in which case a JIRA report will be needed.
Thanks, Elena
------------
160702 10:22:13 [ERROR] mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware.
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail.
Server version: 10.1.15-MariaDB key_buffer_size=268435456 read_buffer_size=524288 max_used_connections=8 max_threads=1001 thread_count=4 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1804821 K bytes of memory Hope that's ok; if not, decrease some variables in the equation.
Thread pointer: 0x0x7f5a89876008 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x7f5ab5dd9058 thread_stack 0x48400 /usr/sbin/mysqld(my_print_stacktrace+0x2b)[0x7f5c9736641b] /usr/sbin/mysqld(handle_fatal_signal+0x475)[0x7f5c96ec29f5] /lib64/libpthread.so.0(+0xf7e0)[0x7f5c964c77e0] /usr/sbin/mysqld(_ZN10SQL_SELECT17test_quick_selectEP3THD6BitmapILj64EEyybbb+0x1fe5)[0x7f5c96fae535]
/usr/sbin/mysqld(+0x49bb90)[0x7f5c96d92b90] /usr/sbin/mysqld(_ZN4JOIN14optimize_innerEv+0x1e2e)[0x7f5c96da1e9e] /usr/sbin/mysqld(_ZN4JOIN8optimizeEv+0x20)[0x7f5c96da2fe0] /usr/sbin/mysqld(_Z12mysql_selectP3THDPPP4ItemP10TABLE_LISTjR4ListIS1_ES2_jP8st_orderSB_S2_SB_yP13select_resultP18st_select_lex_unitP13st_select_lex+0x9d)[0x7f5c96da313d]
/usr/sbin/mysqld(_Z13handle_selectP3THDP3LEXP13select_resultm+0x25d)[0x7f5c96da6add]
/usr/sbin/mysqld(+0x451d52)[0x7f5c96d48d52] /usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x5d7f)[0x7f5c96d5500f] /usr/sbin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x28d)[0x7f5c96d5854d]
/usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x243b)[0x7f5c96d5b1ab]
/usr/sbin/mysqld(_Z10do_commandP3THD+0x16b)[0x7f5c96d5b75b] /usr/sbin/mysqld(_Z26threadpool_process_requestP3THD+0xa7)[0x7f5c96e51777]
mysys/stacktrace.c:247(my_print_stacktrace)[0x7f5c96e9c425] sql/signal_handler.cc:160(handle_fatal_signal)[0x7f5c964bfaa1] sql/opt_range.cc:12843(get_sel_arg_for_keypart)[0x7f5c949a4a8d]
Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query (0x7f5aaf1b5020): is an invalid pointer Connection ID (thread ID): 456 Status: NOT_KILLED
Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=off
------------
Anyone else encountered crashes on 10.1.15 ? Regards
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
Hi Alex, On 07/02/2016 01:35 PM, Alex wrote:
Hi Elena,
I had to downgrade and went back to 10.1.14 which doesn't cause any problems. I may try to upgrade again tonight and turn on the general log.
Thanks, it won't be needed just yet, I think I have a test case for what you've encountered; but when the bug is fixed, maybe you'll want to try the patch and see if it really solves the problem on your side. Regards, Elena
Regards
On 7/2/2016 1:24 PM, Elena Stepanova wrote:
Hi Alex,
On 07/02/2016 11:18 AM, Alex wrote:
Hi,
Last night one of my servers updated MariaDB to 10.1.15 and it started to crash every few minutes since the update ( from 10.1.14 -> 10.1.15 ).
Sorry about that. I assume you'll want to downgrade, but before you do that, could you please enable the general log and check which query it crashes on? It will be the last query from the connection ID which is mentioned in the crash log (xxx):
Some pointers may be invalid and cause the dump to abort. Query (0x7f5aaf1b5020): is an invalid pointer Connection ID (thread ID): xxx Status: NOT_KILLED
We have an open report with a stack trace similar to the visible part of yours, https://jira.mariadb.org/browse/MDEV-7865. As the description says, on some reason the scenario in the bug report didn't cause the crash on 10.x (I just re-checked, still does not, and no valgrind warnings either); but it's quite possible that the problem is there anyway. Maybe with your query we'll be able to create a test case for 10.x -- or maybe it's an entirely different bug, in which case a JIRA report will be needed.
Thanks, Elena
------------
160702 10:22:13 [ERROR] mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware.
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail.
Server version: 10.1.15-MariaDB key_buffer_size=268435456 read_buffer_size=524288 max_used_connections=8 max_threads=1001 thread_count=4 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1804821 K bytes of memory Hope that's ok; if not, decrease some variables in the equation.
Thread pointer: 0x0x7f5a89876008 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x7f5ab5dd9058 thread_stack 0x48400 /usr/sbin/mysqld(my_print_stacktrace+0x2b)[0x7f5c9736641b] /usr/sbin/mysqld(handle_fatal_signal+0x475)[0x7f5c96ec29f5] /lib64/libpthread.so.0(+0xf7e0)[0x7f5c964c77e0] /usr/sbin/mysqld(_ZN10SQL_SELECT17test_quick_selectEP3THD6BitmapILj64EEyybbb+0x1fe5)[0x7f5c96fae535]
/usr/sbin/mysqld(+0x49bb90)[0x7f5c96d92b90] /usr/sbin/mysqld(_ZN4JOIN14optimize_innerEv+0x1e2e)[0x7f5c96da1e9e] /usr/sbin/mysqld(_ZN4JOIN8optimizeEv+0x20)[0x7f5c96da2fe0] /usr/sbin/mysqld(_Z12mysql_selectP3THDPPP4ItemP10TABLE_LISTjR4ListIS1_ES2_jP8st_orderSB_S2_SB_yP13select_resultP18st_select_lex_unitP13st_select_lex+0x9d)[0x7f5c96da313d]
/usr/sbin/mysqld(_Z13handle_selectP3THDP3LEXP13select_resultm+0x25d)[0x7f5c96da6add]
/usr/sbin/mysqld(+0x451d52)[0x7f5c96d48d52] /usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x5d7f)[0x7f5c96d5500f] /usr/sbin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x28d)[0x7f5c96d5854d]
/usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x243b)[0x7f5c96d5b1ab]
/usr/sbin/mysqld(_Z10do_commandP3THD+0x16b)[0x7f5c96d5b75b] /usr/sbin/mysqld(_Z26threadpool_process_requestP3THD+0xa7)[0x7f5c96e51777]
mysys/stacktrace.c:247(my_print_stacktrace)[0x7f5c96e9c425] sql/signal_handler.cc:160(handle_fatal_signal)[0x7f5c964bfaa1] sql/opt_range.cc:12843(get_sel_arg_for_keypart)[0x7f5c949a4a8d]
Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query (0x7f5aaf1b5020): is an invalid pointer Connection ID (thread ID): 456 Status: NOT_KILLED
Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=off
------------
Anyone else encountered crashes on 10.1.15 ? Regards
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
participants (2)
-
Alex -
Elena Stepanova