Would help if I read the very helpful - how to create a core file - link that was even in the output... root@debbie:/home/derick# coredumpctl debug --debugger-arguments="-batch -ex 'thread apply all bt full'" /usr/sbin/mariadbd            PID: 77900 (mariadbd)            UID: 124 (mysql)            GID: 134 (mysql)         Signal: 11 (SEGV)      Timestamp: Sat 2025-06-07 15:01:58 UTC (36s ago)   Command Line: /usr/sbin/mariadbd     Executable: /usr/sbin/mariadbd  Control Group: /system.slice/mariadb.service           Unit: mariadb.service          Slice: system.slice        Boot ID: 73cd144f6e9d4838b017b061212a9eb5     Machine ID: e7027c3ebde74f4a870b311db52087a0       Hostname: debbie.e-learndesign.scot        Storage: /var/lib/systemd/coredump/core.mariadbd.124.73cd144f6e9d4838b017b061212a9eb5.77900.1749308518000000.zst (present)   Size on Disk: 2.0M        Package: mariadb/1:11.4.7+maria~ubu2404       build-id: 082d7769ab844a0690ee7a7fbf3888d73ca70585        Message: Process 77900 (mariadbd) of user 124 dumped core.                 Module file_key_management.so from deb mariadb-1:11.4.7+maria~ubu2404.amd64                 Module libzstd.so.1 from deb libzstd-1.5.5+dfsg2-2build1.1.amd64                 Module libgcc_s.so.1 from deb gcc-14-14.2.0-4ubuntu2~24.04.amd64                 Module libstdc++.so.6 from deb gcc-14-14.2.0-4ubuntu2~24.04.amd64                 Module libsystemd.so.0 from deb systemd-255.4-1ubuntu8.6.amd64                 Module mariadbd from deb mariadb-1:11.4.7+maria~ubu2404.amd64                 Stack trace of thread 77900:                 #0  0x0000701f2b89eb2c __pthread_kill_implementation (libc.so.6 + 0x9eb2c)                 #1  0x0000568f36749aba handle_fatal_signal (mariadbd + 0xaefaba)                 #2  0x0000701f2b845330 __restore_rt (libc.so.6 + 0x45330)                 #3  0x0000568f36290615 n/a (mariadbd + 0x636615)                 #4  0x0000568f36c9d036 n/a (mariadbd + 0x1043036)                 #5  0x0000568f36b1e68d n/a (mariadbd + 0xec468d)                 #6  0x0000568f3627cc5a n/a (mariadbd + 0x622c5a)                 #7  0x0000568f369f5926 n/a (mariadbd + 0xd9b926)                 #8  0x0000568f3674c17c _Z24ha_initialize_handlertonPv (mariadbd + 0xaf217c)                 #9  0x0000568f364705b3 n/a (mariadbd + 0x8165b3)                 #10 0x0000568f364707a7 n/a (mariadbd + 0x8167a7)                 #11 0x0000568f36476e3c _Z11plugin_initPiPPci (mariadbd + 0x81ce3c)                 #12 0x0000568f36315d65 n/a (mariadbd + 0x6bbd65)                 #13 0x0000568f36319f79 _Z11mysqld_mainiPPc (mariadbd + 0x6bff79)                 #14 0x0000701f2b82a1ca __libc_start_call_main (libc.so.6 + 0x2a1ca)                 #15 0x0000701f2b82a28b __libc_start_main_impl (libc.so.6 + 0x2a28b)                 #16 0x0000568f362f3865 _start (mariadbd + 0x699865)                 Stack trace of thread 77902:                 #0  0x0000701f2b898d71 __futex_abstimed_wait_common64 (libc.so.6 + 0x98d71)                 #1  0x0000701f2b89bc8e __pthread_cond_wait_common (libc.so.6 + 0x9bc8e)                 #2  0x0000568f3629a685 psi_cond_timedwait (mariadbd + 0x640685)                 #3  0x0000568f3629a758 n/a (mariadbd + 0x640758)                 #4  0x0000568f3694d953 n/a (mariadbd + 0xcf3953)                 #5  0x0000701f2b89caa4 start_thread (libc.so.6 + 0x9caa4)                 #6  0x0000701f2b929c3c __clone3 (libc.so.6 + 0x129c3c)                 Stack trace of thread 77905:                 #0  0x0000701f2c802ab0 n/a (liburing.so.2 + 0x2ab0)                 #1  0x0000701f2c8030ad __io_uring_get_cqe (liburing.so.2 + 0x30ad)                 #2  0x0000568f36b3fde8 n/a (mariadbd + 0xee5de8)                 #3  0x0000701f2bcecdb4 n/a (libstdc++.so.6 + 0xecdb4)                 #4  0x0000701f2b89caa4 start_thread (libc.so.6 + 0x9caa4)                 #5  0x0000701f2b929c3c __clone3 (libc.so.6 + 0x129c3c)                 Stack trace of thread 77904:                 #0  0x0000701f2b898d71 __futex_abstimed_wait_common64 (libc.so.6 + 0x98d71)                 #1  0x0000701f2b89bc8e __pthread_cond_wait_common (libc.so.6 + 0x9bc8e)                 #2  0x0000568f3629a685 psi_cond_timedwait (mariadbd + 0x640685)                 #3  0x0000568f362545e8 n/a (mariadbd + 0x5fa5e8)                 #4  0x0000568f3694d953 n/a (mariadbd + 0xcf3953)                 #5  0x0000701f2b89caa4 start_thread (libc.so.6 + 0x9caa4)                 #6  0x0000701f2b929c3c __clone3 (libc.so.6 + 0x129c3c)                 Stack trace of thread 77907:                 #0  0x0000701f2b898d71 __futex_abstimed_wait_common64 (libc.so.6 + 0x98d71)                 #1  0x0000701f2b89b7ed __pthread_cond_wait_common (libc.so.6 + 0x9b7ed)                 #2  0x0000568f36aee675 n/a (mariadbd + 0xe94675)                 #3  0x0000701f2bcecdb4 n/a (libstdc++.so.6 + 0xecdb4)                 #4  0x0000701f2b89caa4 start_thread (libc.so.6 + 0x9caa4)                 #5  0x0000701f2b929c3c __clone3 (libc.so.6 + 0x129c3c)                 Stack trace of thread 77906:                 #0  0x0000701f2b91b4cd __GI___poll (libc.so.6 + 0x11b4cd)                 #1  0x0000568f36ae952d n/a (mariadbd + 0xe8f52d)                 #2  0x0000701f2bcecdb4 n/a (libstdc++.so.6 + 0xecdb4)                 #3  0x0000701f2b89caa4 start_thread (libc.so.6 + 0x9caa4)                 #4  0x0000701f2b929c3c __clone3 (libc.so.6 + 0x129c3c)                 ELF object binary architecture: AMD x86-64 warning: Can't open file anon_inode:[io_uring] which was expanded to anon_inode:[io_uring] during file-backed mapping note processing [New LWP 77900] [New LWP 77902] [New LWP 77905] [New LWP 77904] [New LWP 77907] [New LWP 77906] warning: could not find '.gnu_debugaltlink' file for /usr/lib/x86_64-linux-gnu/libtcmalloc.so warning: could not find '.gnu_debugaltlink' file for /lib/x86_64-linux-gnu/liburing.so.2 warning: could not find '.gnu_debugaltlink' file for /lib/x86_64-linux-gnu/libcap.so.2 [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Core was generated by `/usr/sbin/mariadbd'. Program terminated with signal SIGSEGV, Segmentation fault. #0  __pthread_kill_implementation (no_tid=0, signo=11, threadid=<optimized out>) at ./nptl/pthread_kill.c:44 warning: 44    ./nptl/pthread_kill.c: No such file or directory [Current thread is 1 (Thread 0x701f2be95000 (LWP 77900))] Thread 6 (Thread 0x7019a1bff6c0 (LWP 77906)): #0  0x0000701f2b91b4cd in __GI___poll (fds=0x568f37ebbe80, nfds=3, timeout=-1) at ../sysdeps/unix/sysv/linux/poll.c:29         sc_ret = -516         sc_cancel_oldtype = 0         sc_ret = <optimized out> #1  0x0000568f36ae952d in ?? () No symbol table info available. #2  0x0000701f2bcecdb4 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6 No symbol table info available. #3  0x0000701f2b89caa4 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:447         ret = <optimized out>         pd = <optimized out>         out = <optimized out>         unwind_buf = {cancel_jmp_buf = {{jmp_buf = {123255390205632, -8149018567210979406, 123255390205632, -584, 0, 140726198897104, -8149018567231950926, -8149885138259824718}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}         not_first_call = <optimized out> #4  0x0000701f2b929c3c in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78 No locals. Thread 5 (Thread 0x7019a13fe6c0 (LWP 77907)): #0  0x0000701f2b898d71 in __futex_abstimed_wait_common64 (private=28703, cancel=true, abstime=0x0, op=393, expected=0, futex_word=0x568f37560468) at ./nptl/futex-internal.c:57         sc_cancel_oldtype = 0         sc_ret = <optimized out>         resultvar = <optimized out>         __arg6 = <optimized out>         __arg5 = <optimized out>         __arg4 = <optimized out>         __arg3 = <optimized out>         __arg2 = <optimized out>         __arg1 = <optimized out>         _a6 = <optimized out>         _a5 = <optimized out>         _a4 = <optimized out>         _a3 = <optimized out>         _a2 = <optimized out>         _a1 = <optimized out> #1  __futex_abstimed_wait_common (cancel=true, private=28703, abstime=0x0, clockid=0, expected=0, futex_word=0x568f37560468) at ./nptl/futex-internal.c:87         err = <optimized out>         clockbit = 256         op = 393         err = <optimized out>         clockbit = <optimized out>         op = <optimized out> #2  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x568f37560468, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=private@entry=0) at ./nptl/futex-internal.c:139 No locals. #3  0x0000701f2b89b7ed in __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x568f375603c0, cond=0x568f37560440) at ./nptl/pthread_cond_wait.c:503         spin = 0         buffer = {__routine = 0x701f2b89b4b0 <__condvar_cleanup_waiting>, __arg = 0x7019a13fdbf0, __canceltype = -1589650416, __prev = 0x0}         cbuffer = {wseq = 2, cond = 0x568f37560440, mutex = 0x568f375603c0, private = 0}         err = <optimized out>         g = 0         flags = <optimized out>         g1_start = <optimized out>         maxspin = 0         signals = <optimized out>         result = 0         wseq = 2         seq = 1         private = 0         maxspin = <optimized out>         err = <optimized out>         result = <optimized out>         wseq = <optimized out>         g = <optimized out>         seq = <optimized out>         flags = <optimized out>         private = <optimized out>         signals = <optimized out>         done = <optimized out>         g1_start = <optimized out>         spin = <optimized out>         buffer = <optimized out>         cbuffer = <optimized out>         s = <optimized out> #4  ___pthread_cond_wait (cond=0x568f37560440, mutex=0x568f375603c0) at ./nptl/pthread_cond_wait.c:627 No locals. #5  0x0000568f36aee675 in ?? () No symbol table info available. #6  0x0000701f2bcecdb4 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6 No symbol table info available. #7  0x0000701f2b89caa4 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:447         ret = <optimized out>         pd = <optimized out>         out = <optimized out>         unwind_buf = {cancel_jmp_buf = {{jmp_buf = {123255381812928, -8149017468236222542, 123255381812928, -584, 0, 140726198897792, -8149017468257194062, -8149885138259824718}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}         not_first_call = <optimized out> #8  0x0000701f2b929c3c in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78 No locals. Thread 4 (Thread 0x701f233c56c0 (LWP 77904)): #0  0x0000701f2b898d71 in __futex_abstimed_wait_common64 (private=0, cancel=true, abstime=0x701f233c4c90, op=393, expected=0, futex_word=0x568f37e7eca8) at ./nptl/futex-internal.c:57         sc_cancel_oldtype = 0         sc_ret = <optimized out>         resultvar = <optimized out>         __arg6 = <optimized out>         __arg5 = <optimized out>         __arg4 = <optimized out>         __arg3 = <optimized out>         __arg2 = <optimized out>         __arg1 = <optimized out>         _a6 = <optimized out>         _a5 = <optimized out>         _a4 = <optimized out>         _a3 = <optimized out>         _a2 = <optimized out>         _a1 = <optimized out> #1  __futex_abstimed_wait_common (cancel=true, private=0, abstime=0x701f233c4c90, clockid=0, expected=0, futex_word=0x568f37e7eca8) at ./nptl/futex-internal.c:87         err = <optimized out>         clockbit = 256         op = 393         err = <optimized out>         clockbit = <optimized out>         op = <optimized out> #2  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x568f37e7eca8, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x701f233c4c90, private=private@entry=0) at ./nptl/futex-internal.c:139 No locals. #3  0x0000701f2b89bc8e in __pthread_cond_wait_common (abstime=0x701f233c4c90, clockid=0, mutex=0x568f37e7ecc0, cond=0x568f37e7ec80) at ./nptl/pthread_cond_wait.c:503         spin = 0         buffer = {__routine = 0x701f2b89b4b0 <__condvar_cleanup_waiting>, __arg = 0x701f233c4b50, __canceltype = 591154096, __prev = 0x0}         cbuffer = {wseq = 0, cond = 0x568f37e7ec80, mutex = 0x568f37e7ecc0, private = 0}         err = <optimized out>         g = 0         flags = <optimized out>         g1_start = <optimized out>         maxspin = 0         signals = <optimized out>         result = 0         wseq = 0         seq = 0         private = 0         maxspin = <optimized out>         err = <optimized out>         result = <optimized out>         wseq = <optimized out>         g = <optimized out>         seq = <optimized out>         flags = <optimized out>         private = <optimized out>         signals = <optimized out>         done = <optimized out>         g1_start = <optimized out>         spin = <optimized out>         buffer = <optimized out>         cbuffer = <optimized out>         s = <optimized out> #4  ___pthread_cond_timedwait64 (cond=0x568f37e7ec80, mutex=0x568f37e7ecc0, abstime=0x701f233c4c90) at ./nptl/pthread_cond_wait.c:652         flags = <optimized out>         clockid = 0 #5  0x0000568f3629a685 in psi_cond_timedwait () No symbol table info available. #6  0x0000568f362545e8 in ?? () No symbol table info available. #7  0x0000568f3694d953 in ?? () No symbol table info available. #8  0x0000701f2b89caa4 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:447         ret = <optimized out>         pd = <optimized out>         out = <optimized out>         unwind_buf = {cancel_jmp_buf = {{jmp_buf = {123279037454016, -8149866270274900046, 123279037454016, -584, 0, 140726198900064, -8149866270295871566, -8149885138259824718}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}         not_first_call = <optimized out> #9  0x0000701f2b929c3c in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78 No locals. Thread 3 (Thread 0x701f22bc46c0 (LWP 77905)): #0  0x0000701f2c802ab0 in ?? () from /lib/x86_64-linux-gnu/liburing.so.2 No symbol table info available. #1  0x0000701f2c8030ad in __io_uring_get_cqe () from /lib/x86_64-linux-gnu/liburing.so.2 No symbol table info available. #2  0x0000568f36b3fde8 in ?? () No symbol table info available. #3  0x0000701f2bcecdb4 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6 No symbol table info available. #4  0x0000701f2b89caa4 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:447         ret = <optimized out>         pd = <optimized out>         out = <optimized out>         unwind_buf = {cancel_jmp_buf = {{jmp_buf = {123279029061312, -8149869569346654286, 123279029061312, -584, 0, 140726198897312, -8149869569367625806, -8149885138259824718}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}         not_first_call = <optimized out> #5  0x0000701f2b929c3c in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78 No locals. Thread 2 (Thread 0x701f2be916c0 (LWP 77902)): #0  0x0000701f2b898d71 in __futex_abstimed_wait_common64 (private=0, cancel=true, abstime=0x701f2be90c90, op=393, expected=0, futex_word=0x568f37603a4c) at ./nptl/futex-internal.c:57         sc_cancel_oldtype = 0         sc_ret = <optimized out>         resultvar = <optimized out>         __arg6 = <optimized out>         __arg5 = <optimized out>         __arg4 = <optimized out>         __arg3 = <optimized out>         __arg2 = <optimized out>         __arg1 = <optimized out>         _a6 = <optimized out>         _a5 = <optimized out>         _a4 = <optimized out>         _a3 = <optimized out>         _a2 = <optimized out>         _a1 = <optimized out> #1  __futex_abstimed_wait_common (cancel=true, private=0, abstime=0x701f2be90c90, clockid=0, expected=0, futex_word=0x568f37603a4c) at ./nptl/futex-internal.c:87         err = <optimized out>         clockbit = 256         op = 393         err = <optimized out>         clockbit = <optimized out>         op = <optimized out> #2  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x568f37603a4c, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x701f2be90c90, private=private@entry=0) at ./nptl/futex-internal.c:139 No locals. #3  0x0000701f2b89bc8e in __pthread_cond_wait_common (abstime=0x701f2be90c90, clockid=0, mutex=0x568f37603a60, cond=0x568f37603a20) at ./nptl/pthread_cond_wait.c:503         spin = 0         buffer = {__routine = 0x701f2b89b4b0 <__condvar_cleanup_waiting>, __arg = 0x701f2be90b80, __canceltype = 0, __prev = 0x0}         cbuffer = {wseq = 9, cond = 0x568f37603a20, mutex = 0x568f37603a60, private = 0}         err = <optimized out>         g = 1         flags = <optimized out>         g1_start = <optimized out>         maxspin = 0         signals = <optimized out>         result = 0         wseq = 9         seq = 4         private = 0         maxspin = <optimized out>         err = <optimized out>         result = <optimized out>         wseq = <optimized out>         g = <optimized out>         seq = <optimized out>         flags = <optimized out>         private = <optimized out>         signals = <optimized out>         done = <optimized out>         g1_start = <optimized out>         spin = <optimized out>         buffer = <optimized out>         cbuffer = <optimized out>         s = <optimized out> #4  ___pthread_cond_timedwait64 (cond=0x568f37603a20, mutex=0x568f37603a60, abstime=0x701f2be90c90) at ./nptl/pthread_cond_wait.c:652         flags = <optimized out>         clockid = 0 #5  0x0000568f3629a685 in psi_cond_timedwait () No symbol table info available. #6  0x0000568f3629a758 in ?? () No symbol table info available. #7  0x0000568f3694d953 in ?? () No symbol table info available. #8  0x0000701f2b89caa4 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:447         ret = <optimized out>         pd = <optimized out>         out = <optimized out>         unwind_buf = {cancel_jmp_buf = {{jmp_buf = {123279182993088, -8149885419386589262, 123279182993088, -584, 2, 140726198904720, -8149885419407560782, -8149885138259824718}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}         not_first_call = <optimized out> #9  0x0000701f2b929c3c in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78 No locals. Thread 1 (Thread 0x701f2be95000 (LWP 77900)): #0  __pthread_kill_implementation (no_tid=0, signo=11, threadid=<optimized out>) at ./nptl/pthread_kill.c:44         tid = <optimized out>         ret = 0         pd = <optimized out>         old_mask = {__val = {7595448454083406198}}         ret = <optimized out>         pd = <optimized out>         old_mask = <optimized out>         ret = <optimized out>         tid = <optimized out>         ret = <optimized out>         resultvar = <optimized out>         resultvar = <optimized out>         __arg3 = <optimized out>         __arg2 = <optimized out>         __arg1 = <optimized out>         _a3 = <optimized out>         _a2 = <optimized out>         _a1 = <optimized out>         __futex = <optimized out>         resultvar = <optimized out>         __arg3 = <optimized out>         __arg2 = <optimized out>         __arg1 = <optimized out>         _a3 = <optimized out>         _a2 = <optimized out>         _a1 = <optimized out>         __futex = <optimized out>         __private = <optimized out>         __oldval = <optimized out> #1  __pthread_kill_internal (signo=11, threadid=<optimized out>) at ./nptl/pthread_kill.c:78 No locals. #2  __GI___pthread_kill (threadid=<optimized out>, signo=11) at ./nptl/pthread_kill.c:89 No locals. #3  0x0000568f36749aba in handle_fatal_signal () No symbol table info available. #4  <signal handler called> No locals. #5  0x0000568f36290615 in ?? () No symbol table info available. #6  0x0000568f36c9d036 in ?? () No symbol table info available. #7  0x0000568f36b1e68d in ?? () No symbol table info available. #8  0x0000568f3627cc5a in ?? () No symbol table info available. #9  0x0000568f369f5926 in ?? () No symbol table info available. #10 0x0000568f3674c17c in ha_initialize_handlerton(void*) () No symbol table info available. #11 0x0000568f364705b3 in ?? () No symbol table info available. #12 0x0000568f364707a7 in ?? () No symbol table info available. #13 0x0000568f36476e3c in plugin_init(int*, char**, int) () No symbol table info available. #14 0x0000568f36315d65 in ?? () No symbol table info available. #15 0x0000568f36319f79 in mysqld_main(int, char**) () No symbol table info available. #16 0x0000701f2b82a1ca in __libc_start_call_main (main=main@entry=0x568f362b2390 <main>, argc=argc@entry=1, argv=argv@entry=0x7ffd5f18d438) at ../sysdeps/nptl/libc_start_call_main.h:58         self = <optimized out>         result = <optimized out>         unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140726198916152, -7989065198477267022, 1, 0, 95173106871664, 123279196151808, -7989065198491947086, -8149885079415312462}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x568f374b8168, 0x7ffd5f18d438}, data = {prev = 0x0, cleanup = 0x0, canceltype = 927695208}}}         not_first_call = <optimized out> #17 0x0000701f2b82a28b in __libc_start_main_impl (main=0x568f362b2390 <main>, argc=1, argv=0x7ffd5f18d438, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffd5f18d428) at ../csu/libc-start.c:360 No locals. #18 0x0000568f362f3865 in _start () No symbol table info available. On 07/06/2025 15:52, Derick Turner via discuss wrote:

Thanks Sergie

Been a while since I've worked with core dumps.  Gdd is telling me the crash files are not a recognised file format.  Installed systemd-coredump instead and got this from coredumpctl dump --output=core

           PID: 45641 (mariadbd)            UID: 124 (mysql)            GID: 134 (mysql)         Signal: 11 (SEGV)      Timestamp: Sat 2025-06-07 14:43:13 UTC (3s ago)   Command Line: /usr/sbin/mariadbd     Executable: /usr/sbin/mariadbd  Control Group: /system.slice/mariadb.service           Unit: mariadb.service          Slice: system.slice        Boot ID: 73cd144f6e9d4838b017b061212a9eb5     Machine ID: e7027c3ebde74f4a870b311db52087a0       Hostname: debbie.e-learndesign.scot        Storage: /var/lib/systemd/coredump/core.mariadbd.124.73cd144f6e9d4838b017b061212a9eb5.45641.1749307393000000.zst (present)   Size on Disk: 2.0M        Package: mariadb/1:11.4.7+maria~ubu2404       build-id: 082d7769ab844a0690ee7a7fbf3888d73ca70585        Message: Process 45641 (mariadbd) of user 124 dumped core.

                Module file_key_management.so from deb mariadb-1:11.4.7+maria~ubu2404.amd64                 Module libzstd.so.1 from deb libzstd-1.5.5+dfsg2-2build1.1.amd64                 Module libgcc_s.so.1 from deb gcc-14-14.2.0-4ubuntu2~24.04.amd64                 Module libstdc++.so.6 from deb gcc-14-14.2.0-4ubuntu2~24.04.amd64                 Module libsystemd.so.0 from deb systemd-255.4-1ubuntu8.6.amd64                 Module mariadbd from deb mariadb-1:11.4.7+maria~ubu2404.amd64                 Stack trace of thread 45641:                 #0  0x0000702f1cc4553b __kill (libc.so.6 + 0x4553b)                 #1  0x00005fcc1521fa76 handle_fatal_signal (mariadbd + 0xaefa76)                 #2  0x0000702f1cc45330 __restore_rt (libc.so.6 + 0x45330)                 #3  0x00005fcc14d66615 n/a (mariadbd + 0x636615)                 #4  0x00005fcc15773036 n/a (mariadbd + 0x1043036)                 #5  0x00005fcc155f468d n/a (mariadbd + 0xec468d)                 #6  0x00005fcc14d52c5a n/a (mariadbd + 0x622c5a)                 #7  0x00005fcc154cb926 n/a (mariadbd + 0xd9b926)                 #8  0x00005fcc1522217c _Z24ha_initialize_handlertonPv (mariadbd + 0xaf217c)                 #9  0x00005fcc14f465b3 n/a (mariadbd + 0x8165b3)                 #10 0x00005fcc14f467a7 n/a (mariadbd + 0x8167a7)                 #11 0x00005fcc14f4ce3c _Z11plugin_initPiPPci (mariadbd + 0x81ce3c)                 #12 0x00005fcc14debd65 n/a (mariadbd + 0x6bbd65)                 #13 0x00005fcc14deff79 _Z11mysqld_mainiPPc (mariadbd + 0x6bff79)                 #14 0x0000702f1cc2a1ca __libc_start_call_main (libc.so.6 + 0x2a1ca)                 #15 0x0000702f1cc2a28b __libc_start_main_impl (libc.so.6 + 0x2a28b)                 #16 0x00005fcc14dc9865 _start (mariadbd + 0x699865)

                Stack trace of thread 45645:                 #0  0x0000702f1cc98d71 __futex_abstimed_wait_common64 (libc.so.6 + 0x98d71)                 #1  0x0000702f1cc9bc8e __pthread_cond_wait_common (libc.so.6 + 0x9bc8e)                 #2  0x00005fcc14d70685 psi_cond_timedwait (mariadbd + 0x640685)                 #3  0x00005fcc14d2a5e8 n/a (mariadbd + 0x5fa5e8)                 #4  0x00005fcc15423953 n/a (mariadbd + 0xcf3953)                 #5  0x0000702f1cc9caa4 start_thread (libc.so.6 + 0x9caa4)                 #6  0x0000702f1cd29c3c __clone3 (libc.so.6 + 0x129c3c)

                Stack trace of thread 45643:                 #0  0x0000702f1cc98d71 __futex_abstimed_wait_common64 (libc.so.6 + 0x98d71)                 #1  0x0000702f1cc9bc8e __pthread_cond_wait_common (libc.so.6 + 0x9bc8e)                 #2  0x00005fcc14d70685 psi_cond_timedwait (mariadbd + 0x640685)                 #3  0x00005fcc14d70758 n/a (mariadbd + 0x640758)                 #4  0x00005fcc15423953 n/a (mariadbd + 0xcf3953)                 #5  0x0000702f1cc9caa4 start_thread (libc.so.6 + 0x9caa4)                 #6  0x0000702f1cd29c3c __clone3 (libc.so.6 + 0x129c3c)

                Stack trace of thread 45646:                 #0  0x0000702f1db76ab0 n/a (liburing.so.2 + 0x2ab0)                 #1  0x0000702f1db770ad __io_uring_get_cqe (liburing.so.2 + 0x30ad)                 #2  0x00005fcc15615de8 n/a (mariadbd + 0xee5de8)                 #3  0x0000702f1d0ecdb4 n/a (libstdc++.so.6 + 0xecdb4)                 #4  0x0000702f1cc9caa4 start_thread (libc.so.6 + 0x9caa4)                 #5  0x0000702f1cd29c3c __clone3 (libc.so.6 + 0x129c3c)

                Stack trace of thread 45648:                 #0  0x0000702f1cc98d71 __futex_abstimed_wait_common64 (libc.so.6 + 0x98d71)                 #1  0x0000702f1cc9b7ed __pthread_cond_wait_common (libc.so.6 + 0x9b7ed)                 #2  0x00005fcc155c4675 n/a (mariadbd + 0xe94675)                 #3  0x0000702f1d0ecdb4 n/a (libstdc++.so.6 + 0xecdb4)                 #4  0x0000702f1cc9caa4 start_thread (libc.so.6 + 0x9caa4)                 #5  0x0000702f1cd29c3c __clone3 (libc.so.6 + 0x129c3c)

                Stack trace of thread 45647:                 #0  0x0000702f1cd1b4cd __GI___poll (libc.so.6 + 0x11b4cd)                 #1  0x00005fcc155bf52d n/a (mariadbd + 0xe8f52d)                 #2  0x0000702f1d0ecdb4 n/a (libstdc++.so.6 + 0xecdb4)                 #3  0x0000702f1cc9caa4 start_thread (libc.so.6 + 0x9caa4)                 #4  0x0000702f1cd29c3c __clone3 (libc.so.6 + 0x129c3c)                 ELF object binary architecture: AMD x86-64

Before this I also installed mariadb-server-dbgsym and the entry in the error log looks like:

2025-06-07 14:47:21 0 [Note] Starting MariaDB 11.4.7-MariaDB-ubu2404-log source revision 118cfcf82107188f2295631193658b2ef94f4f3f server_uid 6IX+WeD2vga5XxJDcZUY249RO+s= as process 52898 2025-06-07 14:47:21 0 [Note] InnoDB: Compressed tables use zlib 1.3 2025-06-07 14:47:21 0 [Note] InnoDB: Number of transaction pools: 1 2025-06-07 14:47:21 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions 2025-06-07 14:47:21 0 [Note] InnoDB: Using liburing 2025-06-07 14:47:21 0 [Note] InnoDB: innodb_buffer_pool_size_max=22528m, innodb_buffer_pool_size=22528m 2025-06-07 14:47:21 0 [Note] InnoDB: Initialized memory pressure event listener 2025-06-07 14:47:21 0 [Note] InnoDB: Completed initialization of buffer pool 2025-06-07 14:47:21 0 [Note] InnoDB: File system buffers for log disabled (block size=512 bytes) 2025-06-07 14:47:21 0 [Note] InnoDB: Resetting space id's in the doublewrite buffer 2025-06-07 14:47:21 0 [Note] InnoDB: End of log at LSN=43439159458895 2025-06-07 14:47:21 0 [Note] InnoDB: Opened 3 undo tablespaces 2025-06-07 14:47:21 0 [Note] InnoDB: 128 rollback segments in 3 undo tablespaces are active. 2025-06-07 14:47:21 0 [Warning] InnoDB: Cannot free the unused segments in system tablespace because a previous shutdown was not with innodb_fast_shutdown=0 or XA PREPARE transactions exist 2025-06-07 14:47:22 0 [Note] InnoDB: System tablespace defragmentation process starts 2025-06-07 14:47:22 0 [Note] InnoDB: Moving the data from extents 15552 through 2452672 250607 14:47:22 [ERROR] /usr/sbin/mariadbd got signal 11 ; Sorry, we probably made a mistake, and this is a bug.

Your assistance in bug reporting will enable us to fix this for the next release. To report this bug, see https://mariadb.com/kb/en/reporting-bugs about how to report a bug on https://jira.mariadb.org/.

Please include the information from the server start above, to the end of the information below.

Server version: 11.4.7-MariaDB-ubu2404-log source revision: 118cfcf82107188f2295631193658b2ef94f4f3f

The information page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mariadbd/ contains instructions to obtain a better version of the backtrace below. Following these instructions will help MariaDB developers provide a fix quicker.

Attempting backtrace. Include this in the bug report. (note: Retrieving this information may fail)

Thread pointer: 0x0 stack_bottom = 0x0 thread_stack 0x30000 /usr/sbin/mariadbd(my_print_stacktrace+0x30)[0x63a1fcb51560] /usr/sbin/mariadbd(handle_fatal_signal+0x2a1)[0x63a1fc6eaae1] libc_sigaction.c:0(__restore_rt)[0x714484045330] /usr/sbin/mariadbd(+0x636615)[0x63a1fc231615] /usr/sbin/mariadbd(+0x1043036)[0x63a1fcc3e036] /usr/sbin/mariadbd(+0xec468d)[0x63a1fcabf68d] /usr/sbin/mariadbd(+0x622c5a)[0x63a1fc21dc5a] /usr/sbin/mariadbd(+0xd9b926)[0x63a1fc996926] /usr/sbin/mariadbd(_Z24ha_initialize_handlertonPv+0x8c)[0x63a1fc6ed17c] /usr/sbin/mariadbd(+0x8165b3)[0x63a1fc4115b3] /usr/sbin/mariadbd(+0x8167a7)[0x63a1fc4117a7] /usr/sbin/mariadbd(_Z11plugin_initPiPPci+0xbfc)[0x63a1fc417e3c] /usr/sbin/mariadbd(+0x6bbd65)[0x63a1fc2b6d65] /usr/sbin/mariadbd(_Z11mysqld_mainiPPc+0xf79)[0x63a1fc2baf79] x86/libc-start.c:74(__libc_start_call_main)[0x71448402a1ca] csu/libc-start.c:128(call_init)[0x71448402a28b] /usr/sbin/mariadbd(_start+0x25)[0x63a1fc294865] Writing a core file... Working directory at /var/lib/mysql Resource Limits (excludes unlimited resources): Limit                     Soft Limit           Hard Limit           Units Max stack size            8388608 unlimited            bytes Max core file size        0 unlimited            bytes Max processes             139959 139959               processes Max open files            1048576 1048576              files Max locked memory         524288 524288               bytes Max pending signals       139959 139959               signals Max msgqueue size         819200 819200               bytes Max nice priority         0 0 Max realtime priority     0 0 Core pattern: |/usr/lib/systemd/systemd-coredump %P %u %g %s %t 9223372036854775808 %h

Kernel version: Linux version 6.8.0-60-generic (buildd@lcy02-amd64-054) (x86_64-linux-gnu-gcc-13 (Ubuntu 13.3.0-6ubuntu2~24.04) 13.3.0, GNU ld (GNU Binutils for Ubuntu) 2.42) #63-Ubuntu SMP PREEMPT_DYNAMIC Tue Apr 15 19:04:15 UTC 2025

2025-06-07 14:47:29 0 [Note] Starting MariaDB 11.4.7-MariaDB-ubu2404-log source revision 118cfcf82107188f2295631193658b2ef94f4f3f server_uid 6IX+WeD2vga5XxJDcZUY249RO+s= as process 53111 2025-06-07 14:47:29 0 [Note] InnoDB: Compressed tables use zlib 1.3 2025-06-07 14:47:29 0 [Note] InnoDB: Number of transaction pools: 1 2025-06-07 14:47:29 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions 2025-06-07 14:47:29 0 [Note] InnoDB: Using liburing 2025-06-07 14:47:29 0 [Note] InnoDB: innodb_buffer_pool_size_max=22528m, innodb_buffer_pool_size=22528m 2025-06-07 14:47:29 0 [Note] InnoDB: Initialized memory pressure event listener 2025-06-07 14:47:29 0 [Note] InnoDB: Completed initialization of buffer pool 2025-06-07 14:47:29 0 [Note] InnoDB: File system buffers for log disabled (block size=512 bytes) 2025-06-07 14:47:29 0 [Note] InnoDB: Resetting space id's in the doublewrite buffer 2025-06-07 14:47:29 0 [Note] InnoDB: End of log at LSN=43439159458895 2025-06-07 14:47:29 0 [Note] InnoDB: Opened 3 undo tablespaces 2025-06-07 14:47:29 0 [Note] InnoDB: 128 rollback segments in 3 undo tablespaces are active. 2025-06-07 14:47:29 0 [Warning] InnoDB: Cannot free the unused segments in system tablespace because a previous shutdown was not with innodb_fast_shutdown=0 or XA PREPARE transactions exist 2025-06-07 14:47:30 0 [Note] InnoDB: System tablespace defragmentation process starts 2025-06-07 14:47:30 0 [Note] InnoDB: Moving the data from extents 15552 through 2452672 250607 14:47:30 [ERROR] /usr/sbin/mariadbd got signal 11 ; Sorry, we probably made a mistake, and this is a bug.

Your assistance in bug reporting will enable us to fix this for the next release. To report this bug, see https://mariadb.com/kb/en/reporting-bugs about how to report a bug on https://jira.mariadb.org/.

Please include the information from the server start above, to the end of the information below.

Server version: 11.4.7-MariaDB-ubu2404-log source revision: 118cfcf82107188f2295631193658b2ef94f4f3f

The information page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mariadbd/ contains instructions to obtain a better version of the backtrace below. Following these instructions will help MariaDB developers provide a fix quicker.

Attempting backtrace. Include this in the bug report. (note: Retrieving this information may fail)

Thread pointer: 0x0 stack_bottom = 0x0 thread_stack 0x30000 /usr/sbin/mariadbd(my_print_stacktrace+0x30)[0x64ab42e73560] /usr/sbin/mariadbd(handle_fatal_signal+0x2a1)[0x64ab42a0cae1] libc_sigaction.c:0(__restore_rt)[0x7ebd14245330]

Open file limits are off in this.  /etc/security/limits.conf has unlimited for files for mysql and systemd config file has LimitNOFILE=2097152.  Currently trying to track down why that is.

Kind regards

Derick

On 07/06/2025 15:07, Sergei Golubchik via discuss wrote:

...

Hi, Derick,

On Jun 07, Derick Turner via discuss wrote:

...

Hi all,

Patching one of our spare servers this morning and ran into a couple of issues.  The first was a complaint in the service status: ... Thread pointer: 0x0 stack_bottom = 0x0 thread_stack 0x30000 /usr/sbin/mariadbd(my_print_stacktrace+0x30)[0x5a41189db560] /usr/sbin/mariadbd(handle_fatal_signal+0x2a1)[0x5a4118574ae1] /lib/x86_64-linux-gnu/libc.so.6(+0x45330)[0x7650bf845330] /usr/sbin/mariadbd(+0x636615)[0x5a41180bb615] /usr/sbin/mariadbd(+0x1043036)[0x5a4118ac8036] /usr/sbin/mariadbd(+0xec468d)[0x5a411894968d] /usr/sbin/mariadbd(+0x622c5a)[0x5a41180a7c5a] /usr/sbin/mariadbd(+0xd9b926)[0x5a4118820926] /usr/sbin/mariadbd(_Z24ha_initialize_handlertonPv+0x8c)[0x5a411857717c] /usr/sbin/mariadbd(+0x8165b3)[0x5a411829b5b3] /usr/sbin/mariadbd(+0x8167a7)[0x5a411829b7a7] /usr/sbin/mariadbd(_Z11plugin_initPiPPci+0xbfc)[0x5a41182a1e3c] /usr/sbin/mariadbd(+0x6bbd65)[0x5a4118140d65] /usr/sbin/mariadbd(_Z11mysqld_mainiPPc+0xf79)[0x5a4118144f79] /lib/x86_64-linux-gnu/libc.so.6(+0x2a1ca)[0x7650bf82a1ca] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x8b)[0x7650bf82a28b] /usr/sbin/mariadbd(_start+0x25)[0x5a411811e865] Writing a core file... Working directory at /var/lib/mysql

Is there anything I'm missing before I raise this as a bug? yes, please install dbgsym packages to get a better stack trace. or load the core dump (that was written above) in gdb and run `thread apply all bt`. Someone will ask you do it after you raise a bug anyway, so you can as well be prepared.

Regards, Sergei Chief Architect, MariaDB Server andsecurity@mariadb.org _______________________________________________ discuss mailing list --discuss@lists.mariadb.org To unsubscribe send an email todiscuss-leave@lists.mariadb.org -- Derick Turner - He/Him

Managing Director e-Learn Design Ltd | Scotland's Moodle Partner T +44 (0)845 474 4512 | M +44 (0)7808 068 797 |www.e-learndesign.co.uk

_______________________________________________ discuss mailing list --discuss@lists.mariadb.org To unsubscribe send an email todiscuss-leave@lists.mariadb.org

-- Derick Turner - He/Him Managing Director e-Learn Design Ltd | Scotland's Moodle Partner T +44 (0)845 474 4512 | M +44 (0)7808 068 797 |www.e-learndesign.co.uk

On Mon, 9 Jun 2025 at 16:48, Otto Kekäläinen via discuss < discuss@lists.mariadb.org> wrote:

...

Patching one of our spare servers this morning and ran into a couple of issues. The first was a complaint in the service status:

mariadb.service: Referenced but unset environment variable evaluates to an empty string: MYSQLD_OPTS, _WSREP_NEW_CLUSTER

Which was fixed by running systemctl edit mariadb.service and adding

Environment="MYSQLD_OPTS=" Environment="_WSREP_NEW_CLUSTER="

This is just a warning, and does not affect anything. Naturally it should be fixed though. Potential fix posted in https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1105029

A more complete backport of MDEV-19210 commits would be required to have a serviceable Galera with systemd. upstream commits: * 3d2e54ff8c6070800ead60dd1616df30c5936c8b * 105473233dd49b198758432fa0da5d0811f99c08 * aeffec60f6864bae5af04dac1184f2a0f2c77f38 * 4c56c66372c3f578dc75b537a79bf922dbddcafd * 621926e90ac05c33b956cf123f08e56b654d00df * 7936254012784790b7747dc07941917af1bbb83c

...

The second was from the error log which stated that the service was unable to read /etc/mysql/rest/keyfile.passwd, which turned out to be a permissions error on /etc/mysql/rest which was 600 and not 700 (I checked the other servers and theirs were also 600 so I guess it's the fix for MDEV-36229 which caused that)

yes, directories should have execute permissions.