Running msqld with thread sanitizer
Hello, I am trying to build and run a thread sanitized mysqld binary. In the CMake configuration, I see the WITH_ASAN option, however, I do not see an option to build mysqld with the thread sanitizer. After making some custom configurations, I was able to build a mysqld binary with the thread sanitizer (by linking static thread sanitizer), however, when trying to run it, I got the following error: "ThreadSanitizer: sync allocator overflow (65536*1024). Dying." This error seems to be coming from the tsan library and seems this error is caused by too many synchronizations [1]. Reducing the number of threads also didn't help me with this error. So my questions to you experts: 1. Do you run mysqld with the thread sanitizer ? 2. If yes, could you please share the details. (e.g., CMake configurations, compiler/linker flags etc.) 3. If not, any suggestions to overcome the above error ? [1] https://groups.google.com/g/thread-sanitizer/c/AX5twD_IPP4 Thank you. Best Regards, Thejaka Amila Kanewala, PhD https://github.com/thejkane/agm
Hi, Thejaka, On Aug 03, Thejaka Amila J Kanewala via developers wrote:
Hello,
I am trying to build and run a thread sanitized mysqld binary. In the CMake configuration, I see the WITH_ASAN option, however, I do not see an option to build mysqld with the thread sanitizer.
What MariaDB version did you look at? The earliest still supported is 10.4: This is WITH_ASAN option: https://github.com/MariaDB/server/blob/10.4/CMakeLists.txt#L199 This is WITH_TSAN option: https://github.com/MariaDB/server/blob/10.4/CMakeLists.txt#L223
After making some custom configurations, I was able to build a mysqld binary with the thread sanitizer (by linking static thread sanitizer), however, when trying to run it, I got the following error:
"ThreadSanitizer: sync allocator overflow (65536*1024). Dying."
This error seems to be coming from the tsan library and seems this error is caused by too many synchronizations [1]. Reducing the number of threads also didn't help me with this error.
So my questions to you experts: 1. Do you run mysqld with the thread sanitizer ?
No, our CI only runs with ASAN, MSAN, and UBSAN.
2. If yes, could you please share the details. (e.g., CMake configurations, compiler/linker flags etc.) 3. If not, any suggestions to overcome the above error ?
See above. I've just built with -DWITH_TSAN=ON and didn't get that error of yours. On gcc 12.3.1. I did get a lot of warning: 'atomic_thread_fence' is not supported with '-fsanitize=thread' which should've been expected, https://gcc.gnu.org/bugzilla/show_bug.cgi?id=97868 And thread sanitizer was complaining about statistic_increment() macro, which, indeed, increments a shared variable without any locks. Intentionally, though. Regards, Sergei VP of MariaDB Server Engineering and security@mariadb.org
Hello Sergei, Thank you for the reply. I was trying with 10.2 and compiled with gcc-10. Let me try with gcc 12. Thanks. Best Regards, Thejaka Amila Kanewala, PhD https://github.com/thejkane/agm http://valagamba.net/ On Thu, Aug 3, 2023 at 2:52 PM Sergei Golubchik <serg@mariadb.org> wrote:
Hi, Thejaka,
On Aug 03, Thejaka Amila J Kanewala via developers wrote:
Hello,
I am trying to build and run a thread sanitized mysqld binary. In the CMake configuration, I see the WITH_ASAN option, however, I do not see an option to build mysqld with the thread sanitizer.
What MariaDB version did you look at? The earliest still supported is 10.4:
This is WITH_ASAN option: https://github.com/MariaDB/server/blob/10.4/CMakeLists.txt#L199
This is WITH_TSAN option: https://github.com/MariaDB/server/blob/10.4/CMakeLists.txt#L223
After making some custom configurations, I was able to build a mysqld binary with the thread sanitizer (by linking static thread sanitizer), however, when trying to run it, I got the following error:
"ThreadSanitizer: sync allocator overflow (65536*1024). Dying."
This error seems to be coming from the tsan library and seems this error is caused by too many synchronizations [1]. Reducing the number of threads also didn't help me with this error.
So my questions to you experts: 1. Do you run mysqld with the thread sanitizer ?
No, our CI only runs with ASAN, MSAN, and UBSAN.
2. If yes, could you please share the details. (e.g., CMake configurations, compiler/linker flags etc.) 3. If not, any suggestions to overcome the above error ?
See above. I've just built with -DWITH_TSAN=ON and didn't get that error of yours. On gcc 12.3.1.
I did get a lot of
warning: 'atomic_thread_fence' is not supported with '-fsanitize=thread'
which should've been expected, https://gcc.gnu.org/bugzilla/show_bug.cgi?id=97868
And thread sanitizer was complaining about statistic_increment() macro, which, indeed, increments a shared variable without any locks. Intentionally, though.
Regards, Sergei VP of MariaDB Server Engineering and security@mariadb.org
Hi, Thejaka, 10.2 is two years out of support. Any bugs you possibly find in 10.2 will never be fixed - there will be no new 10.2 releases. Better use an active release, 10.4 or later. The latest is 11.2 at the moment, the latest LTS is 10.11. Regards, Sergei VP of MariaDB Server Engineering and security@mariadb.org On Aug 03, Thejaka Amila J Kanewala wrote:
Hello Sergei,
Thank you for the reply. I was trying with 10.2 and compiled with gcc-10.
Let me try with gcc 12.
On Aug 03, Thejaka Amila J Kanewala via developers wrote:
Hello,
I am trying to build and run a thread sanitized mysqld binary. In the CMake configuration, I see the WITH_ASAN option, however, I do not see an option to build mysqld with the thread sanitizer.
What MariaDB version did you look at? The earliest still supported is 10.4:
This is WITH_ASAN option: https://github.com/MariaDB/server/blob/10.4/CMakeLists.txt#L199
This is WITH_TSAN option: https://github.com/MariaDB/server/blob/10.4/CMakeLists.txt#L223
After making some custom configurations, I was able to build a mysqld binary with the thread sanitizer (by linking static thread sanitizer), however, when trying to run it, I got the following error:
"ThreadSanitizer: sync allocator overflow (65536*1024). Dying."
This error seems to be coming from the tsan library and seems this error is caused by too many synchronizations [1]. Reducing the number of threads also didn't help me with this error.
So my questions to you experts: 1. Do you run mysqld with the thread sanitizer ?
No, our CI only runs with ASAN, MSAN, and UBSAN.
2. If yes, could you please share the details. (e.g., CMake configurations, compiler/linker flags etc.) 3. If not, any suggestions to overcome the above error ?
See above. I've just built with -DWITH_TSAN=ON and didn't get that error of yours. On gcc 12.3.1.
I did get a lot of
warning: 'atomic_thread_fence' is not supported with '-fsanitize=thread'
which should've been expected, https://gcc.gnu.org/bugzilla/show_bug.cgi?id=97868
And thread sanitizer was complaining about statistic_increment() macro, which, indeed, increments a shared variable without any locks. Intentionally, though.
participants (2)
-
Sergei Golubchik -
Thejaka Amila J Kanewala