[Maria-discuss] memory allocators and MariaDB
Do MariaDB 10.2 binaries use glibc malloc or get built with one of tcmalloc or jemalloc? Sorry for being lazy, as I have yet to install it. -- Mark Callaghan mdcallag@gmail.com
Hi, MARK! On Jun 20, MARK CALLAGHAN wrote:
Do MariaDB 10.2 binaries use glibc malloc or get built with one of tcmalloc or jemalloc? Sorry for being lazy, as I have yet to install it.
glibc malloc. Only ha_tokudb.so gets linked with jemalloc. Regards, Sergei Chief Architect MariaDB and security@mariadb.org
Am 20.06.2017 um 18:36 schrieb Sergei Golubchik:
Hi, MARK!
On Jun 20, MARK CALLAGHAN wrote:
Do MariaDB 10.2 binaries use glibc malloc or get built with one of tcmalloc or jemalloc? Sorry for being lazy, as I have yet to install it.
glibc malloc. Only ha_tokudb.so gets linked with jemalloc
really and why? the mariadb build below is built without tokudb [harry@srv-rhsoft:~]$ ldd /usr/libexec/mysqld linux-vdso.so.1 (0x00007ffc3896e000) libaio.so.1 => /lib64/libaio.so.1 (0x00007f28ce556000) libz.so.1 => /lib64/libz.so.1 (0x00007f28ce340000) libpcre.so.1 => /lib64/libpcre.so.1 (0x00007f28ce0cd000) libjemalloc.so.2 => /lib64/libjemalloc.so.2 (0x00007f28cde86000) libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007f28cdc50000) libssl.so.10 => /lib64/libssl.so.10 (0x00007f28cd9dc000) libcrypto.so.10 => /lib64/libcrypto.so.10 (0x00007f28cd57b000) libdl.so.2 => /lib64/libdl.so.2 (0x00007f28cd377000) libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f28ccfef000) libm.so.6 => /lib64/libm.so.6 (0x00007f28ccce6000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f28ccac8000) libc.so.6 => /lib64/libc.so.6 (0x00007f28cc700000) /lib64/ld-linux-x86-64.so.2 (0x00005616ed062000) libgomp.so.1 => /lib64/libgomp.so.1 (0x00007f28cc4d2000) libfreebl3.so => /lib64/libfreebl3.so (0x00007f28cc2cf000) libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00007f28cc082000) libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00007f28cbd9c000) libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007f28cbb98000) libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007f28cb965000) libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f28cb74e000) libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x00007f28cb53f000) libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00007f28cb33b000) libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f28cb120000) libselinux.so.1 => /lib64/libselinux.so.1 (0x00007f28caef7000)
Hi, Reindl! On Jun 20, Reindl Harald wrote:
On Jun 20, MARK CALLAGHAN wrote:
Do MariaDB 10.2 binaries use glibc malloc or get built with one of tcmalloc or jemalloc? Sorry for being lazy, as I have yet to install it.
glibc malloc. Only ha_tokudb.so gets linked with jemalloc
really and why?
the mariadb build below is built without tokudb
Is it 10.2? I've removed jemalloc dependency in 10.2.5, commit bf40e8069cd6952 Why do you have kerberos in the list? How did you do that?
[harry@srv-rhsoft:~]$ ldd /usr/libexec/mysqld linux-vdso.so.1 (0x00007ffc3896e000) libaio.so.1 => /lib64/libaio.so.1 (0x00007f28ce556000) libz.so.1 => /lib64/libz.so.1 (0x00007f28ce340000) libpcre.so.1 => /lib64/libpcre.so.1 (0x00007f28ce0cd000) libjemalloc.so.2 => /lib64/libjemalloc.so.2 (0x00007f28cde86000) libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007f28cdc50000) libssl.so.10 => /lib64/libssl.so.10 (0x00007f28cd9dc000) libcrypto.so.10 => /lib64/libcrypto.so.10 (0x00007f28cd57b000) libdl.so.2 => /lib64/libdl.so.2 (0x00007f28cd377000) libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f28ccfef000) libm.so.6 => /lib64/libm.so.6 (0x00007f28ccce6000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f28ccac8000) libc.so.6 => /lib64/libc.so.6 (0x00007f28cc700000) /lib64/ld-linux-x86-64.so.2 (0x00005616ed062000) libgomp.so.1 => /lib64/libgomp.so.1 (0x00007f28cc4d2000) libfreebl3.so => /lib64/libfreebl3.so (0x00007f28cc2cf000) libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00007f28cc082000) libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00007f28cbd9c000) libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007f28cbb98000) libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007f28cb965000) libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f28cb74e000) libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x00007f28cb53f000) libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00007f28cb33b000) libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f28cb120000) libselinux.so.1 => /lib64/libselinux.so.1 (0x00007f28caef7000)
Regards, Sergei Chief Architect MariaDB and security@mariadb.org
Am 20.06.2017 um 19:43 schrieb Sergei Golubchik:
Hi, Reindl!
On Jun 20, Reindl Harald wrote:
On Jun 20, MARK CALLAGHAN wrote:
Do MariaDB 10.2 binaries use glibc malloc or get built with one of tcmalloc or jemalloc? Sorry for being lazy, as I have yet to install it.
glibc malloc. Only ha_tokudb.so gets linked with jemalloc
really and why?
the mariadb build below is built without tokudb
Is it 10.2?
10.1.24
I've removed jemalloc dependency in 10.2.5, commit bf40e8069cd6952
i don't get why it is not used everywhere at all trafficsevrer and dbmail are happy with it
Why do you have kerberos in the list? How did you do that?
most likely by a system library (Fedora 25) because as distributions i call bundeled libraries as root of all evil cmake . \ -DFEATURE_SET="large" \ -DCMAKE_INSTALL_PREFIX="%{_prefix}" \ -DINSTALL_INCLUDEDIR=include/mysql \ -DINSTALL_LAYOUT=RPM \ -DINSTALL_LIBDIR="%{_lib}/mysql" \ -DINSTALL_MANDIR=share/man \ -DINSTALL_MYSQLSHAREDIR=share/mysql \ -DINSTALL_MYSQLTESTDIR=share/mysql-test \ -DINSTALL_PLUGINDIR="%{_lib}/mysql/plugin" \ -DINSTALL_SBINDIR=libexec \ -DINSTALL_SCRIPTDIR=bin \ -DINSTALL_SQLBENCHDIR= \ -DINSTALL_SUPPORTFILESDIR=share/mysql \ -DMYSQL_DATADIR="%{_sharedstatedir}/mysql" \ -DMYSQL_UNIX_ADDR="%{_sharedstatedir}/mysql/mysql.sock" \ -DENABLED_PROFILING=OFF \ -DENABLE_DEBUG_SYNC=OFF \ -DENABLE_DTRACE=OFF \ -DPLUGIN_ARIA=YES \ -DPLUGIN_CSV=YES \ -DPLUGIN_MYISAM=YES \ -DPLUGIN_XTRADB=YES \ -DPLUGIN_ARCHIVE=NO \ -DPLUGIN_BLACKHOLE=NO \ -DPLUGIN_CASSANDRA=NO \ -DPLUGIN_CONNECT=NO \ -DPLUGIN_EXAMPLE=NO \ -DPLUGIN_FEDERATED=NO \ -DPLUGIN_FEDERATEDX=NO \ -DPLUGIN_FEEDBACK=NO \ -DPLUGIN_INNODB=NO \ -DPLUGIN_MROONGA=NO \ -DPLUGIN_MYISAMMRG=NO \ -DPLUGIN_OQGRAPH=NO \ -DPLUGIN_PARTITION=NO \ -DPLUGIN_PERFSCHEMA=NO \ -DPLUGIN_SEMISYNC=NO \ -DPLUGIN_SEQUENCE=NO \ -DPLUGIN_SPHINX=NO \ -DPLUGIN_SPIDER=NO \ -DPLUGIN_TOKUDB=NO \ -DWITHOUT_DYNAMIC_PLUGINS=ON \ -DWITH_ATOMIC_OPS=smp \ -DWITH_EMBEDDED_SERVER=OFF \ -DWITH_INNODB_DISALLOW_WRITES=OFF \ -DWITH_INNODB_BZIP2=OFF \ -DWITH_INNODB_LZ4=OFF \ -DWITH_INNODB_LZMA=OFF \ -DWITH_INNODB_LZO=OFF \ -DSECURITY_HARDENED=OFF \ -DWITH_LIBWRAP=OFF \ -DWITH_MARIABACKUP=OFF \ -DWITH_PIC=NO \ -DWITH_READLINE=OFF \ -DWITH_SAFEMALLOC=OFF \ -DWITH_SYSTEMD=OFF \ -DWITH_VALGRIND=OFF \ -DWITH_WSREP=OFF \ -DWITH_JEMALLOC=system \ -DWITH_SSL=system \ -DWITH_ZLIB=system \ -DCOMPILATION_COMMENT="thelounge" %{__make} %{?_smp_mflags} VERBOSE=1
[harry@srv-rhsoft:~]$ ldd /usr/libexec/mysqld linux-vdso.so.1 (0x00007ffc3896e000) libaio.so.1 => /lib64/libaio.so.1 (0x00007f28ce556000) libz.so.1 => /lib64/libz.so.1 (0x00007f28ce340000) libpcre.so.1 => /lib64/libpcre.so.1 (0x00007f28ce0cd000) libjemalloc.so.2 => /lib64/libjemalloc.so.2 (0x00007f28cde86000) libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007f28cdc50000) libssl.so.10 => /lib64/libssl.so.10 (0x00007f28cd9dc000) libcrypto.so.10 => /lib64/libcrypto.so.10 (0x00007f28cd57b000) libdl.so.2 => /lib64/libdl.so.2 (0x00007f28cd377000) libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f28ccfef000) libm.so.6 => /lib64/libm.so.6 (0x00007f28ccce6000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f28ccac8000) libc.so.6 => /lib64/libc.so.6 (0x00007f28cc700000) /lib64/ld-linux-x86-64.so.2 (0x00005616ed062000) libgomp.so.1 => /lib64/libgomp.so.1 (0x00007f28cc4d2000) libfreebl3.so => /lib64/libfreebl3.so (0x00007f28cc2cf000) libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00007f28cc082000) libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00007f28cbd9c000) libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007f28cbb98000) libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007f28cb965000) libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f28cb74e000) libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x00007f28cb53f000) libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00007f28cb33b000) libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f28cb120000) libselinux.so.1 => /lib64/libselinux.so.1 (0x00007f28caef7000)
Hi, Reindl! On Jun 20, Reindl Harald wrote:
the mariadb build below is built without tokudb
Is it 10.2?
10.1.24
Ah, okay.
I've removed jemalloc dependency in 10.2.5, commit bf40e8069cd6952
i don't get why it is not used everywhere at all trafficsevrer and dbmail are happy with it
And on FreeBSD it's the default system malloc. But on Linux it's an extra dependency that not many users need. (and those who do can use --malloc-lib switch). Some users even wanted to use tcmalloc. Regards, Sergei Chief Architect MariaDB and security@mariadb.org
Am 20.06.2017 um 20:53 schrieb Sergei Golubchik:
Hi, Reindl!
On Jun 20, Reindl Harald wrote:
the mariadb build below is built without tokudb
Is it 10.2?
10.1.24
Ah, okay.
I've removed jemalloc dependency in 10.2.5, commit bf40e8069cd6952
i don't get why it is not used everywhere at all trafficsevrer and dbmail are happy with it
And on FreeBSD it's the default system malloc.
But on Linux it's an extra dependency that not many users need. (and those who do can use --malloc-lib switch). Some users even wanted to use tcmalloc
that still don't explain why "-DWITH_JEMALLOC" should be only for TokuDB and why that can be not a configure option at all for the whole mariadb stuff
On Tue, Jun 20, 2017 at 9:36 AM, Sergei Golubchik <serg@mariadb.org> wrote:
Hi, MARK!
On Jun 20, MARK CALLAGHAN wrote:
Do MariaDB 10.2 binaries use glibc malloc or get built with one of tcmalloc or jemalloc? Sorry for being lazy, as I have yet to install it.
glibc malloc. Only ha_tokudb.so gets linked with jemalloc.
Thanks, I neglected to share a link to a result from the past to show the benefit from jemalloc and tcmalloc with MyRocks. RSS was much larger with glibc malloc because tcmalloc and jemalloc are better at avoiding fragmentation when dealing with something like RocksDB that puts stress on the allocator. This isn't an issue for InnoDB. I will repeat this test soon. http://smalldatum.blogspot.com/2015/10/myrocks-versus-allocators-glibc.html -- Mark Callaghan mdcallag@gmail.com
participants (3)
-
MARK CALLAGHAN -
Reindl Harald -
Sergei Golubchik