[Maria-developers] compile issue and the -lz flag
Hi all: when compiling on x64 systems (i.e. CentOS & Fedora core 10) and using the BUILD/compile-pentium64-max, I'm getting the -lz error "/usr/bin/ld: cannot find -lz" and when I simplify the ./comfigure line to a --without-zlib it works. So, after modifing the configure flags that results into: ./configure --prefix=/u02/local/mysql-5.1.34 --enable-assembler --with-extra-charsets=complex --enable-thread-safe-client --with-big-tables --with-readline --with-ssl --with-plugins=max --with-embedded-server --with-mysqld-ldflags=-all-static --with-client-ldflags=-all-static --without-zlib --enable-local-infile It still has problems with the -lz flag, so does anybody know what is bringing the -lz into the configure? Note: I've also tried --with-zlibdir=/usr/lib64, but same error. Thanks tons.
Don Kehn <dekehn@gmail.com> writes:
when compiling on x64 systems (i.e. CentOS & Fedora core 10) and using the BUILD/compile-pentium64-max, I'm getting the -lz error "/usr/bin/ld: cannot find -lz" and when I simplify the ./comfigure line to a --without-zlib it
Hm, could not reproduce this locally. This was using BUILD/compile-pentium64-max on a 64-bit Ubuntu Hardy (amd64).
works. So, after modifing the configure flags that results into: ./configure --prefix=/u02/local/mysql-5.1.34 --enable-assembler --with-extra-charsets=complex --enable-thread-safe-client --with-big-tables --with-readline --with-ssl --with-plugins=max --with-embedded-server --with-mysqld-ldflags=-all-static --with-client-ldflags=-all-static --without-zlib --enable-local-infile
It still has problems with the -lz flag, so does anybody know what is bringing the -lz into the configure? Note: I've also tried --with-zlibdir=/usr/lib64, but same error.
As far as I can tell, there is no --with-zlib, --without-zlib, or --with-zlibdir option in ./configure. The correct name of the option seems to be --with-zlib-dir. Just that ./configure has this 'nice' habbit of ignoring unknown options :-(. So probably you can get it to compile using --with-zlib-dir=bundled. However, by default (no --with-zlib-dir) option it should use the system libz if available, and fall back to bundled if not, as far as I can tell from sources. So it's strange that it failed for you. Can you show exact link error message? Maybe you have a broken or incompatible version of libz on your system? Do you have a zlib-dev package installed? - Kristian.
On Monday 04 May 2009 16:57:42 Kristian Nielsen wrote:
Don Kehn <dekehn@gmail.com> writes:
when compiling on x64 systems (i.e. CentOS & Fedora core 10) and using the BUILD/compile-pentium64-max, I'm getting the -lz error "/usr/bin/ld: cannot find -lz" and when I simplify the ./comfigure line to a --without-zlib it
Hm, could not reproduce this locally. This was using BUILD/compile-pentium64-max on a 64-bit Ubuntu Hardy (amd64).
64 bit libraries are handled differently between these two. Debian/Ubuntu puts them in /usr/lib (and has a link from /usr/lib64) and CentOS/Fedora/Red Hat/OpenSUSE puts them in /usr/lib64 (/usr/lib is 32 bit). So to reproduce it, you need a distribution that does the same thing as Dons. I only have Kubuntu machines here, so I can't test it. Don, just to be absolutely certain: You do have the zlib devel package installed, right? There should be something like this in your /usr/lib64: shrek> ll /usr/lib64/libz* -rw-r--r-- 1 root root 119778 Mar 18 18:58 /usr/lib64/libz.a lrwxrwxrwx 1 root root 20 Mar 19 07:51 /usr/lib64/libz.so -> /lib/libz.so.1.2.3.3 lrwxrwxrwx 1 root root 15 Feb 25 17:19 /usr/lib64/libzip.so.1 -> libzip.so.1.0.0 -rw-r--r-- 1 root root 37904 Nov 12 2007 /usr/lib64/libzip.so.1.0.0 This might be obvious, but I have to ask. Bo.
Yes the the libraries are installed, and I believe that the --with-zlib-dir=bundled has gotten me on to a difference error, my yum list shown below. [root@m1330 etc] > yum list|grep libz libzip.x86_64 0.8-5.fc9 installed libzip-devel.x86_64 0.8-5.fc9 installed libzdb.i386 2.4-3.fc10 updates libzdb.x86_64 2.4-3.fc10 updates libzdb-devel.i386 2.4-3.fc10 updates libzdb-devel.x86_64 2.4-3.fc10 updates libzip.i386 0.8-5.fc9 fedora New situation: ../cmd-line-utils/readline/libreadline.a(complete.o): In function `rl_username_completion_function': complete.c:(.text+0x140d): warning: Using 'setpwent' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking ../libmysql/.libs/libmysqlclient.a(mf_pack.o): In function `unpack_dirname': mf_pack.c:(.text+0x988): warning: Using 'endpwent' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking ../libmysql/.libs/libmysqlclient.a(my_gethostbyname.o): In function `my_gethostbyname_r': my_gethostbyname.c:(.text+0x10): warning: Using 'gethostbyname_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking ../libmysql/.libs/libmysqlclient.a(libmysql.o): In function `mysql_server_init': libmysql.c:(.text+0x418d): warning: Using 'getservbyname' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking ../cmd-line-utils/readline/libreadline.a(display.o): In function `_rl_move_vert': display.c:(.text+0x282): undefined reference to `tputs' display.c:(.text+0x2e5): undefined reference to `tputs' ../cmd-line-utils/readline/libreadline.a(display.o): In function `_rl_move_cursor_relative': display.c:(.text+0x5f2): undefined reference to `tputs' ../cmd-line-utils/readline/libreadline.a(display.o): In function `_rl_erase_entire_line': display.c:(.text+0x122b): undefined reference to `tputs' display.c:(.text+0x1250): undefined reference to `tputs' ../cmd-line-utils/readline/libreadline.a(display.o):display.c:(.text+0x126b): more undefined references to `tputs' follow ../cmd-line-utils/readline/libreadline.a(display.o): In function `update_line': display.c:(.text+0x2134): undefined reference to `tgoto' display.c:(.text+0x2146): undefined reference to `tputs' display.c:(.text+0x220f): undefined reference to `tputs' display.c:(.text+0x2351): undefined reference to `tputs' display.c:(.text+0x2372): undefined reference to `tputs' display.c:(.text+0x23e5): undefined reference to `tputs' display.c:(.text+0x2486): undefined reference to `tgoto' display.c:(.text+0x2496): undefined reference to `tputs' display.c:(.text+0x24aa): undefined reference to `tputs' ../cmd-line-utils/readline/libreadline.a(display.o): In function `rl_redisplay': display.c:(.text+0x31c2): undefined reference to `tputs' display.c:(.text+0x31f3): undefined reference to `tputs' display.c:(.text+0x322f): undefined reference to `tputs' ../cmd-line-utils/readline/libreadline.a(display.o):display.c:(.text+0x32d0): more undefined references to `tputs' follow ../cmd-line-utils/readline/libreadline.a(terminal.o): In function `_rl_get_screen_size': terminal.c:(.text+0x437): undefined reference to `tgetnum' terminal.c:(.text+0x4b3): undefined reference to `tgetnum' ../cmd-line-utils/readline/libreadline.a(terminal.o): In function `_rl_init_terminal_io': terminal.c:(.text+0x757): undefined reference to `PC' terminal.c:(.text+0x76a): undefined reference to `BC' terminal.c:(.text+0x775): undefined reference to `UP' terminal.c:(.text+0x7ff): undefined reference to `tgetent' terminal.c:(.text+0x820): undefined reference to `tgetstr' terminal.c:(.text+0x85c): undefined reference to `PC' terminal.c:(.text+0x863): undefined reference to `BC' terminal.c:(.text+0x871): undefined reference to `UP' terminal.c:(.text+0x881): undefined reference to `tgetflag' terminal.c:(.text+0x8d0): undefined reference to `tgetflag' terminal.c:(.text+0xa96): undefined reference to `tgetflag' terminal.c:(.text+0xb06): undefined reference to `tgetflag' ../cmd-line-utils/readline/libreadline.a(terminal.o): In function `_rl_set_cursor': terminal.c:(.text+0x50): undefined reference to `tputs' terminal.c:(.text+0x73): undefined reference to `tputs' ../cmd-line-utils/readline/libreadline.a(terminal.o): In function `_rl_control_keypad': terminal.c:(.text+0x9b): undefined reference to `tputs' ../cmd-line-utils/readline/libreadline.a(terminal.o): In function `_rl_enable_meta_key': terminal.c:(.text+0xd1): undefined reference to `tputs' collect2: ld returned 1 exit status make[2]: *** [mysql] Error 1 make[2]: Leaving directory `/u02/devl/mysql-5.1.34/client' make[1]: *** [all] Error 2 make[1]: Leaving directory `/u02/devl/mysql-5.1.34/client' make: *** [all-recursive] Error 1 [dkehn@m1330 mysql-5.1.34] > On Mon, May 4, 2009 at 9:18 AM, Bo Thorsen <bo@sonofthor.dk> wrote:
On Monday 04 May 2009 16:57:42 Kristian Nielsen wrote:
Don Kehn <dekehn@gmail.com> writes:
when compiling on x64 systems (i.e. CentOS & Fedora core 10) and using the BUILD/compile-pentium64-max, I'm getting the -lz error "/usr/bin/ld: cannot find -lz" and when I simplify the ./comfigure line to a --without-zlib it
Hm, could not reproduce this locally. This was using BUILD/compile-pentium64-max on a 64-bit Ubuntu Hardy (amd64).
64 bit libraries are handled differently between these two. Debian/Ubuntu puts them in /usr/lib (and has a link from /usr/lib64) and CentOS/Fedora/Red Hat/OpenSUSE puts them in /usr/lib64 (/usr/lib is 32 bit).
So to reproduce it, you need a distribution that does the same thing as Dons. I only have Kubuntu machines here, so I can't test it.
Don, just to be absolutely certain: You do have the zlib devel package installed, right? There should be something like this in your /usr/lib64:
shrek> ll /usr/lib64/libz* -rw-r--r-- 1 root root 119778 Mar 18 18:58 /usr/lib64/libz.a lrwxrwxrwx 1 root root 20 Mar 19 07:51 /usr/lib64/libz.so -> /lib/libz.so.1.2.3.3 lrwxrwxrwx 1 root root 15 Feb 25 17:19 /usr/lib64/libzip.so.1 -> libzip.so.1.0.0 -rw-r--r-- 1 root root 37904 Nov 12 2007 /usr/lib64/libzip.so.1.0.0
This might be obvious, but I have to ask.
Bo.
On Monday 04 May 2009 17:22:57 Don Kehn wrote:
Yes the the libraries are installed, and I believe that the --with-zlib-dir=bundled has gotten me on to a difference error, my yum list shown below.
[root@m1330 etc] > yum list|grep libz libzip.x86_64 0.8-5.fc9 installed libzip-devel.x86_64 0.8-5.fc9 installed libzdb.i386 2.4-3.fc10 updates libzdb.x86_64 2.4-3.fc10 updates libzdb-devel.i386 2.4-3.fc10 updates libzdb-devel.x86_64 2.4-3.fc10 updates libzip.i386 0.8-5.fc9 fedora
The library packages are usually called zlib and zlib-devel. libzip is a different library. Bo.
Sorry: [root@m1330 etc] > yum list|grep zlib zlib.i386 1.2.3-18.fc9 installed zlib.x86_64 1.2.3-18.fc9 installed zlib-devel.x86_64 1.2.3-18.fc9 installed On Mon, May 4, 2009 at 9:26 AM, Bo Thorsen <bo@sonofthor.dk> wrote:
Yes the the libraries are installed, and I believe that the --with-zlib-dir=bundled has gotten me on to a difference error, my yum
On Monday 04 May 2009 17:22:57 Don Kehn wrote: list
shown below.
[root@m1330 etc] > yum list|grep libz libzip.x86_64 0.8-5.fc9 installed libzip-devel.x86_64 0.8-5.fc9 installed libzdb.i386 2.4-3.fc10 updates libzdb.x86_64 2.4-3.fc10 updates libzdb-devel.i386 2.4-3.fc10 updates libzdb-devel.x86_64 2.4-3.fc10 updates libzip.i386 0.8-5.fc9 fedora
The library packages are usually called zlib and zlib-devel. libzip is a different library.
Bo.
Don Kehn <dekehn@gmail.com> writes:
display.c:(.text+0x282): undefined reference to `tputs'
This sounds like you are missing curses development libraries. On my machine (Ubuntu Hardy), I think they are in the package libncurses5-dev. Hopefully this will get you one step further. (Btw, on Ubuntu/Debian, a quick shortcut to get all required dependencies installed is the following: sudo apt-get build-dep mysql-server Maybe your system has something similar). - Kristian.
Got the ncurses development libs loaded - intersting if I do a 32-bit version all works as it should, but when I do a ./BUILDcompile-pentium64-max I start seeing the -lz error. I feel like this is a library malfunction. On Tue, May 5, 2009 at 7:41 AM, Kristian Nielsen <knielsen@knielsen-hq.org>wrote:
Don Kehn <dekehn@gmail.com> writes:
display.c:(.text+0x282): undefined reference to `tputs'
This sounds like you are missing curses development libraries.
On my machine (Ubuntu Hardy), I think they are in the package libncurses5-dev.
Hopefully this will get you one step further.
(Btw, on Ubuntu/Debian, a quick shortcut to get all required dependencies installed is the following:
sudo apt-get build-dep mysql-server
Maybe your system has something similar).
- Kristian.
This is how I got it to work and there maybe an issue here: using the BUILD/compile-pentium64-max, changed the changed the extra_configs="$pentium_configs $max_configs $static_link" to extra_configs="$pentium_configs $max_configs" (notice the static_link is removed). It is trying to link with --all-static (thus it needs zlib.a libssl.a .....), which is a pain in the ars, I'm wondering if this is an issue for the build team. I believe this is the only *max that has the static_link. libtool ... uses --all-static which leads to the gcc -static ....-lz -lpthread, anything on the commandline after the -static MUST be statically linked thus is not found in my case. Have confirmed this on two different fedora core 10 - 64-bit systems. Anyways thanks all for the input. I finally have a build - on to the next.................. On Tue, May 5, 2009 at 1:14 PM, Don Kehn <dekehn@gmail.com> wrote:
Got the ncurses development libs loaded - intersting if I do a 32-bit version all works as it should, but when I do a ./BUILDcompile-pentium64-max I start seeing the -lz error. I feel like this is a library malfunction.
On Tue, May 5, 2009 at 7:41 AM, Kristian Nielsen <knielsen@knielsen-hq.org
wrote:
Don Kehn <dekehn@gmail.com> writes:
display.c:(.text+0x282): undefined reference to `tputs'
This sounds like you are missing curses development libraries.
On my machine (Ubuntu Hardy), I think they are in the package libncurses5-dev.
Hopefully this will get you one step further.
(Btw, on Ubuntu/Debian, a quick shortcut to get all required dependencies installed is the following:
sudo apt-get build-dep mysql-server
Maybe your system has something similar).
- Kristian.
Don Kehn <dekehn@gmail.com> writes:
This is how I got it to work and there maybe an issue here: using the BUILD/compile-pentium64-max, changed the changed the extra_configs="$pentium_configs $max_configs $static_link" to extra_configs="$pentium_configs $max_configs" (notice the static_link is removed). It is trying to link with --all-static (thus it needs zlib.a libssl.a .....), which is a pain in the ars, I'm wondering if this is an issue for the build team. I believe this is the only *max that has the static_link. libtool ... uses --all-static which leads to the gcc -static ....-lz -lpthread, anything on the commandline after the -static MUST be statically linked thus is not found in my case. Have confirmed this on two different fedora core 10 - 64-bit systems.
Ah, thanks for digging up this, I think I understand now. Probably on your system, you have libz.so but no libz.a. The $static_link expands to --with-mysqld-ldflags=-all-static --with-client-ldflags=-all-static These are extra linker flags added when linking specific executables. So they are _not_ added generally to $LDFLAGS in configure.in (and anyway the checks for --with-mysqld-ldflags and --with-client-ldflags are after the check for libz in configure.in). So these flags are not used when checking for libz, and hence ./configure decides it can use the system libz. But when it then gets to actually link mysqld, it fails due to missing static library for libz. So the BUILD/compile-pentium64-max script is probably not used much. For development people generally will use BUILD/compile-pentium64-debug-max instead, and the release builds are not made with the scripts in BUILD/. That is probably the reason this has not been fixed yet. So the work-around for you could be to just use BUILD/compile-pentium64-debug-max, or remove the $static_link, or add --with-zlib-dir=bundled. Still, it is a bug in configure.in that it wrongly decides to use the system libz when that doesn't actually work when trying to link. It should use the same flags when detecting libz as when actually linking the mysqld and other binaries. So actually the libz check needs to do check 3 different linker options (plain, adding --with-mysqld-ldflags, and adding --with-client-ldflags), and only if system libz works in all three cases should it use that in preference to the bundled. I'll see if I can come up with a patch for this at some point, I may ask you to test it later. (Not sure how important this is. Linking mysqld statically was used historically, but these days there are so many problems with fully static binaries that it does not really work anymore. Still, I would like to get it fixed, I strongly favour ./configure just doing the right thing, MariaDB should compile out of the box on as many systems as possible). Thanks again for your help in getting to the bottom of this! - Kristian.
Hm, so I looked into this a bit further, and it seems not so easy to fix... The problem is that -all-static is really a libtool special option, but configure checks do not use libtool. So configure cannot easily in a generic may make libtool options set by the user affect its decision. In fact, I think passing -all-static to libtool via --with-mysqld-ldflags in the way the MySQL source does is heavy abuse of libtool (also according to automake/libtool docs). But that is another matter... My conclusion is that this should not be fixed in ./configure. If someone decides to pass in -all-static, they should have the responsibility to add any extra options needed to make linking -all-static actually work. In this case --with-zlib-dir=bundled. (Another issue is that your CentOS/Fedora should maybe provide both libz.so and libz.a, don't understand why it does not? But that is another discussion). So the fix I suggest is to simply add --with-zlib-dir=bundled in BUILD/compile-pentium64-max (I already committed this change for review). Or even better, avoid using -all-static, which really creates more problems than it solves (but that also is another discussion). Thanks again, - Kristian. Kristian Nielsen <knielsen@knielsen-hq.org> writes:
Don Kehn <dekehn@gmail.com> writes:
This is how I got it to work and there maybe an issue here: using the BUILD/compile-pentium64-max, changed the changed the extra_configs="$pentium_configs $max_configs $static_link" to extra_configs="$pentium_configs $max_configs" (notice the static_link is removed). It is trying to link with --all-static (thus it needs zlib.a libssl.a .....), which is a pain in the ars, I'm wondering if this is an issue for the build team. I believe this is the only *max that has the static_link. libtool ... uses --all-static which leads to the gcc -static ....-lz -lpthread, anything on the commandline after the -static MUST be statically linked thus is not found in my case. Have confirmed this on two different fedora core 10 - 64-bit systems.
Ah, thanks for digging up this, I think I understand now.
Probably on your system, you have libz.so but no libz.a.
The $static_link expands to
--with-mysqld-ldflags=-all-static --with-client-ldflags=-all-static
These are extra linker flags added when linking specific executables. So they are _not_ added generally to $LDFLAGS in configure.in (and anyway the checks for --with-mysqld-ldflags and --with-client-ldflags are after the check for libz in configure.in). So these flags are not used when checking for libz, and hence ./configure decides it can use the system libz. But when it then gets to actually link mysqld, it fails due to missing static library for libz.
So the BUILD/compile-pentium64-max script is probably not used much. For development people generally will use BUILD/compile-pentium64-debug-max instead, and the release builds are not made with the scripts in BUILD/. That is probably the reason this has not been fixed yet.
So the work-around for you could be to just use BUILD/compile-pentium64-debug-max, or remove the $static_link, or add --with-zlib-dir=bundled.
Still, it is a bug in configure.in that it wrongly decides to use the system libz when that doesn't actually work when trying to link. It should use the same flags when detecting libz as when actually linking the mysqld and other binaries.
So actually the libz check needs to do check 3 different linker options (plain, adding --with-mysqld-ldflags, and adding --with-client-ldflags), and only if system libz works in all three cases should it use that in preference to the bundled.
I'll see if I can come up with a patch for this at some point, I may ask you to test it later.
(Not sure how important this is. Linking mysqld statically was used historically, but these days there are so many problems with fully static binaries that it does not really work anymore. Still, I would like to get it fixed, I strongly favour ./configure just doing the right thing, MariaDB should compile out of the box on as many systems as possible).
Hi!
"Kristian" == Kristian Nielsen <knielsen@knielsen-hq.org> writes:
<cut> Kristian> So the fix I suggest is to simply add --with-zlib-dir=bundled in Kristian> BUILD/compile-pentium64-max (I already committed this change for review). Kristian> Or even better, avoid using -all-static, which really creates more problems Kristian> than it solves (but that also is another discussion). The reason for -all-static are the following: - Slightly faster binary (2-3 % last time I measured) - The binary can be moved between different system (as long as they have the same architecture) - The binary will not stop working during a system upgrade. The above is the reason I prefer to make critical application staticly linked. Regards, Monty
participants (4)
-
Bo Thorsen -
Don Kehn -
Kristian Nielsen -
Michael Widenius