[Maria-developers] ccache with mariadb?
Hello! Has somebody managed to configure ccache correctly so that it would work with MariaDB compilation? https://ccache.samba.org/ I don't seem to get it working as the build always seems to pick up the real gcc binary instead of any of the ccache symlink wrappers I've manually put in place to replace all of gcc, cc1, cc1plus etc.
Hi, Otto! On Dec 23, Otto Kekäläinen wrote:
Hello!
Has somebody managed to configure ccache correctly so that it would work with MariaDB compilation?
I don't seem to get it working as the build always seems to pick up the real gcc binary instead of any of the ccache symlink wrappers I've manually put in place to replace all of gcc, cc1, cc1plus etc.
Sure. See our old build scripts in BUILD/* there's ccache all over. They're from autotool-times, but I believe Monty is still using them. So one can use ccache with cmake. Our build scripts to CC='ccache gcc' CXX='ccache g++'. That's actually not a good idea, a compiler with the space in the middle kind of works in cmake (there's special support for this use case), but I've hit rough edges. But using ccache symlink wrappers should generally work with no problems at all. I remember trying that and it worked. Try CC=gcc CXX=g++ before running cmake. I don't know exactly in what locations cmake is using for a compiler and why it doesn't pick up your symlinks automatically. Regards, Sergei
2014-12-24 14:34 GMT+02:00 Sergei Golubchik <serg@mariadb.org>:
Try CC=gcc CXX=g++ before running cmake.
I don't know exactly in what locations cmake is using for a compiler and why it doesn't pick up your symlinks automatically.
Ok, I managed to figure it out now. The solution was to create this two symbolic links: ll /usr/local/bin/x86_64-linux-gnu-g* lrwxrwxrwx 1 root root 15 joulu 27 17:01 /usr/local/bin/x86_64-linux-gnu-g++ -> /usr/bin/ccache* lrwxrwxrwx 1 root root 15 joulu 27 17:02 /usr/local/bin/x86_64-linux-gnu-gcc -> /usr/bin/ccache* The problem was in two parts: First I thought that the cacheable commands was cc1 and cc1plus, since that are the names of the binaries where the build process spends most of its time. Turns out those cannot be handled by ccache. Instead I needed to wrap the step above them and that meant the invocation of gcc via the name x86_64-linux-gnu-g*. Second issue was that the git-buildpackage fakeroot didn't inherit the PATH of the parent environment, so I instead of manipulating the PATH I simply added the symbolic links to a location where they are read and triggered before /usr/bin, and /usr/local/bin was such a place. The time saving is huge. My full git-buildpackage run went down from 25-33 minutes to about 4 minutes. The size of the cache is less than 0,5 gigs, so I could maybe make it even more faster by placing the cache on a ramdisk: $ ccache -s cache directory /var/cache/ccache [..] files in cache 3325 cache size 426.3 Mbytes max cache size 2.0 Gbytes By the way, why does not the buildbot infrastructure use ccache? It could have a common nfs mounted /var/ccache directory that all the builds share that could presists from build-to-build? Most of the commits just touch a few files so ccache would be able to save lots of compilation time as only the changed files would be compiled.
Hi, Otto! On Dec 28, Otto Kekäläinen wrote:
The time saving is huge. My full git-buildpackage run went down from 25-33 minutes to about 4 minutes.
...
By the way, why does not the buildbot infrastructure use ccache? It could have a common nfs mounted /var/ccache directory that all the builds share that could presists from build-to-build? Most of the commits just touch a few files so ccache would be able to save lots of compilation time as only the changed files would be compiled.
Because nobody implemented that. And, I suspect, that's because compiling is the fastest step in buildbot already. Downloading the sources (from buildmaster to buildslaves) and uploading packages (back to buildmaster) used to be the slowest step (1-2 hours was not uncommon, 3 hours was possible). Testing usually takes about half an hour. Compiling with its mere 15 minutes is simply not worth optimizing. I've recently fixed the downloading step - it wasn't network at all, something was wrong on twisted or buildbot level. Now I need to fix uploading. Then testing will be next in line - we already have a solution for that, GSoC project of this year, implemented by Pablo Estrada with Elena being the mentor. And only when testing will be taking ~5 minutes (ok, below 10), then one can start thinking about optimizing the build. Otherwise that won't make any noticeable difference to the buildbot throughput. Regards, Sergei
participants (2)
-
Otto Kekäläinen -
Sergei Golubchik