Hi Kristian On 26/03/2010, at 7:13 PM, Kristian Nielsen wrote:

I wanted to see if there was something I could do to help getting oqgraph-in-mariadb moving again.

Great!

As I understand it, the issues are the following:

1. Current OQGraph autoconf test enables OQGraph by default in the build if a too old Boost is installed on the machine. This means that a plain ./configure && make will not work on such machines. (In other words, OQGraph must not be enabled by default on machines where it does not compile).

2. OQGraph is affected by a bug in Boost, so it actually requires a patched Boost (until some Boost version is released with the fix included).

3. Even if OQGraph is included properly into the MariaDB source tree, we still need to set up package building so that it will actually be able to build and include OQGraph.

So one possible solution for issue 3 is to include the patched boost headers inside the storage/oqgraph/ directory. I remember that including all of boost was infeasible due to the sheer size of the library. But you mentioned that just the graph part of Boost would do. How big is this subset (number of files / number of lines?).

Nop. Didn't say that. Antony and I thought it might be feasible to simply grab the files from Boost that would be required, reducing the # of files significantly. However, the way the Boost infra works it still requires heaps of files, with conditions based on OS, CPU and compiler platform, etc.

Another solution would be to create an ourdelta package of the patched boost development headers. We could then make this a build dependency of mariadb .debs. But not sure if this is too much work to set up (and not sure about how this would work for .deb either).

That's work for every distro platform: DEB, RPM, etc. Fussy.

A third solution could be to just install the patched boost in /usr/local/include on the build VMs. This should work, though it's not 100% nice in terms of providing full source code (you could always release OQGraph as GPL with a boost linking exception if we're really paranoid).

I don't see this at all. OQGRAPH is already GPL. And the patched library is in the oqgraph project repos on Launchpad. So it's all there anyway.

A fourth solution could be to include the graph part of boost (with bug fix) as a patch in ourdelta/bakery/, just like the other patches already there.

That's possible. As you know, we build a source tarball, and then everything builds from there. So as long as we toss in Boost before we wrap up the source tarball, we're good. Then it's not in the MariaDB repos, but it's all there.

For issue 1 + 2, if you can point me to what the bug in boost is I could look into making an autoconf test for this bug, and only enable OQGraph by default if a boost without the bug is found. Or failing that, if you can supply me with the minimum version of boost needed, I could try for an autoconf test that does not enable OQGraph by defaults on hosts where it breaks the build.

That would simply mean that OQGRAPH does not get built anyway. As I understand from Antony, there currently is no Boost version without the issues, hence him doing the fixes. Of course we'll contribute those upstream, but you understand the lag with that going through into distros; and that'd mostly be future distro versions, not existing/past ones that we build for. So while the upstream contribution is important on its own, it's not really related to us doing our builds. Regards, Arjen. -- Arjen Lentz, Exec.Director @ Open Query (http://openquery.com) Exceptional Services for MySQL at a fixed budget. Follow our blog at http://openquery.com/blog/ OurDelta: packages for MySQL and MariaDB @ http://ourdelta.org

Kristian Nielsen <knielsen@knielsen-hq.org> writes:

I wanted to see if there was something I could do to help getting oqgraph-in-mariadb moving again.

Ok, I got some progress on this. But I have problems with the packaging, need help to proceed. The OQGraph engine seems to be built as a module, oqgraph_engine.so. So for packaging we need to (at least) 1. Include the oqgraph_engine.so somewhere 2. Provide some facilities/documentation for the user to load the engine (or should it be added by default into the mysql.plugin table?) I am going to need some help with this, I do not know .deb and .rpm well enough to know how to proceed with this. It seems plugins are normally installed in /usr/lib/mysql/plugin, for the .deb stuff this directory belongs to the libmariadbclient-dev package. That's not really going to work of course, seems something will have to move around? So what were your ideas for oqgraph with respect to placement/packaging of oqgraph_engine.so and configuring/loading the plugin? (A third thing that is needed is to add the mysql-test/suite/oqgraph stuff to the mariadb-test-5.1 package, but that is easy). ----------------------------------------------------------------------- Things I did so far: 1. I found that the fixes for boost that were made in the ourdelta stuff are now upstream, from boost 1.40.0 on. So we do not need anything fancy for boost, I just make plug.in test for boost >= 1.40.0 (thanks to Serg for helping with this). 2. I added a short README explaining the need for Boost >= 1.40.0 3. I installed boost 1.42.0 on the package build hosts. 4. I fixed an include of a deprecated Boost header. 5. I commented out the DTRACE stuff in Makefile.am, as it breaks on non-dtrace platforms. I see the same stuff in federatedx, also commented out, and the probes.d looks empty anyway (you can sort it out later if you want to re-enable it). 6. I pushed my fixed tree here if you need to look: lp:~knielsen/maria/mariadb-5.1-oqgraph 7. I am running a Buildbot test of the tree here: https://buildbot.askmonty.org/buildbot/grid?branch=mariadb-5.1-knielsen But I'm kind of stuck on the packaging. - Kristian.