[Maria-developers] Cassandra SE: problems including it into release
Hello, I've figured there may be an additional challenge in getting Cassandra storage engine into a *MariaDB release*. The problem is packaging. <contents> Cassandra SE packaging problems Need Thrift library Need recent gcc How did Sphinx SE manage to avoid a similar mess? Possible way out </contents> == Cassandra SE packaging problems == === Need recent gcc === C++ code generated by Thrift uses boost, and so one needs a recent gcc to compile it with MySQL. We have the same issue with oqgraph. See MDEV-501 for details. I am not aware of crashing problems that ograph+boost combination has. Still, recent-gcc requirement may limit the range of platforms we can build for. For example, I never managed to compile on Ubuntu 9.04. === Need Thrift library === Cassandra SE uses Thrift protocol to talk to server. In order to do this, it needs to link againist libthrift.so. There are no Thrift packages in either Fedora or Ubuntu (actually I haven't seen Thrift packages for any OS). Thrift homepage only offers source tarballs. The source tarball uses automake and compiles into 1. thrift compiler binary (we dont actually need it. Source code generated by thrift needs to be modified anyway.) 2. thrift libraries for a dozen of languages, including libthrift.so, which we do need. == How did Sphinx SE manage to avoid a similar mess? == I've asked myself a question how do we manage to include Sphinx SE without having to mess with Sphinx packages. The answer is: Sphinx SE doesn't need any sphinx libraries, either at compile time, or at runtime. It implements its client protocol fully, except for INSERT commands, for which it uses the available mysql client library to talk to Sphinx server using mysql client-server protocol. == An unlikely but possible way out == Cassandra 1.2 (the next version that is promised to be released "in Q4 2012" will introduce native network protocol. A week ago they've pushed first documentation draft. The native network protocol should allow to do everything that Thrift API allows, and more. There are no C++ clients yet, though. If I am faced with a choice betwen 1. implementing a relatively simple network protocol 2. Making Thrift packages for various platforms (and potentially convincing downstream maintainers to accept them) then I will definitely pick #1 (I have implemented network protocols in the past). There is a big risk, though: native protocol support is very new in Cassandra, they pushed it about a week ago. It was pushed into their "trunk" which is under development. I don't know if the code in their trunk is sufficiently stable for us to develop againist. BR Sergei -- Sergei Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog
On Fri, Sep 21, 2012 at 12:27:31PM +0400, Sergei Petrunia wrote:
There is a big risk, though: native protocol support is very new in Cassandra, they pushed it about a week ago. It was pushed into their "trunk" which is under development. I don't know if the code in their trunk is sufficiently stable for us to develop againist.
news on this front: today they started to vote whether to release 1.2.0-beta1 BR Sergei -- Sergei Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog
Hi, Sergei! On Sep 21, Sergei Petrunia wrote:
Hello,
I've figured there may be an additional challenge in getting Cassandra storage engine into a *MariaDB release*. The problem is packaging.
Do we have to do that soon? May be, making a tarball would be enough? Or a tarball with the engine only (without the mysqld and supporting files). And then we'll bundle it into all packages when Cassandra will have its network protocol releases. Regards, Sergei
Hi!
"Sergei" == Sergei Petrunia <psergey@askmonty.org> writes:
Sergei> Hello, Sergei> I've figured there may be an additional challenge in getting Cassandra Sergei> storage engine into a *MariaDB release*. The problem is packaging. Sergei> <contents> <cut> Sergei> If I am faced with a choice betwen Sergei> 1. implementing a relatively simple network protocol Sergei> 2. Making Thrift packages for various platforms (and potentially convincing Sergei> downstream maintainers to accept them) Sergei> then I will definitely pick #1 (I have implemented network protocols in the Sergei> past). Sergei> There is a big risk, though: native protocol support is very new in Cassandra, Sergei> they pushed it about a week ago. It was pushed into their "trunk" which is Sergei> under development. I don't know if the code in their trunk is sufficiently Sergei> stable for us to develop againist. There is another solution: - Do 2) for one very common platform and release this as a 'lab' release. This should be enough for us to get money from Tekes. - As soon we know that #1 is a working solution and we have got people interested in testing the #1 solution, we can then go trough route #1. The benefits are that we spend less hours developing against a probably not stable protocol. Regards, Monty
participants (3)
-
Michael Widenius
-
Sergei Golubchik
-
Sergei Petrunia