- developers - lists.mariadb.org

Re: [Maria-developers] [Commits] Rev 3657: MDEV-4127 : Export additional symbols when building RPM
by Sergei Golubchik 04 Feb '13

04 Feb '13

Hi, Vladislav! May be you'd rather rename symbols with #define, just as redhat does? On Feb 03, Vladislav Vaintroub wrote: > At file:///H:/bzr/5.5/ > > ------------------------------------------------------------ > revno: 3657 > revision-id: wlad(a)montyprogram.com-20130203164516-ex2ftmyca5b507g3 > parent: elenst(a)ubuntu11.home-20130202225357-za9bglcf2ngv36xj > committer: Vladislav Vaintroub <wlad(a)montyprogram.com> > branch nick: 5.5 > timestamp: Sun 2013-02-03 17:45:16 +0100 > message: > MDEV-4127 : Export additional symbols when building RPM, > to enable both recompiling mysqli or odbc from sources in addition > to drop-in replacement functionality. > > The case in question is compiling mysqli from sources, that needs > client_errors via ER() macro. > > Previously, we exported it as mysql_client_errors (compatibly to > Fedora's style symbol renaming, see MDEV-3842). > However, if MariaDB header files are used when compiling mysqli, > client_errors needs to be exported with its original name. Regards, Sergei

2 2

[Maria-developers] Coverity scans of MariaDB source
by Christian Convey 04 Feb '13

04 Feb '13

Coverity has a deal called Coverity Scan, where they offer free static analyses of open-source programs (http://scan.coverity.com/) I've used Coverity Prevent at work, and I was pretty happy with the analyses it performed. Is it an intentional choice that MariaDB isn't making use of Coverity Scan? I'm interested in starting to contribute to MariaDB, and I'm wondering if it would be useful to the project for me to pursue this as a first task. - Christian

2 2

[Maria-developers] CMake vs. autotools on Linux?
by Christian Convey 04 Feb '13

04 Feb '13

Hi guys, I noticed that MariaDB is built (indirectly) with CMake on Windows, and with autotools on Linux. Why are both systems used? I would have assumed that because CMake is pretty good at generating Makefiles on Linux and OS X, there would be little reason to continue using autotools. Thanks, Christian

2 3

Re: [Maria-developers] Summary of the MDEV-3882 (.deb upgrade from mysql to mariadb fails due to apt-get considering it a downgrade)
by Kristian Nielsen 02 Feb '13

02 Feb '13

[Moving the discussion to maria-developers@, please use the public mailing lists for discussions like this] Alexey Botchkov <holyfoot(a)askmonty.org> writes: > No matter what dependencies or new packages i created inside MariaDB, > Debian's first idea was to try to install the 'later' version of > libmysqlclient18. At the moment > that was the Oracle's one. Yes, we could change our version's number to be > 'later' than Oracle's, but that would turn same problems on MySQL, and Not really. It's not "Oracle's", it is the native Debian/Ubuntu package. So if users want MariaDB, they can add our repository. If they want the official Debian/Ubuntu package, they can just not add our repository. So it makes sense that when users explicitly add the MariaDB repository, they get the MariaDB version of all the packages automatically. > would help us only temporarely - to the moment Oracle releases even > later' version. We can use epoch to ensure that our version will always be bigger than the official Debian/Ubuntu version of MySQL: http://www.debian.org/doc/debian-policy/ch-controlfields.html#s-f-Version Something like 1:5.5.29-mariadb1~wheezy. This will always be prefered by apt-get over any official Debian package. > I didn't find anything that could affect apt-get and guys on #Debian > didn't recommend anything. I believe it works to specify desired version explicitly, as mentioned in the bug report as workaround: sudo apt-get install mariadb-server libmysqlclient18=5.5.29-mariadb1~wheezy > So for now i'd do about this: > - change nothing in the existing mariadb-produced debian packages. > - recommend to use 'aptitude' installing the mariadb-server on Debian. > So how do you like my proposal? Any other ideas? I think it is better if we use the epoch to make MariaDB upgrades work automatically with both apt-get and aptitude, like they used to. Did you check how Percona handles the similar problem? - Kristian.

2 1

Re: [Maria-developers] Fwd: [Commits] Rev 3622: Analysis: in file:///home/tsk/mprog/src/5.3-md641/
by Igor Babaev 31 Jan '13

31 Jan '13

On 01/31/2013 04:41 AM, Timour Katchaounov wrote: > Igor, > > Let me know if I can push the patch below. It fixes the bug > according to our last discussion. If the number of group_list > elements was known, I could add a test already in the beginning > of the function, however such a counter doesn't exist, and I > didn't want to complicate the code just for this special case. > That's why the test is done where it is now. > > Timour > > > ------------------------------------------------------------ > revno: 3622 > revision-id: timour(a)askmonty.org-20130131120040-oan9zei8ojyco178 > parent: sergii(a)pisem.net-20130128081223-mp9rsd3t9soz8lly > fixes bug(s): https://mariadb.atlassian.net/browse/MDEV-641 > committer: timour(a)askmonty.org > branch nick: 5.3-md641 > timestamp: Thu 2013-01-31 14:00:40 +0200 > message: > Analysis: > Range analysis discoveres that the query can be executed via loose > index scan for GROUP BY. > Later, GROUP BY analysis fails to confirm that the GROUP operation can > be computed via an > index because there is no logic to handle duplicate field references > in the GROUP clause. > As a result the optimizer the optimizer produces an inconsistent plan. > It constructs a > temporary table, but on the other hand the group fields are not set to > point there. > > Solution: > Make loose scan analysis work in sync with order by analysis. In the > case of duplicate > columns loose scan will not be applicable. This limitation will be > lifted in 10.0 by > removing duplicate columns. > >

1 0

Re: [Maria-developers] what pros/cons of storing binary log in an InnoDB table?
by Kristian Nielsen 31 Jan '13

31 Jan '13

Zardosht Kasheff <zardosht(a)gmail.com> writes: > Instead of having the binary log be stored as a log file, a table with > a transactional engine (like InnoDB) is used. I have NOT done any Yes, this is a tempting idea, is it not? Drizzle even implemented this. But there are some subtle problems, and I think there are better solutions. One problem you will find if you try implementing this is - how are you going to keep the concept of binlog order? A relational table is unordered. You can use an increasing uint64 as the primary key, but then how are you going to ensure that the order in the table is the same as the commit order in InnoDB (InnoDB redo log)? If you take a lock from allocating the sequence number to doing the commit, then you have introduced a serialisation bottleneck and killed group commit. If you accept different order (as Drizzle does), then when you take an XtraBackup, it may not be consistent with the binlog. You will be writing your data *6* times: To the real table, when flushing its buffer pool pages. To the double-write buffer while flushing, and to the redo log before flushing. Then to the binlog table, and before that to the redo log and the doublewrite buffer. I could never reconcile myself with this. So it is tempting, but re-implementing binlog is a huge change (also in terms of end-user visibility), and my personal opinion is that for such huge a change we need to go to the best solution, not second-best. > investigation into whether this is possible, but I'm wondering what > people abstractly think of this idea. > My major motivation is to find a way to have users not be forced to > fsync on every transaction commit on slaves. With crash safety of My preference is MWL#164: http://askmonty.org/worklog/Server-RawIdeaBin/?tid=164 This reduces to one fsync() per group commit. But as you point out, this is still horrendously expensive for a single-threaded slave :-( Abstractly, the proper solution is to store the binlog and the InnoDB redo log (and all other transactional-type logs) in the *same* log. The upper layer would provide a general transactional log facility, and binlog, InnoDB, partitioning, etc. would use this instead of their own logs. But it is hard to see how we could arrive there from current code ... For MariaDB global transaction ID, I support crash-safe without binlog enabled, so there turning off fsync() will be possible. But this does not help if a slave also needs to act as a master. Parallel replication can help a lot, by allowing group commit, amortising the cost of the fsync(). But it would still be nice to solve the orginal problem. As you say, for a slave it is not a problem to loose transactions in a crash as they can be just replicated again. The problem is to be consistent between what is in the InnoDB tables and what is in the binlog. InnoDB already records in a crash-safe way the binlog position of the last commit. So if we can ensure that InnoDB is never ahead of the binlog, we can just cut the binlog after a crash if it has more events than what InnoDB recovered. We could do this by doing binlog writes as async I/O O_DIRECT writes, and sending a signal somehow to InnoDB so it would buffer its own redo log until binlog is safe on disk. Then no fsync() would be needed. But this is so far just a loose idea, it needs more thought to be sure it can work... Anyway, this is my current long-term plan: if we cannot have a proper single database-wide transaction log, then instead coordinate writes between the multiple logs that exist. Delay writes to logs so that they are never ahead of the master log. And at crash, either recover missing pieces of logs from the master log (MWL#164), or cut logs to be consistent. Then we can be consistent without any fsync() delays. > - As I understand it, for two phase commit, storage engines are > currently expected to fsync once after a transaction is prepared and > once after a transaction is committed. If the binary log is an InnoDB > table, then two-phase commit is unnecessary and the number of fsyncs > on the system is cut in half. In MariaDB 10 and MySQL 5.6, the fsync() at commit is no longer needed (InnoDB does not have it). [Note that IIRC, MySQL 5.6 breaks the storage engine API, you will need to implement the skip-fsync-at-commit-stage, or you will loose group commit.] > - With crash safe slaves on MySQL 5.6, one theoretically does not > need to enable fsyncs on transaction commit on slaves. If some data > gets lost after a crash, then the data may be replayed from the > master. However, to get GTIDs working, one must enable the binary log > on slaves, which requires two-phase commit, which require fsyncs. By Yes, requiring binlog on slaves is a serious mistake :-( Ok, as you can perhaps tell, this is something I have thought a lot about over the past years, so I have a lot of ideas but it is a bit difficult to put it all in a single mail reply without it becomming mostly a jumble of loose ideas. You are welcome to ask for more details on any specific points of interest. - Kristian.

5 9

Re: [Maria-developers] reducing fsyncs during handlerton->prepare and handlerton->commit in 5.6
by Kristian Nielsen 31 Jan '13

31 Jan '13

Zardosht Kasheff <zardosht(a)gmail.com> writes: > - in MySQL 5.6 and MariaDB 10.0, what are the fsyncing requirements > for storage engines during prepare and commit? > - are there new APIs that the storage engine must comply with in > order to get these benefits? In MariaDB 10.0, there is a new commit_checkpoint_request handlerton method. If the storage engine implements that, it need not do the fsync() at commit. It must also implement the commit_ordered method from 5.3. Check comments in sql/handler.h for these two methods, and let me know if there are any questions. The intention is that storage engines need not do much extra work for this functionality, eg. the patch for InnoDB support was a couple hundred lines at most IIRC. > - in MySQL 5.6, I see new handlerton methods commit_low and > prepare_low. What do these APIs do? What is their contract? I think MySQL developers should answer for this. But it is related to this new check in InnoDB: thd_requested_durability(trx->mysql_thd) == HA_IGNORE_DURABILITY And the existing flush_logs handlerton method is also used. My guess is that commit_low is just internal server stuff that the storage engine does not need to consider. The MySQL 5.6 mechanism is mostly the same as the MariaDB 10.0 one in what it actually does, but they did not bother to try to be compatible with the storage engine API in MariaDB 5.3. So looks like it is #ifdef time for you if you want to support both. - Kristian.

1 0

[Maria-developers] how to build mariadb RPMs
by Rich Prohaska 30 Jan '13

30 Jan '13

How do you build the mariadb RPMs? Are you using rpmbuild or cpack? Is there a top level script that uses rpmbuild or cpack that I should use? Thanks Rich Prohaska

3 11

[Maria-developers] MariaDB 5.1.67, 5.2.14, 5.3.12, 5.5.29 Release Prep
by Daniel Bartholomew 29 Jan '13

29 Jan '13

All, I've prepped changelog and release notes pages for MariaDB 5.1.67, 5.2.14, and 5.3.12. I'll shortly be doing the same for MariaDB 5.5.29. https://kb.askmonty.org/en/mariadb-5167-release-notes/ https://kb.askmonty.org/en/mariadb-5167-changelog/ https://kb.askmonty.org/en/mariadb-5214-release-notes/ https://kb.askmonty.org/en/mariadb-5214-changelog/ https://kb.askmonty.org/en/mariadb-5312-release-notes/ https://kb.askmonty.org/en/mariadb-5312-changelog/ If there is anything I should be sure to mention in the release notes, please let me know ASAP. I plan on announcing the releases once 5.5.29 has been uploaded to the mirrors. It's still building at the moment so the earliest the release could happen is tomorrow and it could be pushed back to Thursday if there's a delay of some sort. Thanks. -- Daniel Bartholomew Google+ - http://gplus.to/dbart Twitter - http://twitter.com/daniel_bart MariaDB: An Enhanced Drop-in Replacement for MySQL Website - http://mariadb.org Twitter - http://twitter.com/mariadb Google+ - http://google.com/+mariadb Facebook - http://fb.com/MariaDB.dbms Knowledgebase - http://kb.askmonty.org Monty Program - http://montyprogram.com

1 0

Re: [Maria-developers] [Commits] Rev 3653: MDEV-3882 .deb upgrade from mysql to mariadb fails due to apt-get considering it a downgrade. in file:///home/hf/wmar/mdev-3882/
by Kristian Nielsen 29 Jan '13

29 Jan '13

<holyfoot(a)askmonty.org> writes: > message: > MDEV-3882 .deb upgrade from mysql to mariadb fails due to apt-get considering it a downgrade. > The libmysqlclient18 package and all the dependencies on it removed. > The libmariadbclient18 set to provide the libmysqlclient's functions and > conflicts with all the libmysqlclients. > > === modified file 'debian/dist/Debian/control' > --- a/debian/dist/Debian/control 2012-08-23 13:32:03 +0000 > +++ b/debian/dist/Debian/control 2013-01-29 09:56:54 +0000 > @@ -13,7 +13,9 @@ Vcs-Bzr: bzr://lp:maria > Package: libmariadbclient18 > Section: libs > Architecture: any > -Depends: mariadb-common, libmysqlclient18 (= ${source:Version}), ${shlibs:Depends}, ${misc:Depends} > +Depends: mariadb-common, ${shlibs:Depends}, ${misc:Depends} > +Provides: libmysqlclient18 > +Conflicts: libmysqlclient10, libmysqlclient12, libmysqlclient14, libmysqlclient15, libmysqlclient16, libmysqlclient18 Are you sure this works? The original reason for introducing a libmysqlclient18 package in the first place (with all the problems in introduces) is that there are standard packages with a versioned dependency on libmysqlclient18. And versioned dependencies can not (used not to?) be satisfied by a Provides: Eg. the following gives 143 packages with versioned dependencies on libmysqlclient18: apt-cache rdepends libmysqlclient18|(read x; read x; while read x; do apt-cache show $x|grep ^Depends: | perl -nle 'print "'"$x:"' $1" if /libmysqlclient18[ \t]*$([^)]+)$/' ; done) You need to test a number of these that they are still installable when MariaDB is installed, also on old .deb distros (ie. Ubuntu 8.04). On the other hand, if this works then it is a nice solution for sure, and could also solve other issues. A related issue is that we might get the distro mysql-common package or the MariaDB mysql-common package somewhat at random, depending on which has the newer version number on any given day. I believe we made it so we work with the distro mysql-common package as well as the MariaDB one, it it will not necessarily cause any problems. It does seem a bit of a mess though, maybe we should just drop the mysql-common package from our repos? - Kristian.

1 0