- developers - lists.mariadb.org

[Maria-developers] MariaDB build dependencies
by Otto Kekäläinen 03 Feb '14

03 Feb '14

Hello, The original MariaDB debian/control in bzr/mariadb-5.5 lists many more dependencies than what are likely to be needed: Build-Depends: libtool (>= 1.4.2-7), procps | hurd, debhelper, file (>= 3.28), libncurses5-dev (>= 5.0-6), perl (>= 5.6.0), libwrap0-dev (>= 7.6-8.3), zlib1g-dev (>= 1:1.1.3-5), ${LIBREADLINE_DEV}, libevent-dev, libssl-dev, libpam0g-dev, psmisc, po-debconf, chrpath, automake1.9, doxygen, texlive-latex-base, ghostscript | gs-gpl, dpatch, gawk, bison, lsb-release, hardening-wrapper, ${CMAKE_DEP}libaio-dev The source builds error free with just: libaio-dev libncurses5-dev perl libwrap0-dev zlib1g-dev libreadline-gplv2-dev po-debconf psmisc chrpath gawk bison lsb-release hardening-wrapper cmake libssl-dev libboost1.48-dev libpam0g-dev Can anybody tell me how in particular these are supposed to affect the build process? - libevent-dev - doxygen, texlive-latex-base, ghostscript | gs-gpl - gawk Debian MySQL team has a page https://wiki.debian.org/Teams/MySQL/BuildDependencies but obiviously MariaDB is different in these aspects, for example MariaDB needs libpam0g-dev for auth_pam.so which does not exist in MySQL. If anybody can explain the above that would be great, thanks!

3 3

Re: [Maria-developers] Rev 3966: MDEV-4439 ALTER TABLE .. [ADD|DROP] FOREIGN KEY IF [NOT] EXISTS does not work if constraint name is not used
by Sergei Golubchik 01 Feb '14

01 Feb '14

Hi, Holyfoot! On Feb 01, holyfoot(a)askmonty.org wrote: > revno: 3966 > revision-id: holyfoot(a)askmonty.org-20140201114304-7bi0ug2yg3eyyhlr > parent: psergey(a)askmonty.org-20140121100700-9g8hl8vx8nmk1e2w > committer: Alexey Botchkov <holyfoot(a)askmonty.org> > branch nick: 10-hf > timestamp: Sat 2014-02-01 15:43:04 +0400 > message: > MDEV-4439 ALTER TABLE .. [ADD|DROP] FOREIGN KEY IF [NOT] EXISTS does not work if constraint name is not used. > Patches for server and the Innodb engine. > Server is fixed so it does nothing if no indexes left to alter. > Innodb parser is fixed so it looks for the IF [NOT] EXISTS option in a string. > Another change is that it uses the index name for the internal dictionary. > Prior to that it only used the CONSTRAINT name for it. ok to push Regards, Sergei

1 0

Re: [Maria-developers] Rev 3886: MDEV-4439 ALTER TABLE .. [ADD|DROP] FOREIGN KEY IF [NOT] EXISTS does not work if constraint name is not used
by Sergei Golubchik 31 Jan '14

31 Jan '14

Hi, Holyfoot! On Jan 31, holyfoot(a)askmonty.org wrote: > At file:///home/hf/wmar/mdev-4439/ > > ------------------------------------------------------------ > revno: 3886 > revision-id: holyfoot(a)askmonty.org-20140131134044-ej3ftpf240xbn0lc > parent: knielsen(a)knielsen-hq.org-20131107065240-gxxg4ilcs8o2v0q2 > committer: Alexey Botchkov <holyfoot(a)askmonty.org> > branch nick: mdev-4439 > timestamp: Fri 2014-01-31 17:40:44 +0400 > message: > MDEV-4439 ALTER TABLE .. [ADD|DROP] FOREIGN KEY IF [NOT] EXISTS does not work if constraint name is not used. > Patches for server and the Innodb engine. > Server is fixed so it does nothing if no indexes left to alter. > Innodb parser is fixed so it looks for the IF [NOT] EXISTS option, takes into account > the drop->if_not_exists parameter. > Another change is that it uses the index name for the internal dictionary. > Prior to that it only used the CONSTRAINT name for it. That looks pretty much ok, but please add tests that use generated FK names. For example, it doesn't look like you handle the case of ADD FOREIGN KEY IF NOT EXISTS existing_generated_name (...) Regards, Sergei

1 0

[Maria-developers] buildbot for non ~maria-captains branches on launchpad
by Alexander Barkov 31 Jan '14

31 Jan '14

Hello Daniel, Would it be possible to have buildbot watch branches which are on Launchpad, but which are outside of the /~maria-captains/ directory? For example: bzr+ssh://bazaar.launchpad.net/~mroonga/maria/mroonga/ ? Thanks.

3 5

Re: [Maria-developers] Rev 3965: MDEV-5403 - Reduce usage of LOCK_open: tc_count
by Sergei Golubchik 31 Jan '14

31 Jan '14

Hi, Sergey! Please, see below. On Jan 30, Sergey Vojtovich wrote: > revno: 3965 > revision-id: svoj(a)mariadb.org-20140130123642-wsysj31e1wwlqz0o > parent: psergey(a)askmonty.org-20140120123657-g1z4g11xmsdiw2dc > committer: Sergey Vojtovich <svoj(a)mariadb.org> > branch nick: 10.0 > timestamp: Thu 2014-01-30 16:36:42 +0400 > message: > MDEV-5403 - Reduce usage of LOCK_open: tc_count > > Lock-free tc_count. > === modified file 'sql/table_cache.cc' > --- a/sql/table_cache.cc 2013-12-12 17:49:14 +0000 > +++ b/sql/table_cache.cc 2014-01-30 12:36:42 +0000 > @@ -173,7 +175,9 @@ void tc_purge(void) > { > share->tdc.all_tables.remove(table); > purge_tables.push_front(table); > - tc_count--; > + my_atomic_rwlock_wrlock(&LOCK_tdc_atomics); > + my_atomic_add32(&tc_count, -1); > + my_atomic_rwlock_wrunlock(&LOCK_tdc_atomics); Okay, so in this new code tc_count is always equal or greater than the actual number of tables in the cache, right? (because you first remove the table from the cache, then decrement the counter) > } > } > tdc_it.deinit(); > @@ -245,47 +249,52 @@ static void check_unused(THD *thd) > > void tc_add_table(THD *thd, TABLE *table) > { > + bool need_purge; > DBUG_ASSERT(table->in_use == thd); > mysql_mutex_lock(&LOCK_open); > table->s->tdc.all_tables.push_front(table); > + mysql_mutex_unlock(&LOCK_open); > + > /* If we have too many TABLE instances around, try to get rid of them */ > - if (tc_count == tc_size) > + my_atomic_rwlock_wrlock(&LOCK_tdc_atomics); > + need_purge= my_atomic_add32(&tc_count, 1) >= (int32) tc_size; Hmm, but here you increment after adding a table. I'd suggest to keep tc_count consistently either always no smaller than the number of tables in the cache (that is, decrement after removing, increment before adding), or always no larger than that (derement before removing, increment after adding). > + my_atomic_rwlock_wrunlock(&LOCK_tdc_atomics); > + > + if (need_purge) > { > + TABLE *purge_table= 0; > + TABLE_SHARE *share; > TDC_iterator tdc_it; > - mysql_mutex_unlock(&LOCK_open); > > tdc_it.init(); > mysql_mutex_lock(&LOCK_open); > - if (tc_count == tc_size) > - { > - TABLE *purge_table= 0; > - TABLE_SHARE *share; > while ((share= tdc_it.next())) > { > TABLE_SHARE::TABLE_list::Iterator it(share->tdc.free_tables); > TABLE *entry; > while ((entry= it++)) > if (!purge_table || entry->tc_time < purge_table->tc_time) > purge_table= entry; > } > + tdc_it.deinit(); > + > if (purge_table) > { > - tdc_it.deinit(); > purge_table->s->tdc.free_tables.remove(purge_table); > purge_table->s->tdc.all_tables.remove(purge_table); > mysql_rwlock_rdlock(&LOCK_flush); > mysql_mutex_unlock(&LOCK_open); > intern_close_table(purge_table); > mysql_rwlock_unlock(&LOCK_flush); > + > + my_atomic_rwlock_wrlock(&LOCK_tdc_atomics); > + my_atomic_add32(&tc_count, -1); > + my_atomic_rwlock_wrunlock(&LOCK_tdc_atomics); > check_unused(thd); > - return; > } > - } > - tdc_it.deinit(); > - } > - /* Nothing to evict, increment tc_count. */ > - tc_count++; > + else > mysql_mutex_unlock(&LOCK_open); > + } > } > > > @@ -357,25 +366,35 @@ bool tc_release_table(TABLE *table) > DBUG_ASSERT(table->in_use); > DBUG_ASSERT(table->file); > > + if (table->needs_reopen() || tc_records() > tc_size) > + { > + mysql_mutex_lock(&LOCK_open); > + table->in_use= 0; > + goto purge; > + } Why did you move this up? > table->tc_time= my_interval_timer(); > > mysql_mutex_lock(&LOCK_open); > table->in_use= 0; > - if (table->s->has_old_version() || table->needs_reopen() || tc_count > tc_size) > - { > - tc_count--; > + if (table->s->has_old_version()) > + goto purge; > + /* Add table to the list of unused TABLE objects for this share. */ > + table->s->tdc.free_tables.push_front(table); > + mysql_mutex_unlock(&LOCK_open); > + check_unused(thd); > + return false; > + > +purge: > + my_atomic_rwlock_wrlock(&LOCK_tdc_atomics); > + my_atomic_add32(&tc_count, -1); > + my_atomic_rwlock_wrunlock(&LOCK_tdc_atomics); > table->s->tdc.all_tables.remove(table); and now you decrement before removal > mysql_rwlock_rdlock(&LOCK_flush); > mysql_mutex_unlock(&LOCK_open); > intern_close_table(table); > mysql_rwlock_unlock(&LOCK_flush); > return true; > - } > - /* Add table to the list of unused TABLE objects for this share. */ > - table->s->tdc.free_tables.push_front(table); > - mysql_mutex_unlock(&LOCK_open); > - check_unused(thd); > - return false; > } > > Regards, Sergei

2 1

[Maria-developers] FreeBSD compilation problems and a fix
by Alexander Barkov 31 Jan '14

31 Jan '14

Hi guys, I just tried to compile 10.0 on FreeBSD-9.1 RELEASE and noticed two problems: 1. It printed this warning for every source file: /usr/include/sys/timeb.h:42:2: warning: #warning "this file includes <sys/timeb.h> which is deprecated" This diff fixes the problem: -#ifdef HAVE_SYS_TIMEB_H +#if defined(HAVE_SYS_TIMEB_H) && !defined(__FreeBSD__) 2. It failed to compile because of pcre (both libraries and includes) installed in the system. Compiler used the *system* header pcre.h together with the *bundled* library libpcre.a. I had a similar complaint from the MacPorts maintainers. This diff fixes the problem: # Common defines and includes +SET(CMAKE_INCLUDE_DIRECTORIES_PROJECT_BEFORE ON) ADD_DEFINITIONS(-DHAVE_CONFIG_H) INCLUDE_DIRECTORIES(${CMAKE_CURRENT_BINARY_DIR}/include Does this look OK? If so, which version should I push it in? Thanks.

2 3

Re: [Maria-developers] Rev 3966: MDEV-5492 - Reduce usage of LOCK_open: TABLE::in_use
by Sergei Golubchik 30 Jan '14

30 Jan '14

Hi, Sergey! On Jan 29, Sergey Vojtovich wrote: > At lp:maria/10.0 > > ------------------------------------------------------------ > revno: 3966 > revision-id: svoj(a)mariadb.org-20140129083115-jwye7zfyncrq9crs > parent: svoj(a)mariadb.org-20140124075326-d7twer07e0bu42b5 > committer: Sergey Vojtovich <svoj(a)mariadb.org> > branch nick: 10.0 > timestamp: Wed 2014-01-29 12:31:15 +0400 > message: > MDEV-5492 - Reduce usage of LOCK_open: TABLE::in_use > > Move TABLE::in_use out of LOCK_open. > > This is done with assumtion that foreign threads accessing TABLE::in_use > will only need consistent value _after_ marking table for flush and purging > unused table instances. In this case TABLE::in_use will always point to a > valid thread object. I believe I was able to understand what you were doing here :) Seems ok. The assumption above looks a bit risky without any way to enforce it. Perhaps you could make in_use private (renaming to, say, m_in_use) and add an inline getter method TABLE::in_use() that in debug builds will assert that your assumption holds? Regards, Sergei

2 2

[Maria-developers] MDEV-4309, second patch: review
by Sergei Petrunia 29 Jan '14

29 Jan '14

Hello, This is a review for MDEV-4309, second patch: https://mariadb.atlassian.net/secure/attachment/21510/201303271828_sanja_co… === The name "memcmp_func" is confusing, because the function has nothing to do with memcmp. Did you mean memcpy actually? The function should be renamed to something like "memcpy_field_value". memcmp_func() still makes one virtual call. === Consider a scenario: - Item_field::setup_cpy_optimizer() is called. - It calls memcmp_possible(), which returns FALSE, which means field_conv() will be called for every row. - field_conv() will still call memcpy_field_value(), even if we know that it will return false. Why not add a field_conv_incompatible() and make field_conv look like this: find_conv() { if (memcmp_possible()) { ... } return field_conv_incompatible(); } === The name "cpy_optimizer" is very cryptic. Is "Fast_field_copier" better? === class Item_field has this member: Field *cpy_optimizer_setup; I don't see how it is useful without its corresponding 'cpy_optimizer' data. cpy_optimizer is only present inside ORDER structure which has both: cpy_optimizer cpy_optimizer_func; Field *cpy_optimizer_setup; If ORDER has both members, why have the Item_field::cpy_optimizer_setup ? === Let's discuss this on IRC. BR Sergei -- Sergei Petrunia, Software Developer MariaDB | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] single thread performance (was: Re: Need to check if the slowdown is still there)
by Sergei Petrunia 29 Jan '14

29 Jan '14

Hi Axel, On Tue, Jan 28, 2014 at 04:27:10PM +0100, Axel Schwenke wrote: > Hi, > > I have now re-run a benchmark to test single thread performance. This is the > continuation of MDEV-4309. > Thanks for the data. > ... > The numbers demonstrate quite clearly the general observation that with each > new major version of MariaDB/MySQL the code path becomes longer and the > performance suffers. But there is also good news: sometimes during the > maturing of a release the performance may increase again. I.e. I have seen > significant changes from MySQL-5.6.13 to 5.6.14. I agree with the observations. I would even say that MySQL was getting worse in 5.6.11 - 5.6.13 versions, and then fixe something in 5.6.14. > Also from MariaDB-5.5.30 to > 5.5.31 (which was when MDEV-4309 was implemented). Unfortunately between > MariaDB-5.5.33 and 5.5.34 most of that was lost again :( The first step in not letting that happen is to have the benchmark data so that we know if there are any problems. The patch for MDEV-4309 part#2 is not yet ready. We will get back to you when it can be used for testing. BR Sergei -- Sergei Petrunia, Software Developer MariaDB | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] The "Monty Program AB" vendor check (rpm)
by Oden Eriksson 28 Jan '14

28 Jan '14

Hello, I got assigned to look at the rpm packaging and fixed some stuff there, but got stuck at the "Monty Program AB" vendor check in support-files/rpm/server- prein.sh when I wanted to replace the fedora packages with in this case mariadb galera packages. Is this check absolutely nessesary? This prevents you from an easy switch from the fedora mariadb packages to the ones provided by mariadb using yum and a yum repository as provided at http://yum.mariadb.org/. Basically this check prevents me from doing any serious work with the rpm packages because you manually have to, by force deinstall any conflicting packages and then install the packages provided by us. Cheers.

3 3