January 2014 - developers - lists.mariadb.org

[Maria-developers] MDEV-5450 Assertion ... mysql_type_to_time_type(cached_field_type) == ltime.time_type' fails ...
by Alexander Barkov 03 Feb '14

03 Feb '14

Hi Sergei, Please review a patch for mdev-5450. Thanks.

2 3

Re: [Maria-developers] Rev 3886: MDEV-4439 ALTER TABLE .. [ADD|DROP] FOREIGN KEY IF [NOT] EXISTS does not work if constraint name is not used
by Sergei Golubchik 31 Jan '14

31 Jan '14

Hi, Holyfoot! On Jan 31, holyfoot(a)askmonty.org wrote: > At file:///home/hf/wmar/mdev-4439/ > > ------------------------------------------------------------ > revno: 3886 > revision-id: holyfoot(a)askmonty.org-20140131134044-ej3ftpf240xbn0lc > parent: knielsen(a)knielsen-hq.org-20131107065240-gxxg4ilcs8o2v0q2 > committer: Alexey Botchkov <holyfoot(a)askmonty.org> > branch nick: mdev-4439 > timestamp: Fri 2014-01-31 17:40:44 +0400 > message: > MDEV-4439 ALTER TABLE .. [ADD|DROP] FOREIGN KEY IF [NOT] EXISTS does not work if constraint name is not used. > Patches for server and the Innodb engine. > Server is fixed so it does nothing if no indexes left to alter. > Innodb parser is fixed so it looks for the IF [NOT] EXISTS option, takes into account > the drop->if_not_exists parameter. > Another change is that it uses the index name for the internal dictionary. > Prior to that it only used the CONSTRAINT name for it. That looks pretty much ok, but please add tests that use generated FK names. For example, it doesn't look like you handle the case of ADD FOREIGN KEY IF NOT EXISTS existing_generated_name (...) Regards, Sergei

1 0

[Maria-developers] buildbot for non ~maria-captains branches on launchpad
by Alexander Barkov 31 Jan '14

31 Jan '14

Hello Daniel, Would it be possible to have buildbot watch branches which are on Launchpad, but which are outside of the /~maria-captains/ directory? For example: bzr+ssh://bazaar.launchpad.net/~mroonga/maria/mroonga/ ? Thanks.

3 5

Re: [Maria-developers] Rev 3965: MDEV-5403 - Reduce usage of LOCK_open: tc_count
by Sergei Golubchik 31 Jan '14

31 Jan '14

Hi, Sergey! Please, see below. On Jan 30, Sergey Vojtovich wrote: > revno: 3965 > revision-id: svoj(a)mariadb.org-20140130123642-wsysj31e1wwlqz0o > parent: psergey(a)askmonty.org-20140120123657-g1z4g11xmsdiw2dc > committer: Sergey Vojtovich <svoj(a)mariadb.org> > branch nick: 10.0 > timestamp: Thu 2014-01-30 16:36:42 +0400 > message: > MDEV-5403 - Reduce usage of LOCK_open: tc_count > > Lock-free tc_count. > === modified file 'sql/table_cache.cc' > --- a/sql/table_cache.cc 2013-12-12 17:49:14 +0000 > +++ b/sql/table_cache.cc 2014-01-30 12:36:42 +0000 > @@ -173,7 +175,9 @@ void tc_purge(void) > { > share->tdc.all_tables.remove(table); > purge_tables.push_front(table); > - tc_count--; > + my_atomic_rwlock_wrlock(&LOCK_tdc_atomics); > + my_atomic_add32(&tc_count, -1); > + my_atomic_rwlock_wrunlock(&LOCK_tdc_atomics); Okay, so in this new code tc_count is always equal or greater than the actual number of tables in the cache, right? (because you first remove the table from the cache, then decrement the counter) > } > } > tdc_it.deinit(); > @@ -245,47 +249,52 @@ static void check_unused(THD *thd) > > void tc_add_table(THD *thd, TABLE *table) > { > + bool need_purge; > DBUG_ASSERT(table->in_use == thd); > mysql_mutex_lock(&LOCK_open); > table->s->tdc.all_tables.push_front(table); > + mysql_mutex_unlock(&LOCK_open); > + > /* If we have too many TABLE instances around, try to get rid of them */ > - if (tc_count == tc_size) > + my_atomic_rwlock_wrlock(&LOCK_tdc_atomics); > + need_purge= my_atomic_add32(&tc_count, 1) >= (int32) tc_size; Hmm, but here you increment after adding a table. I'd suggest to keep tc_count consistently either always no smaller than the number of tables in the cache (that is, decrement after removing, increment before adding), or always no larger than that (derement before removing, increment after adding). > + my_atomic_rwlock_wrunlock(&LOCK_tdc_atomics); > + > + if (need_purge) > { > + TABLE *purge_table= 0; > + TABLE_SHARE *share; > TDC_iterator tdc_it; > - mysql_mutex_unlock(&LOCK_open); > > tdc_it.init(); > mysql_mutex_lock(&LOCK_open); > - if (tc_count == tc_size) > - { > - TABLE *purge_table= 0; > - TABLE_SHARE *share; > while ((share= tdc_it.next())) > { > TABLE_SHARE::TABLE_list::Iterator it(share->tdc.free_tables); > TABLE *entry; > while ((entry= it++)) > if (!purge_table || entry->tc_time < purge_table->tc_time) > purge_table= entry; > } > + tdc_it.deinit(); > + > if (purge_table) > { > - tdc_it.deinit(); > purge_table->s->tdc.free_tables.remove(purge_table); > purge_table->s->tdc.all_tables.remove(purge_table); > mysql_rwlock_rdlock(&LOCK_flush); > mysql_mutex_unlock(&LOCK_open); > intern_close_table(purge_table); > mysql_rwlock_unlock(&LOCK_flush); > + > + my_atomic_rwlock_wrlock(&LOCK_tdc_atomics); > + my_atomic_add32(&tc_count, -1); > + my_atomic_rwlock_wrunlock(&LOCK_tdc_atomics); > check_unused(thd); > - return; > } > - } > - tdc_it.deinit(); > - } > - /* Nothing to evict, increment tc_count. */ > - tc_count++; > + else > mysql_mutex_unlock(&LOCK_open); > + } > } > > > @@ -357,25 +366,35 @@ bool tc_release_table(TABLE *table) > DBUG_ASSERT(table->in_use); > DBUG_ASSERT(table->file); > > + if (table->needs_reopen() || tc_records() > tc_size) > + { > + mysql_mutex_lock(&LOCK_open); > + table->in_use= 0; > + goto purge; > + } Why did you move this up? > table->tc_time= my_interval_timer(); > > mysql_mutex_lock(&LOCK_open); > table->in_use= 0; > - if (table->s->has_old_version() || table->needs_reopen() || tc_count > tc_size) > - { > - tc_count--; > + if (table->s->has_old_version()) > + goto purge; > + /* Add table to the list of unused TABLE objects for this share. */ > + table->s->tdc.free_tables.push_front(table); > + mysql_mutex_unlock(&LOCK_open); > + check_unused(thd); > + return false; > + > +purge: > + my_atomic_rwlock_wrlock(&LOCK_tdc_atomics); > + my_atomic_add32(&tc_count, -1); > + my_atomic_rwlock_wrunlock(&LOCK_tdc_atomics); > table->s->tdc.all_tables.remove(table); and now you decrement before removal > mysql_rwlock_rdlock(&LOCK_flush); > mysql_mutex_unlock(&LOCK_open); > intern_close_table(table); > mysql_rwlock_unlock(&LOCK_flush); > return true; > - } > - /* Add table to the list of unused TABLE objects for this share. */ > - table->s->tdc.free_tables.push_front(table); > - mysql_mutex_unlock(&LOCK_open); > - check_unused(thd); > - return false; > } > > Regards, Sergei

2 1

[Maria-developers] FreeBSD compilation problems and a fix
by Alexander Barkov 31 Jan '14

31 Jan '14

Hi guys, I just tried to compile 10.0 on FreeBSD-9.1 RELEASE and noticed two problems: 1. It printed this warning for every source file: /usr/include/sys/timeb.h:42:2: warning: #warning "this file includes <sys/timeb.h> which is deprecated" This diff fixes the problem: -#ifdef HAVE_SYS_TIMEB_H +#if defined(HAVE_SYS_TIMEB_H) && !defined(__FreeBSD__) 2. It failed to compile because of pcre (both libraries and includes) installed in the system. Compiler used the *system* header pcre.h together with the *bundled* library libpcre.a. I had a similar complaint from the MacPorts maintainers. This diff fixes the problem: # Common defines and includes +SET(CMAKE_INCLUDE_DIRECTORIES_PROJECT_BEFORE ON) ADD_DEFINITIONS(-DHAVE_CONFIG_H) INCLUDE_DIRECTORIES(${CMAKE_CURRENT_BINARY_DIR}/include Does this look OK? If so, which version should I push it in? Thanks.

2 3

Re: [Maria-developers] Rev 3966: MDEV-5492 - Reduce usage of LOCK_open: TABLE::in_use
by Sergei Golubchik 30 Jan '14

30 Jan '14

Hi, Sergey! On Jan 29, Sergey Vojtovich wrote: > At lp:maria/10.0 > > ------------------------------------------------------------ > revno: 3966 > revision-id: svoj(a)mariadb.org-20140129083115-jwye7zfyncrq9crs > parent: svoj(a)mariadb.org-20140124075326-d7twer07e0bu42b5 > committer: Sergey Vojtovich <svoj(a)mariadb.org> > branch nick: 10.0 > timestamp: Wed 2014-01-29 12:31:15 +0400 > message: > MDEV-5492 - Reduce usage of LOCK_open: TABLE::in_use > > Move TABLE::in_use out of LOCK_open. > > This is done with assumtion that foreign threads accessing TABLE::in_use > will only need consistent value _after_ marking table for flush and purging > unused table instances. In this case TABLE::in_use will always point to a > valid thread object. I believe I was able to understand what you were doing here :) Seems ok. The assumption above looks a bit risky without any way to enforce it. Perhaps you could make in_use private (renaming to, say, m_in_use) and add an inline getter method TABLE::in_use() that in debug builds will assert that your assumption holds? Regards, Sergei

2 2

[Maria-developers] MDEV-4309, second patch: review
by Sergei Petrunia 29 Jan '14

29 Jan '14

Hello, This is a review for MDEV-4309, second patch: https://mariadb.atlassian.net/secure/attachment/21510/201303271828_sanja_co… === The name "memcmp_func" is confusing, because the function has nothing to do with memcmp. Did you mean memcpy actually? The function should be renamed to something like "memcpy_field_value". memcmp_func() still makes one virtual call. === Consider a scenario: - Item_field::setup_cpy_optimizer() is called. - It calls memcmp_possible(), which returns FALSE, which means field_conv() will be called for every row. - field_conv() will still call memcpy_field_value(), even if we know that it will return false. Why not add a field_conv_incompatible() and make field_conv look like this: find_conv() { if (memcmp_possible()) { ... } return field_conv_incompatible(); } === The name "cpy_optimizer" is very cryptic. Is "Fast_field_copier" better? === class Item_field has this member: Field *cpy_optimizer_setup; I don't see how it is useful without its corresponding 'cpy_optimizer' data. cpy_optimizer is only present inside ORDER structure which has both: cpy_optimizer cpy_optimizer_func; Field *cpy_optimizer_setup; If ORDER has both members, why have the Item_field::cpy_optimizer_setup ? === Let's discuss this on IRC. BR Sergei -- Sergei Petrunia, Software Developer MariaDB | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] single thread performance (was: Re: Need to check if the slowdown is still there)
by Sergei Petrunia 29 Jan '14

29 Jan '14

Hi Axel, On Tue, Jan 28, 2014 at 04:27:10PM +0100, Axel Schwenke wrote: > Hi, > > I have now re-run a benchmark to test single thread performance. This is the > continuation of MDEV-4309. > Thanks for the data. > ... > The numbers demonstrate quite clearly the general observation that with each > new major version of MariaDB/MySQL the code path becomes longer and the > performance suffers. But there is also good news: sometimes during the > maturing of a release the performance may increase again. I.e. I have seen > significant changes from MySQL-5.6.13 to 5.6.14. I agree with the observations. I would even say that MySQL was getting worse in 5.6.11 - 5.6.13 versions, and then fixe something in 5.6.14. > Also from MariaDB-5.5.30 to > 5.5.31 (which was when MDEV-4309 was implemented). Unfortunately between > MariaDB-5.5.33 and 5.5.34 most of that was lost again :( The first step in not letting that happen is to have the benchmark data so that we know if there are any problems. The patch for MDEV-4309 part#2 is not yet ready. We will get back to you when it can be used for testing. BR Sergei -- Sergei Petrunia, Software Developer MariaDB | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] The "Monty Program AB" vendor check (rpm)
by Oden Eriksson 28 Jan '14

28 Jan '14

Hello, I got assigned to look at the rpm packaging and fixed some stuff there, but got stuck at the "Monty Program AB" vendor check in support-files/rpm/server- prein.sh when I wanted to replace the fedora packages with in this case mariadb galera packages. Is this check absolutely nessesary? This prevents you from an easy switch from the fedora mariadb packages to the ones provided by mariadb using yum and a yum repository as provided at http://yum.mariadb.org/. Basically this check prevents me from doing any serious work with the rpm packages because you manually have to, by force deinstall any conflicting packages and then install the packages provided by us. Cheers.

3 3

Re: [Maria-developers] Review request for a fix for MDEV-5506
by Sergei Golubchik 28 Jan '14

28 Jan '14

Hi, Alexander! Few questions, see below. > === modified file 'mysql-test/t/timezone2.test' > --- mysql-test/t/timezone2.test 2013-08-08 08:58:28 +0000 > +++ mysql-test/t/timezone2.test 2014-01-23 10:34:46 +0000 > @@ -298,5 +298,11 @@ SELECT CONVERT_TZ(TIME('00:00:00'),'+00: > SELECT CONVERT_TZ(TIME('2010-01-01 00:00:00'),'+00:00','+7:5'); > > --echo # > +--echo # MDEV-5506 safe_mutex: Trying to lock unitialized mutex at safemalloc.c on server shutdown after SELECT with CONVERT_TZ > +--echo # > +SELECT CONVERT_TZ('2001-10-08 00:00:00', MAKE_SET(0,'+01:00'), '+00:00' ); hmm, the bug description was "at shutdown". I don't see a shutdown in your test case. I suppose, you assume that the server will be eventually shut down and that'll do. But perhaps it needs to be shut down immediately, and if mtr will do a lot of work in this server the test case becomes invalid - it won't crash anymore? > === modified file 'sql/item_strfunc.cc' > --- sql/item_strfunc.cc 2013-09-25 12:30:13 +0000 > +++ sql/item_strfunc.cc 2014-01-23 10:17:57 +0000 > @@ -43,7 +43,7 @@ C_MODE_END > /** > @todo Remove this. It is not safe to use a shared String object. > */ I guess, you've fixed that and can remove the comment. > -String my_empty_string("",default_charset_info); > +String_const my_empty_string("", default_charset_info); > > === modified file 'sql/sql_string.h' > --- sql/sql_string.h 2012-11-09 08:11:20 +0000 > +++ sql/sql_string.h 2014-01-23 10:26:13 +0000 > @@ -56,25 +56,52 @@ uint convert_to_printable(char *to, size > > class String > { > + typedef enum > + { > + STRING_FLAG_NONE= 0, > + STRING_FLAG_ALLOCED= 1, > + STRING_FLAG_READ_ONLY= 2, > + STRING_FLAG_NULL_TERMINATED= 4 > + } flag_t; > char *Ptr; > uint32 str_length,Alloced_length, extra_alloc; > - bool alloced; > + flag_t flags; > CHARSET_INFO *str_charset; Okay, I understand what you did. But I don't understand why. What was the problem? Why there was a crash? Regards, Sergei

2 4