- developers - lists.mariadb.org

[Maria-developers] MariaDB Developers Meetup
by Ian Gilfillan 10 Aug '16

10 Aug '16

The MariaDB Developers Meetup will be taking place from the 6th to the 8th of October in Amsterdam. https://mariadb.org/2016-mariadb-developers-meetup/ Details of where to sign up, and where to suggest topics for the meetup are on that page. Please feel free to suggest topic you're interested in presenting, as well as topics you'd like to see covered. The meetup is scheduled to be just after Percona Live, which also takes place in Amsterdam from the 3rd to the 5th. The Foundation has arranged a discount so that anyone signing up for the MariaDB Developers Meetup and that also wants to attend Percona Live will receive a discount for that event (the Developers Meetup has no charge). Percona Live are also looking for talks or sessions, so if you have a topic that would be of interest to the broader community, feel free to submit it at https://www.percona.com/live/plam16/program

1 1

[Maria-developers] Please review MDEV-10468 Assertion `nr >= 0.0' failed in Item_sum_std::val_real()
by Alexander Barkov 08 Aug '16

08 Aug '16

Hello Alexey, Can you please review a patch for MDEV-10468? Thanks!

2 3

[Maria-developers] Please review MDEV-10500 CASE/IF Statement returns multiple values and shifts further result values to the next column
by Alexander Barkov 08 Aug '16

08 Aug '16

Hi Serg, Please review a patch for MDEV-10500. Thanks.

2 2

Re: [Maria-developers] 634e130: MDEV-10424 - Assertion `ticket == __null' failed in MDL_request::set_type
by Sergei Golubchik 08 Aug '16

08 Aug '16

Hi, Sergey! On Aug 04, Sergey Vojtovich wrote: > revision-id: 634e130b00bc900d1b7b8a8df6c13b1701f84b30 (mariadb-5.5.50-24-g634e130) > parent(s): 2f55f0b010619a09870ba704b55dd1426a158f1d > committer: Sergey Vojtovich > timestamp: 2016-08-04 15:51:17 +0400 > message: > > MDEV-10424 - Assertion `ticket == __null' failed in MDL_request::set_type > > Reexecution of prepared "ANALYZE TABLE merge_table, table" may miss to > reinitialize "table" for subsequent execution and trigger assertion failure. > > This happens because MERGE engine may adjust table->next_global chain, which > gets cleared by close_thread_tables()/ha_myisammrg::detach_children() later. > Since reinitilization iterates next_global chain, it won't see tables following > merge table. > > Fixed by appending saved next_global chain after merge children. looks ok to me, but let elena do some extra testing of this, please Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] GSoC weekly reports (Unique indexes for blobs)
by Shubham Barai 04 Aug '16

04 Aug '16

GSoC (week 5) Hello everyone, Here is the list of things I have done in the 5th week of GSoC. 1.implemented unique key violation with a hash collision. (actual row comparison is remaining ). 2.modified hash function for data types like varchar and binary data types. 3.fixed a bug which was causing a server to crash for complex unique keys. 4.added support to allow any number of nulls which will not cause any unique key violation. 5.added test cases for above features. On 22 June 2016 at 16:36, Shubham Barai <shubhambaraiss(a)gmail.com> wrote: > Hi, > > can we discuss on IRC first? > > > Regards, > Shubham > > On 22 June 2016 at 13:21, Jan Lindström <jan.lindstrom(a)mariadb.com> wrote: > >> Hi, >> >> Please commit and push these changes to your git branch I have not yet >> seen them, in my opinion as this is only a working branch you can push >> often. I still fail to see any test cases on InnoDB branch, do you have >> more than one branch and if you have why ? Depending on extent of these >> changes my estimate is that you are behind schedule to complete project in >> time. Based on your progress report you are still missing update and delete >> and redo-logging. For alter table you should start from forcing copy-method >> and then if time permits develop on-line method. This naturally only after >> everything else >> has been completed and tested. >> >> R: Jan >> >> On Tue, Jun 21, 2016 at 11:06 PM, Shubham Barai <shubhambaraiss(a)gmail.com >> > wrote: >> >>> GSoC (week 4) >>> >>> Hello everyone, >>> >>> After working on create table operation,next thing I had to work on was >>> insert operations.So I explored some of the functions like >>> row_ins_scan_index_for_duplicates, btr_pcur_get_rec to get clear >>> understanding about how to implement duplicate search on hash index. >>> There was a problem in hash function that I wrote .It would calculate >>> same hash value for two different keys if the prefix length of blob key >>> part was zero. Now it seems to be working after I checked it in debugger.I >>> still have to modify it for data types like varchar etc. >>> I have added test cases for insert operations in myisam. >>> In MyIsam, I found one problem in update operation. When updating a >>> row,if the key is conflicting then server crashes because some pointer goes >>> invalid in compare_record. I haven't fixed this issue yet. >>> >>> I also modified some functions in dict0load.cc to adjust some members >>> of dict_index_t for a new index type.The main problem is that index entry >>> for hash based index cointains only two fields(hash value and row id) while >>> dict_index_t contains hash field and other user defined fields which are >>> used to calculate hash value.Some of the operations like alter table( e.g. >>> rename column) needs to get access to all fields while other functions like >>> rec_get_offsets and row_build_index_entry_low needs to get access to only >>> hash field and row id. I am still working on this to find efficient >>> solution to this problem. >>> >>> On 16 June 2016 at 23:29, Sergei Golubchik <vuvova(a)gmail.com> wrote: >>> >>>> Hi, Shubham! >>>> >>>> What I wanted to say on IRC was: >>>> >>>> here's what the comment of cmp_dtuple_rec_with_match_low() says: >>>> >>>> ............... If rec has an externally stored field we do not >>>> compare it but return with value 0 if such a comparison should be >>>> made. >>>> >>>> Note that blobs are externally stored fields in InnoDB, so, I think, >>>> this means that you cannot use cmp_dtuple_rec() to compare blobs. >>>> >>>> Regards, >>>> Sergei >>>> Chief Architect MariaDB >>>> and security(a)mariadb.org >>>> >>> >>> >>> _______________________________________________ >>> Mailing list: https://launchpad.net/~maria-developers >>> Post to : maria-developers(a)lists.launchpad.net >>> Unsubscribe : https://launchpad.net/~maria-developers >>> More help : https://help.launchpad.net/ListHelp >>> >>> >> >

1 5

Re: [Maria-developers] bdef42c: MDEV-10365 - Race condition in error handling of INSERT DELAYED
by Sergei Golubchik 03 Aug '16

03 Aug '16

Hi, Sergey! On Aug 03, Sergey Vojtovich wrote: > revision-id: bdef42c54f8938d28d9a421271b11e09e88735a4 (mariadb-5.5.50-19-gbdef42c) > parent(s): 45ec5d673a1f5c736baec8ea9e2065bca4350324 > committer: Sergey Vojtovich > timestamp: 2016-08-03 11:49:35 +0400 > message: > > MDEV-10365 - Race condition in error handling of INSERT DELAYED > > Shared variables of Delayed_insert may be updated without mutex protection > when delayed insert thread gets an error. > > Re-acquire mutex earlier, so that shared variables are protected. ok to push Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] fc42991: MDEV-7901: re-implement analyze table for low impact
by Sergei Golubchik 03 Aug '16

03 Aug '16

Hi, Oleksandr! Looks mostly ok, see few comments below On Aug 01, Oleksandr Byelkin wrote: > revision-id: fc42991720838e165ad448b6707602cded92faa4 (mariadb-10.2.1-9-gfc42991) > parent(s): 08683a726773f8cdf16a4a3dfb3920e5f7842481 > committer: Oleksandr Byelkin > timestamp: 2016-08-01 19:25:45 +0200 > message: > > MDEV-7901: re-implement analyze table for low impact > > Table before collecting engine independent statistics now is reopened in read mode, > InnoDB allow write operations in this case. > > --- > mysql-test/r/stat_tables_innodb_debug.result | 27 +++ > mysql-test/t/stat_tables_innodb_debug.test | 37 ++++ > sql/sql_admin.cc | 295 +++++++++++++++------------ > storage/innobase/handler/ha_innodb.cc | 1 + > storage/xtradb/handler/ha_innodb.cc | 2 +- > 5 files changed, 227 insertions(+), 135 deletions(-) > > diff --git a/mysql-test/r/stat_tables_innodb_debug.result b/mysql-test/r/stat_tables_innodb_debug.result > new file mode 100644 > index 0000000..b02848d > --- /dev/null > +++ b/mysql-test/r/stat_tables_innodb_debug.result > @@ -0,0 +1,27 @@ > +# > +# MDEV-7901: re-implement analyze table for low impact > +# > +create table t1(a int primary key) engine=innodb; > +insert into t1 values (0),(1),(2),(3),(4),(5),(6),(7),(8),(9); you use t1 only to populate t2. You don't need to create a table for thatr, use seq_0_to_9 - https://mariadb.com/kb/en/mariadb/sequence/ > +create table t2 (a int, b int, c int, key IDX1(a), key IDX2(a,b)) engine=innodb; > +insert into t2 select a/10, a/2, a from test.t1; > +SET DEBUG_SYNC='statistics_collection_start1 SIGNAL analyzing WAIT_FOR written'; > +analyze table t2 persistent for all; > +connect con1, localhost, root,,; > +connection con1; > +SET DEBUG_SYNC= 'now WAIT_FOR analyzing'; > +select count(*) from t2; > +count(*) > +10 > +insert into t2 values (333,333,333); > +update t2 set a=1; > +SET DEBUG_SYNC= 'now SIGNAL written'; > +connection default; > +Table Op Msg_type Msg_text > +test.t2 analyze status Engine-independent statistics collected > +test.t2 analyze status OK > +select count(*) from t2; > +count(*) > +11 > +set debug_sync='RESET'; > +drop table t1,t2; good direct test > diff --git a/sql/sql_admin.cc b/sql/sql_admin.cc > index 578b78d..933f3d0 100644 > --- a/sql/sql_admin.cc > +++ b/sql/sql_admin.cc > @@ -812,7 +773,73 @@ static bool mysql_admin_table(THD* thd, TABLE_LIST* tables, > > if (compl_result_code == HA_ADMIN_OK && collect_eis) > { ok, I've quickly checked the code and I think (not sure) that there is no need to reopen for innodb. But as you do, please, show the benefits of it - add a test case for MyISAM too (remember, TL_READ allows concurrent inserts). besides, this reopening code needs a comment explaining why you're doing it. > + enum_sql_command save_sql_command= lex->sql_command; > + trans_commit_stmt(thd); > + trans_commit(thd); > + thd->open_options|= extra_open_options; > + close_thread_tables(thd); > + table->table= NULL; > + thd->mdl_context.release_transactional_locks(); > + lex->sql_command= save_sql_command; where is lex->sql_command changed? > + table->mdl_request.init(MDL_key::TABLE, table->db, table->table_name, > + MDL_SHARED_NO_READ_WRITE, MDL_TRANSACTION); > + table->mdl_request.set_type(MDL_SHARED_READ); > + > + table->lock_type= TL_READ; > + open_error= open_only_one_table(thd, table, > + repair_table_use_frm, > + (view_operator_func != NULL)); can one analyze a view? how does that work? > + thd->open_options&= ~extra_open_options; ... Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

2 1

[Maria-developers] MariaDB memory usage when rebuilding a large table
by Mark Wadham 03 Aug '16

03 Aug '16

Hi, Apologies if this is the wrong place to ask this question but it seemed like the most apt. We are working on operational processes for online schema changes with MariaDB 10.1.16 and have encountered memory issues which are proving difficult to reliably solve. If anyone on this list could explain to me in detail how to calculate the memory footprint of modifying a large table with DDL statements I would be very grateful. Our test case is a 3-node galera cluster on which we want to alter a ~160GB table that has . Each node has 64GB ram and (originally) no swap space allocated. Even with the innodb buffer pool lowered to 16GB we consistently reached an OOM condition trying to add a column to the table. If we lower the innodb buffer pool to 16GB *and* provision 16GB of swap, the DDL completes. With the innodb buffer pool left at 50GB and allocated swap of 16GB, same OOM condition. Since this is an operational procedure we are testing the process under heavy load, but the node that is applying the DDL is first placed into RSU mode so the writesets simply build up in the gcache while it's executing (and obviously traffic from the production environment would be directed away from the node while this is happening). We are also increasing the gcache to 800GB on the node as part of this process to ensure that we have plenty of time for it to complete. I'm fairly sure that the gcache is simply a continuous write-to-disk process rather than anything involving significant memory, so *hope* that's not a contributing factor here. I am however quite surprised that executing DDL on a table can be so memory-intensive. Our next test run will be with the innodb buffer pool set to 0 on the node executing the DDL with no swap allocated to see if it can complete purely in-memory. Logically it should be able to if this is a memory constraint rather than a bug. But fundamentally we need to be able to determine with confidence how much memory we need to successfully execute DDL on a table of {n} size. Thanks, Mark

1 1

Re: [Maria-developers] e252dae: MDEV-9872 - Add common optimized CRC32 function interface
by Sergei Golubchik 03 Aug '16

03 Aug '16

Hi, Sergey! So, what did you do here? Did you change InnoDB's version of crc32 or just moved it to extra/ ? You removed mysys/my_crc32.c - where will crc32 come from on x86? Why did you name new function crc32_vpmsum instead of just naming it crc32 and avoding #ifdef's elsewhere? On Jul 05, Sergey Vojtovich wrote: > revision-id: e252dae86392a2b50d647a380db36fb8b01bf405 (mariadb-10.2.1-5-ge252dae) > parent(s): 72c94872835f27b536f617978dbc664ddb57da06 > committer: Sergey Vojtovich > timestamp: 2016-07-05 18:09:23 +0400 > message: > > MDEV-9872 - Add common optimized CRC32 function interface > > Move crc32-vpmsum to extra. > Compile static crc32-vpmsum instead of adding sources directly. > Make use of crc32-vpmsum via my_checksum(). > > Based on contribution by Daniel Black. Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

2 1

Re: [Maria-developers] f89d1f5: MDEV-10421 duplicate CHECK CONSTRAINTs.
by Sergei Golubchik 03 Aug '16

03 Aug '16

Hi, Alexey! Thanks! See below On Jul 31, Alexey Botchkov wrote: > revision-id: f89d1f55576136fb229392cb9a2997b9c0131241 (mariadb-10.2.1-9-gf89d1f5) > parent(s): 08683a726773f8cdf16a4a3dfb3920e5f7842481 > committer: Alexey Botchkov > timestamp: 2016-07-31 14:04:52 +0400 > message: > > MDEV-10421 duplicate CHECK CONSTRAINTs. > > mysql_prepare_create_table fixed so it doesn't let duplicating > constraint names. Syntax for CONSTRAINT IF NOT EXISTS added > and handled in mysql_alter_table. > > diff --git a/mysql-test/r/alter_table.result b/mysql-test/r/alter_table.result > index 3461038..5c1da48 100644 > --- a/mysql-test/r/alter_table.result > +++ b/mysql-test/r/alter_table.result > @@ -2057,3 +2057,23 @@ t1 CREATE TABLE `t1` ( > KEY `i1` (`a`) COMMENT 'comment2' > ) ENGINE=MyISAM DEFAULT CHARSET=latin1 > DROP TABLE t1; > +# > +# MDEV-10421 duplicate CHECK CONSTRAINTs > +# > +CREATE TABLE t1 (a INT, b INT) engine=myisam; > +ALTER TABLE t1 ADD CONSTRAINT IF NOT EXISTS `min` CHECK (a+b > 100); > +ALTER TABLE t1 ADD CONSTRAINT `min` CHECK (a+b > 100); > +ERROR HY000: Duplicate constraint name 'min' > +ALTER TABLE t1 ADD CONSTRAINT IF NOT EXISTS `min` CHECK (a+b > 100); > +Warnings: > +Note 4027 Duplicate constraint name 'min' > +ALTER TABLE t1 ADD CONSTRAINT `mini` CHECK (a+b > 100); please also add a test like create table (a int, b int, constraint min check (a>5), constraint min check (b>5)); > +SHOW CREATE TABLE t1; > +Table Create Table > +t1 CREATE TABLE `t1` ( > + `a` int(11) DEFAULT NULL, > + `b` int(11) DEFAULT NULL, > + CONSTRAINT `min` CHECK (a+b > 100), > + CONSTRAINT `mini` CHECK (a+b > 100) > +) ENGINE=MyISAM DEFAULT CHARSET=latin1 > +DROP TABLE t1; > diff --git a/sql/field.h b/sql/field.h > index 13a0e91..7b271b0 100644 > --- a/sql/field.h > +++ b/sql/field.h > @@ -606,6 +606,7 @@ class Virtual_column_info: public Sql_alloc > /* Flag indicating that the field is physically stored in the database */ > bool stored_in_db; > bool utf8; /* Already in utf8 */ > + bool create_if_not_exists; Hmm, I don't think it's a good idea. Virtual_column_info is used for storing run-time information about generated columns, defaults, and check constraints. It's kind of wrong to put the a parser-time check-constraint-only flag. > /* The expression to compute the value of the virtual column */ > Item *expr_item; > /* Text representation of the defining expression */ > diff --git a/sql/share/errmsg-utf8.txt b/sql/share/errmsg-utf8.txt > index 8dfa519..261021b 100644 > --- a/sql/share/errmsg-utf8.txt > +++ b/sql/share/errmsg-utf8.txt > @@ -7214,3 +7214,5 @@ ER_CALCULATING_DEFAULT_VALUE > eng "Got an error when calculating default value for %`s" > ER_EXPRESSION_REFERS_TO_UNINIT_FIELD 01000 > eng "Expression for field %`-.64s is refering to uninitialized field %`s" > +ER_DUP_CONSTRAINT_NAME > + eng "Duplicate constraint name '%-.192s'" Say "Duplicate CHECK constraint" here, because there is already ER_FK_DUP_NAME with the text "Duplicate foreign key constraint name" Alternatively, (better, I think) generalize ER_FK_DUP_NAME to say Duplicate %s constraint name: %`s (note correct quoting %` and no need to limit the length, it's an identifier anyway, it cannot be arbitrarily long). Then it can be used like my_error(ER_DUP_CONSTRAINT_NAME, MYF(0), "FOREIGN KEY", name); my_error(ER_DUP_CONSTRAINT_NAME, MYF(0), "CHECK", name); That also means renaming the error from ER_FK_DUP_NAME to ER_DUP_CONSTRAINT_NAME, and putting somewhere a compatibility define #define ER_FK_DUP_NAME ER_DUP_CONSTRAINT_NAME > diff --git a/sql/sql_yacc.yy b/sql/sql_yacc.yy > index 9d7e735..71f3692 100644 > --- a/sql/sql_yacc.yy > +++ b/sql/sql_yacc.yy > @@ -6070,7 +6070,11 @@ key_def: > constraint_def: > opt_constraint check_constraint > { > - Lex->add_constraint(&$1, $2); > + Lex->add_constraint(&$1, $2, FALSE); > + } > + | CONSTRAINT IF_SYM not EXISTS field_ident check_constraint > + { > + Lex->add_constraint(&$5, $6, TRUE); Eh, not here please. IF NOT EXIST should not be recognized inside CREATE TABLE :) > } > ; > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0