- developers - lists.mariadb.org

Re: [Maria-developers] [Commits] 3c7e2540f63: MDEV-17020: Assertion `length > 0' failed in ptr_compare upon ORDER BY with bad conversion
by Sergey Petrunia 12 Sep '18

12 Sep '18

Hi Varun, Looking at the testcase name, I have found Serg's patch for MDEV-4285. His patch skips sorting if the keys have zero length. He does it at a late stage, in Filesort_buffer::sort_buffer(). His patch doesn't cover the example of MDEV-17020, because the testcase in MDEV-17020 uses LIMIT and so uses a Priority Queue (and not Filesort_buffer::sort_buffer which uses qsort). However, your patch doesn't cover all cases either. There are other ways to produce zero-length keys. I have found this one: SELECT * FROM t1 ORDER BY 'foo', substring(pk, 0) LIMIT 2; it crashes in the same way as the test for MDEV-17020 used to do. Possible options: 1. Find all Items that might produce zero-length keys. It's difficult not to miss anything. 2. Instead of #1, add a universal logic "if sort key length is 0, don't sort". Is this doable? 3. Extend the Serg's approach. We cannot use his code directly ("dont sort"), because Priority Queue API doesn't easily allow it, but perhaps we could add a comparison function that will allow to compare zero-length keys? Or make them 1-byte long with a dummy value (do we need the sorting to be stable??) On Fri, Aug 31, 2018 at 12:18:08AM +0530, Varun wrote: > revision-id: 3c7e2540f63c6801a0d0a27fbea7c934bb61221f (mariadb-10.0.36-13-g3c7e2540f63) > parent(s): b3c320bb0b93e516cda4db277cfa3efeef48c988 > author: Varun Gupta > committer: Varun Gupta > timestamp: 2018-08-31 00:14:20 +0530 > message: > > MDEV-17020: Assertion `length > 0' failed in ptr_compare upon ORDER BY with bad conversion > > We have a case here where the the entire sort key is empty and so sorting is not needed. > The crash happens because filesort expects the sort key to have length > 0. > The case here is like ORDER BY CONVERT(a,char(0)), which if evaluated is a constant so > can be removed from the ORDER BY clause > > --- > mysql-test/r/order_by_zerolength-4285.result | 28 +++++++++++++++++----------- > mysql-test/t/order_by_zerolength-4285.test | 13 +++++++++++++ > sql/item_timefunc.h | 6 ++++++ > 3 files changed, 36 insertions(+), 11 deletions(-) > > diff --git a/mysql-test/r/order_by_zerolength-4285.result b/mysql-test/r/order_by_zerolength-4285.result > index f60ce7d90c7..f77b284df3c 100644 > --- a/mysql-test/r/order_by_zerolength-4285.result > +++ b/mysql-test/r/order_by_zerolength-4285.result > @@ -12,15 +12,21 @@ pk > 8 > 9 > 10 > -Warnings: > -Warning 1292 Truncated incorrect CHAR(0) value: '1' > -Warning 1292 Truncated incorrect CHAR(0) value: '2' > -Warning 1292 Truncated incorrect CHAR(0) value: '3' > -Warning 1292 Truncated incorrect CHAR(0) value: '4' > -Warning 1292 Truncated incorrect CHAR(0) value: '5' > -Warning 1292 Truncated incorrect CHAR(0) value: '6' > -Warning 1292 Truncated incorrect CHAR(0) value: '7' > -Warning 1292 Truncated incorrect CHAR(0) value: '8' > -Warning 1292 Truncated incorrect CHAR(0) value: '9' > -Warning 1292 Truncated incorrect CHAR(0) value: '10' > +drop table t1; > +# > +# MDEV-17020: Assertion `length > 0' failed in ptr_compare upon ORDER BY with bad conversion > +# > +set @save_sql_mode= @@sql_mode; > +SET @@sql_mode= ''; > +CREATE TABLE t1 (pk INT PRIMARY KEY); > +INSERT INTO t1 VALUES (1),(2); > +explain > +SELECT * FROM t1 ORDER BY 'foo', CONVERT(pk, CHAR(0)) LIMIT 2; > +id select_type table type possible_keys key key_len ref rows Extra > +1 SIMPLE t1 index NULL PRIMARY 4 NULL 2 Using index > +SELECT * FROM t1 ORDER BY 'foo', Cast(pk as CHAR(0)) LIMIT 2; > +pk > +1 > +2 > +set @@sql_mode= @save_sql_mode; > drop table t1; > diff --git a/mysql-test/t/order_by_zerolength-4285.test b/mysql-test/t/order_by_zerolength-4285.test > index 2fb58edd36d..f03d528320c 100644 > --- a/mysql-test/t/order_by_zerolength-4285.test > +++ b/mysql-test/t/order_by_zerolength-4285.test > @@ -6,3 +6,16 @@ insert into t1 values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10); > select * from t1 order by now(), cast(pk as char(0)); > drop table t1; > > +--echo # > +--echo # MDEV-17020: Assertion `length > 0' failed in ptr_compare upon ORDER BY with bad conversion > +--echo # > + > +set @save_sql_mode= @@sql_mode; > +SET @@sql_mode= ''; > +CREATE TABLE t1 (pk INT PRIMARY KEY); > +INSERT INTO t1 VALUES (1),(2); > +explain > +SELECT * FROM t1 ORDER BY 'foo', CONVERT(pk, CHAR(0)) LIMIT 2; > +SELECT * FROM t1 ORDER BY 'foo', Cast(pk as CHAR(0)) LIMIT 2; > +set @@sql_mode= @save_sql_mode; > +drop table t1; > diff --git a/sql/item_timefunc.h b/sql/item_timefunc.h > index 80b98758fb7..461c3feed8c 100644 > --- a/sql/item_timefunc.h > +++ b/sql/item_timefunc.h > @@ -903,6 +903,12 @@ class Item_char_typecast :public Item_str_func > String *val_str(String *a); > void fix_length_and_dec(); > void print(String *str, enum_query_type query_type); > + table_map used_tables() const > + { > + if (!cast_length) > + return (table_map) 0L; > + return Item_str_func::used_tables(); > + } > }; > > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits -- BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] 17540ad: MDEV-15890 Strange error message if you try to FLUSH TABLES <view> after LOCK TABLES <view>.
by Sergei Golubchik 04 Sep '18

04 Sep '18

Hi, Alexey! On Sep 03, Alexey Botchkov wrote: > revision-id: 17540adc625ff4ac0f00aa10796cb6e763838d18 (mariadb-10.1.35-25-g17540ad) > parent(s): 34e026f35a7fbeeaec412547ee207c880944ae68 > committer: Alexey Botchkov > timestamp: 2018-09-03 16:49:46 +0400 > message: > > MDEV-15890 Strange error message if you try to FLUSH TABLES <view> after LOCK TABLES <view>. > > LOCK view WRITE shouldn't block FLUSH view. > So we set the view's mdl_request type to it's tables. > > diff --git a/sql/sql_view.cc b/sql/sql_view.cc > index 6bd6b6a..d7a2a6b 100644 > --- a/sql/sql_view.cc > +++ b/sql/sql_view.cc > @@ -1534,8 +1534,7 @@ bool mysql_make_view(THD *thd, TABLE_SHARE *share, TABLE_LIST *table, > for (tbl= view_main_select_tables; tbl; tbl= tbl->next_local) > { > tbl->lock_type= table->lock_type; > - tbl->mdl_request.set_type((tbl->lock_type >= TL_WRITE_ALLOW_WRITE) ? > - MDL_SHARED_WRITE : MDL_SHARED_READ); > + tbl->mdl_request.set_type(table->mdl_request.type); > } I agree it's looks like a proper fix. But I'm a bit cautious about possible side effects. A more conservative fix would be only change the lock level for TL_NO_READ_WRITE, like tbl->lock_type == TL_NO_READ_WRITE ? TL_NO_READ_WRITE : tbl->lock_type >= TL_WRITE_ALLOW_WRITE ? MDL_SHARED_WRITE : MDL_SHARED_READ (or with an if(), for readability). But I don't see whether it'll make any difference as compared to your fix. Please, run all tests locally, and I mean *all*, big, ps, embedded. And see if you can think of any cases where this change would cause a change in behavior. After that - ok to push! Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] 2 pull requests
by Ming Lin 04 Sep '18

04 Sep '18

Hi Sergey, Could you take a look at these 2 PRs? MDEV-16768: https://github.com/MariaDB/server/pull/837 You approved this one, but not merged yet. MDEV-15816: https://github.com/MariaDB/server/pull/849 This one was already merged by facebook. Thanks, Ming

1 1

[Maria-developers] Binlog handling of faulty DDL query
by Sachin Setiya 03 Sep '18

03 Sep '18

Hi Everyone! Suppose this case CREATE USER user1@localhost IDENTIFIED BY 'BsG9#9.cem#!85', user2@localhost IDENTIFIED BY 'x'; user2 has too short passowrd which will give error (if we use security plugin) IN the case of *mariadb* we will create user1 and wont create user2 In the case of *mysql (8.0)* we will return error and we will not create any user Our approach create a lots of problem for binary log and replication(MDEV-14716, MDEV-16728) Because in Query_log_event::write() we write the whole query not the parts of it. So we have 2 options to solve this either write modified query into binlog or do what mysql does. So what you think , which approach we should use ? Or there is another better way ? -- Regards sachin

5 6

Re: [Maria-developers] 63ad6a9: MDEV-15890 Strange error message if you try to FLUSH TABLES <view> after LOCK TABLES <view>.
by Sergei Golubchik 02 Sep '18

02 Sep '18

Hi, Alexey! On Sep 02, Alexey Botchkov wrote: > revision-id: 63ad6a9e1a33ddd5547767b2894e09ae66196f69 (mariadb-10.1.35-23-g63ad6a9) > parent(s): 288212f489c8cd88c4cc98f8aecc3366c85a90be > committer: Alexey Botchkov > timestamp: 2018-09-02 09:24:33 +0400 > message: > > MDEV-15890 Strange error message if you try to FLUSH TABLES <view> after LOCK TABLES <view>. > > Check if the argument of the FLUSH TABLE is a VIEW and handle it > accordingly. > > diff --git a/mysql-test/r/flush.result b/mysql-test/r/flush.result > index b643510..2d7b81b 100644 > --- a/mysql-test/r/flush.result > +++ b/mysql-test/r/flush.result > @@ -496,3 +496,27 @@ flush relay logs,relay logs; > ERROR HY000: Incorrect usage of FLUSH and RELAY LOGS > flush slave,slave; > ERROR HY000: Incorrect usage of FLUSH and SLAVE > +# > +# MDEV-15890 Strange error message if you try to > +# FLUSH TABLES <view> after LOCK TABLES <view>. > +# > +CREATE TABLE t1 (qty INT, price INT); > +CREATE VIEW v1 AS SELECT qty, price, qty*price AS value FROM t1; > +LOCK TABLES v1 READ; > +FLUSH TABLES v1; > +ERROR HY000: Table 't1' was locked with a READ lock and can't be updated > +UNLOCK TABLES; > +LOCK TABLES v1 WRITE; > +FLUSH TABLES v1; > +ERROR HY000: Table 't1' was locked with a READ lock and can't be updated Why is that? Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] MariaDB v10 on AIX
by REIX, Tony 31 Aug '18

31 Aug '18

Hi, We are porting MariaDB v10(.3.7/9) on AIX now. First, I'd like to know if there are people porting (or who has ported) MariaDB on AIX in this mailing list. Second, we have found some issue dealing with: AIX, -O0, and inline, which leads to a symbol missing at link time: my_ulonglong2double . It deals with the code below. Since we are beginners with MariaDB, I'd like to have some comments and/or recommendations about how it should be fixed. By using "static __inline" instead of "inline" below, and adding it too to the ma_global.h definition of my_ulonglong2double() , we are able to fix it. However, probably that using: #define my_ulonglong2double(nr) ((double) nr) would be enough. Or adding my_port.c to the MYSYS_SOURCES variable in mysys/CMakeLists.txt would help: IF (AIX) SET (MYSYS_SOURCES ${MYSYS_SOURCES} my_port.c) ENDIF() however, that does not work, and I do not see my_port.c being added to the list of files to be added to mysys. Any idea why "IF (AIX)" does not work? (I'm also a CMake beginner). Thanks/Regards, Tony Reix mysys/my_port.c: #include "mysys_priv.h" #ifdef _AIX double my_ulonglong2double(unsigned long long nr) { return (double) nr; } mysys/mysys_priv.h: #include <my_global.h> mysys/CMakeLists.txt : SET(MYSYS_SOURCES ........ No my_port.c !! include/my_global.h: #ifdef _AIX /* By soren(a)t.dk */ #define ulonglong2double(A) my_ulonglong2double(A) C_MODE_START inline double my_ulonglong2double(unsigned long long A) { return (double)A; } C_MODE_END libmariadb/include/ma_global.h: #ifdef _AIX /* By soren(a)t.dk */ #define ulonglong2double(A) my_ulonglong2double(A) #ifdef __cplusplus extern "C" { #endif double my_ulonglong2double(unsigned long long A); déclaration externe #ifdef __cplusplus Cordialement, Tony Reix tony.reix(a)atos.net ATOS / Bull SAS ATOS Expert IBM Coop Architect & Technical Leader Office : +33 (0) 4 76 29 72 67 1 rue de Provence - 38432 Échirolles - France www.atos.net<https://mail.ad.bull.net/owa/redir.aspx?C=PvphmPvCZkGrAgHVnWGsdMcDKgzl_dEIs…>

2 1

Re: [Maria-developers] 44692f17a62: MDEV-15436: If log_bin and log_bin_index is different SST with rsync fails.
by Sergei Golubchik 31 Aug '18

31 Aug '18

Hi, Jan! On Aug 31, serg(a)mariadb.org wrote: > revision-id: 44692f17a621d7f0261d84b97a7334352463038c (mariadb-10.2.16-6-g44692f17a62) > parent(s): 7d0d934ca642e485b2c008727dc20c83e26cce10 > author: Jan Lindström > committer: Jan Lindström > timestamp: 2018-06-26 12:56:19 +0300 > message: > > MDEV-15436: If log_bin and log_bin_index is different SST with rsync fails. > > Problem was that in SST log_bin_index name and directory was not > handled and passed to rsync SST script. > > mysql-test/suite/galera/r/galera_sst_rsync2.result | 396 +++++++++++++++++++++ > mysql-test/suite/galera/t/galera_sst_rsync2.cnf | 15 + > mysql-test/suite/galera/t/galera_sst_rsync2.test | 12 + I didn't review the test, but I presume you've verified that it fails without the bug fix. > diff --git a/scripts/wsrep_sst_rsync.sh b/scripts/wsrep_sst_rsync.sh > index 05c99880d75..6886d2ce1e1 100644 > --- a/scripts/wsrep_sst_rsync.sh > +++ b/scripts/wsrep_sst_rsync.sh > @@ -132,6 +132,12 @@ then > BINLOG_FILENAME=$(basename $WSREP_SST_OPT_BINLOG) > fi > > +if ! [ -z $WSREP_SST_OPT_BINLOG_INDEX ] > +then > + BINLOG_INDEX_DIRNAME=$(dirname $WSREP_SST_OPT_BINLOG_INDEX) > + BINLOG_INDEX_FILENAME=$(basename $WSREP_SST_OPT_BINLOG_INDEX) > +fi > + > WSREP_LOG_DIR=${WSREP_LOG_DIR:-""} > # if WSREP_LOG_DIR env. variable is not set, try to get it from my.cnf > if [ -z "$WSREP_LOG_DIR" ]; then > @@ -204,12 +210,20 @@ then > OLD_PWD="$(pwd)" > cd $BINLOG_DIRNAME > > - binlog_files_full=$(tail -n $BINLOG_N_FILES ${BINLOG_FILENAME}.index) > + if ! [ -z $WSREP_SST_OPT_BINLOG_INDEX ] first, I think it'd be simpler to do, like if [ -z $WSREP_SST_OPT_BINLOG_INDEX ] then WSREP_SST_OPT_BINLOG_INDEX=$WSREP_SST_OPT_BINLOG.index fi it's enough to do it once before setting BINLOG_INDEX_DIRNAME and BINLOG_INDEX_FILENAME and you woudn't need if's here and below. second, you apparently are trying to take into account that binlog index may be a different directory. Please use this configuration in your test case, put binlog index in a different directory. otherwise ok. please, push after adjusting the test case to put binlog index in a different directory. > + binlog_files_full=$(tail -n $BINLOG_N_FILES ${BINLOG_FILENAME}.index) > + then > + cd $BINLOG_INDEX_DIRNAME > + binlog_files_full=$(tail -n $BINLOG_N_FILES ${BINLOG_INDEX_FILENAME}.index) > + fi > + > + cd $BINLOG_DIRNAME > binlog_files="" > for ii in $binlog_files_full > do > binlog_files="$binlog_files $(basename $ii)" > done > + > if ! [ -z "$binlog_files" ] > then > wsrep_log_info "Preparing binlog files for transfer:" > @@ -391,7 +405,11 @@ EOF > tar -xvf $BINLOG_TAR_FILE >&2 > for ii in $(ls -1 ${BINLOG_FILENAME}.*) > do > - echo ${BINLOG_DIRNAME}/${ii} >> ${BINLOG_FILENAME}.index > + if ! [ -z $WSREP_SST_OPT_BINLOG_INDEX ] > + echo ${BINLOG_DIRNAME}/${ii} >> ${BINLOG_FILENAME}.index > + then > + echo ${BINLOG_DIRNAME}/${ii} >> ${BINLOG_INDEX_DIRNAME}/${BINLOG_INDEX_FILENAME}.index > + fi > done > fi > cd "$OLD_PWD" Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] 6be8bd6: MDEV-15890 Strange error message if you try to FLUSH TABLES <view> after LOCK TABLES <view>.
by Sergei Golubchik 31 Aug '18

31 Aug '18

Hi, Alexey! On Jun 24, Alexey Botchkov wrote: > revision-id: 6be8bd6ee05c6f18b6a6a0375f9bc2b68ef6c43e (mariadb-10.3.7-59-g6be8bd6) > parent(s): 1abd877e2df9e83bc1c2f5195796f427a35bd3f1 > committer: Alexey Botchkov > timestamp: 2018-06-24 23:12:20 +0400 > message: > > MDEV-15890 Strange error message if you try to FLUSH TABLES <view> after LOCK TABLES <view>. > > Check if the argument of the FLUSH TABLE is a VIEW and handle it > accordingly. Looks great! But please add more tests. A test with a write lock, and tests with locking the view and flushing the table or vice versa. After that ok to push! Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] 10.2 and merges (e.g. TokuDB)
by Phil Sweeney 28 Aug '18

28 Aug '18

Hi there, It seems 10.2.6 is still shipping with TokuDB from PS 5.6.35-80 (Feb 8th 2017). I can't see a jira issue for a 10.2 merge since 10.2.4. Perhaps intentional to help stabilise 10.2 for release? 10.0.30 also shipped with the same Feb version (in March). 10.1.23 however was released in May and still has this version from Feb, which seems unfortunate, but I know it's because it has to flow through 10.0 first. I see 10.0.31 is due to get TokuDB from PS 5.6.36-82 And that should flow into 10.1.24 soon too. However, 10.2.7 isn't due until July, so by then TokuDB version will be 5 months old in 10.2 I'm waiting on a fairly important row count fix that should appear in the next Percona Server release. When it does, if the current merging strategy continues, do I have to first wait for 10.0.32 to get it in late July, then 10.1.26 in August, then 10.2 after that? Or with 10.0, 10.1 and 10.2 all being supported and needing merges, is there a new strategy on the cards to shorten this lag? Thanks! Phil

3 7

[Maria-developers] MDEV-371 Long unique Index
by Sachin Setiya 28 Aug '18

28 Aug '18

Hi Serg! Please review the patch for Unique Index For Long (MDEV-371) (It is in bb-10.4-blob-index branch) Architecture:- We will create a virtual column DB_ROW_HASH_X (X is any number which makes this field name unique) for each long unique column which will store hash. At the time of ha_write_row we will check for hash collusions if we find same hash then we will retrieve the blob and then compare byte by byte. key_info for long unique have one extra key_part which will point to hash field. For example unique(a,b) key will have 3 key_parts :- a ,b DB_ROW_HASH_X But on server layer we can just see a,b parts on storage layer we can just see DB_ROW_HASH_X key_part. I have created 2 function setup_table_hash and resetup_table which helps in this transition. If user specify length of blob then we use LEFT function to trim the value. -- Regards Sachin Setiya Software Engineer at MariaDB

1 0