- developers - lists.mariadb.org

Re: [Maria-developers] 17540ad: MDEV-15890 Strange error message if you try to FLUSH TABLES <view> after LOCK TABLES <view>.
by Sergei Golubchik 04 Sep '18

04 Sep '18

Hi, Alexey! On Sep 03, Alexey Botchkov wrote: > revision-id: 17540adc625ff4ac0f00aa10796cb6e763838d18 (mariadb-10.1.35-25-g17540ad) > parent(s): 34e026f35a7fbeeaec412547ee207c880944ae68 > committer: Alexey Botchkov > timestamp: 2018-09-03 16:49:46 +0400 > message: > > MDEV-15890 Strange error message if you try to FLUSH TABLES <view> after LOCK TABLES <view>. > > LOCK view WRITE shouldn't block FLUSH view. > So we set the view's mdl_request type to it's tables. > > diff --git a/sql/sql_view.cc b/sql/sql_view.cc > index 6bd6b6a..d7a2a6b 100644 > --- a/sql/sql_view.cc > +++ b/sql/sql_view.cc > @@ -1534,8 +1534,7 @@ bool mysql_make_view(THD *thd, TABLE_SHARE *share, TABLE_LIST *table, > for (tbl= view_main_select_tables; tbl; tbl= tbl->next_local) > { > tbl->lock_type= table->lock_type; > - tbl->mdl_request.set_type((tbl->lock_type >= TL_WRITE_ALLOW_WRITE) ? > - MDL_SHARED_WRITE : MDL_SHARED_READ); > + tbl->mdl_request.set_type(table->mdl_request.type); > } I agree it's looks like a proper fix. But I'm a bit cautious about possible side effects. A more conservative fix would be only change the lock level for TL_NO_READ_WRITE, like tbl->lock_type == TL_NO_READ_WRITE ? TL_NO_READ_WRITE : tbl->lock_type >= TL_WRITE_ALLOW_WRITE ? MDL_SHARED_WRITE : MDL_SHARED_READ (or with an if(), for readability). But I don't see whether it'll make any difference as compared to your fix. Please, run all tests locally, and I mean *all*, big, ps, embedded. And see if you can think of any cases where this change would cause a change in behavior. After that - ok to push! Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] 2 pull requests
by Ming Lin 04 Sep '18

04 Sep '18

Hi Sergey, Could you take a look at these 2 PRs? MDEV-16768: https://github.com/MariaDB/server/pull/837 You approved this one, but not merged yet. MDEV-15816: https://github.com/MariaDB/server/pull/849 This one was already merged by facebook. Thanks, Ming

1 1

[Maria-developers] Binlog handling of faulty DDL query
by Sachin Setiya 03 Sep '18

03 Sep '18

Hi Everyone! Suppose this case CREATE USER user1@localhost IDENTIFIED BY 'BsG9#9.cem#!85', user2@localhost IDENTIFIED BY 'x'; user2 has too short passowrd which will give error (if we use security plugin) IN the case of *mariadb* we will create user1 and wont create user2 In the case of *mysql (8.0)* we will return error and we will not create any user Our approach create a lots of problem for binary log and replication(MDEV-14716, MDEV-16728) Because in Query_log_event::write() we write the whole query not the parts of it. So we have 2 options to solve this either write modified query into binlog or do what mysql does. So what you think , which approach we should use ? Or there is another better way ? -- Regards sachin

5 6

Re: [Maria-developers] 63ad6a9: MDEV-15890 Strange error message if you try to FLUSH TABLES <view> after LOCK TABLES <view>.
by Sergei Golubchik 02 Sep '18

02 Sep '18

Hi, Alexey! On Sep 02, Alexey Botchkov wrote: > revision-id: 63ad6a9e1a33ddd5547767b2894e09ae66196f69 (mariadb-10.1.35-23-g63ad6a9) > parent(s): 288212f489c8cd88c4cc98f8aecc3366c85a90be > committer: Alexey Botchkov > timestamp: 2018-09-02 09:24:33 +0400 > message: > > MDEV-15890 Strange error message if you try to FLUSH TABLES <view> after LOCK TABLES <view>. > > Check if the argument of the FLUSH TABLE is a VIEW and handle it > accordingly. > > diff --git a/mysql-test/r/flush.result b/mysql-test/r/flush.result > index b643510..2d7b81b 100644 > --- a/mysql-test/r/flush.result > +++ b/mysql-test/r/flush.result > @@ -496,3 +496,27 @@ flush relay logs,relay logs; > ERROR HY000: Incorrect usage of FLUSH and RELAY LOGS > flush slave,slave; > ERROR HY000: Incorrect usage of FLUSH and SLAVE > +# > +# MDEV-15890 Strange error message if you try to > +# FLUSH TABLES <view> after LOCK TABLES <view>. > +# > +CREATE TABLE t1 (qty INT, price INT); > +CREATE VIEW v1 AS SELECT qty, price, qty*price AS value FROM t1; > +LOCK TABLES v1 READ; > +FLUSH TABLES v1; > +ERROR HY000: Table 't1' was locked with a READ lock and can't be updated > +UNLOCK TABLES; > +LOCK TABLES v1 WRITE; > +FLUSH TABLES v1; > +ERROR HY000: Table 't1' was locked with a READ lock and can't be updated Why is that? Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] MariaDB v10 on AIX
by REIX, Tony 31 Aug '18

31 Aug '18

Hi, We are porting MariaDB v10(.3.7/9) on AIX now. First, I'd like to know if there are people porting (or who has ported) MariaDB on AIX in this mailing list. Second, we have found some issue dealing with: AIX, -O0, and inline, which leads to a symbol missing at link time: my_ulonglong2double . It deals with the code below. Since we are beginners with MariaDB, I'd like to have some comments and/or recommendations about how it should be fixed. By using "static __inline" instead of "inline" below, and adding it too to the ma_global.h definition of my_ulonglong2double() , we are able to fix it. However, probably that using: #define my_ulonglong2double(nr) ((double) nr) would be enough. Or adding my_port.c to the MYSYS_SOURCES variable in mysys/CMakeLists.txt would help: IF (AIX) SET (MYSYS_SOURCES ${MYSYS_SOURCES} my_port.c) ENDIF() however, that does not work, and I do not see my_port.c being added to the list of files to be added to mysys. Any idea why "IF (AIX)" does not work? (I'm also a CMake beginner). Thanks/Regards, Tony Reix mysys/my_port.c: #include "mysys_priv.h" #ifdef _AIX double my_ulonglong2double(unsigned long long nr) { return (double) nr; } mysys/mysys_priv.h: #include <my_global.h> mysys/CMakeLists.txt : SET(MYSYS_SOURCES ........ No my_port.c !! include/my_global.h: #ifdef _AIX /* By soren(a)t.dk */ #define ulonglong2double(A) my_ulonglong2double(A) C_MODE_START inline double my_ulonglong2double(unsigned long long A) { return (double)A; } C_MODE_END libmariadb/include/ma_global.h: #ifdef _AIX /* By soren(a)t.dk */ #define ulonglong2double(A) my_ulonglong2double(A) #ifdef __cplusplus extern "C" { #endif double my_ulonglong2double(unsigned long long A); déclaration externe #ifdef __cplusplus Cordialement, Tony Reix tony.reix(a)atos.net ATOS / Bull SAS ATOS Expert IBM Coop Architect & Technical Leader Office : +33 (0) 4 76 29 72 67 1 rue de Provence - 38432 Échirolles - France www.atos.net<https://mail.ad.bull.net/owa/redir.aspx?C=PvphmPvCZkGrAgHVnWGsdMcDKgzl_dEIs…>

2 1

Re: [Maria-developers] 44692f17a62: MDEV-15436: If log_bin and log_bin_index is different SST with rsync fails.
by Sergei Golubchik 31 Aug '18

31 Aug '18

Hi, Jan! On Aug 31, serg(a)mariadb.org wrote: > revision-id: 44692f17a621d7f0261d84b97a7334352463038c (mariadb-10.2.16-6-g44692f17a62) > parent(s): 7d0d934ca642e485b2c008727dc20c83e26cce10 > author: Jan Lindström > committer: Jan Lindström > timestamp: 2018-06-26 12:56:19 +0300 > message: > > MDEV-15436: If log_bin and log_bin_index is different SST with rsync fails. > > Problem was that in SST log_bin_index name and directory was not > handled and passed to rsync SST script. > > mysql-test/suite/galera/r/galera_sst_rsync2.result | 396 +++++++++++++++++++++ > mysql-test/suite/galera/t/galera_sst_rsync2.cnf | 15 + > mysql-test/suite/galera/t/galera_sst_rsync2.test | 12 + I didn't review the test, but I presume you've verified that it fails without the bug fix. > diff --git a/scripts/wsrep_sst_rsync.sh b/scripts/wsrep_sst_rsync.sh > index 05c99880d75..6886d2ce1e1 100644 > --- a/scripts/wsrep_sst_rsync.sh > +++ b/scripts/wsrep_sst_rsync.sh > @@ -132,6 +132,12 @@ then > BINLOG_FILENAME=$(basename $WSREP_SST_OPT_BINLOG) > fi > > +if ! [ -z $WSREP_SST_OPT_BINLOG_INDEX ] > +then > + BINLOG_INDEX_DIRNAME=$(dirname $WSREP_SST_OPT_BINLOG_INDEX) > + BINLOG_INDEX_FILENAME=$(basename $WSREP_SST_OPT_BINLOG_INDEX) > +fi > + > WSREP_LOG_DIR=${WSREP_LOG_DIR:-""} > # if WSREP_LOG_DIR env. variable is not set, try to get it from my.cnf > if [ -z "$WSREP_LOG_DIR" ]; then > @@ -204,12 +210,20 @@ then > OLD_PWD="$(pwd)" > cd $BINLOG_DIRNAME > > - binlog_files_full=$(tail -n $BINLOG_N_FILES ${BINLOG_FILENAME}.index) > + if ! [ -z $WSREP_SST_OPT_BINLOG_INDEX ] first, I think it'd be simpler to do, like if [ -z $WSREP_SST_OPT_BINLOG_INDEX ] then WSREP_SST_OPT_BINLOG_INDEX=$WSREP_SST_OPT_BINLOG.index fi it's enough to do it once before setting BINLOG_INDEX_DIRNAME and BINLOG_INDEX_FILENAME and you woudn't need if's here and below. second, you apparently are trying to take into account that binlog index may be a different directory. Please use this configuration in your test case, put binlog index in a different directory. otherwise ok. please, push after adjusting the test case to put binlog index in a different directory. > + binlog_files_full=$(tail -n $BINLOG_N_FILES ${BINLOG_FILENAME}.index) > + then > + cd $BINLOG_INDEX_DIRNAME > + binlog_files_full=$(tail -n $BINLOG_N_FILES ${BINLOG_INDEX_FILENAME}.index) > + fi > + > + cd $BINLOG_DIRNAME > binlog_files="" > for ii in $binlog_files_full > do > binlog_files="$binlog_files $(basename $ii)" > done > + > if ! [ -z "$binlog_files" ] > then > wsrep_log_info "Preparing binlog files for transfer:" > @@ -391,7 +405,11 @@ EOF > tar -xvf $BINLOG_TAR_FILE >&2 > for ii in $(ls -1 ${BINLOG_FILENAME}.*) > do > - echo ${BINLOG_DIRNAME}/${ii} >> ${BINLOG_FILENAME}.index > + if ! [ -z $WSREP_SST_OPT_BINLOG_INDEX ] > + echo ${BINLOG_DIRNAME}/${ii} >> ${BINLOG_FILENAME}.index > + then > + echo ${BINLOG_DIRNAME}/${ii} >> ${BINLOG_INDEX_DIRNAME}/${BINLOG_INDEX_FILENAME}.index > + fi > done > fi > cd "$OLD_PWD" Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] 6be8bd6: MDEV-15890 Strange error message if you try to FLUSH TABLES <view> after LOCK TABLES <view>.
by Sergei Golubchik 31 Aug '18

31 Aug '18

Hi, Alexey! On Jun 24, Alexey Botchkov wrote: > revision-id: 6be8bd6ee05c6f18b6a6a0375f9bc2b68ef6c43e (mariadb-10.3.7-59-g6be8bd6) > parent(s): 1abd877e2df9e83bc1c2f5195796f427a35bd3f1 > committer: Alexey Botchkov > timestamp: 2018-06-24 23:12:20 +0400 > message: > > MDEV-15890 Strange error message if you try to FLUSH TABLES <view> after LOCK TABLES <view>. > > Check if the argument of the FLUSH TABLE is a VIEW and handle it > accordingly. Looks great! But please add more tests. A test with a write lock, and tests with locking the view and flushing the table or vice versa. After that ok to push! Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] 10.2 and merges (e.g. TokuDB)
by Phil Sweeney 28 Aug '18

28 Aug '18

Hi there, It seems 10.2.6 is still shipping with TokuDB from PS 5.6.35-80 (Feb 8th 2017). I can't see a jira issue for a 10.2 merge since 10.2.4. Perhaps intentional to help stabilise 10.2 for release? 10.0.30 also shipped with the same Feb version (in March). 10.1.23 however was released in May and still has this version from Feb, which seems unfortunate, but I know it's because it has to flow through 10.0 first. I see 10.0.31 is due to get TokuDB from PS 5.6.36-82 And that should flow into 10.1.24 soon too. However, 10.2.7 isn't due until July, so by then TokuDB version will be 5 months old in 10.2 I'm waiting on a fairly important row count fix that should appear in the next Percona Server release. When it does, if the current merging strategy continues, do I have to first wait for 10.0.32 to get it in late July, then 10.1.26 in August, then 10.2 after that? Or with 10.0, 10.1 and 10.2 all being supported and needing merges, is there a new strategy on the cards to shorten this lag? Thanks! Phil

3 7

[Maria-developers] MDEV-371 Long unique Index
by Sachin Setiya 28 Aug '18

28 Aug '18

Hi Serg! Please review the patch for Unique Index For Long (MDEV-371) (It is in bb-10.4-blob-index branch) Architecture:- We will create a virtual column DB_ROW_HASH_X (X is any number which makes this field name unique) for each long unique column which will store hash. At the time of ha_write_row we will check for hash collusions if we find same hash then we will retrieve the blob and then compare byte by byte. key_info for long unique have one extra key_part which will point to hash field. For example unique(a,b) key will have 3 key_parts :- a ,b DB_ROW_HASH_X But on server layer we can just see a,b parts on storage layer we can just see DB_ROW_HASH_X key_part. I have created 2 function setup_table_hash and resetup_table which helps in this transition. If user specify length of blob then we use LEFT function to trim the value. -- Regards Sachin Setiya Software Engineer at MariaDB

1 0

Re: [Maria-developers] [Commits] f522fb9: MDEV-16934 Query with very large IN clause lists runs slowly
by Sergey Petrunia 17 Aug '18

17 Aug '18

Hello Igor, I am debugging with the patch and I see that PARAM::range_count is not reset after the eq_ranges_exceeds_limit() call. This causes different kinds of undesired effects. For example, here create table ten(a int); insert into ten values (0),(1),(2),(3),(4),(5),(6),(7),(8),(9); create table t1 (pk int, kp1 int, kp2 int, kp3 int, col1 int, key(kp1,kp2,kp3), primary key(pk)); insert into t1 select A.a + 10*B.a+100* C.a, A.a, B.a , C.a, 1234 from ten A, ten B, ten C; alter table t1 add key3 int, add key2 int; update t1 set key3=kp1; update t1 set key2=kp1; alter table t1 add key(key2), add key(key3); set eq_range_index_dive_limit=10; explain format=json select 1 from t1 force index (key2, key3) where key3=1 or key2=1; index_merge will use Sort-Union instead of Union. I assume we are targeting only IN (...) cases? Both our and MySQL's fixes will still do many records_in_range calls for queries like keypart1 IN (1,2,3,...) AND keypart2 IN (1,2,3,...) AND keypart3 >= 10 (because here the ranges will not be equality ranges). I assume this is outside of the scope of this fix? If so, then the patch is ok to push after PARAM::range_count issue is resolved. On Wed, Aug 15, 2018 at 07:48:30PM -0700, IgorBabaev wrote: > revision-id: f522fb9f63f88195bccbbc0326a4a318cc2d3db0 (mariadb-10.2.16-50-gf522fb9) > parent(s): 3c141e319ab29afdfe843553da385fe5d981906e > author: Igor Babaev > committer: Igor Babaev > timestamp: 2018-08-15 19:48:30 -0700 > message: > > MDEV-16934 Query with very large IN clause lists runs slowly > > This patch introduces support for the system variable eq_range_index_dive_limit > that existed in MySQL starting from 5.6. The variable sets a limit for > index dives into equality ranges. Index dives are performed by optimizer > to estimate the number of rows in range scans. Index dives usually provide > good estimate but they are pretty expensive. To estimate the number of rows > in equality ranges statistical data on indexes can be employed. Its usage gives > not so good estimates but it's cheap. So if the number of equality dives > required by an index scan exceeds the set limit no dives for equality > ranges are performed by the optimizer for this index. > > As the new system variable is introduced in a stable version the default > value for it is set to a special value meaning there is no limit for the number > of index dives performed by the optimizer. > > The patch partially uses the MySQL code for WL 5957 > 'Statistics-based Range optimization for many ranges'. > > --- > mysql-test/r/mysqld--help.result | 6 +++ > mysql-test/r/range.result | 41 +++++++++++++++++ > mysql-test/r/range_mrr_icp.result | 41 +++++++++++++++++ > .../sys_vars/r/sysvars_server_embedded.result | 14 ++++++ > .../sys_vars/r/sysvars_server_notembedded.result | 14 ++++++ > mysql-test/t/range.test | 33 ++++++++++++++ > sql/multi_range_read.cc | 11 +++++ > sql/opt_range.cc | 27 +++++++++++ > sql/opt_range.h | 4 +- > sql/opt_range_mrr.cc | 53 +++++++++++++++------- > sql/sql_class.h | 1 + > sql/sql_statistics.h | 2 +- > sql/sys_vars.cc | 10 ++++ > 13 files changed, 238 insertions(+), 19 deletions(-) > > diff --git a/mysql-test/r/mysqld--help.result b/mysql-test/r/mysqld--help.result > index 3b7d5e2..51ece33 100644 > --- a/mysql-test/r/mysqld--help.result > +++ b/mysql-test/r/mysqld--help.result > @@ -197,6 +197,11 @@ The following specify which files/extra groups are read (specified before remain > cache, etc) > --enforce-storage-engine=name > Force the use of a storage engine for new tables > + --eq-range-index-dive-limit=# > + The optimizer will use existing index statistics instead > + of doing index dives for equality ranges if the number of > + equality ranges for the index is larger than or equal to > + this number. If set to 0, index dives are always used. > --event-scheduler[=name] > Enable the event scheduler. Possible values are ON, OFF, > and DISABLED (keep the event scheduler completely > @@ -1259,6 +1264,7 @@ encrypt-binlog FALSE > encrypt-tmp-disk-tables FALSE > encrypt-tmp-files FALSE > enforce-storage-engine (No default value) > +eq-range-index-dive-limit 0 > event-scheduler OFF > expensive-subquery-limit 100 > expire-logs-days 0 > diff --git a/mysql-test/r/range.result b/mysql-test/r/range.result > index 3a71d08..0c6be5e 100644 > --- a/mysql-test/r/range.result > +++ b/mysql-test/r/range.result > @@ -3002,5 +3002,46 @@ deallocate prepare stmt; > set optimizer_switch=@save_optimizer_switch; > drop table t1,t2,t3; > # > +# MDEV-16934: using system variable eq_range_index_dive_limit > +# to reduce the number of index dives > +# > +create table t1 (a int, b varchar(31), index idx(a)); > +insert into t1 values > +(7,'xxxx'), (1,'yy'), (3,'aaa'), (1,'bbb'), (2,'zz'), > +(4,'vvvvv'), (7,'ddd'), (9,'zzzzz'), (1,'cc'), (5,'ffff'); > +insert into t1 select a+10, concat(b,'zz') from t1; > +insert into t1 select a+15, concat(b,'yy') from t1; > +insert into t1 select a+100, concat(b,'xx') from t1; > +analyze table t1; > +Table Op Msg_type Msg_text > +test.t1 analyze status OK > +select cast(count(a)/count(distinct a) as unsigned) as rec_per_key from t1; > +rec_per_key > +2 > +set eq_range_index_dive_limit=0; > +explain select * from t1 where a in (8, 15, 31, 1, 9); > +id select_type table type possible_keys key key_len ref rows Extra > +1 SIMPLE t1 range idx idx 5 NULL 7 Using index condition > +select * from t1 where a in (8, 15, 31, 1, 9); > +a b > +1 yy > +1 bbb > +1 cc > +9 zzzzz > +15 ffffzz > +set eq_range_index_dive_limit=2; > +explain select * from t1 where a in (8, 15, 31, 1, 9); > +id select_type table type possible_keys key key_len ref rows Extra > +1 SIMPLE t1 range idx idx 5 NULL 10 Using index condition > +select * from t1 where a in (8, 15, 31, 1, 9); > +a b > +1 yy > +1 bbb > +1 cc > +9 zzzzz > +15 ffffzz > +set eq_range_index_dive_limit=default; > +drop table t1; > +# > # End of 10.2 tests > # > diff --git a/mysql-test/r/range_mrr_icp.result b/mysql-test/r/range_mrr_icp.result > index 799a299..93f414f 100644 > --- a/mysql-test/r/range_mrr_icp.result > +++ b/mysql-test/r/range_mrr_icp.result > @@ -3014,6 +3014,47 @@ deallocate prepare stmt; > set optimizer_switch=@save_optimizer_switch; > drop table t1,t2,t3; > # > +# MDEV-16934: using system variable eq_range_index_dive_limit > +# to reduce the number of index dives > +# > +create table t1 (a int, b varchar(31), index idx(a)); > +insert into t1 values > +(7,'xxxx'), (1,'yy'), (3,'aaa'), (1,'bbb'), (2,'zz'), > +(4,'vvvvv'), (7,'ddd'), (9,'zzzzz'), (1,'cc'), (5,'ffff'); > +insert into t1 select a+10, concat(b,'zz') from t1; > +insert into t1 select a+15, concat(b,'yy') from t1; > +insert into t1 select a+100, concat(b,'xx') from t1; > +analyze table t1; > +Table Op Msg_type Msg_text > +test.t1 analyze status OK > +select cast(count(a)/count(distinct a) as unsigned) as rec_per_key from t1; > +rec_per_key > +2 > +set eq_range_index_dive_limit=0; > +explain select * from t1 where a in (8, 15, 31, 1, 9); > +id select_type table type possible_keys key key_len ref rows Extra > +1 SIMPLE t1 range idx idx 5 NULL 7 Using index condition; Rowid-ordered scan > +select * from t1 where a in (8, 15, 31, 1, 9); > +a b > +1 yy > +1 bbb > +9 zzzzz > +1 cc > +15 ffffzz > +set eq_range_index_dive_limit=2; > +explain select * from t1 where a in (8, 15, 31, 1, 9); > +id select_type table type possible_keys key key_len ref rows Extra > +1 SIMPLE t1 range idx idx 5 NULL 10 Using index condition; Rowid-ordered scan > +select * from t1 where a in (8, 15, 31, 1, 9); > +a b > +1 yy > +1 bbb > +9 zzzzz > +1 cc > +15 ffffzz > +set eq_range_index_dive_limit=default; > +drop table t1; > +# > # End of 10.2 tests > # > set optimizer_switch=@mrr_icp_extra_tmp; > diff --git a/mysql-test/suite/sys_vars/r/sysvars_server_embedded.result b/mysql-test/suite/sys_vars/r/sysvars_server_embedded.result > index f7a5cf5..9d7fa10 100644 > --- a/mysql-test/suite/sys_vars/r/sysvars_server_embedded.result > +++ b/mysql-test/suite/sys_vars/r/sysvars_server_embedded.result > @@ -765,6 +765,20 @@ NUMERIC_BLOCK_SIZE NULL > ENUM_VALUE_LIST NULL > READ_ONLY NO > COMMAND_LINE_ARGUMENT NULL > +VARIABLE_NAME EQ_RANGE_INDEX_DIVE_LIMIT > +SESSION_VALUE 0 > +GLOBAL_VALUE 0 > +GLOBAL_VALUE_ORIGIN COMPILE-TIME > +DEFAULT_VALUE 0 > +VARIABLE_SCOPE SESSION > +VARIABLE_TYPE INT UNSIGNED > +VARIABLE_COMMENT The optimizer will use existing index statistics instead of doing index dives for equality ranges if the number of equality ranges for the index is larger than or equal to this number. If set to 0, index dives are always used. > +NUMERIC_MIN_VALUE 0 > +NUMERIC_MAX_VALUE 4294967295 > +NUMERIC_BLOCK_SIZE 1 > +ENUM_VALUE_LIST NULL > +READ_ONLY NO > +COMMAND_LINE_ARGUMENT REQUIRED > VARIABLE_NAME ERROR_COUNT > SESSION_VALUE 0 > GLOBAL_VALUE NULL > diff --git a/mysql-test/suite/sys_vars/r/sysvars_server_notembedded.result b/mysql-test/suite/sys_vars/r/sysvars_server_notembedded.result > index 034195e..cada139 100644 > --- a/mysql-test/suite/sys_vars/r/sysvars_server_notembedded.result > +++ b/mysql-test/suite/sys_vars/r/sysvars_server_notembedded.result > @@ -779,6 +779,20 @@ NUMERIC_BLOCK_SIZE NULL > ENUM_VALUE_LIST NULL > READ_ONLY NO > COMMAND_LINE_ARGUMENT NULL > +VARIABLE_NAME EQ_RANGE_INDEX_DIVE_LIMIT > +SESSION_VALUE 0 > +GLOBAL_VALUE 0 > +GLOBAL_VALUE_ORIGIN COMPILE-TIME > +DEFAULT_VALUE 0 > +VARIABLE_SCOPE SESSION > +VARIABLE_TYPE INT UNSIGNED > +VARIABLE_COMMENT The optimizer will use existing index statistics instead of doing index dives for equality ranges if the number of equality ranges for the index is larger than or equal to this number. If set to 0, index dives are always used. > +NUMERIC_MIN_VALUE 0 > +NUMERIC_MAX_VALUE 4294967295 > +NUMERIC_BLOCK_SIZE 1 > +ENUM_VALUE_LIST NULL > +READ_ONLY NO > +COMMAND_LINE_ARGUMENT REQUIRED > VARIABLE_NAME ERROR_COUNT > SESSION_VALUE 0 > GLOBAL_VALUE NULL > diff --git a/mysql-test/t/range.test b/mysql-test/t/range.test > index ab95180..ed68cfb 100644 > --- a/mysql-test/t/range.test > +++ b/mysql-test/t/range.test > @@ -2046,6 +2046,39 @@ set optimizer_switch=@save_optimizer_switch; > drop table t1,t2,t3; > > --echo # > +--echo # MDEV-16934: using system variable eq_range_index_dive_limit > +--echo # to reduce the number of index dives > +--echo # > + > +create table t1 (a int, b varchar(31), index idx(a)); > + > +insert into t1 values > + (7,'xxxx'), (1,'yy'), (3,'aaa'), (1,'bbb'), (2,'zz'), > + (4,'vvvvv'), (7,'ddd'), (9,'zzzzz'), (1,'cc'), (5,'ffff'); > +insert into t1 select a+10, concat(b,'zz') from t1; > +insert into t1 select a+15, concat(b,'yy') from t1; > +insert into t1 select a+100, concat(b,'xx') from t1; > + > +analyze table t1; > + > +select cast(count(a)/count(distinct a) as unsigned) as rec_per_key from t1; > + > +let $q= > +select * from t1 where a in (8, 15, 31, 1, 9); > + > +set eq_range_index_dive_limit=0; > +eval explain $q; > +eval $q; > + > +set eq_range_index_dive_limit=2; > +eval explain $q; > +eval $q; > + > +set eq_range_index_dive_limit=default; > + > +drop table t1; > + > +--echo # > --echo # End of 10.2 tests > --echo # > > diff --git a/sql/multi_range_read.cc b/sql/multi_range_read.cc > index 50918d8..94a96c2 100644 > --- a/sql/multi_range_read.cc > +++ b/sql/multi_range_read.cc > @@ -17,6 +17,7 @@ > #include <my_bit.h> > #include "sql_select.h" > #include "key.h" > +#include "sql_statistics.h" > > /**************************************************************************** > * Default MRR implementation (MRR to non-MRR converter) > @@ -67,6 +68,9 @@ handler::multi_range_read_info_const(uint keyno, RANGE_SEQ_IF *seq, > /* Default MRR implementation doesn't need buffer */ > *bufsz= 0; > > + bool use_statistics_for_eq_range= eq_ranges_exceeds_limit(seq, > + seq_init_param); > + > seq_it= seq->init(seq_init_param, n_ranges, *flags); > while (!seq->next(seq_it, &range)) > { > @@ -87,8 +91,15 @@ handler::multi_range_read_info_const(uint keyno, RANGE_SEQ_IF *seq, > min_endp= range.start_key.length? &range.start_key : NULL; > max_endp= range.end_key.length? &range.end_key : NULL; > } > + int keyparts_used= my_count_bits(range.start_key.keypart_map); > if ((range.range_flag & UNIQUE_RANGE) && !(range.range_flag & NULL_RANGE)) > rows= 1; /* there can be at most one row */ > + else if (use_statistics_for_eq_range && > + !(range.range_flag & NULL_RANGE) && > + (range.range_flag & EQ_RANGE) && > + table->key_info[keyno].actual_rec_per_key(keyparts_used - 1) > 0.5) > + rows= > + (ha_rows) table->key_info[keyno].actual_rec_per_key(keyparts_used - 1); > else > { > if (HA_POS_ERROR == (rows= this->records_in_range(keyno, min_endp, > diff --git a/sql/opt_range.cc b/sql/opt_range.cc > index b011c01..c9c37a2 100644 > --- a/sql/opt_range.cc > +++ b/sql/opt_range.cc > @@ -14616,6 +14616,33 @@ void QUICK_GROUP_MIN_MAX_SELECT::add_keys_and_lengths(String *key_names, > } > > > +/* Check whether the number for equality ranges exceeds the set threshold */ > + > +bool eq_ranges_exceeds_limit(RANGE_SEQ_IF *seq, void *seq_init_param) > +{ > + KEY_MULTI_RANGE range; > + range_seq_t seq_it; > + uint count = 0; > + PARAM *param= ((SEL_ARG_RANGE_SEQ*) seq_init_param)->param; > + uint limit= param->thd->variables.eq_range_index_dive_limit; > + > + if (limit == 0) > + { > + /* 'Statistics instead of index dives' feature is turned off */ > + return false; > + } > + seq_it= seq->init(seq_init_param, 0, 0); > + while (!seq->next(seq_it, &range)) > + { > + if ((range.range_flag & EQ_RANGE) && !(range.range_flag & NULL_RANGE)) > + { > + if (++count >= limit) > + return true; > + } > + } > + return false; > +} > + > #ifndef DBUG_OFF > > static void print_sel_tree(PARAM *param, SEL_TREE *tree, key_map *tree_map, > diff --git a/sql/opt_range.h b/sql/opt_range.h > index c1f7079..6698c98 100644 > --- a/sql/opt_range.h > +++ b/sql/opt_range.h > @@ -242,7 +242,7 @@ class SEL_ARG :public Sql_alloc > Number of children of this element in the RB-tree, plus 1 for this > element itself. > */ > - uint16 elements; > + uint32 elements; > /* > Valid only for elements which are RB-tree roots: Number of times this > RB-tree is referred to (it is referred by SEL_ARG::next_key_part or by > @@ -1664,6 +1664,8 @@ SQL_SELECT *make_select(TABLE *head, table_map const_tables, > > bool calculate_cond_selectivity_for_table(THD *thd, TABLE *table, Item **cond); > > +bool eq_ranges_exceeds_limit(RANGE_SEQ_IF *seq, void *seq_init_param); > + > #ifdef WITH_PARTITION_STORAGE_ENGINE > bool prune_partitions(THD *thd, TABLE *table, Item *pprune_cond); > #endif > diff --git a/sql/opt_range_mrr.cc b/sql/opt_range_mrr.cc > index ace6208..7f80a3e 100644 > --- a/sql/opt_range_mrr.cc > +++ b/sql/opt_range_mrr.cc > @@ -272,25 +272,44 @@ bool sel_arg_range_seq_next(range_seq_t rseq, KEY_MULTI_RANGE *range) > key_info= NULL; > else > key_info= &seq->param->table->key_info[seq->real_keyno]; > - > + > /* > - Conditions below: > - (1) - range analysis is used for estimating condition selectivity > - (2) - This is a unique key, and we have conditions for all its > - user-defined key parts. > - (3) - The table uses extended keys, this key covers all components, > - and we have conditions for all key parts. > + This is an equality range (keypart_0=X and ... and keypart_n=Z) if > + (1) - There are no flags indicating open range (e.g., > + "keypart_x > y") or GIS. > + (2) - The lower bound and the upper bound of the range has the > + same value (min_key == max_key). > */ > - if (!(cur->min_key_flag & ~NULL_RANGE) && !cur->max_key_flag && > - (!key_info || // (1) > - ((uint)key_tree->part+1 == key_info->user_defined_key_parts && // (2) > - key_info->flags & HA_NOSAME) || // (2) > - ((key_info->flags & HA_EXT_NOSAME) && // (3) > - (uint)key_tree->part+1 == key_info->ext_key_parts) // (3) > - ) && > - range->start_key.length == range->end_key.length && > - !memcmp(seq->param->min_key,seq->param->max_key,range->start_key.length)) > - range->range_flag= UNIQUE_RANGE | (cur->min_key_flag & NULL_RANGE); > + const uint is_open_range = > + (NO_MIN_RANGE | NO_MAX_RANGE | NEAR_MIN | NEAR_MAX | GEOM_FLAG); > + const bool is_eq_range_pred = > + !(cur->min_key_flag & is_open_range) && // (1) > + !(cur->max_key_flag & is_open_range) && // (1) > + range->start_key.length == range->end_key.length && // (2) > + !memcmp(seq->param->min_key, seq->param->max_key, // (2) > + range->start_key.length); > + > + if (is_eq_range_pred) > + { > + range->range_flag = EQ_RANGE; > + > + /* > + Conditions below: > + (1) - Range analysis is used for estimating condition selectivity > + (2) - This is a unique key, and we have conditions for all its > + user-defined key parts. > + (3) - The table uses extended keys, this key covers all components, > + and we have conditions for all key parts. > + */ > + if ( > + !key_info || // (1) > + ((uint)key_tree->part+1 == key_info->user_defined_key_parts && // (2) > + key_info->flags & HA_NOSAME) || // (2) > + ((key_info->flags & HA_EXT_NOSAME) && // (3) > + (uint)key_tree->part+1 == key_info->ext_key_parts) // (3) > + ) > + range->range_flag |= UNIQUE_RANGE | (cur->min_key_flag & NULL_RANGE); > + } > > if (seq->param->is_ror_scan) > { > diff --git a/sql/sql_class.h b/sql/sql_class.h > index 6663d9e..d2f00b2d 100644 > --- a/sql/sql_class.h > +++ b/sql/sql_class.h > @@ -569,6 +569,7 @@ typedef struct system_variables > ha_rows max_join_size; > ha_rows expensive_subquery_limit; > ulong auto_increment_increment, auto_increment_offset; > + uint eq_range_index_dive_limit; > ulong lock_wait_timeout; > ulong join_cache_level; > ulong max_allowed_packet; > diff --git a/sql/sql_statistics.h b/sql/sql_statistics.h > index f465838..4ff85cd 100644 > --- a/sql/sql_statistics.h > +++ b/sql/sql_statistics.h > @@ -21,7 +21,7 @@ enum enum_use_stat_tables_mode > { > NEVER, > COMPLEMENTARY, > - PEFERABLY, > + PREFERABLY, > } Use_stat_tables_mode; > > typedef > diff --git a/sql/sys_vars.cc b/sql/sys_vars.cc > index e6aec47..650b17a 100644 > --- a/sql/sys_vars.cc > +++ b/sql/sys_vars.cc > @@ -2578,6 +2578,16 @@ static Sys_var_ulong Sys_div_precincrement( > SESSION_VAR(div_precincrement), CMD_LINE(REQUIRED_ARG), > VALID_RANGE(0, DECIMAL_MAX_SCALE), DEFAULT(4), BLOCK_SIZE(1)); > > +static Sys_var_uint Sys_eq_range_index_dive_limit( > + "eq_range_index_dive_limit", > + "The optimizer will use existing index statistics instead of " > + "doing index dives for equality ranges if the number of equality " > + "ranges for the index is larger than or equal to this number. " > + "If set to 0, index dives are always used.", > + SESSION_VAR(eq_range_index_dive_limit), CMD_LINE(REQUIRED_ARG), > + VALID_RANGE(0, UINT_MAX32), DEFAULT(0), > + BLOCK_SIZE(1)); > + > static Sys_var_ulong Sys_range_alloc_block_size( > "range_alloc_block_size", > "Allocation block size for storing ranges during optimization", > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits -- BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] adaafcf5184: MDEV-16930: Crash when VALUES in derived table contains expressions
by Sergei Golubchik 16 Aug '18

16 Aug '18

Hi, Oleksandr! On Aug 16, Oleksandr Byelkin wrote: > revision-id: adaafcf5184375369c0c4b3ccb1e82e846024ef8 (mariadb-10.3.7-137-gadaafcf5184) > parent(s): 8716bb3b72b7f5fed69b6dde413c2138f6d175b2 > author: Oleksandr Byelkin > committer: Oleksandr Byelkin > timestamp: 2018-08-16 13:55:36 +0200 > message: > > MDEV-16930: Crash when VALUES in derived table contains expressions > > Give names to the value constructor columns as in SELECT-list. > > diff --git a/sql/sql_yacc.yy b/sql/sql_yacc.yy > index f915895a260..20cea4959eb 100644 > --- a/sql/sql_yacc.yy > +++ b/sql/sql_yacc.yy > @@ -13306,15 +13306,20 @@ opt_values: > ; > > values: > - values ',' expr_or_default > + values ',' remember_name expr_or_default remember_end > { > - if (unlikely(Lex->insert_list->push_back($3, thd->mem_root))) > + if (unlikely(Lex->insert_list->push_back($4, thd->mem_root))) > MYSQL_YYABORT; > + // give some name in case of using in table value constuctor (TVC) > + $4->set_name(thd, $3, (uint) ($5 - $3), thd->charset()); > + > } > - | expr_or_default > + | remember_name expr_or_default remember_end > { > - if (unlikely(Lex->insert_list->push_back($1, thd->mem_root))) > + if (unlikely(Lex->insert_list->push_back($2, thd->mem_root))) > MYSQL_YYABORT; > + // give some name in case of using in table value constuctor (TVC) > + $2->set_name(thd, $1, (uint) ($3 - $1), thd->charset()); > } > ; Could you please create a special rule for that? Like named_value: remember_name expr_or_default remember_end { if (unlikely(Lex->insert_list->push_back($2, thd->mem_root))) MYSQL_YYABORT; // give some name in case of using in table value constuctor (TVC) $2->set_name(thd, $1, (uint) ($3 - $1), thd->charset()); } values: values ',' named_value | named_value ; if it'll work, that is Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] [Commits] 7fc57734575: MDEV-6439: Server crashes in Explain_union::print_explain with explain in slow log, tis620 charset
by Oleksandr Byelkin 15 Aug '18

15 Aug '18

Hi! See small comment inline. (fix and OK to push) Am 15.08.2018 um 13:36 schrieb Sergey Petrunia: > MDEV-6439: Server crashes in Explain_union::print_explain with explain in slow log, tis620 charset > > Item_subselect::is_expensive() used to return FALSE (Inexpensive) whenever > it saw that one of SELECTs in the Subquery's UNION is degenerate. It > ignored the fact that other parts of the UNION might not be inexpensive, > including the case where pther parts of the UNION have no query plan yet. > > For a subquery in form col >= ANY (SELECT 'foo' UNION SELECT 'bar') > this would cause the query to be considered inexpensive when there is > no query plan for the second part of the UNION, which in turn would > cause the SELECT 'foo' to compute and free itself while still inside > JOIN::optimize for that SELECT (See MDEV comment for full description). > > diff --git a/mysql-test/r/subselect_extra_no_semijoin.result b/mysql-test/r/subselect_extra_no_semijoin.result > index 79bca38..200682b 100644 > --- a/mysql-test/r/subselect_extra_no_semijoin.result > +++ b/mysql-test/r/subselect_extra_no_semijoin.result > @@ -482,3 +482,22 @@ DROP TABLE t1,t2; > set optimizer_switch= @tmp_subselect_extra_derived; > set optimizer_switch= @subselect_extra_no_sj_tmp; > set @optimizer_switch_for_subselect_extra_test=null; > +# > +# MDEV-6439: Server crashes in Explain_union::print_explain with explain in slow log, tis620 charset > +# > +SET NAMES tis620; > +set @tmp= @@global.slow_query_log; > +SET GLOBAL slow_query_log = 1; > +SET long_query_time = 0.000001; > +SET log_slow_verbosity = 'explain'; > +CREATE TABLE t1 (a VARCHAR(3)) ENGINE=MyISAM; > +SELECT * FROM t1 WHERE a >= ANY ( SELECT 'foo'); > +a > +SELECT * FROM t1 WHERE a >= ANY ( SELECT 'foo' UNION SELECT 'bar' ); > +ERROR HY000: Illegal mix of collations (tis620_thai_ci,COERCIBLE) and (latin1_swedish_ci,IMPLICIT) for operation '<=' > +create table t2 (b int); > +insert into t2 values (1),(2),(3); > +SELECT * FROM t1 WHERE a >= ANY ( SELECT 'foo' FROM t2); > +ERROR HY000: Illegal mix of collations (tis620_thai_ci,COERCIBLE) and (latin1_swedish_ci,IMPLICIT) for operation '<=' > +drop table t1,t2; > +SET GLOBAL slow_query_log=@tmp; > diff --git a/mysql-test/t/subselect_extra_no_semijoin.test b/mysql-test/t/subselect_extra_no_semijoin.test > index 8aba3dd..41593fb 100644 > --- a/mysql-test/t/subselect_extra_no_semijoin.test > +++ b/mysql-test/t/subselect_extra_no_semijoin.test > @@ -6,4 +6,34 @@ set @optimizer_switch_for_subselect_extra_test='semijoin=off,firstmatch=off,loo > > set optimizer_switch= @subselect_extra_no_sj_tmp; > > -set @optimizer_switch_for_subselect_extra_test=null; > \ No newline at end of file > +set @optimizer_switch_for_subselect_extra_test=null; > + > +--echo # > +--echo # MDEV-6439: Server crashes in Explain_union::print_explain with explain in slow log, tis620 charset > +--echo # > + > +## Using a separate client connection is easier than restoring state > +connect(con1,localhost,root,,); > + > +SET NAMES tis620; > +set @tmp= @@global.slow_query_log; > +SET GLOBAL slow_query_log = 1; > +SET long_query_time = 0.000001; > +SET log_slow_verbosity = 'explain'; > + please remove above space at the end of line and push. > +CREATE TABLE t1 (a VARCHAR(3)) ENGINE=MyISAM; > +SELECT * FROM t1 WHERE a >= ANY ( SELECT 'foo'); > +--error ER_CANT_AGGREGATE_2COLLATIONS > +SELECT * FROM t1 WHERE a >= ANY ( SELECT 'foo' UNION SELECT 'bar' ); > + > +create table t2 (b int); > +insert into t2 values (1),(2),(3); > + > +--error ER_CANT_AGGREGATE_2COLLATIONS > +SELECT * FROM t1 WHERE a >= ANY ( SELECT 'foo' FROM t2); > + > +drop table t1,t2; > +SET GLOBAL slow_query_log=@tmp; > +disconnect con1; > +connection default; > + > diff --git a/sql/item_subselect.cc b/sql/item_subselect.cc > index a8dfdff..14b2ccd 100644 > --- a/sql/item_subselect.cc > +++ b/sql/item_subselect.cc > @@ -575,7 +575,7 @@ bool Item_subselect::is_expensive() > */ > if (cur_join->optimized && > (cur_join->zero_result_cause || !cur_join->tables_list)) > - return false; > + continue; > > /* > If a subquery is not optimized we cannot estimate its cost. A subquery is

1 0

[Maria-developers] MDEV-16768 and MDEV-16703 patch review
by Ming Lin 10 Aug '18

10 Aug '18

Hi Sergey, Could you help to review below 2 patches please? MDEV-16768: https://github.com/MariaDB/server/pull/837 MDEV-16703: https://github.com/MariaDB/server/pull/839 Thanks, Ming

1 0

Re: [Maria-developers] 6df4e4a855f: MDEV-16217: Assertion `!table || (!table->read_set || bitmap_is_set(table->read_set, field_index))' failed in Field_num::get_date
by Sergei Golubchik 09 Aug '18

09 Aug '18

Hi, Oleksandr! See questions below. There are few changes that I'm not quite sure about. On Aug 08, Oleksandr Byelkin wrote: > revision-id: 6df4e4a855fe223988c12681f8caec6c49b2f629 (mariadb-10.2.16-66-g6df4e4a855f) > parent(s): 4ddcb4eb46c62cf459936554d43351db740ba14d > author: Oleksandr Byelkin > committer: Oleksandr Byelkin > timestamp: 2018-08-08 12:28:15 +0200 > message: > > MDEV-16217: Assertion `!table || (!table->read_set || bitmap_is_set(table->read_set, field_index))' failed in Field_num::get_date > > - clean up DEFAULT() to work only with default value and correctly print > itself. > - fix of DBUG_ASSERT about fields read/write > - fix of firld marking for write based really on the thd->mark_used_columns flag > > --- > mysql-test/r/func_default.result | 2 +- > mysql-test/r/func_time.result | 28 ++++++++++++++++++++++++++++ > mysql-test/t/func_time.test | 21 +++++++++++++++++++++ > sql/field.cc | 21 +++++++++++++++++++-- > sql/field.h | 1 + > sql/item.cc | 25 ++++++++++++++++++++++++- > sql/item.h | 5 +++++ > sql/sql_base.cc | 2 +- > 8 files changed, 100 insertions(+), 5 deletions(-) > > diff --git a/mysql-test/r/func_default.result b/mysql-test/r/func_default.result > index 535be10da86..1f331af287c 100644 > --- a/mysql-test/r/func_default.result > +++ b/mysql-test/r/func_default.result > @@ -8,7 +8,7 @@ explain extended select default(str), default(strnull), default(intg), default(r > id select_type table type possible_keys key key_len ref rows filtered Extra > 1 SIMPLE t1 system NULL NULL NULL NULL 1 100.00 > Warnings: > -Note 1003 select default('') AS `default(str)`,default('') AS `default(strnull)`,default(0) AS `default(intg)`,default(0) AS `default(rel)` from dual > +Note 1003 select default(`str`) AS `default(str)`,default(`strnull`) AS `default(strnull)`,default(`intg`) AS `default(intg)`,default(`rel`) AS `default(rel)` from dual This looks rather fishy. There can be no columns str and strnull in dual. Please add a test case with a view, to make sure you only get this effect (column names and dual) in explain extended (where it doesn't matter), but not in a view (where it does). > select * from t1 where str <> default(str); > str strnull intg rel > 0 0 > diff --git a/mysql-test/r/func_time.result b/mysql-test/r/func_time.result > index 55c2c93eb56..51f51820efc 100644 > --- a/mysql-test/r/func_time.result > +++ b/mysql-test/r/func_time.result > @@ -3260,3 +3260,31 @@ DROP TABLE t1,t2; > # > # End of 10.1 tests > # > +# > +# MDEV-16217: Assertion `!table || (!table->read_set || > +# bitmap_is_set(table->read_set, field_index))' > +# failed in Field_num::get_date > +# > +CREATE TABLE t1 (pk int default 0, a1 date); > +INSERT INTO t1 VALUES (1,'1900-01-01'),(2,NULL),(3,NULL),(4,NULL); > +CREATE VIEW v1 AS > +SELECT t1.pk AS pk, t1.a1 AS a1 FROM t1; > +SELECT a1 BETWEEN (('2018-08-24')) AND (DEFAULT(pk)) FROM v1; > +a1 BETWEEN (('2018-08-24')) AND (DEFAULT(pk)) > +0 > +NULL > +NULL > +NULL > +SELECT a1 BETWEEN (('2018-08-24')) AND (~ DEFAULT(pk)) FROM v1; > +a1 BETWEEN (('2018-08-24')) AND (~ DEFAULT(pk)) > +0 > +NULL > +NULL > +NULL > +Warnings: > +Warning 1292 Incorrect datetime value: '18446744073709551615' > +DROP VIEW v1; > +DROP TABLE t1; > +# > +# End of 10.2 tests > +# > diff --git a/mysql-test/t/func_time.test b/mysql-test/t/func_time.test > index 3503b6d5bc6..c8859feee93 100644 > --- a/mysql-test/t/func_time.test > +++ b/mysql-test/t/func_time.test > @@ -1857,3 +1857,24 @@ DROP TABLE t1,t2; > --echo # > --echo # End of 10.1 tests > --echo # > + > +--echo # > +--echo # MDEV-16217: Assertion `!table || (!table->read_set || > +--echo # bitmap_is_set(table->read_set, field_index))' > +--echo # failed in Field_num::get_date > +--echo # > +CREATE TABLE t1 (pk int default 0, a1 date); > +INSERT INTO t1 VALUES (1,'1900-01-01'),(2,NULL),(3,NULL),(4,NULL); > + > +CREATE VIEW v1 AS > +SELECT t1.pk AS pk, t1.a1 AS a1 FROM t1; > + > +SELECT a1 BETWEEN (('2018-08-24')) AND (DEFAULT(pk)) FROM v1; > +SELECT a1 BETWEEN (('2018-08-24')) AND (~ DEFAULT(pk)) FROM v1; > + > +DROP VIEW v1; > +DROP TABLE t1; > + > +--echo # > +--echo # End of 10.2 tests > +--echo # > diff --git a/sql/field.cc b/sql/field.cc > index 9ca9663f066..e12adbf47b3 100644 > --- a/sql/field.cc > +++ b/sql/field.cc > @@ -70,8 +70,25 @@ const char field_separator=','; > #define BLOB_PACK_LENGTH_TO_MAX_LENGH(arg) \ > ((ulong) ((1LL << MY_MIN(arg, 4) * 8) - 1)) > > -#define ASSERT_COLUMN_MARKED_FOR_READ DBUG_ASSERT(!table || (!table->read_set || bitmap_is_set(table->read_set, field_index))) > -#define ASSERT_COLUMN_MARKED_FOR_WRITE_OR_COMPUTED DBUG_ASSERT(is_stat_field || !table || (!table->write_set || bitmap_is_set(table->write_set, field_index) || (table->vcol_set && bitmap_is_set(table->vcol_set, field_index)))) > +// Column marked for read or the field set to read out or record[0] or [1] > +#define ASSERT_COLUMN_MARKED_FOR_READ \ > + DBUG_ASSERT(!table || \ > + (!table->read_set || \ > + bitmap_is_set(table->read_set, field_index) || \ > + (!(ptr >= table->record[0] && \ > + ptr < table->record[0] + table->s->reclength) && \ > + !(ptr >= table->record[1] && \ > + ptr < table->record[1] + table->s->reclength)))) why did you add record[1] to the assert? I think, having just record[0] would've been enough > + > +#define ASSERT_COLUMN_MARKED_FOR_WRITE_OR_COMPUTED \ > + DBUG_ASSERT(is_stat_field || !table || \ > + (!table->write_set || \ > + bitmap_is_set(table->write_set, field_index) || \ > + (!(ptr >= table->record[0] && \ > + ptr < table->record[0] + table->s->reclength) && \ > + !(ptr >= table->record[1] && \ > + ptr < table->record[1] + table->s->reclength))) || \ > + (table->vcol_set && bitmap_is_set(table->vcol_set, field_index))) > > #define FLAGSTR(S,F) ((S) & (F) ? #F " " : "") > > diff --git a/sql/field.h b/sql/field.h > index 22c276478b6..55c3ed4c4bd 100644 > --- a/sql/field.h > +++ b/sql/field.h > @@ -630,6 +630,7 @@ class Virtual_column_info: public Sql_alloc > bool utf8; /* Already in utf8 */ > Item *expr; > LEX_STRING name; /* Name of constraint */ > + /* see VCOL_* (VCOL_FIELD_REF, ...) */ > uint flags; > > Virtual_column_info() > diff --git a/sql/item.cc b/sql/item.cc > index 58d2b7dbfc0..91fe7ff03aa 100644 > --- a/sql/item.cc > +++ b/sql/item.cc > @@ -8917,8 +8917,19 @@ bool Item_default_value::fix_fields(THD *thd, Item **items) > fixed= 1; > return FALSE; > } > + > + /* > + DEFAULT() do not need table field so should not ask handler to bring > + field value (mark column for read) > + */ > + enum_mark_columns save_mark_used_columns= thd->mark_used_columns; > + thd->mark_used_columns= MARK_COLUMNS_NONE; > if (!arg->fixed && arg->fix_fields(thd, &arg)) > + { > + thd->mark_used_columns= save_mark_used_columns; > goto error; > + } > + thd->mark_used_columns= save_mark_used_columns; Hmm, I thought that the field wasn't marked in read_set, and that failed the assert. Now you're saying that it was? > real_arg= arg->real_item(); > @@ -8938,12 +8949,16 @@ bool Item_default_value::fix_fields(THD *thd, Item **items) > goto error; > memcpy((void *)def_field, (void *)field_arg->field, > field_arg->field->size_of()); > - IF_DBUG(def_field->is_stat_field=1,); // a hack to fool ASSERT_COLUMN_MARKED_FOR_WRITE_OR_COMPUTED ah, good. thanks! > + // If non-constant default value expression > if (def_field->default_value && def_field->default_value->flags) > { > uchar *newptr= (uchar*) thd->alloc(1+def_field->pack_length()); > if (!newptr) > goto error; > + /* > + Even if DEFAULT() do not read tables fields, the default value > + expression can do it. > + */ > fix_session_vcol_expr_for_read(thd, def_field, def_field->default_value); > if (thd->mark_used_columns != MARK_COLUMNS_NONE) > def_field->default_value->expr->walk(&Item::register_field_in_read_map, 1, 0); better call update_used_tables() from here. > @@ -8970,6 +8985,14 @@ void Item_default_value::print(String *str, enum_query_type query_type) > return; > } > str->append(STRING_WITH_LEN("default(")); > + /* > + We take DEFAULT from a field so do not need it value in case of const > + tables but its name so we set QT_NO_DATA_EXPANSION (as we print for > + table definition, also we do not need table and database name) > + */ > + query_type= (enum_query_type) (query_type | QT_NO_DATA_EXPANSION | > + QT_ITEM_IDENT_SKIP_DB_NAMES | > + QT_ITEM_IDENT_SKIP_TABLE_NAMES); in a view definition you might need table and database names. For example, a view that joins two tables from different databases, and they both have a column named `a`. And the query uses DEFAULT(db1.t1.a). > arg->print(str, query_type); > str->append(')'); > } > diff --git a/sql/item.h b/sql/item.h > index 8fad8dadf22..d252a256f76 100644 > --- a/sql/item.h > +++ b/sql/item.h > @@ -5316,6 +5316,11 @@ class Item_default_value : public Item_field > return false; > } > table_map used_tables() const; > + virtual void update_used_tables() > + { > + if (field && field->default_value) > + field->default_value->expr->walk(&Item::register_field_in_read_map, 1, 0); > + } > Field *get_tmp_table_field() { return 0; } > Item *get_tmp_table_item(THD *thd) { return this; } > Item_field *field_for_view_update() { return 0; } Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

2 2

[Maria-developers] Spelling errors in MariaDB sources
by Otto Kekäläinen 31 Jul '18

31 Jul '18

Hello! I recall I've made several big patches on MariaDB that fixes spelling errors. Somehow the same spelling errors seem to creep back in every now and then. I notice them while uploading stuff to Debian and using the Debian QA tool "Lintian". I haven't done git blame and looked deeply in it. I suspect it is related to how we merge old branches to new branches and occasionally loose stuff. Would somebody like to look into this? It could be an relatively easy task for some new contributor? $ grep spelling-error lintian-22e620c534.log I: mariadb-server-10.1: spelling-error-in-binary usr/bin/aria_chk noone no one I: mariadb-server-10.1: spelling-error-in-binary usr/bin/aria_chk thats that's I: mariadb-server-10.1: spelling-error-in-binary usr/bin/aria_chk theres there's I: mariadb-server-10.1: spelling-error-in-binary usr/bin/aria_dump_log noone no one I: mariadb-server-10.1: spelling-error-in-binary usr/bin/aria_dump_log thats that's I: mariadb-server-10.1: spelling-error-in-binary usr/bin/aria_dump_log theres there's I: mariadb-server-10.1: spelling-error-in-binary usr/bin/aria_ftdump noone no one I: mariadb-server-10.1: spelling-error-in-binary usr/bin/aria_ftdump thats that's I: mariadb-server-10.1: spelling-error-in-binary usr/bin/aria_ftdump theres there's I: mariadb-server-10.1: spelling-error-in-binary usr/bin/aria_pack noone no one I: mariadb-server-10.1: spelling-error-in-binary usr/bin/aria_pack thats that's I: mariadb-server-10.1: spelling-error-in-binary usr/bin/aria_pack theres there's I: mariadb-server-10.1: spelling-error-in-binary usr/bin/aria_read_log noone no one I: mariadb-server-10.1: spelling-error-in-binary usr/bin/aria_read_log thats that's I: mariadb-server-10.1: spelling-error-in-binary usr/bin/aria_read_log theres there's I: mariadb-server-10.1: spelling-error-in-binary usr/bin/myisam_ftdump noone no one I: mariadb-server-10.1: spelling-error-in-binary usr/bin/myisam_ftdump thats that's I: mariadb-server-10.1: spelling-error-in-binary usr/bin/myisam_ftdump theres there's I: mariadb-server-10.1: spelling-error-in-binary usr/bin/myisamchk noone no one I: mariadb-server-10.1: spelling-error-in-binary usr/bin/myisamchk thats that's I: mariadb-server-10.1: spelling-error-in-binary usr/bin/myisamchk theres there's I: mariadb-server-10.1: spelling-error-in-binary usr/bin/myisamlog noone no one I: mariadb-server-10.1: spelling-error-in-binary usr/bin/myisamlog thats that's I: mariadb-server-10.1: spelling-error-in-binary usr/bin/myisamlog theres there's I: mariadb-server-10.1: spelling-error-in-binary usr/bin/myisampack noone no one I: mariadb-server-10.1: spelling-error-in-binary usr/bin/myisampack thats that's I: mariadb-server-10.1: spelling-error-in-binary usr/bin/myisampack theres there's I: mariadb-server-10.1: spelling-error-in-binary usr/lib/x86_64-linux-gnu/mariadb18/plugin/ha_federated.so Allows to Allows one to I: mariadb-server-10.1: spelling-error-in-binary usr/lib/x86_64-linux-gnu/mariadb18/plugin/ha_federatedx.so Allows to Allows one to I: mariadb-server-10.1: spelling-error-in-binary usr/lib/x86_64-linux-gnu/mariadb18/plugin/ha_innodb.so allow to allow one to I: libmariadbd18: spelling-error-in-binary usr/lib/x86_64-linux-gnu/libmysqld.so.18 extentions extensions I: libmariadbd18: spelling-error-in-binary usr/lib/x86_64-linux-gnu/libmysqld.so.18 noone no one I: libmariadbd18: spelling-error-in-binary usr/lib/x86_64-linux-gnu/libmysqld.so.18 thats that's I: libmariadbd18: spelling-error-in-binary usr/lib/x86_64-linux-gnu/libmysqld.so.18 theres there's I: libmariadbd18: spelling-error-in-binary usr/lib/x86_64-linux-gnu/libmysqld.so.18 allow to allow one to I: mariadb-client-10.1: spelling-error-in-binary usr/bin/mariabackup yuR your I: mariadb-client-10.1: spelling-error-in-binary usr/bin/mariabackup occured occurred I: mariadb-client-10.1: spelling-error-in-binary usr/bin/mariabackup accessable accessible I: mariadb-client-10.1: spelling-error-in-binary usr/bin/mariabackup extentions extensions I: mariadb-client-10.1: spelling-error-in-binary usr/bin/mariabackup reseting resetting I: mariadb-client-10.1: spelling-error-in-binary usr/bin/mariabackup succeded succeeded I: mariadb-client-10.1: spelling-error-in-binary usr/bin/mariabackup settting setting I: mariadb-client-10.1: spelling-error-in-binary usr/bin/mariabackup postion position I: mariadb-client-10.1: spelling-error-in-binary usr/bin/mariabackup noone no one I: mariadb-client-10.1: spelling-error-in-binary usr/bin/mariabackup thats that's I: mariadb-client-10.1: spelling-error-in-binary usr/bin/mariabackup theres there's I: mariadb-client-10.1: spelling-error-in-binary usr/bin/mariabackup allow to allow one to I: mariadb-client-10.1: spelling-error-in-manpage usr/share/man/man1/innotop.1.gz Additonal Additional I: mariadb-plugin-tokudb: spelling-error-in-binary usr/bin/tokuftdump dont don't I: mariadb-plugin-tokudb: spelling-error-in-binary usr/lib/x86_64-linux-gnu/mariadb18/plugin/ha_tokudb.so dont don't I: mariadb-client-core-10.1: spelling-error-in-binary usr/bin/mysql_embedded extentions extensions I: mariadb-client-core-10.1: spelling-error-in-binary usr/bin/mysql_embedded noone no one I: mariadb-client-core-10.1: spelling-error-in-binary usr/bin/mysql_embedded thats that's I: mariadb-client-core-10.1: spelling-error-in-binary usr/bin/mysql_embedded theres there's I: mariadb-client-core-10.1: spelling-error-in-binary usr/bin/mysql_embedded allow to allow one to I: mariadb-server-core-10.1: spelling-error-in-binary usr/sbin/mysqld extentions extensions I: mariadb-server-core-10.1: spelling-error-in-binary usr/sbin/mysqld reseting resetting I: mariadb-server-core-10.1: spelling-error-in-binary usr/sbin/mysqld succeded succeeded I: mariadb-server-core-10.1: spelling-error-in-binary usr/sbin/mysqld settting setting I: mariadb-server-core-10.1: spelling-error-in-binary usr/sbin/mysqld postion position I: mariadb-server-core-10.1: spelling-error-in-binary usr/sbin/mysqld noone no one I: mariadb-server-core-10.1: spelling-error-in-binary usr/sbin/mysqld thats that's I: mariadb-server-core-10.1: spelling-error-in-binary usr/sbin/mysqld theres there's I: mariadb-server-core-10.1: spelling-error-in-binary usr/sbin/mysqld allow to allow one to I: mariadb-plugin-connect: spelling-error-in-binary usr/lib/x86_64-linux-gnu/mariadb18/plugin/ha_connect.so Mutiple Multiple I: mariadb-plugin-connect: spelling-error-in-binary usr/lib/x86_64-linux-gnu/mariadb18/plugin/ha_connect.so Openning Opening

1 0

[Maria-developers] Missing man pages
by Otto Kekäläinen 31 Jul '18

31 Jul '18

Hello Ian and others! While working on uploading latest MariaDB 10.1 into Debian I noticed it complains about quite a few missing man pages: $ grep manpage lintian-22e620c534.log W: mariadb-server-10.1: binary-without-manpage usr/bin/wsrep_sst_mariabackup W: mariadb-server-10.1: binary-without-manpage usr/bin/wsrep_sst_rsync_wan I: mariadb-client-10.1: spelling-error-in-manpage usr/share/man/man1/innotop.1.gz Additonal Additional W: mariadb-client-10.1: binary-without-manpage usr/bin/mariabackup W: mariadb-client-10.1: binary-without-manpage usr/bin/mbstream W: mariadb-client-10.1: binary-without-manpage usr/bin/mytop W: mariadb-plugin-tokudb: binary-without-manpage usr/bin/tokuft_logprint W: mariadb-client-core-10.1: binary-without-manpage usr/bin/mysql_embedded Could you Ian check if we have these on some branch already and make sure they and up an all the same branches and releases as the binaries themselves? And those who make new binaries, please at least write a stub for your new binary. It makes downstream work much faster when the man page in the source explains directly what the new binaries are and hints how they should be packaged. - Otto

1 0

[Maria-developers] Please review MDEV-16584 SP with a cursor inside a loop wastes THD memory aggressively
by Alexander Barkov 30 Jul '18

30 Jul '18

Hello Sanja, Please review a patch for MDEV-16584. Thanks!

2 1

Re: [Maria-developers] [Commits] 108af298032: MDEV-15454: Nested SELECT IN returns wrong results
by Sergey Petrunia 25 Jul '18

25 Jul '18

On Fri, May 11, 2018 at 03:08:29AM +0530, Varun wrote: > revision-id: 108af298032cb4cea316a08c304c306268a2badf (mariadb-5.5.56-213-g108af298032) > parent(s): 318097bb8f6e12c546b5dcd287416158209dbb39 > author: Varun Gupta > committer: Varun Gupta > timestamp: 2018-05-11 02:56:33 +0530 > message: > > MDEV-15454: Nested SELECT IN returns wrong results > > In this case we are setting the field Item_func_eq::in_eqaulity_no for the semi-join equalities. > This helps us to remove these equalites as the inner tables are not available during parent select execution > while the outer tables are not available during materialization phase. > We only have it set for the equalites for the fields involved with the IN subquery > and reset it for the equalities which do not belong to the IN subquery. > > For example in case of nested IN subqueries: > > SELECT t1.a FROM t1 WHERE t1.a IN > (SELECT t2.a FROM t2 where t2.b IN > (select t3.b from t3 where t3.c=27 )) > > there are two equalites involving the fields of the IN subquery > > 1) t2.b = t3.b : the field Item_func_eq::in_eqaulity_no is set when we merge the grandchild select into the child select > 2) t1.a = t2.a : the field Item_func_eq::in_eqaulity_no is set when we merge the child select into the parent select > > But when we perform case 2) we should ensure that we reset the equalities in the child's WHERE clause. > > --- > mysql-test/r/subselect_sj_mat.result | 46 +++++++++++++++++++++++++ > mysql-test/t/subselect_sj_mat.test | 49 ++++++++++++++++++++++++++ > sql/opt_subselect.cc | 66 ++++++++++++++++++++++++++++++++++++ > 3 files changed, 161 insertions(+) > ... > diff --git a/sql/opt_subselect.cc b/sql/opt_subselect.cc > index a7edd64e68b..31213a5cf1d 100644 > --- a/sql/opt_subselect.cc > +++ b/sql/opt_subselect.cc > @@ -1481,6 +1543,7 @@ static int subq_sj_candidate_cmp(Item_in_subselect* el1, Item_in_subselect* el2, > TRUE Out of memory error > */ > > + > static bool convert_subq_to_sj(JOIN *parent_join, Item_in_subselect *subq_pred) > { > SELECT_LEX *parent_lex= parent_join->select_lex; > @@ -1713,6 +1776,7 @@ static bool convert_subq_to_sj(JOIN *parent_join, Item_in_subselect *subq_pred) > > if (subq_pred->left_expr->cols() == 1) > { > + reset_equality_number_for_subq_conds(sj_nest->sj_on_expr); > /* add left = select_list_element */ > nested_join->sj_outer_expr_list.push_back(&subq_pred->left_expr); > /* > @@ -1735,6 +1799,7 @@ static bool convert_subq_to_sj(JOIN *parent_join, Item_in_subselect *subq_pred) > } > else if (subq_pred->left_expr->type() == Item::ROW_ITEM) > { > + reset_equality_number_for_subq_conds(sj_nest->sj_on_expr); > /* > disassemple left expression and add > left1 = select_list_element1 and left2 = select_list_element2 ... > @@ -1759,6 +1824,7 @@ static bool convert_subq_to_sj(JOIN *parent_join, Item_in_subselect *subq_pred) > } > else > { > + reset_equality_number_for_subq_conds(sj_nest->sj_on_expr); > /* > add row operation > left = (select_list_element1, select_list_element2, ...) So, the code here is if (...) { variant1; } else if (...) { variant2; } else { variant3; } and there is the same call added to all three: > + reset_equality_number_for_subq_conds(sj_nest->sj_on_expr); Can you move it out of the "if" and make it one call instead of three? Otherwise, the patch is ok, ok to push after the above is addressed. BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] [Commits] d277c4682e4: MDEV-16751: Server crashes in st_join_table::cleanup or TABLE_LIST::is_with_table_recursive_reference
by Sergey Petrunia 24 Jul '18

24 Jul '18

On Sun, Jul 22, 2018 at 03:56:37PM +0530, Varun wrote: > revision-id: d277c4682e4c6bb0e708296e6ae520c149dd78f4 (mariadb-5.5.60-47-gd277c4682e4) > parent(s): 9cea4ccf12cb6e8746b9b440d9c62408a9ef04af > author: Varun Gupta > committer: Varun Gupta > timestamp: 2018-07-22 15:56:25 +0530 > message: > > MDEV-16751: Server crashes in st_join_table::cleanup or TABLE_LIST::is_with_table_recursive_reference > with join_cache_level>2 > > During muliple equality propagation for a query in which we have an IN subquery, the items in the select list of the > subquery may not be part of the multiple equality because there might be another occurence of the same field in the > where clause of the subquery. > So we keyuse_is_valid_for_access_in_chosen_plan function which expects the items in the select list of the subquery to > be same to the ones in the multiple equality (through these multiple equalities we create keyuse array). > The solution would be that we expect the same field not the same Item because when we have SEMI JOIN MATERIALIZATION SCAN, > we use copy back technique to copies back the materialised table fields to the original fields of the base tables. > > --- > mysql-test/r/subselect_mat.result | 35 +++++++++++++++++++++++++++++++++++ > mysql-test/r/subselect_sj_mat.result | 35 +++++++++++++++++++++++++++++++++++ > mysql-test/t/subselect_sj_mat.test | 25 +++++++++++++++++++++++++ > sql/sql_select.cc | 4 +++- > 4 files changed, 98 insertions(+), 1 deletion(-) ... > diff --git a/sql/sql_select.cc b/sql/sql_select.cc > index 6a64a0e9952..af6f63a55f1 100644 > --- a/sql/sql_select.cc > +++ b/sql/sql_select.cc > @@ -7438,7 +7438,9 @@ bool JOIN_TAB::keyuse_is_valid_for_access_in_chosen_plan(JOIN *join, > st_select_lex *sjm_sel= emb_sj_nest->sj_subq_pred->unit->first_select(); > for (uint i= 0; i < sjm_sel->item_list.elements; i++) > { > - if (sjm_sel->ref_pointer_array[i] == keyuse->val) > + DBUG_ASSERT(keyuse->val->type() == Item::FIELD_ITEM); ^^(1) > + Field *field = ((Item_field*)sjm_sel->ref_pointer_array[i])->field; ^^(2) > + if (field->eq(((Item_field*)keyuse->val)->field)) > return true; > } > return false; As for typecast (2): As far as I understand, it is valid because of the following: - We get to this point in this function only when Semi-Join-Materialization-Scan is used. - SJM-Scan is only used when the subquery's select list has Item_field objects (This is checked in subquery_types_allow_materialization()). * Please add a comment in subquery_types_allow_materialization() about this function relying on that logic. * Please add here an assert that sjm_sel->ref_pointer_array[i] is an Item_field. The logic is complex and can be potentially broken. As for typecast (1): What if keyuse->val is not an Item_field? If we get to this point in this function, it means it's an expression that only depends on the SJ-Inner table. This cannot just happen due to limitations in subquery_types_allow_materialization(), and I wasn't able to construct an example where it would happen (tried playing with multiple-equalities, but substitute_for_best_equal_item is called after get_best_combination(), tried playing with constant tables but then SJ-Materialization is not used, etc. ) I think we will be safer by changing the assert to be this check: if (keyuse->val->type() == Item::FIELD_ITEM) { ... everything else } The idea is that a KEYUSE that satisfies the requirement will still be available, and we just want to filter out any extra KEYUSE that might pop up due to some optimization. what do you think? Ok to push after the above is addressed. BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] Updated Gtid_slave_pos of Untracked domain creates skipped events
by Sachin Setiya 23 Jul '18

23 Jul '18

Hi Andrei, Kristian! Problem:- This issue is regarding Mdev-9107 , where we have 3 master master with do_domain_ids of replication channel master domain-id . , When one replication channel is stopped and that master is updated , slave server updates the gtid slave_pos from second replication channel. And then when replication channel is brought up again then since we have updated the gtid_slave_pos we skip that event. I have created a abstract test case for this probem. (3) ^ ^ / \ (2)----->(1) 3 and 1 is configured with log slave updates. 3 has 2 replication channel m2_s3(do_domain_id=2 ), m1_s3(do_domain_id=1) 1 has one replication channel m2_s1(do_domain_id=2) I have tried with master_use_gtid == slave_pos and current_pos both give same result. When I stop replication channel m2_s3 and then write one-one event in (2) and (1) server. Server 3 will update its gtid_slave_pos and it will have latest gtid event seq no from domain id 2 and 1. When replication channel m2_s3 is bought up again since gtid_slave_pos of (3) is equal to (2) so we actually skipped the event generated on server 2. (I believe network lag between 2-->3 will have similar effect) (I have attached the test cases) Possible Solution. May be we should have one more option in master_use_gtid = binlog_state ? which will compare its binlog_state to arriving event gtid no ? (It is different from current_pos , current_pos will select the maximum seq_no between binlog_state and slave_pos) -- Regards Sachin Setiya Software Engineer at MariaDB

2 5

[Maria-developers] Please review a patch for MDEV-15758 Split Item_bool_func::get_mm_leaf() into virtual methods in Field and Type_handler
by Alexander Barkov 20 Jul '18

20 Jul '18

Hello Vicentiu, Please review my patch for MDEV-15758. It also fixes this bug: MDEV-15759 Expect "Impossible WHERE" for indexed_int_column=out_of_range_int_constant Thanks!

3 4

[Maria-developers] To MDEV-6608 Slave loses master binlog filename
by andrei.elkin＠pp.inet.fi 13 Jul '18

13 Jul '18

Hello, Kristian. I read through your analysis of the case to agree with the plan and its motivation: MDEV-6608> Generally, it is a very bad idea to destroy data during crash recovery ... try an approach that leaves the crashed binlog file intact, and instead ensures that the crashed binlog can be properly handled by the rest of the code. So in case of a partially written event group (transaction) the short (xid-less therefore) transaction is to be rolled back on the master. So why won't we properly enclose the group in the binlog at the end of recovery which means to append ROLLBACK Query-log-event, naturally before that the last event of the group needs sanity check? Was that that you had in mind actually? Cheers, Andrei

2 1

[Maria-developers] MDEV-13118 Wrong results with LOWER and UPPER and subquery
by Alexander Barkov 12 Jul '18

12 Jul '18

Hello Sergei, Please review a patch for MDEV-13118. Thanks!

2 4

[Maria-developers] Selecting Blob by parts (suppose select 10 to 100 char from blob )
by Sachin Setiya 12 Jul '18

12 Jul '18

Hello everyone! On optimizer call (11-7-2018) it was discussed , instead of selecting the whole blob , sometime we just need to select the subpart of it. And we should extend handler/engine api to facilitate this. Although this looks like a interesting idea , Serg (and I ) did not understand why we need this , what is use case of it. Some use case which I think can be like optimizing substring SELECT SUBSTRING(blob_col, 4,4) FROM t1; And we can push down this SUBSTRING condition directly to storage engine. But apart from this what use case this have ? -- Regards Sachin Setiya Software Engineer at MariaDB

1 0

[Maria-developers] What can MariaDB developers learn from Debian and Ubuntu Quality Assurance systems?
by Otto Kekäläinen 11 Jul '18

11 Jul '18

Hello! I had the presentation with the title in the subject line at the developers unconference some weeks ago. The slides are available here: https://mariadb.org/wp-content/uploads/2018/07/2018-06-27-mariadb-dev-learn… I recommend all developers skim though them. Debian has some really nice QA systems. All presentations from the event can be found at: https://mariadb.org/tampere-mariadb-developers-unconference-reportback-pres…

1 0

Re: [Maria-developers] 1a2748c: MDEV-15473 Isolate/sandbox PAM modules, so that they can't crash the server.
by Sergei Golubchik 10 Jul '18

10 Jul '18

Hi, Alexey! On Jul 10, Alexey Botchkov wrote: > > > Did you test that it works? > Of course i tested the mysql_install_db script a lot. > Didn't test the .rpm and .deb though. As far as i can see they plainly run > mysql_install_db --user=.. under root. could you please test with rpm and deb too? should only take 10-20 minutes in a buildbot VM. ok to push after that (if all goes well :)) Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] 1a2748c: MDEV-15473 Isolate/sandbox PAM modules, so that they can't crash the server.
by Sergei Golubchik 09 Jul '18

09 Jul '18

Hi, Alexey! On Jul 09, Alexey Botchkov wrote: > revision-id: 1a2748ceb0e30770eebc97d82d7baa885b332c49 (mariadb-10.3.6-45-g1a2748c) > parent(s): aa01f51bdef9cc38d8e0a75ea9e2788651e41d16 > committer: Alexey Botchkov > timestamp: 2018-07-09 13:20:42 +0400 > message: > > MDEV-15473 Isolate/sandbox PAM modules, so that they can't crash the server. > > Scripts added to set safe permissions for the auth_pam_tool and it's > directory. > > diff --git a/scripts/mysql_install_db.sh b/scripts/mysql_install_db.sh > index ad7c028..fd26228 100644 > --- a/scripts/mysql_install_db.sh > +++ b/scripts/mysql_install_db.sh > @@ -308,6 +308,7 @@ then > srcpkgdatadir="$srcdir/scripts" > buildpkgdatadir="$builddir/scripts" > plugindir="$builddir/plugin/auth_socket" > + pamtooldir="$builddir/plugin/auth_pam" > elif test -n "$basedir" > then > bindir="$basedir/bin" # only used in the help text > @@ -337,6 +338,7 @@ then > exit 1 > fi > plugindir=`find_in_dirs --dir auth_socket.so $basedir/lib*/plugin $basedir/lib*/mysql/plugin` > + pamtooldir=`find_in_dirs --dir auth_pam.so $basedir/lib*/plugin $basedir/lib*/mysql/plugin` Why not just pamtooldir=$plugindir ? > else > basedir="@prefix@" > bindir="@bindir@" > @@ -445,6 +448,13 @@ done > > if test -n "$user" > then > + chown $user "$pamtooldir/auth_pam_tool_dir" > + if test $? -ne 0 > + then > + echo "Cannot change ownership of the '$pamtooldir\auth_pam_tool_dir' directory" > + echo " to the '$user' user. Check that you have the necessary permissions and try again." > + exit 1 > + fi Looks ok. Did you test that it works? I mean, a test woud be to install from a tarball and run mysql_install_db or install from rpm or deb (they run the install script automatically) and then check that all permissions are correct. Also, I suspect that at additional chown root "$pamtooldir/auth_pam_tool_dir/auth_pam_tool could also be useful here. But not when the script is running in the builddir. Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] [Commits] 765ba2a: MDEV-15473 Isolate/sandbox PAM modules, so that they can't crash the server.
by Sergei Golubchik 06 Jul '18

06 Jul '18

Hi, Alexey! On Jul 05, Alexey Botchkov wrote: > revision-id: 765ba2ac76dab984183bf829dc3407713a4d5d9b (mariadb-10.3.6-40-g765ba2a) > parent(s): 7e704a2308e25953b5f8fb154eb325df3e25c2ec > committer: Alexey Botchkov > timestamp: 2018-07-05 17:00:47 +0400 > message: > > MDEV-15473 Isolate/sandbox PAM modules, so that they can't crash the server. > > Proper access permissions for the auth_pam_tool_dir and auth_pam_tool. > > diff --git a/plugin/auth_pam/CMakeLists.txt b/plugin/auth_pam/CMakeLists.txt > index 4943d57..b9313de 100644 > --- a/plugin/auth_pam/CMakeLists.txt > +++ b/plugin/auth_pam/CMakeLists.txt > @@ -11,7 +11,13 @@ IF(HAVE_PAM_APPL_H) > ADD_DEFINITIONS(-D_GNU_SOURCE) > MYSQL_ADD_PLUGIN(auth_pam_v1 auth_pam_v1.c LINK_LIBRARIES pam MODULE_ONLY) > MYSQL_ADD_PLUGIN(auth_pam auth_pam.c LINK_LIBRARIES pam dl MODULE_ONLY) > - MYSQL_ADD_EXECUTABLE(auth_pam_tool auth_pam_tool.c DESTINATION ${INSTALL_PLUGINDIR}/auth_pam_tool_dir COMPONENT Server) > + MYSQL_ADD_EXECUTABLE(auth_pam_tool auth_pam_tool.c DESTINATION ${INSTALL_PLUGINDIR}/auth_pam_tool_dir COMPONENT Server) > TARGET_LINK_LIBRARIES(auth_pam_tool pam) > + INSTALL(CODE "EXECUTE_PROCESS( > + COMMAND chmod u=rwx,g=,o= auth_pam_tool_dir > + COMMAND chmod u=rwx,g=rx,o=rx auth_pam_tool_dir/auth_pam_tool > + COMMAND chmod +s auth_pam_tool_dir/auth_pam_tool > + WORKING_DIRECTORY \$ENV{DESTDIR}\${CMAKE_INSTALL_PREFIX}/${INSTALL_PLUGINDIR}/)" > + COMPONENT Server) I think it's generally ok. Two comments: minor: you can combine two chmods on auth_pam_tool in one, like u=rwxs major: you still need to make auth_pam_tool_dir to be owned by mysql user. I'm afraid the only way to do it is from a post-install scriptlet or from mysql_install_db. Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] [Commits] af15686: MDEV-16242: MyRocks: parallel slave on a table without PK can stop with ER_KEY_NOT_FOUND
by Kristian Nielsen 05 Jul '18

05 Jul '18

psergey(a)askmonty.org (Sergei Petrunia) writes: > MDEV-16242: MyRocks: parallel slave on a table without PK can stop with ER_KEY_NOT_FOUND > > DRAFT: If RBR event applier uses a secondary key or a full table scan > to locate a row, force waiting for prior commit to complete. I think if you wait_for_prior_commit() also for secondary key case, you should definitely restrict it to be only for conservative parallel replication, not for optimistic/agressive. Using a secondary key seems generally fine (as opposed to full table scan), and this problem should affect only conservative mode, no need to penalise optimistic/aggressive. (But maybe that is what is meant with "DRAFT"). - Kristian.

2 2

Re: [Maria-developers] b88e98df9da: C++11 experiment: check if all builders support it
by Sergei Golubchik 04 Jul '18

04 Jul '18

Hi, Sergey! > Current status is: all staging builders pass, except for Ubuntu Precise. But we no longer build on precise, do we? Any followup MDEVs after this one is pushed? Like migrate atomics ops to C++11 or something? > revision-id: b88e98df9daa586bd6ed1f5908ededaa65a6e464 (mariadb-10.3.6-16-gb88e98df9da) > parent(s): 9988a423d229730b252942b7fa066d1591b7eca9 > author: Sergey Vojtovich > committer: Sergey Vojtovich > timestamp: 2018-06-04 12:41:18 +0400 > message: > > C++11 experiment: check if all builders support it > > diff --git a/CMakeLists.txt b/CMakeLists.txt > index 76b0817f8c7..efeae3d1ca5 100644 > --- a/CMakeLists.txt > +++ b/CMakeLists.txt > @@ -89,6 +89,14 @@ ELSE() > ENDIF() > PROJECT(${MYSQL_PROJECT_NAME}) > > +IF(CMAKE_VERSION VERSION_LESS "3.1") > + IF(CMAKE_CXX_COMPILER_ID STREQUAL "GNU") > + SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=gnu++11") Why gnu++11 and not c++11 ? And why not for clang? > + ENDIF() > +ELSE() > + SET(CMAKE_CXX_STANDARD 11) I'd expect this to mean c++11, not gnu++11. > +ENDIF() > + > SET(CPACK_PACKAGE_NAME "MariaDB") > SET(CPACK_PACKAGE_DESCRIPTION_SUMMARY "MariaDB: a very fast and robust SQL database server") > SET(CPACK_PACKAGE_URL "http://mariadb.org") > diff --git a/storage/tokudb/PerconaFT/portability/toku_crash.cc b/storage/tokudb/PerconaFT/portability/toku_crash.cc > index 0af85342a99..297cc29d9ca 100644 > --- a/storage/tokudb/PerconaFT/portability/toku_crash.cc > +++ b/storage/tokudb/PerconaFT/portability/toku_crash.cc > @@ -70,7 +70,7 @@ run_gdb(pid_t parent_pid, const char *gdb_path) { > "-ex", "thread apply all bt", > "-ex", "thread apply all bt full", > exe_buf, pid_buf, > - NULL); > + (char*) NULL); Why's that? Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

3 3

[Maria-developers] MDEV-13995 MAX(timestamp) returns a wrong result near DST change
by Alexander Barkov 03 Jul '18

03 Jul '18

Hi Sergei, Please review a patch for MDEV-13995. Thanks!

1 0

[Maria-developers] FOR loop documentation
by Alexander Barkov 03 Jul '18

03 Jul '18

Hi Ian, can you please document the FOR loop? I've created an MDEV for it: MDEV-16674 Document FOR loop with links to all relevant tasks. Thanks!

1 0

Re: [Maria-developers] ed2dcf0: MDEV-15473 Isolate/sandbox PAM modules, so that they can't crash the server.
by Sergei Golubchik 01 Jul '18

01 Jul '18

Hi, Alexey! Looks good, few comments/suggestions below. That and my reply to your email (about tests and plugin name). Just push when you're ready, no need to put it In-Review again. Thanks! On Jul 01, Alexey Botchkov wrote: > revision-id: ed2dcf00fd26e363f34638a841ff17a111b7e324 (mariadb-10.3.6-33-ged2dcf0) > parent(s): 4b0cedf82d8d8ba582648dcb4a2620c146862a43 > committer: Alexey Botchkov > timestamp: 2018-07-01 12:23:54 +0400 > message: > > MDEV-15473 Isolate/sandbox PAM modules, so that they can't crash the server. > > The new 'safe' version of the PAM plugin. > It works using the auth_pam_tool executable. That isolates possible > crashes in pam modules and, since we can set the +s bit for the tool, > we can normally run modules that read protected information. > > diff --git a/plugin/auth_pam/CMakeLists.txt b/plugin/auth_pam/CMakeLists.txt > index 5131752..9c0859c 100644 > --- a/plugin/auth_pam/CMakeLists.txt > +++ b/plugin/auth_pam/CMakeLists.txt > @@ -8,6 +8,10 @@ IF(HAVE_PAM_APPL_H) > IF(HAVE_STRNDUP) > ADD_DEFINITIONS(-DHAVE_STRNDUP) > ENDIF(HAVE_STRNDUP) > + ADD_DEFINITIONS(-D_GNU_SOURCE) > MYSQL_ADD_PLUGIN(auth_pam auth_pam.c LINK_LIBRARIES pam MODULE_ONLY) > + MYSQL_ADD_PLUGIN(auth_pam_safe auth_pam_safe.c LINK_LIBRARIES pam dl MODULE_ONLY) perhaps, rename to auth_pam_new.c (and auth_pam_new.so)? > + MYSQL_ADD_EXECUTABLE(auth_pam_tool auth_pam_tool.c DESTINATION ${INSTALL_PLUGINDIR}/auth_pam_tool_dir COMPONENT Server) > + TARGET_LINK_LIBRARIES(auth_pam_tool pam) > ENDIF(HAVE_PAM_APPL_H) > > diff --git a/plugin/auth_pam/auth_pam.c b/plugin/auth_pam/auth_pam.c > index ffc3d6f..c785ba4 100644 > --- a/plugin/auth_pam/auth_pam.c > +++ b/plugin/auth_pam/auth_pam.c > @@ -1,5 +1,5 @@ > /* > - Copyright (c) 2011, 2012, Monty Program Ab > + Copyright (c) 2018 MariaDB Corporation should be "2011, 2018" not just "2018" > This program is free software; you can redistribute it and/or modify > it under the terms of the GNU General Public License as published by > static struct st_mysql_auth info = > diff --git a/plugin/auth_pam/auth_pam_base.c b/plugin/auth_pam/auth_pam_base.c > new file mode 100644 > index 0000000..30b4e01 > --- /dev/null > +++ b/plugin/auth_pam/auth_pam_base.c > @@ -0,0 +1,171 @@ > +/* > + Copyright (c) 2018 MariaDB Corporation > + > + This program is free software; you can redistribute it and/or modify > + it under the terms of the GNU General Public License as published by > + the Free Software Foundation; version 2 of the License. > + > + This program is distributed in the hope that it will be useful, > + but WITHOUT ANY WARRANTY; without even the implied warranty of > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > + GNU General Public License for more details. > + > + You should have received a copy of the GNU General Public License > + along with this program; if not, write to the Free Software > + Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02111-1301 USA */ > + > +// struct param { > + // unsigned char buf[10240], *ptr; > + // MYSQL_PLUGIN_VIO *vio; > +// }; > +//static int write_packet(struct param *param, const unsigned char *buf, > + //int buf_len) > +//static int read_packet(struct param *param, unsigned char **pkt) forgot to remove these comments? > + > +#include <stdio.h> > +#include <string.h> > +#include <security/pam_appl.h> > +#include <security/pam_modules.h> > + > diff --git a/plugin/auth_pam/auth_pam_safe.c b/plugin/auth_pam/auth_pam_safe.c > new file mode 100644 > index 0000000..9dac90d > --- /dev/null > +++ b/plugin/auth_pam/auth_pam_safe.c > @@ -0,0 +1,235 @@ ... > + PAM_DEBUG((stderr, "PAM: Child process prepares pipes.\n")); > + > + if (close(p_to_c[1]) < 0 || > + close(c_to_p[0]) < 0 || > + dup2(p_to_c[0], 0) < 0 || /* Parent's pipe to STDIN. */ > + dup2(c_to_p[1], 1) < 0) /* Sandbox's pipe to STDOUT. */ > + { > + exit(-1); > + } > + > + PAM_DEBUG((stderr, "PAM: check tool directory: %s, %s.\n", > + opt_plugin_dir, tool_name)); > + plugin_dir_len= strlen(opt_plugin_dir); > + if (plugin_dir_len + tool_name_len + 2 > sizeof(toolpath)) > + { > + /* Tool path too long. */ > + exit(-1); > + } > + > + memcpy(toolpath, opt_plugin_dir, plugin_dir_len); > + if (plugin_dir_len && toolpath[plugin_dir_len-1] != FN_LIBCHAR) > + toolpath[plugin_dir_len++]= FN_LIBCHAR; > + memcpy(toolpath+plugin_dir_len, tool_name, tool_name_len+1); check the length before copying? like, toolpath+plugin_dir_len+tool_name_len+1 < sizeof(toolpath) ? or simply strnxfrm(toolpath, sizeof(toolpath), opt_plugin_dir, "/", tool_name, NULL); > + > + PAM_DEBUG((stderr, "PAM: execute pam sandbox [%s].\n", toolpath)); > + (void) execl(toolpath, toolpath, NULL); > + PAM_DEBUG((stderr, "PAM: exec() failed.\n")); > + exit(-1); > + } ... > +static char use_cleartext_plugin; > +static MYSQL_SYSVAR_BOOL(use_cleartext_plugin, use_cleartext_plugin, > + PLUGIN_VAR_NOCMDARG | PLUGIN_VAR_READONLY, > + "Use mysql_cleartext_plugin on the client side instead of the dialog " > + "plugin. This may be needed for compatibility reasons, but it only " > + "supports simple PAM policies that don't require anything besides " > + "a password", NULL, NULL, 0); > + > +#ifndef DBUG_OFF > +static MYSQL_SYSVAR_BOOL(debug, pam_debug, PLUGIN_VAR_OPCMDARG, > + "Log all PAM activity", NULL, NULL, 0); > +#endif still a lot of similar code in auth_pam.c and auth_pam_safe.c. could be moved to auth_pam_common.c? > + > + > diff --git a/plugin/auth_pam/auth_pam_tool.c b/plugin/auth_pam/auth_pam_tool.c > new file mode 100644 > index 0000000..5f19d29 > --- /dev/null > +++ b/plugin/auth_pam/auth_pam_tool.c > @@ -0,0 +1,132 @@ > + > +int main(int argc, char **argv) > +{ > + struct param param; > + MYSQL_SERVER_AUTH_INFO info; > + unsigned char field; > + int res; > + char a_buf[MYSQL_USERNAME_LENGTH + 1 + 1024]; > + > + if (read(0, &field, 1) < 1) > + return -1; > +#ifndef DBUG_OFF > + pam_debug= field; > +#endif > + > + PAM_DEBUG((stderr, "PAM: sandbox started [%s].\n", argv[0])); > + > + info.user_name= a_buf; > + if ((res= read_string(0, info.user_name, sizeof(a_buf))) < 0) > + return -1; > + PAM_DEBUG((stderr, "PAM: sandbox username [%s].\n", info.user_name)); > + > +#ifndef DBUG_OFF > + /* Produce the crash for testing purposes. */ > + if ((res == 14) && > + memcmp(info.user_name, "crash_pam_tool", 14) == 0) > + { > + info.user_name= 0; > + memcpy(info.user_name, a_buf, sizeof(a_buf)); > + return -1; > + } now I think it's better to put it into pam_mariadb_mtr.c, sorry :( then it doesn't need to be in debug-only builds > +#endif > + > + info.auth_string= info.user_name + res + 1; > + if (read_string(0, info.auth_string, sizeof(a_buf) - 1 - res) < 0) > + return -1; > + > + PAM_DEBUG((stderr, "PAM: sandbox auth string [%s].\n", info.auth_string)); Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] 84c9ec5: MDEV-15473 Isolate/sandbox PAM modules, so that they can't crash the server.
by Sergei Golubchik 01 Jul '18

01 Jul '18

Hi, Alexey! On Jun 30, Alexey Botchkov wrote: > > not sure about calling it "safe". I think it's more of a side > > effect, the main feature it that it works, while old pam plugin > > simply doesn't :) unless mysqld is run as root. > > > > and not totally sure about calling it also pam. it means one won't > > be able to load it and the old pam plugin at the same time. I > > suspect it'll still ok and benefits overweight it. > > Well i'm open to any ideas here :) > We can swap 'safe' with somethin else. 'box' for instance. > Or we can instead rename the original version like 'fast' or 'old'. > Finally we can build only one version of the plugin. The 'safe' > whenever possible. I'd rather rename the old version to (old) or (requires root) or something. > I also have a question about the testing. > Now pam*.test-s rely on some mariadb_mtr setup for PAM. How can i see what > is in the expected pam.d configuration file, and what pam modules are used? See plugin/auth_pam/testing/pam_mariadb_mtr.c Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] FYI: 10.3 master branch head builds OK Travis-CI
by Otto Kekäläinen 30 Jun '18

30 Jun '18

Hello! I am happy to announce that of this morning, Travis-CI for MariaDB 10.3 shows all green again! [1] Developers: please rebase your 10.3 branches on top of the latest 10.3 to get the Travis-CI benefits, and please don't break Travis. Last time it was all green in January it took only a few weeks until it was broken again. We don't have protected branches enabled yet, so we rely on all developers to be diligent and not push changes that break the all-green-status on Travis-CI for 10.3. Otto [1] https://travis-ci.org/MariaDB/server/builds [2} https://mariadb.org/protected-branches-ensuring-code-quality-git/

1 0

[Maria-developers] MariaDB 10.3.7 and GSSAPI : Issue
by REIX, Tony 28 Jun '18

28 Jun '18

Hi, We are porting MariaDB (10.3.7 now) on AIX. On some machines we have the following error message when running: + cmake . --trace --debug-trycompile --check-system-vars -GUnix Makefiles -DBUILD_CONFIG=mysql_release -DFEATURE_SET=community -DINSTALL_LAYOUT=RPM -DLOG_LOCATION=/opt/freeware/mariadb/mariadb.log -DPID_FILE_DIR=%{_rundir}/mariadb -DNICE_PROJECT_NAME=MariaDB -DCMAKE_INSTALL_PREFIX=/opt/freeware -DCMAKE_AR=/usr/bin/ar -DCMAKE_SYSTEM_NAME=AIX -DINSTALL_SYSCONFDIR=/opt/freeware/etc -DINSTALL_SYSCONF2DIR=/opt/freeware/etc/my.cnf.d -DINSTALL_INCLUDEDIR=include/mysql -DINSTALL_INFODIR=share/info -DINSTALL_LIBDIR=/opt/freeware/lib64 -DINSTALL_MANDIR=share/man -DINSTALL_MYSQLSHAREDIR=share/mariadb -DINSTALL_PLUGINDIR=/opt/freeware/lib64/mariadb/plugin -DINSTALL_MYSQLTESTDIR=share/mysql-test -DINSTALL_SBINDIR=libexec -DINSTALL_SCRIPTDIR=bin -DINSTALL_SQLBENCHDIR=share -DINSTALL_SUPPORTFILESDIR=share/mariadb -DMYSQL_DATADIR=/opt/freeware/var/lib/mysql -DMYSQL_UNIX_ADDR=/var/lib64/mysql/mysql.sock -DTMPDIR=/var/tmp -DENABLED_LOCAL_INFILE=ON -DSECURITY_HARDENED=ON -DENABLE_DTRACE=ON -DWITH_EMBEDDED_SERVER=ON -DWITH_MARIABACKUP=ON -DWITH_UNIT_TESTS=ON -DCONC_WITH_SSL=ON -DWITH_SSL=system -DWITH_ZLIB=system -DWITH_JEMALLOC=no -DPLUGIN_MROONGA=DYNAMIC -DPLUGIN_OQGRAPH=DYNAMIC -DPLUGIN_CRACKLIB_PASSWORD_CHECK=DYNAMIC -DPLUGIN_ROCKSDB=DYNAMIC -DPLUGIN_SPHINX=DYNAMIC -DPLUGIN_TOKUDB=DYNAMIC -DPLUGIN_CONNECT=DYNAMIC -DCONNECT_WITH_MONGO=OFF -DCONNECT_WITH_JDBC=OFF -DCMAKE_CROSSCOMPILING=FALSE -DWITH_VALGRIND=OFF -DCMAKE_BUILD_TYPE=Debug -DWITH_ASAN=OFF -DWITH_INNODB_EXTRA_DEBUG=ON ...... /opt/freeware/share/cmake/Modules/CheckSymbolExists.cmake(87): try_compile(HAVE_KRB5_FREE_UNPARSED_NAME ${CMAKE_BINARY_DIR} /opt/freeware/src/packages/BUILD/mariadb-10.3.7/64bit/CMakeFiles/CMakeTmp/CheckSymbolExists.cxx COMPILE_DEFINITIONS ${CMAKE_REQUIRED_DEFINITIONS} ${CHECK_SYMBOL_EXISTS_LIBS} CMAKE_FLAGS -DCOMPILE_DEFINITIONS:STRING=${MACRO_CHECK_SYMBOL_EXISTS_FLAGS} ${CMAKE_SYMBOL_EXISTS_INCLUDES} OUTPUT_VARIABLE OUTPUT ) CMake Error at /opt/freeware/src/packages/BUILD/mariadb-10.3.7/64bit/CMakeFiles/CMakeTmp/CMakeLists.txt:14 (add_executable): Target "cmTC_3efe0" links to target "-L/opt/freeware/lib -blibpath:/opt/freeware/lib::/usr/lib:/lib -I/usr/include -I/opt/freeware/include -L/opt/freeware/lib -blibpath:/opt/freeware/lib:/usr/lib:/lib -bmaxdata:0x80000000 -brtl -lgssapi_krb5 -lkrb5 -lk5crypto -lcom_err -lpthreads" but the target was not found. Perhaps a find_package() call is missing for an IMPORTED target, or an ALIAS target is missing? CMake Error at /opt/freeware/share/cmake/Modules/CheckSymbolExists.cmake:87 (try_compile): Failed to generate test project build system. Call Stack (most recent call first): /opt/freeware/share/cmake/Modules/CheckCXXSymbolExists.cmake:39 (__CHECK_SYMBOL_EXISTS_IMPL) plugin/auth_gssapi/CMakeLists.txt:24 (CHECK_CXX_SYMBOL_EXISTS) In file: mariadb-10.3.7/64bit/CMakeFiles/CMakeTmp/CMakeLists.txt , there is: cmake_minimum_required(VERSION 3.11.4.0) set(CMAKE_MODULE_PATH "/opt/freeware/src/packages/BUILD/mariadb-10.3.7/64bit/cmake;/opt/freeware/src/packages/BUILD/mariadb-10.3.7/64bit/plugin/auth_gssapi/cmake") project(CMAKE_TRY_COMPILE CXX) set(CMAKE_VERBOSE_MAKEFILE 1) set(CMAKE_CXX_FLAGS " -pie -fPIC -fPIC -fno-rtti") set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${COMPILE_DEFINITIONS}") set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} ${EXE_LINKER_FLAGS}") include_directories(${INCLUDE_DIRECTORIES}) set(CMAKE_SUPPRESS_REGENERATION 1) link_directories(${LINK_DIRECTORIES}) add_definitions(-DPACKAGE=test -D_LARGEFILE_SOURCE -D_LARGE_FILES -D_FILE_OFFSET_BITS=64 -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS) cmake_policy(SET CMP0065 OLD) set(CMAKE_RUNTIME_OUTPUT_DIRECTORY "/opt/freeware/src/packages/BUILD/mariadb-10.3.7/64bit/CMakeFiles/CMakeTmp") add_executable(cmTC_3efe0 "/opt/freeware/src/packages/BUILD/mariadb-10.3.7/64bit/CMakeFiles/CMakeTmp/CheckSymbolExists.cxx") target_link_libraries(cmTC_3efe0 "-L/opt/freeware/lib -blibpath:/opt/freeware/lib::/usr/lib:/lib -I/usr/include -I/opt/freeware/include -L/opt/freeware/lib -blibpath:/opt/freeware/lib:/usr/lib:/lib -bmaxdata:0x80000000 -brtl -lgssapi_krb5 -lkrb5 -lk5crypto -lcom_err -lpthreads" ) Talking with CMake guys, they said: > I suspect mariadb's source is setting CMAKE_REQUIRED_LIBRARIES > to a space-separated value instead of a ;-separated value. > That's broken here: https://github.com/MariaDB/server/blob/ed0b84a0270bd99b001dd00654875d26e29b…<https://github.com/MariaDB/server/blob/ed0b84a0270bd99b001dd00654875d26e29b…> Do you have an idea about what is broken and what should be changed? either on AIX or for all target? Thanks/Regards, Cordialement, Tony Reix ATOS / Bull SAS ATOS Expert IBM Coop Architect & Technical Leader Office : +33 (0) 4 76 29 72 67 1 rue de Provence - 38432 Échirolles - France www.atos.net<https://mail.ad.bull.net/owa/redir.aspx?C=PvphmPvCZkGrAgHVnWGsdMcDKgzl_dEIs…>

2 3

[Maria-developers] Dead code in class sp_head
by Alexander Barkov 26 Jun '18

26 Jun '18

Hi Aleksey, I found some dead code in sp_head that seem to have appeared in your patch for "IB: 0.2 part IV". If you don't mind, I'll remove it. Please have a look into the attached patch. Thanks.

2 1

Re: [Maria-developers] [Commits] MDEV-16166 RBR breaks with HA_ERR_KEY_NOT_FOUND upon DELETE
by Alexey Botchkov 25 Jun '18

25 Jun '18

Ok to push. 05.06.2018 13:45, Sachin Setiya wrote: > commit bde95bc70d0ac011f55f1b3b55fee1c9276859da > Author: sachin <sachin.setiya(a)maridb.com > <mailto:sachin.setiya@maridb.com>> > Date: Tue Jun 5 15:14:19 2018 +0530 > > MDEV-16166 RBR breaks with HA_ERR_KEY_NOT_FOUND upon DELETE from > table... > with spatial index > So the issue is since it is spatial index , at the time of > searching index > for key (Rows_log_event::find_row) we use wrong field image we use > Field::itRAW while we should be using Field::itMBR > > diff --git a/mysql-test/suite/rpl/r/rpl_row_spatial.result > b/mysql-test/suite/rpl/r/rpl_row_spatial.result > new file mode 100644 > index 00000000000..8f546fc479e > --- /dev/null > +++ b/mysql-test/suite/rpl/r/rpl_row_spatial.result > @@ -0,0 +1,14 @@ > +include/master-slave.inc > +[connection master] > +CREATE TABLE t1 (g POINT NOT NULL, SPATIAL INDEX(g)); > +INSERT INTO t1 VALUES (ST_GEOMFROMTEXT('Point(1 1)')); > +INSERT INTO t1 VALUES (ST_GEOMFROMTEXT('Point(2 1)')); > +INSERT INTO t1 VALUES (ST_GEOMFROMTEXT('Point(1 2)')); > +INSERT INTO t1 VALUES (ST_GEOMFROMTEXT('Point(2 2)')); > +DELETE FROM t1 where MBREqual(g, ST_GEOMFROMTEXT('Point(1 2)')); > +select count(*) from t1; > +count(*) > +3 > +DELETE FROM t1; > +drop table t1; > +include/rpl_end.inc > diff --git a/mysql-test/suite/rpl/t/rpl_row_spatial.test > b/mysql-test/suite/rpl/t/rpl_row_spatial.test > new file mode 100644 > index 00000000000..00c3dd7c54d > --- /dev/null > +++ b/mysql-test/suite/rpl/t/rpl_row_spatial.test > @@ -0,0 +1,17 @@ > +--source include/have_binlog_format_row.inc > +--source include/master-slave.inc > + > +CREATE TABLE t1 (g POINT NOT NULL, SPATIAL INDEX(g)); > +INSERT INTO t1 VALUES (ST_GEOMFROMTEXT('Point(1 1)')); > +INSERT INTO t1 VALUES (ST_GEOMFROMTEXT('Point(2 1)')); > +INSERT INTO t1 VALUES (ST_GEOMFROMTEXT('Point(1 2)')); > +INSERT INTO t1 VALUES (ST_GEOMFROMTEXT('Point(2 2)')); > +DELETE FROM t1 where MBREqual(g, ST_GEOMFROMTEXT('Point(1 2)')); > + > +--sync_slave_with_master > +select count(*) from t1; > + > +--connection master > +DELETE FROM t1; > +drop table t1; > +--source include/rpl_end.inc > diff --git a/sql/key.cc b/sql/key.cc > index 700bf6a05a6..7e5a3309b10 100644 > --- a/sql/key.cc > +++ b/sql/key.cc > @@ -145,7 +145,8 @@ void key_copy(uchar *to_key, uchar *from_record, > KEY *key_info, > { > key_length-= HA_KEY_BLOB_LENGTH; > length= min(key_length, key_part->length); > - uint bytes= key_part->field->get_key_image(to_key, length, > Field::itRAW); > + uint bytes= key_part->field->get_key_image(to_key, length, > +key_info->flags & HA_SPATIAL ? Field::itMBR : Field::itRAW); > if (with_zerofill && bytes < length) > bzero((char*) to_key + bytes, length - bytes); > to_key+= HA_KEY_BLOB_LENGTH; > > > -- > Regards > Sachin Setiya > Software Engineer at MariaDB > > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits

1 0

[Maria-developers] MDEV-12574 MAX(old_decimal) produces a column of the old DECIMAL type
by Alexander Barkov 24 Jun '18

24 Jun '18

Hi Sergei, Please review a patch for MDEV-12574. I suggest we push it into 10.2 (not into 10.3). Thanks!

2 1

Re: [Maria-developers] 37b432c1d59: MDEV-15242 Poor RBR update performance with partitioned tables
by Sergei Golubchik 22 Jun '18

22 Jun '18

Hi, andrei.elkin! On Jun 19, andrei.elkin(a)pp.inet.fi wrote: > revision-id: 37b432c1d59e66b742f97a280cae7784d85e62ec (mariadb-10.0.35-41-g37b432c1d59) > parent(s): d2e1ed8b936a78ceeec0e64231997d7ed18a4daf > author: Andrei Elkin > committer: Andrei Elkin > timestamp: 2018-06-19 18:14:47 +0300 > message: > > MDEV-15242 Poor RBR update performance with partitioned tables > > Observed and described > partitioned engine execution time difference > between master and slave was caused by excessive invocation > of base_engine::rnd_init which was done also for partitions > uninvolved into Rows-event operation. > The bug's slave slowdown therefore scales with the number of partitions. Ok to push > Fixed with applying an upstream patch. > > References: > ---------- > https://bugs.mysql.com/bug.php?id=73648 > Bug#25687813 REPLICATION REGRESSION WITH RBR AND PARTITIONED TABLES > Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] MariaDB Developers Unconference next week
by Otto Kekäläinen 22 Jun '18

22 Jun '18

Hello everybody! If you plan to attend the MariaDB Developers Unconference and related activities next week,* make sure you have signed up <https://www.meetup.com/MariaDB-Developers/events/249024232/> to the meetup* . We will soon be posting important information to attendees and only people who have signed up will receive it. Info: - www.meetup.com/MariaDB-Developers/events/249024232/ <https://www.meetup.com/MariaDB-Developers/events/249024232/> - mariadb.org/2018-2-developers-unconference-in-finland/ - mariadb.org/travel-tips-to-tampere-finland/

1 0

Re: [Maria-developers] 84c9ec5: MDEV-15473 Isolate/sandbox PAM modules, so that they can't crash the server.
by Sergei Golubchik 22 Jun '18

22 Jun '18

Hi, Alexey! Just a couple of comments (args[] vs stdio and how to prevent brute-forcing), see below. Otherwise looks pretty good. On Jun 06, Alexey Botchkov wrote: > revision-id: 84c9ec5552bf14bda826912fb045ae36f34a4490 (mariadb-10.3.6-9-g84c9ec5) > parent(s): 637af7838316a49267e55cde7cf96e23f63e5c9d > committer: Alexey Botchkov > timestamp: 2018-06-06 14:13:07 +0400 > message: > > MDEV-15473 Isolate/sandbox PAM modules, so that they can't crash the server. > > Crash-safe version of the PAM plugin added. It runs the auth_pam_tool > application that actually calls the PAM modules. > > diff --git a/plugin/auth_pam/auth_pam_base.c b/plugin/auth_pam/auth_pam_base.c > new file mode 100644 > index 0000000..885e6f0 > --- /dev/null > +++ b/plugin/auth_pam/auth_pam_base.c > @@ -0,0 +1,173 @@ > +/* > + Copyright (c) 2011, 2012, Monty Program Ab could be updated... > + > + This program is free software; you can redistribute it and/or modify > + it under the terms of the GNU General Public License as published by > + the Free Software Foundation; version 2 of the License. > + > + This program is distributed in the hope that it will be useful, > + but WITHOUT ANY WARRANTY; without even the implied warranty of > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > + GNU General Public License for more details. > + > + You should have received a copy of the GNU General Public License > + along with this program; if not, write to the Free Software > + Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02111-1301 USA */ > + > +// struct param { > + // unsigned char buf[10240], *ptr; > + // MYSQL_PLUGIN_VIO *vio; > +// }; > +//static int write_packet(struct param *param, const unsigned char *buf, > + //int buf_len) > +//static int read_packet(struct param *param, unsigned char **pkt) > + > +#define _GNU_SOURCE 1 /* for strndup */ > + > +#include <stdio.h> > +#include <string.h> > +#include <security/pam_appl.h> > +#include <security/pam_modules.h> > + > +/* It least solaris doesn't have strndup */ > + > +#ifndef HAVE_STRNDUP > +char *strndup(const char *from, size_t length) > +{ > + char *ptr; > + size_t max_length= strlen(from); > + if (length > max_length) > + length= max_length; > + if ((ptr= (char*) malloc(length+1)) != 0) > + { > + memcpy((char*) ptr, (char*) from, length); > + ptr[length]=0; > + } > + return ptr; > +} > +#endif > + > +#ifndef DBUG_OFF > +static char pam_debug = 0; > +#define PAM_DEBUG(X) do { if (pam_debug) { fprintf X; } } while(0) > +#else > +#define PAM_DEBUG(X) /* no-op */ > +#endif > + > +static int conv(int n, const struct pam_message **msg, > + struct pam_response **resp, void *data) > +{ > + struct param *param = (struct param *)data; > + unsigned char *end = param->buf + sizeof(param->buf) - 1; > + int i; > + > + *resp = 0; > + > + for (i = 0; i < n; i++) > + { > + /* if there's a message - append it to the buffer */ > + if (msg[i]->msg) > + { > + int len = strlen(msg[i]->msg); > + if (len > end - param->ptr) > + len = end - param->ptr; > + if (len > 0) > + { > + memcpy(param->ptr, msg[i]->msg, len); > + param->ptr+= len; > + *(param->ptr)++ = '\n'; > + } > + } > + /* if the message style is *_PROMPT_*, meaning PAM asks a question, > + send the accumulated text to the client, read the reply */ > + if (msg[i]->msg_style == PAM_PROMPT_ECHO_OFF || > + msg[i]->msg_style == PAM_PROMPT_ECHO_ON) > + { > + int pkt_len; > + unsigned char *pkt; > + > + /* allocate the response array. > + freeing it is the responsibility of the caller */ > + if (*resp == 0) > + { > + *resp = calloc(sizeof(struct pam_response), n); > + if (*resp == 0) > + return PAM_BUF_ERR; > + } > + > + /* dialog plugin interprets the first byte of the packet > + as the magic number. > + 2 means "read the input with the echo enabled" > + 4 means "password-like input, echo disabled" > + C'est la vie. */ > + param->buf[0] = msg[i]->msg_style == PAM_PROMPT_ECHO_ON ? 2 : 4; > + PAM_DEBUG((stderr, "PAM: conv: send(%.*s)\n", > + (int)(param->ptr - param->buf - 1), param->buf)); > + if (write_packet(param, param->buf, param->ptr - param->buf - 1)) > + return PAM_CONV_ERR; > + > + pkt_len = read_packet(param, &pkt); > + if (pkt_len < 0) > + { > + PAM_DEBUG((stderr, "PAM: conv: recv() ERROR\n")); > + return PAM_CONV_ERR; > + } > + PAM_DEBUG((stderr, "PAM: conv: recv(%.*s)\n", pkt_len, pkt)); > + /* allocate and copy the reply to the response array */ > + if (!((*resp)[i].resp= strndup((char*) pkt, pkt_len))) > + return PAM_CONV_ERR; > + param->ptr = param->buf + 1; > + } > + } > + return PAM_SUCCESS; > +} > + > +#define DO(X) if ((status = (X)) != PAM_SUCCESS) goto end > + > +#if defined(SOLARIS) || defined(__sun) > +typedef void** pam_get_item_3_arg; > +#else > +typedef const void** pam_get_item_3_arg; > +#endif > + > +static int pam_auth_base(struct param *param, MYSQL_SERVER_AUTH_INFO *info) > +{ > + pam_handle_t *pamh = NULL; > + int status; > + const char *new_username= NULL; > + /* The following is written in such a way to make also solaris happy */ > + struct pam_conv pam_start_arg = { &conv, (char*) param }; > + > + /* > + get the service name, as specified in > + > + CREATE USER ... IDENTIFIED WITH pam AS "service" > + */ > + const char *service = info->auth_string && info->auth_string[0] > + ? info->auth_string : "mysql"; > + > + param->ptr = param->buf + 1; > + > + PAM_DEBUG((stderr, "PAM: pam_start(%s, %s)\n", service, info->user_name)); > + DO( pam_start(service, info->user_name, &pam_start_arg, &pamh) ); > + > + PAM_DEBUG((stderr, "PAM: pam_authenticate(0)\n")); > + DO( pam_authenticate (pamh, 0) ); > + > + PAM_DEBUG((stderr, "PAM: pam_acct_mgmt(0)\n")); > + DO( pam_acct_mgmt(pamh, 0) ); > + > + PAM_DEBUG((stderr, "PAM: pam_get_item(PAM_USER)\n")); > + DO( pam_get_item(pamh, PAM_USER, (pam_get_item_3_arg) &new_username) ); > + > + if (new_username && strcmp(new_username, info->user_name)) > + strncpy(info->authenticated_as, new_username, > + sizeof(info->authenticated_as)); > + info->authenticated_as[sizeof(info->authenticated_as)-1]= 0; > + > +end: > + pam_end(pamh, status); > + PAM_DEBUG((stderr, "PAM: status = %d user = %s\n", status, info->authenticated_as)); > + return status == PAM_SUCCESS ? CR_OK : CR_ERROR; > +} > + > diff --git a/plugin/auth_pam/auth_pam_safe.c b/plugin/auth_pam/auth_pam_safe.c > new file mode 100644 > index 0000000..28c10b8 > --- /dev/null > +++ b/plugin/auth_pam/auth_pam_safe.c > @@ -0,0 +1,235 @@ > +/* > + Copyright (c) 2011, 2012, Monty Program Ab could be updated... > + > + This program is free software; you can redistribute it and/or modify > + it under the terms of the GNU General Public License as published by > + the Free Software Foundation; version 2 of the License. > + > + This program is distributed in the hope that it will be useful, > + but WITHOUT ANY WARRANTY; without even the implied warranty of > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > + GNU General Public License for more details. > + > + You should have received a copy of the GNU General Public License > + along with this program; if not, write to the Free Software > + Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02111-1301 USA */ > + > +#define _GNU_SOURCE 1 /* for strndup */ > + > +#include <unistd.h> > +#include <string.h> > +#include <mysql/plugin_auth.h> > +#include "auth_pam_tool.h" > +#include <my_global.h> > + > +#ifndef DBUG_OFF > +static char pam_debug = 0; > +#define PAM_DEBUG(X) do { if (pam_debug) { fprintf X; } } while(0) > +#else > +#define PAM_DEBUG(X) /* no-op */ > +#endif > + > +static char *opt_plugin_dir; /* To be dynamically linked. */ > +static const char *tool_name= "auth_pam_tool"; > +static const int tool_name_len= 13; > + > +static int pam_auth(MYSQL_PLUGIN_VIO *vio, MYSQL_SERVER_AUTH_INFO *info) > +{ > + int p_to_c[2], c_to_p[2]; /* Parent-to-child and child-to-parent pipes. */ > + pid_t proc_id; > + int result= CR_ERROR;; double semicolon :) > + > + PAM_DEBUG((stderr, "PAM: opening pipes.\n")); > + if (pipe(p_to_c) < 0 || pipe(c_to_p) < 0) > + { > + /* Error creating pipes. */ > + return CR_ERROR; > + } > + PAM_DEBUG((stderr, "PAM: forking.\n")); > + if ((proc_id= fork()) < 0) > + { > + /* Error forking. */ > + close(p_to_c[0]); > + close(c_to_p[1]); > + goto error_ret; > + } > + > + if (proc_id == 0) > + { > + /* The 'sandbox' process started. */ > + const char *arg[5]; > + char toolpath[FN_REFLEN]; > + size_t plugin_dir_len; > + > + PAM_DEBUG((stderr, "PAM: Child process prepares pipes.\n")); > + > + if (close(p_to_c[1]) < 0 || > + close(c_to_p[0]) < 0 || > + dup2(p_to_c[0], 0) < 0 || /* Parent's pipe to STDIN. */ > + dup2(c_to_p[1], 1) < 0) /* Sandbox's pipe to STDOUT. */ > + { > + exit(-1); > + } > + > + PAM_DEBUG((stderr, "PAM: check tool directory: %s, %s.\n", > + opt_plugin_dir, tool_name)); > + plugin_dir_len= strlen(opt_plugin_dir); > + if (plugin_dir_len + tool_name_len + 2 > sizeof(toolpath)) > + { > + /* Tool path too long. */ > + exit(-1); > + } > + > + memcpy(toolpath, opt_plugin_dir, plugin_dir_len); > + if (plugin_dir_len && toolpath[plugin_dir_len-1] != FN_LIBCHAR) > + toolpath[plugin_dir_len++]= FN_LIBCHAR; > + memcpy(toolpath+plugin_dir_len, tool_name, tool_name_len+1); > + > + arg[0]= toolpath; > + arg[1]= info->user_name; > + arg[2]= info->auth_string; > + if (pam_debug) > + { > + arg[3]= "DEBUG"; > + arg[4]= 0; > + } > + else > + arg[3]= 0; I'd rather passed that via the pipe, like, in the first three lines. so that it wouldn't be seen in `ps` output. May be it's harmless, but better to avoid this argument altogether. Just do, like fprintf(p_to_c[0], "%s\n%s\n%d\n", info->user_name, info->auth_string, pam_debug); and in the tool, fscanf("%s\n%s\n%d\n"); Hmm. Taking into account that the tool is installed SUID, you need to take extra care to avoid buffer overflows, assuming someone can invoke the tool directly. So, a little bit more complex than fscanf("%s\n%s\n%d\n"), sorry > + > + PAM_DEBUG((stderr, "PAM: execute pam sandbox:[%s] [%s] [%s] [%s].\n", > + arg[0], arg[1], arg[2], > + arg[3] ? arg[3] : "No debug")); > + (void) execv(arg[0], (char **) arg); > + PAM_DEBUG((stderr, "PAM: exec() failed.\n")); > + exit(-1); > + } > + > + /* Parent process continues. */ > + > + PAM_DEBUG((stderr, "PAM: parent continues.\n")); > + if (close(p_to_c[0]) < 0 || > + close(c_to_p[1]) < 0) > + goto error_ret; > + > + for (;;) > + { > + unsigned char field; > + > + PAM_DEBUG((stderr, "PAM: listening to the sandbox.\n")); > + if (read(c_to_p[0], &field, 1) < 1) > + { > + PAM_DEBUG((stderr, "PAM: read failed.\n")); > + goto error_ret; > + } > + > + if (field == AP_EOF) > + { > + PAM_DEBUG((stderr, "PAM: auth OK returned.\n")); > + break; > + } > + > + switch (field) > + { > + case AP_AUTHENTICATED_AS: > + PAM_DEBUG((stderr, "PAM: reading authenticated_as string.\n")); > + if (read_string(c_to_p[0], info->authenticated_as, > + sizeof(info->authenticated_as)) < 0) > + goto error_ret; > + break; > + > + case AP_CONV: > + { > + unsigned char buf[10240]; > + int buf_len; > + unsigned char *pkt; > + > + PAM_DEBUG((stderr, "PAM: getting CONV string.\n")); > + if ((buf_len= read_string(c_to_p[0], (char *) buf, sizeof(buf))) < 0) > + goto error_ret; > + > + PAM_DEBUG((stderr, "PAM: sending CONV string.\n")); > + if (vio->write_packet(vio, buf, buf_len)) > + goto error_ret; > + > + PAM_DEBUG((stderr, "PAM: reading CONV answer.\n")); > + if ((buf_len= vio->read_packet(vio, &pkt)) < 0) > + goto error_ret; > + > + PAM_DEBUG((stderr, "PAM: answering CONV.\n")); > + if (write_string(p_to_c[1], pkt, buf_len)) > + goto error_ret; > + } > + break; > + > + default: > + PAM_DEBUG((stderr, "PAM: unknown sandbox field.\n")); > + goto error_ret; > + } > + } > + result= CR_OK; > + > +error_ret: > + close(p_to_c[1]); > + close(c_to_p[0]); > + > + PAM_DEBUG((stderr, "PAM: auth result %d.\n", result)); > + return result; > +} > + > +static struct st_mysql_auth info = > +{ > + MYSQL_AUTHENTICATION_INTERFACE_VERSION, > + "dialog", > + pam_auth > +}; > + > +static char use_cleartext_plugin; > +static MYSQL_SYSVAR_BOOL(use_cleartext_plugin, use_cleartext_plugin, > + PLUGIN_VAR_NOCMDARG | PLUGIN_VAR_READONLY, > + "Use mysql_cleartext_plugin on the client side instead of the dialog " > + "plugin. This may be needed for compatibility reasons, but it only " > + "supports simple PAM policies that don't require anything besides " > + "a password", NULL, NULL, 0); > + > +#ifndef DBUG_OFF > +static MYSQL_SYSVAR_BOOL(debug, pam_debug, PLUGIN_VAR_OPCMDARG, > + "Log all PAM activity", NULL, NULL, 0); > +#endif > + > + > +static struct st_mysql_sys_var* vars[] = { > + MYSQL_SYSVAR(use_cleartext_plugin), > +#ifndef DBUG_OFF > + MYSQL_SYSVAR(debug), > +#endif > + NULL > +}; > + > + > +static int init(void *p __attribute__((unused))) > +{ > + if (use_cleartext_plugin) > + info.client_auth_plugin= "mysql_clear_password"; > + if (!(opt_plugin_dir= dlsym(RTLD_DEFAULT, "opt_plugin_dir"))) > + return 1; > + > + return 0; > +} > + > +maria_declare_plugin(pam) > +{ > + MYSQL_AUTHENTICATION_PLUGIN, > + &info, > + "pam", > + "MariaDB Corp", > + "PAM based authentication (safe)", not sure about calling it "safe". I think it's more of a side effect, the main feature it that it works, while old pam plugin simply doesn't :) unless mysqld is run as root. and not totally sure about calling it also pam. it means one won't be able to load it and the old pam plugin at the same time. I suspect it'll still ok and benefits overweight it. > + PLUGIN_LICENSE_GPL, > + init, > + NULL, > + 0x0100, > + NULL, > + vars, > + "1.0", > + MariaDB_PLUGIN_MATURITY_STABLE Let's start from beta and change it to stable later :) > +} > +maria_declare_plugin_end; > diff --git a/plugin/auth_pam/auth_pam_tool.c b/plugin/auth_pam/auth_pam_tool.c > new file mode 100644 > index 0000000..e4addd3 > --- /dev/null > +++ b/plugin/auth_pam/auth_pam_tool.c > @@ -0,0 +1,115 @@ > +/* > + Copyright (c) 2011, 2012, Monty Program Ab could be updated... > + > + This program is free software; you can redistribute it and/or modify > + it under the terms of the GNU General Public License as published by > + the Free Software Foundation; version 2 of the License. > + > + This program is distributed in the hope that it will be useful, > + but WITHOUT ANY WARRANTY; without even the implied warranty of > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > + GNU General Public License for more details. > + > + You should have received a copy of the GNU General Public License > + along with this program; if not, write to the Free Software > + Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02111-1301 USA */ > + > +#define _GNU_SOURCE 1 /* for strndup */ you could just set _GNU_SOURCE once in CMakeLists.txt instead of doing it in every file > + > +#include <stdlib.h> > +#include <unistd.h> > +#include <mysql/plugin_auth_common.h> > + > +struct param { > + unsigned char buf[10240], *ptr; > +}; > + > + > +#include "auth_pam_tool.h" > + > + > +static int write_packet(struct param *param __attribute__((unused)), > + const unsigned char *buf, int buf_len) > +{ > + unsigned char b= AP_CONV; > + return write(1, &b, 1) < 1 || > + write_string(1, buf, buf_len); > +} > + > + > +static int read_packet(struct param *param, unsigned char **pkt) > +{ > + *pkt= (unsigned char *) param->buf; > + return read_string(0, (char *) param->buf, (int) sizeof(param->buf)) - 1; > +} > + > + > +typedef struct st_mysql_server_auth_info > +{ > + /** > + User name as sent by the client and shown in USER(). > + NULL if the client packet with the user name was not received yet. > + */ > + char *user_name; > + > + /** > + A corresponding column value from the mysql.user table for the > + matching account name > + */ > + char *auth_string; > + > + /** > + Matching account name as found in the mysql.user table. > + A plugin can override it with another name that will be > + used by MySQL for authorization, and shown in CURRENT_USER() > + */ > + char authenticated_as[MYSQL_USERNAME_LENGTH+1]; > +} MYSQL_SERVER_AUTH_INFO; > + > + > +#include "auth_pam_base.c" > + > + > +int main(int argc, char **argv) > +{ > + struct param param; > + MYSQL_SERVER_AUTH_INFO info; > + unsigned char field; > + int res; > + > + if (argc < 3) > + return -1; > + > + info.user_name= argv[1]; > + info.auth_string= argv[2]; > + > + if (argc >= 4) > + pam_debug= 1; As we discussed, there should be some ways to prevent it being used to brute-force passwords. It'll be suid root, so it cannot be made executable only by "mysql" user. I see two approaches here: 1. getuid() and check that the user name is "mysql" 2. getppid() and check that it matches the value in a pid_file second is heavier (one might need to read my.cnf to find the location of a pid_file). first hard-codes the user name. oh, a third approach. create a new directory under plugin-dir, say auth_tool with permissions r-x------ and owned by "mysql". Put the actual executable there, so nobody besides "mysql" user will be able to get to it. Yes, this seems the easiest, fastest, and most flexible too :) > + > + PAM_DEBUG((stderr, "PAM: sandbox started [%s] [%s] [%s] [%s].\n", argv[0], > + info.user_name, info.auth_string, > + pam_debug ? argv[3] : "No debug")); Hmm, where will it go? messages written by the tool to its stderr > + > + if ((res= pam_auth_base(&param, &info)) != CR_OK) > + { > + PAM_DEBUG((stderr, "PAM: auth failed, sandbox closed.\n")); > + return -1; > + } > + > + if (info.authenticated_as[0]) > + { > + PAM_DEBUG((stderr, "PAM: send authenticated_as field.\n")); > + field= AP_AUTHENTICATED_AS; > + if (write(1, &field, 1) < 1 || > + write_string(1, (unsigned char *) info.authenticated_as, > + strlen(info.authenticated_as))) > + return -1; > + } > + > + PAM_DEBUG((stderr, "PAM: send OK result.\n")); > + field= AP_EOF; > + write(1, &field, 1); > + > + PAM_DEBUG((stderr, "PAM: sandbox closed.\n")); > + return 0; > +} Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

2 1

Re: [Maria-developers] 980a5c31880: MDEV-14572: Assertion `! is_set()' failed in Diagnostics_area::set_eof_status upon EXPLAIN UPDATE in PS
by Sergei Golubchik 19 Jun '18

19 Jun '18

Hi, Oleksandr! On Jan 28, Oleksandr Byelkin wrote: > Hi, Sergei! > > Flag in SELECT_LEX options dropped at the end of execution (in our case > prepare). All commands where EXPLAIN is possible restore the flag in main > SELECT_LEX (in mysql_explain_union()) except UPDATE (because it do > not use mysql_explain_union() > for explain). Without the flag it first try to send OK (as part of update) > then EOF as part of EXPLAIN. Ok. 1. Why does it not use mysql_explain_union() ? 2. Why EXPLAIN flag is need to be removed and restored anyway? 3. Please append the following to the commit comment: " Without it, mysql_update() didn't know that there will be EXPLAIN result set and was sending OK at the end of the update, which conflicted with the EOF sent later by EXPLAIN. " > On Sun, Jan 28, 2018 at 2:45 PM, Sergei Golubchik <serg(a)mariadb.org> wrote: > > > Hi, Oleksandr! > > > > On Dec 22, Oleksandr Byelkin wrote: > > > revision-id: 980a5c318803d6aa7355e019869597976dcf19bf (mariadb-10.0.33-37-g980a5c31880) > > > parent(s): 042f763268c0f209e7c12e0a6a72bb5d204dfe29 > > > author: Oleksandr Byelkin > > > committer: Oleksandr Byelkin > > > timestamp: 2017-12-22 14:00:10 +0100 > > > message: > > > > > > MDEV-14572: Assertion `! is_set()' failed in Diagnostics_area::set_eof_status upon EXPLAIN UPDATE in PS > > > > > > Restore EXPAIN flag in SELECT_LEX before execution multi-update by flag in LEX > > > (the same but in other way made before INSERT/DELETE/SELECT) > > > > Please explain what was the reason for the bug, why the status was set > > twice, and how your change fixes it. Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

2 3

Re: [Maria-developers] 398ea6a: MDEV-11917 enum/set command-line options aren't respecting max-* settings.
by Sergei Golubchik 18 Jun '18

18 Jun '18

Hi, Alexey! On Mar 13, Alexey Botchkov wrote: > revision-id: 398ea6ab012938d08126c8d0e0291000d06f6cd6 (mariadb-10.2.13-43-g398ea6a) > parent(s): 9d95b8665a3d87ab857e77e220effc64454eb881 > committer: Alexey Botchkov > timestamp: 2018-03-13 17:13:27 +0400 > message: > > MDEV-11917 enum/set command-line options aren't respecting max-* settings. > > Code added to make maximum- working for Sys_var_enum > and Sys_var_set. > > diff --git a/sql/set_var.cc b/sql/set_var.cc > index e96e636..32e52a9 100644 > --- a/sql/set_var.cc > +++ b/sql/set_var.cc > @@ -452,6 +452,22 @@ void sys_var::do_deprecated_warning(THD *thd) > > @retval true on error, false otherwise (warning or ok) > */ > + > + > +bool throw_bounds_warning(THD *thd, const char *name,const char *v) > +{ > + if (thd->variables.sql_mode & MODE_STRICT_ALL_TABLES) > + { > + my_error(ER_WRONG_VALUE_FOR_VAR, MYF(0), name, v); This is rather illogical, setting a variable doesn't touch any tables, so MODE_STRICT_ALL_TABLES should not affect it. Also, the error message could be improved. But it's how it was, and it's not subject of this bugfix, I agree. > + return true; > + } > + push_warning_printf(thd, Sql_condition::WARN_LEVEL_WARN, > + ER_TRUNCATED_WRONG_VALUE, > + ER_THD(thd, ER_TRUNCATED_WRONG_VALUE), name, v); > + return false; > +} > + > + > bool throw_bounds_warning(THD *thd, const char *name, > bool fixed, bool is_unsigned, longlong v) > { > @@ -464,14 +480,7 @@ bool throw_bounds_warning(THD *thd, const char *name, > else > llstr(v, buf); > > - if (thd->variables.sql_mode & MODE_STRICT_ALL_TABLES) > - { > - my_error(ER_WRONG_VALUE_FOR_VAR, MYF(0), name, buf); > - return true; > - } > - push_warning_printf(thd, Sql_condition::WARN_LEVEL_WARN, > - ER_TRUNCATED_WRONG_VALUE, > - ER_THD(thd, ER_TRUNCATED_WRONG_VALUE), name, buf); ah, good. only one place to change when we fix the above. thanks. > + return throw_bounds_warning(thd, name, buf); > } > return false; > } > diff --git a/sql/sys_vars.ic b/sql/sys_vars.ic > index bc913b1..adcda5d 100644 > --- a/sql/sys_vars.ic > +++ b/sql/sys_vars.ic all ok, looks good. Just one comment. Please make max_var_ptr() logic to be in one place only. If C++ cannot do it, then, in the worst case, make it a macro, like #define define_max_var_ptr(TYPE) \ TYPE *max_var_ptr() { return scope() == SESSION ? (TYPE*)(((uchar*)&max_system_variables) + offset) : 0; } after that ok to push, thanks! Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] f3dda8e: MDEV-15357 Wrong result with join_cache_level=4, BNLH join
by Sergei Golubchik 15 Jun '18

15 Jun '18

Hi, Igor! On Apr 30, Igor Babaev wrote: > revision-id: f3dda8e2002dc2ad630c07bda0d8c00be7269907 (mariadb-10.2.14-72-gf3dda8e) > parent(s): b4c5e4a717e3ce2c2d434106cc74417fe9a1d3dc > author: Igor Babaev > committer: Igor Babaev > timestamp: 2018-04-30 21:26:38 -0700 > message: > > MDEV-15357 Wrong result with join_cache_level=4, BNLH join > > This bug was introduced by the architectural changes of the patch > for MDEV-11640. The patch moved initialization of join caches after > the call of make_join_readinfo(). As any failure to initialize > a join cache caused denial of its usage the execution code for > the query had to be revised. This revision required rolling back many > actions taken by make_join_readinfo(). It was not done in the patch. > As a result if a denial of join cache happened some join conditions > could be lost. This was exactly the cause of wrong results in the > bug's reported test case. > > Thus the introduced architectural change is not valid and it would be > better to roll it back. At the same time two new methods > adjust_read_set_for_vcol_keyread() and get_covering_index_for_scan() > were added to the class JOIN_TAB to resolve the problems of MDEV-11640. > > diff --git a/sql/sql_join_cache.cc b/sql/sql_join_cache.cc > index 3612cb6..242cb91 100644 > --- a/sql/sql_join_cache.cc > +++ b/sql/sql_join_cache.cc > @@ -226,6 +226,16 @@ void JOIN_CACHE::calc_record_fields() > flag_fields+= MY_TEST(tab->table->maybe_null); > fields+= tab->used_fields; > blobs+= tab->used_blobs; > + if (tab->type == JT_ALL || tab->type == JT_HASH) > + { > + uint idx= tab->get_covering_index_for_scan(); as you moved JOIN_CACHE initialization back, I suspect that this calc_record_fields() now happens too early for get_covering_index_for_scan() to be useful. And why only JT_ALL and JT_HASH? > + if (idx != MAX_KEY) > + { > + fields-= bitmap_bits_set(tab->table->read_set); > + tab->adjust_read_set_for_vcol_keyread(idx); > + fields+= bitmap_bits_set(tab->table->read_set); > + } > + } > } > if ((with_match_flag= join_tab->use_match_flag())) > flag_fields++; > diff --git a/sql/sql_select.cc b/sql/sql_select.cc > index 30ef26f..e5b75ec 100644 > --- a/sql/sql_select.cc > +++ b/sql/sql_select.cc > @@ -11404,6 +11375,61 @@ void JOIN_TAB::remove_redundant_bnl_scan_conds() > } > > > +uint JOIN_TAB::get_covering_index_for_scan() > +{ > + int idx= MAX_KEY; > + if (table->no_keyread) > + return idx; > + if (select && select->quick && > + select->quick->index != MAX_KEY && //not index_merge > + table->covering_keys.is_set(select->quick->index)) > + idx= select->quick->index; > + else if (!table->covering_keys.is_clear_all() && > + !(select && select->quick)) > + { // Only read index tree > + if (loosescan_match_tab) > + idx= loosescan_key; > + else > + { > +#ifdef BAD_OPTIMIZATION > + /* > + It has turned out that the below change, while speeding things > + up for disk-bound loads, slows them down for cases when the data > + is in disk cache (see BUG#35850): > + See bug #26447: "Using the clustered index for a table scan > + is always faster than using a secondary index". > + */ > + if (table->s->primary_key != MAX_KEY && > + table->file->primary_key_is_clustered()) > + idx= table->s->primary_key; > + else > +#endif > + idx= find_shortest_key(table, & table->covering_keys); > + } > + } > + return idx; > +} > + > + > +void JOIN_TAB::adjust_read_set_for_vcol_keyread(uint keyread_idx) > +{ > + if (!table->vcol_set || bitmap_is_clear_all(table->vcol_set)) > + return; > + > + MY_BITMAP *keyread_set= &table->cond_set; // is free for use at call normally one uses tmp_set for this, not hijacks cond_set > + bitmap_clear_all(keyread_set); > + table->mark_columns_used_by_index(keyread_idx, keyread_set); > + bitmap_intersect(keyread_set, table->read_set); > + bitmap_intersect(keyread_set, table->vcol_set); this looks wrong, you probably should do keyread_set INTERSECT (read_set UNION vcol_set) > + if (!bitmap_is_clear_all(keyread_set)) > + { > + bitmap_clear_all(keyread_set); > + table->mark_columns_used_by_index(keyread_idx, keyread_set); > + bitmap_intersect(table->read_set, keyread_set); > + } I don't understand that. 1. What if bitmap_is_clear_all(keyread_set) - what does it mean? 2. Why do you recalculate keyread_set, why not simply copy keyread_set into read_set ? > +} > + > + > /* > Plan refinement stage: do various setup things for the executor > > @@ -11526,7 +11552,10 @@ make_join_readinfo(JOIN *join, ulonglong options, uint no_jbuf_after) > tab->read_first_record= tab->type == JT_SYSTEM ? join_read_system > : join_read_const; > if (table->covering_keys.is_set(tab->ref.key) && !table->no_keyread) > + { > + tab->adjust_read_set_for_vcol_keyread(tab->ref.key); > table->file->ha_start_keyread(tab->ref.key); better combine both into inline void JOIN_TAB::start_keyread() { adjust_read_set_for_vcol_keyread(ref.key); tab->table->file->ha_start_keyread(ref.key); } or even JOIN_TAB::start_keyread_if_needed() putting the if() inside too > + } > else if ((!jcl || jcl > 4) && !tab->ref.is_access_triggered()) > push_index_cond(tab, tab->ref.key); > break; > @@ -11606,35 +11641,19 @@ make_join_readinfo(JOIN *join, ulonglong options, uint no_jbuf_after) > } > } > } > - if (!table->no_keyread) > + uint idx= tab->get_covering_index_for_scan(); > + if (idx != MAX_KEY) > { > if (tab->select && tab->select->quick && > tab->select->quick->index != MAX_KEY && //not index_merge > table->covering_keys.is_set(tab->select->quick->index)) > - table->file->ha_start_keyread(tab->select->quick->index); > + table->file->ha_start_keyread(idx); why do you need an if() here - you already checked that in get_covering_index_for_scan(). I generally don't like this code at all. In JOIN_CACHE you simply use tab->get_covering_index_for_scan(), but here you use it *plus* a bunch of if()s afterwards. It surely creates an impression that JOIN_CACHE and make_join_readinfo may have a different idea about whether keyread is used. Ideally here should be just if (idx != MAX_KEY) tab->start_keyread(idx); and even then it would be kinda questionable (because preconditions might change between two get_covering_index_for_scan() calls). > else if (!table->covering_keys.is_clear_all() && > !(tab->select && tab->select->quick)) > - { // Only read index tree > - if (tab->loosescan_match_tab) > - tab->index= tab->loosescan_key; > - else > - { > -#ifdef BAD_OPTIMIZATION > - /* > - It has turned out that the below change, while speeding things > - up for disk-bound loads, slows them down for cases when the data > - is in disk cache (see BUG#35850): > - See bug #26447: "Using the clustered index for a table scan > - is always faster than using a secondary index". > - */ > - if (table->s->primary_key != MAX_KEY && > - table->file->primary_key_is_clustered()) > - tab->index= table->s->primary_key; > - else > -#endif > - tab->index=find_shortest_key(table, & table->covering_keys); > - } > + { // Only read index tree > + tab->index= idx; > tab->read_first_record= join_read_first; > + table->file->ha_start_keyread(tab->index); > /* Read with index_first / index_next */ > tab->type= tab->type == JT_ALL ? JT_NEXT : JT_HASH_NEXT; > } Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] MDEV-6111: Optimizer trace: let's get rid of "optimizer_trace_enabled"
by Sergey Petrunia 15 Jun '18

15 Jun '18

Hi Zhzhzoo, I see that the current patch uses @@optimizer_trace_enabled variable to control it. Meanwhile, in MySQL the name of the variable is "optimizer_trace". It is a flag set: mysql> select @@optimizer_trace; +-------------------------+ | @@optimizer_trace | +-------------------------+ | enabled=on,one_line=off | +-------------------------+ 1 row in set (0.00 sec) but one can set it to 1 or 0 to just set enabled on/off: mysql> set @@optimizer_trace=1; Query OK, 0 rows affected (0.00 sec) mysql> set @@optimizer_trace=0; Query OK, 0 rows affected (0.00 sec) This obviously carries some legacy but I think we should have it the same way, in order to be compatible with MySQL where possible. Please use the same name/values as MySQL does. BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] MDEV-6111: Optimizer trace: on the first output
by Sergey Petrunia 15 Jun '18

15 Jun '18

On Tue, Jun 12, 2018 at 07:40:08AM +0800, zhzhzoo zhang wrote: > I've done > condition processing and some range analysis. > and for the test input ( > https://dev.mysql.com/doc/internals/en/tracing-example.html) It now > outputs something like > (not very readable from the email so I've made two pastes, from the current tree and from MySQL 8): https://gist.github.com/spetrunia/cb4a1a382a9e81a60491d524dbfc08f6 https://gist.github.com/spetrunia/62347cc3bf26e003909a32b378b6684f > ```{ > "join_optimization": { > "select #": 1, > "steps": { > "inner": { > "select #": 1, Two 'select #' :1 seem to be redundant. There are also two nested "steps" elements which seem redundant as well. > "steps": [ > { > "condition_processing": { > "condition": "WHERE", > "original_condition": "`test`.`alias1`.`pk` <> 0 and > `test`.`alias2`.`pk` = `test`.`alias1`.`col_int_key`", > "steps": [ > { > "transformation": "equality_propagation", > "subselect_evaluation": [], > "resulting_condition": "`test`.`alias1`.`pk` <> 0 and > multiple equal(`test`.`alias2`.`pk`, `test`.`alias1`.`col_int_key`)???" What are the "???" at the end? Please get them removed. also, if you check EXPLAIN FORMAT=JSON output, it prints conditions without overquoting, like so: "attached_condition": "alias1.pk <> 0 and alias1.col_int_key is not null" This is done by using QT_EXPLAIN parameter: item->print(&str, QT_EXPLAIN); Let's print them this way here as well. > }, > { > "transformation": "constant_propagation", > "subselect_evaluation": [], > "resulting_condition": "`test`.`alias1`.`pk` <> 0 and > multiple equal(`test`.`alias2`.`pk`, `test`.`alias1`.`col_int_key`)???" > }, BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] 46e0c8a9219: MDEV-11071: Assertion `thd->transaction.stmt.is_empty()' failed in Locked_tables_list::unlock_locked_table
by Sergei Golubchik 14 Jun '18

14 Jun '18

Hi, Oleksandr! Looks good, but please see a few comments below. On May 08, Oleksandr Byelkin wrote: > revision-id: 46e0c8a921978e7b95adf3f17f3c85b18d9f9ef6 (mariadb-10.2.14-78-g46e0c8a9219) > parent(s): 9bcd0f5fea8ca26742b10d37b95a966c69909ff1 > author: Oleksandr Byelkin > committer: Oleksandr Byelkin > timestamp: 2018-05-08 15:26:26 +0200 > message: > > MDEV-11071: Assertion `thd->transaction.stmt.is_empty()' failed in Locked_tables_list::unlock_locked_table > > fix_length_and_dec now return result (error/OK) > diff --git a/sql/item.h b/sql/item.h > index 8921ee76f6a..e9713730a1c 100644 > --- a/sql/item.h > +++ b/sql/item.h > @@ -4210,7 +4210,7 @@ class Item_func_or_sum: public Item_result_field, > also to make printing of items inherited from Item_sum uniform. > */ > virtual const char *func_name() const= 0; > - virtual void fix_length_and_dec()= 0; > + virtual bool fix_length_and_dec()= 0; 1. I wonder, if you change fix_length_and_dec to void in one of the derived classes, will you get a compiler warning for that? 2. may be also __attribute__ ((warn_unused_result)) here? I don't know how it'll work on virtual methods, may be it won't. both questions are for the use case "what if we merge some fix_length_and_dec related changes from 10.1, will the compiler tell us to update them?" > bool const_item() const { return const_item_cache; } > table_map used_tables() const { return used_tables_cache; } > Item* build_clone(THD *thd, MEM_ROOT *mem_root); > diff --git a/sql/item_cmpfunc.cc b/sql/item_cmpfunc.cc > index 44cc4f3cae9..46cf412e229 100644 > --- a/sql/item_cmpfunc.cc > +++ b/sql/item_cmpfunc.cc > @@ -535,8 +535,9 @@ void Item_bool_rowready_func2::fix_length_and_dec() > we have to check for out of memory conditions here > */ > if (!args[0] || !args[1]) > - return; > - setup_args_and_comparator(current_thd, &cmp); > + return FALSE; > + bool res= setup_args_and_comparator(current_thd, &cmp); > + return res; not that it matters much (no need to change), but why didn't you return setup_args_and_comparator(current_thd, &cmp); directly? > } > > > diff --git a/sql/item_cmpfunc.h b/sql/item_cmpfunc.h > index de1b27cff1a..5efb98bc81d 100644 > --- a/sql/item_cmpfunc.h > +++ b/sql/item_cmpfunc.h > @@ -911,10 +911,12 @@ class Item_func_strcmp :public Item_int_func > longlong val_int(); > uint decimal_precision() const { return 1; } > const char *func_name() const { return "strcmp"; } > - void fix_length_and_dec() > + bool fix_length_and_dec() > { > - agg_arg_charsets_for_comparison(cmp_collation, args, 2); > + if (agg_arg_charsets_for_comparison(cmp_collation, args, 2)) > + return TRUE; > fix_char_length(2); // returns "1" or "0" or "-1" this comment is kinda weird. remove? > + return FALSE; > } > Item *get_copy(THD *thd, MEM_ROOT *mem_root) > { return get_item_copy<Item_func_strcmp>(thd, mem_root, this); } > @@ -1987,10 +1997,10 @@ class Item_func_like :public Item_bool_func2 > const char *func_name() const { return "like"; } > enum precedence precedence() const { return CMP_PRECEDENCE; } > bool fix_fields(THD *thd, Item **ref); > - void fix_length_and_dec() > + bool fix_length_and_dec() > { > max_length= 1; > - agg_arg_charsets_for_comparison(cmp_collation, args, 2); should this one also be __attribute__ ((warn_unused_result)) ? > + return agg_arg_charsets_for_comparison(cmp_collation, args, 2); > } > void cleanup(); > > diff --git a/sql/item_subselect.cc b/sql/item_subselect.cc > index 4214980c36c..ca49e06a9e7 100644 > --- a/sql/item_subselect.cc > +++ b/sql/item_subselect.cc > @@ -904,9 +905,10 @@ Item::Type Item_subselect::type() const > } > > > -void Item_subselect::fix_length_and_dec() > +bool Item_subselect::fix_length_and_dec() > { > engine->fix_length_and_dec(0); what about subselect_engine::fix_length_and_dec() ? should return bool too, I guess, right? > + return FALSE; > } > > > diff --git a/sql/sql_table.cc b/sql/sql_table.cc > index 9e7973b745c..f3cb85f01d3 100644 > --- a/sql/sql_table.cc > +++ b/sql/sql_table.cc > @@ -4894,7 +4894,13 @@ int create_table_impl(THD *thd, > file= mysql_create_frm_image(thd, orig_db, orig_table_name, create_info, > alter_info, create_table_mode, key_info, > key_count, frm); > - if (!file) > + /* > + We have to check thd->is_error() here because it can be set by > + Item::val* for example, and before it will be cought accidentally by > + Item_func::fix_fields() of the next call. Now we removed the check > + from Item_func::fix_fields() > + */ this is a very confusing comment, don't write in comments that "it used to be this way, but now we've changed it and it is that way" and I still don't understand why do you need to check for thd->is_error() here > + if (!file || thd->is_error()) > goto err; > if (rea_create_table(thd, frm, path, db, table_name, create_info, > file, frm_only)) > @@ -7377,7 +7383,8 @@ static bool mysql_inplace_alter_table(THD *thd, > */ > if (mysql_rename_table(db_type, alter_ctx->new_db, alter_ctx->tmp_name, > alter_ctx->db, alter_ctx->alias, > - FN_FROM_IS_TMP | NO_HA_TABLE)) > + FN_FROM_IS_TMP | NO_HA_TABLE) || > + thd->is_error()) and here > { > // Since changes were done in-place, we can't revert them. > (void) quick_rm_table(thd, db_type, Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

2 6

Re: [Maria-developers] 0918ae42a83: MDEV-14014 Multi-Slave Replication Fail: bogus data in log event
by Sergei Golubchik 13 Jun '18

13 Jun '18

Hi, Andrei! On Jun 13, andrei.elkin(a)pp.inet.fi wrote: > revision-id: 0918ae42a8394b6faf28f67b70550894851be3bb (mariadb-10.1.33-36-g0918ae42a83) > parent(s): 26be5072429082e634b8fc102609370975443439 > author: Andrei Elkin > committer: Andrei Elkin > timestamp: 2018-06-13 16:27:00 +0300 > message: > > MDEV-14014 Multi-Slave Replication Fail: bogus data in log event > > MDEV-7257 made a dump thread to read from binlog concurrently with > writers as long as the read bytes are below a water-mark > (MYSQL_BIN_LOG::binlog_end_pos). However it appeared to be possible a > dump thread reader reach out for bytes past the water mark through a > feature of IO_CACHE that fills in the internal buffer and while doing > so it could read what the reader is not supposed to see (the bytes > above MYSQL_BIN_LOG::binlog_end_pos). > > The issue is fixed with constraining the IO_CACHE buffer fill to respect > the watermark. > An added test simulates potentially unconstrained buffer fill and an > assert guards this is not the case anymore. Looks good, thanks! Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] unbundle RocksDB
by Michal Schorm 13 Jun '18

13 Jun '18

Hello, Since we have a rocksdb package ready in Fedora, I tried to unbundle it, using the system package instead. I wasn't successful though. I haven't found any CMake argument which seemed related and from the build log it seems, the code tries to link against the static libraries in any case. Before I'll start to dig further, I'm stopping by on this list, making sure it is possible. - is it currently supported? - have I overlooked any related CMake argument? Links: MariaDB in Fedora: https://src.fedoraproject.org/rpms/mariadb/tree/master -- Michal Schorm Associate Software Engineer Core Services - Databases Team Red Hat

3 3

Re: [Maria-developers] d9913834ceb: MDEV-14014 Multi-Slave Replication Fail: bogus data in log event
by Sergei Golubchik 13 Jun '18

13 Jun '18

Hi, andrei.elkin! On May 30, andrei.elkin(a)pp.inet.fi wrote: > revision-id: d9913834cebe4ed8494b3d76762cd882788a8fd5 (mariadb-10.1.33-28-gd9913834ceb) > parent(s): c1698e8dc50f0c342c1a6886426ba1d43396cd1e > author: Andrei Elkin > committer: Andrei Elkin > timestamp: 2018-05-30 20:06:44 +0300 > message: > > MDEV-14014 Multi-Slave Replication Fail: bogus data in log event > > MDEV-7257 made a dump thread to read from binlog concurrently with > writers as long as the read bytes are below a water-mark > (MYSQL_BIN_LOG::binlog_end_pos). However it appeared to be possible a > dump thread reader reach out for bytes past the water mark through a > feature of IO_CACHE that fills in the internal buffer and while doing > so it could read what the reader is not supposed to see (the bytes > above MYSQL_BIN_LOG::binlog_end_pos). > > The issue is fixed with constraining the IO_CACHE buffer fill to respect > the watermark. > An added test simulates potentially unconstrained buffer fill and an > assert guards this is not the case anymore. What about a much simpler one-liner fix: --- a/sql/sql_repl.cc +++ b/sql/sql_repl.cc @@ -2549,6 +2549,7 @@ static int send_events(binlog_send_info *info, IO_CACHE* return 1; info->last_pos= linfo->pos; + log->end_of_file= end_pos; error= Log_event::read_log_event(log, packet, info->fdev, opt_master_verify_checksum ? info->current_checksum_alg : BINLOG_CHECKSUM_ALG_OFF); It doesn't change IO_CACHE structure and doesn't add any overhead to the IO_CACHE. Did you already test that your commit actually fixes the issue? Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

2 3

Re: [Maria-developers] 20f3479: MDEV-15890 Strange error message if you try to FLUSH TABLES <view> after LOCK TABLES <view>.
by Sergei Golubchik 12 Jun '18

12 Jun '18

Hi, Alexey! On Jun 11, Alexey Botchkov wrote: > revision-id: 20f347974f6a9b4e2c65a5ca79aefb8e118ba6cd (mariadb-10.1.33-33-g20f3479) > parent(s): 3627dd7f6a1cf5bd7151ff23290d64cb6dffea90 > committer: Alexey Botchkov > timestamp: 2018-06-11 14:58:46 +0400 > message: > > MDEV-15890 Strange error message if you try to FLUSH TABLES <view> after LOCK TABLES <view>. > > The reload_acl_and_cache() now looks into the VIEW definition > to check the involved tables if the locked_tables_mode. That's a pretty neat solution :) But it seems that - contrary to what I expected - that MariaDB actually distinguishes between locking a view a locking all its tables. Consider this test case: ---- create table t1 (a int); create view v1 as select * from t1; lock table v1 write; insert v1 values (1); select * from t1; unlock tables; drop view v1; drop table t1; ---- this works and inserts 1 just fine. If you change the lock table statement to lock t1, it'll with Table 'v1' was not locked with LOCK TABLES It'd be great if FLUSH TABLES will exhibit the same behavior, it needs to know if the view is locked, not only if all underlying tables are locked. Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] [Maria-discuss] MariaDB crashes because of "long semaphore wait" after migrating from 10.1 to 10.3
by Vicențiu Ciorbaru 12 Jun '18

12 Jun '18

Hi! I have cc'ed the maria-developers mailing list as this is potentially a dev issue and also the devs responsible for Windows and InnoDB development. There might be some people on vacation right now, but this should get looked into in a week or two at most. Let's find a solution for your problem. :) Vicentiu On Sun, 10 Jun 2018, 19:11 TD, <tdcf(a)gmx.net> wrote: > Hi, > > I have a Windows server that has been running MariaDB 10.1 successfully > for over a year. The server remains mostly idle for long times with some > read access, but occasionally there are transactions that add data (about > 500k rows per commit). There can be up to 10 such transactions (one per > database) at the same time and during those times the server is under quite > some load (the code processing the data resides on the same server as the > database). We're using an 8 core AMD Ryzen, 32 GB RAM and a Samsung SSD. > > Like I said, this worked fine with MariaDB 10.1 all the time. > > We're currently in the process of setting up a new server (specs similar > to the ones above) and because MariaDB 10.3 has been released recently, I > was curious and installed it. The database now constantly shuts down > because of "[FATAL] InnoDB: Semaphore wait has lasted > 600 seconds. We > intentionally crash the server because it appears to be hung." whenever > there is one of the load situations described above. If I install MariaDB > 10.1 on the new server, it runs as fine as the old one. The crashes also > happen when I start with a clean database (no data migrated from 10.1 > databases). > > I'm not a huge MariaDB config pro, as it's only a tool to store the data, > it just needs to run with reasonable performance and has always worked > until now. As far as I know and can judge from comments in the file, the > my.ini in use is (mostly?) based on my-innodb-heavy-4G.cnf, but coworkers > might have made modifications. I've attached the contents below. > > I don't know how I can debug this issue further. Windows event viewer > contains several log warnings with the text "InnoDB: A long semaphore > wait:" but no further information why this wait happens. I couldn't find > any other error logs. I guess it is some configuration error (I read about > MariaDB switching from XtraDB to InnoDB, so I guess that the issue might be > related). > > Migrating to 10.3 is not an absolute must, but it would be nice to be able > to fix the problem. > > Any help would be greatly appreciated and if you need more information, > I'll gladly try to provide it. > > Thanks, > Tom. > > my.ini: > [mysqld] > max_connections = 100 > max_allowed_packet = 16M > binlog_cache_size = 1M > max_heap_table_size = 64M > read_buffer_size = 2M > read_rnd_buffer_size = 16M > sort_buffer_size = 8M > join_buffer_size = 8M > thread_cache_size = 10 > query_cache_size = 64M > query_cache_limit = 2M > thread_stack = 240K > tmp_table_size = 64M > slow_query_log > expire_logs_days = 14 > log-bin = mysql-bin > character-set-server=utf8 > collation-server=utf8_general_ci > > #*** MyISAM Specific options > key_buffer_size = 32M > bulk_insert_buffer_size = 64M > myisam_sort_buffer_size = 128M > myisam_max_sort_file_size = 10G > myisam_repair_threads = 1 > myisam_recover > > # *** INNODB Specific options *** > #innodb_fatal_semaphore_wait_threshold = 6000 #tried that, didn't help > #innodb_adaptive_hash_index = 0 #tried that (suggested in some stack > overflow post, didn't help) > innodb_buffer_pool_size = 2G > innodb_data_file_path = ibdata1:10M:autoextend > innodb_write_io_threads = 8 > innodb_read_io_threads = 8 > innodb_thread_concurrency = 16 > innodb_log_buffer_size = 8M > innodb_log_file_size = 256M > innodb_log_files_in_group = 3 > innodb_max_dirty_pages_pct = 10 > innodb_io_capacity = 2000 > innodb_lock_wait_timeout = 120 > > [mysql] > no-auto-rehash > > [myisamchk] > key_buffer_size = 512M > sort_buffer_size = 512M > read_buffer = 8M > write_buffer = 8M > > [mysqld_safe] > open-files-limit = 8192 > _______________________________________________ > Mailing list: https://launchpad.net/~maria-discuss > Post to : maria-discuss(a)lists.launchpad.net > Unsubscribe : https://launchpad.net/~maria-discuss > More help : https://help.launchpad.net/ListHelp >

3 2

[Maria-developers] MariaDB Developers Unconference in a few weeks!
by Otto Kekäläinen 08 Jun '18

08 Jun '18

Hello everybody! The next developers unconference will be in a few weeks: https://mariadb.org/2018-2-developers-unconference-in-finland/ There are still open slots so we encourage you to sign up now. https://www.meetup.com/MariaDB-Developers/events/249024232/ We have plenty of break-out rooms this time and there is space, so we also encourage you to suggest sessions/discussions you would like to have to ian(a)mariadb.org. Around mid-summer is a great time to visit Finland. The nights that don't get dark are worth experiencing! Getting to the venue in Tampere., Finlnd is also very easy. https://mariadb.org/travel-tips-to-tampere-finland/#why-visit Hope to see you there! - Otto

1 0

Re: [Maria-developers] [Commits] 56d339111a4: MDEV-16374: Filtered shows 0 for materilization scan for a semi join, which makes optimizer always picks
by Sergey Petrunia 05 Jun '18

05 Jun '18

Hi Varun, On Mon, Jun 04, 2018 at 12:34:49PM +0530, Varun wrote: > revision-id: 56d339111a4e5c868ffa3b1f2538c5f101f67767 (mariadb-10.0.30-366-g56d339111a4) > parent(s): a61724a3ca187f6eb44fc67948acb76a94efa783 > author: Varun Gupta > committer: Varun Gupta > timestamp: 2018-06-04 12:26:56 +0530 > message: > > MDEV-16374: Filtered shows 0 for materilization scan for a semi join, which makes optimizer always picks > materialization scan over materialization lookup > > For non-mergeable semi-joins we don't store the estimates of the IN subquery in table->file->stats.records. > In the function TABLE_LIST::fetch_number_of_rows, we store the number of rows in the tables > (estimates in case of derived table/views). > Currently we don't store the estimates for non-mergeable semi-joins, which leads to a problem of selecting > materialization scan over materialization lookup. > Fixed this by storing these estimated appropriately > * This is a change in the optimizer. Should we really do it in 10.0? In my opinion, this should go into 10.3 or 10.4. Let's discuss this, and I think Igor may have something to say, too. * The patch updates selectivity.test but not selectivity_innodb.result (which includes selectivity.test). * Please see my last comments at https://jira.mariadb.org/browse/MDEV-16374 . Here is an alternative patch (it passes the testsuite, and produces the same result for selectivity* tes): diff --git a/sql/sql_select.cc b/sql/sql_select.cc index 1285835..884997c 100644 --- a/sql/sql_select.cc +++ b/sql/sql_select.cc @@ -6602,7 +6602,7 @@ double matching_candidates_in_table(JOIN_TAB *s, bool with_found_constraint, { TABLE *table= s->table; double sel= table->cond_selectivity; - double table_records= (double)table->stat_records(); + double table_records= s->records; // psergey-10 dbl_records= table_records * sel; return dbl_records; } What do you think about it? I would like to get Igor's opinion also because I'm not certain which member variable should be changed here. (Why does derived table modify table->stats.records while JTBM semi-joins modify JOIN_TAB::records? These seem like two solutions for the same problem?) BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] Proposal for Negative Grants Project
by Rutuja Surve 04 Jun '18

04 Jun '18

Hello, Please find the proposal for the Negative Grants project attached along with this e-mail. It would be great to hear your opinions and suggestions on the approach proposed here, before delving into its implementation. Some context about the project: *The current privilege system allows access in this manner:GRANT SELECT ON some_database.* TO a_user@%;Any revoke would revoke access from the entire database. We want to support a revoke that would disable select on a specific table, like:REVOKE SELECT ON some_database.secret_table FROM a_user@%;Reference:[1] https://jira.mariadb.org/browse/MDEV-14443 <https://jira.mariadb.org/browse/MDEV-14443>[2] https://mariadb.com/kb/en/library/google-summer-of-code-2018/ <https://mariadb.com/kb/en/library/google-summer-of-code-2018/>Thanks,Rutuja*

3 6

[Maria-developers] Conservative parallel slave is "too optimistic" for certain engines
by Sergey Petrunia 01 Jun '18

01 Jun '18

Hello, This comes from MDEV-16242. == Symptoms == When one runs a parallel slave (mode=conservative) and replicates DML for MyRocks table without a Primary Key, replication may stop with a ER_KEY_NOT_FOUND error. This may happen even if the queries were run on the master sequentially. == A detail about conservative replication == Suppose the master runs these two statements, sequentially: INSERT INTO t1 VALUES (5),(6),(7); DELETE FROM t1; Parallel slave may schedule the INSERT to Thread1, and the DELETE to Thread2. In conservative parallel replication, the DELETE "will wait" to be executed after the INSERT. One may expect that "will wait" here means that DELETE execution does not start until the INSERT has committed. But it's more subtle than that: actually, DELETE execution will start as soon as the INSERT is ready to do the prepare/commit steps. I assume this was done to increase the parallelism: initial phases of DELETE can be ran in parallel with commit/prepare steps of the INSERT. This is safe to do: 1. The INSERT has acquired locks for all rows it is about to touch. DELETE will not be able to prevent INSERT from committing. 2. The DELETE starts its execution on a database that doesn't include the results of the INSERT (the INSERT has not committed yet!). But this is fine, because the DELETE locks the rows it is about to modify. If it attempts to access a row that the INSERT is about to insert, it will block on a lock until the INSERT finishes. == Applying this to MyRocks == The critical part is #2. It works if the DELETE command will acquire row locks for rows it about to DELETE. This is normally true: If the storage engine supports gap locks, attempting to read a locked gap will cause DELETE to wait for INSERT If the storage engine doesn't support gap locks, it will use Row-Based-Replication. RBR's Delete_rows_event includes Primary Key value of the row that should be deleted. Doing a point lookup on PK will cause the DELETE to block on the row lock. The remaining case is - storage engine that doesn't support gap locks - Row-based-replication of table without primary key In this case, finding the row to delete is done by scanning the table until we find the row where all columns match. The scan does not see the rows that the INSERT is about to INSERT, and DELETE stops with an error, "we could not find the row to delete". == Possible solutions == 1. Implement gap locking in MyRocks. This will take some time. 2. Change parallel slave to wait *for commit*. This should only be done if tables that are updated do not support Gap Locking. This is hard, it will require making risky changes on the SQL layer. 3. Disallow parallel slave execution for tables without Gap Locking and without primary key. (Looks most promising but I am not sure if it is sufficient). BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

3 2

[Maria-developers] Test failure overview for Debian/Ubuntu on 10.3 master branch (June 1st 2018 status report)
by Otto Kekäläinen 01 Jun '18

01 Jun '18

Hello! I reported here about the test failure status in early April and here is a follow-up. My old emails can be found here: https://lists.launchpad.net/maria-developers/msg11171.html https://lists.launchpad.net/maria-developers/msg11170.html If one looks at the picture from April and compares to the status today, the situation is a bit sad as we have even less green than before: https://lists.launchpad.net/maria-developers/pngcpxTB4wjTW.png Current issues we have in addition to the usual mtr test failures are: - On multiple bionic builds the upgrade fails to run with no apparent reason. The builder script does not seem to find any previous installation of anything, so that probably is missing from the images: + dpkg -l + grep -iE 'maria|mysql|galera' + true Thus the build fails. Example log: http://buildbot.askmonty.org/buildbot/builders/kvm-deb-bionic-amd64/bui lds/196/steps/upgrade2/logs/stdio - On bionic and xenial aarch64 mysqld fails to start. In syslog we can see May 28 16:11:25 ubuntu-xenial-aarch64 mysqld: 2018-05-28 16:11:25 0 [ERROR] Incorrect definition of table mysql.event: expected column 'sql_mode' at position 14 to have type set('REAL_AS_FLOAT','PIPES_AS_CONCAT','ANSI_QUOTES','IGNORE_SPACE','IGN ORE_BAD_TABLE_OPTIONS','ONLY_FULL_GROUP_BY','NO_UNSIGNED_SUBTRACTION',' NO_DIR_IN_CREATE','POSTGRESQL','ORACLE','MSSQL','DB2','MAXDB','NO_KEY_O PTIONS','NO_TABLE_OPTIONS','NO_FIELD_OPTIONS','MYSQL323','MYSQL40','ANS I','NO_AUTO_VALUE_ON_ZERO','NO_BACKSLASH_ESCAPES','STRICT_TRANS_TABLES' ,'STRICT_ALL_TABLES','NO_ZERO_IN_DATE','NO_ZERO_DATE','INVALID_DATES',' ERROR_FOR_DIVISION_BY_ZERO','TRADITIONAL','NO_AUTO_CREATE_USER','HIGH_N OT_PRECEDENCE','NO_ENGINE_SUBSTITUTION','PAD_CHAR_TO_FULL_LENGTH','EMPT Y_STRING_IS_NULL','SIMULTANEOUS_ASSIGNMENT'), found type set('REAL_AS_FLOAT','PIPES_AS_CONCAT','ANSI_QUOTES','IGNORE_SPACE','IGN ORE_BAD_TABLE_OPTIONS','ONLY_FULL_GROUP_BY','NO_UNSIGNED_SUBTRACTION',' NO_DIR_IN_CREATE','POSTGRESQL','ORACLE','MSSQL','DB2','MAXDB','NO_KEY_O PTIONS','NO_TABLE_OPTIONS','NO May 28 16:11:25 ubuntu-xenial-aarch64 mysqld: _FIELD_OPTIONS','MYSQL323','MYSQL40','ANSI','NO_AUTO_VALU May 28 16:11:25 ubuntu-xenial-aarch64 mysqld: 2018-05-28 16:11:25 0 [ERROR] mysqld: Event Scheduler: An error occurred when initializing system tables. Disabling the Event Scheduler. May 28 16:11:25 ubuntu-xenial-aarch64 mysqld: 2018-05-28 16:11:25 6 [ERROR] InnoDB: Column last_update in table `mysql`.`innodb_table_stats` is INT UNSIGNED NOT NULL but should be BINARY(4) NOT NULL (type mismatch). May 28 16:11:25 ubuntu-xenial-aarch64 mysqld: 2018-05-28 16:11:25 6 [ERROR] InnoDB: Fetch of persistent statistics requested for table `mysql`.`gtid_slave_pos` but the required system tables mysql.innodb_table_stats and mysql.innodb_index_stats are not present or have unexpected structure. Using transient stats instead. http://buildbot.askmonty.org/buildbot/builders/kvm-deb-xenial-aarch64/b uilds/277/steps/upgrade1/logs/syslog - stretch amd64 fails to compile due to librocksdblib.a(compaction_job.cc.o): In function `rocksdb::ZSTD_TrainDictionary(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<unsigned long, std::allocator<unsigned long> > const&, unsigned long)': ./builddir/storage/rocksdb/./storage/rocksdb/rocksdb/util/compression.h :812: undefined reference to `ZDICT_trainFromBuffer' ./builddir/storage/rocksdb/./storage/rocksdb/rocksdb/util/compression.h :813: undefined reference to `ZDICT_isError' collect2: error: ld returned 1 exit status http://buildbot.askmonty.org/buildbot/builders/kvm-deb-stretch-amd64/bu ilds/3986/steps/compile/logs/stdio - jessie amd64 builds to compile due to /usr/bin/ld.bfd.real: /tmp/ccOIwjFo.ltrans0.ltrans.o: relocation R_X86_64_PC32 against symbol `toku_product_name_strings' can not be used when making a shared object; recompile with -fPIC /usr/bin/ld.bfd.real: final link failed: Bad value collect2: error: ld returned 1 exit status http://buildbot.askmonty.org/buildbot/builders/kvm-deb-jessie-amd64/bui lds/4298/steps/compile/logs/stdio This -fPIC issue might be related to a change in hardening flags I "forward-ported" from Debian and I will look into it. All the other issues are things I cannot do anything about.. :( On the bright side I can report that there are at least no build regressions on Launchpad with current 10.3: https://launchpad.net/~mysql-ubuntu/+archive/ubuntu/mariadb-10.3/+build s?build_text=&build_state=all - Otto

1 0

[Maria-developers] With_sum_func_cache
by Alexander Barkov 31 May '18

31 May '18

Hello Sanja, I recently worked on MDEV-16309 and had hard time to find which Item classes in the hierarchy: - have always with_sum_func==true - have always with_sum_func==false - have variable with_sum_func To make it sure, before actually working on MDEV-16309, I had to create a separate working tree and did with Item::with_sum_func the same change that we previously did for Item::with_subselect in: MDEV-14517 Cleanup for Item::with_subselect and Item::has_subquery() (which you reviewed) - I find the code easier to read this way (instead of searching for all possible assignments, one can search who overrides the method) - Also, some Item can become smaller in size. It's pity to throw the patch away. So perhaps we could just apply it. Can you please have a look? Thanks. I the meanwhile I'll create a new MDEV for it (with a similar description to MDEV-14517)

2 2

[Maria-developers] Open and close a TABLE_LIST at random place in code
by Eugene Kosov 31 May '18

31 May '18

Hello. I need to see a table structure while executing some query (DELETE in my case, but there will be more query types in future). I have a TABLE_LIST which came from a parser. Now I'm opening it with open_table_only_view_structure(), examine table structure and close it, trying to prevent any side effects. And there is a lot of them. Current code looks like this: if (open_tables_only_view_structure(thd, table_list, false)) { return true; } // examine table structure close_mysql_tables(thd); for (Sroutine_hash_entry *e = thd->lex->sroutines_list.first;e; e=e->next) e->mdl_request.ticket= NULL; for (TABLE_LIST *it= table_list; it; it= it->next_global) { it->table= NULL; it->reinit_before_use(thd); //it->next_global= NULL; it->derived_result= NULL; it->derived= NULL; it->view= NULL; it->view_tables= NULL; it->view_sctx= NULL; it->file_version= 0; it->mariadb_version= 0; it->updatable_view= 0; it->view_suid= 0; it->derived_type= 0; it->open_type= OT_BASE_ONLY; it->view_creation_ctx= 0; // it->set_table_ref_id(TABLE_REF_NULL, 0); } Is there a way to close a TABLE_LIST gracefully without touching a lot of it's fields? Or even maybe it's possible to open a table in a simpler manner? -- Eugene

2 3

Re: [Maria-developers] [Commits] de722585c1b: MDEV-16225: wrong resultset from query with semijoin=on
by Sergey Petrunia 27 May '18

27 May '18

Hi Varun, On Thu, May 24, 2018 at 03:17:02AM +0530, Varun wrote: > revision-id: de722585c1b2ad2af99bd1749e8566ea77838764 (mariadb-5.5.56-219-gde722585c1b) > parent(s): 1ada4afb0a51f7283b6187a95019ec2cb80c8a0b > author: Varun Gupta > committer: Varun Gupta > timestamp: 2018-05-24 03:09:42 +0530 > message: > > MDEV-16225: wrong resultset from query with semijoin=on > > For non-semi-join subquery optimization we do a cost based decision between > Materialisation and IN -> EXIST transformation. The issue in this case is that for IN->EXIST transformation > we run JOIN::reoptimize with the IN->EXISt conditions and we come up with a new query plan. But when we compare > the cost with Materialization, we make the decision to chose Materialization so we need to restore the query plan > for Materilization. The restoring of query plan is not handled correctly. > > diff --git a/sql/sql_select.cc b/sql/sql_select.cc > index d6d269a700f..b8d43b9797e 100644 > --- a/sql/sql_select.cc > +++ b/sql/sql_select.cc > @@ -23805,6 +23805,22 @@ void JOIN::restore_query_plan(Join_plan_state *restore_from) > } > > } > + else > + { > + /* > + After running JOIN::reoptimize for IN->EXISTS > + transformation we might add elements to the keyuse array, > + The original query plan was that the keyuse array had no elements, so we > + should restore that state here. > + */ > + while(keyuse.elements) > + delete_dynamic_element(&keyuse, 0); I'm wondering, why do we need the above loop and not just delete everything with delete_dynamic(). But if I don't miss anything, this is not significant. Ok to push either the above variant or one with delete_dynamic(). > + for (uint i= 0; i < table_count; i++) > + { > + join_tab[i].keyuse= NULL; > + join_tab[i].checked_keys.clear_all(); > + } > + } > memcpy((uchar*) best_positions, (uchar*) restore_from->best_positions, > sizeof(POSITION) * (table_count + 1)); > /* Restore SJM nests */ > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits -- BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] [Maria-discuss] Query ambiguity
by Karthick Subramanian 17 May '18

17 May '18

Check whether you really need a comma here: *metrics_5.id <http://metrics_5.id/>,* *(SELECT * Also in your screen shot, AS 'LT CPC`, there is no comma and you use derived table in SELECT and trying to use JOIN in the FROM using this. Sorry to say this, I think you can either break this logic into small simple chunks and use SP to derive the logic in case you are not much comfortable writing a complex SQL. I think there is lot of syntax errors on it. On Thu, May 17, 2018 at 10:05 AM, Dev C <chauhan.devsur(a)gmail.com> wrote: > Here is the full code and picture on what is required. > > SELECT > campaigns_5.name AS `Campaign Name`, > adgroups_5.name AS `Adgroup Name`, > STNegData1W.query, > keywords_5.matchType AS `Match Type`, > Sum(metrics_5.clicks) AS `LT Clicks`, > Sum(metrics_5.costs) AS `LT Spend`, > Sum(metrics_5.orders) AS `LT Orders`, > metrics_5.sales AS `LT Sales`, > If(Sum(metrics_5.sales)=0,0,Sum(metrics_5.costs)/Sum(metrics_5.sales)) > AS `LT ACOS`, > If(Sum(clicks)=0,0,Sum(costs)/Sum(clicks)) AS `LT CPC` > FROM (((keywords_5 INNER JOIN campaigns_5 ON keywords_5.campaignId = > campaigns_5.campaignId) INNER JOIN adgroups_5 ON keywords_5.adGroupId = > adgroups_5.adGroupId) INNER JOIN STNegData1W ON keywords_5.keywordId = > STNegData1W.keywordId) INNER JOIN metrics_5 ON STNegData1W.metricId = *metrics_5.id > <http://metrics_5.id>,* > *(SELECT * > STNegData4W.query, > STNegData4W.metricId, > STNegData4W.keywordId, > metrics_5.impressions > FROM > (SELECT searchterm_5.query, searchterm_5.metricId, > searchterm_5.keywordId, Sum(metrics_5.orders) AS `4W Orders` > FROM searchterm_5 INNER JOIN metrics_5 ON searchterm_5.metricId = > metrics_5.id > GROUP BY searchterm_5.query, searchterm_5.metricId, > searchterm_5.keywordId, metrics_5.created_at > HAVING (((Sum(metrics_5.orders))=0) AND ((metrics_5.created_at) > > DATE_SUB('2018-05-10',INTERVAL 28 DAY)))) AS STNegData4W > INNER JOIN > metrics_5 ON STNegData4W.metricId = metrics_5.id > GROUP BY STNegData4W.query, STNegData4W.metricId, STNegData4W.keywordId, > metrics_5.impressions, metrics_5.created_at > HAVING (((metrics_5.impressions)>0) AND ((metrics_5.created_at)> > DATE_SUB('2018-05-10', INTERVAL 7 DAY)))) AS STNegData1W > GROUP BY campaigns_5.name, adgroups_5.name, STNegData1W.query, > keywords_5.matchType, metrics_5.sales, campaigns_5.user_market_id, Right( > campaigns_5.name,4) > HAVING (((Sum(metrics_5.clicks))>5) AND ((campaigns_5.user_market_id)=12) > AND ((Right(campaigns_5.name,4))="PFUS")) > > On Thu, May 17, 2018 at 4:06 PM, Pantelis Theodosiou <ypercube(a)gmail.com> > wrote: > >> What Rhys said, please post queries as text, not as images. >> >> The issue is likely due to the way you structured your query. You have >> put a derived table in a random place (in the SELECT list). >> >> Please try to read about CTEs. It will help you design those complex >> queries better and make them more readable and easy to edit: >> https://mariadb.com/kb/en/library/with/ >> >> Best regards >> >> Pantelis Theodosiou >> >> On Thu, May 17, 2018 at 11:23 AM, <Rhys.Campbell(a)swisscom.com> wrote: >> >>> You're not making a lot of sense here. Please explain the "ambiguity". >>> That's a fairly complex query so you should include as text. >>> >>> >>> >>> "Does not work" <- In the image. It helps if you include error message. >>> >>> >>> >>> *From:* Maria-discuss [mailto:maria-discuss-bounces+rhys.campbell= >>> swisscom.com(a)lists.launchpad.net] *On Behalf Of *Dev C >>> *Sent:* 17 May 2018 06:41 >>> *To:* maria-discuss(a)lists.launchpad.net; Maria Developers < >>> maria-developers(a)lists.launchpad.net> >>> *Subject:* [Maria-discuss] Query ambiguity >>> >>> >>> >>> Hello >>> >>> >>> >>> I have query ambiguity as attached. Please help me on how to solve that. >>> >>> >>> >>> >>> _______________________________________________ >>> Mailing list: https://launchpad.net/~maria-discuss >>> Post to : maria-discuss(a)lists.launchpad.net >>> Unsubscribe : https://launchpad.net/~maria-discuss >>> More help : https://help.launchpad.net/ListHelp >>> >>> >> > > _______________________________________________ > Mailing list: https://launchpad.net/~maria-discuss > Post to : maria-discuss(a)lists.launchpad.net > Unsubscribe : https://launchpad.net/~maria-discuss > More help : https://help.launchpad.net/ListHelp > >

1 0

Re: [Maria-developers] [Maria-discuss] Query ambiguity
by Pantelis Theodosiou 17 May '18

17 May '18

What Rhys said, please post queries as text, not as images. The issue is likely due to the way you structured your query. You have put a derived table in a random place (in the SELECT list). Please try to read about CTEs. It will help you design those complex queries better and make them more readable and easy to edit: https://mariadb.com/kb/en/library/with/ Best regards Pantelis Theodosiou On Thu, May 17, 2018 at 11:23 AM, <Rhys.Campbell(a)swisscom.com> wrote: > You're not making a lot of sense here. Please explain the "ambiguity". > That's a fairly complex query so you should include as text. > > > > "Does not work" <- In the image. It helps if you include error message. > > > > *From:* Maria-discuss [mailto:maria-discuss-bounces+rhys.campbell= > swisscom.com(a)lists.launchpad.net] *On Behalf Of *Dev C > *Sent:* 17 May 2018 06:41 > *To:* maria-discuss(a)lists.launchpad.net; Maria Developers < > maria-developers(a)lists.launchpad.net> > *Subject:* [Maria-discuss] Query ambiguity > > > > Hello > > > > I have query ambiguity as attached. Please help me on how to solve that. > > > > > _______________________________________________ > Mailing list: https://launchpad.net/~maria-discuss > Post to : maria-discuss(a)lists.launchpad.net > Unsubscribe : https://launchpad.net/~maria-discuss > More help : https://help.launchpad.net/ListHelp > >

2 1

Re: [Maria-developers] [Maria-discuss] Interval in MYsql query
by Peter Laursen 16 May '18

16 May '18

Use the DATE_ADD( ) or DATE_SUB() function with INTERVAL as explained here: https://dev.mysql.com/doc/refman/8.0/en/date-and-time-functions.html#functi… -- Peter. On Wed, May 16, 2018 at 4:07 PM, Dev C <chauhan.devsur(a)gmail.com> wrote: > Hello > > If I use below query > (metrics_5.created_at)>'5/10/2018'- 28 ) > it gives me some result > but if i use below query > (metrics_5.created_at)>'5/10/2018'- INTERVAL 28 DAY ) > > It does not give me any result, any one know why ? > > _______________________________________________ > Mailing list: https://launchpad.net/~maria-discuss > Post to : maria-discuss(a)lists.launchpad.net > Unsubscribe : https://launchpad.net/~maria-discuss > More help : https://help.launchpad.net/ListHelp > >

1 0

[Maria-developers] [question] password input length
by Michal Schorm 16 May '18

16 May '18

Hello everyone, 1) I found out, that passwords longer than 79 characters aren't accepted via password prompt. Is that expected? Morie info and reproducer in RH bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1566179 -- 2) Another password length cap at 213 characters can be found when using sha256_password authentification plugin. Is that expected too? Morie info and reproducer in RH bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1566219 -- 3) Different theme: why aren't tests for mariadb-connector-odbc, that can be found on GitHub, packged to the source tarball ? -- Michal Schorm Associate Software Engineer Core Services - Databases Team Red Hat

2 2

Re: [Maria-developers] a639eff: MDEV-15813 ASAN use-after-poison in hp_hashnr upon HANDLER READ on a versioned HEAP table.
by Sergei Golubchik 16 May '18

16 May '18

Hi, Alexey! On May 16, Alexey Botchkov wrote: > revision-id: a639eff5940ed4147709511aeaee922fa7095284 (mariadb-10.3.6-193-ga639eff) > parent(s): cd15e764a8a4f3e799f1efaede413c95e3233df6 > committer: Alexey Botchkov > timestamp: 2018-05-16 09:44:22 +0400 > message: > > MDEV-15813 ASAN use-after-poison in hp_hashnr upon HANDLER READ on a versioned HEAP table. > > Check index capabilities before executing HANDLER READ command. > > diff --git a/sql/sql_handler.cc b/sql/sql_handler.cc > index 187a746..02d5bbf 100644 > --- a/sql/sql_handler.cc > +++ b/sql/sql_handler.cc > @@ -660,6 +660,18 @@ mysql_ha_fix_cond_and_key(SQL_HANDLER *handler, > Item *item; > key_part_map keypart_map; > uint key_len; > + const KEY *c_key= table->s->key_info + handler->keyno; > + > + if ((c_key->flags & HA_SPATIAL) || > + c_key->algorithm == HA_KEY_ALG_FULLTEXT || > + (ha_rkey_mode != HA_READ_KEY_EXACT && > + (table->file->index_flags(handler->keyno, 0, TRUE) & > + (HA_READ_NEXT | HA_READ_PREV | HA_READ_RANGE)) == 0)) This is strange. Do you mean that SPATIAL and FULLTEXT indexes have HA_READ_NEXT or HA_READ_PREV or HA_READ_RANGE? Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] MDEV-16094 Crash when using AS OF with a stored function
by Alexander Barkov 15 May '18

15 May '18

Hi Sergei, Please review a patch for MDEV-16094. Thanks!

2 2

Re: [Maria-developers] review of MDEV-16101 ADD PARTITION on table partitioned by list does not work …
by Jacob Mathew 14 May '18

14 May '18

Hi Holyfoot, Your fix creates more lists, all of them containing one element, but it does only one reorganization. My fix does a reorganization whenever a 33rd element is processed but creates less lists. I agree that reducing the number of reorganizations to one is a better way to fix the problem. So I will use your fix. Thanks, Jacob Jacob B. Mathew Spider and Server Developer MariaDB Corporation +1 408 655 8999 (mobile) jacob.b.mathew (Skype) jacob.mathew(a)MariaDB.com On Sun, May 13, 2018 at 2:36 PM, Alexey Botchkov <holyfoot(a)mariadb.com> wrote: > Hi, Jacob. > > Your fix: > https://github.com/MariaDB/server/commit/85bbea8dbb9c01defa993e44a5b16c > 43ef851fbd > > I don't like the way your fix work. > So after we get the 32-th item we do the reorganize_into_single_ > field_col_val() > trick that turns one element with 32 values into 32 elements with a single > value. > Then as we get 33-th element, we again put it into that first element, and > do so for > the next 31 element then call the reorganize_ again. > > I think we should just keep the 'one-value' list after the 'reorganize_' > So my fix would be like this: > --- a/sql/partition_info.cc > +++ b/sql/partition_info.cc > @@ -1996,9 +1996,11 @@ part_column_list_val *partition_info::add_column_value(THD > *thd) > into the structure used for 1 column. After this we call > ourselves recursively which should always succeed. > */ > + num_columns= curr_list_object; > if (!reorganize_into_single_field_col_val(thd)) > { > - DBUG_RETURN(add_column_value(thd)); > + if (!init_column_part(thd)) > + DBUG_RETURN(add_column_value(thd)); > } > > > Best regards. > HF > >

1 0

Re: [Maria-developers] 5651e72: MDEV-15465 Server crash or ASAN heap-use-after-free in Item_func_match::cleanup upon using FT search with partitioning.
by Sergei Golubchik 14 May '18

14 May '18

Hi, Alexey! On May 14, Alexey Botchkov wrote: > Hi Sergei, > > I was wrong - this fix is only needed in 10.3, since the FT keys > are available for PARTITION tables. > So going to push it to the 10.3. Is it ok with you? Yes, thanks! Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] [Commits] 4861977ce76: MDEV-14695: Assertion `n < m_size' failed in Bounds_checked_array<Element_type>::operator
by Vicențiu Ciorbaru 14 May '18

14 May '18

Hi Varun! I will take a second look at this but a first minor comment is to not save two characters by shortening fields to flds, it makes reading unnatural. Vicentiu On Sat, 12 May 2018 at 21:45 Varun <varunraiko1803(a)gmail.com> wrote: > revision-id: 4861977ce76761fefb9779a774984b6afa818f2e > (mariadb-10.2.5-636-g4861977ce76) > parent(s): 8b26fea83572cf5c043721b7835c3828937f9c27 > author: Varun Gupta > committer: Varun Gupta > timestamp: 2018-05-13 00:09:15 +0530 > message: > > MDEV-14695: Assertion `n < m_size' failed in > Bounds_checked_array<Element_type>::operator > > In this issue we hit the assert because we are adding addition fields to > the field JOIN::all_fields list. This > is done because HEAP tables can't index BIT fields so we need to use an > additional hidden field for grouping because later it will be > converted to a LONG field. Original field will remain of the BIT type and > will be returned. This happens when we convert DISTINCT to > GROUP BY. > > The solution is to take into account the number of such hidden fields that > would be added to the field JOIN::all_fields list while calculating > the size of the ref_pointer_array. > > --- > sql/sql_base.cc | 9 ++++++--- > sql/sql_base.h | 4 ++-- > sql/sql_delete.cc | 3 ++- > sql/sql_lex.cc | 8 +++++++- > sql/sql_lex.h | 5 +++++ > sql/sql_select.cc | 4 +++- > sql/sql_union.cc | 1 + > 7 files changed, 26 insertions(+), 8 deletions(-) > > diff --git a/sql/sql_base.cc b/sql/sql_base.cc > index 95ff41cb6f4..81ccfe5e514 100644 > --- a/sql/sql_base.cc > +++ b/sql/sql_base.cc > @@ -6919,7 +6919,7 @@ static bool setup_natural_join_row_types(THD *thd, > > int setup_wild(THD *thd, TABLE_LIST *tables, List<Item> &fields, > List<Item> *sum_func_list, > - uint wild_num) > + uint wild_num, uint *hidden_bit_flds) > { > if (!wild_num) > return(0); > @@ -6960,7 +6960,7 @@ int setup_wild(THD *thd, TABLE_LIST *tables, > List<Item> &fields, > else if (insert_fields(thd, ((Item_field*) item)->context, > ((Item_field*) item)->db_name, > ((Item_field*) item)->table_name, &it, > - any_privileges)) > + any_privileges, hidden_bit_flds)) > { > if (arena) > thd->restore_active_arena(arena, &backup); > @@ -7425,7 +7425,7 @@ bool get_key_map_from_key_list(key_map *map, TABLE > *table, > bool > insert_fields(THD *thd, Name_resolution_context *context, const char > *db_name, > const char *table_name, List_iterator<Item> *it, > - bool any_privileges) > + bool any_privileges, uint *hidden_bit_flds) > { > Field_iterator_table_ref field_iterator; > bool found; > @@ -7545,6 +7545,9 @@ insert_fields(THD *thd, Name_resolution_context > *context, const char *db_name, > else > it->after(item); /* Add 'item' to the SELECT list. */ > > + if (item->type() == Item::FIELD_ITEM && item->field_type() == > MYSQL_TYPE_BIT) > + (*hidden_bit_flds)++; > + > #ifndef NO_EMBEDDED_ACCESS_CHECKS > /* > Set privilege information for the fields of newly created views. > diff --git a/sql/sql_base.h b/sql/sql_base.h > index d6063f1b771..8d0dcb2f197 100644 > --- a/sql/sql_base.h > +++ b/sql/sql_base.h > @@ -154,11 +154,11 @@ bool fill_record_n_invoke_before_triggers(THD *thd, > TABLE *table, > enum trg_event_type event); > bool insert_fields(THD *thd, Name_resolution_context *context, > const char *db_name, const char *table_name, > - List_iterator<Item> *it, bool any_privileges); > + List_iterator<Item> *it, bool any_privileges, uint > *hidden_bit_flds); > void make_leaves_list(THD *thd, List<TABLE_LIST> &list, TABLE_LIST > *tables, > bool full_table_list, TABLE_LIST *boundary); > int setup_wild(THD *thd, TABLE_LIST *tables, List<Item> &fields, > - List<Item> *sum_func_list, uint wild_num); > + List<Item> *sum_func_list, uint wild_num, uint * > hidden_bit_flds); > bool setup_fields(THD *thd, Ref_ptr_array ref_pointer_array, > List<Item> &item, enum_mark_columns mark_used_columns, > List<Item> *sum_func_list, List<Item> *pre_fix, > diff --git a/sql/sql_delete.cc b/sql/sql_delete.cc > index 7b7a7e3f804..7bae7736cf3 100644 > --- a/sql/sql_delete.cc > +++ b/sql/sql_delete.cc > @@ -760,7 +760,8 @@ l > select_lex->leaf_tables, FALSE, > DELETE_ACL, SELECT_ACL, TRUE)) > DBUG_RETURN(TRUE); > - if ((wild_num && setup_wild(thd, table_list, field_list, NULL, > wild_num)) || > + if ((wild_num && setup_wild(thd, table_list, field_list, NULL, wild_num, > + &select_lex->hidden_bit_flds)) || > setup_fields(thd, Ref_ptr_array(), > field_list, MARK_COLUMNS_READ, NULL, NULL, 0) || > setup_conds(thd, table_list, select_lex->leaf_tables, conds) || > diff --git a/sql/sql_lex.cc b/sql/sql_lex.cc > index 3e36cac96b9..49150c863fe 100644 > --- a/sql/sql_lex.cc > +++ b/sql/sql_lex.cc > @@ -2143,6 +2143,7 @@ void st_select_lex::init_query() > m_agg_func_used= false; > window_specs.empty(); > window_funcs.empty(); > + hidden_bit_flds= 0; > } > > void st_select_lex::init_select() > @@ -2673,6 +2674,10 @@ ulong st_select_lex::get_table_join_options() > > bool st_select_lex::setup_ref_array(THD *thd, uint order_group_num) > { > + > + if (!((options & SELECT_DISTINCT) && !group_list)) > + hidden_bit_flds= 0; > + > // find_order_in_list() may need some extra space, so multiply by two. > order_group_num*= 2; > > @@ -2687,7 +2692,8 @@ bool st_select_lex::setup_ref_array(THD *thd, uint > order_group_num) > select_n_reserved + > select_n_having_items + > select_n_where_fields + > - order_group_num) * 5; > + order_group_num + > + hidden_bit_flds) * 5; > if (!ref_pointer_array.is_null()) > { > /* > diff --git a/sql/sql_lex.h b/sql/sql_lex.h > index 6d281930d1f..8283496604e 100644 > --- a/sql/sql_lex.h > +++ b/sql/sql_lex.h > @@ -956,6 +956,11 @@ class st_select_lex: public st_select_lex_node > > /* it is for correct printing SELECT options */ > thr_lock_type lock_type; > + /* > + it counts the number of bit fields in the SELECT list. These are used > when DISTINCT is > + converted to a GROUP BY involving BIT fields. > + */ > + uint hidden_bit_flds; > > void init_query(); > void init_select(); > diff --git a/sql/sql_select.cc b/sql/sql_select.cc > index 4cd4754596b..83953661ce3 100644 > --- a/sql/sql_select.cc > +++ b/sql/sql_select.cc > @@ -799,7 +799,9 @@ JOIN::prepare(TABLE_LIST *tables_init, > select_lex != select_lex->master_unit()->global_parameters()) > real_og_num+= select_lex->order_list.elements; > > - if (setup_wild(thd, tables_list, fields_list, &all_fields, wild_num)) > + DBUG_ASSERT(select_lex->hidden_bit_flds == 0); > + if (setup_wild(thd, tables_list, fields_list, &all_fields, wild_num, > + &select_lex->hidden_bit_flds)) > DBUG_RETURN(-1); > if (select_lex->setup_ref_array(thd, real_og_num)) > DBUG_RETURN(-1); > diff --git a/sql/sql_union.cc b/sql/sql_union.cc > index 3fe8a24f8bd..42904e341b6 100644 > --- a/sql/sql_union.cc > +++ b/sql/sql_union.cc > @@ -1474,6 +1474,7 @@ bool st_select_lex::cleanup() > } > inner_refs_list.empty(); > exclude_from_table_unique_test= FALSE; > + hidden_bit_flds= 0; > DBUG_RETURN(error); > } > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits

1 0

[Maria-developers] MDEV-6111: Optimizer trace: features and milestones
by Sergey Petrunia 11 May '18

11 May '18

Hi Zhzhzoo, Please find below some thoughts about features/milestones in the project. Items at the top of the list should be done first. The list is not necesarrily complete, any comments/additions are welcome. - Optimizer trace collection API (is it different from Json_writer that we have? Does MySQL use the same API to produce EXPLAIN FORMAT=JSON and produce optimizer_trace? Should we follow them?) - Basic optimizer trace collection (just print the query and "hello":"world" instead of trace). Clear the trace data at query start, collect the trace if tracing is switched on (and do not collect if it is off) - INFORMATION_SCHEMA.OPTIMIZER_TRACE table in I_S database. Produce optimizer_trace content for features common with MySQL: = condition rewrite = partition pruning = range analysis = ref/eq_ref analysis (update_ref_and_keys) = (TODO what else?) Produce optimizer_trace content for MariaDB-specific features: = merged semi-join conversion = non-merged semi-join conversion = join condition pushdown = (TODO what else?) - optimizer_trace_limit/optimizer_trace_offset (without this feature, just save one last trace) - Ability to produce either a pretty-printed or one-line optimizer trace - Security = INFORMATION_SCHEMA.OPTIMIZER_TRACE.INSUFFICIENT_PRIVILEGES = Who can view optimizer_trace? = Figure out and implement the necessary security restrictions - Memory limits = Honor the @@optimizer_trace_max_mem_size setting. BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

2 1

[Maria-developers] Please review MDEV-16134 Wrong I_S.COLUMNS.CHARACTER_XXX_LENGTH value for compressed columns
by Alexander Barkov 10 May '18

10 May '18

Hi Sergey, Please review a patch for MDEV-16134. Thanks!

1 0

[Maria-developers] MDEV-15935: Connection Redirection Mechanism in MariaDB Client/Server Protocol
by xiangyu hu 09 May '18

09 May '18

Hi developers, There is an ongoing discussion regarding to the proposal of connection redirection in JIRA. Please kindly take a look if you aren't aware of it. Link is: https://jira.mariadb.org/browse/MDEV-15935. Thank you, Xiangyu

1 0

[Maria-developers] MDEV-15450: Add Host Name Field to MariaDB Client Handshake Protocol
by xiangyu hu 09 May '18

09 May '18

Hello devs, We have initiated a proposal in JIRA where we described the necessity of adding host_name into login packet attributes. I would like to broaden the audience here to bring up more discussions. Please take a look at MDEV-15450 (https://jira.mariadb.org/browse/MDEV-15450) [MDEV-15450] Add Host Name Field to MariaDB Client ...<https://jira.mariadb.org/browse/MDEV-15450> jira.mariadb.org Why. In original, one MariaDB server instance is identified by one address (IP + port). That is not enough for a shared proxy for multi servers. Thanks, Xiangyu

1 0

Re: [Maria-developers] eb3e9cd: MDEV-15813 ASAN use-after-poison in hp_hashnr upon HANDLER READ on a versioned HEAP table.
by Sergei Golubchik 09 May '18

09 May '18

Hi, Alexey! On May 08, Alexey Botchkov wrote: > revision-id: eb3e9cdbe86b5a68c3ec8a9f26165259e4ca7eec (mariadb-10.3.6-125-geb3e9cd) > parent(s): bd09c5ca866e273b6cebbd9a15c51c82bfa3ac9b > committer: Alexey Botchkov > timestamp: 2018-05-08 14:19:55 +0400 > message: > > MDEV-15813 ASAN use-after-poison in hp_hashnr upon HANDLER READ on a versioned HEAP table. > > Returns an error if we try to use a key in a query it declared unable to > handle. > > diff --git a/sql/share/errmsg-utf8.txt b/sql/share/errmsg-utf8.txt > index c55ac4f..7446a60 100644 > --- a/sql/share/errmsg-utf8.txt > +++ b/sql/share/errmsg-utf8.txt > @@ -7915,3 +7915,5 @@ ER_UPDATED_COLUMN_ONLY_ONCE > eng "The column %`s.%`s cannot be changed more than once in a single UPDATE statement" > ER_EMPTY_ROW_IN_TVC > eng "Row with no elements is not allowed in table value constructor in this context" > +ER_KEY_DOESNT_SUPPORT > + eng "%s index %`s does not support this operation" I think this can be safely pushed into 10.2. But not earlier, we can only add new error messages in the latest GA version, not in previos GA's. > diff --git a/sql/sql_handler.cc b/sql/sql_handler.cc > index 09883d8..12e4b5a 100644 > --- a/sql/sql_handler.cc > +++ b/sql/sql_handler.cc > @@ -661,12 +661,30 @@ mysql_ha_fix_cond_and_key(SQL_HANDLER *handler, > key_part_map keypart_map; > uint key_len; > > + if (ha_rkey_mode != HA_READ_KEY_EXACT && > + (table->file->index_flags(handler->keyno, 0, TRUE) & > + (HA_READ_NEXT | HA_READ_PREV | HA_READ_RANGE)) == 0) I suspect there can be hypothetical cases where an index supports some specific ha_rkey_mode values and has some specific flags, but not all of them, like you test here. But as the only real use case is HASH vs BTREE now, your check is fine. No need overengineer a complex test that we cannot possibly test now. > + { > + my_error(ER_KEY_DOESNT_SUPPORT, MYF(0), > + table->file->index_type(handler->keyno), keyinfo->name); > + return 1; > + } > + > if (key_expr->elements > keyinfo->user_defined_key_parts) > { > my_error(ER_TOO_MANY_KEY_PARTS, MYF(0), > keyinfo->user_defined_key_parts); > return 1; > } > + else if (key_expr->elements < keyinfo->user_defined_key_parts && > + (table->file->index_flags(handler->keyno, 0, TRUE) & > + HA_ONLY_WHOLE_INDEX)) > + { > + my_error(ER_KEY_DOESNT_SUPPORT, MYF(0), > + table->file->index_type(handler->keyno), keyinfo->name); > + return 1; Good point! > + } This is ok to push. Just two questions: 1. Why did you add it in sql_handler.cc and not in the class handler method, like ha_index_read_map or ha_index_next ? 2. Could you add tests for fulltext/rtree indexes too, please? Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] GSoC 2018 - MDEV-12313 - 30th of April -> 6th of May Update
by Teodor-Vicentiu Niculescu 08 May '18

08 May '18

Greetings, First of all, I want to thank you for accepting me for this years GSoC. I plan to do my best during the following months in order to implement the SQL Statement specified in the proposal. https://docs.google.com/document/d/1dA7YlDPlRG43LhfGlHXnHJiERfjExLG9qS7BqXh… In the last week (30th of April to 6th of May) I have started the first task of the proposal, specifically extending the grammar. After receiving some initial hints from Mr. Ciorbaru, concerning where to start looking in the files ( ANALYZE_SYM in sql/sql_yacc.yy ), I quickly discovered I have no knowledge of GNU Bison. I took my time to look through the GNU Bison online documentation, I have compiled tests and written code in order to better understand parsing. Now, sql_yacc.yy is starting to make sense, which is why I plan to finish the parsing part of the project by Sunday (13th of May). Until 15th of June my schedule will be quite busy because finals are approaching. You can still contact me by e-mail. Advice and constructive criticism is always much appreciated. Thank you for this amazing opportunity. Sincerely, Teodor

1 0

Re: [Maria-developers] fa0b71b0e19: MDEV-10259 mysqld crash with certain statement length and...
by Sergei Golubchik 08 May '18

08 May '18

Hi, Sachin! On May 08, Sachin Setiya wrote: > diff --git a/unittest/sql/mf_iocache-t.cc b/unittest/sql/mf_iocache-t.cc > index 31f98562521..d5e2ffb095e 100644 > --- a/unittest/sql/mf_iocache-t.cc > +++ b/unittest/sql/mf_iocache-t.cc > @@ -187,10 +187,71 @@ void mdev9044() > close_cached_file(&info); > } > > +/* 2 Reads in cache makes second read to fail (only if first read uses temp file) */ > +void mdev10259() > +{ > + int res; > + uchar buf[CACHE_SIZE + 200]; > + memset(buf, FILL, sizeof(buf)); > + > + diag("MDEV-10259- mysqld crash with certain statement length and order with" > + " Galera and encrypt-tmp-files=1"); > + > + init_io_cache_encryption(); > + > + res= open_cached_file(&info, 0, 0, CACHE_SIZE, 0); > + ok(res == 0, "open_cached_file" INFO_TAIL); > + > + res= my_b_write(&info, buf, sizeof(buf)); > + ok(res == 0 && info.pos_in_file == CACHE_SIZE, "large write" INFO_TAIL); > + > + res= my_b_flush_io_cache(&info, 1); > + ok(res == 0, "flush" INFO_TAIL); I think this misses the point. The bug happened when the data is in the cache, and there's no file to read from, so when the data is lost - it's forever. But my_b_flush_io_cache() writes the buffer to a file, so my_b_fill() can read it even if the buffer is emptied. Meaning, I suspect that if you remove your fix from mf_iocache_encr.cc, this unit test will still pass. Please double-check that your test fails without a fix. Ok to push after that. > + res= reinit_io_cache(&info, READ_CACHE, 0, 0, 0); > + ok(res == 0, "reinit READ_CACHE" INFO_TAIL); Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] 7107565: MDEV-10259 mysqld crash with certain statement length and...
by Sergei Golubchik 08 May '18

08 May '18

Hi, Sachin! Great work! On Mar 27, Sachin Setiya wrote: > revision-id: 710756527ec8589bce33bc834a1d8e6604266b6d (mariadb-10.1.31-47-g7107565) > parent(s): 8152c52e1af608da5d288a4e6fc82e2d804e64cd > author: Sachin Setiya > committer: Sachin Setiya > timestamp: 2018-03-27 04:24:57 +0530 > message: > > MDEV-10259 mysqld crash with certain statement length and... > order with Galera and encrypt-tmp-files=1 > > Problem:- If trans_cache (IO_CACHE) uses encrypted tmp file > then on next DML server will crash. > > Case:- > Lets take a case , we have a table t1 , We try to do 2 inserts in t1 > 1. A really long insert so that trans_cache has to use temp_file > 2. Just a small insert > > Analysis:- Actually server crashes from inside of galera > library. > /lib64/libc.so.6(abort+0x175)[0x7fb5ba779dc5] > /usr/lib64/galera/libgalera_smm.so(_ZN6galera3FSMINS_9TrxHandle5State... > mysys/stacktrace.c:247(my_print_stacktrace)[0x7fb5a714940e] > sql/signal_handler.cc:160(handle_fatal_signal)[0x7fb5a715c1bd] > sql/wsrep_hton.cc:257(wsrep_rollback)[0x7fb5bcce923a] > sql/wsrep_hton.cc:268(wsrep_rollback)[0x7fb5bcce9368] > sql/handler.cc:1658(ha_rollback_trans(THD*, bool))[0x7fb5bcd4f41a] > sql/handler.cc:1483(ha_commit_trans(THD*, bool))[0x7fb5bcd4f804] > > but actual issue is not in galera but in mariadb, because for 2nd > insert we should never call rollback. We are calling rollback because > log_and_order fails it fails because write_cache fails , It fails > because after reinit_io_cache(trans_cache) , my_b_tell says 0 so we > look into tmp_file for data , which is obviously wrong since temp > was used for previous insert and it no longer exist. Add here something like: wsrep_write_cache_inc() reads the IO_CACHE in a loop, filling it with my_b_fill() until it returns "0 bytes read". Later MYSQL_BIN_LOG::write_cache() does the same. wsrep_write_cache_inc() assumes that reading a zero bytes past EOF leaves the old data in the cache. Let's do that. > diff --git a/sql/mf_iocache_encr.cc b/sql/mf_iocache_encr.cc > index 149e6fe..41e1333 100644 > --- a/sql/mf_iocache_encr.cc > +++ b/sql/mf_iocache_encr.cc > @@ -49,8 +49,7 @@ static int my_b_encr_read(IO_CACHE *info, uchar *Buffer, size_t Count) > > if (pos_in_file == info->end_of_file) > { here, a comment: /* reading past EOF should not empty the cache */ > - info->read_pos= info->read_end= info->buffer; > - info->pos_in_file= pos_in_file; > + info->read_pos= info->read_end; > info->error= 0; > DBUG_RETURN(MY_TEST(Count)); > } and add a test for it into unittest/sql/mf_iocache-t.cc. Something simple, like, write into a cache, reinit to READ, fill, fill, reinit to READ, verify the data is still there. Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

2 2

Re: [Maria-developers] bea915f: MDEV-15965 Invisible columns and LOAD DATA don't work well together:...
by Sergei Golubchik 07 May '18

07 May '18

Hi, sachin! On Apr 26, sachin wrote: > revision-id: bea915f21a2a2771ce7f9e19dcb9d6893f33f7bd (mariadb-10.3.6-47-gbea915f) > parent(s): 326db1a2aaa9b275a1a21a863e8cd2d9fa1b1d5f > author: Sachin Setiya > committer: Sachin Setiya > timestamp: 2018-04-26 16:49:27 +0530 > message: > > MDEV-15965 Invisible columns and LOAD DATA don't work well together:... > ER_WARN_TOO_FEW_RECORDS > > Fix mysql_load iterator to skip invisible fields. > > diff --git a/sql/sql_load.cc b/sql/sql_load.cc > index cfa92f1..fbb3926 100644 > --- a/sql/sql_load.cc > +++ b/sql/sql_load.cc > @@ -444,6 +444,8 @@ int mysql_load(THD *thd, const sql_exchange *ex, TABLE_LIST *table_list, > field_iterator.set(table_list); > for (; !field_iterator.end_of_fields(); field_iterator.next()) > { > + if (field_iterator.field()->invisible > VISIBLE) > + continue; Oh, that's all? And the syntax with SET just worked automatically? Cool. ok to push! Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] 326db1a: Mdev-14853 Grant does not work correctly when table contains...
by Sergei Golubchik 07 May '18

07 May '18

Hi, Sachin! On Apr 26, sachin wrote: > revision-id: 326db1a2aaa9b275a1a21a863e8cd2d9fa1b1d5f (mariadb-10.3.6-46-g326db1a) > parent(s): 9477a2a9ba17c0db362e2bb39d5048e369096f39 > author: Sachin Setiya > committer: Sachin Setiya > timestamp: 2018-04-26 12:47:25 +0530 > message: > > Mdev-14853 Grant does not work correctly when table contains... > SYSTEM_INVISIBLE or COMPLETELY_INVISIBLE > > This commit does multiple things to solve this mdev > 1st add field into the parameter of check_column_grant_in_table_ref, so that > we can find out field invisibility. > 2nd If field->invisible >= INVISIBLE_SYSTEM skip access check and simple > grant access. Looks ok. A couple of comments about the test: 1. I suppose now you can use system versioning to test system invisible fields, so there's no need to limit this test to debug-only builds. 2. please test a case when a user has *no* privileges on the table at all. To make sure he cannot select always-readable system invisible fields. and change the capitalization in the commit comment to say MDEV not Mdev - I suspect this was the reason why Jira didn't pick up your commit. Thanks! Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] 5651e72: MDEV-15465 Server crash or ASAN heap-use-after-free in Item_func_match::cleanup upon using FT search with partitioning.
by Sergei Golubchik 07 May '18

07 May '18

Hi, Alexey! On May 07, Alexey Botchkov wrote: > Yes, it is an issue in 5.5. > But we don't always fix bugs in the earliest version possible. > > Ok. Do i understand it right that the patch is ok to push into 5.5? Yes. > Should i then propagate the patch upt to 10.3 muself? No, don't bother. Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] 5651e72: MDEV-15465 Server crash or ASAN heap-use-after-free in Item_func_match::cleanup upon using FT search with partitioning.
by Sergei Golubchik 06 May '18

06 May '18

Hi, Alexey! On May 03, Alexey Botchkov wrote: > revision-id: 5651e72b8dd20e29a4963723773ec6bcfac457ba (mariadb-10.3.6-102-g5651e72) > parent(s): 73a10cbcc5178ae5378abb821428d35d3276b4da > committer: Alexey Botchkov > timestamp: 2018-05-03 12:42:56 +0400 > message: > > MDEV-15465 Server crash or ASAN heap-use-after-free in Item_func_match::cleanup upon using FT search with partitioning. > > The Item_func_match::cleanup() uses table's internals so it's not safe > to call it after the close_thread_tables(). Let's call it in > st_select_lex::cleanup(). Oops. Agree, thanks. But why do you fix it in 10.3? Looks like a 5.5+ bug. Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] cc0bc69: MDEV-15813 ASAN use-after-poison in hp_hashnr upon HANDLER READ on a
by Sergei Golubchik 06 May '18

06 May '18

Hi, Alexey! On May 06, Alexey Botchkov wrote: > revision-id: cc0bc69f2d28aace12487f930677abcdffa5ce97 (mariadb-10.3.6-102-gcc0bc69) > parent(s): 73a10cbcc5178ae5378abb821428d35d3276b4da > committer: Alexey Botchkov > timestamp: 2018-05-06 02:37:20 +0400 > message: > > MDEV-15813 ASAN use-after-poison in hp_hashnr upon HANDLER READ on a > versioned HEAP table. > > hp_rkey now returns an error if we try to use HASH key where it doesn't > work. > > diff --git a/sql/share/errmsg-utf8.txt b/sql/share/errmsg-utf8.txt > index c55ac4f..fcd20d8 100644 > --- a/sql/share/errmsg-utf8.txt > +++ b/sql/share/errmsg-utf8.txt > @@ -7915,3 +7915,5 @@ ER_UPDATED_COLUMN_ONLY_ONCE > eng "The column %`s.%`s cannot be changed more than once in a single UPDATE statement" > ER_EMPTY_ROW_IN_TVC > eng "Row with no elements is not allowed in table value constructor in this context" > +ER_KEY_DOESNT_SUPPORT > + eng "HASH index %`s does not support this operation" Use "%s index %`s does not support this operation" please. First parameter could be taken from handler::index_type(). By the way, check what happens when the index is a fulltext or rtree. Is it also a crash or this case is handled somehow? If it does, the same check should take care of HASH indexes, no need to do a separate solution for them. > diff --git a/storage/heap/hp_rkey.c b/storage/heap/hp_rkey.c > index 692e1a0..a7b81ae 100644 > --- a/storage/heap/hp_rkey.c > +++ b/storage/heap/hp_rkey.c > @@ -60,6 +60,14 @@ int heap_rkey(HP_INFO *info, uchar *record, int inx, const uchar *key, > } > else > { > + /* > + HASH key doesn't support search other than equality, > + and doesn't support prefix search. > + */ > + if (find_flag != HA_READ_KEY_EXACT || > + (1UL << share->keydef[inx].keysegs) != keypart_map + 1) > + DBUG_RETURN(my_errno= HA_ERR_KEY_DOESNT_SUPPORT); > + No, please, not here. The parent handler object knows index types, it knows index flags and it can return an error if the corresponding index flag is missing. No need to push this check down to the engine if the server can do it in, say, ha_index_read. Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] MDEV-15813 ... HANDLER READ on a versioned HEAP table
by Sergei Golubchik 05 May '18

05 May '18

Hi, Alexey! On May 04, Alexey Botchkov wrote: > Hi, Sergei, guys. > > This bug iis more like a feature of the HASH indexes. > And i came upon other 'features' while experimenting. > > Firstly the 'range' queries like > HANDLER h READ `PRIMARY` < (5); > cannot be naturally handled by the HASH keys. Only equality queries > ...'PRIMARY' = (5); make sence. > And these 'range' queries just return wrong empty result. > I'd say we need to fire an error in this case. Do you agree? Yes. Like "HASH index `PRIMARY` does not support this operation" ? > Second problem - the SYSTEM VERSIONING. > It adds an extra column to the key, which makes this key virtually > useless. I think for the VERSIONING we should either replace the HASH > key with the RBTREE, or get the row_end data from the table after > getting records from the HASH. > What is your idea how to deal with this? row_end must be a part of every unique key, because uniqueness of current data should not be affected by historical, deleted, rows. In this particular use case, HANDLER is a low level command, it doesn't see high-level features like versioning. So, for HANDLER row_end is just a normal key part, should be used as HANDLER h READ `PRIMARY` = (5, '2038-01-19 03:14:07.999999'); Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] : GSoC Project Optimizer Trace
by Sergey Petrunia 04 May '18

04 May '18

Hi Zhzhzoo, Please find below some thoughts about the optimizer trace project * The MDEV for the optimizer trace task is https://jira.mariadb.org/browse/MDEV-6111. Let's keep it up to date with all the changes. * I think we should target MySQL-like functionality of optimizer_trace. At least, for the first milestone. We may decide to make some things better but working in a similar way to MySQL is a nice first goal. * MariaDB's optimizer trace will contain JSON and produce similar content to MySQL. However, MariaDB's optimizer is different, so one cannot expect that the JSON document will have exactly the same fields/objects/etc. (MariaDB' EXPLAIN FORMAT=JSON output doesn't match the one in MySQL, either. And in MySQL, the output varies across its versions.) * MySQL prints conditions with heavy over-quoting. Example: "original_condition": "((`A`.`a` = `B`.`a`) and ((`A`.`a` + `B`.`a`) < 10))", MariaDB has the code to print the condition without redundant quotes (see how EXPLAIN FORMAT=JSON does it). * I assume MySQL has some class for printing well-formed JSON documents. MariaDB also has one. It is in sql/my_json_writer.h, see class Json_writer. It handles printing into JSON, and also does pretty-printing. When I looked at the code last time, it did pretty-printing better than MySQL's code for EXPLAIN FORMAT=JSON did. It would be nice to use Json_writer for collecting optimizer trace output, too. (there is a caveat: Json_writer was written specifically for producing EXPLAIN FORMAT=JSON output so it may lack some features. You may need to extend it). BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] MariaDB Developers Unconference in June
by Ian Gilfillan 03 May '18

03 May '18

Hi A reminder about the next MariaDB Developers Unconference, which will be held in Tampera, Finland, from 26 June to 29 June. This will be a four-day event; the first day focusing on new contributors, the 2nd and 3rd the regular Unconference sessions and discussions, and the 4th day on getting patches reviewed and implemented, taking advantage of the chance to have in-person discussions. Details, as well as where to register and how to suggest sessions, are at https://mariadb.org/2018-2-developers-unconference-in-finland/

1 0

Re: [Maria-developers] [External] Re: Questions regarding closing partitions in MDEV-11084
by Mattias Jonsson 30 Apr '18

30 Apr '18

Hi Holyfoot, Sergei and Jacob, Thank you for looking into this. Jacob there are no Jira ticket (that I am aware of) also I do see it as two different problems with the same source. 1) Spider engine does not handle opening/closing specific partitions (as I understand it currently relies on opening/closing all partitions at once in order). Currently this is a crashing bug! 2) MDEV-11084 makes explicit partition selection work like FLUSH TABLE, due to the closing partitions part. I think it would be nice if you could create tickets for those two issues (or at least for the first one). Regards Mattias On Thu, Apr 26, 2018 at 11:26 AM, Alexey Botchkov <holyfoot(a)mariadb.com> wrote: > No, there's no JIRA task for that problem. > So i guess you can create one :) > > Best regards. > HF > > On Thu, Apr 26, 2018 at 4:50 AM, Jacob Mathew <jacob.mathew(a)mariadb.com> > wrote: > >> Hi Mattias and Holyfoot, >> >> Is there a Jira bug for this problem? If not I can create a bug for it >> and assign it to Holyfoot. >> >> Thanks, >> Jacob >> >> Jacob B. Mathew >> Spider and Server Developer >> MariaDB Corporation >> +1 408 655 8999 (mobile) >> jacob.b.mathew (Skype) >> jacob.mathew(a)MariaDB.com >> >> >> On Tue, Apr 17, 2018 at 9:11 AM, Mattias Jonsson < >> mattias.jonsson(a)booking.com> wrote: >> >>> Hi Holyfoot, >>> >>> On Mon, Apr 16, 2018 at 3:38 PM, Alexey Botchkov <holyfoot(a)askmonty.org> >>> wrote: >>> > Hi, Mattias, guys! >>> > >>> > While investigating the crash, i'd like to discuss that >>> > >>> >> it seems to close partitions whenever it >>> >> is not used in a statement (i.e. require it to be reopened in the next >>> >> statement that would use another partition >>> > >>> > >>> > Yes, it does that, handling statements with the specified 'PARTITION' >>> > option. >>> > The patch supposed to solve the problem when there are too many >>> partitions >>> > opened, >>> > so i think it must close the unused partitions sometime. >>> > No, it doesn't have to happen that often. I planned to check the >>> > table_open_cache >>> > variable before the forced close. But decided not to do that initially >>> - as >>> > it simplified testing, >>> > and i thought if someone uses the PARTITION option, he would stick to >>> using >>> > this partition >>> > anyway. And i forgot about that issue. >>> >>> The reason for not closing partitions in this case is that it turns >>> the 'SELECT col FROM t PARTITION(p)' almost into a 'FLUSH TABLES t' >>> which is kind of unexpected. >>> I did a test mixing PK selects with and without 'PARTITION (p)' clause >>> and that shows it will close all but 1 partition and then (re)open all >>> but 1 partition. >>> >>> Think of the case when a server runs in production with a heavily >>> partitioned table serving simple PK queries and then someone runs a >>> query with explicit PARTITION selection, then it will introduce a >>> short stall. First for the query itself (closing all but 1 partitions) >>> and then for the next simple PK query using the same table (opening >>> all but 1 partition). >>> >>> As I read the bug report: the reporter wants to avoid opening all >>> partitions. Not that it keeps the partitions open in the table open >>> cache (which is an issue on the architectural level of partitioning >>> not really fitting into the open table cache). >>> >>> I attached a diff with the test and results (I also added handler >>> status variables to show my point). The diff is against >>> b4a2baffa82e5c07b96a1c752228560dcac1359b. >>> >>> Here is the part of the result file that shows what I mean with extra >>> comments prepended by MJ>: >>> CREATE TABLE t1 (a int PRIMARY KEY) >>> ENGINE = InnoDB >>> PARTITION BY HASH (a) PARTITIONS 1000; >>> INSERT INTO t1 VALUES (0), (1), (2), (3); >>> FLUSH STATUS; >>> >>> SELECT a FROM t1 PARTITION(p0) WHERE a = 0; >>> a >>> 0 >>> SHOW SESSION STATUS >>> WHERE VARIABLE_NAME LIKE 'HANDLER_%' AND VALUE > 0; >>> Variable_name Value >>> MJ> Here it closes all but one partitions. >>> Handler_close 999 >>> >>> SELECT a FROM t1 WHERE a = 0; >>> a >>> 0 >>> SHOW SESSION STATUS >>> WHERE VARIABLE_NAME LIKE 'HANDLER_%' AND VALUE > 0; >>> Variable_name Value >>> Handler_close 1000 >>> Handler_external_lock 2006 >>> MJ> and here it re-opens those 'all but one' partitions. >>> Handler_open 999 >>> >>> >>> Regards >>> Mattias >>> > >>> > >>> > Best regards. >>> > HF >>> > >>> > >>> > >>> > >>> > 13.04.2018 19:07, Mattias Jonsson wrote: >>> >> >>> >> Hi MariaDB Devs, >>> >> >>> >> I tried to evaluate spider engine and found an issue where it crashes, >>> >> most likely due to MDEV-11084 (Stacktrace and reproducible test case >>> >> attached). >>> >> >>> >> That also leads me to wonder about the performance for partitioned >>> >> tables after MDEV-11084, when it seems to close partitions whenever it >>> >> is not used in a statement (i.e. require it to be reopened in the next >>> >> statement that would use another partition, effectively not using the >>> >> open table cache). >>> >> >>> >> I cannot see anything mentioned in the final commit message hinting on >>> >> that it closes partitions not used in the current query, but in the >>> >> previous patches it was mentioned without any reason. >>> >> >>> >> Why does it close the already opened partitions? >>> >> https://github.com/MariaDB/server/blob/10.3/sql/ha_partition.cc#L8365 >>> >> >>> >> I would not mind opening the partitions only when they are to be used >>> >> (even though there are engines that need to be tested more), but >>> >> closing them makes no sense to me performance wise. Also notice that >>> >> the partitions first will be put back into the open table cache and >>> >> then on the next query the non-used partitions will be closed and the >>> >> needed ones be (re)-opened. >>> >> >>> >> Regards >>> >> Mattias Jonsson >>> > >>> > >>> > >>> > >>> >> >> > -- Mattias Jonsson IT Team Lead - Development Booking.com B.V. Herengracht 597 Amsterdam 1017 CE Netherlands Direct +31207125646 [image: Booking.com] <http://www.booking.com/> The world's #1 accommodation site 43 languages, 198+ offices worldwide, 120,000+ global destinations, 1,550,000+ room nights booked every day No booking fees, best price always guaranteed Subsidiary of Booking Holdings Inc. (NASDAQ: BKNG)

3 4

Re: [Maria-developers] [Commits] 326db1a: Mdev-14853 Grant does not work correctly when table contains...
by Vicențiu Ciorbaru 26 Apr '18

26 Apr '18

Hi Sachin! Did you run this test on embedded server? Usually grant related tests require some form of not_embedded.inc include. (check roles suite for examples) Vicentiu On Thu, 26 Apr 2018 at 00:20 sachin <sachin.setiya(a)mariadb.com> wrote: > revision-id: 326db1a2aaa9b275a1a21a863e8cd2d9fa1b1d5f > (mariadb-10.3.6-46-g326db1a) > parent(s): 9477a2a9ba17c0db362e2bb39d5048e369096f39 > author: Sachin Setiya > committer: Sachin Setiya > timestamp: 2018-04-26 12:47:25 +0530 > message: > > Mdev-14853 Grant does not work correctly when table contains... > SYSTEM_INVISIBLE or COMPLETELY_INVISIBLE > > This commit does multiple things to solve this mdev > 1st add field into the parameter of check_column_grant_in_table_ref, so > that > we can find out field invisibility. > 2nd If field->invisible >= INVISIBLE_SYSTEM skip access check and simple > grant access. > > --- > mysql-test/main/invisible_field_grant.result | 111 > +++++++++++++++++++++++++++ > mysql-test/main/invisible_field_grant.test | 77 +++++++++++++++++++ > sql/sp_rcontext.cc | 9 ++- > sql/sql_acl.cc | 11 ++- > sql/sql_acl.h | 2 +- > sql/sql_base.cc | 4 +- > 6 files changed, 206 insertions(+), 8 deletions(-) > > diff --git a/mysql-test/main/invisible_field_grant.result > b/mysql-test/main/invisible_field_grant.result > new file mode 100644 > index 0000000..c3ccbb1 > --- /dev/null > +++ b/mysql-test/main/invisible_field_grant.result > @@ -0,0 +1,111 @@ > +set @old_debug= @@debug_dbug; > +create user user_1; > +show grants for user_1; > +Grants for user_1@% > +GRANT USAGE ON *.* TO 'user_1'@'%' > +# create user > +create database d; > +use d; > + > +#System_Invisible > +set debug_dbug= "+d,test_pseudo_invisible"; > +create table t1(a int); > +set debug_dbug=@old_debug; > +insert into t1 values(1); > +select a,invisible from t1; > +a invisible > +1 9 > +grant insert(a) on t1 to user_1; > +grant update(a) on t1 to user_1; > +grant select(a) on t1 to user_1; > +grant delete on t1 to user_1; > +connect con1, localhost, user_1,,test; > +connection con1; > +select user(); > +user() > +user_1@localhost > +use d; > +select * from t1; > +a > +1 > +insert into t1 values(2); > +select * from t1; > +a > +1 > +2 > +insert into t1(a) values(3); > +select * from t1; > +a > +1 > +2 > +3 > +select invisible,a from t1; > +invisible a > +9 1 > +9 2 > +9 3 > +delete from t1 where a =1; > +update t1 set a=1 where a=3; > +select * from t1; > +a > +2 > +1 > +disconnect con1; > + > +#Cleanup > +connection default; > +drop table t1; > +REVOKE ALL PRIVILEGES, GRANT OPTION FROM user_1; > + > +#Completely Invisible > +set debug_dbug= "+d,test_completely_invisible"; > +create table t1(a int); > +insert into t1 values(1); > +select a,invisible from t1; > +a invisible > +1 9 > +set debug_dbug=@old_debug; > +grant insert(a) on t1 to user_1; > +grant update(a) on t1 to user_1; > +grant select(a) on t1 to user_1; > +grant delete on t1 to user_1; > +connect con1, localhost, user_1,,test; > +connection con1; > +select user(); > +user() > +user_1@localhost > +use d; > +select * from t1; > +a > +1 > +insert into t1 values(2); > +select * from t1; > +a > +1 > +2 > +insert into t1(a) values(3); > +select * from t1; > +a > +1 > +2 > +3 > +select invisible,a from t1; > +ERROR 42S22: Unknown column 'invisible' in 'field list' > +delete from t1 where a =1; > +update t1 set a=1 where a=3; > +select * from t1; > +a > +2 > +1 > +disconnect con1; > + > +#Final Cleanup > +connection default; > +set debug_dbug= "+d,test_completely_invisible"; > +select a,invisible from t1; > +a invisible > +2 9 > +1 9 > +drop user user_1; > +drop database d; > +set @old_debug= @@debug_dbug; > diff --git a/mysql-test/main/invisible_field_grant.test > b/mysql-test/main/invisible_field_grant.test > new file mode 100644 > index 0000000..0d627e5 > --- /dev/null > +++ b/mysql-test/main/invisible_field_grant.test > @@ -0,0 +1,77 @@ > +--source include/have_debug.inc > +##TEST for invisible coloumn level 2 > +set @old_debug= @@debug_dbug; > +create user user_1; > +show grants for user_1; > +--echo # create user > +create database d; > +use d; > + > +--echo > +--echo #System_Invisible > +set debug_dbug= "+d,test_pseudo_invisible"; > +create table t1(a int); > +set debug_dbug=@old_debug; > +insert into t1 values(1); > +select a,invisible from t1; > +grant insert(a) on t1 to user_1; > +grant update(a) on t1 to user_1; > +grant select(a) on t1 to user_1; > +grant delete on t1 to user_1; > +connect (con1, localhost, user_1,,test); > +connection con1; > +select user(); > +use d; > +select * from t1; > +insert into t1 values(2); > +select * from t1; > +insert into t1(a) values(3); > +select * from t1; > +select invisible,a from t1; > +delete from t1 where a =1; > +update t1 set a=1 where a=3; > +select * from t1; > +disconnect con1; > +--source include/wait_until_disconnected.inc > + > +--echo > +--echo #Cleanup > +--connection default > +drop table t1; > +REVOKE ALL PRIVILEGES, GRANT OPTION FROM user_1; > + > +--echo > +--echo #Completely Invisible > +set debug_dbug= "+d,test_completely_invisible"; > +create table t1(a int); > +insert into t1 values(1); > +select a,invisible from t1; > +set debug_dbug=@old_debug; > +grant insert(a) on t1 to user_1; > +grant update(a) on t1 to user_1; > +grant select(a) on t1 to user_1; > +grant delete on t1 to user_1; > +connect (con1, localhost, user_1,,test); > +connection con1; > +select user(); > +use d; > +select * from t1; > +insert into t1 values(2); > +select * from t1; > +insert into t1(a) values(3); > +select * from t1; > +--error ER_BAD_FIELD_ERROR > +select invisible,a from t1; > +delete from t1 where a =1; > +update t1 set a=1 where a=3; > +select * from t1; > +disconnect con1; > +--source include/wait_until_disconnected.inc > +--echo > +--echo #Final Cleanup > +connection default; > +set debug_dbug= "+d,test_completely_invisible"; > +select a,invisible from t1; > +drop user user_1; > +drop database d; > +set @old_debug= @@debug_dbug; > diff --git a/sql/sp_rcontext.cc b/sql/sp_rcontext.cc > index 2e9ae23..dc103fa 100644 > --- a/sql/sp_rcontext.cc > +++ b/sql/sp_rcontext.cc > @@ -196,11 +196,12 @@ bool sp_rcontext::init_var_table(THD *thd, > */ > static inline bool > check_column_grant_for_type_ref(THD *thd, TABLE_LIST *table_list, > - const char *str, size_t length) > + const char *str, size_t length, > + Field *fld) > { > #ifndef NO_EMBEDDED_ACCESS_CHECKS > table_list->table->grant.want_privilege= SELECT_ACL; > - return check_column_grant_in_table_ref(thd, table_list, str, length); > + return check_column_grant_in_table_ref(thd, table_list, str, length, > fld); > #else > return false; > #endif > @@ -238,7 +239,7 @@ bool Qualified_column_ident::resolve_type_ref(THD > *thd, Column_definition *def) > { > if (!(rc= check_column_grant_for_type_ref(thd, table_list, > m_column.str, > - m_column.length))) > + m_column.length, src))) > { > *def= Column_definition(thd, src, NULL/*No defaults,no > constraints*/); > def->flags&= (uint) ~NOT_NULL_FLAG; > @@ -302,7 +303,7 @@ bool Table_ident::resolve_table_rowtype_ref(THD *thd, > LEX_CSTRING tmp= src[0]->field_name; > Spvar_definition *def; > if ((rc= check_column_grant_for_type_ref(thd, table_list, > - tmp.str, tmp.length)) || > + tmp.str, > tmp.length,src[0])) || > (rc= !(src[0]->field_name.str= thd->strmake(tmp.str, > tmp.length))) || > (rc= !(def= new (thd->mem_root) Spvar_definition(thd, *src)))) > break; > diff --git a/sql/sql_acl.cc b/sql/sql_acl.cc > index de4e201..1c154a1 100644 > --- a/sql/sql_acl.cc > +++ b/sql/sql_acl.cc > @@ -7775,6 +7775,8 @@ bool check_grant_column(THD *thd, GRANT_INFO *grant, > table_ref table reference where to check the field > name name of field to check > length length of name > + fld use fld object to check invisibility when it is > + not 0, not_found_field, view_ref_found > > DESCRIPTION > Check the access rights to a column depending on the type of table > @@ -7789,13 +7791,17 @@ bool check_grant_column(THD *thd, GRANT_INFO > *grant, > */ > > bool check_column_grant_in_table_ref(THD *thd, TABLE_LIST * table_ref, > - const char *name, size_t length) > + const char *name, size_t length, > + Field *fld) > { > GRANT_INFO *grant; > const char *db_name; > const char *table_name; > Security_context *sctx= table_ref->security_ctx ? > table_ref->security_ctx : thd->security_ctx; > + if (fld && fld != not_found_field && fld != view_ref_found > + && fld->invisible >= INVISIBLE_SYSTEM) > + return false; > > if (table_ref->view || table_ref->field_translation) > { > @@ -7871,6 +7877,9 @@ bool check_grant_all_columns(THD *thd, ulong > want_access_arg, > > for (; !fields->end_of_fields(); fields->next()) > { > + if (fields->field() && > + fields->field()->invisible >= INVISIBLE_SYSTEM) > + continue; > LEX_CSTRING *field_name= fields->name(); > > if (table_name != fields->get_table_name()) > diff --git a/sql/sql_acl.h b/sql/sql_acl.h > index a608ef0..6da7d4d 100644 > --- a/sql/sql_acl.h > +++ b/sql/sql_acl.h > @@ -239,7 +239,7 @@ bool check_grant_column (THD *thd, GRANT_INFO *grant, > const char *db_name, const char *table_name, > const char *name, size_t length, > Security_context *sctx); > bool check_column_grant_in_table_ref(THD *thd, TABLE_LIST * table_ref, > - const char *name, size_t length); > + const char *name, size_t length, > Field *fld); > bool check_grant_all_columns(THD *thd, ulong want_access, > Field_iterator_table_ref *fields); > bool check_grant_routine(THD *thd, ulong want_access, > diff --git a/sql/sql_base.cc b/sql/sql_base.cc > index 0081365..383341c 100644 > --- a/sql/sql_base.cc > +++ b/sql/sql_base.cc > @@ -5880,7 +5880,7 @@ find_field_in_table_ref(THD *thd, TABLE_LIST > *table_list, > #ifndef NO_EMBEDDED_ACCESS_CHECKS > /* Check if there are sufficient access rights to the found field. */ > if (check_privileges && > - check_column_grant_in_table_ref(thd, *actual_table, name, length)) > + check_column_grant_in_table_ref(thd, *actual_table, name, length, > fld)) > fld= WRONG_GRANT; > else > #endif > @@ -6057,7 +6057,7 @@ find_field_in_tables(THD *thd, Item_ident *item, > #ifndef NO_EMBEDDED_ACCESS_CHECKS > /* Check if there are sufficient access rights to the found field. > */ > if (found && check_privileges && > - check_column_grant_in_table_ref(thd, table_ref, name, length)) > + check_column_grant_in_table_ref(thd, table_ref, name, length, > found)) > found= WRONG_GRANT; > #endif > } > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits

2 1

Re: [Maria-developers] Questions regarding closing partitions in MDEV-11084
by Alexey Botchkov 26 Apr '18

26 Apr '18

Hi, Mattias, guys! While investigating the crash, i'd like to discuss that > it seems to close partitions whenever it > is not used in a statement (i.e. require it to be reopened in the next > statement that would use another partition Yes, it does that, handling statements with the specified 'PARTITION' option. The patch supposed to solve the problem when there are too many partitions opened, so i think it must close the unused partitions sometime. No, it doesn't have to happen that often. I planned to check the table_open_cache variable before the forced close. But decided not to do that initially - as it simplified testing, and i thought if someone uses the PARTITION option, he would stick to using this partition anyway. And i forgot about that issue. Best regards. HF 13.04.2018 19:07, Mattias Jonsson wrote: > Hi MariaDB Devs, > > I tried to evaluate spider engine and found an issue where it crashes, > most likely due to MDEV-11084 (Stacktrace and reproducible test case > attached). > > That also leads me to wonder about the performance for partitioned > tables after MDEV-11084, when it seems to close partitions whenever it > is not used in a statement (i.e. require it to be reopened in the next > statement that would use another partition, effectively not using the > open table cache). > > I cannot see anything mentioned in the final commit message hinting on > that it closes partitions not used in the current query, but in the > previous patches it was mentioned without any reason. > > Why does it close the already opened partitions? > https://github.com/MariaDB/server/blob/10.3/sql/ha_partition.cc#L8365 > > I would not mind opening the partitions only when they are to be used > (even though there are engines that need to be tested more), but > closing them makes no sense to me performance wise. Also notice that > the partitions first will be put back into the open table cache and > then on the next query the non-used partitions will be closed and the > needed ones be (re)-opened. > > Regards > Mattias Jonsson

3 2

Re: [Maria-developers] 331034cb6e1: MDEV-11975: SQLCOM_PREPARE of EXPLAIN & ANALYZE statement do not return correct metadata info
by Sergei Golubchik 25 Apr '18

25 Apr '18

Hi, Oleksandr! On Apr 20, Oleksandr Byelkin wrote: > revision-id: 331034cb6e1552e1ab7c12b6f7341f557f76b807 (mariadb-10.3.6-16-g331034cb6e1) > parent(s): cff60be7fe159fdcb2517ce8441610ad512aa7d0 > author: Oleksandr Byelkin > committer: Oleksandr Byelkin > timestamp: 2018-04-20 14:40:14 +0200 > message: > > MDEV-11975: SQLCOM_PREPARE of EXPLAIN & ANALYZE statement do not return correct metadata info > > Added metadate info after prepare EXPLAIN/ANALYZE. > > diff --git a/sql/sql_class.cc b/sql/sql_class.cc > index 96485ed15ec..0ec95e11829 100644 > --- a/sql/sql_class.cc > +++ b/sql/sql_class.cc > @@ -2659,18 +2659,32 @@ CHANGED_TABLE_LIST* THD::changed_table_dup(const char *key, size_t key_length) > } > > > -int THD::send_explain_fields(select_result *result, uint8 explain_flags, bool is_analyze) > +void THD::prepare_explain_fields(select_result *result, > + List<Item> *field_list, > + uint8 explain_flags, > + bool is_analyze) > { > - List<Item> field_list; > + ; forgot to remove? otherwise, ok to push... Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] MDEV-13695: INTERSECT precedence is not in line with Oracle even in SQL_MODE=Oracle
by Sergei Golubchik 25 Apr '18

25 Apr '18

Hi, Oleksandr! On Apr 24, Oleksandr Byelkin wrote: > revision-id: cb6aade130b75f3dc4b7c3a1dd4d33864adffd90 (mariadb-10.3.6-43-gcb6aade130b) > parent(s): f79c5a658cc33a10d7744a748a4328254e2cbaf7 > author: Oleksandr Byelkin > committer: Oleksandr Byelkin > timestamp: 2018-04-24 12:04:59 +0200 > message: > > MDEV-13695: INTERSECT precedence is not in line with Oracle even in SQL_MODE=Oracle > > Switch off automatic INTERSECT priority for ORACLE MODE ok to push Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] 10.3-MDEV-11953
by Alexander Barkov 25 Apr '18

25 Apr '18

Hello Igor and Sanja, In your branch you stopped to use the global function check_simple_select(). It's not used any more. Please remove it from sql_parse.h and sql_parse.cc. Thanks.

1 0

Re: [Maria-developers] 57673bc: MDEV-14024 PCRE2
by Sergei Golubchik 24 Apr '18

24 Apr '18

Hi, Alexey! On Apr 24, Alexey Botchkov wrote: > > But for mroonga you've just changed the include path. I was > > surprised that it was enough. Perhaps mroonga doesn't use pcre at > > all? > > No, mroonga doesn't use PCRE, just mentions in comfigure scripts. > As far as i can see. At least it compiles fine with my changes. If it doesn't use PCRE, could you remove those lines with the path at all? Would be less confusing. Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] 57673bc: MDEV-14024 PCRE2
by Sergei Golubchik 23 Apr '18

23 Apr '18

Hi, Alexey! On Apr 23, Alexey Botchkov wrote: > > >> diff --git a/sql/item_cmpfunc.h b/sql/item_cmpfunc.h > >> index 3d11a22..d3f52ed 100644 > >... > >> #define PCRE_STATIC 1 /* Important on Windows */ > >> -#include "pcre.h" /* pcre header file */ > >> +#define PCRE2_CODE_UNIT_WIDTH 8 > > > >you sure, you want it here and not, say, in one universally-included > >headers like my_config.h? You don't define it in other places now, so > >they include different function names and macros. > > Since it's the only place where we include the 'pcre2.h', it probably > makes sence to keep all pcre2-related stuff in one place. No, that was precisely my point - you included pcre2.h in many (more than one) files, but defined PCRE2_CODE_UNIT_WIDTH only here. Otherwise I wouldn't have asked :) > Still i moved the #define to the my_config.h. Right, thanks. > >> +if(EXISTS "${MYSQL_SOURCE_DIR}/pcre2") > >> + set(MYSQL_REGEX_INCLUDE_DIR "${MYSQL_SOURCE_DIR}/pcre2/src") > > > >eh, is that enough? > >there were a lot more changes in the server, than just correcting the > >path. > > Do you mean there's more to be done? Not sure i understand right. To make the server use pcre2 you had to change something in the sources, like, API function names, types names, etc. But for mroonga you've just changed the include path. I was surprised that it was enough. Perhaps mroonga doesn't use pcre at all? Otherwise how could've it compiled without you changing function names and types? Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] sql_mode=oracle : Rollback on error
by jerome brauge 23 Apr '18

23 Apr '18

Hello Alexander, I've found one another difference between Mariadb and Oracle : oracle always rollback the active transaction if an error is raised, Maria leave the transaction active. Possible workaround in stored procedure: add an exit handler which do a rollback and resignal the error. What do you think about this ? Regards

4 7

[Maria-developers] Deprecate INT(M) in SHOW?
by Alexander Barkov 23 Apr '18

23 Apr '18

Hello, We have an optional length for integer data types: TINYINT(N) SMALLINT(N) MEDIUMINT(N) INT(N) BIGINT(N) The number given in the parentheses imposes no any limits on the supported range. So for example BIGINT(1) is still a full-featured data type that can store and retrieve any 64-bit number (btw, I find this confusing). If I declare a column without N, it's later displayed in SHOW CREATE TABLE with the default length, so: CREATE TABLE t1 (a INT) becomes CREATE TABLE t1 (a INT(11)); The length is really important only in two cases: 1. In combination with ZEROFILL. 2. When the explicitly specified length is larger than the default length for the data type. For example, INT(40). In this case on cast to string, it's converted to VARCHAR(40) rather than to VARCHAR(11). In all other cases the length does not matter and has no any visible effects. Does anybody see any problems if in 10.4 we fix all metadata statements, such as: - SHOW CREATE TABLE t1 - DESCRIBE t1 - SELECT * FROM INFORMATION_SCHEMA.COLUMNS not to print the default length for non-ZEROFILL columns?

4 5

Re: [Maria-developers] Analysis of the same name of field and name check constraints
by Anel Husakovic 23 Apr '18

23 Apr '18

Hi Vincentiu, Sorry but I have seen now, after sending the mail, that example in this mail is not relevant, please take a look in a example in PR. Regards, Anel On Mon, Apr 23, 2018 at 12:45 PM, Anel Husakovic <anel(a)mariadb.org> wrote: > Hi Vicentiu, > > Regarding the check constraints with *field *and *table *constraint that > have the same name for example: > *`create table t (a int, b check(b>0), constraint b check (b>10));`* > > when new rows are inserted and an error occurs, generated error is not > pointing to exact constraint. For example: > *`insert into t values (-1);`* > ERROR 4025 (23000): CONSTRAINT `b` failed for `test`.`t` > but this means different constraint. > > In order to solve this, here are my opinions: > 1. approach is not to allow this scenario while creating the table, > 2. approach is to generate different message one message for *table *constraint > second for *field *constraint depending where error occurs- and you told > me to go with this one. > > Out of curiosity I also tried with 1. approach problem is that in > *sql/sql_table.cc **mysql_prepare_create_table 4221* > *create/alter_info.check_constraint_list *is tied to table constraints. > However there is another attribute *field_check_constraints *for > *create_info *object but it is always *0*. It is updated in > * sql/unireg.cc* *783*. Problem is that *mysql_prepare_create_table *is > called first and after that *field_check_constraints *is updated,so > there is no mechanism to allow check in early stage for *table*-*field *check > constraint. > After that there is a validation according to the each constraint in > *sql/table.cc* *verify_constraints 5232* - and this is 2. approach. > > I finished 2. approach and what I have that each test already written in > main directory failed because of different error message for a specific > case. > So here is a patch for 2. approach: > https://github.com/MariaDB/server/pull/715 > > In this approach we are handling each error independently. > Also we need to change each test regarding the ER_CONSTRAINT_FAILED where > should be raised newly created error ER_FIELD_CONSTRAINT_FAILED. > > > Also I have updated PR *IS.check_constraint *, rebased everything in one > file > https://github.com/MariaDB/server/pull/689/ > > Best regards, > Anel > > > >

1 0

Re: [Maria-developers] [Commits] a20e1ce85ec: MDEV-13232: Assertion `(&(&share->intern_lock)->m_mutex)->count > 0 && pthread_equal(pthread_self(), (&(&share->intern_lock)->m_mutex)->thread)' failed in _ma_state_info_write
by Alexander Barkov 23 Apr '18

23 Apr '18

Hi Sanja, the patch is OK to push. Thanks. On 04/23/2018 12:37 PM, Oleksandr Byelkin wrote: > revision-id: a20e1ce85ec56616fa048e9ba39c25c65b098fce (mariadb-10.3.6-36-ga20e1ce85ec) > parent(s): c6ba758d1d41c11466b8f9b61b4546efc95aa689 > author: Oleksandr Byelkin > committer: Oleksandr Byelkin > timestamp: 2018-04-23 10:35:33 +0200 > message: > > MDEV-13232: Assertion `(&(&share->intern_lock)->m_mutex)->count > 0 && pthread_equal(pthread_self(), (&(&share->intern_lock)->m_mutex)->thread)' failed in _ma_state_info_write > > Limit length of result of "negative" operation to something reasonable > > --- > mysql-test/main/union.result | 14 ++++++++++++++ > mysql-test/main/union.test | 15 +++++++++++++++ > sql/item_func.cc | 3 +++ > 3 files changed, 32 insertions(+) > > diff --git a/mysql-test/main/union.result b/mysql-test/main/union.result > index 4e5f9312e03..39cd68ae851 100644 > --- a/mysql-test/main/union.result > +++ b/mysql-test/main/union.result > @@ -2500,5 +2500,19 @@ t2 CREATE TABLE `t2` ( > DROP TABLE t2; > DROP TABLE t1; > # > +# MDEV-13232: Assertion `(&(&share->intern_lock)->m_mutex)->count > 0 && > +# pthread_equal(pthread_self(), (&(&share->intern_lock)->m_mutex)-> > +# thread)' failed in _ma_state_info_write > +# > +CREATE TABLE t1 (c1 CHAR(8)); > +INSERT INTO t1 VALUES ('10'),('-10'); > +CREATE TABLE t2 (c2 CHAR); > +SET @a= CAST('10' AS CHAR); > +SELECT c1 FROM t1 UNION SELECT - @a FROM t2; > +c1 > +10 > +-10 > +drop table t1,t2; > +# > # End of 10.3 tests > # > diff --git a/mysql-test/main/union.test b/mysql-test/main/union.test > index f86cae87524..b9b38271f83 100644 > --- a/mysql-test/main/union.test > +++ b/mysql-test/main/union.test > @@ -1745,6 +1745,21 @@ SHOW CREATE TABLE t2; > DROP TABLE t2; > DROP TABLE t1; > > +--echo # > +--echo # MDEV-13232: Assertion `(&(&share->intern_lock)->m_mutex)->count > 0 && > +--echo # pthread_equal(pthread_self(), (&(&share->intern_lock)->m_mutex)-> > +--echo # thread)' failed in _ma_state_info_write > +--echo # > + > +CREATE TABLE t1 (c1 CHAR(8)); > +INSERT INTO t1 VALUES ('10'),('-10'); > + > +CREATE TABLE t2 (c2 CHAR); > +SET @a= CAST('10' AS CHAR); > + > +SELECT c1 FROM t1 UNION SELECT - @a FROM t2; > + > +drop table t1,t2; > > --echo # > --echo # End of 10.3 tests > diff --git a/sql/item_func.cc b/sql/item_func.cc > index e2740272385..ae6c61e83f4 100644 > --- a/sql/item_func.cc > +++ b/sql/item_func.cc > @@ -1935,6 +1935,9 @@ void Item_func_neg::fix_length_and_dec_double() > set_handler(&type_handler_double); > decimals= args[0]->decimals; // Preserve NOT_FIXED_DEC > max_length= args[0]->max_length + 1; > + // Limit length with something reasonable > + uint32 mlen= type_handler()->max_display_length(this); > + set_if_smaller(max_length, mlen); > unsigned_flag= false; > } > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits >

1 0

Re: [Maria-developers] ce222a7: MDEV-15915 Add Feature_json status variable.
by Sergei Golubchik 23 Apr '18

23 Apr '18

Hi, Alexey! On Apr 23, Alexey Botchkov wrote: > > Elena commented in MDEV about "Feature_json" being possibly confusing. > I've seen Elena's comment. Can also add GEOJSON functions to her list. > Still i can't figure out anything better than Feature_json as the name > for this variable. > The self-explaining name is either too long or possible misleading. Feature_json_functions ? But I think Feature_json is fine too, one hardly thinks of "EXPLAIN JSON" when talking about "JSON support". Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] ce222a7: MDEV-15915 Add Feature_json status variable.
by Sergei Golubchik 22 Apr '18

22 Apr '18

Hi, Alexey! On Apr 22, Alexey Botchkov wrote: > revision-id: ce222a71bf75b2125cc588371a3debb3e4a22d7f (mariadb-10.3.6-34-gce222a7) > parent(s): 9f84451d87ba1924539b7d2ab69619b71d0155b9 > committer: Alexey Botchkov > timestamp: 2018-04-22 16:01:18 +0400 > message: > > MDEV-15915 Add Feature_json status variable. > > --- > mysql-test/main/status.test | 7 +++++++ > sql/item_create.cc | 26 ++++++++++++++++++++++++++ > sql/mysqld.cc | 1 + > sql/sql_class.h | 1 + > 4 files changed, 35 insertions(+) > > diff --git a/sql/item_create.cc b/sql/item_create.cc > index 901dfa0..8b8476b 100644 > --- a/sql/item_create.cc > +++ b/sql/item_create.cc > @@ -5118,6 +5118,7 @@ Create_func_json_exists Create_func_json_exists::s_singleton; > Item* > Create_func_json_exists::create_2_arg(THD *thd, Item *arg1, Item *arg2) > { > + status_var_increment(current_thd->status_var.feature_json); > return new (thd->mem_root) Item_func_json_exists(thd, arg1, arg2); I believe you've added that to all JSON functions, I didn't try to double-check that :) > } > > diff --git a/sql/mysqld.cc b/sql/mysqld.cc > index 6e1a599..ae14c80 100644 > --- a/sql/mysqld.cc > +++ b/sql/mysqld.cc > @@ -8553,6 +8553,7 @@ SHOW_VAR status_vars[]= { > {"Feature_fulltext", (char*) offsetof(STATUS_VAR, feature_fulltext), SHOW_LONG_STATUS}, > {"Feature_gis", (char*) offsetof(STATUS_VAR, feature_gis), SHOW_LONG_STATUS}, > {"Feature_invisible_columns", (char*) offsetof(STATUS_VAR, feature_invisible_columns), SHOW_LONG_STATUS}, > + {"Feature_json", (char*) offsetof(STATUS_VAR, feature_json), SHOW_LONG_STATUS}, > {"Feature_locale", (char*) offsetof(STATUS_VAR, feature_locale), SHOW_LONG_STATUS}, > {"Feature_subquery", (char*) offsetof(STATUS_VAR, feature_subquery), SHOW_LONG_STATUS}, > {"Feature_timezone", (char*) offsetof(STATUS_VAR, feature_timezone), SHOW_LONG_STATUS}, Elena commented in MDEV about "Feature_json" being possibly confusing. Does your using "Feature_json" name mean that * you considered it and intentionally kept the old name, thinking it's fine or * you didn't see the comment, and now you might reconsider? Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] 67260fd: MDEV-15130 Assertion `table->s->null_bytes == 0' failed in ...
by Sergei Golubchik 20 Apr '18

20 Apr '18

Hi, Sachin! On Apr 19, sachin wrote: > diff --git a/storage/perfschema/ha_perfschema.cc b/storage/perfschema/ha_perfschema.cc > index e8e9581..d5846f2 100644 > --- a/storage/perfschema/ha_perfschema.cc > +++ b/storage/perfschema/ha_perfschema.cc > @@ -64,6 +64,16 @@ static int pfs_discover_table(handlerton *hton, THD *thd, TABLE_SHARE *share) > { > const PFS_engine_table_share *pfs_share; > > + DBUG_EXECUTE_IF("test_pseudo_invisible", { > + my_error(ER_INTERNAL_ERROR, MYF(0), "Don't query performance_schema when" > + "test_pseudo_invisible is on"); > + return 1; > + }); > + DBUG_EXECUTE_IF("test_completely_invisible", { > + my_error(ER_INTERNAL_ERROR, MYF(0), "Don't query performance_schema when" > + "test_completely_invisible is on"); > + return 1; > + }); > if ((pfs_share= find_table_share(share->db.str, share->table_name.str))) > return share->init_from_sql_statement_string(thd, false, > pfs_share->sql.str, Better put it inside TABLE_SHARE::init_from_sql_statement_string to cover Connect and FederatedX too. Or - I'd say it's an option too - just close it as not a bug. DBUG_EXECUTE_IF can cause a crash, can make a server to misbehave, etc. That's normal, it's a debug hack that intentionally makes server to do something weird. I wouldn't spend too much time on making it user friendly and error-proof. Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

2 1

Re: [Maria-developers] 346c8ab9533: MDEV-11975: SQLCOM_PREPARE of EXPLAIN & ANALYZE statement do not return correct metadata info
by Sergei Golubchik 20 Apr '18

20 Apr '18

Hi, Oleksandr! On Apr 18, Oleksandr Byelkin wrote: > revision-id: 346c8ab9533a3de6a4cb348428402ffee2aa8da2 (mariadb-10.3.6-16-g346c8ab9533) > parent(s): cff60be7fe159fdcb2517ce8441610ad512aa7d0 > author: Oleksandr Byelkin > committer: Oleksandr Byelkin > timestamp: 2018-04-18 19:34:12 +0200 > message: > > MDEV-11975: SQLCOM_PREPARE of EXPLAIN & ANALYZE statement do not return correct metadata info > > Added metadate info after prepare EXPLAIN/ANALYZE. > > --- > mysql-test/main/mysql_client_test.result | 122 ++++++++++++ > mysql-test/main/mysql_client_test.test | 7 +- > sql/sql_class.cc | 18 +- > sql/sql_class.h | 3 +- > sql/sql_explain.cc | 3 +- > sql/sql_parse.cc | 5 +- > sql/sql_prepare.cc | 24 ++- > sql/sql_prepare.h | 2 + > tests/mysql_client_test.c | 323 ++++++++++++++++++++++++++++++- > 9 files changed, 493 insertions(+), 14 deletions(-) > > diff --git a/mysql-test/main/mysql_client_test.result b/mysql-test/main/mysql_client_test.result > index 83ef8d442b3..20385acfa00 100644 > --- a/mysql-test/main/mysql_client_test.result > +++ b/mysql-test/main/mysql_client_test.result > @@ -122,5 +122,127 @@ EOF > mysql_stmt_next_result(): 0; field_count: 0 > # ------------------------------------ > > +# cat MYSQL_TMP_DIR/test_explain_meta.out.log > +# ------------------------------------ okay, altought I'd rather add prepare metadata output to mysqltest. Like, either print prepare metadata on --enable_metadata or introduce a new command --enable_prepare_metadata > +SELECT number of fields: 1 > +EXPALIN number of fields: 10 > + - 0: name: 'id'/''; table: ''/''; db: ''; catalog: 'def'; length: 3; max_length: 0; type: 8; decimals: 0 > + - 1: name: 'select_type'/''; table: ''/''; db: ''; catalog: 'def'; length: 57; max_length: 0; type: 253; decimals: 39 > + - 2: name: 'table'/''; table: ''/''; db: ''; catalog: 'def'; length: 192; max_length: 0; type: 253; decimals: 39 > + - 3: name: 'type'/''; table: ''/''; db: ''; catalog: 'def'; length: 30; max_length: 0; type: 253; decimals: 39 > + - 4: name: 'possible_keys'/''; table: ''/''; db: ''; catalog: 'def'; length: 12288; max_length: 0; type: 253; decimals: 39 > + - 5: name: 'key'/''; table: ''/''; db: ''; catalog: 'def'; length: 192; max_length: 0; type: 253; decimals: 39 > + - 6: name: 'key_len'/''; table: ''/''; db: ''; catalog: 'def'; length: 12288; max_length: 0; type: 253; decimals: 39 > + - 7: name: 'ref'/''; table: ''/''; db: ''; catalog: 'def'; length: 6144; max_length: 0; type: 253; decimals: 39 > + - 8: name: 'rows'/''; table: ''/''; db: ''; catalog: 'def'; length: 10; max_length: 0; type: 8; decimals: 0 > + - 9: name: 'Extra'/''; table: ''/''; db: ''; catalog: 'def'; length: 765; max_length: 0; type: 253; decimals: 39 > +EXPALIN JSON number of fields: 1 > + - 0: name: 'EXPLAIN'/''; table: ''/''; db: ''; catalog: 'def'; length: 234; max_length: 0; type: 253; decimals: 39 > +ANALYZE number of fields: 13 > + - 0: name: 'id'/''; table: ''/''; db: ''; catalog: 'def'; length: 3; max_length: 0; type: 8; decimals: 0 > + - 1: name: 'select_type'/''; table: ''/''; db: ''; catalog: 'def'; length: 57; max_length: 0; type: 253; decimals: 39 > + - 2: name: 'table'/''; table: ''/''; db: ''; catalog: 'def'; length: 192; max_length: 0; type: 253; decimals: 39 > + - 3: name: 'type'/''; table: ''/''; db: ''; catalog: 'def'; length: 30; max_length: 0; type: 253; decimals: 39 > + - 4: name: 'possible_keys'/''; table: ''/''; db: ''; catalog: 'def'; length: 12288; max_length: 0; type: 253; decimals: 39 > + - 5: name: 'key'/''; table: ''/''; db: ''; catalog: 'def'; length: 192; max_length: 0; type: 253; decimals: 39 > + - 6: name: 'key_len'/''; table: ''/''; db: ''; catalog: 'def'; length: 12288; max_length: 0; type: 253; decimals: 39 > + - 7: name: 'ref'/''; table: ''/''; db: ''; catalog: 'def'; length: 6144; max_length: 0; type: 253; decimals: 39 > + - 8: name: 'rows'/''; table: ''/''; db: ''; catalog: 'def'; length: 10; max_length: 0; type: 8; decimals: 0 > + - 9: name: 'r_rows'/''; table: ''/''; db: ''; catalog: 'def'; length: 4; max_length: 0; type: 5; decimals: 10 > + - 10: name: 'filtered'/''; table: ''/''; db: ''; catalog: 'def'; length: 4; max_length: 0; type: 5; decimals: 2 > + - 11: name: 'r_filtered'/''; table: ''/''; db: ''; catalog: 'def'; length: 4; max_length: 0; type: 5; decimals: 2 > + - 12: name: 'Extra'/''; table: ''/''; db: ''; catalog: 'def'; length: 765; max_length: 0; type: 253; decimals: 39 > +ANALYZE JSON number of fields: 1 > + - 0: name: 'ANALYZE'/''; table: ''/''; db: ''; catalog: 'def'; length: 234; max_length: 0; type: 253; decimals: 39 > +EXPALIN INSERT number of fields: 10 > + - 0: name: 'id'/''; table: ''/''; db: ''; catalog: 'def'; length: 3; max_length: 0; type: 8; decimals: 0 > + - 1: name: 'select_type'/''; table: ''/''; db: ''; catalog: 'def'; length: 57; max_length: 0; type: 253; decimals: 39 > + - 2: name: 'table'/''; table: ''/''; db: ''; catalog: 'def'; length: 192; max_length: 0; type: 253; decimals: 39 > + - 3: name: 'type'/''; table: ''/''; db: ''; catalog: 'def'; length: 30; max_length: 0; type: 253; decimals: 39 > + - 4: name: 'possible_keys'/''; table: ''/''; db: ''; catalog: 'def'; length: 12288; max_length: 0; type: 253; decimals: 39 > + - 5: name: 'key'/''; table: ''/''; db: ''; catalog: 'def'; length: 192; max_length: 0; type: 253; decimals: 39 > + - 6: name: 'key_len'/''; table: ''/''; db: ''; catalog: 'def'; length: 12288; max_length: 0; type: 253; decimals: 39 > + - 7: name: 'ref'/''; table: ''/''; db: ''; catalog: 'def'; length: 6144; max_length: 0; type: 253; decimals: 39 > + - 8: name: 'rows'/''; table: ''/''; db: ''; catalog: 'def'; length: 10; max_length: 0; type: 8; decimals: 0 > + - 9: name: 'Extra'/''; table: ''/''; db: ''; catalog: 'def'; length: 765; max_length: 0; type: 253; decimals: 39 > +EXPALIN JSON INSERT number of fields: 1 > + - 0: name: 'EXPLAIN'/''; table: ''/''; db: ''; catalog: 'def'; length: 234; max_length: 0; type: 253; decimals: 39 > +ANALYZE INSERT number of fields: 13 > + - 0: name: 'id'/''; table: ''/''; db: ''; catalog: 'def'; length: 3; max_length: 0; type: 8; decimals: 0 > + - 1: name: 'select_type'/''; table: ''/''; db: ''; catalog: 'def'; length: 57; max_length: 0; type: 253; decimals: 39 > + - 2: name: 'table'/''; table: ''/''; db: ''; catalog: 'def'; length: 192; max_length: 0; type: 253; decimals: 39 > + - 3: name: 'type'/''; table: ''/''; db: ''; catalog: 'def'; length: 30; max_length: 0; type: 253; decimals: 39 > + - 4: name: 'possible_keys'/''; table: ''/''; db: ''; catalog: 'def'; length: 12288; max_length: 0; type: 253; decimals: 39 > + - 5: name: 'key'/''; table: ''/''; db: ''; catalog: 'def'; length: 192; max_length: 0; type: 253; decimals: 39 > + - 6: name: 'key_len'/''; table: ''/''; db: ''; catalog: 'def'; length: 12288; max_length: 0; type: 253; decimals: 39 > + - 7: name: 'ref'/''; table: ''/''; db: ''; catalog: 'def'; length: 6144; max_length: 0; type: 253; decimals: 39 > + - 8: name: 'rows'/''; table: ''/''; db: ''; catalog: 'def'; length: 10; max_length: 0; type: 8; decimals: 0 > + - 9: name: 'r_rows'/''; table: ''/''; db: ''; catalog: 'def'; length: 4; max_length: 0; type: 5; decimals: 10 > + - 10: name: 'filtered'/''; table: ''/''; db: ''; catalog: 'def'; length: 4; max_length: 0; type: 5; decimals: 2 > + - 11: name: 'r_filtered'/''; table: ''/''; db: ''; catalog: 'def'; length: 4; max_length: 0; type: 5; decimals: 2 > + - 12: name: 'Extra'/''; table: ''/''; db: ''; catalog: 'def'; length: 765; max_length: 0; type: 253; decimals: 39 > +ANALYZE JSON INSERT number of fields: 1 > + - 0: name: 'ANALYZE'/''; table: ''/''; db: ''; catalog: 'def'; length: 234; max_length: 0; type: 253; decimals: 39 > +EXPALIN UPDATE number of fields: 10 > + - 0: name: 'id'/''; table: ''/''; db: ''; catalog: 'def'; length: 3; max_length: 0; type: 8; decimals: 0 > + - 1: name: 'select_type'/''; table: ''/''; db: ''; catalog: 'def'; length: 57; max_length: 0; type: 253; decimals: 39 > + - 2: name: 'table'/''; table: ''/''; db: ''; catalog: 'def'; length: 192; max_length: 0; type: 253; decimals: 39 > + - 3: name: 'type'/''; table: ''/''; db: ''; catalog: 'def'; length: 30; max_length: 0; type: 253; decimals: 39 > + - 4: name: 'possible_keys'/''; table: ''/''; db: ''; catalog: 'def'; length: 12288; max_length: 0; type: 253; decimals: 39 > + - 5: name: 'key'/''; table: ''/''; db: ''; catalog: 'def'; length: 192; max_length: 0; type: 253; decimals: 39 > + - 6: name: 'key_len'/''; table: ''/''; db: ''; catalog: 'def'; length: 12288; max_length: 0; type: 253; decimals: 39 > + - 7: name: 'ref'/''; table: ''/''; db: ''; catalog: 'def'; length: 6144; max_length: 0; type: 253; decimals: 39 > + - 8: name: 'rows'/''; table: ''/''; db: ''; catalog: 'def'; length: 10; max_length: 0; type: 8; decimals: 0 > + - 9: name: 'Extra'/''; table: ''/''; db: ''; catalog: 'def'; length: 765; max_length: 0; type: 253; decimals: 39 > +EXPALIN JSON UPDATE number of fields: 1 > + - 0: name: 'EXPLAIN'/''; table: ''/''; db: ''; catalog: 'def'; length: 234; max_length: 0; type: 253; decimals: 39 > +ANALYZE UPDATE number of fields: 13 > + - 0: name: 'id'/''; table: ''/''; db: ''; catalog: 'def'; length: 3; max_length: 0; type: 8; decimals: 0 > + - 1: name: 'select_type'/''; table: ''/''; db: ''; catalog: 'def'; length: 57; max_length: 0; type: 253; decimals: 39 > + - 2: name: 'table'/''; table: ''/''; db: ''; catalog: 'def'; length: 192; max_length: 0; type: 253; decimals: 39 > + - 3: name: 'type'/''; table: ''/''; db: ''; catalog: 'def'; length: 30; max_length: 0; type: 253; decimals: 39 > + - 4: name: 'possible_keys'/''; table: ''/''; db: ''; catalog: 'def'; length: 12288; max_length: 0; type: 253; decimals: 39 > + - 5: name: 'key'/''; table: ''/''; db: ''; catalog: 'def'; length: 192; max_length: 0; type: 253; decimals: 39 > + - 6: name: 'key_len'/''; table: ''/''; db: ''; catalog: 'def'; length: 12288; max_length: 0; type: 253; decimals: 39 > + - 7: name: 'ref'/''; table: ''/''; db: ''; catalog: 'def'; length: 6144; max_length: 0; type: 253; decimals: 39 > + - 8: name: 'rows'/''; table: ''/''; db: ''; catalog: 'def'; length: 10; max_length: 0; type: 8; decimals: 0 > + - 9: name: 'r_rows'/''; table: ''/''; db: ''; catalog: 'def'; length: 4; max_length: 0; type: 5; decimals: 10 > + - 10: name: 'filtered'/''; table: ''/''; db: ''; catalog: 'def'; length: 4; max_length: 0; type: 5; decimals: 2 > + - 11: name: 'r_filtered'/''; table: ''/''; db: ''; catalog: 'def'; length: 4; max_length: 0; type: 5; decimals: 2 > + - 12: name: 'Extra'/''; table: ''/''; db: ''; catalog: 'def'; length: 765; max_length: 0; type: 253; decimals: 39 > +ANALYZE JSON UPDATE number of fields: 1 > + - 0: name: 'ANALYZE'/''; table: ''/''; db: ''; catalog: 'def'; length: 234; max_length: 0; type: 253; decimals: 39 > +EXPALIN DELETE number of fields: 10 > + - 0: name: 'id'/''; table: ''/''; db: ''; catalog: 'def'; length: 3; max_length: 0; type: 8; decimals: 0 > + - 1: name: 'select_type'/''; table: ''/''; db: ''; catalog: 'def'; length: 57; max_length: 0; type: 253; decimals: 39 > + - 2: name: 'table'/''; table: ''/''; db: ''; catalog: 'def'; length: 192; max_length: 0; type: 253; decimals: 39 > + - 3: name: 'type'/''; table: ''/''; db: ''; catalog: 'def'; length: 30; max_length: 0; type: 253; decimals: 39 > + - 4: name: 'possible_keys'/''; table: ''/''; db: ''; catalog: 'def'; length: 12288; max_length: 0; type: 253; decimals: 39 > + - 5: name: 'key'/''; table: ''/''; db: ''; catalog: 'def'; length: 192; max_length: 0; type: 253; decimals: 39 > + - 6: name: 'key_len'/''; table: ''/''; db: ''; catalog: 'def'; length: 12288; max_length: 0; type: 253; decimals: 39 > + - 7: name: 'ref'/''; table: ''/''; db: ''; catalog: 'def'; length: 6144; max_length: 0; type: 253; decimals: 39 > + - 8: name: 'rows'/''; table: ''/''; db: ''; catalog: 'def'; length: 10; max_length: 0; type: 8; decimals: 0 > + - 9: name: 'Extra'/''; table: ''/''; db: ''; catalog: 'def'; length: 765; max_length: 0; type: 253; decimals: 39 > +EXPALIN JSON DELETE number of fields: 1 > + - 0: name: 'EXPLAIN'/''; table: ''/''; db: ''; catalog: 'def'; length: 234; max_length: 0; type: 253; decimals: 39 > +ANALYZE DELETE number of fields: 13 > + - 0: name: 'id'/''; table: ''/''; db: ''; catalog: 'def'; length: 3; max_length: 0; type: 8; decimals: 0 > + - 1: name: 'select_type'/''; table: ''/''; db: ''; catalog: 'def'; length: 57; max_length: 0; type: 253; decimals: 39 > + - 2: name: 'table'/''; table: ''/''; db: ''; catalog: 'def'; length: 192; max_length: 0; type: 253; decimals: 39 > + - 3: name: 'type'/''; table: ''/''; db: ''; catalog: 'def'; length: 30; max_length: 0; type: 253; decimals: 39 > + - 4: name: 'possible_keys'/''; table: ''/''; db: ''; catalog: 'def'; length: 12288; max_length: 0; type: 253; decimals: 39 > + - 5: name: 'key'/''; table: ''/''; db: ''; catalog: 'def'; length: 192; max_length: 0; type: 253; decimals: 39 > + - 6: name: 'key_len'/''; table: ''/''; db: ''; catalog: 'def'; length: 12288; max_length: 0; type: 253; decimals: 39 > + - 7: name: 'ref'/''; table: ''/''; db: ''; catalog: 'def'; length: 6144; max_length: 0; type: 253; decimals: 39 > + - 8: name: 'rows'/''; table: ''/''; db: ''; catalog: 'def'; length: 10; max_length: 0; type: 8; decimals: 0 > + - 9: name: 'r_rows'/''; table: ''/''; db: ''; catalog: 'def'; length: 4; max_length: 0; type: 5; decimals: 10 > + - 10: name: 'filtered'/''; table: ''/''; db: ''; catalog: 'def'; length: 4; max_length: 0; type: 5; decimals: 2 > + - 11: name: 'r_filtered'/''; table: ''/''; db: ''; catalog: 'def'; length: 4; max_length: 0; type: 5; decimals: 2 > + - 12: name: 'Extra'/''; table: ''/''; db: ''; catalog: 'def'; length: 765; max_length: 0; type: 253; decimals: 39 > +ANALYZE JSON DELETE number of fields: 1 > + - 0: name: 'ANALYZE'/''; table: ''/''; db: ''; catalog: 'def'; length: 234; max_length: 0; type: 253; decimals: 39 > +# ------------------------------------ > + > + > SET @@global.general_log= @old_general_log; > SET @@global.slow_query_log= @old_slow_query_log; > diff --git a/sql/sql_class.cc b/sql/sql_class.cc > index 96485ed15ec..349ec576264 100644 > --- a/sql/sql_class.cc > +++ b/sql/sql_class.cc > @@ -2659,18 +2660,27 @@ CHANGED_TABLE_LIST* THD::changed_table_dup(const char *key, size_t key_length) > } > > > -int THD::send_explain_fields(select_result *result, uint8 explain_flags, bool is_analyze) > +int THD::send_explain_fields(select_result *result, > + uint8 explain_flags, > + bool is_analyze, > + Prepared_statement *stmt) > { > List<Item> field_list; > + int rc; > if (lex->explain_json) > make_explain_json_field_list(field_list, is_analyze); > else > make_explain_field_list(field_list, explain_flags, is_analyze); > > result->prepare(field_list, NULL); > - return (result->send_result_set_metadata(field_list, > - Protocol::SEND_NUM_ROWS | > - Protocol::SEND_EOF)); > + if (stmt) > + rc= send_prep_stmt(stmt, result->field_count(field_list)) || > + result->send_result_set_metadata(field_list, Protocol::SEND_EOF); > + else > + rc= result->send_result_set_metadata(field_list, > + Protocol::SEND_NUM_ROWS | > + Protocol::SEND_EOF); > + return(rc); > } dunno, I'd rather keep prepare functionality in sql_prepare.cc. Like, void THD::prepare_explain_fields(select_result *result, List<Item> *field_list, uint8 explain_flags, bool is_analyze) { if (lex->explain_json) make_explain_json_field_list(field_list, is_analyze); else make_explain_field_list(field_list, explain_flags, is_analyze); result->prepare(field_list, NULL); } int THD::send_explain_fields(select_result *result, uint8 explain_flags, bool is_analyze) { List<Item> field_list; prepare_explain_fields(result, field_list, explain_flags, is_analyze); return result->send_result_set_metadata(field_list, Protocol::SEND_NUM_ROWS | Protocol::SEND_EOF); } and in sql_prepare.cc you do List<Item> field_list; thd->prepare_explain_fields(result, field_list, explain_flags, is_analyze); return send_prep_stmt(stmt, result->field_count(field_list)) || result->send_result_set_metadata(field_list, Protocol::SEND_EOF); Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

2 3

Re: [Maria-developers] [Commits] 602865f9f24: MDEV-13727 rpl.rpl_concurrency_error failed
by Alexander Barkov 20 Apr '18

20 Apr '18

Hello Andrei, In LOGGER::slow_log_print() there is a piece of this code: query_utime= (current_utime - thd->start_utime); lock_utime= (thd->utime_after_lock - thd->start_utime); my_hrtime_t current_time= { hrtime_from_time(thd->start_time) + thd->start_time_sec_part + query_utime }; If we don't reset thd->start_utime in the beginning of sp_instr_stmt::execute(), then this code will calculate something useless. It seems that this function: bool LOGGER::slow_log_print( THD *thd, const char *query, size_t query_length, ulonglong current_utime) should not use thd->start_time, thd->start_utime, thd->start_time_sec_part. and should get this information from the argument: bool LOGGER::slow_log_print( THD *thd, const char *query, size_t query_length, const QUERY_START_TIME_INFO *start, ulonglong current_utime) And sp_instr_stmt::execute() should not do thd->set_time(). The whole logic should reside in sql_parse.cc. 1. Remember the current time 2. Execute the statement 3. Log the query if it was slow, using the time remembered at step 1. Is my understanding correct? Thanks. On 04/19/2018 03:42 PM, andrei.elkin(a)pp.inet.fi wrote: > revision-id: 602865f9f242a7ed5e3679af60a78698c03d93f5 (mariadb-10.3.6-14-g602865f9f24) > parent(s): 02e897ca57168059c6a74d1129158afea93aa875 > author: Andrei Elkin > committer: Andrei Elkin > timestamp: 2018-04-18 13:04:46 +0300 > message: > > MDEV-13727 rpl.rpl_concurrency_error failed > > The test actually revealed a flaw in MDEV-8305 > which inadvertently enrolled the trigger and > stored function into slow query reporting aimed exclusively > to the stored procedure. > Specifically to the test, a query on the master was logged > with a timestamp of the query's top-level statement but its (post > update) trigger computed one more (later) timestamp which got > inserted into another table. > > Master-vs-slave discrepancy became evident thanks to > different execution time of the trigger combined with the fact of the > logged master timestamp in the micro-second fraction format was > truncated on the slave. The latter means the slave execution of the query > always started earlier than on the master as far as the timestamp was concerned. > And when the fractional part was close to 1 the 2nd timestamp acquired by > MDEV-8305 piece of logics could overflow to next integer value. > That's how the master table appeared to be temporarly behind the slave's one. > > Fixed with restoring the pre-MDEV-8305 behavior not to reset > the current statement's timestamp when the query is not from a stored procedure. > That is the trigger's or stored function's queries continue to use > the top-level statement's timestamp as they are supposed to. > > --- > mysql-test/main/func_time.result | 26 ++++++++++++++++++ > mysql-test/main/func_time.test | 59 ++++++++++++++++++++++++++++++++++++++++ > sql/sp_head.cc | 4 +-- > 3 files changed, 87 insertions(+), 2 deletions(-) > > diff --git a/mysql-test/main/func_time.result b/mysql-test/main/func_time.result > index 2772f850ce9..9babfc6a48b 100644 > --- a/mysql-test/main/func_time.result > +++ b/mysql-test/main/func_time.result > @@ -3484,3 +3484,29 @@ t1 CREATE TABLE `t1` ( > `c5` varchar(100) DEFAULT NULL > ) ENGINE=MyISAM DEFAULT CHARSET=latin1 > DROP TABLE t1; > +CREATE TABLE t_ts (a timestamp(6)); > +CREATE TABLE t_trig (a timestamp(6)); > +CREATE FUNCTION fn_sleep_before_now() returns int > +BEGIN > +INSERT INTO t_ts SET a= current_timestamp(6); > +RETURN 0; > +END// > +CREATE TRIGGER trg_insert_t_ts after INSERT on t_ts for each row > +BEGIN > +INSERT into t_trig set a= current_timestamp(6); > +END// > +SET @sav_slow_query_log= @@session.slow_query_log; > +SET @@session.slow_query_log= ON; > +SELECT current_timestamp(6),fn_sleep_before_now() INTO @ts_cur, @ts_func; > +SELECT a FROM t_ts LIMIT 1 into @ts_func; > +SELECT a FROM t_trig LIMIT 1 into @ts_trig; > +DELETE FROM t_ts; > +DELETE FROM t_trig; > +SET @@session.slow_query_log= OFF; > +SELECT current_timestamp(6),fn_sleep_before_now() INTO @ts_cur, @func_ts; > +SELECT a FROM t_ts LIMIT 1 into @ts_func; > +SELECT a FROM t_trig LIMIT 1 into @ts_trig; > +SET @@session.slow_query_log= @sav_slow_query_log; > +DROP FUNCTION fn_sleep_before_now; > +DROP TRIGGER trg_insert_t_ts; > +DROP TABLE t_ts, t_trig; > diff --git a/mysql-test/main/func_time.test b/mysql-test/main/func_time.test > index 5417cb20a92..ad207218969 100644 > --- a/mysql-test/main/func_time.test > +++ b/mysql-test/main/func_time.test > @@ -2057,3 +2057,62 @@ EXECUTE IMMEDIATE > USING NULL, '10', 10, 10.0, 10e0, TIME'10:20:30'; > SHOW CREATE TABLE t1; > DROP TABLE t1; > + > + > +############ > +# MDEV-13727 > +# Current timestamp functions inside stored functions must return the > +# value of the top-level statement's timestamp (its start time). > +# This must hold regardless of @@slow_query_log option. > +# > + > +CREATE TABLE t_ts (a timestamp(6)); > +CREATE TABLE t_trig (a timestamp(6)); > +delimiter //; > +CREATE FUNCTION fn_sleep_before_now() returns int > +BEGIN > + INSERT INTO t_ts SET a= current_timestamp(6); > + RETURN 0; > +END// > +CREATE TRIGGER trg_insert_t_ts after INSERT on t_ts for each row > +BEGIN > + INSERT into t_trig set a= current_timestamp(6); > +END// > +delimiter ;// > + > +SET @sav_slow_query_log= @@session.slow_query_log; > + > +# @@slow_query_log ON check > +SET @@session.slow_query_log= ON; > +SELECT current_timestamp(6),fn_sleep_before_now() INTO @ts_cur, @ts_func; > + > +SELECT a FROM t_ts LIMIT 1 into @ts_func; > +SELECT a FROM t_trig LIMIT 1 into @ts_trig; > +if (!`SELECT @ts_cur = @ts_func and @ts_func = @ts_trig`) > +{ > + SELECT @ts_cur, @ts_func, @ts_trig; > + --die Error: timestamps must be equal but they diverge > +} > +DELETE FROM t_ts; > +DELETE FROM t_trig; > + > +# @@slow_query_log OFF check > +SET @@session.slow_query_log= OFF; > +SELECT current_timestamp(6),fn_sleep_before_now() INTO @ts_cur, @func_ts; > +SELECT a FROM t_ts LIMIT 1 into @ts_func; > +SELECT a FROM t_trig LIMIT 1 into @ts_trig; > +if (!`SELECT @ts_cur = @ts_func and @ts_func = @ts_trig`) > +{ > + SELECT @ts_cur, @ts_func, @ts_trig; > + --die Error: timestamps must be equal but they diverge > +} > + > +# Cleanup > +SET @@session.slow_query_log= @sav_slow_query_log; > +DROP FUNCTION fn_sleep_before_now; > +DROP TRIGGER trg_insert_t_ts; > +DROP TABLE t_ts, t_trig; > + > +# > +# End of MDEV-13727 > +################### > diff --git a/sql/sp_head.cc b/sql/sp_head.cc > index f7847bae89d..90fe77e6158 100644 > --- a/sql/sp_head.cc > +++ b/sql/sp_head.cc > @@ -3471,7 +3471,7 @@ sp_instr_stmt::execute(THD *thd, uint *nextp) > thd->profiling.set_query_source(m_query.str, m_query.length); > #endif > > - if ((save_enable_slow_log= thd->enable_slow_log)) > + if ((save_enable_slow_log= thd->enable_slow_log) && !thd->in_sub_stmt) > { > /* > Save start time info for the CALL statement and overwrite it with the > @@ -3544,7 +3544,7 @@ sp_instr_stmt::execute(THD *thd, uint *nextp) > } > } > /* Restore the original query start time */ > - if (thd->enable_slow_log) > + if (thd->enable_slow_log && !thd->in_sub_stmt) > thd->restore_query_start_time(&time_info); > > DBUG_RETURN(res || thd->is_error()); > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits >

1 0

Re: [Maria-developers] Mdev-10664 Add statuses about optimistic parallel replication stalls.
by Sachin Setiya 19 Apr '18

19 Apr '18

Hi Kristian! Sorry for late reply , I was on vacations. On Tue, Jan 10, 2017 at 7:31 PM, Kristian Nielsen <knielsen(a)knielsen-hq.org> wrote: > Sachin Setiya <sachin.setiya(a)mariadb.com> writes: > > >> Right, but you still need _some_ way to synchronise multiple threads > > > I used atomic builtins my_atomic_add64_explicit. > > That seems fine (but see comment below). > > > Documentation:- > > > > Slave_DDL_Events:- > > > > Description: Number of DDL statements slave have executed in > > replication channel X. > > Scope: Per Replication Channel > > Data Type: numeric > > Introduced: MariaDB 10.1 > > > > Slave_Non_Transactional_Events:- > > > > Description: Number of Non Transactional "no idea what should be the > > next word " slave have executed in replication channel X. > > Scope: Per Replication Channel > > Data Type: numeric > > Introduced: MariaDB 10.1 > > I checked, the term "event group" is used already multiple places in the > documentation, so I think it can be used here also. An event group can be a > transaction (possibly with multiple transactional DML statements), or a DDL > or non-transactional statement. > > What you are doing is introducing status variables to help understand > optimistic (and aggressive) parallel replication modes. > > In optimistic parallel replication, a non-transactional event group is not > safe to run in parallel with other non-transactional events. So that event > is made to wait for all prior event groups to reach commit state, before > being allowed to execute. However, other transactional events are not > delayed by the non-transactional event group. > > Slave_Non_Transactional_Events counts the occurences of this. > > Then in case of a DDL statement, this can potentially be unsafe to run in > parallel even with other transactional events. So for a DDL, the DDL has to > wait for all prior event groups, _and_ subsequent event groups have to wait > for the DDL. > > Slave_DDL_Events counts these occurences. > > This is what the documentation, and commit messages, should explain, I > think. > > I think the status variables describe the values for > @@default_master_connection?, this should also be explained in the > documentation. > > Why did you decide to put this information into a status variable? > Normally, > slave status is seen in SHOW SLAVE STATUS, which supports showing status > for > a particular connection without using @@default_master_connection. > > Sorry for this , I guess It was a confusion I was looking at Slave_Running this is avaliable in show status as well as show slave status. So I thought for if I want to show some information in SHOW slave status this has to be in show status. And this is wrong. Now we can see non trans events / ddl events in show slave status. Show I remove this vars from Show Status, or it is okay ? > > diff --git a/sql/log_event.h b/sql/log_event.h > > index 90900f6..579607f 100644 > > --- a/sql/log_event.h > > +++ b/sql/log_event.h > > @@ -50,6 +50,7 @@ > > #endif > > > > #include "rpl_gtid.h" > > +#include "my_atomic.h" > > Why include my_atomic.h in the header file, when you only use it in the .cc > file? > Changed. > > > + mysql_mutex_lock(&LOCK_active_mi); > > + if (master_info_index) > > + { > > + mi= master_info_index-> > > + get_master_info(&thd->variables.default_master_connection, > > + Sql_condition::WARN_LEVEL_NOTE); > > + if (mi) > > + tmp= mi->total_ddl_events; > > > + if (mi) > > + tmp= mi->total_non_trans_events; > > Better use an atomic primitive here to read the value. Taking a mutex does > not help synchronise against my_atomic_add64_explicit(). > > Changed. > > + } > > + mysql_mutex_unlock(&LOCK_active_mi); > > + if (mi) > > + *((longlong *)buff)= tmp; > > + else > > + var->type= SHOW_UNDEF; > > + > > + return 0; > > +} > > #endif /* HAVE_REPLICATION */ > > > > static int show_open_tables(THD *thd, SHOW_VAR *var, char *buff, > > > index 9365c06..a474aa5 100644 > > --- a/sql/rpl_mi.h > > +++ b/sql/rpl_mi.h > > @@ -303,6 +303,12 @@ class Master_info : public > Slave_reporting_capability > > > > /* The parallel replication mode. */ > > enum_slave_parallel_mode parallel_mode; > > + > > + /* No of DDL statements */ > > + long total_ddl_events; > > + > > + /* No of non-transactional statements*/ > > + long total_non_trans_events; > > "long" is not the appropriate type here. Certainly not since you are using > my_atomic_add64_explicit(). Use a type that is the same size on all > platforms (and why use a signed type?). > > Changed to uint64. > - Kristian. > Attached a newer version of patch , please review it. -- Regards Sachin

3 7

[Maria-developers] CREATE SEQUENCE is coming
by Michael Widenius 18 Apr '18

18 Apr '18

Hi! For those that are trying to port applications to MariaDB from other databases or need more features than the current AUTO_INCREMENT provides, the following may be of interest: First version of sequence is pushed to bb-10.2-sequence. It should be merged to bb-10.2-compatilbity and then to 10.3 shortly (1-2 weeks). What is working for the moment: CREATE OR REPLACE [TEMPORARY] SEQUENCE [IF NOT EXISTS] name [ INCREMENT [ BY | = ] increment ] [ MINVALUE [=] minvalue | NO MINVALUE ] [ MAXVALUE [=] maxvalue | NO MAXVALUE ] [ START [ WITH | = ] start ] [ CACHE [=] cache ] [ [ NO ] CYCLE ] ENGINE=xxx COMMENT=".." SELECT NEXT VALUE FOR sequence_name; SELECT NEXTVAL(sequence_name); SELECT PREVIOUS VALUE FOR sequence_name; SELECT LASTVAL(sequence_name); SHOW CREATE SEQUENCE sequence_name; SHOW CREATE TABLE sequence_name; CREATE TABLE sequence-structure ... SEQUENCE=1 ALTER TABLE sequence RENAME TO sequence2; RENAME TABLE sequence RENAME TO sequence2; DROP [TEMPORARY] SEQUENCE sequence_name [IF EXISTS] See https://jira.mariadb.org/browse/MDEV-10139 for progress. See commit messages in bb-10.2-sequence for what is still to be done. Some documentation can be found at: https://mariadb.com/kb/en/mariadb/create-sequence/ This will be improved as the work progress. One of the goals with the SEQUENCE implementation is that all old tools, like mysqldump, should work unchanged, while still keeping normal usage of sequence standard compatibly. The make this possible, the sequence is currently implemented as a table with a few exclusive properties. The main disadvantage of having sequence as a table is that it uses the same name space as tables. In other words, you can't have a table and a sequence of the same name. The benefit is that sequences shows up in 'show tables', one can create a sequence also with 'create table' and drop it with 'drop table'. One can select from it as from any other table. This ensures that all old tools that works with tables should work with sequences. The special properties for sequence tables are: - A sequence table has always one row. - When one creates a sequence, either with CREATE TABLE or CREATE SEQUENCE, one row will be inserted. - Normal table options works for CREATE SEQUENCE. One can use ENGINE=xxx, COMMENT=xxx etc. - If one tries to insert into a sequence table, the single row will be updated. This allows mysqldump to work but also gives the additional benefit that one can change all properties of a sequence with a single insert. New applications can of course also use ALTER SEQUENCE. - Updates to the sequence table will change the single row. - Doing a select on the sequence shows the current state of the sequence, except the values that are reserved in the cache. The column 'next_value' shows the next value not reserved by the cache. - Truncate on a sequence table will give an error. - Alter table and rename works on sequences. - If one creates a sequence with INCREMENT 0, then the sequence will use auto_increment_increment and auto_increment_offset for the sequence, just like AUTO_INCREMENT. This allows sequences to work reliable in a master-master environment and with Galera. Internally sequence tables are created as a normal table without rollback (InnoDB, Aria and MySAM supports this) with is wrapped by a sequence engine. This allowed me to create sequences with almost no performance impact for normal tables. (The cost is one 'if' per insert if binary log is enabled). MariaDB 10.3 will support both the ANSI SQL and Oracle syntax for creating and accessing sequences. As ANSI SQL doesn't have an easy access to the last generated value, 103 also supports 'PREVIOUS VALUE FOR sequence_name', like IBM DB2, and LASTVAL(sequence_name) as PostgreSQL. I want to thank Jianwe Zhao from Aliyun for his work on SEQUENCE in AliSQL which gave me ideas and inspiration for this work. I also want to thank Marko Mäkelä for his help in the InnoDB part of the code. There is still a lot of work to fix edge cases, but in the current implementation most major things seams to work... Comments, suggestions or questions? Regards, Monty

2 1

Re: [Maria-developers] 5e94fe1: MDEV-15754 Server crashes in fill_record_n_invoke_before_triggers upon
by Sergei Golubchik 17 Apr '18

17 Apr '18

Hi, Sachin! Looks good, thanks! Ok to push. On Apr 16, sachin wrote: > revision-id: 5e94fe189d6325028613b2c830082e07ff303acb (mariadb-10.3.5-156-g5e94fe1) > parent(s): f0e4f94c230326a2f2e608e4119530d775c37b21 > author: Sachin Setiya > committer: Sachin Setiya > timestamp: 2018-04-16 16:27:11 +0530 > message: > > MDEV-15754 Server crashes in fill_record_n_invoke_before_triggers upon ... > insert into table with TIMESTAMP INVISIBLE > > Problem:- The segfault occurs because value is null but since timestamp field > is VISIBLE it expects a value , and it tries to call value->save_in_field(.. > Timestamp field should not be visible this is the problem. > > Solution:- While we clone field for record0_field we don't honor the field > _visibility , this patch changes that. > > diff --git a/sql/sql_trigger.cc b/sql/sql_trigger.cc > index 4c1e2a5..d523779 100644 > --- a/sql/sql_trigger.cc > +++ b/sql/sql_trigger.cc > @@ -1240,6 +1240,7 @@ bool Table_triggers_list::prepare_record_accessors(TABLE *table) > return 1; > > f->flags= (*fld)->flags; > + f->invisible= (*fld)->invisible; > f->null_ptr= null_ptr; > f->null_bit= null_bit; > if (null_bit == 128) Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] Questions about AS OF (possibly bugs found)
by Alexander Barkov 16 Apr '18

16 Apr '18

Hi Sergei, I'm looking at this use rule in sql_yacc.yy: history_point: temporal_literal { $$= Vers_history_point(VERS_TIMESTAMP, $1); } | function_call_keyword_timestamp { $$= Vers_history_point(VERS_TIMESTAMP, $1); } | opt_history_unit simple_expr { $$= Vers_history_point($1, $2); } ; AS OF DATE 'xxx' - returns Vers_history_point(VERS_TIMESTAMP) AS OF DATE('xxx') - returns Vers_history_point(VERS_UNDEFINED) AS OF TIMESTAMP('2001-01-01') - returns Vers_history_point(VERS_TIMESTAMP) AS OF COALESCE(TIMESTAMP('2001-01-01')) - returns Vers_history_point(VERS_UNDEFINED) Perhaps this is not expected. Also, this code looks suspicious: void Vers_history_point::resolve_unit(bool timestamps_only) { if (item && unit == VERS_UNDEFINED) { if (item->type() == Item::FIELD_ITEM || timestamps_only) unit= VERS_TIMESTAMP; else if (item->result_type() == INT_RESULT || item->result_type() == REAL_RESULT) unit= VERS_TRX_ID; else unit= VERS_TIMESTAMP; } } Why DECIMAL_RESULT is not handled? Why only Item::FIELD_ITEM is checked? Can't "history_point" be simplified to something like: history_point: simple_expr { $$= Vers_history_point(VERS_UNDEFINED, $1); } | TRANSACTION_SYM simple_expr { $$= Vers_history_point(VERS_TRX_ID, $2); } ; I'm saying "something like" because TRANSACTION_SYM will cause a conflict with simple_expr. So probably it should be cought somehow else, by catching Item_ident and checking it name. Btw, an SP variable with name "TRANSACTION" does not work well: This script works fine: DELIMITER $$ BEGIN NOT ATOMIC DECLARE TRANSACTION int DEFAULT 10; SELECT TRANSACTION; END; $$ DELIMITER ; This script: DELIMITER $$ BEGIN NOT ATOMIC DECLARE TRANSACTION int DEFAULT 10; SELECT * FROM t1 FOR SYSTEM_TIME AS OF TRANSACTION; END; $$ DELIMITER ; returns an error: ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '; Note, if I rename "TRANSACTION" to "TRANSACTION1", it works fine. Thanks!

3 8

[Maria-developers] innobase/sync0policy.h converts opaque pthread_t to ULINT
by Teodor Mircea Ionita 16 Apr '18

16 Apr '18

Hi, While trying to build on macOS 10.13.4 with clang 9.1.0 I originally came across this error: -- .../storage/innobase/include/sync0policy.h:79:4: error: cannot initialize a parameter of type 'int64' (aka 'long long') with an rvalue of type 'os_thread_id_t' (aka '_opaque_pthread_t *') my_atomic_storelint(&m_thread_id, os_thread_get_curr_id()); --- This problem was originally reported in MDEV-15778. Since pthread_t on macOS is a pointer to a struct named _opaque_pthread_t, the "obvious" solution for this was to explicitly cast to ULINT which in turn is defined as size_t, fine so far. Solution was tested in PR#691. However, upon testing my patch with buildbot, on slave kvm-fulltest2, which is 64bit qemu vm with Ubuntu precise-i386 (32bit) running and gcc compiling in -march=i686. The above "fix" now yields: -- /home/buildbot/buildbot/build/mariadb-10.3.6/storage/innobase/include/sync0policy.h:79:4: error: invalid cast from type ‘os_thread_id_t {aka long unsigned int}’ to type ‘ulint {aka unsigned int}’ --- Now reconsidering more carefully, this looks more serious then initially thought, posix pthread_t(3) is defined as: -- POSIX.1 allows an implementation wide freedom in choosing the type used to represent a thread ID; for example, representation using either an arithmetic type or a structure is permitted. Therefore, variables of type pthread_t can't portably be compared using the C equality operator (==); use pthread_equal(3)instead. --- So in this particular platform spec, where pthread_t on Linux is defined as unsigned long int integral type and if LP64 model is used, this implies pthread_t always being 64bit in either both 32 and 64 arch and trying to convert to size_t on a 32bit arch will fail. Now given that the standard particularly states pthread_t as being opaque and no assumptions should be made in regards to the actual type, not even an initializer type (which ULINT_UNDEFINED defines as -1), trying to cast m_thread_id to any other type ( (ULINT) ) either implicitly or explicit is not portable. Is this a bug that needs fixing or just impossible to handle pthread id atomically in any other fashion? How about a test case I can run to confirm/infirm erratic behavior? Since I don't have access to the particular bb slave instance to check with sizeof() maybe somebody else can help me here to confirm this? Relevant links bellow: https://jira.mariadb.org/browse/MDEV-15778 https://buildbot.askmonty.org/buildbot/builders/kvm-fulltest2/builds/12317/… https://linux.die.net/man/3/pthread_self https://github.com/MariaDB/server/pull/691 Cheers, Teodor

4 5

Re: [Maria-developers] a91fdd1: Fixed the bug mdev-15575 different results when using CTE and big_tables=1.
by Sergei Golubchik 16 Apr '18

16 Apr '18

Hi, Igor On Apr 14, IgorBabaev wrote: > revision-id: a91fdd1288b3cbc5616f2ef9e0ce04bf7f0ef9ce (mariadb-10.2.14-35-ga91fdd1) > parent(s): 740fc2ae084f8f81990de557d696aefbc507752d > author: Igor Babaev > committer: Igor Babaev > timestamp: 2018-04-14 21:32:41 -0700 > message: > > Fixed the bug mdev-15575 different results when using CTE and big_tables=1. please, make the first line MDEV-15575 different results when using CTE and big_tables=1 most other developers use that pattern, so your commits really stand out in the history, e.g.: 4c7a1a1 MDEV-15780 : mariabackup does not handle absolute names in for system tablespaces 9124590 MDEV-12903: encryption.innodb_encryption_discard_import fails in buildbot with FOUND vs NOT FOUND dd12779 MDEV-15832 With innodb_fast_shutdown=3, skip the rollback of connected transactions 740fc2a Fixed mdev-15765 BETWEEN not working in certain cases 8334ace MDEV-14380 Reduce possibility to timing-induced error in test and the reason for that is that many tools show only first 50 characters of the commit summary (first line), so starting it with the "Fixed the bug " does not add new information, but only pushes the relevant content to the right to be truncated. > This bug happened due to a defect of the implementation of the handler > function ha_delete_all_rows() for the ARIA engine. > The function maria_delete_all_rows() truncated the table, but it didn't > touch the write cache, so the cache's write offset was not reset. > In the scenario like in the function st_select_lex_unit::exec_recursive > when first all records were deleted from the table and then several new > records were added some metadata became inconsistent with the state of > the cache. As a result the table scan function could not read records > at the end of the table. > The same defect could be found in the implementation of ha_delete_all_rows() > for the MYISAM engine mi_delete_all_rows(). This is ok to push > Additionally made late instantiation for the temporary table used to store > rows that were used for each new iteration when executing a recursive CTE. I didn't review this part. Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] Questions regarding closing partitions in MDEV-11084
by Mattias Jonsson 16 Apr '18

16 Apr '18

Hi MariaDB Devs, I tried to evaluate spider engine and found an issue where it crashes, most likely due to MDEV-11084 (Stacktrace and reproducible test case attached). That also leads me to wonder about the performance for partitioned tables after MDEV-11084, when it seems to close partitions whenever it is not used in a statement (i.e. require it to be reopened in the next statement that would use another partition, effectively not using the open table cache). I cannot see anything mentioned in the final commit message hinting on that it closes partitions not used in the current query, but in the previous patches it was mentioned without any reason. Why does it close the already opened partitions? https://github.com/MariaDB/server/blob/10.3/sql/ha_partition.cc#L8365 I would not mind opening the partitions only when they are to be used (even though there are engines that need to be tested more), but closing them makes no sense to me performance wise. Also notice that the partitions first will be put back into the open table cache and then on the next query the non-used partitions will be closed and the needed ones be (re)-opened. Regards Mattias Jonsson

1 0

Re: [Maria-developers] Fwd: [Commits] b32031991e4: initial oracle parser fix
by Alexander Barkov 16 Apr '18

16 Apr '18

Hello Sanja, I reviewed your recent changes in "10.3-MDEV-11953" (and the attached additional patch for sql_yacc_ora.yy) I have some proposals: 1. Can you please move huge pieces of the code from sql_yacc.yy to LEX or other relevant classes? It makes the grammar much more readable (patches are aslo much more readable). I'd move the relevant pieces of the code to LEX as a separate patch, even before fixing the grammar. 2. You're adding too many main_select_push() and pop_select(). Please move them to upper level rules (it should be possible in many cases). Add new helper rules when needed. For example, this piece of code repeats many times: + { + if (Lex->main_select_push()) + MYSQL_YYABORT; + } + expr + { + Lex->pop_select(); //main select + $$= $3; It deserved a rule, say, expr_with_select_push_pop. You can find a better name :) - Serg and I spent a lot of time working on this task: MDEV-8909 union parser cleanup (and its 13 dependency tasks, and 3 related tasks, see the "Issue links" section in MDEV). We think that it should be the parser who disallows bad grammar, instead of post-analysis with raising errors like ER_CANT_USE_OPTION_HERE. Please keep using the same approach. Thanks! On 04/06/2018 07:37 PM, Oleksandr Byelkin wrote: > > > > -------- Weitergeleitete Nachricht -------- > Betreff: [Commits] b32031991e4: initial oracle parser fix > Datum: Fri, 06 Apr 2018 17:34:31 +0200 > Von: Oleksandr Byelkin <sanja(a)mariadb.com> > Antwort an: maria-developers(a)lists.launchpad.net > An: commits(a)mariadb.org > > > > revision-id: b32031991e4ba270f5c221f7f6068732d09efd53 > (mariadb-10.3.5-110-gb32031991e4) > parent(s): 60a502eb7efbe87bd8e3af3e3ece49a9eb83f8b3 > author: Oleksandr Byelkin > committer: Oleksandr Byelkin > timestamp: 2018-04-06 17:05:34 +0200 > message: > > initial oracle parser fix > > --- > sql/sql_yacc.yy | 8 +- > sql/sql_yacc_ora.yy | 2085 > +++++++++++++++++++++++++++++---------------------- > 2 files changed, 1183 insertions(+), 910 deletions(-) > > diff --git a/sql/sql_yacc.yy b/sql/sql_yacc.yy > index e4cbff7114e..9372269b05d 100644 > --- a/sql/sql_yacc.yy > +++ b/sql/sql_yacc.yy > @@ -904,7 +904,7 @@ bool my_yyoverflow(short **a, YYSTYPE **b, size_t > *yystacksize); > %parse-param { THD *thd } > %lex-param { THD *thd } > /* > - Currently there are 96 shift/reduce conflicts. > + Currently there are 135 shift/reduce conflicts. > We should not introduce new conflicts any more. > */ > %expect 135 > @@ -9244,7 +9244,7 @@ table_value_constructor: > lex->push_select(sel)) > MYSQL_YYABORT; > sel->init_select(); > - sel->braces= FALSE; > + sel->braces= FALSE; // just initialisation > } > values_list > { > @@ -13763,8 +13763,10 @@ single_multi: > } > | table_wild_list > { > + /* XXX > if (Lex->main_select_push()) > MYSQL_YYABORT; > + */ > mysql_init_multi_delete(Lex); > YYPS->m_lock_type= TL_READ_DEFAULT; > YYPS->m_mdl_type= MDL_SHARED_READ; > @@ -13779,8 +13781,10 @@ single_multi: > } > | FROM table_alias_ref_list > { > + /* XXX > if (Lex->main_select_push()) > MYSQL_YYABORT; > + */ > mysql_init_multi_delete(Lex); > YYPS->m_lock_type= TL_READ_DEFAULT; > YYPS->m_mdl_type= MDL_SHARED_READ; > diff --git a/sql/sql_yacc_ora.yy b/sql/sql_yacc_ora.yy > index c1512434d72..4a3ee348f89 100644 > --- a/sql/sql_yacc_ora.yy > +++ b/sql/sql_yacc_ora.yy > @@ -186,6 +186,20 @@ void ORAerror(THD *thd, const char *s) > LEX_CSTRING name; > uint offset; > } sp_cursor_name_and_offset; > + struct > + { > + enum sub_select_type unit_type; > + bool distinct; > + } unit_operation; > + struct > + { > + SELECT_LEX *first; > + SELECT_LEX *prev_last; > + } select_list; > + SQL_I_List<ORDER> *select_order; > + Lex_select_lock select_lock; > + Lex_select_limit select_limit; > + Lex_order_limit_lock *order_limit_lock; > > /* pointers */ > Create_field *create_field; > @@ -231,6 +245,7 @@ void ORAerror(THD *thd, const char *s) > > handlerton *db_type; > st_select_lex *select_lex; > + st_select_lex_unit *select_lex_unit; > struct p_elem_val *p_elem_value; > class Window_frame *window_frame; > class Window_frame_bound *window_frame_bound; > @@ -240,7 +255,6 @@ void ORAerror(THD *thd, const char *s) > /* enums */ > enum enum_sp_suid_behaviour sp_suid; > enum enum_view_suid view_suid; > - enum sub_select_type unit_type; > enum Condition_information_item::Name cond_info_item_name; > enum enum_diag_condition_item_name diag_condition_item_name; > enum Diagnostics_information::Which_area diag_area; > @@ -277,10 +291,10 @@ bool my_yyoverflow(short **a, YYSTYPE **b, size_t > *yystacksize); > %parse-param { THD *thd } > %lex-param { THD *thd } > /* > - Currently there are 104 shift/reduce conflicts. > + Currently there are 99 shift/reduce conflicts. > We should not introduce new conflicts any more. > */ > -%expect 104 > +%expect 99 > > /* > Comments for TOKENS. > @@ -599,6 +613,9 @@ bool my_yyoverflow(short **a, YYSTYPE **b, size_t > *yystacksize); > %token LEAVES > %token LEAVE_SYM > %token LEFT /* SQL-2003-R */ > +%token LEFT_PAREN_ALT /* INTERNAL */ > +%token LEFT_PAREN_WITH /* INTERNAL */ > +%token LEFT_PAREN_LIKE /* INTERNAL */ > %token LESS_SYM > %token LEVEL_SYM > %token LEX_HOSTNAME > @@ -1063,7 +1080,8 @@ bool my_yyoverflow(short **a, YYSTYPE **b, size_t > *yystacksize); > NCHAR_STRING > > %type <lex_str_ptr> > - opt_table_alias > + opt_table_alias_clause > + table_alias_clause > > %type <lex_string_with_pos> > ident ident_with_tok_start > @@ -1111,7 +1129,7 @@ bool my_yyoverflow(short **a, YYSTYPE **b, size_t > *yystacksize); > opt_temporary all_or_any opt_distinct opt_glimit_clause > opt_ignore_leaves fulltext_options union_option > opt_not > - select_derived_init transaction_access_mode_types > + transaction_access_mode_types > opt_natural_language_mode opt_query_expansion > opt_ev_status opt_ev_on_completion ev_on_completion opt_ev_comment > ev_alter_on_schedule_completion opt_ev_rename_to opt_ev_sql_stmt > @@ -1231,9 +1249,8 @@ bool my_yyoverflow(short **a, YYSTYPE **b, size_t > *yystacksize); > join_table_list join_table > table_factor table_ref esc_table_ref > table_primary_ident table_primary_derived > - select_derived derived_table_list > - select_derived_union > - derived_query_specification > + derived_table_list table_reference_list_parens > + nested_table_reference_list join_table_parens > %type <date_time_type> date_time_type; > %type <interval> interval > > @@ -1276,12 +1293,16 @@ bool my_yyoverflow(short **a, YYSTYPE **b, > size_t *yystacksize); > UNDERSCORE_CHARSET > > %type <select_lex> subselect > - get_select_lex get_select_lex_derived > query_specification > - query_term_union_not_ready > - query_term_union_ready > + table_value_constructor > + simple_table > + query_primary > + query_primary_parens > + > +%type <select_lex_unit> > query_expression_body > - select_paren_derived > + query_expression > + query_expression_unit > > %type <boolfunc2creator> comp_op > > @@ -1293,7 +1314,21 @@ bool my_yyoverflow(short **a, YYSTYPE **b, size_t > *yystacksize); > > %type <virtual_column> opt_check_constraint check_constraint > virtual_column_func > column_default_expr > -%type <unit_type> unit_type_decl > + > +%type <unit_operation> unit_type_decl > + > +%type <select_lock> > + opt_select_lock_type > + select_lock_type > + opt_lock_wait_timeout_new > + > +%type <select_limit> opt_limit_clause limit_clause limit_options > + > +%type <order_limit_lock> > + query_expression_tail > + order_or_limit > + > +%type <select_order> opt_order_clause order_clause order_list > > %type <NONE> > analyze_stmt_command > @@ -1313,7 +1348,7 @@ bool my_yyoverflow(short **a, YYSTYPE **b, size_t > *yystacksize); > assign_to_keycache_parts > preload_list preload_list_or_parts preload_keys preload_keys_parts > select_item_list select_item values_list no_braces > - opt_limit_clause delete_limit_clause fields opt_values values > + delete_limit_clause fields opt_values values > procedure_list procedure_list2 procedure_item > field_def handler opt_generated_always > opt_ignore opt_column opt_restrict > @@ -1333,9 +1368,8 @@ bool my_yyoverflow(short **a, YYSTYPE **b, size_t > *yystacksize); > table_to_table_list table_to_table opt_table_list opt_as > handler_rkey_function handler_read_or_scan > single_multi table_wild_list table_wild_one opt_wild > - union_clause union_list > - subselect_start opt_and charset > - subselect_end select_var_list select_var_list_init help > + opt_and charset > + select_var_list select_var_list_init help > opt_extended_describe shutdown > opt_format_json > prepare prepare_src execute deallocate > @@ -1606,14 +1640,22 @@ deallocate_or_drop: > ; > > prepare: > - PREPARE_SYM ident FROM prepare_src > + PREPARE_SYM > + { > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > + } > + ident FROM prepare_src > { > LEX *lex= thd->lex; > if (lex->table_or_sp_used()) > my_yyabort_error((ER_SUBQUERIES_NOT_SUPPORTED, MYF(0), > "PREPARE..FROM")); > + if (Lex->check_main_unit_semantics()) > + MYSQL_YYABORT; > lex->sql_command= SQLCOM_PREPARE; > - lex->prepared_stmt_name= $2; > + lex->prepared_stmt_name= $3; > + Lex->pop_select(); //main select > } > ; > > @@ -1632,10 +1674,21 @@ execute: > LEX *lex= thd->lex; > lex->sql_command= SQLCOM_EXECUTE; > lex->prepared_stmt_name= $2; > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > } > execute_using > - {} > - | EXECUTE_SYM IMMEDIATE_SYM prepare_src > + { > + Lex->pop_select(); //main select > + if (Lex->check_main_unit_semantics()) > + MYSQL_YYABORT; > + } > + | EXECUTE_SYM IMMEDIATE_SYM > + { > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > + } > + prepare_src > { > if (Lex->table_or_sp_used()) > my_yyabort_error((ER_SUBQUERIES_NOT_SUPPORTED, MYF(0), > @@ -1643,7 +1696,11 @@ execute: > Lex->sql_command= SQLCOM_EXECUTE_IMMEDIATE; > } > execute_using > - {} > + { > + Lex->pop_select(); //main select > + if (Lex->check_main_unit_semantics()) > + MYSQL_YYABORT; > + } > ; > > execute_using: > @@ -1928,13 +1985,20 @@ connection_name: > /* create a table */ > > create: > - create_or_replace opt_temporary TABLE_SYM opt_if_not_exists > table_ident > + create_or_replace opt_temporary TABLE_SYM opt_if_not_exists > { > LEX *lex= thd->lex; > lex->create_info.init(); > + if (lex->main_select_push()) > + MYSQL_YYABORT; > + lex->current_select->parsing_place= BEFORE_OPT_LIST; > if (lex->set_command_with_check(SQLCOM_CREATE_TABLE, $2, $1 > | $4)) > MYSQL_YYABORT; > - if (!lex->builtin_select.add_table_to_list(thd, $5, NULL, > + } > + table_ident > + { > + LEX *lex= thd->lex; > + if (!lex->builtin_select.add_table_to_list(thd, $6, NULL, > TL_OPTION_UPDATING, > TL_WRITE, > MDL_EXCLUSIVE)) > MYSQL_YYABORT; > @@ -1960,13 +2024,16 @@ create: > ER_WARN_USING_OTHER_HANDLER, > ER_THD(thd, > ER_WARN_USING_OTHER_HANDLER), > > hton_name(lex->create_info.db_type)->str, > - $5->table.str); > + $6->table.str); > } > create_table_set_open_action_and_adjust_tables(lex); > + Lex->pop_select(); //main select > } > | create_or_replace opt_temporary SEQUENCE_SYM opt_if_not_exists > table_ident > { > LEX *lex= thd->lex; > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > lex->create_info.init(); > if (lex->set_command_with_check(SQLCOM_CREATE_SEQUENCE, $2, > $1 | $4)) > MYSQL_YYABORT; > @@ -2021,42 +2088,69 @@ create: > $5->table.str); > } > create_table_set_open_action_and_adjust_tables(lex); > + Lex->pop_select(); //main select > } > - | create_or_replace opt_unique INDEX_SYM opt_if_not_exists ident > + | create_or_replace opt_unique INDEX_SYM opt_if_not_exists > + { > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > + } > + ident > opt_key_algorithm_clause > ON table_ident > { > - if (Lex->add_create_index_prepare($8)) > + if (Lex->add_create_index_prepare($9)) > MYSQL_YYABORT; > - if (Lex->add_create_index($2, &$5, $6, $1 | $4)) > + if (Lex->add_create_index($2, &$6, $7, $1 | $4)) > MYSQL_YYABORT; > } > '(' key_list ')' opt_lock_wait_timeout normal_key_options > - opt_index_lock_algorithm { } > - | create_or_replace fulltext INDEX_SYM opt_if_not_exists ident > + opt_index_lock_algorithm > + { > + Lex->pop_select(); //main select > + } > + | create_or_replace fulltext INDEX_SYM > + { > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > + } > + opt_if_not_exists ident > ON table_ident > { > - if (Lex->add_create_index_prepare($7)) > + if (Lex->add_create_index_prepare($8)) > MYSQL_YYABORT; > - if (Lex->add_create_index($2, &$5, HA_KEY_ALG_UNDEF, $1 | $4)) > + if (Lex->add_create_index($2, &$6, HA_KEY_ALG_UNDEF, $1 | $5)) > MYSQL_YYABORT; > } > '(' key_list ')' opt_lock_wait_timeout fulltext_key_options > - opt_index_lock_algorithm { } > - | create_or_replace spatial INDEX_SYM opt_if_not_exists ident > + opt_index_lock_algorithm > + { > + Lex->pop_select(); //main select > + } > + | create_or_replace spatial INDEX_SYM > + { > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > + } > + opt_if_not_exists ident > ON table_ident > { > - if (Lex->add_create_index_prepare($7)) > + if (Lex->add_create_index_prepare($8)) > MYSQL_YYABORT; > - if (Lex->add_create_index($2, &$5, HA_KEY_ALG_UNDEF, $1 | $4)) > + if (Lex->add_create_index($2, &$6, HA_KEY_ALG_UNDEF, $1 | $5)) > MYSQL_YYABORT; > } > '(' key_list ')' opt_lock_wait_timeout spatial_key_options > - opt_index_lock_algorithm { } > + opt_index_lock_algorithm > + { > + Lex->pop_select(); //main select > + } > | create_or_replace DATABASE opt_if_not_exists ident > { > Lex->create_info.default_table_charset= NULL; > Lex->create_info.used_fields= 0; > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > } > opt_create_database_options > { > @@ -2064,51 +2158,94 @@ create: > if (lex->set_command_with_check(SQLCOM_CREATE_DB, 0, $1 | $3)) > MYSQL_YYABORT; > lex->name= $4; > + Lex->pop_select(); //main select > } > | create_or_replace definer_opt opt_view_suid VIEW_SYM > opt_if_not_exists table_ident > { > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > if (Lex->add_create_view(thd, $1 | $5, > DTYPE_ALGORITHM_UNDEFINED, $3, $6)) > MYSQL_YYABORT; > } > view_list_opt AS view_select > - { } > + { > + Lex->pop_select(); //main select > + } > | create_or_replace view_algorithm definer_opt opt_view_suid > VIEW_SYM > opt_if_not_exists table_ident > { > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > if (Lex->add_create_view(thd, $1 | $6, $2, $4, $7)) > MYSQL_YYABORT; > } > view_list_opt AS view_select > - { } > + { > + Lex->pop_select(); //main select > + } > | create_or_replace definer_opt TRIGGER_SYM > - { Lex->create_info.set($1); } > + { > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > + Lex->create_info.set($1); > + } > trigger_tail > - { } > + { > + Lex->pop_select(); //main select > + } > | create_or_replace definer_opt PROCEDURE_SYM > - { Lex->create_info.set($1); } > + { > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > + Lex->create_info.set($1); > + } > sp_tail_standalone > - { } > + { > + Lex->pop_select(); //main select > + } > | create_or_replace definer_opt EVENT_SYM > - { Lex->create_info.set($1); } > + { > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > + Lex->create_info.set($1); > + } > event_tail > - { } > + { > + Lex->pop_select(); //main select > + } > | create_or_replace definer FUNCTION_SYM > - { Lex->create_info.set($1); } > + { > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > + Lex->create_info.set($1); > + } > sf_tail_standalone > - { } > + { > + Lex->pop_select(); //main select > + } > | create_or_replace no_definer FUNCTION_SYM > - { Lex->create_info.set($1); } > + { > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > + Lex->create_info.set($1); > + } > create_function_tail > - { } > + { > + Lex->pop_select(); //main select > + } > | create_or_replace no_definer AGGREGATE_SYM FUNCTION_SYM > { > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > Lex->create_info.set($1); > Lex->udf.type= UDFTYPE_AGGREGATE; > } > udf_tail > - { } > + { > + Lex->pop_select(); //main select > + } > | create_or_replace USER_SYM opt_if_not_exists clear_privileges > grant_list > opt_require_clause opt_resource_options > { > @@ -2815,8 +2952,13 @@ call: > { > if (Lex->call_statement_start(thd, $2)) > MYSQL_YYABORT; > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > + } > + opt_sp_cparam_list > + { > + Lex->pop_select(); //main select > } > - opt_sp_cparam_list {} > ; > > /* CALL parameters */ > @@ -3346,10 +3488,16 @@ raise_stmt: > ; > > signal_stmt: > - SIGNAL_SYM signal_value opt_set_signal_information > + SIGNAL_SYM > { > - if (Lex->add_signal_statement(thd, $2)) > + if (Lex->main_select_push()) > + YYABORT; > + } > + signal_value opt_set_signal_information > + { > + if (Lex->add_signal_statement(thd, $3)) > MYSQL_YYABORT; > + Lex->pop_select(); //main select > } > ; > > @@ -3475,9 +3623,14 @@ resignal_stmt: > ; > > get_diagnostics: > - GET_SYM which_area DIAGNOSTICS_SYM diagnostics_information > + GET_SYM which_area DIAGNOSTICS_SYM > + { > + if (Lex->main_select_push()) > + YYABORT; > + } > + diagnostics_information > { > - Diagnostics_information *info= $4; > + Diagnostics_information *info= $5; > > info->set_which_da($2); > > @@ -3486,6 +3639,7 @@ get_diagnostics: > > if (Lex->m_sql_cmd == NULL) > MYSQL_YYABORT; > + Lex->pop_select(); //main select > } > ; > > @@ -3666,8 +3820,26 @@ sp_decl_idents: > > sp_opt_default: > /* Empty */ { $$ = NULL; } > - | DEFAULT expr { $$ = $2; } > - | SET_VAR expr { $$ = $2; } > + | DEFAULT > + { > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > + } > + expr > + { > + Lex->pop_select(); //main select > + $$ = $3; > + } > + | SET_VAR > + { > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > + } > + expr > + { > + Lex->pop_select(); //main select > + $$ = $3; > + } > ; > > sp_proc_stmt: > @@ -3784,10 +3956,15 @@ sp_proc_stmt_statement: > > sp_proc_stmt_return: > RETURN_SYM > - { Lex->sphead->reset_lex(thd); } > + { > + Lex->sphead->reset_lex(thd); > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > + } > expr > { > LEX *lex= Lex; > + lex->pop_select(); //main select > sp_head *sp= lex->sphead; > if (sp->m_handler->add_instr_freturn(thd, sp, lex->spcont, > $3, lex) || > @@ -3902,6 +4079,8 @@ assignment_source_expr: > { > DBUG_ASSERT(thd->free_list == NULL); > Lex->sphead->reset_lex(thd, $1); > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > } > expr > { > @@ -3910,6 +4089,7 @@ assignment_source_expr: > $$->sp_lex_in_use= true; > $$->set_item_and_free_list($3, thd->free_list); > thd->free_list= NULL; > + Lex->pop_select(); //main select > if ($$->sphead->restore_lex(thd)) > MYSQL_YYABORT; > } > @@ -4030,9 +4210,14 @@ sp_fetch_list: > ; > > sp_if: > - { Lex->sphead->reset_lex(thd); } > + { > + Lex->sphead->reset_lex(thd); > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > + } > expr THEN_SYM > { > + Lex->pop_select(); //main select > LEX *lex= Lex; > sp_head *sp= lex->sphead; > sp_pcontext *ctx= lex->spcont; > @@ -4144,12 +4329,18 @@ case_stmt_specification: > ; > > case_stmt_body: > - { Lex->sphead->reset_lex(thd); /* For expr $2 */ } > + { > + Lex->sphead->reset_lex(thd); /* For expr $2 */ > + > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > + } > expr > { > if (Lex->case_stmt_action_expr($2)) > MYSQL_YYABORT; > > + Lex->pop_select(); //main select > if (Lex->sphead->restore_lex(thd)) > MYSQL_YYABORT; > } > @@ -4173,6 +4364,9 @@ simple_when_clause: > WHEN_SYM > { > Lex->sphead->reset_lex(thd); /* For expr $3 */ > + > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > } > expr > { > @@ -4181,6 +4375,8 @@ simple_when_clause: > LEX *lex= Lex; > if (lex->case_stmt_action_when($3, true)) > MYSQL_YYABORT; > + > + lex->pop_select(); //main select > /* For expr $3 */ > if (lex->sphead->restore_lex(thd)) > MYSQL_YYABORT; > @@ -4197,12 +4393,17 @@ searched_when_clause: > WHEN_SYM > { > Lex->sphead->reset_lex(thd); /* For expr $3 */ > + > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > } > expr > { > LEX *lex= Lex; > if (lex->case_stmt_action_when($3, false)) > MYSQL_YYABORT; > + > + lex->pop_select(); //main select > /* For expr $3 */ > if (lex->sphead->restore_lex(thd)) > MYSQL_YYABORT; > @@ -4442,6 +4643,7 @@ while_body: > LEX *lex= Lex; > if (lex->sp_while_loop_expression(thd, $1)) > MYSQL_YYABORT; > + Lex->pop_select(); //main select pushed before while_body use > if (lex->sphead->restore_lex(thd)) > MYSQL_YYABORT; > } > @@ -4454,7 +4656,12 @@ while_body: > > repeat_body: > sp_proc_stmts1 UNTIL_SYM > - { Lex->sphead->reset_lex(thd); } > + { > + Lex->sphead->reset_lex(thd); > + > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > + } > expr END REPEAT_SYM > { > LEX *lex= Lex; > @@ -4465,6 +4672,8 @@ repeat_body: > if (i == NULL || > lex->sphead->add_instr(i)) > MYSQL_YYABORT; > + > + lex->pop_select(); //main select > if (lex->sphead->restore_lex(thd)) > MYSQL_YYABORT; > /* We can shortcut the cont_backpatch here */ > @@ -4493,8 +4702,12 @@ sp_labeled_control: > if (Lex->sp_push_loop_label(thd, &$1)) > MYSQL_YYABORT; > Lex->sphead->reset_lex(thd); > + > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > } > while_body pop_sp_loop_label > + > { } > | labels_declaration_oracle FOR_SYM > { > @@ -4546,9 +4759,13 @@ sp_unlabeled_control: > if (Lex->sp_push_loop_empty_label(thd)) > MYSQL_YYABORT; > Lex->sphead->reset_lex(thd); > + > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > } > while_body > { > + // while body pop main select > Lex->sp_pop_loop_empty_label(thd); > } > | FOR_SYM > @@ -4981,20 +5198,10 @@ size_number: > */ > > create_body: > - '(' create_field_list ')' > + create_field_list_parens > { Lex->create_info.option_list= NULL; } > opt_create_table_options opt_create_partitioning > opt_create_select {} > | opt_create_table_options opt_create_partitioning > opt_create_select {} > - /* > - the following rule is redundant, but there's a shift/reduce > - conflict that prevents the rule above from parsing a syntax like > - CREATE TABLE t1 (SELECT 1); > - */ > - | '(' create_select_query_specification ')' > - | '(' create_select_query_specification ')' > - { Select->set_braces(1);} union_list {} > - | '(' create_select_query_specification ')' > - { Select->set_braces(1);} union_order_or_limit {} > | create_like > { > > @@ -5010,7 +5217,7 @@ create_body: > > create_like: > LIKE table_ident { $$= $2; } > - | '(' LIKE table_ident ')' { $$= $3; } > + | LEFT_PAREN_LIKE LIKE table_ident ')' { $$= $3; } > ; > > opt_create_select: > @@ -5019,23 +5226,42 @@ opt_create_select: > ; > > create_select_query_expression: > - opt_with_clause SELECT_SYM create_select_part2 > opt_table_expression > - create_select_part4 > - { > - Select->set_braces(0); > - Select->set_with_clause($1); > + query_expression > + { > + SELECT_LEX *first_select= $1->first_select(); > + > + if (Lex->sql_command == SQLCOM_INSERT || > + Lex->sql_command == SQLCOM_REPLACE) > + { > + if (Lex->sql_command == SQLCOM_INSERT) > + Lex->sql_command= SQLCOM_INSERT_SELECT; > + else > + Lex->sql_command= SQLCOM_REPLACE_SELECT; > + } > + Lex->insert_select_hack(first_select); > + if (Lex->check_main_unit_semantics()) > + MYSQL_YYABORT; > + > } > - union_clause > - | opt_with_clause SELECT_SYM create_select_part2 > - create_select_part3_union_not_ready create_select_part4 > + | LEFT_PAREN_WITH with_clause query_expression_body ')' > { > - Select->set_with_clause($1); > + SELECT_LEX *first_select= $3->first_select(); > + $3->set_with_clause($2); > + $2->attach_to(first_select); > + > + Lex->insert_select_hack(first_select); > + if (Lex->check_main_unit_semantics()) > + MYSQL_YYABORT; > + > + if (Lex->sql_command == SQLCOM_INSERT || > + Lex->sql_command == SQLCOM_REPLACE) > + { > + if (Lex->sql_command == SQLCOM_INSERT) > + Lex->sql_command= SQLCOM_INSERT_SELECT; > + else > + Lex->sql_command= SQLCOM_REPLACE_SELECT; > + } > } > - | '(' create_select_query_specification ')' > - | '(' create_select_query_specification ')' > - { Select->set_braces(1);} union_list {} > - | '(' create_select_query_specification ')' > - { Select->set_braces(1);} union_order_or_limit {} > ; > > opt_create_partitioning: > @@ -5126,8 +5352,13 @@ partition_entry: > We enter here when opening the frm file to translate > partition info string into part_info data structure. > */ > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > + } > + partition > + { > + Lex->pop_select(); //main select > } > - partition {} > ; > > partition: > @@ -5742,56 +5973,6 @@ opt_part_option: > End of partition parser part > */ > > -create_select_query_specification: > - opt_with_clause SELECT_SYM create_select_part2 > create_select_part3 > - create_select_part4 > - { > - Select->set_with_clause($1); > - } > - ; > - > -create_select_part2: > - { > - LEX *lex=Lex; > - if (lex->sql_command == SQLCOM_INSERT) > - lex->sql_command= SQLCOM_INSERT_SELECT; > - else if (lex->sql_command == SQLCOM_REPLACE) > - lex->sql_command= SQLCOM_REPLACE_SELECT; > - /* > - The following work only with the local list, the global list > - is created correctly in this case > - */ > - > lex->current_select->table_list.save_and_clear(&lex->save_list); > - mysql_init_select(lex); > - lex->current_select->parsing_place= SELECT_LIST; > - } > - select_options select_item_list > - { > - Select->parsing_place= NO_MATTER; > - } > - ; > - > -create_select_part3: > - opt_table_expression > - | create_select_part3_union_not_ready > - ; > - > -create_select_part3_union_not_ready: > - table_expression order_or_limit > - | order_or_limit > - ; > - > -create_select_part4: > - opt_select_lock_type > - { > - /* > - The following work only with the local list, the global list > - is created correctly in this case > - */ > - Lex->current_select->table_list.push_front(&Lex->save_list); > - } > - ; > - > opt_as: > /* empty */ {} > | AS {} > @@ -6195,6 +6376,13 @@ create_field_list: > } > ; > > +create_field_list_parens: > + LEFT_PAREN_ALT field_list ')' > + { > + Lex->create_last_non_select_table= Lex->last_table(); > + } > + ; > + > field_list: > field_list_item > | field_list ',' field_list_item > @@ -6462,6 +6650,8 @@ parse_vcol_expr: > Prevent the end user from invoking this command. > */ > MYSQL_YYABORT_UNLESS(Lex->parse_vcol_expr); > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > } > expr > { > @@ -6469,13 +6659,29 @@ parse_vcol_expr: > if (!v) > MYSQL_YYABORT; > Lex->last_field->vcol_info= v; > + Lex->pop_select(); //main select > } > ; > > parenthesized_expr: > - subselect > + remember_tok_start > + query_expression > { > - $$= new (thd->mem_root) Item_singlerow_subselect(thd, $1); > + if (!Lex->expr_allows_subselect || > + Lex->sql_command == (int)SQLCOM_PURGE) > + { > + thd->parse_error(ER_SYNTAX_ERROR, $1); > + MYSQL_YYABORT; > + } > + > + // Add the subtree of subquery to the current SELECT_LEX > + SELECT_LEX *curr_sel= Lex->select_stack_head(); > + DBUG_ASSERT(Lex->current_select == curr_sel); > + curr_sel->register_unit($2, &curr_sel->context); > + curr_sel->add_statistics($2); > + > + $$= new (thd->mem_root) > + Item_singlerow_subselect(thd, $2->first_select()); > if ($$ == NULL) > MYSQL_YYABORT; > } > @@ -7474,6 +7680,8 @@ alter: > Lex->alter_info.reset(); > Lex->no_write_to_binlog= 0; > Lex->create_info.storage_media= HA_SM_DEFAULT; > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > DBUG_ASSERT(!Lex->m_sql_cmd); > } > alter_options TABLE_SYM table_ident opt_lock_wait_timeout > @@ -7494,12 +7702,15 @@ alter: > Lex->m_sql_cmd= new (thd->mem_root) Sql_cmd_alter_table(); > if (Lex->m_sql_cmd == NULL) > MYSQL_YYABORT; > + Lex->pop_select(); //main select > } > } > | ALTER DATABASE ident_or_empty > { > Lex->create_info.default_table_charset= NULL; > Lex->create_info.used_fields= 0; > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > } > create_database_options > { > @@ -7508,6 +7719,7 @@ alter: > lex->name= $3; > if (lex->name.str == NULL && lex->copy_db_to(&lex->name)) > MYSQL_YYABORT; > + Lex->pop_select(); //main select > } > | ALTER DATABASE ident UPGRADE_SYM DATA_SYM DIRECTORY_SYM NAME_SYM > { > @@ -7523,6 +7735,8 @@ alter: > > if (lex->sphead) > my_yyabort_error((ER_SP_NO_DROP_SP, MYF(0), "PROCEDURE")); > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > lex->sp_chistics.init(); > } > sp_a_chistics > @@ -7531,6 +7745,9 @@ alter: > > lex->sql_command= SQLCOM_ALTER_PROCEDURE; > lex->spname= $3; > + Lex->pop_select(); //main select > + if (Lex->check_main_unit_semantics()) > + MYSQL_YYABORT; > } > | ALTER FUNCTION_SYM sp_name > { > @@ -7538,6 +7755,8 @@ alter: > > if (lex->sphead) > my_yyabort_error((ER_SP_NO_DROP_SP, MYF(0), "FUNCTION")); > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > lex->sp_chistics.init(); > } > sp_a_chistics > @@ -7546,14 +7765,23 @@ alter: > > lex->sql_command= SQLCOM_ALTER_FUNCTION; > lex->spname= $3; > + Lex->pop_select(); //main select > + if (Lex->check_main_unit_semantics()) > + MYSQL_YYABORT; > } > | ALTER view_algorithm definer_opt opt_view_suid VIEW_SYM > table_ident > { > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > if (Lex->add_alter_view(thd, $2, $4, $6)) > MYSQL_YYABORT; > } > view_list_opt AS view_select > - {} > + { > + Lex->pop_select(); //main select > + if (Lex->check_main_unit_semantics()) > + MYSQL_YYABORT; > + } > | ALTER definer_opt opt_view_suid VIEW_SYM table_ident > /* > We have two separate rules for ALTER VIEW rather that > @@ -7561,14 +7789,22 @@ alter: > with the ALTER EVENT below. > */ > { > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > if (Lex->add_alter_view(thd, VIEW_ALGORITHM_INHERIT, $3, $5)) > MYSQL_YYABORT; > } > view_list_opt AS view_select > - {} > + { > + Lex->pop_select(); //main select > + if (Lex->check_main_unit_semantics()) > + MYSQL_YYABORT; > + } > | ALTER definer_opt remember_name EVENT_SYM sp_name > { > - /* > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > + /* > It is safe to use Lex->spname because > ALTER EVENT xxx RENATE TO yyy DO ALTER EVENT RENAME TO > is not allowed. Lex->spname is used in the case of RENAME TO > @@ -7600,6 +7836,8 @@ alter: > */ > Lex->sql_command= SQLCOM_ALTER_EVENT; > Lex->stmt_definition_end= (char*)YYLIP->get_cpp_ptr(); > + > + Lex->pop_select(); //main select > } > | ALTER TABLESPACE alter_tablespace_info > { > @@ -7643,6 +7881,8 @@ alter: > lex->create_info.init(); > lex->no_write_to_binlog= 0; > DBUG_ASSERT(!lex->m_sql_cmd); > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > } > table_ident > { > @@ -7660,6 +7900,9 @@ alter: > Lex->m_sql_cmd= new (thd->mem_root) > Sql_cmd_alter_sequence($3); > if (Lex->m_sql_cmd == NULL) > MYSQL_YYABORT; > + Lex->pop_select(); //main select > + if (Lex->check_main_unit_semantics()) > + MYSQL_YYABORT; > } > ; > > @@ -8336,9 +8579,13 @@ checksum: > lex->sql_command = SQLCOM_CHECKSUM; > /* Will be overridden during execution. */ > YYPS->m_lock_type= TL_UNLOCK; > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > } > table_list opt_checksum_type > - {} > + { > + Lex->pop_select(); //main select > + } > ; > > opt_checksum_type: > @@ -8364,6 +8611,8 @@ repair: > lex->alter_info.reset(); > /* Will be overridden during execution. */ > YYPS->m_lock_type= TL_UNLOCK; > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > } > repair_table_or_view > { > @@ -8372,6 +8621,7 @@ repair: > lex->m_sql_cmd= new (thd->mem_root) Sql_cmd_repair_table(); > if (lex->m_sql_cmd == NULL) > MYSQL_YYABORT; > + Lex->pop_select(); //main select > } > ; > > @@ -8400,6 +8650,8 @@ analyze: > ANALYZE_SYM opt_no_write_to_binlog table_or_tables > { > LEX *lex=Lex; > + if (lex->main_select_push()) > + YYABORT; > lex->sql_command = SQLCOM_ANALYZE; > lex->no_write_to_binlog= $2; > lex->check_opt.init(); > @@ -8414,6 +8666,7 @@ analyze: > lex->m_sql_cmd= new (thd->mem_root) Sql_cmd_analyze_table(); > if (lex->m_sql_cmd == NULL) > MYSQL_YYABORT; > + Lex->pop_select(); //main select > } > ; > > @@ -8530,6 +8783,8 @@ check: CHECK_SYM > lex->alter_info.reset(); > /* Will be overridden during execution. */ > YYPS->m_lock_type= TL_UNLOCK; > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > } > check_view_or_table > { > @@ -8540,6 +8795,7 @@ check: CHECK_SYM > lex->m_sql_cmd= new (thd->mem_root) Sql_cmd_check_table(); > if (lex->m_sql_cmd == NULL) > MYSQL_YYABORT; > + Lex->pop_select(); //main select > } > ; > > @@ -8577,6 +8833,8 @@ optimize: > lex->alter_info.reset(); > /* Will be overridden during execution. */ > YYPS->m_lock_type= TL_UNLOCK; > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > } > table_list opt_lock_wait_timeout > { > @@ -8585,6 +8843,7 @@ optimize: > lex->m_sql_cmd= new (thd->mem_root) Sql_cmd_optimize_table(); > if (lex->m_sql_cmd == NULL) > MYSQL_YYABORT; > + Lex->pop_select(); //main select > } > ; > > @@ -8598,9 +8857,13 @@ rename: > RENAME table_or_tables > { > Lex->sql_command= SQLCOM_RENAME_TABLE; > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > } > table_to_table_list > - {} > + { > + Lex->pop_select(); //main select > + } > | RENAME USER_SYM clear_privileges rename_list > { > Lex->sql_command = SQLCOM_RENAME_USER; > @@ -8694,10 +8957,14 @@ preload: > LEX *lex=Lex; > lex->sql_command=SQLCOM_PRELOAD_KEYS; > lex->alter_info.reset(); > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > } > preload_list_or_parts > - {} > - ; > + { > + Lex->pop_select(); //main select > + } > + ; > > preload_list_or_parts: > preload_keys_parts > @@ -8763,192 +9030,353 @@ opt_ignore_leaves: > > > select: > - opt_with_clause select_init > + query_expression > { > - LEX *lex= Lex; > - lex->sql_command= SQLCOM_SELECT; > - lex->current_select->set_with_clause($1); > + Lex->selects_allow_into= TRUE; > + Lex->selects_allow_procedure= TRUE; > + Lex->set_main_unit($1); > + if (Lex->check_main_unit_semantics()) > + MYSQL_YYABORT; > + > + SELECT_LEX *fake= Lex->unit.fake_select_lex; > + if (fake) > + { > + fake->no_table_names_allowed= 1; > + Lex->push_select(fake); > + } > + else > + Lex->push_select(Lex->first_select_lex()); > + Lex->sql_command= SQLCOM_SELECT; > } > ; > > -select_init: > - SELECT_SYM select_options_and_item_list select_init3 > - | '(' select_paren ')' > - | '(' select_paren ')' union_list > - | '(' select_paren ')' union_order_or_limit > - ; > > -union_list_part2: > - SELECT_SYM select_options_and_item_list > select_init3_union_query_term > - | '(' select_paren_union_query_term ')' > - | '(' select_paren_union_query_term ')' union_list > - | '(' select_paren_union_query_term ')' union_order_or_limit > +simple_table: > + query_specification { $$= $1; } > + | table_value_constructor { $$= $1; } > ; > - > -select_paren: > + > +table_value_constructor: > + VALUES > + { > + LEX *lex= Lex; > + SELECT_LEX *sel; > + //lex->field_list.empty(); > + lex->many_values.empty(); > + lex->insert_list=0; > + if (!(sel= lex->alloc_select(TRUE)) || > + lex->push_select(sel)) > + MYSQL_YYABORT; > + sel->init_select(); > + sel->braces= FALSE; // just initialisation > + } > + values_list > + { > + LEX *lex=Lex; > + $$= lex->pop_select(); // above TVC select > + if (!($$->tvc= > + new (lex->thd->mem_root) > table_value_constr(lex->many_values, > + $$, > + > $$->options))) > + MYSQL_YYABORT; > + lex->many_values.empty(); > + } > + ; > + > +query_specification: > + SELECT_SYM > { > - /* > - In order to correctly parse UNION's global ORDER BY we > need to > - set braces before parsing the clause. > - */ > - Lex->current_select->set_braces(true); > + SELECT_LEX *sel; > + LEX *lex= Lex; > + if (!(sel= lex->alloc_select(TRUE)) || > + lex->push_select(sel)) > + MYSQL_YYABORT; > + sel->init_select(); > + sel->braces= FALSE; > } > - SELECT_SYM select_options_and_item_list select_part3 > - opt_select_lock_type > + select_options > { > - DBUG_ASSERT(Lex->current_select->braces); > + Select->parsing_place= SELECT_LIST; > } > - | '(' select_paren ')' > - ; > - > -select_paren_union_query_term: > + select_item_list > { > - /* > - In order to correctly parse UNION's global ORDER BY we > need to > - set braces before parsing the clause. > - */ > - Lex->current_select->set_braces(true); > + Select->parsing_place= NO_MATTER; > } > - SELECT_SYM select_options_and_item_list > select_part3_union_query_term > - opt_select_lock_type > + opt_into > + opt_from_clause > + opt_where_clause > + opt_group_clause > + opt_having_clause > + opt_window_clause > { > - DBUG_ASSERT(Lex->current_select->braces); > + $$= Lex->pop_select(); > } > - | '(' select_paren_union_query_term ')' > ; > > -select_paren_view: > - { > - /* > - In order to correctly parse UNION's global ORDER BY we > need to > - set braces before parsing the clause. > - */ > - Lex->current_select->set_braces(true); > - } > - SELECT_SYM select_options_and_item_list select_part3_view > - opt_select_lock_type > - { > - DBUG_ASSERT(Lex->current_select->braces); > - } > - | '(' select_paren_view ')' > +opt_from_clause: > + /* Empty */ > + | from_clause > + ; > + > +query_primary: > + simple_table > + { $$= $1; } > + | query_primary_parens > + { $$= $1; } > ; > > -/* The equivalent of select_paren for nested queries. */ > -select_paren_derived: > +query_primary_parens: > + '(' query_expression_unit > { > - Lex->current_select->set_braces(true); > + SELECT_LEX *last= $2->pre_last_parse->next_select(); > + int cmp= > cmp_unit_op($2->first_select()->next_select()->linkage, > + last->linkage); > + if (cmp < 0) > + { > + if (!check_intersect_prefix($2->first_select())) > + { > + if (Lex->pop_new_select_and_wrap() == NULL) > + MYSQL_YYABORT; > + } > + } > + Lex->push_select($2->fake_select_lex); > } > - SELECT_SYM select_part2_derived > - opt_table_expression > - opt_order_clause > - opt_limit_clause > - opt_select_lock_type > + query_expression_tail ')' > { > - DBUG_ASSERT(Lex->current_select->braces); > - $$= Lex->current_select->master_unit()->first_select(); > + Lex->pop_select(); > + if ($4) > + { > + ($4)->set_to($2->fake_select_lex); > + } > + $$= $2->first_select(); > } > - | '(' select_paren_derived ')' { $$= $2; } > - ; > - > -select_init3: > - opt_table_expression > - opt_select_lock_type > + | '(' query_primary > { > - /* Parentheses carry no meaning here */ > - Lex->current_select->set_braces(false); > + Lex->push_select($2); > } > - union_clause > - | select_part3_union_not_ready > - opt_select_lock_type > + query_expression_tail ')' > { > - /* Parentheses carry no meaning here */ > - Lex->current_select->set_braces(false); > + Lex->pop_select(); > + $$= $2; > + $$->braces= TRUE; > + if ($4) > + { > + if ($2->next_select()) > + { > + SELECT_LEX_UNIT *unit= $2->master_unit(); > + if (!unit) > + unit= Lex->create_unit($2); > + if (!unit) > + YYABORT; > + if (!unit->fake_select_lex->is_set_query_expr_tail) > + $4->set_to(unit->fake_select_lex); > + else > + { > + $$= Lex->wrap_unit_into_derived(unit); > + if (!$$) > + YYABORT; > + $4->set_to($$); > + } > + } > + else if (!$2->is_set_query_expr_tail) > + { > + $4->set_to($2); > + } > + else > + { > + SELECT_LEX_UNIT *unit= Lex->create_unit($2); > + if (!unit) > + YYABORT; > + $$= Lex->wrap_unit_into_derived(unit); > + if (!$$) > + YYABORT; > + $4->set_to($$); > + } > + } > } > ; > > - > -select_init3_union_query_term: > - opt_table_expression > - opt_select_lock_type > +query_expression_unit: > + query_primary > + unit_type_decl > + query_primary > { > - /* Parentheses carry no meaning here */ > - Lex->current_select->set_braces(false); > + SELECT_LEX *sel1; > + SELECT_LEX *sel2; > + if (!$1->next_select()) > + sel1= $1; > + else > + { > + sel1= Lex->wrap_unit_into_derived($1->master_unit()); > + if (!sel1) > + YYABORT; > + } > + if (!$3->next_select()) > + sel2= $3; > + else > + { > + sel2= Lex->wrap_unit_into_derived($3->master_unit()); > + if (!sel2) > + YYABORT; > + } > + sel1->link_neighbour(sel2); > + sel2->set_linkage_and_distinct($2.unit_type, $2.distinct); > + $$= Lex->create_unit(sel1); > + $$->pre_last_parse= sel1; > + if ($$ == NULL) > + YYABORT; > } > - union_clause > - | select_part3_union_not_ready_noproc > - opt_select_lock_type > + | query_expression_unit > + unit_type_decl > + query_primary > { > - /* Parentheses carry no meaning here */ > - Lex->current_select->set_braces(false); > + SELECT_LEX *sel1; > + if (!$3->next_select()) > + sel1= $3; > + else > + { > + sel1= Lex->wrap_unit_into_derived($3->master_unit()); > + if (!sel1) > + YYABORT; > + } > + SELECT_LEX *last= $1->pre_last_parse->next_select(); > + > + int cmp= cmp_unit_op($2.unit_type, last->linkage); > + if (cmp == 0) > + { > + // do nothing, this part will be just connected > + } > + else if (cmp > 0) > + { > + // Store beginning and continue to connect parts > + if (Lex->push_new_select($1->pre_last_parse)) > + MYSQL_YYABORT; > + } > + else /* cmp < 0 */ > + { > + // wrap stored part in a select, then continue to connect > parts > + if (!check_intersect_prefix($1->first_select())) > + { > + if ((last= Lex->pop_new_select_and_wrap()) == NULL) > + MYSQL_YYABORT; > + last->set_master_unit($1); > + } > + } > + last->link_neighbour(sel1); > + sel1->set_master_unit($1); > + sel1->set_linkage_and_distinct($2.unit_type, $2.distinct); > + $$= $1; > + $$->pre_last_parse= last; > } > ; > > - > -select_init3_view: > - opt_table_expression opt_select_lock_type > +query_expression_body: > + query_primary > { > - Lex->current_select->set_braces(false); > + Lex->push_select($1); > } > - | opt_table_expression opt_select_lock_type > + query_expression_tail > { > - Lex->current_select->set_braces(false); > + Lex->pop_select(); > + SELECT_LEX *sel= $1; > + if ($3) > + { > + if ($1->next_select()) > + { > + SELECT_LEX_UNIT *unit= $1->master_unit(); > + if (!unit) > + unit= Lex->create_unit($1); > + if (!unit) > + YYABORT; > + if (!unit->fake_select_lex->is_set_query_expr_tail) > + $3->set_to(unit->fake_select_lex); > + else > + { > + SELECT_LEX *sel= Lex->wrap_unit_into_derived(unit); > + if (!sel) > + YYABORT; > + $3->set_to(sel); > + } > + } > + else if (!$1->is_set_query_expr_tail) > + $3->set_to($1); > + else > + { > + SELECT_LEX_UNIT *unit= $1->master_unit(); > + if (!unit) > + unit= Lex->create_unit($1); > + if (!unit) > + YYABORT; > + sel= Lex->wrap_unit_into_derived(unit); > + if (!sel) > + YYABORT; > + $3->set_to(sel); > + } > + } > + $$= Lex->create_unit(sel); > + if ($$ == NULL) > + YYABORT; > } > - union_list_view > - | order_or_limit opt_select_lock_type > + | query_expression_unit > { > - Lex->current_select->set_braces(false); > + SELECT_LEX *last= $1->pre_last_parse->next_select(); > + int cmp= > cmp_unit_op($1->first_select()->next_select()->linkage, > + last->linkage); > + if (cmp < 0) > + { > + if (!check_intersect_prefix($1->first_select())) > + { > + if (Lex->pop_new_select_and_wrap() == NULL) > + MYSQL_YYABORT; > + } > + } > + Lex->push_select($1->fake_select_lex); > } > - | table_expression order_or_limit opt_select_lock_type > + query_expression_tail > { > - Lex->current_select->set_braces(false); > + Lex->pop_select(); > + if ($3) > + { > + ($3)->set_to($1->fake_select_lex); > + } > + $$= $1; > } > ; > > -/* > - The SELECT parts after select_item_list that cannot be followed by > UNION. > -*/ > - > -select_part3: > - opt_table_expression > - | select_part3_union_not_ready > - ; > - > -select_part3_union_query_term: > - opt_table_expression > - | select_part3_union_not_ready_noproc > - ; > - > -select_part3_view: > - opt_table_expression > - | order_or_limit > - | table_expression order_or_limit > +query_expression: > + opt_with_clause > + query_expression_body > + { > + if ($1) > + { > + $2->set_with_clause($1); > + $1->attach_to($2->first_select()); > + } > + $$= $2; > + } > ; > > -select_part3_union_not_ready: > - select_part3_union_not_ready_noproc > - | table_expression procedure_clause > - | table_expression order_or_limit procedure_clause > - ; > +subselect: > + remember_tok_start > + query_expression > + { > + if (!Lex->expr_allows_subselect || > + Lex->sql_command == (int)SQLCOM_PURGE) > + { > + thd->parse_error(ER_SYNTAX_ERROR, $1); > + MYSQL_YYABORT; > + } > > -select_part3_union_not_ready_noproc: > - order_or_limit > - | into opt_table_expression opt_order_clause opt_limit_clause > - | table_expression into > - | table_expression order_or_limit > - | table_expression order_or_limit into > - ; > + // Add the subtree of subquery to the current SELECT_LEX > + SELECT_LEX *curr_sel= Lex->select_stack_head(); > + DBUG_ASSERT(Lex->current_select == curr_sel); > + if (curr_sel) > + { > + curr_sel->register_unit($2, &curr_sel->context); > + curr_sel->add_statistics($2); > + } > > -select_options_and_item_list: > - { > - LEX *lex= Lex; > - SELECT_LEX *sel= lex->current_select; > - if (sel->linkage != UNION_TYPE) > - mysql_init_select(lex); > - lex->current_select->parsing_place= SELECT_LIST; > - } > - select_options select_item_list > - { > - Select->parsing_place= NO_MATTER; > + $$= $2->first_select(); > } > ; > > @@ -8956,18 +9384,6 @@ select_options_and_item_list: > /** > <table expression>, as in the SQL standard. > */ > -table_expression: > - from_clause > - opt_where_clause > - opt_group_clause > - opt_having_clause > - opt_window_clause > - ; > - > -opt_table_expression: > - /* Empty */ > - | table_expression > - ; > > from_clause: > FROM table_reference_list > @@ -9005,59 +9421,63 @@ select_option: > query_expression_option > | SQL_NO_CACHE_SYM > { > - /* > - Allow this flag only on the first top-level SELECT > statement, if > - SQL_CACHE wasn't specified, and only once per query. > - */ > - if (Lex->current_select != &Lex->builtin_select) > - my_yyabort_error((ER_CANT_USE_OPTION_HERE, MYF(0), > "SQL_NO_CACHE")); > - if (Lex->builtin_select.sql_cache == SELECT_LEX::SQL_CACHE) > - my_yyabort_error((ER_WRONG_USAGE, MYF(0), "SQL_CACHE", > "SQL_NO_CACHE")); > - if (Lex->builtin_select.sql_cache == SELECT_LEX::SQL_NO_CACHE) > + /* > + Allow this flag once per query. > + */ > + if (Select->options & OPTION_NO_QUERY_CACHE) > my_yyabort_error((ER_DUP_ARGUMENT, MYF(0), "SQL_NO_CACHE")); > - > - Lex->safe_to_cache_query=0; > - Lex->builtin_select.options&= ~OPTION_TO_QUERY_CACHE; > - Lex->builtin_select.sql_cache= SELECT_LEX::SQL_NO_CACHE; > + Select->options|= OPTION_NO_QUERY_CACHE; > } > | SQL_CACHE_SYM > { > - /* > - Allow this flag only on the first top-level SELECT > statement, if > - SQL_NO_CACHE wasn't specified, and only once per query. > - */ > - if (Lex->current_select != &Lex->builtin_select) > - my_yyabort_error((ER_CANT_USE_OPTION_HERE, MYF(0), > "SQL_CACHE")); > - if (Lex->builtin_select.sql_cache == SELECT_LEX::SQL_NO_CACHE) > - my_yyabort_error((ER_WRONG_USAGE, MYF(0), "SQL_NO_CACHE", > "SQL_CACHE")); > - if (Lex->builtin_select.sql_cache == SELECT_LEX::SQL_CACHE) > + /* > + Allow this flag once per query. > + */ > + if (Select->options & OPTION_TO_QUERY_CACHE) > my_yyabort_error((ER_DUP_ARGUMENT, MYF(0), "SQL_CACHE")); > - > - Lex->safe_to_cache_query=1; > - Lex->builtin_select.options|= OPTION_TO_QUERY_CACHE; > - Lex->builtin_select.sql_cache= SELECT_LEX::SQL_CACHE; > + Select->options|= OPTION_TO_QUERY_CACHE; > } > ; > > opt_select_lock_type: > /* empty */ > - | FOR_SYM UPDATE_SYM opt_lock_wait_timeout > + { $$.empty(); } > + | select_lock_type > + { $$= $1; } > + ; > + > +select_lock_type: > + FOR_SYM UPDATE_SYM opt_lock_wait_timeout_new > { > - LEX *lex=Lex; > - lex->current_select->lock_type= TL_WRITE; > - lex->current_select->set_lock_for_tables(TL_WRITE); > - lex->safe_to_cache_query=0; > + $$= $3; > + $$.defined_lock= TRUE; > + $$.update_lock= TRUE; > } > - | LOCK_SYM IN_SYM SHARE_SYM MODE_SYM opt_lock_wait_timeout > + | LOCK_SYM IN_SYM SHARE_SYM MODE_SYM opt_lock_wait_timeout_new > { > - LEX *lex=Lex; > - lex->current_select->lock_type= TL_READ_WITH_SHARED_LOCKS; > - lex->current_select-> > - set_lock_for_tables(TL_READ_WITH_SHARED_LOCKS); > - lex->safe_to_cache_query=0; > + $$= $5; > + $$.defined_lock= TRUE; > + $$.update_lock= FALSE; > } > ; > > +opt_lock_wait_timeout_new: > + /* empty */ > + { > + $$.empty(); > + } > + | WAIT_SYM ulong_num > + { > + $$.defined_timeout= TRUE; > + $$.timeout= $2; > + } > + | NOWAIT_SYM > + { > + $$.defined_timeout= TRUE; > + $$.timeout= 0; > + } > + ; > + > select_item_list: > select_item_list ',' select_item > | select_item > @@ -11361,10 +11781,15 @@ esc_table_ref: > /* Equivalent to <table reference list> in the SQL:2003 standard. */ > /* Warning - may return NULL in case of incomplete SELECT */ > derived_table_list: > - esc_table_ref { $$=$1; } > + esc_table_ref > + { > + $$=$1; > + Select->add_joined_table($1); > + } > | derived_table_list ',' esc_table_ref > { > MYSQL_YYABORT_UNLESS($1 && ($$=$3)); > + Select->add_joined_table($3); > } > ; > > @@ -11383,11 +11808,18 @@ join_table: > left-associative joins. > */ > table_ref normal_join table_ref %prec TABLE_REF_PRIORITY > - { MYSQL_YYABORT_UNLESS($1 && ($$=$3)); $3->straight=$2; } > + { > + MYSQL_YYABORT_UNLESS($1 && ($$=$3)); > + Select->add_joined_table($1); > + Select->add_joined_table($3); > + $3->straight=$2; > + } > | table_ref normal_join table_ref > ON > { > MYSQL_YYABORT_UNLESS($1 && $3); > + Select->add_joined_table($1); > + Select->add_joined_table($3); > /* Change the current name resolution context to a local > context. */ > if (push_new_name_resolution_context(thd, $1, $3)) > MYSQL_YYABORT; > @@ -11404,6 +11836,8 @@ join_table: > USING > { > MYSQL_YYABORT_UNLESS($1 && $3); > + Select->add_joined_table($1); > + Select->add_joined_table($3); > } > '(' using_list ')' > { > @@ -11414,6 +11848,8 @@ join_table: > | table_ref NATURAL inner_join table_factor > { > MYSQL_YYABORT_UNLESS($1 && ($$=$4)); > + Select->add_joined_table($1); > + Select->add_joined_table($4); > $4->straight=$3; > add_join_natural($1,$4,NULL,Select); > } > @@ -11423,6 +11859,8 @@ join_table: > ON > { > MYSQL_YYABORT_UNLESS($1 && $5); > + Select->add_joined_table($1); > + Select->add_joined_table($5); > /* Change the current name resolution context to a local > context. */ > if (push_new_name_resolution_context(thd, $1, $5)) > MYSQL_YYABORT; > @@ -11439,6 +11877,8 @@ join_table: > | table_ref LEFT opt_outer JOIN_SYM table_factor > { > MYSQL_YYABORT_UNLESS($1 && $5); > + Select->add_joined_table($1); > + Select->add_joined_table($5); > } > USING '(' using_list ')' > { > @@ -11449,6 +11889,8 @@ join_table: > | table_ref NATURAL LEFT opt_outer JOIN_SYM table_factor > { > MYSQL_YYABORT_UNLESS($1 && $6); > + Select->add_joined_table($1); > + Select->add_joined_table($6); > add_join_natural($1,$6,NULL,Select); > $6->outer_join|=JOIN_TYPE_LEFT; > $$=$6; > @@ -11459,6 +11901,8 @@ join_table: > ON > { > MYSQL_YYABORT_UNLESS($1 && $5); > + Select->add_joined_table($1); > + Select->add_joined_table($5); > /* Change the current name resolution context to a local > context. */ > if (push_new_name_resolution_context(thd, $1, $5)) > MYSQL_YYABORT; > @@ -11476,6 +11920,8 @@ join_table: > | table_ref RIGHT opt_outer JOIN_SYM table_factor > { > MYSQL_YYABORT_UNLESS($1 && $5); > + Select->add_joined_table($1); > + Select->add_joined_table($5); > } > USING '(' using_list ')' > { > @@ -11487,6 +11933,8 @@ join_table: > | table_ref NATURAL RIGHT opt_outer JOIN_SYM table_factor > { > MYSQL_YYABORT_UNLESS($1 && $6); > + Select->add_joined_table($1); > + Select->add_joined_table($6); > add_join_natural($6,$1,NULL,Select); > LEX *lex= Lex; > if (!($$= lex->current_select->convert_right_join())) > @@ -11521,40 +11969,70 @@ use_partition: > $$= $3; > } > ; > - > -/* > - This is a flattening of the rules <table factor> and <table primary> > - in the SQL:2003 standard, since we don't have <sample clause> > > - I.e. > - <table factor> ::= <table primary> [ <sample clause> ] > -*/ > -/* Warning - may return NULL in case of incomplete SELECT */ > table_factor: > - table_primary_ident > - | table_primary_derived > + table_primary_ident { $$= $1; } > + | table_primary_derived { $$= $1; } > + | join_table_parens { $$= $1; } > + | table_reference_list_parens { $$= $1; } > ; > > +table_reference_list_parens: > + '(' table_reference_list_parens ')' { $$= $2; } > + | '(' nested_table_reference_list ')' > + { > + if (!($$= Select->end_nested_join(thd))) > + MYSQL_YYABORT; > + } > + ; > + > +nested_table_reference_list: > + table_ref ',' table_ref > + { > + if (Select->init_nested_join(thd)) > + MYSQL_YYABORT; > + Select->add_joined_table($1); > + Select->add_joined_table($3); > + $$= $1->embedding; > + } > + | nested_table_reference_list ',' table_ref > + { > + Select->add_joined_table($3); > + $$= $1; > + } > + ; > + > +join_table_parens: > + '(' join_table_parens ')' { $$= $2; } > + | '(' join_table ')' > + { > + LEX *lex= Lex; > + if (!($$= lex->current_select->nest_last_join(thd))) > + { > + thd->parse_error(); > + MYSQL_YYABORT; > + } > + } > + ; > + > + > table_primary_ident: > + table_ident opt_use_partition > + opt_table_alias_clause opt_key_definition > { > SELECT_LEX *sel= Select; > sel->table_join_options= 0; > - } > - table_ident opt_use_partition opt_table_alias opt_key_definition > - { > - if (!($$= Select->add_table_to_list(thd, $2, $4, > + if (!($$= Select->add_table_to_list(thd, $1, $3, > > Select->get_table_join_options(), > YYPS->m_lock_type, > YYPS->m_mdl_type, > Select->pop_index_hints(), > - $3))) > + $2))) > MYSQL_YYABORT; > - Select->add_joined_table($$); > } > ; > > > - > /* > Represents a flattening of the following rules from the SQL:2003 > standard. This sub-rule corresponds to the sub-rule > @@ -11572,242 +12050,56 @@ table_primary_ident: > */ > > table_primary_derived: > - '(' get_select_lex select_derived_union ')' opt_table_alias > + query_primary_parens table_alias_clause > { > - /* Use $2 instead of Lex->current_select as derived table will > - alter value of Lex->current_select. */ > - if (!($3 || $5) && $2->embedding && > - !$2->embedding->nested_join->join_list.elements) > + LEX *lex=Lex; > + lex->derived_tables|= DERIVED_SUBQUERY; > + $1->linkage= DERIVED_TABLE_TYPE; > + $1->braces= FALSE; > + // Add the subtree of subquery to the current SELECT_LEX > + SELECT_LEX *curr_sel= Lex->select_stack_head(); > + DBUG_ASSERT(Lex->current_select == curr_sel); > + SELECT_LEX_UNIT *unit= $1->master_unit(); > + if (!unit) > { > - /* we have a derived table ($3 == NULL) but no alias, > - Since we are nested in further parentheses so we > - can pass NULL to the outer level parentheses > - Permits parsing of "((((select ...))) as xyz)" */ > - $$= 0; > + unit= Lex->create_unit($1); > + if (!unit) > + YYABORT; > } > - else if (!$3) > - { > - /* Handle case of derived table, alias may be NULL if there > - are no outer parentheses, add_table_to_list() will throw > - error in this case */ > - LEX *lex=Lex; > - lex->check_automatic_up(UNSPECIFIED_TYPE); > - SELECT_LEX *sel= lex->current_select; > - SELECT_LEX_UNIT *unit= sel->master_unit(); > - lex->current_select= sel= unit->outer_select(); > - Table_ident *ti= new (thd->mem_root) Table_ident(unit); > - if (ti == NULL) > - MYSQL_YYABORT; > - if (!($$= sel->add_table_to_list(thd, > - ti, $5, 0, > - TL_READ, MDL_SHARED_READ))) > + curr_sel->register_unit(unit, &curr_sel->context); > + curr_sel->add_statistics(unit); > > - MYSQL_YYABORT; > - sel->add_joined_table($$); > - //lex->pop_context("derived"); > - lex->nest_level--; > - } > - else if ($5 != NULL) > - { > - /* > - Tables with or without joins within parentheses cannot > - have aliases, and we ruled out derived tables above. > - */ > - thd->parse_error(); > - MYSQL_YYABORT; > - } > - else > - { > - /* nested join: FROM (t1 JOIN t2 ...), > - nest_level is the same as in the outer query */ > - $$= $3; > - } > - /* > - Fields in derived table can be used in upper select in > - case of merge. We do not add HAVING fields because we do > - not merge such derived. We do not add union because > - also do not merge them > - */ > - if ($$ && $$->derived && > - !$$->derived->first_select()->next_select()) > - > $$->select_lex->add_where_field($$->derived->first_select()); > - } > - /* Represents derived table with WITH clause */ > - | '(' get_select_lex subselect_start > - with_clause query_expression_body > - subselect_end ')' opt_table_alias > - { > - LEX *lex=Lex; > - SELECT_LEX *sel= $2; > - SELECT_LEX_UNIT *unit= $5->master_unit(); > Table_ident *ti= new (thd->mem_root) Table_ident(unit); > if (ti == NULL) > MYSQL_YYABORT; > - $5->set_with_clause($4); > - lex->current_select= sel; > - if (!($$= sel->add_table_to_list(lex->thd, > - ti, $8, 0, > - TL_READ, MDL_SHARED_READ))) > + if (!($$= curr_sel->add_table_to_list(lex->thd, > + ti, $2, 0, > + TL_READ, > MDL_SHARED_READ))) > MYSQL_YYABORT; > - sel->add_joined_table($$); > - } > - ; > - > -/* > - This rule accepts just about anything. The reason is that we have > - empty-producing rules in the beginning of rules, in this case > - subselect_start. This forces bison to take a decision which rules to > - reduce by long before it has seen any tokens. This approach ties us > - to a very limited class of parseable languages, and unfortunately > - SQL is not one of them. The chosen 'solution' was this rule, which > - produces just about anything, even complete bogus statements, for > - instance ( table UNION SELECT 1 ). > - Fortunately, we know that the semantic value returned by > - select_derived is NULL if it contained a derived table, and a pointer to > - the base table's TABLE_LIST if it was a base table. So in the rule > - regarding union's, we throw a parse error manually and pretend it > - was bison that did it. > - > - Also worth noting is that this rule concerns query expressions in > - the from clause only. Top level select statements and other types of > - subqueries have their own union rules. > -*/ > -select_derived_union: > - select_derived > - | select_derived union_order_or_limit > - { > - if ($1) > - { > - thd->parse_error(); > - MYSQL_YYABORT; > - } > - } > - | select_derived union_head_non_top > - { > - if ($1) > - { > - thd->parse_error(); > - MYSQL_YYABORT; > - } > } > - union_list_derived_part2 > - | derived_query_specification opt_select_lock_type > - | derived_query_specification order_or_limit opt_select_lock_type > - | derived_query_specification opt_select_lock_type > union_list_derived > - ; > - > -union_list_derived_part2: > - query_term_union_not_ready { Lex->pop_context(); } > - | query_term_union_ready { Lex->pop_context(); } > - | query_term_union_ready { Lex->pop_context(); } > union_list_derived > - ; > - > -union_list_derived: > - union_head_non_top union_list_derived_part2 > - ; > - > - > -/* The equivalent of select_init2 for nested queries. */ > -select_init2_derived: > - select_part2_derived > - { > - Select->set_braces(0); > - } > - ; > - > -/* The equivalent of select_part2 for nested queries. */ > -select_part2_derived: > - { > - LEX *lex= Lex; > - SELECT_LEX *sel= lex->current_select; > - if (sel->linkage != UNION_TYPE) > - mysql_init_select(lex); > - lex->current_select->parsing_place= SELECT_LIST; > - } > - opt_query_expression_options select_item_list > - { > - Select->parsing_place= NO_MATTER; > - } > - ; > - > -/* handle contents of parentheses in join expression */ > -select_derived: > - get_select_lex_derived derived_table_list > + | '(' > + query_expression > + ')' table_alias_clause > { > - LEX *lex= Lex; > - /* for normal joins, $2 != NULL and end_nested_join() != NULL, > - for derived tables, both must equal NULL */ > + LEX *lex=Lex; > + lex->derived_tables|= DERIVED_SUBQUERY; > + $2->first_select()->linkage= DERIVED_TABLE_TYPE; > > - if (!($$= $1->end_nested_join(lex->thd)) && $2) > - MYSQL_YYABORT; > - if (!$2 && $$) > - { > - thd->parse_error(); > - MYSQL_YYABORT; > - } > - } > - ; > > -/* > - Similar to query_specification, but for derived tables. > - Example: the inner parenthesized SELECT in this query: > - SELECT * FROM (SELECT * FROM t1); > -*/ > -derived_query_specification: > - SELECT_SYM select_derived_init select_derived2 > - { > - if ($2) > - Select->set_braces(1); > - $$= NULL; > - } > - ; > + // Add the subtree of subquery to the current SELECT_LEX > + SELECT_LEX *curr_sel= Lex->select_stack_head(); > + DBUG_ASSERT(Lex->current_select == curr_sel); > + curr_sel->register_unit($2, &curr_sel->context); > + curr_sel->add_statistics($2); > > -select_derived2: > - { > - LEX *lex= Lex; > - lex->derived_tables|= DERIVED_SUBQUERY; > - if (!lex->expr_allows_subselect || > - lex->sql_command == (int)SQLCOM_PURGE) > - { > - thd->parse_error(); > - MYSQL_YYABORT; > - } > - if (lex->current_select->linkage == GLOBAL_OPTIONS_TYPE || > - mysql_new_select(lex, 1, NULL)) > + Table_ident *ti= new (thd->mem_root) Table_ident($2); > + if (ti == NULL) > MYSQL_YYABORT; > - mysql_init_select(lex); > - lex->current_select->set_linkage(DERIVED_TABLE_TYPE); > - lex->current_select->parsing_place= SELECT_LIST; > - } > - select_options select_item_list > - { > - Select->parsing_place= NO_MATTER; > - } > - opt_table_expression > - ; > - > -get_select_lex: > - /* Empty */ { $$= Select; } > - ; > - > -get_select_lex_derived: > - get_select_lex > - { > - LEX *lex= Lex; > - if ($1->init_nested_join(lex->thd)) > + if (!($$= curr_sel->add_table_to_list(lex->thd, > + ti, $4, 0, > + TL_READ, > MDL_SHARED_READ))) > MYSQL_YYABORT; > } > - ; > - > -select_derived_init: > - { > - LEX *lex= Lex; > - > - TABLE_LIST *embedding= lex->current_select->embedding; > - $$= embedding && > - !embedding->nested_join->join_list.elements; > - /* return true if we are deeply nested */ > - } > ; > > opt_outer: > @@ -11939,9 +12231,14 @@ table_alias: > | '=' > ; > > -opt_table_alias: > +opt_table_alias_clause: > /* empty */ { $$=0; } > - | table_alias ident > + > + | table_alias_clause { $$= $1; } > + ; > + > +table_alias_clause: > + table_alias ident > { > $$= (LEX_CSTRING*) thd->memdup(&$2,sizeof(LEX_STRING)); > if ($$ == NULL) > @@ -12108,7 +12405,7 @@ opt_window_partition_clause: > > opt_window_order_clause: > /* empty */ { } > - | ORDER_SYM BY order_list > + | ORDER_SYM BY order_list { Select->order_list= *($3); } > ; > > opt_window_frame_clause: > @@ -12232,64 +12529,35 @@ alter_order_item: > > opt_order_clause: > /* empty */ > + { $$= NULL; } > | order_clause > + { $$= $1; } > ; > > order_clause: > ORDER_SYM BY > { > - LEX *lex=Lex; > - SELECT_LEX *sel= lex->current_select; > - SELECT_LEX_UNIT *unit= sel-> master_unit(); > - if (sel->linkage != GLOBAL_OPTIONS_TYPE && > - sel->olap != UNSPECIFIED_OLAP_TYPE && > - (sel->linkage != UNION_TYPE || sel->braces)) > - { > - my_error(ER_WRONG_USAGE, MYF(0), > - "CUBE/ROLLUP", "ORDER BY"); > - MYSQL_YYABORT; > - } > - if (lex->sql_command != SQLCOM_ALTER_TABLE && > - !unit->fake_select_lex) > - { > - /* > - A query of the of the form (SELECT ...) ORDER BY > order_list is > - executed in the same way as the query > - SELECT ... ORDER BY order_list > - unless the SELECT construct contains ORDER BY or LIMIT > clauses. > - Otherwise we create a fake SELECT_LEX if it has not > been created > - yet. > - */ > - SELECT_LEX *first_sl= unit->first_select(); > - if (!unit->is_unit_op() && > - (first_sl->order_list.elements || > - first_sl->select_limit) && > - unit->add_fake_select_lex(thd)) > - MYSQL_YYABORT; > - } > - if (sel->master_unit()->is_unit_op() && !sel->braces) > - { > - /* > - At this point we don't know yet whether this is the last > - select in union or not, but we move ORDER BY to > - fake_select_lex anyway. If there would be one more select > - in union mysql_new_select will correctly throw error. > - */ > - DBUG_ASSERT(sel->master_unit()->fake_select_lex); > - lex->current_select= sel->master_unit()->fake_select_lex; > - } > + thd->where= "ORDER clause"; > } > order_list > { > - > + $$= $4; > } > ; > > order_list: > order_list ',' order_ident order_dir > - { if (add_order_to_list(thd, $3,(bool) $4)) MYSQL_YYABORT; } > + { > + $$= $1; > + if (add_to_list(thd, *$$, $3,(bool) $4)) > + MYSQL_YYABORT; > + } > | order_ident order_dir > - { if (add_order_to_list(thd, $1,(bool) $2)) MYSQL_YYABORT; } > + { > + $$= new (thd->mem_root) SQL_I_List<ORDER>(); > + if (add_to_list(thd, *$$, $1, (bool) $2)) > + MYSQL_YYABORT; > + } > ; > > order_dir: > @@ -12299,63 +12567,61 @@ order_dir: > ; > > opt_limit_clause: > - /* empty */ {} > - | limit_clause {} > + /* empty */ > + { $$.empty(); } > + | limit_clause > + { $$= $1; } > ; > > -limit_clause_init: > - LIMIT > - { > - SELECT_LEX *sel= Select; > - if (sel->master_unit()->is_unit_op() && !sel->braces) > - { > - /* Move LIMIT that belongs to UNION to fake_select_lex */ > - Lex->current_select= sel->master_unit()->fake_select_lex; > - DBUG_ASSERT(Select); > - } > - } > - ; > - > limit_clause: > - limit_clause_init limit_options > + LIMIT limit_options > { > - SELECT_LEX *sel= Select; > - if (!sel->select_limit->basic_const_item() || > - sel->select_limit->val_int() > 0) > + $$= $2; > + if (!$$.select_limit->basic_const_item() || > + $$.select_limit->val_int() > 0) > Lex->set_stmt_unsafe(LEX::BINLOG_STMT_UNSAFE_LIMIT); > } > - | limit_clause_init limit_options > + | LIMIT limit_options > ROWS_SYM EXAMINED_SYM limit_rows_option > { > + $$= $2; > Lex->set_stmt_unsafe(LEX::BINLOG_STMT_UNSAFE_LIMIT); > } > - | limit_clause_init ROWS_SYM EXAMINED_SYM limit_rows_option > + | LIMIT ROWS_SYM EXAMINED_SYM limit_rows_option > { > + $$.select_limit= 0; > + $$.offset_limit= 0; > + $$.explicit_limit= 1; > Lex->set_stmt_unsafe(LEX::BINLOG_STMT_UNSAFE_LIMIT); > } > ; > > +opt_global_limit_clause: > + opt_limit_clause > + { > + Select->explicit_limit= $1.explicit_limit; > + Select->select_limit= $1.select_limit; > + Select->offset_limit= $1.offset_limit; > + } > + > limit_options: > limit_option > { > - SELECT_LEX *sel= Select; > - sel->select_limit= $1; > - sel->offset_limit= 0; > - sel->explicit_limit= 1; > + $$.select_limit= $1; > + $$.offset_limit= 0; > + $$.explicit_limit= 1; > } > | limit_option ',' limit_option > { > - SELECT_LEX *sel= Select; > - sel->select_limit= $3; > - sel->offset_limit= $1; > - sel->explicit_limit= 1; > + $$.select_limit= $3; > + $$.offset_limit= $1; > + $$.explicit_limit= 1; > } > | limit_option OFFSET_SYM limit_option > { > - SELECT_LEX *sel= Select; > - sel->select_limit= $1; > - sel->offset_limit= $3; > - sel->explicit_limit= 1; > + $$.select_limit= $1; > + $$.offset_limit= $3; > + $$.explicit_limit= 1; > } > ; > > @@ -12424,6 +12690,66 @@ delete_limit_clause: > | LIMIT limit_option ROWS_SYM EXAMINED_SYM { thd->parse_error(); > MYSQL_YYABORT; } > ; > > +query_expression_tail: > + /* empty */ { $$= NULL; } > + | order_or_limit opt_select_lock_type > + { > + $$= $1; > + $$->lock= $2; > + } > + | order_or_limit procedure_or_into opt_select_lock_type > + { > + $$= $1; > + $$->lock= $3; > + } > + | procedure_or_into opt_select_lock_type > + { > + $$= new(thd->mem_root) Lex_order_limit_lock; > + if (!$$) > + YYABORT; > + $$->order_list= NULL; > + $$->limit.empty(); > + $$->lock= $2; > + } > + | select_lock_type > + { > + $$= new(thd->mem_root) Lex_order_limit_lock; > + if (!$$) > + YYABORT; > + $$->order_list= NULL; > + $$->limit.empty(); > + $$->lock= $1; > + } > + ; > + > +procedure_or_into: > + procedure_clause > + | into > + | procedure_clause into > + ; > + > +order_or_limit: > + order_clause opt_limit_clause > + { > + $$= new(thd->mem_root) Lex_order_limit_lock; > + if (!$$) > + YYABORT; > + $$->order_list= $1; > + $$->limit= $2; > + } > + | limit_clause > + { > + Lex_order_limit_lock *op= $$= new(thd->mem_root) > Lex_order_limit_lock; > + if (!$$) > + YYABORT; > + op->order_list= NULL; > + op->limit= $1; > + $$->order_list= NULL; > + $$->limit= $1; > + } > + ; > + > + > opt_plus: > /* empty */ > | '+' > @@ -12493,14 +12819,11 @@ bool: > | TRUE_SYM { $$= 1; } > | FALSE_SYM { $$= 0; } > > - > procedure_clause: > PROCEDURE_SYM ident /* Procedure name */ > { > LEX *lex=Lex; > > - DBUG_ASSERT(&lex->builtin_select == lex->current_select); > - > lex->proc_list.elements=0; > lex->proc_list.first=0; > lex->proc_list.next= &lex->proc_list.first; > @@ -12520,6 +12843,7 @@ procedure_clause: > parameters are reduced. > */ > Lex->expr_allows_subselect= false; > + Select->options|= OPTION_PROCEDURE_CLAUSE; > } > '(' procedure_list ')' > { > @@ -12600,8 +12924,21 @@ select_outvar: > } > ; > > +opt_into: > + /* empty */ > + | into > + ; > into: > INTO into_destination > + { > + if (!(Select->options & OPTION_INTO_CLAUSE)) > + Select->options|= OPTION_INTO_CLAUSE; > + else > + { > + my_error(ER_CANT_USE_OPTION_HERE, MYF(0), "INTO"); > + MYSQL_YYABORT; > + } > + } > ; > > into_destination: > @@ -12647,10 +12984,15 @@ do: > LEX *lex=Lex; > lex->sql_command = SQLCOM_DO; > mysql_init_select(lex); > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > } > expr_list > { > Lex->insert_list= $3; > + Lex->pop_select(); //main select > + if (Lex->check_main_unit_semantics()) > + MYSQL_YYABORT; > } > ; > > @@ -12882,17 +13224,24 @@ insert: > { > LEX *lex= Lex; > lex->sql_command= SQLCOM_INSERT; > - lex->duplicates= DUP_ERROR; > - mysql_init_select(lex); > + lex->duplicates= DUP_ERROR; > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > + mysql_init_select(lex); > + lex->current_select->parsing_place= BEFORE_OPT_LIST; > } > insert_lock_option > opt_ignore insert2 > { > Select->set_lock_for_tables($3); > - Lex->current_select= &Lex->builtin_select; > + Lex->current_select= Lex->first_select_lex(); > } > insert_field_spec opt_insert_update > - {} > + { > + Lex->pop_select(); //main select > + if (Lex->check_main_unit_semantics()) > + MYSQL_YYABORT; > + } > ; > > replace: > @@ -12901,15 +13250,22 @@ replace: > LEX *lex=Lex; > lex->sql_command = SQLCOM_REPLACE; > lex->duplicates= DUP_REPLACE; > - mysql_init_select(lex); > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > + mysql_init_select(lex); > + lex->current_select->parsing_place= BEFORE_OPT_LIST; > } > replace_lock_option insert2 > { > Select->set_lock_for_tables($3); > - Lex->current_select= &Lex->builtin_select; > + Lex->current_select= Lex->first_select_lex(); > } > insert_field_spec > - {} > + { > + Lex->pop_select(); //main select > + if (Lex->check_main_unit_semantics()) > + MYSQL_YYABORT; > + } > ; > > insert_lock_option: > @@ -12955,35 +13311,47 @@ insert_table: > table_name_with_opt_use_partition > { > LEX *lex=Lex; > - lex->field_list.empty(); > + //lex->field_list.empty(); > lex->many_values.empty(); > lex->insert_list=0; > }; > > insert_field_spec: > insert_values {} > - | '(' ')' insert_values {} > - | '(' fields ')' insert_values {} > + | insert_field_list insert_values {} > | SET > { > LEX *lex=Lex; > if (!(lex->insert_list= new (thd->mem_root) List_item) || > lex->many_values.push_back(lex->insert_list, > thd->mem_root)) > MYSQL_YYABORT; > + lex->current_select->parsing_place= NO_MATTER; > } > ident_eq_list > ; > > +insert_field_list: > + LEFT_PAREN_ALT opt_fields ')' > + { > + Lex->current_select->parsing_place= AFTER_LIST; > + } > + ; > + > +opt_fields: > + /* empty */ > + | fields > + ; > + > fields: > fields ',' insert_ident > { Lex->field_list.push_back($3, thd->mem_root); } > | insert_ident { Lex->field_list.push_back($1, thd->mem_root); } > ; > > + > + > insert_values: > - VALUES values_list {} > - | VALUE_SYM values_list {} > - | create_select_query_expression {} > + create_select_query_expression {} > ; > > values_list: > @@ -13093,6 +13461,8 @@ update: > UPDATE_SYM > { > LEX *lex= Lex; > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > mysql_init_select(lex); > lex->sql_command= SQLCOM_UPDATE; > lex->duplicates= DUP_ERROR; > @@ -13118,7 +13488,14 @@ update: > */ > Select->set_lock_for_tables($3); > } > - opt_where_clause opt_order_clause delete_limit_clause {} > + opt_where_clause opt_order_clause delete_limit_clause > + { > + if ($10) > + Select->order_list= *($10); > + Lex->pop_select(); //main select > + if (Lex->check_main_unit_semantics()) > + MYSQL_YYABORT; > + } > ; > > update_list: > @@ -13164,6 +13541,8 @@ delete: > mysql_init_select(lex); > YYPS->m_lock_type= TL_WRITE_DEFAULT; > YYPS->m_mdl_type= MDL_SHARED_WRITE; > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > > lex->ignore= 0; > lex->builtin_select.init_order(); > @@ -13185,7 +13564,12 @@ single_multi: > } > opt_where_clause opt_order_clause > delete_limit_clause {} > - opt_select_expressions {} > + opt_select_expressions > + { > + if ($6) > + Select->order_list= *($6); > + Lex->pop_select(); //main select > + } > | table_wild_list > { > mysql_init_multi_delete(Lex); > @@ -13196,6 +13580,9 @@ single_multi: > { > if (multi_delete_set_locks_and_link_aux_tables(Lex)) > MYSQL_YYABORT; > + Lex->pop_select(); //main select > + if (Lex->check_main_unit_semantics()) > + MYSQL_YYABORT; > } > | FROM table_alias_ref_list > { > @@ -13207,6 +13594,9 @@ single_multi: > { > if (multi_delete_set_locks_and_link_aux_tables(Lex)) > MYSQL_YYABORT; > + Lex->pop_select(); //main select > + if (Lex->check_main_unit_semantics()) > + MYSQL_YYABORT; > } > ; > > @@ -13284,6 +13674,7 @@ truncate: > lex->m_sql_cmd= new (thd->mem_root) Sql_cmd_truncate_table(); > if (lex->m_sql_cmd == NULL) > MYSQL_YYABORT; > + Lex->pop_select(); //main select > } > opt_truncate_table_storage_clause { } > ; > @@ -13365,6 +13756,8 @@ show: > LEX *lex=Lex; > lex->wild=0; > lex->ident= null_clex_str; > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > mysql_init_select(lex); > lex->current_select->parsing_place= SELECT_LIST; > lex->create_info.init(); > @@ -13372,6 +13765,7 @@ show: > show_param > { > Select->parsing_place= NO_MATTER; > + Lex->pop_select(); //main select > } > ; > > @@ -13387,7 +13781,7 @@ show_param: > { > LEX *lex= Lex; > lex->sql_command= SQLCOM_SHOW_TABLES; > - lex->builtin_select.db= $3; > + lex->first_select_lex()->db= $3; > if (prepare_schema_table(thd, lex, 0, SCH_TABLE_NAMES)) > MYSQL_YYABORT; > } > @@ -13395,7 +13789,7 @@ show_param: > { > LEX *lex= Lex; > lex->sql_command= SQLCOM_SHOW_TRIGGERS; > - lex->builtin_select.db= $3; > + lex->first_select_lex()->db= $3; > if (prepare_schema_table(thd, lex, 0, SCH_TRIGGERS)) > MYSQL_YYABORT; > } > @@ -13403,7 +13797,7 @@ show_param: > { > LEX *lex= Lex; > lex->sql_command= SQLCOM_SHOW_EVENTS; > - lex->builtin_select.db= $2; > + lex->first_select_lex()->db= $2; > if (prepare_schema_table(thd, lex, 0, SCH_EVENTS)) > MYSQL_YYABORT; > } > @@ -13411,7 +13805,7 @@ show_param: > { > LEX *lex= Lex; > lex->sql_command= SQLCOM_SHOW_TABLE_STATUS; > - lex->builtin_select.db= $3; > + lex->first_select_lex()->db= $3; > if (prepare_schema_table(thd, lex, 0, SCH_TABLES)) > MYSQL_YYABORT; > } > @@ -13419,7 +13813,7 @@ show_param: > { > LEX *lex= Lex; > lex->sql_command= SQLCOM_SHOW_OPEN_TABLES; > - lex->builtin_select.db= $3; > + lex->first_select_lex()->db= $3; > if (prepare_schema_table(thd, lex, 0, SCH_OPEN_TABLES)) > MYSQL_YYABORT; > } > @@ -13469,12 +13863,13 @@ show_param: > LEX *lex= Lex; > lex->sql_command= SQLCOM_SHOW_BINLOG_EVENTS; > } > - opt_limit_clause > + opt_global_limit_clause > | RELAYLOG_SYM optional_connection_name EVENTS_SYM binlog_in > binlog_from > { > LEX *lex= Lex; > lex->sql_command= SQLCOM_SHOW_RELAYLOG_EVENTS; > - } opt_limit_clause > + } > + opt_global_limit_clause > | keys_or_index from_or_in table_ident opt_db opt_where_clause > { > LEX *lex= Lex; > @@ -13516,13 +13911,13 @@ show_param: > LEX_CSTRING var= {STRING_WITH_LEN("error_count")}; > (void) create_select_for_variable(thd, &var); > } > - | WARNINGS opt_limit_clause > + | WARNINGS opt_global_limit_clause > { Lex->sql_command = SQLCOM_SHOW_WARNS;} > - | ERRORS opt_limit_clause > + | ERRORS opt_global_limit_clause > { Lex->sql_command = SQLCOM_SHOW_ERRORS;} > | PROFILES_SYM > { Lex->sql_command = SQLCOM_SHOW_PROFILES; } > - | PROFILE_SYM opt_profile_defs opt_profile_args opt_limit_clause > + | PROFILE_SYM opt_profile_defs opt_profile_args > opt_global_limit_clause > { > LEX *lex= Lex; > lex->sql_command= SQLCOM_SHOW_PROFILE; > @@ -13829,7 +14224,7 @@ describe: > explainable_command > { > LEX *lex=Lex; > - lex->builtin_select.options|= SELECT_DESCRIBE; > + lex->first_select_lex()->options|= SELECT_DESCRIBE; > } > ; > > @@ -13855,6 +14250,8 @@ analyze_stmt_command: > > opt_extended_describe: > EXTENDED_SYM { Lex->describe|= DESCRIBE_EXTENDED; } > + | EXTENDED_SYM ALL > + { Lex->describe|= DESCRIBE_EXTENDED | DESCRIBE_EXTENDED2; } > | PARTITIONS_SYM { Lex->describe|= DESCRIBE_PARTITIONS; } > | opt_format_json {} > ; > @@ -13897,8 +14294,7 @@ flush: > lex->type= 0; > lex->no_write_to_binlog= $2; > } > - flush_options > - {} > + flush_options {} > ; > > flush_options: > @@ -13915,6 +14311,7 @@ flush_options: > opt_table_list opt_flush_lock > {} > | flush_options_list > + {} > ; > > opt_flush_lock: > @@ -14070,9 +14467,13 @@ purge: > LEX *lex=Lex; > lex->type=0; > lex->sql_command = SQLCOM_PURGE; > + if (lex->main_select_push()) > + MYSQL_YYABORT; > } > purge_options > - {} > + { > + Lex->pop_select(); //main select > + } > ; > > purge_options: > @@ -14090,6 +14491,8 @@ purge_option: > lex->value_list.empty(); > lex->value_list.push_front($2, thd->mem_root); > lex->sql_command= SQLCOM_PURGE_BEFORE; > + if (Lex->check_main_unit_semantics()) > + MYSQL_YYABORT; > } > ; > > @@ -14099,6 +14502,8 @@ kill: > KILL_SYM > { > LEX *lex=Lex; > + if (lex->main_select_push()) > + YYABORT; > lex->value_list.empty(); > lex->users_list.empty(); > lex->sql_command= SQLCOM_KILL; > @@ -14107,6 +14512,7 @@ kill: > kill_type kill_option kill_expr > { > Lex->kill_signal= (killed_state) ($3 | $4); > + Lex->pop_select(); //main select > } > ; > > @@ -14167,6 +14573,8 @@ load: > $2 == FILETYPE_CSV ? "LOAD DATA" : "LOAD XML"); > MYSQL_YYABORT; > } > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > } > load_data_lock opt_local INFILE TEXT_STRING_filesystem > { > @@ -14193,7 +14601,11 @@ load: > opt_xml_rows_identified_by > opt_field_term opt_line_term opt_ignore_lines > opt_field_or_var_spec > opt_load_data_set_spec > - {} > + { > + Lex->pop_select(); //main select > + if (Lex->check_main_unit_semantics()) > + MYSQL_YYABORT; > + } > ; > > data_or_xml: > @@ -14597,17 +15009,21 @@ opt_with_clause: > > > with_clause: > - WITH opt_recursive > + WITH opt_recursive > { > + LEX *lex= Lex; > With_clause *with_clause= > new With_clause($2, Lex->curr_with_clause); > if (with_clause == NULL) > MYSQL_YYABORT; > - Lex->derived_tables|= DERIVED_WITH; > - Lex->curr_with_clause= with_clause; > + lex->derived_tables|= DERIVED_WITH; > + lex->curr_with_clause= with_clause; > with_clause->add_to_list(Lex->with_clauses_list_last_next); > + if (lex->current_select && > + lex->current_select->parsing_place == BEFORE_OPT_LIST) > + lex->current_select->parsing_place= NO_MATTER; > } > - with_list > + with_list > { > $$= Lex->curr_with_clause; > Lex->curr_with_clause= Lex->curr_with_clause->pop(); > @@ -14636,9 +15052,9 @@ with_list_element: > MYSQL_YYABORT; > Lex->with_column_list.empty(); > } > - AS '(' remember_name subselect remember_end ')' > + AS '(' remember_name query_expression remember_end ')' > { > - With_element *elem= new With_element($1, *$2, > $7->master_unit()); > + With_element *elem= new With_element($1, *$2, $7); > if (elem == NULL || Lex->curr_with_clause->add_with_element(elem)) > MYSQL_YYABORT; > if (elem->set_unparsed_spec(thd, $6+1, $8)) > @@ -15589,14 +16005,22 @@ set: > SET > { > LEX *lex=Lex; > + if (lex->main_select_push()) > + MYSQL_YYABORT; > lex->set_stmt_init(); > lex->var_list.empty(); > sp_create_assignment_lex(thd, yychar == YYEMPTY); > } > start_option_value_list > - {} > + { > + Lex->pop_select(); //main select > + if (Lex->check_main_unit_semantics()) > + MYSQL_YYABORT; > + } > | SET STATEMENT_SYM > { > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > Lex->set_stmt_init(); > } > set_stmt_option_value_following_option_type_list > @@ -15606,6 +16030,9 @@ set: > my_yyabort_error((ER_SUBQUERIES_NOT_SUPPORTED, MYF(0), > "SET STATEMENT")); > lex->stmt_var_list= lex->var_list; > lex->var_list.empty(); > + Lex->pop_select(); //main select > + if (Lex->check_main_unit_semantics()) > + MYSQL_YYABORT; > } > FOR_SYM verb_clause > {} > @@ -16038,9 +16465,13 @@ lock: > if (lex->sphead) > my_yyabort_error((ER_SP_BADSTATEMENT, MYF(0), "LOCK")); > lex->sql_command= SQLCOM_LOCK_TABLES; > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > } > table_lock_list opt_lock_wait_timeout > - {} > + { > + Lex->pop_select(); //main select > + } > ; > > opt_lock_wait_timeout: > @@ -16071,7 +16502,7 @@ table_lock_list: > ; > > table_lock: > - table_ident opt_table_alias lock_option > + table_ident opt_table_alias_clause lock_option > { > thr_lock_type lock_type= (thr_lock_type) $3; > bool lock_for_write= (lock_type >= TL_WRITE_ALLOW_WRITE); > @@ -16105,9 +16536,13 @@ unlock: > if (lex->sphead) > my_yyabort_error((ER_SP_BADSTATEMENT, MYF(0), "UNLOCK")); > lex->sql_command= SQLCOM_UNLOCK_TABLES; > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > } > table_or_tables > - {} > + { > + Lex->pop_select(); //main select > + } > ; > > /* > @@ -16115,25 +16550,36 @@ unlock: > */ > > handler: > - HANDLER_SYM table_ident OPEN_SYM opt_table_alias > + HANDLER_SYM > + { > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > + } > + handler_tail > + { > + Lex->pop_select(); //main select > + } > + > +handler_tail: > + table_ident OPEN_SYM opt_table_alias_clause > { > LEX *lex= Lex; > if (lex->sphead) > my_yyabort_error((ER_SP_BADSTATEMENT, MYF(0), "HANDLER")); > lex->sql_command = SQLCOM_HA_OPEN; > - if (!lex->current_select->add_table_to_list(thd, $2, $4, 0)) > + if (!lex->current_select->add_table_to_list(thd, $1, $3, 0)) > MYSQL_YYABORT; > } > - | HANDLER_SYM table_ident_nodb CLOSE_SYM > + | table_ident_nodb CLOSE_SYM > { > LEX *lex= Lex; > if (lex->sphead) > my_yyabort_error((ER_SP_BADSTATEMENT, MYF(0), "HANDLER")); > lex->sql_command = SQLCOM_HA_CLOSE; > - if (!lex->current_select->add_table_to_list(thd, $2, 0, 0)) > + if (!lex->current_select->add_table_to_list(thd, $1, 0, 0)) > MYSQL_YYABORT; > } > - | HANDLER_SYM table_ident_nodb READ_SYM > + | table_ident_nodb READ_SYM > { > LEX *lex=Lex; > if (lex->sphead) > @@ -16141,20 +16587,24 @@ handler: > lex->expr_allows_subselect= FALSE; > lex->sql_command = SQLCOM_HA_READ; > lex->ha_rkey_mode= HA_READ_KEY_EXACT; /* Avoid purify > warnings */ > - Item *one= new (thd->mem_root) Item_int(thd, (int32) 1); > - if (one == NULL) > - MYSQL_YYABORT; > - lex->current_select->select_limit= one; > - lex->current_select->offset_limit= 0; > - lex->limit_rows_examined= 0; > - if (!lex->current_select->add_table_to_list(thd, $2, 0, 0)) > + if (!lex->current_select->add_table_to_list(thd, $1, 0, 0)) > MYSQL_YYABORT; > } > - handler_read_or_scan opt_where_clause opt_limit_clause > + handler_read_or_scan opt_where_clause opt_global_limit_clause > { > - Lex->expr_allows_subselect= TRUE; > + LEX *lex=Lex; > + lex->expr_allows_subselect= TRUE; > + if (!lex->current_select->explicit_limit) > + { > + Item *one= new (thd->mem_root) Item_int(thd, (int32) 1); > + if (one == NULL) > + MYSQL_YYABORT; > + lex->current_select->select_limit= one; > + lex->current_select->offset_limit= 0; > + lex->limit_rows_examined= 0; > + } > /* Stored functions are not supported for HANDLER READ. */ > - if (Lex->uses_stored_routines()) > + if (lex->uses_stored_routines()) > { > my_error(ER_NOT_SUPPORTED_YET, MYF(0), > "stored functions in HANDLER ... READ"); > @@ -16800,83 +17250,16 @@ release: > */ > > unit_type_decl: > - UNION_SYM > - { $$= UNION_TYPE; } > + UNION_SYM union_option > + { $$.unit_type= UNION_TYPE; $$.distinct= $2; } > | INTERSECT_SYM > - { $$= INTERSECT_TYPE; } > + { $$.unit_type= INTERSECT_TYPE; $$.distinct= 1; } > | EXCEPT_SYM > - { $$= EXCEPT_TYPE; } > - > - > -union_clause: > - /* empty */ {} > - | union_list > - ; > - > -union_list: > - unit_type_decl union_option > - { > - if (Lex->add_select_to_union_list((bool)$2, $1, TRUE)) > - MYSQL_YYABORT; > - } > - union_list_part2 > - { > - /* > - Remove from the name resolution context stack the context > of the > - last select in the union. > - */ > - Lex->pop_context(); > - } > - ; > - > -union_list_view: > - unit_type_decl union_option > - { > - if (Lex->add_select_to_union_list((bool)$2, $1, TRUE)) > - MYSQL_YYABORT; > - } > - query_expression_body_view > - { > - Lex->pop_context(); > - } > - ; > - > -union_order_or_limit: > - { > - LEX *lex= thd->lex; > - DBUG_ASSERT(lex->current_select->linkage != > GLOBAL_OPTIONS_TYPE); > - SELECT_LEX *sel= lex->current_select; > - SELECT_LEX_UNIT *unit= sel->master_unit(); > - SELECT_LEX *fake= unit->fake_select_lex; > - if (fake) > - { > - fake->no_table_names_allowed= 1; > - lex->current_select= fake; > - } > - thd->where= "global ORDER clause"; > - } > - order_or_limit > - { > - thd->lex->current_select->no_table_names_allowed= 0; > - thd->where= ""; > - } > - ; > - > -order_or_limit: > - order_clause opt_limit_clause > - | limit_clause > - ; > + { $$.unit_type= EXCEPT_TYPE; $$.distinct= 1; } > > /* > Start a UNION, for non-top level query expressions. > */ > -union_head_non_top: > - unit_type_decl union_option > - { > - if (Lex->add_select_to_union_list((bool)$2, $1, FALSE)) > - MYSQL_YYABORT; > - } > - ; > > union_option: > /* empty */ { $$=1; } > @@ -16884,110 +17267,10 @@ union_option: > | ALL { $$=0; } > ; > > -/* > - Corresponds to the SQL Standard > - <query specification> ::= > - SELECT [ <set quantifier> ] <select list> <table expression> > - > - Notes: > - - We allow more options in addition to <set quantifier> > - - <table expression> is optional in MariaDB > -*/ > -query_specification: > - SELECT_SYM select_init2_derived opt_table_expression > - { > - $$= Lex->current_select->master_unit()->first_select(); > - } > - ; > - > -query_term_union_not_ready: > - query_specification order_or_limit opt_select_lock_type { $$= > $1; } > - | '(' select_paren_derived ')' union_order_or_limit { $$= > $2; } > - ; > - > -query_term_union_ready: > - query_specification opt_select_lock_type { $$= > $1; } > - | '(' select_paren_derived ')' { $$= > $2; } > - ; > - > -query_expression_body: > - query_term_union_not_ready { > $$= $1; } > - | query_term_union_ready { > $$= $1; } > - | query_term_union_ready union_list_derived { > $$= $1; } > - ; > - > -/* Corresponds to <query expression> in the SQL:2003 standard. */ > -subselect: > - subselect_start opt_with_clause query_expression_body > subselect_end > - { > - $3->set_with_clause($2); > - $$= $3; > - } > - ; > - > -subselect_start: > - { > - LEX *lex=Lex; > - if (!lex->expr_allows_subselect || > - lex->sql_command == (int)SQLCOM_PURGE) > - { > - thd->parse_error(); > - MYSQL_YYABORT; > - } > - /* > - we are making a "derived table" for the parenthesis > - as we need to have a lex level to fit the union > - after the parenthesis, e.g. > - (SELECT .. ) UNION ... becomes > - SELECT * FROM ((SELECT ...) UNION ...) > - */ > - if (mysql_new_select(Lex, 1, NULL)) > - MYSQL_YYABORT; > - } > - ; > - > -subselect_end: > - { > - LEX *lex=Lex; > - > - lex->check_automatic_up(UNSPECIFIED_TYPE); > - lex->pop_context(); > - SELECT_LEX *child= lex->current_select; > - lex->current_select = > lex->current_select->return_after_parsing(); > - lex->nest_level--; > - lex->current_select->n_child_sum_items += child->n_sum_items; > - /* > - A subselect can add fields to an outer select. Reserve > space for > - them. > - */ > - lex->current_select->select_n_where_fields+= > - child->select_n_where_fields; > - > - /* > - Aggregate functions in having clause may add fields to an > outer > - select. Count them also. > - */ > - lex->current_select->select_n_having_items+= > - child->select_n_having_items; > - } > - ; > - > -opt_query_expression_options: > - /* empty */ > - | query_expression_option_list > - ; > - > -query_expression_option_list: > - query_expression_option_list query_expression_option > - | query_expression_option > - ; > - > query_expression_option: > STRAIGHT_JOIN { Select->options|= SELECT_STRAIGHT_JOIN; } > | HIGH_PRIORITY > { > - if (check_simple_select()) > - MYSQL_YYABORT; > YYPS->m_lock_type= TL_READ_HIGH_PRIORITY; > YYPS->m_mdl_type= MDL_SHARED_READ; > Select->options|= SELECT_HIGH_PRIORITY; > @@ -16996,18 +17279,8 @@ query_expression_option: > | UNIQUE_SYM { Select->options|= SELECT_DISTINCT; } > | SQL_SMALL_RESULT { Select->options|= SELECT_SMALL_RESULT; } > | SQL_BIG_RESULT { Select->options|= SELECT_BIG_RESULT; } > - | SQL_BUFFER_RESULT > - { > - if (check_simple_select()) > - MYSQL_YYABORT; > - Select->options|= OPTION_BUFFER_RESULT; > - } > - | SQL_CALC_FOUND_ROWS > - { > - if (check_simple_select()) > - MYSQL_YYABORT; > - Select->options|= OPTION_FOUND_ROWS; > - } > + | SQL_BUFFER_RESULT { Select->options|= OPTION_BUFFER_RESULT; } > + | SQL_CALC_FOUND_ROWS { Select->options|= OPTION_FOUND_ROWS; } > | ALL { Select->options|= SELECT_ALL; } > ; > > @@ -17095,32 +17368,28 @@ view_select: > lex->parsing_options.allows_variable= FALSE; > lex->create_view->select.str= (char *) YYLIP->get_cpp_ptr(); > } > - opt_with_clause query_expression_body_view view_check_option > + query_expression > + view_check_option > { > LEX *lex= Lex; > + SQL_I_List<TABLE_LIST> *save= > &lex->first_select_lex()->table_list; > + lex->set_main_unit($2); > + if (lex->check_main_unit_semantics()) > + MYSQL_YYABORT; > + lex->first_select_lex()->table_list.push_front(save); > + lex->current_select= Lex->first_select_lex(); > size_t len= YYLIP->get_cpp_ptr() - > lex->create_view->select.str; > void *create_view_select= > thd->memdup(lex->create_view->select.str, len); > lex->create_view->select.length= len; > lex->create_view->select.str= (char *) create_view_select; > + size_t not_used; > trim_whitespace(thd->charset(), > - &lex->create_view->select); > - lex->create_view->check= $4; > + &lex->create_view->select, &not_used); > + lex->create_view->check= $3; > lex->parsing_options.allows_variable= TRUE; > - lex->current_select->set_with_clause($2); > } > ; > > -/* > - SQL Standard <query expression body> for VIEWs. > - Does not include INTO and PROCEDURE clauses. > -*/ > -query_expression_body_view: > - SELECT_SYM select_options_and_item_list select_init3_view > - | '(' select_paren_view ')' > - | '(' select_paren_view ')' union_order_or_limit > - | '(' select_paren_view ')' union_list_view > - ; > - > view_check_option: > /* empty */ { $$= VIEW_CHECK_NONE; } > | WITH CHECK_SYM OPTION { $$= VIEW_CHECK_CASCADED; } > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits >

2 5

[Maria-developers] MDEV-14448
by Badrul Chowdhury 16 Apr '18

16 Apr '18

Hello, Is anyone working on https://jira.mariadb.org/browse/MDEV-14448? Pointers to documentation on the procedure to adopt upstream fixes would be appreciated also. Thanks, Badrul

2 2

Re: [Maria-developers] Observation for CHECK_CONSTRAINTS MDEV 14474
by Vicențiu Ciorbaru 14 Apr '18

14 Apr '18

Hi Anel! I've reviewed your patch and generally things look good. You do some good analysis usually and I think we should highlight your work to the broader developer scene. I've thus CC'ed this email to the maria-developers mailing list. This gives us a chance to get feedback from outside, particularly as this feature is of general interest to the community. Please include the .result files in future versions as it makes reviewing a lot easier. Serg: coding style and implementation aside, I'll do the review on that bit, please have a look at Anel's patch and suggest if you think the current table header for I_S.check_constraints is good enough for our needs or if you have more input on this task. Relevant bit: +ST_FIELD_INFO check_constraints_fields_info[]= +{ + {"CONSTRAINT_CATALOG", FN_REFLEN, MYSQL_TYPE_STRING, 0, 0, 0, OPEN_FULL_TABLE}, + {"CONSTRAINT_SCHEMA", NAME_CHAR_LEN, MYSQL_TYPE_STRING, 0, 0, 0, + OPEN_FULL_TABLE}, + {"CONSTRAINT_NAME", NAME_CHAR_LEN, MYSQL_TYPE_STRING, 0, 0, 0, + OPEN_FULL_TABLE}, + {"TABLE_SCHEMA", NAME_CHAR_LEN, MYSQL_TYPE_STRING, 0, 0, 0, OPEN_FULL_TABLE}, + {"TABLE_NAME", NAME_CHAR_LEN, MYSQL_TYPE_STRING, 0, 0, 0, OPEN_FULL_TABLE}, + {"CHECK_CLAUSE", NAME_CHAR_LEN, MYSQL_TYPE_STRING, 0, 0, 0, + OPEN_FULL_TABLE}, + {0, 0, MYSQL_TYPE_STRING, 0, 0, 0, SKIP_OPEN_TABLE} +}; Vicențiu On Thu, 29 Mar 2018 at 12:01 Anel Husakovic <anel(a)mariadb.org> wrote: > Hi Vin, > > I'm pleased to announce my new patch prototype for MDEV 14474 where are > implemented field and table constraints: > > https://github.com/an3l/server/commit/f92fd77d2695be52faf280245faa014c44617… > > According to the standard there are only 4 column to represent. > CONSTRAINT_CATALOG, CONSTRAINT_SCHEMA, CONSTRAINT_NAME,CHECK_CLAUSE > But because Serg mentioned that alternative would be to change > *information_schema.table_constraints* with adding only 1 column ( > *check_clause*) I also used 2 additional columns which are represented > also in *table_constraints *and they are TABLE_SHEMA , TABLE_NAME. > > Function of interest is: get_check_constraints_record > > Also I added 2 test cases: > > One that works with only 1 table: MDEV14474-one_table_play.test > <https://github.com/an3l/server/commit/f92fd77d2695be52faf280245faa014c44617…> > Second that works with more tables: MDEV14474-I_S_check_constraint.test > <https://github.com/an3l/server/commit/f92fd77d2695be52faf280245faa014c44617…> > > Last night I had a problem with field constraints. > As a reference I used only way how currently all check constraints are > shown and its obtained via* show create table t;* > > Bellow is shown that field constraints were not visible as well as that > name used for them is *NULL*, at least this is how *show create table* > represented them ( there is no way to give a name to a field constraint* > for example: f int constraint name check(f<0)*). According to my opinion, > because this constraint belongs to the field, it should be also called as a > name of a field and not NULL. > As an argument to my opinion here is mariadb kb > https://mariadb.com/kb/en/library/constraint/ > Please take a look at *constraint **name*, which is field constraint and > error message. > > INSERT INTO t2(name, start_date, end_date) VALUES('Io', '2003-12-15', '2014-11-09');ERROR 4022 (23000): CONSTRAINT `name` failed for `test`.`t2` > > If the name of constraint = name of the field , than definitely it is not > consistent with* show create table* output, and *show create table* > output *needs to be changed*, , but this is question that I would like > to discuss with you ?! > > When you try to use show create table x > [image: image.png] > > > I'm waiting on your reply for global temporary tables as well as > check_constraints and how to proceed further > > *Not understand yet a concept*: > > In *sql/table.h* > There is a struct* st_schema_table *which has fill data and process data. > > How they work, and how to use that on other database for example PSI. > > Regards, > > Anel >

2 1

Re: [Maria-developers] [Commits] a74e3ef17e7: MDEV-14551 Can't find record in table on multi-table update with ORDER BY
by Sergey Petrunia 13 Apr '18

13 Apr '18

Hi Serg, Please find some cosmetic input below. (additionally, we've found a failing example while chatting) On Tue, Apr 10, 2018 at 02:10:17PM +0200, serg(a)mariadb.org wrote: > revision-id: a74e3ef17e7a8693d68001290a8d91b5b206804d (mariadb-10.3.5-138-ga74e3ef17e7) > parent(s): 9bd3af97dffa411657879bbdfd4dfef38c99f7cc > author: Sergei Golubchik > committer: Sergei Golubchik > timestamp: 2018-04-10 14:09:17 +0200 > message: > > MDEV-14551 Can't find record in table on multi-table update with ORDER BY ... > diff --git a/mysql-test/main/multi_update.test b/mysql-test/main/multi_update.test > index 5feebe87a5a..96e78dab82b 100644 > --- a/mysql-test/main/multi_update.test > +++ b/mysql-test/main/multi_update.test > @@ -914,3 +914,14 @@ update t1 set c1=NULL; > update t1, t2 set t1.c1=t2.c3 where t1.c3=t2.c3 order by t1.c3 desc limit 2; > select * from t1; > drop table t1, t2; > + > +# > +# MDEV-14551 Can't find record in table on multi-table update with ORDER BY > +# > +CREATE TABLE t1 (i INT) ENGINE=MEMORY; > +INSERT t1 VALUES (1),(2); > +CREATE TABLE t2 (f INT) ENGINE=MyISAM; > +INSERT t2 VALUES (1),(2); > +UPDATE t1, t2 SET f = 126 ORDER BY f LIMIT 2; > +SELECT * FROM t2; > +DROP TABLE t1, t2; > diff --git a/sql/item.h b/sql/item.h > index 9574bdc63bf..e391e7810c4 100644 > --- a/sql/item.h > +++ b/sql/item.h > @@ -2023,6 +2023,7 @@ class Item: public Value_source, > { > marker &= ~EXTRACTION_MASK; > } > + virtual TABLE *rowid_table() const { return 0; } I am concerned about this approach bloating class Item's vtable. There is (and likely will ever be) only one class with a different implementation, the check for this is made only in one place - why not check Item's type() or functype(), etc? > }; > > MEM_ROOT *get_thd_memroot(THD *thd); > diff --git a/sql/item_strfunc.cc b/sql/item_strfunc.cc > index c6b0ae7fba8..9097ffca867 100644 > --- a/sql/item_strfunc.cc > +++ b/sql/item_strfunc.cc > @@ -5204,3 +5204,23 @@ String *Item_func_dyncol_list::val_str(String *str) > my_free(names); > return NULL; > } > + > +Item_temptable_rowid::Item_temptable_rowid(TABLE *table_arg) > + : Item_str_func(table_arg->in_use), table(table_arg) > +{ > + max_length= table->file->ref_length; > +} > + > +void Item_temptable_rowid::fix_length_and_dec() > +{ > + used_tables_cache= table->map; > + const_item_cache= false; > +} > + > +String *Item_temptable_rowid::val_str(String *str) > +{ > + if (!((null_value= table->null_row))) > + table->file->position(table->record[0]); > + str_value.set((char*)(table->file->ref), max_length, &my_charset_bin); > + return &str_value; > +} > diff --git a/sql/item_strfunc.h b/sql/item_strfunc.h > index 18cda491efd..49de5568696 100644 > --- a/sql/item_strfunc.h > +++ b/sql/item_strfunc.h > @@ -1748,5 +1748,20 @@ class Item_func_dyncol_list: public Item_str_func > { return get_item_copy<Item_func_dyncol_list>(thd, this); } > }; > > -#endif /* ITEM_STRFUNC_INCLUDED */ Please add a note that this is not the "_rowid" that we support in the parser. > +class Item_temptable_rowid :public Item_str_func > +{ > + TABLE *table; > +public: > + Item_temptable_rowid(TABLE *table_arg); > + const Type_handler *type_handler() const { return &type_handler_string; } > + Field *create_tmp_field(bool group, TABLE *table) > + { return create_table_field_from_handler(table); } > + String *val_str(String *str); > + const char *func_name() const { return "<rowid>"; } > + void fix_length_and_dec(); > + Item *get_copy(THD *thd) > + { return get_item_copy<Item_temptable_rowid>(thd, this); } > + TABLE *rowid_table() const { return table; } > +}; > > +#endif /* ITEM_STRFUNC_INCLUDED */ ... > diff --git a/sql/sql_select.cc b/sql/sql_select.cc > index 796ea569e64..22301319680 100644 > --- a/sql/sql_select.cc > +++ b/sql/sql_select.cc > @@ -2650,6 +2650,21 @@ bool JOIN::add_having_as_table_cond(JOIN_TAB *tab) > } > > > +bool JOIN::add_fields_for_current_rowid(JOIN_TAB *cur, List<Item> *table_fields) I think ths is actually "current rowids", in plural. As the loop shows, there are multiple current rowids (from different tables). > +{ > + for (JOIN_TAB *tab=join_tab; tab < cur; tab++) Please add a comment: this will not walk into semi-join materialization nests but this is ok because we will never need to save current rowids for those. > + { > + if (!tab->keep_current_rowid) > + continue; > + Item *item= new (thd->mem_root) Item_temptable_rowid(tab->table); > + item->fix_fields(thd, 0); > + table_fields->push_back(item, thd->mem_root); > + cur->tmp_table_param->func_count++; > + } > + return 0; > +} > + > + > /** > Set info for aggregation tables > > diff --git a/sql/sql_select.h b/sql/sql_select.h > index f8911fbba01..1da87bb9d50 100644 > --- a/sql/sql_select.h > +++ b/sql/sql_select.h > @@ -1438,6 +1438,9 @@ class JOIN :public Sql_alloc > > enum { QEP_NOT_PRESENT_YET, QEP_AVAILABLE, QEP_DELETED} have_query_plan; > > + // if keep_current_rowid=true, whether they should be saved in temporary table > + bool tmp_table_keep_current_rowid; Same input as with add_fields_for_current_rowid - use plural. > + > /* > Additional WHERE and HAVING predicates to be considered for IN=>EXISTS > subquery transformation of a JOIN object. > diff --git a/sql/sql_union.cc b/sql/sql_union.cc > index 0149c2848c2..0e29f817e56 100644 > --- a/sql/sql_union.cc > +++ b/sql/sql_union.cc > @@ -507,14 +507,14 @@ void select_union_recursive::cleanup() > bool select_union_direct::change_result(select_result *new_result) > { > result= new_result; > - return (result->prepare(unit->types, unit) || result->prepare2()); > + return (result->prepare(unit->types, unit) || result->prepare2(0)); Do we really continue to use '0' when 'NULL' should be used? I prefer "NULL". > } > > > bool select_union_direct::postponed_prepare(List<Item> &types) > { > if (result != NULL) > - return (result->prepare(types, unit) || result->prepare2()); > + return (result->prepare(types, unit) || result->prepare2(0)); > else > return false; > } > diff --git a/sql/sql_update.cc b/sql/sql_update.cc > index 38638d3aa1d..4ae7e33b4e3 100644 > --- a/sql/sql_update.cc > +++ b/sql/sql_update.cc > @@ -2205,10 +2195,41 @@ multi_update::initialize_tables(JOIN *join) > DBUG_RETURN(1); > tmp_tables[cnt]->file->extra(HA_EXTRA_WRITE_CACHE); > } > + join->tmp_table_keep_current_rowid= TRUE; > DBUG_RETURN(0); > } > (Follwing explanations on Slack) Please add a comment there, saying that Here we walk through the aggegation temptable to find where rowids are stored. And then for each of the multi-update tables, we get them to copy the data from the rowid field in the aggregation temptable. > + > +int multi_update::prepare2(JOIN *join) > +{ > + if (!join->need_tmp || !join->tmp_table_keep_current_rowid) > + return 0; > + > + // there cannot be many tmp tables in multi-update > + JOIN_TAB *tmptab= join->join_tab + join->exec_join_tab_cnt(); > + > + for (Item **it= tmptab->tmp_table_param->items_to_copy; *it ; it++) > + { > + TABLE *tbl= (*it)->rowid_table(); > + if (!tbl) > + continue; > + for (uint i= 0; i < table_count; i++) > + { > + for (Item **it2= tmp_table_param[i].items_to_copy; *it2; it2++) > + { > + if ((*it2)->rowid_table() != tbl) > + continue; > + *it2= new (thd->mem_root) Item_field(thd, (*it)->get_tmp_table_field()); > + if (!*it2) > + return 1; > + } > + } > + } > + return 0; > +} > + > + > multi_update::~multi_update() > { > TABLE_LIST *table; BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

2 5

[Maria-developers] FOR UPDATE behavior change in 10.3-MDEV-11953
by Alexander Barkov 12 Apr '18

12 Apr '18

Hi Sanja, and Igor, (resending with Igor on CC) I noticed a FOR UPDATE behavior change in 10.3-MDEV-11953 In 10.3 I run this script in a client session: CREATE OR REPLACE TABLE t1 (a INT NOT NULL PRIMARY KEY); CREATE OR REPLACE TABLE t2 (a INT NOT NULL PRIMARY KEY); INSERT INTO t1 VALUES (1); INSERT INTO t2 VALUES (2); BEGIN; SELECT * FROM t1 UNION SELECT * FROM t2 FOR UPDATE; Notice no COMMIT yet! It returns this result: +---+ | a | +---+ | 1 | | 2 | +---+ Now I open a new console, start a new client session and run this script: BEGIN; SELECT * FROM t2 FOR UPDATE; The second session gets locked, as expected. Now I return to the first console and run "COMMIT;". The second console gets unlocked and returns +---+ | a | +---+ | 2 | +---+ Now if I do the same in 10.3-MDEV-11953, the second session does not get locked, it returns the result immediately, even before I typed "COMMIT;" in the first session. Note, if in 10.3-MDEV-11953 I start a transaction with a simpler query (without UNION) in the first session: BEGIN; SELECT * FROM t2 FOR UPDATE; and run the same script in the second session BEGIN; SELECT * FROM t2 FOR UPDATE; then the second session gets locked as expected. Is this behavior change expected? Thanks!

2 1

Re: [Maria-developers] FOR UPDATE behavior change in 10.3-MDEV-11953
by Alexander Barkov 12 Apr '18

12 Apr '18

Hi Igor, Please do "reply all" when replying. On 04/12/2018 01:28 AM, Igor Babaev wrote: > Alexander, > > If you have the same behaviour you have to use > > SELECT * FROM t1 UNION (SELECT * FROM t2 FOR UPDATE). Thanks. This worked fine for me. > > Otherwise it's considered as > > (SELECT * FROM t1 UNION SELECT * FROM t2 FOR UPDATE). What does this syntax mean 10.3-MDEV-11953? Which records should be locked? From t1, from t2, or from both? It seems that it locks nothing. If so, shouldn't this be disallowed? This query (without parentheses) also locks nothing: SELECT * FROM t1 UNION SELECT * FROM t2 FOR UPDATE; > > MySQL manual says that 'FOR UPDATE' must follow ORDER BY, > > but it does not tell you how to interpret FOR UPDATE in unions. The old behavior was to interpret UNION parts separately. This locked only t2 in 10.3: SELECT * FROM t1 UNION SELECT * FROM t2 FOR UPDATE; In 10.3-MDEV-11953 it locks nothing. Upgrade looks dangerous. Can we do this way: 1. In 10.4: disallow FOR UPDATE applied to the entire UNION. 2. In 10.3: Add a warning which tells that this feature is deprecated and suggests to use parentheses around union parts when a non-parenthesized union term is used with FOR UPDATE ? > > Anyway you can't use anymore: > SELECT c FROM t1 ORDER BY a > UNION > SELECT c FROM t2 ORDER BY b > ORDER BY c. > > You have to put it so: > (SELECT c FROM t1 ORDER BY a) > UNION > (SELECT c FROM t2 ORDER BY b) > ORDER BY c. > > (see the standard). I like this change. > > It means that the ORDER BY clause followed a union relates to the union. > The same is true for 'FOR UPDATE' because it follows optional ORDER BY/ > LIMIT clauses. > > > Regards, > Igor. > > > On 04/11/2018 09:10 AM, Alexander Barkov wrote: >> Hi Sanja, and Igor, >> >> >> (resending with Igor on CC) >> >> >> I noticed a FOR UPDATE behavior change in 10.3-MDEV-11953 >> >> In 10.3 I run this script in a client session: >> >> CREATE OR REPLACE TABLE t1 (a INT NOT NULL PRIMARY KEY); >> CREATE OR REPLACE TABLE t2 (a INT NOT NULL PRIMARY KEY); >> INSERT INTO t1 VALUES (1); >> INSERT INTO t2 VALUES (2); >> BEGIN; >> SELECT * FROM t1 UNION SELECT * FROM t2 FOR UPDATE; >> >> Notice no COMMIT yet! It returns this result: >> +---+ >> | a | >> +---+ >> | 1 | >> | 2 | >> +---+ >> >> Now I open a new console, start a new client session and run this script: >> >> BEGIN; >> SELECT * FROM t2 FOR UPDATE; >> >> The second session gets locked, as expected. >> >> Now I return to the first console and run "COMMIT;". >> The second console gets unlocked and returns >> +---+ >> | a | >> +---+ >> | 2 | >> +---+ >> >> >> Now if I do the same in 10.3-MDEV-11953, the second session does not get >> locked, it returns the result immediately, even before I typed "COMMIT;" >> in the first session. >> >> >> Note, if in 10.3-MDEV-11953 I start a transaction with a simpler query >> (without UNION) in the first session: >> >> BEGIN; >> SELECT * FROM t2 FOR UPDATE; >> >> and run the same script in the second session >> >> BEGIN; >> SELECT * FROM t2 FOR UPDATE; >> >> then the second session gets locked as expected. >> >> >> Is this behavior change expected? >> >> >> Thanks! >> >> _______________________________________________ >> Mailing list: https://launchpad.net/~maria-developers >> Post to : maria-developers(a)lists.launchpad.net >> Unsubscribe : https://launchpad.net/~maria-developers >> More help : https://help.launchpad.net/ListHelp >>

2 3

[Maria-developers] Issue with setting up of MariaDB and solve issue MDEV-14896
by Aman Jain 11 Apr '18

11 Apr '18

Hello developers I am setting up the codebase for MariaDB on my system to understand the codebase and solve some issues. But I am facing a problem which I am not able to solve when I run "make" command. Error log: https://pastebin.com/u5fAhMHX OS: MacOS 10.13.3 Python version: 2.7.14 Please let me know how can I set up so that I can run and test the changes. Also, I want to solve issue MDEV-14896. Please connect me to the owner of the issue so that I can discuss the changes. ( https://jira.mariadb.org/browse/MDEV-14896 ) Thanks Aman Jain

3 5

[Maria-developers] 10.3-MDEV-11953
by Alexander Barkov 11 Apr '18

11 Apr '18

Hi Sanja and Igor, In 10.3-MDEV-11953 you're adding a number of main_select_push() / pop_select() around "expr" and other rules that contain "expr". How do you decide when push/pop is needed. Why, for example, LIMIT and ORDER clause don't have their own push/pop. They do create some expressions (Items). What is the rule? Thanks.

1 0

[Maria-developers] FOR UPDATE behavior change in 10.3-MDEV-11953
by Alexander Barkov 11 Apr '18

11 Apr '18

Hi Sanja, I noticed a FOR UPDATE behavior change in 10.3-MDEV-11953 In 10.3 I run this script in a client session: CREATE OR REPLACE TABLE t1 (a INT NOT NULL PRIMARY KEY); CREATE OR REPLACE TABLE t2 (a INT NOT NULL PRIMARY KEY); INSERT INTO t1 VALUES (1); INSERT INTO t2 VALUES (2); BEGIN; SELECT * FROM t1 UNION SELECT * FROM t2 FOR UPDATE; Notice no COMMIT yet! It returns this result: +---+ | a | +---+ | 1 | | 2 | +---+ Now I open a new console, start a new client session and run this script: BEGIN; SELECT * FROM t2 FOR UPDATE; The second session gets locked, as expected. Now I return to the first console and run "COMMIT;". The second console gets unlocked and returns +---+ | a | +---+ | 2 | +---+ Now if I do the same in 10.3-MDEV-11953, the second session does not get locked, it returns the result immediately, even before I typed "COMMIT;" in the first session. Note, if in 10.3-MDEV-11953 I start a transaction with a simpler query (without UNION) in the first session: BEGIN; SELECT * FROM t2 FOR UPDATE; and run the same script in the second session BEGIN; SELECT * FROM t2 FOR UPDATE; then the second session gets locked as expected. Is this behavior change expected? Thanks!

1 0

Re: [Maria-developers] 57673bc: MDEV-14024 PCRE2
by Sergei Golubchik 11 Apr '18

11 Apr '18

Hi, Alexey! On Apr 03, Alexey Botchkov wrote: > revision-id: 57673bcfeecf51d1dd8f4907adc8ca11d84796d3 (mariadb-10.3.5-76-g57673bc) > parent(s): d614d4b2e61552a5b74259f91b3e14b23707743f > committer: Alexey Botchkov > timestamp: 2018-04-03 02:59:51 +0400 > message: > > MDEV-14024 PCRE2. > > server ocde switched to the PCRE2 features. ^^^ s/ocde/code/ > diff --git a/cmake/pcre.cmake b/cmake/pcre.cmake > index 4c11392..271dd06 100644 > --- a/cmake/pcre.cmake > +++ b/cmake/pcre.cmake > @@ -5,24 +5,16 @@ SET(WITH_PCRE "auto" CACHE STRING > > + IF(NOT HAVE_PCRE2 OR WITH_PCRE STREQUAL "bundled") > IF (WITH_PCRE STREQUAL "system") > - MESSAGE(FATAL_ERROR "system pcre is not found or unusable") > + MESSAGE(FATAL_ERROR "system pcre2-8 is not found or unusable") better "pcre2-8 library" or simply "pcre2" > diff --git a/sql/item_cmpfunc.cc b/sql/item_cmpfunc.cc > index 89aa307..08aa293 100644 > --- a/sql/item_cmpfunc.cc > +++ b/sql/item_cmpfunc.cc > @@ -5481,15 +5481,6 @@ int Regexp_processor_pcre::default_regex_flags() > return default_regex_flags_pcre(current_thd); > } > > -void Regexp_processor_pcre::set_recursion_limit(THD *thd) > -{ > - long stack_used; > - DBUG_ASSERT(thd == current_thd); > - stack_used= available_stack_size(thd->thread_stack, &stack_used); > - m_pcre_extra.match_limit_recursion= > - (ulong)((my_thread_stack_size - STACK_MIN_SIZE - stack_used)/my_pcre_frame_size); > -} > - > > /** > Convert string to lib_charset, if needed. > @@ -5537,19 +5528,32 @@ bool Regexp_processor_pcre::compile(String *pattern, bool send_error) > if (!(pattern= convert_if_needed(pattern, &pattern_converter))) > return true; > > - m_pcre= pcre_compile(pattern->c_ptr_safe(), m_library_flags, > - &pcreErrorStr, &pcreErrorOffset, NULL); > + m_pcre= pcre2_compile((PCRE2_SPTR8) pattern->ptr(), pattern->length(), > + m_library_flags, > + &pcreErrorNumber, &pcreErrorOffset, NULL); > > if (m_pcre == NULL) > { > if (send_error) > { > char buff[MAX_FIELD_WIDTH]; > - my_snprintf(buff, sizeof(buff), "%s at offset %d", pcreErrorStr, pcreErrorOffset); > + int lmsg= pcre2_get_error_message(pcreErrorNumber, > + (PCRE2_UCHAR8 *)buff, sizeof(buff)); > + if (lmsg >= 0) > + my_snprintf(buff+lmsg, sizeof(buff)-lmsg, > + " at offset %d", pcreErrorOffset); if lmsg < 0, does it guarantee that the buffer is 0-terminated? > my_error(ER_REGEXP_ERROR, MYF(0), buff); > } > return true; > } > + > + m_pcre_match_data= pcre2_match_data_create_from_pattern(m_pcre, NULL); > + if (m_pcre_match_data == NULL) > + { > + my_error(ER_OUT_OF_RESOURCES, MYF(0)); > + return true; > + } > + > return false; > } > > @@ -5570,124 +5574,43 @@ bool Regexp_processor_pcre::compile(Item *item, bool send_error) > */ > void Regexp_processor_pcre::pcre_exec_warn(int rc) const > { ... > - push_warning_printf(thd, Sql_condition::WARN_LEVEL_WARN, > - ER_REGEXP_ERROR, ER_THD(thd, ER_REGEXP_ERROR), errmsg); > + int errlen= pcre2_get_error_message(rc, buf, sizeof(buf)); > + if (errlen >= 0) > + push_warning_printf(thd, Sql_condition::WARN_LEVEL_WARN, > + ER_REGEXP_ERROR, ER_THD(thd, ER_REGEXP_ERROR), buf); perhaps, issue some generic message if pcre2_get_error_message fails? > } > @@ -5697,14 +5620,12 @@ bool Regexp_processor_pcre::exec(String *str, int offset, > { > if (!(str= convert_if_needed(str, &subject_converter))) > return true; > - m_pcre_exec_rc= pcre_exec_with_warn(m_pcre, &m_pcre_extra, > - str->c_ptr_safe(), str->length(), > - offset, 0, > - m_SubStrVec, array_elements(m_SubStrVec)); > + m_pcre_exec_rc= pcre_exec_with_warn(m_pcre, m_pcre_match_data, > + str->ptr(), str->length(), offset, 0); > if (m_pcre_exec_rc > 0) > { > uint i; > - for (i= 0; i < n_result_offsets_to_convert; i++) > + for (i= 0; i < n_result_offsets_to_convert && i < m_pcre_exec_rc-1; i++) hmm, what does that mean? that both n_result_offsets_to_convert < m_pcre_exec_rc-1 and m_pcre_exec_rc-1 < n_result_offsets_to_convert are possible? > { > /* > Convert byte offset into character offset. > diff --git a/sql/item_cmpfunc.h b/sql/item_cmpfunc.h > index 3d11a22..d3f52ed 100644 > --- a/sql/item_cmpfunc.h > +++ b/sql/item_cmpfunc.h > @@ -25,7 +25,8 @@ > > #include "item_func.h" /* Item_int_func, Item_bool_func */ > #define PCRE_STATIC 1 /* Important on Windows */ > -#include "pcre.h" /* pcre header file */ > +#define PCRE2_CODE_UNIT_WIDTH 8 you sure, you want it here and not, say, in one universally-included headers like my_config.h? You don't define it in other places now, so they include different function names and macros. > +#include "pcre2.h" /* pcre header file */ > #include "item.h" > > extern Item_result item_cmp_type(Item_result a,Item_result b); > diff --git a/sql/mysqld.cc b/sql/mysqld.cc > index fe2b9c8..dcf0996 100644 > --- a/sql/mysqld.cc > +++ b/sql/mysqld.cc > @@ -111,7 +111,7 @@ > #include "sp_rcontext.h" > #include "sp_cache.h" > #include "sql_reload.h" // reload_acl_and_cache > -#include "pcre.h" > +#include "pcre2.h" do you still need to include pcre header here? > > #ifdef HAVE_POLL_H > #include <poll.h> > diff --git a/storage/mroonga/CMakeLists.txt b/storage/mroonga/CMakeLists.txt > index 5d8e8c1..57d12a1 100644 > --- a/storage/mroonga/CMakeLists.txt > +++ b/storage/mroonga/CMakeLists.txt > @@ -189,8 +189,8 @@ else() > set(MYSQL_VARIANT "MySQL") > endif() > > -if(EXISTS "${MYSQL_SOURCE_DIR}/pcre") > - set(MYSQL_REGEX_INCLUDE_DIR "${MYSQL_SOURCE_DIR}/pcre") > +if(EXISTS "${MYSQL_SOURCE_DIR}/pcre2") > + set(MYSQL_REGEX_INCLUDE_DIR "${MYSQL_SOURCE_DIR}/pcre2/src") eh, is that enough? there were a lot more changes in the server, than just correcting the path. > else() > set(MYSQL_REGEX_INCLUDE_DIR "${MYSQL_SOURCE_DIR}/regex") > endif() Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] FYI: 10.3 master branch head builds OK Launchpad all architectures
by Otto Kekäläinen 11 Apr '18

11 Apr '18

Hello! Just for the record, I've done some small tweaks to 10.3 so that it now builds perfectly and passes build tests on all Launchpad architectures that I have access to. https://launchpad.net/~mysql-ubuntu/+archive/ubuntu/mariadb-10.3/+build s?build_text=&build_state=all Let's hope it stays like this for the rest of the current 10.3 cycle and will not regress by before it is announced GA.. On buildbot the Debian sid and Ubuntu Bionic builders pass now. There are however some sporadic failures left. http://buildbot.askmonty.org/buildbot/grid?branch=10.3&category=main&ca tegory=experimental On Travis everything else passes apart from one test, which has been consistently failing for some time. Everything was green in January when I wrote the protected branches blog post, so this one is a consistently failing regression. https://travis-ci.org/MariaDB/server/builds/362466962 - Otto

2 5

[Maria-developers] Temporary table bug
by Badrul Chowdhury 09 Apr '18

09 Apr '18

Hi, Is anyone actively working on https://jira.mariadb.org/browse/MDEV-12459? If not, I would like to take a look. -- Thanks, Badrul

2 2

[Maria-developers] Test failure overview for Debian/Ubuntu on 10.3 master branch
by Otto Kekäläinen 05 Apr '18

05 Apr '18

(posting this here so to ensure it is recorded in mailing list archive) On Travis the consistently failing tests are: main.type_datetime_hires main.select_pkeycache main.select main.select_jcl6 main.type_float main.func_str main.type_time_hires main.type_timestamp_hires Log: https://api.travis-ci.org/v3/job/362466968/log.txt On buildbot the consistelty failing tests for Debian/Ubuntu are: galera.galera_sst_mysqldump Log: http://buildbot.askmonty.org/buildbot/builders/kvm-deb-jessie-x86/build s/4033/steps/mtr/logs/stdio and http://buildbot.askmonty.org/buildbot/builders/kvm-deb-trusty-ppc64le/b uilds/4546/steps/mtr/logs/stdio Single ones: encryption.innodb_encryption Log: http://buildbot.askmonty.org/buildbot/builders/kvm-deb-sid-x86/builds/6 560/steps/mtr/logs/stdio rocksdb.drop_index_inplace Log: http://buildbot.askmonty.org/buildbot/builders/kvm-deb-trusty-ppc64le/b uilds/4546/steps/mtr/logs/stdio - Otto

1 0

Re: [Maria-developers] Regarding bugs (mysqldump)
by Vicențiu Ciorbaru 01 Apr '18

01 Apr '18

Hi Rutuja! Great analysis again and lets share this with the maria-developers mailing list as well. I've tested your proposed change and the change for dump_all_tables_in_db seems to not be necessary. If you look at how dump_all_tables_in_db gets called, it's followed by dump_all_views_in_db. The actual view creation happens in dump_all_views_in_db, while dump_all_tables_in_db will create some dummy tables for views. We create dummy tables first, so as to avoid having to check dependencies within views. There's more details in the comments of why we do that in the code of get_table_structure: /* Create a table with the same name as the view and with columns of the same name in order to satisfy views that depend on this view. The table will be removed when the actual view is created. The properties of each column, are not preserved in this temporary table, because they are not necessary. This will not be necessary once we can determine dependencies between views and can simply dump them in the appropriate order. */ For dump_selected_tables, your proposed change should be sufficient and according to my tests it works. Please submit a pull request for this. Looking forward to having this fixed soon. :) Unrelated to your analysis: There are plenty of edge cases where views won't work: For example: If we have 2 databases and one view uses something from a different database, if you're not careful about the order of things, it's going to fail. To create a "truly" always working dump one would have to create a full dependency graph and then do a topological sort for all objects in the database that depend on the objects we're dumping. This is outside of the scope of this bug, however it could prove to be a "better way to dump stuff". Regards, Vicentiu On Sun, 1 Apr 2018 at 06:28 Rutuja Surve <rutuja.r.surve(a)gmail.com> wrote: > Hello, > Could you review my following approach regarding the mysqldump bug ( > https://jira.mariadb.org/browse/MDEV-15021) > > I went through the code and observed the following: > 1. dump_all_tables_in_db calls dump_table first (for the list of all > tables), then dump_table calls get_view_structure. > 2. get_view_structure dumps the view for the corresponding table to > mysqldump > 3. After this call, dump_all_tables_in_db calls dump_routines_for_db > > dump_selected_tables also behaves in the exact same way as dump_all_tables_in_db, > following steps 1,2,3, except that it does it for a list of given tables > instead of doing it for all tables. > > We want the routines to get dumped before the views for the dump to become > importable. > There are two solutions for this: > 1. Make dump_all_tables_in_db call dump_routines_for_db before it calls > dump_table. This will dump routines first and views later. > This can be done by shifting the call to dump_routines_for_db before the > loop in which dump_table is called. I.E: > Shift : > if (opt_routines && mysql_get_server_version(mysql) >= 50009) > { > DBUG_PRINT("info", ("Dumping routines for database %s", database)); > *dump_routines_for_db*(database); > } > Before this loop: > while ((table= getTableName(0))) > { > char *end= strmov(afterdot, table); > if (include_table((uchar*) hash_key, end - hash_key)) > { > *dump_table*(table,database); > my_free(order_by); > .... > } > } > > We should do this for both dump_all_tables_in_db and dump_selected_tables. > > 2. Approach 2: > As suggested in the jira thread, we can use negative logic to call > dump_table on tables we intend to skip. > This appears to be a workaround. In the place where dump_all_tables_in_db > is called, we'd rather call dump_selected_tables > with a list of tables that don't use the routines we'll be listing after > that. This would fix the issue but is not a good approach. > > If dump_table doesn't alter the values for opt_routines, the first > approach should fix the issue. > > Thanks, > Rutuja > > > > On Fri, Mar 23, 2018 at 10:36 PM, Rutuja Surve <rutuja.r.surve(a)gmail.com> > wrote: > >> Hello, >> With respect to https://jira.mariadb.org/browse/MDEV-15021 : >> From what I understood from the thread, since the routines are placed >> below the tables/views, they cannot be used in views while importing. >> I went through the problematic_dump.sql file that's attached there. What >> does the '@' in the following mean: >> SET @saved_cs_client = @@character_set_client; >> (I'm not familiar with this syntax) >> Also, what does the following piece of code do: (what does create definer >> mean and does this function simply return 1 as there are no other >> statements after begin? This is a routine defined below test_view) >> CREATE DEFINER=`root`@`localhost` FUNCTION `test_function`() RETURNS >> int(11) >> BEGIN >> >> RETURN 1; >> END ;; >> DELIMITER ; >> From my understanding, test_view uses test_function and test_view2 would >> use test_function2. >> In the mysqldump.c file, I went through the dump_routines_for_db >> function. We'd require to modify that in order to fix the bug. >> >> From the beginner bugs list, I found these 2 bugs that are also related >> to mysql dump that I could take up: >> 1. https://jira.mariadb.org/browse/MDEV-9883 >> (Flag for letting mysqldump support the 'auto' option for character set). >> Where in the code base can I find the file where the ' --default-character-set' >> flag is defined?) >> 2. https://jira.mariadb.org/browse/MDEV-8765 >> The solution for this has already been proposed in the thread. For >> mysqldump to not corrupt 4 byte UTF8 data, "change >> include/my_global.h:#define MYSQL_UNIVERSAL_CLIENT_CHARSET to utf8mb4" >> >> Thanks, >> Rutuja >> > >

1 0

Re: [Maria-developers] d975226a0d6: MDEV-15241: make SIGNAL maximum MESSAGE_TEXT length a larger value
by Sergei Golubchik 01 Apr '18

01 Apr '18

Hi, Varun! On Mar 22, Varun wrote: > revision-id: d975226a0d668775062151c9f30676db3d81c93b (mariadb-10.3.0-642-gd975226a0d6) > parent(s): 85ddd9e8ce11bef089ccc6c86745142d09639c14 > author: Varun Gupta > committer: Varun Gupta > timestamp: 2018-03-22 12:45:55 +0530 > message: > > MDEV-15241: make SIGNAL maximum MESSAGE_TEXT length a larger value > > Increasing the length of MESSAGE_TEXT from 128 to MYSQL_ERRMSG_SIZE > which is the max length of the error message ok to push. Don't forget to document your change, please. > diff --git a/sql/sql_signal.cc b/sql/sql_signal.cc > index 1da8b145151..13dc3db1f10 100644 > --- a/sql/sql_signal.cc > +++ b/sql/sql_signal.cc > @@ -262,12 +262,12 @@ int Sql_cmd_common_signal::eval_signal_informations(THD *thd, Sql_condition *con > } > /* > Enforce that SET MESSAGE_TEXT = <value> evaluates the value > - as VARCHAR(128) CHARACTER SET UTF8. > + as VARCHAR(MYSQL_ERRMSG_SIZE) CHARACTER SET UTF8. > */ > bool truncated; > String utf8_text; > str= set->val_str(& str_value); > - truncated= assign_fixed_string(thd->mem_root, & my_charset_utf8_bin, 128, > + truncated= assign_fixed_string(thd->mem_root, & my_charset_utf8_bin, MYSQL_ERRMSG_SIZE, > & utf8_text, str); > if (truncated) > { Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] MDEV-14929 - AddressSanitizer: memcpy-param-overlap in Field_longstr::compress
by Sergei Golubchik 31 Mar '18

31 Mar '18

Hi, Sergey, > From 6de643a580b4e706e0201927e43a033f5cb98970 Mon Sep 17 00:00:00 2001 > From: Sergey Vojtovich <svoj(a)mariadb.org> > Date: Wed, 21 Mar 2018 13:54:15 +0400 > Subject: [PATCH] MDEV-14929 - AddressSanitizer: memcpy-param-overlap in > Field_longstr::compress > > Handle overlaping "from" and Field_blob_compressed::value for compressed > blobs similarily to regular blobs. > --- > mysql-test/r/column_compression.result | 13 +++++++++++++ > mysql-test/t/column_compression.test | 12 ++++++++++++ > sql/field.cc | 16 +++++++++++----- > 3 files changed, 36 insertions(+), 5 deletions(-) > > diff --git a/sql/field.cc b/sql/field.cc > index 9da65526e145..1b044ec740e2 100644 > --- a/sql/field.cc > +++ b/sql/field.cc > @@ -8718,17 +8718,23 @@ int Field_blob_compressed::store(const char *from, size_t length, > { > ASSERT_COLUMN_MARKED_FOR_WRITE_OR_COMPUTED; > uint to_length= (uint)MY_MIN(max_data_length(), field_charset->mbmaxlen * length + 1); > + String tmp(from, length, cs); > int rc; > > + if (from >= value.ptr() && from <= value.ptr() + value.length() && > + tmp.copy(from, length, cs)) minor style comment: this could be if (from >= value.ptr() && from <= value.end() && tmp.copy()) Bigger question: Field_blob::store() also checks whether charset conversion is necessary. Why is it not applicable here? And, it'd be good to have a test case where from > value.ptr() (with a substring() function). > + goto oom; > + > if (value.alloc(to_length)) > - { > - set_ptr((uint32) 0, NULL); > - return -1; > - } > + goto oom; > > - rc= compress((char*) value.ptr(), &to_length, from, (uint)length, cs); > + rc= compress((char*) value.ptr(), &to_length, tmp.ptr(), (uint) length, cs); > set_ptr(to_length, (uchar*) value.ptr()); > return rc; > + > +oom: > + set_ptr((uint32) 0, NULL); > + return -1; > } Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

2 1

[Maria-developers] Patch for pad functions
by jerome brauge 30 Mar '18

30 Mar '18

Hello Alexander, Can you open an MDEV for rpad/lpad (return NULL instead of empty string when sql_mode=oracle) and review this patch ? Regards.

2 1

Re: [Maria-developers] [Commits] c9ae1234814: MDEV-14592: Custom Aggregates Usage Status Variable
by Vicențiu Ciorbaru 26 Mar '18

26 Mar '18

Hi Varun This generally looks good, 2 comments inline. Vicentiu diff --git a/mysql-test/t/custom_aggregate_functions.test b/mysql-test/t/custom_aggregate_functions.test index 20fcc35f39f..a93ec545f01 100644 --- a/mysql-test/t/custom_aggregate_functions.test +++ b/mysql-test/t/custom_aggregate_functions.test @@ -16,7 +16,9 @@ create table t1 (sal int(10),id int(10)); INSERT INTO t1 (sal,id) VALUES (5000,1); INSERT INTO t1 (sal,id) VALUES (2000,1); INSERT INTO t1 (sal,id) VALUES (1000,1); # Please test that this works with subqueries, views, ctes and when calling from # other stored procedures. +show global status like "Feature_custom_aggregate_functions"; select f1(sal) from t1 where id>= 1; +show global status like "Feature_custom_aggregate_functions"; select * from t2; drop table t2; drop function f1; diff --git a/sql/item_sum.cc b/sql/item_sum.cc index d4abdfc614f..7c0390e543f 100644 --- a/sql/item_sum.cc +++ b/sql/item_sum.cc @@ -313,6 +313,8 @@ bool Item_sum::check_sum_func(THD *thd, Item **ref) } } aggr_sel->set_agg_func_used(true); + if (sum_func() == SP_AGGREGATE_FUNC) + aggr_sel->set_custom_agg_func_used(true); update_used_tables(); thd->lex->in_sum_func= in_sum_func; return FALSE; diff --git a/sql/mysqld.cc b/sql/mysqld.cc index c7fbf0a594c..75ccda408b1 100644 --- a/sql/mysqld.cc +++ b/sql/mysqld.cc @@ -8491,6 +8491,7 @@ SHOW_VAR status_vars[]= { {"Executed_triggers", (char*) offsetof(STATUS_VAR, executed_triggers), SHOW_LONG_STATUS}, {"Feature_check_constraint", (char*) &feature_check_constraint, SHOW_LONG }, {"Feature_delay_key_write", (char*) &feature_files_opened_with_delayed_keys, SHOW_LONG }, Please move this up so the variable names are ordered alphabetically. + {"Feature_custom_aggregate_functions", (char*) offsetof(STATUS_VAR, feature_custom_aggregate_functions), SHOW_LONG_STATUS}, {"Feature_dynamic_columns", (char*) offsetof(STATUS_VAR, feature_dynamic_columns), SHOW_LONG_STATUS}, {"Feature_fulltext", (char*) offsetof(STATUS_VAR, feature_fulltext), SHOW_LONG_STATUS}, {"Feature_gis", (char*) offsetof(STATUS_VAR, feature_gis), SHOW_LONG_STATUS}, On Sat, 9 Dec 2017 at 08:53 Varun <varunraiko1803(a)gmail.com> wrote: > revision-id: c9ae12348143f59e875a52e2eb285c5ed7d7af56 > (mariadb-10.3.0-285-gc9ae1234814) > parent(s): c60095a818dce92838940525899a13a05633d148 > author: Varun Gupta > committer: Varun Gupta > timestamp: 2017-12-09 12:22:26 +0530 > message: > > MDEV-14592: Custom Aggregates Usage Status Variable > > --- > mysql-test/r/custom_aggregate_functions.result | 6 ++++++ > mysql-test/t/custom_aggregate_functions.test | 2 ++ > sql/item_sum.cc | 2 ++ > sql/mysqld.cc | 1 + > sql/sql_lex.cc | 2 ++ > sql/sql_lex.h | 3 +++ > sql/sql_select.cc | 2 ++ > 7 files changed, 18 insertions(+) > > diff --git a/mysql-test/r/custom_aggregate_functions.result > b/mysql-test/r/custom_aggregate_functions.result > index ca8612ba652..f70675fa4db 100644 > --- a/mysql-test/r/custom_aggregate_functions.result > +++ b/mysql-test/r/custom_aggregate_functions.result > @@ -11,6 +11,9 @@ create table t1 (sal int(10),id int(10)); > INSERT INTO t1 (sal,id) VALUES (5000,1); > INSERT INTO t1 (sal,id) VALUES (2000,1); > INSERT INTO t1 (sal,id) VALUES (1000,1); > +show global status like "Feature_custom_aggregate_functions"; > +Variable_name Value > +Feature_custom_aggregate_functions 0 > select f1(sal) from t1 where id>= 1; > f1(sal) > 0 > @@ -18,6 +21,9 @@ Warnings: > Note 4093 At line 5 in test.f1 > Note 4093 At line 5 in test.f1 > Note 4093 At line 5 in test.f1 > +show global status like "Feature_custom_aggregate_functions"; > +Variable_name Value > +Feature_custom_aggregate_functions 1 > select * from t2; > sal > 5000 > diff --git a/mysql-test/t/custom_aggregate_functions.test > b/mysql-test/t/custom_aggregate_functions.test > index 20fcc35f39f..a93ec545f01 100644 > --- a/mysql-test/t/custom_aggregate_functions.test > +++ b/mysql-test/t/custom_aggregate_functions.test > @@ -16,7 +16,9 @@ create table t1 (sal int(10),id int(10)); > INSERT INTO t1 (sal,id) VALUES (5000,1); > INSERT INTO t1 (sal,id) VALUES (2000,1); > INSERT INTO t1 (sal,id) VALUES (1000,1); > +show global status like "Feature_custom_aggregate_functions"; > select f1(sal) from t1 where id>= 1; > +show global status like "Feature_custom_aggregate_functions"; > select * from t2; > drop table t2; > drop function f1; > diff --git a/sql/item_sum.cc b/sql/item_sum.cc > index d4abdfc614f..7c0390e543f 100644 > --- a/sql/item_sum.cc > +++ b/sql/item_sum.cc > @@ -313,6 +313,8 @@ bool Item_sum::check_sum_func(THD *thd, Item **ref) > } > } > aggr_sel->set_agg_func_used(true); > + if (sum_func() == SP_AGGREGATE_FUNC) > + aggr_sel->set_custom_agg_func_used(true); > update_used_tables(); > thd->lex->in_sum_func= in_sum_func; > return FALSE; > diff --git a/sql/mysqld.cc b/sql/mysqld.cc > index c7fbf0a594c..75ccda408b1 100644 > --- a/sql/mysqld.cc > +++ b/sql/mysqld.cc > @@ -8491,6 +8491,7 @@ SHOW_VAR status_vars[]= { > {"Executed_triggers", (char*) offsetof(STATUS_VAR, > executed_triggers), SHOW_LONG_STATUS}, > {"Feature_check_constraint", (char*) &feature_check_constraint, > SHOW_LONG }, > {"Feature_delay_key_write", (char*) > &feature_files_opened_with_delayed_keys, SHOW_LONG }, > + {"Feature_custom_aggregate_functions", (char*) offsetof(STATUS_VAR, > feature_custom_aggregate_functions), SHOW_LONG_STATUS}, > {"Feature_dynamic_columns", (char*) offsetof(STATUS_VAR, > feature_dynamic_columns), SHOW_LONG_STATUS}, > {"Feature_fulltext", (char*) offsetof(STATUS_VAR, > feature_fulltext), SHOW_LONG_STATUS}, > {"Feature_gis", (char*) offsetof(STATUS_VAR, feature_gis), > SHOW_LONG_STATUS}, > diff --git a/sql/sql_lex.cc b/sql/sql_lex.cc > index 6cb065c3985..277469cc0aa 100644 > --- a/sql/sql_lex.cc > +++ b/sql/sql_lex.cc > @@ -2241,6 +2241,7 @@ void st_select_lex::init_query() > select_list_tables= 0; > m_non_agg_field_used= false; > m_agg_func_used= false; > + m_custom_agg_func_used= false; > window_specs.empty(); > window_funcs.empty(); > tvc= 0; > @@ -2278,6 +2279,7 @@ void st_select_lex::init_select() > merged_into= 0; > m_non_agg_field_used= false; > m_agg_func_used= false; > + m_custom_agg_func_used= false; > name_visibility_map= 0; > with_dep= 0; > join= 0; > diff --git a/sql/sql_lex.h b/sql/sql_lex.h > index 069b75628e3..9b40529d3e2 100644 > --- a/sql/sql_lex.h > +++ b/sql/sql_lex.h > @@ -1213,9 +1213,11 @@ class st_select_lex: public st_select_lex_node > */ > bool non_agg_field_used() const { return m_non_agg_field_used; } > bool agg_func_used() const { return m_agg_func_used; } > + bool custom_agg_func_used() const { return m_custom_agg_func_used; } > > void set_non_agg_field_used(bool val) { m_non_agg_field_used= val; } > void set_agg_func_used(bool val) { m_agg_func_used= val; } > + void set_custom_agg_func_used(bool val) { m_custom_agg_func_used= val; } > inline void set_with_clause(With_clause *with_clause); > With_clause *get_with_clause() > { > @@ -1259,6 +1261,7 @@ class st_select_lex: public st_select_lex_node > private: > bool m_non_agg_field_used; > bool m_agg_func_used; > + bool m_custom_agg_func_used; > > /* current index hint kind. used in filling up index_hints */ > enum index_hint_type current_index_hint_type; > diff --git a/sql/sql_select.cc b/sql/sql_select.cc > index 96aa227b439..c0d7c720910 100644 > --- a/sql/sql_select.cc > +++ b/sql/sql_select.cc > @@ -2909,6 +2909,8 @@ bool JOIN::make_aggr_tables_info() > /* Count that we're using window functions. */ > status_var_increment(thd->status_var.feature_window_functions); > } > + if (select_lex->custom_agg_func_used()) > + > status_var_increment(thd->status_var.feature_custom_aggregate_functions); > > fields= curr_fields_list; > // Reset before execution > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits

1 0

[Maria-developers] MDEV 10754 - Patch for trim functions
by jerome brauge 26 Mar '18

26 Mar '18

Hello Alexander, Can you review this patch for trim functions for Oracle ? Regards, Jérôme.

2 2

Re: [Maria-developers] [Commits] 2950feb5c09: MDEV-9959: A serious MariaDB server performance bug
by Sergey Petrunia 26 Mar '18

26 Mar '18

Hi Varun, Please find my feedback below. As disucssed before: please also run the MDEV's test case with this patch and post a comment with the result. On Fri, Mar 23, 2018 at 05:17:57PM +0530, Varun wrote: > revision-id: 2950feb5c09033843a6a32c7431421b45d5812dd (mariadb-10.3.0-646-g2950feb5c09) > parent(s): 9ef7ab221d70630155206910de8a6053db097164 > author: Varun Gupta > committer: Varun Gupta > timestamp: 2018-03-23 17:17:09 +0530 > message: > > MDEV-9959: A serious MariaDB server performance bug > > step#1: if a derived table has SELECT DISTINCT, provide index statistics for it so that the join optimizer in the > upper select knows that ref access to the table will produce one row. > > Added handling for multiple selects in the derived table > > --- > mysql-test/r/mdev9959.result | 64 ++++++++++++++++++++++++++++++++++++++++++++ > mysql-test/t/mdev9959.test | 28 +++++++++++++++++++ > sql/sql_lex.h | 2 ++ > sql/sql_union.cc | 36 +++++++++++++++++++++++++ > sql/table.cc | 26 +++++++++++++++--- > 5 files changed, 153 insertions(+), 3 deletions(-) > > diff --git a/mysql-test/r/mdev9959.result b/mysql-test/r/mdev9959.result > new file mode 100644 > index 00000000000..5269ba9b734 > --- /dev/null > +++ b/mysql-test/r/mdev9959.result > @@ -0,0 +1,64 @@ > +create table t1(a int); > +insert into t1 values (1),(2),(3),(4),(5),(6); > +create table t2(a int, b int,c int); > +insert into t2(a,b,c) values (1,1,2),(2,2,3),(3,1,4),(4,2,2),(5,1,1),(6,2,5); > +create table t3(a int, b int); > +insert into t3(a,b) values (1,1),(2,2),(2,1),(1,2),(5,1),(9,2); > +analyze select * from t1 , ( (select distinct t2.a from t2 group by c))q where t1.a=q.a; > +id select_type table type possible_keys key key_len ref rows r_rows filtered r_filtered Extra > +1 PRIMARY t1 ALL NULL NULL NULL NULL 6 6.00 100.00 100.00 Using where > +1 PRIMARY <derived2> ref key0 key0 5 test.t1.a 1 0.83 100.00 100.00 > +2 DERIVED t2 ALL NULL NULL NULL NULL 6 6.00 100.00 100.00 Using temporary; Using filesort It is not at all clear which parts of the above output matter and which don't, for example, does #rows matter for table t1? Please add a comment (into .test file with --echo # ) describing that table "<derived2>" should have type=ref and rows=1 > +select * from t1 , ( (select distinct t2.a from t2 group by c))q where t1.a=q.a; > +a a > +1 1 > +2 2 > +3 3 > +5 5 > +6 6 > +analyze select * from t1 , ( (select t2.a from t2 group by c))q where t1.a=q.a; > +id select_type table type possible_keys key key_len ref rows r_rows filtered r_filtered Extra > +1 PRIMARY t1 ALL NULL NULL NULL NULL 6 6.00 100.00 100.00 Using where > +1 PRIMARY <derived2> ref key0 key0 5 test.t1.a 2 0.83 100.00 100.00 > +2 DERIVED t2 ALL NULL NULL NULL NULL 6 6.00 100.00 100.00 Using temporary; Using filesort > +select * from t1 , ( (select t2.a from t2 group by c))q where t1.a=q.a; > +a a > +1 1 > +2 2 > +3 3 > +5 5 > +6 6 > +analyze select * from t1 , ( (select distinct t2.a from t2 group by c) union all (select distinct t2.a from t2 group by c))q where t1.a=q.a; > +id select_type table type possible_keys key key_len ref rows r_rows filtered r_filtered Extra > +1 PRIMARY t1 ALL NULL NULL NULL NULL 6 6.00 100.00 100.00 Using where > +1 PRIMARY <derived2> ref key0 key0 5 test.t1.a 2 1.67 100.00 100.00 > +2 DERIVED t2 ALL NULL NULL NULL NULL 6 6.00 100.00 100.00 Using temporary; Using filesort > +3 UNION t2 ALL NULL NULL NULL NULL 6 6.00 100.00 100.00 Using temporary; Using filesort What does the above test show? We've got type=ref, #rows = 2 for the derived table. The estimate is correct but where does it come from? As far as I understand this patch, it only sets the estimate rows=1 for derived tables. How do we arrive at the value of 2? (If we would have gotten rows=2 without this patch, what's the point of having the testcase as part of this patch?) > +select * from t1 , ( (select distinct t2.a from t2 group by c) union all (select distinct t2.a from t2 group by c))q where t1.a=q.a; > +a a > +1 1 > +1 1 > +2 2 > +2 2 > +3 3 > +3 3 > +5 5 > +5 5 > +6 6 > +6 6 > +analyze select * from t1 , ( (select distinct t2.a from t2 group by c) union (select distinct t2.a from t2 group by c) union (select t3.a from t3 group by b))q where t1.a=q.a; > +id select_type table type possible_keys key key_len ref rows r_rows filtered r_filtered Extra > +1 PRIMARY t1 ALL NULL NULL NULL NULL 6 6.00 100.00 100.00 Using where > +1 PRIMARY <derived2> ref key0 key0 5 test.t1.a 2 0.83 100.00 100.00 > +2 DERIVED t2 ALL NULL NULL NULL NULL 6 6.00 100.00 100.00 Using temporary; Using filesort > +3 UNION t2 ALL NULL NULL NULL NULL 6 6.00 100.00 100.00 Using temporary; Using filesort > +4 UNION t3 ALL NULL NULL NULL NULL 6 6.00 100.00 100.00 Using temporary; Using filesort > +NULL UNION RESULT <union2,3,4> ALL NULL NULL NULL NULL NULL 5.00 NULL NULL > +select * from t1 , ( (select distinct t2.a from t2 group by c) union (select distinct t2.a from t2 group by c) union (select t3.a from t3 group by b))q where t1.a=q.a; > +a a > +1 1 > +2 2 > +3 3 > +5 5 > +6 6 > +drop table t1,t2,t3; > diff --git a/mysql-test/t/mdev9959.test b/mysql-test/t/mdev9959.test > new file mode 100644 > index 00000000000..77970f363ac > --- /dev/null > +++ b/mysql-test/t/mdev9959.test > @@ -0,0 +1,28 @@ > +create table t1(a int); > +insert into t1 values (1),(2),(3),(4),(5),(6); > +create table t2(a int, b int,c int); > +insert into t2(a,b,c) values (1,1,2),(2,2,3),(3,1,4),(4,2,2),(5,1,1),(6,2,5); > +create table t3(a int, b int); > +insert into t3(a,b) values (1,1),(2,2),(2,1),(1,2),(5,1),(9,2); > + > +# one select in derived table > + > +#with distinct > +analyze select * from t1 , ( (select distinct t2.a from t2 group by c))q where t1.a=q.a; > +select * from t1 , ( (select distinct t2.a from t2 group by c))q where t1.a=q.a; > +# without distinct > +analyze select * from t1 , ( (select t2.a from t2 group by c))q where t1.a=q.a; > +select * from t1 , ( (select t2.a from t2 group by c))q where t1.a=q.a; > + > +# multiple selects in derived table > + > +#UNION ALL > +analyze select * from t1 , ( (select distinct t2.a from t2 group by c) union all (select distinct t2.a from t2 group by c))q where t1.a=q.a; > +select * from t1 , ( (select distinct t2.a from t2 group by c) union all (select distinct t2.a from t2 group by c))q where t1.a=q.a; > + > +#UNION > +analyze select * from t1 , ( (select distinct t2.a from t2 group by c) union (select distinct t2.a from t2 group by c) union (select t3.a from t3 group by b))q where t1.a=q.a; > +select * from t1 , ( (select distinct t2.a from t2 group by c) union (select distinct t2.a from t2 group by c) union (select t3.a from t3 group by b))q where t1.a=q.a; > + > +drop table t1,t2,t3; > + > diff --git a/sql/sql_lex.h b/sql/sql_lex.h > index f0241a32acf..4128c1dd300 100644 > --- a/sql/sql_lex.h > +++ b/sql/sql_lex.h > @@ -761,6 +761,7 @@ class st_select_lex_unit: public st_select_lex_node { > Procedure *last_procedure; /* Pointer to procedure, if such exists */ > > bool columns_are_renamed; > + bool union_all; /* TRUE if we have UNION ALL operation */ This variable is checked but it is never set? > > void init_query(); > st_select_lex* outer_select(); > @@ -800,6 +801,7 @@ class st_select_lex_unit: public st_select_lex_node { > bool union_needs_tmp_table(); > > void set_unique_exclude(); > + bool check_distinct_in_union(); > > friend struct LEX; > friend int subselect_union_engine::exec(); > diff --git a/sql/sql_union.cc b/sql/sql_union.cc > index 857c9a117f5..43ce0418fe5 100644 > --- a/sql/sql_union.cc > +++ b/sql/sql_union.cc > @@ -1960,3 +1960,39 @@ void st_select_lex_unit::set_unique_exclude() > } > } > } > + > +/* > + Check if the selects in the derived table can give distinct rows I think "can" is the wrong word here. UNION/INTERSECT/EXCEPT output "can" be distinct if the underlying selects produce distinct sets of rows. What we are checking is whether the output will be distinct regardless of what data is there in the queried tables. > + > + for example: > + select * from > + ((select t1.a from t1) op (select t2.a from t2) op (select t3.a from t3)); > + the op here being UNION/INTERSECT/EXCEPT > + > + so this function would check if the derived table like the case above > + can give distinct rows or not. Please describe the criteria and provide a couple of examples (either here or in the comments inside the function). > + > + @retval false Distinct rows are not guarenteed > + @retval true Distinct rows are guanrenteed Typos in the above lines. > + > +*/ > + > +bool st_select_lex_unit::check_distinct_in_union() > +{ > + bool is_intersect_present=FALSE; > + st_select_lex* first= first_select(); > + for(st_select_lex *sl=first; sl ; sl=sl->next_select()) > + is_intersect_present|= (sl->linkage == INTERSECT_TYPE); > + > + if (!union_all) > + return true; > + else > + { > + if (union_select) > + { > + if (!is_intersect_present && !union_select->next_select()) > + return true; > + } > + } > + return false; > +} > diff --git a/sql/table.cc b/sql/table.cc > index f42455a8413..c474d2ed4d7 100644 > --- a/sql/table.cc > +++ b/sql/table.cc > @@ -7113,9 +7113,29 @@ bool TABLE::add_tmp_key(uint key, uint key_parts, > if (!keyinfo->rec_per_key) > return TRUE; > bzero(keyinfo->rec_per_key, sizeof(ulong)*key_parts); > - st_select_lex* lex= pos_in_table_list ? pos_in_table_list->derived->first_select(): NULL; > - if (lex && (lex->options & SELECT_DISTINCT)) > - keyinfo->rec_per_key[key_parts-1]=1; > + > + /* > + For the case when there is a derived table that would give distinct rows, > + the index statistics are passed to the join optimizer to tell that > + a ref access to the derived table will produce only one row. > + */ > + > + st_select_lex_unit* derived= pos_in_table_list ? pos_in_table_list->derived: NULL; > + if (derived) > + { > + /* > + This handles the case when we have a single select in the derived table > + */ > + st_select_lex* first= derived->first_select(); > + if (first && !first->is_part_of_union() && > + first->options & SELECT_DISTINCT) > + keyinfo->rec_per_key[key_parts-1]=1; > + else > + { > + if (check_distinct_in_union()) > + keyinfo->rec_per_key[key_parts-1]=1; > + } I think it would be better if there was a single "if", like so: if ((this is a non-union select with SELECT_DISTINCT || check_distinct_in_union()) { keyinfo->rec_per_key[key_parts-1]=1; } > + } > keyinfo->read_stats= NULL; > keyinfo->collected_stats= NULL; > BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] [Commits] b718e87caaf: MDEV-11274: Executing EXPLAIN of complex query over join limit causes server to crash
by Sergey Petrunia 26 Mar '18

26 Mar '18

Hi Varun, On Sat, Mar 24, 2018 at 03:31:59AM +0530, Varun wrote: > revision-id: b718e87caaf2d0a60c19cd042dfc04868f4bb0b4 (mariadb-5.5.56-187-gb718e87caaf) > parent(s): dfc79ae949f6d831ff65a08a5d32d14b59ba06e6 > author: Varun Gupta > committer: Varun Gupta > timestamp: 2018-03-24 03:31:18 +0530 > message: > > MDEV-11274: Executing EXPLAIN of complex query over join limit causes server to crash > > For this case we have a view that is mergeable but we are not able to merge it in the > parent select because that would exceed the maximum tables allowed in the join list, so we > materialise this view > TABLE_LIST::dervied is NULL for such views, it is only set for views which have ALGORITHM=TEMPTABLE > Fixed by making sure TABLE_LIST::derived is set for views that could not be merged > > --- > mysql-test/r/view.result | 197 +++++++++++++++++++++++++++++++++++++++++++++++ > mysql-test/t/view.test | 196 ++++++++++++++++++++++++++++++++++++++++++++++ > sql/table.h | 2 + > 3 files changed, 395 insertions(+) > > diff --git a/sql/table.h b/sql/table.h > index 98f8c7ad73f..7b58f62e659 100644 > --- a/sql/table.h > +++ b/sql/table.h > @@ -46,6 +46,7 @@ struct TABLE_LIST; > class ACL_internal_schema_access; > class ACL_internal_table_access; > class Field; > +struct LEX; > > /* > Used to identify NESTED_JOIN structures within a join (applicable only to What is the above needed for? Please remove this change. Ok to push after this is addressed. > @@ -2125,6 +2126,7 @@ struct TABLE_LIST > DBUG_PRINT("enter", ("Alias: '%s' Unit: %p", > (alias ? alias : "<NULL>"), > get_unit())); > + derived= get_unit(); > derived_type= ((derived_type & (derived ? DTYPE_MASK : DTYPE_VIEW)) | > DTYPE_TABLE | DTYPE_MATERIALIZE); > set_check_materialized(); BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] 108cdbf: MDEV-10871 Add logging capability to pam_user_map.c
by Sergei Golubchik 24 Mar '18

24 Mar '18

Hi, Alexey! On Mar 23, Alexey Botchkov wrote: > > Technically yes, there can be many arguments. > BTW i think some other arguments make sence. > We can specify the configuration file name in arguments (now it's hardcoded > as "/etc/security/user_map.conf"), > then we can specify some 'user: user_map' pairs as arguments, > to make an addition to what is in the user_map.conf or we can go by > withouth the .conf > file at all. > Is it worth adding the todo-mdev? Agree. File name — makes sense, user:map pairs probably not, I don't remember any other pam module doing that. Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] MDEV-15620 Crash when using "SET @@NEW.a=expr" inside a trigger
by Alexander Barkov 23 Mar '18

23 Mar '18

Hello Sanja, Please review a patch for MDEV-15620. Thanks!

1 0

Re: [Maria-developers] 108cdbf: MDEV-10871 Add logging capability to pam_user_map.c
by Sergei Golubchik 22 Mar '18

22 Mar '18

Hi, Alexey! Looks good, of course. See few minor comments below. On Mar 18, Alexey Botchkov wrote: > revision-id: 108cdbf854df6af1ed2c0fc3a63e1786f6b4b7b4 (mariadb-10.1.31-61-g108cdbf) > parent(s): a0c722d853071e605ea025168da1b01bfe21092c > committer: Alexey Botchkov > timestamp: 2018-03-18 12:03:59 +0400 > message: > > MDEV-10871 Add logging capability to pam_user_map.c. > > 'debug' option added to the pam_user_map.so. > It writes the verbose report to the syslog. don't indent the commit comment, please > > --- > plugin/auth_pam/mapper/pam_user_map.c | 74 +++++++++++++++++++++++++++++++++-- > 1 file changed, 71 insertions(+), 3 deletions(-) > > diff --git a/plugin/auth_pam/mapper/pam_user_map.c b/plugin/auth_pam/mapper/pam_user_map.c > index e62be94..1706630 100644 > --- a/plugin/auth_pam/mapper/pam_user_map.c > +++ b/plugin/auth_pam/mapper/pam_user_map.c > @@ -26,10 +26,13 @@ top: accounting > > #include <stdlib.h> > #include <stdio.h> > +#include <ctype.h> > +#include <string.h> > #include <syslog.h> > #include <grp.h> > #include <pwd.h> > > +#include <security/pam_ext.h> > #include <security/pam_modules.h> > > #define FILENAME "/etc/security/user_map.conf" > @@ -90,9 +93,40 @@ static int user_in_group(const gid_t *user_groups, int ng,const char *group) > } > > > +static void print_groups(pam_handle_t *pamh, const gid_t *user_groups, int ng) > +{ > + char buf[256]; > + char *c_buf= buf, *buf_end= buf+sizeof(buf)-1; > + struct group *gr; > + > + for (; ng > 0; user_groups++, ng--) > + { > + char *c; > + if (!(gr= getgrgid(*user_groups)) || > + !(c= gr->gr_name)) > + continue; > + while (*c) > + { > + if (c_buf == buf_end) > + break; > + *(c_buf++)= *c; > + } > + if (c_buf == buf_end) > + break; > + *(c_buf++)= ','; > + } > + *c_buf= 0; > + pam_syslog(pamh, LOG_DEBUG, "User belongs to groups [%s].\n", buf); s/pam_syslog/SYSLOG_DEBUG/ ? > +} > + > + > +static const char debug_keyword[]= "debug"; > +#define SYSLOG_DEBUG if (mode_debug) pam_syslog > + > int pam_sm_authenticate(pam_handle_t *pamh, int flags, > int argc, const char *argv[]) > { > + int mode_debug= 0; > int pam_err, line= 0; > const char *username; > char buf[256]; > @@ -101,6 +135,14 @@ int pam_sm_authenticate(pam_handle_t *pamh, int flags, > gid_t *groups= group_buffer; > int n_groups= -1; > > + for (; argc > 0; argv++) > + { > + if (strcasecmp(*argv, debug_keyword) == 0) > + mode_debug= 1; > + } can there be many arguments? > + > + SYSLOG_DEBUG(pamh, LOG_DEBUG, "Opening file '%s'.\n", FILENAME); > > + > f= fopen(FILENAME, "r"); > if (f == NULL) > { > @@ -110,12 +152,18 @@ int pam_sm_authenticate(pam_handle_t *pamh, int flags, > > pam_err = pam_get_item(pamh, PAM_USER, (const void**)&username); > if (pam_err != PAM_SUCCESS) > + { > + pam_syslog(pamh, LOG_ERR, "Cannot get username.\n"); > goto ret; > + } > + > + SYSLOG_DEBUG(pamh, LOG_DEBUG, "Incoming username '%s'.\n", username); > > while (fgets(buf, sizeof(buf), f) != NULL) > { > char *s= buf, *from, *to, *end_from, *end_to; > int check_group; > + int cmp_result; > > line++; > > @@ -124,7 +172,11 @@ int pam_sm_authenticate(pam_handle_t *pamh, int flags, > if ((check_group= *s == '@')) > { > if (n_groups < 0) > + { > n_groups= populate_user_groups(username, &groups); > + if (mode_debug) > + print_groups(pamh, groups, n_groups); > + } > s++; > } > from= s; > @@ -139,14 +191,29 @@ int pam_sm_authenticate(pam_handle_t *pamh, int flags, > if (end_to == to) goto syntax_error; > > *end_from= *end_to= 0; > - if (check_group ? > - user_in_group(groups, n_groups, from) : > - (strcmp(username, from) == 0)) > + > + cmp_result= 0; looks redundant ^^^ > + if (check_group) > + { > + cmp_result= user_in_group(groups, n_groups, from); > + SYSLOG_DEBUG(pamh, LOG_DEBUG, "Check if user is in gourp '%s': %s\n", s/gourp/group/ > + from, cmp_result ? "YES":"NO"); > + } > + else > + { > + cmp_result= (strcmp(username, from) == 0); > + SYSLOG_DEBUG(pamh, LOG_DEBUG, "Check if username '%s': %s\n", > + from, cmp_result ? "YES":"NO"); > + } > + if (cmp_result) > { > pam_err= pam_set_item(pamh, PAM_USER, to); > + SYSLOG_DEBUG(pamh, LOG_DEBUG, "User authenticated as '%s'\n", to); may be "user mapped to %s" ? > goto ret; > } > } > + > + SYSLOG_DEBUG(pamh, LOG_DEBUG, "User not found in the list.\n"); > pam_err= PAM_AUTH_ERR; > goto ret; > > @@ -162,6 +229,7 @@ int pam_sm_authenticate(pam_handle_t *pamh, int flags, > return pam_err; > } > > + > int pam_sm_setcred(pam_handle_t *pamh, int flags, > int argc, const char *argv[]) > { Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] GSoC 2018 - Histograms with equal-width bins
by Teodor Niculescu 22 Mar '18

22 Mar '18

Greetings, My name is Teodor Niculescu and I am a second year undergraduate student at the The Faculty of Automatic Control and Computers, University Politehnica of Bucharest, Romania. I am interested in contributing to MariaDB as part of this years Google Summer of Code. I am interested in the project: MDEV-12313 Histograms with equal-width bins. I would like to ask you about a good place to start reading the codebase, in order to help me get an idea about what I need to do for this project. I look forward to hearing from you. Sincerely, Teodor.

2 1

Re: [Maria-developers] MDEV-15492: Subquery crash similar to MDEV-10050
by Sergei Golubchik 21 Mar '18

21 Mar '18

Hi, Oleksandr! On Mar 21, Oleksandr Byelkin wrote: > revision-id: 49a443c53a7f01812299f20e94c7209579efd881 (mariadb-5.5.59-45-g49a443c53a7) > parent(s): 0943b33de3daa0fcbf58803be8e991941de63218 > author: Oleksandr Byelkin > committer: Oleksandr Byelkin > timestamp: 2018-03-21 12:13:37 +0100 > message: > > MDEV-15492: Subquery crash similar to MDEV-10050 > > Detection of first execution of PS fixed. > More debug info. > > diff --git a/sql/sql_prepare.cc b/sql/sql_prepare.cc > index a3bf9d6c93c..369a9f3a804 100644 > --- a/sql/sql_prepare.cc > +++ b/sql/sql_prepare.cc > @@ -3819,6 +3819,7 @@ bool Prepared_statement::execute(String *expanded_query, bool open_cursor) > Statement stmt_backup; > Query_arena *old_stmt_arena; > bool error= TRUE; > + bool qc_executed; > > char saved_cur_db_name_buf[SAFE_NAME_LEN+1]; > LEX_STRING saved_cur_db_name= > @@ -3931,12 +3932,14 @@ bool Prepared_statement::execute(String *expanded_query, bool open_cursor) > 1); > error= mysql_execute_command(thd); > MYSQL_QUERY_EXEC_DONE(error); > + qc_executed= FALSE; > } > else > { > thd->lex->sql_command= SQLCOM_SELECT; > status_var_increment(thd->status_var.com_stat[SQLCOM_SELECT]); > thd->update_stats(); > + qc_executed= TRUE; > } > } mostly ok, but this leaves one code path where qc_executed stays uninitialized. Better assign a default value of FALSE when declaring a variable, and set it to TRUE when really executed. ok to push after that. > > @@ -3960,7 +3963,7 @@ bool Prepared_statement::execute(String *expanded_query, bool open_cursor) > thd->set_statement(&stmt_backup); > thd->stmt_arena= old_stmt_arena; > > - if (state == Query_arena::STMT_PREPARED) > + if (state == Query_arena::STMT_PREPARED && !qc_executed) > state= Query_arena::STMT_EXECUTED; > > if (error == 0 && this->lex->sql_command == SQLCOM_CALL) Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] [Commits] b960f68b102: MDEV-11274: Executing EXPLAIN of complex query over join limit causes server to crash
by Sergey Petrunia 21 Mar '18

21 Mar '18

Hi Varun, On Tue, Jan 23, 2018 at 10:49:37AM +0530, Varun wrote: > revision-id: b960f68b102ec72cf75252b357760e99c5dffc18 (mariadb-5.5.56-144-gb960f68b102) > parent(s): a7a4519a40c58947796c6d9b2e4e58acc18aeef8 > author: Varun Gupta > committer: Varun Gupta > timestamp: 2018-01-23 10:31:02 +0530 > message: > > MDEV-11274: Executing EXPLAIN of complex query over join limit causes server to crash > > For this case we have a view that is mergeable but we are not able to merge it in the > parent select because that would exceed the maximum tables allowed in the join list, so we > materialise this view > TABLE_LIST::dervied is NULL for such views, it is only set for views which have ALGORITHM=TEMPTABLE > Fixed by using SELECT_LEX_UNIT representing the result of the query within the view definition ... > diff --git a/sql/table.cc b/sql/table.cc > index 9cade76cb78..8f377d5bda8 100644 > --- a/sql/table.cc > +++ b/sql/table.cc > @@ -6817,9 +6817,8 @@ int TABLE_LIST::fetch_number_of_rows() > if (jtbm_subselect) > return 0; > if (is_materialized_derived() && !fill_me) > - > { > - table->file->stats.records= ((select_union*)derived->result)->records; > + table->file->stats.records= ((select_union*)get_unit()->result)->records; > set_if_bigger(table->file->stats.records, 2); So, regular VIEWs with algorithm=TEMPTABLE have derived == view->unit This is set in mysql_make_view(): ... table->derived_type= VIEW_ALGORITHM_TMPTABLE; DBUG_PRINT("info", ("algorithm: TEMPORARY TABLE")); view_select->linkage= DERIVED_TABLE_TYPE; ... However, the view in the example has algorithm=MERGE so the execution doesn't reach that point. The decision to materialized is made in mysql_derived_merge. It calls derived->set_materialized_derived(); which calls set_check_materialized(); I have discussed this with Sanja, and the outcome is that a better solution would be to set TABLE_LIST::derived when we decide that this view is going to be handled with temp.table strategy. set_check_materialized() seems to be the right place to do it (although its name looks confusing. What does one "check"? Probably the name is a leftover from some feature backport from MySQL). Could you change the patch to set TABLE_LIST::derived and see if that works? BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] [Commits] 0f35480f3d6: MDEV-6707: Wrong result (extra row) with group by, multi-part key
by Sergey Petrunia 20 Mar '18

20 Mar '18

Hi Varun, Nice to see feedback for the first review fixed, but there's more to do. Please find more input below. On Fri, Jan 19, 2018 at 01:07:20AM +0530, Varun wrote: > revision-id: 0f35480f3d60fac91bf1f91f8140ac5f55724139 (mariadb-5.5.56-142-g0f35480f3d6) > parent(s): fafdac3365f4943e73bcefd0e0d07d69997a9724 > author: Varun Gupta > committer: Varun Gupta > timestamp: 2018-01-18 23:53:35 +0530 > message: > > MDEV-6707: Wrong result (extra row) with group by, multi-part key > > This case involves using a composite key,few parts of which are involved in GROUP BY and few in the MIN/MAX > functions in the select list. > Ranges are created in accordance with the where condition, so during the execution of such queries, > we try to find a prefix using the fields involved in GROUP BY > and we then check if this newly returned prefix lies within the range we had calculated earlier. > > For queries which use composite key, few parts of which are involved in GROUP BY and few in the MIN/MAX functtions > in the select list, we try to find the prefix of the ranges by using the fields involved in the group by clause. > We get extra rows in the output when we have same partial ranges created for the fields in the GROUP BY clause > > This issue can be fixed if we compare such partial ranges and don't lookup if we see the same prefix again. > > --- > mysql-test/r/range.result | 14 +++++++++++++ > mysql-test/r/range_mrr_icp.result | 14 +++++++++++++ > mysql-test/t/range.test | 13 ++++++++++++ > sql/opt_range.cc | 44 ++++++++++++++++++++++++++++++++++++--- > sql/opt_range.h | 5 +++-- > 5 files changed, 85 insertions(+), 5 deletions(-) > > diff --git a/mysql-test/r/range.result b/mysql-test/r/range.result > index 630a692cef6..72b31de4df4 100644 > --- a/mysql-test/r/range.result > +++ b/mysql-test/r/range.result > @@ -2144,3 +2144,17 @@ value1 1000685 12345 > value1 1003560 12345 > value1 1004807 12345 > drop table t1; > +# > +# MDEV-6707: Wrong result (extra row) with group by, multi-part key > +# > +CREATE TABLE t1 (f1 INT, f2 VARCHAR(1), KEY(f2,f1)) ENGINE=InnoDB; > +INSERT INTO t1 VALUES > +(7,'v'),(0,'s'),(9,'l'),(4,'c'); > +explain > +SELECT MAX(f1), f2 FROM t1 WHERE f2 LIKE 'c%' AND f1 <> 9 GROUP BY f2; > +id select_type table type possible_keys key key_len ref rows Extra > +1 SIMPLE t1 range f2 f2 9 NULL 2 Using where; Using index for group-by > +SELECT MAX(f1), f2 FROM t1 WHERE f2 LIKE 'c%' AND f1 <> 9 GROUP BY f2; > +MAX(f1) f2 > +4 c > +DROP TABLE t1; > diff --git a/mysql-test/r/range_mrr_icp.result b/mysql-test/r/range_mrr_icp.result > index 3f5de5b0189..1050bfcd887 100644 > --- a/mysql-test/r/range_mrr_icp.result > +++ b/mysql-test/r/range_mrr_icp.result > @@ -2146,4 +2146,18 @@ value1 1000685 12345 > value1 1003560 12345 > value1 1004807 12345 > drop table t1; > +# > +# MDEV-6707: Wrong result (extra row) with group by, multi-part key > +# > +CREATE TABLE t1 (f1 INT, f2 VARCHAR(1), KEY(f2,f1)) ENGINE=InnoDB; > +INSERT INTO t1 VALUES > +(7,'v'),(0,'s'),(9,'l'),(4,'c'); > +explain > +SELECT MAX(f1), f2 FROM t1 WHERE f2 LIKE 'c%' AND f1 <> 9 GROUP BY f2; > +id select_type table type possible_keys key key_len ref rows Extra > +1 SIMPLE t1 range f2 f2 9 NULL 2 Using where; Using index for group-by > +SELECT MAX(f1), f2 FROM t1 WHERE f2 LIKE 'c%' AND f1 <> 9 GROUP BY f2; > +MAX(f1) f2 > +4 c > +DROP TABLE t1; > set optimizer_switch=@mrr_icp_extra_tmp; > diff --git a/mysql-test/t/range.test b/mysql-test/t/range.test > index 393ca68e945..62e907b7b4a 100644 > --- a/mysql-test/t/range.test > +++ b/mysql-test/t/range.test > @@ -1718,3 +1718,16 @@ where (key1varchar='value1' AND (key2int <=1 OR key2int > 1)); > --echo # The following must show col1=12345 for all rows: > select * from t1; > drop table t1; > + > +--echo # > +--echo # MDEV-6707: Wrong result (extra row) with group by, multi-part key > +--echo # > + > +CREATE TABLE t1 (f1 INT, f2 VARCHAR(1), KEY(f2,f1)) ENGINE=InnoDB; > +INSERT INTO t1 VALUES > +(7,'v'),(0,'s'),(9,'l'),(4,'c'); > + > +explain > +SELECT MAX(f1), f2 FROM t1 WHERE f2 LIKE 'c%' AND f1 <> 9 GROUP BY f2; > +SELECT MAX(f1), f2 FROM t1 WHERE f2 LIKE 'c%' AND f1 <> 9 GROUP BY f2; > +DROP TABLE t1; > diff --git a/sql/opt_range.cc b/sql/opt_range.cc > index 25a9e729a8b..be959764b16 100644 > --- a/sql/opt_range.cc > +++ b/sql/opt_range.cc > @@ -11249,6 +11249,7 @@ int QUICK_RANGE_SELECT::get_next() > @param prefix_length length of cur_prefix > @param group_key_parts The number of key parts in the group prefix > @param cur_prefix prefix of a key to be searched for > + @param save_last_range INOUT Saving the last range we encountered > > Each subsequent call to the method retrieves the first record that has a > prefix with length prefix_length and which is different from cur_prefix, > @@ -11271,7 +11272,8 @@ int QUICK_RANGE_SELECT::get_next() > > int QUICK_RANGE_SELECT::get_next_prefix(uint prefix_length, > uint group_key_parts, > - uchar *cur_prefix) > + uchar *cur_prefix, > + QUICK_RANGE **save_last_range) > { > DBUG_ENTER("QUICK_RANGE_SELECT::get_next_prefix"); > const key_part_map keypart_map= make_prev_keypart_map(group_key_parts); > @@ -11301,8 +11303,43 @@ int QUICK_RANGE_SELECT::get_next_prefix(uint prefix_length, > last_range= 0; > DBUG_RETURN(HA_ERR_END_OF_FILE); > } > + > last_range= *(cur_range++); > > + /* > + While calculating these prefixes we might encounter a case where there > + woud be same partial ranges for multiple ranges. Typo. > + An example would be > + select max(f1), f2 from t1 where f2 ='c' AND f1 <> 9 group by f2; > + so the ranges would be > + (NULL,c <= f1,f2 <= c,9) > + (c,9 <= f1,f2 <= c,+infinity) This looks as if index was defined on (f1, f2) ? Then the comment doesn't make sense because loose index scan does not support queries in form: select max(key_part1) ... group by key_part2 It only can handle them when the index is on (f2,f1), but this is counterintuitive for the reader that is not familiar with the testcase for this bug. Please rephrase the comment. Please use key_part1 and key_part2 as column names. Tuple comparisons should look like this (note the braces): (a,b) < (c,d) . > + > + In this case for calculating prefixes with the group by field we take up the > + partial ranges involving field f2 those would be > + c<= f2 <=c > + c<= f2 <=c > + > + So we lookup rows with the same prefix in all such ranges and > + then we check for the other part(in this case f1) in ALL the ranges. > + So if a record lies in a range, then it would satisfy both the partial > + ranges in this case and therefore there would be multiple outputs for > + the same row. > + For such cases we should calculate the prefix only when we have the next > + partial range different from the previous one. > + */ > + > + > + if (*save_last_range) > + { > + if (!key_tuple_cmp(key_part_info, (*save_last_range)->min_key, > + last_range->min_key, prefix_length)) > + { > + last_range=NULL; > + continue; > + } > + } > + *save_last_range= last_range; > key_range start_key, end_key; > last_range->make_min_endpoint(&start_key, prefix_length, keypart_map); > last_range->make_max_endpoint(&end_key, prefix_length, keypart_map); > @@ -13315,7 +13352,7 @@ QUICK_GROUP_MIN_MAX_SELECT(TABLE *table, JOIN *join_arg, bool have_min_arg, > seen_first_key(FALSE), doing_key_read(FALSE), min_max_arg_part(min_max_arg_part_arg), > key_infix(key_infix_arg), key_infix_len(key_infix_len_arg), > min_functions_it(NULL), max_functions_it(NULL), > - is_index_scan(is_index_scan_arg) > + is_index_scan(is_index_scan_arg), save_last_range(NULL) This is not the right place for this initialization. It needs to be in QUICK_GROUP_MIN_MAX_SELECT::reset(), as the quick select may be executed multiple times. Testcase: create table ten(a int); insert into ten values (0),(1),(2),(3),(4),(5),(6),(7),(8),(9); MariaDB [test]> select (SELECT MAX(f1) as MAXVAL FROM t1 WHERE f2 LIKE 'c%' AND f1 <> 9 GROUP BY f2) AS SUBQ from ten; +------+ | SUBQ | +------+ | 4 | | 4 | | 4 | | 4 | | 4 | | 4 | | 4 | | 4 | | 4 | | 4 | +------+ 10 rows in set (0.00 sec) Now, let's add a HAVING clause (which always evaluates to TRUE but forces the subquery to be recomputed: MariaDB [test]> select (SELECT MAX(f1) as MAXVAL FROM t1 WHERE f2 LIKE 'c%' AND f1 <> 9 GROUP BY f2 having maxval<100+ten.a) AS SUBQ from ten; +------+ | SUBQ | +------+ | 4 | | NULL | | NULL | | NULL | | NULL | | NULL | | NULL | | NULL | | NULL | | NULL | +------+ 10 rows in set (0.00 sec) To make sure this is loose index, try with IGNORE INDEX: MariaDB [test]> select (SELECT MAX(f1) as MAXVAL FROM t1 ignore index(f2) WHERE f2 LIKE 'c%' AND f1 <> 9 GROUP BY f2 having maxval<100+ten.a) AS SUBQ from ten; +------+ | SUBQ | +------+ | 4 | | 4 | | 4 | | 4 | | 4 | | 4 | | 4 | | 4 | | 4 | | 4 | +------+ 10 rows in set (0.00 sec) Please fix this and add the above into the testcase. > { > head= table; > index= use_index; > @@ -13968,7 +14005,8 @@ int QUICK_GROUP_MIN_MAX_SELECT::next_prefix() > uchar *cur_prefix= seen_first_key ? group_prefix : NULL; > if ((result= quick_prefix_select->get_next_prefix(group_prefix_len, > group_key_parts, > - cur_prefix))) > + cur_prefix, > + &save_last_range))) > DBUG_RETURN(result); > seen_first_key= TRUE; > } > diff --git a/sql/opt_range.h b/sql/opt_range.h > index b8b46ae5ab1..206d65f6da4 100644 > --- a/sql/opt_range.h > +++ b/sql/opt_range.h > @@ -477,7 +477,7 @@ class QUICK_RANGE_SELECT : public QUICK_SELECT_I > int get_next(); > void range_end(); > int get_next_prefix(uint prefix_length, uint group_key_parts, > - uchar *cur_prefix); > + uchar *cur_prefix, QUICK_RANGE **save_last_range); > bool reverse_sorted() { return 0; } > bool unique_key_range(); > int init_ror_merged_scan(bool reuse_handler, MEM_ROOT *alloc); > @@ -916,7 +916,8 @@ class QUICK_GROUP_MIN_MAX_SELECT : public QUICK_SELECT_I > Use index scan to get the next different key instead of jumping into it > through index read > */ > - bool is_index_scan; > + bool is_index_scan; > + QUICK_RANGE *save_last_range; Please add a comment describing the new member variable. > public: > /* > The following two members are public to allow easy access from BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] GSoC 2018 Student Self Introduction & Question
by zhzhzoo zhang 20 Mar '18

20 Mar '18

Hello everybody, I am a first year graduate in Peking University major in math. I started to take part in algorithm contests at grade 7 so I have a lot of programming experience and I am very well trained in algorithms and data structures. In my childhood memory MySQL was the most popular DBMS choice for personal website (LAMP stack). However it was acquired by Oracle as I grew up. In this year's GSoC projects, `optimizer trace' interested me particularly. It would be great for MariaDB to have this functionality. This functionality is already implemented in MySQL. So my question is, can I read MySQL's source code implementing this function? Would that cause legal problems? Thanks very much! Regards, Zhang Shuai.

3 2

Re: [Maria-developers] [Commits] a99dcdf7e9e: MDEV-6731: Valgrind warnings 'Invalid read' in subselect_engine::calc_const_tables with SQ in WHERE and
by Sergey Petrunia 16 Mar '18

16 Mar '18

Hi Varun, Please fix the MDEV# in the commit comment. Ok to push after that. On Wed, Mar 14, 2018 at 12:11:20PM +0530, Varun wrote: > revision-id: a99dcdf7e9ede3b96466eb1294284141805747fc (mariadb-10.0.30-308-ga99dcdf7e9e) > parent(s): 48c11d407b409a1291c84782518c69ee9138ce72 > author: Varun Gupta > committer: Varun Gupta > timestamp: 2018-03-14 12:09:49 +0530 > message: > > MDEV-6731: Valgrind warnings 'Invalid read' in subselect_engine::calc_const_tables with SQ in WHERE and > HAVING, ORDER BY, materialization+semijoin > > During cleanup a pointer to the materialised table that was freed was not set to NULL > > --- > mysql-test/r/having.result | 14 ++++++++++++++ > mysql-test/t/having.test | 18 ++++++++++++++++++ > sql/sql_select.cc | 4 +++- > 3 files changed, 35 insertions(+), 1 deletion(-) > > diff --git a/mysql-test/r/having.result b/mysql-test/r/having.result > index 0b933427303..99059c25fbd 100644 > --- a/mysql-test/r/having.result > +++ b/mysql-test/r/having.result > @@ -723,4 +723,18 @@ SELECT * FROM t1 JOIN t2 ON c1 = c2 HAVING c2 > 'a' ORDER BY c2 LIMIT 1; > c1 c2 > x x > DROP TABLE t1,t2; > +# > +# MDEV-6736: Valgrind warnings 'Invalid read' in subselect_engine::calc_const_tables with SQ > +# in WHERE and HAVING, ORDER BY, materialization+semijoin > +# > +CREATE TABLE t1 (a INT) ENGINE=MyISAM; > +INSERT INTO t1 VALUES (3),(8); > +CREATE TABLE t2 (b INT) ENGINE=MyISAM; > +INSERT INTO t2 VALUES (2),(1); > +SELECT a FROM t1 > +WHERE 9 IN ( SELECT MIN( a ) FROM t1 ) > +HAVING a <> ( SELECT COUNT(*) FROM t2 ) > +ORDER BY a; > +a > +DROP TABLE t1,t2; > End of 10.0 tests > diff --git a/mysql-test/t/having.test b/mysql-test/t/having.test > index 1682fe5b874..e624cd630dd 100644 > --- a/mysql-test/t/having.test > +++ b/mysql-test/t/having.test > @@ -759,4 +759,22 @@ SELECT * FROM t1 JOIN t2 ON c1 = c2 HAVING c2 > 'a' ORDER BY c2 LIMIT 1; > > DROP TABLE t1,t2; > > +--echo # > +--echo # MDEV-6736: Valgrind warnings 'Invalid read' in subselect_engine::calc_const_tables with SQ > +--echo # in WHERE and HAVING, ORDER BY, materialization+semijoin > +--echo # > + > +CREATE TABLE t1 (a INT) ENGINE=MyISAM; > +INSERT INTO t1 VALUES (3),(8); > + > +CREATE TABLE t2 (b INT) ENGINE=MyISAM; > +INSERT INTO t2 VALUES (2),(1); > + > +SELECT a FROM t1 > +WHERE 9 IN ( SELECT MIN( a ) FROM t1 ) > +HAVING a <> ( SELECT COUNT(*) FROM t2 ) > +ORDER BY a; > + > +DROP TABLE t1,t2; > + > --echo End of 10.0 tests > diff --git a/sql/sql_select.cc b/sql/sql_select.cc > index fd8ff6eb016..37d68c730dd 100644 > --- a/sql/sql_select.cc > +++ b/sql/sql_select.cc > @@ -11476,13 +11476,15 @@ void JOIN_TAB::cleanup() > } > else > { > + TABLE_LIST *tmp= table->pos_in_table_list; > end_read_record(&read_record); > - table->pos_in_table_list->jtbm_subselect->cleanup(); > + tmp->jtbm_subselect->cleanup(); > /* > The above call freed the materializedd temptable. Set it to NULL so > that we don't attempt to touch it if JOIN_TAB::cleanup() is invoked > multiple times (it may be) > */ > + tmp->table= NULL; > table=NULL; > } > DBUG_VOID_RETURN; > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits -- BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] d8e3cfacebc: MDEV-14616: WSREP has not yet prepared node for application use error
by Sergei Golubchik 15 Mar '18

15 Mar '18

Hi, Jan! On Feb 13, jan wrote: > revision-id: d8e3cfacebc47849532ef3e7bbf18f9da53bab88 (mariadb-10.1.31-20-gd8e3cfacebc) > parent(s): 474f3df092cc73342b2da03aa03ba5c66930f596 > author: Jan Lindström > committer: Jan Lindström > timestamp: 2018-02-13 10:04:48 +0200 > message: > > MDEV-14616: WSREP has not yet prepared node for application use error > > Contains following Galera fixes: > > MW-405 Make sure wsrep is ready in wait_until_connected_again.inc What is "MW-405", can I see the original bug report somewhere? > wait_until_connected_again issues 'SHOW STATUS' query repeatedly > until mysqld replies without errors. However, SHOW STATUS is > treated specially by wsrep in that it is allowed to proceed > even if wsrep is not yet in ready state. As a consequence, > after returning from wait_until_connected_again, wsrep may > not be ready yet and subsequent queries may fail with error > "1047 WSREP has not yet prepared node for application use". > To avoid those errors, the patch includes > wait_wsrep_ready.inc at the end of the wait_until_connected_again.inc I don't quite understand the issue. Old code did wait_wsrep_ready.inc after wait_until_connected_again.inc. Now wait_wsrep_ready.inc is included in wait_until_connected_again.inc. This seems to imply that the bug was "there were some cases where wait_wsrep_ready.inc was not included after wait_until_connected_again.inc". There are, of course, two ways to fix such a bug. Either include wait_wsrep_ready.inc after wait_until_connected_again.inc, or include wait_wsrep_ready.inc in wait_until_connected_again.inc. The latter fix implies that there are no possible cases when one would want to wait for a server to be connected, but before wsrep is ready. Are there? For the purpose of testing, that could be useful, could it not? > kill_galera.inc can no longer rely on wait_until_connected_again.inc. > This is because wait_until_connected_again now tries to make sure > that the server it is connected eventually transition to ready state. > Whereas some tests may need to kill galera while the server is in a > non-primary view. Which means that there are cases when a test needs the server connected, but before wsrep is ready, right? > MW-408 Fix 'WSREP error while trying to determine node state' > > mysql-test-run.pl sporadically reports 'WSREP error while > trying to determine node state' right after starting servers > for test execution. This happens because we try to execute a > SELECT statement that queries the current value of status variable > wsrep_ready. If this statement fails, the above message is reported. > The failure is due to fact that wsrep may return error > ER_LOCK_WAIT_TIMEOUT (on any SELECT statement) if it is not ready > and wsrep_sync_wait enabled for SELECTs. The fix is to disable > wsrep_sync_wait for the session that issues those SELECT statements. 1. Could you put this fix in a separate commit please? 2. Sorry, I don't understand. Where do "we try to execute a SELECT statement that queries the current value of status variable wsrep_ready" ? > --- > mysql-test/include/restart_mysqld.inc | 3 - > mysql-test/include/start_mysqld.inc | 3 - > mysql-test/include/wait_until_connected_again.inc | 11 ++- > mysql-test/mysql-test-run.pl | 104 ++++++++++++++++++++++ > mysql-test/suite/galera/include/kill_galera.inc | 1 - > 5 files changed, 111 insertions(+), 11 deletions(-) > > diff --git a/mysql-test/include/restart_mysqld.inc b/mysql-test/include/restart_mysqld.inc > index a0447280ff5..940e081c431 100644 > --- a/mysql-test/include/restart_mysqld.inc > +++ b/mysql-test/include/restart_mysqld.inc > @@ -50,9 +50,6 @@ if (!$restart_parameters) > # Call script that will poll the server waiting for it to be back online again > --source include/wait_until_connected_again.inc > > -# Wait for wsrep > ---source include/wait_wsrep_ready.inc > - > # Turn off reconnect again > --disable_reconnect > > diff --git a/mysql-test/include/start_mysqld.inc b/mysql-test/include/start_mysqld.inc > index 04dff714d49..e31f26aad8c 100644 > --- a/mysql-test/include/start_mysqld.inc > +++ b/mysql-test/include/start_mysqld.inc > @@ -16,9 +16,6 @@ if (!$restart_parameters) > # Call script that will poll the server waiting for it to be back online again > --source include/wait_until_connected_again.inc > > -# Wait for wsrep > ---source include/wait_wsrep_ready.inc > - > # Turn off reconnect again > --disable_reconnect > > diff --git a/mysql-test/include/wait_until_connected_again.inc b/mysql-test/include/wait_until_connected_again.inc > index 6f64ef45440..4958d276ebd 100644 > --- a/mysql-test/include/wait_until_connected_again.inc > +++ b/mysql-test/include/wait_until_connected_again.inc > @@ -11,10 +11,7 @@ let $counter= 5000; > let $mysql_errno= 9999; > while ($mysql_errno) > { > - # Strangely enough, the server might return "Too many connections" > - # while being shutdown, thus 1040 is an "allowed" error > - # See BUG#36228 > - --error 0,1040,1053,2002,2003,2005,2006,2013,1927 > + --error 0,ER_SERVER_SHUTDOWN,ER_CONNECTION_KILLED,2002,2003,2006,2013 > show status; > > dec $counter; > @@ -26,3 +23,9 @@ while ($mysql_errno) > } > --enable_query_log > --enable_result_log > + > +# WSREP: SHOW STATUS queries are allowed even if wsrep > +# is not ready. Make sure wsrep is ready before > +# returning from this script > + > +--source include/wait_wsrep_ready.inc > diff --git a/mysql-test/mysql-test-run.pl b/mysql-test/mysql-test-run.pl > index eaec51b82b4..324f2e98ebd 100755 > --- a/mysql-test/mysql-test-run.pl > +++ b/mysql-test/mysql-test-run.pl > @@ -2883,6 +2883,49 @@ sub mysql_server_wait { > $warn_seconds); > } > > +sub have_wsrep() { > + my $wsrep_on= $mysqld_variables{'wsrep-on'}; > + return defined $wsrep_on > +} > + > + > +sub check_wsrep_support() { > + if (have_wsrep()) > + { > + mtr_report(" - binaries built with wsrep patch"); > + > + # ADD scripts to $PATH to that wsrep_sst_* can be found > + my ($path) = grep { -f "$_/wsrep_sst_rsync"; } "$::bindir/scripts", $::path_client_bindir; > + mtr_error("No SST scripts") unless $path; > + $ENV{PATH}="$path:$ENV{PATH}"; > + > + # Check whether WSREP_PROVIDER environment variable is set. > + if (defined $ENV{'WSREP_PROVIDER'}) { > + if ((mtr_file_exists($ENV{'WSREP_PROVIDER'}) eq "") && > + ($ENV{'WSREP_PROVIDER'} ne "none")) { > + mtr_error("WSREP_PROVIDER env set to an invalid path"); > + } > + # WSREP_PROVIDER is valid; set to a valid path or "none"). > + mtr_verbose("WSREP_PROVIDER env set to $ENV{'WSREP_PROVIDER'}"); > + } else { > + # WSREP_PROVIDER env not defined. Lets try to locate the wsrep provider > + # library. > + my $file_wsrep_provider= > + mtr_file_exists("/usr/lib/galera/libgalera_smm.so", > + "/usr/lib64/galera/libgalera_smm.so"); > + > + if ($file_wsrep_provider ne "") { > + # wsrep provider library found ! > + mtr_verbose("wsrep provider library found : $file_wsrep_provider"); > + $ENV{'WSREP_PROVIDER'}= $file_wsrep_provider; > + } else { > + mtr_verbose("Could not find wsrep provider library, setting it to 'none'"); > + $ENV{'WSREP_PROVIDER'}= "none"; > + } > + } > + } > +} > + > sub create_config_file_for_extern { > my %opts= > ( > @@ -3341,6 +3384,62 @@ sub run_query { > return $res > } > > +sub run_query_output { > + my ($mysqld, $query, $outfile)= @_; > + > + my $args; > + mtr_init_args(\$args); > + mtr_add_arg($args, "--defaults-file=%s", $path_config_file); > + mtr_add_arg($args, "--defaults-group-suffix=%s", $mysqld->after('mysqld')); > + > + mtr_add_arg($args, "--silent"); > + mtr_add_arg($args, "--execute=%s", $query); > + > + my $res= My::SafeProcess->run > + ( > + name => "run_query_output -> ".$mysqld->name(), > + path => $exe_mysql, > + args => \$args, > + output => $outfile, > + error => $outfile > + ); > + > + return $res > +} > + > +sub wait_wsrep_ready($$) { > + my ($tinfo, $mysqld)= @_; > + > + my $sleeptime= 100; # Milliseconds > + my $loops= ($opt_start_timeout * 1000) / $sleeptime; > + > + my $name= $mysqld->name(); > + my $outfile= "$opt_vardir/tmp/$name.wsrep_ready"; > + my $query= "SET SESSION wsrep_sync_wait = 0; > + SELECT VARIABLE_VALUE > + FROM INFORMATION_SCHEMA.GLOBAL_STATUS > + WHERE VARIABLE_NAME = 'wsrep_ready'"; > + > + for (my $loop= 1; $loop <= $loops; $loop++) > + { > + if (run_query_output($mysqld, $query, $outfile) != 0) > + { > + $tinfo->{logfile}= "WSREP error while trying to determine node state"; > + return 0; > + } > + > + if (mtr_grab_file($outfile) =~ /^ON/) > + { > + unlink($outfile); > + return 1; > + } > + > + mtr_milli_sleep($sleeptime); > + } > + > + $tinfo->{logfile}= "WSREP did not transition to state READY"; > + return 0; > +} > > sub do_before_run_mysqltest($) > { > @@ -5380,6 +5479,11 @@ sub start_servers($) { > $tinfo->{comment}= "Failed to start ".$_->name() . "\n"; > return 1; > } > + > + if (have_wsrep() && !wait_wsrep_ready($tinfo, $_)) > + { > + return 1; > + } > } > return 0; > } > diff --git a/mysql-test/suite/galera/include/kill_galera.inc b/mysql-test/suite/galera/include/kill_galera.inc > index c61bad8e19d..f95dccf2185 100644 > --- a/mysql-test/suite/galera/include/kill_galera.inc > +++ b/mysql-test/suite/galera/include/kill_galera.inc > @@ -18,4 +18,3 @@ > exit(0); > EOF > > ---source include/wait_until_disconnected.inc > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] [Commits] dc3d6c9edb3: MDEV-15477: SESSION_SYSVARS_TRACKER does not track last_gtid
by Sergei Golubchik 09 Mar '18

09 Mar '18

Hi, Oleksandr! On Mar 09, Oleksandr Byelkin wrote: > revision-id: dc3d6c9edb3536e5f0c87ac1e9c04891d50fc420 (mariadb-10.2.13-32-gdc3d6c9edb3) > parent(s): 6cf25e5eb76923550173a0059036d80c80a053d6 > author: Oleksandr Byelkin > committer: Oleksandr Byelkin > timestamp: 2018-03-09 14:39:40 +0100 > message: > > MDEV-15477: SESSION_SYSVARS_TRACKER does not track last_gtid > > register changes of last_gtid > > diff --git a/mysql-test/t/session_tracker_last_gtid.test b/mysql-test/t/session_tracker_last_gtid.test > --- /dev/null > +++ b/mysql-test/t/session_tracker_last_gtid.test > @@ -0,0 +1,18 @@ > + > +--source include/have_innodb.inc > +--source include/have_binlog_format_mixed_or_statement.inc Why "mixed or statement"? > diff --git a/sql/log.cc b/sql/log.cc > --- a/sql/log.cc > +++ b/sql/log.cc > @@ -5945,7 +5945,20 @@ MYSQL_BIN_LOG::write_gtid_event(THD *thd, bool standalone, > } > if (err) > DBUG_RETURN(true); > + bool changed_gtid= (thd->last_commit_gtid.seq_no != gtid.seq_no); > thd->last_commit_gtid= gtid; > + if (changed_gtid && > + thd->session_tracker.get_tracker(SESSION_SYSVARS_TRACKER)->is_enabled()) > + { > + sys_var *svar; > + mysql_mutex_lock(&LOCK_plugin); > + if ((svar= find_sys_var_ex(thd, "last_gtid", > + sizeof("last_gtid") - 1, > + false, true))) > + thd->session_tracker.get_tracker(SESSION_SYSVARS_TRACKER)-> > + mark_as_changed(thd, (LEX_CSTRING*)svar); > + mysql_mutex_unlock(&LOCK_plugin); > + } No, please. You svar will always be Sys_last_gtid. There is no need to search for it on every write_gtid_event(), under a mutex. See in sys_vars.cc around Sys_autocommit how to do it easier. Also, perhaps it'd be safer to rename last_commit_gtid to m_last_commit_gtid, make it private and provide a setter and a getter. The setter will update the tracker. I'll make sure that last_commit_gtid is not changed anywhere else bypassing the tracker. Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] 6cf25e5eb76: Session tracking info support in mysqltest
by Sergei Golubchik 09 Mar '18

09 Mar '18

Hi, Oleksandr! On Mar 09, Oleksandr Byelkin wrote: > revision-id: 6cf25e5eb76923550173a0059036d80c80a053d6 (mariadb-10.2.13-31-g6cf25e5eb76) > parent(s): f2b91c878cf4ec53c292caba7f5907d1ef724de4 > author: Oleksandr Byelkin > committer: Oleksandr Byelkin > timestamp: 2018-03-09 14:27:13 +0100 > message: > > Session tracking info support in mysqltest Add a test case for it. Don't forget to test whether ONCE modifier works. Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] f2b91c878cf: Reset connection support in mysqltest
by Sergei Golubchik 09 Mar '18

09 Mar '18

Hi, Oleksandr! On Mar 09, Oleksandr Byelkin wrote: > revision-id: f2b91c878cf4ec53c292caba7f5907d1ef724de4 (mariadb-10.2.13-30-gf2b91c878cf) > parent(s): 8f98835bb86550a68a6c412d8417cc169dfdc206 > author: Oleksandr Byelkin > committer: Oleksandr Byelkin > timestamp: 2018-03-09 14:26:10 +0100 > message: > > Reset connection support in mysqltest You add a feature - add a test case for it too. Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] MariaDB 5.5, Jemalloc, TokuDB, RHEL
by Michal Schorm 08 Mar '18

08 Mar '18

Hello, so far we packed MariaDB 5.5 for RHEL 7 without TokuDB and without Jemalloc. Recently, a customer asked about the availability of TokuDB, so I'm doing a research, if its possible to pack it without any issues for the users updating. I have a several questions: 1) Is Jemmaloc used elsewhere from TokuDB ? 2) If yes, what all would be affected? Could any issues arise during update from build without Jemalloc to build with it? Any expected performance changes? 3) Would be the use of the bundled Jemalloc (on of the) best option(s)? 4) Anything else that I missed? -- Michal Schorm Associate Software Engineer Core Services - Databases Team Red Hat

2 1

[Maria-developers] Introduction as GSoC 2018 aspirant
by Arshdeep Singh Sachdeva 05 Mar '18

05 Mar '18

Dear MariaDB Developers, I am Arshdeep Singh Sachdeva, currently studying BTech ( C.S.E. ) from Punjab Engineering College, Chandigarh, India. I am interested in contibuting to MariaDB and participating in GSoC 2018 under your guidance. I am proficient in C++, python, linux, mysql. I have already used MariaDB in my projects in college. Also I am presently going through documentation and tutorials. Thanks.

1 0

[Maria-developers] Introduction as a GSoC aspirant
by Aman Jain 05 Mar '18

05 Mar '18

Hello developers and mentors I am Aman and I am interested in contributing to MariaDB as a GSoC'18 aspirant. I am familiar with C++, Web development, SQL, MongoDB and shell scripting and I have set up the MariaDB server. Please suggest me where I can find the source code so that I can get a better understanding of the project ideas and solve some issues to get started. Thanks Aman Jain

2 1

Re: [Maria-developers] [Commit] MDEV-12746 out-of-order retry
by Kristian Nielsen 17 Feb '18

17 Feb '18

andrei.elkin(a)pp.inet.fi writes: > commit 3cebb54e6387a7eace1757c82ed0efd6e11590b9 > Author: Andrei Elkin <andrei.elkin(a)mariadb.com> > Date: Fri Feb 9 15:00:23 2018 +0200 > > MDEV-12746 rpl.rpl_parallel_optimistic_nobinlog fails committing > out of order at retry > > The 2nd issue was out of order commit by transactions that > are ordered after an erroring out transaction. The latter was possible > in the test thanks to the above retry failure. > Out-of-order committing was done despite the slave was stopping. Let me see if I understand this correctly: Normally, a transaction in optimistic parallel replication cannot commit out of order because it either waits for the prior transaction to commit first, or, if the prior transaction has stopped with an error earlier, then the following transaction will not even start. But in this case, during retry, there was missing the check that the prior transaction stopped with an error. Therefore, the following transaction's wait_for_prior_commit() does nothing, and it wrongly goes ahead and runs and even commits, even though an earlier transaction failed and replication is stopping. Correct? That's a nice catch of a subtle, but potentially serious bug. I wonder how this is possible though. As I recall, when a transaction fails, this failure is marked in the wait_for_prior_commit data structures. So the retrying transaction should have gotten an error in wait_for_prior_commit() and aborted the retries. So why does this not happen in this case? > The test failures were of two sorts. One is that the number of potential > temporary errors actually exceeds the default value of the slave retry > option. > Increased @@global.slave_transaction_retries fixes that part. Hm, this shouldn't be needed, I think. The retry waits for all prior transactions to commit first, so after the first retry no further retries should be caused by other parallel replication workers. So I would expect this problem to be a side-effect of the other, real issue, and this increase should not be needed (and might hide another problem later, if left in). > diff --git a/sql/rpl_parallel.cc b/sql/rpl_parallel.cc > index 34f9113f7fe..9f1d23a1161 100644 > --- a/sql/rpl_parallel.cc > +++ b/sql/rpl_parallel.cc > @@ -745,7 +761,25 @@ retry_event_group(rpl_group_info *rgi, rpl_parallel_thread *rpt, > for (;;) > { > mysql_mutex_lock(&entry->LOCK_parallel_entry); > - register_wait_for_prior_event_group_commit(rgi, entry); > + if (entry->stop_on_error_sub_id == (uint64) ULONGLONG_MAX || > +#ifndef DBUG_OFF > + (DBUG_EVALUATE_IF("simulate_mdev_12746", 1, 0)) || > +#endif > + rgi->gtid_sub_id < entry->stop_on_error_sub_id) > + { > + register_wait_for_prior_event_group_commit(rgi, entry); > + } > + else > + { > + /* > + An earlier transaction by another worker may have not had not > + have the current one in subsequent committers list and thus > + the actual induced error status could not have been sumbitted. > + So it should be set by ourselves now. > + */ This comment is hard to understand - what do you mean here? > + if (!rgi->worker_error) > + rgi->worker_error= 1; > + } I would have expected an abort (goto err) here (or well just below, after unlocking the mutex). Is this not needed, and if so, why not? > @@ -716,12 +724,20 @@ retry_event_group(rpl_group_info *rgi, rpl_parallel_thread *rpt, > unregistering (and later re-registering) the wait. > */ > if(thd->wait_for_commit_ptr) > - thd->wait_for_commit_ptr->unregister_wait_for_prior_commit(); > + thd->wait_for_commit_ptr->unregister_wait_for_prior_commit(); Indentation? - Kristian.

2 5

[Maria-developers] Developers Unconference in New York
by Ian Gilfillan 16 Feb '18

16 Feb '18

A final reminder about the Developers Unconference happening in New York on Sat 24 and Sun 25 February, just before M|18. https://mariadb.org/2018-mariadb-developers-unconference/ Please sign up on the Meetup page if you are going to be attending: https://www.meetup.com/MariaDB-Developers/events/244375856/ Hope to see you there!

1 0

[Maria-developers] Google Summer of Code 2018 - MariaDB Foundation accepted
by Colin Charles 14 Feb '18

14 Feb '18

Hi, I just noticed this piece of good news — MariaDB Foundation is accepted to participate in the Google Summer of Code 2018 again — https://summerofcode.withgoogle.com/ There is an already healthy list of ideas: https://mariadb.com/kb/en/library/google-summer-of-code-2018/ Many already have mentors from MariaDB Corporation or MariaDB Foundation. Some are looking for mentors. Happy coding this summer! cheers, -colin -- Colin Charles, http://bytebot.net/blog/ twitter: @bytebot | skype: colincharles "First they ignore you, then they laugh at you, then they fight you, then you win." -- Mohandas Gandhi

1 0

Re: [Maria-developers] [Commits] f4f32ab71d5: MDEV-9744: session optimizer_use_condition_selectivity=5 causing SQL Error (1918):
by Sergey Petrunia 13 Feb '18

13 Feb '18

Hi Varun, On Sat, Feb 03, 2018 at 06:37:21PM +0530, Varun wrote: > revision-id: f4f32ab71d5c96fbec87f8b6a5c057af2bc2398b (mariadb-10.2.5-413-gf4f32ab71d5) > parent(s): 9390ff53fc39e34976cf051ce33649f9316bb8e6 > author: Varun Gupta > committer: Varun Gupta > timestamp: 2018-02-03 18:24:49 +0530 > message: > > MDEV-9744: session optimizer_use_condition_selectivity=5 causing SQL Error (1918): > Encountered illegal value '' when converting to DECIMAL > > The issue was that EITS data was allocated but then not read for some reason (one being to avoid a deadlock), > then the optimizer was using these bzero'ed buffers as EITS statistics. > This should not be allowed, we should use statistcs for a table only when we have successfully loaded/read > the stats from the statistical tables. > > --- > mysql-test/r/statistics.result | 16 ++++++++++++++++ > mysql-test/t/statistics.test | 19 +++++++++++++++++++ > sql/sql_statistics.cc | 23 ++++++++++++++++------- > 3 files changed, 51 insertions(+), 7 deletions(-) > > diff --git a/mysql-test/r/statistics.result b/mysql-test/r/statistics.result > index ffaaf09acc8..1486a8ab5e6 100644 > --- a/mysql-test/r/statistics.result > +++ b/mysql-test/r/statistics.result > @@ -1716,5 +1716,21 @@ b > 0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.004,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.004,0.000,0.000,0.004,0.000,0.000,0.000,0.004,0.000,0.000,0.000,0.004,0.000,0.000,0.000,0.004,0.000,0.004,0.000,0.000,0.004,0.000,0.004,0.000,0.004,0.000,0.004,0.004,0.004,0.000,0.004,0.000,0.004,0.004,0.004,0.004,0.004,0.000,0.004,0.004,0.004,0.004,0.004,0.004,0.008,0.004,0.008,0.008,0.008,0.008,0.020,0.004,0.016,0.020,0.016,0.016,0.051,0.031,0.027,0.031,0.043,0.047,0.043,0.043,0.055,0.051,0.071,0.043,0.043,0.043,0.020,0.024,0.024,0.020,0.016,0.016,0.008,0.008,0.012,0.000 > DROP TABLE t1; > # > +# MDEB-9744: session optimizer_use_condition_selectivity=5 causing SQL Error (1918): Please change to "MDEV" > +# Encountered illegal value '' when converting to DECIMAL > +# > +set @save_optimizer_use_condition_selectivity= @@optimizer_use_condition_selectivity; > +set optimizer_use_condition_selectivity=3, use_stat_tables=preferably; > +create table a (id int(10),cost decimal(9,2)) engine=innodb; > +ANALYZE TABLE a PERSISTENT FOR ALL; > +Table Op Msg_type Msg_text > +test.a analyze status Engine-independent statistics collected > +test.a analyze status OK > +create temporary table b (id int); > +insert into b (id) select id from a where cost > 0; > +drop table a,b; Please follow the convention and use table names t1,t2,... Ok to push after this is addressed. BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] [Commits] MDEV-15167 Server crashes in in bitmap_bits_set upon REPAIR PARTITION ...
by Sergei Golubchik 10 Feb '18

10 Feb '18

Hi, Sachin! On Feb 09, Sachin Setiya wrote: > commit 3e49ebcacf9022931f0dfefcb44c13973128f8c4 > Author: Sachin Setiya <sachinsetia1001(a)gmail.com> > Date: Fri Feb 9 17:44:04 2018 +0530 > > MDEV-15167 Server crashes in in bitmap_bits_set upon REPAIR PARTITION > ... > > diff --git a/sql/sql_partition.cc b/sql/sql_partition.cc > index a4d9406..3d53056 100644 > --- a/sql/sql_partition.cc > +++ b/sql/sql_partition.cc > @@ -4583,6 +4583,11 @@ uint prep_alter_part_table(THD *thd, TABLE *table, > Alter_info *alter_info, > Alter_info::ALTER_TABLE_REORG | > Alter_info::ALTER_REBUILD_PARTITION)) > { > + /* > + You can't add column when we are doing alter related to partition > + */ > + DBUG_EXECUTE_IF("test_pseudo_invisible",{DBUG_ASSERT(0);}); > + DBUG_EXECUTE_IF("test_completely_invisible",{DBUG_ASSERT(0);}); May be return error? Like with P_S? DBUG_EXECUTE_IF("test_pseudo_invisible", {DBUG_RETURN(1)}); but test it, may be you'll need my_error too, like DBUG_EXECUTE_IF("test_pseudo_invisible", { my_error(ER_INTERNAL_ERROR, MYF(0), "Don't to it with test_pseudo_invisible"); DBUG_RETURN(1) }); Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] Oracle strings functions compatibility : substr / MDEV 10574
by jerome brauge 09 Feb '18

09 Feb '18

Hello Alexander, If you remember, there was still a problem with our function substr_oracle when length is less or equal to zero. Can you review this very small patch for this ? For MDEV-10574, If we overload only functions required for Oracle compatibility (ltrim/rtrim and lpad/rpad), could I make a similar patch ? (by adding functions ltrim_oracle, ... like for substr) It's for us the last blocking point. Regards, Jérôme. > -----Message d'origine----- > De : Alexander Barkov [mailto:bar@mariadb.org] > Envoyé : mardi 19 septembre 2017 13:16 > À : jerome brauge > Objet : Re: Oracle strings functions compatibility : substr > > Hi Jerome, > > I noticed one more difference in SUBSTR: > > > SELECT SUBSTR('aaa',1,-1) FROM DUAL; > > MariaDB returns an empty string. > Oracle returns NULL. > > Here's Oracle's documentation: > > https://docs.oracle.com/cd/B19306_01/server.102/b14200/functions162.htm > > > I think we should do this in the same patch, to avoid behavior change in the > future. > > Would you like to try doing this additional change? > Or would you like me to make an incremental patch on top of yours? > > Thanks! > > On 09/19/2017 10:33 AM, jerome brauge wrote: > > Hello Alexander, > > It's done. > > > > Do you have news from Sergei for MDEV-12874, MDEV-13417 and MDEV- > 13418 ? > > I asked Sergei, awaiting for his answer. I'll let you know. > > > > > Thank you very much. > > > > > > > >> -----Message d'origine----- > >> De : Alexander Barkov [mailto:bar@mariadb.org] Envoyé : lundi 18 > >> septembre 2017 18:12 À : jerome brauge Objet : Re: Oracle strings > >> functions compatibility : substr > >> > >> Hello Jerome, > >> > >> > >> On 09/18/2017 05:14 PM, jerome brauge wrote: > >>> Hello Alexander, > >>> I don't understand how I've missed such an obvious solution ! > >>> Here is the new patch. > >> > >> Thanks. Now it looks simpler. > >> > >> Can you please do one small thing: > >> move get_position() from "public:" to "protected:". > >> > >> Ok to make a pull request after this change. > >> > >> Thank you very much! > >> > >> > >>> > >>> Regards. > >>> Jérôme. > >>> > >>> > >>>> -----Message d'origine----- > >>>> De : Alexander Barkov [mailto:bar@mariadb.org] Envoyé : lundi 18 > >>>> septembre 2017 11:30 À : jerome brauge Objet : Re: Oracle strings > >>>> functions compatibility : substr > >>>> > >>>> Hello Jerome, > >>>> > >>>> > >>>> On 08/17/2017 04:43 PM, jerome brauge wrote: > >>>>> Hello Alexander, > >>>>> Here is a patch for function SUBSTR in sql_mode=oracle (If > >>>>> position is 0, > >>>> then it is treated as 1). > >>>> > >>>> I'm thinking of how to get a smaller patch. > >>>> > >>>> Wouldn't it be possible to introduce: > >>>> > >>>> virtual longlong get_position(); > >>>> > >>>> > >>>> > >>>> longlong Item_func_substr::get_position() { > >>>> return args[1]->val_int(); > >>>> } > >>>> > >>>> longlong Item_func_substr_oracle::get_position() > >>>> { > >>>> longlong pos= args[1]->val_int(); > >>>> return pos == 0 ? 1 : pos; > >>>> } > >>>> > >>>> > >>>> and just replace all calls for args[1]->val_int() to get_position() ? > >>>> > >>>> Thanks! > >>>> > >>>> > >>>>> > >>>>> Regards, > >>>>> Jérôme. > >>>>>

1 1

Re: [Maria-developers] 9bc03294c89: MDEV-15033: Node fails to join mdb cluster with wsrep_sst_mariabackup due to dependency from socat
by Sergei Golubchik 06 Feb '18

06 Feb '18

Hi, Jan! No, sorry, you cannot do it. First, we cannot add new RPM dependencies in GA (we tried once and had to revert that change). Second, there was already socat dependency in RPM, added by Nirbhay in MDEV-6211 MariaDB-Galera-server uses 'socat', but 'socat' is not in the dependency list commit bd5ca5e6093046d7e69cc59f0950a7af2c0a7bf2 removed by Nirbhay in MDEV-7233: Fix issue with missing dependency socat when installing MariaDB-galera-server on RedhatEL/OracleEL/(Others?) RPM based commit 41367e4023d0823624a461597813e6902437f096 I don't see a good solution for this issue. Adding socat to mariabackup package aren't much better, as mariabackup users don't really need socat either. May be just a better error message in the script? On Feb 06, jan wrote: > revision-id: 9bc03294c89b16df5f556b7d9fa4cb1eadae451a (mariadb-10.1.30-31-g9bc03294c89) > parent(s): 1cd1db9602476ec6864f4a521de97d788fe93fac > author: Jan Lindström > committer: Jan Lindström > timestamp: 2018-02-06 10:59:27 +0200 > message: > > MDEV-15033: Node fails to join mdb cluster with wsrep_sst_mariabackup due to dependency from socat > > Add socat dependency to cpack_rpm.cmake. Similar dependency already > is on debian/control. > > diff --git a/cmake/cpack_rpm.cmake b/cmake/cpack_rpm.cmake > index e80fb199f03..5b9c6726b5e 100644 > --- a/cmake/cpack_rpm.cmake > +++ b/cmake/cpack_rpm.cmake > @@ -170,7 +170,7 @@ SETA(CPACK_RPM_server_PACKAGE_REQUIRES > IF(WITH_WSREP) > SETA(CPACK_RPM_server_PACKAGE_REQUIRES > "galera" "rsync" "lsof" "grep" "gawk" "iproute" > - "coreutils" "findutils" "tar") > + "coreutils" "findutils" "tar" "socat") > IF (RPM MATCHES "sles11") > SETA(CPACK_RPM_server_PACKAGE_REQUIRES "util-linux") > ELSE() Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

2 2

[Maria-developers] Triangular FKs - Cascade delete leaves DB in state with broken referential integrity
by Oliver Schonrock 05 Feb '18

05 Feb '18

Initially posted on "discuss" list... I have come across a very strange symptom with triangular FKs (see below) where a cascade delete of the central (member) record leaves the DB in a state with broken referential integrity. The FK structure is a little bit demanding and somewhat "triangular/circular", but has been in operation in our application for many years without problems. After recent upgrade from 10.1 to 10.2.12, we promptly got broken FKs. (I have not yet retested with 10.1). How to reproduce: (example reduced to the bare essentials with 3 tables with 1 record each). SET FOREIGN_KEY_CHECKS=0; CREATE TABLE member ( id int(11) NOT NULL AUTO_INCREMENT, default_address_id int(11) DEFAULT NULL, default_card_id int(11) DEFAULT NULL, PRIMARY KEY (id), KEY member_FI_2 (default_address_id), KEY member_FI_3 (default_card_id), CONSTRAINT member_FK_2 FOREIGN KEY (default_address_id) REFERENCES address (id) ON DELETE SET NULL ON UPDATE CASCADE, CONSTRAINT member_FK_3 FOREIGN KEY (default_card_id) REFERENCES payment_method (id) ON DELETE SET NULL ON UPDATE CASCADE ) ENGINE=InnoDB; INSERT INTO member VALUES (1,2,3); CREATE TABLE address ( id int(11) NOT NULL AUTO_INCREMENT, member_id int(11) NOT NULL, PRIMARY KEY (id), KEY address_FI_1 (member_id), CONSTRAINT address_FK_1 FOREIGN KEY (member_id) REFERENCES member (id) ON DELETE CASCADE ON UPDATE CASCADE ) ENGINE=InnoDB; INSERT INTO address VALUES (2,1); CREATE TABLE payment_method ( id int(11) NOT NULL AUTO_INCREMENT, member_id int(11) NOT NULL, cardholder_address_id int(11) DEFAULT NULL, PRIMARY KEY (id), KEY payment_method_FI_1 (member_id), KEY payment_method_FI_2 (cardholder_address_id), CONSTRAINT payment_method_FK_1 FOREIGN KEY (member_id) REFERENCES member (id) ON DELETE CASCADE ON UPDATE CASCADE, CONSTRAINT payment_method_FK_2 FOREIGN KEY (cardholder_address_id) REFERENCES address (id) ON DELETE SET NULL ON UPDATE CASCADE ) ENGINE=InnoDB; INSERT INTO payment_method VALUES (3,1,2); SET FOREIGN_KEY_CHECKS=1; SELECT * FROM member; +----+--------------------+-----------------+ | id | default_address_id | default_card_id | +----+--------------------+-----------------+ | 1 | 2 | 3 | +----+--------------------+-----------------+ SELECT * FROM address; +----+-----------+ | id | member_id | +----+-----------+ | 2 | 1 | +----+-----------+ SELECT * FROM payment_method; +----+-----------+-----------------------+ | id | member_id | cardholder_address_id | +----+-----------+-----------------------+ | 3 | 1 | 2 | +----+-----------+-----------------------+ DELETE FROM member WHERE id = 1; SELECT * FROM member; [ empty ] SELECT * FROM address; [ empty ] SELECT * FROM payment_method; +----+-----------+-----------------------+ | id | member_id | cardholder_address_id | +----+-----------+-----------------------+ | 3 | 1 | NULL | +----+-----------+-----------------------+ ------------ DB is now in an inconsistent state: payment_method.member_id=1 references a non-existent member record. Am I missing something, or should this never happen? Thanks -- Oliver Schönrock

1 6

[Maria-developers] Partial indexes?
by Jared Beck 04 Feb '18

04 Feb '18

Dear Maria, :p Thanks for all your hard work! Any plans to implement partial indexes? (https://www.postgresql.org/docs/10/static/indexes-partial.html) I hope this is the right place to suggest new features. Please redirect me if not. -- Jared Beck • (607) 216-5373 • jared(a)jaredbeck.com

3 7

[Maria-developers] MDEV-14425: Is appending never slower than writing to a preallocated file?
by Marko Mäkelä 02 Feb '18

02 Feb '18

I would like to redesign the InnoDB redo log format for better performance in MariaDB 10.4. Part of this would involve minimizing write amplification and optimizing for journaled file systems. Jun Su from Microsoft suggested to me that regular writes (as opposed to appends) could cause some write amplification inside journaled file systems. InnoDB traditionally pre-allocates both data and log files. Maybe it was a good idea in 1994 when the code was initially conceived. But we have had journaled or copy-on-write file systems and also SSDs for quite some time now. I wrote two test programs that write a 2GiB file in 2KiB blocks, either pre-allocating the file upfront, or appending to the file. On the two SSDs that I tested (with ext4fs), appending was always faster. The programs are attached to https://jira.mariadb.org/browse/MDEV-14425 I would appreciate it if someone can provide a counterexample where writing to a preallocated file would be faster than appending, on a modern file system. Also, I would like to see how HDDs would perform. With best regards, Marko -- Marko Mäkelä, Lead Developer InnoDB MariaDB Corporation DON’T MISS M|18 MariaDB User Conference February 26 - 27, 2018 New York City https://m18.mariadb.com/

2 1

Re: [Maria-developers] [Commits] a0ea3a58224: MDEV-14857: problem with 10.2.11 server crashing when executing stored procedure
by Sergey Petrunia 31 Jan '18

31 Jan '18

Hi Sanja, Please find my review below. Lots of typos, but the two important comments are at the bottom, for the changes in sql_select.cc and sql_trigger.cc On Sat, Jan 27, 2018 at 11:39:10AM +0100, Oleksandr Byelkin wrote: > revision-id: a0ea3a5822486fa1702d6615f1be30684c3d412f (mariadb-10.1.30-26-ga0ea3a58224) > parent(s): 524221e7a34d42214e82dd868348b453f2ef1591 > author: Oleksandr Byelkin > committer: Oleksandr Byelkin > timestamp: 2018-01-27 11:38:38 +0100 > message: > > MDEV-14857: problem with 10.2.11 server crashing when executing stored procedure > > Counter for select numbering made stored with the statement (before was global) > So now it does have always accurate value which does not depend on > interruption of statement prepare by errors like lack of table in > a view definition. > > --- > diff --git a/sql/sql_class.h b/sql/sql_class.h > index 4556487bdfe..2dc2fb61e9d 100644 > --- a/sql/sql_class.h > +++ b/sql/sql_class.h > @@ -1027,6 +1027,21 @@ class Statement: public ilink, public Query_arena > > LEX_STRING name; /* name for named prepared statements */ > LEX *lex; // parse tree descriptor > + /* > + LEX whith represent curent statement (conventional, SP or PS) s/whith/which/ s/curent/current/ s/represent/represents/ > + > + For example during view parsing THD::lex will point to the views LEX and > + THD::stmt_lex will point on LEX of the statement where the view will be s/point on/point to/ > + included > + > + Now used to have alwais correct select numbering inside statemet s/Now used/Currently it is used/ s/alwais/always/ s/statemet/statement/ > + (LEX::current_select_number) without storing and restoring a global > + counter which was THD::select_number. > + > + TODO: make some unified statement representation (now SP has different) > + to store such data like LEX::current_select_number. > + */ > + LEX *stmt_lex; > /* > Points to the query associated with this statement. It's const, but > we need to declare it char * because all table handlers are written > diff --git a/sql/sql_lex.h b/sql/sql_lex.h > index b57fba08b47..a2da6f672ca 100644 > --- a/sql/sql_lex.h > +++ b/sql/sql_lex.h > @@ -727,7 +727,13 @@ class st_select_lex: public st_select_lex_node > Item *prep_having;/* saved HAVING clause for prepared statement processing */ > /* Saved values of the WHERE and HAVING clauses*/ > Item::cond_result cond_value, having_value; > - /* point on lex in which it was created, used in view subquery detection */ > + /* > + Point on lex in which it was created, used in view subquery detection. "Points to the LEX...". Also please shift the whole comment 1 position to the left. > + > + TODO: make also st_select_lex::parent_stmt_lex (see THD::stmt_lex) > + and use st_select_lex::parent_lex & st_select_lex::parent_stmt_lex > + instead of global (from THD) references where it is possible. > + */ > LEX *parent_lex; > enum olap_type olap; > /* FROM clause - points to the beginning of the TABLE_LIST::next_local list. */ > @@ -2452,7 +2458,7 @@ struct LEX: public Query_tables_list > /** SELECT of CREATE VIEW statement */ > LEX_STRING create_view_select; > > - uint number_of_selects; // valid only for view > + uint current_select_number; // valid for statmet LEX (not view) s/statmet/statement/ > > /** Start of 'ON table', in trigger statements. */ > const char* raw_trg_on_table_name_begin; > diff --git a/sql/sql_parse.cc b/sql/sql_parse.cc > index 6084c59a257..8d3116cd8b3 100644 > --- a/sql/sql_parse.cc > +++ b/sql/sql_parse.cc > @@ -6903,7 +6903,12 @@ void THD::reset_for_next_command(bool do_clear_error) > clear_error(1); > > thd->free_list= 0; > - thd->select_number= 1; > + /* > + The same thd->stmt_lex made in lex_start, but during bootstrup this "We also assign thd->stmt_lex in lex_start()" s/bootstrup/bootstrap > + code executed first code is executed > + */ > + thd->stmt_lex= &main_lex; thd->stmt_lex->current_select_number= 1; > + DBUG_PRINT("info", ("Lex %p stmt_lex: %p", thd->lex, thd->stmt_lex)); > /* > Those two lines below are theoretically unneeded as > THD::cleanup_after_query() should take care of this already. > diff --git a/sql/sql_select.cc b/sql/sql_select.cc > index b9fe8f3162a..7a6a028ee9c 100644 > --- a/sql/sql_select.cc > +++ b/sql/sql_select.cc > @@ -2462,6 +2462,17 @@ void JOIN::save_explain_data(Explain_query *output, bool can_overwrite, > bool need_tmp_table, bool need_order, > bool distinct) > { > + /* > + If there is SELECT in this statemet with the same number it must be the > + same SELECT > + */ > + DBUG_ASSERT(select_lex->select_number == UINT_MAX || > + select_lex->select_number == INT_MAX || > + !output || Is it really possible that output==NULL? If yes, that's probably a bug. > + !output->get_select(select_lex->select_number) || > + output->get_select(select_lex->select_number)->select_lex == > + select_lex); > + > if (select_lex->select_number != UINT_MAX && > select_lex->select_number != INT_MAX /* this is not a UNION's "fake select */ && > have_query_plan != JOIN::QEP_NOT_PRESENT_YET && > @@ -24601,6 +24612,11 @@ int JOIN::save_explain_data_intern(Explain_query *output, bool need_tmp_table, > { > explain= new (output->mem_root) Explain_select(output->mem_root, > thd->lex->analyze_stmt); > + if (!explain) > + DBUG_RETURN(1); // EoM > +#ifndef DBUG_OFF > + explain->select_lex= select_lex; > +#endif > join->select_lex->set_explain_type(true); > > explain->select_id= join->select_lex->select_number; > diff --git a/sql/sql_trigger.cc b/sql/sql_trigger.cc > index 0b4978b2862..878cd5a3b57 100644 > --- a/sql/sql_trigger.cc > +++ b/sql/sql_trigger.cc > @@ -1378,7 +1378,7 @@ bool Table_triggers_list::check_n_load(THD *thd, const char *db, > ulonglong save_sql_mode= thd->variables.sql_mode; > LEX_STRING *on_table_name; > > - thd->lex= &lex; > + thd->lex= thd->stmt_lex= &lex; > The same comment as before: we change thd->stmt_lex here. The function has code below that restores the value of thd->lex: // QQ: anything else ? lex_end(&lex); thd->lex= old_lex; But the value of thd->stmt_lex is not restored? > save_db.str= thd->db; > save_db.length= thd->db_length; BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] Wording suggestion?
by Sergei Golubchik 30 Jan '18

30 Jan '18

Hi, Russell! On Jan 30, Russell J.T. Dyer wrote: > How about concurrent_assign ? I think "concurrent" already has a clear meaning, at least in MariaDB - operations are done "concurrently" if they're executed truly in parallel, usually in different threads. For example, both MyISAM and InnoDB allow concurrent SELECTs, but only InnoDB allows concurrent UPDATEs. I considered simultaneous_assign, as a slightly shorter alternative to simultaneous_assignment. But it looked somewhat unnatural to me. Still, this is the case when I happily yield to the opinion of a documentation writer :) If you think that simultaneous_assign looks just fine, I'll use it. > > It's not only for update. There is a patch to fix SELECT ... INTO, > > and I suppose we'll have "simultaneous assignment" behavior > > everywhere eventually. > > > > So, I'm leaning towards either simultaneous_assignment or > > simultaneous_set as mode names. The first is more exact, but it's > > rather long (sql_mode names range from 3 to 26 characters, and this > > one is 23). The second (with 16) fits right in the middle of the > > range, length-wise. But there's no SET keyword in SELECT ... INTO. > > On the other hand, there's no ASSIGNMENT keyword anywhere either :) > > Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] Mdev-15085 Invisible Column Non-constant Default value results...
by Sergei Golubchik 30 Jan '18

30 Jan '18

Hi, Sachin! ok to push! On Jan 30, Sachin Setiya wrote: > commit a2e6d3a2fe6842bd3c35dffb1cd6f6d8a3117107 > Author: Sachin Setiya <sachin.setiya(a)mariadb.com> > Date: Tue Jan 30 14:04:05 2018 +0530 > > Mdev-15085 Invisible Column Non-constant Default value results... > > Problem:- If we create table field with dynamic default value then that > field always gets NULL value. > > Analyze:- This is because in fill_record we simple continue at Invisible > column because we though that share->default_values(default value is > always copied into table->record[0] before insert) will have a default > value for them(which is true for constant defaults , but not for dynamic > defaults). > > Solution:- We simple set all_fields_have_value to null , and this will > make call to update_default_fields (in the case of dynamic default), And > default expr will be evaluted and value will be set in field. Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] Wording suggestion?
by Sergei Golubchik 30 Jan '18

30 Jan '18

Hi, Russell! On Jan 29, Russell J.T. Dyer wrote: > Is this mode related only to UPDATE and this feature? If so, maybe it > could be something like one of these: > > update_static > update_stasis > update_isolation > > These would focus on the fact that it's related just to UPDATE and that the > values held in a state temporarily, or in isolation of each other. Sorry, I should've said earlier. It's not only for update. There is a patch to fix SELECT ... INTO, and I suppose we'll have "simultaneous assignment" behavior everywhere eventually. So, I'm leaning towards either simultaneous_assignment or simultaneous_set as mode names. The first is more exact, but it's rather long (sql_mode names range from 3 to 26 characters, and this one is 23). The second (with 16) fits right in the middle of the range, length-wise. But there's no SET keyword in SELECT ... INTO. On the other hand, there's no ASSIGNMENT keyword anywhere either :) Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

2 2

[Maria-developers] Wording suggestion?
by Sergei Golubchik 30 Jan '18

30 Jan '18

Hi, Ian, Russell, As you know, UPDATE in MariaDB is executed left-to-right, say, in the case of UPDATE t1 SET a=b, b=a both columns will have the same value, old value of `b`. We're now introducing a new sql_mode to have standard compatible behavior, where all assignments are executed kind of "simultaneously" (and the statement above would swap the values). What could the name for this new sql_mode be? Regards, Sergei

2 2

Re: [Maria-developers] [Commits] Mdev-15085 Rebased to latest 10.3
by Sergei Golubchik 30 Jan '18

30 Jan '18

Hi, Sachin! On Jan 29, Sachin Setiya wrote: > commit 905806689979ae91c89115f18b107ebacbc489e2 > Author: Sachin Setiya <sachin.setiya(a)maridb.com> > Date: Fri Jan 26 22:58:34 2018 +0530 > > Mdev-15085 Invisible Column Non-constant Default value results... > > Problem:- If we create table field with dynamic default value then that > field always gets NULL value. > > Analyze:- This is because in fill_record we simple continue at Invisible > column because we though that share->default_values(default value is > always copied into table->record[0] before insert) will have a default > value for them(which is true for constant defaults , but not for dynamic > defaults). > > Solution:- In case of invisible field we will assign default value to 'value' > variable, but we do this only for columns which have non dynamic default value. > We do not have to worry for Auto_increment field , storage engine will take > of it. > > diff --git a/sql/sql_base.cc b/sql/sql_base.cc > index 45efa63..17ff7eb 100644 > --- a/sql/sql_base.cc > +++ b/sql/sql_base.cc > @@ -8382,7 +8382,12 @@ fill_record(THD *thd, TABLE *table, Field **ptr, > List<Item> &values, > > if (field->invisible) > { > - continue; > + if(field->flags & AUTO_INCREMENT_FLAG || > + !field->default_value) > + continue; > + else > + value= new (thd->mem_root) Item_default_value(thd, > + &thd->lex->select_lex.context); No-no, you cannot create items in fill_record(). It would create a new item for every inserted row. I think the only thing you need to do here (and it doesn't depend on AUTO_INCREMENT_FLAG or field->default_value), you need to set all_fields_have_values to false. Like if (field->invisible) { all_fields_have_values= false; continue; } Indeed, hidden columns did not get a value, so all_fields_have_values should be false. I'd expect the rest will work automatically :) > } > else > value=v++; Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

2 1

Re: [Maria-developers] MDEV-14849 CREATE + ALTER with user-invisible columns produce ...
by Sergei Golubchik 29 Jan '18

29 Jan '18

Hi, Sachin! On Jan 29, Sachin Setiya wrote: > commit 5bd97f22094f3b6a658d503f98022ccd9076bb19 > Author: Sachin Setiya <sachin.setiya(a)maridb.com> > Date: Mon Jan 29 12:31:07 2018 +0530 > > MDEV-14849 CREATE + ALTER with user-invisible columns produce ... > > Problem:- > create or replace table t1 (pk int auto_increment primary key invisible, i int); > alter table t1 modify pk int invisible; > This last alter makes a invisible column which is not null and does not > have default value. > > Analysis:- > This is caused because our error check for NOT_NULL_FLAG and > NO_DEFAULT_VALUE_FLAG flag misses this sql_field , but this is not the fault > of error check :).Actually this field come via mysql_prepare_alter_table > and it does not have NO_DEFAULT_VALUE_FLAG flag turned on. (If it was create > table NO_DEFAULT_VALUE_FLAG would have turned on Column_definition::check) > and this would have generated error. > > Solution:- > I have moved the error check to kind last of mysql_prepare_create_table > because upto this point we have applied NO_DEFAULT_VALUE_FLAG to required > column. ok to push! Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] MDEV-14785 SYSTEM_INVISIBLE behaviour not consistent ---Initial Patch
by Sergei Golubchik 29 Jan '18

29 Jan '18

Hi, Sachin! On Jan 29, Sachin Setiya wrote: > commit 5e2f5aa78a5f53b000a2b9c6989c9ed7076e2527 > Author: Sachin Setiya <sachin.setiya(a)maridb.com> Note the typo in your email ^^^^^ > Date: Mon Jan 29 17:09:32 2018 +0530 > > MDEV-14785 SYSTEM_INVISIBLE behaviour not consistent > > TODO Commit message > > diff --git a/sql/item.h b/sql/item.h > index d178d9f..0072dfd 100644 > --- a/sql/item.h > +++ b/sql/item.h > @@ -2907,11 +2907,17 @@ class Item_field :public Item_ident > bool check_vcol_func_processor(void *arg) > { > context= 0; > - if (field && (field->unireg_check == Field::NEXT_NUMBER)) > + if (field) > { > - // Auto increment fields are unsupported > - return mark_unsupported_function(field_name.str, arg, VCOL_FIELD_REF | VCOL_AUTO_INC); > + if (field->unireg_check == Field::NEXT_NUMBER) > + // Auto increment fields are unsupported > + return mark_unsupported_function(field_name.str, arg, > + VCOL_FIELD_REF | VCOL_AUTO_INC); > + if (field->invisible > INVISIBLE_USER) > + return mark_unsupported_function(field_name.str, arg, > + VCOL_FIELD_REF | VCOL_SYSTEM_INVISIBLE); Why would you do that, instead of hiding system invisible fields in CREATE TABLE? From the user point of view they aren't fields, they don't exist in the table, and the error should be "no such field", not "Function or expression 'on SYSTEM_INVISIBLE column' cannot be used" > } > + > return mark_unsupported_function(field_name.str, arg, VCOL_FIELD_REF); > } > bool set_fields_as_dependent_processor(void *arg) > diff --git a/sql/sql_base.cc b/sql/sql_base.cc > index 17ff7eb..89ba2ed 100644 > --- a/sql/sql_base.cc > +++ b/sql/sql_base.cc > @@ -8128,6 +8128,11 @@ fill_record(THD *thd, TABLE *table_arg, List<Item> &fields, List<Item> &values, > value=v++; > DBUG_ASSERT(value); > Field *rfield= field->field; > + if (rfield->invisible == INVISIBLE_SYSTEM) > + { > + my_error(ER_BAD_FIELD_ERROR, MYF(0), rfield->field_name.str, thd->where); The error is correct. But do you need to do it in fill_record()? I'd thought you can do it when the statement is prepared, once, not in fill_record() which is called for every row, many times per statement. > + goto err; > + } > TABLE* table= rfield->table; > if (table->next_number_field && > rfield->field_index == table->next_number_field->field_index) > diff --git a/sql/sql_select.cc b/sql/sql_select.cc > index d2fd2ce..e4971d7 100644 > --- a/sql/sql_select.cc > +++ b/sql/sql_select.cc > @@ -16639,6 +16639,7 @@ Field *create_tmp_field_from_field(THD *thd, Field *org_field, > new_field->next_equal_field= NULL; > new_field->option_list= NULL; > new_field->option_struct= NULL; > + new_field->invisible= org_field->invisible; What is it for? It'd mean that temporary tables can have invisible fields. What bugs does this change fix? > } > return new_field; > } > diff --git a/sql/sql_update.cc b/sql/sql_update.cc > index 7b4938f..7bdedc7 100644 > --- a/sql/sql_update.cc > +++ b/sql/sql_update.cc > @@ -1497,6 +1497,9 @@ int mysql_multi_update_prepare(THD *thd) > TABLE_LIST *table_list= lex->query_tables; > TABLE_LIST *tl; > List<Item> *fields= &lex->select_lex.item_list; > + List_iterator<Item> it(*fields); > + Item *item; > + Item_field *field; > table_map tables_for_update; > bool update_view= 0; > /* > @@ -1564,7 +1567,21 @@ int mysql_multi_update_prepare(THD *thd) > { > DBUG_RETURN(TRUE); > } > - > + //Check if we got any INVISIBLE_SYSTEM field > + while ((item= it++)) > + { > + if (Item::FIELD_ITEM == item->type()) > + { > + field= (Item_field *)item; > + if (field->field->invisible == INVISIBLE_SYSTEM) > + { > + my_error(ER_BAD_FIELD_ERROR,MYF(0), > + field->field->field_name.str, > + field->field->table->s->table_name.str); > + DBUG_RETURN(TRUE); > + } > + } > + } Dunno. What if I do UPDATE ... SET normal_field = row_start you'll have an invisible field, but it's not updated, so should be ok, right? May be something like if (field->field->invisible == INVISIBLE_SYSTEM && bitmap_is_set(field->table->write_set, field->field_index)) { ... my_error } And why here? I'd think this belongs into check_fields() function. > thd->table_map_for_update= tables_for_update= get_table_map(fields); Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] MDEV-13790 UNHEX() of a somewhat complicated CONCAT() returns NULL
by Alexander Barkov 29 Jan '18

29 Jan '18

Hello Sergei, Please review a patch for MDEV-13790. It removes the attempt to search the best suitable buffer and just returns the result in "str" passed to val_str(). I'm quite sure this optimization gives nothing. In can be useful only under very rare conditions, but generates so many bugs. Moreover, in 10.3 (or 10.4) we'll work on reentrant items. So all Item_xxx::tmp_buffer will be gone anyway, together with this optimization. I think an optimization proposal. Why not to introduce a new method: class Item { public: .. virtual append_to_string(String *str); .. } So for example Field_varchar could write directly to "str" passed to Item_func_concat::val_str(). This will make sure that the first allocation is done on the target "str", and "str" will always be the best buffer instead of any Item_xxx::tmp_value. Thanks!

2 3

Re: [Maria-developers] 980a5c31880: MDEV-14572: Assertion `! is_set()' failed in Diagnostics_area::set_eof_status upon EXPLAIN UPDATE in PS
by Sergei Golubchik 28 Jan '18

28 Jan '18

Hi, Oleksandr! On Dec 22, Oleksandr Byelkin wrote: > revision-id: 980a5c318803d6aa7355e019869597976dcf19bf (mariadb-10.0.33-37-g980a5c31880) > parent(s): 042f763268c0f209e7c12e0a6a72bb5d204dfe29 > author: Oleksandr Byelkin > committer: Oleksandr Byelkin > timestamp: 2017-12-22 14:00:10 +0100 > message: > > MDEV-14572: Assertion `! is_set()' failed in Diagnostics_area::set_eof_status upon EXPLAIN UPDATE in PS > > Restore EXPAIN flag in SELECT_LEX before execution multi-update by flag in LEX > (the same but in other way made before INSERT/DELETE/SELECT) Please explain what was the reason for the bug, why the status was set twice, and how your change fixes it. Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] [Commits] ee8755e3c51: MDEV-15012: ASAN: numerous test failures in PS
by Sergei Golubchik 26 Jan '18

26 Jan '18

Hi, Oleksandr! Nice one! Ok to push! On Jan 24, Oleksandr Byelkin wrote: > revision-id: ee8755e3c51a1da8fcf108ad0257a7e62fc94347 (mariadb-5.5.59-32-gee8755e3c51) > parent(s): 76577e1e2602f3c30859a176808c433a263e1b0a > author: Oleksandr Byelkin > committer: Oleksandr Byelkin > timestamp: 2018-01-24 14:42:52 +0100 > message: > > MDEV-15012: ASAN: numerous test failures in PS > > First roll back changes, then free Items which can lead to memory freeing. > Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] Patch for MDEV-13417 - UPDATE produces wrong values if an updated column is later used as an update source
by jerome brauge 24 Jan '18

24 Jan '18

Hello Sergei, Here is a new version of the patch following your comments. I let you decide if SET_CONSISTENCY must also be part of TRADITIONAL and ANSI. Regards, Jérôme. > -----Message d'origine----- > De : Sergei Golubchik [mailto:serg@mariadb.org] > Envoyé : jeudi 11 janvier 2018 23:17 > À : jerome brauge > Cc : bar(a)mariadb.org > Objet : Re: Patch for MDEV-13417 - UPDATE produces wrong values if an > updated column is later used as an update source > > Hi, Jérôme! > > Happy New Year! > > Sorry for the delay. > > But I've finally got around to reviewing your MDEV-13417 > > The patch was pretty good and with good tests too! > > I had only a few minor comments, see below. > If you'd like, I can apply your patch and do these changes myself. > > See the review below. > > On Dec 28, jerome brauge wrote: > > Hello Sergei, > > Do you have time to review my patch for this mdev ? > > This is very important for us because I don't see any workaround > > (without unique key and cursor). > > > diff --git a/sql/sql_class.h b/sql/sql_class.h index f30b442..a5e2104 > > 100644 > > --- a/sql/sql_class.h > > +++ b/sql/sql_class.h > > @@ -139,6 +139,7 @@ enum enum_binlog_row_image { > > #define MODE_HIGH_NOT_PRECEDENCE (1ULL << 29) > > #define MODE_NO_ENGINE_SUBSTITUTION (1ULL << 30) > > #define MODE_PAD_CHAR_TO_FULL_LENGTH (1ULL << 31) > > +#define MODE_SET_CONSISTENCY (1ULL << 32) > > I don't particularly like "SET_CONSISTENCY" name, but cannot come up with a > good alternative. It doesn't matter, renaming the mode is a trivial change > that doesn't affect anything else. > So, unless you have a strong preference for some specific mode name - just > ignore this naming issue completely. > And I'll ask our documentation guys if they could suggest something. > > > > > /* Bits for different old style modes */ > > #define OLD_MODE_NO_DUP_KEY_WARNINGS_WITH_IGNORE (1 << 0) > > diff --git a/sql/sql_base.cc b/sql/sql_base.cc index 60247c8..debb81a > > 100644 > > --- a/sql/sql_base.cc > > +++ b/sql/sql_base.cc > > @@ -7952,15 +7954,41 @@ fill_record(THD *thd, TABLE *table_arg, > List<Item> &fields, List<Item> &values, > > ER_THD(thd, > ER_WARNING_NON_DEFAULT_VALUE_FOR_VIRTUAL_COLUMN), > > rfield->field_name.str, table->s->table_name.str); > > } > > - if (rfield->stored_in_db() && > > - (value->save_in_field(rfield, 0)) < 0 && !ignore_errors) > > + if (rfield->stored_in_db()) > > { > > - my_message(ER_UNKNOWN_ERROR, ER_THD(thd, > ER_UNKNOWN_ERROR), MYF(0)); > > - goto err; > > + if (value->save_in_field(rfield, 0) < 0 && !ignore_errors) > > + { > > + my_message(ER_UNKNOWN_ERROR, ER_THD(thd, > ER_UNKNOWN_ERROR), MYF(0)); > > + goto err; > > + } > > + /* > > + ** In sql MODE_SET_CONSISTENCY, > > + ** move field pointer on value stored in record[1] > > + ** which contains row before update (see MDEV-13417) > > + */ > > + if (update && thd->variables.sql_mode & MODE_SET_CONSISTENCY) > > + rfield->move_field_offset((my_ptrdiff_t) (table->record[1] - > > + table->record[0])); > > Cool! A simple and nice solution :) > > > } > > rfield->set_explicit_default(value); > > } > > > > + if (update && thd->variables.sql_mode & MODE_SET_CONSISTENCY) { > > + // restore fields pointers on record[0] > > + f.rewind(); > > + while ((fld= f++)) > > + { > > + rfield= fld->field_for_view_update()->field; > > + if (rfield->stored_in_db()) > > + { > > + table= rfield->table; > > + rfield->move_field_offset((my_ptrdiff_t) (table->record[0] - > > + table->record[1])); > > + } > > + } > > + } > > + > > if (!update && table_arg->default_field && > > table_arg->update_default_fields(0, ignore_errors)) > > goto err; > > diff --git a/sql/sql_update.cc b/sql/sql_update.cc index > > 568dd40..5796b96 100644 > > --- a/sql/sql_update.cc > > +++ b/sql/sql_update.cc > > @@ -148,7 +148,25 @@ bool compare_record(const TABLE *table) > > we make temporary copy of Item_field, to avoid influence of changing > > result_field on Item_ref which refer on this field > > */ > > thd->change_item_tree(it.ref(), new (thd->mem_root) Item_field(thd, > field)); > > } > > + } > > + > > + if (thd->variables.sql_mode & MODE_SET_CONSISTENCY) { > > + // Make sure that a column is updated only once > > + List_iterator_fast<Item> it(items); > > + List<Item> ul; > > + ul.empty(); > > + while ((item= it++) && !ul.add_unique(item, &cmp_items)) ; > > Dunno, it's O(N²) and item->eq() is not trivial either. > perhaps it'd be faster, say, mark fields some way, like > > while ((item= it++)) > { > Field *f= item->field_for_view_update()->field; > if (f->has_explicit_value()) > { > my_error(ER_UPDATED_COLUMN_ONLY_ONCE, MYF(0), > f->table_name, f->field_name.str); > return TRUE; > } > f->set_has_explicit_value(); > } > > note that this assumes the change in the error message, %s -> %`s.%`s > > > + > > + if (item) > > + { > > + // same field appear more than once in a single update statement. > > + my_error(ER_UPDATED_COLUMN_ONLY_ONCE, MYF(0), item- > >full_name()); > > + return TRUE; > > + } > > + } > > return FALSE; > > } > > diff --git a/mysql-test/r/set_consistency.result > > b/mysql-test/r/set_consistency.result > > new file mode 100644 > > index 0000000..747b6b3 > > --- /dev/null > > +++ b/mysql-test/r/set_consistency.result > > @@ -0,0 +1,185 @@ > > +SET > > > +sql_mode='STRICT_TRANS_TABLES,ERROR_FOR_DIVISION_BY_ZERO,NO_A > UTO_CREA > > +TE_USER,NO_ENGINE_SUBSTITUTION,SET_CONSISTENCY'; > > shouldn't SET_CONSISTENCY be part of TRADITIONAL, ORACLE, and ANSI? > > > +# > > +# MDEV-13417 UPDATE produces wrong values if an UPDATEd column is > > +later used as an UPDATE source # MDEV-13418 Compatibility: The order > > +of evaluation of SELECT..INTO assignments # CREATE TABLE t1 (c1 > > +INTEGER, c2 INTEGER, c3 INTEGER) ENGINE=InnoDb; INSERT INTO > > +t1(c1,c2,c3) VALUES (1,1,1); CREATE TABLE t2 (c1 INTEGER, c2 > > +INTEGER, c3 INTEGER) ENGINE=InnoDb; INSERT INTO t2(c1,c2,c3) VALUES > > +(1,1,1); > > + > > +Check that a column is only updated once. > > + > > please, comment out these messages, just like the MDEV numbers above > > > +UPDATE t1 > > +SET c1 = 1, > > +c1 = 2; > > +ERROR HY000: The column test.t1.c1 cannot be changed more than once > > +in a single UPDATE statement > > Regards, > Sergei > Chief Architect MariaDB > and security(a)mariadb.org

2 1

Re: [Maria-developers] [Commits] 16298d9e82e: MDEV-7533: COLUMN_JSON() doesn't escape control characters in string values
by Sergei Golubchik 22 Jan '18

22 Jan '18

Hi, Oleksandr! On Jan 22, Oleksandr Byelkin wrote: > revision-id: 16298d9e82eb507c5e0ca6d097e6a7016582ccbc (mariadb-10.0.33-47-g16298d9e82e) > parent(s): 5e87f49a9927bf3fd0bd9af789bbdd329e4b4416 > author: Oleksandr Byelkin > committer: Oleksandr Byelkin > timestamp: 2018-01-22 16:53:10 +0100 > message: > > MDEV-7533: COLUMN_JSON() doesn't escape control characters in string values > > escape all charecters less or equal 0x1F (control symbols) > (shorter sequence are not used to make code simple, long encoding is > always legal according to the rfc4627) ok to push Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] MDEV-14603 signal 11 with short stacktrace
by Alexander Barkov 22 Jan '18

22 Jan '18

Hi Sergei, can you please review a patch for MDEV-14603? Thanks!

2 5

Re: [Maria-developers] [JIRA] (MDEV-14185) CREATE TEMPORARY TABLE AS SELECT causes error 1290 with read_only and InnoDB
by Sergei Golubchik 22 Jan '18

22 Jan '18

Hi, Vesa! On Jan 16, Vesa Pentti (JIRA) wrote: > > Vesa Pentti updated MDEV-14185: > ------------------------------- > Status: In Review (was: In Progress) > Assignee: Sergei Golubchik (was: Vesa Pentti) > > commit d4366c4a2c9b41901b11ca180d3ddbf79e992490 Please send commit emails next time. This time I had to go and download the patch from github and paste it in an email. > diff --git a/sql/handler.cc b/sql/handler.cc > index 174034a7eb55..85b73719f0e8 100644 > --- a/sql/handler.cc > +++ b/sql/handler.cc > @@ -4387,8 +4390,11 @@ handler::ha_create_partitioning_metadata(const char *name, const char *old_name, > partitions. > */ > DBUG_ASSERT(m_lock_type == F_UNLCK || > - (!old_name && strcmp(name, table_share->path.str))); > - mark_trx_read_write(); > + (!old_name && table_share && strcmp(name, table_share->path.str))); > + if (should_mark_rw) > + { > + mark_trx_read_write(); > + } Just remove mark_trx_read_write() from ha_create_partitioning_metadata(). It's a non-transactional operation anyway. It's always a part of DDL, so will have the same protection as DDL does. > > return create_partitioning_metadata(name, old_name, action_flag); > } Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] 47bb2dcbcc8: MDEV-14786: Server crashes in Item_cond::transform on 2nd execution of SP querying from a view
by Sergei Golubchik 22 Jan '18

22 Jan '18

Hi, Oleksandr! On Jan 21, Oleksandr Byelkin wrote: > revision-id: 47bb2dcbcc8c3cdca43baf6259f2434a746c7b03 (mariadb-5.5.58-42-g47bb2dcbcc8) > parent(s): c0e964dcf1787a101f6e9c115751c0d02b0fac1b > author: Oleksandr Byelkin > committer: Oleksandr Byelkin > timestamp: 2018-01-21 23:48:32 +0100 > message: > > MDEV-14786: Server crashes in Item_cond::transform on 2nd execution of SP querying from a view > > MDEV-14957: JOIN::prepare gets unusable "conds" as argument > > Do not touch merged derived (it is irreversible) > > Fix first argument of in_optimizer for calls possible before fix_fields() > > diff --git a/sql/item.cc b/sql/item.cc > index 332e027adf1..562c0bec78d 100644 > --- a/sql/item.cc > +++ b/sql/item.cc > @@ -10010,7 +10010,7 @@ const char *dbug_print_item(Item *item) > if (!item) > return "(Item*)NULL"; > item->print(&str ,QT_ORDINARY); > - if (str.c_ptr() == buf) > + if (str.ptr() == buf) c_ptr_safe() here. It's MDEV-14981. > return buf; > else > return "Couldn't fit into buffer"; > diff --git a/sql/item.h b/sql/item.h > index 4d0860af547..add408602fc 100644 > --- a/sql/item.h > +++ b/sql/item.h > @@ -4478,4 +4478,11 @@ class Item_iterator_row: public Item_iterator > void close() {} > }; > > +#ifdef DBUG_OFF > +static inline const char *dbug_print_item(Item *item) { return NULL; } > +#else > +extern const char *dbug_print_item(Item *item); > +#endif Put it not at the end, but at the beginning - above 'class Protocol;' That's where someone had it in 10.2, it'll be easier to merge if you add it in the same place. And no 'extern' in the #else branch, just as in 10.2. > #endif /* SQL_ITEM_INCLUDED */ > diff --git a/sql/item_cmpfunc.cc b/sql/item_cmpfunc.cc > index 62e76922c0e..a40340baae9 100644 > --- a/sql/item_cmpfunc.cc > +++ b/sql/item_cmpfunc.cc > @@ -1443,6 +1443,33 @@ bool Item_in_optimizer::eval_not_null_tables(uchar *opt_arg) > } > > > +void Item_in_optimizer::print(String *str, enum_query_type query_type) > +{ > + restore_first_argumet(); > + Item_func::print(str, query_type); > +} > + > + > +/** > + "Restore" first argument before fix_fields() call (after it is harmless). > + > + @Note: Main pointer to left part of IN/ALL/ANY subselect is subselect's > + lest_expr (see Item_in_optimizer::fix_left) so changes made during > + fix_fields will be rolled back there which can make > + Item_in_optimizer::args[0] unusable on second execution before fix_left() > + call. This call fix the pointer. > +*/ > + > +void Item_in_optimizer::restore_first_argumet() > +{ > + if (args[1]->type() == Item::SUBSELECT_ITEM && > + ((Item_subselect *)args[1])->is_in_predicate()) > + { > + args[0]= ((Item_in_subselect *)args[1])->left_expr; > + } > +} Any way to trigger the bug without dbug_print_item() ? Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] Mdev-14586 Assertion `0' failed in retrieve_auto_increment upon attempt to drop PK
by Sergei Golubchik 19 Jan '18

19 Jan '18

Hi, Sachin! I agree with the patch, I think it's correct. Good tests too. I wasn't able to understand the bug from the commit comment though, see below. Ok to push (preferrably with the improved commit message). On Jan 19, Sachin Setiya wrote: > Hi serg , please review the latest patch > > commit d9641478bb8f6045039a3d942f2925ada334907e > Author: Sachin Setiya <sachin.setiya(a)maridb.com> > Date: Fri Jan 19 19:33:45 2018 +0530 > > MDEV-14586 Assertion `0' failed in retrieve_auto_increment ... > > Problem:- > If we create table using myisam/aria then this crashes the server. > CREATE TABLE t1(a bit(1), b int auto_increment , index(a,b)); > insert into t1 values(1,1); > Or this query > CREATE TABLE t1 (b BIT(1), pk INTEGER AUTO_INCREMENT PRIMARY KEY); > ALTER TABLE t1 ADD INDEX(b,pk); > INSERT INTO t1 VALUES (1,b'1'); > ALTER TABLE t1 DROP PRIMARY KEY; > > Reason:- > The reason for this is > 1st- Since bit field is stored in record null bits itself , so > find_ref_key in open_binary_frm will wrongly set share->next_number_key_offset > and share->next_number_keypart (both set to 0). I would say that find_ref_key() finds what key an auto_increment field belongs to by comparing key_part->offset and field->ptr. But BIT fields might have zero length in the record, so a key might have many key parts with the same offset. That is, comparing offsets cannot uniquely identify the correct key part. > 2nd- Since next_number_key_offset is zero it myisam/aria will think that > auto_increment is in first part of key. > 3nd- myisam/aria will call retrieve_auto_key which will see first key_part > field as a bit field and call assert(0) > > Solution:- Many key parts might have the same offset, but BIT fields do not support auto_increment. So, we can skip all key parts over BIT fields, and then comparing offsets will be unambiguous. > Although the bit field is stored in record null_byte but in key file(.MYI) > this does not hold true. Bit(1-7) will take one byte of memory. So the correct > approach will be to never call retrieve_auto_increment in the case when > bit field is in the starting of index. So I have changed the find_ref_key > function to take account that if first N fields is Bit field then simply go > to next field which is not Bit field. > Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] [Commits] 60d1803ec64: MDEV-6707: Wrong result (extra row) with group by, multi-part key
by Sergey Petrunia 18 Jan '18

18 Jan '18

On Thu, Jan 11, 2018 at 02:35:55AM +0530, Varun wrote: > revision-id: 60d1803ec645b6d388aac7125ac6381ecee4d548 (mariadb-5.5.56-115-g60d1803ec64) > parent(s): 9c9cf556a15af84f9133ef859b50596085f7ae8e > author: Varun Gupta > committer: Varun Gupta > timestamp: 2018-01-11 01:51:23 +0530 > message: > > MDEV-6707: Wrong result (extra row) with group by, multi-part key > > Range optimizer creates ranges according to the where conditions. So this case involves using a composite key, > few parts of which are involved in GROUP BY and few in the MIN/MAX functions in the select list. > So during the execution of such queries, we try to find a prefix using the fields involved in GROUP BY > and we then check if this newly returned prefix lies within the range we had calculated earlier. > After we find a row that satisfies the partial range(involving fields of GROUP BY), then we look at the other fields > of the where clause. We look in all the ranges if we can satisfy the above retured row. > > The issue is while calculating these prefixes we might encounter same partial ranges for multiple ranges. > An example would be > where condition is f2 ='c' AND f1 <> 9 > so the ranges would be > (NULL,c <= f1,f2 <= c,9) > (c,9 <= f1,f2 <= c,+infinity) > > In this case for calculating prefixes with the group by field we take up the > partial ranges involving field f2 those would be > c<= f2 <=c > c<= f2 <=c > So we lookup rows with the same prefix in all such ranges and then we check for the other > part(in this case f1) in all the ranges. So lets take a row of the table to be (4,'c'), that would have > a prefix 'c' and value for f1 to be 4, so it would lie in the first range mentioned above. But we would > get this same record again when we read the partial range and this would again satisfy the first range. > This issue can be fixed if we compare such partial ranges and don't lookup if we see the same prefix again. This is a huge and long comment. Please make it smaller, and move at least part of it into the code. Good places would be: - The function comment for QUICK_RANGE_SELECT::get_next_prefix - the new lines in QUICK_RANGE_SELECT::get_next_prefix which use/update the value of save_last_range > > --- > mysql-test/r/range.result | 14 ++++++++++++++ > mysql-test/r/range_mrr_icp.result | 14 ++++++++++++++ > mysql-test/t/range.test | 13 +++++++++++++ > sql/opt_range.cc | 20 +++++++++++++++++--- > sql/opt_range.h | 6 ++++-- > 5 files changed, 62 insertions(+), 5 deletions(-) > > diff --git a/mysql-test/r/range.result b/mysql-test/r/range.result > index 630a692cef6..72b31de4df4 100644 > --- a/mysql-test/r/range.result > +++ b/mysql-test/r/range.result > @@ -2144,3 +2144,17 @@ value1 1000685 12345 > value1 1003560 12345 > value1 1004807 12345 > drop table t1; > +# > +# MDEV-6707: Wrong result (extra row) with group by, multi-part key > +# > +CREATE TABLE t1 (f1 INT, f2 VARCHAR(1), KEY(f2,f1)) ENGINE=InnoDB; > +INSERT INTO t1 VALUES > +(7,'v'),(0,'s'),(9,'l'),(4,'c'); > +explain > +SELECT MAX(f1), f2 FROM t1 WHERE f2 LIKE 'c%' AND f1 <> 9 GROUP BY f2; > +id select_type table type possible_keys key key_len ref rows Extra > +1 SIMPLE t1 range f2 f2 9 NULL 2 Using where; Using index for group-by > +SELECT MAX(f1), f2 FROM t1 WHERE f2 LIKE 'c%' AND f1 <> 9 GROUP BY f2; > +MAX(f1) f2 > +4 c > +DROP TABLE t1; > diff --git a/mysql-test/r/range_mrr_icp.result b/mysql-test/r/range_mrr_icp.result > index 3f5de5b0189..1050bfcd887 100644 > --- a/mysql-test/r/range_mrr_icp.result > +++ b/mysql-test/r/range_mrr_icp.result > @@ -2146,4 +2146,18 @@ value1 1000685 12345 > value1 1003560 12345 > value1 1004807 12345 > drop table t1; > +# > +# MDEV-6707: Wrong result (extra row) with group by, multi-part key > +# > +CREATE TABLE t1 (f1 INT, f2 VARCHAR(1), KEY(f2,f1)) ENGINE=InnoDB; > +INSERT INTO t1 VALUES > +(7,'v'),(0,'s'),(9,'l'),(4,'c'); > +explain > +SELECT MAX(f1), f2 FROM t1 WHERE f2 LIKE 'c%' AND f1 <> 9 GROUP BY f2; > +id select_type table type possible_keys key key_len ref rows Extra > +1 SIMPLE t1 range f2 f2 9 NULL 2 Using where; Using index for group-by > +SELECT MAX(f1), f2 FROM t1 WHERE f2 LIKE 'c%' AND f1 <> 9 GROUP BY f2; > +MAX(f1) f2 > +4 c > +DROP TABLE t1; > set optimizer_switch=@mrr_icp_extra_tmp; > diff --git a/mysql-test/t/range.test b/mysql-test/t/range.test > index 393ca68e945..62e907b7b4a 100644 > --- a/mysql-test/t/range.test > +++ b/mysql-test/t/range.test > @@ -1718,3 +1718,16 @@ where (key1varchar='value1' AND (key2int <=1 OR key2int > 1)); > --echo # The following must show col1=12345 for all rows: > select * from t1; > drop table t1; > + > +--echo # > +--echo # MDEV-6707: Wrong result (extra row) with group by, multi-part key > +--echo # > + > +CREATE TABLE t1 (f1 INT, f2 VARCHAR(1), KEY(f2,f1)) ENGINE=InnoDB; > +INSERT INTO t1 VALUES > +(7,'v'),(0,'s'),(9,'l'),(4,'c'); > + > +explain > +SELECT MAX(f1), f2 FROM t1 WHERE f2 LIKE 'c%' AND f1 <> 9 GROUP BY f2; > +SELECT MAX(f1), f2 FROM t1 WHERE f2 LIKE 'c%' AND f1 <> 9 GROUP BY f2; > +DROP TABLE t1; > diff --git a/sql/opt_range.cc b/sql/opt_range.cc > index 25a9e729a8b..e985aba1140 100644 > --- a/sql/opt_range.cc > +++ b/sql/opt_range.cc > @@ -11249,6 +11249,7 @@ int QUICK_RANGE_SELECT::get_next() > @param prefix_length length of cur_prefix > @param group_key_parts The number of key parts in the group prefix > @param cur_prefix prefix of a key to be searched for > + @save_last_range Saving the last range we encountered You've meant "@param save_last_range" ? Also please use " INOUT " to indicate this parameter is used to pass info both from the caller and to the caller. > > Each subsequent call to the method retrieves the first record that has a > prefix with length prefix_length and which is different from cur_prefix, > @@ -11271,7 +11272,8 @@ int QUICK_RANGE_SELECT::get_next() > > int QUICK_RANGE_SELECT::get_next_prefix(uint prefix_length, > uint group_key_parts, > - uchar *cur_prefix) > + uchar *cur_prefix, > + QUICK_RANGE **save_last_range) > { > DBUG_ENTER("QUICK_RANGE_SELECT::get_next_prefix"); > const key_part_map keypart_map= make_prev_keypart_map(group_key_parts); > @@ -11301,8 +11303,19 @@ int QUICK_RANGE_SELECT::get_next_prefix(uint prefix_length, > last_range= 0; > DBUG_RETURN(HA_ERR_END_OF_FILE); > } > + > last_range= *(cur_range++); > > + if (*save_last_range) > + { > + if (!key_tuple_cmp(key_part_info, (*save_last_range)->min_key, > + last_range->min_key, prefix_length)) > + { > + last_range=NULL; > + continue; > + } > + } > + *save_last_range= last_range; > key_range start_key, end_key; > last_range->make_min_endpoint(&start_key, prefix_length, keypart_map); > last_range->make_max_endpoint(&end_key, prefix_length, keypart_map); > @@ -13315,7 +13328,7 @@ QUICK_GROUP_MIN_MAX_SELECT(TABLE *table, JOIN *join_arg, bool have_min_arg, > seen_first_key(FALSE), doing_key_read(FALSE), min_max_arg_part(min_max_arg_part_arg), > key_infix(key_infix_arg), key_infix_len(key_infix_len_arg), > min_functions_it(NULL), max_functions_it(NULL), > - is_index_scan(is_index_scan_arg) > + is_index_scan(is_index_scan_arg), save_last_range(NULL) > { > head= table; > index= use_index; > @@ -13968,7 +13981,8 @@ int QUICK_GROUP_MIN_MAX_SELECT::next_prefix() > uchar *cur_prefix= seen_first_key ? group_prefix : NULL; > if ((result= quick_prefix_select->get_next_prefix(group_prefix_len, > group_key_parts, > - cur_prefix))) > + cur_prefix, > + &save_last_range))) > DBUG_RETURN(result); > seen_first_key= TRUE; > } > diff --git a/sql/opt_range.h b/sql/opt_range.h > index b8b46ae5ab1..f4fb0642590 100644 > --- a/sql/opt_range.h > +++ b/sql/opt_range.h > @@ -455,6 +455,7 @@ class QUICK_RANGE_SELECT : public QUICK_SELECT_I > QUICK_RANGE **cur_range; /* current element in ranges */ > > QUICK_RANGE *last_range; > + QUICK_RANGE *save_last_range; So, now both QUICK_RANGE_SELECT and QUICK_GROUP_MIN_MAX_SELECT have 'save_last_range'. Does QUICK_RANGE_SELECT need one? > > KEY_PART *key_parts; > KEY_PART_INFO *key_part_info; > @@ -477,7 +478,7 @@ class QUICK_RANGE_SELECT : public QUICK_SELECT_I > int get_next(); > void range_end(); > int get_next_prefix(uint prefix_length, uint group_key_parts, > - uchar *cur_prefix); > + uchar *cur_prefix, QUICK_RANGE **save_last_range); > bool reverse_sorted() { return 0; } > bool unique_key_range(); > int init_ror_merged_scan(bool reuse_handler, MEM_ROOT *alloc); > @@ -916,7 +917,8 @@ class QUICK_GROUP_MIN_MAX_SELECT : public QUICK_SELECT_I > Use index scan to get the next different key instead of jumping into it > through index read > */ > - bool is_index_scan; > + bool is_index_scan; > + QUICK_RANGE *save_last_range; > public: > /* > The following two members are public to allow easy access from BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] [Commits] 6ffd535e415: MDEV-14229: Stack trace is not resolved for shared objects
by Sergei Golubchik 18 Jan '18

18 Jan '18

Hi, Vicentiu! Looks mostly ok. Does it mean, it will replace my changes in 10.0+ related to calculating the offset? struct link_map *lm = (struct link_map*) dlopen(0, RTLD_NOW); etc As far as LIBDL and dependencies are concerned... 1. Don't add LIBDL to the link list. The server has it and clients should not link with my_addr_resove (and it's good to get a linker error, if by some mistake they do) 2. Check for dladdr in configure.cmake: CHECK_FUNCTION_EXISTS (dladdr HAVE_DLADDR) 3. handle dladdr in my_global.h (search for dlopen or dlerror) But, I think, you won't need any ifdefs in my_addr_resolve.c, because my_global.h will take care of that. Ok to push after that, thanks! On Jan 16, vicentiu(a)mariadb.org wrote: > revision-id: 6ffd535e415028f414b9d1c9e9efbfb81bf25798 (mariadb-5.5.58-42-g6ffd535e415) > parent(s): 6267be460ab5147e3bc0fffd03e690b3650f2fe2 > author: Vicențiu Ciorbaru > committer: Vicențiu Ciorbaru > timestamp: 2018-01-16 22:57:52 +0200 > message: > > MDEV-14229: Stack trace is not resolved for shared objects > > Resolving a stacktrace including functions in dynamic libraries requires > us to look inside the libraries for the symbols. Addr2line needs to be > started with the correct binary for each address on the stack. To do this, > figure out which library it is using dladdr, then if the addr2line > binary was started with a different binary, fork it again with the > correct one. > > We only have one addr2line process running at any point during the > stacktrace resolving step. The maximum number of forks for addr2line should > generally be around 6. > > One for server stacktrace code, one for plugin code, one when going back > into server code, one for pthread library, one for libc, one for the > _start function in the server. More can come up if plugin calls server > function which goes back to a plugin, etc. > > --- > mysys/CMakeLists.txt | 2 +- > mysys/my_addr_resolve.c | 97 +++++++++++++++++++++++++++++++++---------------- > 2 files changed, 67 insertions(+), 32 deletions(-) > > diff --git a/mysys/CMakeLists.txt b/mysys/CMakeLists.txt > index cb86850c2de..15f3fbad9bf 100644 > --- a/mysys/CMakeLists.txt > +++ b/mysys/CMakeLists.txt > @@ -67,7 +67,7 @@ ENDIF() > > ADD_CONVENIENCE_LIBRARY(mysys ${MYSYS_SOURCES}) > TARGET_LINK_LIBRARIES(mysys dbug strings ${ZLIB_LIBRARY} > - ${LIBNSL} ${LIBM} ${LIBRT} ${LIBSOCKET} ${LIBEXECINFO}) > + ${LIBNSL} ${LIBM} ${LIBRT} ${LIBSOCKET} ${LIBEXECINFO} ${LIBDL}) > DTRACE_INSTRUMENT(mysys) > > IF(HAVE_BFD_H) > diff --git a/mysys/my_addr_resolve.c b/mysys/my_addr_resolve.c > index 90e6f43f390..f831ad5121f 100644 > --- a/mysys/my_addr_resolve.c > +++ b/mysys/my_addr_resolve.c > @@ -132,15 +132,79 @@ const char *my_addr_resolve_init() > > #include <m_string.h> > #include <ctype.h> > + > +#include <sys/wait.h> > + > static int in[2], out[2]; > -static int initialized= 0; > +static pid_t pid; > +static char addr2line_binary[1024]; > static char output[1024]; > + > +int start_addr2line_fork(const char *binary_path) > +{ > + > + if (pid > 0) > + { > + /* Don't leak FDs */ > + close(in[1]); > + close(out[0]); > + /* Don't create zombie processes. */ > + waitpid(pid, NULL, 0); > + } > + > + if (pipe(in) < 0) > + return 1; > + if (pipe(out) < 0) > + return 1; > + > + pid = fork(); > + if (pid == -1) > + return 1; > + > + if (!pid) /* child */ > + { > + dup2(in[0], 0); > + dup2(out[1], 1); > + close(in[0]); > + close(in[1]); > + close(out[0]); > + close(out[1]); > + execlp("addr2line", "addr2line", "-C", "-f", "-e", binary_path, NULL); > + exit(1); > + } > + > + close(in[0]); > + close(out[1]); > + > + return 0; > +} > + > int my_addr_resolve(void *ptr, my_addr_loc *loc) > { > char input[32], *s; > size_t len; > > - len= my_snprintf(input, sizeof(input), "%p\n", ptr); > + Dl_info info; > + void *offset; > + > + if (!dladdr(ptr, &info)) > + return 1; > + > + if (strcmp(addr2line_binary, info.dli_fname)) > + { > + /* We use dli_fname in case the path is longer than the length of our static > + string. We don't want to allocate anything dynamicaly here as we are in > + a "crashed" state. */ > + if (start_addr2line_fork(info.dli_fname)) > + { > + addr2line_binary[0] = '\0'; > + return 1; > + } > + /* Save result for future comparisons. */ > + strnmov(addr2line_binary, info.dli_fname, sizeof(addr2line_binary)); > + } > + offset = info.dli_fbase; > + len= my_snprintf(input, sizeof(input), "%p\n", ptr - offset); > if (write(in[1], input, len) <= 0) > return 1; > if (read(out[0], output, sizeof(output)) <= 0) > @@ -168,35 +232,6 @@ int my_addr_resolve(void *ptr, my_addr_loc *loc) > > const char *my_addr_resolve_init() > { > - if (!initialized) > - { > - pid_t pid; > - > - if (pipe(in) < 0) > - return "pipe(in)"; > - if (pipe(out) < 0) > - return "pipe(out)"; > - > - pid = fork(); > - if (pid == -1) > - return "fork"; > - > - if (!pid) /* child */ > - { > - dup2(in[0], 0); > - dup2(out[1], 1); > - close(in[0]); > - close(in[1]); > - close(out[0]); > - close(out[1]); > - execlp("addr2line", "addr2line", "-C", "-f", "-e", my_progname, NULL); > - exit(1); > - } > - > - close(in[0]); > - close(out[1]); > - initialized= 1; > - } > return 0; > } > #endif Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] a doubt about locks on supremum
by hongxiang jiang 18 Jan '18

18 Jan '18

2 1

Re: [Maria-developers] [Commits] 4b4267288cb: MDEV-14743: Server crashes in Item_func_match::init_search
by Sergei Golubchik 12 Jan '18

12 Jan '18

Hi, Oleksandr! Two thoughts. 1. Please use if (!ifm->fixed) then you'll only need one #ifdef. 2. How does it work with the iterator? if you remove the current element from the list and then iterator with jump to the next one, won't it cause one element to be skipped? On Jan 10, Oleksandr Byelkin wrote: > revision-id: 4b4267288cb2668717cc70537eb7e79cecaff58a (mariadb-5.5.58-24-g4b4267288cb) > parent(s): a408e881cf73d06fc92097fce6ef9584e16edf77 > author: Oleksandr Byelkin > committer: Oleksandr Byelkin > timestamp: 2018-01-10 12:22:56 +0100 > message: > > MDEV-14743: Server crashes in Item_func_match::init_search > > Remove non prepared (and so belonging to removed clauses FT functions) from the list. > > in later version it will be fixed by building the list during preparation. > > diff --git a/sql/sql_base.cc b/sql/sql_base.cc > index c06c4fcff29..26e302ded35 100644 > --- a/sql/sql_base.cc > +++ b/sql/sql_base.cc > @@ -9550,7 +9550,19 @@ int init_ftfuncs(THD *thd, SELECT_LEX *select_lex, bool no_order) > DBUG_PRINT("info",("Performing FULLTEXT search")); > > while ((ifm=li++)) > - ifm->init_search(no_order); > +#if MYSQL_VERSION_ID < 100213 > + if (ifm->fixed) > +#endif > + ifm->init_search(no_order); > +#if MYSQL_VERSION_ID < 100213 > + else > + /* > + it mean that clause where was FT function was removed, so we have > + to remove the function from the list. > + */ > + li.remove(); > +#endif > + > } > return 0; > } Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

2 1

[Maria-developers] Eliminating or replacing the InnoDB row_prebuilt_t::fetch_cache
by Marko Mäkelä 12 Jan '18

12 Jan '18

The handler API in MySQL and MariaDB is operating on a single record at a time. There are some ‘batched’ primitives that in my understanding were developed for the NDB cluster, but generally InnoDB does not use them. The only nontrivial thing that InnoDB does during reads is the Index Condition Pushdown (ICP) and the end-of-range detection (DsMrr) that was ported from the never-released MySQL 6.0 or 5.2 to MySQL 5.6. To improve the performance of range scans, InnoDB uses a caching mechanism to prefetch results. Starting with the 4th consecutive read on the same ha_innobase handle, InnoDB would prefetch 8 records into row_prebuilt_t::fetch_cache. There are several drawbacks of this: - The parameters of this cache are hard-coded, and these caches can make wrong guesses (for SELECT * FROM t LIMIT 5, it would unnecessarily read 8 records, possibly causing unnecessary page accesses) - The caches waste memory, especially on partitioned tables, where each partition handle would have its own cache. (The main benefit of ha_innopart or InnoDB native partitioning in MySQL 5.7 is that the fetch_cache of the table is shared between partitions. It is only a benefit of saving memory, because the cache will be emptied when switching partitions.) As far as I understand, the entire purpose of the prebuilt->fetch_cache is to reduce the cost of acquiring the page latch and repositioning the cursor on each read request. If we did not call mtr_t::commit() between each request, maybe we could remove this cache altogether. When I was in the InnoDB team at MySQL, I did bring up a few times the idea of eliminating the fetch_cache, but the optimizer developers were in a separate team, and to do this, we would have needed support from both managers. I am hoping that with MariaDB this would merely be a technical challenge to first write a proof-of-concept prototype, and then to polish and test it. When I discussed this with the late Olav Sandstå some years ago, there was some thinking that we should stop hard-coding the TABLE::record[0] in various parts of the code, and the storage engine should be able to return multiple records to a larger TABLE::record[] buffer. I wonder if a simpler approach would work, and if we could do without any prefetching: 1. Remove row_sel_dequeue_cached_row_for_mysql(), prebuilt->fetch_cache, and related code. 2. Add prebuilt->mtr and let it be reused across multiple subsequent calls to row_search_mvcc(), until the query executor stops the current read batch. 3. A call to btr_pcur_store_position() and mtr_t::commit() will be necessary before any potentially long delay, such as when the result is being sent to a client connection, or when the query executor is going to switch to read records from another table. 4. The "end of read batch" could be signalled by invoking handler::extra() with a parameter. With such changes, less copying would take place, and there should be less contention on dict_index_t::lock and the upper-level buf_block_t::lock, due to fewer calls to btr_pcur_restore_position(). While that function does have a fast path ("optimistic restore"), it cannot switch to the next page without a key lookup. Range scans would be more efficient if we can always simply advance to the right leaf page and release the lock on the left page. I would love to do this in InnoDB. But I think that the effort should be lead from the optimizer side. Someone should first implement the "end of batched read" calls and some DBUG_ASSERT() that would catch any wrong-doing. That is, whenever we start to send data to a client or start to write into a temporary table, or switch to read records from another cursor, we must assert that the "end of batched read" flag is set on all cursor handles. Similarly, at the end and start of a statement, and at commit or rollback, all cursor handles must be at "end of batched read". Who would take the challenge? Best regards, Marko -- Marko Mäkelä, Lead Developer InnoDB MariaDB Corporation DON’T MISS M|18 MariaDB User Conference February 26 - 27, 2018 New York City https://m18.mariadb.com/

1 0

[Maria-developers] Check constraints
by Martijn Tonies (Upscene Productions) 10 Jan '18

10 Jan '18

Hi, Where can I find meta information for check constraints on tables? With regards, Martijn Tonies Upscene Productions http://www.upscene.com Database Workbench - developer tool for Oracle, MS SQL Server, PostgreSQL, SQL Anywhere, MySQL, InterBase, NexusDB and Firebird.

3 4

[Maria-developers] subscribe
by Martijn Tonies (Upscene Productions) 09 Jan '18

09 Jan '18

subscribe With regards, Martijn Tonies Upscene Productions http://www.upscene.com Database Workbench - developer tool for Oracle, MS SQL Server, PostgreSQL, SQL Anywhere, MySQL, InterBase, NexusDB and Firebird.

1 0

Re: [Maria-developers] [Commits] fbe0fc2: Fixed mdev-14879 Lost rows for query using recursive CTE
by Oleksandr Byelkin 09 Jan '18

09 Jan '18

Hi, Igor! Am 06.01.2018 um 18:32 schrieb IgorBabaev: > revision-id: fbe0fc2d87cb1bdb9f7040d44ad3c58a94857e20 (mariadb-10.2.12-14-gfbe0fc2) > parent(s): 15b1840f43be8c660382458826e9d5a47d148a67 > author: Igor Babaev > committer: Igor Babaev > timestamp: 2018-01-06 09:32:47 -0800 > message: > > Fixed mdev-14879 Lost rows for query using recursive CTE > with recursive reference in subquery > > If a recursive CTE uses a subquery with recursive reference then > the virtual function reset() must be called after each iteration > performed at the execution of the CTE. OK to push. [skip]

1 0

Re: [Maria-developers] [Commits] 81965f0: Fixed mdev-14852 Fails to reopen temp table within standard CTE
by Oleksandr Byelkin 05 Jan '18

05 Jan '18

Am 05.01.2018 um 08:40 schrieb IgorBabaev: > revision-id: 81965f03065b84322f2d03d784893179e8b79fd9 (mariadb-10.2.12-6-g81965f0) > parent(s): 0f253d3e644d3027face26371eb1bb59e8280728 > author: Igor Babaev > committer: Igor Babaev > timestamp: 2018-01-04 23:40:37 -0800 > message: > > Fixed mdev-14852 Fails to reopen temp table within standard CTE > > If the specification of a CTE contains a reference to a temporary table > then THD::open_temporary_table() must be called for this reference for > any occurrence of the CTE in the query. By mistake this was done only > for the first occurrences of CTEs. > The patch fixes this problem in With_element::clone_parsed_spec(). > It also moves there the call of check_dependencies_in_with_clauses() > to its proper place before the call of check_table_access(). > Additionally the patch optimizes the number of calls of the > function check_dependencies_in_with_clauses(). OK to push! > > --- > mysql-test/r/cte_nonrecursive.result | 30 ++++++++++++++++++++++++++++++ > mysql-test/t/cte_nonrecursive.test | 29 +++++++++++++++++++++++++++++ > sql/sql_cte.cc | 7 +++++++ > sql/sql_parse.cc | 22 +++++----------------- > sql/sql_prepare.cc | 11 +++-------- > 5 files changed, 74 insertions(+), 25 deletions(-) > > diff --git a/mysql-test/r/cte_nonrecursive.result b/mysql-test/r/cte_nonrecursive.result > index b1b41b1..c1f4c9f 100644 > --- a/mysql-test/r/cte_nonrecursive.result > +++ b/mysql-test/r/cte_nonrecursive.result > @@ -1265,3 +1265,33 @@ a > 4 > deallocate prepare stmt; > drop table t1; > +# > +# MDEV-14852: CTE using temporary table in query > +# with two references to the CTE > +# > +create temporary table t1 (i int); > +insert into t1 values (5),(4),(1),(2),(3); > +with > +c1 as (select i from t1), > +c2 as (select i from c1 where c1.i=2) > +select i from c1 where i > 3 union select i from c2; > +i > +5 > +4 > +2 > +drop table t1; > +create table t1 (term char(10)); > +create temporary table t2 (term char(10)); > +insert into t1 values ('TERM01'),('TERM02'),('TERM03'); > +insert into t2 values ('TERM02'),('TERM03'),('TERM04'); > +with c1 as (select * from t1), c2 as (select * from t2) > +(select * from c1 left outer join c2 on c1.term = c2.term) > +union all > +(select * from c1 right outer join c2 on c1.term = c2.term > +where c1.term is null); > +term term > +TERM02 TERM02 > +TERM03 TERM03 > +TERM01 NULL > +NULL TERM04 > +drop table t1,t2; > diff --git a/mysql-test/t/cte_nonrecursive.test b/mysql-test/t/cte_nonrecursive.test > index 1f21dbc..9436665 100644 > --- a/mysql-test/t/cte_nonrecursive.test > +++ b/mysql-test/t/cte_nonrecursive.test > @@ -853,3 +853,32 @@ execute stmt; > deallocate prepare stmt; > > drop table t1; > + > +--echo # > +--echo # MDEV-14852: CTE using temporary table in query > +--echo # with two references to the CTE > +--echo # > + > +create temporary table t1 (i int); > +insert into t1 values (5),(4),(1),(2),(3); > + > +with > +c1 as (select i from t1), > +c2 as (select i from c1 where c1.i=2) > +select i from c1 where i > 3 union select i from c2; > + > +drop table t1; > + > +create table t1 (term char(10)); > +create temporary table t2 (term char(10)); > + > +insert into t1 values ('TERM01'),('TERM02'),('TERM03'); > +insert into t2 values ('TERM02'),('TERM03'),('TERM04'); > + > +with c1 as (select * from t1), c2 as (select * from t2) > +(select * from c1 left outer join c2 on c1.term = c2.term) > +union all > +(select * from c1 right outer join c2 on c1.term = c2.term > + where c1.term is null); > + > +drop table t1,t2; > diff --git a/sql/sql_cte.cc b/sql/sql_cte.cc > index 601c192..d12948f 100644 > --- a/sql/sql_cte.cc > +++ b/sql/sql_cte.cc > @@ -817,12 +817,19 @@ st_select_lex_unit *With_element::clone_parsed_spec(THD *thd, > parse_status= parse_sql(thd, &parser_state, 0); > if (parse_status) > goto err; > + > + if (check_dependencies_in_with_clauses(lex->with_clauses_list)) > + goto err; > + > spec_tables= lex->query_tables; > spec_tables_tail= 0; > for (TABLE_LIST *tbl= spec_tables; > tbl; > tbl= tbl->next_global) > { > + if (!tbl->derived && !tbl->schema_table && > + thd->open_temporary_table(tbl)) > + goto err; > spec_tables_tail= tbl; > } > if (check_table_access(thd, SELECT_ACL, spec_tables, FALSE, UINT_MAX, FALSE)) > diff --git a/sql/sql_parse.cc b/sql/sql_parse.cc > index f00c74b..6d068eb 100644 > --- a/sql/sql_parse.cc > +++ b/sql/sql_parse.cc > @@ -2990,6 +2990,9 @@ mysql_execute_command(THD *thd) > thd->get_stmt_da()->opt_clear_warning_info(thd->query_id); > } > > + if (check_dependencies_in_with_clauses(thd->lex->with_clauses_list)) > + DBUG_RETURN(1); > + > #ifdef HAVE_REPLICATION > if (unlikely(thd->slave_thread)) > { > @@ -3446,14 +3449,6 @@ mysql_execute_command(THD *thd) > ulong privileges_requested= lex->exchange ? SELECT_ACL | FILE_ACL : > SELECT_ACL; > > - /* > - The same function must be called for DML commands > - when CTEs are supported in DML statements > - */ > - res= check_dependencies_in_with_clauses(thd->lex->with_clauses_list); > - if (res) > - break; > - > if (all_tables) > res= check_table_access(thd, > privileges_requested, > @@ -3879,8 +3874,7 @@ mysql_execute_command(THD *thd) > /* Copy temporarily the statement flags to thd for lock_table_names() */ > uint save_thd_create_info_options= thd->lex->create_info.options; > thd->lex->create_info.options|= create_info.options; > - if (!(res= check_dependencies_in_with_clauses(lex->with_clauses_list))) > - res= open_and_lock_tables(thd, create_info, lex->query_tables, TRUE, 0); > + res= open_and_lock_tables(thd, create_info, lex->query_tables, TRUE, 0); > thd->lex->create_info.options= save_thd_create_info_options; > if (res) > { > @@ -4493,8 +4487,7 @@ mysql_execute_command(THD *thd) > > unit->set_limit(select_lex); > > - if (!(res= check_dependencies_in_with_clauses(lex->with_clauses_list)) && > - !(res=open_and_lock_tables(thd, all_tables, TRUE, 0))) > + if (!(res=open_and_lock_tables(thd, all_tables, TRUE, 0))) > { > MYSQL_INSERT_SELECT_START(thd->query()); > /* > @@ -4821,9 +4814,6 @@ mysql_execute_command(THD *thd) > { > List<set_var_base> *lex_var_list= &lex->var_list; > > - if (check_dependencies_in_with_clauses(thd->lex->with_clauses_list)) > - goto error; > - > if ((check_table_access(thd, SELECT_ACL, all_tables, FALSE, UINT_MAX, FALSE) > || open_and_lock_tables(thd, all_tables, TRUE, 0))) > goto error; > @@ -6376,8 +6366,6 @@ static bool execute_sqlcom_select(THD *thd, TABLE_LIST *all_tables) > new (thd->mem_root) Item_int(thd, > (ulonglong) thd->variables.select_limit); > } > - if (check_dependencies_in_with_clauses(lex->with_clauses_list)) > - return 1; > > if (!(res= open_and_lock_tables(thd, all_tables, TRUE, 0))) > { > diff --git a/sql/sql_prepare.cc b/sql/sql_prepare.cc > index be78a76..390b708 100644 > --- a/sql/sql_prepare.cc > +++ b/sql/sql_prepare.cc > @@ -1573,8 +1573,6 @@ static int mysql_test_select(Prepared_statement *stmt, > lex->select_lex.context.resolve_in_select_list= TRUE; > > ulong privilege= lex->exchange ? SELECT_ACL | FILE_ACL : SELECT_ACL; > - if (check_dependencies_in_with_clauses(lex->with_clauses_list)) > - goto error; > if (tables) > { > if (check_table_access(thd, privilege, tables, FALSE, UINT_MAX, FALSE)) > @@ -1841,9 +1839,6 @@ static bool mysql_test_create_table(Prepared_statement *stmt) > if (create_table_precheck(thd, tables, create_table)) > DBUG_RETURN(TRUE); > > - if (check_dependencies_in_with_clauses(lex->with_clauses_list)) > - DBUG_RETURN(TRUE); > - > if (select_lex->item_list.elements) > { > /* Base table and temporary table are not in the same name space. */ > @@ -2234,9 +2229,6 @@ static bool mysql_test_insert_select(Prepared_statement *stmt, > if (insert_precheck(stmt->thd, tables)) > return 1; > > - if (check_dependencies_in_with_clauses(lex->with_clauses_list)) > - return 1; > - > /* store it, because mysql_insert_select_prepare_tester change it */ > first_local_table= lex->select_lex.table_list.first; > DBUG_ASSERT(first_local_table != 0); > @@ -2339,6 +2331,9 @@ static bool check_prepared_statement(Prepared_statement *stmt) > if (tables) > thd->get_stmt_da()->opt_clear_warning_info(thd->query_id); > > + if (check_dependencies_in_with_clauses(thd->lex->with_clauses_list)) > + goto error; > + > if (sql_command_flags[sql_command] & CF_HA_CLOSE) > mysql_ha_rm_tables(thd, tables); > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits

1 0

Re: [Maria-developers] [Commits] de21526: This is a full cost-based implementation of the optimization that employs
by Oleksandr Byelkin 29 Dec '17

29 Dec '17

Hi, Igor! Am 29.12.2017 um 01:47 schrieb IgorBabaev: > revision-id: de215264682a9f3291ca028fb3ae052ecb191ca3 (mariadb-10.3.3-4-gde21526) > parent(s): 7a66e0ab8f52f3bd32850463daa05f9a2401e6b1 > author: Igor Babaev > committer: Igor Babaev > timestamp: 2017-12-28 16:47:25 -0800 > message: > > This is a full cost-based implementation of the optimization that employs > splitting technique for equi-joins of materialized derived tables/views/CTEs. > (see mdev-13369 and mdev-13389). I have to make special mention of really good description. As we agreed OK to push after adding test of double execution. [skip]

1 0

Re: [Maria-developers] a01f9ff: MDEV-11071 - Assertion `thd->transaction.stmt.is_empty()' failed in
by Sergei Golubchik 27 Dec '17

27 Dec '17

Hi, Sergey! On Dec 26, Sergey Vojtovich wrote: > revision-id: a01f9ff80b892bf32dd2754da17438e8884c003a (mariadb-10.2.11-51-ga01f9ff) > parent(s): c3bd0b0301936d0dcd0918bb8c966c272610d61b > committer: Sergey Vojtovich > timestamp: 2017-12-26 14:38:45 +0400 > message: > > MDEV-11071 - Assertion `thd->transaction.stmt.is_empty()' failed in > Locked_tables_list::unlock_locked_tables > > Similarly to regular DROP TABLE, don't leave locked tables mode if CREATE OR > REPLACE dropped temporary table but failed to cerate new one. > > The problem is that there's no track of which temporary table was "locked" by > LOCK TABLES. Looks reasonable. Could you also add a test where t2 is locked and t2 is dropped? Just to show that pos_in_locked_tables works for temporary tables. Ok to push after that. Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] Please document HOUR() vs EXTRACT(HOUR)
by Alexander Barkov 23 Dec '17

23 Dec '17

Hello Ian, Can you please document the difference between functions HOUR() and EXTRACT(HOUR)? Please see details here: https://jira.mariadb.org/browse/MDEV-14731 Thanks!

1 0

[Maria-developers] mysql and mariadb got different result
by 落寞回头不如华丽转身 21 Dec '17

21 Dec '17

mysql but mariadb

2 1

[Maria-developers] Fwd: Need Help
by Deepak Gupta 20 Dec '17

20 Dec '17

Hi There, I am working on a project where I have some wired requirement for my project. Issue : I am looking for the generic script that I can run with the help of Admin rights on the data base and I want the return output in such a way that it will give me the complete data base structure along with the table relation ship. For an example , there is one HR schema , with in that schema there might be 100 tables example , Employee / Salary / Address / Department etc and every table has primary and foreign key concept. So I need the script should be such a way that it will fetch all these information and as soon as I run the same script in my local system it will replicate the same data base structure for me in my local system. Kindly suggest how can I achieve this requirement ? Any help or suggestions are more than welcome. Looking foreword to here you soon. Thanks D

2 1

Re: [Maria-developers] [Commits] 203a658: MDEV-11084 Select statement with partition selection against MyISAM table opens all partitions.
by Sergei Golubchik 19 Dec '17

19 Dec '17

Hi, Alexey! On Dec 18, Alexey Botchkov wrote: > revision-id: 203a6585d3deedb36e30d5ebf388de44b6810403 (mariadb-10.1.23-51-g203a658) > parent(s): e4d10e09cf318aad237143254c45458d16009f70 > committer: Alexey Botchkov > timestamp: 2017-12-18 19:31:26 +0400 > message: > > MDEV-11084 Select statement with partition selection against MyISAM table opens all partitions. > > Now SELECT FROM t PARTITION(x) only opens the 'x' file. > The table->partition_names parameter is sent to ha_open > so it only opens the required partitions. > If the table is reopened, the > change_partition_to_open(partition_names) is called that > closes and opens partitions specified for the query. Looks ok to me, thanks! Please add some tests, can status variables show the difference in the number of opened partitions? And make sure that the whole tree compiles without partitioning. Then ok to push into 10.3 Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] [Commits] 7a7ad82: MDEV-13478 Full SST sync fails because of the error in the cleaning part
by Sergei Golubchik 19 Dec '17

19 Dec '17

Hi, Sachin! Unfortunately, your comment is rather difficult to understand. What about this one: Problem: The command was: find $paths -mindepth 1 -regex $cpat -prune -o -exec rm -rf {} \+ Which was supposed to work as * skipping $paths directories themselves (-mindepth 1) * see if the dir/file name matches $cpat (-regex) * if yes - don't dive into the directory, skip it (-prune) * otherwise (-o) * remove it and everything inside (-exec) Now -exec ... \+ works like this: every new found path is appended to the end of the command line. when accumulated command line length reaches `getconf ARG_MAX` (~2Gb) it's executed, and find continues, appending to a new command line. What happens here, find appends some directory to the command line, then dives into it, and starts appending files from that directory. At some point command line overflows, rm -rf gets executed and removes the whole directory. Now find tries to continue scanning the directory that was already removed. Fix: don't dive into directories that will be recursively removed anyway, use -prune for them. Basically, we should be pruning both paths that have matched $cpat and paths that have not matched it. This is achived by pruning unconditionally, before the regex is tested: find $paths -mindepth 1 -prune -regex $cpat -o -exec rm -rf {} \+ On Dec 19, sachin wrote: > revision-id: 7a7ad82029a6c78d31d6736a562c12d02c4d968c (mariadb-galera-5.5.58-3-g7a7ad82) > parent(s): e6e026ae51a77969749de201d491a176483bbc69 > author: Sachin Setiya > committer: Sachin Setiya > timestamp: 2017-12-19 22:30:43 +0530 > message: > > MDEV-13478 Full SST sync fails because of the error in the cleaning part > > Problem:- The problem is in wsrep_sst_xtrabackup-v2.sh we use > find $ib_home_dir $ib_log_dir $ib_undo_dir $DATA -mindepth 1 -regex $cpat -prune -o -exec rm -rfv {} 1>&2 \+ > the problem is that since we have '\+' in end that means all output > will be expanded after rm -rfv . If we have really large database( > quite a no of tables with big names) then this create a problem. > This will result in calling 'rm -rvf' two time(or may be more). So non > deterministicly this might that upto directory name went to rm and > remaining was truncated. For example consider a folder xyz with lots > of files. We executed > find xyz -exec rm -rfv {} \+ > Since we have like millions of file in rm, So it will be greater then > ARG_MAX , so there will be multiple rm invocation. Say in nth invocation > this might happen rm ....... xyz/ {So it get truncated at xyz/} > Above will remove the whole xyz directory and remaining invocation > of rm will return error. How ever this type of error is non deterministic > so is bug. > Solution:- > In above example if instead of removing each file in xyz if we remove > xyz then we have our solution :). > Actually if we shift the -prune term in find we can get to solution. > Why currently find is working like this > find ( -regex && -prune) || -exec > that means if -regex is true (-prune will always return true) then exec > (and hence rm )wont work. But if regex fails then exec is applied with > out -prune which makes rm delete each single file instead of folder. > So we change the position of regex and prune then prune will always be > applied whether regex is true or not. > find (-prune && -regex) || -exec > > Patch Credit:- Serg > > --- > scripts/wsrep_sst_xtrabackup-v2.sh | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/scripts/wsrep_sst_xtrabackup-v2.sh b/scripts/wsrep_sst_xtrabackup-v2.sh > index 327c92f..00d8fe2 100644 > --- a/scripts/wsrep_sst_xtrabackup-v2.sh > +++ b/scripts/wsrep_sst_xtrabackup-v2.sh > @@ -863,7 +863,7 @@ then > > > wsrep_log_info "Cleaning the existing datadir and innodb-data/log directories" > - find $ib_home_dir $ib_log_dir $ib_undo_dir $DATA -mindepth 1 -regex $cpat -prune -o -exec rm -rfv {} 1>&2 \+ > + find $ib_home_dir $ib_log_dir $ib_undo_dir $DATA -mindepth 1 -prune -regex $cpat -o -exec rm -rfv {} 1>&2 \+ > > tempdir=$(parse_cnf mysqld log-bin "") > if [[ -n ${tempdir:-} ]];then Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

2 1

Re: [Maria-developers] 665402a: MDEV-14265 - RPMLint warning: shared-lib-calls-exit
by Sergei Golubchik 18 Dec '17

18 Dec '17

Hi, Sergey! On Dec 18, Sergey Vojtovich wrote: > revision-id: 665402aa9b83565c1cef3caa06cd998d3808f0d7 (mariadb-10.1.29-31-g665402a) > parent(s): ef9e78c9d41a5ac644f08068e3dabad948b0e30a > committer: Sergey Vojtovich > timestamp: 2017-12-18 20:23:36 +0400 > message: > > MDEV-14265 - RPMLint warning: shared-lib-calls-exit > > find_type_with_warning() client helper did exit(1) on error, exit(1) moved to > clients. It was find_type_or_exit(), and you could've had less changes if you'd replaced it with an inline function. Basically, copied the function into the header and made it static inline. > mysql_read_default_options() did exit(1) on error, error is passed through and > handled now. ok > my_str_malloc_default() did exit(1) on error, replaced my_str_ allocator > functions with normal my_malloc()/my_realloc()/my_free(). I think you need to replace my_str_malloc_default() with my_malloc(..., MYF(MY_FAE)), not my_malloc(..., MYF(0)) > sql_connect.cc did many exit(1) on hash initialisation failure. Removed error > check since my_hash_init() never fails. > > my_malloc() did exit(1) on error. Replaced with abort(). > > my_load_defaults() did exit(1) on error, replaced with abort(). I'd say, return 2 instead of exit/abort. my_load_defaults already does it, because my_search_option_files returns 2 for OOM. > my_load_defaults() still does exit(0) when invoked with --print-defaults. dunno, return -1, for example, and exit in the caller? Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

2 1

[Maria-developers] MDEV-14031 Password policy causes replication failure
by Alexander Barkov 18 Dec '17

18 Dec '17

Hi Sergei, Please review a patch for MDEV-14031. Thanks!

1 0

Re: [Maria-developers] [Commits] 2603dc0: MDEV-11084 Select statement with partition selection against MyISAM table opens all partitions.
by Sergei Golubchik 15 Dec '17

15 Dec '17

Hi, Alexey! That looked reasonable enough. I didn't find anything big. Small comments - see below. On Jun 01, Alexey Botchkov wrote: > revision-id: 2603dc0d56ec90f308678a3c002f1b75af69f554 (mariadb-10.1.23-51-g2603dc0) > parent(s): e4d10e09cf318aad237143254c45458d16009f70 > committer: Alexey Botchkov > timestamp: 2017-06-01 17:33:34 +0400 > message: > > MDEV-11084 Select statement with partition selection against MyISAM table opens all partitions. > > Now SELECT FROM t PARTITION(x) only opens the 'x' file. > The table->partition_names parameter is sent to ha_open > so it only opens the required partitions. > If the table is reopened, the > change_partition_to_open(partition_names) is called that > closes and opens partitions specified for the query. > > diff --git a/sql/ha_partition.cc b/sql/ha_partition.cc > index 29054c7..53a128c 100644 > --- a/sql/ha_partition.cc > +++ b/sql/ha_partition.cc > @@ -3481,12 +3498,16 @@ int ha_partition::open(const char *name, int mode, uint test_if_locked) > } > name_buffer_ptr+= strlen(name_buffer_ptr) + 1; > } > + file_sample= m_file[0]; are you sure that m_file[0] is opened? > } > else > { > file= m_file; > do > { > + if (!bitmap_is_set(&m_opened_partitions, file - m_file)) > + continue; > + > create_partition_name(name_buff, name, name_buffer_ptr, NORMAL_PART_NAME, > FALSE); > table->s->connect_string = m_connect_string[(uint)(file-m_file)]; > @@ -3494,20 +3515,21 @@ int ha_partition::open(const char *name, int mode, uint test_if_locked) > test_if_locked | HA_OPEN_NO_PSI_CALL))) > goto err_handler; > bzero(&table->s->connect_string, sizeof(LEX_STRING)); > - if (m_file == file) > - m_num_locks= (*file)->lock_count(); > + if (!file_sample) > + m_num_locks= (file_sample= (*file))->lock_count(); > DBUG_ASSERT(m_num_locks == (*file)->lock_count()); > - name_buffer_ptr+= strlen(name_buffer_ptr) + 1; > - } while (*(++file)); > + } while (name_buffer_ptr+= strlen(name_buffer_ptr) + 1, *(++file)); why? > } > > file= m_file; > - ref_length= (*file)->ref_length; > - check_table_flags= (((*file)->ha_table_flags() & > + ref_length= file_sample->ref_length; > + check_table_flags= ((file_sample->ha_table_flags() & > ~(PARTITION_DISABLED_TABLE_FLAGS)) | > (PARTITION_ENABLED_TABLE_FLAGS)); > while (*(++file)) > { > + if (!bitmap_is_set(&m_opened_partitions, file - m_file)) > + continue; > /* MyISAM can have smaller ref_length for partitions with MAX_ROWS set */ > set_if_bigger(ref_length, ((*file)->ref_length)); > /* > @@ -6782,6 +6809,63 @@ void ha_partition::get_dynamic_partition_info(PARTITION_STATS *stat_info, > } > > > +void ha_partition::set_partitions_to_open(List<String> *partition_names) > +{ > + m_partitions_to_open= partition_names; > +} > + > + > +int ha_partition::change_partitions_to_open(List<String> *partition_names) > +{ > + char name_buff[FN_REFLEN]; > + handler **file; > + char *name_buffer_ptr; > + int error= 0; > + > + if (m_is_clone_of) > + return 0; > + > + m_partitions_to_open= partition_names; > + if ((error= m_part_info->set_partition_bitmaps(partition_names))) > + goto err_handler; > + > + if (bitmap_cmp(&m_opened_partitions, &m_part_info->read_partitions) != 0) > + return 0; > + > + if ((error= read_par_file(table->s->normalized_path.str))) > + goto err_handler; > + > + name_buffer_ptr= m_name_buffer_ptr; > + file= m_file; > + do > + { > + int n_file= file-m_file; > + int is_open= bitmap_is_set(&m_opened_partitions, n_file); > + int should_be_open= bitmap_is_set(&m_part_info->read_partitions, n_file); > + > + if (is_open && !should_be_open) > + (*file)->ha_close(); > + else if (!is_open && should_be_open) > + { > + create_partition_name(name_buff, table->s->normalized_path.str, > + name_buffer_ptr, NORMAL_PART_NAME, FALSE); > + table->s->connect_string = m_connect_string[(uint)(file-m_file)]; > + if ((error= (*file)->ha_open(table, name_buff, m_mode, > + m_open_test_lock | HA_OPEN_NO_PSI_CALL))) > + goto err_handler; > + bzero(&table->s->connect_string, sizeof(LEX_STRING)); > + } > + } while (name_buffer_ptr+= strlen(name_buffer_ptr) + 1, *(++file)); Could you reuse that, instead of copying from ::open? Perhaps ::open() can reset the bitmap to be empty and then call ::change_partitions_to_open() ? Or the common code could be moved into a third function that these both will use... > + > + bitmap_copy(&m_opened_partitions, &m_part_info->read_partitions); > + > + clear_handler_file(); > + > +err_handler: > + return error; > +} > + > + > /** > General function to prepare handler for certain behavior. > > diff --git a/sql/partition_info.cc b/sql/partition_info.cc > index bc0db9e..7fb8f70 100644 > --- a/sql/partition_info.cc > +++ b/sql/partition_info.cc > @@ -283,6 +281,27 @@ bool partition_info::set_partition_bitmaps(TABLE_LIST *table_list) > > > /** > + Set read/lock_partitions bitmap over non pruned partitions > + > + @param table_list Possible TABLE_LIST which can contain > + list of partition names to query > + > + @return Operation status > + @retval FALSE OK > + @retval TRUE Failed to allocate memory for bitmap or list of partitions > + did not match > + > + @note OK to call multiple times without the need for free_bitmaps. > +*/ > +bool partition_info::set_partition_bitmaps_from_table(TABLE_LIST *table_list) > +{ > + List<String> *partition_names= table_list ? > + NULL : table_list->partition_names; > + return set_partition_bitmaps(partition_names); > +} This looks unused. > + > + > +/** > Checks if possible to do prune partitions on insert. > > @param thd Thread context > diff --git a/sql/sql_base.cc b/sql/sql_base.cc > index 9c56b7d..2f9ec3e 100644 > --- a/sql/sql_base.cc > +++ b/sql/sql_base.cc > @@ -2533,6 +2533,12 @@ bool open_table(THD *thd, TABLE_LIST *table_list, Open_table_context *ot_ctx) > { > DBUG_ASSERT(table->file != NULL); > MYSQL_REBIND_TABLE(table->file); > + if (table->part_info) > + { > + /* Set all [named] partitions as used. */ > + if (table->file->change_partitions_to_open(table_list->partition_names)) > + DBUG_RETURN(true); > + } > } > else > { > @@ -2549,7 +2555,8 @@ bool open_table(THD *thd, TABLE_LIST *table_list, Open_table_context *ot_ctx) > HA_TRY_READ_ONLY), > (READ_KEYINFO | COMPUTE_TYPES | > EXTRA_RECORD), > - thd->open_options, table, FALSE); > + thd->open_options, table, FALSE, > + table_list->partition_names); Please, make sure you code compiles without partitioning too. (in this case I wouldn't make the last argument of open_table_from_share conditional, but I'd rather make table_list->partition_names unconditional). > > if (error) > { > @@ -2598,9 +2605,11 @@ bool open_table(THD *thd, TABLE_LIST *table_list, Open_table_context *ot_ctx) > #ifdef WITH_PARTITION_STORAGE_ENGINE > if (table->part_info) > { > +#ifdef CODE_TO_REMOVE > /* Set all [named] partitions as used. */ > if (table->part_info->set_partition_bitmaps(table_list)) > DBUG_RETURN(true); > +#endif /*CODE_TO_REMOVE*/ why not to remove at once? > } > else if (table_list->partition_names) > { Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] 14acb68: MDEV-14593 human-readable XA RECOVER.
by Sergei Golubchik 14 Dec '17

14 Dec '17

Hi, Alexey! On Dec 12, Alexey Botchkov wrote: > revision-id: 14acb687199f0d30444f8c0849c0d1c1284f3e2b (mariadb-10.2.11-26-g14acb68) > parent(s): 4d016e6ed2d32298977a66e125cbfcae39e23847 > committer: Alexey Botchkov > timestamp: 2017-12-12 14:31:41 +0400 > message: > > MDEV-14593 human-readable XA RECOVER. > > The 'data' field in the XA RECOVER resultset changed > to be charset_bin. It seems to me right and also > --binary-as-hex starts working. > The XA RECOVER FORMAT='SQL' option implemented. > It returns the XID string that fits to be an argument for the > XA ... statements. Looks ok, just a couple of comments: > +xa recover format='SQL'; Please add a test for format=SQL (without quotes). We normally document EXPLAIN FORMAT=JSON without quotes (although it works with quotes too), so XA RECOVER should work the same way. > +ER_UNKNOWN_XID_FORMAT > + eng "Unknown XID format name: '%s'" Don't add a new error message, reuse ER_UNKNOWN_EXPLAIN_FORMAT: - eng "Unknown EXPLAIN format name: '%s'" - rus "Неизвестное имя формата команды EXPLAIN: '%s'" + eng "Unknown %s format name: '%s'" + rus "Неизвестное имя формата команды %s: '%s'" note, you'll need either to remove/rephrase the russian error message, or use "XA RECOVER" as the first argument, not "XID". Ok to push (into bb-10.2-ext) with these changes. Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] [Commits] 8048670: MDEV-10715 Galera: Replicate MariaDB GTID to other nodes in the cluster
by Jan Lindström 12 Dec '17

12 Dec '17

On Mon, Dec 11, 2017 at 9:40 AM, sachin <sachin.setiya(a)mariadb.com> wrote: > revision-id: 8048670e25e5cda915ad7ce16dbd66afa6227aa5 > (mariadb-10.1.29-24-g8048670) > parent(s): de76cbdcb0728b80c0f7b453b24b0b81f653e450 > author: Sachin Setiya > committer: Sachin Setiya > timestamp: 2017-12-11 13:05:08 +0530 > message: > > MDEV-10715 Galera: Replicate MariaDB GTID to other nodes in the cluster > > Problem:- Gtid are not transferred in Galera Cluster. > > Solution:- We need to transfer gtid in the case on either when cluster is > slave/master in async replication. In normal Gtid replication gtid are > generated on > recieving node itself and it is always on sync with other nodes. Because > galera keeps > node in sync , So all nodes get same no of event groups. So the issue > arises when > say galera is slave in async replication. > A > | (Async replication) > D <-> E <-> F {Galera replication} > So what should happen is that all node should apply the master gtid but > this does > node happen, becuase node E, F does not recieve gtid from D in write set , > So what E(or F) > does is that it applies wsrep_gtid_domain_id, D server-id , E gtid next > seq no. This > generated gtid does not always work when say A has different domain id. > > So In this commit, on galera node when we see that this event is recieved > from master > we simply write Gtid_Log_Event in write_set and send it to other nodes. > > --- > mysql-test/suite/galera/r/galera_gtid_slave.result | 26 +++ > .../r/galera_gtid_slave_sst_mysqldump.result | 133 +++++++++++++ > .../galera/r/galera_gtid_slave_sst_rsync.result | 130 +++++++++++++ > mysql-test/suite/galera/t/galera_gtid_slave.cnf | 18 ++ > mysql-test/suite/galera/t/galera_gtid_slave.test | 78 ++++++++ > .../galera/t/galera_gtid_slave_sst_mysqldump.cnf | 19 ++ > .../galera/t/galera_gtid_slave_sst_mysqldump.test | 208 > +++++++++++++++++++++ > .../suite/galera/t/galera_gtid_slave_sst_rsync.cnf | 18 ++ > .../galera/t/galera_gtid_slave_sst_rsync.test | 207 > ++++++++++++++++++++ > mysql-test/suite/galera_3nodes/galera_2x3nodes.cnf | 122 ++++++++++++ > .../galera_3nodes/r/galera_gtid_2_cluster.result | 117 ++++++++++++ > .../galera_3nodes/t/galera_gtid_2_cluster.cnf | 28 +++ > .../galera_3nodes/t/galera_gtid_2_cluster.test | 143 ++++++++++++++ > sql/log.cc | 51 ++++- > sql/wsrep_mysqld.cc | 10 + > 15 files changed, 1306 insertions(+), 2 deletions(-) > > diff --git a/mysql-test/suite/galera/r/galera_gtid_slave.result > b/mysql-test/suite/galera/r/galera_gtid_slave.result > new file mode 100644 > index 0000000..40f3f1c > --- /dev/null > +++ b/mysql-test/suite/galera/r/galera_gtid_slave.result > @@ -0,0 +1,26 @@ > +START SLAVE; > +CREATE TABLE t1 (f1 INTEGER PRIMARY KEY) ENGINE=InnoDB; > +CREATE TABLE t2 (f1 INTEGER PRIMARY KEY) ENGINE=InnoDB; > +INSERT INTO t1 VALUES(1); > +begin; > +insert into t2 values(21); > +insert into t2 values(22); > +commit; > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-4 > +INSERT INTO t1 VALUES(2); > +INSERT INTO t1 VALUES(3); > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-4,2-2-2 > +INSERT INTO t1 VALUES(4); > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-4,2-2-2,2-3-3 > +DROP TABLE t1,t2; > +reset master; > +STOP SLAVE; > +RESET SLAVE ALL; > +reset master; > +reset master; > diff --git a/mysql-test/suite/galera/r/galera_gtid_slave_sst_mysqldump.result > b/mysql-test/suite/galera/r/galera_gtid_slave_sst_mysqldump.result > new file mode 100644 > index 0000000..78a0a17 > --- /dev/null > +++ b/mysql-test/suite/galera/r/galera_gtid_slave_sst_mysqldump.result > @@ -0,0 +1,133 @@ > +#Connection 2 > +START SLAVE; > +include/wait_for_slave_to_start.inc > +#Connection 1 > +CREATE TABLE t2 (f1 INTEGER PRIMARY KEY, f2 int unique) ENGINE=InnoDB; > +INSERT INTO t2 VALUES(1,11); > +INSERT INTO t2 VALUES(2,22); > +INSERT INTO t2 VALUES(3,33); > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-4 > +include/save_master_gtid.inc > +#Connection 2 > +include/sync_with_master_gtid.inc > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-4 > +INSERT INTO t2 VALUES(4,44); > +INSERT INTO t2 VALUES(5,55); > +INSERT INTO t2 VALUES(6,66); > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-4,2-2-3 > +#Connection 3 > +INSERT INTO t2 VALUES(7,77); > +INSERT INTO t2 VALUES(8,88); > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-4,2-2-3,2-3-5 > +#Connection 1 > +CREATE TABLE t1 (f1 CHAR(255)) ENGINE=InnoDB; > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node1_committed_before'); > +INSERT INTO t1 VALUES ('node1_committed_before'); > +COMMIT; > +include/save_master_gtid.inc > +#Connection 2 > +include/sync_with_master_gtid.inc > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node2_committed_before'); > +INSERT INTO t1 VALUES ('node2_committed_before'); > +COMMIT; > +#Connection 3 > +Shutting down server ... > +#Connection 2 > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node1_committed_during'); > +INSERT INTO t1 VALUES ('node1_committed_during'); > +COMMIT; > +#Connection 3 > +Starting server ... > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node3_committed_after'); > +INSERT INTO t1 VALUES ('node3_committed_after'); > +COMMIT; > +#Connection 2 > +Select * from t1 order by f1; > +f1 > +node1_committed_before > +node1_committed_before > +node1_committed_during > +node1_committed_during > +node2_committed_before > +node2_committed_before > +node3_committed_after > +node3_committed_after > +#Connection 3 > +Select * from t1 order by f1; > +f1 > +node1_committed_before > +node1_committed_before > +node1_committed_during > +node1_committed_during > +node2_committed_before > +node2_committed_before > +node3_committed_after > +node3_committed_after > +#Connection 2 > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-6,2-2-7,2-3-8 > +#Connection 3 > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-6,2-2-7,2-3-8 > +#Connection 1 > +SET AUTOCOMMIT=ON; > +#Connection 2 > +SET AUTOCOMMIT=ON; > +#Connection 3 > +SET AUTOCOMMIT=ON; > +#Connection 2 > +STOP slave; > +include/wait_for_slave_to_stop.inc > +INSERT INTO t1 VALUES ('node2_slave_stoped'); > +#Connection 1 > +INSERT INTO t1 VALUES ('node1_normal_entry'); > +include/save_master_gtid.inc > +#Connection 2 > +INSERT INTO t1 VALUES ('node2_slave_stoped_inserted'); > +start slave; > +include/wait_for_slave_to_start.inc > +include/sync_with_master_gtid.inc > +INSERT INTO t1 VALUES ('node2_slave_started'); > +SELECT count(*) from t1; > +count(*) > +12 > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-7,2-3-8,2-2-11 > +#Connection 3 > +SELECT count(*) from t1; > +count(*) > +12 > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-7,2-3-8,2-2-11 > +#Connection 1 > +DROP TABLE t2,t1; > +#Connection 2 > +#Connection 3 > +#Connection 2 > +STOP SLAVE; > +RESET SLAVE; > +reset master; > +#Connection 3 > +reset master; > +#Connection 1 > +reset master; > diff --git a/mysql-test/suite/galera/r/galera_gtid_slave_sst_rsync.result > b/mysql-test/suite/galera/r/galera_gtid_slave_sst_rsync.result > new file mode 100644 > index 0000000..81fae57 > --- /dev/null > +++ b/mysql-test/suite/galera/r/galera_gtid_slave_sst_rsync.result > @@ -0,0 +1,130 @@ > +#Connection 2 > +START SLAVE; > +#Connection 1 > +CREATE TABLE t2 (f1 INTEGER PRIMARY KEY, f2 int unique) ENGINE=InnoDB; > +INSERT INTO t2 VALUES(1,11); > +INSERT INTO t2 VALUES(2,22); > +INSERT INTO t2 VALUES(3,33); > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-4 > +include/save_master_gtid.inc > +#Connection 2 > +include/sync_with_master_gtid.inc > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-4 > +INSERT INTO t2 VALUES(4,44); > +INSERT INTO t2 VALUES(5,55); > +INSERT INTO t2 VALUES(6,66); > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-4,2-2-3 > +#Connection 3 > +INSERT INTO t2 VALUES(7,77); > +INSERT INTO t2 VALUES(8,88); > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-4,2-2-3,2-3-5 > +#Connection 1 > +CREATE TABLE t1 (f1 CHAR(255)) ENGINE=InnoDB; > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node1_committed_before'); > +INSERT INTO t1 VALUES ('node1_committed_before'); > +COMMIT; > +include/save_master_gtid.inc > +#Connection 2 > +include/sync_with_master_gtid.inc > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node2_committed_before'); > +INSERT INTO t1 VALUES ('node2_committed_before'); > +COMMIT; > +#Connection 3 > +Shutting down server ... > +#Connection 2 > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node1_committed_during'); > +INSERT INTO t1 VALUES ('node1_committed_during'); > +COMMIT; > +#Connection 3 > +Starting server ... > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node3_committed_after'); > +INSERT INTO t1 VALUES ('node3_committed_after'); > +COMMIT; > +#Connection 2 > +Select * from t1 order by f1; > +f1 > +node1_committed_before > +node1_committed_before > +node1_committed_during > +node1_committed_during > +node2_committed_before > +node2_committed_before > +node3_committed_after > +node3_committed_after > +#Connection 3 > +Select * from t1 order by f1; > +f1 > +node1_committed_before > +node1_committed_before > +node1_committed_during > +node1_committed_during > +node2_committed_before > +node2_committed_before > +node3_committed_after > +node3_committed_after > +#Connection 2 > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-6,2-2-7,2-3-8 > +#Connection 3 > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-6,2-2-7,2-3-8 > +#Connection 1 > +SET AUTOCOMMIT=ON; > +#Connection 2 > +SET AUTOCOMMIT=ON; > +#Connection 3 > +SET AUTOCOMMIT=ON; > +#Connection 2 > +STOP slave; > +INSERT INTO t1 VALUES ('node2_slave_stoped'); > +#Connection 1 > +INSERT INTO t1 VALUES ('node1_normal_entry'); > +include/save_master_gtid.inc > +#Connection 2 > +INSERT INTO t1 VALUES ('node2_slave_stoped_inserted'); > +start slave; > +include/sync_with_master_gtid.inc > +INSERT INTO t1 VALUES ('node2_slave_started'); > +SELECT count(*) from t1; > +count(*) > +12 > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-7,2-3-8,2-2-11 > +#Connection 3 > +SELECT count(*) from t1; > +count(*) > +12 > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-7,2-3-8,2-2-11 > +#Connection 1 > +DROP TABLE t2,t1; > +#Connection 2 > +#Connection 3 > +#Connection 2 > +STOP SLAVE; > +RESET SLAVE ALL; > +reset master; > +#Connection 3 > +reset master; > +#Connection 1 > +reset master; > diff --git a/mysql-test/suite/galera/t/galera_gtid_slave.cnf > b/mysql-test/suite/galera/t/galera_gtid_slave.cnf > new file mode 100644 > index 0000000..409d0d1 > --- /dev/null > +++ b/mysql-test/suite/galera/t/galera_gtid_slave.cnf > @@ -0,0 +1,18 @@ > +!include ../galera_2nodes_as_slave.cnf > + > +[mysqld] > +log-bin=mysqld-bin > +log-slave-updates > +binlog-format=ROW > + > +[mysqld.1] > +gtid-domain-id=1 > +[mysqld.2] > +gtid-domain-id=2 > +wsrep_gtid_mode=1 > +wsrep_gtid_domain_id=2 > +[mysqld.3] > +gtid-domain-id=2 > +wsrep_gtid_mode=1 > +wsrep_gtid_domain_id=2 > + > diff --git a/mysql-test/suite/galera/t/galera_gtid_slave.test > b/mysql-test/suite/galera/t/galera_gtid_slave.test > new file mode 100644 > index 0000000..8ef8745 > --- /dev/null > +++ b/mysql-test/suite/galera/t/galera_gtid_slave.test > @@ -0,0 +1,78 @@ > +# > +# Test Galera as a slave to a MariaDB master using GTIDs > +# > +# suite/galera/galera_2nodes_as_slave.cnf describes the setup of the > nodes > +# suite/galera/t/galera_as_slave_gtid.cnf has the GTID options > +# > +# In addition to performing DDL and DML, we check that the gtid of the > master is preserved inside the cluster > +# > + > +--source include/have_innodb.inc > + > +# As node #1 is not a Galera node, we connect to node #2 in order to run > include/galera_cluster.inc > +--connect node_2a, 127.0.0.1, root, , test, $NODE_MYPORT_2 > +--source include/galera_cluster.inc > + > +--connection node_2 > +--disable_query_log > +--eval CHANGE MASTER TO MASTER_HOST='127.0.0.1', MASTER_USER='root', > MASTER_PORT=$NODE_MYPORT_1; > +--enable_query_log > +START SLAVE; > + > +--connection node_1 > +CREATE TABLE t1 (f1 INTEGER PRIMARY KEY) ENGINE=InnoDB; > +CREATE TABLE t2 (f1 INTEGER PRIMARY KEY) ENGINE=InnoDB; > +INSERT INTO t1 VALUES(1); > + > +#multi stmt trans > +begin; > +insert into t2 values(21); > +insert into t2 values(22); > +commit; > + > +SELECT @@global.gtid_binlog_state; > + > +--connection node_2 > +--let $wait_condition = SELECT COUNT(*) = 1 FROM > INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = 't1'; > +--source include/wait_condition.inc > + > +--let $wait_condition = SELECT COUNT(*) = 1 FROM t1; > +--source include/wait_condition.inc > +--sleep 1 > +INSERT INTO t1 VALUES(2); > +INSERT INTO t1 VALUES(3); > +SELECT @@global.gtid_binlog_state; > + > +--connect node_3, 127.0.0.1, root, , test, $NODE_MYPORT_3 > +--let $wait_condition = SELECT COUNT(*) = 3 FROM t1; > +--source include/wait_condition.inc > + > +INSERT INTO t1 VALUES(4); > +SELECT @@global.gtid_binlog_state; > + > +--connection node_1 > +DROP TABLE t1,t2; > +reset master; > +# > +# Unfortunately without the sleep below the following statement fails > with "query returned no rows", which > +# is difficult to understand given that it is an aggregate query. A > "query execution was interrupted" > +# warning is also reported by MTR, which is also weird. > +# > + > +--sleep 1 > + > +--connection node_2 > +--let $wait_condition = SELECT COUNT(*) = 0 FROM > INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = 't1'; > +--source include/wait_condition.inc > + > +--connection node_3 > +--let $wait_condition = SELECT COUNT(*) = 0 FROM > INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = 't1'; > +--source include/wait_condition.inc > + > +--connection node_2 > +STOP SLAVE; > +RESET SLAVE ALL; > +reset master; > + > +--connection node_3 > +reset master; > diff --git a/mysql-test/suite/galera/t/galera_gtid_slave_sst_mysqldump.cnf > b/mysql-test/suite/galera/t/galera_gtid_slave_sst_mysqldump.cnf > new file mode 100644 > index 0000000..0b4a776 > --- /dev/null > +++ b/mysql-test/suite/galera/t/galera_gtid_slave_sst_mysqldump.cnf > @@ -0,0 +1,19 @@ > +!include ../galera_2nodes_as_slave.cnf > + > +[mysqld] > +log-bin=mysqld-bin > +log-slave-updates > +binlog-format=ROW > +wsrep_sst_method=xtrabackup > + > +[mysqld.1] > +gtid-domain-id=1 > +[mysqld.2] > +gtid-domain-id=2 > +wsrep_gtid_mode=1 > +wsrep_gtid_domain_id=2 > +[mysqld.3] > +gtid-domain-id=2 > +wsrep_gtid_mode=1 > +wsrep_gtid_domain_id=2 > + > diff --git a/mysql-test/suite/galera/t/galera_gtid_slave_sst_mysqldump.test > b/mysql-test/suite/galera/t/galera_gtid_slave_sst_mysqldump.test > new file mode 100644 > index 0000000..b507698 > --- /dev/null > +++ b/mysql-test/suite/galera/t/galera_gtid_slave_sst_mysqldump.test > @@ -0,0 +1,208 @@ > +# > +# Test Galera as a slave to a MariaDB master using GTIDs > +# > +# suite/galera/galera_2nodes_as_slave.cnf describes the setup of the > nodes > +# > +# In addition to performing DDL and DML, we check that the gtid of the > master is preserved inside the cluster > +# > + > +--source include/big_test.inc > +--source include/have_innodb.inc > +# As node #1 is not a Galera node, we connect to node #2 in order to run > include/galera_cluster.inc > +--connect node_2a, 127.0.0.1, root, , test, $NODE_MYPORT_2 > +--source include/galera_cluster.inc > + > +--echo #Connection 2 > +--connection node_2 > +--disable_query_log > +--eval CHANGE MASTER TO MASTER_HOST='127.0.0.1', MASTER_USER='root', > MASTER_PORT=$NODE_MYPORT_1,master_use_gtid=slave_pos; > +--enable_query_log > +START SLAVE; > +--source include/wait_for_slave_to_start.inc > + > +--echo #Connection 1 > +--connection node_1 > +CREATE TABLE t2 (f1 INTEGER PRIMARY KEY, f2 int unique) ENGINE=InnoDB; > +INSERT INTO t2 VALUES(1,11); > +INSERT INTO t2 VALUES(2,22); > +INSERT INTO t2 VALUES(3,33); > + > +SELECT @@global.gtid_binlog_state; > +--source include/save_master_gtid.inc > + > +--echo #Connection 2 > +--connection node_2 > +--source include/sync_with_master_gtid.inc > +SELECT @@global.gtid_binlog_state; > +INSERT INTO t2 VALUES(4,44); > +INSERT INTO t2 VALUES(5,55); > +INSERT INTO t2 VALUES(6,66); > +SELECT @@global.gtid_binlog_state; > + > +--echo #Connection 3 > +--connect node_3, 127.0.0.1, root, , test, $NODE_MYPORT_3 > +--let $wait_condition = SELECT COUNT(*) = 1 FROM > INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME= 't2'; > +--source include/wait_condition.inc > +--let $wait_condition = SELECT COUNT(*) = 6 FROM t2; > +--source include/wait_condition.inc > + > +INSERT INTO t2 VALUES(7,77); > +INSERT INTO t2 VALUES(8,88); > +SELECT @@global.gtid_binlog_state; > + > +#Perform SST > +--echo #Connection 1 > +--connection node_1 > +CREATE TABLE t1 (f1 CHAR(255)) ENGINE=InnoDB; > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node1_committed_before'); > +INSERT INTO t1 VALUES ('node1_committed_before'); > +COMMIT; > +--source include/save_master_gtid.inc > + > +--echo #Connection 2 > +--connection node_2 > +--source include/sync_with_master_gtid.inc > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node2_committed_before'); > +INSERT INTO t1 VALUES ('node2_committed_before'); > +COMMIT; > + > +--echo #Connection 3 > +--connection node_3 > +--let $wait_condition = SELECT COUNT(*) = 1 FROM > INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME= 't1'; > +--source include/wait_condition.inc > +--let $wait_condition = SELECT COUNT(*) = 4 FROM t1; > +--source include/wait_condition.inc > +--let $node_1= node_2 > +--let $node_2= node_3 > +--source include/auto_increment_offset_save.inc > +--echo Shutting down server ... > +--source include/shutdown_mysqld.inc > + > + > +--echo #Connection 2 > +--connection node_2 > +--let $wait_condition = SELECT VARIABLE_VALUE = 1 FROM > INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME = > 'wsrep_cluster_size' > +--source include/wait_condition.inc > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node1_committed_during'); > +INSERT INTO t1 VALUES ('node1_committed_during'); > +COMMIT; > + > +--echo #Connection 3 > +--connection node_3 > +--echo Starting server ... > +--source include/start_mysqld.inc > +--source include/wait_until_ready.inc > +--source include/auto_increment_offset_restore.inc > + > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node3_committed_after'); > +INSERT INTO t1 VALUES ('node3_committed_after'); > +COMMIT; > + > +--echo #Connection 2 > +--connection node_2 > +--let $wait_condition = SELECT COUNT(*) = 8 FROM t1; > +--source include/wait_condition.inc > +Select * from t1 order by f1; > + > +--echo #Connection 3 > +--connection node_3 > +Select * from t1 order by f1; > + > +#SST Done > +--sleep 1 > +--echo #Connection 2 > +--connection node_2 > +SELECT @@global.gtid_binlog_state; > + > +--echo #Connection 3 > +--connection node_3 > +SELECT @@global.gtid_binlog_state; > + > +--echo #Connection 1 > +--connection node_1 > +SET AUTOCOMMIT=ON; > +#drop table t1; > +#CREATE TABLE t1 (f1 CHAR(255)) ENGINE=InnoDB; > + > +--echo #Connection 2 > +--connection node_2 > +SET AUTOCOMMIT=ON; > +--echo #Connection 3 > +--connection node_3 > +SET AUTOCOMMIT=ON; > + > +# > +#stop slave on node 2 > +--echo #Connection 2 > +--connection node_2 > +STOP slave; > +--source include/wait_for_slave_to_stop.inc > +--sleep 1 > +INSERT INTO t1 VALUES ('node2_slave_stoped'); > + > +--echo #Connection 1 > +--connection node_1 > +INSERT INTO t1 VALUES ('node1_normal_entry'); > +--source include/save_master_gtid.inc > + > +#start slave > +--echo #Connection 2 > +--connection node_2 > +INSERT INTO t1 VALUES ('node2_slave_stoped_inserted'); > +start slave; > +--source include/wait_for_slave_to_start.inc > +--source include/sync_with_master_gtid.inc > +INSERT INTO t1 VALUES ('node2_slave_started'); > +SELECT count(*) from t1; > +SELECT @@global.gtid_binlog_state; > + > +--echo #Connection 3 > +--connection node_3 > +--let $wait_condition = SELECT COUNT(*) = 12 FROM t1; > +--source include/wait_condition.inc > +SELECT count(*) from t1; > +SELECT @@global.gtid_binlog_state; > + > +--echo #Connection 1 > +--connection node_1 > +DROP TABLE t2,t1; > + > +# Unfortunately without the sleep below the following statement fails > with "query returned no rows", which > +# is difficult to understand given that it is an aggregate query. A > "query execution was interrupted" > +# warning is also reported by MTR, which is also weird. > +# > + > +--sleep 3 > + > +--echo #Connection 2 > +--connection node_2 > +--let $wait_condition = SELECT COUNT(*) = 0 FROM > INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = 't2'; > +--source include/wait_condition.inc > + > +--echo #Connection 3 > +--connection node_3 > +--let $wait_condition = SELECT COUNT(*) = 0 FROM > INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = 't1'; > +--source include/wait_condition.inc > + > +--echo #Connection 2 > +--connection node_2 > +STOP SLAVE; > +RESET SLAVE; > +reset master; > + > +--echo #Connection 3 > +--connection node_3 > +reset master; > + > +--echo #Connection 1 > +--connection node_1 > +reset master; > + > diff --git a/mysql-test/suite/galera/t/galera_gtid_slave_sst_rsync.cnf > b/mysql-test/suite/galera/t/galera_gtid_slave_sst_rsync.cnf > new file mode 100644 > index 0000000..bb9c8e8 > --- /dev/null > +++ b/mysql-test/suite/galera/t/galera_gtid_slave_sst_rsync.cnf > @@ -0,0 +1,18 @@ > +!include ../galera_2nodes_as_slave.cnf > + > +[mysqld] > +log-bin=mysqld-bin > +log-slave-updates > +binlog-format=ROW > +wsrep_sst_method=rsync > +[mysqld.1] > +gtid-domain-id=1 > +[mysqld.2] > +gtid-domain-id=2 > +wsrep_gtid_mode=1 > +wsrep_gtid_domain_id=2 > +[mysqld.3] > +gtid-domain-id=2 > +wsrep_gtid_mode=1 > +wsrep_gtid_domain_id=2 > + > diff --git a/mysql-test/suite/galera/t/galera_gtid_slave_sst_rsync.test > b/mysql-test/suite/galera/t/galera_gtid_slave_sst_rsync.test > new file mode 100644 > index 0000000..3fe94ad > --- /dev/null > +++ b/mysql-test/suite/galera/t/galera_gtid_slave_sst_rsync.test > @@ -0,0 +1,207 @@ > +# > +# Test Galera as a slave to a MariaDB master using GTIDs > +# > +# suite/galera/galera_2nodes_as_slave.cnf describes the setup of the > nodes > +# > +# In addition to performing DDL and DML, we check that the gtid of the > master is preserved inside the cluster > +# > + > +--source include/big_test.inc > +--source include/have_innodb.inc > +# As node #1 is not a Galera node, we connect to node #2 in order to run > include/galera_cluster.inc > +--connect node_2a, 127.0.0.1, root, , test, $NODE_MYPORT_2 > +--source include/galera_cluster.inc > + > +--echo #Connection 2 > +--connection node_2 > +--disable_query_log > +--eval CHANGE MASTER TO MASTER_HOST='127.0.0.1', MASTER_USER='root', > MASTER_PORT=$NODE_MYPORT_1,master_use_gtid=slave_pos; > +--enable_query_log > +START SLAVE; > +--sleep 1 > + > + > +--echo #Connection 1 > +--connection node_1 > +CREATE TABLE t2 (f1 INTEGER PRIMARY KEY, f2 int unique) ENGINE=InnoDB; > +INSERT INTO t2 VALUES(1,11); > +INSERT INTO t2 VALUES(2,22); > +INSERT INTO t2 VALUES(3,33); > + > +SELECT @@global.gtid_binlog_state; > +--source include/save_master_gtid.inc > + > +--echo #Connection 2 > +--connection node_2 > +--source include/sync_with_master_gtid.inc > +SELECT @@global.gtid_binlog_state; > + > +INSERT INTO t2 VALUES(4,44); > +INSERT INTO t2 VALUES(5,55); > +INSERT INTO t2 VALUES(6,66); > +SELECT @@global.gtid_binlog_state; > + > +--echo #Connection 3 > +--connect node_3, 127.0.0.1, root, , test, $NODE_MYPORT_3 > +--let $wait_condition = SELECT COUNT(*) = 1 FROM > INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME= 't2'; > +--source include/wait_condition.inc > +--let $wait_condition = SELECT COUNT(*) = 6 FROM t2; > +--source include/wait_condition.inc > + > +INSERT INTO t2 VALUES(7,77); > +INSERT INTO t2 VALUES(8,88); > +SELECT @@global.gtid_binlog_state; > + > +#Perform SST > +--echo #Connection 1 > +--connection node_1 > +CREATE TABLE t1 (f1 CHAR(255)) ENGINE=InnoDB; > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node1_committed_before'); > +INSERT INTO t1 VALUES ('node1_committed_before'); > +COMMIT; > +--source include/save_master_gtid.inc > + > +--echo #Connection 2 > +--connection node_2 > +--source include/sync_with_master_gtid.inc > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node2_committed_before'); > +INSERT INTO t1 VALUES ('node2_committed_before'); > +COMMIT; > + > +--echo #Connection 3 > +--connection node_3 > +--let $wait_condition = SELECT COUNT(*) = 1 FROM > INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME= 't1'; > +--source include/wait_condition.inc > +--let $wait_condition = SELECT COUNT(*) = 4 FROM t1; > +--source include/wait_condition.inc > +--let $node_1= node_2 > +--let $node_2= node_3 > +--source include/auto_increment_offset_save.inc > +--echo Shutting down server ... > +--source include/shutdown_mysqld.inc > + > + > +--echo #Connection 2 > +--connection node_2 > +--let $wait_condition = SELECT VARIABLE_VALUE = 1 FROM > INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME = > 'wsrep_cluster_size' > +--source include/wait_condition.inc > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node1_committed_during'); > +INSERT INTO t1 VALUES ('node1_committed_during'); > +COMMIT; > + > +--echo #Connection 3 > +--connection node_3 > +--echo Starting server ... > +--source include/start_mysqld.inc > +--source include/wait_until_ready.inc > +--source include/auto_increment_offset_restore.inc > + > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node3_committed_after'); > +INSERT INTO t1 VALUES ('node3_committed_after'); > +COMMIT; > + > +--echo #Connection 2 > +--connection node_2 > +--let $wait_condition = SELECT COUNT(*) = 8 FROM t1; > +--source include/wait_condition.inc > +Select * from t1 order by f1; > + > +--echo #Connection 3 > +--connection node_3 > +Select * from t1 order by f1; > + > +#SST Done > +--sleep 1 > +--echo #Connection 2 > +--connection node_2 > +SELECT @@global.gtid_binlog_state; > + > +--echo #Connection 3 > +--connection node_3 > +SELECT @@global.gtid_binlog_state; > + > +--echo #Connection 1 > +--connection node_1 > +SET AUTOCOMMIT=ON; > +#drop table t1; > +#CREATE TABLE t1 (f1 CHAR(255)) ENGINE=InnoDB; > + > +--echo #Connection 2 > +--connection node_2 > +SET AUTOCOMMIT=ON; > +--echo #Connection 3 > +--connection node_3 > +SET AUTOCOMMIT=ON; > + > +# > +#stop slave on node 2 > +--echo #Connection 2 > +--connection node_2 > +STOP slave; > +--sleep 1 > +INSERT INTO t1 VALUES ('node2_slave_stoped'); > + > +--echo #Connection 1 > +--connection node_1 > +INSERT INTO t1 VALUES ('node1_normal_entry'); > +--source include/save_master_gtid.inc > + > +#start slave > +--echo #Connection 2 > +--connection node_2 > +INSERT INTO t1 VALUES ('node2_slave_stoped_inserted'); > +start slave; > +--source include/sync_with_master_gtid.inc > +INSERT INTO t1 VALUES ('node2_slave_started'); > +SELECT count(*) from t1; > +SELECT @@global.gtid_binlog_state; > + > +--echo #Connection 3 > +--connection node_3 > +--let $wait_condition = SELECT COUNT(*) = 12 FROM t1; > +--source include/wait_condition.inc > +SELECT count(*) from t1; > +SELECT @@global.gtid_binlog_state; > + > +--echo #Connection 1 > +--connection node_1 > +DROP TABLE t2,t1; > + > +# Unfortunately without the sleep below the following statement fails > with "query returned no rows", which > +# is difficult to understand given that it is an aggregate query. A > "query execution was interrupted" > +# warning is also reported by MTR, which is also weird. > +# > + > +--sleep 3 > + > +--echo #Connection 2 > +--connection node_2 > +--let $wait_condition = SELECT COUNT(*) = 0 FROM > INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = 't2'; > +--source include/wait_condition.inc > + > +--echo #Connection 3 > +--connection node_3 > +--let $wait_condition = SELECT COUNT(*) = 0 FROM > INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = 't1'; > +--source include/wait_condition.inc > + > +--echo #Connection 2 > +--connection node_2 > +STOP SLAVE; > +RESET SLAVE ALL; > +reset master; > + > +--echo #Connection 3 > +--connection node_3 > +reset master; > + > +--echo #Connection 1 > +--connection node_1 > +reset master; > diff --git a/mysql-test/suite/galera_3nodes/galera_2x3nodes.cnf > b/mysql-test/suite/galera_3nodes/galera_2x3nodes.cnf > new file mode 100644 > index 0000000..3f39b82 > --- /dev/null > +++ b/mysql-test/suite/galera_3nodes/galera_2x3nodes.cnf > @@ -0,0 +1,122 @@ > +# Use default setting for mysqld processes > +!include include/default_mysqld.cnf > + > +[mysqld] > +log-bin > +log-slave-updates > +binlog-format=row > +innodb-autoinc-lock-mode=2 > +default-storage-engine=innodb > +wsrep_gtid_mode=1 > +gtid_ignore_duplicates > + > +wsrep-on=1 > +wsrep-provider=(a)ENV.WSREP_PROVIDER > +wsrep_node_address=127.0.0.1 > +# enforce read-committed characteristics across the cluster > +# wsrep-causal-reads=ON > +# wsrep-sync-wait=15 > + > +[mysqld.1] > +#galera_port=(a)OPT.port > +#ist_port=(a)OPT.port > +#sst_port=(a)OPT.port > +wsrep-cluster-address='gcomm://' > +wsrep_provider_options='base_port=(a)mysqld.1.#galera_port; > evs.suspect_timeout=PT300S;evs.inactive_timeout=PT1000M; > evs.install_timeout=PT155S;evs.keepalive_period = PT100S' > + > +wsrep_sst_receive_address=127.0.0.2:@mysqld.1.#sst_port > +wsrep_node_incoming_address=127.0.0.1:@mysqld.1.port > +wsrep_sst_receive_address='127.0.0.1:@mysqld.1.#sst_port' > + > +[mysqld.2] > +#galera_port=(a)OPT.port > +#ist_port=(a)OPT.port > +#sst_port=(a)OPT.port > +wsrep_cluster_address='gcomm://127.0.0.1:@mysqld.1.#galera_port' > +wsrep_provider_options='base_port=(a)mysqld.2.#galera_port; > evs.suspect_timeout=PT300S;evs.inactive_timeout=PT1000M; > evs.install_timeout=PT155S;evs.keepalive_period = PT100S' > + > +wsrep_sst_receive_address=127.0.0.2:@mysqld.2.#sst_port > +wsrep_node_incoming_address=127.0.0.1:@mysqld.2.port > +wsrep_sst_receive_address='127.0.0.1:@mysqld.2.#sst_port' > + > +[mysqld.3] > +#galera_port=(a)OPT.port > +#ist_port=(a)OPT.port > +#sst_port=(a)OPT.port > +wsrep_cluster_address='gcomm://127.0.0.1:@mysqld.1.#galera_port' > +wsrep_provider_options='base_port=(a)mysqld.3.#galera_port; > evs.suspect_timeout=PT300S;evs.inactive_timeout=PT1000M; > evs.install_timeout=PT155S;evs.keepalive_period = PT100S' > + > +wsrep_sst_receive_address=127.0.0.2:@mysqld.3.#sst_port > +wsrep_node_incoming_address=127.0.0.1:@mysqld.3.port > +wsrep_sst_receive_address='127.0.0.1:@mysqld.3.#sst_port' > + > + > +[mysqld.4] > +wsrep_cluster_name=cluster2 > +#galera_port=(a)OPT.port > +#ist_port=(a)OPT.port > +#sst_port=(a)OPT.port > + > +wsrep-cluster-address='gcomm://' > +wsrep_provider_options='base_port=(a)mysqld.4.#galera_port; > evs.suspect_timeout=PT300S;evs.inactive_timeout=PT1000M; > evs.install_timeout=PT155S;evs.keepalive_period = PT100S' > + > +wsrep_sst_receive_address=127.0.0.2:@mysqld.4.#sst_port > +wsrep_node_incoming_address=127.0.0.1:@mysqld.4.port > +wsrep_sst_receive_address='127.0.0.1:@mysqld.4.#sst_port' > + > +[mysqld.5] > +wsrep_cluster_name=cluster2 > +#galera_port=(a)OPT.port > +#ist_port=(a)OPT.port > +#sst_port=(a)OPT.port > +wsrep_cluster_address='gcomm://127.0.0.1:@mysqld.4.#galera_port' > +wsrep_provider_options='base_port=(a)mysqld.5.#galera_port; > evs.suspect_timeout=PT300S;evs.inactive_timeout=PT1000M; > evs.install_timeout=PT155S;evs.keepalive_period = PT100S' > + > +wsrep_sst_receive_address=127.0.0.2:@mysqld.5.#sst_port > +wsrep_node_incoming_address=127.0.0.1:@mysqld.5.port > +wsrep_sst_receive_address='127.0.0.1:@mysqld.5.#sst_port' > + > +[mysqld.6] > +wsrep_cluster_name=cluster2 > +#galera_port=(a)OPT.port > +#ist_port=(a)OPT.port > +#sst_port=(a)OPT.port > +wsrep_cluster_address='gcomm://127.0.0.1:@mysqld.4.#galera_port' > +wsrep_provider_options='base_port=(a)mysqld.6.#galera_port; > evs.suspect_timeout=PT300S;evs.inactive_timeout=PT1000M; > evs.install_timeout=PT155S;evs.keepalive_period = PT100S' > + > +wsrep_sst_receive_address=127.0.0.2:@mysqld.6.#sst_port > +wsrep_node_incoming_address=127.0.0.1:@mysqld.6.port > +wsrep_sst_receive_address='127.0.0.1:@mysqld.6.#sst_port' > + > +[ENV] > +NODE_MYPORT_1= @mysqld.1.port > +NODE_MYSOCK_1= @mysqld.1.socket > + > +NODE_MYPORT_2= @mysqld.2.port > +NODE_MYSOCK_2= @mysqld.2.socket > + > +NODE_MYPORT_3= @mysqld.3.port > +NODE_MYSOCK_3= @mysqld.3.socket > + > +NODE_MYPORT_4= @mysqld.4.port > +NODE_MYSOCK_4= @mysqld.4.socket > + > +NODE_MYPORT_5= @mysqld.5.port > +NODE_MYSOCK_5= @mysqld.5.socket > + > +NODE_MYPORT_6= @mysqld.6.port > +NODE_MYSOCK_6= @mysqld.6.socket > + > +NODE_GALERAPORT_1= @mysqld.1.#galera_port > +NODE_GALERAPORT_2= @mysqld.2.#galera_port > +NODE_GALERAPORT_3= @mysqld.3.#galera_port > +NODE_GALERAPORT_4= @mysqld.4.#galera_port > +NODE_GALERAPORT_5= @mysqld.5.#galera_port > +NODE_GALERAPORT_6= @mysqld.6.#galera_port > + > +NODE_SSTPORT_1= @mysqld.1.#sst_port > +NODE_SSTPORT_2= @mysqld.2.#sst_port > +NODE_SSTPORT_3= @mysqld.3.#sst_port > +NODE_SSTPORT_4= @mysqld.4.#sst_port > +NODE_SSTPORT_5= @mysqld.5.#sst_port > +NODE_SSTPORT_6= @mysqld.6.#sst_port > diff --git a/mysql-test/suite/galera_3nodes/r/galera_gtid_2_cluster.result > b/mysql-test/suite/galera_3nodes/r/galera_gtid_2_cluster.result > new file mode 100644 > index 0000000..52c82ab > --- /dev/null > +++ b/mysql-test/suite/galera_3nodes/r/galera_gtid_2_cluster.result > @@ -0,0 +1,117 @@ > +cluster 1 node 1 > +SHOW STATUS LIKE 'wsrep_cluster_size'; > +Variable_name Value > +wsrep_cluster_size 3 > +cluster 1 node 2 > +SHOW STATUS LIKE 'wsrep_cluster_size'; > +Variable_name Value > +wsrep_cluster_size 3 > +cluster 1 node 3 > +SHOW STATUS LIKE 'wsrep_cluster_size'; > +Variable_name Value > +wsrep_cluster_size 3 > +cluster 2 node 1 > +SHOW STATUS LIKE 'wsrep_cluster_size'; > +Variable_name Value > +wsrep_cluster_size 3 > +cluster 2 node 2 > +SHOW STATUS LIKE 'wsrep_cluster_size'; > +Variable_name Value > +wsrep_cluster_size 3 > +cluster 2 node 3 > +SHOW STATUS LIKE 'wsrep_cluster_size'; > +Variable_name Value > +wsrep_cluster_size 3 > +change master to master_host='127.0.0.1', master_user='root', > master_port=16003, master_use_gtid=current_pos, ignore_server_ids=(12,13);; > +start slave; > +include/wait_for_slave_to_start.inc > +select @@gtid_binlog_state; > +@@gtid_binlog_state > + > +select @@gtid_slave_pos; > +@@gtid_slave_pos > + > +change master to master_host='127.0.0.1', master_user='root', > master_port=16000, master_use_gtid=current_pos, ignore_server_ids=(22,23);; > +start slave; > +include/wait_for_slave_to_start.inc > +select @@gtid_binlog_state; > +@@gtid_binlog_state > + > +select @@gtid_slave_pos; > +@@gtid_slave_pos > + > +cluster 1 node 1 > +create table t1 (cluster_domain_id int ,node_server_id int, seq_no int); > +insert into t1 values (1, 11, 2); > +select @@gtid_binlog_state; > +@@gtid_binlog_state > +1-11-2 > +#wait for sync cluster 1 and 2 > +include/save_master_gtid.inc > +include/sync_with_master_gtid.inc > +cluster 2 node 1 > +insert into t1 values (2, 21, 1); > +select @@gtid_binlog_state; > +@@gtid_binlog_state > +1-11-2,2-21-1 > +select * from t1; > +cluster_domain_id node_server_id seq_no > +1 11 2 > +2 21 1 > +#wait for sync cluster 1 and 2 > +include/save_master_gtid.inc > +include/sync_with_master_gtid.inc > +cluster 1 node 2 > +select @@gtid_binlog_state; > +@@gtid_binlog_state > +1-11-2,2-21-1 > +insert into t1 values (1, 12, 3); > +select @@gtid_binlog_state; > +@@gtid_binlog_state > +1-11-2,1-12-3,2-21-1 > +#wait for sync cluster 1 and 2 > +include/save_master_gtid.inc > +include/sync_with_master_gtid.inc > +cluster 1 node 3 > +select @@gtid_binlog_state; > +@@gtid_binlog_state > +1-11-2,1-12-3,2-21-1 > +insert into t1 values (1, 13, 4); > +select @@gtid_binlog_state; > +@@gtid_binlog_state > +1-12-3,1-11-2,1-13-4,2-21-1 > +#wait for sync cluster 1 and 2 > +include/save_master_gtid.inc > +include/sync_with_master_gtid.inc > +cluster 2 node 2 > +select @@gtid_binlog_state; > +@@gtid_binlog_state > +1-12-3,1-11-2,1-13-4,2-21-1 > +insert into t1 values (2, 22, 2); > +select @@gtid_binlog_state; > +@@gtid_binlog_state > +1-12-3,1-11-2,1-13-4,2-21-1,2-22-2 > +#wait for sync cluster 2 and 1 > +include/save_master_gtid.inc > +include/sync_with_master_gtid.inc > +cluster 2 node 3 > +select @@gtid_binlog_state; > +@@gtid_binlog_state > +1-12-3,1-11-2,1-13-4,2-21-1,2-22-2 > +insert into t1 values (2, 23, 3); > +select @@gtid_binlog_state; > +@@gtid_binlog_state > +1-12-3,1-11-2,1-13-4,2-21-1,2-22-2,2-23-3 > +#wait for sync cluster 2 and 1 > +include/save_master_gtid.inc > +include/sync_with_master_gtid.inc > +cluster 1 node 1 > +select @@gtid_binlog_state; > +@@gtid_binlog_state > +1-12-3,1-11-2,1-13-4,2-21-1,2-22-2,2-23-3 > +drop table t1; > +stop slave; > +change master to master_use_gtid=no, ignore_server_ids=(); > +cluster 2 node 1 > +stop slave; > +change master to master_use_gtid=no, ignore_server_ids=(); > diff --git a/mysql-test/suite/galera_3nodes/t/galera_gtid_2_cluster.cnf > b/mysql-test/suite/galera_3nodes/t/galera_gtid_2_cluster.cnf > new file mode 100644 > index 0000000..dc5535e > --- /dev/null > +++ b/mysql-test/suite/galera_3nodes/t/galera_gtid_2_cluster.cnf > @@ -0,0 +1,28 @@ > +# We need a dedicated .cnf file, even if empty, in order to force this > test to run > +# alone on a freshly started cluster. Otherwise there are adverse > interactions with > +# following tests such as galera_3nodes.galera_var_dirty_reads2 > + > +!include ../galera_2x3nodes.cnf > +[mysqld.1] > +wsrep_gtid_domain_id=1 > +server-id=11 > + > +[mysqld.2] > +wsrep_gtid_domain_id=1 > +server-id=12 > + > +[mysqld.3] > +wsrep_gtid_domain_id=1 > +server-id=13 > + > +[mysqld.4] > +wsrep_gtid_domain_id=2 > +server-id=21 > + > +[mysqld.5] > +wsrep_gtid_domain_id=2 > +server-id=22 > + > +[mysqld.6] > +wsrep_gtid_domain_id=2 > +server-id=23 > diff --git a/mysql-test/suite/galera_3nodes/t/galera_gtid_2_cluster.test > b/mysql-test/suite/galera_3nodes/t/galera_gtid_2_cluster.test > new file mode 100644 > index 0000000..4c9e611 > --- /dev/null > +++ b/mysql-test/suite/galera_3nodes/t/galera_gtid_2_cluster.test > @@ -0,0 +1,143 @@ > +# > +# This test creates 2x 3 nodes galera cluster. > +# The whole test case > +# A <-> B <-> C {Galera cluster 1} > +# | {Circular Async replication} > +# D <-> E <-> F {Galera cluster 2} > +# We will write on any random node to see if gtid is consitent or not > +# Then we will kill node D and set up the replication between A and E > +# To see whether fail over works or not. > + > +--source include/big_test.inc > +--source include/galera_cluster.inc > +--source include/have_innodb.inc > + > +--connection node_1 > +--echo cluster 1 node 1 > +SHOW STATUS LIKE 'wsrep_cluster_size'; > + > +--connection node_2 > +--echo cluster 1 node 2 > +SHOW STATUS LIKE 'wsrep_cluster_size'; > + > +--connect node_3, 127.0.0.1, root, , test, $NODE_MYPORT_3 > +--connection node_3 > +--echo cluster 1 node 3 > +SHOW STATUS LIKE 'wsrep_cluster_size'; > + > +--connect node_4, 127.0.0.1, root, , test, $NODE_MYPORT_4 > +--connection node_4 > +--echo cluster 2 node 1 > +SHOW STATUS LIKE 'wsrep_cluster_size'; > + > +--connect node_5, 127.0.0.1, root, , test, $NODE_MYPORT_5 > +--connection node_5 > +--echo cluster 2 node 2 > +SHOW STATUS LIKE 'wsrep_cluster_size'; > + > +--connect node_6, 127.0.0.1, root, , test, $NODE_MYPORT_6 > +--connection node_6 > +--echo cluster 2 node 3 > +SHOW STATUS LIKE 'wsrep_cluster_size'; > + > +--connection node_1 > +--eval change master to master_host='127.0.0.1', master_user='root', > master_port=$NODE_MYPORT_4, master_use_gtid=current_pos, > ignore_server_ids=(12,13); > +start slave; > +--source include/wait_for_slave_to_start.inc > +select @@gtid_binlog_state; > +select @@gtid_slave_pos; > +#--query_vertical SHOW SLAVE STATUS; > + > +--connection node_4 > +--eval change master to master_host='127.0.0.1', master_user='root', > master_port=$NODE_MYPORT_1, master_use_gtid=current_pos, > ignore_server_ids=(22,23); > +start slave; > +--source include/wait_for_slave_to_start.inc > +select @@gtid_binlog_state; > +select @@gtid_slave_pos; > +#--query_vertical SHOW SLAVE STATUS; > + > +--echo cluster 1 node 1 > +--connection node_1 > +create table t1 (cluster_domain_id int ,node_server_id int, seq_no int); > +insert into t1 values (1, 11, 2); > +select @@gtid_binlog_state; > + > +--echo #wait for sync cluster 1 and 2 > +--connection node_1 > +--source include/save_master_gtid.inc > +--connection node_4 > +--source include/sync_with_master_gtid.inc > + > +--echo cluster 2 node 1 > +--connection node_4 > +insert into t1 values (2, 21, 1); > +select @@gtid_binlog_state; > +select * from t1; > + > +--echo #wait for sync cluster 1 and 2 > +--connection node_1 > +--source include/save_master_gtid.inc > +--connection node_4 > +--source include/sync_with_master_gtid.inc > + > + > +--echo cluster 1 node 2 > +--connection node_2 > +select @@gtid_binlog_state; > +insert into t1 values (1, 12, 3); > +select @@gtid_binlog_state; > + > +--echo #wait for sync cluster 1 and 2 > +--connection node_1 > +--source include/save_master_gtid.inc > +--connection node_4 > +--source include/sync_with_master_gtid.inc > + > +--echo cluster 1 node 3 > +--connection node_3 > +select @@gtid_binlog_state; > +insert into t1 values (1, 13, 4); > +select @@gtid_binlog_state; > + > +--echo #wait for sync cluster 1 and 2 > +--connection node_1 > +--source include/save_master_gtid.inc > +--connection node_4 > +--source include/sync_with_master_gtid.inc > + > +--echo cluster 2 node 2 > +--connection node_5 > +select @@gtid_binlog_state; > +insert into t1 values (2, 22, 2); > +select @@gtid_binlog_state; > + > +--echo #wait for sync cluster 2 and 1 > +--connection node_4 > +--source include/save_master_gtid.inc > +--connection node_1 > +--source include/sync_with_master_gtid.inc > + > +--echo cluster 2 node 3 > +--connection node_6 > +select @@gtid_binlog_state; > +insert into t1 values (2, 23, 3); > +select @@gtid_binlog_state; > + > +--echo #wait for sync cluster 2 and 1 > +--connection node_4 > +--source include/save_master_gtid.inc > +--connection node_1 > +--source include/sync_with_master_gtid.inc > + > + > +--echo cluster 1 node 1 > +--connection node_1 > +select @@gtid_binlog_state; > +drop table t1; > +stop slave; > +change master to master_use_gtid=no, ignore_server_ids=(); > + > +--echo cluster 2 node 1 > +--connection node_4 > +stop slave; > +change master to master_use_gtid=no, ignore_server_ids=(); > diff --git a/sql/log.cc b/sql/log.cc > index 07d4074..bb8b1d3 100644 > --- a/sql/log.cc > +++ b/sql/log.cc > @@ -5544,7 +5544,36 @@ THD::binlog_start_trans_and_stmt() > cache_mngr->trx_cache.get_prev_position() == MY_OFF_T_UNDEF) > { > this->binlog_set_stmt_begin(); > - if (in_multi_stmt_transaction_mode()) > + bool mstmt_mode= in_multi_stmt_transaction_mode(); > +#ifdef WITH_WSREP > + /* Write Gtid > + Get domain id only when gtid mode is set > + If this event is replicate through a master then , > + we will forward the same gtid another nodes > + We have to do this only one time in mysql transaction. > + Since this function is called multiple times , We will check for > + ha_info->is_started() > + */ > + Ha_trx_info *ha_info; > + ha_info= this->ha_data[binlog_hton->slot].ha_info + (mstmt_mode ? 1 > : 0); > + > + if (!ha_info->is_started() && wsrep_gtid_mode && > this->variables.gtid_seq_no) > + { > + binlog_cache_mngr *const cache_mngr= > + (binlog_cache_mngr*) thd_get_ha_data(this, binlog_hton); > + > + IO_CACHE *file= > + cache_mngr->get_binlog_cache_log(use_trans_cache(this, true)); > + Log_event_writer writer(file); > + Gtid_log_event gtid_event(this, this->variables.gtid_seq_no, > + this->variables.gtid_domain_id, > + true, LOG_EVENT_SUPPRESS_USE_F, > + true, 0); > + gtid_event.server_id= this->variables.server_id; > + writer.write(&gtid_event); > + } > +#endif > + if (mstmt_mode) > trans_register_ha(this, TRUE, binlog_hton); > trans_register_ha(this, FALSE, binlog_hton); > /* > @@ -5826,7 +5855,7 @@ MYSQL_BIN_LOG::write_gtid_event(THD *thd, bool > standalone, > DBUG_PRINT("enter", ("standalone: %d", standalone)); > > #ifdef WITH_WSREP > - if (WSREP(thd) && thd->wsrep_trx_meta.gtid.seqno != -1 && > wsrep_gtid_mode) > + if (WSREP(thd) && thd->wsrep_trx_meta.gtid.seqno != -1 && > wsrep_gtid_mode && !thd->variables.gtid_seq_no) > { > domain_id= wsrep_gtid_domain_id; > } else { > @@ -6840,6 +6869,24 @@ int MYSQL_BIN_LOG::write_cache(THD *thd, IO_CACHE > *cache) > mysql_mutex_assert_owner(&LOCK_log); > if (reinit_io_cache(cache, READ_CACHE, 0, 0, 0)) > DBUG_RETURN(ER_ERROR_ON_WRITE); > + uchar *read_pos= cache->read_pos; > + > +#ifdef WITH_WSREP > + /* > + In the cache we have gtid event if , below condition is true, > + then we will simply increment the read position of cache by > + the length of Gtid_log_event. > + */ > + if (wsrep_gtid_mode && thd->variables.gtid_seq_no) > + { > + uchar *head= read_pos; > + uint data_len= uint4korr(head + EVENT_LEN_OFFSET); > + uint event_type= (uchar)head[EVENT_TYPE_OFFSET]; > + if (event_type == GTID_LOG_EVENT) > + cache->read_pos+= data_len; > + } > +#endif > + > uint length= my_b_bytes_in_cache(cache), group, carry, hdr_offs; > long val; > ulong end_log_pos_inc= 0; // each event processed adds > BINLOG_CHECKSUM_LEN 2 t > diff --git a/sql/wsrep_mysqld.cc b/sql/wsrep_mysqld.cc > index bdc8491..285bb52 100644 > --- a/sql/wsrep_mysqld.cc > +++ b/sql/wsrep_mysqld.cc > @@ -1227,6 +1227,16 @@ int wsrep_to_buf_helper( > if (!ret && writer.write(&gtid_ev)) ret= 1; > } > #endif /* GTID_SUPPORT */ > + if (wsrep_gtid_mode && thd->variables.gtid_seq_no) > + { > + Gtid_log_event gtid_event(thd, thd->variables.gtid_seq_no, > + thd->variables.gtid_domain_id, > + true, LOG_EVENT_SUPPRESS_USE_F, > + true, 0); > + gtid_event.server_id= thd->variables.server_id; > + if (!gtid_event.is_valid()) ret= 0; > + ret= writer.write(&gtid_event); > + } > > /* if there is prepare query, add event for it */ > if (!ret && thd->wsrep_TOI_pre_query) > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits

1 0

[Maria-developers] Issue with commit 0f43279c / MDEV-13936
by jerome brauge 12 Dec '17

12 Dec '17

Hi All, Since some time ago, my app crashed the server (10.2.-ext) in some case (repeatable) with the following stack trace : On Linux Centos 7: ./sql/mysqld(my_print_stacktrace+0x38)[0x7f463f49a093] ./sql/mysqld(handle_fatal_signal+0x3a3)[0x7f463ed2747a] /lib64/libpthread.so.0(+0xf370)[0x7f463dffd370] ./sql/mysqld(_ZN13Explain_query10get_selectEj+0x40)[0x7f463ebf4d06] ./sql/mysqld(_ZN13Explain_query8add_nodeEP12Explain_node+0x203)[0x7f463ebf4f15] ./sql/mysqld(_ZN4JOIN24save_explain_data_internEP13Explain_querybbbPKc+0x959)[0x7f463eaf3db9] ./sql/mysqld(_ZN4JOIN17save_explain_dataEP13Explain_querybbbb+0x14b)[0x7f463eabb963] ./sql/mysqld(_ZN4JOIN13build_explainEv+0xe2)[0x7f463eab3b70] ./sql/mysqld(_ZN4JOIN8optimizeEv+0xb6)[0x7f463eab3d2e] ./sql/mysqld(_ZN13st_select_lex31optimize_unflattened_subqueriesEb+0x263)[0x7f463ea57b5d] ./sql/mysqld(_ZN4JOIN31optimize_unflattened_subqueriesEv+0x24)[0x7f463ec1b746] ./sql/mysqld(_ZN4JOIN15optimize_stage2Ev+0x1cdb)[0x7f463eab7171] ./sql/mysqld(_ZN4JOIN14optimize_innerEv+0x14dd)[0x7f463eab542d] ./sql/mysqld(_ZN4JOIN8optimizeEv+0x7a)[0x7f463eab3cf2] ./sql/mysqld(_Z12mysql_selectP3THDP10TABLE_LISTjR4ListI4ItemEPS4_jP8st_orderS9_S7_S9_yP13select_resultP18st_select_lex_unitP13st_select_lex+0x263)[0x7f463eabccd2] ./sql/mysqld(_Z13handle_selectP3THDP3LEXP13select_resultm+0x16c)[0x7f463eab13e6] ./sql/mysqld(_Z21mysql_execute_commandP3THD+0x4b5b)[0x7f463ea76fb4] ./sql/mysqld(_ZN13sp_instr_stmt9exec_coreEP3THDPj+0x80)[0x7f463eeaafb0] ./sql/mysqld(_ZN13sp_lex_keeper23reset_lex_and_exec_coreEP3THDPjbP8sp_instr+0x1d1)[0x7f463eeaa549] ./sql/mysqld(_ZN13sp_instr_stmt7executeEP3THDPj+0x1b5)[0x7f463eeaabf9] ./sql/mysqld(_ZN7sp_head7executeEP3THDb+0x754)[0x7f463eea4ff6] ./sql/mysqld(_ZN7sp_head17execute_procedureEP3THDP4ListI4ItemE+0x971)[0x7f463eea74df] ./sql/mysqld(+0x645959)[0x7f463ea71959] ./sql/mysqld(_ZN12Sql_cmd_call7executeEP3THD+0x270)[0x7f463ea7240c] ./sql/mysqld(_Z21mysql_execute_commandP3THD+0x9f95)[0x7f463ea7c3ee] ./sql/mysqld(_ZN18Prepared_statement7executeEP6Stringb+0x508)[0x7f463ea9e468] ./sql/mysqld(_ZN18Prepared_statement12execute_loopEP6StringbPhS2_+0x200)[0x7f463ea9c6fa] ./sql/mysqld(+0x66e113)[0x7f463ea9a113] ./sql/mysqld(_Z19mysqld_stmt_executeP3THDPcj+0x97)[0x7f463ea99cd7] ./sql/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcjbb+0xc9e)[0x7f463ea6e68d] ./sql/mysqld(_Z10do_commandP3THD+0x700)[0x7f463ea6d315] ./sql/mysqld(_Z24do_handle_one_connectionP7CONNECT+0x1ca)[0x7f463ebbe3d2] ./sql/mysqld(handle_one_connection+0x30)[0x7f463ebbe125] ./sql/mysqld(+0x1012852)[0x7f463f43e852] /lib64/libpthread.so.0(+0x7dc5)[0x7f463dff5dc5] /lib64/libc.so.6(clone+0x6d)[0x7f463c01673d] Or on Windows : Unhandled exception thrown: read access violation. Dynamic_array<Explain_select *>::at(...) returned 0xDB925000. > mysqld.exe!Explain_query::~Explain_query() Line 71 C++ mysqld.exe!Explain_query::`scalar deleting destructor'(unsigned int) C++ mysqld.exe!delete_explain_query(LEX * lex) Line 2397 C++ mysqld.exe!sp_lex_keeper::reset_lex_and_exec_core(THD * thd, unsigned int * nextp, bool open_tables, sp_instr * instr) Line 3130 C++ mysqld.exe!sp_instr_stmt::execute(THD * thd, unsigned int * nextp) Line 3266 C++ mysqld.exe!sp_head::execute(THD * thd, bool merge_da_on_success) Line 1213 C++ mysqld.exe!sp_head::execute_procedure(THD * thd, List<Item> * args) Line 2072 C++ mysqld.exe!do_execute_sp(THD * thd, sp_head * sp) Line 2907 C++ mysqld.exe!Sql_cmd_call::execute(THD * thd) Line 3147 C++ mysqld.exe!mysql_execute_command(THD * thd) Line 6223 C++ mysqld.exe!Prepared_statement::execute(String * expanded_query, bool open_cursor) Line 4682 C++ mysqld.exe!Prepared_statement::execute_loop(String * expanded_query, bool open_cursor, unsigned char * packet, unsigned char * packet_end) Line 4111 C++ mysqld.exe!mysql_stmt_execute_common(THD * thd, unsigned long stmt_id, unsigned char * packet, unsigned char * packet_end, unsigned long cursor_flags, bool bulk_op, bool read_types) Line 3132 C++ mysqld.exe!mysqld_stmt_execute(THD * thd, char * packet_arg, unsigned int packet_length) Line 3028 C++ mysqld.exe!dispatch_command(enum_server_command command, THD * thd, char * packet, unsigned int packet_length, bool is_com_multi, bool is_next_command) Line 1763 C++ mysqld.exe!do_command(THD * thd) Line 1369 C++ mysqld.exe!threadpool_process_request(THD * thd) Line 366 C++ mysqld.exe!tp_callback(TP_connection * c) Line 192 C++ mysqld.exe!tp_callback(_TP_CALLBACK_INSTANCE * instance, void * context) Line 377 C++ mysqld.exe!work_callback(_TP_CALLBACK_INSTANCE * instance, void * context, _TP_WORK * work) Line 451 C++ The sql statement involved (in a stored procedure): Query (0x7f457df65bb8): INSERT INTO qnwtc( enrqnwtc, priqnwtc, lprqnwtc, cprqnwtc, nuiqnwtc) SELECT 'PRI' ,occgtpar ,intgtpar ,ch1gtpar , NAME_CONST('wpio_ident',110325)FROM gtpara WHERE padgtpar = 'PRIQNTAC' AND numgtets = NAME_CONST('wsysets',_utf8'IFR' COLLATE 'utf8_bin')AND etagtpar = 'A' Where gtpara is a view. If the stored procedure is called alone, it works fine. I've revert commit 0f43279c (MDEV-13936) and now my app works fine again. Regards.

1 0

Re: [Maria-developers] Mdev-10715 Initial Patch
by Sachin Setiya 11 Dec '17

11 Dec '17

Hi Andrei! Thanks for the review. On Thu, Oct 12, 2017 at 11:48 PM, <andrei.elkin(a)pp.inet.fi> wrote: > Howdy, Sachin! > > I've checked the patch mostly from the high-level design pov to gather > few suggestions. Find them on the top of diff hunks (hopefully the diff > remains applicable) below indented with '*review*>'. > > > > revision-id: daf9a67ea13c5c7309f85a104e7c07ec68f65e88 (mariadb-10.1.28-10-gdaf9a67) > parent(s): 5eb666ad37ff80e5f8dc714bb68fcb1e269b797a > author: Sachin Setiya > committer: Sachin Setiya > timestamp: 2017-10-10 17:45:21 +0530 > message: > > Mdev-10715 Initial Patch > *review*> The initial patch is a very good time to write down the work > intent and its key features. Done > Could you please do that and highlight how WSREP has really > become multi-source compatible. > > > --- > mysql-test/suite/galera/r/galera_gtid_slave.result | 18 +++ > .../r/galera_gtid_slave_sst_mysqldump.result | 92 ++++++++++++ > .../galera/r/galera_gtid_slave_sst_rsync.result | 92 ++++++++++++ > mysql-test/suite/galera/t/galera_gtid_slave.cnf | 18 +++ > mysql-test/suite/galera/t/galera_gtid_slave.test | 67 +++++++++ > .../galera/t/galera_gtid_slave_sst_mysqldump.cnf | 18 +++ > .../galera/t/galera_gtid_slave_sst_mysqldump.test | 156 +++++++++++++++++++++ > .../suite/galera/t/galera_gtid_slave_sst_rsync.cnf | 19 +++ > .../galera/t/galera_gtid_slave_sst_rsync.test | 156 +++++++++++++++++++++ > sql/log.cc | 37 ++++- > sql/wsrep_mysqld.cc | 9 ++ > 11 files changed, 681 insertions(+), 1 deletion(-) > > diff --git a/mysql-test/suite/galera/r/galera_gtid_slave.result b/mysql-test/suite/galera/r/galera_gtid_slave.result > new file mode 100644 > index 0000000..7ab10c7 > --- /dev/null > +++ b/mysql-test/suite/galera/r/galera_gtid_slave.result > @@ -0,0 +1,18 @@ > +START SLAVE; > +CREATE TABLE t1 (f1 INTEGER PRIMARY KEY) ENGINE=InnoDB; > +INSERT INTO t1 VALUES(1); > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-2 > +INSERT INTO t1 VALUES(2); > +INSERT INTO t1 VALUES(3); > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-2,2-2-2 > +INSERT INTO t1 VALUES(4); > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-2,2-2-2,2-3-3 > +DROP TABLE t1; > +STOP SLAVE; > +RESET SLAVE ALL; > diff --git a/mysql-test/suite/galera/r/galera_gtid_slave_sst_mysqldump.result b/mysql-test/suite/galera/r/galera_gtid_slave_sst_mysqldump.result > new file mode 100644 > index 0000000..8e8e2e0 > --- /dev/null > +++ b/mysql-test/suite/galera/r/galera_gtid_slave_sst_mysqldump.result > @@ -0,0 +1,92 @@ > +START SLAVE; > +CREATE TABLE t2 (f1 INTEGER PRIMARY KEY, f2 int unique) ENGINE=InnoDB; > +INSERT INTO t2 VALUES(1,11); > +INSERT INTO t2 VALUES(2,22); > +INSERT INTO t2 VALUES(3,33); > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-4 > +INSERT INTO t2 VALUES(4,44); > +INSERT INTO t2 VALUES(5,55); > +INSERT INTO t2 VALUES(6,66); > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-4,2-2-3 > +INSERT INTO t2 VALUES(7,77); > +INSERT INTO t2 VALUES(8,88); > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-4,2-2-3,2-3-5 > +CREATE TABLE t1 (f1 CHAR(255)) ENGINE=InnoDB; > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node1_committed_before'); > +INSERT INTO t1 VALUES ('node1_committed_before'); > +COMMIT; > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node2_committed_before'); > +INSERT INTO t1 VALUES ('node2_committed_before'); > +COMMIT; > +Shutting down server ... > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node1_committed_during'); > +INSERT INTO t1 VALUES ('node1_committed_during'); > +COMMIT; > +Starting server ... > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node3_committed_after'); > +INSERT INTO t1 VALUES ('node3_committed_after'); > +COMMIT; > +Select * from t1 order by f1; > +f1 > +node1_committed_before > +node1_committed_before > +node1_committed_during > +node1_committed_during > +node2_committed_before > +node2_committed_before > +node3_committed_after > +node3_committed_after > +Select * from t1 order by f1; > +f1 > +node1_committed_before > +node1_committed_before > +node1_committed_during > +node1_committed_during > +node2_committed_before > +node2_committed_before > +node3_committed_after > +node3_committed_after > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-6,2-2-7,2-3-8 > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-6,2-2-7,2-3-8 > +SET AUTOCOMMIT=ON; > +SET AUTOCOMMIT=ON; > +SET AUTOCOMMIT=ON; > +STOP slave; > +INSERT INTO t1 VALUES ('node2_slave_stoped'); > +INSERT INTO t1 VALUES ('node1_slave_stoped'); > +INSERT INTO t1 VALUES ('node2_slave_stoped_inserted'); > +start slave; > +INSERT INTO t1 VALUES ('node2_slave_started'); > +SELECT COUNT(*) from t1; > +COUNT(*) > +11 > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-6,2-3-8,2-2-11 > +SELECT COUNT(*) from t1; > +COUNT(*) > +11 > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-6,2-3-8,2-2-11 > +DROP TABLE t2,t1; > +STOP SLAVE; > +RESET SLAVE ALL; > diff --git a/mysql-test/suite/galera/r/galera_gtid_slave_sst_rsync.result b/mysql-test/suite/galera/r/galera_gtid_slave_sst_rsync.result > new file mode 100644 > index 0000000..9ab4601 > --- /dev/null > +++ b/mysql-test/suite/galera/r/galera_gtid_slave_sst_rsync.result > @@ -0,0 +1,92 @@ > +START SLAVE; > +CREATE TABLE t2 (f1 INTEGER PRIMARY KEY, f2 int unique) ENGINE=InnoDB; > +INSERT INTO t2 VALUES(1,11); > +INSERT INTO t2 VALUES(2,22); > +INSERT INTO t2 VALUES(3,33); > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-4 > +INSERT INTO t2 VALUES(4,44); > +INSERT INTO t2 VALUES(5,55); > +INSERT INTO t2 VALUES(6,66); > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-4,2-2-3 > +INSERT INTO t2 VALUES(7,77); > +INSERT INTO t2 VALUES(8,88); > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-4,2-2-3,2-3-5 > +CREATE TABLE t1 (f1 CHAR(255)) ENGINE=InnoDB; > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node1_committed_before'); > +INSERT INTO t1 VALUES ('node1_committed_before'); > +COMMIT; > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node2_committed_before'); > +INSERT INTO t1 VALUES ('node2_committed_before'); > +COMMIT; > +Shutting down server ... > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node1_committed_during'); > +INSERT INTO t1 VALUES ('node1_committed_during'); > +COMMIT; > +Starting server ... > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node3_committed_after'); > +INSERT INTO t1 VALUES ('node3_committed_after'); > +COMMIT; > +Select * from t1 order by f1; > +f1 > +node1_committed_before > +node1_committed_before > +node1_committed_during > +node1_committed_during > +node2_committed_before > +node2_committed_before > +node3_committed_after > +node3_committed_after > +Select * from t1 order by f1; > +f1 > +node1_committed_before > +node1_committed_before > +node1_committed_during > +node1_committed_during > +node2_committed_before > +node2_committed_before > +node3_committed_after > +node3_committed_after > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-6,2-2-7,2-3-8 > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-6,2-2-7,2-3-8 > +SET AUTOCOMMIT=ON; > +SET AUTOCOMMIT=ON; > +SET AUTOCOMMIT=ON; > +STOP slave; > +INSERT INTO t1 VALUES ('node2_slave_stoped'); > +INSERT INTO t1 VALUES ('node1_slave_stoped'); > +INSERT INTO t1 VALUES ('node2_slave_stoped_inserted'); > +start slave; > +INSERT INTO t1 VALUES ('node2_slave_started'); > +SELECT COUNT(*) from t1; > +COUNT(*) > +11 > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-7,2-3-8,2-2-11 > +SELECT COUNT(*) from t1; > +COUNT(*) > +12 > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-7,2-3-8,2-2-11 > +DROP TABLE t2,t1; > +STOP SLAVE; > +RESET SLAVE ALL; > diff --git a/mysql-test/suite/galera/t/galera_gtid_slave.cnf b/mysql-test/suite/galera/t/galera_gtid_slave.cnf > new file mode 100644 > index 0000000..409d0d1 > --- /dev/null > +++ b/mysql-test/suite/galera/t/galera_gtid_slave.cnf > @@ -0,0 +1,18 @@ > +!include ../galera_2nodes_as_slave.cnf > + > +[mysqld] > +log-bin=mysqld-bin > +log-slave-updates > +binlog-format=ROW > + > +[mysqld.1] > +gtid-domain-id=1 > +[mysqld.2] > +gtid-domain-id=2 > +wsrep_gtid_mode=1 > +wsrep_gtid_domain_id=2 > +[mysqld.3] > +gtid-domain-id=2 > +wsrep_gtid_mode=1 > +wsrep_gtid_domain_id=2 > + > diff --git a/mysql-test/suite/galera/t/galera_gtid_slave.test b/mysql-test/suite/galera/t/galera_gtid_slave.test > new file mode 100644 > index 0000000..3b3f6f3 > --- /dev/null > +++ b/mysql-test/suite/galera/t/galera_gtid_slave.test > @@ -0,0 +1,67 @@ > +# > +# Test Galera as a slave to a MariaDB master using GTIDs > +# > +# suite/galera/galera_2nodes_as_slave.cnf describes the setup of the nodes > +# suite/galera/t/galera_as_slave_gtid.cnf has the GTID options > +# > +# In addition to performing DDL and DML, we check that the gtid of the master is preserved inside the cluster > +# > + > +--source include/have_innodb.inc > + > +# As node #1 is not a Galera node, we connect to node #2 in order to run include/galera_cluster.inc > +--connect node_2a, 127.0.0.1, root, , test, $NODE_MYPORT_2 > +--source include/galera_cluster.inc > + > +--connection node_2 > +--disable_query_log > +--eval CHANGE MASTER TO MASTER_HOST='127.0.0.1', MASTER_USER='root', MASTER_PORT=$NODE_MYPORT_1; > +--enable_query_log > +START SLAVE; > + > +--connection node_1 > +CREATE TABLE t1 (f1 INTEGER PRIMARY KEY) ENGINE=InnoDB; > +INSERT INTO t1 VALUES(1); > + > +SELECT @@global.gtid_binlog_state; > + > +--connection node_2 > +--let $wait_condition = SELECT COUNT(*) = 1 FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = 't1'; > +--source include/wait_condition.inc > + > +--let $wait_condition = SELECT COUNT(*) = 1 FROM t1; > +--source include/wait_condition.inc > +INSERT INTO t1 VALUES(2); > +INSERT INTO t1 VALUES(3); > +SELECT @@global.gtid_binlog_state; > + > +--connect node_3, 127.0.0.1, root, , test, $NODE_MYPORT_3 > +--sleep 1 > +--let $wait_condition = SELECT COUNT(*) = 3 FROM t1; > +--source include/wait_condition.inc > + > +INSERT INTO t1 VALUES(4); > +SELECT @@global.gtid_binlog_state; > + > +--connection node_1 > +DROP TABLE t1; > + > +# > +# Unfortunately without the sleep below the following statement fails with "query returned no rows", which > +# is difficult to understand given that it is an aggregate query. A "query execution was interrupted" > +# warning is also reported by MTR, which is also weird. > +# > + > +--sleep 1 > + > +--connection node_2 > +--let $wait_condition = SELECT COUNT(*) = 0 FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = 't1'; > +--source include/wait_condition.inc > + > +--connection node_3 > +--let $wait_condition = SELECT COUNT(*) = 0 FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = 't1'; > +--source include/wait_condition.inc > + > +--connection node_2 > +STOP SLAVE; > +RESET SLAVE ALL; > diff --git a/mysql-test/suite/galera/t/galera_gtid_slave_sst_mysqldump.cnf b/mysql-test/suite/galera/t/galera_gtid_slave_sst_mysqldump.cnf > new file mode 100644 > index 0000000..409d0d1 > --- /dev/null > +++ b/mysql-test/suite/galera/t/galera_gtid_slave_sst_mysqldump.cnf > @@ -0,0 +1,18 @@ > +!include ../galera_2nodes_as_slave.cnf > + > +[mysqld] > +log-bin=mysqld-bin > +log-slave-updates > +binlog-format=ROW > + > +[mysqld.1] > +gtid-domain-id=1 > +[mysqld.2] > +gtid-domain-id=2 > +wsrep_gtid_mode=1 > +wsrep_gtid_domain_id=2 > +[mysqld.3] > +gtid-domain-id=2 > +wsrep_gtid_mode=1 > +wsrep_gtid_domain_id=2 > + > diff --git a/mysql-test/suite/galera/t/galera_gtid_slave_sst_mysqldump.test b/mysql-test/suite/galera/t/galera_gtid_slave_sst_mysqldump.test > new file mode 100644 > index 0000000..67aad21 > --- /dev/null > +++ b/mysql-test/suite/galera/t/galera_gtid_slave_sst_mysqldump.test > @@ -0,0 +1,156 @@ > +# > +# Test Galera as a slave to a MariaDB master using GTIDs > +# > +# suite/galera/galera_2nodes_as_slave.cnf describes the setup of the nodes > +# suite/galera/t/galera_as_slave_gtid.cnf has the GTID options > +# > +# In addition to performing DDL and DML, we check that the gtid of the master is preserved inside the cluster > +# > + > +--source include/big_test.inc > +--source include/have_innodb.inc > +# As node #1 is not a Galera node, we connect to node #2 in order to run include/galera_cluster.inc > +--connect node_2a, 127.0.0.1, root, , test, $NODE_MYPORT_2 > +--source include/galera_cluster.inc > + > +--connection node_2 > +--disable_query_log > +--eval CHANGE MASTER TO MASTER_HOST='127.0.0.1', MASTER_USER='root', MASTER_PORT=$NODE_MYPORT_1; > +--enable_query_log > +START SLAVE; > + > +--connection node_1 > +CREATE TABLE t2 (f1 INTEGER PRIMARY KEY, f2 int unique) ENGINE=InnoDB; > +INSERT INTO t2 VALUES(1,11); > +INSERT INTO t2 VALUES(2,22); > +INSERT INTO t2 VALUES(3,33); > + > +SELECT @@global.gtid_binlog_state; > + > +--connection node_2 > +--let $wait_condition = SELECT COUNT(*) = 1 FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = 't2'; > +--source include/wait_condition.inc > + > +--let $wait_condition = SELECT COUNT(*) = 3 FROM t2; > +--source include/wait_condition.inc > +INSERT INTO t2 VALUES(4,44); > +INSERT INTO t2 VALUES(5,55); > +INSERT INTO t2 VALUES(6,66); > +SELECT @@global.gtid_binlog_state; > + > +--connect node_3, 127.0.0.1, root, , test, $NODE_MYPORT_3 > +--sleep 1 > +--let $wait_condition = SELECT COUNT(*) = 6 FROM t2; > +--source include/wait_condition.inc > + > +INSERT INTO t2 VALUES(7,77); > +INSERT INTO t2 VALUES(8,88); > +SELECT @@global.gtid_binlog_state; > + > +#Perform SST > +--connection node_1 > +CREATE TABLE t1 (f1 CHAR(255)) ENGINE=InnoDB; > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node1_committed_before'); > +INSERT INTO t1 VALUES ('node1_committed_before'); > +COMMIT; > + > +--connection node_2 > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node2_committed_before'); > +INSERT INTO t1 VALUES ('node2_committed_before'); > +COMMIT; > + > +--connection node_3 > +--echo Shutting down server ... > +--source include/shutdown_mysqld.inc > + > + > +--connection node_2 > +--let $wait_condition = SELECT VARIABLE_VALUE = 1 FROM INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME = 'wsrep_cluster_size' > +--source include/wait_condition.inc > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node1_committed_during'); > +INSERT INTO t1 VALUES ('node1_committed_during'); > +COMMIT; > + > +--connection node_3 > +--echo Starting server ... > +--source include/start_mysqld.inc > +--source include/wait_until_ready.inc > + > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node3_committed_after'); > +INSERT INTO t1 VALUES ('node3_committed_after'); > +COMMIT; > + > +--connection node_2 > +Select * from t1 order by f1; > + > +--connection node_3 > +Select * from t1 order by f1; > + > +#SST Done > +--sleep 1 > +--connection node_2 > +SELECT @@global.gtid_binlog_state; > + > +--connection node_3 > +SELECT @@global.gtid_binlog_state; > + > +# > +#stop slave on node 2 > +--connection node_1 > +SET AUTOCOMMIT=ON; > +#drop table t1; > +#CREATE TABLE t1 (f1 CHAR(255)) ENGINE=InnoDB; > + > +--connection node_2 > +SET AUTOCOMMIT=ON; > +--connection node_3 > +SET AUTOCOMMIT=ON; > + > +--connection node_2 > +STOP slave; > +--sleep 1 > +INSERT INTO t1 VALUES ('node2_slave_stoped'); > + > +--connection node_1 > +INSERT INTO t1 VALUES ('node1_slave_stoped'); > +#start slave > +--connection node_2 > +INSERT INTO t1 VALUES ('node2_slave_stoped_inserted'); > +start slave; > +INSERT INTO t1 VALUES ('node2_slave_started'); > +SELECT COUNT(*) from t1; > +SELECT @@global.gtid_binlog_state; > + > +--connection node_3 > +SELECT COUNT(*) from t1; > +SELECT @@global.gtid_binlog_state; > + > +--connection node_1 > +DROP TABLE t2,t1; > + > +# Unfortunately without the sleep below the following statement fails with "query returned no rows", which > +# is difficult to understand given that it is an aggregate query. A "query execution was interrupted" > +# warning is also reported by MTR, which is also weird. > +# > + > +--sleep 3 > + > +--connection node_2 > +--let $wait_condition = SELECT COUNT(*) = 0 FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = 't2'; > +--source include/wait_condition.inc > + > +--connection node_3 > +--let $wait_condition = SELECT COUNT(*) = 0 FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = 't1'; > +--source include/wait_condition.inc > + > +--connection node_2 > +STOP SLAVE; > +RESET SLAVE ALL; > diff --git a/mysql-test/suite/galera/t/galera_gtid_slave_sst_rsync.cnf b/mysql-test/suite/galera/t/galera_gtid_slave_sst_rsync.cnf > new file mode 100644 > index 0000000..2c82531 > --- /dev/null > +++ b/mysql-test/suite/galera/t/galera_gtid_slave_sst_rsync.cnf > @@ -0,0 +1,19 @@ > +!include ../galera_2nodes_as_slave.cnf > + > +[mysqld] > +log-bin=mysqld-bin > +log-slave-updates > +binlog-format=ROW > +wsrep_sst_method=rsync > + > +[mysqld.1] > +gtid-domain-id=1 > +[mysqld.2] > +gtid-domain-id=2 > +wsrep_gtid_mode=1 > +wsrep_gtid_domain_id=2 > +[mysqld.3] > +gtid-domain-id=2 > +wsrep_gtid_mode=1 > +wsrep_gtid_domain_id=2 > + > diff --git a/mysql-test/suite/galera/t/galera_gtid_slave_sst_rsync.test b/mysql-test/suite/galera/t/galera_gtid_slave_sst_rsync.test > new file mode 100644 > index 0000000..67aad21 > --- /dev/null > +++ b/mysql-test/suite/galera/t/galera_gtid_slave_sst_rsync.test > @@ -0,0 +1,156 @@ > +# > +# Test Galera as a slave to a MariaDB master using GTIDs > +# > +# suite/galera/galera_2nodes_as_slave.cnf describes the setup of the nodes > +# suite/galera/t/galera_as_slave_gtid.cnf has the GTID options > +# > +# In addition to performing DDL and DML, we check that the gtid of the master is preserved inside the cluster > +# > + > +--source include/big_test.inc > +--source include/have_innodb.inc > +# As node #1 is not a Galera node, we connect to node #2 in order to run include/galera_cluster.inc > +--connect node_2a, 127.0.0.1, root, , test, $NODE_MYPORT_2 > +--source include/galera_cluster.inc > + > +--connection node_2 > +--disable_query_log > +--eval CHANGE MASTER TO MASTER_HOST='127.0.0.1', MASTER_USER='root', MASTER_PORT=$NODE_MYPORT_1; > +--enable_query_log > +START SLAVE; > + > +--connection node_1 > +CREATE TABLE t2 (f1 INTEGER PRIMARY KEY, f2 int unique) ENGINE=InnoDB; > +INSERT INTO t2 VALUES(1,11); > +INSERT INTO t2 VALUES(2,22); > +INSERT INTO t2 VALUES(3,33); > + > +SELECT @@global.gtid_binlog_state; > + > +--connection node_2 > +--let $wait_condition = SELECT COUNT(*) = 1 FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = 't2'; > +--source include/wait_condition.inc > + > +--let $wait_condition = SELECT COUNT(*) = 3 FROM t2; > +--source include/wait_condition.inc > +INSERT INTO t2 VALUES(4,44); > +INSERT INTO t2 VALUES(5,55); > +INSERT INTO t2 VALUES(6,66); > +SELECT @@global.gtid_binlog_state; > + > +--connect node_3, 127.0.0.1, root, , test, $NODE_MYPORT_3 > +--sleep 1 > +--let $wait_condition = SELECT COUNT(*) = 6 FROM t2; > +--source include/wait_condition.inc > + > +INSERT INTO t2 VALUES(7,77); > +INSERT INTO t2 VALUES(8,88); > +SELECT @@global.gtid_binlog_state; > + > +#Perform SST > +--connection node_1 > +CREATE TABLE t1 (f1 CHAR(255)) ENGINE=InnoDB; > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node1_committed_before'); > +INSERT INTO t1 VALUES ('node1_committed_before'); > +COMMIT; > + > +--connection node_2 > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node2_committed_before'); > +INSERT INTO t1 VALUES ('node2_committed_before'); > +COMMIT; > + > +--connection node_3 > +--echo Shutting down server ... > +--source include/shutdown_mysqld.inc > + > + > +--connection node_2 > +--let $wait_condition = SELECT VARIABLE_VALUE = 1 FROM INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME = 'wsrep_cluster_size' > +--source include/wait_condition.inc > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node1_committed_during'); > +INSERT INTO t1 VALUES ('node1_committed_during'); > +COMMIT; > + > +--connection node_3 > +--echo Starting server ... > +--source include/start_mysqld.inc > +--source include/wait_until_ready.inc > + > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node3_committed_after'); > +INSERT INTO t1 VALUES ('node3_committed_after'); > +COMMIT; > + > +--connection node_2 > +Select * from t1 order by f1; > + > +--connection node_3 > +Select * from t1 order by f1; > + > +#SST Done > +--sleep 1 > +--connection node_2 > +SELECT @@global.gtid_binlog_state; > + > +--connection node_3 > +SELECT @@global.gtid_binlog_state; > + > +# > +#stop slave on node 2 > +--connection node_1 > +SET AUTOCOMMIT=ON; > +#drop table t1; > +#CREATE TABLE t1 (f1 CHAR(255)) ENGINE=InnoDB; > + > +--connection node_2 > +SET AUTOCOMMIT=ON; > +--connection node_3 > +SET AUTOCOMMIT=ON; > + > +--connection node_2 > +STOP slave; > +--sleep 1 > +INSERT INTO t1 VALUES ('node2_slave_stoped'); > + > +--connection node_1 > +INSERT INTO t1 VALUES ('node1_slave_stoped'); > +#start slave > +--connection node_2 > +INSERT INTO t1 VALUES ('node2_slave_stoped_inserted'); > +start slave; > +INSERT INTO t1 VALUES ('node2_slave_started'); > +SELECT COUNT(*) from t1; > +SELECT @@global.gtid_binlog_state; > + > +--connection node_3 > +SELECT COUNT(*) from t1; > +SELECT @@global.gtid_binlog_state; > + > +--connection node_1 > +DROP TABLE t2,t1; > + > +# Unfortunately without the sleep below the following statement fails with "query returned no rows", which > +# is difficult to understand given that it is an aggregate query. A "query execution was interrupted" > +# warning is also reported by MTR, which is also weird. > +# > + > +--sleep 3 > + > +--connection node_2 > +--let $wait_condition = SELECT COUNT(*) = 0 FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = 't2'; > +--source include/wait_condition.inc > + > +--connection node_3 > +--let $wait_condition = SELECT COUNT(*) = 0 FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = 't1'; > +--source include/wait_condition.inc > + > +--connection node_2 > +STOP SLAVE; > +RESET SLAVE ALL; > > *review*> The better timing for the non-zero seq_no Gtid write deems to > inside THD::binlog_start_trans_and_stmt() invoked few instructions > earlier. This method is invoked once per *transaction* which > guarantees no duplication which the current chosen point seems to > concede. Perhaps you change its singature to pass the writer. Done as we discussed. this is added into binlog_statrt_trans_and_stmt() And I am checking for ha_info->is_started(). > Also I think we may write the non-zero seq_no Gtid generally, regardless > of WITH_WSREP (naturally to block doing that later at binlog commit > phase, also commented below). This commit will go into 10.1 so as we discussed , we can do this later in 10.3. > > diff --git a/sql/log.cc b/sql/log.cc > index a9f486d..53eeadf 100644 > --- a/sql/log.cc > +++ b/sql/log.cc > @@ -5655,6 +5655,23 @@ int THD::binlog_write_table_map(TABLE *table, bool is_transactional, > DBUG_RETURN(error); > } > } > +#ifdef WITH_WSREP > + /* Get domain id only when gtid mode is set > + If this event is replicate through a master then , > + we will forward the same gtid another nodes > + */ > + if (wsrep_gtid_mode && this->variables.gtid_seq_no) > + { > + Gtid_log_event gtid_event(this, this->variables.gtid_seq_no, > + this->variables.gtid_domain_id, > + true, LOG_EVENT_SUPPRESS_USE_F, > + true, 0); > + gtid_event.server_id= this->variables.server_id; > + > + if ((error= writer.write(&gtid_event))) > + DBUG_RETURN(error); > + } > +#endif > if ((error= writer.write(&the_event))) > DBUG_RETURN(error); > > @@ -5826,7 +5843,7 @@ MYSQL_BIN_LOG::write_gtid_event(THD *thd, bool standalone, > DBUG_PRINT("enter", ("standalone: %d", standalone)); > > #ifdef WITH_WSREP > - if (WSREP(thd) && thd->wsrep_trx_meta.gtid.seqno != -1 && wsrep_gtid_mode) > + if (WSREP(thd) && thd->wsrep_trx_meta.gtid.seqno != -1 && wsrep_gtid_mode && !thd->variables.gtid_seq_no) > { > domain_id= wsrep_gtid_domain_id; > } else { > > *review*> I'm suggest we check the non-zero seq_no status in > MYSQL_BIN_LOG::write_gtid_event() > > *review*>diff --git a/sql/log.cc b/sql/log.cc > *review*>index 2fcb6f6dbf6..8229426a5bd 100644 > *review*>--- a/sql/log.cc > *review*>+++ b/sql/log.cc > *review*>@@ -5867,7 +5867,6 @@ MYSQL_BIN_LOG::write_gtid_event(THD *thd, bool standalone, > *review*> err= rpl_global_gtid_binlog_state.update_with_next_gtid(domain_id, > *review*> local_server_id, &gtid); > *review*> seq_no= gtid.seq_no; > *review*>- } > *review*> if (err) > *review*> DBUG_RETURN(true); > *review*> thd->last_commit_gtid= gtid; > *review*>@@ -5880,6 +5879,7 @@ MYSQL_BIN_LOG::write_gtid_event(THD *thd, bool standalone, > *review*> DBUG_ASSERT(this == &mysql_bin_log); > *review*> if (write_event(&gtid_event)) > *review*> DBUG_RETURN(true); > *review*>+ } > *review*> status_var_add(thd->status_var.binlog_bytes_written, gtid_event.data_written); > *review*> > *review*> DBUG_RETURN(false); > > *review*> so only the zero auto-evaluated seq_no Gtid would make to the > method you're trying to patch. And the hunk is unneeded in this > case. Like above we generalize WITH_WSREP away. > ^^^^^^^ I am not sure about this hunk , I think this was mistake. I have reverted this change. > @@ -6718,6 +6735,24 @@ int MYSQL_BIN_LOG::write_cache(THD *thd, IO_CACHE *cache) > mysql_mutex_assert_owner(&LOCK_log); > if (reinit_io_cache(cache, READ_CACHE, 0, 0, 0)) > DBUG_RETURN(ER_ERROR_ON_WRITE); > + uchar *read_pos= cache->read_pos; > + > +#ifdef WITH_WSREP > + /* > + In the cache we have gtid event if , below condition is true, > + then we will simply increment the read position of cache by > + the length of Gtid_log_event. > + */ > + if (wsrep_gtid_mode && thd->variables.gtid_seq_no) > + { > + uchar *head= read_pos; > + uint data_len= uint4korr(head + EVENT_LEN_OFFSET); > + uint event_type= (uchar)head[EVENT_TYPE_OFFSET]; > + if (event_type == GTID_LOG_EVENT) > + cache->read_pos+= data_len; > + } > +#endif > + > uint length= my_b_bytes_in_cache(cache), group, carry, hdr_offs; > long val; > ulong end_log_pos_inc= 0; // each event processed adds BINLOG_CHECKSUM_LEN 2 t > > *review*> The following block is fine *but* we shall explain > what is GTID_SUPPORT, whether it actually for us. > Well , there is no such thing as GTID_SUPPORT :( , we are using ifdef without even defining gtid_support ╭─sachin@sachin-desk ~/10715/server ‹bb-10.1-10715*› ╰─$ grep -Rnie 'GTID_SUPPORT' sql/wsrep_sst.cc:290:#ifdef GTID_SUPPORT sql/wsrep_sst.cc:292:#endif /* GTID_SUPPORT */ sql/wsrep_applier.cc:130:#ifdef GTID_SUPPORT sql/wsrep_applier.cc:141:#endif /* GTID_SUPPORT */ sql/wsrep_applier.cc:309:#ifdef GTID_SUPPORT sql/wsrep_applier.cc:311:#endif /* GTID_SUPPORT */ sql/wsrep_mysqld.cc:50:#ifdef GTID_SUPPORT sql/wsrep_mysqld.cc:53:#endif /* GTID_SUPPORT */ sql/wsrep_mysqld.cc:256:#ifdef GTID_SUPPORT sql/wsrep_mysqld.cc:274:#endif /* GTID_SUPPORT */ sql/wsrep_mysqld.cc:413:#ifdef GTID_SUPPORT sql/wsrep_mysqld.cc:415:#endif /* GTID_SUPPORT */ sql/wsrep_mysqld.cc:1222:#ifdef GTID_SUPPORT sql/wsrep_mysqld.cc:1229:#endif /* GTID_SUPPORT */ sql/sql_class.h:4085:#ifdef GTID_SUPPORT sql/sql_class.h:4087:#endif /* GTID_SUPPORT */ sql/wsrep_mysqld.h:261:#ifdef GTID_SUPPORT sql/wsrep_mysqld.h:263:#endif /* GTID_SUPPORT */ > diff --git a/sql/wsrep_mysqld.cc b/sql/wsrep_mysqld.cc > index 33689a9..65751f1 100644 > --- a/sql/wsrep_mysqld.cc > +++ b/sql/wsrep_mysqld.cc > @@ -1227,6 +1227,15 @@ int wsrep_to_buf_helper( > if (!ret && writer.write(&gtid_ev)) ret= 1; > } > #endif /* GTID_SUPPORT */ > + if (wsrep_gtid_mode && thd->variables.gtid_seq_no) > + { > + Gtid_log_event gtid_event(thd, thd->variables.gtid_seq_no, > + thd->variables.gtid_domain_id, > + true, LOG_EVENT_SUPPRESS_USE_F, > + true, 0); > + gtid_event.server_id= thd->variables.server_id; > + ret= writer.write(&gtid_event); > + } > > /* if there is prepare query, add event for it */ > if (!ret && thd->wsrep_TOI_pre_query) -- Regards Sachin Setiya Software Engineer at MariaDB

1 0

[Maria-developers] Mysterious files after build
by Michal Schorm 09 Dec '17

09 Dec '17

Hello, I'd like to know more about some files, that are left after MariaDB build in Fedora. I do not ship them, however I haven't found any explanation from previous maintainers. I gathered some clues and thoughts about them. Questions are: What are those files and what they does?; Is it safe to ship them?; Should I start to ship them? /usr/bin/mysql_embedded - you (MariaDB) ship this file in client package %{_datadir}/mariadb/binary-configure - this script creates the MySQL system tables and starts the server. %{_datadir}/mariadb/magic - FS files first-bytes recoginiton? %{_datadir}/mariadb/mysql.server %{_datadir}/%{pkg_name}/mysqld_multi.server - those shouldn't be needed on systemd OSs ? /usr/bin/mytop - you (mariaDB) do not ship this file -- Michal Schorm Associate Software Engineer Core Services - Databases Team Red Hat

2 1

Re: [Maria-developers] b610986: MDEV-14593 human-readable XA RECOVER
by Sergei Golubchik 08 Dec '17

08 Dec '17

Hi, Alexey! Please, see few comments below. On Dec 07, Alexey Botchkov wrote: > revision-id: b610986c9c226e9b9cbe7c80ceaead1de0a55535 (mariadb-10.2.11-26-gb610986) > parent(s): 4d016e6ed2d32298977a66e125cbfcae39e23847 > committer: Alexey Botchkov > timestamp: 2017-12-07 23:49:12 +0400 > message: > > MDEV-14593 human-readable XA RECOVER. > > diff --git a/sql/handler.cc b/sql/handler.cc > index 7eed722..128721c 100644 > --- a/sql/handler.cc > +++ b/sql/handler.cc > @@ -1978,6 +1978,97 @@ int ha_recover(HASH *commit_list) > } > > /** > + return the XID as it appears in the SQL function's arguments. > + So this string can be passed to XA START, XA PREPARE etc... > + > + @note > + the 'buf' has to have space for at least SQL_XIDSIZE bytes. > +*/ > + > + > +/* > + 'a'..'z' 'A'..'Z', '0'..'9' > + and '-' '_' ' ' symbols don't have to be > + converted. I'd say it's overcomplication, why not to convert everything to hex? > +*/ > + > +static const char xid_needs_conv[128]= > +{ > + 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1, > + 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1, > + 0,1,1,1,1,1,1,1,1,1,1,1,1,0,1,1, > + 0,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1, > + 1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, > + 0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,0, > + 1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, > + 0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1 > +}; > + > +uint get_sql_xid(XID *xid, char *buf) > +{ > + int tot_len= xid->gtrid_length + xid->bqual_length; > + int i; > + const char *orig_buf= buf; > + > + for (i=0; i<tot_len; i++) > + { > + uchar c= ((uchar *) xid->data)[i] > + if (c >= 128 || xid_needs_conv[c]) > + break; > + } > + > + if (i >= tot_len) > + { > + /* No need to convert characters to hexadecimals. */ > + *buf++= '\''; > + memcpy(buf, xid->data, xid->gtrid_length); > + buf+= xid->gtrid_length; > + *buf++= '\''; > + if (xid->bqual_length > 0 || xid->formatID != 1) > + { > + *buf++= ','; > + *buf++= '\''; > + memcpy(buf, xid->data+xid->gtrid_length, xid->bqual_length); > + buf+= xid->bqual_length; > + *buf++= '\''; > + } > + } > + else > + { > + *buf++= 'X'; > + *buf++= '\''; > + for (i= 0; i < xid->gtrid_length; i++) > + { > + *buf++=_dig_vec_lower[((uchar*) xid->data)[i] >> 4]; > + *buf++=_dig_vec_lower[((uchar*) xid->data)[i] & 0x0f]; > + } > + *buf++= '\''; > + if (xid->bqual_length > 0 || xid->formatID != 1) > + { > + *buf++= ','; > + *buf++= 'X'; > + *buf++= '\''; > + for (; i < tot_len; i++) > + { > + *buf++=_dig_vec_lower[((uchar*) xid->data)[i] >> 4]; > + *buf++=_dig_vec_lower[((uchar*) xid->data)[i] & 0x0f]; > + } > + *buf++= '\''; > + } > + } > + > + if (xid->formatID != 1) > + { > + *buf++= ','; > + buf+= my_longlong10_to_str_8bit(&my_charset_bin, buf, > + MY_INT64_NUM_DECIMAL_DIGITS, -10, xid->formatID); > + } > + > + return buf - orig_buf; > +} > + > + > +/** > return the list of XID's to a client, the same way SHOW commands do. > > @note > diff --git a/sql/handler.h b/sql/handler.h > index 486ba56..91e3168 100644 > --- a/sql/handler.h > +++ b/sql/handler.h > @@ -628,6 +628,16 @@ struct xid_t { > }; > typedef struct xid_t XID; > > +/* > + The size of XID string representation in the form > + 'gtrid', 'bqual', formatID > + see xid_t::get_sql_string() for details. > +*/ > +#define SQL_XIDSIZE (XIDDATASIZE * 2 + 8 + MY_INT64_NUM_DECIMAL_DIGITS) > +/* The 'buf' has to have space for at least SQL_XIDSIZE bytes. */ > +uint get_sql_xid(XID *xid, char *buf); Not sure you need this as an extern function in the header. Why not to keep it static in handler.cc? > /* for recover() handlerton call */ > #define MIN_XID_LIST_SIZE 128 > #define MAX_XID_LIST_SIZE (1024*128) > diff --git a/sql/sql_yacc.yy b/sql/sql_yacc.yy > index 17e6904..d0fa642 100644 > --- a/sql/sql_yacc.yy > +++ b/sql/sql_yacc.yy > @@ -1705,6 +1705,7 @@ bool my_yyoverflow(short **a, YYSTYPE **b, ulong *yystacksize); > %token WRITE_SYM /* SQL-2003-N */ > %token X509_SYM > %token XA_SYM > +%token XID_SYM > %token XML_SYM > %token XOR > %token YEAR_MONTH_SYM You forgot to add XID_SYM to the keyword_sp rule. Also I don't see why we should have CONVERT XID syntax if the output is incompatible. Perhaps it'd be better to have a different syntax? FORMAT=SQL? Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] Crash on view with union
by jerome brauge 08 Dec '17

08 Dec '17

Hi all, The following script crashs the server (built with last bb-10.2-ext sources). use test; CREATE or replace TABLE `gtfmt` ( `numgtfmt` char(10) COLLATE utf8_bin NOT NULL ) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin; create or replace view gtfmtl(numgtfmt) as select 'x' from gtfmt union select 'x' from gtfmt ; SELECT * FROM gtfmtl WHERE numgtfmt = NAME_CONST('wnumgtfmt',_utf8'QEDITIONS' COLLATE 'utf8_bin'); Stack : > mysqld.exe!my_sigabrt_handler(int sig) Line 488 C mysqld.exe!raise(int signum) Line 516 C++ mysqld.exe!abort() Line 64 C++ mysqld.exe!common_assert_to_stderr_direct(const wchar_t * const expression, const wchar_t * const file_name, const unsigned int line_number) Line 123 C++ mysqld.exe!common_assert_to_stderr<wchar_t>(const wchar_t * const expression, const wchar_t * const file_name, const unsigned int line_number) Line 142 C++ mysqld.exe!_wassert(const wchar_t * expression, const wchar_t * file_name, unsigned int line_number) Line 404 C++ mysqld.exe!Item_func::fix_fields(THD * thd, Item * * ref) Line 330 C++ mysqld.exe!Item_str_func::fix_fields(THD * thd, Item * * ref) Line 106 C++ mysqld.exe!Item_name_const::fix_fields(THD * thd, Item * * ref) Line 2186 C++ mysqld.exe!Item_func::fix_fields(THD * thd, Item * * ref) Line 361 C++ mysqld.exe!JOIN::optimize_inner() Line 1311 C++ mysqld.exe!JOIN::optimize() Line 1121 C++ mysqld.exe!st_select_lex_unit::optimize() Line 1344 C++ mysqld.exe!mysql_derived_optimize(THD * thd, LEX * lex, TABLE_LIST * derived) Line 887 C++ mysqld.exe!mysql_handle_single_derived(LEX * lex, TABLE_LIST * derived, unsigned int phases) Line 197 C++ mysqld.exe!JOIN::optimize_inner() Line 1356 C++ mysqld.exe!JOIN::optimize() Line 1121 C++ mysqld.exe!mysql_select(THD * thd, TABLE_LIST * tables, unsigned int wild_num, List<Item> & fields, Item * conds, unsigned int og_num, st_order * order, st_order * group, Item * having, st_order * proc_param, unsigned __int64 select_options, select_result * result, st_select_lex_unit * unit, st_select_lex * select_lex) Line 3758 C++ mysqld.exe!handle_select(THD * thd, LEX * lex, select_result * result, unsigned long setup_tables_done_option) Line 364 C++ mysqld.exe!execute_sqlcom_select(THD * thd, TABLE_LIST * all_tables) Line 6473 C++ mysqld.exe!mysql_execute_command(THD * thd) Line 3717 C++ mysqld.exe!mysql_parse(THD * thd, char * rawbuf, unsigned int length, Parser_state * parser_state, bool is_com_multi, bool is_next_command) Line 7935 C++ mysqld.exe!dispatch_command(enum_server_command command, THD * thd, char * packet, unsigned int packet_length, bool is_com_multi, bool is_next_command) Line 1820 C++ mysqld.exe!do_command(THD * thd) Line 1369 C++ mysqld.exe!threadpool_process_request(THD * thd) Line 366 C++ mysqld.exe!tp_callback(TP_connection * c) Line 192 C++ mysqld.exe!tp_callback(_TP_CALLBACK_INSTANCE * instance, void * context) Line 377 C++ mysqld.exe!work_callback(_TP_CALLBACK_INSTANCE * instance, void * context, _TP_WORK * work) Line 451 C++ Best regard, Jérôme.

1 0