April 2022 - developers - lists.mariadb.org

Re: [Maria-developers] 349283c5e7a: MDEV-17124: mariadb 10.1.34, views and prepared statements: ERROR 1615 (HY000): Prepared statement needs to be re-prepared
by Sergei Golubchik 27 Sep '22

27 Sep '22

Hi, Oleksandr, On Jan 27, Oleksandr Byelkin wrote: > commit 349283c5e7a > Author: Oleksandr Byelkin <sanja(a)mariadb.com> > Date: Wed Apr 17 15:50:59 2019 +0200 > > MDEV-17124: mariadb 10.1.34, views and prepared statements: ERROR 1615 (HY000): Prepared statement needs to be re-prepared > > The problem is that if table definition cache (TDC) is full of real tables > which are in tables cache, view definition can not stay there so will be > removed by its own underlying tables. > In situation above old mechanism of detection matching definition in PS > and current version always require reprepare and so prevent executing > the PS. > > One work around is to increase TDC, other - improve version check for > views/triggers (which is done here). Now in suspicious cases we check: > - timestamp (ms) of the view to be sure that version really have changed; > - time (ms) of creation of a trigger related to time (ms) of statement > preparation. not "ms" but "us" or "µs". microseconds, not millisecons. > diff --git a/mysql-test/r/ps_ddl.result b/mysql-test/r/ps_ddl.result > index e69c6e06193..005c78a0319 100644 > --- a/mysql-test/r/ps_ddl.result > +++ b/mysql-test/r/ps_ddl.result > @@ -831,8 +831,8 @@ a b c > 20 40 100 > 30 60 150 > call p_verify_reprepare_count(1); > -SUCCESS > - > +ERROR > +Expected: 1, actual: 0 oops, forgot to update? > # Check that we properly handle ALTER VIEW statements. > execute stmt; > a b c > @@ -882,9 +882,9 @@ a b c > 10 50 60 > 20 100 120 > 30 150 180 > -call p_verify_reprepare_count(1); > -SUCCESS > - > +call p_verify_reprepare_count(0); > +ERROR > +Expected: 0, actual: 1 oops, updated too much? > execute stmt; > a b c > 10 50 60 > diff --git a/sql/sql_class.h b/sql/sql_class.h > index 3f0fba8fc10..b9bb9c978fb 100644 > --- a/sql/sql_class.h > +++ b/sql/sql_class.h > @@ -1079,6 +1079,7 @@ class Statement: public ilink, public Query_arena > > LEX_STRING name; /* name for named prepared statements */ > LEX *lex; // parse tree descriptor > + ulonglong ms_prepare_time; // time of preparation in microseconds "ms" here too :( Better just say my_hrtime_t prepare_time; > /* > Points to the query associated with this statement. It's const, but > we need to declare it char * because all table handlers are written > diff --git a/sql/sql_prepare.cc b/sql/sql_prepare.cc > index 64e4cd30561..a4a24cc1a80 100644 > --- a/sql/sql_prepare.cc > +++ b/sql/sql_prepare.cc > @@ -4177,6 +4177,9 @@ bool Prepared_statement::prepare(const char *packet, uint packet_len) > Query_arena *old_stmt_arena; > DBUG_ENTER("Prepared_statement::prepare"); > DBUG_ASSERT(m_sql_mode == thd->variables.sql_mode); > + > + // The same format as for triggers to compare > + ms_prepare_time= my_hrtime().val; do it at the end instead? > /* > If this is an SQLCOM_PREPARE, we also increase Com_prepare_sql. > However, it seems handy if com_stmt_prepare is increased always, > @@ -4519,6 +4523,22 @@ Prepared_statement::execute_loop(String *expanded_query, > DBUG_ASSERT(thd->get_stmt_da()->sql_errno() == ER_NEED_REPREPARE); > thd->clear_error(); > > + { > + /* > + Check if we too fast with reprepare: > + we can be so fast that: > + 1) make change of a trigger, > + 2) prepare, > + 3) try to exacute and reprepare > + in 1 microsecond, so we will wait till > + next microsecond before last reprepare > + */ > + while (first_prepared == my_hrtime().val) > + { > + pthread_yield(); > + } > + } I wouldn't bother. A more likely case is when a system time goes back, e.g. if there's an ntpd running and your loop doesn't help there. I suggest not to do anything special in this case. And if an ::execute() will open a trigger or a view with a timestamp >= than ms_prepare_time, it will cause reprepare every time. Which is exactly the behavior before your bugfix - opening of a trigger or a view causes reprepare - and it is not a problem unless one has a very small table definition cache. > + > error= reprepare(); > > if (! error) /* Success */ > diff --git a/sql/parse_file.cc b/sql/parse_file.cc > index 999f53bd681..92e7d82af1c 100644 > --- a/sql/parse_file.cc > +++ b/sql/parse_file.cc > @@ -174,11 +174,11 @@ write_parameter(IO_CACHE *file, uchar* base, File_option *parameter) > { > /* string have to be allocated already */ > LEX_STRING *val_s= (LEX_STRING *)(base + parameter->offset); > - time_t tm= my_time(0); > - > - get_date(val_s->str, GETDATE_DATE_TIME|GETDATE_GMT|GETDATE_FIXEDLENGTH, > - tm); > - val_s->length= PARSE_FILE_TIMESTAMPLENGTH; > + ulonglong tm= my_hrtime().val; add a comment here, "number of microseconds since Epoch, timezone-independent" or something like that > + // Paded to 19 characters for compatibility > + val_s->length= snprintf(val_s->str, MICROSECOND_TIMESTAMP_BUFFER_SIZE, > + "%019lld", tm); > + DBUG_ASSERT(val_s->length == MICROSECOND_TIMESTAMP_BUFFER_SIZE-1); > if (my_b_write(file, (const uchar *)val_s->str, > PARSE_FILE_TIMESTAMPLENGTH)) > DBUG_RETURN(TRUE); > @@ -835,15 +835,15 @@ File_parser::parse(uchar* base, MEM_ROOT *mem_root, > /* string have to be allocated already */ > LEX_STRING *val= (LEX_STRING *)(base + parameter->offset); > /* yyyy-mm-dd HH:MM:SS = 19(PARSE_FILE_TIMESTAMPLENGTH) characters */ a comment is now wrong > - if (ptr[PARSE_FILE_TIMESTAMPLENGTH] != '\n') > + if (ptr[MICROSECOND_TIMESTAMP_BUFFER_SIZE-1] != '\n') > { > my_error(ER_FPARSER_ERROR_IN_PARAMETER, MYF(0), > parameter->name.str, line); > DBUG_RETURN(TRUE); > } > - memcpy(val->str, ptr, PARSE_FILE_TIMESTAMPLENGTH); > - val->str[val->length= PARSE_FILE_TIMESTAMPLENGTH]= '\0'; > - ptr+= (PARSE_FILE_TIMESTAMPLENGTH+1); > + memcpy(val->str, ptr, MICROSECOND_TIMESTAMP_BUFFER_SIZE-1); > + val->str[val->length= MICROSECOND_TIMESTAMP_BUFFER_SIZE-1]= '\0'; > + ptr+= MICROSECOND_TIMESTAMP_BUFFER_SIZE; > break; > } > case FILE_OPTIONS_STRLIST: > diff --git a/sql/table.h b/sql/table.h > index 14d6b787b4e..82508388f84 100644 > --- a/sql/table.h > +++ b/sql/table.h > @@ -2159,7 +2175,7 @@ struct TABLE_LIST > LEX_STRING source; /* source of CREATE VIEW */ > LEX_STRING view_db; /* saved view database */ > LEX_STRING view_name; /* saved view name */ > - LEX_STRING timestamp; /* GMT time stamp of last operation */ > + LEX_STRING ms_timestamp; /* GMT time stamp of last operation */ better keep "timestamp" name. Or "hrtimestamp" if you'd like > st_lex_user definer; /* definer of view */ > ulonglong file_version; /* version of file's field set */ > ulonglong mariadb_version; /* version of server on creation */ > diff --git a/sql/table.cc b/sql/table.cc > index ca6ce02e4f2..24412966b36 100644 > --- a/sql/table.cc > +++ b/sql/table.cc > @@ -8480,6 +8480,53 @@ bool TABLE_LIST::is_with_table() > } > > > +bool TABLE_LIST::is_table_ref_id_equal(THD* thd, TABLE_SHARE *s) > +{ > + enum enum_table_ref_type tp= s->get_table_ref_type(); > + if (m_table_ref_type == tp) > + { > + bool res= m_table_ref_version == s->get_table_ref_version(); > + > + /* > + If definition is different check content version > + */ > + if (tabledef_version.length && > + tabledef_version.length == s->tabledef_version.length && > + memcmp(tabledef_version.str, s->tabledef_version.str, > + tabledef_version.length) == 0) > + { > + if (table && table->triggers) > + { > + > + ulonglong ms_stmt_prepare= thd->ms_prepare_time; > + if (ms_stmt_prepare) > + for(uint i= 0; i < TRG_EVENT_MAX; i++) > + for (uint j= 0; j < TRG_ACTION_MAX; j++) > + { > + Trigger *tr= > + table->triggers->get_trigger((trg_event_type)i, > + (trg_action_time_type)j); > + if (tr) > + if (ms_stmt_prepare <= tr->ms_create_time) > + { > + set_tabledef_version(s); > + return FALSE; > + } > + } > + } > + set_table_id(s); > + return TRUE; > + } > + else > + tabledef_version.length= 0; > + return res; > + } > + else > + set_tabledef_version(s); > + return FALSE; > +} this is now a very confusing function. it doesn't only check is_table_ref_id_equal, it also sets some of them (tabledef or both tabledef and ref_id), and under unclear conditions. Could you please comment it? And you don't check that view's timestamp is in the past. If it's not, it can be changed still within the same microsecond. > + > + > uint TABLE_SHARE::actual_n_key_parts(THD *thd) > { > return use_ext_keys && > diff --git a/sql/sql_trigger.h b/sql/sql_trigger.h > index e2e6e1b7acc..70f246d4e0d 100644 > --- a/sql/sql_trigger.h > +++ b/sql/sql_trigger.h > @@ -112,7 +112,7 @@ class Trigger :public Sql_alloc > GRANT_INFO subject_table_grants; > sql_mode_t sql_mode; > /* Store create time. Can't be mysql_time_t as this holds also sub seconds */ > - ulonglong create_time; > + ulonglong ms_create_time; // Create time timestamp in microseconds my_hrtime_t create_time; > trg_event_type event; > trg_action_time_type action_time; > uint action_order; > @@ -195,7 +195,7 @@ class Table_triggers_list: public Sql_alloc > */ > List<ulonglong> definition_modes_list; > /** Create times for triggers */ > - List<ulonglong> create_times; > + List<ulonglong> ms_create_times; ditto > > List<LEX_STRING> definers_list; > > @@ -331,4 +331,14 @@ bool mysql_create_or_drop_trigger(THD *thd, TABLE_LIST *tables, bool create); > extern const char * const TRG_EXT; > extern const char * const TRN_EXT; > > + > +/** > + Make time compatible with MySQL 5.7 trigger time. > +*/ > + > +inline ulonglong make_ms_time(my_time_t time, ulong time_sec_part) make_hrtime. and better put it together with hrtime_to_time, hrtime_from_time, etc > +{ > + return ((ulonglong) time)*1000000 + time_sec_part; > +} > + > #endif /* SQL_TRIGGER_INCLUDED */ > diff --git a/sql/sql_view.cc b/sql/sql_view.cc > index 020830483c5..3b51f6d2451 100644 > --- a/sql/sql_view.cc > +++ b/sql/sql_view.cc > @@ -1131,7 +1141,31 @@ static int mysql_register_view(THD *thd, TABLE_LIST *view, > DBUG_RETURN(error); > } > > +/** > + Check is TABLE_LEST and SHARE match > + @param[in] view TABLE_LIST of the view > + @param[in] share Share object of view > + > + @return false on error or misspatch > +*/ > > +bool mariadb_view_version_get(TABLE_SHARE *share) > +{ > + DBUG_ASSERT(share->is_view); > + > + if (!(share->tabledef_version.str= > + (uchar*) alloc_root(&share->mem_root, > + MICROSECOND_TIMESTAMP_BUFFER_SIZE))) > + return TRUE; > + > + DBUG_ASSERT(share->view_def != NULL); > + if (share->view_def->parse((uchar *) &share->tabledef_version, NULL, > + view_timestamp_parameters, 1, > + &file_parser_dummy_hook)) > + return TRUE; > + DBUG_ASSERT(share->tabledef_version.length == MICROSECOND_TIMESTAMP_BUFFER_SIZE-1); If you open an old view with YYYY-MM-DD... timestamp, where do you convert it? If you open a view without a timestamp at all, where do you handle it? > + return FALSE; > +} > > /** > read VIEW .frm and create structures > diff --git a/sql/sql_show.cc b/sql/sql_show.cc > index 9483db9eff9..138fa4bc631 100644 > --- a/sql/sql_show.cc > +++ b/sql/sql_show.cc > @@ -6724,13 +6724,15 @@ static bool store_trigger(THD *thd, Trigger *trigger, > table->field[14]->store(STRING_WITH_LEN("OLD"), cs); > table->field[15]->store(STRING_WITH_LEN("NEW"), cs); > > - if (trigger->create_time) > + if (trigger->ms_create_time) > { > + /* timestamp is in microseconds */ > table->field[16]->set_notnull(); > thd->variables.time_zone->gmt_sec_to_TIME(&timestamp, > - (my_time_t)(trigger->create_time/100)); > - /* timestamp is with 6 digits */ > - timestamp.second_part= (trigger->create_time % 100) * 10000; > + (my_time_t) > + (trigger->ms_create_time/ > + 1000000)); > + timestamp.second_part= trigger->ms_create_time % 1000000; hrtime_to_time() and hrtime_sec_part() > ((Field_temporal_with_date*) table->field[16])->store_time_dec(&timestamp, > 2); > } > @@ -9810,12 +9812,14 @@ static bool show_create_trigger_impl(THD *thd, Trigger *trigger) > trigger->db_cl_name.length, > system_charset_info); > > - if (trigger->create_time) > + if (trigger->ms_create_time) > { > MYSQL_TIME timestamp; > thd->variables.time_zone->gmt_sec_to_TIME(&timestamp, > - (my_time_t)(trigger->create_time/100)); > - timestamp.second_part= (trigger->create_time % 100) * 10000; > + (my_time_t) > + (trigger->ms_create_time/ > + 1000000)); > + timestamp.second_part= trigger->ms_create_time % 1000000; ditto > p->store(&timestamp, 2); > } > else Regards, Sergei VP of MariaDB Server Engineering and security(a)mariadb.org

2 5

Re: [Maria-developers] b3844107287: MDEV-16546 System versioning setting to allow history modification
by Sergei Golubchik 24 Aug '22

24 Aug '22

Hi, Aleksey! On Jun 08, Aleksey Midenkov wrote: > revision-id: b3844107287 (mariadb-10.5.2-424-gb3844107287) > parent(s): 27d66d644cf > author: Aleksey Midenkov <midenok(a)gmail.com> > committer: Aleksey Midenkov <midenok(a)gmail.com> > timestamp: 2021-03-01 17:43:34 +0300 > message: > > MDEV-16546 System versioning setting to allow history modification > > 1. force_fields_visible session variable makes system-invisible and > user-invisible fields visible on next table open. FLUSH TABLES > required for already opened tables. * system_versioning_insert_history * allows pseudocolumns ROW_START and ROW_END be specified in INSERT * doesn't affect invisible columns * FLUSH TABLES requirememnt is rather confusing, but let's have it, at least until I could suggest something better > 2. If secure_timestamp allows to modify timestamp variable then > following DML commands: INSERT, INSERT..SELECT and LOAD DATA can > specify row_start and row_end system field values. when system_versioning_insert_history=1 > 3. Cleaned up select_insert::send_data() from setting vers_write as > this parameter is now set on TABLE initialization. I don't see where it's set. This commit removes two table->vers_write= table->versioned(); and adds one from_field->table->vers_write= false; in Item_field::fix_fields. But where is vers_write set to true? Was it done in some other commit? > diff --git a/mysql-test/suite/versioning/r/insert.result b/mysql-test/suite/versioning/r/insert.result > index 2645d0184e8..00d1bb705c4 100644 > --- a/mysql-test/suite/versioning/r/insert.result > +++ b/mysql-test/suite/versioning/r/insert.result > @@ -112,3 +112,95 @@ x y > 9 9001 > drop table t1; > drop table t2; > +# > +# MDEV-16546 System versioning setting to allow history modification > +# > +# restart: --secure-timestamp=NO > +set @@session.time_zone='+00:00'; > +create table t1(x int) with system versioning; > +insert into t1(x) values (1); > +insert into t1(x, row_start, row_end) values (2, '1980-01-01 00:00:00', '1980-01-01 00:00:01'); > +ERROR 42S22: Unknown column 'row_start' in 'field list' > +set session force_fields_visible= 1; > +flush tables; > +show create table t1; > +Table Create Table > +t1 CREATE TABLE `t1` ( > + `x` int(11) DEFAULT NULL, > + `row_start` timestamp(6) GENERATED ALWAYS AS ROW START, > + `row_end` timestamp(6) GENERATED ALWAYS AS ROW END, shouldn't be visible here, only in insert and load. > + PERIOD FOR SYSTEM_TIME (`row_start`, `row_end`) > +) ENGINE=DEFAULT_ENGINE DEFAULT CHARSET=latin1 WITH SYSTEM VERSIONING > +insert into t1(x, row_start, row_end) values (3, '1980-01-01 00:00:00', '1980-01-01 00:00:01'); please, add tests for insert into t1 values (4) this should work, row_start/row_end must be mentioned explicitly. Also insert into t1 set x=5, row_start=..., row_end=... this should work too. And when only one row_start or row_end is mentioned as well. But update t1 set row_start=... insert into t1 (x, row_start, row_end) values (...) on duplicate key update row_start=... this should fail. While replace load data ... replace should work, but as DELETE+INSERT (I mean that DELETE in versioned tables doesn't actually delete history). If that's too difficult, then an error too. > +update t1 set x= x + 1; > +select x, check_row_ts(row_start, row_end) from t1 for system_time all order by x; ... > diff --git a/mysql-test/suite/versioning/t/insert.opt b/mysql-test/suite/versioning/t/insert.opt > new file mode 100644 > index 00000000000..a74d68957ef > --- /dev/null > +++ b/mysql-test/suite/versioning/t/insert.opt > @@ -0,0 +1 @@ > +--secure-timestamp=yes why is this? > -- source suite/versioning/common_finish.inc > diff --git a/mysql-test/suite/versioning/t/insert2.opt b/mysql-test/suite/versioning/t/insert2.opt > new file mode 100644 > index 00000000000..a74d68957ef > --- /dev/null > +++ b/mysql-test/suite/versioning/t/insert2.opt > @@ -0,0 +1 @@ > +--secure-timestamp=yes and this? > diff --git a/sql/sql_class.cc b/sql/sql_class.cc > index d23b190b2c5..93f8a8434e4 100644 > --- a/sql/sql_class.cc > +++ b/sql/sql_class.cc > @@ -7604,6 +7604,20 @@ void THD::set_last_commit_gtid(rpl_gtid &gtid) > #endif > } > > +bool THD::vers_modify_sys_field() const don't call it vers_modify_sys_field, your previous name vers_modify_history was better. There may be other sys fields that will never be directly writable (like ROWID, for example). > +{ > + if (lex->sql_command != SQLCOM_INSERT && > + lex->sql_command != SQLCOM_INSERT_SELECT && > + lex->sql_command != SQLCOM_LOAD) > + return false; I don't think this is enough to reject insert ... on duplicate key update row_start=xxx > + if (opt_secure_timestamp >= SECTIME_REPL || this is fixed in a later commit, so ok > + (opt_secure_timestamp == SECTIME_SUPER && > + !(security_ctx->master_access & SUPER_ACL))) > + return false; > + return true; > +} > + > + > void > wait_for_commit::reinit() > { > diff --git a/sql/sql_class.h b/sql/sql_class.h > index 2913a4a8350..520ea1d63bf 100644 > --- a/sql/sql_class.h > +++ b/sql/sql_class.h > @@ -820,6 +820,7 @@ typedef struct system_variables > > vers_asof_timestamp_t vers_asof_timestamp; > ulong vers_alter_history; > + my_bool force_fields_visible; system_versioning_insert_history > } SV; > > /** > @@ -5400,6 +5401,7 @@ class THD: public THD_count, /* this must be first */ > Item *sp_prepare_func_item(Item **it_addr, uint cols= 1); > bool sp_eval_expr(Field *result_field, Item **expr_item_ptr); > > + bool vers_modify_sys_field() const; vers_modify_history > }; > > > diff --git a/sql/sql_insert.cc b/sql/sql_insert.cc > index f0b84ab126f..84c2ae4ab36 100644 > --- a/sql/sql_insert.cc > +++ b/sql/sql_insert.cc > @@ -2014,7 +2014,7 @@ int write_record(THD *thd, TABLE *table, COPY_INFO *info, select_result *sink) > !table->file->referenced_by_foreign_key() && > (!table->triggers || !table->triggers->has_delete_triggers())) > { > - if (table->versioned(VERS_TRX_ID)) > + if (table->versioned(VERS_TRX_ID) && table->vers_write) does it even make sense? I mean direct insertion of transaction ids. > { > bitmap_set_bit(table->write_set, table->vers_start_field()->field_index); > table->file->column_bitmaps_signal(); > @@ -2029,7 +2029,7 @@ int write_record(THD *thd, TABLE *table, COPY_INFO *info, select_result *sink) > info->deleted++; > if (!table->file->has_transactions()) > thd->transaction->stmt.modified_non_trans_table= TRUE; > - if (table->versioned(VERS_TIMESTAMP)) > + if (table->versioned(VERS_TIMESTAMP) && table->vers_write) This (and the previous hunk) are in the DUP_REPLACE branch. There should be tests for these changes. > { > store_record(table, record[2]); > error= vers_insert_history_row(table); > diff --git a/sql/sys_vars.cc b/sql/sys_vars.cc > index f817f9086a9..4429308a097 100644 > --- a/sql/sys_vars.cc > +++ b/sql/sys_vars.cc > @@ -6737,3 +6737,7 @@ static Sys_var_ulonglong Sys_max_rowid_filter_size( > SESSION_VAR(max_rowid_filter_size), CMD_LINE(REQUIRED_ARG), > VALID_RANGE(1024, (ulonglong)~(intptr)0), DEFAULT(128*1024), > BLOCK_SIZE(1)); > + > +static Sys_var_mybool Sys_force_fields_visible( > + "force_fields_visible", "Make invisible fields visible on next table open", "system_versioning_insert_history" "Allows direct inserts into ROW_START and ROW_END columns" > + SESSION_VAR(force_fields_visible), CMD_LINE(OPT_ARG), DEFAULT(FALSE)); > diff --git a/sql/table.cc b/sql/table.cc > index f4bdbdeac5a..520f3738c9e 100644 > --- a/sql/table.cc > +++ b/sql/table.cc > @@ -4024,6 +4024,9 @@ enum open_frm_error open_table_from_share(THD *thd, TABLE_SHARE *share, > { > if (!((*field_ptr)= share->field[i]->clone(&outparam->mem_root, outparam))) > goto err; > + if (thd->variables.force_fields_visible && > + (*field_ptr)->invisible <= INVISIBLE_SYSTEM) > + (*field_ptr)->invisible= VISIBLE; Not really, system_versioning_insert_history should not affect visibility of anything besides ROW_START and ROW_END pseudocolumns in the INSERT and LOAD DATA statements. Regards, Sergei VP of MariaDB Server Engineering and security(a)mariadb.org

2 5

Re: [Maria-developers] 49ecf935415: MDEV-27009 Add UCA-14.0.0 collations
by Sergei Golubchik 16 Jun '22

16 Jun '22

Hi, Alexander, On Mar 14, Alexander Barkov wrote: > revision-id: 49ecf935415 (mariadb-10.6.1-335-g49ecf935415) > parent(s): c67789f63c8 > author: Alexander Barkov > committer: Alexander Barkov > timestamp: 2022-02-28 14:04:58 +0400 > message: > > MDEV-27009 Add UCA-14.0.0 collations please, list all user visible changes there. Mainly that collations are now decoupled from charsets. New syntax in CREATE TABLE, changes in I_S tables, etc. > diff --git a/mysql-test/include/ctype_utf_uca1400_ids.inc b/mysql-test/include/ctype_utf_uca1400_ids.inc > new file mode 100644 > index 00000000000..09cf49fc0e7 > --- /dev/null > +++ b/mysql-test/include/ctype_utf_uca1400_ids.inc > @@ -0,0 +1,17 @@ file names are confusing. better rename ctype_ucs_uca1400_ids.inc to something like ctype_convert_uca1400_ids and ctype_utf_uca1400_ids to ctype_set_names_uca1400_ids or something like that, to show what they do. > + > +--disable_ps_protocol > +--enable_metadata > +DELIMITER $$; > +FOR rec IN (SELECT COLLATION_NAME > + FROM INFORMATION_SCHEMA.COLLATION_CHARACTER_SET_APPLICABILITY > + WHERE CHARACTER_SET_NAME=@charset > + AND COLLATION_NAME RLIKE 'uca1400' > + ORDER BY ID) > +DO > + EXECUTE IMMEDIATE CONCAT('SET NAMES ',@charset,' COLLATE ', rec.COLLATION_NAME); > + SELECT rec.COLLATION_NAME; > +END FOR; > +$$ > +DELIMITER ;$$ > +--disable_metadata > +--enable_ps_protocol > diff --git a/include/m_ctype.h b/include/m_ctype.h > index 4c6628b72b3..706764ead2a 100644 > --- a/include/m_ctype.h > +++ b/include/m_ctype.h > @@ -34,7 +34,9 @@ enum loglevel { > extern "C" { > #endif > > -#define MY_CS_NAME_SIZE 32 > +#define MY_CS_CHARACTER_SET_NAME_SIZE 32 > +#define MY_CS_COLLATION_NAME_SIZE 64 That's FULL_COLLATION_NAME_SIZE, right? > + > #define MY_CS_CTYPE_TABLE_SIZE 257 > #define MY_CS_TO_LOWER_TABLE_SIZE 256 > #define MY_CS_TO_UPPER_TABLE_SIZE 256 > @@ -240,6 +242,46 @@ typedef enum enum_repertoire_t > } my_repertoire_t; > > > +/* ID compatibility */ > +typedef enum enum_collation_id_type > +{ > + MY_COLLATION_ID_TYPE_PRECISE= 0, > + MY_COLLATION_ID_TYPE_COMPAT_100800= 1 > +} my_collation_id_type_t; > + > + > +/* Collation name display modes */ > +typedef enum enum_collation_name_mode > +{ > + MY_COLLATION_NAME_MODE_FULL= 0, > + MY_COLLATION_NAME_MODE_CONTEXT= 1 > +} my_collation_name_mode_t; > + > + > +/* Level flags */ > +#define MY_CS_LEVEL_BIT_PRIMARY 0x00 > +#define MY_CS_LEVEL_BIT_SECONDARY 0x01 > +#define MY_CS_LEVEL_BIT_TERTIARY 0x02 > +#define MY_CS_LEVEL_BIT_QUATERNARY 0x03 > + > +#define MY_CS_COLL_LEVELS_S1 (1<<MY_CS_LEVEL_BIT_PRIMARY) > + > +#define MY_CS_COLL_LEVELS_AI_CS (1<<MY_CS_LEVEL_BIT_PRIMARY)| \ > + (1<<MY_CS_LEVEL_BIT_TERTIARY) > + > +#define MY_CS_COLL_LEVELS_S2 (1<<MY_CS_LEVEL_BIT_PRIMARY)| \ > + (1<<MY_CS_LEVEL_BIT_SECONDARY) > + > +#define MY_CS_COLL_LEVELS_S3 (1<<MY_CS_LEVEL_BIT_PRIMARY)| \ > + (1<<MY_CS_LEVEL_BIT_SECONDARY) | \ > + (1<<MY_CS_LEVEL_BIT_TERTIARY) AI_CS and S3 don't seem to be used yet > + > +#define MY_CS_COLL_LEVELS_S4 (1<<MY_CS_LEVEL_BIT_PRIMARY)| \ > + (1<<MY_CS_LEVEL_BIT_SECONDARY) | \ > + (1<<MY_CS_LEVEL_BIT_TERTIARY) | \ > + (1<<MY_CS_LEVEL_BIT_QUATERNARY) > + > + > /* Flags for strxfrm */ > #define MY_STRXFRM_LEVEL1 0x00000001 /* for primary weights */ > #define MY_STRXFRM_LEVEL2 0x00000002 /* for secondary weights */ > diff --git a/sql/sql_alter.cc b/sql/sql_alter.cc > index 86c6e9a27f8..9ddd482ad57 100644 > --- a/sql/sql_alter.cc > +++ b/sql/sql_alter.cc > @@ -546,6 +546,7 @@ bool Sql_cmd_alter_table::execute(THD *thd) > > result= mysql_alter_table(thd, &select_lex->db, &lex->name, > &create_info, > + lex->create_info.default_charset_collation, I don't see why you need a new argument here. It's create_info.default_charset_collation, so, mysql_alter_table already gets it in create_info. All other mysql_alter_table invocations also take create_info argument and can get default_charset_collation from there > first_table, > &alter_info, > select_lex->order_list.elements, > diff --git a/sql/sql_partition_admin.cc b/sql/sql_partition_admin.cc > index fb1ae0d5fc7..4188dde252b 100644 > --- a/sql/sql_partition_admin.cc > +++ b/sql/sql_partition_admin.cc > @@ -211,6 +211,7 @@ bool compare_table_with_partition(THD *thd, TABLE *table, TABLE *part_table, > part_table->use_all_columns(); > table->use_all_columns(); > if (unlikely(mysql_prepare_alter_table(thd, part_table, &part_create_info, > + Lex_maybe_default_charset_collation(), Same. Can be in part_create_info > &part_alter_info, &part_alter_ctx))) > { > my_error(ER_TABLES_DIFFERENT_METADATA, MYF(0)); > diff --git a/sql/sql_i_s.h b/sql/sql_i_s.h > index bed2e886718..5ff06d32231 100644 > --- a/sql/sql_i_s.h > +++ b/sql/sql_i_s.h > @@ -162,6 +162,11 @@ class Yesno: public Varchar > { > public: > Yesno(): Varchar(3) { } > + static LEX_CSTRING value(bool val) > + { > + return val ? Lex_cstring(STRING_WITH_LEN("Yes")) : > + Lex_cstring(); > + } eh... please, rename the class from Yesno to something like Yesempty or Yes_or_empty, something that says that the second should not be Lex_cstring(STRING_WITH_LEN("No")) > }; > > > diff --git a/sql/table.cc b/sql/table.cc > index a683a78ff49..c28cb2bd928 100644 > --- a/sql/table.cc > +++ b/sql/table.cc > @@ -3491,6 +3493,16 @@ int TABLE_SHARE::init_from_sql_statement_string(THD *thd, bool write, > else > thd->set_n_backup_active_arena(arena, &backup); > > + /* > + THD::reset_db() does not set THD::db_charset, > + so it keeps pointing to the character set and collation > + of the current database, rather than the database of the > + new initialized table. Hmm, is that correct? Could you check other invocation of thd->reset_db()? Perhaps they all need to switch charset? In that case it should be done inside THD::reset_db(). Or may be they have to use mysql_change_db_impl() instead? > + Let's call get_default_db_collation() before reset_db(). > + This forces the db.opt file to be loaded. > + */ > + db_cs= get_default_db_collation(thd, db.str); > + > thd->reset_db(&db); > lex_start(thd); > > @@ -3498,6 +3510,11 @@ int TABLE_SHARE::init_from_sql_statement_string(THD *thd, bool write, > sql_unusable_for_discovery(thd, hton, sql_copy)))) > goto ret; > > + if (!(thd->lex->create_info.default_table_charset= > + thd->lex->create_info.default_charset_collation. > + resolved_to_character_set(db_cs, db_cs))) > + DBUG_RETURN(true); How could this (and similar if()'s in other files) fail? > + > thd->lex->create_info.db_type= hton; > #ifdef WITH_PARTITION_STORAGE_ENGINE > thd->work_part_info= 0; // For partitioning > diff --git a/sql/mysys_charset.h b/sql/mysys_charset.h > new file mode 100644 > index 00000000000..86eaeedd432 > --- /dev/null > +++ b/sql/mysys_charset.h > @@ -0,0 +1,44 @@ > +#ifndef MYSYS_CHARSET > +#define MYSYS_CHARSET > + > +/* Copyright (c) 2021, MariaDB Corporation. > + > + This program is free software; you can redistribute it and/or modify > + it under the terms of the GNU General Public License as published by > + the Free Software Foundation; version 2 of the License. > + > + This program is distributed in the hope that it will be useful, > + but WITHOUT ANY WARRANTY; without even the implied warranty of > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > + GNU General Public License for more details. > + > + You should have received a copy of the GNU General Public License > + along with this program; if not, write to the Free Software > + Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1335 USA */ > + > + > +#include "my_sys.h" > + > + > +class Charset_loader_mysys: public MY_CHARSET_LOADER > +{ > +public: > + Charset_loader_mysys() > + { > + my_charset_loader_init_mysys(this); > + } > + void raise_unknown_collation_error(const char *name, > + CHARSET_INFO *name_cs) const; > + CHARSET_INFO *get_charset(const char *cs_name, uint cs_flags, myf my_flags); > + CHARSET_INFO *get_exact_collation(const char *name, myf utf8_flag); > + CHARSET_INFO *get_contextually_typed_collation(CHARSET_INFO *cs, > + const char *name); > + CHARSET_INFO *get_contextually_typed_collation(const char *name); > + CHARSET_INFO *get_contextually_typed_collation_or_error(CHARSET_INFO *cs, > + const char *name); > + CHARSET_INFO *find_default_collation(CHARSET_INFO *cs); > + CHARSET_INFO *find_bin_collation_or_error(CHARSET_INFO *cs); > +}; you can have C++ code in mysys too, you know, no need to put it in sql/mysys* > + > +#endif // MYSYS_CHARSET > + > diff --git a/strings/ctype-simple.c b/strings/ctype-simple.c > index b579f0af203..d09dfba86ed 100644 > --- a/strings/ctype-simple.c > +++ b/strings/ctype-simple.c > @@ -1940,13 +1941,26 @@ my_bool my_propagate_complex(CHARSET_INFO *cs __attribute__((unused)), > } > > > +void my_ci_set_strength(struct charset_info_st *cs, uint strength) > +{ > + DBUG_ASSERT(strength > 0 && strength <= MY_STRXFRM_NLEVELS); don't use && in asserts, please create two separate asserts instead: DBUG_ASSERT(strength > 0); DBUG_ASSERT(strength <= MY_STRXFRM_NLEVELS); > + cs->levels_for_order= ((1 << strength) - 1); why do you still use the old concept of "strength"? Why not to use bitmap consistently everywhere? > +} > + > + > +void my_ci_set_level_flags(struct charset_info_st *cs, uint flags) > +{ > + DBUG_ASSERT(flags < (1<<MY_STRXFRM_NLEVELS)); > + cs->levels_for_order= flags; > +} > + > /* > Normalize strxfrm flags > > SYNOPSIS: > my_strxfrm_flag_normalize() > + cs - the CHARSET_INFO pointer > flags - non-normalized flags > - nlevels - number of levels > > NOTES: > If levels are omitted, then 1-maximum is assumed. > diff --git a/sql/handler.h b/sql/handler.h > index 8ad521e189a..1e82f37b1e7 100644 > --- a/sql/handler.h > +++ b/sql/handler.h > @@ -2409,7 +2386,32 @@ struct Table_specification_st: public HA_CREATE_INFO, > { > HA_CREATE_INFO::options= 0; > DDL_options_st::init(); > + default_charset_collation.init(); > + } > + > + bool > + add_alter_list_item_convert_to_charset(const Lex_charset_collation_st &cl) > + { > + /* > + cs cannot be NULL, as sql_yacc.yy translates > + CONVERT TO CHARACTER SET DEFAULT > + to > + CONVERT TO CHARACTER SET <character-set-of-the-current-database> > + TODO: Shouldn't we postpone resolution of DEFAULT until the > + character set of the table owner database is loaded from its db.opt? > + */ > + DBUG_ASSERT(cl.charset_collation()); > + DBUG_ASSERT(!cl.is_contextually_typed_collation()); > + alter_table_convert_to_charset= cl.charset_collation(); > + default_charset_collation.Lex_charset_collation_st::operator=(cl); looks quite ugly. can you do, like, default_charset_collation.set(cl) ? > + used_fields|= (HA_CREATE_USED_CHARSET | HA_CREATE_USED_DEFAULT_CHARSET); > + return false; > } > + bool add_table_option_default_charset(CHARSET_INFO *cs); > + bool add_table_option_default_collation(const Lex_charset_collation_st &cl); > + bool resolve_db_charset_and_collation(THD *thd, > + const LEX_CSTRING &db, > + bool is_alter); > }; > > > diff --git a/strings/ctype-uca1400data.h b/strings/ctype-uca1400data.h > new file mode 100644 > index 00000000000..da95dcfde54 > --- /dev/null > +++ b/strings/ctype-uca1400data.h > @@ -0,0 +1,44151 @@ > +/* > + Generated from allkeys.txt version '14.0.0' > +*/ if it's generated, do you need to check it in? perhaps it should be generated during the build? you've checked in allkeys1400.txt anyway. > +static const uint16 uca1400_p000[]= { /* 0000 (4 weights per char) */ > +0x0000,0x0000,0x0000,0x0000, 0x0000,0x0000,0x0000,0x0000, /* 0000 */ > +0x0000,0x0000,0x0000,0x0000, 0x0000,0x0000,0x0000,0x0000, /* 0002 */ > diff --git a/sql/sql_lex.cc b/sql/sql_lex.cc > index 6ca10267187..d115401a855 100644 > --- a/sql/sql_lex.cc > +++ b/sql/sql_lex.cc > @@ -542,6 +542,30 @@ bool LEX::add_alter_list(LEX_CSTRING name, LEX_CSTRING new_name, bool exists) > } > > > +bool LEX::add_alter_list_item_convert_to_charset( > + THD *thd, > + CHARSET_INFO *cs, > + const Lex_charset_collation_st &cl) > +{ > + if (!cs) > + { > + Lex_charset_collation_st tmp; > + tmp.set_charset_collate_default(thd->variables.collation_database); Hmm, what if one is doing ALTER TABLE db.test CHARSET DEFAULT and current db is not `db` but `test` ? > + if (!(cs= tmp.charset_collation())) > + return true; // Should not actually happen assert? > + } > + > + Lex_explicit_charset_opt_collate tmp(cs, false); > + if (tmp.merge_opt_collate_or_error(cl) || > + create_info.add_alter_list_item_convert_to_charset( > + Lex_charset_collation(tmp))) > + return true; > + > + alter_info.flags|= ALTER_CONVERT_TO; > + return false; > +} > + > + > void LEX::init_last_field(Column_definition *field, > const LEX_CSTRING *field_name) > { > @@ -11871,29 +11869,41 @@ CHARSET_INFO *Lex_collation_st::find_default_collation(CHARSET_INFO *cs) > "def" is the upper level CHARACTER SET clause (e.g. of a table) > */ > CHARSET_INFO * > -Lex_collation_st::resolved_to_character_set(CHARSET_INFO *def) const > +Lex_charset_collation_st::resolved_to_character_set(CHARSET_INFO *def) const > { > DBUG_ASSERT(def); > - if (m_type != TYPE_CONTEXTUALLY_TYPED) > - { > - if (!m_collation) > - return def; // Empty - not typed at all > - return m_collation; // Explicitly typed > + > + switch (m_type) { > + case TYPE_EMPTY: > + return def; > + case TYPE_CHARACTER_SET: > + DBUG_ASSERT(m_ci); > + return m_ci; > + case TYPE_COLLATE_EXACT: > + DBUG_ASSERT(m_ci); > + return m_ci; > + case TYPE_COLLATE_CONTEXTUALLY_TYPED: > + break; > } > > // Contextually typed > - DBUG_ASSERT(m_collation); > + DBUG_ASSERT(m_ci); > > - if (m_collation == &my_charset_bin) // CHAR(10) BINARY > - return find_bin_collation(def); > + Charset_loader_mysys loader; > + if (is_contextually_typed_binary_style()) // CHAR(10) BINARY > + return loader.find_bin_collation_or_error(def); > > - if (m_collation == &my_charset_latin1) // CHAR(10) COLLATE DEFAULT > - return find_default_collation(def); > + if (is_contextually_typed_collate_default()) // CHAR(10) COLLATE DEFAULT > + return loader.find_default_collation(def); > + > + const LEX_CSTRING context_name= collation_name_context_suffix(); I'd rather put this in assert, not in if(). Like - if (!strncasecmp(context_name.str, STRING_WITH_LEN("uca1400_"))) + DBUG_ASSERT(!strncasecmp(context_cl_name.str, STRING_WITH_LEN("uca1400_"))); > + if (!strncasecmp(context_name.str, STRING_WITH_LEN("uca1400_"))) > + return loader.get_contextually_typed_collation_or_error(def, > + context_name.str); > > /* > - Non-binary and non-default contextually typed collation. > + Non-binary, non-default, non-uca1400 contextually typed collation. > We don't have such yet - the parser cannot produce this. > - But will have soon, e.g. "uca1400_as_ci". > */ > DBUG_ASSERT(0); > return NULL; > @@ -11944,58 +11972,106 @@ bool Lex_collation_st:: > CHAR(10) BINARY .. COLLATE latin1_bin > CHAR(10) COLLATE uca1400_as_ci .. COLLATE latin1_bin > */ > - if (collation() == &my_charset_latin1 && > - !(cl.collation()->state & MY_CS_PRIMARY)) > + if (is_contextually_typed_collate_default() && > + !cl.charset_collation()->default_flag()) > { > - my_error(ER_CONFLICTING_DECLARATIONS, MYF(0), > - "COLLATE ", "DEFAULT", "COLLATE ", > - cl.collation()->coll_name.str); > + error_conflicting_collations_or_styles(*this, cl); > return true; > } > - if (collation() == &my_charset_bin && > - !(cl.collation()->state & MY_CS_BINSORT)) > + > + if (is_contextually_typed_binary_style() && > + !cl.charset_collation()->binsort_flag()) > { > - my_error(ER_CONFLICTING_DECLARATIONS, MYF(0), > - "", "BINARY", "COLLATE ", cl.collation()->coll_name.str); > + error_conflicting_collations_or_styles(*this, cl); > return true; > } > *this= cl; > return false; > } > > - if (cl.is_contextually_typed_collation()) > - { > + DBUG_ASSERT(0); > + return false; > +} > + > + > +bool > +Lex_explicit_charset_opt_collate:: > + merge_collate_or_error(const Lex_charset_collation_st &cl) > +{ > + DBUG_ASSERT(cl.type() != Lex_charset_collation_st::TYPE_CHARACTER_SET); > + > + switch (cl.type()) { > + case Lex_charset_collation_st::TYPE_EMPTY: > + return false; > + case Lex_charset_collation_st::TYPE_CHARACTER_SET: > + DBUG_ASSERT(0); > + return false; > + case Lex_charset_collation_st::TYPE_COLLATE_EXACT: > /* > - EXPLICIT + CONTEXT > - CHAR(10) COLLATE latin1_bin .. COLLATE DEFAULT - not supported > - CHAR(10) COLLATE latin1_bin .. COLLATE uca1400_as_ci - not yet > + EXPLICIT + EXPLICIT > + CHAR(10) CHARACTER SET latin1 .. COLLATE latin1_bin > + CHAR(10) CHARACTER SET latin1 COLLATE latin1_bin .. COLLATE latin1_bin > + CHAR(10) COLLATE latin1_bin .. COLLATE latin1_bin > + CHAR(10) COLLATE latin1_bin .. COLLATE latin1_bin > + CHAR(10) CHARACTER SET latin1 BINARY .. COLLATE latin1_bin > */ > - DBUG_ASSERT(0); // Not possible yet > + if (m_with_collate && m_ci != cl.charset_collation()) > + { > + my_error(ER_CONFLICTING_DECLARATIONS, MYF(0), > + "COLLATE ", m_ci->coll_name.str, > + "COLLATE ", cl.charset_collation()->coll_name.str); > + return true; > + } > + if (!my_charset_same(m_ci, cl.charset_collation())) > + { > + my_error(ER_COLLATION_CHARSET_MISMATCH, MYF(0), > + cl.charset_collation()->coll_name.str, m_ci->cs_name.str); > + return true; > + } > + m_ci= cl.charset_collation(); > + m_with_collate= true; > return false; > - } > > - /* > - EXPLICIT + EXPLICIT > - CHAR(10) CHARACTER SET latin1 .. COLLATE latin1_bin > - CHAR(10) CHARACTER SET latin1 COLLATE latin1_bin .. COLLATE latin1_bin > - CHAR(10) COLLATE latin1_bin .. COLLATE latin1_bin > - CHAR(10) COLLATE latin1_bin .. COLLATE latin1_bin > - CHAR(10) CHARACTER SET latin1 BINARY .. COLLATE latin1_bin > - */ > - if (type() == TYPE_EXPLICIT && collation() != cl.collation()) > - { > - my_error(ER_CONFLICTING_DECLARATIONS, MYF(0), > - "COLLATE ", collation()->coll_name.str, > - "COLLATE ", cl.collation()->coll_name.str); > - return true; > - } > - if (!my_charset_same(collation(), cl.collation())) > - { > - my_error(ER_COLLATION_CHARSET_MISMATCH, MYF(0), > - cl.collation()->coll_name.str, collation()->cs_name.str); > - return true; > + case Lex_charset_collation_st::TYPE_COLLATE_CONTEXTUALLY_TYPED: > + if (cl.is_contextually_typed_collate_default()) > + { > + /* > + SET NAMES latin1 COLLATE DEFAULT; > + ALTER TABLE t1 CONVERT TO CHARACTER SET latin1 COLLATE DEFAULT; > + */ > + CHARSET_INFO *tmp= Charset_loader_mysys().find_default_collation(m_ci); > + if (!tmp) > + return true; > + m_ci= tmp; > + m_with_collate= true; > + return false; > + } > + else > + { > + /* > + EXPLICIT + CONTEXT > + CHAR(10) COLLATE latin1_bin .. COLLATE DEFAULT not possible yet > + CHAR(10) COLLATE latin1_bin .. COLLATE uca1400_as_ci > + */ > + > + const LEX_CSTRING context_cl_name= cl.collation_name_context_suffix(); > + if (!strncasecmp(context_cl_name.str, STRING_WITH_LEN("uca1400_"))) Like above, better DBUG_ASSERT(!strncasecmp(context_cl_name.str, STRING_WITH_LEN("uca1400_"))) > + { > + CHARSET_INFO *tmp; > + Charset_loader_mysys loader; > + if (!(tmp= loader.get_contextually_typed_collation_or_error(m_ci, > + context_cl_name.str))) > + return true; > + m_with_collate= true; > + m_ci= tmp; > + return false; > + } > + > + DBUG_ASSERT(0); // Not possible yet > + return false; > + } > } > - *this= cl; > + DBUG_ASSERT(0); > return false; > } > > diff --git a/strings/ctype-uca.c b/strings/ctype-uca.c > index b89916f3b20..3e6b4e4ce43 100644 > --- a/strings/ctype-uca.c > +++ b/strings/ctype-uca.c > @@ -30542,7 +30613,7 @@ static const char vietnamese[]= > Myanmar, according to CLDR Revision 8900. > http://unicode.org/cldr/trac/browser/trunk/common/collation/my.xml > */ > -static const char myanmar[]= "[shift-after-method expand][version 5.2.0]" > +static const char myanmar[]= "[shift-after-method expand]" What's going on with myanmar? You removed a version here and added &my_uca_v520 below in its charset_info_st. What does this change mean? > /* Tones */ > "&\\u108C" > "<\\u1037" > @@ -37627,7 +37825,7 @@ struct charset_info_st my_charset_utf32_myanmar_uca_ci= > NULL, /* to_lower */ > NULL, /* to_upper */ > NULL, /* sort_order */ > - NULL, /* uca */ > + &my_uca_v520, /* uca */ What does this change? > NULL, /* tab_to_uni */ > NULL, /* tab_from_uni */ > &my_unicase_unicode520,/* caseinfo */ Regards, Sergei VP of MariaDB Server Engineering and security(a)mariadb.org

2 3

Re: [Maria-developers] 9d6be81c0da: MDEV-27832 disable binary logging for SQL SERVICE.
by Sergei Golubchik 05 May '22

05 May '22

Hi, Alexey, On Mar 02, Alexey Botchkov wrote: > revision-id: 9d6be81c0da (mariadb-10.7.2-8-g9d6be81c0da) > parent(s): 33fd136c61b > author: Alexey Botchkov > committer: Alexey Botchkov > timestamp: 2022-02-15 13:37:59 +0400 > message: > > MDEV-27832 disable binary logging for SQL SERVICE. > > Binary logging is now disabled for the queries run by SQL SERVICE. > The binlogging can be turned on with the 'SET SQL_LOG_BIN=On' query. > > diff --git a/mysql-test/suite/plugins/r/test_sql_service.result b/mysql-test/suite/plugins/r/test_sql_service.result > index 00f0411b665..943bb239e5f 100644 > --- a/mysql-test/suite/plugins/r/test_sql_service.result > +++ b/mysql-test/suite/plugins/r/test_sql_service.result > @@ -2,6 +2,19 @@ install plugin test_sql_service soname 'test_sql_service'; > show status like 'test_sql_service_passed'; > Variable_name Value > Test_sql_service_passed 1 > +set global test_sql_service_execute_sql_global= 'create table test.t1 select 1 as a, @@SQL_LOG_BIN'; > +set global test_sql_service_execute_sql_local= 'insert into test.t1 select 2 as a, @@SQL_LOG_BIN'; > +set global test_sql_service_execute_sql_global= 'SET SQL_LOG_BIN=1'; > +set global test_sql_service_execute_sql_global= 'insert into test.t1 select 3 as a, @@SQL_LOG_BIN'; > +set global test_sql_service_execute_sql_global= 'SET SQL_LOG_BIN=0'; > +set global test_sql_service_execute_sql_global= 'insert into test.t1 select 4 as a, @@SQL_LOG_BIN'; > +select * from t1 order by a; this is not a test. Please, enable binlog and see whether queries are actually logged. Also, add a test for, say, sql_auto_is_null: set global test_sql_service_execute_sql_global='set sql_auto_is_null=1'; set global test_sql_service_execute_sql_global='insert test.t1 select 5, @@sql_auto_is_null'; select * from t1 where a=5; > +a @@SQL_LOG_BIN > +1 0 > +2 0 > +3 1 > +4 0 > +drop table t1; > set global test_sql_service_run_test= 1; > show status like 'test_sql_service_passed'; > Variable_name Value > diff --git a/sql/sql_prepare.cc b/sql/sql_prepare.cc > index e025147c71e..48681ac0b6f 100644 > --- a/sql/sql_prepare.cc > +++ b/sql/sql_prepare.cc > @@ -5617,12 +5617,30 @@ class Protocol_local : public Protocol_text > THD *new_thd; > Security_context empty_ctx; > > + my_bool do_log_bin; > + > Protocol_local(THD *thd_arg, THD *new_thd_arg, ulong prealloc) : > Protocol_text(thd_arg, prealloc), > cur_data(0), first_data(0), data_tail(&first_data), alloc(0), > - new_thd(new_thd_arg) > + new_thd(new_thd_arg), do_log_bin(FALSE) > {} > > + void set_binlog_vars(my_bool *sav_log_bin, ulonglong *sav_bits) > + { > + *sav_log_bin= thd->variables.sql_log_bin; > + *sav_bits= thd->variables.option_bits; > + > + if ((thd->variables.sql_log_bin= do_log_bin)) > + thd->variables.option_bits|= OPTION_BIN_LOG; > + else > + thd->variables.option_bits&= ~OPTION_BIN_LOG; > + } > + void restore_binlog_vars(my_bool &sav_log_bin, ulonglong &sav_bits) > + { > + do_log_bin= thd->variables.sql_log_bin; > + thd->variables.sql_log_bin= sav_log_bin; > + thd->variables.option_bits= sav_bits; that's what sql_auto_is_null test is for.You destroy all other option_bits here. Better don't save/restore option_bits, but use the same if() as in set_binlog_vars(). Or, even better, reuse fix_sql_log_bin_after_update(). I'd suggest to make it a method in THD > + } > protected: > bool net_store_data(const uchar *from, size_t length); > bool net_store_data_cs(const uchar *from, size_t length, > @@ -6230,12 +6248,17 @@ loc_advanced_command(MYSQL *mysql, enum enum_server_command command, > Ed_connection con(p->thd); > Security_context *ctx_orig= p->thd->security_ctx; > MYSQL_LEX_STRING sql_text; > + my_bool log_bin_orig; > + ulonglong option_bits_orig; > + p->set_binlog_vars(&log_bin_orig, &option_bits_orig); > + > DBUG_ASSERT(current_thd == p->thd); > sql_text.str= (char *) arg; > sql_text.length= arg_length; > p->thd->security_ctx= &p->empty_ctx; > result= con.execute_direct(p, sql_text); > p->thd->security_ctx= ctx_orig; > + p->restore_binlog_vars(log_bin_orig, option_bits_orig); > } > if (skip_check) > result= 0; > @@ -6391,6 +6414,9 @@ extern "C" MYSQL *mysql_real_connect_local(MYSQL *mysql) > new_thd->security_ctx->skip_grants(); > new_thd->query_cache_is_applicable= 0; > new_thd->variables.wsrep_on= 0; > + new_thd->variables.sql_log_bin= 0; > + new_thd->variables.option_bits&= ~OPTION_BIN_LOG; > + > /* > TOSO: decide if we should turn the auditing off > for such threads. Regards, Sergei VP of MariaDB Server Engineering and security(a)mariadb.org

2 1

Re: [Maria-developers] 1c0731e0544: MDEV-26695: Number of an invalid row is not calculated for table value constructor
by Sergei Golubchik 26 Apr '22

26 Apr '22

Hi, Rucha, Looks good. please also check that row_number values match what's in a warning, and then ok to push! On Apr 26, Rucha Deodhar wrote: > On Tue, Apr 26, 2022 at 12:41 AM Sergei Golubchik <serg(a)mariadb.org> wrote: > > what if you combine it with union? > > create table t (a char(1)) values ('a'),('b') union values ('foo'); > > > > please, add a test for that. > > > > > It gives 3 as row the number (in warning). Added a test for it. > Please find it here: 1c0731e0544db4d2f5c107473776bbd6d4d3e2f0 > > > On Apr 25, Rucha Deodhar wrote: > > > revision-id: 1c0731e0544 (mariadb-10.7.1-33-g1c0731e0544) > > > parent(s): 5e6fd4e8043 > > > author: Rucha Deodhar > > > committer: Rucha Deodhar > > > timestamp: 2022-01-23 13:48:18 +0530 > > > message: > > > > > > MDEV-26695: Number of an invalid row is not calculated for table value > > > constructor > > > > > > Analysis: counter does not increment while sending rows for table value > > > constructor and so row_number assumes the default value (0 in this case). > > > Fix: Increment the counter to avoid counter using default value. > > > > > > diff --git a/sql/sql_tvc.cc b/sql/sql_tvc.cc > > > index bdaf6829fbd..7db0ab217ca 100644 > > > --- a/sql/sql_tvc.cc > > > +++ b/sql/sql_tvc.cc > > > @@ -422,7 +422,9 @@ bool table_value_constr::exec(SELECT_LEX *sl) > > > DBUG_ENTER("table_value_constr::exec"); > > > List_iterator_fast<List_item> li(lists_of_values); > > > List_item *elem; > > > + THD *cur_thd= sl->parent_lex->thd; > > > ha_rows send_records= 0; > > > + int rc=0; > > > > > > if (select_options & SELECT_DESCRIBE) > > > DBUG_RETURN(false); > > > @@ -438,10 +440,10 @@ bool table_value_constr::exec(SELECT_LEX *sl) > > > > > > while ((elem= li++)) > > > { > > > + cur_thd->get_stmt_da()->inc_current_row_for_warning(); > > > if (send_records >= sl->master_unit()->lim.get_select_limit()) > > > break; > > > - int rc= > > > - result->send_data_with_check(*elem, sl->master_unit(), > > send_records); > > > + rc= result->send_data_with_check(*elem, sl->master_unit(), > > send_records); > > > > you don't need to do that anymore (it was needed in your first patch, > > but not here) > > > > Removed. > > > > if (!rc) > > > send_records++; > > > else if (rc > 0) > > Regards, Sergei VP of MariaDB Server Engineering and security(a)mariadb.org

1 0

Re: [Maria-developers] 1c0731e0544: MDEV-26695: Number of an invalid row is not calculated for table value constructor
by Sergei Golubchik 25 Apr '22

25 Apr '22

Hi, Rucha, what if you combine it with union? create table t (a char(1)) values ('a'),('b') union values ('foo'); please, add a test for that. On Apr 25, Rucha Deodhar wrote: > revision-id: 1c0731e0544 (mariadb-10.7.1-33-g1c0731e0544) > parent(s): 5e6fd4e8043 > author: Rucha Deodhar > committer: Rucha Deodhar > timestamp: 2022-01-23 13:48:18 +0530 > message: > > MDEV-26695: Number of an invalid row is not calculated for table value > constructor > > Analysis: counter does not increment while sending rows for table value > constructor and so row_number assumes the default value (0 in this case). > Fix: Increment the counter to avoid counter using default value. > > diff --git a/sql/sql_tvc.cc b/sql/sql_tvc.cc > index bdaf6829fbd..7db0ab217ca 100644 > --- a/sql/sql_tvc.cc > +++ b/sql/sql_tvc.cc > @@ -422,7 +422,9 @@ bool table_value_constr::exec(SELECT_LEX *sl) > DBUG_ENTER("table_value_constr::exec"); > List_iterator_fast<List_item> li(lists_of_values); > List_item *elem; > + THD *cur_thd= sl->parent_lex->thd; > ha_rows send_records= 0; > + int rc=0; > > if (select_options & SELECT_DESCRIBE) > DBUG_RETURN(false); > @@ -438,10 +440,10 @@ bool table_value_constr::exec(SELECT_LEX *sl) > > while ((elem= li++)) > { > + cur_thd->get_stmt_da()->inc_current_row_for_warning(); > if (send_records >= sl->master_unit()->lim.get_select_limit()) > break; > - int rc= > - result->send_data_with_check(*elem, sl->master_unit(), send_records); > + rc= result->send_data_with_check(*elem, sl->master_unit(), send_records); you don't need to do that anymore (it was needed in your first patch, but not here) > if (!rc) > send_records++; > else if (rc > 0) Regards, Sergei VP of MariaDB Server Engineering and security(a)mariadb.org

1 0

Re: [Maria-developers] 585cd1f52e7: MDEV-26875: Wrong user in SET DEFAULT ROLE error
by Sergei Golubchik 25 Apr '22

25 Apr '22

Hi, Anel, On Apr 25, Anel Husakovic wrote: > revision-id: 585cd1f52e7 (mariadb-10.5.14-10-g585cd1f52e7) > parent(s): 52b32c60c26 > author: Anel Husakovic > committer: Anel Husakovic > timestamp: 2022-02-14 13:59:24 +0100 > message: > > MDEV-26875: Wrong user in SET DEFAULT ROLE error > > - Caused by 7c02e8717de5, where 957cb7b7ba35 introduced the bug. > > Reviewed by: > > diff --git a/sql/sql_acl.cc b/sql/sql_acl.cc > index c4e66cf5d73..e83bc5635dc 100644 > --- a/sql/sql_acl.cc > +++ b/sql/sql_acl.cc > @@ -3277,10 +3277,14 @@ static int check_user_can_set_role(THD *thd, const char *user, > check_role_is_granted_callback, > NULL) == -1)) > { > - /* Role is not granted but current user can see the role */ > - my_printf_error(ER_INVALID_ROLE, "User %`s@%`s has not been granted role %`s", > - MYF(0), thd->security_ctx->priv_user, > - thd->security_ctx->priv_host, rolename); > + /* If the SET ROLE is applied on the anonymous user, host is null */ > + if (!host) > + my_printf_error(ER_INVALID_ROLE, "User %`s@%`s has not been granted role %`s", > + MYF(0), thd->security_ctx->priv_user, thd->security_ctx->priv_host, rolename); I don't understand it. 1) what does it mean? why for anonymous user you print priv_user@priv_host ? 2) do you have any tests for that? There are none in the commit > + else > + /* Role is not granted but current user can see the role */ > + my_printf_error(ER_INVALID_ROLE, "User %`s@%`s has not been granted role %`s", > + MYF(0), user, host, rolename); > } > else > { Regards, Sergei VP of MariaDB Server Engineering and security(a)mariadb.org

1 0

Re: [Maria-developers] 5f8ffc7271a: MDEV-20207: Assertion `! is_set()' failed in Diagnostics_area::set_eof_status upon HANDLER READ
by Sergei Golubchik 25 Apr '22

25 Apr '22

Hi, Rucha, The fix is ok to push. But as a general comment, better try to define variables in the smallest scope where they're used. In particular here it could've been { MY_BITMAP *old_map= dbug_tmp_use_all_columns(table, &table->write_set); - (void) item->save_in_field(key_part->field, 1); + int res= item->save_in_field(key_part->field, 1); because you don't need `res` outside of that scope. Makes the code easier to read. Of course, there can be exceptions. For example, if the variable is an object and its initialization is very expensive. But whether you move `int res` declaration inside or not - it's ok to push, no need to ask for another review. Regards, Sergei VP of MariaDB Server Engineering and security(a)mariadb.org On Apr 25, Rucha Deodhar wrote: > revision-id: 5f8ffc7271a (mariadb-10.2.40-251-g5f8ffc7271a) > parent(s): 2cbf92522b5 > author: Rucha Deodhar > committer: Rucha Deodhar > timestamp: 2022-01-27 01:18:19 +0530 > message: > > MDEV-20207: Assertion `! is_set()' failed in Diagnostics_area::set_eof_status > upon HANDLER READ > > Analysis: The error state is not stored while checking condition and key name. > Fix: Return true while checking condition and key name if error is reported > because geometry object can't be created from the data in the index value for > HANDLER READ. > > diff --git a/sql/sql_handler.cc b/sql/sql_handler.cc > index ec3756eceba..7c002a68d9e 100644 > --- a/sql/sql_handler.cc > +++ b/sql/sql_handler.cc > @@ -554,6 +554,7 @@ mysql_ha_fix_cond_and_key(SQL_HANDLER *handler, > { > THD *thd= handler->thd; > TABLE *table= handler->table; > + int res= 0; > if (cond) > { > /* This can only be true for temp tables */ > @@ -613,8 +614,10 @@ mysql_ha_fix_cond_and_key(SQL_HANDLER *handler, > if (!in_prepare) > { > MY_BITMAP *old_map= dbug_tmp_use_all_columns(table, &table->write_set); > - (void) item->save_in_field(key_part->field, 1); > + res= item->save_in_field(key_part->field, 1); > dbug_tmp_restore_column_map(&table->write_set, old_map); > + if (res) > + return 1; > } > key_len+= key_part->store_length; > keypart_map= (keypart_map << 1) | 1;

1 0

Re: [Maria-developers] 14cc679c95e: MDEV-27831 Let the SQL SERVICE user set the current user name.
by Sergei Golubchik 22 Apr '22

22 Apr '22

Hi, Alexey, On Mar 04, Alexey Botchkov wrote: > revision-id: 14cc679c95e (mariadb-10.7.2-8-g14cc679c95e) > parent(s): 33fd136c61b > author: Alexey Botchkov > committer: Alexey Botchkov > timestamp: 2022-02-15 11:35:18 +0400 > message: > > MDEV-27831 Let the SQL SERVICE user set the current user name. > > The 'user' argument added to the mysql_real_connect_local. I think this is wrong on many levels. A plugin name is already known inside the plugin, you shouldn't force the plugin to pass its own name as an argument in multiple places, the server should determine it automatically. You should not set current_user to an arbitrary string for audit plugin to see it. current_user is the name of the user account and it's used in many places as such. Try, for example, to create a view or a stored procedure. Who will be a definer? Setting only user() might be ok. Setting @@proxy_user or @@external_user is even better, if your audit plugin can show them. @@external_user would be the best, I think it's purely informational. Regards, Sergei VP of MariaDB Server Engineering and security(a)mariadb.org

2 3

Re: [Maria-developers] 96b40a5e2f5: MDEV-27744 InnoDB: Failing assertion: !cursor->index->is_committed() in row0ins.cc (from row_ins_sec_index_entry_by_modify) | Assertion `0' failed in row_upd_sec_index_entry (debug) | Corruption
by Sergei Golubchik 22 Apr '22

22 Apr '22

Hi, Alexander, Some comments/questions here. I'll need to look at the patch again after I understand why you did it this way better. On Apr 18, Alexander Barkov wrote: > revision-id: 96b40a5e2f5 (mariadb-10.3.33-61-g96b40a5e2f5) > parent(s): 7355f7b1f5c > author: Alexander Barkov > committer: Alexander Barkov > timestamp: 2022-04-07 16:22:17 +0400 > message: > > MDEV-27744 InnoDB: Failing assertion: !cursor->index->is_committed() in row0ins.cc (from row_ins_sec_index_entry_by_modify) | Assertion `0' failed in row_upd_sec_index_entry (debug) | Corruption A detailed comment here is a must. What was the problem, how it was fixed. > diff --git a/sql/item_func.h b/sql/item_func.h > index 754b1cd1eb2..d844e45280a 100644 > --- a/sql/item_func.h > +++ b/sql/item_func.h > @@ -35,6 +35,36 @@ extern "C" /* Bug in BSDI include file */ > #include <cmath> > > > +template<class FUNC> > +FUNC* item_func_create_2_3_args(THD *thd, const LEX_CSTRING &name, > + List<Item> *item_list) > +{ > + int arg_count= item_list ? item_list->elements : 0; > + > + switch (arg_count) { > + case 2: > + { > + Item *param_1= item_list->pop(); > + Item *param_2= item_list->pop(); > + return new (get_thd_memroot(thd)) FUNC(thd, param_1, param_2); > + } > + case 3: > + { > + Item *param_1= item_list->pop(); > + Item *param_2= item_list->pop(); > + Item *param_3= item_list->pop(); > + return new (get_thd_memroot(thd)) FUNC(thd, param_1, param_2, param_3); A constructor that takes a List<Item> wouldn't need a switch. And wouldn't need a template either, your create_check_args_ge() isn't a template. > + break; > + } > + default: > + my_error(ER_WRONG_PARAMCOUNT_TO_NATIVE_FCT, MYF(0), name.str); > + break; > + } > + > + return NULL; > +} > + > + > class Item_func :public Item_func_or_sum > { > void sync_with_sum_func_and_with_field(List<Item> &list); > @@ -56,8 +86,41 @@ class Item_func :public Item_func_or_sum > bool check_argument_types_can_return_text(uint start, uint end) const; > bool check_argument_types_can_return_date(uint start, uint end) const; > bool check_argument_types_can_return_time(uint start, uint end) const; > + > + void print_schema_name_if_needed(String *to) const > + { > + const LEX_CSTRING schema_name= schema_name_cstring(); > + if (schema_name.length) > + { > + to->append(schema_name); > + to->append('.'); > + } this should check that schema_name isn't the same as the schema in the path (implicit path, assumed by the current sql_mode). > + } > + void print_maybe_qualified_name(String *to) const > + { > + print_schema_name_if_needed(to); > + to->append(func_name()); > + } > + > public: > > + // Print an error message for a builtin-schema qualified function call > + static void wrong_param_count_error(const LEX_CSTRING &schema_name, > + const LEX_CSTRING &func_name); > + > + // Check that the number of arguments is greater or equal to "expected" > + static bool create_check_args_ge(const LEX_CSTRING &name, > + const List<Item> *item_list, > + uint expected) > + { > + if (!item_list || item_list->elements < expected) > + { > + my_error(ER_WRONG_PARAMCOUNT_TO_NATIVE_FCT, MYF(0), name.str); what's the point of your wrong_param_count_error() if you aren't using it? > + return true; > + } > + return false; > + } > + > table_map not_null_tables_cache; > > enum Functype { UNKNOWN_FUNC,EQ_FUNC,EQUAL_FUNC,NE_FUNC,LT_FUNC,LE_FUNC, > diff --git a/sql/item_strfunc.cc b/sql/item_strfunc.cc > index 75abd9906cd..e45fe6ef6d7 100644 > --- a/sql/item_strfunc.cc > +++ b/sql/item_strfunc.cc > @@ -3204,6 +3239,39 @@ static String *default_pad_str(String *pad_str, CHARSET_INFO *collation) > return pad_str; > } > > + > +Item_func_lpad* > +Item_func_lpad::create(THD *thd, const LEX_CSTRING &name, > + List<Item> *item_list) > +{ > + return item_func_create_2_3_args<Item_func_lpad>(thd, name, item_list); may be item_func_create_2_3_args should be in Item_func? Like create_check_args_ge() is? and may be create_check_args_ge() should be a template too? > +} > + > + > +Item_func_lpad_oracle* > +Item_func_lpad_oracle::create(THD *thd, const LEX_CSTRING &name, > + List<Item> *item_list) > +{ > + return item_func_create_2_3_args<Item_func_lpad_oracle>(thd, name, item_list); > +} > + > + > +Item_func_rpad* > +Item_func_rpad::create(THD *thd, const LEX_CSTRING &name, > + List<Item> *item_list) > +{ > + return item_func_create_2_3_args<Item_func_rpad>(thd, name, item_list); > +} > + > + > +Item_func_rpad_oracle* > +Item_func_rpad_oracle::create(THD *thd, const LEX_CSTRING &name, > + List<Item> *item_list) > +{ > + return item_func_create_2_3_args<Item_func_rpad_oracle>(thd, name, item_list); > +} > + > + > bool Item_func_pad::fix_length_and_dec() > { > if (arg_count == 3) > diff --git a/sql/sql_lex.cc b/sql/sql_lex.cc > index 7f1b362d91d..6b0a351f76e 100644 > --- a/sql/sql_lex.cc > +++ b/sql/sql_lex.cc > @@ -1542,7 +1541,18 @@ int Lex_input_stream::lex_one_token(YYSTYPE *yylval, THD *thd) > if (lex->parsing_options.lookup_keywords_after_qualifier) > next_state= MY_LEX_IDENT_OR_KEYWORD; > else > - next_state= MY_LEX_IDENT_START; // Next is ident (not keyword) > + { > + /* > + Next is: > + - A qualified func with a special syntax: > + mariadb_schema.REPLACE('a','b','c') > + mariadb_schema.SUSTRING('a',1,2) > + mariadb_schema.TRIM('a') > + - Or an identifier otherwise. No keyword lookup is done, > + all keywords are treated as identifiers. > + */ > + next_state= MY_LEX_IDENT_OR_QUALIFIED_SPECIAL_FUNC; I don't understand why you did it in the lexer and not in the parser, like - | REPLACE '(' expr ',' expr ',' expr ')' + | opt_schema REPLACE '(' expr ',' expr ',' expr ')' or (if the above won't work in LARL(1) parser) - | ident_cli '.' ident_cli '(' opt_expr_list ')' + | ident_cli '.' func_2nd_part with + func_2nd_part: ident_cli '(' opt_expr_list ')' + | REPLACE '(' expr ',' expr ',' expr ')' etc (not reviewing lexer changes anymore, until I understand the above) > + } > if (!ident_map[(uchar) yyPeek()]) // Probably ` or " > next_state= MY_LEX_START; > return((int) c); Regards, Sergei VP of MariaDB Server Engineering and security(a)mariadb.org

2 4