February 2017 - developers - lists.mariadb.org

[Maria-developers] Windows line endings in MariaDB sources again (
by Otto Kekäläinen 23 Jul '19

23 Jul '19

Hello! While trying to release 10.0.26 for Debian and Ubuntu I noticed git chokes again on Windows line endings. As our .gitattributes states that by default git should clean up the line endings, why do we keep getting new Windows line endings in new code? mariadb-10.0$ cat .gitattributes # Set the default behavior, in case people don't have core.autocrlf set. * text=auto ... storage/connect/mysql-test/connect/std_data/*.txt -text mariadb-10.0.26$ find . | xargs file | grep CRLF ./storage/connect/mysql-test/connect/std_data/boyswin.txt: ASCII text, with CRLF line terminators ./storage/connect/mysql-test/connect/std_data/expenses.txt: ASCII text, with CRLF line terminators ./storage/connect/mysql-test/connect/std_data/emp.txt: ASCII text, with CRLF line terminators ./mysql-test/std_data/loaddata7.dat: ASCII text, with CRLF line terminators ./mysql-test/r/loadxml.result: ASCII text, with CRLF, LF line terminators ./mysql-test/r/perror-win.result: ASCII text, with CRLF, LF line terminators ./mysql-test/r/mysql_binary_mode.result: ASCII English text, with CRLF, LF line terminators ./mysql-test/r/func_regexp_pcre.result: UTF-8 Unicode C++ program text, with CRLF, CR, LF line terminators ./pcre/testdata/grepoutputN: ASCII text, with CRLF, CR, LF line terminators ./pcre/testdata/greppatN4: ASCII text, with CRLF line terminators This leads for me to: dpkg-source: info: building mariadb-10.0 using existing ./mariadb-10.0_10.0.26.orig.tar.gz dpkg-source: info: local changes detected, the modified files are: mariadb-10.0/storage/connect/mysql-test/connect/std_data/boyswin.txt mariadb-10.0/storage/connect/mysql-test/connect/std_data/emp.txt mariadb-10.0/storage/connect/mysql-test/connect/std_data/expenses.txt dpkg-source: error: aborting due to unexpected upstream changes, see /tmp/mariadb-10.0_10.0.26-1.diff.Yzwwhw Which means I have to do extra work with importing the upstream sources.. - Otto

2 4

Re: [Maria-developers] Mdev-10664 Add statuses about optimistic parallel replication stalls.
by Sachin Setiya 19 Apr '18

19 Apr '18

Hi Kristian! Sorry for late reply , I was on vacations. On Tue, Jan 10, 2017 at 7:31 PM, Kristian Nielsen <knielsen(a)knielsen-hq.org> wrote: > Sachin Setiya <sachin.setiya(a)mariadb.com> writes: > > >> Right, but you still need _some_ way to synchronise multiple threads > > > I used atomic builtins my_atomic_add64_explicit. > > That seems fine (but see comment below). > > > Documentation:- > > > > Slave_DDL_Events:- > > > > Description: Number of DDL statements slave have executed in > > replication channel X. > > Scope: Per Replication Channel > > Data Type: numeric > > Introduced: MariaDB 10.1 > > > > Slave_Non_Transactional_Events:- > > > > Description: Number of Non Transactional "no idea what should be the > > next word " slave have executed in replication channel X. > > Scope: Per Replication Channel > > Data Type: numeric > > Introduced: MariaDB 10.1 > > I checked, the term "event group" is used already multiple places in the > documentation, so I think it can be used here also. An event group can be a > transaction (possibly with multiple transactional DML statements), or a DDL > or non-transactional statement. > > What you are doing is introducing status variables to help understand > optimistic (and aggressive) parallel replication modes. > > In optimistic parallel replication, a non-transactional event group is not > safe to run in parallel with other non-transactional events. So that event > is made to wait for all prior event groups to reach commit state, before > being allowed to execute. However, other transactional events are not > delayed by the non-transactional event group. > > Slave_Non_Transactional_Events counts the occurences of this. > > Then in case of a DDL statement, this can potentially be unsafe to run in > parallel even with other transactional events. So for a DDL, the DDL has to > wait for all prior event groups, _and_ subsequent event groups have to wait > for the DDL. > > Slave_DDL_Events counts these occurences. > > This is what the documentation, and commit messages, should explain, I > think. > > I think the status variables describe the values for > @@default_master_connection?, this should also be explained in the > documentation. > > Why did you decide to put this information into a status variable? > Normally, > slave status is seen in SHOW SLAVE STATUS, which supports showing status > for > a particular connection without using @@default_master_connection. > > Sorry for this , I guess It was a confusion I was looking at Slave_Running this is avaliable in show status as well as show slave status. So I thought for if I want to show some information in SHOW slave status this has to be in show status. And this is wrong. Now we can see non trans events / ddl events in show slave status. Show I remove this vars from Show Status, or it is okay ? > > diff --git a/sql/log_event.h b/sql/log_event.h > > index 90900f6..579607f 100644 > > --- a/sql/log_event.h > > +++ b/sql/log_event.h > > @@ -50,6 +50,7 @@ > > #endif > > > > #include "rpl_gtid.h" > > +#include "my_atomic.h" > > Why include my_atomic.h in the header file, when you only use it in the .cc > file? > Changed. > > > + mysql_mutex_lock(&LOCK_active_mi); > > + if (master_info_index) > > + { > > + mi= master_info_index-> > > + get_master_info(&thd->variables.default_master_connection, > > + Sql_condition::WARN_LEVEL_NOTE); > > + if (mi) > > + tmp= mi->total_ddl_events; > > > + if (mi) > > + tmp= mi->total_non_trans_events; > > Better use an atomic primitive here to read the value. Taking a mutex does > not help synchronise against my_atomic_add64_explicit(). > > Changed. > > + } > > + mysql_mutex_unlock(&LOCK_active_mi); > > + if (mi) > > + *((longlong *)buff)= tmp; > > + else > > + var->type= SHOW_UNDEF; > > + > > + return 0; > > +} > > #endif /* HAVE_REPLICATION */ > > > > static int show_open_tables(THD *thd, SHOW_VAR *var, char *buff, > > > index 9365c06..a474aa5 100644 > > --- a/sql/rpl_mi.h > > +++ b/sql/rpl_mi.h > > @@ -303,6 +303,12 @@ class Master_info : public > Slave_reporting_capability > > > > /* The parallel replication mode. */ > > enum_slave_parallel_mode parallel_mode; > > + > > + /* No of DDL statements */ > > + long total_ddl_events; > > + > > + /* No of non-transactional statements*/ > > + long total_non_trans_events; > > "long" is not the appropriate type here. Certainly not since you are using > my_atomic_add64_explicit(). Use a type that is the same size on all > platforms (and why use a signed type?). > > Changed to uint64. > - Kristian. > Attached a newer version of patch , please review it. -- Regards Sachin

3 7

[Maria-developers] Please review MDEV-11292 Item_func_sp::fix_length_and_dec() does not set collation derivation properly
by Alexander Barkov 20 Nov '17

20 Nov '17

Hello Alexey, Please review a patch for MDEV-11292 (for 10.3). Thanks.

2 2

[Maria-developers] MDEV-10306 Wrong results with combination of CONCAT, SUBSTR and CONVERT in subquery
by Alexander Barkov 16 Jun '17

16 Jun '17

Hello Sergei, Please review a fix for MDEV-10306. Thanks!

4 13

[Maria-developers] MDEV-10142 - bb-10.2-compatibility / MDEV-10574
by jerome brauge 28 Apr '17

28 Apr '17

Hello Alexander, Sometime ago, when I ask you about plan for MDEV-10574, you replied : >The current plan is to do these transformations: > >1. Transform Insert >- insert values ("") -> insert values (null) > >2. Transform Select > >- where v=x => (v <> "" and V=X) >- where v is null => (v="" or v is null) > >We didn't plan to change functions yet. Thanks for bringing this up. >We'll discuss this. I've done some tests just by changing : - insert an Item_null instead of an Item_string when $1.length==0 in rule text_literal of sql_yacc_ora.yy - return null instead of an empty string in Item_str_func::make_empty_result My first tests seem promising. Of course this solution does not allow to "see" the records created with empty strings as null values. I don't see the importance of being able to do this in a transparent way. We can explicitly select these row by adding rtrim on these columns. If you are interesting, I can begin to write a test to evaluate the coverage of this solution. Best regard. Jérôme.

2 6

[Maria-developers] Item_func_group_concat::fix_fields - indentation and check positioning?
by Daniel Black 18 Apr '17

18 Apr '17

I came across the following gcc warning (6.3.1) sql/item_sum.cc: In member function ‘virtual bool Item_func_group_concat::fix_fields(THD*, Item**)’: sql/item_sum.cc:3478:5: warning: this ‘if’ clause does not guard... [-Wmisleading-indentation] if ((!args[i]->fixed && ^~ sql/item_sum.cc:3482:7: note: ...this statement, but the latter is misleadingly indented as if it is guarded by the ‘if’ with_subselect|= args[i]->with_subselect; ^~~~~~~~~~~~~~ The with_subselect line was added in 950abd526 In context it is: Item_func_group_concat::fix_fields(THD *thd, Item **ref) { ... for (i=0 ; i < arg_count ; i++) { if ((!args[i]->fixed && args[i]->fix_fields(thd, args + i)) || args[i]->check_cols(1)) return TRUE; with_subselect|= args[i]->with_subselect; } While removing the indent would fix the warning, I'd like someone to check this line should be after the IF statement rather than before.

2 1

[Maria-developers] MDEV-10598 - bb-10.2-compatibility
by jerome brauge 13 Mar '17

13 Mar '17

Hello Alexander, I've done this patch for MDEV-10598. Can you review it ? Regards, Jérôme.

2 11

[Maria-developers] INFORMATION_SCHEMA.SCHEMATA and INFORMATION_SCHEMA.SCHEMA_PRIVILEGES
by Witold Filipczyk 09 Mar '17

09 Mar '17

Hi, I'm newbie, I have problem with INFORMATION_SCHEMA. There is mariadb-10.0 with around 100000 databases. Disks are slow, but most users have one database, a few have more than one, but it is not a big number. Queries like SHOW DATABASES or SELECT * FROM INFORMATION_SCHEMA.SCHEMATA bring to knees whole server. The idea to resolve it is: in find_files(...) instead of the for loop execute query: SELECT REPLACE(TABLE_SCHEMA, '\\', '') FROM INFORMATION_SCHEMA.SCHEMA_PRIVILEGES WHERE PRIVILEGE_TYPE = 'SELECT'; There are databases with names with _ in them and \_ in GRANTS. This is theory, but I don't know how to do it. That's mean how to freeze the current query, how to run new query, read results, and fill files in the find_files function and resume the current query. Could you help? Or do you have ideas how to solve it? -- Witold Filipczyk

2 3

[Maria-developers] MDEV-11829 RFC3339 Support
by Seth Shelnutt 05 Mar '17

05 Mar '17

Hello, This is in response to the comments on the initial pull request I made for adding RFC3339 support [1]. The proposal is to add support for parsing RFC3339 and then use the given timezone offset to shift the datetime. I think most users would expect the behavior to be converted to the timezone of the mariadb server / session as defined in THD. Most timezone operations are depended on the session specified timezone, whether that is set or inherited from global timezone, or system timezone. In order to convert from RFC3339 timezone to session timezone, we will need to return the timezone from the str_to_datetime function. As mentioned there is no timezone member of the MY_TIME struct. I believe a change to the MY_TIME struct would be the most beneficial and cleanest approach. This would allow for handling RFC3339 and future additions, such as a datetime with time zone field, similar to the ansi timestamp with time zone. The struct is used in a several ways, including packing into longlong, all of those cases would need to be handled. This would be a much larger and more involved patchset. An alternative approach, that is less involved, would be use the MYSQL_TIME_STATUS struct to indicate that a timezone offset exists. An offset field and indicator could be added right to the status struct, or a new warning can be used to indicate it exists in the raw string. Then the caller of str_to_datetime can handle the timezone conversion. A third approach, would be to change the function signature of str_to_datetime and provide a variable to store the offset in, if exists. I'm sure what the general stance is on altering entire function calls, especially one that is older and quite important. With any approach, the timezone could then be handled outside of my_time.c, where we have THD and the timezone class, which can easily handle the conversion. I will start working on this once I receive some feedback on the general direction that would result in an acceptable patchset. [1] https://github.com/MariaDB/server/pull/290 Thank you, Seth Shelnutt

3 4

Re: [Maria-developers] e8ca684: MDEV-11221 - main.events_restart failed in bb
by Sergei Golubchik 28 Feb '17

28 Feb '17

Hi, Sergey! Thanks! Ok to push! On Feb 28, Sergey Vojtovich wrote: > revision-id: e8ca684c6d5ce14eb021b4b691be50a8bf9c8281 (mariadb-10.0.29-38-ge8ca684) > parent(s): b54566d73bb96ef9eb7d734742d3743fb3369a40 > committer: Sergey Vojtovich > timestamp: 2017-02-28 15:23:44 +0400 > message: > > MDEV-11221 - main.events_restart failed in bb > > Applied lost in a merge revision: next time, please specify a commit you're applying. Like Applied commit 7f38a070baa that was lost in a merge: > MDEV-10043 - main.events_restart fails sporadically in buildbot (crashes upon > shutdown) > > There was race condition between shutdown thread and event worker threads. > > Shutdown thread waits for thread_count to become 0 in close_connections(). It > may happen so that event worker thread was started but didn't increment > thread_count by this time. In this case shutdown thread may miss wait for this > working thread and continue deinitialization. Worker thread in turn may continue > execution and crash on deinitialized data. > > Fixed by incrementing thread_count before thread is actually created like it is > done for connection threads. > > Also let event scheduler not to inc/dec running threads counter for symmetry > with other "service" threads. > Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0