- developers - lists.mariadb.org

Re: [Maria-developers] [GSoC] self-tuning optimizer
by Sergei Golubchik 13 Jun '14

13 Jun '14

Hi, Anshu! On Jun 13, Anshu Avinash wrote: > > So I have refactored Cost_factors::init() into Cost_factors::init() and > Cost_factors::re_init(). The diff is at: > https://github.com/igniting/server/commit/41e360309c067d072d788495e4e948054… Looks great. I've added one comment there. > I had started writing a test for checking whether if loading a new > storage engine works or not. I loaded 'Example' storage engine, but > was unable to test the current t1 table on it, as t1 contains a > primary key. Also I need to write test cases for 'time_for_compare'. > Any hints on that? Hm. Yes. Example engine, is pretty useless for you, as you cannot create tables in it. You can use InnoDB plugin. But create a separate test file for that. For example (I'll use the short name below, don't commit it): a.test: ============================================== if (!$HA_INNODB_SO) { skip Need InnoDB plugin; } install plugin innodb soname 'ha_innodb'; ... do your tests here uninstall plugin innodb; ============================================== you will also need a.opt file: ============================================== --ignore-builtin-innodb --loose-innodb ============================================== As for time_for_compare... Here's a universal approach, you can do it yourself. This time_for_compare is 5 by default, right? Change it to be, say, 500, and run existing tests (start from the main suite, and there from optimizer tests, e.g. from select.test). If some test will fail - here's your query, extract it, simplify, perhaps, and put in your test file. That's exactly what I just did :) And here's a simple test: create table t1 (a blob, index(a(20))); create table t2 (a blob, index(a(20))); insert into t1 values ('one'),('two'),('three'),('four'),('five'); insert into t2 values ('one'),('two'),('three'),('four'),('five'); insert into t2 values ('one'),('two'),('three'),('four'),('five'); drop table t1, t2; Perhaps you can get it down to one table - I didn't try. > Also I think I can start with the next part, i.e. collect statistics. We > can have a structure like: > > struct Cost_factor_stats > { > char *cost_factor_name; > double cost_factor_value; > double total_queries; total_queries should be ulonglong, not double. > double total_query_time; > double total_query_time_squared; > }; > > We will be updating these structures after every query per THD and write > the data into a persistent table on disconnect. Is this workflow correct? Almost. You need a Cost_factor_stats structure per cost factor (and per engine where appropriate). You don't need the name there. You have these structures per THD and globally. Basically, instead of your current double time_for_compare_val; you will have Cost_factor_stats time_for_compare_val; hmm, perhaps better to name the structure simply "Cost_factor": Cost_factor time_for_compare; And you put them all in one structure: struct Engine_cost_factors { Cost_factor read_time; Cost_factor scan_time; }; struct Global_cost_factors { Cost_factor time_for_compare; Cost_factor time_for_compare_rowid; }; class Cost_factors { Global_cost_factors global; Engine_cost_factors engine[MAX_HA]; double time_for_compare() const { return global.read_time.value; } double time_for_compare_rowid() const; double read_time(const handler*) const; double scan_time(const handler*) const; }; Now, you stick this structure in the THD, and also declare one instance globally. On connect you initialize THD::Cost_factor values from the global Cost_factor object. On disconnect you add THD::Cost_factor values back to the global Cost_factor object. There's no writing to disk so far, we'll think about it later (you don't want to write to disk on *every* disconnect). > Few more questions: > 1.What will we do if the query had used more than one cost factor? > 2.How would joins be handled? Yes, a query will certainly use more than one cost factor. In fact, this is something we might want to experiment with. I see two approaches - exact and approximate. Approximate one is described in MDEV. Say, you've counted 500 index reads, 20,000 sequential row reads (table scan), some 10,000,000 comparisons, etc. And you know how long the query took. For another query you've counted different numbers, and so on. Basically, you'll have a system of linear equations that one can solve to know how the time was split. The problem with this approach - the server is doing much more than purely index reads, table scans, and comparisons. So this unaccounted time will be split between cost factors. I don't know if it's a problem. The exact approach - we only measure times spent on the actual operation. Basically, you measure how long *each row read* took, how long *each index read* took, etc. This will show no unaccounted time. But it's more expensive, it might be too expensive to be practically useful - I don't know that. These two approaches need different data structures, the first one needs more memory, because you cannot start aggregating before you solve the system of equations. Basically, if you have four constant (time_for_compare, time_for_compare_rowid, read_time, scan_time, only one engine), you'll need four equations, that is, per THD you need to store four measurements: struct measurement { ulong time_for_compare; ulong time_for_compare_rowid; struct per_engine { ulong scan_time; ulong read_time; } per_engine[MAX_HA]; double total_time; }; and in THD you need struct measurement data[4]; Also, note that I'm writing per_engine[MAX_HA], but it's a simplification, good enough for a prototype. But in the final code we cannot allocate that much memory per THD, as most of the time we'll have only few engines, not MAX_HA. Regards, Sergei

1 0

[Maria-developers] WebScaleSQL: Adjust linking flags to not embed some libs
by Sergey Vojtovich 13 Jun '14

13 Jun '14

Hi Sergei, during our recent discussion re WebScaleSQL patches we agreed to accept $subj: https://github.com/webscalesql/webscalesql-5.6/commit/361e3b83145402945b737… I studied our binaries and it looks like we embed zlib and ssl libraries only if WITH_ZLIB=bundled/WITH_SSL=bundled. E.g. DEB/RPM packages use system libs and do not embed zlib/ssl, whereas tarball uses bundled zlib/yassl and embeds them. Not embedding yassl into the client library will make the latter almost useless. Same probably applies to zlib even though it is normally offered by OS-es. Said the above I believe our build scripts behave properly and we should skip this patch. What's your take? Regards, Sergey

2 1

[Maria-developers] Java Connector Coding till 12th June
by Puneet Dewan 13 Jun '14

13 Jun '14

Hi Everyone, This week, I was able to send the com_stmt_execute command from the *Driver * in the way it is accepted by the MariaDB server So, Now I am able to insert a row in the database using com_stmt_execute command. But from the console when I try to see the values I see *NULL* I used *Wireshark* to capture the packets,but I can see the value(Varchar /Int/..etc) is passed in the packet which wireshark captures. I am investigating the code but could not find anything till now. In Wireshark it shows* [Expert Info (warn/undecoded) :FIXME: Excute Dissector Incomplete]* Now I think I am not sending the values in a correct format.Will look and try to resolve it. Regards Puneet. On Thu, Jun 5, 2014 at 10:42 PM, Puneet Dewan <puneetd30(a)gmail.com> wrote: > Hi Everyone, > > The work done till 5th June: > 1.I started studying the connector immediately after my semester exams got > over(5th May). > 2. Understood the relationship between different classes. > 3.Read the MySql documentation for text protocol and map the commands from > documentation onto code.This thing gave a clear idea about how things are > moving from client to server. > 4.Setup the code on Ubuntu using eclipse and tested against database. > Setup the Team branch in launchpad. > 5.Debug the test classes to see how different classes are interlinked and > understood that MySQLProtocol.java is the heart of our JDBC driver. > 6.Then I started reading Client Server Protocol documentation i.e > ServerSide prepared statement and try to see how and where I have to code > it. > 7.Then I saw that there is a class MySqlServerSidePreparedStatement.java, > Initially I coded the prepareStatement() ,close(),and execute() in that > class seperately. > But after (Code review) discussion with Massimo, we donot need to use that > class. > Instead use MySqlPreparedStatement.java to do the same work.So made > changes accordingly. > 8.I had pushed some of the code which I have written to launchpad and > discussed with Massimo last week. > 9.Last week I was struck with Wireshark on windows and ubuntu both. > I was unable to see the packets going to database and seeing the response. > 10.On Monday the problem was resolved as I was using the wrong interface.I > should have used loopback.io(127.0.0.1) instead on ubuntu(thanks to > Georg). > Unfortunately the thing does not work on windows.Tried a lot many times. > 11.Now I am working on the execute method of client server protocol as I > am getting exception while using that method.Wireshark is playing a great > role in how the packets are flowing. > > From now I will do weekly blogging and keep everyone updated. > > Thanks & Regards > Puneet. >

1 0

[Maria-developers] Allow to turn off max_connect_errors
by Honza Horak 13 Jun '14

13 Jun '14

Hi guys, there are apparently some tools out there [1], that check if the server is up quite often, while not closing the connection properly. It eventually ends in 'many connection errors', because max_connect_errors is always limited now. I understand that this way of checking may be wrong, but there may be scenarios where we do not want to check for `max_connect_errors` at all. So, would it be acceptable for mariadb to change behaviour of max_connect_errors option, so that it accepts also 0 as a possible value, which would mean 'do not check connect errors at all'? I'm bringing the idea here first, but will submit a report and possibly patch if it does not seem to be undesired behaviour. [1] https://bugzilla.redhat.com/show_bug.cgi?id=1104957 TIA and regards, Honza

4 4

Re: [Maria-developers] [Commits] Rev 4018: MDEV-4784 - merge test cases from 5.6 in lp:maria/10.0
by Sergei Golubchik 12 Jun '14

12 Jun '14

Hi, Sergey! > === added file 'mysql-test/include/have_daemon_example_plugin.inc' > --- a/mysql-test/include/have_daemon_example_plugin.inc 1970-01-01 00:00:00 +0000 > +++ b/mysql-test/include/have_daemon_example_plugin.inc 2014-04-14 09:20:24 +0000 > @@ -0,0 +1,16 @@ > +disable_query_log; > +# > +# Check if server has support for loading plugins > +# > +if (`SELECT @@have_dynamic_loading != 'YES'`) { > + --skip daemon example plugin requires dynamic loading > +} > + > +# > +# Check if the variable DAEMONEXAMPLE is set > +# > +if (!$LIBDAEMON_EXAMPLE_SO) { > + --skip daemon_example plugin requires the environment variable \$DAEMONEXAMPLE to be set (normally done by mtr) Eh? It was in your first patch. I thought it's an additional patch, but it looks like it includes the first one? Okay, I will copy all my comments from the first patch here to have a complete review in one email. So, here I said ************************ it's always done by mtr, a better error message should be "No libdaemon_example.so was found" or simply "Need daemon_example plugin" and, btw, the first if() is redundant, if no dynamic loading was enabled no dynamic plugins will be built whatsoever. > +} > + > +enable_query_log; > > === added file 'mysql-test/include/ib_logfile_size_check.inc' > --- a/mysql-test/include/ib_logfile_size_check.inc 1970-01-01 00:00:00 +0000 > +++ b/mysql-test/include/ib_logfile_size_check.inc 2014-04-14 09:20:24 +0000 > @@ -0,0 +1,14 @@ I'm not a great fan of obscure include files that are only used once in one single test. Perhaps it's better to inline it? > +perl; > + my $dir = $ENV{'MYSQLD_DATADIR'}; > + my $size; > + opendir(DIR, $dir) or die $!; > + while (my $file = readdir(DIR)) > + { > + > + next unless ($file =~ m/\ib_logfile.$/); > + $size = -s "$dir/$file"; > + print "The size of the ib_logfile(0/1): $size \n"; > + } > + close(DIR); > + exit(0) > +EOF > > === added file 'mysql-test/include/vardir_size_check.inc' > --- a/mysql-test/include/vardir_size_check.inc 1970-01-01 00:00:00 +0000 > +++ b/mysql-test/include/vardir_size_check.inc 2014-04-14 09:20:24 +0000 > @@ -0,0 +1,26 @@ Same here. Although it's worse than with the previous file :) ib_logfile_size_check.inc is only used in innodb_log_file_size_functionality.test. This file - too. But in 5.6 this one is also used in query_cache_type_functionality, table_open_cache_functionality, and table_definition_cache_functionality, sort_buffer_size_functionality, query_cache_size_functionality. I suppose somebody just copied innodb_log_file_size_functionality over and edited it. I don't see why would anyone really want to check the vardir size in all these other tests. So, yes, better to inline this file too, to avoid silly mistakes like that. > +--echo Check the size of the vardir > +--echo The output size is in unit blocks > +perl; > +#!/usr/bin/perl > +use warnings; > +use strict; > +use File::Find; > +my $dir = $ENV{'MYSQLTEST_VARDIR'}; > +my $size = 0; > +find( sub { $size += -f $_ ? -s _ : 0 }, $dir ); > + > +if ( $size < 1717987000 ) > + { print "TRUE", "\n";} > +else > + { print "FALSE $size", "\n";} > + > + > +## Expected size for the VAR dir being changed for the PB2 runs > + > +##if ( $size =~ /^496[\d]{5}$/ ) > +## { > +## { print "TRUE", "\n";} > +## } > +##else > +## { print "FALSE $size", "\n";} > +EOF > > === modified file 'mysql-test/mysql-test-run.pl' > --- a/mysql-test/mysql-test-run.pl 2014-02-26 14:28:07 +0000 > +++ b/mysql-test/mysql-test-run.pl 2014-04-14 09:20:24 +0000 > @@ -5350,6 +5350,14 @@ sub mysqld_arguments ($$$) { > mtr_add_arg($args, "--gdb"); > } > > + # Enable the debug sync facility, set default wait timeout. > + # Facility stays disabled if timeout value is zero. > + mtr_add_arg($args, "--loose-debug-sync-timeout=%s", > + $opt_debug_sync_timeout) unless $opt_user_args; Why was it moved? > + > + # Options specified in .opt files should be added last so they can > + # override defaults above. > + > my $found_skip_core= 0; > foreach my $arg ( @$extra_opts ) > { > === added file 'mysql-test/suite/archive/archive_no_symlink-master.opt' > --- a/mysql-test/suite/archive/archive_no_symlink-master.opt 1970-01-01 00:00:00 +0000 > +++ b/mysql-test/suite/archive/archive_no_symlink-master.opt 2014-04-14 09:20:24 +0000 > @@ -0,0 +1 @@ > +--skip-symbolic-links just fyi: in MariaDB the file name can simply be archive_no_symlink.opt -master and -slave are only needed for replication tests to have different options for the master and the slave. > > === added file 'mysql-test/t/bug12969156.test' > --- a/mysql-test/t/bug12969156.test 1970-01-01 00:00:00 +0000 > +++ b/mysql-test/t/bug12969156.test 2014-04-14 09:20:24 +0000 > @@ -0,0 +1,18 @@ > +--source include/have_daemon_example_plugin.inc > +--source include/not_embedded.inc > +# TODO: the windows emulation of pthreads doesn't behave predictably > +--source include/not_windows.inc > + > +--echo # > +--echo # Bug #12969156 : SEGMENTATION FAULT ON UNINSTALLING > +--echo # DAEMON_EXAMPLE PLUGIN > +--echo # > + > +let $counter= 0; > +while ($counter < 10) > +{ > +--replace_result $LIBDAEMON_EXAMPLE_SO DAEMONEXAMPLE > + eval INSTALL PLUGIN daemon_example SONAME '$LIBDAEMON_EXAMPLE_SO'; in our own tests (not copied from 5.6) better use install soname 'libdaemon_example'; it's shorter, doesn't need --eval and doesn't need --replace_result > + UNINSTALL PLUGIN daemon_example; > + inc $counter; > +} > > === added file 'mysql-test/t/innodb_log_file_size_functionality.test' > --- a/mysql-test/t/innodb_log_file_size_functionality.test 1970-01-01 00:00:00 +0000 > +++ b/mysql-test/t/innodb_log_file_size_functionality.test 2014-04-14 09:20:24 +0000 > @@ -0,0 +1,124 @@ This should be renamed to sys_vars.innodb_log_file_size_func test > +############################################################################### > +# # > +# Variable Name: innodb_log_file_size # > +# Scope: Global # > +# Access Type: Static # > +# Data Type: numeric # > +# # > +# # > +# Creation Date: 2012-08-20 # > +# Author : Tanjot Singh Uppal # > +# # > +# # > +# Description:Test Cases of Static System Variable innodb_log_file_size # > +# that checks the behavior of this variable in the following ways # > +# * Value Check # > +# * Scope Check # > +# * Functionality Check # > +# * Accessability Check # > +# # > +# This test does not perform the crash recovery on this variable # > +# For crash recovery test on default change please run the ibtest # > +############################################################################### Regards, Sergei

1 0

Re: [Maria-developers] [Commits] Rev 4018: MDEV-4784 - merge test cases from 5.6 in lp:maria/10.0
by Sergei Golubchik 12 Jun '14

12 Jun '14

Hi, Sergey! > === added file 'mysql-test/include/have_daemon_example_plugin.inc' > --- a/mysql-test/include/have_daemon_example_plugin.inc 1970-01-01 00:00:00 +0000 > +++ b/mysql-test/include/have_daemon_example_plugin.inc 2014-03-11 09:28:16 +0000 > @@ -0,0 +1,16 @@ > +disable_query_log; > +# > +# Check if server has support for loading plugins > +# > +if (`SELECT @@have_dynamic_loading != 'YES'`) { > + --skip daemon example plugin requires dynamic loading > +} > + > +# > +# Check if the variable DAEMONEXAMPLE is set > +# > +if (!$LIBDAEMON_EXAMPLE_SO) { > + --skip daemon_example plugin requires the environment variable \$DAEMONEXAMPLE to be set (normally done by mtr) it's always done by mtr (not just "normally"). So a better error message should be "No libdaemon_example.so was found" or simply "Need daemon_example plugin" and, btw, the first if() is redundant, if no dynamic loading was enabled no dynamic plugins will be built whatsoever. > +} > + > +enable_query_log; > > === added file 'mysql-test/include/ib_logfile_size_check.inc' > --- a/mysql-test/include/ib_logfile_size_check.inc 1970-01-01 00:00:00 +0000 > +++ b/mysql-test/include/ib_logfile_size_check.inc 2014-03-11 09:28:16 +0000 > @@ -0,0 +1,14 @@ I'm not a great fan of obscure include files that are only used once in one single test. Perhaps it's better to inline it? > +perl; > + my $dir = $ENV{'MYSQLD_DATADIR'}; > + my $size; > + opendir(DIR, $dir) or die $!; > + while (my $file = readdir(DIR)) > + { > + > + next unless ($file =~ m/\ib_logfile.$/); > + $size = -s "$dir/$file"; > + print "The size of the ib_logfile(0/1): $size \n"; > + } > + close(DIR); > + exit(0) > +EOF > > === added file 'mysql-test/include/vardir_size_check.inc' > --- a/mysql-test/include/vardir_size_check.inc 1970-01-01 00:00:00 +0000 > +++ b/mysql-test/include/vardir_size_check.inc 2014-03-11 09:28:16 +0000 > @@ -0,0 +1,26 @@ Same here. Although it's worse than with the previous file :) ib_logfile_size_check.inc is only used in innodb_log_file_size_functionality.test. This file - too. But in 5.6 this one is also used in query_cache_type_functionality, table_open_cache_functionality, and table_definition_cache_functionality, sort_buffer_size_functionality, query_cache_size_functionality. I suppose somebody just copied innodb_log_file_size_functionality over and edited it without much thinking. I don't see why would anyone really want to check the vardir size in all these other tests. So, yes, better to inline this file too, to avoid silly mistakes like that. > +--echo Check the size of the vardir > +--echo The output size is in unit blocks > +perl; > +#!/usr/bin/perl > +use warnings; > +use strict; > +use File::Find; > +my $dir = $ENV{'MYSQLTEST_VARDIR'}; > +my $size = 0; > +find( sub { $size += -f $_ ? -s _ : 0 }, $dir ); > + > +if ( $size < 1717987000 ) > + { print "TRUE", "\n";} > +else > + { print "FALSE $size", "\n";} > + > + > +## Expected size for the VAR dir being changed for the PB2 runs > + > +##if ( $size =~ /^496[\d]{5}$/ ) > +## { > +## { print "TRUE", "\n";} > +## } > +##else > +## { print "FALSE $size", "\n";} > +EOF > > === added file 'mysql-test/suite/archive/archive_no_symlink-master.opt' > --- a/mysql-test/suite/archive/archive_no_symlink-master.opt 1970-01-01 00:00:00 +0000 > +++ b/mysql-test/suite/archive/archive_no_symlink-master.opt 2014-03-11 09:28:16 +0000 > @@ -0,0 +1 @@ just fyi: in MariaDB the file name can simply be archive_no_symlink.opt -master and -slave are only needed for replication tests to have different options for the master and the slave. > +--skip-symbolic-links > > === added file 'mysql-test/t/bug12969156.test' > --- a/mysql-test/t/bug12969156.test 1970-01-01 00:00:00 +0000 > +++ b/mysql-test/t/bug12969156.test 2014-03-11 09:28:16 +0000 > @@ -0,0 +1,18 @@ > +--source include/have_daemon_example_plugin.inc > +--source include/not_embedded.inc > +# TODO: the windows emulation of pthreads doesn't behave predictably > +--source include/not_windows.inc > + > +--echo # > +--echo # Bug #12969156 : SEGMENTATION FAULT ON UNINSTALLING > +--echo # DAEMON_EXAMPLE PLUGIN > +--echo # > + > +let $counter= 0; > +while ($counter < 10) > +{ > +--replace_result $LIBDAEMON_EXAMPLE_SO DAEMONEXAMPLE > + eval INSTALL PLUGIN daemon_example SONAME '$LIBDAEMON_EXAMPLE_SO'; in our own tests (not copied from 5.6) better use install soname 'libdaemon_example'; it's shorter, doesn't need --eval and doesn't need --replace_result > + UNINSTALL PLUGIN daemon_example; > + inc $counter; > +} > > === added file 'mysql-test/t/innodb_log_file_size_functionality.test' > --- a/mysql-test/t/innodb_log_file_size_functionality.test 1970-01-01 00:00:00 +0000 > +++ b/mysql-test/t/innodb_log_file_size_functionality.test 2014-03-11 09:28:16 +0000 > @@ -0,0 +1,124 @@ This should be renamed to sys_vars.innodb_log_file_size_func test > +############################################################################### > +# # > +# Variable Name: innodb_log_file_size # > +# Scope: Global # > +# Access Type: Static # > +# Data Type: numeric # > +# # > +# # > +# Creation Date: 2012-08-20 # > +# Author : Tanjot Singh Uppal # > +# # > +# # > +# Description:Test Cases of Static System Variable innodb_log_file_size # > +# that checks the behavior of this variable in the following ways # > +# * Value Check # > +# * Scope Check # > +# * Functionality Check # > +# * Accessability Check # > +# # > +# This test does not perform the crash recovery on this variable # > +# For crash recovery test on default change please run the ibtest # > +############################################################################### > + > +-- source include/have_innodb.inc > +-- source include/not_embedded.inc > +-- source include/have_innodb_16k.inc > + > +echo '#________________________VAR_09_INNODB_LOG_FILE_SIZE__________________#' > +echo '##' > +--echo '#---------------------WL6372_VAR_9_01----------------------#' > +#################################################################### > +# Checking default value # > +#################################################################### > +SELECT COUNT(@@GLOBAL.innodb_log_file_size); > +--echo 1 Expected > + > +SELECT @@GLOBAL.innodb_log_file_size; > +--echo 5242880 Expected > + > + > + > +--echo '#---------------------WL6372_VAR_9_02----------------------#' > +#################################################################### > +# Checking Value can be set - Static # > +#################################################################### > +--error ER_INCORRECT_GLOBAL_LOCAL_VAR > +SET @@local.innodb_log_file_size=1; > +--echo Expected error 'Read only variable' > + > +--error ER_INCORRECT_GLOBAL_LOCAL_VAR > +SET @@session.innodb_log_file_size=1; > +--echo Expected error 'Read only variable' > + > +--error ER_INCORRECT_GLOBAL_LOCAL_VAR > +SET @@GLOBAL.innodb_log_file_size=1; > +--echo Expected error 'Read only variable' > + > +--error ER_INCORRECT_GLOBAL_LOCAL_VAR > +SET @@GLOBAL.innodb_log_file_size=DEFAULT; > +--echo Expected error 'Read only variable' > + > +SELECT COUNT(@@GLOBAL.innodb_log_file_size); > +--echo 1 Expected > + > + > + > +--echo '#---------------------WL6372_VAR_9_03----------------------#' > +################################################################# > +# Check if the value in GLOBAL Table matches value in variable # > +################################################################# > +SELECT @@GLOBAL.innodb_log_file_size = VARIABLE_VALUE > +FROM INFORMATION_SCHEMA.GLOBAL_VARIABLES > +WHERE VARIABLE_NAME='innodb_log_file_size'; > +--echo 1 Expected > + > +SELECT COUNT(@@GLOBAL.innodb_log_file_size); > +--echo 1 Expected > + > +SELECT COUNT(VARIABLE_VALUE) > +FROM INFORMATION_SCHEMA.GLOBAL_VARIABLES > +WHERE VARIABLE_NAME='innodb_log_file_size'; > +--echo 1 Expected > + > + > + > +--echo '#---------------------WL6372_VAR_9_04----------------------#' > +################################################################################ > +# Checking Variable Scope # > +################################################################################ > +SELECT @@innodb_log_file_size = @@GLOBAL.innodb_log_file_size; > +--echo 1 Expected > + > +--Error ER_INCORRECT_GLOBAL_LOCAL_VAR > +SELECT COUNT(@@local.innodb_log_file_size); > +--echo Expected error 'Variable is a GLOBAL variable' > + > +--Error ER_INCORRECT_GLOBAL_LOCAL_VAR > +SELECT COUNT(@@SESSION.innodb_log_file_size); > +--echo Expected error 'Variable is a GLOBAL variable' > + > +SELECT COUNT(@@GLOBAL.innodb_log_file_size); > +--echo 1 Expected > + > +--Error ER_BAD_FIELD_ERROR > +SELECT innodb_log_file_size = @@SESSION.innodb_log_file_size; > +--echo Expected error 'Unknown column innodb_log_file_size in field list' > + > + > + > +--echo '#---------------------WL6372_VAR_9_05----------------------#' > +############################################################################### > +# Checking the /Var directory size # > +############################################################################### > +-- source include/vardir_size_check.inc > +--echo TRUE Expected > + > +--echo '#---------------------WL6372_VAR_9_06----------------------#' > +################################################################################# > +# Checking the size of ib_logfile # > +################################################################################# > +let MYSQLD_DATADIR=`SELECT @@datadir`; > +--source include/ib_logfile_size_check.inc > +--echo 5242880 expected > + > Regards, Sergei

1 0

[Maria-developers] PERFORMANCE_SCHEMA to be disabled in 10.0 (10.0.12)
by Colin Charles 12 Jun '14

12 Jun '14

Hi all, Recently there was chat about how PERFORMANCE_SCHEMA was enabled by mistake and it should be disabled in the 10.0 series. I'm curious - how many of you are using PERFORMANCE_SCHEMA? Is it a problem to turn it on, if you use it? I'm referring to: https://mariadb.atlassian.net/browse/MDEV-6249 Looking from the webscalesql list, there are examples of performance degradation: https://groups.google.com/d/msg/webscalesql/B-4bPIb8mHI/eUNsl0lSfbMJ https://groups.google.com/d/msg/webscalesql/B-4bPIb8mHI/-YZMMCuPw3QJ quote: "Our perf testing agrees with your assessment (we see about a 5%-6% perf hit when it's included and on, and a 2%-3% hit when it's included but off)" Please discuss this, either here or on MDEV-6249 Thanks cheers, -colin -- Colin Charles, Chief Evangelist, SkySQL - The MariaDB Company blog: http://bytebot.net/blog/| t: +6-012-204-3201 | Skype: colincharles

6 10

[Maria-developers] Extendet FederatedX storage engine to support updating Sphinx RT indexes
by Markus Lidel 12 Jun '14

12 Jun '14

Hello, the Sphinx Search engine has RT indexes, which could be updated with a subset of SQL commands, refered to as SphinxQL: http://sphinxsearch.com/docs/2.1.1/sphinxql-reference.html Because the FederatedX storage engine uses some commands, which Sphinx does not know, here is a patch, wich extends the FederatedX storage engine to support this subset of commands. Now its possible to update Sphinx RT indexes directly from MariaDB. To create a connection to the RT index in the Sphinx default RT-index: index rt { type = rt path = @CONFDIR@/data/rt rt_field = title rt_field = content rt_attr_string = title rt_attr_string = content rt_attr_uint = gid } you just have to call: CREATE TABLE `rt` ( `id` BIGINT UNSIGNED NOT NULL, `title` TEXT, `content` TEXT, `gid` INT UNSIGNED ) ENGINE=FEDERATED CONNECTION='sphinxql://root@localhost:9306/rt'; It would be great, if the attached patch could be integrated into MariaDB. If you have any suggestions, please feel free to contact me. Best regards, Markus Lidel

4 8

Re: [Maria-developers] [GSoC] Accepted student ready to work : )
by Pablo Estrada 10 Jun '14

10 Jun '14

Hello all, I'm still interested on a call, (I would make sure to write down everything as part of an email to keep it clearly organized here), but as Elena suggested, I'll try to look at Sergei's bzr repo first, and all the documentation. In any case, I already have a decent idea of how the data is arranged. I can see how it is difficult to trace test results back to code changes directly, but it might be possible. I still haven't figured out how to recognize what code changes pertain to the same 'history', or 'line' (I think that you were not considering this in your experiments... and it might turn out to not be relevant at all, but I still want to consider it, to try to make a more complete model). So I have just two questions that should be easy to answer by email: 1. In your graphs, exactly, what do you mean by Precision? What about Recall? And cutoff? 2. I looked at your launchpad profile but I can't seem to find the repo with your tests (if you even pushed it to lp at all). Is it available? That's it for today : ) Until next time. Pablo On Thu, Apr 24, 2014 at 12:28 AM, Sergei Golubchik <serg(a)skysql.com> wrote: > On April 23, 2014 10:07:23 AM CEST, Pablo Estrada wrote: > > >If it's not too inconvenient, do you think we could set up a Google > >Hangout > >or a Skype call on Monday to go over a few questions quickly? > > > Definitely. This week I'm on vacations and without a laptop, back home on > Sunday. So, Monday is fine. But please if you have specific questions - ask > them now in emails. I'll prepare answers for Monday, but it'll take time to > remember everything. That is, if you will wait with questions till the > call, I might not be able to remember the answer on the spot. > Regards, > Sergei >

4 17

Re: [Maria-developers] Rev 4234: Fixed bug mdev-6071
by Sergei Golubchik 10 Jun '14

10 Jun '14

Hi, Igor! On Jun 09, Igor Babaev wrote: > revno: 4234 > revision-id: igor(a)askmonty.org-20140610050716-0x8xaquf3itvofjc > parent: bar(a)mariadb.org-20140606062952-x2jwt4mgrgr6tpw6 > committer: Igor Babaev <igor(a)askmonty.org> > branch nick: maria-10.0-bugs > timestamp: Mon 2014-06-09 22:07:16 -0700 > message: > Fixed bug mdev-6071. > The method JOIN_CACHE::init may fail (return 1) if some conditions on the > used join buffer is not satisfied. For example it fails if join_buffer_size > is greater than join_buffer_space_limit. The conditions should be checked > when running the EXPLAIN command for the query. That's why the method > JOIN_CACHE::init has to be called for EXPLAIN commands as well. It's an improvement over the old behavior - as EXPLAIN shows correctly whether BKA is used. But I think there's still not enough diagnostics. In the bug report Sergey wrote that " I think it's a big gotcha that @join_buffer_size must be lower than @@join_buffer_space_limit when optimizer_space_limit optimization is off. " So, I'd believe users will be very confused that in some cases BKA is suddenly turned off, when they expect it to be used. Of course, without this bugfix they would've not even noticed that, which is even worse. So, this bugfix is good. But it would be nice to have some kind of an explanation of why BKA is not used. At the very least in the manual. Better - here as a warning, like if (for_explain) push_warning("BKA was not used because ..."; goto fail; > === modified file 'sql/sql_select.cc' > --- a/sql/sql_select.cc 2014-06-05 22:07:27 +0000 > +++ b/sql/sql_select.cc 2014-06-10 05:07:16 +0000 > @@ -10541,7 +10541,7 @@ > if (cache_level == 1) > prev_cache= 0; > if ((tab->cache= new JOIN_CACHE_BNL(join, tab, prev_cache)) && > - ((options & SELECT_DESCRIBE) || !tab->cache->init())) > + !tab->cache->init(MY_TEST(options & SELECT_DESCRIBE))) MY_TEST here (and in other similar places) is redundant, you could've simply written !tab->cache->init(options & SELECT_DESCRIBE)) > { > tab->icp_other_tables_ok= FALSE; > return (2 - MY_TEST(!prev_cache)); and while it wasn't part of your bug fix, I cannot help but noticing that MY_TEST() is redundant here too. !prev_cache is boolean, it's always 0 or 1. Regards, Sergei

1 0

Re: [Maria-developers] Google Code for Encryption of MariaDB
by Elmar Eperiesi-Beck 09 Jun '14

09 Jun '14

Hi, 1. what about an NDA between us, to give me the chance to have a pre-view. I will make sure, that your architecture and development fits into our work. Everything we can think of in an early stage does not make lot of work, but… 2. like. 3. We will post our concept next week into the MariaDB Developers community. 4. … or 1. Regards Elmar Am 09.06.2014 um 11:14 schrieb Jonas Oreland : > Hi there, > > 1) repo: as I said in my mail, the code is currently not yet published, > but the guy doing this said that he expects it to be with in a couple of weeks or so. > before this, i don't think you can see it. > > once it's published, I think you can find it at https://code.google.com, > but I'll check and let you know. > > 2) This is my personal email address, so I prefer if we keep the conversation > @maria-developers where I'm subscribed with work email address, if this is ok > by you ? > > 3) concept paper: sure > > 4) we don't really have a concept paper, but the code is complete and being rolled out. > I can answer questions about it, but I think it's easiest once it has been published. > > /Jonas > > > > On Mon, Jun 9, 2014 at 11:02 AM, Elmar Eperiesi-Beck wrote: > Jonas, > is there a repository, where I can find the actual source code of your encryption implementation? > Or do we have to sign an NDA to exchange these kind of information? > > We can have a look at your code and merge it with our existing implementation. > Additionally I would like to send you our concept paper, to give you an overview of our ideas. > > Regards > Elmar >

1 0

[Maria-developers] [GSoC] Week 3!
by sriram patil 09 Jun '14

09 Jun '14

Hi all, Here is my post for the 3rd week into GSoC project. http://goo.gl/bhl5ZW Suggestions and review are welcome! Thanks, Sriram

1 0

[Maria-developers] MariaDB 5.5.38 Release Prep
by Daniel Bartholomew 07 Jun '14

07 Jun '14

All, I've begun prep work on the MariaDB 5.5.38 release. Draft release notes and changelog are here: - https://mariadb.com/kb/en/mariadb-5538-release-notes/ - https://mariadb.com/kb/en/mariadb-5538-changelog/ Current plan is to activate and announce the release on Mon, 9 jun 2014. Thanks. -- Daniel Bartholomew, MariaDB Release Manager MariaDB | http://mariadb.com

1 0

[Maria-developers] MDEV-5533 - Backport Threadpool improvements from Percona Server
by Sergey Vojtovich 04 Jun '14

04 Jun '14

Hi Sergei, Since MDEV-5533 is marked critical for 10.0.12 it is a good time to start this discussion. IIRC we agreed that our intention is to accept default value change for thread_pool_max_threads into 10.0 and accept priority queue throttling into 10.1. So default value for thread_pool_max_threads has bumped in percona server from 500 to 100 000. The reason is: deadlocks and uneven distribution of worker threads between thread groups in case of stalled connections. But the above is contradicting to suggestions in KB: https://mariadb.com/kb/en/threadpool-in-55/ ... so ideally, there should be a single active thread for each CPU on the machine. ... If your application constantly pegs at 500 threads, it might be a strong indicator for high contention in the application, and threadpool does not help much. ... One way to solve the problem would be to set thread_pool_max_threads to a very high value, but this might be less than ideal, and negatively affect performance. ... Said the above, do you still think it is a good idea to raise default value of thread_pool_max_threads? If so, what should KB article state wrt this variable? Thanks, Sergey

1 0

Re: [Maria-developers] MDEV-6213: Query cache is not invalidated on cascade delete when database name contains special symbols
by Sergei Golubchik 03 Jun '14

03 Jun '14

Hi, Sanja! Please add in your revision a test case for MySQL bug#13919851 http://bazaar.launchpad.net/~mysql/mysql-server/5.6/revision/4514 and make sure that your fix doesn't break it. On May 09, sanja(a)askmonty.org wrote: > revno: 4207 > revision-id: sanja(a)askmonty.org-20140509104240-ksrk10tef5dq157z > branch nick: work-maria-10.0-MDEV-6213 > timestamp: Fri 2014-05-09 13:42:40 +0300 > message: > MDEV-6213: Query cache is not invalidated on cascade delete when > database name contains special symbols. > > Innodb should de-escape db/table names when sending them to QC Regards, Sergei

1 0

[Maria-developers] Please review a patch fixing a few MDEV-6001 blockers
by Alexander Barkov 03 Jun '14

03 Jun '14

Hi Sergei, can you please review my patch fixing a few bugs that are blockers for MDEV-6001? - MDEV-4858 Wrong results for a huge unsigned value inserted into a TIME column - MDEV-6097 Inconsistent results for CAST(int,decimal,double AS DATETIME) - MDEV-6099 Bad results for DATE_ADD(.., INTERVAL 2000000000000000000.0 SECOND) - MDEV-6100 No warning on CAST(9000000 AS TIME) - MDEV-6101 Hybrid functions do not add CURRENT_DATE when converting TIME to DATETIME - MDEV-6102 Comparison between TIME and DATETIME does not use CURRENT_DATE Thanks.

2 10

[Maria-developers] Review request[major 5.3/5.5 bug]: MDEV-4511 Assertion `scale <= precision' fails on GROUP BY TIMEDIFF with incorrect types, , Agile Board, , Export
by Alexander Barkov 03 Jun '14

03 Jun '14

Hello Sergei, A few weeks ago we fixed this bug for 10.0 by changing temporal items not to have decimals=NOT_FIXED_DEC (more MySQL-5.6 compatibility), and agreed to have a separate patch for the pre-10.0 branches. This patch is for 5.3 and 5.5 only. It back-ports the mdev-4511 tests from 10.0 as is. But it does not touch the NOT_FIXED_DEC behavior of the temporal items and fixes the problem in a different way. Greetings.

1 0

[Maria-developers] POWER interest
by Colin Charles 03 Jun '14

03 Jun '14

Hi, It seems we have a POWER8 box and soon will start doing builds on this platform Stewart Smith (currently at IBM) seems to be doing some work on MySQL 5.6 and MySQL 5.7 for POWER8 systems - there doesn't seem to be much attention being paid to MariaDB 10. Ideally, our fixes that are being picked up from Stewart or focused on POWER should be targeted at 10.1 I'm thinking (unless there's customer interest for the 10.0 series -- which may very likely be the case) So far, I've noted GIS precise has an open failure, and maybe its a great idea for Holyfoot to use the POWER8 system to start fixing it: https://mariadb.atlassian.net/browse/MDEV-4153 From MySQL bugs system we see multiple examples from Stewart's blog posts - some are InnoDB specific so we will inherit them during a merge - some aren't like mysql#72715 (character set code endianness dependent on CPU type rather than endianness of CP) https://www.flamingspork.com/blog/2014/06/03/1-million-sql-queries-per-seco… https://www.flamingspork.com/blog/2014/06/03/mysql-5-7-on-power/ Relevant stuff that we may want to look at: http://bugs.mysql.com/search.php?cmd=display&status=Active&tags=PowerPC cheers, -colin -- Colin Charles, Chief Evangelist, SkySQL - The MariaDB Company blog: http://bytebot.net/blog/| t: +6-012-204-3201 | Skype: colincharles

1 0

[Maria-developers] [GSoC] First two weeks into coding!
by sriram patil 03 Jun '14

03 Jun '14

Hi all, I have created a blog to write the details of the work that I do as part of GSoC'14. Following is the link to my first blog post explaining the details of the work that I have done during the last two weeks. http://goo.gl/PYoVZb I will be writing these blog posts every weekend from now on and will share it on the developer mailing list every Monday. Suggestions/reviews are welcome. Thanks, Sriram

2 2

[Maria-developers] Possible race condition or NULL pointer triggered by OQGraph
by Andrew McDonnell 02 Jun '14

02 Jun '14

Hi all (Cross-posted to oqgraph-developers, maria-developers) I am trying to track down a segfault apparently triggered by concurrent execution of queries through OQGraph - for context, see: https://mariadb.atlassian.net/browse/MDEV-6282 I am however a bit confused as to what is going on, because at least some of the information leads me to suspect that there could be more going on here than just a simple race condition. --Immediate Cause-- On the surface, a segfault appears to be caused in a method in sql_class.h, Statement::check_limit_rows_examined() dereferencing a NULL pointer, `lex`. The method check_limit_rows_examined() was called from sql_class.h method handler::increment_statistics() . --Analysis-- A quick inspection reveals that a NULL value for `lex` is actually perfectly valid some of the time, although I am nowhere near familiar enough with the code to really understand when or why. Specifically, `lex` is a member of the base class Statement, which is initialised to either point to THD::main_lex via a constructor , or intentionally to NULL from construction of an object Prepared_statement() (sql_prepare.cc) To my mind, if there is going to be a possibility of a pointer being NULL, then all uses of that pointer should be checking for that. I am hoping that the bug reporter can prove or otherwise my hypothesis with his particular scenario, although I am suspicious that there may be additional issues to be dealt with; it is quite possible that lex being NULL is a red herring, being caused by a race condition memory overrun elsewhere. However, I think it is still a good idea to NULL check pointers that could be set to NULL intentionally. I can submit a pull request if the following patch is agreeable: diff --git a/sql/sql_class.h b/sql/sql_class.h index 5898d9e..957acf3 100644 --- a/sql/sql_class.h +++ b/sql/sql_class.h @@ -2475,8 +2475,10 @@ public: */ void check_limit_rows_examined() { - if (++accessed_rows_and_keys > lex->limit_rows_examined_cnt) + ++accessed_rows_and_keys; + if (lex && accessed_rows_and_keys > lex->limit_rows_examined_cnt) { killed= ABORT_QUERY; + } } --Aside-- Immediately prior to the check_limit_rows_examined() call is a call to macro status_var_increment(). Comments around the macro indicate it is often likely to be called in a mutli-threaded context, but the data is such that it is possible to disable protection with the only side effect being an occasionally 'inaccurate' result. Is it possible however, that check_limit_rows_examined() is not thread-safe and should have some protection? cheers, Andrew --A

3 3

[Maria-developers] MDEV-4051 INET6_ATON() and INET6_NTOA()
by Alexander Barkov 02 Jun '14

02 Jun '14

Hello Ian, I have merged these functions from MySQL-5.6 into 10.0: INET6_ATON() INET6_NTOA() IS_IPV4() IS_IPV4_COMPAT() IS_IPV4_MAPPED() IS_IPV6() Can you please document them? The corresponding article in the MySQL manual is here: http://dev.mysql.com/doc/refman/5.6/en/miscellaneous-functions.html Thanks.

2 2

[Maria-developers] KB page on switching from bazaar to git.
by Sergey Petrunia 27 May '14

27 May '14

Hi, I was doing some stuff in git already, and so I have created a git cheatsheet with equivalents of bzr commands that were useful in MariaDB development: https://mariadb.com/kb/en/using-git/ It is not finished, any additions are welcome. Elena has agreed to look at creating a commit trigger that sends commit emails (currently, github sends emails on push (not commit), and these do not include diffs) BR Sergei -- Sergei Petrunia, Software Developer MariaDB | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] CREATE OR REPLACE DATABASE algorithm
by sriram patil 27 May '14

27 May '14

Hi Alexander and all, Following are the steps that I am following for implementing "CREATE OR REPLACE" for DATABASE "..." signifies, there exists one or more lines of code in that block of the function. mysql_create_db(...) ... ... if(db exists) { if(create_or_replace) { if(DROP_ACL access not granted) { // Error 1 DBUG_RETURN(-1) } if(mysql_rm_db(...) unsuccessful) { // Error 2 DBUG_RETURN(-1) } Reset m_status from diagnostics cause it is changed by mysql_rm_db. Proceed with normal create db. } else if(if not exists) { ... } else ... } ... ... } Is the above algorithm correct? Also, I am not yet sure which errors should be displayed at both the places "Error 1" and "Error 2". I have written and tested the above code. It is working fine. Testing involves running all the existing test cases and executing "CREATE OR REPLACE DATABASE" command on the console manually. "IF NOT EXISTS" gives error if used with "CREATE OR REPLACE". Any other test cases I need to consider? Thanks, Sriram

2 2

[Maria-developers] FOR JSON
by Alexander Barkov 26 May '14

26 May '14

Hello Monty, We discussed extracting data in JSON format today, to make it possible to return data as a single JSON value (as a result set consisting of a single row and a single column): SELECT id,name,address FROM users FOR JSON; Note, in some cases the result of a such query can obviously be very huge. So it would be nice to be able to use "FOR JSON" in subselects, like this: SELECT id, (SELECT name,address FROM users AS b WHERE a.id= b.id FOR JSON) AS RowJSON FROM users AS a; This query will return a result set with multiple rows (one row per one record) and with two columns: - the ID, as a regular number - the JSON value with "name" and "address" embedded into it. Or, another example, if one does not need "id" as a separate column: SELECT (SELECT id,name,address FROM users AS b WHERE a.id= b.id FOR JSON) AS RowJSON FROM users AS a; This query will return a result set with multiple rows (one row per one record) with a single JSON value embedding all three columns: "id", "name" and "address".

2 4

[Maria-developers] MariaDB 10.1 is now on github!
by Sergei Golubchik 26 May '14

26 May '14

Hi! I'm happy to announce that MariaDB-10.1 tree has been completely migrated to github. Since now on we'll use github for the new development in MariaDB. It's https://github.com/MariaDB/server, go on, fork it, hack around, submit pull requests. Have fun! Older trees (10.0, 5.5, 5.3, 5.2, 5.1) are not on github - we do hope to migrate them too eventually, but at the moment they are still on launchpad. If you're a maria-captain on launchpad - for you to get write access to the mariadb repository on github you need to tell me your github account name. Regards, Sergei P.S.: Don't forget that 10.1 is still *pre-alpha*. You've been warned.

4 5