- developers - lists.mariadb.org

Re: [Maria-developers] [Commits] b2db8e8: MDEV-7949: Item_field::used_tables() takes 0.29% in OLTP RO
by Sergey Vojtovich 21 May '15

21 May '15

Hi Sanja, this patch looks Ok, just one minor suggestion inline. Though effect for given workload is quite moderate: Item_field::used_tables 0.21% -> 0.20% Item_basic_constant::used_tables 0.11% -> 0.11% Item_func::used_tables 0.05% -> 0.05% Item_sum::used_tables 0.00% -> 0.00% On Wed, May 13, 2015 at 04:17:26PM +0200, sanja(a)mariadb.com wrote: > revision-id: b2db8e85422a455a10cd28ef7e44977182528f70 > parent(s): b22959903b89e798f8804ec9a815c88f75915cd9 > committer: Oleksandr Byelkin > branch nick: server > timestamp: 2015-05-13 16:17:22 +0200 > message: > > MDEV-7949: Item_field::used_tables() takes 0.29% in OLTP RO > > small sixes of used_tables() usage > ...skip... > diff --git a/sql/item_func.h b/sql/item_func.h > index 0d57c2b..c48cd84 100644 > --- a/sql/item_func.h > +++ b/sql/item_func.h > @@ -1337,7 +1337,7 @@ class Item_func_sleep :public Item_int_func > const char *func_name() const { return "sleep"; } > table_map used_tables() const > { > - return Item_int_func::used_tables() | RAND_TABLE_BIT; > + return used_tables_cache | RAND_TABLE_BIT; > } > bool is_expensive() { return 1; } > longlong val_int(); > @@ -1591,7 +1591,7 @@ class Item_func_get_lock :public Item_int_func > void fix_length_and_dec() { max_length=1; maybe_null=1;} > table_map used_tables() const > { > - return Item_int_func::used_tables() | RAND_TABLE_BIT; > + return used_tables_cache | RAND_TABLE_BIT; > } > bool const_item() const { return 0; } > bool is_expensive() { return 1; } > @@ -1611,7 +1611,7 @@ class Item_func_release_lock :public Item_int_func > void fix_length_and_dec() { max_length= 1; maybe_null= 1;} > table_map used_tables() const > { > - return Item_int_func::used_tables() | RAND_TABLE_BIT; > + return used_tables_cache | RAND_TABLE_BIT; > } > bool const_item() const { return 0; } > bool is_expensive() { return 1; } > @@ -1724,7 +1724,7 @@ class Item_func_set_user_var :public Item_func > void fix_length_and_dec(); > table_map used_tables() const > { > - return Item_func::used_tables() | RAND_TABLE_BIT; > + return used_tables_cache | RAND_TABLE_BIT; > } > bool const_item() const { return 0; } > bool is_expensive() { return 1; } Was there any reason for this change? Compiler should be clever enough to inline used_tables() of base class here. Regards, Sergey

3 2

[Maria-developers] Misnamed libmariadb-dev packages on aptitude
by pcrews 20 May '15

20 May '15

Hello, Today, I was experimenting with building mariadb images and encountered this issue: E: Unable to locate package libmariadb-dev pcrews@erlking-dev:~/git/dibtest$ aptitude search libmariadb p libmariadbd-dev - MariaDB embedded database development files p libmariadbd-dev:i386 - MariaDB embedded database development files Note how these are listed as libmariadbd vs. libmariadb. I am writing to see if this can be fixed or if this might be some change in naming (which would require some updating of other tools' references, etc). Thank you, Patrick Crews

1 0

[Maria-developers] Step#6: MDEV-7950 Item_func::type() takes 0.26% in OLTP RO
by Alexander Barkov 19 May '15

19 May '15

Hi Sergey, Please review the next iteration for MDEV-7950. This one splits the function add_key_fields() into a method in Item. This change removes about 3 virtual calls item->type(), as well as some virtual calls item_func->functype(), and adds one virtual call item->add_key_fields() instead. Thanks.

2 2

Re: [Maria-developers] [Commits] 8b30c0f: MDEV-7949: Item_field::used_tables() takes 0.29% in OLTP RO
by Sergey Vojtovich 19 May '15

19 May '15

Hi Sanja, There're 3 bug fixes and 4 revisions concerning these lines: https://bugs.mysql.com/bug.php?id=16716 git: 1991a87d831bb48b31989c622e35acda11df7fef git: ca22a81b1c84ce81e1e9ea2c3ace7be1848027d8 (source for all used_tables()) https://bugs.mysql.com/bug.php?id=36488 git: 66367aeea83a0cd7f583a0194651a235b63975a9 https://bugs.mysql.com/bug.php?id=46815 git: 10406ae65871de074e807e626f9ede686e9321d4 If we don't have these bugs, why not to revert all the code introduced by these patches? Or we're not affected by the need to use used_tables() as an indication of prepared statement? If so, why? Thanks, Sergey On Sun, May 17, 2015 at 03:10:48PM +0200, sanja(a)mariadb.com wrote: > revision-id: 8b30c0f26c7b35fc73429f85aa35eb3614d60bd8 > parent(s): b2db8e85422a455a10cd28ef7e44977182528f70 > committer: Oleksandr Byelkin > branch nick: server > timestamp: 2015-05-17 15:10:45 +0200 > message: > > MDEV-7949: Item_field::used_tables() takes 0.29% in OLTP RO > > Part 2: removed hack workaround for bug we do not have. > > --- > sql/item_strfunc.cc | 7 +++---- > 1 file changed, 3 insertions(+), 4 deletions(-) > > diff --git a/sql/item_strfunc.cc b/sql/item_strfunc.cc > index 4bf8dd5..7b9169e 100644 > --- a/sql/item_strfunc.cc > +++ b/sql/item_strfunc.cc > @@ -626,8 +626,7 @@ String *Item_func_concat::val_str(String *str) > if (!(res=args[0]->val_str(str))) > goto null; > use_as_buff= &tmp_value; > - /* Item_subselect in --ps-protocol mode will state it as a non-const */ > - is_const= args[0]->const_item() || !args[0]->used_tables(); > + is_const= args[0]->const_item(); > for (i=1 ; i < arg_count ; i++) > { > if (res->length() == 0) > @@ -639,7 +638,7 @@ String *Item_func_concat::val_str(String *str) > non-empty argument. Because of this we need is_const to be > evaluated only for it. > */ > - is_const= args[i]->const_item() || !args[i]->used_tables(); > + is_const= args[i]->const_item(); > } > else > { > @@ -975,7 +974,7 @@ String *Item_func_concat_ws::val_str(String *str) > for (i=1; i < arg_count; i++) > if ((res= args[i]->val_str(str))) > { > - is_const= args[i]->const_item() || !args[i]->used_tables(); > + is_const= args[i]->const_item(); > break; > } > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits

1 0

[Maria-developers] MDEV-7937: Enforce SSL when --ssl client option is used
by Vicențiu Ciorbaru 18 May '15

18 May '15

Hi Sergei! I've done some work on this issue. I've read MySQL's implementation of this and have looked at our implementation. They have done a bit of refactoring, introducing an enforce_ssl flag, as well as changing the C interface a bit, to allow setting this flag programatically. I've created a patch here that changes the minimum amount possible, in order to implement what MDEV-7937 requires. That being said, I agree with (most of) MySQL's refactoring in this case. They've moved all the SSL related connection code into its own separate function, before actually calling send_client_reply_packet. I can work towards implementing things the way MySQL does, but since I saw that you've actually done most of the work in that area of the code, I figured I'd ask for your input on it. There are two more things that I'm not sure of: 1. Specifying --ssl as a command line parameter to the mysql client is not enough to enforce ssl and the client's code in this case just ignores the option. We need to provide at least one of the additional ones like --ssl-key or --ssl-ca. My patch will not cause the client to report an error in this case. Is this acceptable behaviour or not? 2. Do we want mysql's enforce_ssl feature? Regards, Vicențiu

2 1

Re: [Maria-developers] MDEV-8112: PATCH: no slave left behind
by Kristian Nielsen 18 May '15

18 May '15

Hi Jonas, Thanks for the patch! I read through the patch. I am not that familiar with the semi-sync plugin code, but it looks reasonable. But the patch causes a test failure in sys_vars.all_vars. This is because not all newly introduced variables have corresponding test cases in mysql-test/suite/sys_vars/t/*_basic.test. Also, do you have some documentation for the feature? (There is a bit in the Jira entry, but for example the new variable rpl_semi_sync_master_slave_lag_heartbeat_frequency_us seems to be not described at all... If you can fix the above two points (and if Buildbot does not point out more test suite issues), then I will push it into the MariaDB tree. Thanks, - Kristian. > patch has been ported to 10.1 > you can use it under new (also called "3-clause") BSD license > i have done internal benchmarks...but you are most welcome to do some too. > > DESCRIPTION: > no slave left behind > > this patch implements master throttling based on slave lag, > aka no slave left behind. the core feature works as follows > 1) the semi-sync-reply is ammended to also report back SQL-thread > position (aka exec position) > 2) transactions are not removed from the "active-transaction-list" > in the semi-sync-master plugin until atleast one slave has reported > that it has executed this transaction. the slave lag can then > be estimated by calculating how long the oldest transaction has been > lingering in the active-transaction-list. > 3) client-threads are forced to wait before commit until slave lag > has decreased to acceptable value. > > the following variables are introduced on master: > > rpl_semi_sync_master_max_slave_lag (global) > rpl_semi_sync_master_slave_lag_wait_timeout (session) > > the following status variables are introduced on master: > > rpl_semi_sync_master_slave_lag_wait_sessions > rpl_semi_sync_master_estimated_slave_lag > rpl_semi_sync_master_trx_slave_lag_wait_time > rpl_semi_sync_master_trx_slave_lag_wait_num > rpl_semi_sync_master_avg_trx_slave_lag_wait_time > > the following variables are introduced on slave: > > rpl_semi_sync_slave_lag_enabled (global) > > in addition to this, 2 optimizations that decreases overhead of semi-sync > is introduced. > 1) the idea of this is that if when a slave should send and transaction, > it checks if it should be semi-synced, but rather > than semi-sync:ing each transaction (which is done currently) the code > will skip semi-syncing transaction if there already is newer transactions > committed. But, since this can mean that semi-syncing is delayed indefinitely > a cap is set using 2 new master variables: > > rpl_semi_sync_master_max_unacked_event_bytes (global) > rpl_semi_sync_master_max_unacked_event_count (global) > 2) rpl_semi_sync_master_group_commit which makes the semi-sync > plugin only semi-sync the last transaction in a group commit. > diff --git a/mysql-test/suite/rpl/r/rpl_semi_sync_slave_lag.result b/mysql-test/suite/rpl/r/rpl_semi_sync_slave_lag.result > new file mode 100644 > index 0000000..f3545a4 > --- /dev/null > +++ b/mysql-test/suite/rpl/r/rpl_semi_sync_slave_lag.result > @@ -0,0 +1,150 @@ > +include/master-slave.inc > +[connection master] > +set global rpl_semi_sync_master_enabled = 1; > +set global rpl_semi_sync_master_max_slave_lag = 10; > +show variables like 'rpl_semi_sync_master_%slave_lag%'; > +Variable_name Value > +rpl_semi_sync_master_max_slave_lag 10 > +rpl_semi_sync_master_slave_lag_heartbeat_frequency_us 500000 > +rpl_semi_sync_master_slave_lag_wait_timeout 50 > +include/stop_slave.inc > +set global rpl_semi_sync_slave_enabled = 1; > +set global rpl_semi_sync_slave_lag_enabled = 1; > +include/start_slave.inc > +# create non-root user for testing READ_ONLY > +grant SELECT, INSERT on *.* to test@localhost; > +CREATE TABLE t1 (i INT NOT NULL AUTO_INCREMENT PRIMARY KEY, f varchar(8)) > +ENGINE=innodb; > +# > +# Check basic behaviour > +# > +INSERT INTO t1 (f) VALUES ('1'),('2'),('3'); > +# Now wait for slave lag to decrease to 0 > +# [ on slave ] > +STOP SLAVE SQL_THREAD; > +# [ on master ] > +INSERT INTO t1 (f) VALUES ('4'),('5'),('6'); > +# Now wait for slave lag to increase to > 0 > +# [ on slave ] > +START SLAVE SQL_THREAD; > +# [ on master ] > +# Now wait for slave lag to decrease to 0 > +# [ on slave ] > +STOP SLAVE SQL_THREAD; > +# [ on master ] > +set session rpl_semi_sync_master_slave_lag_wait_timeout = 5; > +# First transaction should succeed. slave_lag is zero when it commits > +INSERT INTO t1 (f) VALUES ('7'),('8'),('9'); > +# Now wait for slave lag to increase to > 10s > +# Check that estimated_slave_lag is > 10s > +SELECT VARIABLE_VALUE > 10000000 as should_be_1 > +FROM INFORMATION_SCHEMA.GLOBAL_STATUS > +WHERE VARIABLE_NAME = 'rpl_semi_sync_master_estimated_slave_lag'; > +should_be_1 > +1 > +# Second transaction should now fail. slave_lag is >10s when it commits > +INSERT INTO t1 (f) VALUES ('a'),('b'),('c'); > +ERROR HY000: Slave-lag timeout > +# [ on slave ] > +START SLAVE SQL_THREAD; > +# [ on master ] > +# Now wait for slave lag to decrease to < 10s > +# And now it should succeed again > +INSERT INTO t1 (f) VALUES ('d'),('e'),('f'); > +SELECT * > +FROM t1 > +ORDER BY 1; > +i f > +1 1 > +2 2 > +3 3 > +4 4 > +5 5 > +6 6 > +7 7 > +8 8 > +9 9 > +13 d > +14 e > +15 f > +# [ on slave ] > +SELECT * > +FROM t1 > +ORDER BY 1; > +i f > +1 1 > +2 2 > +3 3 > +4 4 > +5 5 > +6 6 > +7 7 > +8 8 > +9 9 > +13 d > +14 e > +15 f > +# > +# Test interaction with READ_ONLY > +# > +# [ on slave ] > +STOP SLAVE SQL_THREAD; > +# [ on master ] > +INSERT INTO t1 (f) VALUES ('g'),('h'),('i'); > +# Now wait for slave lag to increase to > 10s > +# [ on con1 ] > +BEGIN; > +INSERT INTO t1 (f) VALUES ('g'),('h'),('i'); > +# [ on master ] > +set global read_only = 1; > +# [ on con1 ] > +set session rpl_semi_sync_master_slave_lag_wait_timeout = 5; > +# read-only is check *before* slave lag > +COMMIT; > +ERROR HY000: The MariaDB server is running with the --read-only option so it cannot execute this statement > +# [ on slave ] > +START SLAVE SQL_THREAD; > +# [ on master ] > +# Now wait for slave lag to decrease to 0 > +set global read_only = 0; > +# > +# check slave_lag > 0 but less than rpl_semi_sync_master_max_slave_lag > +# > +# [ on slave ] > +STOP SLAVE SQL_THREAD; > +# [ on master ] > +INSERT INTO t1 (f) VALUES ('j'),('k'),('l'); > +# Now wait for slave lag to increase to > 0 > +# Capture rpl_semi_sync_master_tx_slave_lag_waits before transaction > +select @count_before := VARIABLE_VALUE > +FROM INFORMATION_SCHEMA.GLOBAL_STATUS > +WHERE VARIABLE_NAME = 'rpl_semi_sync_master_tx_slave_lag_waits'; > +# Set maximum allowed slave lag to 24h > +set global rpl_semi_sync_master_max_slave_lag = 86400; > +INSERT INTO t1 (f) VALUES ('m'),('n'),('o'); > +# Capture rpl_semi_sync_master_tx_slave_lag_waits after transaction > +select @count_after := VARIABLE_VALUE > +FROM INFORMATION_SCHEMA.GLOBAL_STATUS > +WHERE VARIABLE_NAME = 'rpl_semi_sync_master_tx_slave_lag_waits'; > +# There should have been no wait, since maximum allowed is very high > +select @count_before = @count_after as should_be_1; > +should_be_1 > +1 > +# [ on slave ] > +START SLAVE SQL_THREAD; > +# [ on master ] > +# Now wait for slave lag to decrease to 0 > +# > +# Clean up > +# > +# [ on master ] > +set global rpl_semi_sync_master_max_slave_lag = default; > +set session rpl_semi_sync_master_slave_lag_wait_timeout = default; > +DROP USER test@localhost; > +include/stop_slave.inc > +set global rpl_semi_sync_slave_enabled = 0; > +set global rpl_semi_sync_slave_lag_enabled = default; > +set global rpl_semi_sync_master_enabled = 0; > +include/start_slave.inc > +DROP TABLE t1; > +include/rpl_end.inc > diff --git a/mysql-test/suite/rpl/t/rpl_semi_sync_slave_lag.test b/mysql-test/suite/rpl/t/rpl_semi_sync_slave_lag.test > new file mode 100644 > index 0000000..f9bdbd9 > --- /dev/null > +++ b/mysql-test/suite/rpl/t/rpl_semi_sync_slave_lag.test > @@ -0,0 +1,252 @@ > +source include/have_semisync.inc; > +source include/not_embedded.inc; > +source include/have_innodb.inc; > +source include/master-slave.inc; > + > +connection master; > +set global rpl_semi_sync_master_enabled = 1; > +set global rpl_semi_sync_master_max_slave_lag = 10; > + > +show variables like 'rpl_semi_sync_master_%slave_lag%'; > + > +connection slave; > +source include/stop_slave.inc; > +set global rpl_semi_sync_slave_enabled = 1; > +set global rpl_semi_sync_slave_lag_enabled = 1; > + > +source include/start_slave.inc; > + > +connection master; > + > +--echo # create non-root user for testing READ_ONLY > +grant SELECT, INSERT on *.* to test@localhost; > +connect (con1,localhost,test,,test); > + > +CREATE TABLE t1 (i INT NOT NULL AUTO_INCREMENT PRIMARY KEY, f varchar(8)) > +ENGINE=innodb; > + > +--echo # > +--echo # Check basic behaviour > +--echo # > +INSERT INTO t1 (f) VALUES ('1'),('2'),('3'); > + > +--echo # Now wait for slave lag to decrease to 0 > +let $wait_condition= SELECT VARIABLE_VALUE = 0 > +FROM INFORMATION_SCHEMA.GLOBAL_STATUS > +WHERE VARIABLE_NAME = 'rpl_semi_sync_master_estimated_slave_lag'; > +--source include/wait_condition.inc > + > +--echo # [ on slave ] > +connection slave; > +STOP SLAVE SQL_THREAD; > + > +--echo # [ on master ] > +connection master; > +INSERT INTO t1 (f) VALUES ('4'),('5'),('6'); > + > +--echo # Now wait for slave lag to increase to > 0 > +let $wait_condition= SELECT VARIABLE_VALUE > 0 > +FROM INFORMATION_SCHEMA.GLOBAL_STATUS > +WHERE VARIABLE_NAME = 'rpl_semi_sync_master_estimated_slave_lag'; > +--source include/wait_condition.inc > + > +--echo # [ on slave ] > +connection slave; > +START SLAVE SQL_THREAD; > + > +--echo # [ on master ] > +connection master; > + > +--echo # Now wait for slave lag to decrease to 0 > +let $wait_condition= SELECT VARIABLE_VALUE = 0 > +FROM INFORMATION_SCHEMA.GLOBAL_STATUS > +WHERE VARIABLE_NAME = 'rpl_semi_sync_master_estimated_slave_lag'; > +--source include/wait_condition.inc > + > +--echo # [ on slave ] > +connection slave; > +STOP SLAVE SQL_THREAD; > + > +--echo # [ on master ] > +connection master; > + > +set session rpl_semi_sync_master_slave_lag_wait_timeout = 5; > + > +--echo # First transaction should succeed. slave_lag is zero when it commits > +INSERT INTO t1 (f) VALUES ('7'),('8'),('9'); > + > +--echo # Now wait for slave lag to increase to > 10s > +let $wait_condition= SELECT VARIABLE_VALUE > 10000000 > +FROM INFORMATION_SCHEMA.GLOBAL_STATUS > +WHERE VARIABLE_NAME = 'rpl_semi_sync_master_estimated_slave_lag'; > +--source include/wait_condition.inc > + > +--echo # Check that estimated_slave_lag is > 10s > +SELECT VARIABLE_VALUE > 10000000 as should_be_1 > +FROM INFORMATION_SCHEMA.GLOBAL_STATUS > +WHERE VARIABLE_NAME = 'rpl_semi_sync_master_estimated_slave_lag'; > + > +--echo # Second transaction should now fail. slave_lag is >10s when it commits > +--error ER_ERROR_DURING_COMMIT > +INSERT INTO t1 (f) VALUES ('a'),('b'),('c'); > + > +--echo # [ on slave ] > +connection slave; > +START SLAVE SQL_THREAD; > + > +--echo # [ on master ] > +connection master; > + > +--echo # Now wait for slave lag to decrease to < 10s > +let $wait_condition= SELECT VARIABLE_VALUE < 10000000 > +FROM INFORMATION_SCHEMA.GLOBAL_STATUS > +WHERE VARIABLE_NAME = 'rpl_semi_sync_master_estimated_slave_lag'; > +--source include/wait_condition.inc > + > +--echo # And now it should succeed again > +INSERT INTO t1 (f) VALUES ('d'),('e'),('f'); > + > +SELECT * > +FROM t1 > +ORDER BY 1; > + > +sync_slave_with_master; > + > +--echo # [ on slave ] > +connection slave; > + > +SELECT * > +FROM t1 > +ORDER BY 1; > + > +--echo # > +--echo # Test interaction with READ_ONLY > +--echo # > + > +--echo # [ on slave ] > +connection slave; > +STOP SLAVE SQL_THREAD; > + > +--echo # [ on master ] > +connection master; > + > +INSERT INTO t1 (f) VALUES ('g'),('h'),('i'); > + > +--echo # Now wait for slave lag to increase to > 10s > +let $wait_condition= SELECT VARIABLE_VALUE > 10000000 > +FROM INFORMATION_SCHEMA.GLOBAL_STATUS > +WHERE VARIABLE_NAME = 'rpl_semi_sync_master_estimated_slave_lag'; > +--source include/wait_condition.inc > + > +connection con1; > +--echo # [ on con1 ] > +BEGIN; > +INSERT INTO t1 (f) VALUES ('g'),('h'),('i'); > + > +connection master; > +--echo # [ on master ] > +set global read_only = 1; > + > +connection con1; > +--echo # [ on con1 ] > +set session rpl_semi_sync_master_slave_lag_wait_timeout = 5; > + > +--echo # read-only is check *before* slave lag > +--error ER_OPTION_PREVENTS_STATEMENT > +COMMIT; > + > +disconnect con1; > + > +--echo # [ on slave ] > +connection slave; > +START SLAVE SQL_THREAD; > + > +connection master; > +--echo # [ on master ] > + > +--echo # Now wait for slave lag to decrease to 0 > +let $wait_condition= SELECT VARIABLE_VALUE = 0 > +FROM INFORMATION_SCHEMA.GLOBAL_STATUS > +WHERE VARIABLE_NAME = 'rpl_semi_sync_master_estimated_slave_lag'; > +--source include/wait_condition.inc > + > +set global read_only = 0; > + > +--echo # > +--echo # check slave_lag > 0 but less than rpl_semi_sync_master_max_slave_lag > +--echo # > + > +--echo # [ on slave ] > +connection slave; > +STOP SLAVE SQL_THREAD; > + > +connection master; > +--echo # [ on master ] > +INSERT INTO t1 (f) VALUES ('j'),('k'),('l'); > + > +--echo # Now wait for slave lag to increase to > 0 > +let $wait_condition= SELECT VARIABLE_VALUE > 0 > +FROM INFORMATION_SCHEMA.GLOBAL_STATUS > +WHERE VARIABLE_NAME = 'rpl_semi_sync_master_estimated_slave_lag'; > +--source include/wait_condition.inc > + > +--echo # Capture rpl_semi_sync_master_tx_slave_lag_waits before transaction > +--disable_result_log > +select @count_before := VARIABLE_VALUE > +FROM INFORMATION_SCHEMA.GLOBAL_STATUS > +WHERE VARIABLE_NAME = 'rpl_semi_sync_master_tx_slave_lag_waits'; > +--enable_result_log > + > +--echo # Set maximum allowed slave lag to 24h > +set global rpl_semi_sync_master_max_slave_lag = 86400; # 24h > + > +INSERT INTO t1 (f) VALUES ('m'),('n'),('o'); > + > +--echo # Capture rpl_semi_sync_master_tx_slave_lag_waits after transaction > +--disable_result_log > +select @count_after := VARIABLE_VALUE > +FROM INFORMATION_SCHEMA.GLOBAL_STATUS > +WHERE VARIABLE_NAME = 'rpl_semi_sync_master_tx_slave_lag_waits'; > +--enable_result_log > + > +--echo # There should have been no wait, since maximum allowed is very high > +select @count_before = @count_after as should_be_1; > + > +--echo # [ on slave ] > +connection slave; > +START SLAVE SQL_THREAD; > + > +connection master; > +--echo # [ on master ] > + > +--echo # Now wait for slave lag to decrease to 0 > +let $wait_condition= SELECT VARIABLE_VALUE = 0 > +FROM INFORMATION_SCHEMA.GLOBAL_STATUS > +WHERE VARIABLE_NAME = 'rpl_semi_sync_master_estimated_slave_lag'; > +--source include/wait_condition.inc > + > +--echo # > +--echo # Clean up > +--echo # > +connection master; > +--echo # [ on master ] > +set global rpl_semi_sync_master_max_slave_lag = default; > +set session rpl_semi_sync_master_slave_lag_wait_timeout = default; > +DROP USER test@localhost; > + > +connection slave; > +source include/stop_slave.inc; > +set global rpl_semi_sync_slave_enabled = 0; > +set global rpl_semi_sync_slave_lag_enabled = default; > + > +connection master; > +set global rpl_semi_sync_master_enabled = 0; > + > +connection slave; > +source include/start_slave.inc; > + > +connection master; > + > +DROP TABLE t1; > +sync_slave_with_master; > +--source include/rpl_end.inc > diff --git a/plugin/semisync/semisync.cc b/plugin/semisync/semisync.cc > index 4a80360..fc02b9d 100644 > --- a/plugin/semisync/semisync.cc > +++ b/plugin/semisync/semisync.cc > @@ -20,6 +20,7 @@ > > const unsigned char ReplSemiSyncBase::kPacketMagicNum = 0xef; > const unsigned char ReplSemiSyncBase::kPacketFlagSync = 0x01; > +const unsigned char ReplSemiSyncBase::kPacketFlagSyncAndReport = 0x02; > > > const unsigned long Trace::kTraceGeneral = 0x0001; > @@ -29,3 +30,6 @@ const unsigned long Trace::kTraceFunction = 0x0040; > > const unsigned char ReplSemiSyncBase::kSyncHeader[2] = > {ReplSemiSyncBase::kPacketMagicNum, 0}; > + > +const char* const ReplSemiSyncBase::kRplSemiSyncSlaveReportExec = > + "rpl_semi_sync_slave_report_exec"; > diff --git a/plugin/semisync/semisync.h b/plugin/semisync/semisync.h > index 2857729..78faba9 100644 > --- a/plugin/semisync/semisync.h > +++ b/plugin/semisync/semisync.h > @@ -75,13 +75,23 @@ class ReplSemiSyncBase > > /* Constants in network packet header. */ > static const unsigned char kPacketMagicNum; > + /* this event should be semisync acked */ > static const unsigned char kPacketFlagSync; > + /* this event should be semisync acked including the current SQL position */ > + static const unsigned char kPacketFlagSyncAndReport; > + > + /* user variable for enabling exec-pos reporting */ > + static const char* const kRplSemiSyncSlaveReportExec; > }; > > /* The layout of a semisync slave reply packet: > 1 byte for the magic num > 8 bytes for the binlog positon > - n bytes for the binlog filename, terminated with a '\0' > + n bytes for the binlog filename, NOT terminated with a '\0' > + [ optionally ] > + 1 byte == 0 > + 8 bytes for the sql-thread position > + n bytes for the sql-thread filename, terminated with a '\0' > */ > #define REPLY_MAGIC_NUM_LEN 1 > #define REPLY_BINLOG_POS_LEN 8 > diff --git a/plugin/semisync/semisync_master.cc b/plugin/semisync/semisync_master.cc > index c88c162..3ee6e26 100644 > --- a/plugin/semisync/semisync_master.cc > +++ b/plugin/semisync/semisync_master.cc > @@ -22,6 +22,9 @@ > #define TIME_MILLION 1000000 > #define TIME_BILLION 1000000000 > > +/* thd_key for per slave thread state */ > +static MYSQL_THD_KEY_T thd_key; > + > /* This indicates whether semi-synchronous replication is enabled. */ > char rpl_semi_sync_master_enabled; > unsigned long rpl_semi_sync_master_wait_point = > @@ -45,6 +48,18 @@ unsigned long long rpl_semi_sync_master_net_wait_time = 0; > unsigned long long rpl_semi_sync_master_trx_wait_time = 0; > char rpl_semi_sync_master_wait_no_slave = 1; > > +unsigned long rpl_semi_sync_master_max_unacked_event_count = 0; > +unsigned long rpl_semi_sync_master_max_unacked_event_bytes = 4096; > + > +unsigned long rpl_semi_sync_master_slave_lag_clients = 0; > +unsigned long long rpl_semi_sync_master_estimated_slave_lag = 0; > +unsigned long rpl_semi_sync_master_slave_lag_heartbeat_frequency_us = 500000; > +unsigned long rpl_semi_sync_master_max_slave_lag = 0; > +unsigned long rpl_semi_sync_master_slave_lag_wait_sessions = 0; > + > +unsigned long rpl_semi_sync_master_avg_trx_slave_lag_wait_time = 0; > +unsigned long long rpl_semi_sync_master_trx_slave_lag_wait_num = 0; > +unsigned long long rpl_semi_sync_master_trx_slave_lag_wait_time = 0; > > static int getWaitTime(const struct timespec& start_ts); > > @@ -150,6 +165,15 @@ int ActiveTranx::insert_tranx_node(const char *log_file_name, > ins_node->log_name_[FN_REFLEN-1] = 0; /* make sure it ends properly */ > ins_node->log_pos_ = log_file_pos; > > + { > + /** > + * set trans commit time > + * this is called when writing into binlog, which is not > + * exactly right, but close enough for our purposes > + */ > + ins_node->tranx_commit_time_us = my_hrtime().val; > + } > + > if (!trx_front_) > { > /* The list is empty. */ > @@ -193,12 +217,11 @@ int ActiveTranx::insert_tranx_node(const char *log_file_name, > return function_exit(kWho, result); > } > > -bool ActiveTranx::is_tranx_end_pos(const char *log_file_name, > - my_off_t log_file_pos) > +TranxNode* ActiveTranx::lookup_tranx_end_pos(const char *log_file_name, > + my_off_t log_file_pos) > { > - const char *kWho = "ActiveTranx::is_tranx_end_pos"; > + const char *kWho = "ActiveTranx::lookup_tranx_end_pos"; > function_enter(kWho); > - > unsigned int hash_val = get_hash_value(log_file_name, log_file_pos); > TranxNode *entry = trx_htb_[hash_val]; > > @@ -211,38 +234,24 @@ bool ActiveTranx::is_tranx_end_pos(const char *log_file_name, > } > > if (trace_level_ & kTraceDetail) > - sql_print_information("%s: probe (%s, %lu) in entry(%u)", kWho, > - log_file_name, (unsigned long)log_file_pos, hash_val); > + sql_print_information("%s: probe (%s, %lu)", kWho, > + log_file_name, (unsigned long)log_file_pos); > > function_exit(kWho, (entry != NULL)); > - return (entry != NULL); > + return entry; > } > > -int ActiveTranx::clear_active_tranx_nodes(const char *log_file_name, > - my_off_t log_file_pos) > +int ActiveTranx::clear_active_tranx_nodes() > { > - const char *kWho = "ActiveTranx::::clear_active_tranx_nodes"; > - TranxNode *new_front; > + set_new_front(NULL); > + return 0; > +} > > +void ActiveTranx::set_new_front(TranxNode *new_front) > +{ > + const char *kWho = "ActiveTranx::set_new_front"; > function_enter(kWho); > > - if (log_file_name != NULL) > - { > - new_front = trx_front_; > - > - while (new_front) > - { > - if (compare(new_front, log_file_name, log_file_pos) > 0) > - break; > - new_front = new_front->next_; > - } > - } > - else > - { > - /* If log_file_name is NULL, clear everything. */ > - new_front = NULL; > - } > - > if (new_front == NULL) > { > /* No active transaction nodes after the call. */ > @@ -257,7 +266,6 @@ int ActiveTranx::clear_active_tranx_nodes(const char *log_file_name, > trx_front_ = NULL; > trx_rear_ = NULL; > } > - > if (trace_level_ & kTraceDetail) > sql_print_information("%s: cleared all nodes", kWho); > } > @@ -291,14 +299,40 @@ int ActiveTranx::clear_active_tranx_nodes(const char *log_file_name, > > trx_front_ = new_front; > allocator_.free_nodes_before(trx_front_); > - > if (trace_level_ & kTraceDetail) > sql_print_information("%s: cleared %d nodes back until pos (%s, %lu)", > kWho, n_frees, > trx_front_->log_name_, (unsigned long)trx_front_->log_pos_); > } > + function_exit(kWho, 0); > +} > > - return function_exit(kWho, 0); > +bool ActiveTranx::prune_active_tranx_nodes( > + LogPosPtr pos, > + ulonglong *oldest_tranx_commit_time_us) > +{ > + TranxNode *old_front = trx_front_; > + TranxNode *new_front; > + > + new_front = trx_front_; > + while (new_front) > + { > + if (compare(new_front, pos.file_name, pos.file_pos) > 0) > + break; > + new_front = new_front->next_; > + } > + > + set_new_front(new_front); > + > + if (oldest_tranx_commit_time_us) > + { > + if (trx_front_ == NULL) > + *oldest_tranx_commit_time_us = 0; > + else > + *oldest_tranx_commit_time_us = trx_front_->tranx_commit_time_us; > + } > + > + return ! (old_front == trx_front_); > } > > > @@ -334,7 +368,8 @@ ReplSemiSyncMaster::ReplSemiSyncMaster() > wait_file_pos_(0), > master_enabled_(false), > wait_timeout_(0L), > - state_(0) > + state_(0), > + oldest_unapplied_tranx_commit_time_us_(0) > { > strcpy(reply_file_name_, ""); > strcpy(wait_file_name_, ""); > @@ -362,11 +397,19 @@ int ReplSemiSyncMaster::initObject() > mysql_cond_init(key_ss_cond_COND_binlog_send_, > &COND_binlog_send_, NULL); > > + /* Mutex initialization can only be done after MY_INIT(). */ > + mysql_mutex_init(key_ss_mutex_LOCK_slave_lag_, > + &LOCK_slave_lag_, MY_MUTEX_INIT_FAST); > + mysql_cond_init(key_ss_cond_COND_slave_lag_, > + &COND_slave_lag_, NULL); > + > if (rpl_semi_sync_master_enabled) > result = enableMaster(); > else > result = disableMaster(); > > + thd_key_create(&thd_key); > + > return result; > } > > @@ -437,6 +480,8 @@ void ReplSemiSyncMaster::cleanup() > { > mysql_mutex_destroy(&LOCK_binlog_); > mysql_cond_destroy(&COND_binlog_send_); > + mysql_mutex_destroy(&LOCK_slave_lag_); > + mysql_cond_destroy(&COND_slave_lag_); > init_done_= 0; > } > > @@ -473,7 +518,34 @@ void ReplSemiSyncMaster::add_slave() > { > lock(); > rpl_semi_sync_master_clients++; > + if (has_semi_sync_slave_lag()) > + rpl_semi_sync_master_slave_lag_clients++; > unlock(); > + > + if (has_semi_sync_slave_lag()) > + { > + int null_val = 0; > + longlong new_val = > + rpl_semi_sync_master_slave_lag_heartbeat_frequency_us * 1000; > + longlong old_val = new_val + 1; > + > + get_user_var_int("master_heartbeat_period", &old_val, &null_val); > + if (old_val > new_val || null_val) > + { > + /* if there no old value or it's bigger than what we want */ > + int res = set_user_var_int("master_heartbeat_period",new_val, &old_val); > + if (res == -1) > + { > + sql_print_error( > + "Repl_semi_sync::failed to set master_heartbeat_period"); > + } > + } > + } > + > + /** > + * create per slave-state and store it in thread-local-storage */ > + ReplSemiSyncMasterPerSlaveState *state = new ReplSemiSyncMasterPerSlaveState; > + thd_setspecific(current_thd, thd_key, state); > } > > void ReplSemiSyncMaster::remove_slave() > @@ -492,7 +564,31 @@ void ReplSemiSyncMaster::remove_slave() > rpl_semi_sync_master_clients == 0) > switch_off(); > } > + > + bool no_slave_lag_clients = false; > + if (has_semi_sync_slave_lag()) > + { > + if (--rpl_semi_sync_master_slave_lag_clients == 0) > + { > + no_slave_lag_clients = true; > + } > + } > + > unlock(); > + > + ReplSemiSyncMasterPerSlaveState *state = > + (ReplSemiSyncMasterPerSlaveState*)thd_getspecific(current_thd, thd_key); > + thd_setspecific(current_thd, thd_key, NULL); > + > + if (state != NULL) > + { > + delete state; > + } > + > + if (no_slave_lag_clients) > + { > + wake_slave_lag_waiters(0); > + } > } > > bool ReplSemiSyncMaster::is_semi_sync_slave() > @@ -503,14 +599,115 @@ bool ReplSemiSyncMaster::is_semi_sync_slave() > return val; > } > > +bool ReplSemiSyncMaster::has_semi_sync_slave_lag() > +{ > + int null_value; > + long long val= 0; > + get_user_var_int(kRplSemiSyncSlaveReportExec, &val, &null_value); > + return val; > +} > + > +int ReplSemiSyncMaster::checkSyncReq(const LogPosPtr *log_pos) > +{ > + if (log_pos == NULL) > + { > + /* heartbeat events does not have logpos (since they are not actually > + * stored in the binlog). > + */ > + if (!has_semi_sync_slave_lag()) > + { > + /* don't semi-sync them if we haven't enabled slave-lag handling */ > + return 0; > + } > + else > + { > + /* else ask for both IO and exec position */ > + return 2; > + } > + } > + > + /** > + * check if this log-pos is a candidate for semi-syncing event > + */ > + TranxNode *entry = active_tranxs_->lookup_tranx_end_pos(log_pos->file_name, > + log_pos->file_pos); > + > + if (entry == NULL) > + return 0; > + > + ReplSemiSyncMasterPerSlaveState *state = > + (ReplSemiSyncMasterPerSlaveState*)thd_getspecific(current_thd, > + thd_key); > + do > + { > + state->unacked_event_count_++; > + > + if (active_tranxs_->is_rear(entry)) > + { > + /* always ask for ack on last event in tranx list */ > + break; > + } > + > + if (state->unacked_event_count_ >= > + rpl_semi_sync_master_max_unacked_event_count) > + { > + /* enough events passed that it's time for another ack */ > + break; > + } > + > + if (!state->sync_req_pos_.IsInited()) > + { > + /* first event => time for ack */ > + break; > + } > + > + if (strcmp(log_pos->file_name, state->sync_req_pos_.file_name) != 0) > + { > + /* new file => time for ack */ > + break; > + } > + > + if (log_pos->file_pos >= (state->sync_req_pos_.file_pos + > + rpl_semi_sync_master_max_unacked_event_bytes)) > + { > + /* enough bytes => time for ack */ > + break; > + } > + > + /* we skip asking for semi-sync ack on this event */ > + return 0; > + > + } while (0); > + > + /* keep track on when we last asked for semi-sync-ack */ > + state->unacked_event_count_ = 0; > + state->sync_req_pos_.Assign(log_pos); > + > + /** > + * check if this slave can report back exec position > + */ > + if (!has_semi_sync_slave_lag()) > + { > + /* slave can't report back SQL position */ > + return 1; > + } > + > + /* ask for both IO and SQL position */ > + return 2; > +} > + > int ReplSemiSyncMaster::reportReplyBinlog(uint32 server_id, > const char *log_file_name, > - my_off_t log_file_pos) > + my_off_t log_file_pos, > + const LogPos *exec_pos) > { > const char *kWho = "ReplSemiSyncMaster::reportReplyBinlog"; > int cmp; > bool can_release_threads = false; > bool need_copy_send_pos = true; > + bool pruned_trx_list = false; > + ulonglong oldest_tranx_commit_time_us = 0; > + > > if (!(getMasterEnabled())) > return 0; > @@ -559,15 +756,29 @@ int ReplSemiSyncMaster::reportReplyBinlog(uint32 server_id, > reply_file_pos_ = log_file_pos; > reply_file_name_inited_ = true; > > - /* Remove all active transaction nodes before this point. */ > - assert(active_tranxs_ != NULL); > - active_tranxs_->clear_active_tranx_nodes(log_file_name, log_file_pos); > - > if (trace_level_ & kTraceDetail) > sql_print_information("%s: Got reply at (%s, %lu)", kWho, > log_file_name, (unsigned long)log_file_pos); > } > > + assert(active_tranxs_ != NULL); > + if (exec_pos != NULL) > + { > + /* prune using exec_pos */ > + LogPosPtr ptr(*exec_pos); > + pruned_trx_list = active_tranxs_->prune_active_tranx_nodes( > + ptr, &oldest_tranx_commit_time_us); > + } > + else if (rpl_semi_sync_master_slave_lag_clients == 0 && need_copy_send_pos) > + { > + /** > + * if we don't have any slaves that can do exec_pos reporting, > + * prune by IO position as "plain old semi sync" > + */ > + LogPosPtr ptr(log_file_name, log_file_pos); > + active_tranxs_->prune_active_tranx_nodes(ptr, NULL); > + } > + > if (rpl_semi_sync_master_wait_sessions > 0) > { > /* Let us check if some of the waiting threads doing a trx > @@ -596,6 +807,15 @@ int ReplSemiSyncMaster::reportReplyBinlog(uint32 server_id, > cond_broadcast(); > } > > + if (pruned_trx_list) > + { > + /** > + * if we did prune trx list, it might be that we should wake up > + * threads waiting for slave-lag to decrease > + */ > + wake_slave_lag_waiters(oldest_tranx_commit_time_us); > + } > + > return function_exit(kWho, 0); > } > > @@ -743,14 +963,6 @@ int ReplSemiSyncMaster::commitTrx(const char* trx_wait_binlog_name, > } > } > > - /* > - At this point, the binlog file and position of this transaction > - must have been removed from ActiveTranx. > - */ > - assert(thd_killed(NULL) || > - !active_tranxs_->is_tranx_end_pos(trx_wait_binlog_name, > - trx_wait_binlog_pos)); > - > l_end: > /* Update the status counter. */ > if (is_on()) > @@ -794,7 +1006,7 @@ int ReplSemiSyncMaster::switch_off() > > /* Clear the active transaction list. */ > assert(active_tranxs_ != NULL); > - result = active_tranxs_->clear_active_tranx_nodes(NULL, 0); > + result = active_tranxs_->clear_active_tranx_nodes(); > > rpl_semi_sync_master_off_times++; > wait_file_name_inited_ = false; > @@ -884,7 +1096,7 @@ int ReplSemiSyncMaster::updateSyncHeader(unsigned char *packet, > { > const char *kWho = "ReplSemiSyncMaster::updateSyncHeader"; > int cmp = 0; > - bool sync = false; > + int sync = 0; > > /* If the semi-sync master is not enabled, or the slave is not a semi-sync > * target, do not request replies from the slave. > @@ -905,6 +1117,13 @@ int ReplSemiSyncMaster::updateSyncHeader(unsigned char *packet, > /* semi-sync is ON */ > /* sync= false; No sync unless a transaction is involved. */ > > + if (log_file_name == NULL) > + { > + /* this is heartbeat, request io_pos and exec_pos */ > + sync = checkSyncReq(0); > + goto l_end; > + } > + > if (reply_file_name_inited_) > { > cmp = ActiveTranx::compare(log_file_name, log_file_pos, > @@ -933,12 +1152,12 @@ int ReplSemiSyncMaster::updateSyncHeader(unsigned char *packet, > */ > if (cmp >= 0) > { > - /* > + /* > * We only wait if the event is a transaction's ending event. > */ > assert(active_tranxs_ != NULL); > - sync = active_tranxs_->is_tranx_end_pos(log_file_name, > - log_file_pos); > + LogPosPtr pos(log_file_name, log_file_pos); > + sync = checkSyncReq(&pos); > } > } > else > @@ -951,7 +1170,7 @@ int ReplSemiSyncMaster::updateSyncHeader(unsigned char *packet, > } > else > { > - sync = true; > + sync = 1; > } > } > > @@ -966,10 +1185,14 @@ int ReplSemiSyncMaster::updateSyncHeader(unsigned char *packet, > /* We do not need to clear sync flag because we set it to 0 when we > * reserve the packet header. > */ > - if (sync) > + if (sync == 1) > { > (packet)[2] = kPacketFlagSync; > } > + else if (sync == 2) > + { > + (packet)[2] = kPacketFlagSyncAndReport; > + } > > return function_exit(kWho, 0); > } > @@ -1018,7 +1241,8 @@ int ReplSemiSyncMaster::writeTranxInBinlog(const char* log_file_name, > if (is_on()) > { > assert(active_tranxs_ != NULL); > - if(active_tranxs_->insert_tranx_node(log_file_name, log_file_pos)) > + bool empty = active_tranxs_->is_empty(); > + if (active_tranxs_->insert_tranx_node(log_file_name, log_file_pos)) > { > /* > if insert tranx_node failed, print a warning message > @@ -1028,6 +1252,14 @@ int ReplSemiSyncMaster::writeTranxInBinlog(const char* log_file_name, > log_file_name, (ulong)log_file_pos); > switch_off(); > } > + else if (empty && rpl_semi_sync_master_slave_lag_clients > 0) > + { > + /* if the list of transactions was empty, > + * we need to init the oldest_tranx_commit_time_us > + */ > + oldest_unapplied_tranx_commit_time_us_ = > + active_tranxs_->get_oldest_tranx_commit_time_us(); > + } > } > > l_end: > @@ -1037,10 +1269,10 @@ int ReplSemiSyncMaster::writeTranxInBinlog(const char* log_file_name, > } > > int ReplSemiSyncMaster::readSlaveReply(NET *net, uint32 server_id, > - const char *event_buf) > + const char *event_buf_) > { > const char *kWho = "ReplSemiSyncMaster::readSlaveReply"; > - const unsigned char *packet; > + const unsigned char *packet, *packet_start; > char log_file_name[FN_REFLEN]; > my_off_t log_file_pos; > ulong log_file_len = 0; > @@ -1048,12 +1280,15 @@ int ReplSemiSyncMaster::readSlaveReply(NET *net, uint32 server_id, > int result = -1; > struct timespec start_ts; > ulong trc_level = trace_level_; > + const unsigned char *event_buf = (const unsigned char*)event_buf_; > + bool exec_pos_present = false; // is SQL exec pos present in reply > + LogPos exec_pos; // position of SQL thread > LINT_INIT_STRUCT(start_ts); > > function_enter(kWho); > > - assert((unsigned char)event_buf[1] == kPacketMagicNum); > - if ((unsigned char)event_buf[2] != kPacketFlagSync) > + assert(event_buf[1] == kPacketMagicNum); > + if ((event_buf[2] & (kPacketFlagSync | kPacketFlagSyncAndReport)) == 0) > { > /* current event does not require reply */ > result = 0; > @@ -1111,28 +1346,60 @@ int ReplSemiSyncMaster::readSlaveReply(NET *net, uint32 server_id, > goto l_end; > } > > - packet = net->read_pos; > + packet_start = packet = net->read_pos; > if (packet[REPLY_MAGIC_NUM_OFFSET] != ReplSemiSyncMaster::kPacketMagicNum) > { > sql_print_error("Read semi-sync reply magic number error"); > goto l_end; > } > > + /* we determine if this semisync ack contains a sql-thread exec-pos > + * by checking if last byte == 0, since the packet then contains > + * \0-terminated filenames */ > + exec_pos_present = packet[packet_len - 1] == 0; > + > log_file_pos = uint8korr(packet + REPLY_BINLOG_POS_OFFSET); > - log_file_len = packet_len - REPLY_BINLOG_NAME_OFFSET; > + if (exec_pos_present == false) > + { > + log_file_len = packet_len - REPLY_BINLOG_NAME_OFFSET; > + } > + else > + { > + log_file_len = strnlen((char*)packet + REPLY_BINLOG_NAME_OFFSET, > + MY_MIN((ulong)FN_REFLEN, > + packet_len - REPLY_BINLOG_NAME_OFFSET)); > + } > if (log_file_len >= FN_REFLEN) > { > sql_print_error("Read semi-sync reply binlog file length too large"); > goto l_end; > } > - strncpy(log_file_name, (const char*)packet + REPLY_BINLOG_NAME_OFFSET, log_file_len); > + packet+= REPLY_BINLOG_NAME_OFFSET; > + > + strncpy(log_file_name, (const char*)packet, log_file_len); > log_file_name[log_file_len] = 0; > > + if (exec_pos_present) > + { > + packet += log_file_len + 1; > + if (packet + 8 + 1 >= (packet_start + packet_len)) > + { > + sql_print_error("Read semi-sync reply binlog. " > + "Packet to short to contain exec-position!"); > + goto l_end; > + } > + exec_pos.file_pos = uint8korr(packet); > + packet += 8; > + strncpy(exec_pos.file_name, (char*)packet, > + (packet_start + packet_len) - packet); > + } > + > if (trc_level & kTraceDetail) > sql_print_information("%s: Got reply (%s, %lu)", > kWho, log_file_name, (ulong)log_file_pos); > > - result = reportReplyBinlog(server_id, log_file_name, log_file_pos); > + result = reportReplyBinlog(server_id, log_file_name, log_file_pos, > + exec_pos_present ? &exec_pos : NULL); > > l_end: > return function_exit(kWho, result); > @@ -1154,6 +1421,16 @@ int ReplSemiSyncMaster::resetMaster() > wait_file_name_inited_ = false; > reply_file_name_inited_ = false; > commit_file_name_inited_ = false; > + if (active_tranxs_ != NULL) > + { > + /** > + * make sure to empty transaction hash/list > + * with slave-lag reporting this container does > + * not have to be empty even if no transaction is > + * currently running > + */ > + active_tranxs_->clear_active_tranx_nodes(); > + } > > rpl_semi_sync_master_yes_transactions = 0; > rpl_semi_sync_master_no_transactions = 0; > @@ -1168,6 +1445,13 @@ int ReplSemiSyncMaster::resetMaster() > > unlock(); > > + mysql_mutex_lock(&LOCK_slave_lag_); > + rpl_semi_sync_master_slave_lag_wait_sessions = 0; > + oldest_unapplied_tranx_commit_time_us_ = 0; > + rpl_semi_sync_master_trx_slave_lag_wait_num = 0; > + rpl_semi_sync_master_trx_slave_lag_wait_time = 0; > + mysql_mutex_unlock(&LOCK_slave_lag_); > + > return function_exit(kWho, result); > } > > @@ -1186,6 +1470,29 @@ void ReplSemiSyncMaster::setExportStats() > ((double)rpl_semi_sync_master_net_wait_num)) : 0); > > unlock(); > + > + if (oldest_unapplied_tranx_commit_time_us_ != 0) > + { > + rpl_semi_sync_master_estimated_slave_lag = my_hrtime().val - > + oldest_unapplied_tranx_commit_time_us_; > + } > + else > + { > + rpl_semi_sync_master_estimated_slave_lag = 0; > + } > + > + mysql_mutex_lock(&LOCK_slave_lag_); > + if (rpl_semi_sync_master_trx_slave_lag_wait_num) > + { > + rpl_semi_sync_master_avg_trx_slave_lag_wait_time = > + (unsigned long)((double)rpl_semi_sync_master_trx_slave_lag_wait_time / > + (double)rpl_semi_sync_master_trx_slave_lag_wait_num); > + } > + else > + { > + rpl_semi_sync_master_avg_trx_slave_lag_wait_time = 0; > + } > + mysql_mutex_unlock(&LOCK_slave_lag_); > } > > /* Get the waiting time given the wait's staring time. > @@ -1213,3 +1520,117 @@ static int getWaitTime(const struct timespec& start_ts) > > return (int)(end_usecs - start_usecs); > } > + > +void ReplSemiSyncMaster::wake_slave_lag_waiters( > + ulonglong oldest_unapplied_tranx_commit_time_us) > +{ > + mysql_mutex_lock(&LOCK_slave_lag_); > + oldest_unapplied_tranx_commit_time_us_ = > + oldest_unapplied_tranx_commit_time_us; > + > + if (rpl_semi_sync_master_slave_lag_wait_sessions > 0) > + { > + mysql_cond_broadcast(&COND_slave_lag_); > + } > + mysql_mutex_unlock(&LOCK_slave_lag_); > +} > + > +int ReplSemiSyncMaster::wait_slave_lag(ulong timeout_sec) > +{ > + int error = 0; > + PSI_stage_info old_stage; > + > + /* slave lag waiting not enabled, return directly */ > + if (rpl_semi_sync_master_max_slave_lag == 0) > + return 0; > + > + /* there is no slave that can report slave lag, return directly */ > + if (rpl_semi_sync_master_slave_lag_clients == 0) > + return 0; > + > + /* compute start_time and end_time */ > + struct timespec end_time; > + set_timespec(end_time, 0); > + ulonglong start_time_us = timespec_to_usec(&end_time); > + end_time.tv_sec += timeout_sec; > + > + mysql_mutex_lock(&LOCK_slave_lag_); > + > + if (oldest_unapplied_tranx_commit_time_us_ == 0) > + { > + /* no slave lag, atleast one slave is up to date */ > + mysql_mutex_unlock(&LOCK_slave_lag_); > + return 0; > + } > + > + if (rpl_semi_sync_master_max_slave_lag == 0) > + { > + /* slave lag waiting not enabled */ > + mysql_mutex_unlock(&LOCK_slave_lag_); > + return 0; > + } > + > + /* This must be called after acquired the lock */ > + THD_ENTER_COND(NULL, &COND_slave_lag_, &LOCK_slave_lag_, > + &stage_waiting_for_semi_sync_slave_lag, > + &old_stage); > + > + bool waited = false; > + ulonglong lag = 0; > + ulonglong max_lag = 0; > + while (oldest_unapplied_tranx_commit_time_us_ != 0) > + { > + /* check kill_level after THD_ENTER_COND but *before* cond_wait > + * to avoid missing kills */ > + if (! (getMasterEnabled() && is_on() && > + thd_kill_level(NULL) == THD_IS_NOT_KILLED)) > + break; > + > + lag = start_time_us - oldest_unapplied_tranx_commit_time_us_; > + max_lag = 1000000 * rpl_semi_sync_master_max_slave_lag; > + if (lag <= max_lag) > + break; > + > + waited = true; > + rpl_semi_sync_master_slave_lag_wait_sessions++; > + int wait_result = mysql_cond_timedwait(&COND_slave_lag_, &LOCK_slave_lag_, > + &end_time); > + rpl_semi_sync_master_slave_lag_wait_sessions--; > + > + bool thd_was_killed = thd_kill_level(NULL) != THD_IS_NOT_KILLED; > + if (wait_result != 0 || thd_was_killed) > + { > + break; > + } > + } > + > + if (thd_kill_level(NULL) != THD_IS_NOT_KILLED) > + { > + /* Return error to client. */ > + error = 1; > + my_printf_error(ER_ERROR_DURING_COMMIT, > + "Killed while waiting for replication semi-sync slave-lag.", > + MYF(0)); > + } > + else if (lag > max_lag) > + { > + /* Return error to client. */ > + error = 1; > + my_printf_error(ER_ERROR_DURING_COMMIT, > + "Slave-lag timeout", > + MYF(0)); > + } > + > + if (waited) > + { > + rpl_semi_sync_master_trx_slave_lag_wait_num++; > + rpl_semi_sync_master_trx_slave_lag_wait_time += > + (my_hrtime().val - start_time_us); > + } > + > + /* The lock held will be released by thd_exit_cond, so no need to > + call unlock() here */ > + THD_EXIT_COND(NULL, & old_stage); > + > + return error; > +} > diff --git a/plugin/semisync/semisync_master.h b/plugin/semisync/semisync_master.h > index d9dc4ce..e5c0de6 100644 > --- a/plugin/semisync/semisync_master.h > +++ b/plugin/semisync/semisync_master.h > @@ -24,17 +24,101 @@ > #ifdef HAVE_PSI_INTERFACE > extern PSI_mutex_key key_ss_mutex_LOCK_binlog_; > extern PSI_cond_key key_ss_cond_COND_binlog_send_; > + > +extern PSI_mutex_key key_ss_mutex_LOCK_slave_lag_; > +extern PSI_cond_key key_ss_cond_COND_slave_lag_; > #endif > > extern PSI_stage_info stage_waiting_for_semi_sync_ack_from_slave; > +extern PSI_stage_info stage_waiting_for_semi_sync_slave_lag; > > struct TranxNode { > char log_name_[FN_REFLEN]; > - my_off_t log_pos_; > + my_off_t log_pos_; > + ulonglong tranx_commit_time_us; > struct TranxNode *next_; /* the next node in the sorted list */ > struct TranxNode *hash_next_; /* the next node during hash collision */ > }; > > +struct LogPos; > + > +/* This represent a log position */ > +struct LogPosPtr { > + LogPosPtr() { Uninit();} > + LogPosPtr(const char *name, my_off_t pos) : file_name(name), file_pos(pos){} > + explicit LogPosPtr(const LogPos& pos) { Assign(&pos); } > + > + const char *file_name; > + my_off_t file_pos; > + > + LogPosPtr& Assign(const LogPosPtr *src) { > + file_name = src->file_name; > + file_pos = src->file_pos; > + return *this; > + } > + > + LogPosPtr& Assign(const LogPos *src); > + > + void Uninit() { file_name = NULL;} > + bool IsInited() const { return file_name != NULL; } > +}; > + > +struct LogPos { > + char file_name[FN_REFLEN]; > + my_off_t file_pos; > + > + LogPos() { Uninit(); } > + > + LogPosPtr ToLogPosPtr() const { > + if (IsInited()){ > + LogPosPtr p(file_name, file_pos); > + return p; > + } else { > + LogPosPtr p; > + return p; > + } > + } > + > + LogPos& Assign(const LogPosPtr *src) { > + if (src->IsInited()) { > + strcpy(file_name, src->file_name); > + file_pos = src->file_pos; > + } else { > + Uninit(); > + } > + return *this; > + } > + > + LogPos& Assign(const LogPos* src) { > + LogPosPtr p = src->ToLogPosPtr(); > + Assign(&p); > + return *this; > + } > + > + void Uninit() { file_name[0] = 0; } > + bool IsInited() const { return file_name[0] != 0; } > +}; > + > +inline LogPosPtr& LogPosPtr::Assign(const LogPos* src) { > + LogPosPtr p = src->ToLogPosPtr(); > + Assign(&p); > + return *this; > +} > + > +inline int CompareLogPos(const LogPosPtr *pos1, const LogPosPtr *pos2) { > + int cmp = strcmp(pos1->file_name, pos2->file_name); > + > + if (cmp != 0) > + return cmp; > + > + if (pos1->file_pos > pos2->file_pos) > + return 1; > + else if (pos1->file_pos < pos2->file_pos) > + return -1; > + else > + return 0; > +} > + > /** > @class TranxNodeAllocator > > @@ -329,10 +413,14 @@ class ActiveTranx > node2->log_name_, node2->log_pos_); > } > > + void set_new_front(TranxNode* new_front); > + > public: > ActiveTranx(mysql_mutex_t *lock, unsigned long trace_level); > ~ActiveTranx(); > > + bool is_empty() const { return trx_front_ == NULL; } > + > /* Insert an active transaction node with the specified position. > * > * Return: > @@ -340,21 +428,42 @@ class ActiveTranx > */ > int insert_tranx_node(const char *log_file_name, my_off_t log_file_pos); > > - /* Clear the active transaction nodes until(inclusive) the specified > - * position. > - * If log_file_name is NULL, everything will be cleared: the sorted > + /* Clear the active transaction > + * Everything will be cleared: the sorted > * list and the hash table will be reset to empty. > - * > + * > * Return: > - * 0: success; non-zero: error > + * 0 success; non-zero: error > + */ > + int clear_active_tranx_nodes(); > + > + /* Prune the active transaction nodes until the specified > + * position (inclusive). > + * > + * Return: > + * true if any transaction was removed > + * false if list was left unchanged > */ > - int clear_active_tranx_nodes(const char *log_file_name, > - my_off_t log_file_pos); > + bool prune_active_tranx_nodes(LogPosPtr logpos, > + ulonglong *oldest_tranx_commit_time_us); > > - /* Given a position, check to see whether the position is an active > - * transaction's ending position by probing the hash table. > + /* Lookup a transaction's ending position by probing the hash table. > + * > + * return entry if found or NULL otherwise > */ > - bool is_tranx_end_pos(const char *log_file_name, my_off_t log_file_pos); > + TranxNode* lookup_tranx_end_pos(const char *log_file_name, > + my_off_t log_file_pos); > + > + > + /* Check if an entry is rear (i.e last) */ > + bool is_rear(TranxNode* entry) const { return entry == trx_rear_; } > + > + /** > + * return timestamp of oldest transaction in list > + */ > + ulonglong get_oldest_tranx_commit_time_us() const { > + return trx_front_->tranx_commit_time_us; > + } > > /* Given two binlog positions, compare which one is bigger based on > * (file_name, file_position). > @@ -365,6 +474,24 @@ class ActiveTranx > }; > > /** > + * State that semisync master keeps per slave > + */ > +struct ReplSemiSyncMasterPerSlaveState > +{ > + ReplSemiSyncMasterPerSlaveState() : unacked_event_count_(0) {} > + > + /** > + * No of events that has not been semi-sync acked > + */ > + unsigned unacked_event_count_; > + > + /** > + * Position of last event that was semisync'ed > + */ > + LogPos sync_req_pos_; > +}; > + > +/** > The extension class for the master of semi-synchronous replication > */ > class ReplSemiSyncMaster > @@ -432,6 +559,16 @@ class ReplSemiSyncMaster > > bool state_; /* whether semi-sync is switched */ > > + /* This cond variable is signaled when slave lag has decreased */ > + mysql_cond_t COND_slave_lag_; > + > + /* Mutex that protects oldest_unapplied_tranx_commit_time_us */ > + mysql_mutex_t LOCK_slave_lag_; > + > + /* this is commit time of oldest transaction that has not been applied > + * on any slave */ > + ulonglong oldest_unapplied_tranx_commit_time_us_; > + > void lock(); > void unlock(); > void cond_broadcast(); > @@ -493,6 +630,9 @@ class ReplSemiSyncMaster > /* Is the slave servered by the thread requested semi-sync */ > bool is_semi_sync_slave(); > > + /* Does this slave have slave lag reporting capabilities */ > + bool has_semi_sync_slave_lag(); > + > /* In semi-sync replication, reports up to which binlog position we have > * received replies from the slave indicating that it already get the events. > * > @@ -501,13 +641,15 @@ class ReplSemiSyncMaster > * log_file_name - (IN) binlog file name > * end_offset - (IN) the offset in the binlog file up to which we have > * the replies from the slave > + * exec_position - (IN) position of SQL thread or NULL if not present > * > * Return: > * 0: success; non-zero: error > */ > int reportReplyBinlog(uint32 server_id, > const char* log_file_name, > - my_off_t end_offset); > + my_off_t end_offset, > + const LogPos *exec_position); > > /* Commit a transaction in the final step. This function is called from > * InnoDB before returning from the low commit. If semi-sync is switch on, > @@ -540,6 +682,16 @@ class ReplSemiSyncMaster > */ > int reserveSyncHeader(unsigned char *header, unsigned long size); > > + /* > + * check if an event should be semi synced and optionally > + * if it should report back position of SQL thread on slave > + * > + * return 0 - no semi sync > + * 1 - semi sync > + * 2 - semi sync and report exec position > + */ > + int checkSyncReq(const LogPosPtr *log_pos); > + > /* Update the sync bit in the packet header to indicate to the slave whether > * the master will wait for the reply of the event. If semi-sync is switched > * off and we detect that the slave is catching up, we switch semi-sync on. > @@ -592,6 +744,21 @@ class ReplSemiSyncMaster > * go off for that. > */ > int resetMaster(); > + > + /** > + * wake potential slave-lag waiters > + * called by binlog dump-thread(s) > + */ > + void wake_slave_lag_waiters(ulonglong oldest_unapplied_tranx_commit_time_us); > + > + /** > + * wait for slave lag to get below threshold > + * called by user-thread(s) > + * > + * return 0 - success > + * 1 - timeout > + */ > + int wait_slave_lag(ulong max_wait_time_sec); > }; > > enum rpl_semi_sync_master_wait_point_t { > @@ -621,6 +788,17 @@ extern unsigned long long rpl_semi_sync_master_trx_wait_num; > extern unsigned long long rpl_semi_sync_master_net_wait_time; > extern unsigned long long rpl_semi_sync_master_trx_wait_time; > > +extern unsigned long rpl_semi_sync_master_max_unacked_event_count; > +extern unsigned long rpl_semi_sync_master_max_unacked_event_bytes; > +extern unsigned long rpl_semi_sync_master_max_slave_lag; > +extern unsigned long rpl_semi_sync_master_slave_lag_heartbeat_frequency_us; > +extern unsigned long rpl_semi_sync_master_slave_lag_wait_sessions; > +extern unsigned long long rpl_semi_sync_master_estimated_slave_lag; > + > +extern unsigned long rpl_semi_sync_master_avg_trx_slave_lag_wait_time; > +extern unsigned long long rpl_semi_sync_master_trx_slave_lag_wait_num; > +extern unsigned long long rpl_semi_sync_master_trx_slave_lag_wait_time; > + > /* > This indicates whether we should keep waiting if no semi-sync slave > is available. > diff --git a/plugin/semisync/semisync_master_plugin.cc b/plugin/semisync/semisync_master_plugin.cc > index 7bb0eea..3282158 100644 > --- a/plugin/semisync/semisync_master_plugin.cc > +++ b/plugin/semisync/semisync_master_plugin.cc > @@ -21,6 +21,11 @@ > > static ReplSemiSyncMaster repl_semisync; > > +// forward declaration > +static inline ulong get_slave_lag_wait_timeout(THD* thd); > + > +static char rpl_semi_sync_master_group_commit = 0; > + > C_MODE_START > > int repl_semi_report_binlog_update(Binlog_storage_param *param, > @@ -31,6 +36,13 @@ int repl_semi_report_binlog_update(Binlog_storage_param *param, > > if (repl_semisync.getMasterEnabled()) > { > + if (rpl_semi_sync_master_group_commit && > + ((flags & BINLOG_GROUP_COMMIT_TRAILER) == 0)) > + { > + /** there are transactions more coming... */ > + return 0; > + } > + > /* > Let us store the binlog file name and the position, so that > we know how long to wait for the binlog to the replicated to > @@ -43,8 +55,11 @@ int repl_semi_report_binlog_update(Binlog_storage_param *param, > return error; > } > > -int repl_semi_request_commit(Trans_param *param) > +int repl_semi_before_commit(Trans_param *param, int *error) > { > + *error = repl_semisync.wait_slave_lag( > + get_slave_lag_wait_timeout(current_thd)); > + > return 0; > } > > @@ -53,6 +68,14 @@ int repl_semi_report_binlog_sync(Binlog_storage_param *param, > my_off_t log_pos, uint32 flags) > { > int error= 0; > + > + if (rpl_semi_sync_master_group_commit && > + ((flags & BINLOG_GROUP_COMMIT_TRAILER) == 0)) > + { > + /** there are transactions more coming... */ > + return 0; > + } > + > if (rpl_semi_sync_master_wait_point == > SEMI_SYNC_MASTER_WAIT_POINT_AFTER_BINLOG_SYNC) > { > @@ -100,7 +123,7 @@ int repl_semi_binlog_dump_start(Binlog_transmit_param *param, > Let's assume this semi-sync slave has already received all > binlog events before the filename and position it requests. > */ > - repl_semisync.reportReplyBinlog(param->server_id, log_file, log_pos); > + repl_semisync.reportReplyBinlog(param->server_id, log_file, log_pos, NULL); > } > sql_print_information("Start %s binlog_dump to slave (server_id: %d), pos(%s, %lu)", > semi_sync_slave ? "semi-sync" : "asynchronous", > @@ -242,15 +265,72 @@ static MYSQL_SYSVAR_ULONG(trace_level, rpl_semi_sync_master_trace_level, > &fix_rpl_semi_sync_master_trace_level, // update > 32, 0, ~0UL, 1); > > +static MYSQL_SYSVAR_ULONG(max_unacked_event_count, > + rpl_semi_sync_master_max_unacked_event_count, > + PLUGIN_VAR_OPCMDARG, > + "Maximum unacked replication events", > + NULL, // check > + NULL, // update > + rpl_semi_sync_master_max_unacked_event_count, 0, ~0UL, 1); > + > +static MYSQL_SYSVAR_ULONG(max_unacked_event_bytes, > + rpl_semi_sync_master_max_unacked_event_bytes, > + PLUGIN_VAR_OPCMDARG, > + "Maximum unacked replication bytes", > + NULL, // check > + NULL, // update > + rpl_semi_sync_master_max_unacked_event_bytes, 0, ~0UL, 1); > + > +static MYSQL_SYSVAR_ULONG(max_slave_lag, rpl_semi_sync_master_max_slave_lag, > + PLUGIN_VAR_OPCMDARG, > + "Maximum allowed lag of fastest semi-sync slave (in seconds), " > + "checked before commit.", > + NULL, // check > + NULL, // update > + rpl_semi_sync_master_max_slave_lag, 0, ~0UL, 1); > + > +static MYSQL_THDVAR_ULONG(slave_lag_wait_timeout, > + PLUGIN_VAR_RQCMDARG, > + "Timeout in seconds a rw-transaction may wait for max slave lag before " > + "being rolled back.", > + NULL, NULL, 50, 1, 1024 * 1024 * 1024, 0); > + > +static MYSQL_SYSVAR_ULONG( > + slave_lag_heartbeat_frequency_us, > + rpl_semi_sync_master_slave_lag_heartbeat_frequency_us, > + PLUGIN_VAR_RQCMDARG, > + "Heartbeat frequency when slave-lag is enabled (in microseconds).", > + NULL, // check > + NULL, // update > + 500000, /* 500 ms */ > + 1, ~0UL, 1); > + > +static MYSQL_SYSVAR_BOOL(group_commit, rpl_semi_sync_master_group_commit, > + PLUGIN_VAR_OPCMDARG, > + "Group commit for semi sync", > + NULL, // check > + NULL, > + 0); > + > static SYS_VAR* semi_sync_master_system_vars[]= { > MYSQL_SYSVAR(enabled), > MYSQL_SYSVAR(wait_point), > MYSQL_SYSVAR(timeout), > MYSQL_SYSVAR(wait_no_slave), > MYSQL_SYSVAR(trace_level), > + MYSQL_SYSVAR(max_unacked_event_count), > + MYSQL_SYSVAR(max_unacked_event_bytes), > + MYSQL_SYSVAR(max_slave_lag), > + MYSQL_SYSVAR(slave_lag_wait_timeout), > + MYSQL_SYSVAR(slave_lag_heartbeat_frequency_us), > + MYSQL_SYSVAR(group_commit), > NULL, > }; > > +static inline ulong get_slave_lag_wait_timeout(THD* thd) > +{ > + return THDVAR(thd, slave_lag_wait_timeout); > +} > > static void fix_rpl_semi_sync_master_timeout(MYSQL_THD thd, > SYS_VAR *var, > @@ -297,6 +377,7 @@ Trans_observer trans_observer = { > > repl_semi_report_commit, // after_commit > repl_semi_report_rollback, // after_rollback > + repl_semi_before_commit, // before commit > }; > > Binlog_storage_observer storage_observer = { > @@ -339,7 +420,11 @@ DEF_SHOW_FUNC(net_wait_time, SHOW_LONGLONG) > DEF_SHOW_FUNC(net_wait_num, SHOW_LONGLONG) > DEF_SHOW_FUNC(avg_net_wait_time, SHOW_LONG) > DEF_SHOW_FUNC(avg_trx_wait_time, SHOW_LONG) > - > +DEF_SHOW_FUNC(slave_lag_wait_sessions, SHOW_LONG) > +DEF_SHOW_FUNC(estimated_slave_lag, SHOW_LONGLONG) > +DEF_SHOW_FUNC(trx_slave_lag_wait_time, SHOW_LONGLONG) > +DEF_SHOW_FUNC(trx_slave_lag_wait_num, SHOW_LONGLONG) > +DEF_SHOW_FUNC(avg_trx_slave_lag_wait_time, SHOW_LONG) > > /* plugin status variables */ > static SHOW_VAR semi_sync_master_status_vars[]= { > @@ -385,32 +470,55 @@ static SHOW_VAR semi_sync_master_status_vars[]= { > {"Rpl_semi_sync_master_net_avg_wait_time", > (char*) &SHOW_FNAME(avg_net_wait_time), > SHOW_SIMPLE_FUNC}, > + {"Rpl_semi_sync_master_slave_lag_wait_sessions", > + (char*) &SHOW_FNAME(slave_lag_wait_sessions), > + SHOW_SIMPLE_FUNC}, > + {"Rpl_semi_sync_master_estimated_slave_lag", > + (char*) &SHOW_FNAME(estimated_slave_lag), > + SHOW_SIMPLE_FUNC}, > + {"Rpl_semi_sync_master_tx_slave_lag_wait_time", > + (char*) &SHOW_FNAME(trx_slave_lag_wait_time), > + SHOW_SIMPLE_FUNC}, > + {"Rpl_semi_sync_master_tx_slave_lag_waits", > + (char*) &SHOW_FNAME(trx_slave_lag_wait_num), > + SHOW_SIMPLE_FUNC}, > + {"Rpl_semi_sync_master_tx_avg_slave_lag_wait_time", > + (char*) &SHOW_FNAME(avg_trx_slave_lag_wait_time), > + SHOW_SIMPLE_FUNC}, > {NULL, NULL, SHOW_LONG}, > }; > > #ifdef HAVE_PSI_INTERFACE > PSI_mutex_key key_ss_mutex_LOCK_binlog_; > +PSI_mutex_key key_ss_mutex_LOCK_slave_lag_; > > static PSI_mutex_info all_semisync_mutexes[]= > { > - { &key_ss_mutex_LOCK_binlog_, "LOCK_binlog_", 0} > + { &key_ss_mutex_LOCK_binlog_, "LOCK_binlog_", 0 }, > + { &key_ss_mutex_LOCK_slave_lag_, "LOCK_slave_lag_", 0 } > }; > > PSI_cond_key key_ss_cond_COND_binlog_send_; > +PSI_cond_key key_ss_cond_COND_slave_lag_; > > static PSI_cond_info all_semisync_conds[]= > { > - { &key_ss_cond_COND_binlog_send_, "COND_binlog_send_", 0} > + { &key_ss_cond_COND_binlog_send_, "COND_binlog_send_", 0 }, > + { &key_ss_cond_COND_slave_lag_, "COND_slave_lag_", 0 } > }; > #endif /* HAVE_PSI_INTERFACE */ > > PSI_stage_info stage_waiting_for_semi_sync_ack_from_slave= > { 0, "Waiting for semi-sync ACK from slave", 0}; > > +PSI_stage_info stage_waiting_for_semi_sync_slave_lag= > +{ 0, "Waiting for semi-sync slave lag", 0}; > + > #ifdef HAVE_PSI_INTERFACE > PSI_stage_info *all_semisync_stages[]= > { > - & stage_waiting_for_semi_sync_ack_from_slave > + & stage_waiting_for_semi_sync_ack_from_slave, > + & stage_waiting_for_semi_sync_slave_lag > }; > > static void init_semisync_psi_keys(void) > @@ -492,4 +600,3 @@ maria_declare_plugin(semisync_master) > MariaDB_PLUGIN_MATURITY_GAMMA > } > maria_declare_plugin_end; > - > diff --git a/plugin/semisync/semisync_slave.cc b/plugin/semisync/semisync_slave.cc > index 5f98472..839e0cc 100644 > --- a/plugin/semisync/semisync_slave.cc > +++ b/plugin/semisync/semisync_slave.cc > @@ -20,6 +20,7 @@ > char rpl_semi_sync_slave_enabled; > char rpl_semi_sync_slave_status= 0; > unsigned long rpl_semi_sync_slave_trace_level; > +char rpl_semi_sync_slave_lag_enabled= 0; > > int ReplSemiSyncSlave::initObject() > { > @@ -42,7 +43,7 @@ int ReplSemiSyncSlave::initObject() > > int ReplSemiSyncSlave::slaveReadSyncHeader(const char *header, > unsigned long total_len, > - bool *need_reply, > + unsigned char *need_reply, > const char **payload, > unsigned long *payload_len) > { > @@ -52,7 +53,7 @@ int ReplSemiSyncSlave::slaveReadSyncHeader(const char *header, > > if ((unsigned char)(header[0]) == kPacketMagicNum) > { > - *need_reply = (header[1] & kPacketFlagSync); > + *need_reply = (header[1] & (kPacketFlagSync | kPacketFlagSyncAndReport)); > *payload_len = total_len - 2; > *payload = header + 2; > > @@ -95,16 +96,20 @@ int ReplSemiSyncSlave::slaveStop(Binlog_relay_IO_param *param) > return 0; > } > > -int ReplSemiSyncSlave::slaveReply(MYSQL *mysql, > - const char *binlog_filename, > - my_off_t binlog_filepos) > +int ReplSemiSyncSlave::slaveReply(unsigned char header_byte, > + MYSQL *mysql, > + const char *binlog_filename, > + my_off_t binlog_filepos, > + Master_info * mi) > { > const char *kWho = "ReplSemiSyncSlave::slaveReply"; > NET *net= &mysql->net; > - uchar reply_buffer[REPLY_MAGIC_NUM_LEN > - + REPLY_BINLOG_POS_LEN > - + REPLY_BINLOG_NAME_LEN]; > + uchar reply_buffer[REPLY_MAGIC_NUM_LEN + > + 2 * ( REPLY_BINLOG_POS_LEN + > + REPLY_BINLOG_NAME_LEN + > + /* '\0' */ 1) ]; > int reply_res, name_len = strlen(binlog_filename); > + int msg_len = name_len + REPLY_BINLOG_NAME_OFFSET; > > function_enter(kWho); > > @@ -119,10 +124,29 @@ int ReplSemiSyncSlave::slaveReply(MYSQL *mysql, > sql_print_information("%s: reply (%s, %lu)", kWho, > binlog_filename, (ulong)binlog_filepos); > > + if (header_byte & kPacketFlagSyncAndReport) > + { > + /** > + * master requests that we also report back SQL-thread position > + */ > + > + // where to store sql filename/position > + char *bufptr = (char*)reply_buffer + msg_len; > + bufptr[0] = 0; // '\0' terminate previous filename > + bufptr++; > + > + my_off_t sql_file_pos; > + // get file/position and store the filename directly info bufptr+8 > + size_t name_len2 = get_master_log_pos(mi, bufptr + 8, &sql_file_pos); > + int8store(bufptr, sql_file_pos); // store position > + > + msg_len += /* '\0' */ 1 + /* position */ 8 + name_len2 + /* '\0' */ 1; > + } > + > net_clear(net, 0); > /* Send the reply. */ > - reply_res = my_net_write(net, reply_buffer, > - name_len + REPLY_BINLOG_NAME_OFFSET); > + reply_res = my_net_write(net, reply_buffer, msg_len); > + > if (!reply_res) > { > reply_res = net_flush(net); > diff --git a/plugin/semisync/semisync_slave.h b/plugin/semisync/semisync_slave.h > index 1bf8cf3..c91847d 100644 > --- a/plugin/semisync/semisync_slave.h > +++ b/plugin/semisync/semisync_slave.h > @@ -60,23 +60,30 @@ class ReplSemiSyncSlave > * Return: > * 0: success; non-zero: error > */ > - int slaveReadSyncHeader(const char *header, unsigned long total_len, bool *need_reply, > + int slaveReadSyncHeader(const char *header, unsigned long total_len, > + unsigned char *need_reply_byte, > const char **payload, unsigned long *payload_len); > > /* A slave replies to the master indicating its replication process. It > * indicates that the slave has received all events before the specified > * binlog position. > - * > + * > * Input: > + * need_reply_byte - (IN) the header byte > * mysql - (IN) the mysql network connection > * binlog_filename - (IN) the reply point's binlog file name > * binlog_filepos - (IN) the reply point's binlog file offset > + * master_info - (IN) the master info struct so that we can get more > + * info if needed > * > * Return: > * 0: success; non-zero: error > */ > - int slaveReply(MYSQL *mysql, const char *binlog_filename, > - my_off_t binlog_filepos); > + int slaveReply(unsigned char need_reply_byte, > + MYSQL *mysql, > + const char *binlog_filename, > + my_off_t binlog_filepos, > + Master_info* master_info); > > int slaveStart(Binlog_relay_IO_param *param); > int slaveStop(Binlog_relay_IO_param *param); > @@ -93,5 +100,6 @@ class ReplSemiSyncSlave > extern char rpl_semi_sync_slave_enabled; > extern unsigned long rpl_semi_sync_slave_trace_level; > extern char rpl_semi_sync_slave_status; > +extern char rpl_semi_sync_slave_lag_enabled; > > #endif /* SEMISYNC_SLAVE_H */ > diff --git a/plugin/semisync/semisync_slave_plugin.cc b/plugin/semisync/semisync_slave_plugin.cc > index 572ead2..0bf03be 100644 > --- a/plugin/semisync/semisync_slave_plugin.cc > +++ b/plugin/semisync/semisync_slave_plugin.cc > @@ -28,7 +28,7 @@ static ReplSemiSyncSlave repl_semisync; > event read is the last event of a transaction. And the value is > checked in repl_semi_slave_queue_event. > */ > -bool semi_sync_need_reply= false; > +unsigned char semi_sync_need_reply= 0; > > C_MODE_START > > @@ -81,6 +81,23 @@ int repl_semi_slave_request_dump(Binlog_relay_IO_param *param, > return 1; > } > mysql_free_result(mysql_store_result(mysql)); > + > + if (rpl_semi_sync_slave_lag_enabled) > + { > + char buf[100]; > + /* > + Tell master that we can do exec-position reporting > + */ > + snprintf(buf, sizeof(buf), "SET @%s= 1", > + ReplSemiSyncBase::kRplSemiSyncSlaveReportExec); > + if (mysql_real_query(mysql, buf, strlen(buf))) > + { > + sql_print_error("query: %s on master failed", buf); > + return 1; > + } > + mysql_free_result(mysql_store_result(mysql)); > + } > + > rpl_semi_sync_slave_status= 1; > return 0; > } > @@ -110,9 +127,11 @@ int repl_semi_slave_queue_event(Binlog_relay_IO_param *param, > should not cause the slave IO thread to stop, and the error > messages are already reported. > */ > - (void) repl_semisync.slaveReply(param->mysql, > + (void) repl_semisync.slaveReply(semi_sync_need_reply, > + param->mysql, > param->master_log_name, > - param->master_log_pos); > + param->master_log_pos, > + param->mi); > } > return 0; > } > @@ -164,9 +183,17 @@ static MYSQL_SYSVAR_ULONG(trace_level, rpl_semi_sync_slave_trace_level, > &fix_rpl_semi_sync_trace_level, // update > 32, 0, ~0UL, 1); > > +static MYSQL_SYSVAR_BOOL(lag_enabled, rpl_semi_sync_slave_lag_enabled, > + PLUGIN_VAR_OPCMDARG, > + "Enable semi-synchronous replication slave lag reporting. ", > + NULL, // check > + NULL, // update > + 0); > + > static SYS_VAR* semi_sync_slave_system_vars[]= { > MYSQL_SYSVAR(enabled), > MYSQL_SYSVAR(trace_level), > + MYSQL_SYSVAR(lag_enabled), > NULL, > }; > > @@ -230,4 +257,3 @@ maria_declare_plugin(semisync_slave) > MariaDB_PLUGIN_MATURITY_GAMMA > } > maria_declare_plugin_end; > - > diff --git a/sql/handler.cc b/sql/handler.cc > index 3ca9ec3..3e6cd65 100644 > --- a/sql/handler.cc > +++ b/sql/handler.cc > @@ -1364,12 +1364,30 @@ int ha_commit_trans(THD *thd, bool all) > uint rw_ha_count= ha_check_and_coalesce_trx_read_only(thd, ha_info, all); > /* rw_trans is TRUE when we in a transaction changing data */ > bool rw_trans= is_real_trans && (rw_ha_count > 0); > + bool mdl_request_initialized= false; > MDL_request mdl_request; > DBUG_PRINT("info", ("is_real_trans: %d rw_trans: %d rw_ha_count: %d", > is_real_trans, rw_trans, rw_ha_count)); > > if (rw_trans) > { > + /* check READ-ONLY just before before_commit hook to decrease likelihood > + * of having threads hanging waiting for slave-lag only to be aborted > + * due to read-only. > + */ > + if (opt_readonly && > + !(thd->security_ctx->master_access & SUPER_ACL) && > + !thd->slave_thread) > + { > + my_error(ER_OPTION_PREVENTS_STATEMENT, MYF(0), "--read-only"); > + goto err; > + } > + > + if (RUN_HOOK(transaction, before_commit, (thd))) > + { > + goto err; > + } > + > /* > Acquire a metadata lock which will ensure that COMMIT is blocked > by an active FLUSH TABLES WITH READ LOCK (and vice versa: > @@ -1378,6 +1396,7 @@ int ha_commit_trans(THD *thd, bool all) > We allow the owner of FTWRL to COMMIT; we assume that it knows > what it does. > */ > + mdl_request_initialized= true; > mdl_request.init(MDL_key::COMMIT, "", "", MDL_INTENTION_EXCLUSIVE, > MDL_EXPLICIT); > > @@ -1486,7 +1505,7 @@ int ha_commit_trans(THD *thd, bool all) > ha_rollback_trans(thd, all); > > end: > - if (rw_trans && mdl_request.ticket) > + if (rw_trans && mdl_request_initialized && mdl_request.ticket) > { > /* > We do not always immediately release transactional locks > diff --git a/sql/replication.h b/sql/replication.h > index 4731c22..309bdb4 100644 > --- a/sql/replication.h > +++ b/sql/replication.h > @@ -110,6 +110,21 @@ typedef struct Trans_observer { > @retval 1 Failure > */ > int (*after_rollback)(Trans_param *param); > + > + /** > + This callback is called before transaction commit > + If function does not return *error == 0 transaction will > + not be committed but error code will be returned to client > + > + @note *error!=0 and return code 0 shall be used by plugin to signal > + that transaction should be aborted. > + If returning non-zero transaction will also be aborted and an error > + will be printed to error log. > + > + @retval 0 Sucess > + @retval non-zero error > + */ > + int (*before_commit)(Trans_param *param, int *error); > } Trans_observer; > > /** > @@ -294,6 +309,8 @@ enum Binlog_relay_IO_flags { > }; > > > +class Master_info; > + > /** > Replication binlog relay IO observer parameter > */ > @@ -309,8 +326,20 @@ typedef struct Binlog_relay_IO_param { > my_off_t master_log_pos; > > MYSQL *mysql; /* the connection to master */ > + > + Master_info * mi; /* master info handle */ > } Binlog_relay_IO_param; > > + > +/* get the master log given a Master_info > + * and store it in filename_buf/filepos > + * return length of filename (excluding \0) > + * > + * note: filename_buf should be a minimum FN_REFLEN > + */ > +size_t get_master_log_pos(const Master_info *mi, > + char *filename_buf, my_off_t *filepos); > + > /** > Observes and extends the service of slave IO thread. > */ > @@ -561,7 +590,21 @@ int get_user_var_str(const char *name, > char *value, unsigned long len, > unsigned int precision, int *null_value); > > - > + > +/** > + Set or replace the value of user variable as to an ulonglong > + > + @param name user variable name > + @param value the value > + @param old_value pointer to where old value will be stored (or NULL) > + > + @retval 0 Success, no prior value found > + @retval 1 Success, old_value populated > + @retval -1 Fail > +*/ > +int set_user_var_int(const char *name, > + long long int value, > + long long int *old_value); > > #ifdef __cplusplus > } > diff --git a/sql/rpl_handler.cc b/sql/rpl_handler.cc > index 3962600..209a809 100644 > --- a/sql/rpl_handler.cc > +++ b/sql/rpl_handler.cc > @@ -23,6 +23,7 @@ > #include "rpl_filter.h" > #include <my_dir.h> > #include "rpl_handler.h" > +#include "sql_prepare.h" > > Trans_delegate *transaction_delegate; > Binlog_storage_delegate *binlog_storage_delegate; > @@ -88,6 +89,42 @@ int get_user_var_str(const char *name, char *value, > return 0; > } > > +int set_user_var_int(const char *name, > + long long int value, > + long long int *old_value) > +{ > + THD* thd= current_thd; > + bool null_val; > + user_var_entry *entry= > + (user_var_entry*) my_hash_search(&thd->user_vars, > + (uchar*) name, strlen(name)); > + if (entry != NULL) > + { > + if (old_value != NULL) > + *old_value= entry->val_int(&null_val); > + } > + > + Ed_connection con(thd); > + > + char buf[256]; > + int res= snprintf(buf, sizeof(buf), "SET @%s=%lld", name, value); > + if (/* error */ res < 0 || > + /* truncated */ res >= sizeof(buf)) > + { > + return -1; > + } > + > + LEX_STRING str; > + lex_string_set(&str, buf); > + > + if (con.execute_direct(str)) > + { > + return -1; > + } > + > + return entry == NULL ? 0 : 1; > +} > + > int delegates_init() > { > static my_aligned_storage<sizeof(Trans_delegate), MY_ALIGNOF(long)> trans_mem; > @@ -249,6 +286,17 @@ int Trans_delegate::after_rollback(THD *thd, bool all) > return ret; > } > > +int Trans_delegate::before_commit(THD *thd) > +{ > + int ret= 0, error= 0; > + Trans_param param; > + param.flags= 0; > + param.log_file= 0; > + param.log_pos= 0; > + FOREACH_OBSERVER(ret, before_commit, thd, (&param, &error)); > + return error; > +} > + > int Binlog_storage_delegate::after_flush(THD *thd, > const char *log_file, > my_off_t log_pos, > @@ -374,17 +422,19 @@ int Binlog_transmit_delegate::reserve_header(THD *thd, ushort flags, > > int Binlog_transmit_delegate::before_send_event(THD *thd, ushort flags, > String *packet, > - const char *log_file, > + const char *log_file_path, > my_off_t log_pos) > { > Binlog_transmit_param param; > param.flags= flags; > > int ret= 0; > + const char* log_file_name= log_file_path != NULL ? > + log_file_path + dirname_length(log_file_path) : NULL; > FOREACH_OBSERVER(ret, before_send_event, false, > (&param, (uchar *)packet->c_ptr(), > packet->length(), > - log_file+dirname_length(log_file), log_pos)); > + log_file_name, log_pos)); > return ret; > } > > @@ -414,6 +464,7 @@ int Binlog_transmit_delegate::after_reset_master(THD *thd, ushort flags) > void Binlog_relay_IO_delegate::init_param(Binlog_relay_IO_param *param, > Master_info *mi) > { > + param->mi = mi; > param->mysql= mi->mysql; > param->user= mi->user; > param->host= mi->host; > @@ -540,6 +591,20 @@ int unregister_binlog_relay_io_observer(Binlog_relay_IO_observer *observer, void > { > return binlog_relay_io_delegate->remove_observer(observer, (st_plugin_int *)p); > } > + > +/* get master log pos for a Master_info struct */ > +size_t get_master_log_pos(const Master_info* mi, > + char *filename_buf, my_off_t *filepos) > +{ > + mysql_mutex_t *mutex= &mi->rli.data_lock; > + > + mysql_mutex_lock(mutex); > + *filepos= mi->rli.group_master_log_pos; > + strncpy(filename_buf, mi->rli.group_master_log_name, FN_REFLEN); > + mysql_mutex_unlock(mutex); > + return strnlen(filename_buf, FN_REFLEN); > +} > + > #else > int register_binlog_transmit_observer(Binlog_transmit_observer *observer, void *p) > { > @@ -560,4 +625,13 @@ int unregister_binlog_relay_io_observer(Binlog_relay_IO_observer *observer, void > { > return 0; > } > + > +size_t get_master_log_pos(const Master_info* mi, > + char *filename_buf, my_off_t *filepos) > +{ > + *filepos= 0; > + filename_buf[0]= 0; > + return 0; > +} > + > #endif /* HAVE_REPLICATION */ > diff --git a/sql/rpl_handler.h b/sql/rpl_handler.h > index afcfd9d..5119ee4 100644 > --- a/sql/rpl_handler.h > +++ b/sql/rpl_handler.h > @@ -142,7 +142,7 @@ class Trans_delegate > :public Delegate { > public: > typedef Trans_observer Observer; > - int before_commit(THD *thd, bool all); > + int before_commit(THD *thd); > int before_rollback(THD *thd, bool all); > int after_commit(THD *thd, bool all); > int after_rollback(THD *thd, bool all); > diff --git a/sql/rpl_rli.h b/sql/rpl_rli.h > index 56300c6..d60a122 100644 > --- a/sql/rpl_rli.h > +++ b/sql/rpl_rli.h > @@ -154,7 +154,9 @@ class Relay_log_info : public Slave_reporting_capability > standard lock acquisition order to avoid deadlocks: > run_lock, data_lock, relay_log.LOCK_log, relay_log.LOCK_index > */ > - mysql_mutex_t data_lock, run_lock; > + mutable mysql_mutex_t data_lock; > + mysql_mutex_t run_lock; > + > /* > start_cond is broadcast when SQL thread is started > stop_cond - when stopped > diff --git a/sql/sql_repl.cc b/sql/sql_repl.cc > index d9ae6ca..9662058 100644 > --- a/sql/sql_repl.cc > +++ b/sql/sql_repl.cc > @@ -818,6 +818,13 @@ static int send_heartbeat_event(binlog_send_info *info, > packet->append(b, sizeof(b)); > } > > + if (RUN_HOOK(binlog_transmit, before_send_event, > + (info->thd, info->flags, packet, 0, 0))) > + { > + info->error= ER_UNKNOWN_ERROR; > + DBUG_RETURN(-1); > + } > + > if (my_net_write(net, (uchar*) packet->ptr(), packet->length()) || > net_flush(net)) > { > @@ -825,6 +832,13 @@ static int send_heartbeat_event(binlog_send_info *info, > DBUG_RETURN(-1); > } > > + if (RUN_HOOK(binlog_transmit, after_send_event, > + (info->thd, info->flags, packet))) > + { > + info->error= ER_UNKNOWN_ERROR; > + DBUG_RETURN(-1); > + } > + > DBUG_RETURN(0); > } >

1 0

Re: [Maria-developers] [Commits] 51926a1: Fix assertion failure.
by Sergei Golubchik 16 May '15

16 May '15

Hi, Jan! On May 16, Jan Lindström wrote: > revision-id: 51926a1d2ed756902257fe802b1a01abb89a8fa3 > parent(s): 8d2117c0fd0a2211f1ec55f1d61fc5ae822c0ccf > committer: Jan Lindström > branch nick: 10.1-encrypt > timestamp: 2015-05-16 08:58:03 +0300 > message: > > Fix assertion failure. > > --- > mysql-test/suite/encryption/r/encrypt_and_grep.result | 6 +----- > mysql-test/suite/encryption/t/encrypt_and_grep.test | 10 +--------- > storage/innobase/fil/fil0crypt.cc | 9 +++++++-- > storage/xtradb/fil/fil0crypt.cc | 9 +++++++-- > 4 files changed, 16 insertions(+), 18 deletions(-) > > diff --git a/mysql-test/suite/encryption/t/encrypt_and_grep.test b/mysql-test/suite/encryption/t/encrypt_and_grep.test > index 1968b71..0301aa2 100644 > --- a/mysql-test/suite/encryption/t/encrypt_and_grep.test > +++ b/mysql-test/suite/encryption/t/encrypt_and_grep.test > @@ -52,7 +52,7 @@ insert t3 values (repeat('dummy', 42)); > SET GLOBAL innodb_encrypt_tables = off; > SET GLOBAL innodb_encryption_threads = 4; > > ---echo # Wait max 10 min for key encryption threads to decrypt all space > +--echo # Wait max 10 min for key encryption threads to decrypt all spacew A typo? Did you mean "spaces" ? > --let $wait_timeout= 600 > --let $wait_condition=SELECT COUNT(*) = 1 FROM INFORMATION_SCHEMA.INNODB_TABLESPACES_ENCRYPTION WHERE MIN_KEY_VERSION <> 0; > --source include/wait_condition.inc > @@ -69,14 +69,6 @@ SET GLOBAL innodb_encryption_threads = 4; > --echo # t3 no on expecting FOUND > -- let SEARCH_FILE=$t3_IBD > -- source include/search_pattern_in_file.inc > ---let SEARCH_PATTERN=foobar > ---echo # ibdata1 expecting NOT FOUND > --- let SEARCH_FILE=$ib1_IBD > --- source include/search_pattern_in_file.inc > ---let SEARCH_PATTERN=temp > ---echo # ibdata1 expecting NOT FOUND > --- let SEARCH_FILE=$ib1_IBD > --- source include/search_pattern_in_file.inc Why? > --echo # Now turn on encryption and wait for threads to encrypt all spaces > SET GLOBAL innodb_encryption_threads = 4; > diff --git a/storage/innobase/fil/fil0crypt.cc b/storage/innobase/fil/fil0crypt.cc > index 65be860..0c2280f 100644 > --- a/storage/innobase/fil/fil0crypt.cc > +++ b/storage/innobase/fil/fil0crypt.cc > @@ -2128,8 +2128,13 @@ fil_crypt_complete_rotate_space( > crypt_data->rotate_state.end_lsn = state->end_lsn; > } > > - ut_a(crypt_data->rotate_state.active_threads > 0); > - crypt_data->rotate_state.active_threads--; > + /* Remember that we can try to rotate tablespaces with > + FIL_SPACE_ENCRYPTION_OFF and we might want to scrub them > + also. */ Do you mean that when rotating FIL_SPACE_ENCRYPTION_OFF spaces crypt_data->rotate_state.active_threads == 0? > + if (crypt_data->rotate_state.active_threads > 0) { > + crypt_data->rotate_state.active_threads--; > + } > + > bool last = crypt_data->rotate_state.active_threads == 0; Regards, Sergei

1 0

[Maria-developers] Need help with review/suggestions for MDEV-8075 (was Re: [Commits] c325a34: MDEV-8075 ...)
by Kristian Nielsen 15 May '15

15 May '15

Hi Serg, Svoj, I need some help with MDEV-8075, as it involves code around transaction start/end which I am not familiar with. I hope one of you know enough about that part of the server to help me? I have a partial patch (below), but it still has some problems remaining and I am not sure how to solve it properly. I tried to base the patch on some corresponding MySQL code, which was only partially merged to MariaDB 10.1. The basic problem is to remember during a transaction whether DDL was used, and then at the end of the transaction in binlogging, to set a flag in the GTID event if DDL is used. The concrete problem is with this kind of SQL: BEGIN; INSERT INTO innodb_table VALUES (1); ALTER TABLE temporary_table ADD b INT; In this case the ALTER TABLE does not get the "ddl" mark in my patch. The reason is that during mysql_alter_table(), THD_TRANS::reset() gets called, which clears the flag DID_DDL that was set in mysql_execute_command(). Call stack: #0 THD_TRANS::reset #1 trans_commit_implicit #2 ha_enable_transaction #3 mysql_trans_commit_alter_copy_data #4 copy_data_between_tables #5 mysql_alter_table #6 Sql_cmd_alter_table::execute #7 mysql_execute_command (An earlier version of the patch looked directly at thd->lex->sql_command during binlogging to see if it was a DDL. That caused another problem: the INSERT INTO got wrongly marked as DDL, because it is implicitly committed by an ALTER TABLE statement). But the more general problem is that I am very unsure about the proper places to put in code to clear the required flags at the start of a transaction, and to copy relevant flags from "stmt" to "all" transaction. I tried using what I found in MySQL code, and to look for where the existing MariaDB code copies the existing THD_TRANS::modified_non_trans_table flag. But I am very unsure that I choose the right places, the existing code seems rather ad-hoc (I'm wondering if it is even correct). I also replaced some add-hoc code that clears THD_TRANS::modified_non_trans_table directly with calls to THD_TRANS::reset(); that seems more appropriate when more flags are affected, but it does more things (notably clearing no_2pc), so again I'm very unsure if it is correct... Any suggestions welcomed. The underlying bug makes optimistic parallel replication quite broken with temporary tables and statement-based binlogging in 10.1 (can probably even crash the server), so it is of some importance to get fixed.... - Kristian. ----------------------------------------------------------------------- revision-id: c325a34d93b931d1c927255c18df94e1560af6eb parent(s): 53382ac128d9571b5f779bf625567fc3ea0b475e committer: Kristian Nielsen branch nick: mariadb timestamp: 2015-05-15 14:56:54 +0200 message: MDEV-8075: DROP TEMPORARY TABLE not marked as ddl, causing optimistic parallel replication to fail CREATE/DROP TEMPORARY TABLE are not safe to optimistically replicate in parallel with other transactions, so they need to be marked as "ddl" in the binlog. This was already done for stand-alone CREATE/DROP TEMPORARY. But temporary tables can also be created and dropped inside a BEGIN...END transaction, and such transactions were not marked as ddl. Nor was the DROP TEMPORARY TABLE statement emitted implicitly when a client connection is closed. So this patch adds such ddl mark for the missing cases. --- .../include/binlog_parallel_replication_marks.test | 83 ++++++++++++++++++ .../r/binlog_parallel_replication_marks_row.result | 98 ++++++++++++++++++++++ ...inlog_parallel_replication_marks_stm_mix.result | 95 +++++++++++++++++++++ .../t/binlog_parallel_replication_marks_row.test | 3 + .../binlog_parallel_replication_marks_stm_mix.test | 3 + .../suite/rpl/r/rpl_parallel_optimistic.result | 27 ++++++ .../suite/rpl/t/rpl_parallel_optimistic.test | 25 ++++++ sql/handler.h | 14 ++++ sql/log_event.cc | 6 +- sql/sql_base.cc | 1 + sql/sql_class.h | 10 +++ sql/sql_insert.cc | 8 ++ sql/sql_parse.cc | 4 +- sql/sql_table.cc | 22 +++-- sql/transaction.cc | 25 +++--- 15 files changed, 400 insertions(+), 24 deletions(-) diff --git a/mysql-test/include/binlog_parallel_replication_marks.test b/mysql-test/include/binlog_parallel_replication_marks.test new file mode 100644 index 0000000..4775e30 --- /dev/null +++ b/mysql-test/include/binlog_parallel_replication_marks.test @@ -0,0 +1,83 @@ +# Test the markings on GTID events (ddl, waited, trans, +# @@skip_parallel_replication) that are used to control parallel +# replication on the slave. + +--source include/have_innodb.inc + +RESET MASTER; +--source include/wait_for_binlog_checkpoint.inc + +--let $stable_stamp= `SELECT UNIX_TIMESTAMP("2020-01-21 15:32:22")` +eval set timestamp=$stable_stamp; + +CREATE TABLE t1 (a INT PRIMARY KEY, b INT) ENGINE=InnoDB; +--let $binlog_pos1=query_get_value(SHOW MASTER STATUS, Position, 1) +/* GTID */ INSERT INTO t1 VALUES (1,0); +/* GTID */ BEGIN; +/* GTID */ INSERT INTO t1 VALUES (2,0); +/* GTID */ ALTER TABLE t1 ADD c INT; +/* GTID */ INSERT INTO t1 VALUES (3,0,0); +/* GTID */ COMMIT; +/* GTID */ BEGIN; +/* GTID */ UPDATE t1 SET b=1, c=1 WHERE a=2; +/* GTID */ CREATE TEMPORARY TABLE t2 (a INT PRIMARY KEY, b INT) ENGINE=InnoDB; +/* GTID */ INSERT INTO t2 VALUES (4,10), (5,20); +/* GTID */ INSERT INTO t1 SELECT a, 2, b FROM t2; +/* GTID */ DROP TEMPORARY TABLE t2; +/* GTID */ INSERT INTO t1 VALUES (6, 3, 0); +/* GTID */ COMMIT; +/* GTID */ CREATE TEMPORARY TABLE t3 (a INT PRIMARY KEY) ENGINE=InnoDB; +/* GTID */ BEGIN; +/* GTID */ DELETE FROM t1 WHERE a=5; +/* GTID */ INSERT INTO t3 VALUES (7); +/* GTID */ INSERT INTO t1 SELECT a, 4, 0 FROM t3; +/* GTID */ UPDATE t1 SET c=1 WHERE a=7; +/* GTID */ DROP TEMPORARY TABLE t3; +/* GTID */ COMMIT; +/* GTID */ CREATE TEMPORARY TABLE t4 (a INT PRIMARY KEY) ENGINE=InnoDB; +/* GTID */ BEGIN; +/* GTID */ INSERT INTO t1 VALUES (8, 5, 0); +/* GTID */ ALTER TABLE t4 ADD b INT; +/* GTID */ INSERT INTO t1 VALUES (9, 5, 1); +/* GTID */ COMMIT; +connect (tmp_con,localhost,root,,); +eval set timestamp=$stable_stamp; +/* GTID */ INSERT INTO t1 VALUES (10, 6, 0); +/* GTID */ BEGIN; +/* GTID */ CREATE TEMPORARY TABLE t5 (a INT PRIMARY KEY) ENGINE=InnoDB; +/* GTID */ INSERT INTO t1 VALUES (11, 7, 0); +/* GTID */ COMMIT; +--let $before_drop_pos=query_get_value(SHOW MASTER STATUS, Position, 1) +disconnect tmp_con; +connection default; + +# We need to wait for the implicit DROP TEMPORARY TABLE to be logged after +# tmp_con disconnect, otherwise we get sporadic test failures. +--let $wait_condition= SELECT variable_value > $before_drop_pos FROM information_schema.global_status WHERE variable_name = 'binlog_snapshot_position' +--source include/wait_condition.inc + +--let $binlog_pos2=query_get_value(SHOW MASTER STATUS, Position, 1) + +--let $binlog_file= query_get_value(SHOW MASTER STATUS, File, 1) +FLUSH LOGS; + +--let $MYSQLD_DATADIR= `select @@datadir` +--let $file= $MYSQLTEST_VARDIR/tmp/binlog_parallel_replication_marks.out +--let OUTPUT_FILE=$file +exec $MYSQL_BINLOG --start_position=$binlog_pos1 --stop_position=$binlog_pos2 $MYSQLD_DATADIR/$binlog_file > $file; + +perl; +my $file= $ENV{'OUTPUT_FILE'}; +open F, "<", $file + or die "Unable to open file '$file': $!\n"; +while (<F>) { + s/GTID \d+-\d+-\d+/GTID #-#-#/; + s/end_log_pos \d+/end_log_pos #/; + s/table id \d+/table id #/; + s/mapped to number \d+/mapped to number #/; + print if /GTID|BEGIN|COMMIT|Table_map|Write_rows|Update_rows|Delete_rows|generated by server|40005 TEMPORARY/; +} +close F; +EOF + +DROP TABLE t1; diff --git a/mysql-test/suite/binlog/r/binlog_parallel_replication_marks_row.result b/mysql-test/suite/binlog/r/binlog_parallel_replication_marks_row.result new file mode 100644 index 0000000..b9c8666 --- /dev/null +++ b/mysql-test/suite/binlog/r/binlog_parallel_replication_marks_row.result @@ -0,0 +1,98 @@ +RESET MASTER; +set timestamp=1579617142; +CREATE TABLE t1 (a INT PRIMARY KEY, b INT) ENGINE=InnoDB; +/* GTID */ INSERT INTO t1 VALUES (1,0); +/* GTID */ BEGIN; +/* GTID */ INSERT INTO t1 VALUES (2,0); +/* GTID */ ALTER TABLE t1 ADD c INT; +/* GTID */ INSERT INTO t1 VALUES (3,0,0); +/* GTID */ COMMIT; +/* GTID */ BEGIN; +/* GTID */ UPDATE t1 SET b=1, c=1 WHERE a=2; +/* GTID */ CREATE TEMPORARY TABLE t2 (a INT PRIMARY KEY, b INT) ENGINE=InnoDB; +/* GTID */ INSERT INTO t2 VALUES (4,10), (5,20); +/* GTID */ INSERT INTO t1 SELECT a, 2, b FROM t2; +/* GTID */ DROP TEMPORARY TABLE t2; +/* GTID */ INSERT INTO t1 VALUES (6, 3, 0); +/* GTID */ COMMIT; +/* GTID */ CREATE TEMPORARY TABLE t3 (a INT PRIMARY KEY) ENGINE=InnoDB; +/* GTID */ BEGIN; +/* GTID */ DELETE FROM t1 WHERE a=5; +/* GTID */ INSERT INTO t3 VALUES (7); +/* GTID */ INSERT INTO t1 SELECT a, 4, 0 FROM t3; +/* GTID */ UPDATE t1 SET c=1 WHERE a=7; +/* GTID */ DROP TEMPORARY TABLE t3; +/* GTID */ COMMIT; +/* GTID */ CREATE TEMPORARY TABLE t4 (a INT PRIMARY KEY) ENGINE=InnoDB; +/* GTID */ BEGIN; +/* GTID */ INSERT INTO t1 VALUES (8, 5, 0); +/* GTID */ ALTER TABLE t4 ADD b INT; +/* GTID */ INSERT INTO t1 VALUES (9, 5, 1); +/* GTID */ COMMIT; +set timestamp=1579617142; +/* GTID */ INSERT INTO t1 VALUES (10, 6, 0); +/* GTID */ BEGIN; +/* GTID */ CREATE TEMPORARY TABLE t5 (a INT PRIMARY KEY) ENGINE=InnoDB; +/* GTID */ INSERT INTO t1 VALUES (11, 7, 0); +/* GTID */ COMMIT; +FLUSH LOGS; +#200121 15:32:22 server id 1 end_log_pos # GTID #-#-# trans +BEGIN +#200121 15:32:22 server id 1 end_log_pos # Table_map: `test`.`t1` mapped to number # +#200121 15:32:22 server id 1 end_log_pos # Write_rows: table id # flags: STMT_END_F +COMMIT/*!*/; +#200121 15:32:22 server id 1 end_log_pos # GTID #-#-# trans +BEGIN +#200121 15:32:22 server id 1 end_log_pos # Table_map: `test`.`t1` mapped to number # +#200121 15:32:22 server id 1 end_log_pos # Write_rows: table id # flags: STMT_END_F +COMMIT/*!*/; +#200121 15:32:22 server id 1 end_log_pos # GTID #-#-# ddl +/* GTID */ ALTER TABLE t1 ADD c INT +#200121 15:32:22 server id 1 end_log_pos # GTID #-#-# trans +BEGIN +#200121 15:32:22 server id 1 end_log_pos # Table_map: `test`.`t1` mapped to number # +#200121 15:32:22 server id 1 end_log_pos # Write_rows: table id # flags: STMT_END_F +COMMIT/*!*/; +#200121 15:32:22 server id 1 end_log_pos # GTID #-#-# ddl +BEGIN +#200121 15:32:22 server id 1 end_log_pos # Table_map: `test`.`t1` mapped to number # +#200121 15:32:22 server id 1 end_log_pos # Update_rows: table id # flags: STMT_END_F +#200121 15:32:22 server id 1 end_log_pos # Table_map: `test`.`t1` mapped to number # +#200121 15:32:22 server id 1 end_log_pos # Write_rows: table id # flags: STMT_END_F +DROP TEMPORARY TABLE IF EXISTS `test`.`t2` /* generated by server */ +#200121 15:32:22 server id 1 end_log_pos # Table_map: `test`.`t1` mapped to number # +#200121 15:32:22 server id 1 end_log_pos # Write_rows: table id # flags: STMT_END_F +COMMIT/*!*/; +#200121 15:32:22 server id 1 end_log_pos # GTID #-#-# ddl +BEGIN +#200121 15:32:22 server id 1 end_log_pos # Table_map: `test`.`t1` mapped to number # +#200121 15:32:22 server id 1 end_log_pos # Delete_rows: table id # flags: STMT_END_F +#200121 15:32:22 server id 1 end_log_pos # Table_map: `test`.`t1` mapped to number # +#200121 15:32:22 server id 1 end_log_pos # Write_rows: table id # flags: STMT_END_F +#200121 15:32:22 server id 1 end_log_pos # Table_map: `test`.`t1` mapped to number # +#200121 15:32:22 server id 1 end_log_pos # Update_rows: table id # flags: STMT_END_F +DROP TEMPORARY TABLE IF EXISTS `test`.`t3` /* generated by server */ +COMMIT/*!*/; +#200121 15:32:22 server id 1 end_log_pos # GTID #-#-# trans +BEGIN +#200121 15:32:22 server id 1 end_log_pos # Table_map: `test`.`t1` mapped to number # +#200121 15:32:22 server id 1 end_log_pos # Write_rows: table id # flags: STMT_END_F +COMMIT/*!*/; +#200121 15:32:22 server id 1 end_log_pos # GTID #-#-# trans +BEGIN +#200121 15:32:22 server id 1 end_log_pos # Table_map: `test`.`t1` mapped to number # +#200121 15:32:22 server id 1 end_log_pos # Write_rows: table id # flags: STMT_END_F +COMMIT/*!*/; +#200121 15:32:22 server id 1 end_log_pos # GTID #-#-# trans +BEGIN +#200121 15:32:22 server id 1 end_log_pos # Table_map: `test`.`t1` mapped to number # +#200121 15:32:22 server id 1 end_log_pos # Write_rows: table id # flags: STMT_END_F +COMMIT/*!*/; +#200121 15:32:22 server id 1 end_log_pos # GTID #-#-# trans +BEGIN +#200121 15:32:22 server id 1 end_log_pos # Table_map: `test`.`t1` mapped to number # +#200121 15:32:22 server id 1 end_log_pos # Write_rows: table id # flags: STMT_END_F +COMMIT/*!*/; +#200121 15:32:22 server id 1 end_log_pos # GTID #-#-# ddl +DROP /*!40005 TEMPORARY */ TABLE IF EXISTS `t5` +DROP TABLE t1; diff --git a/mysql-test/suite/binlog/r/binlog_parallel_replication_marks_stm_mix.result b/mysql-test/suite/binlog/r/binlog_parallel_replication_marks_stm_mix.result new file mode 100644 index 0000000..c3dc00d --- /dev/null +++ b/mysql-test/suite/binlog/r/binlog_parallel_replication_marks_stm_mix.result @@ -0,0 +1,95 @@ +RESET MASTER; +set timestamp=1579617142; +CREATE TABLE t1 (a INT PRIMARY KEY, b INT) ENGINE=InnoDB; +/* GTID */ INSERT INTO t1 VALUES (1,0); +/* GTID */ BEGIN; +/* GTID */ INSERT INTO t1 VALUES (2,0); +/* GTID */ ALTER TABLE t1 ADD c INT; +/* GTID */ INSERT INTO t1 VALUES (3,0,0); +/* GTID */ COMMIT; +/* GTID */ BEGIN; +/* GTID */ UPDATE t1 SET b=1, c=1 WHERE a=2; +/* GTID */ CREATE TEMPORARY TABLE t2 (a INT PRIMARY KEY, b INT) ENGINE=InnoDB; +/* GTID */ INSERT INTO t2 VALUES (4,10), (5,20); +/* GTID */ INSERT INTO t1 SELECT a, 2, b FROM t2; +/* GTID */ DROP TEMPORARY TABLE t2; +/* GTID */ INSERT INTO t1 VALUES (6, 3, 0); +/* GTID */ COMMIT; +/* GTID */ CREATE TEMPORARY TABLE t3 (a INT PRIMARY KEY) ENGINE=InnoDB; +/* GTID */ BEGIN; +/* GTID */ DELETE FROM t1 WHERE a=5; +/* GTID */ INSERT INTO t3 VALUES (7); +/* GTID */ INSERT INTO t1 SELECT a, 4, 0 FROM t3; +/* GTID */ UPDATE t1 SET c=1 WHERE a=7; +/* GTID */ DROP TEMPORARY TABLE t3; +/* GTID */ COMMIT; +/* GTID */ CREATE TEMPORARY TABLE t4 (a INT PRIMARY KEY) ENGINE=InnoDB; +/* GTID */ BEGIN; +/* GTID */ INSERT INTO t1 VALUES (8, 5, 0); +/* GTID */ ALTER TABLE t4 ADD b INT; +/* GTID */ INSERT INTO t1 VALUES (9, 5, 1); +/* GTID */ COMMIT; +set timestamp=1579617142; +/* GTID */ INSERT INTO t1 VALUES (10, 6, 0); +/* GTID */ BEGIN; +/* GTID */ CREATE TEMPORARY TABLE t5 (a INT PRIMARY KEY) ENGINE=InnoDB; +/* GTID */ INSERT INTO t1 VALUES (11, 7, 0); +/* GTID */ COMMIT; +FLUSH LOGS; +#200121 15:32:22 server id 1 end_log_pos # GTID #-#-# trans +BEGIN +/* GTID */ INSERT INTO t1 VALUES (1,0) +COMMIT/*!*/; +#200121 15:32:22 server id 1 end_log_pos # GTID #-#-# trans +BEGIN +/* GTID */ INSERT INTO t1 VALUES (2,0) +COMMIT/*!*/; +#200121 15:32:22 server id 1 end_log_pos # GTID #-#-# ddl +/* GTID */ ALTER TABLE t1 ADD c INT +#200121 15:32:22 server id 1 end_log_pos # GTID #-#-# trans +BEGIN +/* GTID */ INSERT INTO t1 VALUES (3,0,0) +COMMIT/*!*/; +#200121 15:32:22 server id 1 end_log_pos # GTID #-#-# ddl +BEGIN +/* GTID */ UPDATE t1 SET b=1, c=1 WHERE a=2 +/* GTID */ CREATE TEMPORARY TABLE t2 (a INT PRIMARY KEY, b INT) ENGINE=InnoDB +/* GTID */ INSERT INTO t2 VALUES (4,10), (5,20) +/* GTID */ INSERT INTO t1 SELECT a, 2, b FROM t2 +DROP TEMPORARY TABLE `t2` /* generated by server */ +/* GTID */ INSERT INTO t1 VALUES (6, 3, 0) +COMMIT/*!*/; +#200121 15:32:22 server id 1 end_log_pos # GTID #-#-# ddl +/* GTID */ CREATE TEMPORARY TABLE t3 (a INT PRIMARY KEY) ENGINE=InnoDB +#200121 15:32:22 server id 1 end_log_pos # GTID #-#-# ddl +BEGIN +/* GTID */ DELETE FROM t1 WHERE a=5 +/* GTID */ INSERT INTO t3 VALUES (7) +/* GTID */ INSERT INTO t1 SELECT a, 4, 0 FROM t3 +/* GTID */ UPDATE t1 SET c=1 WHERE a=7 +DROP TEMPORARY TABLE `t3` /* generated by server */ +COMMIT/*!*/; +#200121 15:32:22 server id 1 end_log_pos # GTID #-#-# ddl +/* GTID */ CREATE TEMPORARY TABLE t4 (a INT PRIMARY KEY) ENGINE=InnoDB +#200121 15:32:22 server id 1 end_log_pos # GTID #-#-# trans +BEGIN +/* GTID */ INSERT INTO t1 VALUES (8, 5, 0) +COMMIT/*!*/; +#200121 15:32:22 server id 1 end_log_pos # GTID #-#-# ddl +/* GTID */ ALTER TABLE t4 ADD b INT +#200121 15:32:22 server id 1 end_log_pos # GTID #-#-# trans +BEGIN +/* GTID */ INSERT INTO t1 VALUES (9, 5, 1) +COMMIT/*!*/; +#200121 15:32:22 server id 1 end_log_pos # GTID #-#-# trans +BEGIN +/* GTID */ INSERT INTO t1 VALUES (10, 6, 0) +COMMIT/*!*/; +#200121 15:32:22 server id 1 end_log_pos # GTID #-#-# ddl +BEGIN +/* GTID */ CREATE TEMPORARY TABLE t5 (a INT PRIMARY KEY) ENGINE=InnoDB +/* GTID */ INSERT INTO t1 VALUES (11, 7, 0) +COMMIT/*!*/; +#200121 15:32:22 server id 1 end_log_pos # GTID #-#-# ddl +DROP /*!40005 TEMPORARY */ TABLE IF EXISTS `t5` +DROP TABLE t1; diff --git a/mysql-test/suite/binlog/t/binlog_parallel_replication_marks_row.test b/mysql-test/suite/binlog/t/binlog_parallel_replication_marks_row.test new file mode 100644 index 0000000..8289848 --- /dev/null +++ b/mysql-test/suite/binlog/t/binlog_parallel_replication_marks_row.test @@ -0,0 +1,3 @@ +--source include/have_log_bin.inc +--source include/have_binlog_format_row.inc +--source include/binlog_parallel_replication_marks.test diff --git a/mysql-test/suite/binlog/t/binlog_parallel_replication_marks_stm_mix.test b/mysql-test/suite/binlog/t/binlog_parallel_replication_marks_stm_mix.test new file mode 100644 index 0000000..15042b3 --- /dev/null +++ b/mysql-test/suite/binlog/t/binlog_parallel_replication_marks_stm_mix.test @@ -0,0 +1,3 @@ +--source include/have_log_bin.inc +--source include/have_binlog_format_mixed_or_statement.inc +--source include/binlog_parallel_replication_marks.test diff --git a/mysql-test/suite/rpl/r/rpl_parallel_optimistic.result b/mysql-test/suite/rpl/r/rpl_parallel_optimistic.result index 7593b49..2da7830 100644 --- a/mysql-test/suite/rpl/r/rpl_parallel_optimistic.result +++ b/mysql-test/suite/rpl/r/rpl_parallel_optimistic.result @@ -455,6 +455,33 @@ a b include/stop_slave.inc SET GLOBAL debug_dbug= @old_debug; include/start_slave.inc +*** MDEV-8075: DROP TEMPORARY TABLE not marked as ddl, causing optimistic parallel replication to fail *** +include/stop_slave.inc +INSERT INTO t1 VALUES (40, 10); +CREATE TEMPORARY TABLE t1 (a INT PRIMARY KEY) ENGINE=InnoDB; +INSERT INTO t1 VALUES (41); +BEGIN; +INSERT INTO t2 SELECT a, 20 FROM t1; +DROP TEMPORARY TABLE t1; +COMMIT; +INSERT INTO t1 VALUES (42, 10); +include/save_master_gtid.inc +SELECT * FROM t1 WHERE a >= 40 ORDER BY a; +a b +40 10 +42 10 +SELECT * FROM t2 WHERE a >= 40 ORDER BY a; +a b +41 20 +include/start_slave.inc +include/sync_with_master_gtid.inc +SELECT * FROM t1 WHERE a >= 40 ORDER BY a; +a b +40 10 +42 10 +SELECT * FROM t2 WHERE a >= 40 ORDER BY a; +a b +41 20 include/stop_slave.inc SET GLOBAL slave_parallel_mode=@old_parallel_mode; SET GLOBAL slave_parallel_threads=@old_parallel_threads; diff --git a/mysql-test/suite/rpl/t/rpl_parallel_optimistic.test b/mysql-test/suite/rpl/t/rpl_parallel_optimistic.test index e7d4f18..376c8d6 100644 --- a/mysql-test/suite/rpl/t/rpl_parallel_optimistic.test +++ b/mysql-test/suite/rpl/t/rpl_parallel_optimistic.test @@ -429,6 +429,31 @@ SET GLOBAL debug_dbug= @old_debug; --source include/start_slave.inc +--echo *** MDEV-8075: DROP TEMPORARY TABLE not marked as ddl, causing optimistic parallel replication to fail *** + +--connection server_2 +--source include/stop_slave.inc + +--connection server_1 +INSERT INTO t1 VALUES (40, 10); +CREATE TEMPORARY TABLE t1 (a INT PRIMARY KEY) ENGINE=InnoDB; +INSERT INTO t1 VALUES (41); +BEGIN; +INSERT INTO t2 SELECT a, 20 FROM t1; +DROP TEMPORARY TABLE t1; +COMMIT; +INSERT INTO t1 VALUES (42, 10); +--source include/save_master_gtid.inc +SELECT * FROM t1 WHERE a >= 40 ORDER BY a; +SELECT * FROM t2 WHERE a >= 40 ORDER BY a; + +--connection server_2 +--source include/start_slave.inc +--source include/sync_with_master_gtid.inc +SELECT * FROM t1 WHERE a >= 40 ORDER BY a; +SELECT * FROM t2 WHERE a >= 40 ORDER BY a; + + # Clean up. --connection server_2 diff --git a/sql/handler.h b/sql/handler.h index 279bfaf..8faec7f 100644 --- a/sql/handler.h +++ b/sql/handler.h @@ -1444,16 +1444,30 @@ struct THD_TRANS static unsigned int const CREATED_TEMP_TABLE= 0x02; static unsigned int const DROPPED_TEMP_TABLE= 0x04; static unsigned int const DID_WAIT= 0x08; + static unsigned int const DID_DDL= 0x10; void mark_created_temp_table() { DBUG_PRINT("debug", ("mark_created_temp_table")); m_unsafe_rollback_flags|= CREATED_TEMP_TABLE; } + void mark_dropped_temp_table() + { + DBUG_PRINT("debug", ("mark_dropped_temp_table")); + m_unsafe_rollback_flags|= DROPPED_TEMP_TABLE; + } + bool has_created_dropped_temp_table() const { + return + (m_unsafe_rollback_flags & (CREATED_TEMP_TABLE|DROPPED_TEMP_TABLE)) != 0; + } void mark_trans_did_wait() { m_unsafe_rollback_flags|= DID_WAIT; } bool trans_did_wait() const { return (m_unsafe_rollback_flags & DID_WAIT) != 0; } + void mark_trans_did_ddl() { m_unsafe_rollback_flags|= DID_DDL; } + bool trans_did_ddl() const { + return (m_unsafe_rollback_flags & DID_DDL) != 0; + } }; diff --git a/sql/log_event.cc b/sql/log_event.cc index aa34996..98cc4b1 100644 --- a/sql/log_event.cc +++ b/sql/log_event.cc @@ -6405,8 +6405,10 @@ Gtid_log_event::Gtid_log_event(THD *thd_arg, uint64 seq_no_arg, if (thd_arg->transaction.stmt.trans_did_wait() || thd_arg->transaction.all.trans_did_wait()) flags2|= FL_WAITED; - if (sql_command_flags[thd->lex->sql_command] & - (CF_DISALLOW_IN_RO_TRANS | CF_AUTO_COMMIT_TRANS)) + if (thd_arg->transaction.stmt.trans_did_ddl() || + thd_arg->transaction.stmt.has_created_dropped_temp_table() || + thd_arg->transaction.all.trans_did_ddl() || + thd_arg->transaction.all.has_created_dropped_temp_table()) flags2|= FL_DDL; else if (is_transactional) flags2|= FL_TRANSACTIONAL; diff --git a/sql/sql_base.cc b/sql/sql_base.cc index c400598..6ef46ca 100644 --- a/sql/sql_base.cc +++ b/sql/sql_base.cc @@ -1253,6 +1253,7 @@ bool close_temporary_tables(THD *thd) thd->variables.character_set_client= cs_save; thd->get_stmt_da()->set_overwrite_status(true); + thd->transaction.stmt.mark_dropped_temp_table(); if ((error= (mysql_bin_log.write(&qinfo) || error))) { /* diff --git a/sql/sql_class.h b/sql/sql_class.h index a39c8cd..11eab4e 100644 --- a/sql/sql_class.h +++ b/sql/sql_class.h @@ -3953,6 +3953,16 @@ class THD :public Statement, { main_lex.restore_set_statement_var(); } + /* Copy relevant `stmt` transaction flags to `all` transaction. */ + void merge_unsafe_rollback_flags() + { + if (transaction.stmt.modified_non_trans_table) + transaction.all.modified_non_trans_table= TRUE; + transaction.all.m_unsafe_rollback_flags|= + (transaction.stmt.m_unsafe_rollback_flags & + (THD_TRANS::DID_WAIT | THD_TRANS::CREATED_TEMP_TABLE | + THD_TRANS::DROPPED_TEMP_TABLE | THD_TRANS::DID_DDL)); + } }; diff --git a/sql/sql_insert.cc b/sql/sql_insert.cc index 24d75d9..90e68d5 100644 --- a/sql/sql_insert.cc +++ b/sql/sql_insert.cc @@ -4226,6 +4226,14 @@ void select_create::store_values(List<Item> &values) bool select_create::send_eof() { + /* + The routine that writes the statement in the binary log + is in select_insert::send_eof(). For that reason, we + mark the flag at this point. + */ + if (create_info->options & HA_LEX_CREATE_TMP_TABLE) + thd->transaction.stmt.mark_created_temp_table(); + if (select_insert::send_eof()) { abort_result_set(); diff --git a/sql/sql_parse.cc b/sql/sql_parse.cc index 5eca972..142596f 100644 --- a/sql/sql_parse.cc +++ b/sql/sql_parse.cc @@ -2798,6 +2798,7 @@ mysql_execute_command(THD *thd) goto error; } } + thd->transaction.stmt.mark_trans_did_ddl(); } #ifndef DBUG_OFF @@ -6781,8 +6782,7 @@ void THD::reset_for_next_command() if (!thd->in_multi_stmt_transaction_mode()) { thd->variables.option_bits&= ~OPTION_KEEP_LOG; - thd->transaction.all.modified_non_trans_table= FALSE; - thd->transaction.all.m_unsafe_rollback_flags&= ~THD_TRANS::DID_WAIT; + thd->transaction.all.reset(); } DBUG_ASSERT(thd->security_ctx== &thd->main_security_ctx); thd->thread_specific_used= FALSE; diff --git a/sql/sql_table.cc b/sql/sql_table.cc index 1f96793..9abe012 100644 --- a/sql/sql_table.cc +++ b/sql/sql_table.cc @@ -2559,6 +2559,9 @@ int mysql_rm_table_no_locks(THD *thd, TABLE_LIST *tables, bool if_exists, if (non_trans_tmp_table_deleted || trans_tmp_table_deleted || non_tmp_table_deleted) { + if (non_trans_tmp_table_deleted || trans_tmp_table_deleted) + thd->transaction.stmt.mark_dropped_temp_table(); + query_cache_invalidate3(thd, tables, 0); if (!dont_log_query && mysql_bin_log.is_open()) { @@ -5014,6 +5017,9 @@ bool mysql_create_table(THD *thd, TABLE_LIST *create_table, if (thd->is_current_stmt_binlog_format_row() && create_info->tmp_table()) DBUG_RETURN(result); + if (create_info->options & HA_LEX_CREATE_TMP_TABLE) + thd->transaction.stmt.mark_created_temp_table(); + /* Write log if no error or if we already deleted a table */ if (!result || thd->log_current_statement) { @@ -5491,13 +5497,17 @@ bool mysql_create_like_table(THD* thd, TABLE_LIST* table, DBUG_PRINT("info", ("res: %d tmp_table: %d create_info->table: %p", res, create_info->tmp_table(), local_create_info.table)); - if (!res && create_info->tmp_table() && local_create_info.table) + if (create_info->tmp_table()) { - /* - Remember that tmp table creation was logged so that we know if - we should log a delete of it. - */ - local_create_info.table->s->table_creation_was_logged= 1; + thd->transaction.stmt.mark_created_temp_table(); + if (!res && local_create_info.table) + { + /* + Remember that tmp table creation was logged so that we know if + we should log a delete of it. + */ + local_create_info.table->s->table_creation_was_logged= 1; + } } do_logging= TRUE; } diff --git a/sql/transaction.cc b/sql/transaction.cc index f84c4e5..fa6d338 100644 --- a/sql/transaction.cc +++ b/sql/transaction.cc @@ -154,8 +154,7 @@ bool trans_begin(THD *thd, uint flags) The following set should not be needed as the flag should always be 0 when we come here. We should at some point change this to an assert. */ - thd->transaction.all.modified_non_trans_table= FALSE; - thd->transaction.all.m_unsafe_rollback_flags&= ~THD_TRANS::DID_WAIT; + thd->transaction.all.reset(); thd->has_waiter= false; thd->waiting_on_group_commit= false; @@ -251,8 +250,7 @@ bool trans_commit(THD *thd) else (void) RUN_HOOK(transaction, after_commit, (thd, FALSE)); thd->variables.option_bits&= ~(OPTION_BEGIN | OPTION_KEEP_LOG); - thd->transaction.all.modified_non_trans_table= FALSE; - thd->transaction.all.m_unsafe_rollback_flags&= ~THD_TRANS::DID_WAIT; + thd->transaction.all.reset(); thd->lex->start_transaction_opt= 0; DBUG_RETURN(MY_TEST(res)); @@ -299,8 +297,7 @@ bool trans_commit_implicit(THD *thd) } thd->variables.option_bits&= ~(OPTION_BEGIN | OPTION_KEEP_LOG); - thd->transaction.all.modified_non_trans_table= FALSE; - thd->transaction.all.m_unsafe_rollback_flags&= ~THD_TRANS::DID_WAIT; + thd->transaction.all.reset(); /* Upon implicit commit, reset the current transaction @@ -345,8 +342,7 @@ bool trans_rollback(THD *thd) thd->variables.option_bits&= ~(OPTION_BEGIN | OPTION_KEEP_LOG); /* Reset the binlog transaction marker */ thd->variables.option_bits&= ~OPTION_GTID_BEGIN; - thd->transaction.all.modified_non_trans_table= FALSE; - thd->transaction.all.m_unsafe_rollback_flags&= ~THD_TRANS::DID_WAIT; + thd->transaction.all.reset(); thd->lex->start_transaction_opt= 0; DBUG_RETURN(MY_TEST(res)); @@ -390,8 +386,7 @@ bool trans_rollback_implicit(THD *thd) preserve backward compatibility. */ thd->variables.option_bits&= ~(OPTION_KEEP_LOG); - thd->transaction.all.modified_non_trans_table= false; - thd->transaction.all.m_unsafe_rollback_flags&= ~THD_TRANS::DID_WAIT; + thd->transaction.all.reset(); /* Rollback should clear transaction_rollback_request flag. */ DBUG_ASSERT(! thd->transaction_rollback_request); @@ -427,6 +422,8 @@ bool trans_commit_stmt(THD *thd) */ DBUG_ASSERT(! thd->in_sub_stmt); + thd->merge_unsafe_rollback_flags(); + if (thd->transaction.stmt.ha_list) { if (WSREP_ON) @@ -481,6 +478,8 @@ bool trans_rollback_stmt(THD *thd) */ DBUG_ASSERT(! thd->in_sub_stmt); + thd->merge_unsafe_rollback_flags(); + if (thd->transaction.stmt.ha_list) { if (WSREP_ON) @@ -913,8 +912,7 @@ bool trans_xa_commit(THD *thd) } thd->variables.option_bits&= ~(OPTION_BEGIN | OPTION_KEEP_LOG); - thd->transaction.all.modified_non_trans_table= FALSE; - thd->transaction.all.m_unsafe_rollback_flags&= ~THD_TRANS::DID_WAIT; + thd->transaction.all.reset(); thd->server_status&= ~(SERVER_STATUS_IN_TRANS | SERVER_STATUS_IN_TRANS_READONLY); DBUG_PRINT("info", ("clearing SERVER_STATUS_IN_TRANS")); @@ -969,8 +967,7 @@ bool trans_xa_rollback(THD *thd) res= xa_trans_force_rollback(thd); thd->variables.option_bits&= ~(OPTION_BEGIN | OPTION_KEEP_LOG); - thd->transaction.all.modified_non_trans_table= FALSE; - thd->transaction.all.m_unsafe_rollback_flags&= ~THD_TRANS::DID_WAIT; + thd->transaction.all.reset(); thd->server_status&= ~(SERVER_STATUS_IN_TRANS | SERVER_STATUS_IN_TRANS_READONLY); DBUG_PRINT("info", ("clearing SERVER_STATUS_IN_TRANS")); _______________________________________________ commits mailing list commits(a)mariadb.org https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits

1 0

[Maria-developers] Step#5: MDEV-7950, , Item_func::type() takes 0.26% in OLTP RO
by Alexander Barkov 14 May '15

14 May '15

Hi Sergey, please review the next step for MDEV-7950. This step changes the function remove_eq_conds() into a method in Item. It removes 6 virtual calls for Item_func::type(), and adds only 2 virtual calls for Item***::remove_eq_conds(). Thanks.

1 0

Re: [Maria-developers] 8a0f331: MDEV-7943 - pthread_getspecific() takes 0.76% in OLTP RO
by Sergei Golubchik 13 May '15

13 May '15

Hi, Sergey! On Apr 22, svoj(a)mariadb.org wrote: > revision-id: 8a0f3310275f7b4fa445f907140f677910e18999 > parent(s): 7feee74dd30c96bd50d1c90e4ce3b06a656b17a5 > committer: Sergey Vojtovich > branch nick: mariadb > timestamp: 2015-04-22 14:18:51 +0400 > message: > > MDEV-7943 - pthread_getspecific() takes 0.76% in OLTP RO > > Avoid calling current_thd from thd_kill_level(). This reduces number of > pthread_getspecific() calls from 776 to 354. > > Also thd_kill_level(NULL) is not permitted anymore: this saves one condition. > > --- > plugin/semisync/semisync_master.cc | 4 ++-- > sql/sql_class.cc | 20 +++++++++----------- > 2 files changed, 11 insertions(+), 13 deletions(-) > > diff --git a/plugin/semisync/semisync_master.cc b/plugin/semisync/semisync_master.cc > index c88c162..b1f7fbd 100644 > --- a/plugin/semisync/semisync_master.cc > +++ b/plugin/semisync/semisync_master.cc > @@ -635,7 +635,7 @@ int ReplSemiSyncMaster::commitTrx(const char* trx_wait_binlog_name, > (int)is_on()); > } > > - while (is_on() && !thd_killed(NULL)) > + while (is_on() && !thd_killed(current_thd)) thd_killed() is a function of the kill_statement service (see include/mysql/service_kill_statement.h). It is created for plugins to use. But current_thd is not. Plugins generally have no access to it. > { > if (reply_file_name_inited_) > { > diff --git a/sql/sql_class.cc b/sql/sql_class.cc > index f273974..d32b14d 100644 > --- a/sql/sql_class.cc > +++ b/sql/sql_class.cc > @@ -4208,20 +4208,18 @@ void THD::restore_backup_open_tables_state(Open_tables_backup *backup) > */ > extern "C" enum thd_kill_levels thd_kill_level(const MYSQL_THD thd) > { > - THD* current= current_thd; > - > - if (!thd) > - thd= current; > - > - if (thd == current) I wonder if there's a cheaper way to check "if THD is current_thd". E.g. THD stores the "end of stack" value. If it'll also store "beginning of stack", one can easily check whether a local variable is within these limits, which would mean the THD is current. > - { > - Apc_target *apc_target= (Apc_target*)&thd->apc_target; > - if (apc_target->have_apc_requests()) > - apc_target->process_apc_requests(); > - } > + DBUG_ASSERT(thd); > > if (likely(thd->killed == NOT_KILLED)) > + { > + Apc_target *apc_target= (Apc_target*) &thd->apc_target; > + if (unlikely(apc_target->have_apc_requests())) > + { > + if (thd == current_thd) > + apc_target->process_apc_requests(); > + } > return THD_IS_NOT_KILLED; > + } Why did you put process_apc_requests() under thd->killed == NOT_KILLED ? > return thd->killed & KILL_HARD_BIT ? THD_ABORT_ASAP : THD_ABORT_SOFTLY; > } Regards, Sergei

2 4

[Maria-developers] Step#4: MDEV-7950 Item_func::type() takes 0.26% in OLTP RO
by Alexander Barkov 13 May '15

13 May '15

Hi Sergey, Please review a patch for the next step for MDEV-7950 (one small thing at a time, to avoid huge unclear patches) Thanks.

2 4

Re: [Maria-developers] 7feee74: MDEV-7943 - pthread_getspecific() takes 0.76% in OLTP RO
by Sergei Golubchik 12 May '15

12 May '15

Hi, Sergey! On Apr 22, svoj(a)mariadb.org wrote: > revision-id: 7feee74dd30c96bd50d1c90e4ce3b06a656b17a5 > parent(s): 696188fe2f0756a98914df5b95ba302f7e5c55df > committer: Sergey Vojtovich > branch nick: mariadb > timestamp: 2015-04-22 13:29:56 +0400 > message: > > MDEV-7943 - pthread_getspecific() takes 0.76% in OLTP RO > > Added THD argument to select_result and all derivative classes. This > reduces number of pthread_getspecific calls from 796 to 776 per OLTP > RO transaction. Lots of changes for a little gain :) Ok to push. An idea. Create a variant of Sql_alloc (say, Sql_alloc_explicit, a parent of Sql_alloc) that does not use current_thd but requires MEM_ROOT to be specified explicitly. And use it for classes where you want it. Regards, Sergei

2 1

[Maria-developers] MariaDB Debian deprecation policy
by Daniel Bartholomew 11 May '15

11 May '15

Looking for input on this. On the MariaDB Deprecation Policy page we list planned deprecation dates for most of the platforms we build on. https://mariadb.com/kb/en/mariadb/deprecation-policy/ The various Debian releases we build for are listed, but there are no deprecation dates. According to https://www.debian.org/releases/ with the release of Debian 8 "Jessie" both Squeeze and Wheezy are now "obsolete". Should we deprecate both of them and stop providing builds? Or should we just deprecate Squeeze? Thanks. -- Daniel Bartholomew, MariaDB Release Manager MariaDB | http://mariadb.com

3 3

Re: [Maria-developers] 0520a00: MDEV-7964 - delete_dynamic() takes 0.12% in OLTP RO
by Sergei Golubchik 10 May '15

10 May '15

Hi, Sergey! On Apr 16, svoj(a)mariadb.org wrote: > revision-id: 0520a009c7139c940ab92345054bf0dba7808b11 > parent(s): eb059cd41e5553597f81fd90ad55c07d76d317ce > committer: Sergey Vojtovich > branch nick: mariadb > timestamp: 2015-04-16 18:38:26 +0400 > message: > > MDEV-7964 - delete_dynamic() takes 0.12% in OLTP RO ... > diff --git a/sql/sql_base.cc b/sql/sql_base.cc > index 0e9f668..d36de7f 100644 > --- a/sql/sql_base.cc > +++ b/sql/sql_base.cc > @@ -4240,9 +4233,17 @@ thr_lock_type read_lock_type_for_table(THD *thd, > DBUG_RETURN(true); > } > > - if (! (flags & MYSQL_OPEN_SKIP_SCOPED_MDL_LOCK) && > - schema_set.insert(table)) > - DBUG_RETURN(TRUE); > + /* Scoped locks: Take intention exclusive locks on all involved schemas. */ > + if (!(flags & MYSQL_OPEN_SKIP_SCOPED_MDL_LOCK)) > + { > + MDL_request *schema_request= new (thd->mem_root) MDL_request; > + if (schema_request == NULL) > + DBUG_RETURN(TRUE); > + schema_request->init(MDL_key::SCHEMA, table->db, "", > + MDL_INTENTION_EXCLUSIVE, > + MDL_TRANSACTION); > + mdl_requests.push_front(schema_request); > + } As far as I understand, the Hash_set was used here to make only one MDL request per schema. Now if you'll have 10 tables all in one schema, you'll make push 10 MDL requests instead of one. Which may be fine, if MDL subsystem places only one lock in this case (iirc, it does). But what is cheaper - collapse identical schemas into one mdl request here (with Hash_set) or let mdl sort it out later? > mdl_requests.push_front(&table->mdl_request); > } Regards, Sergei

2 4

Re: [Maria-developers] [Commits] 845edb6: MDEV-7067: Server outputs Galera (WSREP) information, even if Galera is disabled
by Sergei Golubchik 09 May '15

09 May '15

Hi, Nirbhay! On Apr 08, Nirbhay Choubey wrote: > revision-id: 845edb6259559bed39eb7365860aa5d588990775 > parent(s): 3674c363a7e77e534c3e0d28659dc614c53fbcbc > committer: Nirbhay Choubey > branch nick: 10.1-wsrep > timestamp: 2015-04-08 16:49:57 -0400 > message: > > MDEV-7067: Server outputs Galera (WSREP) information, even if Galera is disabled > > - mysqld_safe: Skip wsrep position recovery if wsrep_on variable > (mandatory since 10.1) is not specified ok > - Do not load "dummy" wsrep provider if wsrep_on is not ON see below > - Remove "-wsrep" from server version ok I'd also move "wsrep_25.10.r4144" from version_comment to a separate variable, like, wsrep_version. > diff --git a/sql/wsrep_var.cc b/sql/wsrep_var.cc > index ca921b7..1cd0a54 100644 > --- a/sql/wsrep_var.cc > +++ b/sql/wsrep_var.cc > @@ -295,6 +295,8 @@ bool wsrep_provider_options_check(sys_var *self, THD* thd, set_var* var) > > bool wsrep_provider_options_update(sys_var *self, THD* thd, enum_var_type type) > { > + if (!wsrep) return false; > + What about that comment by Alexey Yurchenko? The one where he says that checking for wsrep being not NULL inside wsrep functions is never needed, because wsrep functions should be never even called if wsrep is not enabled. Regards, Sergei

2 1

Re: [Maria-developers] ceae9ba: MDEV-7999 - PROFILING routines take 0.2% when profiling disabled
by Sergei Golubchik 09 May '15

09 May '15

Hi, Sergey! On Apr 15, svoj(a)mariadb.org wrote: > revision-id: ceae9ba7c8872d6558b331ef1ed1ebef42ecdc19 > parent(s): 367f043248112394bd1dc05c72d2844d217a4d55 > committer: Sergey Vojtovich > branch nick: mariadb > timestamp: 2015-04-15 18:12:23 +0400 > message: > > MDEV-7999 - PROFILING routines take 0.2% when profiling disabled Ok to push. One question: > diff --git a/sql/sql_profile.cc b/sql/sql_profile.cc > index 2eaaea4..9fa4d64 100644 > --- a/sql/sql_profile.cc > +++ b/sql/sql_profile.cc > @@ -720,4 +665,10 @@ int PROFILING::fill_statistics_info(THD *thd_arg, TABLE_LIST *tables, Item *cond > DBUG_RETURN(0); > } > + > + > +void PROFILING::reset() > +{ > + enabled= thd->variables.option_bits & OPTION_PROFILING; > +} Why is this not inlined? Regards, Sergei

2 1

Re: [Maria-developers] eb059cd: MDEV-8001 - mysql_reset_thd_for_next_command() takes 0.04% in OLTP RO
by Sergei Golubchik 09 May '15

09 May '15

Hi, Sergey! On Apr 15, svoj(a)mariadb.org wrote: > revision-id: eb059cd41e5553597f81fd90ad55c07d76d317ce > parent(s): ceae9ba7c8872d6558b331ef1ed1ebef42ecdc19 > committer: Sergey Vojtovich > branch nick: mariadb > timestamp: 2015-04-15 18:32:34 +0400 > message: > > MDEV-8001 - mysql_reset_thd_for_next_command() takes 0.04% in OLTP RO ok to push > Removed mysql_reset_thd_for_next_command(). Call THD::reset_for_next_command() > directly instead. > > mysql_reset_thd_for_next_command() overhead dropped 0.04% -> out of radar. > THD::reset_for_next_command() overhead didn't increase. Sure, why would it increase? Regards, Sergei

2 1

[Maria-developers] what could be better than %1 OLTP test gains?
by Daniel Black 09 May '15

09 May '15

Appreciate all the effort recently to save <1% on OLTP tests. MDEV-6735 / MDEV-7786 has a 3 line patch in progress that saves a 6700% (0.1 seconds to 11 min 10.44 sec) by changing optimizer decision based on something more meaningful than an absolute number of rows on one side of a join (a specific tuning for some test at some time?). Sure it might not make someone's blog (though it should), but it will make it possible to move one customer of mine beyond 5.5.32 which is near where a regression happened and for all those Magento users who've chosen MariaDB where performance has just broken to unusable when 100 categories was exceeded. -- -- Daniel Black, Engineer @ Open Query (http://openquery.com.au) Remote expertise & maintenance for MySQL/MariaDB server environments.

1 0

Re: [Maria-developers] 696188f: MDEV-7951 - sql_alloc() takes 0.25% in OLTP RO
by Sergei Golubchik 08 May '15

08 May '15

Hi, Sergey! On Apr 17, svoj(a)mariadb.org wrote: > revision-id: 696188fe2f0756a98914df5b95ba302f7e5c55df > parent(s): 0520a009c7139c940ab92345054bf0dba7808b11 > committer: Sergey Vojtovich > branch nick: mariadb > timestamp: 2015-04-17 14:30:15 +0400 > message: > > MDEV-7951 - sql_alloc() takes 0.25% in OLTP RO > > sql_alloc() has additional costs compared to direct mem_root allocation: > - function call: it is defined in a separate translation unit and can't be > inlined > - it needs to call pthread_getspecific() to get THD::mem_root > > It is called dozens of times implicitely at least by: > - List<>::push_back() > - List<>::push_front() > - new (for Sql_alloc derived classes) > - sql_memdup() > > Replaced lots of implicit sql_alloc() calls with direct mem_root allocation, > passing through THD pointer whenever it is needed. > > Number of sql_alloc() calls reduced 345 -> 41 per OLTP RO transaction. > pthread_getspecific() overhead dropped 0.76 -> 0.59 > sql_alloc() overhed dropped 0.25 -> 0.06 ok to push Regards, Sergei

1 0

Re: [Maria-developers] d116e02: MDEV-7945 - THD::enter_stage() takes 0.48% in OLTP RO
by Sergei Golubchik 08 May '15

08 May '15

Hi, Sergey! On Apr 14, svoj(a)mariadb.org wrote: > revision-id: d116e028a9545af8bd9eeb03f2ce0e56a9c455f9 > parent(s): 201c985be56f375677df9275d38f18a20cf41029 > committer: Sergey Vojtovich > branch nick: mariadb > timestamp: 2015-04-14 17:52:08 +0400 > message: > > MDEV-7945 - THD::enter_stage() takes 0.48% in OLTP RO > > THD::enter_stage() optimizations: > - stage backup code moved to THD::backup_stage(), saves one condition > - moved check for "new_stage" out to callers that actually need it > - remnants of enter_stage() moved to sql_class.h so it can be inlined > > THD::enter_stage() overhead dropped 0.48% -> 0.07%. > > PROFILING::status_change() optimizations: > - no need to check "status_arg": it is checked by caller anyway > - no need to check "enabled": !enabled && current is impossible > - remnants of status_change() moved to sql_profile.h so it can be inlined > > PROFILING::status_change() overhead dropped 0.1% -> out of radar. ok to push Regards, Sergei

1 0

Re: [Maria-developers] 334e7c9: MDEV-7895 - HANDLER READ doesn't upgrade metadata lock from S to SR
by Sergei Golubchik 08 May '15

08 May '15

Hi, Sergey! On May 07, Sergey Vojtovich wrote: > revision-id: 334e7c9bccbc6892efbb35f5930f80ccf1d770a8 > parent(s): cbc5157feb9801310e458f7ed10983ad478c881e > committer: Sergey Vojtovich > branch nick: mariadb > timestamp: 2015-04-01 14:07:02 +0400 > message: > > MDEV-7895 - HANDLER READ doesn't upgrade metadata lock from S to SR > > Change code for HANDLER READ statements to upgrade S metadata lock to > SR metadata lock for the duration of read. This allows us properly > isolate HANDLER READ from LOCK TABLES WRITE and makes metadata locking > for these statements consistent with locking for other DML. > > HANDLER-related tests had to be adjusted to take into account that > HANDLER READ will wait for and acquire SR lock. code-wise it's ok, but I'm not sure I like the resulting behavior. Here, the very first test: con1: handler t1 open; con2: drop table t1; -- waits con1: handler t1 read; -- deadlock as a user I'd expect that after handler open, I can use the handler. I don't care about locks, but supposedly handler open should take them all or handler read should solve it somehow, why I, a user, should bother? So, indeed, one option would be to acquire SR lock on handler open. Also, note the comment for MDL_SHARED: /* A shared metadata lock. To be used in cases when we are interested in object metadata only and there is no intention to access object data (e.g. for stored routines or during preparing prepared statements). We also mis-use this type of lock for open HANDLERs, since lock acquired by this statement has to be compatible with lock acquired by LOCK TABLES ... WRITE statement, i.e. SNRW (We can't get by by acquiring S lock at HANDLER ... OPEN time and upgrading it to SR lock for HANDLER ... READ as it doesn't solve problem with need to abort DML statements which wait on table level lock while having open HANDLER in the same connection). To avoid deadlock which may occur when SNRW lock is being upgraded to X lock for table on which there is an active S lock which is owned by thread which waits in its turn for table-level lock owned by thread performing upgrade we have to use thr_abort_locks_for_thread() facility in such situation. This problem does not arise for locks on stored routines as we don't use SNRW locks for them. It also does not arise when S locks are used during PREPARE calls as table-level locks are not acquired in this case. */ MDL_SHARED, Which begs more questions: - why open handler lock has to be compatible with SNRW? - why using S and upgrading to SR "didn't solve the problem" and now it suddenly does? Regards, Sergei

2 1

[Maria-developers] Windows line endings in source files (Re: [Maria-discuss] MariaDB 10.0.18 now available)
by Otto Kekäläinen 08 May '15

08 May '15

While preparing the Debian updates I noticed git complaning: The file will have its original line endings in your working directory. warning: CRLF will be replaced by LF in storage/spider/mysql-test/spider/t/have_partition.inc. The file will have its original line endings in your working directory. warning: CRLF will be replaced by LF in storage/spider/mysql-test/spider/t/have_trigger.inc. The file will have its original line endings in your working directory. warning: CRLF will be replaced by LF in storage/spider/mysql-test/spider/t/spider3_fixes.test. The file will have its original line endings in your working directory. warning: CRLF will be replaced by LF in storage/spider/mysql-test/spider/t/spider3_fixes_part.test. Turns out a lot of source files in this release has Windows line endings: mariadb-10.0/mysql-test/r/func_regexp_pcre.result mariadb-10.0/mysql-test/r/loadxml.result mariadb-10.0/mysql-test/r/mysql_binary_mode.result mariadb-10.0/mysql-test/r/perror-win.result mariadb-10.0/mysql-test/std_data/loaddata7.dat mariadb-10.0/mysql-test/std_data/loadxml2.dat mariadb-10.0/packaging/WiX/CPackWixConfig.cmake mariadb-10.0/packaging/WiX/create_msi.cmake.in mariadb-10.0/packaging/WiX/custom_ui.wxs mariadb-10.0/packaging/WiX/extra.wxs.in mariadb-10.0/packaging/WiX/mysql_server.wxs.in mariadb-10.0/pcre/RunTest.bat mariadb-10.0/pcre/testdata/greppatN4 mariadb-10.0/sql/message.rc mariadb-10.0/sql/winservice.h mariadb-10.0/storage/connect/colblk.cpp mariadb-10.0/storage/connect/mysql-test/connect/my.cnf mariadb-10.0/storage/connect/mysql-test/connect/std_data/biblio.json mariadb-10.0/storage/connect/mysql-test/connect/std_data/bookstore.xml mariadb-10.0/storage/connect/mysql-test/connect/std_data/boyswin.txt mariadb-10.0/storage/connect/mysql-test/connect/std_data/emp.txt mariadb-10.0/storage/connect/mysql-test/connect/std_data/expense.json mariadb-10.0/storage/connect/mysql-test/connect/std_data/expenses.txt mariadb-10.0/storage/connect/mysql-test/connect/std_data/mulexp3.json mariadb-10.0/storage/connect/mysql-test/connect/std_data/mulexp4.json mariadb-10.0/storage/connect/mysql-test/connect/std_data/mulexp5.json mariadb-10.0/storage/connect/mysql-test/connect/std_data/sexe.csv mariadb-10.0/storage/connect/mysql-test/connect/std_data/sitmat.csv mariadb-10.0/storage/connect/mysql-test/connect/t/alter.test mariadb-10.0/storage/connect/mysql-test/connect/t/alter_xml.test mariadb-10.0/storage/connect/mysql-test/connect/t/bin.test mariadb-10.0/storage/connect/mysql-test/connect/t/datest.test mariadb-10.0/storage/connect/mysql-test/connect/t/fmt.test mariadb-10.0/storage/connect/mysql-test/connect/t/general.test mariadb-10.0/storage/connect/mysql-test/connect/t/json.test mariadb-10.0/storage/connect/mysql-test/connect/t/json_udf.test mariadb-10.0/storage/connect/mysql-test/connect/t/mrr.test mariadb-10.0/storage/connect/mysql-test/connect/t/mul.test mariadb-10.0/storage/connect/mysql-test/connect/t/myconn.inc mariadb-10.0/storage/connect/mysql-test/connect/t/myconn_cleanup.inc mariadb-10.0/storage/connect/mysql-test/connect/t/mysql.test mariadb-10.0/storage/connect/mysql-test/connect/t/mysql_discovery.test mariadb-10.0/storage/connect/mysql-test/connect/t/mysql_exec.test mariadb-10.0/storage/connect/mysql-test/connect/t/mysql_grant.test mariadb-10.0/storage/connect/mysql-test/connect/t/mysql_new.test mariadb-10.0/storage/connect/mysql-test/connect/t/null.test mariadb-10.0/storage/connect/mysql-test/connect/t/occur.test mariadb-10.0/storage/connect/mysql-test/connect/t/odbc_sqlite3.test mariadb-10.0/storage/connect/mysql-test/connect/t/part_file.test mariadb-10.0/storage/connect/mysql-test/connect/t/part_table.test mariadb-10.0/storage/connect/mysql-test/connect/t/pivot.test mariadb-10.0/storage/connect/mysql-test/connect/t/tbl.test mariadb-10.0/storage/connect/mysql-test/connect/t/unsigned.test mariadb-10.0/storage/connect/mysql-test/connect/t/upd.test mariadb-10.0/storage/connect/mysql-test/connect/t/updelx.test mariadb-10.0/storage/connect/mysql-test/connect/t/updelx2.test mariadb-10.0/storage/connect/mysql-test/connect/t/xcol.test mariadb-10.0/storage/connect/mysql-test/connect/t/xml_mdev5261.test mariadb-10.0/storage/connect/rcmsg.c mariadb-10.0/storage/connect/tabsys.h mariadb-10.0/storage/federatedx/README.windows mariadb-10.0/storage/spider/mysql-test/spider/bg/include/ha_deinit_child3_1.inc mariadb-10.0/storage/spider/mysql-test/spider/bg/include/ha_deinit_child3_2.inc mariadb-10.0/storage/spider/mysql-test/spider/bg/include/ha_deinit_child3_3.inc mariadb-10.0/storage/spider/mysql-test/spider/bg/include/ha_init_child2_1.inc mariadb-10.0/storage/spider/mysql-test/spider/bg/include/ha_init_child2_2.inc mariadb-10.0/storage/spider/mysql-test/spider/bg/include/ha_init_child2_3.inc mariadb-10.0/storage/spider/mysql-test/spider/bg/include/ha_init_child3_1.inc mariadb-10.0/storage/spider/mysql-test/spider/bg/include/ha_init_child3_2.inc mariadb-10.0/storage/spider/mysql-test/spider/bg/include/ha_init_child3_3.inc mariadb-10.0/storage/spider/mysql-test/spider/bg/include/hs_init_child2_1.inc mariadb-10.0/storage/spider/mysql-test/spider/bg/include/hs_init_child2_2.inc mariadb-10.0/storage/spider/mysql-test/spider/bg/include/init_child2_1.inc mariadb-10.0/storage/spider/mysql-test/spider/bg/include/init_child2_2.inc mariadb-10.0/storage/spider/mysql-test/spider/bg/include/init_child2_3.inc mariadb-10.0/storage/spider/mysql-test/spider/bg/include/init_slave1_1.inc mariadb-10.0/storage/spider/mysql-test/spider/bg/t/direct_aggregate.test mariadb-10.0/storage/spider/mysql-test/spider/bg/t/direct_aggregate_part.test mariadb-10.0/storage/spider/mysql-test/spider/bg/t/have_partition.inc mariadb-10.0/storage/spider/mysql-test/spider/bg/t/have_trigger.inc mariadb-10.0/storage/spider/mysql-test/spider/bg/t/spider3_fixes.test mariadb-10.0/storage/spider/mysql-test/spider/bg/t/spider3_fixes_part.test mariadb-10.0/storage/spider/mysql-test/spider/handler/include/ha_deinit_child3_1.inc mariadb-10.0/storage/spider/mysql-test/spider/handler/include/ha_deinit_child3_2.inc mariadb-10.0/storage/spider/mysql-test/spider/handler/include/ha_deinit_child3_3.inc mariadb-10.0/storage/spider/mysql-test/spider/handler/include/ha_init_child2_1.inc mariadb-10.0/storage/spider/mysql-test/spider/handler/include/ha_init_child2_2.inc mariadb-10.0/storage/spider/mysql-test/spider/handler/include/ha_init_child2_3.inc mariadb-10.0/storage/spider/mysql-test/spider/handler/include/ha_init_child3_1.inc mariadb-10.0/storage/spider/mysql-test/spider/handler/include/ha_init_child3_2.inc mariadb-10.0/storage/spider/mysql-test/spider/handler/include/ha_init_child3_3.inc mariadb-10.0/storage/spider/mysql-test/spider/handler/include/hs_init_child2_1.inc mariadb-10.0/storage/spider/mysql-test/spider/handler/include/hs_init_child2_2.inc mariadb-10.0/storage/spider/mysql-test/spider/handler/include/init_child2_1.inc mariadb-10.0/storage/spider/mysql-test/spider/handler/include/init_child2_2.inc mariadb-10.0/storage/spider/mysql-test/spider/handler/include/init_child2_3.inc mariadb-10.0/storage/spider/mysql-test/spider/handler/include/init_slave1_1.inc mariadb-10.0/storage/spider/mysql-test/spider/handler/t/direct_aggregate.test mariadb-10.0/storage/spider/mysql-test/spider/handler/t/direct_aggregate_part.test mariadb-10.0/storage/spider/mysql-test/spider/handler/t/have_partition.inc mariadb-10.0/storage/spider/mysql-test/spider/handler/t/have_trigger.inc mariadb-10.0/storage/spider/mysql-test/spider/handler/t/spider3_fixes.test mariadb-10.0/storage/spider/mysql-test/spider/handler/t/spider3_fixes_part.test mariadb-10.0/storage/spider/mysql-test/spider/include/deinit_child2_1.inc mariadb-10.0/storage/spider/mysql-test/spider/include/deinit_child2_2.inc mariadb-10.0/storage/spider/mysql-test/spider/include/deinit_child2_3.inc mariadb-10.0/storage/spider/mysql-test/spider/include/deinit_child3_1.inc mariadb-10.0/storage/spider/mysql-test/spider/include/deinit_child3_2.inc mariadb-10.0/storage/spider/mysql-test/spider/include/deinit_child3_3.inc mariadb-10.0/storage/spider/mysql-test/spider/include/deinit_engine.inc mariadb-10.0/storage/spider/mysql-test/spider/include/ha_deinit_child3_1.inc mariadb-10.0/storage/spider/mysql-test/spider/include/ha_deinit_child3_2.inc mariadb-10.0/storage/spider/mysql-test/spider/include/ha_deinit_child3_3.inc mariadb-10.0/storage/spider/mysql-test/spider/include/ha_init_child2_1.inc mariadb-10.0/storage/spider/mysql-test/spider/include/ha_init_child2_2.inc mariadb-10.0/storage/spider/mysql-test/spider/include/ha_init_child2_3.inc mariadb-10.0/storage/spider/mysql-test/spider/include/ha_init_child3_1.inc mariadb-10.0/storage/spider/mysql-test/spider/include/ha_init_child3_2.inc mariadb-10.0/storage/spider/mysql-test/spider/include/ha_init_child3_3.inc mariadb-10.0/storage/spider/mysql-test/spider/include/hs_init_child2_1.inc mariadb-10.0/storage/spider/mysql-test/spider/include/hs_init_child2_2.inc mariadb-10.0/storage/spider/mysql-test/spider/include/init_child2_1.inc mariadb-10.0/storage/spider/mysql-test/spider/include/init_child2_2.inc mariadb-10.0/storage/spider/mysql-test/spider/include/init_child2_3.inc mariadb-10.0/storage/spider/mysql-test/spider/include/init_child3_1.inc mariadb-10.0/storage/spider/mysql-test/spider/include/init_child3_2.inc mariadb-10.0/storage/spider/mysql-test/spider/include/init_child3_3.inc mariadb-10.0/storage/spider/mysql-test/spider/include/init_slave1_1.inc mariadb-10.0/storage/spider/mysql-test/spider/oracle/include/deinit_child3_1.inc mariadb-10.0/storage/spider/mysql-test/spider/oracle/include/deinit_child3_2.inc mariadb-10.0/storage/spider/mysql-test/spider/oracle/include/deinit_child3_3.inc mariadb-10.0/storage/spider/mysql-test/spider/oracle/include/ha_deinit_child3_1.inc mariadb-10.0/storage/spider/mysql-test/spider/oracle/include/ha_deinit_child3_2.inc mariadb-10.0/storage/spider/mysql-test/spider/oracle/include/ha_deinit_child3_3.inc mariadb-10.0/storage/spider/mysql-test/spider/oracle/include/ha_init_child2_1.inc mariadb-10.0/storage/spider/mysql-test/spider/oracle/include/ha_init_child2_2.inc mariadb-10.0/storage/spider/mysql-test/spider/oracle/include/ha_init_child2_3.inc mariadb-10.0/storage/spider/mysql-test/spider/oracle/include/ha_init_child3_1.inc mariadb-10.0/storage/spider/mysql-test/spider/oracle/include/ha_init_child3_2.inc mariadb-10.0/storage/spider/mysql-test/spider/oracle/include/ha_init_child3_3.inc mariadb-10.0/storage/spider/mysql-test/spider/oracle/include/hs_init_child2_1.inc mariadb-10.0/storage/spider/mysql-test/spider/oracle/include/hs_init_child2_2.inc mariadb-10.0/storage/spider/mysql-test/spider/oracle/include/init_child2_1.inc mariadb-10.0/storage/spider/mysql-test/spider/oracle/include/init_child2_2.inc mariadb-10.0/storage/spider/mysql-test/spider/oracle/include/init_child2_3.inc mariadb-10.0/storage/spider/mysql-test/spider/oracle/include/init_child3_1.inc mariadb-10.0/storage/spider/mysql-test/spider/oracle/include/init_child3_2.inc mariadb-10.0/storage/spider/mysql-test/spider/oracle/include/init_child3_3.inc mariadb-10.0/storage/spider/mysql-test/spider/oracle/include/init_slave1_1.inc mariadb-10.0/storage/spider/mysql-test/spider/oracle/t/direct_aggregate.test mariadb-10.0/storage/spider/mysql-test/spider/oracle/t/direct_aggregate_part.test mariadb-10.0/storage/spider/mysql-test/spider/oracle/t/have_partition.inc mariadb-10.0/storage/spider/mysql-test/spider/oracle/t/have_trigger.inc mariadb-10.0/storage/spider/mysql-test/spider/oracle/t/spider3_fixes.test mariadb-10.0/storage/spider/mysql-test/spider/oracle/t/spider3_fixes_part.test mariadb-10.0/storage/spider/mysql-test/spider/oracle2/include/deinit_child3_1.inc mariadb-10.0/storage/spider/mysql-test/spider/oracle2/include/deinit_child3_2.inc mariadb-10.0/storage/spider/mysql-test/spider/oracle2/include/deinit_child3_3.inc mariadb-10.0/storage/spider/mysql-test/spider/oracle2/include/ha_deinit_child3_1.inc mariadb-10.0/storage/spider/mysql-test/spider/oracle2/include/ha_deinit_child3_2.inc mariadb-10.0/storage/spider/mysql-test/spider/oracle2/include/ha_deinit_child3_3.inc mariadb-10.0/storage/spider/mysql-test/spider/oracle2/include/ha_init_child2_1.inc mariadb-10.0/storage/spider/mysql-test/spider/oracle2/include/ha_init_child2_2.inc mariadb-10.0/storage/spider/mysql-test/spider/oracle2/include/ha_init_child2_3.inc mariadb-10.0/storage/spider/mysql-test/spider/oracle2/include/ha_init_child3_1.inc mariadb-10.0/storage/spider/mysql-test/spider/oracle2/include/ha_init_child3_2.inc mariadb-10.0/storage/spider/mysql-test/spider/oracle2/include/ha_init_child3_3.inc mariadb-10.0/storage/spider/mysql-test/spider/oracle2/include/hs_init_child2_1.inc mariadb-10.0/storage/spider/mysql-test/spider/oracle2/include/hs_init_child2_2.inc mariadb-10.0/storage/spider/mysql-test/spider/oracle2/include/init_child2_1.inc mariadb-10.0/storage/spider/mysql-test/spider/oracle2/include/init_child2_2.inc mariadb-10.0/storage/spider/mysql-test/spider/oracle2/include/init_child2_3.inc mariadb-10.0/storage/spider/mysql-test/spider/oracle2/include/init_child3_1.inc mariadb-10.0/storage/spider/mysql-test/spider/oracle2/include/init_child3_2.inc mariadb-10.0/storage/spider/mysql-test/spider/oracle2/include/init_child3_3.inc mariadb-10.0/storage/spider/mysql-test/spider/oracle2/include/init_slave1_1.inc mariadb-10.0/storage/spider/mysql-test/spider/oracle2/t/direct_aggregate.test mariadb-10.0/storage/spider/mysql-test/spider/oracle2/t/direct_aggregate_part.test mariadb-10.0/storage/spider/mysql-test/spider/oracle2/t/have_partition.inc mariadb-10.0/storage/spider/mysql-test/spider/oracle2/t/have_trigger.inc mariadb-10.0/storage/spider/mysql-test/spider/oracle2/t/spider3_fixes.test mariadb-10.0/storage/spider/mysql-test/spider/oracle2/t/spider3_fixes_part.test mariadb-10.0/storage/spider/mysql-test/spider/t/direct_aggregate.test mariadb-10.0/storage/spider/mysql-test/spider/t/direct_aggregate_part.test mariadb-10.0/storage/spider/mysql-test/spider/t/have_partition.inc mariadb-10.0/storage/spider/mysql-test/spider/t/have_trigger.inc mariadb-10.0/storage/spider/mysql-test/spider/t/spider3_fixes.test mariadb-10.0/storage/spider/mysql-test/spider/t/spider3_fixes_part.test mariadb-10.0/win/packaging/COPYING.rtf mariadb-10.0/win/packaging/CPackWixConfig.cmake mariadb-10.0/win/packaging/ca/CustomAction.cpp mariadb-10.0/win/packaging/custom_ui.wxs mariadb-10.0/win/packaging/extra.wxs.in mariadb-10.0/win/packaging/heidisql.cmake mariadb-10.0/win/packaging/heidisql.wxi.in mariadb-10.0/win/packaging/heidisql_feature.wxi.in mariadb-10.0/win/packaging/mysql_server.wxs.in mariadb-10.0/win/upgrade_wizard/stdafx.h mariadb-10.0/win/upgrade_wizard/targetver.h mariadb-10.0/win/upgrade_wizard/upgrade.cpp mariadb-10.0/win/upgrade_wizard/upgrade.h mariadb-10.0/win/upgrade_wizard/upgrade.rc mariadb-10.0/win/upgrade_wizard/upgradeDlg.h mariadb-10.0/win/upgrade_wizard/upgrade_wizard.exe.manifest This is the first time in two years I've found CRLF line endings in MariaDB source files. I hope I won't see them again..

3 2

[Maria-developers] Proposal accepted for GSoc'15
by sriram patil 08 May '15

08 May '15

Hi All, I am Sriram from IIIT Hyderabad, India. I am a Masters' student here. I have worked with MariaDB in GSoC'14 with Alexander Barkov. I had great time hacking on MariaDB last year, so, here I am again. :) This year I will be working on MaxScale filters with Markus Makela. The project title is "MaxScale filter to capture incoming operations for consumption in external sources". (MXS-2) I am excited to get started. Hoping to have a great summer this year as well. :) Thanks, Sriram Patil

2 2

Re: [Maria-developers] [Commits] 313a970: MDEV-7922 - ERROR 1939 (HY000): Engine PERFORMANCE_SCHEMA failed to discover
by Sergei Golubchik 07 May '15

07 May '15

Hi, Sergey! On Apr 08, svoj(a)mariadb.org wrote: > revision-id: 313a970cbe9ac523b0584d293e40bff66d4fbafc > parent(s): 7d9e94e2e6c32b906d1d8469e58bc04b8da1c121 > committer: Sergey Vojtovich > branch nick: mariadb > timestamp: 2015-04-08 10:55:51 +0400 > message: > > MDEV-7922 - ERROR 1939 (HY000): Engine PERFORMANCE_SCHEMA failed to discover > table > > Performance schema discovery fails if connection has no active database set. > > This happened due to restriction in SQL parser: table name with no database name > is ambiguous in such case. > > Fixed by temporary substitution of default database with being discovered table > database. ok to push Regards, Sergei

1 0

[Maria-developers] MDEV-7807 information_schema.processlist truncates queries with binary strings
by Alexander Barkov 07 May '15

07 May '15

Hi Sergei, Please review the patch for: MDEV-7807 information_schema.processlist truncates queries with binary strings The patch is for 10.1, as agreed with the reporter. Previously in 10.1 we fixed binary->string conversion behaviour not to truncate on bad bytes, and replace bad bytes to question marks instead. That fixed half of the problem reported. This patch adds a new new column "PROCESSLIST.INFO_BINARY blob". Thanks.

2 1

Re: [Maria-developers] [Commits] 93e746a: MDEV-7793 - Race condition between XA COMMIT/ROLLBACK and disconnect
by Sergei Golubchik 07 May '15

07 May '15

Hi, Sergey! On Mar 17, svoj(a)mariadb.org wrote: > revision-id: 93e746afb2d1a23cb933291ad736a20858b5ac3e > parent(s): ccc7297fe94af1129c717f91d31fa075d54a0371 > committer: Sergey Vojtovich > branch nick: mariadb > timestamp: 2015-03-17 19:49:04 +0400 > message: > > MDEV-7793 - Race condition between XA COMMIT/ROLLBACK and disconnect > > XA COMMIT/ROLLBACK of XA transaction owned by different thread may access > freed memory if that thread disconnects at the same time. > > Also concurrent XA COMMIT/ROLLBACK of recovered XA transaction were not > serialized properly. > > diff --git a/sql/sql_class.cc b/sql/sql_class.cc > index 3071ba6..f18231d 100644 > --- a/sql/sql_class.cc > +++ b/sql/sql_class.cc > @@ -5355,7 +5378,7 @@ bool xid_cache_insert(THD *thd, XID_STATE *xid_state) > switch (res) > { > case 0: > - xid_state->xid_cache_element->mark_initialized(); > + xid_state->xid_cache_element->set(XID_cache_element::ACQUIRED); Why not mark_acquired() here? Can you be sure that no other thread has already acquired this xid_cache_element after you've inserted it? > break; > case 1: > my_error(ER_XAER_DUPID, MYF(0)); Regards, Sergei

2 2

Re: [Maria-developers] [Commits] 54e9df4: MDEV-7985: MySQL Users Break when Migrating to MariaDB, part 2
by Sergei Golubchik 05 May '15

05 May '15

Hi, Vicentiu! Two comments to the patch - see below. One general comment - you've done quite a bit of refactoring and this refactoring is *not* in 10.1 style. How should it be merged into 10.1? Should your changes be ignored (and not merged)? Should they be merged as is? Aren't they mostly unnecessary in 10.1? On May 05, vicentiu(a)mariadb.org wrote: > revision-id: 54e9df42d1f5837ae0df51e511ba48d3682be20d > parent(s): ae18a28500974351cf42fa3cac67c83e0647d510 > committer: Vicențiu Ciorbaru > branch nick: server > timestamp: 2015-05-05 21:19:53 +0300 > message: > > MDEV-7985: MySQL Users Break when Migrating to MariaDB, part 2 > > Gave priority to password field when using a native authentication > plugin. > > Also, prevented a user from setting an invalid auth_string, when using > native authentication. > > diff --git a/sql/sql_acl.cc b/sql/sql_acl.cc > --- a/sql/sql_acl.cc > +++ b/sql/sql_acl.cc > @@ -862,6 +862,17 @@ static char *fix_plugin_ptr(char *name) > return name; > } > > +static bool plugin_is_native(const char* plugin) > +{ > + if (!plugin) > + return false; > + if ((my_strcasecmp(system_charset_info, plugin, > + native_password_plugin_name.str) == 0) || > + (my_strcasecmp(system_charset_info, plugin, > + old_password_plugin_name.str) == 0)) > + return true; > + return false; > +} I tried to compare plugin names almost always by pointers, not by comparing strings. That is, the string (as coming from the table or from the user) is strcmp'ed *once* and then replaced with a pointer to a static plugin name string. So I'd prefer it if you'd remove plugin_is_native() and instead used fix_user_plugin_ptr(). And then you remove fix_user_plugin_ptr() from acl_update_user() and acl_insert_user(). > /** > Fix ACL::plugin pointer to point to a hard-coded string, if appropriate > > @@ -1973,6 +2020,10 @@ static void acl_update_user(const char *user, const char *host, > acl_user->auth_string.length= auth->length; > if (fix_user_plugin_ptr(acl_user)) > acl_user->plugin.str= strmake_root(&acl_memroot, plugin->str, plugin->length); > + else > + set_user_salt(acl_user, acl_user->auth_string.str, > + acl_user->auth_string.length); what if auth_string is empty, how does the password take precedence in this case? > + > } > else > if (password[0]) Regards, Sergei

1 0

[Maria-developers] Please review: MDEV-7778 impossible create copy of table, if table contain default value for timestamp field
by Alexander Barkov 04 May '15

04 May '15

Hi Sergei, Please review a patch for MDEV-7778. As agreed on IRC, this is a merge from the latest MySQL, which has already fixed the problem. Greetings.

2 1

[Maria-developers] connect on IBM s390
by Alexander Barkov 04 May '15

04 May '15

Hi Olivier, MariaDB fails with some Connect tests on IBM s390. Please have a look into Debian build bot results: https://buildd.debian.org/status/fetch.php?pkg=mariadb-10.0&arch=s390x&ver=… I currently have a temporary access to an IMB s390 machine. It would be nice to fix the failures. Please find the mtr log attached. This is what I got on this test machine. The failures are very similar to what Debian build bot has. If you don't mind, I'd like to start fixing failures with "bin" and "dbf". 1. dbf.bin fails because Connect writes the DBF file header using the current platform endiannes, while from my understanding it should always write the header using little endianness. This should be fixed in the relevant DBF code in Connect. Can you please do this? 2. connect.bin fails because Connect treats columns in the current machine byte order, while Testbal.dat was created on a little endian machine. I can see a few options to fix this test failure: a. Disable on big endian machines. Bad idea. b. Make it possible to specify endiannes of a data file (or even per column), so we can tell in the tests that Testbal.dat (or its relevant columns) are in little endian format. Perhaps it's already possible in the latest Connect versions. Is it? If it's not, how difficult would be to implement this? c. Have two pre-created files Testbal_le.dat and Testbal_be.dat and load the one that suites the current machine endiannes. I'm thinking of going "c" now, with a hope to have both "b" and "c" in the future. If you agree, we'll need a big-endian version of Testbal.dat. 3. It seems that some other tests fail for the same reasons with #2. Having a big-endian version of the test data files, or having the endian flag in CREATE TABLE, would fix some of them. I suggest we fix "bin" first, and make sure it works fine an all platforms. Then do the same thing for the other tests. Thanks.

1 0

Re: [Maria-developers] [Commits] 4bff8b0: MDEV-7004 - Merge scalability fixes from 10.0-power
by jocelyn fournier 02 May '15

02 May '15

Hi Sergey, Just to be sure, currently MariaDB doesn't implement any Hardware Transactional Memory Built-in Functions ? If so, is there any plan to evaluate the gain they could offer to MariaDB ? Thanks and regards, Jocelyn Fournier Le 04/12/2014 14:45, svoj(a)mariadb.org a écrit : > revision-id: 4bff8b0b03ba82cd0ae6817b54de1780e89047ca > parent(s): 3526fe1b6000b2558290007e5cc3a1018850787c > committer: Sergey Vojtovich > branch nick: 10.1 > timestamp: 2014-12-04 17:44:46 +0400 > message: > > MDEV-7004 - Merge scalability fixes from 10.0-power > > Preallocate dynamic array and bitmap on mem_root to avoid expensive malloc. > This reduces number of allocations from 39 to 31 per OLTP RO transaction. > > --- > sql/opt_range.cc | 12 ++++++------ > 1 file changed, 6 insertions(+), 6 deletions(-) > > diff --git a/sql/opt_range.cc b/sql/opt_range.cc > index 1a9ae3d..813508c 100644 > --- a/sql/opt_range.cc > +++ b/sql/opt_range.cc > @@ -1838,8 +1838,6 @@ void SQL_SELECT::cleanup() > index= key_nr; > head= table; > key_part_info= head->key_info[index].key_part; > - my_init_dynamic_array(&ranges, sizeof(QUICK_RANGE*), 16, 16, > - MYF(MY_THREAD_SPECIFIC)); > > /* 'thd' is not accessible in QUICK_RANGE_SELECT::reset(). */ > mrr_buf_size= thd->variables.mrr_buff_size; > @@ -1857,9 +1855,12 @@ void SQL_SELECT::cleanup() > file= head->file; > record= head->record[0]; > > - /* Allocate a bitmap for used columns (Q: why not on MEM_ROOT?) */ > - if (!(bitmap= (my_bitmap_map*) my_malloc(head->s->column_bitmap_size, > - MYF(MY_WME | MY_THREAD_SPECIFIC)))) > + my_init_dynamic_array2(&ranges, sizeof(QUICK_RANGE*), > + thd->alloc(sizeof(QUICK_RANGE*) * 16), 16, 16, > + MYF(MY_THREAD_SPECIFIC)); > + > + /* Allocate a bitmap for used columns */ > + if (!(bitmap= (my_bitmap_map*) thd->alloc(head->s->column_bitmap_size))) > { > column_bitmap.bitmap= 0; > *create_error= 1; > @@ -1923,7 +1924,6 @@ void QUICK_RANGE_SELECT::range_end() > } > delete_dynamic(&ranges); /* ranges are allocated in alloc */ > free_root(&alloc,MYF(0)); > - my_free(column_bitmap.bitmap); > } > my_free(mrr_buf_desc); > DBUG_VOID_RETURN; > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits

2 1

[Maria-developers] [GSoC 2015] Accepted Student- Self Introduction
by Delveri chick 30 Apr '15

30 Apr '15

Hello everyone ! I am Delveri Chick, a junior software engineering student from Cameroon. I have been accepted into GSoC 2015 to hack on project titled : Improved Temporary Tables. Here <http://www.google-melange.com/gsoc/proposal/review/student/google/gsoc2015/…> is my GSoC proposal. Sergei Golubchik will be mentoring me for this project. I am really excited to spend my summer hacking on MariaDB, and hoping to learn a lot of things from the developer community. Thanks. Delveri

2 1

Re: [Maria-developers] [Commits] 13ecde8: Fix compiler warnings.
by Vicențiu Ciorbaru 30 Apr '15

30 Apr '15

Hi Jan! I have one concern with this patch. See below: On Thu, 30 Apr 2015 at 11:45 Jan Lindström <jan.lindstrom(a)mariadb.com> wrote: > revision-id: 13ecde851f08a7952f774707b19734f351bb6598 > parent(s): 5027e0b035f96e2583eff2d1ac71c7c44a093ac8 > committer: Jan Lindström > branch nick: 10.0-git > timestamp: 2015-04-30 11:44:34 +0300 > message: > > Fix compiler warnings. > > --- > storage/innobase/sync/sync0sync.cc | 27 +++++++++++++-------------- > storage/xtradb/sync/sync0sync.cc | 27 +++++++++++++-------------- > 2 files changed, 26 insertions(+), 28 deletions(-) > > diff --git a/storage/innobase/sync/sync0sync.cc > b/storage/innobase/sync/sync0sync.cc > index 121f8e9..78d86f3 100644 > --- a/storage/innobase/sync/sync0sync.cc > +++ b/storage/innobase/sync/sync0sync.cc > @@ -862,10 +862,10 @@ sync_thread_levels_g( > { > ulint i; > > - for (i = 0; i < arr->n_elems; i++) { > + for (i = 0; i < arr->size(); i++) { > const sync_level_t* slot; > > I think this here has to be arr->at(i), or &(*arr)[i]. Doing &arr[i] will actually get the i'th std::vector<> which does not exist, since arr is a pointer to only one std::vector<>. > - slot = &arr->elems[i]; > + slot = (const sync_level_t*)&arr[i]; > > if (slot->latch != NULL && slot->level <= limit) { > if (warn) { > @@ -897,10 +897,10 @@ sync_thread_levels_contain( > { > ulint i; > > - for (i = 0; i < arr->n_elems; i++) { > + for (i = 0; i < arr->size(); i++) { > const sync_level_t* slot; > > Same here. > - slot = &arr->elems[i]; + slot = (const sync_level_t*)&arr[i]; > > if (slot->latch != NULL && slot->level == level) { > > @@ -944,10 +944,10 @@ sync_thread_levels_contains( > > arr = thread_slot->levels; > > - for (i = 0; i < arr->n_elems; i++) { > + for (i = 0; i < arr->size(); i++) { > sync_level_t* slot; > Same here. > - slot = &arr->elems[i]; > + slot = (sync_level_t*)&arr[i]; > > if (slot->latch != NULL && slot->level == level) { > > @@ -993,10 +993,10 @@ sync_thread_levels_nonempty_gen( > > arr = thread_slot->levels; > > - for (i = 0; i < arr->n_elems; ++i) { > + for (i = 0; i < arr->size(); ++i) { > const sync_level_t* slot; > Same here. > - slot = &arr->elems[i]; > + slot = (const sync_level_t*)&arr[i]; > > if (slot->latch != NULL > && (!dict_mutex_allowed > @@ -1053,10 +1053,10 @@ sync_thread_levels_nonempty_trx( > > arr = thread_slot->levels; > > - for (i = 0; i < arr->n_elems; ++i) { > + for (i = 0; i < arr->size(); ++i) { > const sync_level_t* slot; > Same here. > - slot = &arr->elems[i]; > + slot = (const sync_level_t*)&arr[i]; > > if (slot->latch != NULL > && (!has_search_latch > @@ -1331,7 +1331,7 @@ sync_thread_add_level( > > sync_level.latch = latch; > sync_level.level = level; > - array->elems.push_back(sync_level); > + array->push_back(sync_level); > > mutex_exit(&sync_thread_mutex); > } > @@ -1377,14 +1377,14 @@ sync_thread_reset_level( > > array = thread_slot->levels; > - for (std::vector<sync_level_t>::iterator it = array->elems.begin(); > it != array->elems.end(); ++it) { > + for (std::vector<sync_level_t>::iterator it = array->begin(); it > != array->end(); ++it) { > sync_level_t level = *it; > > if (level.latch != latch) { > continue; > } > > - array->elems.erase(it); > + array->erase(it); > mutex_exit(&sync_thread_mutex); > return(TRUE); > } > @@ -1471,7 +1471,6 @@ sync_thread_level_arrays_free(void) > > /* If this slot was allocated then free the slot memory > too. */ > if (slot->levels != NULL) { > - > slot->levels->elems.erase(slot->levels->elems.begin(),slot->levels->elems.end()); > delete slot->levels; > } > } > diff --git a/storage/xtradb/sync/sync0sync.cc > b/storage/xtradb/sync/sync0sync.cc > index d0e3b71..9dcdd9c 100644 > --- a/storage/xtradb/sync/sync0sync.cc > +++ b/storage/xtradb/sync/sync0sync.cc > @@ -970,10 +970,10 @@ sync_thread_levels_g( > { > ulint i; > > - for (i = 0; i < arr->n_elems; i++) { > + for (i = 0; i < arr->size(); i++) { > const sync_level_t* slot; > > Same here. > - slot = &arr->elems[i]; > + slot = (const sync_level_t*)&arr[i]; > > if (slot->latch != NULL && slot->level <= limit) { > if (warn) { > @@ -1005,10 +1005,10 @@ sync_thread_levels_contain( > { > ulint i; > > - for (i = 0; i < arr->n_elems; i++) { > + for (i = 0; i < arr->size(); i++) { > const sync_level_t* slot; > Same here. > - slot = &arr->elems[i]; > + slot = (const sync_level_t*)&arr[i]; > > if (slot->latch != NULL && slot->level == level) { > > @@ -1052,10 +1052,10 @@ sync_thread_levels_contains( > > arr = thread_slot->levels; > > - for (i = 0; i < arr->n_elems; i++) { > + for (i = 0; i < arr->size(); i++) { > sync_level_t* slot; > > Same here. > - slot = &arr->elems[i]; > + slot = (sync_level_t*)&arr[i]; > > if (slot->latch != NULL && slot->level == level) { > > @@ -1101,10 +1101,10 @@ sync_thread_levels_nonempty_gen( > > arr = thread_slot->levels; > > - for (i = 0; i < arr->n_elems; ++i) { > + for (i = 0; i < arr->size(); ++i) { > const sync_level_t* slot; > > Same here. > - slot = &arr->elems[i]; > + slot = (const sync_level_t*)&arr[i]; > > if (slot->latch != NULL > && (!dict_mutex_allowed > @@ -1161,10 +1161,10 @@ sync_thread_levels_nonempty_trx( > > arr = thread_slot->levels; > > - for (i = 0; i < arr->n_elems; ++i) { > + for (i = 0; i < arr->size(); ++i) { > const sync_level_t* slot; > > Same here. > - slot = &arr->elems[i]; > + slot = (const sync_level_t*)&arr[i]; > > if (slot->latch != NULL > && (!has_search_latch > @@ -1456,7 +1456,7 @@ sync_thread_add_level( > > sync_level.latch = latch; > sync_level.level = level; > - array->elems.push_back(sync_level); > + array->push_back(sync_level); > > mutex_exit(&sync_thread_mutex); > } > @@ -1502,14 +1502,14 @@ sync_thread_reset_level( > > array = thread_slot->levels; > > - for (std::vector<sync_level_t>::iterator it = > array->elems.begin(); it != array->elems.end(); ++it) { > + for (std::vector<sync_level_t>::iterator it = array->begin(); it > != array->end(); ++it) { > sync_level_t level = *it; > > if (level.latch != latch) { > continue; > } > > - array->elems.erase(it); > + array->erase(it); > mutex_exit(&sync_thread_mutex); > return(TRUE); > } > @@ -1597,7 +1597,6 @@ sync_thread_level_arrays_free(void) > > /* If this slot was allocated then free the slot memory > too. */ > if (slot->levels != NULL) { > - > slot->levels->elems.erase(slot->levels->elems.begin(),slot->levels->elems.end()); > delete slot->levels; > } > } >

1 0

Re: [Maria-developers] [Commits] 8a18bbc: MDEV-8079: Crash when running MariaDB Debug with InnoDB on Windows
by Vicențiu Ciorbaru 30 Apr '15

30 Apr '15

Here is a patch that fixes all compilation related issues. Unfortunately it's based on 10.1. It might need a bit of tweaking to merge directly into 10.0

1 0

[Maria-developers] GSoC 2015 student
by Blessing Atie 30 Apr '15

30 Apr '15

Hello everyone, My surname is Blessing and I am a student at University of Buea. I am very happy to be selected by MariaDb to part take in GSoC 2015. I will be working on Improving table Functions mentored Sergei Golubchik and I look forward to know him soon on #maria as well as the other students. I equally look forward on using this as an opportunity to learn more about open source and MariaDB community. Can't wait to get hands dirty on MariaDB source. Thanks. Blessing

2 1

Re: [Maria-developers] MDEV-7774: Crash when dropping user within rebuild_role_grants
by Sergei Golubchik 29 Apr '15

29 Apr '15

Hi, Vicentiu! On Mar 13, vicentiu(a)mariadb.org wrote: > revision-id: 22884d35e39ba078dddf5c15b42dd7e2ebb39c07 > parent(s): 184f718fef0101a7559364cb97e22ee568e64c12 > committer: Vicențiu Ciorbaru > branch nick: server > timestamp: 2015-03-13 20:12:22 +0200 > message: > > MDEV-7774: Crash when dropping user within rebuild_role_grants > > The issue comes from not taking all possibilities to match an entry > within the roles_mapping HASH, when updating the data structure. ok to push Regards, Sergei

1 0

[Maria-developers] Likely a dead code in build_equal_items_for_cond()
by Alexander Barkov 29 Apr '15

29 Apr '15

Hi Sanja and Igor, Can you please help to understand this pieces of code in sql_select.cc in 10.0: > else if (cond->type() == Item::FUNC_ITEM || > cond->real_item()->type() == Item::FIELD_ITEM) > { It seems that the "cond->real_item()->type() == Item::FIELD_ITEM" part of this condition is a dead code. I added a DBUG_ASSERT in the very beginning of this block, like this: > else if (cond->type() == Item::FUNC_ITEM || > cond->real_item()->type() == Item::FIELD_ITEM) > { > DBUG_ASSERT(cond->real_item()->type() != Item::FIELD_ITEM); and run all tests. Nothing crashed. I suspect that the following scenario could happen: 1. In April 2011 Igor fixed bugs #717577, #724942 with this commit: > commit 8d9dd21d85e257051b45b2f779dcd9bf696bf9e1 > Author: Igor Babaev <igor(a)askmonty.org> Wed Apr 27 06:58:41 2011 > Committer: Igor Babaev <igor(a)askmonty.org> Wed Apr 27 06:58:41 2011 > > Fixed LP bugs #717577, #724942. and this line was probably to handle conditions like "WHERE field". 2. In July 2011 Sanja introduced a new function normalize_cond(), with this commit: > commit 20a2e1d0ac46e375ee7246576b541624c6f8f028 > Author: unknown <sanja(a)askmonty.org> Thu Jul 21 11:20:55 2011 > unknown <sanja(a)askmonty.org> Thu Jul 21 11:20:55 2011 so all conditions "WHERE/HAVING field" are now replaced to "WHERE/HAVING field<>0". So now, by the time of a build_equal_items_for_cond() call all conditions are already replaced and no FIELD_ITEM are actually possible. Can you please confirm this? If my guess is correct then this part can now be safely removed: > || > cond->real_item()->type() == Item::FIELD_ITEM Thanks.

1 0

[Maria-developers] MDEV-7802 adding status variables - backport to 10.0 request
by Daniel Black 29 Apr '15

29 Apr '15

Hi, I'm requesting a backport of MDEV-7802 https://github.com/MariaDB/server/pull/30/files to 10.0 It added status variables which where important to determining what operational values to set binlog_commit_wait_usec / binlog_commit_wait_count variables to. With parallel replication efficiency and general group commit IO reduction dependent on the tradeoff of these settings with the commit delay, the status variables binlog_group_commit_trigger_count, binlog_group_commit_trigger_timeout and binlog_group_commit_trigger_lock_wait that this patch adds are operationally important. The changes added a few variables and didn't affect the flow of the program at all. In the binary log execution path, no locks where added and only a few increments of static variables where added. There was an additional lock usage of LOCK_prepare_ordered in the show global status for the period to copy the added variables into the global status structures. I'm quite aware of the dangers of show global status holding locks wrongly (MDEV-7126) I don't suspect this does. On the balance of risk/benefit can I request that this feature be merged to 10.0. -- -- Daniel Black, Engineer @ Open Query (http://openquery.com.au) Remote expertise & maintenance for MySQL/MariaDB server environments.

4 4

[Maria-developers] GSoC 2015 Accepted Student
by arlene seuleu 28 Apr '15

28 Apr '15

Hi everyone, I am Arlene Flavie a computer engineering student at the university of Buea and I just got confirmation mail that I was selected to participate in the Google summer of code 2015 under MariaDB. I wish to use this opportunity to thank the mentors for giving me this opportunity to contribute to opensource. I wish to learn much in this experience as I will be Mentored by Sanda Byelkin on the project Import and Export data formats into and from dynamic columns. I look forward to do a similar introduction of myself on #maria. Thanks Arlene

1 0

[Maria-developers] MDEV-7950 Item_func::type() takes 0.26% in OLTP RO
by Alexander Barkov 28 Apr '15

28 Apr '15

Hi Sergey, Can you please review the "step 2" part of MDEV-7950? This is rather a mechanical cut-and-paste change. It does the following things: 1. Removes the function build_equal_items_for_cond() and introduces a new method Item::build_equal_items() instead, with specific implementations in the following Items: Item (the default implementation) Item_ident_or_func_or_sum Item_cond Item_cond_and 2. Adds a new abstract class Item_ident_or_func_or_sum, a common parent for Item_ident and Item_func_or_sum, as they have exactly the same build_equal_items(). 3. Renames Item_cond_and::cond_equal to Item_cond_and::m_cond_equal, to avoid confusion between the member and local variables named "cond_equal". The main purpose of this patch is to prepare to the next step in MDEV-7950. But it already improves performance per se: it gets rid of some virtual calls: - The former call for "((Item_cond*) cond)->functype()" is not needed any more, because Item_cond and Item_cond_and now have different implementations of the method build_equal_items(). - This condition: "else if (cond->type() == Item::FUNC_ITEM || cond->real_item()->type() == Item::FIELD_ITEM)" (which used to cover Item_func, Item_ident and Item_ref) consisted of three virtual calls. It's now replaced by a single virtual call for Item_ident_or_func_or_sum::build_equal_items(), which covers all three item types. Thanks.

2 2

[Maria-developers] Mariadb java client issue.
by 김경남 28 Apr '15

28 Apr '15

Hi, I hava a issue of using mariadb java client. I can't get column alias from query result set. My query as below. select col1 as column_alias1 from table1; reuslt: --------------------- col1 ------------------------ xxx ... In case of mysql connectorJ, there was a same issue. so I added a option like "useOldAliasMetadataBehavior=true" to connection string and that issue solved. but I cant't solve this issue in case of using mariadb java client. Is there a correct way? Regards, Kyungnam kim 김경남 부책임연구원 솔루션사업본부 정보기술 연구원 신기술개발팀 02-2099-6177 / 010-2208-3006 _____ 상기 메시지와 첨부화일 내에는 비밀정보가 포함되어 있을 수 있으며, 지정된 수신자에 한하여 조회 및 사용될 수 있습니다. 만약 송신자의 실수로 인하여 상기 메시지를 수신하였다면, 송신자에게 메시지를 반송해 주시고, 원본 메시지와 모든 사본을 폐기해 주시기 바랍니다. 상기 메시지의 전체 또는 일부에 대해 무단 열람, 사용, 공개, 배포하는 것은 금지되어 있습니다.(주)LG CNS. This message and its attachments may contain confidential information, and they are intended to be viewed or used by only the individuals specified in the message. If you have received this message in an error from the sender, please contact the sender immediately to notify the error and delete all of the message and its copies. It is prohibited to view, use, make public and/or distribute part or whole of this message without written permission.

1 0

[Maria-developers] MariaDB / Galera BUG
by Andrew W Elble 27 Apr '15

27 Apr '15

Environment: MariaDB-10.0.16 Galera-3-25.3.9 This statement does nasty things to a MariaDB+Galera cluster: CREATE TABLE u91rw_zoo_version (`version` varchar(255) NOT NULL) ENGINE=innoDB DEFAULT CHARSET=utf8 SELECT '3.3.3' as version; Statement fails to replicate to the other nodes because the target table doesn't exist. Cluster then disassembles itself. This patch makes the problem disappear. Please evaluate? (note: patch based against github.com/MariaDB/server.git) ============= diff --git a/sql/sql_parse.cc b/sql/sql_parse.cc index be57be9b223a..d0521ed896b8 100644 --- a/sql/sql_parse.cc +++ b/sql/sql_parse.cc @@ -3325,6 +3325,11 @@ mysql_execute_command(THD *thd) /* Store reference to table in case of LOCK TABLES */ create_info.table= create_table->table; + if (WSREP(thd) && (!thd->is_current_stmt_binlog_format_row() || + !create_info.tmp_table())) + { + WSREP_TO_ISOLATION_BEGIN(create_table->db, create_table->table_name, NULL) + } /* select_create is currently not re-execution friendly and needs to be created for every execution of a PS/SP. ============= We have a full environment setup in which to test. debug/trace available on request. Will open bug if required for your workflow. Thanks, Andy -- Andrew W. Elble aweits(a)discipline.rit.edu Infrastructure Engineer, Communications Technical Lead Rochester Institute of Technology PGP: BFAD 8461 4CCF DC95 DA2C B0EB 965B 082E 863E C912

2 6

[Maria-developers] MariaDB finally in Debian officially (Fwd: Debian 8 "Jessie" released)
by Otto Kekäläinen 26 Apr '15

26 Apr '15

Hello! Debian 8 "Jessie" has been released to today. It is the first official Debian release that ships with MariaDB (it has MariaDB 10.0.16 and MySQL 5.5.42). MariaDB client for C (the LGPL version) is also included. Release notes[1] state that in Debian 9 only one MySQL variant will be shipped. Hopefully the decision will tip towards MariaDB. The MariaDB "request for packaging" issue[2] was opened in bugs.debian.org in 2010. I started doing MariaDB packaging for Debian in 2013. The path here has been long and I am glad that this milestone is finally reached. [1] https://www.debian.org/releases/jessie/amd64/release-notes/ch-whats-new.en.… [2] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=565308 ---------- Forwarded message ---------- From: Paul Wise <pabs(a)debian.org> Date: 2015-04-26 5:28 GMT+03:00 Subject: Debian 8 "Jessie" released To: debian-announce(a)lists.debian.org ------------------------------------------------------------------------ The Debian Project https://www.debian.org/ Debian 8 "Jessie" released press(a)debian.org April 25th, 2015 https://www.debian.org/News/2015/20150426 ------------------------------------------------------------------------ After almost 24 months of constant development the Debian project is proud to present its new stable version 8 (code name "Jessie"), which will be supported for the next 5 years thanks to the combined work of the Debian Security team [1] and of the Debian Long Term Support [2] team. 1: http://security-team.debian.org/ 2: https://wiki.debian.org/LTS "Jessie" ships with a new default init system, systemd. The systemd suite provides many exciting features such as faster boot times, cgroups for services, and the possibility of isolating part of the services. The sysvinit init system is still available in "Jessie". The UEFI ("Unified Extensible Firmware Interface") support introduced in "Wheezy" has also been greatly improved in Jessie. This includes workarounds for many known firmware bugs, support for UEFI on 32-bit systems, and support for 64-bit kernels with 32-bit UEFI firmware (with the latter being included only on our amd64/i386 "multi-arch" installation media). Since the previous release, members of the Debian project have also made important improvements to our supporting services. One of these is a browsable view of all source code shipped in Debian [3] currently available at sources.debian.net [4]. Of course, with over 20,000 source packages, it can be quite daunting to locate the right file. Therefore, we are also very pleased to present Debian Code Search [5], available at codesearch.debian.net [6]. Both services are complemented by a completely rewritten and more reponsive package tracking system [7]. 3: https://www.debian.org/News/weekly/2013/14/#sources 4: https://sources.debian.net 5: https://www.debian.org/News/weekly/2014/17/#DCS 6: https://codesearch.debian.net/ 7: https://tracker.debian.org/ This release includes numerous updated software packages, such as: * Apache 2.4.10 * Asterisk 11.13.1 * GIMP 2.8.14 * an updated version of the GNOME desktop environment 3.14 * GNU Compiler Collection 4.9.2 * Icedove 31.6.0 (an unbranded version of Mozilla Thunderbird) * Iceweasel 31.6.0esr (an unbranded version of Mozilla Firefox) * KDE Plasma Workspaces and KDE Applications 4.11.13 * LibreOffice 4.3.3 * Linux 3.16.7-ctk9 * MariaDB 10.0.16 and MySQL 5.5.42 * Nagios 3.5.1 * OpenJDK 7u75 * Perl 5.20.2 * PHP 5.6.7 * PostgreSQL 9.4.1 * Python 2.7.9 and 3.4.2 * Samba 4.1.17 * Tomcat 7.0.56 and 8.0.14 * Xen Hypervisor 4.4.1 * the Xfce 4.10 desktop environment * more than 43,000 other ready-to-use software packages, built from nearly 20,100 source packages. With this broad selection of packages and its traditional wide architecture support, Debian once again stays true to its goal of being the universal operating system. It is suitable for many different use cases: from desktop systems to netbooks; from development servers to cluster systems; and for database, web, or storage servers. At the same time, additional quality assurance efforts like automatic installation and upgrade tests for all packages in Debian's archive ensure that "Jessie" fulfills the high expectations that users have of a stable Debian release. A total of ten architectures are supported: 32-bit PC / Intel IA-32 (i386), 64-bit PC / Intel EM64T / x86-64 (amd64), Motorola/IBM PowerPC (powerpc for older hardware and ppc64el for the new 64-bit (little- endian)), MIPS (mips (big-endian) and mipsel (little-endian)), IBM S/390 (64-bit s390x) and for ARM, armel and armhf for old and new 32-bit hardware, plus arm64 for the new 64-bit "AArch64" architecture. Want to give it a try? ---------------------- If you simply want to try Debian 8 "Jessie" without having to install it, you can use a special image, known as a live image, available for CDs, USB sticks, and netboot setups. Initially, these images are provided for the amd64 and i386 architectures only. It is also possible to use these live images to install Debian. More information is available from the Debian Live homepage [8]. 8: http://live.debian.net/ If, instead, you want to install it to your computer's permanent storage, you can choose from a range of installation media, such as Blu- ray Discs, DVDs, CDs, and USB sticks, or from the network. Several desktop environments — GNOME, KDE Plasma Desktop and Applications, Xfce, and LXDE — may be installed through CD images; the desired one may be chosen from the boot menus of the CDs/DVDs. In addition, multi- architecture CDs and DVDs are available which support installation of multiple architectures from a single disc. Or you can always create bootable USB installation media (see the Installation Guide [9] for more details). For cloud users Debian also offers pre-built OpenStack images [10], ready to use. 9: https://www.debian.org/releases/jessie/installmanual 10: http://cdimage.debian.org/cdimage/openstack/current/ The installation images may be downloaded right now via bittorrent [11] (the recommended method), jigdo [12], or HTTP [13]; see Debian on CDs [14] for further information. "Jessie" will soon be available on physical DVD, CD-ROM, and Blu-ray Discs from numerous vendors [15] too. 11: https://www.debian.org/CD/torrent-cd/ 12: https://www.debian.org/CD/jigdo-cd/#which 13: https://www.debian.org/CD/http-ftp/ 14: https://www.debian.org/CD/ 15: https://www.debian.org/CD/vendors Upgrading Debian ---------------- Upgrades to Debian 8 from the previous release, Debian 7 (codenamed "Wheezy"), are automatically handled by the apt-get package management tool for most configurations. As always, Debian systems may be upgraded painlessly, in place, without any forced downtime, but it is strongly recommended to read the release notes [16] as well as the installation guide [17] for possible issues, and for detailed instructions on installing and upgrading. The release notes will be further improved and translated to additional languages in the weeks after the release. 16: https://www.debian.org/releases/jessie/releasenotes 17: https://www.debian.org/releases/jessie/installmanual About Debian ------------ Debian is a free operating system, developed by thousands of volunteers from all over the world who collaborate via the Internet. The Debian project's key strengths are its volunteer base, its dedication to the Debian Social Contract and Free Software, and its commitment to provide the best operating system possible. Debian 8 is another important step in that direction. Contact Information ------------------- For further information, please visit the Debian web pages at https://www.debian.org/ or send mail to <press(a)debian.org>. -- To UNSUBSCRIBE, email to debian-announce-REQUEST(a)lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmaster(a)lists.debian.org Archive: https://lists.debian.org/20150426022805.GA7860@chianamo -- Check out our blog at http://seravo.fi/blog and follow @ottokekalainen

2 2

Re: [Maria-developers] Rev 4407: MDEV-6916: Upgrade from MySQL to MariaDB breaks already created views
by Sergei Golubchik 25 Apr '15

25 Apr '15

Hi, Sanja! On Feb 09, sanja(a)askmonty.org wrote: > At file:///home/bell/maria/bzr/work-maria-5.5-MDEV-6916-check_view/ > > ------------------------------------------------------------ > revno: 4407 > revision-id: sanja(a)askmonty.org-20150209014828-kx3cr36hgrvjhtrg > parent: svoj(a)mariadb.org-20150114135038-v50g2cul4vce63h8 > committer: sanja(a)askmonty.org > branch nick: work-maria-5.5-MDEV-6916-check_view > timestamp: Mon 2015-02-09 02:48:28 +0100 > message: > MDEV-6916: Upgrade from MySQL to MariaDB breaks already created views > > CHECK/REPAIR commands and mysql_upgradesupport for upgrade from MySQL server support. > === modified file 'client/mysql_upgrade.c' > --- a/client/mysql_upgrade.c 2014-03-27 21:26:58 +0000 > +++ b/client/mysql_upgrade.c 2015-02-09 01:48:28 +0000 > @@ -150,6 +151,14 @@ static struct my_option my_long_options[ > &opt_not_used, &opt_not_used, 0, GET_BOOL, NO_ARG, 1, 0, 0, 0, 0, 0}, > {"version", 'V', "Output version information and exit.", 0, 0, 0, > GET_NO_ARG, NO_ARG, 0, 0, 0, 0, 0, 0}, > + {"mysql-upgrade", 'y', > + "Skip automatic detection MySQL and assume that we upgrade it", > + &opt_mysql_upgrade, &opt_mysql_upgrade, 0, GET_BOOL, NO_ARG, > + 0, 0, 0, 0, 0, 0}, > + {"skip-mysql-upgrade", 'Y', > + "Skip view algorithm upgrade from MySQL", > + &opt_skip_mysql_upgrade, &opt_skip_mysql_upgrade, 0, GET_BOOL, NO_ARG, > + 0, 0, 0, 0, 0, 0}, I remember I've already commented on that in my previous review. Here you are, again: === Those two are very bad option names. First, $ mysql_upgrade --mysql-upgrade looks plain weird. $ mysql_upgrade --skip-mysql-upgrade is even worse. And what about $ mysql_upgrade --mysql-upgrade --skip-mysql-upgrade Second, "skip" is the prefix automatically recognized by my_getopt. Normally, --skip-xxx is the same as --xxx=0. But now you're introducing --skip-mysql-upgrade which is very different from --skip-mysql-upgrade. While simultaneously adding support for --skip-skip-mysql-upgrade. A better option would be --fix-view-algorithm = { AUTO | ALWAYS | NEVER } === In fact, I'd rather remove this option completely. You can simply add if (!is_mysql()) return; to run_mysqlcheck_views() > {"version-check", 'k', "Run this program only if its \'server version\' " > "matches the version of the server to which it's connecting, (enabled by " > "default); use --skip-version-check to avoid this check. Note: the \'server " > @@ -344,6 +353,14 @@ get_one_option(int optid, const struct m > case OPT_DEFAULT_AUTH: /* --default-auth */ > add_one_option(&conn_args, opt, argument); > break; > + case 'y': > + opt_mysql_upgrade= 1; > + add_option= FALSE; > + break; > + case 'Y': > + opt_skip_mysql_upgrade= 1; > + add_option= FALSE; This is a wrong way of implementing command-line options. Let my_getopt do its job and don't set values manually. > + break; > } > > if (add_option) > @@ -754,6 +771,23 @@ static int run_mysqlcheck_upgrade(void) > NULL); > } > > +static int run_mysqlcheck_views(void) > +{ > + if (!opt_mysql_upgrade) > + return 0; > + verbose("Phase 0: Fixing views"); Again, was in my previous review: === No "Phase 0" please, renumber all phases to start from 1. === > + print_conn_args("mysqlcheck"); > + return run_tool(mysqlcheck_path, > + NULL, /* Send output from mysqlcheck directly to screen */ > + "--no-defaults", > + ds_args.str, > + "--all-databases", > + "--mysql-upgrade", > + opt_verbose ? "--verbose": "", > + opt_silent ? "--silent": "", > + "2>&1", > + NULL); > +} > > static int run_mysqlcheck_fixnames(void) > { > === modified file 'client/mysqlcheck.c' > --- a/client/mysqlcheck.c 2014-02-17 10:00:51 +0000 > +++ b/client/mysqlcheck.c 2015-02-09 01:48:28 +0000 > @@ -196,6 +197,9 @@ static struct my_option my_long_options[ > NO_ARG, 0, 0, 0, 0, 0, 0}, > {"version", 'V', "Output version information and exit.", 0, 0, 0, GET_NO_ARG, > NO_ARG, 0, 0, 0, 0, 0, 0}, > + {"mysql-upgrade", 'y', > + "Fix view algorithm view field if it is not new MariaDB view.", > + 0, 0, 0, GET_NO_ARG, NO_ARG, 0, 0, 0, 0, 0, 0}, > {0, 0, 0, 0, 0, 0, GET_NO_ARG, NO_ARG, 0, 0, 0, 0, 0, 0} > }; > > @@ -332,7 +336,13 @@ get_one_option(int optid, const struct m > case 'v': > verbose++; > break; > - case 'V': print_version(); exit(0); > + case 'V': > + print_version(); exit(0); > + break; > + case 'y': > + what_to_do= DO_REPAIR; > + opt_mysql_upgrade= 1; > + break; See above. > case OPT_MYSQL_PROTOCOL: > opt_protocol= find_type_or_exit(argument, &sql_protocol_typelib, > opt->name); > @@ -595,7 +605,15 @@ static int process_all_tables_in_db(char > for (end = tables + 1; (row = mysql_fetch_row(res)) ;) > { > if ((num_columns == 2) && (strcmp(row[1], "VIEW") == 0)) > - continue; > + { > + if (!opt_mysql_upgrade) > + continue; > + } > + else > + { > + if (opt_mysql_upgrade) > + continue; Eh? If --mysql-upgrade is specified you skip all tables and only upgrade views? Then the option should absolutely be called --fix-view-algorithm > + } > > end= fix_table_name(end, row[0]); > *end++= ','; > @@ -756,10 +782,12 @@ static int handle_request_for_tables(cha > if (opt_upgrade) end = strmov(end, " FOR UPGRADE"); > break; > case DO_REPAIR: > - op= (opt_write_binlog) ? "REPAIR" : "REPAIR NO_WRITE_TO_BINLOG"; > + op= ((opt_write_binlog || opt_mysql_upgrade) ? > + "REPAIR" : "REPAIR NO_WRITE_TO_BINLOG"); Really? For view upgrades you want it to be written to binlog? This is a very questionable idea. > if (opt_quick) end = strmov(end, " QUICK"); > if (opt_extended) end = strmov(end, " EXTENDED"); > if (opt_frm) end = strmov(end, " USE_FRM"); > + if (opt_mysql_upgrade) end = strmov(end, " FROM MYSQL"); > break; > case DO_ANALYZE: > op= (opt_write_binlog) ? "ANALYZE" : "ANALYZE NO_WRITE_TO_BINLOG"; > @@ -776,14 +804,17 @@ static int handle_request_for_tables(cha > if (opt_all_in_1) > { > /* No backticks here as we added them before */ > - query_length= sprintf(query, "%s TABLE %s %s", op, tables, options); > + query_length= sprintf(query, "%s %s %s %s", op, > + (opt_mysql_upgrade ? "VIEW" : "TABLE"), this is wrong, because mysqlcheck --mysql-upgrade (or, really, mysqcheck --fix-view-algorithm=xxx) should check/repair *both* tables and views. > + tables, options); > table_name= tables; > } > else > { > char *ptr, *org; > > - org= ptr= strmov(strmov(query, op), " TABLE "); > + org= ptr= strmov(strmov(query, op), > + (opt_mysql_upgrade ? " VIEW " : " TABLE ")); > ptr= fix_table_name(ptr, tables); > strmake(table_name_buff, org, min((int) sizeof(table_name_buff)-1, > (int) (ptr - org))); > > === modified file 'mysql-test/r/view.result' > --- a/mysql-test/r/view.result 2014-12-21 18:23:28 +0000 > +++ b/mysql-test/r/view.result 2015-02-09 01:48:28 +0000 ... > +check view v1,v2,v3,v4,v5 FOR UPGRADE; > +Table Op Msg_type Msg_text > +test.v1 check Note view 'test.v1' has no field mariadb server in its .frm file > +test.v1 check status needs repair > +test.v2 check Note view 'test.v2' has no field mariadb server in its .frm file > +test.v2 check status needs repair > +test.v3 check Note view 'test.v3' has no field mariadb server in its .frm file > +test.v3 check status needs repair > +test.v4 check note View text checksum failed > +test.v5 check status OK > +repair view v1,v2,v3,v4,v5 FROM MYSQL; where's the test for "repair view" without FROM MYSQL ? > +Table Op Msg_type Msg_text > +test.v1 repair status view is repaired > +test.v2 repair status view is repaired > +test.v3 repair status view is repaired > +test.v4 repair status view is repaired > +test.v5 repair status OK > > === modified file 'sql/handler.h' > --- a/sql/handler.h 2015-01-13 18:28:03 +0000 > +++ b/sql/handler.h 2015-02-09 01:48:28 +0000 > @@ -42,6 +42,7 @@ > > // the following is for checking tables > > +#define HA_ADMIN_VIEW_REPAIR_IS_DONE 2 > #define HA_ADMIN_ALREADY_DONE 1 > #define HA_ADMIN_OK 0 > #define HA_ADMIN_NOT_IMPLEMENTED -1 > @@ -56,6 +57,7 @@ > #define HA_ADMIN_NEEDS_UPGRADE -10 > #define HA_ADMIN_NEEDS_ALTER -11 > #define HA_ADMIN_NEEDS_CHECK -12 > +#define HA_ADMIN_NEEDS_REPAIR -13 > > /* Bits in table_flags() to show what database can do */ > > === modified file 'sql/share/errmsg-utf8.txt' > --- a/sql/share/errmsg-utf8.txt 2014-05-27 06:45:01 +0000 > +++ b/sql/share/errmsg-utf8.txt 2015-02-09 01:48:28 +0000 > @@ -6565,3 +6565,9 @@ ER_QUERY_EXCEEDED_ROWS_EXAMINED_LIMIT > ER_NO_SUCH_TABLE_IN_ENGINE 42S02 > eng "Table '%-.192s.%-.192s' doesn't exist in engine" > swe "Det finns ingen tabell som heter '%-.192s.%-.192s' i handlern" > +ER_NO_MARIADB_SERVER_FIELD > + eng "view '%-.192s.%-.192s' has no field mariadb server in its .frm file" First, it's "mariadb_version", not "mariadb_server". Second, I'm not sure such a specific condition needs a dedicated error code. And last - we absolutely cannot add new error messages in 5.5, because 10.0 is already GA. I've already written that in my previous review. > +ER_VIEW_REPAIR_IS_DONE > + eng "view is repaired" > +ER_NEEDS_REPAIR > + eng "needs repair" > > === modified file 'sql/sql_admin.cc' > --- a/sql/sql_admin.cc 2014-05-03 16:12:17 +0000 > +++ b/sql/sql_admin.cc 2015-02-09 01:48:28 +0000 > @@ -867,6 +879,22 @@ send_result_message: > fatal_error=1; > break; > } > + case HA_ADMIN_VIEW_REPAIR_IS_DONE: Why did you need that, what's wrong with HA_ADMIN_OK? Btw, I've already written that in my previous review. > + { > + protocol->store(STRING_WITH_LEN("status"), system_charset_info); > + protocol->store(ER(ER_VIEW_REPAIR_IS_DONE), > + strlen(ER(ER_VIEW_REPAIR_IS_DONE)), > + system_charset_info); > + break; > + } > + case HA_ADMIN_NEEDS_REPAIR: Why did you need that, what's wrong with HA_ADMIN_NEEDS_UPGRADE? > + { > + protocol->store(STRING_WITH_LEN("status"), system_charset_info); > + protocol->store(ER(ER_NEEDS_REPAIR), > + strlen(ER(ER_NEEDS_REPAIR)), > + system_charset_info); > + break; > + } > > default: // Probably HA_ADMIN_INTERNAL_ERROR > { > === modified file 'sql/sql_base.cc' > --- a/sql/sql_base.cc 2014-10-31 13:07:29 +0000 > +++ b/sql/sql_base.cc 2015-02-09 01:48:28 +0000 > @@ -3019,12 +3020,17 @@ bool open_table(THD *thd, TABLE_LIST *ta > else if (table_list->open_strategy == TABLE_LIST::OPEN_STUB) > DBUG_RETURN(FALSE); > > + second empty line? > retry_share: > > mysql_mutex_lock(&LOCK_open); > > if (!(share= get_table_share_with_discover(thd, table_list, key, > - key_length, OPEN_VIEW, > + key_length, > + (OPEN_VIEW | > + ((table_list->required_type == > + FRMTYPE_VIEW) ? > + OPEN_VIEW_ONLY : 0)), > &error, > hash_value))) > { > > === modified file 'sql/sql_table.cc' > --- a/sql/sql_table.cc 2015-01-14 11:10:13 +0000 > +++ b/sql/sql_table.cc 2015-02-09 01:48:28 +0000 > @@ -28,7 +28,7 @@ > #include "sql_base.h" // open_table_uncached, lock_table_names > #include "lock.h" // mysql_unlock_tables > #include "strfunc.h" // find_type2, find_set > -#include "sql_view.h" // view_checksum > +#include "sql_view.h" // view_check You've modified only the comment, not the actual C++ code that uses view_checksum in this file. So, either the comment is wrong (view_checksum is not used in this file) and should be removed, or the whole #include is unnecessary. > #include "sql_truncate.h" // regenerate_locked_table > #include "sql_partition.h" // mem_alloc_error, > // generate_partition_syntax, > > === modified file 'sql/sql_view.cc' > --- a/sql/sql_view.cc 2014-12-21 18:23:28 +0000 > +++ b/sql/sql_view.cc 2015-02-09 01:48:28 +0000 > @@ -729,6 +729,26 @@ err: > } > > > +static void make_view_filename(LEX_STRING *dir, char *dir_buff, > + size_t dir_buff_len, > + LEX_STRING *path, char *path_buff, > + size_t path_buff_len, > + LEX_STRING *file, > + TABLE_LIST *view) > +{ > + /* print file name */ > + dir->length= build_table_filename(dir_buff, dir_buff_len - 1, > + view->db, "", "", 0); > + dir->str= dir_buff; > + > + path->length= build_table_filename(path_buff, path_buff_len - 1, > + view->db, view->table_name, reg_ext, 0); > + path->str= path_buff; > + > + file->str= path->str + dir->length; > + file->length= path->length - dir->length; > +} > + > /* number of required parameters for making view */ > static const int required_view_parameters= 15; > > @@ -791,6 +811,81 @@ static File_option view_parameters[]= > static LEX_STRING view_file_type[]= {{(char*) STRING_WITH_LEN("VIEW") }}; > > > +int mariadb_fix_view(THD *thd, TABLE_LIST *view, bool wrong_checksum, > + bool swap_alg) > +{ > + char dir_buff[FN_REFLEN + 1], path_buff[FN_REFLEN + 1]; > + LEX_STRING dir, file, path; > + DBUG_ENTER("mariadb_fix_view"); > + > + if (view->algorithm == VIEW_ALGORITHM_UNDEFINED && > + !wrong_checksum && view->mariadb_version) > + DBUG_RETURN(HA_ADMIN_OK); Eh? Why do you care what view->algorithm is? It doesn't matter whether it's undefined, merge, or temptable - if mariadb_version is set, the algorithm is always correct. > + make_view_filename(&dir, dir_buff, sizeof(dir_buff), > + &path, path_buff, sizeof(path_buff), > + &file, view); > + /* init timestamp */ > + if (!view->timestamp.str) > + view->timestamp.str= view->timestamp_buffer; > + > + /* check old .frm */ > + { > + char path_buff[FN_REFLEN]; > + LEX_STRING path; > + File_parser *parser; > + > + path.str= path_buff; > + fn_format(path_buff, file.str, dir.str, "", MY_UNPACK_FILENAME); > + path.length= strlen(path_buff); > + if (access(path.str, F_OK)) > + DBUG_RETURN(HA_ADMIN_INVALID); > + > + if (!(parser= sql_parse_prepare(&path, thd->mem_root, 0))) > + DBUG_RETURN(HA_ADMIN_INTERNAL_ERROR); > + > + if (!parser->ok() || !is_equal(&view_type, parser->type())) > + DBUG_RETURN(HA_ADMIN_INVALID); > + } > + > + if (swap_alg && view->algorithm != VIEW_ALGORITHM_UNDEFINED) only if mariadb_version is not set. > + { > + DBUG_ASSERT(view->algorithm == VIEW_ALGORITHM_MERGE || > + view->algorithm == VIEW_ALGORITHM_TMPTABLE); > + if (view->algorithm == VIEW_ALGORITHM_MERGE) > + view->algorithm= VIEW_ALGORITHM_TMPTABLE; > + else > + view->algorithm= VIEW_ALGORITHM_MERGE; > + } > + if (wrong_checksum) > + { > + if (view->md5.length != 32) > + { > + if ((view->md5.str= (char *)thd->alloc(32 + 1)) == NULL) > + DBUG_RETURN(HA_ADMIN_FAILED); > + } > + view->calc_md5(view->md5.str); > + view->md5.length= 32; > + } > + view->mariadb_version= MYSQL_VERSION_ID; > + > + if (sql_create_definition_file(&dir, &file, view_file_type, > + (uchar*)view, view_parameters)) > + { > + sql_print_error("View '%-.192s'.'%-.192s': algorithm swap error.", > + view->db, view->table_name); > + DBUG_RETURN(HA_ADMIN_INTERNAL_ERROR); > + } > + sql_print_information("View '%-.192s'.'%-.192s': algorithm swapped to '%s'", > + view->db, view->table_name, > + (view->algorithm == VIEW_ALGORITHM_MERGE)? > + "MERGE":"TEMPTABLE"); > + > + > + DBUG_RETURN(HA_ADMIN_VIEW_REPAIR_IS_DONE); > +} > + > + > /* > Register VIEW (write .frm & process .frm's history backups) > > @@ -927,17 +1022,9 @@ static int mysql_register_view(THD *thd, > } > loop_out: > /* print file name */ > - dir.length= build_table_filename(dir_buff, sizeof(dir_buff) - 1, > - view->db, "", "", 0); > - dir.str= dir_buff; > - > - path.length= build_table_filename(path_buff, sizeof(path_buff) - 1, > - view->db, view->table_name, reg_ext, 0); > - path.str= path_buff; > - > - file.str= path.str + dir.length; > - file.length= path.length - dir.length; > - > + make_view_filename(&dir, dir_buff, sizeof(dir_buff), > + &path, path_buff, sizeof(path_buff), > + &file, view); > /* init timestamp */ > if (!view->timestamp.str) > view->timestamp.str= view->timestamp_buffer; > @@ -1941,6 +2028,63 @@ int view_checksum(THD *thd, TABLE_LIST * > HA_ADMIN_OK); > } > > +/** > + Check view > + > + @param thd thread handle > + @param view view for check > + @param check_opt check options > + > + @retval HA_ADMIN_OK OK > + @retval HA_ADMIN_NOT_IMPLEMENTED it is not VIEW > + @retval HA_ADMIN_WRONG_CHECKSUM check sum is wrong > +*/ > +int view_check(THD *thd, TABLE_LIST *view, HA_CHECK_OPT *check_opt) > +{ > + int res; > + DBUG_ENTER("view_check"); > + if ((res= view_checksum(thd, view)) != HA_ADMIN_OK) > + DBUG_RETURN(res); > + if (((check_opt->sql_flags & TT_FOR_UPGRADE) && > + !view->mariadb_version)) > + { > + push_warning_printf(thd, MYSQL_ERROR::WARN_LEVEL_NOTE, > + ER_NO_MARIADB_SERVER_FIELD, > + ER(ER_NO_MARIADB_SERVER_FIELD), > + view->db, > + view->table_name); > + DBUG_RETURN(HA_ADMIN_NEEDS_REPAIR); > + } > + DBUG_RETURN(HA_ADMIN_OK); > +} > + > + > +/** > + Repair view > + > + @param thd thread handle > + @param view view for check > + @param check_opt check options > + > + @retval HA_ADMIN_OK OK > + @retval HA_ADMIN_NOT_IMPLEMENTED it is not VIEW > + @retval HA_ADMIN_WRONG_CHECKSUM check sum is wrong > +*/ > + > +int view_repair(THD *thd, TABLE_LIST *view, HA_CHECK_OPT *check_opt) > +{ > + DBUG_ENTER("view_repair"); > + bool swap_alg= > + ((check_opt->sql_flags & TT_FROM_MYSQL) && > + (!view->mariadb_version)); > + bool wrong_checksum= view_checksum(thd, view); > + if (wrong_checksum || swap_alg) > + { > + DBUG_RETURN(mariadb_fix_view(thd, view, wrong_checksum, swap_alg)); First, don't call functions from DBUG_RETURN, this produces weird debug traces and somebody will have to fix this later. Always int res= mariadb_fix_view(thd, view, wrong_checksum, swap_alg); DBUG_RETURN(res); Alternatively, feel free to fix dbug macros to support function calls in DBUG_RETURN() :) Second, REPAIR VIEW viewname - without FROM MYSQL - should work too, it should add mariadb_version field without changing the algorithm. > + } > + DBUG_RETURN(HA_ADMIN_OK); > +} > + > /* > rename view > > === modified file 'sql/sql_yacc.yy' > --- a/sql/sql_yacc.yy 2014-11-08 18:54:42 +0000 > +++ b/sql/sql_yacc.yy 2015-02-09 01:48:28 +0000 > @@ -1145,6 +1145,7 @@ bool my_yyoverflow(short **a, YYSTYPE ** > %token MULTIPOINT > %token MULTIPOLYGON > %token MUTEX_SYM > +%token MYSQL_SYM > %token MYSQL_ERRNO_SYM > %token NAMES_SYM /* SQL-2003-N */ > %token NAME_SYM /* SQL-2003-N */ > @@ -7191,11 +7192,16 @@ opt_checksum_type: > ; > > repair: > - REPAIR opt_no_write_to_binlog table_or_tables > + REPAIR opt_no_write_to_binlog table_or_view > { > LEX *lex=Lex; > lex->sql_command = SQLCOM_REPAIR; > lex->no_write_to_binlog= $2; > + if (lex->no_write_to_binlog && lex->only_view) > + { > + my_parse_error(ER(ER_SYNTAX_ERROR)); > + MYSQL_YYABORT; Why? REPAIR NO_WRITE_TO_BINLOG VIEW makes perfect sense to me, why did you want to disallow it? > + } > lex->check_opt.init(); > lex->alter_info.reset(); > /* Will be overriden during execution. */ Regards, Sergei

4 8

[Maria-developers] MDEV-7912 review and related concerns
by Vicențiu Ciorbaru 24 Apr '15

24 Apr '15

Hi, I'd like to report my findings during the investigation of MDEV-7912, ask for a review on my patch and see your opinion on the status of the related code. The patch and all the issues that I am raising are for the 5.5 branch. With the attached patch, the server crash is eliminated. The problem we had was caused by casting a 64bit unsigned int to a 32bit unsigned int, representing a number of bytes to allocate. When the ranges were too big, it caused an overflow and we allocated too little space. On a related note, while looking at the code in the uniques.cc file, I've discovered a lot of potential issues, that are almost guaranteed to fail, if we actually do use more than 4 GB of memory. This holds true especially for Windows, because ulong is 32 bits for a 64 bit compilation, while on Linux, it is 64 bit. I've tried to convert most of the code that made use of uint or ulong or ulonglong for address space indexing (arrays, indices in arrays, memory sizes) to size_t, so that we escape the portability problem when compiling for 32 bit or 64 bit. Unfortunately, the task branches out into various other files that make use of those values. An example of a such a problem is in Unique::Unique constructor, line 90, where init_tree will get passed an overflown parameter (max_in_memory_size). It just so happens that there is a guard within the init_tree function that sets a minimum value, in case we do happen to pass an overflown value. This is however a lucky occurrence. Unique::get_use_cost can output a bad cost value, because get_merge_many_buffs_cost casts the same ulonglong memory size parameter to a ulong. There are many examples like this. I'd like to get your input on this issue, is this something we plan to fix in the near future? Should I embark on the journey of slowly fixing at least parts of it? Also, this issue seems to plague much of the code base, as I get a huge wall of similar size_t -> uint32 warnings across most files. Regards, Vicențiu

2 1

Re: [Maria-developers] [server] MDEV-7130 MASTER_POS_WAIT does not respect timeout when connection_name is specificed (#32)
by Kristian Nielsen 24 Apr '15

24 Apr '15

f4rnham <notifications(a)github.com> writes: > https://mariadb.atlassian.net/browse/MDEV-7130 > MASTER_POS_WAIT(log_name,log_pos,timeout,"connection_name") hangs, does not respect the timeout > > Changed also arg_count check for connection_name to prevent same bug if fifth argument is introduced in future > > Tests can be reduced to just one master_pos_wait select, it may be better since each this select waits one second timeout (except one for non-existing connection, line 23) Right, I cleaned up the test case and integrated it into the existing rpl.rpl_master_pos_wait test. Pushed into 10.0, thanks f4rnham! - Kristian.

1 0

[Maria-developers] TRUE vs true (was: [Commits] ac05f41: postreview fix)
by Oleksandr Byelkin 24 Apr '15

24 Apr '15

Hi, Jan! On 23.04.15 08:38, Jan Lindström wrote: > Sanja, > > Why you use TRUE/FALSE instead of true/false for bool ? Historical reasons. I was told to use TRUE/FALSE long time ago and nobody have told something else. We have no our code style doc so I use what I remember from old MySQL one. [skip]

4 6

[Maria-developers] MDEV-7649 wrong result when comparing utf8 column with an invalid literal
by Alexander Barkov 22 Apr '15

22 Apr '15

Hi Sergei, I tested comparison behaviour for various different situations. Please find the comment with the summary table in the end of: https://mariadb.atlassian.net/browse/MDEV-7649 Currently there are 5 possible reactions on bad bytes on comparison, depending on the collation, presence of an index, and character_set_connection value: - truncate on bad byte, compare only the well-formed prefix (#1) - treat bad bytes as '?' (#7 and #8) - empty set with no warning (#3) - empty set with a warning (#2) - error (#5 and #6) I'm in doubts what to dofor 10.1 and for 5.5. Please suggest. 1. For 10.1 or 10.2 I think it would be nice to make all these cases work exactly the same way. I am not sure which way would be the best. Any ideas? 2. For 5.5 we need to fix the bug with minimal changes. Do you agree? I suggest we fix only #1 and maybe #2. How we fix 5.5 now should probably depend on how we'll fix 10.1, to avoid changing behaviour many times. Please suggest. Thanks.

2 3

[Maria-developers] packaging code - mariadb and jemalloc.
by Daniel Black 22 Apr '15

22 Apr '15

Daniel, or anyone else. Where is the source/build code for the jemalloc build in the mariadb repo (specifically for ubuntu)? Where is the code that triggers making the rpm and deb packages that form the current mariadb and galera releases? My goal is to get these packages whitelisted by travis-ci so container builds of mariadb and running mtr on github pull rquests/branches can occur for everyone without a buildbot. -- -- Daniel Black, Engineer @ Open Query (http://openquery.com.au) Remote expertise & maintenance for MySQL/MariaDB server environments.

2 3

[Maria-developers] documentation
by Daniel Black 22 Apr '15

22 Apr '15

Seems there are a fair few variables and options changed in the course of development. These don't seem to make it to the documentation, especially pages like: https://mariadb.com/kb/en/mariadb/server-system-variables/ https://mariadb.com/kb/en/mariadb/server-status-variables https://mariadb.com/kb/en/mariadb/mysqld-options And their per subsection equivalents. Please make an effort to add/update these pages after a commit. After all, if a user can't see a change is made, or is referring to out of date documentation, then they aren't making use of the effort you put into making a change. If you made the change you are in the best position to describe it. git log sql/sys_vars.cc has quite a number of examples including 2ab49689c689b1c1daeb080ea568c268c5256952 cdb8e6ed38ce641a6cb90a4a93bc5ff15f3ba9f2 a978bdda1e2de35c79e432f026869e163657a263 -- -- Daniel Black, Engineer @ Open Query (http://openquery.com.au) Remote expertise & maintenance for MySQL/MariaDB server environments.

1 0

[Maria-developers] Coverity scanning
by Daniel Black 22 Apr '15

22 Apr '15

Hi, Seems MariaDB has an active Coverity project https://scan.coverity.com/projects/284 ... but it hasn't been looked at for a while. >From my list of bugs I suspect it would of found issues like: * MDEV-7977 * MDEV-7864 * MDEV-4980 In correcting these before release it would of benefited MariaDB by: * Not having SEGV / assertions faults occurring on user machines (even if they where test machines) * hours of time by Elena and me to get a test cases produced for obvious logic bugs -- -- Daniel Black, Engineer @ Open Query (http://openquery.com.au) Remote expertise & maintenance for MySQL/MariaDB server environments.

2 1

[Maria-developers] MDEV-8024 Remove excessive update_used_tables() calls
by Alexander Barkov 21 Apr '15

21 Apr '15

Hi Sergei, Please review a patch for MDEV-8024. It's needed to as a pre-requisite for "MDEV-7950 Item_func::type() takes 0.26% in OLTP RO" Thanks.

2 1

[Maria-developers] MDEV-8024 Remove excessive update_used_tables() calls
by Alexander Barkov 21 Apr '15

21 Apr '15

Hi Sergei, Please review a patch for MDEV-8024. It's needed to as a pre-requisite for "MDEV-7950 Item_func::type() takes 0.26% in OLTP RO" Thanks.

1 0

[Maria-developers] Need help merging MDEV-7387 from 10.0 to 10.1
by Kristian Nielsen 20 Apr '15

20 Apr '15

Hi Bar, Can you please merge the following patch 10.0 -> 10.1? commit bc902a2bfc46add0708896c07621e3707f66d95f Author: Alexander Barkov <bar(a)mariadb.org> Date: Fri Mar 13 16:12:54 2015 +0400 MDEV-7387 [PATCH] Alter table xxx CHARACTER SET utf8, CONVERT TO CHARACTER SET latin1 should fail Eg. run `git merge bc902a2bfc46add0708896c07621e3707f66d95f` in a 10.1 tree. Or alternatively, tell me how to resolve the below conflict in the merge, which is seen in sql/handler.h, so I can do the merge? (This conflict is blocking me from pushing MDEV-7249, MDEV-5289, MDEV-7668, and MDEV-7802 into 10.1...) - Kristian. ----------------------------------------------------------------------- sql/handler.h: <<<<<<< HEAD void init() { bzero(this, sizeof(*this)); } bool tmp_table() const { return options & HA_LEX_CREATE_TMP_TABLE; } void use_default_db_type(THD *thd) { db_type= tmp_table() ? ha_default_tmp_handlerton(thd) : ha_default_handlerton(thd); } }; /** This struct is passed to handler table routines, e.g. ha_create(). It does not include the "OR REPLACE" and "IF NOT EXISTS" parts, as these parts are handled on the SQL level and are not needed on the handler level. */ struct HA_CREATE_INFO: public Table_scope_and_contents_source_st, public Schema_specification_st { void init() { Table_scope_and_contents_source_st::init(); Schema_specification_st::init(); } }; /** This struct is passed to mysql_create_table() and similar creation functions, as well as to show_create_table(). */ struct Table_specification_st: public HA_CREATE_INFO, public DDL_options_st { // Deep initialization void init() { HA_CREATE_INFO::init(); DDL_options_st::init(); } void init(DDL_options_st::Options options) { HA_CREATE_INFO::init(); DDL_options_st::init(options); } /* Quick initialization, for parser. Most of the HA_CREATE_INFO is left uninitialized. It gets fully initialized in sql_yacc.yy, only when the parser scans a related keyword (e.g. CREATE, ALTER). */ void lex_start() { HA_CREATE_INFO::options= 0; DDL_options_st::init(); } ||||||| merged common ancestors bool tmp_table() { return options & HA_LEX_CREATE_TMP_TABLE; } ======= bool tmp_table() { return options & HA_LEX_CREATE_TMP_TABLE; } bool check_conflicting_charset_declarations(CHARSET_INFO *cs); bool add_table_option_default_charset(CHARSET_INFO *cs) { // cs can be NULL, e.g.: CREATE TABLE t1 (..) CHARACTER SET DEFAULT; if (check_conflicting_charset_declarations(cs)) return true; default_table_charset= cs; used_fields|= HA_CREATE_USED_DEFAULT_CHARSET; return false; } bool add_alter_list_item_convert_to_charset(CHARSET_INFO *cs) { /* cs cannot be NULL, as sql_yacc.yy translates CONVERT TO CHARACTER SET DEFAULT to CONVERT TO CHARACTER SET <character-set-of-the-current-database> TODO: Should't we postpone resolution of DEFAULT until the character set of the table owner database is loaded from its db.opt? */ DBUG_ASSERT(cs); if (check_conflicting_charset_declarations(cs)) return true; table_charset= default_table_charset= cs; used_fields|= (HA_CREATE_USED_CHARSET | HA_CREATE_USED_DEFAULT_CHARSET); return false; } >>>>>>> bc902a2bfc46add0708896c07621e3707f66d95f

3 6

Re: [Maria-developers] MariaDB 10 and MaxScale binlog router
by Kristian Nielsen 20 Apr '15

20 Apr '15

Massimiliano Pinto <massimiliano.pinto(a)mariadb.com> writes: First, Massimiliano, please use the maria-developers@ list for all discussions about development on mariadb and related projects. It is very important to keep all communication in the open and work against mariadb becoming a proprietary SkySQL project rather than an open community project. > we started working with MariaDB 10 master & slaves setup with MaxScale > binlog router (which was developed for MySQL 5.6 without GTID support) > Trying first MariaDB 10 setup we found this statement sent from Slave > to Master (maxsacle here) > > > (1) SET @mariadb_slave_capability=4 Check the MARIA_SLAVE_CAPABILITY_* #defines in sql/log_event.h. When connecting, a slave announces its capabilities (which is basically its version). This tells the master about which events are understood by that version of the slave. The master will check each event if it can be handled by the slave. If not, the event will be converted on-the-fly to something the older slave can understand. For example, a MariaDB 5.5 (or MySQL) slave does not understand GTID events. So a MariaDB 10.0 master will convert the GTID event into a BEGIN query event for slaves that have capability < MARIA_SLAVE_CAPABILITY_GTID (4). If you are accepting connections from a slave, you can do the same conversion. If you do not, then older version slaves might break if they receive events generated by newer version master. You can decide which approach makes sense for your project. If you are connecting to a master, then you need to announce your capability. If you do not, you will be interpreted by the master as having an old version, and newer events (like GTID) will be filtered out and not seen. > We soon got another one statement not handled: > > (2) SELECT binlog_gtid_pos('mst-bin.000001',310) This is used by the slave to obtain the correct GTID position corresponding to the position at which it is starting, when it is connecting in non-GTID mode. When the slave has this information, it becomes easy for the DBA to switch to a new master using GTID: STOP SLAVE; CHANGE MASTER TO master_host='new_master', master_use_gtid=slave_pos; START MASTER; This works even if the slave was not using GTID mode prior to the CHANGE MASTER, thanks to that SELECT binlog_gtid_pos(). > we are not handling it (just return the MaxScale default error message > to the connected client) Then the connected client/slave will assume this is an old master that does not support GTID. So the slave will not know the correct GTID position, and the easy CHANGE MASTER above may not be sufficient. Again, maybe that is fine for you, if you do not plan to support GTID. > We would like to hear your point of view, for (1) > @mariadb_slave_capability, how to handle it? What does it mean? Any > possible issues with different values? > Will this prevent us the use of Binlog router with MariADB 10? > > And for (2), is that ok just to ignore it? Would we get issues? I hope the above answers it (else ask again). Basically, you can ignore (1) and return error in (2), and things should work similar to MariaDB 5.5. But there will be reduced functionality compared to MariaDB 10.0/10.1. - Kristian.

3 5

[Maria-developers] Ubuntu Snappy and MariaDB packaging
by Otto Kekäläinen 19 Apr '15

19 Apr '15

Hello! Cool so see a discussion on the list about how to make the most optimal Docker packaging of MariaDB. I also stumbled across that Ubuntu has this completely novel OS structure called Snappy. Please post in this list if any of you start working on a Ubuntu Snappy version of MariaDB packaging.. Snappy packages are basically just a tar package of the binary and files plus a meta.yml description file: https://developer.ubuntu.com/en/snappy/tutorials/build-snaps/

1 0

[Maria-developers] Oracle April security notices and MariaDB
by Otto Kekäläinen 19 Apr '15

19 Apr '15

Hello! Debian security team is pressing me on the information about which recent Oracle CVEs affect MariaDB and which not. They default to assuming all affect so we need to prove otherwise. The Debian CVE tracker: https://security-tracker.debian.org/tracker/source-package/mariadb-10.0 None of these recent CVEs are listed at the MariaDB.org tracker: https://mariadb.com/kb/en/mariadb/security/ Could somebody please update the MariaDB.org CVE overview page? Thanks! Otto

1 0

Re: [Maria-developers] [Commits] eb47b22: MDEV-7820 Server crashes in in my_strcasecmp_utf8 on subquery in ORDER BY clause of GROUP_CONCAT
by Oleksandr Byelkin 15 Apr '15

15 Apr '15

On 15.04.15 15:26, vicentiu(a)mariadb.org wrote: > revision-id: eb47b226d268e7820a9153672798cf991a8fd0fa > parent(s): cc84ac3be41d9d6ac480d55449d5bf4e324cca10 > committer: Vicențiu Ciorbaru > branch nick: server > timestamp: 2015-04-15 16:23:43 +0300 > message: > > MDEV-7820 Server crashes in in my_strcasecmp_utf8 on subquery in ORDER BY clause of GROUP_CONCAT > > It is possible for Item_field to have a NULL field_name. This is true if > the Item_field is created based on a field in a temporary table that has > no name. It is thus necessary to do a null check before attempting a > strcmp. Thank you. OK to push. (do not forget to run full test suite) [skip]

1 0

[Maria-developers] CREATE/ALTER USER syntax in MySQL 5.7.5/5.7.6
by Peter Laursen 13 Apr '15

13 Apr '15

Please read MySQL 5.7 documentation on CREATE USER and ALTER USER syntax: https://dev.mysql.com/doc/refman/5.7/en/create-user.html https://dev.mysql.com/doc/refman/5.7/en/alter-user.html (note before/after version 5.7.5/5.7.6 changes) .. as well as this blog: http://mysqlblog.fivefarmers.com/2015/04/10/improved-alter-user-syntax-supp… "With changes made in MySQL Server 5.7.6, a better distinction is made between privilege-level attributes (those which are managed via GRANT and REVOKE statements) and account-level attributes (those managed using CREATE USER and ALTER USER statements). MySQL has a long history of confusing these – for example, requiring a GRANTcommand to set account resource limits or require SSL. " I think this is a very valid observation and thus also change. Does MariaDB have plans to implement the full CREATE/ALTER USER syntax of MySQL 5.7.5/5.7.6? -- Peter -- Webyog

3 2

[Maria-developers] MDEV-7660 - MySQL WL#6671 "Improve scalability by not using thr_lock.c locks for InnoDB tables"
by Sergey Vojtovich 09 Apr '15

09 Apr '15

Hi Jan, we're trying to adapt MySQL WL#6671 to MariaDB, which is expected to boost concurrent single table workloads. MySQL solution is fully based on MDL. It caused substential changes to MDL, which would be nice to avoid. We'd like to understand if we can achieve the same effect with InnoDB locking. According to WL#6671 InnoDB relied on THR_LOCK in following cases: 1. "To isolate HANDLER READ statements from LOCK TABLES WRITE statements." It is likely not a big problem if we decide to go non-MDL way. 2. "To isolate LOCK TABLES statements that lock tables for write implicitly or for read, both explicitly or implicitly, from concurrent DML statements." That's something that we're looking alternatives for. See below. 3. "Due to coding mistake thr_lock.c lock was necessary to isolate ALTER TABLE IMPORT/DISCARD TABLESPACE under LOCK TABLES from concurrent I_S queries/ open HANDLERs." I agree with this patch, it is merged and pending approval (MDEV-7894). 4. "To indicate that InnoDB tables don't support LOCK TABLES READ LOCAL by upgrading TL_READ lock requested by statement to TL_READ_NO_INSERT." It is likely not a big problem if we decide to go non-MDL way. The question is if InnoDB can do the following with reasonable effort: - LOCK TABLES ... READ - protect table data from concurrent writers - LOCK TABLES ... WRITE - pretect table data from concurrent readers and writers - honor SELECT HIGH_PRIORITY when there's pending LOCK TABLES ... WRITE (do we need this?) - honor LOCK TABLES ... WRITE LOW_PRIORITY when there're concurrent readers (do we need this?) Thanks, Sergey

1 0

[Maria-developers] MDEV-7540 commit.
by Delveri chick 09 Apr '15

09 Apr '15

diff --git a/sql/sql_show.cc b/sql/sql_show.cc index d6d1bd2..5576875 100644 --- a/sql/sql_show.cc +++ b/sql/sql_show.cc @@ -347,6 +347,119 @@ static int fill_spatial_ref_sys(THD *thd, TABLE_LIST *tables, COND *cond) table->field[3]->store(STRING_WITH_LEN(""), cs);/*SRTEXT*/ if (schema_table_store_record(thd, table)) goto exit; + + table->field[0]->store(0, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(0, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN(""), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2041, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2041, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"Abidjan 1987 / UTM zone 30N\",GEOGCS[\"Abidjan 1987\",DATUM[\"Abidjan_1987\",SPHEROID[\"Clarke 1880 (RGS)\",6378249.145,293.465,AUTHORITY[\"EPSG\",\"7012\"]],TOWGS84[-124.76,53,466.79,0,0,0,0],AUTHORITY[\"EPSG\",\"6143\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.01745329251994328,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4143\"]],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],PROJECTION[\"Transverse_Mercator\"],PARAMETER[\"latitude_of_origin\",0],PARAMETER[\"central_meridian\",-3],PARAMETER[\"scale_factor\",0.9996],PARAMETER[\"false_easting\",500000],PARAMETER[\"false_northing\",0],AUTHORITY[\"EPSG\",\"2041\"],AXIS[\"Easting\",EAST],AXIS[\"Northing\",NORTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2042, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2042, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"Locodjo 1965 / UTM zone 29N\",GEOGCS[\"Locodjo 1965\",DATUM[\"Locodjo_1965\",SPHEROID[\"Clarke 1880 (RGS)\",6378249.145,293.465,AUTHORITY[\"EPSG\",\"7012\"]],TOWGS84[-125,53,467,0,0,0,0],AUTHORITY[\"EPSG\",\"6142\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.01745329251994328,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4142\"]],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],PROJECTION[\"Transverse_Mercator\"],PARAMETER[\"latitude_of_origin\",0],PARAMETER[\"central_meridian\",-9],PARAMETER[\"scale_factor\",0.9996],PARAMETER[\"false_easting\",500000],PARAMETER[\"false_northing\",0],AUTHORITY[\"EPSG\",\"2042\"],AXIS[\"Easting\",EAST],AXIS[\"Northing\",NORTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2043, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2043, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"Abidjan 1987 / UTM zone 29N\",GEOGCS[\"Abidjan 1987\",DATUM[\"Abidjan_1987\",SPHEROID[\"Clarke 1880 (RGS)\",6378249.145,293.465,AUTHORITY[\"EPSG\",\"7012\"]],TOWGS84[-124.76,53,466.79,0,0,0,0],AUTHORITY[\"EPSG\",\"6143\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.01745329251994328,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4143\"]],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],PROJECTION[\"Transverse_Mercator\"],PARAMETER[\"latitude_of_origin\",0],PARAMETER[\"central_meridian\",-9],PARAMETER[\"scale_factor\",0.9996],PARAMETER[\"false_easting\",500000],PARAMETER[\"false_northing\",0],AUTHORITY[\"EPSG\",\"2043\"],AXIS[\"Easting\",EAST],AXIS[\"Northing\",NORTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2044, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2044, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"Hanoi 1972 / Gauss-Kruger zone 18\",GEOGCS[\"Hanoi 1972\",DATUM[\"Hanoi_1972\",SPHEROID[\"Krassowsky 1940\",6378245,298.3,AUTHORITY[\"EPSG\",\"7024\"]],TOWGS84[-17.51,-108.32,-62.39,0,0,0,0],AUTHORITY[\"EPSG\",\"6147\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.01745329251994328,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4147\"]],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],PROJECTION[\"Transverse_Mercator\"],PARAMETER[\"latitude_of_origin\",0],PARAMETER[\"central_meridian\",105],PARAMETER[\"scale_factor\",1],PARAMETER[\"false_easting\",18500000],PARAMETER[\"false_northing\",0],AUTHORITY[\"EPSG\",\"2044\"],AXIS[\"Y\",EAST],AXIS[\"X\",NORTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2045, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2045, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"Hanoi 1972 / Gauss-Kruger zone 19\",GEOGCS[\"Hanoi 1972\",DATUM[\"Hanoi_1972\",SPHEROID[\"Krassowsky 1940\",6378245,298.3,AUTHORITY[\"EPSG\",\"7024\"]],TOWGS84[-17.51,-108.32,-62.39,0,0,0,0],AUTHORITY[\"EPSG\",\"6147\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.01745329251994328,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4147\"]],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],PROJECTION[\"Transverse_Mercator\"],PARAMETER[\"latitude_of_origin\",0],PARAMETER[\"central_meridian\",111],PARAMETER[\"scale_factor\",1],PARAMETER[\"false_easting\",19500000],PARAMETER[\"false_northing\",0],AUTHORITY[\"EPSG\",\"2045\"],AXIS[\"Y\",EAST],AXIS[\"X\",NORTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2046, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2046, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"Hartebeesthoek94 / Lo15\",GEOGCS[\"Hartebeesthoek94\",DATUM[\"Hartebeesthoek94\",SPHEROID[\"WGS 84\",6378137,298.257223563,AUTHORITY[\"EPSG\",\"7030\"]],TOWGS84[0,0,0,0,0,0,0],AUTHORITY[\"EPSG\",\"6148\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.01745329251994328,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4148\"]],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],PROJECTION[\"Transverse_Mercator_South_Orientated\"],PARAMETER[\"latitude_of_origin\",0],PARAMETER[\"central_meridian\",15],PARAMETER[\"scale_factor\",1],PARAMETER[\"false_easting\",0],PARAMETER[\"false_northing\",0],AUTHORITY[\"EPSG\",\"2046\"],AXIS[\"Y\",WEST],AXIS[\"X\",SOUTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2047, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2047, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"Hartebeesthoek94 / Lo17\",GEOGCS[\"Hartebeesthoek94\",DATUM[\"Hartebeesthoek94\",SPHEROID[\"WGS 84\",6378137,298.257223563,AUTHORITY[\"EPSG\",\"7030\"]],TOWGS84[0,0,0,0,0,0,0],AUTHORITY[\"EPSG\",\"6148\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.01745329251994328,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4148\"]],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],PROJECTION[\"Transverse_Mercator_South_Orientated\"],PARAMETER[\"latitude_of_origin\",0],PARAMETER[\"central_meridian\",17],PARAMETER[\"scale_factor\",1],PARAMETER[\"false_easting\",0],PARAMETER[\"false_northing\",0],AUTHORITY[\"EPSG\",\"2047\"],AXIS[\"Y\",WEST],AXIS[\"X\",SOUTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2048, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2048, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"Hartebeesthoek94 / Lo19\",GEOGCS[\"Hartebeesthoek94\",DATUM[\"Hartebeesthoek94\",SPHEROID[\"WGS 84\",6378137,298.257223563,AUTHORITY[\"EPSG\",\"7030\"]],TOWGS84[0,0,0,0,0,0,0],AUTHORITY[\"EPSG\",\"6148\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.01745329251994328,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4148\"]],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],PROJECTION[\"Transverse_Mercator_South_Orientated\"],PARAMETER[\"latitude_of_origin\",0],PARAMETER[\"central_meridian\",19],PARAMETER[\"scale_factor\",1],PARAMETER[\"false_easting\",0],PARAMETER[\"false_northing\",0],AUTHORITY[\"EPSG\",\"2048\"],AXIS[\"Y\",WEST],AXIS[\"X\",SOUTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2049, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2049, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"Hartebeesthoek94 / Lo21\",GEOGCS[\"Hartebeesthoek94\",DATUM[\"Hartebeesthoek94\",SPHEROID[\"WGS 84\",6378137,298.257223563,AUTHORITY[\"EPSG\",\"7030\"]],TOWGS84[0,0,0,0,0,0,0],AUTHORITY[\"EPSG\",\"6148\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.01745329251994328,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4148\"]],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],PROJECTION[\"Transverse_Mercator_South_Orientated\"],PARAMETER[\"latitude_of_origin\",0],PARAMETER[\"central_meridian\",21],PARAMETER[\"scale_factor\",1],PARAMETER[\"false_easting\",0],PARAMETER[\"false_northing\",0],AUTHORITY[\"EPSG\",\"2049\"],AXIS[\"Y\",WEST],AXIS[\"X\",SOUTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2050, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2050, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"Hartebeesthoek94 / Lo23\",GEOGCS[\"Hartebeesthoek94\",DATUM[\"Hartebeesthoek94\",SPHEROID[\"WGS 84\",6378137,298.257223563,AUTHORITY[\"EPSG\",\"7030\"]],TOWGS84[0,0,0,0,0,0,0],AUTHORITY[\"EPSG\",\"6148\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.01745329251994328,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4148\"]],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],PROJECTION[\"Transverse_Mercator_South_Orientated\"],PARAMETER[\"latitude_of_origin\",0],PARAMETER[\"central_meridian\",23],PARAMETER[\"scale_factor\",1],PARAMETER[\"false_easting\",0],PARAMETER[\"false_northing\",0],AUTHORITY[\"EPSG\",\"2050\"],AXIS[\"Y\",WEST],AXIS[\"X\",SOUTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2051, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2051, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"Hartebeesthoek94 / Lo25\",GEOGCS[\"Hartebeesthoek94\",DATUM[\"Hartebeesthoek94\",SPHEROID[\"WGS 84\",6378137,298.257223563,AUTHORITY[\"EPSG\",\"7030\"]],TOWGS84[0,0,0,0,0,0,0],AUTHORITY[\"EPSG\",\"6148\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.01745329251994328,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4148\"]],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],PROJECTION[\"Transverse_Mercator_South_Orientated\"],PARAMETER[\"latitude_of_origin\",0],PARAMETER[\"central_meridian\",25],PARAMETER[\"scale_factor\",1],PARAMETER[\"false_easting\",0],PARAMETER[\"false_northing\",0],AUTHORITY[\"EPSG\",\"2051\"],AXIS[\"Y\",WEST],AXIS[\"X\",SOUTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2052, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2052, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"Hartebeesthoek94 / Lo27\",GEOGCS[\"Hartebeesthoek94\",DATUM[\"Hartebeesthoek94\",SPHEROID[\"WGS 84\",6378137,298.257223563,AUTHORITY[\"EPSG\",\"7030\"]],TOWGS84[0,0,0,0,0,0,0],AUTHORITY[\"EPSG\",\"6148\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.01745329251994328,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4148\"]],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],PROJECTION[\"Transverse_Mercator_South_Orientated\"],PARAMETER[\"latitude_of_origin\",0],PARAMETER[\"central_meridian\",27],PARAMETER[\"scale_factor\",1],PARAMETER[\"false_easting\",0],PARAMETER[\"false_northing\",0],AUTHORITY[\"EPSG\",\"2052\"],AXIS[\"Y\",WEST],AXIS[\"X\",SOUTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2053, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2053, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"Hartebeesthoek94 / Lo29\",GEOGCS[\"Hartebeesthoek94\",DATUM[\"Hartebeesthoek94\",SPHEROID[\"WGS 84\",6378137,298.257223563,AUTHORITY[\"EPSG\",\"7030\"]],TOWGS84[0,0,0,0,0,0,0],AUTHORITY[\"EPSG\",\"6148\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.01745329251994328,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4148\"]],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],PROJECTION[\"Transverse_Mercator_South_Orientated\"],PARAMETER[\"latitude_of_origin\",0],PARAMETER[\"central_meridian\",29],PARAMETER[\"scale_factor\",1],PARAMETER[\"false_easting\",0],PARAMETER[\"false_northing\",0],AUTHORITY[\"EPSG\",\"2053\"],AXIS[\"Y\",WEST],AXIS[\"X\",SOUTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2054, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2054, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"Hartebeesthoek94 / Lo31\",GEOGCS[\"Hartebeesthoek94\",DATUM[\"Hartebeesthoek94\",SPHEROID[\"WGS 84\",6378137,298.257223563,AUTHORITY[\"EPSG\",\"7030\"]],TOWGS84[0,0,0,0,0,0,0],AUTHORITY[\"EPSG\",\"6148\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.01745329251994328,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4148\"]],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],PROJECTION[\"Transverse_Mercator_South_Orientated\"],PARAMETER[\"latitude_of_origin\",0],PARAMETER[\"central_meridian\",31],PARAMETER[\"scale_factor\",1],PARAMETER[\"false_easting\",0],PARAMETER[\"false_northing\",0],AUTHORITY[\"EPSG\",\"2054\"],AXIS[\"Y\",WEST],AXIS[\"X\",SOUTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2055, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2055, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"Hartebeesthoek94 / Lo33\",GEOGCS[\"Hartebeesthoek94\",DATUM[\"Hartebeesthoek94\",SPHEROID[\"WGS 84\",6378137,298.257223563,AUTHORITY[\"EPSG\",\"7030\"]],TOWGS84[0,0,0,0,0,0,0],AUTHORITY[\"EPSG\",\"6148\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.01745329251994328,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4148\"]],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],PROJECTION[\"Transverse_Mercator_South_Orientated\"],PARAMETER[\"latitude_of_origin\",0],PARAMETER[\"central_meridian\",33],PARAMETER[\"scale_factor\",1],PARAMETER[\"false_easting\",0],PARAMETER[\"false_northing\",0],AUTHORITY[\"EPSG\",\"2055\"],AXIS[\"Y\",WEST],AXIS[\"X\",SOUTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + result= 0; exit:

1 0

[Maria-developers] MDEV-5536 / MDEV-427 / MDEV-5713 - systemd socket activation, notify
by Daniel Black 07 Apr '15

07 Apr '15

This adds systemd socket activation (listening file descriptors passed on on startup), and systemd notification parts to the core mysql and bits of innodb that may take a bit of time. Scripts to convert from my.cnf settings to a socket activation file are included. support-files/mariadb.service has been included which replaces every bit of mysqld_safe with prebuilt systemd functionality. Systemd can be enabled at compile time and still will support non-system. Even when systemd compile time options are enabled, mysqld will still run in an existing init script or like an existing service. For review: https://github.com/MariaDB/server/pull/26 Thanks Jan for the innodb side of the review so far. -- -- Daniel Black, Engineer @ Open Query (http://openquery.com.au) Remote expertise & maintenance for MySQL/MariaDB server environments.

1 0

[Maria-developers] spelling and code comment fixes for review
by Daniel Black 07 Apr '15

07 Apr '15

No code changes. For review: Fixing comments https://github.com/MariaDB/server/pull/24 And spelling in comments: https://github.com/MariaDB/server/pull/36 -- -- Daniel Black, Engineer @ Open Query (http://openquery.com.au) Remote expertise & maintenance for MySQL/MariaDB server environments.

1 0

[Maria-developers] MDEV-7781 Allow {un, }install plugins during bootstrap/skip-grant-tables
by Daniel Black 07 Apr '15

07 Apr '15

For review: https://github.com/MariaDB/server/pull/29 Simple one, this allows plugins to be removed/installed during bootstrap (opt_noacl=true) but still enforces access control in normal operation. -- -- Daniel Black, Engineer @ Open Query (http://openquery.com.au) Remote expertise & maintenance for MySQL/MariaDB server environments.

1 0

Re: [Maria-developers] ae2cb44: MDEV-7922 - ERROR 1939 (HY000): Engine PERFORMANCE_SCHEMA failed to discover
by Sergei Golubchik 07 Apr '15

07 Apr '15

Hi, Sergey! On Apr 07, svoj(a)mariadb.org wrote: > revision-id: ae2cb445ea2a51657f1d2581db77db2ecbf70bc1 > parent(s): 7d9e94e2e6c32b906d1d8469e58bc04b8da1c121 > committer: Sergey Vojtovich > branch nick: mariadb > timestamp: 2015-04-07 14:59:25 +0400 > message: > > MDEV-7922 - ERROR 1939 (HY000): Engine PERFORMANCE_SCHEMA failed to discover > table > > Performance schema discovery fails if connection has no active database set. > > This happened due to restriction in SQL parser: table name with no database name > is ambiguous in such case. > > Now PFS discovery statements specify database name explicitely. No, I don't like this solution. This is an issue in discovery, there is nothing wrong with how P_S uses it. I'd rather fix it in TABLE_SHARE::init_from_sql_statement_string() by temporarily setting THD::db to this->db (which is a db where the table will be created). Regards, Sergei

2 3

[Maria-developers] Not possible to use FIFO file as a general_log file
by Honza Horak 07 Apr '15

07 Apr '15

This is just a re-sending the issue reported already at https://mariadb.atlassian.net/browse/MDEV-6870 to make it more visible, since there is no respond yet. There was an issue [1] in mysql and mariadb prior to 5.5.40 if socket or fifo files were used as general_log and slow query log files. That issue was fixed by a commit in [2], so currently it is not possible to use other than regular files as log files. I'd like to bring this issue to attention of mariadb developers as well, since our customer is asking for specifically this functionality – to be able to use FIFO file as general_log file. Is it really unacceptable? [1] http://bugs.mysql.com/bug.php?id=67088 [2] http://bazaar.launchpad.net/~mysql/mysql-server/5.5/revision/4688

2 1

[Maria-developers] Passwordless MariaDB root login with auth_socket in Debian continues..
by Otto Kekäläinen 06 Apr '15

06 Apr '15

Hello Daniel B. and others, I decided to write to you via the mailing list so that we are inclusive if somebody wants to join our effort to create a solid passwordless (Debian/Ubuntu) packaging. Thanks Daniel for all the pull requests you've made. Unfortunately there are still some bugs. Now it ends with 'ERROR 1524 (HY000): Plugin 'unix_socket' is not loaded' I have built packages from the latest commit at my Debian packaging repo https://github.com/ottok/mariadb-10.0/commits/master (bac031c9796ebf08c651bc863bda97fcf2cce9c4) so that all of your pull requests regarding this are included. The dev repo addess is 'deb http://labs.seravo.fi/~otto/mariadb-repo/ mariadb-10.0-sid-amd64/' in case you forgot (see https://wiki.debian.org/Teams/MySQL/MariaDB) $ apt-get purge 'mysql-*' 'mariadb-*'; rm -rf /etc/mysql /var/lib/mysql; apt-get clean; apt-get update [..] $ apt-get install mariadb-server [..] Fetched 12.6 MB in 1min 34s (133 kB/s) Preconfiguring packages ... Selecting previously unselected package mysql-common. Preparing to unpack .../mysql-common_5.5.42-1_all.deb ... Unpacking mysql-common (5.5.42-1) ... Selecting previously unselected package libmysqlclient18:amd64. Preparing to unpack .../libmysqlclient18_5.5.42-1_amd64.deb ... Unpacking libmysqlclient18:amd64 (5.5.42-1) ... Selecting previously unselected package mariadb-common. Preparing to unpack .../mariadb-common_10.0.17-1~exp2_all.deb ... Unpacking mariadb-common (10.0.17-1~exp2) ... Selecting previously unselected package mariadb-client-core-10.0. Preparing to unpack .../mariadb-client-core-10.0_10.0.17-1~exp2_amd64.deb ... Unpacking mariadb-client-core-10.0 (10.0.17-1~exp2) ... Selecting previously unselected package mariadb-client-10.0. Preparing to unpack .../mariadb-client-10.0_10.0.17-1~exp2_amd64.deb ... Unpacking mariadb-client-10.0 (10.0.17-1~exp2) ... Selecting previously unselected package mariadb-server-core-10.0. Preparing to unpack .../mariadb-server-core-10.0_10.0.17-1~exp2_amd64.deb ... Unpacking mariadb-server-core-10.0 (10.0.17-1~exp2) ... Processing triggers for man-db (2.7.0.2-3) ... Setting up mysql-common (5.5.42-1) ... Setting up mariadb-common (10.0.17-1~exp2) ... Notice: configure-symlinks trigger could not be called. Please manually make /etc/mysql/my.cnf a symlink that points to mariadb.cnf. Selecting previously unselected package mariadb-server-10.0. Preparing to unpack .../mariadb-server-10.0_10.0.17-1~exp2_amd64.deb ... Unpacking mariadb-server-10.0 (10.0.17-1~exp2) ... Selecting previously unselected package libdbd-mysql-perl. Preparing to unpack .../libdbd-mysql-perl_4.028-2+b1_amd64.deb ... Unpacking libdbd-mysql-perl (4.028-2+b1) ... Selecting previously unselected package mariadb-server. Preparing to unpack .../mariadb-server_10.0.17-1~exp2_all.deb ... Unpacking mariadb-server (10.0.17-1~exp2) ... Processing triggers for man-db (2.7.0.2-3) ... Setting up libmysqlclient18:amd64 (5.5.42-1) ... Setting up mariadb-client-core-10.0 (10.0.17-1~exp2) ... Setting up mariadb-client-10.0 (10.0.17-1~exp2) ... Setting up mariadb-server-core-10.0 (10.0.17-1~exp2) ... Setting up mariadb-server-10.0 (10.0.17-1~exp2) ... + . /usr/share/debconf/confmodule ++ '[' '!' '' ']' ++ PERL_DL_NONLAZY=1 ++ export PERL_DL_NONLAZY ++ '[' '' ']' ++ exec /usr/share/debconf/frontend /var/lib/dpkg/info/mariadb-server-10.0.postinst configure '' + . /usr/share/debconf/confmodule ++ '[' '!' 1 ']' ++ '[' -z '' ']' ++ exec ++ '[' '' ']' ++ exec ++ DEBCONF_REDIR=1 ++ export DEBCONF_REDIR + '[' -n '' ']' + export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/sbin:/usr/sbin:/bin:/usr/bin + PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/sbin:/usr/sbin:/bin:/usr/bin + ERR_LOGGER='logger -p daemon.err -t mysqld_safe -i' + set -o pipefail + MYSQL_BOOTSTRAP='/usr/sbin/mysqld --bootstrap --user=mysql --disable-log-bin --skip-grant-tables --default-storage-engine=myisam --plugin-load-add=auth_socket' + set +e + invoke stop + '[' -x /usr/sbin/invoke-rc.d ']' + invoke-rc.d mysql stop Stopping MariaDB database server: mysqld. + set -e + case "$1" in + mysql_statedir=/usr/share/mysql + mysql_datadir=/var/lib/mysql + mysql_logdir=/var/log/mysql + mysql_rundir=/var/run/mysqld + mysql_cfgdir=/etc/mysql + mysql_upgradedir=/var/lib/mysql-upgrade + '[' '!' -d /usr/share/mysql -a '!' -L /usr/share/mysql ']' + '[' '!' -d /var/lib/mysql -a '!' -L /var/lib/mysql ']' + '[' '!' -d /var/log/mysql -a '!' -L /var/log/mysql ']' + set +e + chown -R 0:0 /usr/share/mysql + chown -R mysql /var/lib/mysql + chown -R mysql:adm /var/log/mysql + chmod 2750 /var/log/mysql + set -e + db_set mariadb-server/postrm_remove_database false + _db_cmd 'SET mariadb-server/postrm_remove_database' false + _db_internal_IFS=' ' + IFS=' ' + printf '%s\n' 'SET mariadb-server/postrm_remove_database false' + IFS=' ' + IFS=' ' + read -r _db_internal_line + RET='10 mariadb-server/postrm_remove_database doesn'\''t exist' + case ${_db_internal_line%%[ ]*} in + return 10 + true + rm -f '/var/lib/mysql/debian-*.flag' + touch /var/lib/mysql/debian-10.0.flag + set +e + logger -p daemon.err -t mysqld_safe -i + bash /usr/bin/mysql_install_db --rpm --cross-bootstrap --user=mysql --disable-log-bin + set -e + dc=/etc/mysql/debian.cnf + '[' '!' -e /etc/mysql/debian.cnf -o 'grep -sq "^[^#]*user[^#=]*=[^#=]*debian-sys-maint" /etc/mysql/debian.cnf' ']' + '[' '!' -d /etc/mysql ']' + '[' -e /etc/mysql/debian.cnf ']' + oldconf= + umask 066 + cat /dev/null + umask 022 + echo '# Automatically generated for Debian scripts. DO NOT TOUCH!' + echo '[client]' + echo 'host = localhost' + echo 'user = root' + echo 'password = ' + echo 'socket = /var/run/mysqld/mysqld.sock' + echo '[mysql_upgrade]' + echo 'host = localhost' + echo 'user = root' + echo 'password = ' + echo 'socket = /var/run/mysqld/mysqld.sock' + echo 'basedir = /usr' + chown 0:0 /etc/mysql/debian.cnf + chmod 0600 /etc/mysql/debian.cnf ++ /bin/echo -e 'USE mysql;\n' 'SET sql_log_bin=0;\n' 'ALTER TABLE user CHANGE Password Password char(41) character set latin1 collate latin1_bin DEFAULT '\'''\'' NOT NULL;' + password_column_fix_query='USE mysql; SET sql_log_bin=0; ALTER TABLE user CHANGE Password Password char(41) character set latin1 collate latin1_bin DEFAULT '\'''\'' NOT NULL;' + echo 'USE mysql; SET sql_log_bin=0; ALTER TABLE user CHANGE Password Password char(41) character set latin1 collate latin1_bin DEFAULT '\'''\'' NOT NULL;' + logger -p daemon.err -t mysqld_safe -i + /usr/sbin/mysqld --bootstrap --user=mysql --disable-log-bin --skip-grant-tables --default-storage-engine=myisam --plugin-load-add=auth_socket ++ /bin/echo -e 'SET sql_mode='\'''\'', sql_log_bin=0;\n' 'DELETE FROM mysql.user WHERE user='\''debian-sys-maint'\'' AND host='\''localhost'\'';' + replace_query='SET sql_mode='\'''\'', sql_log_bin=0; DELETE FROM mysql.user WHERE user='\''debian-sys-maint'\'' AND host='\''localhost'\'';' ++ /bin/echo -e 'USE mysql;\n' 'SET sql_log_bin=0;\n' 'CREATE TABLE IF NOT EXISTS plugin (name char(64) COLLATE utf8_bin NOT NULL DEFAULT '\'''\'', ' ' dl char(128) COLLATE utf8_bin NOT NULL DEFAULT '\'''\'', ' ' PRIMARY KEY (name)) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_bin COMMENT='\''MySQL plugins'\'';\n' 'INSTALL PLUGIN unix_socket SONAME '\''auth_socket'\'';\n' + install_plugins='USE mysql; SET sql_log_bin=0; CREATE TABLE IF NOT EXISTS plugin (name char(64) COLLATE utf8_bin NOT NULL DEFAULT '\'''\'', dl char(128) COLLATE utf8_bin NOT NULL DEFAULT '\'''\'', PRIMARY KEY (name)) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_bin COMMENT='\''MySQL plugins'\''; INSTALL PLUGIN unix_socket SONAME '\''auth_socket'\'';' + set +e + echo 'USE mysql; SET sql_log_bin=0; CREATE TABLE IF NOT EXISTS plugin (name char(64) COLLATE utf8_bin NOT NULL DEFAULT '\'''\'', dl char(128) COLLATE utf8_bin NOT NULL DEFAULT '\'''\'', PRIMARY KEY (name)) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_bin COMMENT='\''MySQL plugins'\''; INSTALL PLUGIN unix_socket SONAME '\''auth_socket'\'';' + logger -p daemon.err -t mysqld_safe -i + /usr/sbin/mysqld --bootstrap --user=mysql --disable-log-bin --skip-grant-tables --default-storage-engine=myisam --plugin-load-add=auth_socket + set -e + set_mysql_rootpw ++ mktemp + tfile=/tmp/tmp.RyQCYdtKwf + '[' '!' -f /tmp/tmp.RyQCYdtKwf ']' + cat + '[' '' = online ']' + /usr/sbin/mysqld --bootstrap --user=mysql --disable-log-bin --skip-grant-tables --default-storage-engine=myisam --plugin-load-add=auth_socket + retval=0 + rm -f /tmp/tmp.RyQCYdtKwf + return 0 + '[' -e ']' + rm -f + set +e + echo 'SET sql_mode='\'''\'', sql_log_bin=0; DELETE FROM mysql.user WHERE user='\''debian-sys-maint'\'' AND host='\''localhost'\'';' + logger -p daemon.err -t mysqld_safe -i + /usr/sbin/mysqld --bootstrap --user=mysql --disable-log-bin --skip-grant-tables --default-storage-engine=myisam --plugin-load-add=auth_socket + set -e + '[' configure = configure ']' + test_passwordless_root_login + mysql --no-defaults -u root -h localhost + '[' '' = yes ']' + db_stop + echo STOP + '[' -x /etc/init.d/mysql ']' + update-rc.d mysql defaults 19 21 + invoke-rc.d mysql start Starting MariaDB database server: mysqld. + exit 0 ERROR 1524 (HY000): Plugin 'unix_socket' is not loaded Setting up libdbd-mysql-perl (4.028-2+b1) ... Setting up mariadb-server (10.0.17-1~exp2) ... Processing triggers for libc-bin (2.19-13) ... $ mysql -u root ERROR 1524 (HY000): Plugin 'unix_socket' is not loaded /var/log/mysql/error.log is pasted at http://paste.debian.net/161259/ Please figure out what is still missing and open new pull requests to fix it :) The mariadb-10.0.17-exp1 at the moment in Debian experimental is broken due to inclusion of an early version of this change, so I hope we could get it completed soon so that I don't have to revert the whole passwordless root account feature (which has been requested by many). - Otto

5 19

[Maria-developers] [MDEV-7922] pfs is not cahce definition info at db start (ver 10.1.13)
by 유승훈 06 Apr '15

06 Apr '15

Hello, i did some test about pfs on Mariadb 10.1.13 ver (ubuntu 14.04) So I just add some issue about pfs in mariadb ver 10.1.13 like below jira link simple context : 1) db start 2) just query about pfs => error => but "use pfs" then sql run => no error https://mariadb.atlassian.net/browse/MDEV-7922 Any body some idea about this, reply mail or write on jira. Thanks for your time and attention. Best Regards! SeungHoon. _____ 상기 메시지와 첨부화일 내에는 비밀정보가 포함되어 있을 수 있으며, 지정된 수신자에 한하여 조회 및 사용될 수 있습니다. 만약 송신자의 실수로 인하여 상기 메시지를 수신하였다면, 송신자에게 메시지를 반송해 주시고, 원본 메시지와 모든 사본을 폐기해 주시기 바랍니다. 상기 메시지의 전체 또는 일부에 대해 무단 열람, 사용, 공개, 배포하는 것은 금지되어 있습니다.(주)LG CNS. This message and its attachments may contain confidential information, and they are intended to be viewed or used by only the individuals specified in the message. If you have received this message in an error from the sender, please contact the sender immediately to notify the error and delete all of the message and its copies. It is prohibited to view, use, make public and/or distribute part or whole of this message without written permission.

2 2

Re: [Maria-developers] [Commits] Rev 4428: MDEV-6916: Upgrade from MySQL to MariaDB breaks already created views in file:///home/bell/maria/bzr/work-maria-5.5-MDEV-6916-repair_view/
by Daniel Black 05 Apr '15

05 Apr '15

Nice work Oleksandr, Be good to see this committed. I really like the no options bit of mysql_upgrade and the use of is_mysql function. ----- Original Message ----- > At file:///home/bell/maria/bzr/work-maria-5.5-MDEV-6916-repair_view/ > > ------------------------------------------------------------ > revno: 4428 > revision-id: sanja(a)askmonty.org-20150213124726-9rn55t3qqx7yzy9q > parent: jplindst(a)mariadb.org-20150210121723-1gxyej083cf1nv8h > committer: sanja(a)askmonty.org > branch nick: work-maria-5.5-MDEV-6916-repair_view > timestamp: Fri 2015-02-13 13:47:26 +0100 > message: > MDEV-6916: Upgrade from MySQL to MariaDB breaks already created views > > CHECK/REPAIR commands and mysql_upgrade support for upgrade from MySQL > server support. > > > [Text File:patch-4428.diff] > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits -- -- Daniel Black, Engineer @ Open Query (http://openquery.com.au) Remote expertise & maintenance for MySQL/MariaDB server environments.

1 0

[Maria-developers] MDEV-7540 commits.
by Delveri chick 03 Apr '15

03 Apr '15

diff --git a/libmysql/errmsg.c b/libmysql/errmsg.c index 84884f9..abcf3f3 100644 --- a/libmysql/errmsg.c +++ b/libmysql/errmsg.c @@ -31,7 +31,7 @@ "Can't connect to MySQL server on '%-.100s' (%M)", "Can't create TCP/IP socket (%M)", "Unknown MySQL server host '%-.100s' (%d)", - "MySQL server has gone away", + "MySQL server has gone away due to wait_timeout limit reached", "Protocol mismatch; server version = %d, client version = %d", "MySQL client ran out of memory", "Wrong host info", diff --git a/sql/sql_show.cc b/sql/sql_show.cc index c6d73bf..9b2d894 100644 --- a/sql/sql_show.cc +++ b/sql/sql_show.cc @@ -661,6 +661,166 @@ static int fill_spatial_ref_sys(THD *thd, TABLE_LIST *tables, COND *cond) if (schema_table_store_record(thd, table)) goto exit; + table->field[0]->store(2018, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2018, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"NAD27(76) / MTM zone 9\",GEOGCS[\"NAD27(76)\",DATUM[\"North_American_Datum_1927_1976\",SPHEROID[\"Clarke 1866\",6378206.4,294.9786982138982,AUTHORITY[\"EPSG\",\"7008\"]],AUTHORITY[\"EPSG\",\"6608\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.01745329251994328,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4608\"]],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],PROJECTION[\"Transverse_Mercator\"],PARAMETER[\"latitude_of_origin\",0],PARAMETER[\"central_meridian\",-76.5],PARAMETER[\"scale_factor\",0.9999],PARAMETER[\"false_easting\",304800],PARAMETER[\"false_northing\",0],AUTHORITY[\"EPSG\",\"2018\"],AXIS[\"X\",EAST],AXIS[\"Y\",NORTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2019, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2019, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"NAD27(76) / MTM zone 10\",GEOGCS[\"NAD27(76)\",DATUM[\"North_American_Datum_1927_1976\",SPHEROID[\"Clarke 1866\",6378206.4,294.9786982138982,AUTHORITY[\"EPSG\",\"7008\"]],AUTHORITY[\"EPSG\",\"6608\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.01745329251994328,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4608\"]],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],PROJECTION[\"Transverse_Mercator\"],PARAMETER[\"latitude_of_origin\",0],PARAMETER[\"central_meridian\",-79.5],PARAMETER[\"scale_factor\",0.9999],PARAMETER[\"false_easting\",304800],PARAMETER[\"false_northing\",0],AUTHORITY[\"EPSG\",\"2019\"],AXIS[\"X\",EAST],AXIS[\"Y\",NORTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2020, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2020, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"NAD27(76) / MTM zone 11\",GEOGCS[\"NAD27(76)\",DATUM[\"North_American_Datum_1927_1976\",SPHEROID[\"Clarke 1866\",6378206.4,294.9786982138982,AUTHORITY[\"EPSG\",\"7008\"]],AUTHORITY[\"EPSG\",\"6608\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.01745329251994328,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4608\"]],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],PROJECTION[\"Transverse_Mercator\"],PARAMETER[\"latitude_of_origin\",0],PARAMETER[\"central_meridian\",-82.5],PARAMETER[\"scale_factor\",0.9999],PARAMETER[\"false_easting\",304800],PARAMETER[\"false_northing\",0],AUTHORITY[\"EPSG\",\"2020\"],AXIS[\"Easting\",EAST],AXIS[\"Northing\",NORTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2021, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2021, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"NAD27(76) / MTM zone 12\",GEOGCS[\"NAD27(76)\",DATUM[\"North_American_Datum_1927_1976\",SPHEROID[\"Clarke 1866\",6378206.4,294.9786982138982,AUTHORITY[\"EPSG\",\"7008\"]],AUTHORITY[\"EPSG\",\"6608\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.01745329251994328,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4608\"]],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],PROJECTION[\"Transverse_Mercator\"],PARAMETER[\"latitude_of_origin\",0],PARAMETER[\"central_meridian\",-81],PARAMETER[\"scale_factor\",0.9999],PARAMETER[\"false_easting\",304800],PARAMETER[\"false_northing\",0],AUTHORITY[\"EPSG\",\"2021\"],AXIS[\"Easting\",EAST],AXIS[\"Northing\",NORTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2022, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2022, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"NAD27(76) / MTM zone 13\",GEOGCS[\"NAD27(76)\",DATUM[\"North_American_Datum_1927_1976\",SPHEROID[\"Clarke 1866\",6378206.4,294.9786982138982,AUTHORITY[\"EPSG\",\"7008\"]],AUTHORITY[\"EPSG\",\"6608\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.01745329251994328,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4608\"]],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],PROJECTION[\"Transverse_Mercator\"],PARAMETER[\"latitude_of_origin\",0],PARAMETER[\"central_meridian\",-84],PARAMETER[\"scale_factor\",0.9999],PARAMETER[\"false_easting\",304800],PARAMETER[\"false_northing\",0],AUTHORITY[\"EPSG\",\"2022\"],AXIS[\"Easting\",EAST],AXIS[\"Northing\",NORTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2023, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2023, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"NAD27(76) / MTM zone 14\",GEOGCS[\"NAD27(76)\",DATUM[\"North_American_Datum_1927_1976\",SPHEROID[\"Clarke 1866\",6378206.4,294.9786982138982,AUTHORITY[\"EPSG\",\"7008\"]],AUTHORITY[\"EPSG\",\"6608\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.01745329251994328,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4608\"]],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],PROJECTION[\"Transverse_Mercator\"],PARAMETER[\"latitude_of_origin\",0],PARAMETER[\"central_meridian\",-87],PARAMETER[\"scale_factor\",0.9999],PARAMETER[\"false_easting\",304800],PARAMETER[\"false_northing\",0],AUTHORITY[\"EPSG\",\"2023\"],AXIS[\"Easting\",EAST],AXIS[\"Northing\",NORTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2024, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2024, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"NAD27(76) / MTM zone 15\",GEOGCS[\"NAD27(76)\",DATUM[\"North_American_Datum_1927_1976\",SPHEROID[\"Clarke 1866\",6378206.4,294.9786982138982,AUTHORITY[\"EPSG\",\"7008\"]],AUTHORITY[\"EPSG\",\"6608\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.01745329251994328,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4608\"]],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],PROJECTION[\"Transverse_Mercator\"],PARAMETER[\"latitude_of_origin\",0],PARAMETER[\"central_meridian\",-90],PARAMETER[\"scale_factor\",0.9999],PARAMETER[\"false_easting\",304800],PARAMETER[\"false_northing\",0],AUTHORITY[\"EPSG\",\"2024\"],AXIS[\"Easting\",EAST],AXIS[\"Northing\",NORTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2025, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2025, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"NAD27(76) / MTM zone 16\",GEOGCS[\"NAD27(76)\",DATUM[\"North_American_Datum_1927_1976\",SPHEROID[\"Clarke 1866\",6378206.4,294.9786982138982,AUTHORITY[\"EPSG\",\"7008\"]],AUTHORITY[\"EPSG\",\"6608\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.01745329251994328,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4608\"]],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],PROJECTION[\"Transverse_Mercator\"],PARAMETER[\"latitude_of_origin\",0],PARAMETER[\"central_meridian\",-93],PARAMETER[\"scale_factor\",0.9999],PARAMETER[\"false_easting\",304800],PARAMETER[\"false_northing\",0],AUTHORITY[\"EPSG\",\"2025\"],AXIS[\"Easting\",EAST],AXIS[\"Northing\",NORTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2026, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2026, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"NAD27(76) / MTM zone 17\",GEOGCS[\"NAD27(76)\",DATUM[\"North_American_Datum_1927_1976\",SPHEROID[\"Clarke 1866\",6378206.4,294.9786982138982,AUTHORITY[\"EPSG\",\"7008\"]],AUTHORITY[\"EPSG\",\"6608\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.01745329251994328,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4608\"]],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],PROJECTION[\"Transverse_Mercator\"],PARAMETER[\"latitude_of_origin\",0],PARAMETER[\"central_meridian\",-96],PARAMETER[\"scale_factor\",0.9999],PARAMETER[\"false_easting\",304800],PARAMETER[\"false_northing\",0],AUTHORITY[\"EPSG\",\"2026\"],AXIS[\"Easting\",EAST],AXIS[\"Northing\",NORTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2027, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2027, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"NAD27(76) / UTM zone 15N\",GEOGCS[\"NAD27(76)\",DATUM[\"North_American_Datum_1927_1976\",SPHEROID[\"Clarke 1866\",6378206.4,294.9786982138982,AUTHORITY[\"EPSG\",\"7008\"]],AUTHORITY[\"EPSG\",\"6608\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.01745329251994328,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4608\"]],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],PROJECTION[\"Transverse_Mercator\"],PARAMETER[\"latitude_of_origin\",0],PARAMETER[\"central_meridian\",-93],PARAMETER[\"scale_factor\",0.9996],PARAMETER[\"false_easting\",500000],PARAMETER[\"false_northing\",0],AUTHORITY[\"EPSG\",\"2027\"],AXIS[\"Easting\",EAST],AXIS[\"Northing\",NORTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2028, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2028, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"NAD27(76) / UTM zone 16N\",GEOGCS[\"NAD27(76)\",DATUM[\"North_American_Datum_1927_1976\",SPHEROID[\"Clarke 1866\",6378206.4,294.9786982138982,AUTHORITY[\"EPSG\",\"7008\"]],AUTHORITY[\"EPSG\",\"6608\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.01745329251994328,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4608\"]],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],PROJECTION[\"Transverse_Mercator\"],PARAMETER[\"latitude_of_origin\",0],PARAMETER[\"central_meridian\",-87],PARAMETER[\"scale_factor\",0.9996],PARAMETER[\"false_easting\",500000],PARAMETER[\"false_northing\",0],AUTHORITY[\"EPSG\",\"2028\"],AXIS[\"Easting\",EAST],AXIS[\"Northing\",NORTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2029, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2029, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"NAD27(76) / UTM zone 17N\",GEOGCS[\"NAD27(76)\",DATUM[\"North_American_Datum_1927_1976\",SPHEROID[\"Clarke 1866\",6378206.4,294.9786982138982,AUTHORITY[\"EPSG\",\"7008\"]],AUTHORITY[\"EPSG\",\"6608\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.01745329251994328,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4608\"]],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],PROJECTION[\"Transverse_Mercator\"],PARAMETER[\"latitude_of_origin\",0],PARAMETER[\"central_meridian\",-81],PARAMETER[\"scale_factor\",0.9996],PARAMETER[\"false_easting\",500000],PARAMETER[\"false_northing\",0],AUTHORITY[\"EPSG\",\"2029\"],AXIS[\"Easting\",EAST],AXIS[\"Northing\",NORTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2030, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2030, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"NAD27(76) / UTM zone 18N\",GEOGCS[\"NAD27(76)\",DATUM[\"North_American_Datum_1927_1976\",SPHEROID[\"Clarke 1866\",6378206.4,294.9786982138982,AUTHORITY[\"EPSG\",\"7008\"]],AUTHORITY[\"EPSG\",\"6608\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.01745329251994328,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4608\"]],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],PROJECTION[\"Transverse_Mercator\"],PARAMETER[\"latitude_of_origin\",0],PARAMETER[\"central_meridian\",-75],PARAMETER[\"scale_factor\",0.9996],PARAMETER[\"false_easting\",500000],PARAMETER[\"false_northing\",0],AUTHORITY[\"EPSG\",\"2030\"],AXIS[\"Easting\",EAST],AXIS[\"Northing\",NORTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2031, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2031, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"NAD27(CGQ77) / UTM zone 17N\",GEOGCS[\"NAD27(CGQ77)\",DATUM[\"North_American_Datum_1927_CGQ77\",SPHEROID[\"Clarke 1866\",6378206.4,294.9786982138982,AUTHORITY[\"EPSG\",\"7008\"]],AUTHORITY[\"EPSG\",\"6609\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.01745329251994328,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4609\"]],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],PROJECTION[\"Transverse_Mercator\"],PARAMETER[\"latitude_of_origin\",0],PARAMETER[\"central_meridian\",-81],PARAMETER[\"scale_factor\",0.9996],PARAMETER[\"false_easting\",500000],PARAMETER[\"false_northing\",0],AUTHORITY[\"EPSG\",\"2031\"],AXIS[\"Easting\",EAST],AXIS[\"Northing\",NORTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2032, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2032, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"NAD27(CGQ77) / UTM zone 18N\",GEOGCS[\"NAD27(CGQ77)\",DATUM[\"North_American_Datum_1927_CGQ77\",SPHEROID[\"Clarke 1866\",6378206.4,294.9786982138982,AUTHORITY[\"EPSG\",\"7008\"]],AUTHORITY[\"EPSG\",\"6609\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.01745329251994328,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4609\"]],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],PROJECTION[\"Transverse_Mercator\"],PARAMETER[\"latitude_of_origin\",0],PARAMETER[\"central_meridian\",-75],PARAMETER[\"scale_factor\",0.9996],PARAMETER[\"false_easting\",500000],PARAMETER[\"false_northing\",0],AUTHORITY[\"EPSG\",\"2032\"],AXIS[\"Easting\",EAST],AXIS[\"Northing\",NORTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2033, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2033, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"NAD27(CGQ77) / UTM zone 19N\",GEOGCS[\"NAD27(CGQ77)\",DATUM[\"North_American_Datum_1927_CGQ77\",SPHEROID[\"Clarke 1866\",6378206.4,294.9786982138982,AUTHORITY[\"EPSG\",\"7008\"]],AUTHORITY[\"EPSG\",\"6609\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.01745329251994328,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4609\"]],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],PROJECTION[\"Transverse_Mercator\"],PARAMETER[\"latitude_of_origin\",0],PARAMETER[\"central_meridian\",-69],PARAMETER[\"scale_factor\",0.9996],PARAMETER[\"false_easting\",500000],PARAMETER[\"false_northing\",0],AUTHORITY[\"EPSG\",\"2033\"],AXIS[\"Easting\",EAST],AXIS[\"Northing\",NORTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2034, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2034, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"NAD27(CGQ77) / UTM zone 20N\",GEOGCS[\"NAD27(CGQ77)\",DATUM[\"North_American_Datum_1927_CGQ77\",SPHEROID[\"Clarke 1866\",6378206.4,294.9786982138982,AUTHORITY[\"EPSG\",\"7008\"]],AUTHORITY[\"EPSG\",\"6609\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.01745329251994328,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4609\"]],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],PROJECTION[\"Transverse_Mercator\"],PARAMETER[\"latitude_of_origin\",0],PARAMETER[\"central_meridian\",-63],PARAMETER[\"scale_factor\",0.9996],PARAMETER[\"false_easting\",500000],PARAMETER[\"false_northing\",0],AUTHORITY[\"EPSG\",\"2034\"],AXIS[\"Easting\",EAST],AXIS[\"Northing\",NORTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2035, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2035, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"NAD27(CGQ77) / UTM zone 21N\",GEOGCS[\"NAD27(CGQ77)\",DATUM[\"North_American_Datum_1927_CGQ77\",SPHEROID[\"Clarke 1866\",6378206.4,294.9786982138982,AUTHORITY[\"EPSG\",\"7008\"]],AUTHORITY[\"EPSG\",\"6609\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.01745329251994328,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4609\"]],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],PROJECTION[\"Transverse_Mercator\"],PARAMETER[\"latitude_of_origin\",0],PARAMETER[\"central_meridian\",-57],PARAMETER[\"scale_factor\",0.9996],PARAMETER[\"false_easting\",500000],PARAMETER[\"false_northing\",0],AUTHORITY[\"EPSG\",\"2035\"],AXIS[\"Easting\",EAST],AXIS[\"Northing\",NORTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2036, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2036, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"NAD83(CSRS98) / New Brunswick Stereo (deprecated)\",GEOGCS[\"NAD83(CSRS98)\",DATUM[\"NAD83_Canadian_Spatial_Reference_System\",SPHEROID[\"GRS 1980\",6378137,298.257222101,AUTHORITY[\"EPSG\",\"7019\"]],TOWGS84[0,0,0,0,0,0,0],AUTHORITY[\"EPSG\",\"6140\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.0174532925199433,AUTHORITY[\"EPSG\",\"9108\"]],AUTHORITY[\"EPSG\",\"4140\"]],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],PROJECTION[\"Oblique_Stereographic\"],PARAMETER[\"latitude_of_origin\",46.5],PARAMETER[\"central_meridian\",-66.5],PARAMETER[\"scale_factor\",0.999912],PARAMETER[\"false_easting\",2500000],PARAMETER[\"false_northing\",7500000],AUTHORITY[\"EPSG\",\"2036\"],AXIS[\"Easting\",EAST],AXIS[\"Northing\",NORTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2037, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2037, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"NAD83(CSRS98) / UTM zone 19N (deprecated)\",GEOGCS[\"NAD83(CSRS98)\",DATUM[\"NAD83_Canadian_Spatial_Reference_System\",SPHEROID[\"GRS 1980\",6378137,298.257222101,AUTHORITY[\"EPSG\",\"7019\"]],TOWGS84[0,0,0,0,0,0,0],AUTHORITY[\"EPSG\",\"6140\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.0174532925199433,AUTHORITY[\"EPSG\",\"9108\"]],AUTHORITY[\"EPSG\",\"4140\"]],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],PROJECTION[\"Transverse_Mercator\"],PARAMETER[\"latitude_of_origin\",0],PARAMETER[\"central_meridian\",-69],PARAMETER[\"scale_factor\",0.9996],PARAMETER[\"false_easting\",500000],PARAMETER[\"false_northing\",0],AUTHORITY[\"EPSG\",\"2037\"],AXIS[\"Easting\",EAST],AXIS[\"Northing\",NORTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2038, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2038, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"NAD83(CSRS98) / UTM zone 20N (deprecated)\",GEOGCS[\"NAD83(CSRS98)\",DATUM[\"NAD83_Canadian_Spatial_Reference_System\",SPHEROID[\"GRS 1980\",6378137,298.257222101,AUTHORITY[\"EPSG\",\"7019\"]],TOWGS84[0,0,0,0,0,0,0],AUTHORITY[\"EPSG\",\"6140\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.0174532925199433,AUTHORITY[\"EPSG\",\"9108\"]],AUTHORITY[\"EPSG\",\"4140\"]],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],PROJECTION[\"Transverse_Mercator\"],PARAMETER[\"latitude_of_origin\",0],PARAMETER[\"central_meridian\",-63],PARAMETER[\"scale_factor\",0.9996],PARAMETER[\"false_easting\",500000],PARAMETER[\"false_northing\",0],AUTHORITY[\"EPSG\",\"2038\"],AXIS[\"Easting\",EAST],AXIS[\"Northing\",NORTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2039, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2039, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"Israel / Israeli TM Grid\",GEOGCS[\"Israel\",DATUM[\"Israel\",SPHEROID[\"GRS 1980\",6378137,298.257222101,AUTHORITY[\"EPSG\",\"7019\"]],TOWGS84[-48,55,52,0,0,0,0],AUTHORITY[\"EPSG\",\"6141\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.01745329251994328,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4141\"]],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],PROJECTION[\"Transverse_Mercator\"],PARAMETER[\"latitude_of_origin\",31.73439361111111],PARAMETER[\"central_meridian\",35.20451694444445],PARAMETER[\"scale_factor\",1.0000067],PARAMETER[\"false_easting\",219529.584],PARAMETER[\"false_northing\",626907.39],AUTHORITY[\"EPSG\",\"2039\"],AXIS[\"Easting\",EAST],AXIS[\"Northing\",NORTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2040, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2040, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"Locodjo 1965 / UTM zone 30N\",GEOGCS[\"Locodjo 1965\",DATUM[\"Locodjo_1965\",SPHEROID[\"Clarke 1880 (RGS)\",6378249.145,293.465,AUTHORITY[\"EPSG\",\"7012\"]],TOWGS84[-125,53,467,0,0,0,0],AUTHORITY[\"EPSG\",\"6142\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.01745329251994328,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4142\"]],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],PROJECTION[\"Transverse_Mercator\"],PARAMETER[\"latitude_of_origin\",0],PARAMETER[\"central_meridian\",-3],PARAMETER[\"scale_factor\",0.9996],PARAMETER[\"false_easting\",500000],PARAMETER[\"false_northing\",0],AUTHORITY[\"EPSG\",\"2040\"],AXIS[\"Easting\",EAST],AXIS[\"Northing\",NORTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; table->field[0]->store(3819, TRUE); /*SRID*/ table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ @@ -674,6 +834,7 @@ static int fill_spatial_ref_sys(THD *thd, TABLE_LIST *tables, COND *cond) if (schema_table_store_record(thd, table)) goto exit; + result= 0; exit:

1 0

[Maria-developers] [GSoC CONC] Non trivial patch
by arlene seuleu 03 Apr '15

03 Apr '15

diff --git a/libmariadb/dbug.c b/libmariadb/dbug.c index fb7c52b..9855f97 100644 --- a/libmariadb/dbug.c +++ b/libmariadb/dbug.c @@ -428,9 +428,9 @@ static void read_lock_stack(CODE_STATE *cs) */ static void unlock_stack(CODE_STATE *cs) { - if (cs->stack == &init_settings) + if (cs->stack == &init_settings && --(cs->m_read_lock_count) == 0) { - if (--(cs->m_read_lock_count) == 0) + pthread_mutex_unlock(&THR_LOCK_init_settings); } }

2 1

Re: [Maria-developers] [GSoC 2015] patch.
by Oleksandr Byelkin 03 Apr '15

03 Apr '15

Hi! On 03.04.15 12:25, arlene seuleu wrote: > Hi Byelkin > I just did it for consistency with the other storage macros to to > facilitate readability. I do not see any consistency except hiding cause why other macro was written. Please do not make changes in sake of changes. The same about formating, if you do not make some meaningful changes in that like do not change formating of it even if it is wrong.

1 0

[Maria-developers] More explicit Error message
by Blessing Atie 03 Apr '15

03 Apr '15

diff --git a/libmysql/errmsg.c b/libmysql/errmsg.c index 84884f9..abcf3f3 100644 --- a/libmysql/errmsg.c +++ b/libmysql/errmsg.c @@ -31,7 +31,7 @@ "Can't connect to MySQL server on '%-.100s' (%M)", "Can't create TCP/IP socket (%M)", "Unknown MySQL server host '%-.100s' (%d)", - "MySQL server has gone away", + "MySQL server has gone away due to wait_timeout limit reached", "Protocol mismatch; server version = %d, client version = %d", "MySQL client ran out of memory", "Wrong host info",

1 0

[Maria-developers] Patch
by Blessing Atie 03 Apr '15

03 Apr '15

Added generic build instructions from KB to INSTALL-SOURCE diff --git a/INSTALL-SOURCE b/INSTALL-SOURCE index 32cfa97..809d69b 100644 --- a/INSTALL-SOURCE +++ b/INSTALL-SOURCE @@ -1,3 +1,86 @@ -Instructions for building MariaDB can be found at: +The instructions in this file will help you compile MariaDB from source. Links to more complete instructions for specific platforms can be found at https://kb.askmonty.org/en/compiling-mariadb-from-source/ +First, get a copy of the MariaDB source. + +Next, prepare your system to be able to compile the source. + +If you don't want to run MariaDB as yourself, then you should create a mysql user. The example below uses this user. + +MariaDB starting with 5.5 + +Using cmake + +MariaDB 5.5 and above is compiled using cmake. You can configure your build simply by running cmake without any special options, like +cmake . +but if you want it to be configured exactly as we do for our releases, use +cmake . -DBUILD_CONFIG=mysql_release +All cmake configuration options for MariaDB can be displayed with: +cmake . -LH +To build and install MariaDB after running cmake use +make +sudo make install +If the commands above fail, you can enable more compilation information by doing: +make VERBOSE=1 +MariaDB until 5.3 +Building MariaDB with the BUILD scripts +The easiest way of building MariaDB would be to use the appropriate BUILD script that matches your configuration. For example: +cd $maria-source-dir # ex: ~/repos/maria/trunk + +BUILD/compile-pentium64-max # Build for intel/amd 64 bit with all options +BUILD/compile-pentium-max # Build for 32 bit pentum with all options + +BUILD/compile-pentium64-debug-max # Build for intel/amd 64 bit with debugging +BUILD/compile-pentium64-valgrind-max # Build for testing with valgrind + +There are many different BUILD scripts which cover many different configurations and processor architectures. +To see what a build scripts does, you can run it with the --print option. +After building MariaDB with a BUILD script, install MariaDB with: +sudo make install +Building MariaDB Manually +If there is no appropriate script, you can also execute the build commands directly: + +cd $maria-source-dir # ex: ~/repos/maria/trunk +BUILD/autorun.sh +./configure +make +sudo make install +Max builds configure line +The following configure line is the one we used to use in the -max builds (which includes all features we think are relevant). This is also the configure line we use with most of our binary builds: + +./configure --prefix=/usr/local/mysql --enable-assembler \ + --with-extra-charsets=complex --enable-thread-safe-client --with-big-tables \ + --with-plugin-maria --with-aria-tmp-tables --without-plugin-innodb_plugin \ + --with-mysqld-ldflags=-static --with-client-ldflags=-static --with-readline \ + --with-ssl --with-plugins=max-no-ndb --with-embedded-server --with-libevent \ + --with-mysqld-ldflags=-all-static --with-client-ldflags=-all-static \ + --with-zlib-dir=bundled --enable-local-infile + +For a full list of options, you can do: + +./configure --help + +Starting MariaDB for the first time +After installing MariaDB (using sudo make install), but prior to starting MariaDB for the first time, one should: + +ensure the directory where you installed MariaDB is owned by the mysql user (if the user doesn't exist, you'll need to create it) +run the mysql_install_db script to generate the needed system tables +Here is an example: + +# The following assumes that the 'mysql' user exists and that we installed MariaDB + +# in /usr/local/mysql +chown -R mysql /usr/local/mysql/ +scripts/mysql_install_db --user=mysql +/usr/local/mysql/bin/mysqld_safe --user=mysql & + + +Testing MariaDB + +If you want to test your compiled MariaDB, you can do either of: + +make test +or + +cd mysql-test ; ./mysql-test-run --force +Each of the above are run from the source directory. There is no need to 'sudo make install' MariaDB prior to running them. \ No newline at end of file

1 0

[Maria-developers] [GSoC 2015] patch.
by arlene seuleu 03 Apr '15

03 Apr '15

diff --git a/mysys/ma_dyncol.c b/mysys/ma_dyncol.c index a7a048a..0789350 100644 --- a/mysys/ma_dyncol.c +++ b/mysys/ma_dyncol.c @@ -329,20 +329,20 @@ static my_bool type_and_offset_store_num(uchar *place, size_t offset_size, case 1: if (offset >= 0x1f) /* all 1 value is reserved */ return TRUE; - place[0]= (uchar)val; + int1store(place,val); break; case 2: - if (offset >= 0x1fff) /* all 1 value is reserved */ + if (offset >= 0x1fff) /* all 2 values are reserved */ return TRUE; int2store(place, val); break; case 3: - if (offset >= 0x1fffff) /* all 1 value is reserved */ + if (offset >= 0x1fffff) /* all 3 values are reserved */ return TRUE; int3store(place, val); break; case 4: - if (offset >= 0x1fffffff) /* all 1 value is reserved */ + if (offset >= 0x1fffffff) /* all 4 values are reserved */ return TRUE; int4store(place, val); break; diff --git a/mysys/mysys_priv.h b/mysys/mysys_priv.h index 4ea6d08..962dc5a 100644 --- a/mysys/mysys_priv.h +++ b/mysys/mysys_priv.h @@ -16,6 +16,10 @@ #include <my_global.h> #include <my_sys.h> +#define int1store(T,A) do { uint def_temp= (uint) (A) ;\ + *((uchar*) (T))= (uchar)(def_temp);\ + }while(0) + #ifdef USE_SYSTEM_WRAPPERS #include "system_wrappers.h" #endif

2 1

Re: [Maria-developers] [Commits] fc31f6d: MDEV-7833:ANALYZE FORMAT=JSON and Range checked for each record
by Sergey Petrunia 02 Apr '15

02 Apr '15

Hi Sanja, Please find review comments below. On Thu, Apr 02, 2015 at 06:19:41PM +0200, sanja(a)mariadb.com wrote: > revision-id: fc31f6d95720b4b946b8b68c816026d65831f347 > parent(s): 01d7da6785284383b2c04f2d4474feccebb0bb6f > committer: Oleksandr Byelkin > branch nick: server > timestamp: 2015-04-02 18:19:33 +0200 > message: > > MDEV-7833:ANALYZE FORMAT=JSON and Range checked for each record > > --- > mysql-test/r/analyze_format_json.result | 62 +++++++++++++++++++++++++++++ > mysql-test/t/analyze_format_json.test | 30 ++++++++++++++ > sql/sql_explain.cc | 26 +++++++++++-- > sql/sql_explain.h | 12 +++++- > sql/sql_select.cc | 69 ++++++++++++++++++++++++++++++--- > sql/sql_select.h | 7 +++- > 6 files changed, 195 insertions(+), 11 deletions(-) > > diff --git a/mysql-test/r/analyze_format_json.result b/mysql-test/r/analyze_format_json.result > index 9022d8c..b3ef075 100644 > --- a/mysql-test/r/analyze_format_json.result > +++ b/mysql-test/r/analyze_format_json.result > @@ -264,3 +264,65 @@ ANALYZE > } > } > drop table t1; > +# > +# MDEV-7833:ANALYZE FORMAT=JSON and Range checked for each record > +# > +create table t3(a int); > +insert into t3 values (0),(1),(2),(3),(4),(5),(6),(7),(8),(9); > +create table t4(a int); > +insert into t4 select A.a + B.a* 10 + C.a * 100 from t3 A, t3 B, t3 C; > +create table t1 (lb1 int, rb1 int, lb2 int, rb2 int, c1 int, c2 int); > +insert into t1 values (1,2,10,20,15,15); > +insert into t1 values (3,5,10,20,15,15); > +insert into t1 values (10,20,10,20,15,15); > +insert into t1 values (10,20,1,2,15,15); > +insert into t1 values (10,20,10,20,1,3); > +create table t2 (key1 int, key2 int, key3 int, key4 int, col1 int, > +key(key1), key(key2), key(key3), key(key4)); > +insert into t2 select a,a,a,a,a from t3; > +insert into t2 select 15,15,15,15,15 from t4; > +analyze format=json > +select * from t1, t2 where (t2.key1 between t1.lb1 and t1.rb1) and > +(t2.key2 between t1.lb2 and t1.rb2) and > +(t2.key3=t1.c1 OR t2.key4=t1.c2); > +ANALYZE > +{ > + "query_block": { > + "select_id": 1, > + "r_loops": 1, > + "r_total_time_ms": "REPLACED", > + "table": { > + "table_name": "t1", > + "access_type": "ALL", > + "r_loops": 1, > + "rows": 5, > + "r_rows": 5, > + "r_total_time_ms": "REPLACED", > + "filtered": 100, > + "r_filtered": 100 > + }, > + "range-checked-for-each-record": { > + "keys": ["key1", "key2", "key3", "key4"], > + "r_keys_none": 1, > + "r_keys_index_merge": 1, > + "r_keys": { > + "key1": 2, > + "key2": 1, > + "key3": 0, > + "key4": 0 > + }, It seems wrong to have multiple r_XXX elements. I think, having one will be better. How about: r_keys: { "none": 1, ## Maybe, change to "full_table_scan" ? (btw can it be a full index scan instead I wonder?) "index_merge": nnn, "range" : { "key1" : nnn, "key2" : nnn, "key3" : nnn } } Let's discuss it. > + "table": { > + "table_name": "t2", > + "access_type": "ALL", > + "possible_keys": ["key1", "key2", "key3", "key4"], > + "r_loops": 5, > + "rows": 1010, > + "r_rows": 203.8, > + "r_total_time_ms": "REPLACED", > + "filtered": 100, > + "r_filtered": 98.135 > + } > + } > + } > +} > +drop table t1,t2,t3,t4; > diff --git a/mysql-test/t/analyze_format_json.test b/mysql-test/t/analyze_format_json.test > index 64bafd7..0ae9d4f 100644 > --- a/mysql-test/t/analyze_format_json.test > +++ b/mysql-test/t/analyze_format_json.test > @@ -75,3 +75,33 @@ disconnect con1; > connection default; > drop table t1; > > + > +--echo # > +--echo # MDEV-7833:ANALYZE FORMAT=JSON and Range checked for each record > +--echo # > +create table t3(a int); > +insert into t3 values (0),(1),(2),(3),(4),(5),(6),(7),(8),(9); > + > +create table t4(a int); > +insert into t4 select A.a + B.a* 10 + C.a * 100 from t3 A, t3 B, t3 C; > + > +create table t1 (lb1 int, rb1 int, lb2 int, rb2 int, c1 int, c2 int); > + > +insert into t1 values (1,2,10,20,15,15); > +insert into t1 values (3,5,10,20,15,15); > +insert into t1 values (10,20,10,20,15,15); > +insert into t1 values (10,20,1,2,15,15); > +insert into t1 values (10,20,10,20,1,3); > + > +create table t2 (key1 int, key2 int, key3 int, key4 int, col1 int, > + key(key1), key(key2), key(key3), key(key4)); > +insert into t2 select a,a,a,a,a from t3; > +insert into t2 select 15,15,15,15,15 from t4; > + > +--replace_regex /"r_total_time_ms": [0-9]*[.]?[0-9]*/"r_total_time_ms": "REPLACED"/ > +analyze format=json > +select * from t1, t2 where (t2.key1 between t1.lb1 and t1.rb1) and > + (t2.key2 between t1.lb2 and t1.rb2) and > + (t2.key3=t1.c1 OR t2.key4=t1.c2); > + > +drop table t1,t2,t3,t4; > diff --git a/sql/sql_explain.cc b/sql/sql_explain.cc > index 9d82f4f..edbc2bf 100644 > --- a/sql/sql_explain.cc > +++ b/sql/sql_explain.cc > @@ -1076,15 +1076,15 @@ int Explain_table_access::print_explain(select_result_sink *output, uint8 explai > } > > Please add a comment: @return ptr - pointer to the added string NULL - out of memory error Does it make sense to change the return type to 'const char*' ? > -bool String_list::append_str(MEM_ROOT *mem_root, const char *str) > +char *String_list::append_str(MEM_ROOT *mem_root, const char *str) > { > size_t len= strlen(str); > char *cp; > if (!(cp = (char*)alloc_root(mem_root, len+1))) > - return 1; > + return NULL; > memcpy(cp, str, len+1); > push_back(cp); > - return 0; > + return cp; > } > > > @@ -1209,6 +1209,26 @@ void Explain_table_access::print_explain_json(Explain_query *query, > { > writer->add_member("range-checked-for-each-record").start_object(); > add_json_keyset(writer, "keys", &possible_keys); > + if (is_analyze) > + { > + writer->add_member("r_keys_none").add_ll(range_checked_fer->none); > + writer->add_member("r_keys_index_merge").add_ll(range_checked_fer-> > + index_merge); > + if (range_checked_fer->keys_stat) > + { > + writer->add_member("r_keys").start_object(); > + for (uint i= 0; i < range_checked_fer->keys; i++) > + { > + if (range_checked_fer->keys_stat_names[i]) > + { > + writer->add_member(range_checked_fer-> > + keys_stat_names[i]).add_ll(range_checked_fer-> > + keys_stat[i]); > + } > + } > + writer->end_object(); > + } > + } If we move this code into Range_checked_fer::print_json(), will it be possible to make its members private? > } > > if (full_scan_on_null_key) > diff --git a/sql/sql_explain.h b/sql/sql_explain.h > index e3b41ee..f86886d 100644 > --- a/sql/sql_explain.h > +++ b/sql/sql_explain.h > @@ -54,7 +54,7 @@ it into the slow query log. > class String_list: public List<char> > { > public: > - bool append_str(MEM_ROOT *mem_root, const char *str); > + char *append_str(MEM_ROOT *mem_root, const char *str); > }; > > class Json_writer; > @@ -627,6 +627,16 @@ class Explain_range_checked_fer : public Sql_alloc > public: > String_list key_set; > key_map keys_map; > + > + ha_rows none, index_merge; 'none' looks weird. Would a 'full_scan' be better. > + ha_rows *keys_stat; > + char **keys_stat_names; > + uint keys; This needs to be documented. > + > + Explain_range_checked_fer() > + :Sql_alloc(), none(0), index_merge(0), > + keys_stat(0), keys_stat_names(0), keys(0) > + {} > }; > > /* > diff --git a/sql/sql_select.cc b/sql/sql_select.cc > index 5c3e116..4eb3d81 100644 > --- a/sql/sql_select.cc > +++ b/sql/sql_select.cc > @@ -11398,6 +11398,7 @@ void JOIN_TAB::cleanup() > table->reginfo.join_tab= 0; > } > end_read_record(&read_record); > + explain_plan= 0; It's a pointer, right? Please change s/0/NULL/ > DBUG_VOID_RETURN; > } > > @@ -18762,9 +18763,30 @@ test_if_quick_select(JOIN_TAB *tab) > > delete tab->select->quick; > tab->select->quick=0; > - return tab->select->test_quick_select(tab->join->thd, tab->keys, > - (table_map) 0, HA_POS_ERROR, 0, > - FALSE, /*remove where parts*/FALSE); > + int res= tab->select->test_quick_select(tab->join->thd, tab->keys, > + (table_map) 0, HA_POS_ERROR, 0, > + FALSE, /*remove where parts*/FALSE); > + if (tab->explain_plan && tab->explain_plan->range_checked_fer) > + { > + if (tab->select->quick) > + { > + QUICK_SELECT_I *quick= tab->select->quick; > + if (quick->index == MAX_KEY) > + tab->explain_plan->range_checked_fer->index_merge++; > + else > + { > + DBUG_ASSERT(quick->index < tab->explain_plan->range_checked_fer->keys); > + DBUG_ASSERT(tab->explain_plan->range_checked_fer->keys_stat); > + DBUG_ASSERT(tab->explain_plan->range_checked_fer->keys_stat_names); > + DBUG_ASSERT(tab->explain_plan->range_checked_fer->keys_stat_names[ > + quick->index]); > + tab->explain_plan->range_checked_fer->keys_stat[quick->index]++; > + } > + } > + else > + tab->explain_plan->range_checked_fer->none++; > + } > + return res; I think it's bad that join execution code becomes littered with details about how things are stored in the Explain data structures. Can you add a set of methods to range_checked_fer: count_full_scan(); count_index_merge(); count_range_scan(); and then only call these methods from here? > } > > > @@ -23360,6 +23382,32 @@ int append_possible_keys(MEM_ROOT *alloc, String_list &list, TABLE *table, > return 0; > } > What does this function do? AFAIU it does two things: - copies indexes from possible_keys into a String_list. - allocates an empty array of ha_rows(). The set of data structures used by Range_checked_fer is quite peculiar. This function is only useful for Range_checked_fer. Do you think it would make sense to made this a method of Range_checked_fer? > +int append_possible_keys_stat(MEM_ROOT *alloc, String_list &list, > + ha_rows **stat, char ***names, > + TABLE *table, key_map possible_keys) > +{ > + uint j; > + multi_alloc_root(alloc, stat, sizeof(ha_rows) * table->s->keys, > + names, sizeof(char *) * table->s->keys, NULL); > + if ((!(*stat)) || (!(*names))) > + { > + (*stat)= NULL; (*names)= NULL; > + return 1; > + } > + bzero(*stat, sizeof(ha_rows) * table->s->keys); > + for (j= 0; j < table->s->keys; j++) > + { > + if (possible_keys.is_set(j)) > + { > + char *nm= list.append_str(alloc, table->key_info[j].name); > + (*names)[j]= nm; > + } > + else > + (*names)[j]= NULL; > + } > + return 0; > +} > + > /* > TODO: this function is only applicable for the first non-const optimization > join tab. > @@ -23400,6 +23448,8 @@ void JOIN_TAB::save_explain_data(Explain_table_access *eta, table_map prefix_tab > quick_type= -1; > QUICK_SELECT_I *quick= NULL; > > + > + explain_plan= eta; > eta->key.clear(); > eta->quick_info= NULL; > > @@ -23663,9 +23713,16 @@ void JOIN_TAB::save_explain_data(Explain_table_access *eta, table_map prefix_tab > { > eta->push_extra(ET_RANGE_CHECKED_FOR_EACH_RECORD); > eta->range_checked_fer= new (thd->mem_root) Explain_range_checked_fer; > - eta->range_checked_fer->keys_map= tab->keys; > - append_possible_keys(thd->mem_root, eta->range_checked_fer->key_set, > - table, tab->keys); > + if (eta->range_checked_fer) > + { > + eta->range_checked_fer->keys_map= tab->keys; > + eta->range_checked_fer->keys= table->s->keys; > + append_possible_keys_stat(thd->mem_root, > + eta->range_checked_fer->key_set, > + &eta->range_checked_fer->keys_stat, > + &eta->range_checked_fer->keys_stat_names, > + table, tab->keys); > + } > } > else if (tab->select->cond || > (tab->cache_select && tab->cache_select->cond)) > diff --git a/sql/sql_select.h b/sql/sql_select.h > index 0557e32..ef92690 100644 > --- a/sql/sql_select.h > +++ b/sql/sql_select.h > @@ -362,7 +362,12 @@ typedef struct st_join_table { > SJ_TMP_TABLE *check_weed_out_table; > /* for EXPLAIN only: */ > SJ_TMP_TABLE *first_weedout_table; > - > + > + /** > + reference to saved plan and execution statistics > + */ > + Explain_table_access *explain_plan; > + > /* > If set, means we should stop join enumeration after we've got the first > match and return to the specified join tab. May point to BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] [GSoC] MDEV-4674 Port InnoDB memcached interface to MariaDB
by Piotr Jurkiewicz 02 Apr '15

02 Apr '15

Hello. My name is Piotr Jurkiewicz. I am a last year MSc student at AGH University of Science and Technology in Krakow, Poland. I already contributed to MySQL InnoDB Memcached plugin. I know it well, from user and developer points of view. I have been using MySQL InnoDB Memcached plugin since half year in a production system I administrate at my university. However, I must admit that quality of this plugin is rather poor. I have run several times into bugs, from the simple ones, like incorrect results of arithmetic operations, to the serious, like crashes of the whole MySQL process. I reported some of these bugs to Oracle and contributed patches fixing them [1] [2]. However, the more I was diving into the plugin code, the more I was realizing that the problem is not only buggy code and insufficient testing. The core mistakes had been made at the very beginning, when plugin architecture was set up. The most prominent result of these mistakes is lack of access security. After connecting to Memcached port, a user has access to all databases and tables exported by the plugin. Fixing this would require complete remodeling of plugin architecture and the way users configure and use it. This is not possible in MySQL, as it would break existing deployments. Therefore, I was very excited, when I saw that porting of InnoDB Memcached plugin is among MariaDB proposed GSoC ideas. This gives a possibility to build the plugin in a proper way. During the last months I have been thinking a lot how this plugin should be structured. I think I worked out a clear concept of that. With the financing from Google I will be able to dedicate sufficient amount of time for its implementation, what should result in fully featured, secure and robust Memcached plugin for MariaDB. You can review current version of my proposal in the Google Melange system: https://www.google-melange.com/gsoc/proposal/review/student/google/gsoc2015… As for now, it is only visible for MariaDB organization members, who are registered in Melange system. On March 27th, after student application deadline, I will make it visible for non-registered users as well and send its copy to this list. I am waiting for your remarks and hints. [1]: http://bugs.mysql.com/bug.php?id=74874 [2]: http://bugs.mysql.com/bug.php?id=75864 Piotr

4 8

Re: [Maria-developers] [server] Mdev 7802 binlog groupcommit stats (#30)
by Kristian Nielsen 02 Apr '15

02 Apr '15

Daniel Black <notifications(a)github.com> writes: Thanks for your patch. I agree that these kinds of status information could be useful. But I have some parts that are not clear, and I want you to think about the questions below and answer with your thoughts. This is to make sure you understand yourself the issue in detail; to help me understand it; and in some cases maybe some changes should be made. > The following global status variables where added: > * binlog_group_commit_reason_count > * binlog_group_commit_reason_usec So, these names seem a bit odd. Neither "count" nor "usec" makes much sense as a reason for something to happen. "_timeout" would be a better reason that "_usec", but I am not sure of a better name for "_count". The good thing about these names of course is that they match the corresponding --binlog-commit-wait-* option. What do you think of something like binlog_commit_trigger_count binlog_commit_trigger_timeout instead ? > * binlog_group_commit_reason_transaction > * binlog_group_commit_reason_immediate > binlog_group_commit_reason_transaction is a result of ordered > transaction that need to occur in the same order on the slave and can't > be parallelised. > > binlog_group_commit_reason_immediate is caused to prevent stalls with > row locks as described in log.cc:binlog_report_wait_for. This immediate > count is also counted a second time in binlog_group_commit_reason_transaction. Please re-read that explanation, and consider: do you think an average user/DBA will fully understand what these status variables mean? Honestly, I am wondering if you can even yourself explain exactly what the difference is between the two :-). Can you give two simple examples of transactions, one example of which will increment the _transaction status variable and one of which will increment the _immediate? Do you think the distinction between the two is useful for the user? If not, maybe just have a single status variable, like binlog_commit_trigger_lock_wait for example? > Overall binlog_group_commits = binlog_group_commit_reason_count + > binlog_group_commit_reason_usec + binlog_group_commit_reason_transaction Is this also true if --binlog-commit-wait-count=0? If so, what will the values of the different status variables be in this case? Now, to the patches. > + { > + if (++count >= opt_binlog_commit_wait_count) > + { > + group_commit_reason_count++; > + return; > + } > + if (unlikely(e->thd->has_waiter)) > + { > + group_commit_reason_transaction++; > return; > + } > + } So this increments the global status variables directly, I think? How does the thread locking work? Is it possible for a reader to see a corrupt value (eg. one word of the old value and one word of the new) on 32-bit platform? How is this handled for other similar status variables? It would not be good if we had to take some LOCK_status or something inside the critical group commit code. With respect to the test cases: Can you please comment yourself on the different places where you added output of the status variables, and explain why this output will always be the same, no matter how threads are scheduled on the machine running the test? (This is the critical part of most tests related to replication; because of the inherently multi-threaded nature of such tests, a lot of care is needed to make sure the test will not produce different output on different test runs depending on the speed of the host or how threads are scheduled on a loaded machine). - Kristian.

2 2

[Maria-developers] [MDEV-7540] Information Schema SPATIAL_REF_SYS contents don't match the expected contents
by Delveri chick 02 Apr '15

02 Apr '15

diff --git a/sql/sql_show.cc b/sql/sql_show.cc index dce2f7b..c6d73bf 100644 --- a/sql/sql_show.cc +++ b/sql/sql_show.cc @@ -355,6 +355,324 @@ static int fill_spatial_ref_sys(THD *thd, TABLE_LIST *tables, COND *cond) if (schema_table_store_record(thd, table)) goto exit; + table->field[0]->store(1, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("SR-ORG"), cs); /*AUTH_NAME*/ + table->field[2]->store(1, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"unnamed\",\ + GEOGCS[\"GRS 1980(IUGG, 1980)\",DATUM[\"unknown\",SPHEROID[\"GRS80\",\ + 6378137,298.257222101]],PRIMEM[\"Greenwich\",0],UNIT[\"degree\",\ + 0.0174532925199433]],PROJECTION[\"Lambert_Conformal_Conic_2SP\"],\ + PARAMETER[\"standard_parallel_1\",42.6833319],PARAMETER[\"standard_parallel_2\",\ + 41.71666666666667],PARAMETER[\"latitude_of_origin\",41],PARAMETER[\"central_meridian\",\ + -71.5],PARAMETER[\"false_easting\",200000],PARAMETER[\"false_northing\",\ + 750000],UNIT[\"Meter\",1]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2000, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2000, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"Anguilla 1957 / British West Indies Grid\ + \",GEOGCS[\"Anguilla 1957\",DATUM[\"Anguilla_1957\",SPHEROID[\ + \"Clarke 1880 (RGS)\",6378249.145,293.465,AUTHORITY[\"EPSG\",\"7012\ + \"]],AUTHORITY[\"EPSG\",\"6600\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\ + \"8901\"]],UNIT[\"degree\",0.01745329251994328,AUTHORITY[\"EPSG\",\"9122\ + \"]],AUTHORITY[\"EPSG\",\"4600\"]],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\ + \"9001\"]],PROJECTION[\"Transverse_Mercator\"],PARAMETER[\"latitude_of_origin\ + \",0],PARAMETER[\"central_meridian\",-62],PARAMETER[\"scale_factor\",0.9995],PARAMETER[\ + \"false_easting\",400000],PARAMETER[\"false_northing\",0],AUTHORITY[\"EPSG\",\ + \"2000\"],AXIS[\"Easting\",EAST],AXIS[\"Northing\",NORTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2001, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2001, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"Antigua 1943 / British West Indies Grid\ + \",GEOGCS[\"Antigua 1943\",DATUM[\"Antigua_1943\",SPHEROID[\ + \"Clarke 1880 (RGS)\",6378249.145,293.465,AUTHORITY[\"EPSG\",\"7012\ + \"]],AUTHORITY[\"EPSG\",\"6601\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\ + \"8901\"]],UNIT[\"degree\",0.01745329251994328,AUTHORITY[\"EPSG\",\"9122\ + \"]],AUTHORITY[\"EPSG\",\"4601\"]],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\ + \"9001\"]],PROJECTION[\"Transverse_Mercator\"],PARAMETER[\"latitude_of_origin\ + \",0],PARAMETER[\"central_meridian\",-62],PARAMETER[\"scale_factor\ + \",0.9995],PARAMETER[\"false_easting\",400000],PARAMETER[\"false_northing\ + \",0],AUTHORITY[\"EPSG\",\"2001\"],AXIS[\"Easting\",EAST],AXIS[\"Northing\ + \",NORTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2002, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2002, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"Dominica 1945 / British West Indies Grid\ + \",GEOGCS[\"Dominica 1945\",DATUM[\"Dominica_1945\",SPHEROID[\ + \"Clarke 1880 (RGS)\",6378249.145,293.465,AUTHORITY[\"EPSG\",\"7012\ + \"]],TOWGS84[725,685,536,0,0,0,0],AUTHORITY[\"EPSG\",\"6602\"]],PRIMEM[\ + \"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\ + \",0.01745329251994328,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\ + \"4602\"]],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],PROJECTION[\ + \"Transverse_Mercator\"],PARAMETER[\"latitude_of_origin\",0],PARAMETER[\ + \"central_meridian\",-62],PARAMETER[\"scale_factor\",0.9995],PARAMETER[\ + \"false_easting\",400000],PARAMETER[\"false_northing\",0],AUTHORITY[\"EPSG\",\ + \"2002\"],AXIS[\"Easting\",EAST],AXIS[\"Northing\",NORTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2003, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2003, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"Grenada 1953 / British West Indies Grid\ + \",GEOGCS[\"Grenada 1953\",DATUM[\"Grenada_1953\",SPHEROID[\"Clarke 1880 (RGS)\ + \",6378249.145,293.465,AUTHORITY[\"EPSG\",\"7012\"]],TOWGS84[72,213.7,93,0,0,0,0],AUTHORITY[\ + \"EPSG\",\"6603\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\ + \"degree\",0.01745329251994328,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\ + \",\"4603\"]],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],PROJECTION[\ + \"Transverse_Mercator\"],PARAMETER[\"latitude_of_origin\",0],PARAMETER[\ + \"central_meridian\",-62],PARAMETER[\"scale_factor\",0.9995],PARAMETER[\ + \"false_easting\",400000],PARAMETER[\"false_northing\",0],AUTHORITY[\"EPSG\",\ + \"2003\"],AXIS[\"Easting\",EAST],AXIS[\"Northing\",NORTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2004, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2004, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"Montserrat 1958 / British West Indies Grid\ + \",GEOGCS[\"Montserrat 1958\",DATUM[\"Montserrat_1958\",SPHEROID[\ + \"Clarke 1880 (RGS)\",6378249.145,293.465,AUTHORITY[\"EPSG\",\"7012\ + \"]],TOWGS84[174,359,365,0,0,0,0],AUTHORITY[\"EPSG\",\"6604\"]],PRIMEM[\ + \"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.01745329251994328,AUTHORITY[\ + \"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4604\"]],UNIT[\"metre\ + \",1,AUTHORITY[\"EPSG\",\"9001\"]],PROJECTION[\"Transverse_Mercator\ + \"],PARAMETER[\"latitude_of_origin\",0],PARAMETER[\"central_meridian\ + \",-62],PARAMETER[\"scale_factor\",0.9995],PARAMETER[\"false_easting\ + \",400000],PARAMETER[\"false_northing\",0],AUTHORITY[\"EPSG\",\"2004\ + \"],AXIS[\"Easting\",EAST],AXIS[\"Northing\",NORTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2005, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2005, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"St. Kitts 1955 / British West Indies Grid\ + \",GEOGCS[\"St. Kitts 1955\",DATUM[\"St_Kitts_1955\",SPHEROID[\"Clarke 1880 (RGS)\ + \",6378249.145,293.465,AUTHORITY[\"EPSG\",\"7012\"]],AUTHORITY[\"EPSG\",\ + \"6605\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\ + \"degree\",0.01745329251994328,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\ + \"EPSG\",\"4605\"]],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],PROJECTION[\ + \"Transverse_Mercator\"],PARAMETER[\"latitude_of_origin\",0],PARAMETER[\ + \"central_meridian\",-62],PARAMETER[\"scale_factor\",0.9995],PARAMETER[\ + \"false_easting\",400000],PARAMETER[\"false_northing\",0],AUTHORITY[\"EPSG\",\ + \"2005\"],AXIS[\"Easting\",EAST],AXIS[\"Northing\",NORTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2006, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2006, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"St. Lucia 1955 / British West Indies Grid\ + \",GEOGCS[\"St. Lucia 1955\",DATUM[\"St_Lucia_1955\",SPHEROID[\"Clarke 1880 (RGS)\ + \",6378249.145,293.465,AUTHORITY[\"EPSG\",\"7012\"]],TOWGS84[-149,128,296,0,0,0,0],AUTHORITY[\ + \"EPSG\",\"6606\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\ + \"]],UNIT[\"degree\",0.01745329251994328,AUTHORITY[\"EPSG\",\"9122\ + \"]],AUTHORITY[\"EPSG\",\"4606\"]],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\ + \"9001\"]],PROJECTION[\"Transverse_Mercator\"],PARAMETER[\"latitude_of_origin\ + \",0],PARAMETER[\"central_meridian\",-62],PARAMETER[\"scale_factor\",0.9995],PARAMETER[\ + \"false_easting\",400000],PARAMETER[\"false_northing\",0],AUTHORITY[\"EPSG\",\ + \"2006\"],AXIS[\"Easting\",EAST],AXIS[\"Northing\",NORTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2007, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2007, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"St. Vincent 45 / British West Indies Grid\ + \",GEOGCS[\"St. Vincent 1945\",DATUM[\"St_Vincent_1945\",SPHEROID[\ + \"Clarke 1880 (RGS)\",6378249.145,293.465,AUTHORITY[\"EPSG\",\"7012\ + \"]],TOWGS84[195.671,332.517,274.607,0,0,0,0],AUTHORITY[\"EPSG\",\"6607\ + \"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\ + \",0.01745329251994328,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\ + \"4607\"]],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],PROJECTION[\ + \"Transverse_Mercator\"],PARAMETER[\"latitude_of_origin\",0],PARAMETER[\ + \"central_meridian\",-62],PARAMETER[\"scale_factor\",0.9995],PARAMETER[\ + \"false_easting\",400000],PARAMETER[\"false_northing\",0],AUTHORITY[\"EPSG\",\ + \"2007\"],AXIS[\"Easting\",EAST],AXIS[\"Northing\",NORTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2008, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2008, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"NAD27(CGQ77) / SCoPQ zone 2\ + \",GEOGCS[\"NAD27(CGQ77)\",DATUM[\"North_American_Datum_1927_CGQ77\",SPHEROID[\ + \"Clarke 1866\",6378206.4,294.9786982138982,AUTHORITY[\"EPSG\",\"7008\"]],AUTHORITY[\ + \"EPSG\",\"6609\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\ + \",0.01745329251994328,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\ + \"4609\"]],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],PROJECTION[\ + \"Transverse_Mercator\"],PARAMETER[\"latitude_of_origin\",0],PARAMETER[\ + \"central_meridian\",-55.5],PARAMETER[\"scale_factor\",0.9999],PARAMETER[\ + \"false_easting\",304800],PARAMETER[\"false_northing\",0],AUTHORITY[\"EPSG\",\ + \"2008\"],AXIS[\"X\",EAST],AXIS[\"Y\",NORTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2009, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2009, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"NAD27(CGQ77) / SCoPQ zone 3\ + \",GEOGCS[\"NAD27(CGQ77)\",DATUM[\"North_American_Datum_1927_CGQ77\",SPHEROID[\ + \"Clarke 1866\",6378206.4,294.9786982138982,AUTHORITY[\"EPSG\",\"7008\"]],AUTHORITY[\ + \"EPSG\",\"6609\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\ + \"degree\",0.01745329251994328,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\ + \"4609\"]],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],PROJECTION[\"Transverse_Mercator\ + \"],PARAMETER[\"latitude_of_origin\",0],PARAMETER[\"central_meridian\",-58.5],PARAMETER[\ + \"scale_factor\",0.9999],PARAMETER[\"false_easting\",304800],PARAMETER[\"false_northing\ + \",0],AUTHORITY[\"EPSG\",\"2009\"],AXIS[\"X\",EAST],AXIS[\"Y\",NORTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2010, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2010, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"NAD27(CGQ77) / SCoPQ zone 4\ + \",GEOGCS[\"NAD27(CGQ77)\",DATUM[\"North_American_Datum_1927_CGQ77\",SPHEROID[\ + \"Clarke 1866\",6378206.4,294.9786982138982,AUTHORITY[\"EPSG\",\"7008\"]],AUTHORITY[\ + \"EPSG\",\"6609\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\ + \",0.01745329251994328,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4609\"]],UNIT[\ + \"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],PROJECTION[\"Transverse_Mercator\"],PARAMETER[\ + \"latitude_of_origin\",0],PARAMETER[\"central_meridian\",-61.5],PARAMETER[\ + \"scale_factor\",0.9999],PARAMETER[\"false_easting\",304800],PARAMETER[\ + \"false_northing\",0],AUTHORITY[\"EPSG\",\"2010\"],AXIS[\"X\",EAST],AXIS[\ + \"Y\",NORTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2011, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2011, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"NAD27(CGQ77) / SCoPQ zone 5\ + \",GEOGCS[\"NAD27(CGQ77)\",DATUM[\"North_American_Datum_1927_CGQ77\ + \",SPHEROID[\"Clarke 1866\",6378206.4,294.9786982138982,AUTHORITY[\"EPSG\ + \",\"7008\"]],AUTHORITY[\"EPSG\",\"6609\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\ + \"EPSG\",\"8901\"]],UNIT[\"degree\",0.01745329251994328,AUTHORITY[\"EPSG\ + \",\"9122\"]],AUTHORITY[\"EPSG\",\"4609\"]],UNIT[\"metre\",1,AUTHORITY[\ + \"EPSG\",\"9001\"]],PROJECTION[\"Transverse_Mercator\"],PARAMETER[\ + \"latitude_of_origin\",0],PARAMETER[\"central_meridian\",-64.5],PARAMETER[\ + \"scale_factor\",0.9999],PARAMETER[\"false_easting\",304800],PARAMETER[\ + \"false_northing\",0],AUTHORITY[\"EPSG\",\"2011\"],AXIS[\"X\",EAST],AXIS[\ + \"Y\",NORTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2012, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2012, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"NAD27(CGQ77) / SCoPQ zone 6\ + \",GEOGCS[\"NAD27(CGQ77)\",DATUM[\"North_American_Datum_1927_CGQ77\ + \",SPHEROID[\"Clarke 1866\",6378206.4,294.9786982138982,AUTHORITY[\"EPSG\ + \",\"7008\"]],AUTHORITY[\"EPSG\",\"6609\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\ + \"EPSG\",\"8901\"]],UNIT[\"degree\",0.01745329251994328,AUTHORITY[\"EPSG\ + \",\"9122\"]],AUTHORITY[\"EPSG\",\"4609\"]],UNIT[\"metre\",1,AUTHORITY[\ + \"EPSG\",\"9001\"]],PROJECTION[\"Transverse_Mercator\"],PARAMETER[\"latitude_of_origin\ + \",0],PARAMETER[\"central_meridian\",-67.5],PARAMETER[\"scale_factor\",0.9999],PARAMETER[\ + \"false_easting\",304800],PARAMETER[\"false_northing\",0],AUTHORITY[\"EPSG\",\ + \"2012\"],AXIS[\"X\",EAST],AXIS[\"Y\",NORTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2013, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2013, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"NAD27(CGQ77) / SCoPQ zone 7\ + \",GEOGCS[\"NAD27(CGQ77)\",DATUM[\"North_American_Datum_1927_CGQ77\ + \",SPHEROID[\"Clarke 1866\",6378206.4,294.9786982138982,AUTHORITY[\"EPSG\ + \",\"7008\"]],AUTHORITY[\"EPSG\",\"6609\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\ + \"EPSG\",\"8901\"]],UNIT[\"degree\",0.01745329251994328,AUTHORITY[\"EPSG\ + \",\"9122\"]],AUTHORITY[\"EPSG\",\"4609\"]],UNIT[\"metre\",1,AUTHORITY[\ + \"EPSG\",\"9001\"]],PROJECTION[\"Transverse_Mercator\"],PARAMETER[\"latitude_of_origin\ + \",0],PARAMETER[\"central_meridian\",-70.5],PARAMETER[\"scale_factor\",0.9999],PARAMETER[\ + \"false_easting\",304800],PARAMETER[\"false_northing\",0],AUTHORITY[\"EPSG\ + \",\"2013\"],AXIS[\"X\",EAST],AXIS[\"Y\",NORTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2014, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2014, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"NAD27(CGQ77) / SCoPQ zone 8\ + \",GEOGCS[\"NAD27(CGQ77)\",DATUM[\"North_American_Datum_1927_CGQ77\",SPHEROID[\ + \"Clarke 1866\",6378206.4,294.9786982138982,AUTHORITY[\"EPSG\",\"7008\ + \"]],AUTHORITY[\"EPSG\",\"6609\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\ + \"8901\"]],UNIT[\"degree\",0.01745329251994328,AUTHORITY[\"EPSG\",\"9122\ + \"]],AUTHORITY[\"EPSG\",\"4609\"]],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\ + \"9001\"]],PROJECTION[\"Transverse_Mercator\"],PARAMETER[\"latitude_of_origin\ + \",0],PARAMETER[\"central_meridian\",-73.5],PARAMETER[\"scale_factor\ + \",0.9999],PARAMETER[\"false_easting\",304800],PARAMETER[\"false_northing\ + \",0],AUTHORITY[\"EPSG\",\"2014\"],AXIS[\"X\",EAST],AXIS[\"Y\",NORTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2015, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2015, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"NAD27(CGQ77) / SCoPQ zone 9\ + \",GEOGCS[\"NAD27(CGQ77)\",DATUM[\"North_American_Datum_1927_CGQ77\",SPHEROID[\ + \"Clarke 1866\",6378206.4,294.9786982138982,AUTHORITY[\"EPSG\",\"7008\"]],AUTHORITY[\ + \"EPSG\",\"6609\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\ + \",0.01745329251994328,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\ + \"4609\"]],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],PROJECTION[\"Transverse_Mercator\ + \"],PARAMETER[\"latitude_of_origin\",0],PARAMETER[\"central_meridian\",-76.5],PARAMETER[\ + \"scale_factor\",0.9999],PARAMETER[\"false_easting\",304800],PARAMETER[\ + \"false_northing\",0],AUTHORITY[\"EPSG\",\"2015\"],AXIS[\"X\",EAST],AXIS[\ + \"Y\",NORTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2016, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2016, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"NAD27(CGQ77) / SCoPQ zone 10\ + \",GEOGCS[\"NAD27(CGQ77)\",DATUM[\"North_American_Datum_1927_CGQ77\ + \",SPHEROID[\"Clarke 1866\",6378206.4,294.9786982138982,AUTHORITY[\"EPSG\ + \",\"7008\"]],AUTHORITY[\"EPSG\",\"6609\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\ + \"EPSG\",\"8901\"]],UNIT[\"degree\",0.01745329251994328,AUTHORITY[\"EPSG\",\ + \"9122\"]],AUTHORITY[\"EPSG\",\"4609\"]],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\ + \"9001\"]],PROJECTION[\"Transverse_Mercator\"],PARAMETER[\"latitude_of_origin\ + \",0],PARAMETER[\"central_meridian\",-79.5],PARAMETER[\"scale_factor\ + \",0.9999],PARAMETER[\"false_easting\",304800],PARAMETER[\"false_northing\ + \",0],AUTHORITY[\"EPSG\",\"2016\"],AXIS[\"X\",EAST],AXIS[\"Y\",NORTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + table->field[0]->store(2017, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(2017, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("PROJCS[\"NAD27(76) / MTM zone 8\ + \",GEOGCS[\"NAD27(76)\",DATUM[\"North_American_Datum_1927_1976\",SPHEROID[\ + \"Clarke 1866\",6378206.4,294.9786982138982,AUTHORITY[\"EPSG\",\"7008\ + \"]],AUTHORITY[\"EPSG\",\"6608\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\ + \"8901\"]],UNIT[\"degree\",0.01745329251994328,AUTHORITY[\"EPSG\",\"9122\ + \"]],AUTHORITY[\"EPSG\",\"4608\"]],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\ + \"9001\"]],PROJECTION[\"Transverse_Mercator\"],PARAMETER[\"latitude_of_origin\ + \",0],PARAMETER[\"central_meridian\",-73.5],PARAMETER[\"scale_factor\ + \",0.9999],PARAMETER[\"false_easting\",304800],PARAMETER[\"false_northing\ + \",0],AUTHORITY[\"EPSG\",\"2017\"],AXIS[\"X\",EAST],AXIS[\"Y\",NORTH]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; + + + table->field[0]->store(3819, TRUE); /*SRID*/ + table->field[1]->store(STRING_WITH_LEN("EPSG"), cs); /*AUTH_NAME*/ + table->field[2]->store(3819, TRUE); /*AUTH_SRID*/ + table->field[3]->store(STRING_WITH_LEN("GEOGCS[\"HD1909\",DATUM[\"Hungarian Datum 1909\",\ + SPHEROID[\"Bessel 1841\",6377397.155,299.1528128,AUTHORITY[\"EPSG\",\"7004\"]],\ + TOWGS84[595.48,121.69,515.35,4.115,2.9383,-0.853,-0.7029504596901044],\ + AUTHORITY[\"EPSG\",\"1024\"]],PRIMEM[\"Greenwich\",0.0,AUTHORITY[\"EPSG\",\ + \"8901\"]],UNIT[\"degree\",0.017453292519943295],AXIS[\"Geodetic latitude\",\ + NORTH],AXIS[\"Geodetic longitude\",EAST],AUTHORITY[\"EPSG\",\"3819\"]]"), cs);/*SRTEXT*/ + if (schema_table_store_record(thd, table)) + goto exit; result= 0;

1 0

Re: [Maria-developers] 622611b: MDEV-7012: SET STATEMENT character_set_client, character_set_connection, character_set_filesystem, collation_connection have no effect
by Sergei Golubchik 02 Apr '15

02 Apr '15

Hi, Sanja! On Mar 17, sanja(a)mariadb.com wrote: > revision-id: 622611bc6a9371803f349d8cf4de6538ebbcdb37 > parent(s): a21ef88d2a358904f0cdf9cadc701cb263b5e317 > committer: Oleksandr Byelkin > branch nick: server > timestamp: 2015-03-17 14:40:25 +0100 > message: > > MDEV-7012: SET STATEMENT character_set_client, character_set_connection, character_set_filesystem, collation_connection have no effect > > Variables of simple SET STATEMENT set just after assigning > > (changes in test suite result checked) While it's a neat innovative trick, I don't think we should execute statements (and setting variables is executing) from the parser. It's very fragile and breaks in various corner cases, resulting in unfixable gotchas. For example, how would the following work: CREATE PROCEDURE foo BEGIN SET STATEMENT CHARACTER_SET_CONNECTION = latin5 FOR SELECT 'фи'; END So, I'd prefer a simple rule "first - the statement is parsed, then it is executed", meaning no effects from the statement execution can affect how it was parsed. Regards, Sergei

1 0

[Maria-developers] Please review MDEV-7284 INDEX: CREATE OR REPLACE
by Alexander Barkov 02 Apr '15

02 Apr '15

Hi Sergei, Please review a patch for MDEV-7284 Thanks.

2 1

Re: [Maria-developers] [Commits] 135d926d3: MDEV-6714 mysqldump slow with tables in big databases
by Sergei Golubchik 02 Apr '15

02 Apr '15

Hi, Vicentiu! Just one minor change, see the comment in get_actual_table_name(). After that please push. Well done, thanks! Regards, Sergei On Mar 19, vicentiu(a)mariadb.org wrote: > revision-id: 135d926d347443672626f1e655709fd5bf6dec80 > parent(s): 197afb413fcc9f06b5e5e6ef41ce980d108b354f > committer: Vicențiu Ciorbaru > branch nick: server > timestamp: 2015-03-19 15:16:22 +0200 > message: > > MDEV-6714 mysqldump slow with tables in big databases > > mysqldump now attempts to make use of the INFORMATION_SCHEMA tables. > If the table name is not found with a case sensitive search, it > fallbacks to a case insensitive search. > > --- > client/mysqldump.c | 90 ++++++++++++++++++++++++++++++++++++++++++------------ > 1 file changed, 71 insertions(+), 19 deletions(-) > > diff --git a/client/mysqldump.c b/client/mysqldump.c > index 2da4ce6..c265217 100644 > --- a/client/mysqldump.c > +++ b/client/mysqldump.c > @@ -3341,8 +3341,10 @@ static int dump_triggers_for_table(char *table_name, char *db_name) > /* Get list of triggers. */ > > my_snprintf(query_buff, sizeof(query_buff), > - "SHOW TRIGGERS LIKE %s", > - quote_for_like(table_name, name_buff)); > + "SELECT TRIGGER_NAME FROM INFORMATION_SCHEMA.TRIGGERS " > + "WHERE EVENT_OBJECT_SCHEMA = DATABASE() AND " > + "EVENT_OBJECT_TABLE = \"%s\"", > + table_name); You still need to quote table_name, it might contain a double quote inside. quote_names() almost does what you need, but you don't need a check for MASK_ANSI_QUOTES. So you might either create a quote_names_helper, again, or simply create a new function for this, it's just a few lines anyway. UPD: There are other similar places in mysqldump where it doesn't quote table names, e.g. in get_table_structure(), for show_fields_stmt query. So, please create a separate bug report for this and then it'll fix everything at once. > if (mysql_query_with_error_report(mysql, &show_triggers_rs, query_buff)) > goto done; > @@ -4695,29 +4697,37 @@ static my_bool dump_all_views_in_db(char *database) > > > /* > - get_actual_table_name -- executes a SHOW TABLES LIKE '%s' to get the actual > - table name from the server for the table name given on the command line. > - we do this because the table name given on the command line may be a > - different case (e.g. T1 vs t1) > - > - RETURN > - pointer to the table name > - 0 if error > + See get_actual_table_name. Used to retrieve the correct table name > + from the database schema. > */ > - > -static char *get_actual_table_name(const char *old_table_name, MEM_ROOT *root) > +static char *get_actual_table_name_helper(const char *old_table_name, > + my_bool case_sensitive, > + MEM_ROOT *root) > { > char *name= 0; > MYSQL_RES *table_res; > MYSQL_ROW row; > char query[50 + 2*NAME_LEN]; > char show_name_buff[FN_REFLEN]; > - DBUG_ENTER("get_actual_table_name"); > + DBUG_ENTER("get_actual_table_name_helper"); > > /* Check memory for quote_for_like() */ > DBUG_ASSERT(2*sizeof(old_table_name) < sizeof(show_name_buff)); > - my_snprintf(query, sizeof(query), "SHOW TABLES LIKE %s", > - quote_for_like(old_table_name, show_name_buff)); > + > + if (case_sensitive) > + { > + DBUG_PRINT("info", ("case sensitive search")); > + my_snprintf(query, sizeof(query), > + "SELECT table_name FROM INFORMATION_SCHEMA.TABLES " > + "WHERE table_schema = DATABASE() AND table_name = \"%s\"", > + old_table_name); > + } > + else > + { > + DBUG_PRINT("info", ("case insensitive search")); > + my_snprintf(query, sizeof(query), "SHOW TABLES LIKE %s", > + quote_for_like(old_table_name, show_name_buff)); > + } > > if (mysql_query_with_error_report(mysql, 0, query)) > return NullS; > @@ -4742,6 +4752,46 @@ static char *get_actual_table_name(const char *old_table_name, MEM_ROOT *root) > DBUG_RETURN(name); > } > > +/* > + get_actual_table_name -- executes a SELECT .. FROM I_S.tables to check > + if the table name given on the command line matches the one in the database. > + If the table is not found, it falls back to a slower SHOW TABLES LIKE '%s' to > + get the actual table name from the server. > + > + We do this because the table name given on the command line may be a > + different case (e.g. T1 vs t1), but checking this takes a long time > + when there are many tables present. > + > + RETURN > + pointer to the table name > + 0 if error > +*/ > + > +static char *get_actual_table_name(const char *old_table_name, MEM_ROOT *root) > +{ > + char *name= 0; > + MYSQL_RES *table_res; > + MYSQL_ROW row; > + int lower_case_table_names; > + const char *show_var_query = "SHOW VARIABLES LIKE 'lower_case_table_names'"; > + DBUG_ENTER("get_actual_table_name"); > + > + if (mysql_query_with_error_report(mysql, &table_res, show_var_query)) > + return NullS; I'd prefer if you do it only once, not for every get_actual_table_name() call. > + > + if ((row= mysql_fetch_row(table_res))) > + { > + lower_case_table_names= atoi(row[1]); > + mysql_free_result(table_res); > + } > + > + name= get_actual_table_name_helper(old_table_name, TRUE, root); > + if (!name && !lower_case_table_names) > + name= get_actual_table_name_helper(old_table_name, FALSE, root); > + DBUG_RETURN(name); > +} > + > + > > static int dump_selected_tables(char *db, char **table_names, int tables) > {

1 0

[Maria-developers] Problem of accessing pfs table in maria version 10.1.3
by 김경남 01 Apr '15

01 Apr '15

Hi, I'm developing monitoring tool in daemon plugin I'm tyring to access pfs table via low level interface not using sql. (see below) ------------------------------------------------------------------------ ------------- ..... tables.init_one_table(STRING_WITH_LEN("performance_schema"), STRING_WITH_LEN("events_statements_current"), "events_statements_current", TL_READ); bool result = open_and_lock_tables(ctx->thd, &tables, FALSE, MYSQL_LOCK_IGNORE_TIMEOUT); if (result) { sql_print_error("monm plugin error: Can't open the %s.%s table.",dbname,tblname); } ..... /* read routine */ ------------------------------------------------------------ In open_and_lock_table function, I hava a error. I found the reason of this error by debugging. In table_cache.cc, open_table_def() requires .frm file as a table definition. but above version 10.1.3, there is no frm file for pfs table as I know. right? Let me know a correct way to get data from pfs table not using sql. Regards, Kyungnam. _____ 상기 메시지와 첨부화일 내에는 비밀정보가 포함되어 있을 수 있으며, 지정된 수신자에 한하여 조회 및 사용될 수 있습니다. 만약 송신자의 실수로 인하여 상기 메시지를 수신하였다면, 송신자에게 메시지를 반송해 주시고, 원본 메시지와 모든 사본을 폐기해 주시기 바랍니다. 상기 메시지의 전체 또는 일부에 대해 무단 열람, 사용, 공개, 배포하는 것은 금지되어 있습니다.(주)LG CNS. This message and its attachments may contain confidential information, and they are intended to be viewed or used by only the individuals specified in the message. If you have received this message in an error from the sender, please contact the sender immediately to notify the error and delete all of the message and its copies. It is prohibited to view, use, make public and/or distribute part or whole of this message without written permission.

1 0

[Maria-developers] Using THD instance in a daemon thread.
by 김경남 31 Mar '15

31 Mar '15

Hi, I'm writing mariadb daemon type plugin. I hava a question. In a daemon thread, I want to use THD instance. for example, ------------------------------------------------------------------------ ----------- class THD; static int plugin_init() { ... pthread_create(&id,&attr,my_func,...); } void my_func(void* p) { THD* thd = new THD; thd->thread_stack= (char*) &thd; thd->store_globals(); tables.init_one_table("performance_schema", 18, "events_statements_current", 25, "events_statements_current", TL_READ); open_and_lock_tables(thd, &tables, FALSE, MYSQL_LOCK_IGNORE_TIMEOUT); ... } ------------------------------------------------------------------------ -------------------- but In my_func(), new THD cause error. I guess thread problem. So I moved that code to plugin_init() for test as below -------------------------------------------------------------- class THD; static int plugin_init() { ... pthread_create(&id,&attr,my_func,...); THD* thd = new THD; thd->thread_stack= (char*) &thd; thd->store_globals(); tables.init_one_table("performance_schema", 18, "events_statements_current", 25, "events_statements_current", TL_READ); open_and_lock_tables(thd, &tables, FALSE, MYSQL_LOCK_IGNORE_TIMEOUT); ... } ------------------------------------------------------------------------ ----- and then there is no problem. but I need a daemon thread. I think that the reason of this error is assertion of thread. So I changed my code as below. ------------------------------------------------------------------------ --------- class THD; THD* thd; static int plugin_init() { ... pthread_create(&id,&attr,my_func,...); THD* thd = new THD; // moved here from my_func() for avoiding error. } void my_func(void* p) { thd->thread_stack= (char*) &thd; thd->store_globals(); tables.init_one_table("performance_schema", 18, "events_statements_current", 25, "events_statements_current", TL_READ); open_and_lock_tables(thd, &tables, FALSE, MYSQL_LOCK_IGNORE_TIMEOUT); ... } ------------------------------------------------------------------------ ----------------- At this time, thd->store_globals() cause error. In sql_class.cc , mysys_var->id= thread_id cause error because mysys_var is null. Let me know how to use THD instance in daemon thread. Regards, Kyungnam 김경남 부책임연구원 솔루션사업본부 정보기술 연구원 신기술개발팀 02-2099-6177 / 010-2208-3006 _____ 상기 메시지와 첨부화일 내에는 비밀정보가 포함되어 있을 수 있으며, 지정된 수신자에 한하여 조회 및 사용될 수 있습니다. 만약 송신자의 실수로 인하여 상기 메시지를 수신하였다면, 송신자에게 메시지를 반송해 주시고, 원본 메시지와 모든 사본을 폐기해 주시기 바랍니다. 상기 메시지의 전체 또는 일부에 대해 무단 열람, 사용, 공개, 배포하는 것은 금지되어 있습니다.(주)LG CNS. This message and its attachments may contain confidential information, and they are intended to be viewed or used by only the individuals specified in the message. If you have received this message in an error from the sender, please contact the sender immediately to notify the error and delete all of the message and its copies. It is prohibited to view, use, make public and/or distribute part or whole of this message without written permission.

2 1

[Maria-developers] ok to change FixVersion to 10.1 for MDEV-7649 wrong result when comparing utf8 column with an invalid literal
by Alexander Barkov 31 Mar '15

31 Mar '15

Hi Sergei, Do you think it's ok to change FixVersion for MDEV-7649 to 10.1? It requires some changes in sql_lex.cc, sql_yacc.yy and needs a new Item_string-alike Item to handle such malformed strings properly. Too big change to 10.0 or 5.5.

2 1

[Maria-developers] TokuDB test suite failures when building at Launchpad
by Otto Kekäläinen 29 Mar '15

29 Mar '15

Hello! Are here any TokuDB devs who could help me debug why TokuDB does repeatedly fails to pass build suite tests when built on Launchpad.net? I repeatedly get these test failures when running MariaDB builds on Launchpad: rpl-tokudb.tokudb_innodb_xa_crash tokudb_bugs.mdev4533 tokudb.simple_join_tokudb_innodb tokudb.truncate_txn_rollback_innodb tokudb_bugs.5733_innodb The build run log can be viewed at https://launchpad.net/~mysql-ubuntu/+archive/ubuntu/mariadb-10.0/+builds?bu… and it links to the individual build logs. All amd64 builds fail while all i386 (where TokuDB is disabled) pass OK. Sounds very much like a TokuDB-related issue. Or could the problem be something else and the TokuDB tests are just the first victim? Example of a recent log: https://launchpadlibrarian.net/196425361/buildlog_ubuntu-vivid-amd64.mariad… One example of the test outputs: *** rpl-tokudb.tokudb_innodb_xa_crash 'mix,xtradb' [ fail ] Test ended at 2015-02-02 22:29:06 CURRENT_TEST: rpl-tokudb.tokudb_innodb_xa_crash Failed to start mysqld.1 mysqltest failed but provided no output - saving '/build/buildd/mariadb-10.0-10.0.16/builddir/mysql-test/var/log/rpl-tokudb.tokudb_innodb_xa_crash-mix,xtradb/' to '/build/buildd/mariadb-10.0-10.0.16/builddir/mysql-test/var/log/rpl-tokudb.tokudb_innodb_xa_crash-mix,xtradb/' - found 'core' (1/5) Trying 'dbx' to get a backtrace Trying 'gdb' to get a backtrace Compressed file /build/buildd/mariadb-10.0-10.0.16/builddir/mysql-test/var/log/rpl-tokudb.tokudb_innodb_xa_crash-mix,xtradb/mysqld.1/data/core ***Warnings generated in error logs during shutdown after running tests: rpl-tokudb.tokudb_innodb_xa_crash 150202 22:29:05 [ERROR] mysqld got signal 11 ; Attempting backtrace. You can use the following information to find out 150202 22:29:05 [ERROR] mysqld got signal 11 ; Attempting backtrace. You can use the following information to find out ***

4 15

[Maria-developers] GSoC Proposal Submitted.
by arlene seuleu 28 Mar '15

28 Mar '15

Hello everyone, I am Arlene Flavie, currently a Sofeware Engineering student and I have submitted a proposal for the CONC-125 task: "Imporrt and Export popular data formats from and to dynamic columns". I had been in contact with the mentor and other mariadb developpers(serg, danblack) whose help was immense in writing my proposal. However I am still available to discuss it here and on #maria as Arlene. Hoping to get feedback from the community soon. Best Regards Arlene Flavie

1 0

[Maria-developers] MySQL 5.7.6 adds "statement digest" independent of P_S
by Peter Laursen 27 Mar '15

27 Mar '15

.. as well as configurable digest length. http://mysqlserverteam.com/mysql-statement-digests-configurable-max-lengths/ http://mysqlserverteam.com/mysql-statement-digests/ Will this be ported to MariaDB? IMO this is/was one of the (not so many) useful things in P_S. And better now. -- Peter -- Webyog

1 0

[Maria-developers] In daemon plugin, THD problem.
by 김경남 27 Mar '15

27 Mar '15

Hi, Sergei! I'm writing daemon plugin . I hava a problem and don't find soultion yet. There is my code as below. ex)plugin.cc ----------------------------------------------------- pthread_handler_t monm_monitor(void *p) { .... THD* thd; thd = new THD; thd->thread_stack= (char*) &thd; tables.init_one_table("performance_schema", 18, "events_statements_current", 25, "events_statements_current", TL_READ); open_and_lock_tables(thd, &tables, FALSE, MYSQL_LOCK_IGNORE_TIMEOUT); ..... } static int monm_plugin_init(void *p __attribute__ ((unused))) { ... pthread_create(&mon_thread, &attr, monm_monitor,(void *) ctx)!= 0) .... } ------------------------------------------------------------- In monm_monitor function, THD* thd cause server crash. so I examined coredump. #6 0x0000000000598752 in THD::THD (this=0x7f9b64000930, is_wsrep_applier=false) at /home/windfree/workspace/ws-mariadb/maria-mon/sql/sql_class.cc:1015 I don't know the reason of crash. What I just want to is read events_statements_current's row by using internal interface in a daemon plugin. Is there a another way ? let me know the solution. Regards, Kyungnam. 김경남 부책임연구원 솔루션사업본부 정보기술 연구원 신기술개발팀 02-2099-6177 / 010-2208-3006 _____ 상기 메시지와 첨부화일 내에는 비밀정보가 포함되어 있을 수 있으며, 지정된 수신자에 한하여 조회 및 사용될 수 있습니다. 만약 송신자의 실수로 인하여 상기 메시지를 수신하였다면, 송신자에게 메시지를 반송해 주시고, 원본 메시지와 모든 사본을 폐기해 주시기 바랍니다. 상기 메시지의 전체 또는 일부에 대해 무단 열람, 사용, 공개, 배포하는 것은 금지되어 있습니다.(주)LG CNS. This message and its attachments may contain confidential information, and they are intended to be viewed or used by only the individuals specified in the message. If you have received this message in an error from the sender, please contact the sender immediately to notify the error and delete all of the message and its copies. It is prohibited to view, use, make public and/or distribute part or whole of this message without written permission.

2 2

Re: [Maria-developers] [Org FW][RE]Re: init_one_table problem.
by sergei 25 Mar '15

25 Mar '15

Hi, 김경남! On Mar 25, 김경남 wrote: > Hi, Sergei ! > > below contents is my previous question . > > After sending email , I see the debug message.like below. > > "Assertion thd == current_thd() failed." > > My code contains. > > THD thd = new THD; > > so I changed my code as > > THD thd = current_thd; > > but thd is null. > > how to I fix it? Note that sql_acl.cc (for example) has THD *thd=new THD; thd->thread_stack= (char*) &thd; thd->store_globals(); All three lines are important. In particular, THD::store_global sets current_thd. Regards, Sergei

1 0

[Maria-developers] init_one_table problem.
by 김경남 25 Mar '15

25 Mar '15

Hi ~ I hava a problem below code in my plugin. ---------------------------------- TABLE_LIST tables; TABLE table; tables.init_one_table(C_STRING_WITH_LEN("performance_schema"),C_STRING_W ITH_LEN("events_statements_current"), events_statements_current, TL_READ); if(table = tables.table) {...} ------------------------------------------------------- During debugging, I found that tables.table is null. Is there a additional task before my code? Regards, Kyungnam. 김경남 부책임연구원 솔루션사업본부 정보기술 연구원 신기술개발팀 02-2099-6177 / 010-2208-3006 _____ 상기 메시지와 첨부화일 내에는 비밀정보가 포함되어 있을 수 있으며, 지정된 수신자에 한하여 조회 및 사용될 수 있습니다. 만약 송신자의 실수로 인하여 상기 메시지를 수신하였다면, 송신자에게 메시지를 반송해 주시고, 원본 메시지와 모든 사본을 폐기해 주시기 바랍니다. 상기 메시지의 전체 또는 일부에 대해 무단 열람, 사용, 공개, 배포하는 것은 금지되어 있습니다.(주)LG CNS. This message and its attachments may contain confidential information, and they are intended to be viewed or used by only the individuals specified in the message. If you have received this message in an error from the sender, please contact the sender immediately to notify the error and delete all of the message and its copies. It is prohibited to view, use, make public and/or distribute part or whole of this message without written permission.

2 1

Re: [Maria-developers] [RE]Re: I wonder what elements add to CMakeLists.txt
by Sergei Golubchik 25 Mar '15

25 Mar '15

Hi, kyungnam! On Mar 24, kyungnam wrote: > > I hava a question. > > I'm developing mariadb plugin . and I make a development environment > according to below url. > > https://mariadb.com/blog/compiling-debugging-mariadband-mysql-eclipse-scrat… > > I seperate source folder and package folder. > > During developing, I hava a trouble with too much build time. > > for example, after I edit sql_parse.cc (add one or two lines) , I run > command "make" under source root folder. > > The build time takes about 5 ~ 10min. and then I run command "make > package". That time takes about 10min. > > At first, I copied libmysqld.so.18 to package folder. but had no > effect. finally I run command "make package" and reinstall mariadb. > > I think I'm doing wrong things. > > please let me know the efficient way to build and debug. I'm not using Eclipse, so I'm afraid I cannot help you with that. If you just modify sql_parse.cc, the compilation should be fast and only that file has to be recompiled (and few binaries re-linked). If you see a lot of files recompiled than either you modified more than sql_parse.cc or your IDE is doing that for you. Make package needs to copy many files so it's a bit slow if you do it very often. I don't - unless I specifically work on packaging I run "make package" very rarely. You don't have to compile libmysqld every time. Run cmake -DWITH_EMBEDDED_SERVER=OFF in your build directory. Regards, Sergei

1 0

[Maria-developers] Merge 10.0 -> 10.1
by Kristian Nielsen 25 Mar '15

25 Mar '15

I have some patches in 10.0 that have not yet been merged to 10.1. Today I attempted the merge, but there were a few conflicts that I was not sure how to handle. Some charset-related functions touching fields that had moved to another struct, some error handling functions in XtraDB with new extra arguments. In case someone else does the merge, I just wanted to point out one conflict in one of my patches in the 10.0 -> 10.1 merge, as the correct resolution is not obvious: <<<<<<< HEAD if (mysql_bin_log.is_open()) return true; ||||||| merged common ancestors ======= if (mysql_bin_log.is_open() && opt_binlog_commit_wait_count > 0) return true; >>>>>>> origin/10.0 The correct conflict resolution here is to leave the code as it was in 10.1: if (mysql_bin_log.is_open()) return true; Or I can do the complete 10.0 -> 10.1 merge if necessary, figuring out the other conflicts, but I did not want to do that without coordinating, to avoid duplicating efforts. - Kristian.

2 1

[Maria-developers] GSoC 2015: Table functions (a.k.a UDF returning arrays / result set) MDEV-5199
by Diwas Joshi 25 Mar '15

25 Mar '15

Hello, I am Diwas, a Masters' student at IIIT Hyderabad, India. I am interested to work on "Table functions (a.k.a UDF returning arrays / result set)". I have downloaded and built MariaDB 10.1 source code from github. I started going through the code. I realized UDF's for scalar values is already implemented in the code, so i began to analyze from there. I got myself acquainted with the query work-flow. I believe the work will be similar to create_function from sql_udf.cc for some part. I understand that I'll need some real storage mechanism for executing the functions in queries so I'll need to use the handler class for this. Please tell me how much of the project I have understood correctly and if I am on the right lines here and how can i proceed towards writing my proposal for this project. regards Diwas Joshi

2 1

[Maria-developers] Port InnoDB memcached interface to MariaDB (GSOC 2015)
by Aalekh Nigam 24 Mar '15

24 Mar '15

Hello, I am Aalekh Nigam, 3rd Year Undergraduate student at JIIT Noida, India. I would like to propose my participation for Port InnoDB MemcacheInterface to MariaDB (https://mariadb.atlassian.net/browse/MDEV-4674) as GSOC 2015 Student. For the purpose of security following is the approach I have planned to accomplish for this project: The plugin will contain a xtradb_memcached.sql table for configuation option, these configuration options will also contain a username, password , tcp/udp address, port no. as column objects which will help us authenticate the connection for a specified user, thus on authentication the server would fetch corresponding tcp/udp address and port to connect with the memcached server. Hence this will not just prove an improvement to the existing mysql plugin but also provide a better security to the database. Nextly after connecting with a memcached server, our database will act as a NoSql system providing rapid user query transaction. Also please follow my proposal here: https://www.google-melange.com/gsoc/proposal/review/student/google/gsoc2015… Waiting for your comments regarding the approach. ---Aalekh

1 1

Re: [Maria-developers] I wonder what elements add to CMakeLists.txt
by 김경남 24 Mar '15

24 Mar '15

Hi, Sergei! Thanks for your advice and I agree with you. Accoring to your advice, It's not a good plan to pass some data to my plugin agent from sql_parser.cc because it needs to compile mariadb server. So I hava a another concept. Instead of put some code to sql_parse.cc, plugin collects a some data like below sample conceptual code. I_List_iterator<THD> it(threads); while ((tmp_thd=it++)) { if ((tmp_thd->get_command() == COM_QUERY) ||(tmp_thd->get_command() == COM_STMT_EXECUTE) ) continue; char* sql = tmp_thd->query(); if(sql) { ... send_data(); } } In this case, I though that it doesn't need to compile maria server, but it need to recompile whenever structure of THD is changed. Can you accept that? Otherwise, if i hardcode directly into the server, can I contribute this community ? Additionally, you asked me about what information I want to collect. I want to collect data like below. * events_statements_current * events_stages_current * events_waits_current * events_statements_summary_by_digest These data easily collected by sql . but it's hard to collect by access memory.(THD or something else.) I wonder if it is possible to collect these data without using pfs (or psi) data structure. If it is possible, Is THD class is entry point or something else? If you don't mind, please let me know the best way to collect these data. Dear, kyungnam. ps: my email address changed from gmail to my company's mail address. ------------------------------------------------------------------------ ---- From: Sergei Golubchik <serg(a)mariadb.org> Date: 2015-03-24 0:11 GMT+09:00 Subject: Re: [Maria-developers] I wonder what elements add to CMakeLists.txt To: kyung nam Kim <knamkim(a)gmail.com> Cc: maria-developers(a)lists.launchpad.net Hi, kyung! On Mar 23, kyung nam Kim wrote: > Hi, Sergei! > Thanks for your reply. > You asked me about information which I want to send to my plugin. > I have a plan to develop monitoring tool for mariadb. > I want to develop a agent as a mariadb plugin. > Agent collect data from sql_parse.cc (I' don't know exactly where more > infomation about performance .) and sends that data to APM (Application > Performance Monitoring) Server. and Users use databoard to see that data > via APM server. > > We have already have a APM Server, Dashboard and many kinds of agent. (for > server, was). I want to add agent for mariaDB and contribute this community > if possible. Right. That's exactly why I ask. We cannot accept your plugin and it'll be of little use to the community (not "of no use", it'll be used, but less than it could've been) if one would need to modify and recompile MariaDB to use your plugin. If you'd like, let's try to figure out whether you can collect your infomation without patching the server. So what data do you need, exactly? > Your advice is right. It's easier my header file to sql folder. > But When I want to develop my agent as a plugin type, I meet error as > follows. > ============================================ > [ 98%] Built target mysqlserver > Linking CXX shared library libmysqld.so > ../../sql_parse.cc:linenum: undefined reference to 'Mymon::Mymon()' > ../../sql_parse.cc:linenum: undefined reference to 'Mymon::putData(THD* > thd); Of course. Because you want sql_parse.cc to call your plugin, your plugin *always must be in the server*. You cannot have it as a separate shared object. To do that, add the MANDATORY keyword to your MYSQL_ADD_PLUGIN() declaration in CMakeLists.txt Regards, Sergei _____ 상기 메시지와 첨부화일 내에는 비밀정보가 포함되어 있을 수 있으며, 지정된 수신자에 한하여 조회 및 사용될 수 있습니다. 만약 송신자의 실수로 인하여 상기 메시지를 수신하였다면, 송신자에게 메시지를 반송해 주시고, 원본 메시지와 모든 사본을 폐기해 주시기 바랍니다. 상기 메시지의 전체 또는 일부에 대해 무단 열람, 사용, 공개, 배포하는 것은 금지되어 있습니다.(주)LG CNS. This message and its attachments may contain confidential information, and they are intended to be viewed or used by only the individuals specified in the message. If you have received this message in an error from the sender, please contact the sender immediately to notify the error and delete all of the message and its copies. It is prohibited to view, use, make public and/or distribute part or whole of this message without written permission.

2 3

[Maria-developers] I wonder what elements add to CMakeLists.txt
by kyung nam Kim 23 Mar '15

23 Mar '15

Dears. I'm a newer in maria develper community. I'm develping a simple plugin for monitoring. If I access from other module to my plugin interface, I wonder how and what elements add to CMakeLists.txt. for example, My plugin is daemon type and myplugin.h has a definition of public methodA. I want to use methodA in my plugin from sql/sql_parse.cc . I think that information of library is needed at build time. ( -L, -l option) I don't know how to add that elements to CMakeLists.txt. plz,let me know. Thanks in advance.

2 5

Re: [Maria-developers] [server] Execute sync_with_master before SELECT in rpl_gtid_basic.test (#31)
by Kristian Nielsen 23 Mar '15

23 Mar '15

f4rnham <notifications(a)github.com> writes: > Usually slave synchronized on its own before select, but not always. Those two lines were probably swapped by accident > You can view, comment on, or merge this pull request online at: Indeed, silly mistake... This was already fixed in 10.0, so it will be merged to 10.1 that way. But thanks for looking into this anyway, it's good to be aware of these kinds of races in replication tests, they are quite common. - Kristian.

1 0

Re: [Maria-developers] [JIRA] (MDEV-7557) Option to put into slow log any transaction that was retried in parallel replication
by Kristian Nielsen 23 Mar '15

23 Mar '15

"Sergei Golubchik (JIRA)" <jira(a)mariadb.atlassian.net> writes: >> Key: MDEV-7557 >> URL: https://mariadb.atlassian.net/browse/MDEV-7557 >> In optimistic parallel replication (MDEV-6676), we speculatively apply >> transactions in parallel, even if they might conflict. If too many conflicts >> happen, excessive retry might degrade performance. >> The DBA needs some way to monitor for such problems. One good way would seem >> to have an option to add to the slow query log any transactions that were >> retried due to parallel replication conflicts. Possibly with extra information >> such as duration of transaction and similar. > I'm not sure that slow log is a good match for this kind of events Right. So do you have a suggestion for a better way? I was thinking on --log-queries-not-using-indexes, which similarly uses the slow query log to expose certain kinds of problematic queries. So queries that cause deadlocks could be another useful filter for the slow log (it should then probably be for all queries that deadlock, not just replication queries). In non-parallel replication, we put deadlock errors as warnings in the error log. But I disabled that for parallel replication. In non-parallel replication, you normally do not expect any deadlocks (since there is only one thread, how can it deadlock?). But in parallel replication, deadlocks are generally unavoidable, and not harmful. And DBAs do not like noise in the error logs that does not indicate a problem and cannot be removed. One option might be to log deadlocks as INFO level in the error log, and then users can temporarily raise verbosity level to see them? Another option could be to add an option to enable/disable logging deadlocks as warnings in parallel replication. (Though probably you will think as me, that this feels a bit like too ad-hoc an option to add.) Any other ideas? - Kristian.

2 1

[Maria-developers] GSoC 2015: Indexes on Virtual Columns and Array Based UDFs
by Richa Sehgal 20 Mar '15

20 Mar '15

Dear mentors, I am currently pursuing Master's in University of Illinois at Urbana Champaign, USA and completed by B.Tech from Indian Institute of Technology - Delhi (IIT- Delhi) This is regarding GSoC 2015. I am really interested in databases, and I was very excited to see all these projects listed here. The exciting part was that some of the projects are really “hard” as in they have challenged the database community since a long time, and thus it would be very interesting to solve some of these challenges as part of GSoC. I want to discuss 2 projects: A. Indexes on virtual columns Materialization gives us two things: 1. A name to the column which we can use in queries 2. A formal "regular" column which is stored and indexed in the regular fashion - Disadvantage: Extra memory requirements for the materialized column. My initial thoughts on this project are the following: We do need the name of the column which can be used to query. So maybe we can expose a command such as: create virtual_index <name> on <column_name> <expression> What this would do would run a regular query which evaluates expressions (like in WHERE clause) and the feed the result into the indexer. This index can then be stored in the regular fashion. B. Having UDFs returning an array/set There are three approaches that I can think of: 1. Supporting array/set as native datatype inside MariaDB (like int64, double, etc) - This might be hard and touches all levels of stack. 2. Have the array/set pass in serialized form to the above node of query execution and have appropriate deserializer when we want to interpret the result - Coming up with ser/deser strategy might be tough and this would be expensive too. 3. The query execution would be in a Tree structure where each node must be exposing functions like init(), next(), read(int col_index), etc. Maybe we can use this to emulate the evaluation of UDF against row. I think this is the suggestion that is listed in the project. I would like to get some direction on this from the mentors. I would like to discuss these and then decide on one of them. Am I approaching this in the right direction? Can you please point me to the next steps? Thanks Richa

3 9

[Maria-developers] Tasks for newcomer
by Martin Kaluznik 19 Mar '15

19 Mar '15

Hello, I am student interested in GSoC project Automatic provisioning of slave ( https://mariadb.atlassian.net/browse/MDEV-7502 ). Who would like to contribute to codebase. I will appreciate if somebody could suggest me something to work on. I found something really small for myself (https://mariadb.atlassian.net/browse/MDEV-7780 ) and implemented it in most basic way. https://github.com/f4rnham/server/commits/MDEV-7780 Config entry for case specified in report would be fake_version=5.5.25 It will probably require test case too, and I am not sure if this feature will be actually safe, it may create some inconsistencies. I have never worked with bzr before, but if it will be required, I will look into that. Thanks for feedback, Martin Kaluznik ( martin.kaluznik(a)gmail.com ). I can be also found on IRC under nick Farnham

2 1

Re: [Maria-developers] 1cda762: MDEV-6069: Remove old logic for 3.23-to-higher upgrades from upgrade SQL scripts
by Sergei Golubchik 19 Mar '15

19 Mar '15

Hi, Nirbhay! On Mar 17, Nirbhay Choubey wrote: > revision-id: 1cda762d3f1132e6696e1c451435746db0a1420a > parent(s): b0542b78c866fb32383f759914b10f060d0e14cd > committer: Nirbhay Choubey > branch nick: server > timestamp: 2015-03-17 11:20:49 -0400 > message: > > MDEV-6069: Remove old logic for 3.23-to-higher upgrades from upgrade SQL scripts > > Fix for failing tests. > > * Update mysql_system_tables_fix.sql to makeup the differences in system > tables in 5.1.17 (main.system_mysql_db_fix50117) I don't quite understand that. Why these ALTER TABLEs were not needed before? > * Removed system_mysql_db tests for versions 5.0.30 & 4.1.23. ok Regards, Sergei

3 3

[Maria-developers] [GSoC] MDEV-7502 Automatic provisioning of slave
by Martin Kaluznik 18 Mar '15

18 Mar '15

Hello, I am student interested in GSoC project Automatic provisioning of slave ( https://mariadb.atlassian.net/browse/MDEV-7502 ). I have read current code and put together my idea of how it could be implemented. Please correct me if there is better solution for some steps or I just misunderstood something. Slave starts regular sql thread and something really similar to current io thread (maybe best to begin with will be copy of handle_slave_io and related functions, it may still end with some duplicate code, which at the end can be moved to new common functions). Slave will receive gtids of current master as response to copy-modified version of COM_BINLOG_DUMP, they will mark position from which binlogs will be sent to slave. Master proceeds with building of list containing what needs to be sent to slave. Here exact implementation will depend on whether the master itself will block any DDL queries during provisioning process or not. If he will (probably won’t – it’s not necessary and it will just slow him down), we will have guaranteed that internal containers with databases/tables won’t change and they can be just iterated through safely (not true for procedure/trigger containers?). In other case, it will be probably required to prefetch list of databases/tables to copy to slave (it can be done lazily with tables/triggers/procedures/…, holding only list of tables/… for database which is currently being processed). In the second case, slave will have to detect DDL event and interrupt connection. The range scans on primary key mentioned in description, I assume it would look like SELECT * FROM tbl WHERE pk > @lastChunkKey ORDER BY pk LIMIT @chunkSize; just done with internal functions more efficiently, and the result will be converted to Rows_log_event which will be sent to slave. To restrictions I would add at least one more, provisioning cannot be continued if master is restarted during process (or can we actually store information about provisioning process in database?). Another possible problem could be slave reconnect. I found kill_zombie_dump_threads function, in this case, new thread could take information about provisioning progress from old thread and with help of fake gtids bundled with chunks somehow figure out which chunk was last successfully received by slave before disconnect. It would require additional memory to store few (or more?) most recently sent chunks and wouldn't guarantee success. Great way to solve this would be, to be able to map gtid sequence number to exact chunk of data, but I can’t see how it can be done without assuming something about maximum amount of rows in table / number of tables / triggers / … Thanks for feedback, Martin Kaluznik ( martin.kaluznik [at] gmail.com ). I can be also found on IRC under nick Farnham

2 1

Re: [Maria-developers] [Commits] 2bdbfd3: Fix assertion failure seen on Buildbot win32-debug
by Sergei Golubchik 18 Mar '15

18 Mar '15

Hi, Jan! On Mar 18, Jan Lindström wrote: > revision-id: 2bdbfd334bba93f75b2952e8e76f20ed29fb2bd4 > parent(s): c14d9c21f0a6974d9909e00a23522114caad3bfc > committer: Jan Lindström > branch nick: 10.0-git > timestamp: 2015-03-18 12:18:39 +0200 > message: > > Fix assertion failure seen on Buildbot win32-debug > > There is a bug in Visual Studio 2010 > Visual Studio has a feature "Checked Iterators". In a debug build, every > iterator operation is checked at runtime for errors, e g, out of range. > Disable this "Checked Iterators" for Windows and Debug if defined. This comment, > +/* There is a bug in Visual Studio 2010 > +Visual Studio has a feature "Checked Iterators". In a debug build, every > +iterator operation is checked at runtime for errors, e g, out of range. > +Disable this "Checked Iterators" for Windows and Debug if defined. > +*/ and this comment don't explain what bug is it. They create an impression that Visual Studio has run-time checks for an iterator running out of range. And that our code fails this check (because the iterator runs out of range). And that your "fix" is to disable the check instead of fixing the buggy out-of-range iterator. I'm not saying that's what happens, I'm saying that your comment suggests that this happens, because you didn't describe what was the actual bug. A better comment should say that our iterator is correct, but a buggy check erroneously marks it as out-of-range. It should explain what iterator is it, exactly, why it's correct, etc. Could you please fix the comment please? (or the iterator, whatever should be fixed) > +#ifdef UNIV_DEBUG > +#ifdef __WIN__ > +#ifdef _ITERATOR_DEBUG_LEVEL > +#define _ITERATOR_DEBUG_LEVEL 0 this will produce a warning "macro _ITERATOR_DEBUG_LEVEL is redefined" > +#endif /* _ITERATOR_DEBUG_LEVEL */ > +#endif /* __WIN__*/ > +#endif /* UNIV_DEBUG */ Regards, Sergei

1 0

Re: [Maria-developers] [Commits] 97037da: Replace static usage of AES_CTR with current encryption algorithm.
by Jeremy Cole 18 Mar '15

18 Mar '15

Howdy Jan, > But the whole encryption_algorithm stuff seems not well thought out in any >> case. >> > > This was introduced because we can't assume that all installations will > install openSSL especially on Windows, thus encryption should work > with bundled yaSSL also. Do you have some better suggestion ? > Yes, I assumed so, but the current implementation is lazy and user-hostile. And, to me, having a single global variable controlling "the" encryption algorithm (for what, exactly?) is weird. Presumably if it makes sense to have it changeable, it also makes sense to set different algorithms for different objects. At a minimum, IMHO: store the encryption_algorithm selection with each encrypted object so that data can be migrated in the future. Then, implement sufficient support in the various necessary places so that it can be safely changed. Currently as far as I can tell it is globally set once, and impossible to change in the future. This could be done via stealing some bits from either the stored encryption scheme (1 byte) or key version (4 bytes) or adding a new encryption algorithm field which is stored alongside those everywhere (this is probably the best strategy IMHO, but takes a bit of thought). Alternatively as an absolute (and even lower) minimum: ensure that changing it once the database is initialized produces a loud and very clear error message to the user. Regards, Jeremy

1 0

[Maria-developers] Fwd: Port InnoDB memcached interface to MariaDB (GSOC 2015)
by Aalekh Nigam 18 Mar '15

18 Mar '15

Hello, I am Aalekh Nigam, 3rd Year Undergraduate student at JIIT Noida, India. I would like to propose my participation for Port InnoDB Memcache Interface to MariaDB (https://mariadb.atlassian.net/browse/MDEV-4674) as GSOC 2015 Student. Although I have been continuously involved with discussion regarding the project with the assigned mentor (Colin Charles), but would also like to propose my raw architecture for community participation , thus I would like to *request* you all to please go through the given architecture and follow up with your comments and remarks.Since the feature is critical to MariaDB, thus I will work hard on your comments and remarks to make this project a success. Architecture: According to my understanding this is the working that should be done for the project : Following are the main advantages which will come after integrating memcache to XtraDB (MariaDB Engine): - This memcache interfacing with MaraDB would help in faster execution of query in MaraDB Server. - Data can be unstructured or structured, depending on the type of application. You can make an all-new table for the data, or map the NoSQL-style processing to one or more existing tables. - Data that is requested through the memcached protocol is transparently queried from an XtraDB table, without going through the MariaDB SQL layer. - Because memcached consumes relatively little CPU, and its memory footprint is easy to control, it can run comfortably alongside a MariaDB instance on the same system. - Because memcached consumes relatively little CPU, and its memory footprint will be easy to control, it would run comfortably alongside a MariaDB instance on the same system. Detail Architecture: - Will have multi column key value support. - Will control memcache parameters through parameters like: read/write batch size, commit interval. - Direct access to XtraDb tables, bypassing the MariaDB SQL parser, the optimizer, and even the Handler API layer Thank You *Aalekh*

1 0

Re: [Maria-developers] [Commits] 97037da: Replace static usage of AES_CTR with current encryption algorithm.
by Jeremy Cole 17 Mar '15

17 Mar '15

> > I don't get it. Why is example_key_management_plugin special here? If >> you're going to support "none", why not here as well? >> > > This is exactly similar to file_key_management_plugin i.e. do not > initialize if NONE. > Heh, okay. > + >>> + my_aes_init_dynamic_encrypt(current_aes_dynamic_method); >>> >> >> Isn't this redundant? It should already be initialized by the call in >> init_server_components... >> >> >> > Maybe, but what if you load this plugin after server has started ? > Why would it matter? Unless I miss something, my_aes_encrypt_dynamic/my_aes_decrypt_dynamic are global. But the whole encryption_algorithm stuff seems not well thought out in any case. Regards, Jeremy

2 1

Re: [Maria-developers] [Commits] 97037da: Replace static usage of AES_CTR with current encryption algorithm.
by Jeremy Cole 17 Mar '15

17 Mar '15

Jan, > - my_aes_init_dynamic_encrypt(MY_AES_ALGORITHM_CTR); > + if (current_aes_dynamic_method == MY_AES_ALGORITHM_NONE) > + { > + sql_print_error("No encryption method choosen with > --encryption-algorithm. " > + "example_key_management_plugin disabled"); > + return 1; > + } > I don't get it. Why is example_key_management_plugin special here? If you're going to support "none", why not here as well? > + > + my_aes_init_dynamic_encrypt(current_aes_dynamic_method); > Isn't this redundant? It should already be initialized by the call in init_server_components... Regards, Jeremy

1 0

[Maria-developers] MDEV-6714: A couple of questions
by Vicențiu Ciorbaru 16 Mar '15

16 Mar '15

Hi Sergei, I've achieved the desired performance speedup related to MDEV-6714. You can check the preliminary patch on the commits mailing-list. I'll clean up the code and then submit a final version for review. I do however have two concerns: * By using SHOW TABLES LIKE %s, we performed a case insensitive search to find the table's correct name case. This command takes a long time with a lot of tables. Using an equality operator in I_D speeds it up, but it no longer matches if we have a different case. Possible solution: Try to search for the table with the provided string via INFORMATION_SCHEMA using the equals operator. If we can not find the table, fall back to a case insensitive search that takes longer. * The I_S.triggers and I_S.tables are available since 5.5 and 5.1 respectively. I suppose it is safe to assume we will have these tables available when running mysqldump. I can code all the changes to fall back to the previous query if the one that uses I_S tables fails. Regards, Vicentiu

3 3

Re: [Maria-developers] 53b73a7: MDEV-6714 mysqldump slow with tables in big databases
by Sergei Golubchik 16 Mar '15

16 Mar '15

Hi, Vicentiu! On Mar 16, vicentiu(a)mariadb.org wrote: > revision-id: 53b73a7940039809a02184a69502142d767b0741 > parent(s): 197afb413fcc9f06b5e5e6ef41ce980d108b354f > committer: Vicențiu Ciorbaru > branch nick: server > timestamp: 2015-03-16 21:59:26 +0200 > message: > > MDEV-6714 mysqldump slow with tables in big databases > > First draft commit. Performance speedup achieved. > > --- > client/mysqldump.c | 15 ++++++++++----- > 1 file changed, 10 insertions(+), 5 deletions(-) > > diff --git a/client/mysqldump.c b/client/mysqldump.c > index 2da4ce6..6138e94 100644 > --- a/client/mysqldump.c > +++ b/client/mysqldump.c > @@ -3341,7 +3341,8 @@ static int dump_triggers_for_table(char *table_name, char *db_name) > /* Get list of triggers. */ > > my_snprintf(query_buff, sizeof(query_buff), > - "SHOW TRIGGERS LIKE %s", > + "SELECT TRIGGER_NAME from INFORMATION_SCHEMA.TRIGGERS " > + "where TRIGGER_SCHEMA = DATABASE() and EVENT_OBJECT_TABLE = %s", This is wrong, should be EVENT_OBJECT_SCHEMA, not TRIGGER_SCHEMA. Why? That's how I_S optimizations work - for file-based objects it might either use readdir to get all files and then select those that it needs. Or it can open the requested file directly, if it knows the database (= directory name) and the object name (= file name). It kind of works like an index over (db_name, object_name). And what fields are "indexed" is specified in the ST_SCHEMA_TABLE structure for the I_S table in question. For I_S.TRIGGERS it's {"TRIGGERS", triggers_fields_info, 0, get_all_tables, make_old_format, get_schema_triggers_record, 5, 6, 0, OPEN_TRIGGER_ONLY|OPTIMIZE_I_S_TABLE}, where triggers_fields_info is the list of fields and 5,6 are "indexed" fields. Looking at triggers_fields_info you'll see that 5,6 fields are EVENT_OBJECT_SCHEMA and EVENT_OBJECT_TABLE. > quote_for_like(table_name, name_buff)); > > if (mysql_query_with_error_report(mysql, &show_triggers_rs, query_buff)) > @@ -4716,8 +4717,12 @@ static char *get_actual_table_name(const char *old_table_name, MEM_ROOT *root) > > /* Check memory for quote_for_like() */ > DBUG_ASSERT(2*sizeof(old_table_name) < sizeof(show_name_buff)); > - my_snprintf(query, sizeof(query), "SHOW TABLES LIKE %s", > + my_snprintf(query, sizeof(query), > + "select table_name from information_schema.tables " > + "where table_schema = DATABASE() " > + "and table_name = %s", > quote_for_like(old_table_name, show_name_buff)); This is wrong, here and everywhere. quote_for_like() quotes, exactly, for LIKE. It is a wrong kind of quoting for equality operator. > + verbose_msg(query); > Regards, Sergei

1 0

Re: [Maria-developers] Gsoc 2015 - Enhancing mysqlbinlog
by Kristian Nielsen 16 Mar '15

16 Mar '15

Christos Daskalakis <daskalakis.chris(a)gmail.com> writes: > i found this very interesting project on gsoc 2015 and with another > colleague (Stelios Agapiou steagap06(a)gmail.com) we would be very > interested and happy to work on Enhancing mysqlbinlog , as it is referred > to in mariadb.com . I assume you are refering to MDEV-4989: https://mariadb.atlassian.net/browse/MDEV-4989 Rohit Kashyap also showed some interest in this project, and it was discussed a bit on the maria-developers@ mailing list (Cc:ed). So let's coordinate the efforts here. I updated the MDEV-4989 with what was discussed so far, it now better reflects what the project entails. It also now has some pointers to suggested background reading. Thank you for your interest in MariaDB, and MariaDB GTID! - Kristian.

2 1

Re: [Maria-developers] 7c21ea9: MDEV-7772: SIGSEGV on my_aes_encrypt_cbc when -DWITH_SSL=bundled
by Sergei Golubchik 14 Mar '15

14 Mar '15

Hi, Jan! On Mar 14, Jan Lindström wrote: > > and thanks for the review, some comments/questions below: > > > 1. Does file_key_management_plugin work with CTR? > > > Yes it does, but only with openSSL. Then I'd suggest to * add aes_ctr combination to have_file_key_management_plugin.combinations * use only one combinations file, not two identical ones (it can be called, say, encryption_algorithms.inc) and included into both have_*_key_management_plugin.inc files * disable aes_ctr combination from suite.pm, now it has $skip{'include/have_openssl_ctr.inc'} = 'no or too old openssl' unless $::mysqld_variables{'version-ssl-library'} =~ /OpenSSL (\S+)/ and $1 ge "1.0.1"; which disables the complete have_openssl_ctr.inc, it can be changed to disable just one combination (like for have_innodb.inc): $skip{'include/encryption_algorithms.combinations'} = [ 'aes_ctr' ] unless $::mysqld_variables{'version-ssl-library'} =~ /OpenSSL (\S+)/ and $1 ge "1.0.1"; > > This doesn't make a lot of sense now, because > > example_key_management_plugin forces CTR: > > > > my_aes_init_dynamic_encrypt(MY_AES_ALGORITHM_CTR); > > > Is there some reason why it does this here? I will change that on separate > fix to current_aes_dynamic_method if != MY_AES_ALGORITHM_NONE I suspect it was in the original google patch and nobody fixed it yet. It would be good to change it. Regards, Sergei

1 0

Re: [Maria-developers] 7ef6793: MDEV-6650 - LINT_INIT emits code in non-debug builds
by Sergei Golubchik 13 Mar '15

13 Mar '15

Hi, Sergey! On Feb 10, svoj(a)mariadb.org wrote: > revision-id: 7ef679355dfe07eba88964e78a3303e6eddd618c > parent(s): f13939061d468a47985dee0268652a6fe2db7862 > committer: Sergey Vojtovich > branch nick: 10.1 > timestamp: 2015-02-10 14:05:49 +0400 > message: > > MDEV-6650 - LINT_INIT emits code in non-debug builds > > Replaced all references to LINT_INIT with UNINIT_VAR and LINT_INIT_STRUCT. > Removed LINT_INIT macro. Ok to push Regards, Sergei

1 0

Re: [Maria-developers] 7c21ea9: MDEV-7772: SIGSEGV on my_aes_encrypt_cbc when -DWITH_SSL=bundled
by Sergei Golubchik 13 Mar '15

13 Mar '15

Hi, Jan! On Mar 13, Jan Lindström wrote: > revision-id: 7c21ea9f502ded155c12a0ee3c5ff0602e9d4c9a > parent(s): 4d0e52189ca945c94146035c2733d85c9c6fd18d > committer: Jan Lindström > branch nick: 10.1-innodb > timestamp: 2015-03-13 14:18:07 +0200 > message: > > MDEV-7772: SIGSEGV on my_aes_encrypt_cbc when -DWITH_SSL=bundled > > Two problems: > - Read/Write outside of buffer at memcpy() because of incorrect parameters > . OPENSSL_assert(EVP_CIPHER_CTX_iv_length(&ctx.ctx) == iv_length); // ECB does not use IV, thus incorrect assertion > > Added: > mysql-test/include/have_file_key_management_plugin.combinations ( to run with aes_cbc and aes_ecb) > mysql-test/include/have_openssl_ctr.combinations ( to run with aes_cbc, aes_ecb and aes_ctr) The patch is ok, of course. Just a few suggestions, see below. Ok to push after that. > diff --git a/mysql-test/include/have_file_key_management_plugin.combinations b/mysql-test/include/have_file_key_management_plugin.combinations > new file mode 100644 > index 0000000..6a63b5a > --- /dev/null > +++ b/mysql-test/include/have_file_key_management_plugin.combinations > @@ -0,0 +1,6 @@ > +[aes_cbc] > +encryption-algorithm=aes_cbc > + > +[aes_ecb] > +encryption-algorithm=aes_ecb > + 1. Does file_key_management_plugin work with CTR? 2. suggestion: use shorter names for combinations (I'd remove "aes_" prefix) test names are long enough already. See replication combinations - they are 3-4 letters only (rbr/mix/stmt). > diff --git a/mysql-test/include/have_openssl_ctr.combinations b/mysql-test/include/have_openssl_ctr.combinations > new file mode 100644 > index 0000000..2c49c78 > --- /dev/null > +++ b/mysql-test/include/have_openssl_ctr.combinations > @@ -0,0 +1,9 @@ > +[aes_cbc] > +encryption-algorithm=aes_cbc > + > +[aes_ecb] > +encryption-algorithm=aes_ecb > + > +[aes_ctr] > +encryption-algorithm=aes_ctr > + This doesn't make a lot of sense now, because example_key_management_plugin forces CTR: my_aes_init_dynamic_encrypt(MY_AES_ALGORITHM_CTR); so it always uses ctr, no matter what encryption-algorithm is. which is very confusing, SHOW VARIABLES will show, say, aes_ecb, but in fact it will be aes_ctr under the hood. furthermore, it doesn't check for the return value (so it'll fail later or will misbehave on old openssl versions that don't support CTR). This all is a separate bug (in Jira already), but for now I'm just saying that it's kind of pointless to run example_key_management_plugin with different encryption-algorithm values. > diff --git a/mysql-test/suite/innodb/t/innodb_scrub_background.test b/mysql-test/suite/innodb/t/innodb_scrub_background.test > index 44cb16b..881c124 100644 > --- a/mysql-test/suite/innodb/t/innodb_scrub_background.test > +++ b/mysql-test/suite/innodb/t/innodb_scrub_background.test > @@ -1,6 +1,7 @@ > -- source include/have_innodb.inc > -- source include/not_embedded.inc > -- source include/have_example_key_management_plugin.inc > +-- source include/have_openssl_ctr.inc that's fine, although you could've simply included have_openssl_ctr.inc from have_example_key_management_plugin.inc :) > > let $MYSQLD_DATADIR=`select @@datadir`; > let ib1_IBD = $MYSQLD_DATADIR/ibdata1; > diff --git a/storage/innobase/fil/fil0crypt.cc b/storage/innobase/fil/fil0crypt.cc > index 823c74a..10d49bb 100644 > --- a/storage/innobase/fil/fil0crypt.cc > +++ b/storage/innobase/fil/fil0crypt.cc > @@ -734,7 +734,7 @@ fil_space_encrypt(ulint space, ulint offset, lsn_t lsn, > uint32 dstlen; > > if (page_compressed) { > - srclen = page_size - FIL_PAGE_DATA;; > + srclen = page_size - FIL_PAGE_DATA; I'd suggest to move this and all following changes to a separate commit. > } > > int rc = (* my_aes_encrypt_dynamic)(src, srclen, Regards, Sergei

1 0

Re: [Maria-developers] 20e5c37: MDEV-7728 - Spiral patch 032_mariadb-10.0.15.scale_registering_xid.diff
by Sergei Golubchik 13 Mar '15

13 Mar '15

Hi, Sergey! On Mar 12, svoj(a)mariadb.org wrote: > revision-id: 20e5c37ccfac9d95a2db1f295f211656df1ce160 > parent(s): 190858d996e7dc90e01fe8a5e7daec6af0012b23 > committer: Sergey Vojtovich > branch nick: 10.1 > timestamp: 2015-03-12 18:15:08 +0400 > message: > > MDEV-7728 - Spiral patch 032_mariadb-10.0.15.scale_registering_xid.diff > > XID cache is now based on lock-free hash. > Also fixed lf_hash_destroy() to call alloc destructor. > > Note that previous implementation had race condition when thread was accessing > XA owned by different thread. This new implementation doesn't fix it either. > > diff --git a/include/lf.h b/include/lf.h > index 88024bc..19bdafc 100644 > --- a/include/lf.h > +++ b/include/lf.h > @@ -117,7 +117,12 @@ void lf_pinbox_init(LF_PINBOX *pinbox, uint free_ptr_offset, > #define lf_alloc_free(PINS, PTR) lf_pinbox_free((PINS), (PTR)) > #define lf_alloc_get_pins(A) lf_pinbox_get_pins(&(A)->pinbox) > #define lf_alloc_put_pins(PINS) lf_pinbox_put_pins(PINS) > -#define lf_alloc_direct_free(ALLOC, ADDR) my_free((ADDR)) > +#define lf_alloc_direct_free(ALLOC, ADDR) \ > + do { \ > + if ((ALLOC)->destructor) \ > + (ALLOC)->destructor((uchar*) ADDR); \ > + my_free(ADDR); \ > + } while(0) > > void *lf_alloc_new(LF_PINS *pins); > > diff --git a/mysql-test/suite/innodb/r/innodb_bug59641.result b/mysql-test/suite/innodb/r/innodb_bug59641.result > index 5062c69..476385f 100644 > --- a/mysql-test/suite/innodb/r/innodb_bug59641.result > +++ b/mysql-test/suite/innodb/r/innodb_bug59641.result > @@ -43,8 +43,8 @@ COMMIT; > XA RECOVER; > formatID gtrid_length bqual_length data > 1 3 0 789 > -1 3 0 456 > 1 3 0 123 > +1 3 0 456 > XA ROLLBACK '123'; > XA ROLLBACK '456'; > XA COMMIT '789'; > diff --git a/mysql-test/suite/perfschema/r/server_init.result b/mysql-test/suite/perfschema/r/server_init.result > index 20dc130..1bdb988 100644 > --- a/mysql-test/suite/perfschema/r/server_init.result > +++ b/mysql-test/suite/perfschema/r/server_init.result > @@ -120,10 +120,6 @@ where name like "wait/synch/mutex/sql/LOCK_audit_mask"; > count(name) > 1 > select count(name) from mutex_instances > -where name like "wait/synch/mutex/sql/LOCK_xid_cache"; > -count(name) > -1 > -select count(name) from mutex_instances > where name like "wait/synch/mutex/sql/LOCK_plugin"; > count(name) > 1 > diff --git a/mysql-test/suite/perfschema/r/setup_instruments_defaults.result b/mysql-test/suite/perfschema/r/setup_instruments_defaults.result > index e502376..b0570d7 100644 > --- a/mysql-test/suite/perfschema/r/setup_instruments_defaults.result > +++ b/mysql-test/suite/perfschema/r/setup_instruments_defaults.result > @@ -5,14 +5,14 @@ SELECT * FROM performance_schema.setup_instruments > WHERE name IN ( > 'wait/synch/mutex/sql/LOCK_user_conn', > 'wait/synch/mutex/sql/LOCK_uuid_generator', > -'wait/synch/mutex/sql/LOCK_xid_cache', > +'wait/synch/mutex/sql/LOCK_plugin', > 'stage/sql/creating table') > AND enabled = 'yes' AND timed = 'no' > ORDER BY name; > NAME ENABLED TIMED > stage/sql/creating table YES NO > +wait/synch/mutex/sql/LOCK_plugin YES NO > wait/synch/mutex/sql/LOCK_user_conn YES NO > -wait/synch/mutex/sql/LOCK_xid_cache YES NO > SELECT * FROM performance_schema.setup_instruments > WHERE name = 'wait/synch/mutex/sql/LOCK_thread_count' > AND enabled = 'no' AND timed = 'no'; > diff --git a/mysql-test/suite/perfschema/t/server_init.test b/mysql-test/suite/perfschema/t/server_init.test > index d5a7e18..c6d25f1 100644 > --- a/mysql-test/suite/perfschema/t/server_init.test > +++ b/mysql-test/suite/perfschema/t/server_init.test > @@ -118,9 +118,6 @@ select count(name) from mutex_instances > where name like "wait/synch/mutex/sql/LOCK_audit_mask"; > > select count(name) from mutex_instances > - where name like "wait/synch/mutex/sql/LOCK_xid_cache"; > - > -select count(name) from mutex_instances > where name like "wait/synch/mutex/sql/LOCK_plugin"; > > # Not a global variable, may be destroyed already. > diff --git a/mysql-test/suite/perfschema/t/setup_instruments_defaults-master.opt b/mysql-test/suite/perfschema/t/setup_instruments_defaults-master.opt > index 408eb5c..ed6702e 100644 > --- a/mysql-test/suite/perfschema/t/setup_instruments_defaults-master.opt > +++ b/mysql-test/suite/perfschema/t/setup_instruments_defaults-master.opt > @@ -12,7 +12,7 @@ > --loose-performance-schema-instrument='wait/synch/mutex/sql/LOCK_thread_count=OFF' > --loose-performance-schema-instrument=' wait/synch/mutex/sql/LOCK_user_conn = COUNTED' > --loose-performance-schema-instrument='wait%/synch/mutex/sql/LOCK_uu%_genera%/= COUNTED' > ---loose-performance-schema-instrument='%%wait/synch/mutex/sql/LOCK_xid_cache=COUNTED' > +--loose-performance-schema-instrument='%%wait/synch/mutex/sql/LOCK_plugin=COUNTED' > --loose-performance-schema-instrument='%=FOO' > --loose-performance-schema-instrument='%=%' > --loose-performance-schema-instrument='%' > diff --git a/mysql-test/suite/perfschema/t/setup_instruments_defaults.test b/mysql-test/suite/perfschema/t/setup_instruments_defaults.test > index e1f6140..5e0a3a5 100644 > --- a/mysql-test/suite/perfschema/t/setup_instruments_defaults.test > +++ b/mysql-test/suite/perfschema/t/setup_instruments_defaults.test > @@ -15,7 +15,7 @@ SELECT * FROM performance_schema.setup_instruments > WHERE name IN ( > 'wait/synch/mutex/sql/LOCK_user_conn', > 'wait/synch/mutex/sql/LOCK_uuid_generator', > - 'wait/synch/mutex/sql/LOCK_xid_cache', > + 'wait/synch/mutex/sql/LOCK_plugin', > 'stage/sql/creating table') > AND enabled = 'yes' AND timed = 'no' > ORDER BY name; > diff --git a/sql/handler.cc b/sql/handler.cc > index 12d7ffb..a45419f 100644 > --- a/sql/handler.cc > +++ b/sql/handler.cc > @@ -1931,12 +1931,28 @@ int ha_recover(HASH *commit_list) > so mysql_xa_recover does not filter XID's to ensure uniqueness. > It can be easily fixed later, if necessary. > */ > + > +static my_bool xa_recover_callback(XID_STATE *xs, Protocol *protocol) > +{ > + if (xs->xa_state == XA_PREPARED) > + { > + protocol->prepare_for_resend(); > + protocol->store_longlong((longlong) xs->xid.formatID, FALSE); > + protocol->store_longlong((longlong) xs->xid.gtrid_length, FALSE); > + protocol->store_longlong((longlong) xs->xid.bqual_length, FALSE); > + protocol->store(xs->xid.data, xs->xid.gtrid_length + xs->xid.bqual_length, > + &my_charset_bin); > + if (protocol->write()) > + return TRUE; > + } > + return FALSE; > +} > + > + > bool mysql_xa_recover(THD *thd) > { > List<Item> field_list; > Protocol *protocol= thd->protocol; > - int i=0; > - XID_STATE *xs; > DBUG_ENTER("mysql_xa_recover"); > > field_list.push_back(new Item_int("formatID", 0, MY_INT32_NUM_DECIMAL_DIGITS)); > @@ -1948,26 +1964,9 @@ bool mysql_xa_recover(THD *thd) > Protocol::SEND_NUM_ROWS | Protocol::SEND_EOF)) > DBUG_RETURN(1); > > - mysql_mutex_lock(&LOCK_xid_cache); > - while ((xs= (XID_STATE*) my_hash_element(&xid_cache, i++))) > - { > - if (xs->xa_state==XA_PREPARED) > - { > - protocol->prepare_for_resend(); > - protocol->store_longlong((longlong)xs->xid.formatID, FALSE); > - protocol->store_longlong((longlong)xs->xid.gtrid_length, FALSE); > - protocol->store_longlong((longlong)xs->xid.bqual_length, FALSE); > - protocol->store(xs->xid.data, xs->xid.gtrid_length+xs->xid.bqual_length, > - &my_charset_bin); > - if (protocol->write()) > - { > - mysql_mutex_unlock(&LOCK_xid_cache); > - DBUG_RETURN(1); > - } > - } > - } > - > - mysql_mutex_unlock(&LOCK_xid_cache); > + if (xid_cache_iterate(thd, (my_hash_walk_action) xa_recover_callback, > + protocol)) > + DBUG_RETURN(1); > my_eof(thd); > DBUG_RETURN(0); > } > diff --git a/sql/handler.h b/sql/handler.h > index 5ef9208..5ab67c0 100644 > --- a/sql/handler.h > +++ b/sql/handler.h > @@ -612,11 +612,11 @@ struct xid_t { > return sizeof(formatID)+sizeof(gtrid_length)+sizeof(bqual_length)+ > gtrid_length+bqual_length; > } > - uchar *key() > + uchar *key() const > { > return (uchar *)&gtrid_length; > } > - uint key_length() > + uint key_length() const > { > return sizeof(gtrid_length)+sizeof(bqual_length)+gtrid_length+bqual_length; > } > diff --git a/sql/mysqld.cc b/sql/mysqld.cc > index b97742d..6c06240 100644 > --- a/sql/mysqld.cc > +++ b/sql/mysqld.cc > @@ -4889,11 +4889,7 @@ static int init_server_components() > my_charset_error_reporter= charset_error_reporter; > #endif > > - if (xid_cache_init()) > - { > - sql_print_error("Out of memory"); > - unireg_abort(1); > - } > + xid_cache_init(); > > /* > initialize delegates for extension observers, errors have already > diff --git a/sql/sql_class.cc b/sql/sql_class.cc > index 5b0d04d..d59ef52 100644 > --- a/sql/sql_class.cc > +++ b/sql/sql_class.cc > @@ -914,7 +914,8 @@ bool Drop_table_error_handler::handle_condition(THD *thd, > wait_for_commit_ptr(0), > main_da(0, false, false), > m_stmt_da(&main_da), > - tdc_hash_pins(0) > + tdc_hash_pins(0), > + xid_hash_pins(0) > #ifdef WITH_WSREP > , > wsrep_applier(is_wsrep_applier), > @@ -1593,7 +1594,7 @@ void THD::cleanup(void) > > transaction.xid_state.xa_state= XA_NOTR; > trans_rollback(this); > - xid_cache_delete(&transaction.xid_state); > + xid_cache_delete(this, &transaction.xid_state); > > DBUG_ASSERT(open_tables == NULL); > /* > @@ -1704,6 +1705,8 @@ void THD::cleanup(void) > main_da.free_memory(); > if (tdc_hash_pins) > lf_hash_put_pins(tdc_hash_pins); > + if (xid_hash_pins) > + lf_hash_put_pins(xid_hash_pins); > /* Ensure everything is freed */ > if (status_var.local_memory_used != 0) > { > @@ -5106,120 +5109,205 @@ void mark_transaction_to_rollback(THD *thd, bool all) > /*************************************************************************** > Handling of XA id cacheing > ***************************************************************************/ > +class XID_cache_element > +{ > + uint m_state; shouldn't it be uint32 ? That's the type that my_atomic_add32/cas32 work with. > + static const uint DELETED= 1 << 31; > +public: > + XID_STATE *m_xid_state; O-okay. I *think* your hand-written locking logic works. But please add a comment, explaining how it works, all four methods, the meaning of DELETED, etc. Perhaps DELETED should rather be NOT_INITIALIZED? > + bool lock() > + { > + if (my_atomic_add32_explicit(&m_state, 1, MY_MEMORY_ORDER_ACQUIRE) & DELETED) > + { > + unlock(); > + return false; > + } > + return true; > + } > + void unlock() > + { > + my_atomic_add32_explicit(&m_state, -1, MY_MEMORY_ORDER_RELEASE); > + } > + void mark_deleted() > + { > + uint old= 0; > + while (!my_atomic_cas32_weak_explicit(&m_state, &old, DELETED, > + MY_MEMORY_ORDER_RELAXED, > + MY_MEMORY_ORDER_RELAXED)) usually one should use LF_BACKOFF inside spin-loops. > + old= 0; > + } > + void mark_not_deleted() > + { > + DBUG_ASSERT(m_state & DELETED); > + my_atomic_add32_explicit(&m_state, -DELETED, MY_MEMORY_ORDER_RELAXED); > + } > + static void lf_hash_initializer(LF_HASH *hash __attribute__((unused)), > + XID_cache_element *element, > + XID_STATE *xid_state) > + { > + element->m_xid_state= xid_state; > + xid_state->xid_cache_element= element; > + } > + static void lf_alloc_constructor(uchar *ptr) > + { > + XID_cache_element *element= (XID_cache_element*) (ptr + LF_HASH_OVERHEAD); > + element->m_state= DELETED; > + } > + static void lf_alloc_destructor(uchar *ptr) > + { > + XID_cache_element *element= (XID_cache_element*) (ptr + LF_HASH_OVERHEAD); > + if (element->m_state != DELETED) How can this happen? You mark an element DELETED before lf_hash_delete() > + { > + DBUG_ASSERT(!element->m_xid_state->in_thd); > + my_free(element->m_xid_state); > + } > + } > + static uchar *key(const XID_cache_element *element, size_t *length, > + my_bool not_used __attribute__((unused))) > + { > + *length= element->m_xid_state->xid.key_length(); > + return element->m_xid_state->xid.key(); > + } > +}; > > -mysql_mutex_t LOCK_xid_cache; > -HASH xid_cache; > > -extern "C" uchar *xid_get_hash_key(const uchar *, size_t *, my_bool); > -extern "C" void xid_free_hash(void *); > +static LF_HASH xid_cache; > +static bool xid_cache_inited; > > -uchar *xid_get_hash_key(const uchar *ptr, size_t *length, > - my_bool not_used __attribute__((unused))) > -{ > - *length=((XID_STATE*)ptr)->xid.key_length(); > - return ((XID_STATE*)ptr)->xid.key(); > -} > > -void xid_free_hash(void *ptr) > +bool THD::fix_xid_hash_pins() > { > - if (!((XID_STATE*)ptr)->in_thd) > - my_free(ptr); > + if (!xid_hash_pins) > + xid_hash_pins= lf_hash_get_pins(&xid_cache); > + return !xid_hash_pins; > } > > -#ifdef HAVE_PSI_INTERFACE > -static PSI_mutex_key key_LOCK_xid_cache; > > -static PSI_mutex_info all_xid_mutexes[]= > +void xid_cache_init() > { > - { &key_LOCK_xid_cache, "LOCK_xid_cache", PSI_FLAG_GLOBAL} > -}; > - > -static void init_xid_psi_keys(void) > -{ > - const char* category= "sql"; > - int count; > - > - if (PSI_server == NULL) > - return; > - > - count= array_elements(all_xid_mutexes); > - PSI_server->register_mutex(category, all_xid_mutexes, count); > + xid_cache_inited= true; > + lf_hash_init(&xid_cache, sizeof(XID_cache_element), LF_HASH_UNIQUE, 0, 0, > + (my_hash_get_key) XID_cache_element::key, &my_charset_bin); > + xid_cache.alloc.constructor= XID_cache_element::lf_alloc_constructor; > + xid_cache.alloc.destructor= XID_cache_element::lf_alloc_destructor; > + xid_cache.initializer= > + (lf_hash_initializer) XID_cache_element::lf_hash_initializer; > } > -#endif /* HAVE_PSI_INTERFACE */ > - > -bool xid_cache_init() > -{ > -#ifdef HAVE_PSI_INTERFACE > - init_xid_psi_keys(); > -#endif > > - mysql_mutex_init(key_LOCK_xid_cache, &LOCK_xid_cache, MY_MUTEX_INIT_FAST); > - return my_hash_init(&xid_cache, &my_charset_bin, 100, 0, 0, > - xid_get_hash_key, xid_free_hash, 0) != 0; > -} > > void xid_cache_free() > { > - if (my_hash_inited(&xid_cache)) > + if (xid_cache_inited) > { > - my_hash_free(&xid_cache); > - mysql_mutex_destroy(&LOCK_xid_cache); > + lf_hash_destroy(&xid_cache); > + xid_cache_inited= false; > } > } > > -XID_STATE *xid_cache_search(XID *xid) > + > +XID_STATE *xid_cache_search(THD *thd, XID *xid) > { > - mysql_mutex_lock(&LOCK_xid_cache); > - XID_STATE *res=(XID_STATE *)my_hash_search(&xid_cache, xid->key(), > - xid->key_length()); > - mysql_mutex_unlock(&LOCK_xid_cache); > - return res; > + DBUG_ASSERT(thd->xid_hash_pins); > + XID_cache_element *element= > + (XID_cache_element*) lf_hash_search(&xid_cache, thd->xid_hash_pins, > + xid->key(), xid->key_length()); > + if (element) > + { > + lf_hash_search_unpin(thd->xid_hash_pins); > + return element->m_xid_state; Normally, one should not access the element after it's unpinned, so the protocol is XID_STATE *state= element->m_xid_state; lf_hash_search_unpin(thd->xid_hash_pins); return state; Perhaps your locking/deleted m_state guarantees that element is safe to access? But I failed to see that :( > + } > + return 0; > } > > > bool xid_cache_insert(XID *xid, enum xa_states xa_state) > { > XID_STATE *xs; > - my_bool res; > - mysql_mutex_lock(&LOCK_xid_cache); > - if (my_hash_search(&xid_cache, xid->key(), xid->key_length())) > - res=0; > - else if (!(xs=(XID_STATE *)my_malloc(sizeof(*xs), MYF(MY_WME)))) > - res=1; > - else > + LF_PINS *pins; > + int res= 1; > + > + if (!(pins= lf_hash_get_pins(&xid_cache))) > + return true; > + > + if ((xs= (XID_STATE*) my_malloc(sizeof(*xs), MYF(MY_WME)))) > { > xs->xa_state=xa_state; > xs->xid.set(xid); > xs->in_thd=0; > xs->rm_error=0; > - res=my_hash_insert(&xid_cache, (uchar*)xs); > + > + if ((res= lf_hash_insert(&xid_cache, pins, xs))) > + my_free(xs); > + else > + xs->xid_cache_element->mark_not_deleted(); > + if (res == 1) > + res= 0; > } > - mysql_mutex_unlock(&LOCK_xid_cache); > + lf_hash_put_pins(pins); > return res; > } > > > -bool xid_cache_insert(XID_STATE *xid_state) > +bool xid_cache_insert(THD *thd, XID_STATE *xid_state) > { > - mysql_mutex_lock(&LOCK_xid_cache); > - if (my_hash_search(&xid_cache, xid_state->xid.key(), > - xid_state->xid.key_length())) > + if (thd->fix_xid_hash_pins()) > + return true; > + > + int res= lf_hash_insert(&xid_cache, thd->xid_hash_pins, xid_state); > + switch (res) > { > - mysql_mutex_unlock(&LOCK_xid_cache); > + case 0: > + xid_state->xid_cache_element->mark_not_deleted(); > + break; > + case 1: > my_error(ER_XAER_DUPID, MYF(0)); > - return true; > + default: > + xid_state->xid_cache_element= 0; > } > - bool res= my_hash_insert(&xid_cache, (uchar*)xid_state); > - mysql_mutex_unlock(&LOCK_xid_cache); > return res; > } > > > -void xid_cache_delete(XID_STATE *xid_state) > +void xid_cache_delete(THD *thd, XID_STATE *xid_state) > +{ > + if (xid_state->xid_cache_element) > + { > + DBUG_ASSERT(thd->xid_hash_pins); > + xid_state->xid_cache_element->mark_deleted(); > + lf_hash_delete(&xid_cache, thd->xid_hash_pins, > + xid_state->xid.key(), xid_state->xid.key_length()); > + xid_state->xid_cache_element= 0; > + if (!xid_state->in_thd) > + my_free(xid_state); > + } > +} > + > + > +struct xid_cache_iterate_arg > +{ > + my_hash_walk_action action; > + void *argument; > +}; > + > +static my_bool xid_cache_iterate_callback(XID_cache_element *element, > + xid_cache_iterate_arg *arg) > +{ > + my_bool res= FALSE; > + if (element->lock()) > + { > + res= arg->action(element->m_xid_state, arg->argument); > + element->unlock(); > + } > + return res; > +} > + > +int xid_cache_iterate(THD *thd, my_hash_walk_action action, void *arg) > { > - mysql_mutex_lock(&LOCK_xid_cache); > - my_hash_delete(&xid_cache, (uchar *)xid_state); > - mysql_mutex_unlock(&LOCK_xid_cache); > + xid_cache_iterate_arg argument= { action, arg }; > + return thd->fix_xid_hash_pins() ? -1 : > + lf_hash_iterate(&xid_cache, thd->xid_hash_pins, > + (my_hash_walk_action) xid_cache_iterate_callback, > + &argument); > } > > > diff --git a/sql/sql_class.h b/sql/sql_class.h > index bd145bd..200f541 100644 > --- a/sql/sql_class.h > +++ b/sql/sql_class.h > @@ -1105,6 +1105,7 @@ struct st_savepoint { > > enum xa_states {XA_NOTR=0, XA_ACTIVE, XA_IDLE, XA_PREPARED, XA_ROLLBACK_ONLY}; > extern const char *xa_state_names[]; > +class XID_cache_element; > > typedef struct st_xid_state { > /* For now, this is only used to catch duplicated external xids */ > @@ -1113,16 +1114,16 @@ struct st_savepoint { > bool in_thd; > /* Error reported by the Resource Manager (RM) to the Transaction Manager. */ > uint rm_error; > + XID_cache_element *xid_cache_element; > } XID_STATE; > > -extern mysql_mutex_t LOCK_xid_cache; > -extern HASH xid_cache; > -bool xid_cache_init(void); > +void xid_cache_init(void); > void xid_cache_free(void); > -XID_STATE *xid_cache_search(XID *xid); > +XID_STATE *xid_cache_search(THD *thd, XID *xid); > bool xid_cache_insert(XID *xid, enum xa_states xa_state); > -bool xid_cache_insert(XID_STATE *xid_state); > -void xid_cache_delete(XID_STATE *xid_state); > +bool xid_cache_insert(THD *thd, XID_STATE *xid_state); > +void xid_cache_delete(THD *thd, XID_STATE *xid_state); > +int xid_cache_iterate(THD *thd, my_hash_walk_action action, void *argument); > > /** > @class Security_context > @@ -3785,6 +3786,8 @@ class THD :public Statement, > } > > LF_PINS *tdc_hash_pins; > + LF_PINS *xid_hash_pins; > + bool fix_xid_hash_pins(); > > inline ulong wsrep_binlog_format() const > { > diff --git a/sql/transaction.cc b/sql/transaction.cc > index 22e3ad7..d6ef160 100644 > --- a/sql/transaction.cc > +++ b/sql/transaction.cc > @@ -738,7 +738,7 @@ bool trans_xa_start(THD *thd) > thd->transaction.xid_state.xa_state= XA_ACTIVE; > thd->transaction.xid_state.rm_error= 0; > thd->transaction.xid_state.xid.set(thd->lex->xid); > - if (xid_cache_insert(&thd->transaction.xid_state)) > + if (xid_cache_insert(thd, &thd->transaction.xid_state)) > { > thd->transaction.xid_state.xa_state= XA_NOTR; > thd->transaction.xid_state.xid.null(); > @@ -801,7 +801,7 @@ bool trans_xa_prepare(THD *thd) > my_error(ER_XAER_NOTA, MYF(0)); > else if (ha_prepare(thd)) > { > - xid_cache_delete(&thd->transaction.xid_state); > + xid_cache_delete(thd, &thd->transaction.xid_state); > thd->transaction.xid_state.xa_state= XA_NOTR; > my_error(ER_XA_RBROLLBACK, MYF(0)); > } > @@ -830,6 +830,11 @@ bool trans_xa_commit(THD *thd) > > if (!thd->transaction.xid_state.xid.eq(thd->lex->xid)) > { > + if (thd->fix_xid_hash_pins()) > + { > + my_error(ER_OUT_OF_RESOURCES, MYF(0)); > + DBUG_RETURN(TRUE); > + } > /* > xid_state.in_thd is always true beside of xa recovery procedure. > Note, that there is no race condition here between xid_cache_search > @@ -840,7 +845,7 @@ bool trans_xa_commit(THD *thd) > xa_cache_insert(XID, xa_states), which is called before starting > client connections, and thus is always single-threaded. > */ > - XID_STATE *xs= xid_cache_search(thd->lex->xid); > + XID_STATE *xs= xid_cache_search(thd, thd->lex->xid); > res= !xs || xs->in_thd; > if (res) > my_error(ER_XAER_NOTA, MYF(0)); > @@ -848,7 +853,7 @@ bool trans_xa_commit(THD *thd) > { > res= xa_trans_rolled_back(xs); > ha_commit_or_rollback_by_xid(thd->lex->xid, !res); > - xid_cache_delete(xs); > + xid_cache_delete(thd, xs); > } > DBUG_RETURN(res); > } > @@ -911,7 +916,7 @@ bool trans_xa_commit(THD *thd) > thd->server_status&= > ~(SERVER_STATUS_IN_TRANS | SERVER_STATUS_IN_TRANS_READONLY); > DBUG_PRINT("info", ("clearing SERVER_STATUS_IN_TRANS")); > - xid_cache_delete(&thd->transaction.xid_state); > + xid_cache_delete(thd, &thd->transaction.xid_state); > thd->transaction.xid_state.xa_state= XA_NOTR; > > DBUG_RETURN(res); > @@ -935,14 +940,20 @@ bool trans_xa_rollback(THD *thd) > > if (!thd->transaction.xid_state.xid.eq(thd->lex->xid)) > { > - XID_STATE *xs= xid_cache_search(thd->lex->xid); > + if (thd->fix_xid_hash_pins()) > + { > + my_error(ER_OUT_OF_RESOURCES, MYF(0)); > + DBUG_RETURN(TRUE); > + } > + > + XID_STATE *xs= xid_cache_search(thd, thd->lex->xid); > if (!xs || xs->in_thd) > my_error(ER_XAER_NOTA, MYF(0)); > else > { > xa_trans_rolled_back(xs); > ha_commit_or_rollback_by_xid(thd->lex->xid, 0); > - xid_cache_delete(xs); > + xid_cache_delete(thd, xs); > } > DBUG_RETURN(thd->get_stmt_da()->is_error()); > } > @@ -961,7 +972,7 @@ bool trans_xa_rollback(THD *thd) > thd->server_status&= > ~(SERVER_STATUS_IN_TRANS | SERVER_STATUS_IN_TRANS_READONLY); > DBUG_PRINT("info", ("clearing SERVER_STATUS_IN_TRANS")); > - xid_cache_delete(&thd->transaction.xid_state); > + xid_cache_delete(thd, &thd->transaction.xid_state); > thd->transaction.xid_state.xa_state= XA_NOTR; > > DBUG_RETURN(res); > diff --git a/storage/spider/spd_table.cc b/storage/spider/spd_table.cc > index 36768ae..411c7ae 100644 > --- a/storage/spider/spd_table.cc > +++ b/storage/spider/spd_table.cc > @@ -41,11 +41,13 @@ I'd prefer to have spider changes in a separate commit > #include "spd_malloc.h" > > ulong *spd_db_att_thread_id; > +#if MYSQL_VERSION_ID < 100103 > #ifdef XID_CACHE_IS_SPLITTED > uint *spd_db_att_xid_cache_split_num; > #endif > pthread_mutex_t *spd_db_att_LOCK_xid_cache; > HASH *spd_db_att_xid_cache; > +#endif > struct charset_info_st *spd_charset_utf8_bin; > const char **spd_defaults_extra_file; > const char **spd_defaults_file; > @@ -6263,7 +6265,7 @@ int spider_db_init( > "?LOCK_xid_cache@@3PAUst_mysql_mutex@@A")); > spd_db_att_xid_cache = *((HASH **) > GetProcAddress(current_module, "?xid_cache@@3PAUst_hash@@A")); > -#else > +#elif MYSQL_VERSION_ID < 100103 > spd_db_att_LOCK_xid_cache = (pthread_mutex_t *) > #if MYSQL_VERSION_ID < 50500 > GetProcAddress(current_module, > @@ -6289,7 +6291,7 @@ int spider_db_init( > spd_db_att_xid_cache_split_num = &opt_xid_cache_split_num; > spd_db_att_LOCK_xid_cache = LOCK_xid_cache; > spd_db_att_xid_cache = xid_cache; > -#else > +#elif MYSQL_VERSION_ID < 100103 > spd_db_att_LOCK_xid_cache = &LOCK_xid_cache; > spd_db_att_xid_cache = &xid_cache; > #endif > diff --git a/storage/spider/spd_trx.cc b/storage/spider/spd_trx.cc > index a66fa5a..1b02bb8 100644 > --- a/storage/spider/spd_trx.cc > +++ b/storage/spider/spd_trx.cc > @@ -38,11 +38,13 @@ > #include "spd_ping_table.h" > #include "spd_malloc.h" > > +#if MYSQL_VERSION_ID < 100103 > #ifdef XID_CACHE_IS_SPLITTED > extern uint *spd_db_att_xid_cache_split_num; > #endif > extern pthread_mutex_t *spd_db_att_LOCK_xid_cache; > extern HASH *spd_db_att_xid_cache; > +#endif > extern struct charset_info_st *spd_charset_utf8_bin; > > extern handlerton *spider_hton_ptr; > @@ -1641,6 +1643,13 @@ int spider_xa_lock( > int error_num; > const char *old_proc_info; > DBUG_ENTER("spider_xa_lock"); > +#if MYSQL_VERSION_ID >= 100103 > + old_proc_info = thd_proc_info(thd, "Locking xid by Spider"); > + error_num= 0; > + if (xid_cache_insert(thd, xid_state)) > + error_num= thd->get_stmt_da()->sql_errno() == ER_XAER_DUPID ? > + ER_SPIDER_XA_LOCKED_NUM : HA_ERR_OUT_OF_MEM; > +#else > #ifdef SPIDER_HAS_HASH_VALUE_TYPE > my_hash_value_type hash_value = my_calc_hash(spd_db_att_xid_cache, > (uchar*) xid_state->xid.key(), xid_state->xid.key_length()); > @@ -1699,6 +1708,7 @@ int spider_xa_lock( > #else > pthread_mutex_unlock(spd_db_att_LOCK_xid_cache); > #endif > +#endif > thd_proc_info(thd, old_proc_info); > DBUG_RETURN(error_num); > } > @@ -1709,6 +1719,10 @@ int spider_xa_unlock( > THD *thd = current_thd; > const char *old_proc_info; > DBUG_ENTER("spider_xa_unlock"); > +#if MYSQL_VERSION_ID >= 100103 > + old_proc_info = thd_proc_info(thd, "Unlocking xid by Spider"); > + xid_cache_delete(thd, xid_state); > +#else > #if defined(SPIDER_HAS_HASH_VALUE_TYPE) && defined(HASH_UPDATE_WITH_HASH_VALUE) > my_hash_value_type hash_value = my_calc_hash(spd_db_att_xid_cache, > (uchar*) xid_state->xid.key(), xid_state->xid.key_length()); > @@ -1738,6 +1752,7 @@ int spider_xa_unlock( > #else > pthread_mutex_unlock(spd_db_att_LOCK_xid_cache); > #endif > +#endif > thd_proc_info(thd, old_proc_info); > DBUG_RETURN(0); > } > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits Regards, Sergei

2 1

Re: [Maria-developers] b723778: MDEV-6997: SET STATEMENT last_insert_id FOR ... does not affect the value written to the binlog
by Sergei Golubchik 13 Mar '15

13 Mar '15

Hi, Sanja! Ok to push, thanks. But see one confusing typo below. On Mar 13, sanja(a)mariadb.com wrote: > revision-id: b7237786f5b8e74ba9558db8325d29a64f1a1d9b > parent(s): ba3573cae8f0d269452cc6213f1a8243e455830e > committer: Oleksandr Byelkin > branch nick: server > timestamp: 2015-03-13 16:32:47 +0100 > message: > > MDEV-6997: SET STATEMENT last_insert_id FOR ... does not affect the value written to the binlog > > Problem was in reinitialization of first_successful_insert_id_in_prev_stmt_for_binlog after setting variables. > > diff --git a/sql/sql_parse.cc b/sql/sql_parse.cc > index 0837a70..41df891 100644 > --- a/sql/sql_parse.cc > +++ b/sql/sql_parse.cc > @@ -2743,6 +2743,18 @@ static bool do_execute_sp(THD *thd, sp_head *sp) > lex->restore_set_statement_var(); > goto error; > } > + /* > + The value of last_insert_id is remembered in THD to be written to binlog Hmm, I thought you'll put it in the commit comment. But ok, having it in the code is fine too, whatever you prefer. > + when it's used *the first time* in the statement. But SET STATEMENT > + must read the old value of last_insert_id to be able to restore it at > + the end. This should not count at "reading of last_insert_id" and > + should not remember last_insert_id for binlog. That is, it should set typo: it should *reset* (or *clear*) not *set* > + stmt_depends_on_first_successful_insert_id_in_prev_stmt flag. > + */ > + if (!thd->in_sub_stmt) > + { > + thd->stmt_depends_on_first_successful_insert_id_in_prev_stmt= 0; > + } > } Regards, Sergei

1 0

Re: [Maria-developers] 3c941d4: MDEV-5214 Status variables for number of global/db/table/column/role grants
by Sergei Golubchik 13 Mar '15

13 Mar '15

Hi, Vicențiu! That's pretty good, thanks! Although I would like to suggest a somewhat different approach, see below. And there are few minor comments too. On Mar 13, vicentiu(a)mariadb.org wrote: > revision-id: 3c941d471b79ba5d9939a4a1ac61ce5ed8bebaef > parent(s): 197afb413fcc9f06b5e5e6ef41ce980d108b354f > committer: Vicențiu Ciorbaru > branch nick: server > timestamp: 2015-03-13 17:45:07 +0200 > message: > > MDEV-5214 Status variables for number of global/db/table/column/role grants > > Implemented the status variables for use with the feedback plugin. > > --- > mysql-test/suite/plugins/r/acl_statistics.result | 107 +++++++++++++++++++++++ > mysql-test/suite/plugins/t/acl_statistics.test | 70 +++++++++++++++ > sql/mysqld.cc | 34 +++++++ > sql/sql_acl.cc | 34 +++++++ > sql/sql_acl.h | 10 +++ > 5 files changed, 255 insertions(+) > > diff --git a/mysql-test/suite/plugins/t/acl_statistics.test b/mysql-test/suite/plugins/t/acl_statistics.test > new file mode 100644 > index 0000000..e10d907 > --- /dev/null > +++ b/mysql-test/suite/plugins/t/acl_statistics.test > @@ -0,0 +1,70 @@ > +# Test case for validating acl statistics for the feedback plugin. > +--source include/not_embedded.inc > +if (`select count(*) = 0 from information_schema.plugins where plugin_name = 'feedback' and plugin_status='active'`) > +{ > + --skip Feedback plugin is not active > +} Don't bother using feedback plugin for this test. There is nothing in the patch that depends on or uses feedback plugin, so there's no need to make the test depending on it. Please, use SHOW STATUS (or SELECT ... FROM INFORMATION_SCHEMA.GLOBAL_STATUS). And then move the test out of plugins suite. > + > +# First get a baseline of the initial statistics. ... > diff --git a/mysql-test/suite/plugins/r/acl_statistics.result b/mysql-test/suite/plugins/r/acl_statistics.result > new file mode 100644 > index 0000000..dad37bf > --- /dev/null > +++ b/mysql-test/suite/plugins/r/acl_statistics.result > @@ -0,0 +1,107 @@ > +SELECT * FROM information_schema.feedback > +WHERE variable_name like '%Acl%'; > +VARIABLE_NAME VARIABLE_VALUE > +ACL_COUNT_COLUMN_GRANTS 0 > +ACL_COUNT_DATABASE_GRANTS 2 > +ACL_COUNT_FUNCTION_GRANTS 0 > +ACL_COUNT_PROCEDURE_GRANTS 0 > +ACL_COUNT_PROXY_USERS 2 > +ACL_COUNT_ROLE_MAPPINGS 0 > +ACL_COUNT_ROLES 0 > +ACL_COUNT_TABLE_GRANTS 0 > +ACL_COUNT_USERS 4 > +SELECT count(*) COLUMN_GRANTS_COUNT from mysql.columns_priv; > +COLUMN_GRANTS_COUNT > +0 > +SELECT count(*) DATABASE_GRANTS_COUNT from mysql.db; > +DATABASE_GRANTS_COUNT > +2 > +SELECT count(*) FUNCTION_GRANTS_COUNT from mysql.procs_priv where routine_type='FUNCTION'; > +FUNCTION_GRANTS_COUNT > +0 > +SELECT count(*) PROCEDURE_GRANTS_COUNT from mysql.procs_priv where routine_type='PROCEDURE'; > +PROCEDURE_GRANTS_COUNT > +0 > +SELECT count(*) PROXY_USERS from mysql.proxies_priv; > +PROXY_USERS > +2 > +SELECT count(*) ROLE_GRANTS_COUNT from mysql.user where is_role='Y'; > +ROLE_GRANTS_COUNT > +0 > +SELECT count(*) ROLE_MAPPINGS_COUNT from mysql.roles_mapping; > +ROLE_MAPPINGS_COUNT > +0 > +SELECT count(*) TABLE_GRANTS_COUNT from mysql.tables_priv; > +TABLE_GRANTS_COUNT > +0 > +SELECT count(*) USERS_COUNT from mysql.user where is_role='N'; > +USERS_COUNT > +4 > +CREATE USER u1; > +CREATE ROLE r1; > +CREATE ROLE r2; > +GRANT PROXY ON root TO u1; > +GRANT SELECT ON *.* to u1; > +GRANT SELECT ON *.* to r1; > +GRANT DELETE ON mysql.* to u1; > +GRANT DELETE ON mysql.* to r1; > +GRANT INSERT ON mysql.user to u1; > +GRANT INSERT ON mysql.user to r1; > +GRANT UPDATE (host) ON mysql.user to u1; > +GRANT UPDATE (host) ON mysql.user to r1; > +GRANT r1 to u1; > +GRANT r2 to r1; > +create procedure mysql.test_proc (OUT param1 INT) > +begin > +select COUNT(*) into param1 from mysql.roles_mapping; > +end| > +GRANT EXECUTE ON PROCEDURE mysql.test_proc TO r1; > +GRANT EXECUTE ON PROCEDURE mysql.test_proc TO u1; > +CREATE FUNCTION mysql.test_func (param INT) RETURNS INT > +RETURN (SELECT COUNT(*) FROM mysql.user); > +GRANT EXECUTE ON FUNCTION mysql.test_func TO r1; > +GRANT EXECUTE ON FUNCTION mysql.test_func TO u1; > +SELECT * FROM information_schema.feedback > +WHERE variable_name like '%Acl%'; > +VARIABLE_NAME VARIABLE_VALUE > +ACL_COUNT_COLUMN_GRANTS 2 > +ACL_COUNT_DATABASE_GRANTS 4 > +ACL_COUNT_FUNCTION_GRANTS 2 > +ACL_COUNT_PROCEDURE_GRANTS 2 > +ACL_COUNT_PROXY_USERS 3 > +ACL_COUNT_ROLE_MAPPINGS 4 > +ACL_COUNT_ROLES 2 > +ACL_COUNT_TABLE_GRANTS 2 > +ACL_COUNT_USERS 5 > +SELECT count(*) COLUMN_GRANTS_COUNT from mysql.columns_priv; > +COLUMN_GRANTS_COUNT > +2 > +SELECT count(*) DATABASE_GRANTS_COUNT from mysql.db; > +DATABASE_GRANTS_COUNT > +4 > +SELECT count(*) FUNCTION_GRANTS_COUNT from mysql.procs_priv where routine_type='FUNCTION'; > +FUNCTION_GRANTS_COUNT > +2 > +SELECT count(*) PROCEDURE_GRANTS_COUNT from mysql.procs_priv where routine_type='PROCEDURE'; > +PROCEDURE_GRANTS_COUNT > +2 > +SELECT count(*) PROXY_USERS from mysql.proxies_priv; > +PROXY_USERS > +3 > +SELECT count(*) ROLE_GRANTS_COUNT from mysql.user where is_role='Y'; > +ROLE_GRANTS_COUNT > +2 > +SELECT count(*) ROLE_MAPPINGS_COUNT from mysql.roles_mapping; > +ROLE_MAPPINGS_COUNT > +4 Please, swap these two selects (ROLE_GRANTS_COUNT and ROLE_MAPPINGS_COUNT) to have values go in the same order as in the I_S table (it's easier to compare numbers that way). Same for the previous set of selects. > +SELECT count(*) TABLE_GRANTS_COUNT from mysql.tables_priv; > +TABLE_GRANTS_COUNT > +2 > +SELECT count(*) USERS_COUNT from mysql.user where is_role='N'; > +USERS_COUNT > +5 > +DROP PROCEDURE mysql.test_proc; > +DROP FUNCTION mysql.test_func; > +DROP ROLE r2; > +DROP ROLE r1; > +DROP USER u1; > diff --git a/sql/mysqld.cc b/sql/mysqld.cc > index aa85a02..a7be18f 100644 > --- a/sql/mysqld.cc > +++ b/sql/mysqld.cc > @@ -8149,6 +8149,39 @@ int show_threadpool_idle_threads(THD *thd, SHOW_VAR *var, char *buff, > } > #endif > > +int show_acl_statistics(THD *thd, SHOW_VAR *var, char *buff, > + enum enum_var_type scope) > +{ > + > + var->type= SHOW_ARRAY; > + var->value= buff; > + > + SHOW_VAR *v= (SHOW_VAR *)buff; > + static ACL_statistics acl_stats; > + acl_stats= get_acl_statistics(thd); > + static SHOW_VAR statistics_array[]= { > + {"column_grants", (char *)&acl_stats.column_grants, SHOW_ULONG}, > + {"database_grants", (char *)&acl_stats.database_grants, SHOW_ULONG}, > + {"function_grants", (char *)&acl_stats.function_grants, SHOW_ULONG}, > + {"procedure_grants", (char *)&acl_stats.procedure_grants, SHOW_ULONG}, > + {"proxy_users", (char *)&acl_stats.proxy_users, SHOW_ULONG}, > + {"role_mappings", (char *)&acl_stats.role_mappings, SHOW_ULONG}, rename to "role_grants" please. > + {"roles", (char *)&acl_stats.roles, SHOW_ULONG}, > + {"table_grants", (char *)&acl_stats.table_grants, SHOW_ULONG}, > + {"users", (char *)&acl_stats.users, SHOW_ULONG}, > + {NullS, NullS, SHOW_LONG} > + }; > + v->name= "count"; the name doesn't matter, no need to set it. > + v->value = statistics_array; > + v->type= SHOW_ARRAY; > + v++; > + v->name= NullS; > + v->value= NullS; > + v->type= SHOW_LONG; > + > + return 0; > +} > + > /* > Variables shown by SHOW STATUS in alphabetical order > */ > @@ -8157,6 +8190,7 @@ SHOW_VAR status_vars[]= { > {"Aborted_clients", (char*) &aborted_threads, SHOW_LONG}, > {"Aborted_connects", (char*) &aborted_connects, SHOW_LONG}, > {"Access_denied_errors", (char*) offsetof(STATUS_VAR, access_denied_errors), SHOW_LONG_STATUS}, > + {"Acl", (char*) &show_acl_statistics, SHOW_FUNC}, > {"Binlog_bytes_written", (char*) offsetof(STATUS_VAR, binlog_bytes_written), SHOW_LONGLONG_STATUS}, > {"Binlog_cache_disk_use", (char*) &binlog_cache_disk_use, SHOW_LONG}, > {"Binlog_cache_use", (char*) &binlog_cache_use, SHOW_LONG}, > diff --git a/sql/sql_acl.cc b/sql/sql_acl.cc > index aaa1276..e97d094 100644 > --- a/sql/sql_acl.cc > +++ b/sql/sql_acl.cc > @@ -10340,6 +10340,14 @@ applicable_roles_insert(ACL_USER_BASE *grantee, ACL_ROLE *role, void *ptr) > return 0; > } > > +static my_bool count_column_privileges(void *grant_table, > + void *current_count) > +{ > + HASH hash_columns = ((GRANT_TABLE *)grant_table)->hash_columns; > + *(ulong *) current_count+= hash_columns.records; > + return 0; > +} > + > #else > bool check_grant(THD *, ulong, TABLE_LIST *, bool, uint, bool) > { > @@ -10347,6 +10355,32 @@ bool check_grant(THD *, ulong, TABLE_LIST *, bool, uint, bool) > } > #endif /*NO_EMBEDDED_ACCESS_CHECKS */ > > +ACL_statistics get_acl_statistics(THD* thd) > +{ > + ACL_statistics result; > + bzero(&result, sizeof(result)); > +#ifndef NO_EMBEDDED_ACCESS_CHECKS > + mysql_rwlock_rdlock(&LOCK_grant); > + mysql_mutex_lock(&acl_cache->lock); > + > + result.users= acl_users.elements; > + result.roles= acl_roles.records; > + result.database_grants= acl_dbs.elements; > + result.function_grants= proc_priv_hash.records; > + result.procedure_grants= proc_priv_hash.records; > + result.proxy_users= acl_proxy_users.elements; > + result.role_mappings= acl_roles_mappings.records; > + result.table_grants= column_priv_hash.records; > + my_hash_iterate(&column_priv_hash, count_column_privileges, > + &result.column_grants); > + > + mysql_mutex_unlock(&acl_cache->lock); > + mysql_rwlock_unlock(&LOCK_grant); > + > +#endif > + return result; > +} > + > int fill_schema_enabled_roles(THD *thd, TABLE_LIST *tables, COND *cond) > { > TABLE *table= tables->table; > diff --git a/sql/sql_acl.h b/sql/sql_acl.h > index 55b00a9..3abf5d7 100644 > --- a/sql/sql_acl.h > +++ b/sql/sql_acl.h > @@ -406,4 +406,14 @@ int acl_set_default_role(THD *thd, const char *host, const char *user, > extern ulong role_global_merges, role_db_merges, role_table_merges, > role_column_merges, role_routine_merges; > #endif > +/** > + Specify which statistical information (counts) to retrieve from > + the ACL data structures. > +*/ > +struct ACL_statistics > +{ > + ulong column_grants, database_grants, function_grants, procedure_grants, > + proxy_users, role_mappings, roles, table_grants, users; > +}; > +ACL_statistics get_acl_statistics(THD *thd); > #endif /* SQL_ACL_INCLUDED */ I would do it slightly differently. No ACL_statistics structure and no get_acl_statistics function, but instead I'd use SHOW_ARRAY in mysqld.cc: {"Acl", (char*) &show_acl_statistics, SHOW_ARRAY}, and in sql_acl.cc I'd have an array static SHOW_VAR statistics_array[]= { {"column_grants", (char *)&get_column_grants, SHOW_SIMPLE_FUNC}, {"database_grants", (char *)&acl_dbs.elements, SHOW_ULONG}, {"function_grants", (char *)&proc_priv_hash.records, SHOW_ULONG}, {"procedure_grants", (char *)&proc_priv_hash.records, SHOW_ULONG}, {"proxy_users", (char *)&acl_proxy_users.elements, SHOW_ULONG}, {"role_grants", (char *)&acl_roles_mappings.records, SHOW_ULONG}, {"roles", (char *)&acl_roles.records, SHOW_ULONG}, {"table_grants", (char *)&column_priv_hash.records, SHOW_ULONG}, {"users", (char *)&acl_users.elements, SHOW_ULONG}, {NullS, NullS, SHOW_LONG} }; That is, only column_grants need a function call (with mutex locks), and it's SHOW_SIMPLE_FUNC, not SHOW_FUNC. The different is that a typical (for example) SHOW STATUS LIKE '%sort%' will have to invoke all SHOW_FUNC functions, because otherwise it cannot know names of the variables and cannot match them. But SHOW_SIMPLE_FUNC never returns a SHOW_ARRAY, so it can be matched without a function call (so, no mutex lock unless the user specifically asked for Acl_column_grants value). Also, status variables are allowed to be "slightly imprecise", so they're often incremented without a mutex. That is, you aren't required to protect them here with a mutex either. Regards, Sergei

1 0

[Maria-developers] MDEV-6566 Different INSERT behaviour on bad bytes with and without character set conversion
by Alexander Barkov 13 Mar '15

13 Mar '15

Hi Sergei, Please review a patch for MDEV-6566. Thanks.

2 5

[Maria-developers] Enabling feedback pluging for MariaDB 10.1.4
by Michael Widenius 12 Mar '15

12 Mar '15

Hi! We had originally planned to enable the feedback plugin by default for the Alpha and Beta stages of MariaDB 10.1, but we forgot to do this for the alpha so I suggested Sergei today that we should enable it for the beta period of MariaDB 10.0 We agreed that before doing this we would have a discussion on the maria developer list to explain why we would like to do this. The reason we would like to enable it is to get information about which features people are testing and using in MariaDB 10.1, so that we can concentrate our testing and bug fixing during the beta phase for those areas that people are actually using. This would help our development work a lot and would also help us to plan for MariaDB 10.2 development. Anyone will be able to disable the feedback plugin by adding: --feedback=OFF to their my.cnf file. In MariaDB 10.1 gamma/release we would make it default OFF again. We will of course inform everyone about this change, including how to disable the feedback, in the MariaDB changelog and release notes. We do of course hope that as many as possible will have it on, even after gamma, to help us with our development work! Information about the MariaDB feedback plugin and the statistics it provides can be found at: http://mariadb.org/feedback_plugin https://mariadb.com/kb/feedback-plugin As most MariaDB users should know, the feedback is totally anonymous and no private or sensitive information is being sent. Any comments, suggestions or recommendations? Regards, Michael Widenius Creator of MySQL and MariaDB

10 14

Re: [Maria-developers] 6c772ae: MDEV-6956:SET STATEMENT default_master_connection = ... has no effect
by Sergei Golubchik 12 Mar '15

12 Mar '15

Hi, Sanja! On Mar 06, sanja(a)mariadb.com wrote: > revision-id: 6c772ae62a0a43ea227f491509c02efb99c5ce92 > parent(s): da181fee4ec849985b15a758c6b26058a0f5e318 > committer: Oleksandr Byelkin > branch nick: server > timestamp: 2015-03-06 15:51:51 +0100 > message: > > MDEV-6956:SET STATEMENT default_master_connection = ... has no effect > > the problem was in assigning default value during parsing. Ok to push, but see below. > diff --git a/sql/sql_yacc.yy b/sql/sql_yacc.yy > index 8025340..8255306 100644 > --- a/sql/sql_yacc.yy > +++ b/sql/sql_yacc.yy > @@ -2445,7 +2445,9 @@ optional_connection_name: > /* empty */ > { > LEX *lex= thd->lex; > - lex->mi.connection_name= thd->variables.default_master_connection; > + lex->mi.connection_name.str= NullS; > + lex->mi.connection_name.length= 0; > + //thd->variables.default_master_connection; you've forgot to remove this commented-out line. and prefer to use null_lex_str: lex->mi.connection_name= null_lex_str; Regards, Sergei

2 2

Re: [Maria-developers] 7ceff99: MDEV-6954: SET STATEMENT rand_seedX = ...FOR ... makes the next rand() to return 0
by Sergei Golubchik 11 Mar '15

11 Mar '15

Hi, Sanja! On Mar 11, sanja(a)mariadb.com wrote: > revision-id: 7ceff99ff25b74bb188848ce0e5f9e2f1ddac3d0 > parent(s): 9f4ee16bf351ea2969d73abcdea92e70538c723c > committer: Oleksandr Byelkin > branch nick: server > timestamp: 2015-03-11 11:02:13 +0100 > message: > > MDEV-6954: SET STATEMENT rand_seedX = ...FOR ... makes the next rand() to return 0 > > The variables now return real values. They can't be made usual variables because they do not have DEFAULT value. > > diff --git a/mysql-test/suite/sys_vars/inc/sysvars_server.inc b/mysql-test/suite/sys_vars/inc/sysvars_server.inc > index fb6121d..3a9a312 100644 > --- a/mysql-test/suite/sys_vars/inc/sysvars_server.inc > +++ b/mysql-test/suite/sys_vars/inc/sysvars_server.inc > @@ -17,6 +17,7 @@ select * from information_schema.system_variables > where variable_name not like 'aria%' and > variable_name not like 'debug%' and > variable_name not like 'wsrep%' and > + variable_name not like 'rand_seed%%' and 1. why two %% ? :) 2. there are only two variables here, better list them explicilty in the "not in" list 3. if you remove variables from the output here, please add them to the select below. > variable_name not in ( > 'have_openssl', > 'have_symlink', Regards, Sergei

1 0

Re: [Maria-developers] 60c1dc1: MDEV-6997: SET STATEMENT last_insert_id FOR ... does not affect the value written to the binlog
by Sergei Golubchik 11 Mar '15

11 Mar '15

Hi, Sanja! On Feb 25, sanja(a)mariadb.com wrote: > revision-id: 60c1dc1ab24bfe9b3f4a898fe5e7cb070ee02cf7 > parent(s): d0d6284cab70af148269d95744f615454cac24bb > committer: Oleksandr Byelkin > branch nick: work-maria-10.1-MDEV-6997-real > timestamp: 2015-02-25 19:00:57 +0100 > message: > > MDEV-6997: SET STATEMENT last_insert_id FOR ... does not affect the value written to the binlog > > Problem was in reinitialization of > first_successful_insert_id_in_prev_stmt_for_binlog after setting > variables. > > diff --git a/sql/sql_parse.cc b/sql/sql_parse.cc > index 5af700b..54abfd0 100644 > --- a/sql/sql_parse.cc > +++ b/sql/sql_parse.cc > @@ -2728,6 +2728,10 @@ static bool do_execute_sp(THD *thd, sp_head *sp) > my_error(ER_WRONG_ARGUMENTS, MYF(0), "SET"); > goto error; > } > + if (!thd->in_sub_stmt) /* stored functions and triggers are a special case */ > + { > + thd->stmt_depends_on_first_successful_insert_id_in_prev_stmt= 0; > + } > } This seems to be a pretty complicated matter, and your comment isn't very helpful. Try this: The value of last_insert_id is remembered in THD to be written to binlog when it's used *the first time* in the statement. But SET STATEMENT must read the old value of last_insert_id to be able to restore it at the end. This should not count at "reading of last_insert_id" and should not remember last_insert_id for binlog. That is, it should set stmt_depends_on_first_successful_insert_id_in_prev_stmt flag. And try the test case where SET STATEMENT last_insert_id is used inside a trigger or a stored function. I don't think you handle this case correctly. Try both cases, when stmt_depends... is 0 and is 1 at the point of this if(). I suspect you should not set stmt_depends_on_first_successful_insert_id_in_prev_stmt to 0, but you need to make sure it didn't change. That is, you need to set it to its old value (which may or may not be 0). Regards, Sergei

1 0

Re: [Maria-developers] [Commits] 828a7b6: MDEV-6858: enforce_storage_engine option
by Sergei Golubchik 11 Mar '15

11 Mar '15

Hi, Jan! This looks good, thanks! There's just one strange thing I've noticed, see sysvars_server_notembedded test below. I don't understand why it's happening, could you investigate, please? On Mar 10, Jan Lindström wrote: > revision-id: 828a7b67768c7e7a72c2a687f5a3ff77bc61536b > parent(s): f7d196bb2e605766668895027c48728f03803734 > committer: Jan Lindström > branch nick: 10.1-innodb > timestamp: 2015-03-10 08:28:51 +0200 > message: > > MDEV-6858: enforce_storage_engine option > > Merge from Percona Server enforced use of a specific storage engine > authored by Stewart Smith. > > Modified to be session variable and modifiable only by SUPER. Use > similar implementation as default_storage_engine. > > diff --git a/mysql-test/suite/sys_vars/r/sysvars_server_notembedded.result b/mysql-test/suite/sys_vars/r/sysvars_server_notembedded.result > index 83c0196..bf8f868 100644 > --- a/mysql-test/suite/sys_vars/r/sysvars_server_notembedded.result > +++ b/mysql-test/suite/sys_vars/r/sysvars_server_notembedded.result > @@ -721,6 +721,20 @@ NUMERIC_BLOCK_SIZE NULL > ENUM_VALUE_LIST OFF,ON > READ_ONLY NO > COMMAND_LINE_ARGUMENT OPTIONAL > +VARIABLE_NAME ENFORCE_STORAGE_ENGINE > +SESSION_VALUE > +GLOBAL_VALUE NULL This looks strange. Why session value is empty string, but global value is NULL? I believe they should be identical, shouldn't they? Preferrably, both should be empty string - I don't think we have system variables that can be NULL. > +GLOBAL_VALUE_ORIGIN COMPILE-TIME > +DEFAULT_VALUE NULL > +VARIABLE_SCOPE SESSION ONLY > +VARIABLE_TYPE VARCHAR > +VARIABLE_COMMENT Force the use of a storage engine for new tables > +NUMERIC_MIN_VALUE NULL > +NUMERIC_MAX_VALUE NULL > +NUMERIC_BLOCK_SIZE NULL > +ENUM_VALUE_LIST NULL > +READ_ONLY NO > +COMMAND_LINE_ARGUMENT NULL > VARIABLE_NAME ERROR_COUNT > SESSION_VALUE 0 > GLOBAL_VALUE NULL > diff --git a/mysql-test/t/enforce_storage_engine.test b/mysql-test/t/enforce_storage_engine.test > new file mode 100644 > index 0000000..80839e2 > --- /dev/null > +++ b/mysql-test/t/enforce_storage_engine.test > @@ -0,0 +1,64 @@ > +-- source include/have_innodb.inc You can keep it this way, but I'd recommend to change to either have_xtradb.inc or (better) not use innodb at all and test with myisam and memory engines. with have_innodb.inc the test will be run twice - for innodb and for xtradb, which is a waste of time, because it doesn't do anything engine-specific here. > +-- source include/not_embedded.inc > + > +--disable_warnings > +drop table if exists t1; > +--enable_warnings Regards, Sergei

2 1

Re: [Maria-developers] [Commits] Rev 4382: MDEV-6892: WHERE does not apply in file:///home/bell/maria/bzr/work-maria-5.5-MDEV-6892/
by Sergey Petrunia 11 Mar '15

11 Mar '15

Hi Sanja, Please find my feedback below. We should discuss it. On Fri, Dec 19, 2014 at 01:24:12PM +0000, sanja(a)askmonty.org wrote: > At file:///home/bell/maria/bzr/work-maria-5.5-MDEV-6892/ > > ------------------------------------------------------------ > revno: 4382 > revision-id: sanja(a)askmonty.org-20141219132350-6bo1g7q6gbivqdky > parent: svoj(a)mariadb.org-20141201105829-ctaoe194abtorhz2 > committer: sanja(a)askmonty.org > branch nick: work-maria-5.5-MDEV-6892 > timestamp: Fri 2014-12-19 14:23:50 +0100 > message: > MDEV-6892: WHERE does not apply > > Taking into account implicit dependence of constant view field from nullable table of left join added. > > Fixed finding real table to check if ut turned to NULL (materialized view & derived taken into account) ... > === modified file 'sql/item.cc' > --- a/sql/item.cc 2014-10-06 17:53:55 +0000 > +++ b/sql/item.cc 2014-12-19 13:23:50 +0000 > @@ -9790,12 +9790,36 @@ void Item_ref::update_used_tables() > (*ref)->update_used_tables(); > } > > +void Item_direct_view_ref::update_used_tables() > +{ > + if (view->is_inner_table_of_outer_join()) > + { > + null_ref_table= NULL; > + check_null_ref(); update_used_tables() is called during optimization. Why does it call check_null_ref() which may check null_ref_table->null_row? Is check_null_ref() function an execution-time function (and so it computes "current" value of an item) or is it an optimization-time function (and so it computes static attributes and can be called at any point in the query) It looks like it's a bit of both, which is really confusing. Could you please clarify? check_null_ref() was introduced by you in another views+outer joins bugfix. Can we get it documented? > + } > + Item_direct_ref::update_used_tables(); > +} Overall, I don't understand why some checks are in used_tables() and some are in update_used_tables(). Could you clarify? > + > table_map Item_direct_view_ref::used_tables() const > { > + TABLE *null_ref= null_ref_table; > + > + if (view->is_inner_table_of_outer_join()) > + { > + if (null_ref == NULL) > + null_ref= view->get_real_join_table(); > + else if (null_ref == NO_NULL_TABLE) > + null_ref= NULL; > + } > + else > + null_ref= NULL; > + > return get_depended_from() ? > OUTER_REF_TABLE_BIT : > ((view->is_merged_derived() || view->merged || !view->table) ? > - (*ref)->used_tables() : > + ((*ref)->used_tables() ? > + (*ref)->used_tables() : > + (null_ref ? null_ref->map : (table_map)0 )) : > view->table->map); Two-level nesting of conditional expressions is too complicated. Please change into 'if' statement and a comment about what's going on. How can Item_direct_view_ref() deped on an outer table?? > } > > > === modified file 'sql/item.h' > --- a/sql/item.h 2014-11-18 14:42:40 +0000 > +++ b/sql/item.h 2014-12-19 13:23:50 +0000 > @@ -3319,7 +3319,8 @@ class Item_direct_view_ref :public Item_ > { > if (null_ref_table == NULL) > { > - if (!(null_ref_table= view->get_real_join_table())) > + if (!view->is_inner_table_of_outer_join() || > + !(null_ref_table= view->get_real_join_table())) > null_ref_table= NO_NULL_TABLE; > } > if (null_ref_table != NO_NULL_TABLE && null_ref_table->null_row) > @@ -3329,6 +3330,7 @@ class Item_direct_view_ref :public Item_ > } > return FALSE; > } > + > public: > Item_direct_view_ref(Name_resolution_context *context_arg, Item **item, > const char *table_name_arg, > @@ -3353,8 +3355,10 @@ public: > bool subst_argument_checker(uchar **arg); > Item *equal_fields_propagator(uchar *arg); > Item *replace_equal_field(uchar *arg); > - table_map used_tables() const; > + table_map used_tables() const; > + void update_used_tables(); > table_map not_null_tables() const; > + bool const_item() const { return used_tables() == 0; } > bool walk(Item_processor processor, bool walk_subquery, uchar *arg) > { > return (*ref)->walk(processor, walk_subquery, arg) || > > === modified file 'sql/table.cc' > --- a/sql/table.cc 2014-10-29 10:22:48 +0000 > +++ b/sql/table.cc 2014-12-19 13:23:50 +0000 The function get_real_join_table() lacks comments, which makes it difficult to review. Do I understand correctly that its action can be described by this comment: /* @brief Given a merged view (or a merged derived table), find its first* base** table. @detail (*) first means as defined in the syntax (**) base means we need a physical table, one that has a bit in table->map and is present in select_lex->leaf_tables. Example: t1 join ( (select * from t2, t3 ) as derivedA join (select * from t5, t6) as derivedC ) as derivedB here, derivedB->get_real_join_table()="t2" */ > @@ -5012,7 +5012,8 @@ TABLE *TABLE_LIST::get_real_join_table() > TABLE_LIST *tbl= this; > while (tbl->table == NULL || tbl->table->reginfo.join_tab == NULL) I'm quite surprised that joins are run over tables that don't have reginfo.join_tab set appropriately... > { > - if (tbl->view == NULL && tbl->derived == NULL) > + if ((tbl->view == NULL && tbl->derived == NULL) || > + tbl->is_materialized_derived()) I'm wondering why part of conditions are in while(...) and the other part is here. > break; > /* we do not support merging of union yet */ > DBUG_ASSERT(tbl->view == NULL || > The code below this line looks bizarre (and it is still from your recent fixes to VIEWs): { List_iterator_fast<TABLE_LIST> ti; { List_iterator_fast<TABLE_LIST> ti(tbl->view != NULL ? why define 'ti' and then immediately redefine it? BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

2 1

Re: [Maria-developers] f954bb7: MDEV-7671: Cache VIEW definitions in the TDC
by Sergei Golubchik 11 Mar '15

11 Mar '15

Hi, Sanja! On Mar 10, sanja(a)mariadb.com wrote: > revision-id: f954bb7fdc527cf816253fe0e7adb4239c368624 > parent(s): f7d196bb2e605766668895027c48728f03803734 > committer: Oleksandr Byelkin > branch nick: server > timestamp: 2015-03-10 12:52:45 +0100 > message: > > MDEV-7671: Cache VIEW definitions in the TDC > > (changes of backported patch are very small: strlen removed, error > processing fixed, view open statistics added) Ok to push Regards, Sergei

1 0

[Maria-developers] GSOC 2015: Support for popular formats on Dynamic columns.
by arlene seuleu 11 Mar '15

11 Mar '15

Hi I am Arlene Flavie a software Engineering student and I wish to make my contribution to opensource through Mariadb. I have build mariadb from a git clone and ran the tests. I am interested in working on adding routines to import and export popular data formats to and from dynamic columns. I am currently studying the dynamic column data structures in mysql/ma_dyncol.** and I think what Oleksandr Byeklin proposed makes sense to me. Thats using methods like xxx_init() and xxx_named() to play with the old dynamic column blob format. I will be applying to work on this task for Gsoc 2015 and im hoping to learn much in open source development. Cheers.

2 1

[Maria-developers] Interest in GSoc
by Acha Bill 11 Mar '15

11 Mar '15

Hello, I am Acha Bill, Computer Engineering, University of Buea, Cameroon. Second year undergraduate. I am very much interested in the project; Import and export popular data formats from and to dynamic columns. for Gsoc 15. I have experience with C. Any pointers as to what next will be appreciated. Thanks.

2 1

Re: [Maria-developers] [Maria-discuss] Enabling feedback pluging for MariaDB 10.1.4
by Justin Swanhart 11 Mar '15

11 Mar '15

It tells you how much a sample used and you don't know what fraction of the total that represents, or if their usage is representative of the larger base. I maintain they are not meaningful but you can feel free to disagree. Sent from my iPhone > On Mar 10, 2015, at 7:42 PM, Michael Widenius <michael.widenius(a)gmail.com> wrote: > > Hi! > > <cut> > >> I agree with Kristian. Given the way it works, the statistics are really meaningless and I feel you shouldn't drive important choices based on bad statistics. > > From a statistics point of view, the automatic feedback will tell us a > lot about how people use MariaDB and is far from meaningless! > (We only need a small statistical sample to know how the majority is > using the server). > > The most important things for our developers, that the phone home > feature will tell us, are probably: > - Features that are used (and how much they are used) > - Features that not used. > - Combinations of features used. > - The stability of the servers. > - How often upgrades are done > > This will allow us to better concentrate the few development resources > on features that are important to our users. > > In the past, our main problem for allocating resources has always been > "does anyone really use or care about this". Now we finally will get > some information to make these kind of decisions easier. > > Regards, > Monty

2 1

[Maria-developers] GSOC 2015 :UDFs returning arrays
by Blessing Atie 10 Mar '15

10 Mar '15

Hi My name is Blessing Atie and i would love to participate in the Google summer of code 2015 under Mariadb. I am interested in the project UDF returning arrays. I have build mariadb from source and and i also ran the test suites. I am currently studing the sql/sql.** files to understand UDF's and i also downloaded some ebook on mysql internals and mysql plugin development which im reading while hoping to find a breakthrough on this project in them. I would love to know if there are any bugs related to this task on JIRA I could work on as a means of preparing for the project. Please give me the link. Regards Blessing.

1 0

Re: [Maria-developers] [Commits] 83659a6: MDEV-7006 MDEV-7007: SET STATEMENT and slow log
by Sergei Golubchik 10 Mar '15

10 Mar '15

Hi, Sanja! On Feb 17, sanja(a)mariadb.com wrote: > revision-id: 83659a68642da2eb6e9327f17a0ef730f512aa96 > parent(s): 3e2849d2a01b06a61407b00989c3f981e62dd183 > committer: Oleksandr Byelkin > branch nick: server > timestamp: 2015-02-17 12:54:51 +0100 > message: > > MDEV-7006 MDEV-7007: SET STATEMENT and slow log > fixed embedded server tests > MDEV-7009: SET STATEMENT min_examined_row_limit has no effect > MDEV-6948:SET STATEMENT gtid_domain_id = ... FOR has no effect (same for gtid_seq_no and server_id) > > old values of SET STATENENT variables now saved in its own Query_arena and restored later Looks good to me. I didn't check, though, whether you've put every restore correctly. See minor comments below: > diff --git a/mysql-test/t/set_statement_notembedded_binlog.test b/mysql-test/t/set_statement_notembedded_binlog.test > new file mode 100644 > index 0000000..c9c2334 > --- /dev/null > +++ b/mysql-test/t/set_statement_notembedded_binlog.test > @@ -0,0 +1,22 @@ > +--source include/have_log_bin.inc > +--source include/not_embedded.inc > + > +--disable_warnings > +drop table if exists t1; > +drop view if exists t1; > +--enable_warnings This was useful in our early MySQL days, but it's not anymore. I've stopped adding these drop...if exists at the beginning of test files a couple of years ago. mtr has a after-test check to ensure that no test leaves its tables behind. So one can rely on every test starting clean. > + > +--echo # > +--echo # MDEV-6948: SET STATEMENT gtid_domain_id = ... FOR has no effect > +--echo # (same for gtid_seq_no and server_id) > +--echo # > +reset master; > +create table t1 (i int); > +set gtid_domain_id = 10; > +insert into t1 values (1),(2); > +set statement gtid_domain_id = 20 for insert into t1 values (3),(4); > + > +--replace_column 1 x 2 x 3 x 4 x 5 x > +show binlog events limit 5,5; > + > +drop table t1; > diff --git a/sql/sql_lex.cc b/sql/sql_lex.cc > index 0c1c3b3..ee956ed 100644 > --- a/sql/sql_lex.cc > +++ b/sql/sql_lex.cc > @@ -4223,6 +4224,56 @@ int LEX::print_explain(select_result_sink *output, uint8 explain_flags, > return res; > } > > +void LEX::set_arena_for_set_stmt(Query_arena *backup) > +{ > + DBUG_ENTER("LEX::set_arena_for_set_stmt"); > + DBUG_ASSERT(arena_for_set_stmt== 0); > + if (!mem_root_for_set_stmt) > + { > + mem_root_for_set_stmt= new MEM_ROOT(); > + if (!(mem_root_for_set_stmt)) > + DBUG_VOID_RETURN; What happens in OOM case? You return early, but does the caller know that? Or it'll continue, thinking that everything's ok? > + init_sql_alloc(mem_root_for_set_stmt, ALLOC_ROOT_SET, ALLOC_ROOT_SET, > + MYF(MY_THREAD_SPECIFIC)); > + } > + if (!(arena_for_set_stmt= new Query_arena(mem_root_for_set_stmt, Why are you allocating memory for Query_arena, instead of creating it on mem_root_for_set_stmt? > + Query_arena::STMT_INITIALIZED))) > + DBUG_VOID_RETURN; > + DBUG_PRINT("info", ("mem_root: 0x%lx arena: 0x%lx", > + (ulong) mem_root_for_set_stmt, > + (ulong) arena_for_set_stmt)); > + thd->set_n_backup_active_arena(arena_for_set_stmt, backup); > + DBUG_VOID_RETURN; > +} > + > + > +void LEX::reset_arena_for_set_stmt(Query_arena *backup) > +{ > + DBUG_ENTER("LEX::reset_arena_for_set_stmt"); > + DBUG_ASSERT(arena_for_set_stmt); > + thd->restore_active_arena(arena_for_set_stmt, backup); > + DBUG_PRINT("info", ("mem_root: 0x%lx arena: 0x%lx", > + (ulong) arena_for_set_stmt->mem_root, > + (ulong) arena_for_set_stmt)); > + DBUG_VOID_RETURN; > +} > + > + > +void LEX::free_arena_for_set_stmt() > +{ > + DBUG_ENTER("LEX::free_arena_for_set_stmt"); > + if (!arena_for_set_stmt) > + return; > + DBUG_PRINT("info", ("mem_root: 0x%lx arena: 0x%lx", > + (ulong) arena_for_set_stmt->mem_root, > + (ulong) arena_for_set_stmt)); > + arena_for_set_stmt->free_items(); > + delete(arena_for_set_stmt); > + free_root(mem_root_for_set_stmt, MYF(MY_KEEP_PREALLOC)); > + arena_for_set_stmt= 0; > + DBUG_VOID_RETURN; > +} > + > void LEX::restore_set_statement_var() > { > DBUG_ENTER("LEX::restore_set_statement_var"); > diff --git a/sql/sql_lex.h b/sql/sql_lex.h > index 46d667c..ed3c4a0 100644 > --- a/sql/sql_lex.h > +++ b/sql/sql_lex.h > @@ -2677,10 +2688,22 @@ struct LEX: public Query_tables_list > limit_rows_examined_cnt= ULONGLONG_MAX; > } > > + > + inline void free_set_stmt_mem_root() > + { Please, add DBUG_ASSERT(!is_arena_for_set_stmt()); > + if (mem_root_for_set_stmt) > + { > + free_root(mem_root_for_set_stmt, MYF(0)); > + delete mem_root_for_set_stmt; > + mem_root_for_set_stmt= 0; > + } > + } > + > LEX(); > > virtual ~LEX() > { > + free_set_stmt_mem_root(); > destroy_query_tables_list(); > plugin_unlock_list(NULL, (plugin_ref *)plugins.buffer, plugins.elements); > delete_dynamic(&plugins); Regards, Sergei

2 1

[Maria-developers] GSOC 2015: Indexes on virtual columns
by Sichom Lionel 10 Mar '15

10 Mar '15

Hello, My name is Lionel Sichom, 2nd year undergraduate student at the University of Buea. I am a newbie in Open source community and I am interested in participating at the Google Summer of Code 2015 with MariaDB. I am mostly interested with the project Indexes on virtual columns. As for now I have 5.5.41-MariaDB version and I have already downloded the code, and I am already compiling the code. Can someone please guide me through the steps next to follow. Thanks Lionel Sichom

2 1

[Maria-developers] Question regarding MDEV-7682
by Vicențiu Ciorbaru 10 Mar '15

10 Mar '15

Hi Holyfoot! I've made some changes regarding key_length computation to fix MDEV-6838. Changing this seems to have lead to us discovering another issue outlined in MDEV-7682 (that I've just created). It seems that the spatial key returns a key_length of 0. I've spoken to Sergei Petrunia and we are not sure about this particular case. pack_length returns a bigger number (12). According to the other comments in the code, pack_length also returns the length field associated with the column, but we are only interested in the length of the column field, without any metadata information. The function TABLE:create_key_part_by_field adds the metadata length itself afterwards. Do you have any input on this issue, namely why the key_length is returned as 0 for the LINESTRING column? Regards, Vicentiu

2 1

[Maria-developers] [GSoC'15] Running MaxScale and MariaDB servers on single machine
by sriram patil 09 Mar '15

09 Mar '15

Hi All, I tried running 2 MariaDB servers and MaxScale on my laptop. I gave different configurations to both MariaDB servers with separate data and log directories. Referred this <http://www.ducea.com/2009/01/19/running-multiple-instances-of-mysql-on-the-…> for it. Also have set up MaxScale from "MaxScale for rest of us" series on MariaDB blog. 1. https://mariadb.com/blog/maxscale-rest-us-part-1 2. https://mariadb.com/blog/maxscale-rest-us-part-2 3. https://mariadb.com/blog/maxscale-rest-us-part-3 4. https://mariadb.com/blog/maxscale-rest-us-part-4 So, i tried to set up the readwritesplit router with 1 master and 1 slave. Here, the replication mechanism worked fine but the queries are always executed on the master server. Example output for a read query: mysql -h localhost -u maxuser -p test -e "select c1, @@server_id from t1" Enter password: +------+-------------+ | c1 | @@server_id | +------+-------------+ | 1 | 1 | +------+-------------+ server_id for master is 1 whereas for slave it is 11, which is never displayed in the output. Can someone help me with this? Should I try the same config with one more slave? Are there any better resources for setting up MaxScale? Thanks, Sriram

2 5

[Maria-developers] MariaDB Maxscale Project (GSOC 2015)
by Aalekh Nigam 09 Mar '15

09 Mar '15

Hello, I am Aalekh Nigam, 3rd year Undergraduate Student at JIIT, Noida. I have recently successfully completed Mozilla Winter Of Security and Owasp Winter of Code, and is aiming for MaxScale projects for this GSOC. Can someone please guide me through starter’s bug for Maxscale to get my hands on with MaxScale development ? Thank You —Aalekh

2 1

[Maria-developers] MDEV-7643 review
by Elena Stepanova 09 Mar '15

09 Mar '15

Hi Sergei, http://lists.askmonty.org/pipermail/commits/2015-March/007508.html Could you please check patch for https://mariadb.atlassian.net/browse/MDEV-7643 (MTR creates nested links when tests are run with --mem)? It's in the area that you looked at recently, so you might think of something that I could have missed. Thanks, Elena

2 1

Re: [Maria-developers] [Commits] e464bc6: MDEV-7107 Sporadic test failure in multi_source.multisource
by Kristian Nielsen 09 Mar '15

09 Mar '15

elenst(a)montyprogram.com writes: > revision-id: e464bc6fe3d7566831da5e26447d90e758b5aa2a > committer: Elena Stepanova > timestamp: 2015-03-09 02:07:47 +0200 > message: > > MDEV-7107 Sporadic test failure in multi_source.multisource > > Extend show_slave_status.inc to run SHOW ALL SLAVES STATUS and > SHOW SLAVE 'name' STATUS on demand, and make the test use > the include file instead of direct SHOW statements Looks fine, thanks! Ok to push. I agree with your changes to show_slave_status.inc. I think they will be useful for futute test cases also. - Kristian.

1 0

[Maria-developers] MDEV-7107 review
by Elena Stepanova 09 Mar '15

09 Mar '15

Hi Kristian, Could you please take a quick look at the patch for MDEV-7107 (Sporadic test failure in multi_source.multisource). http://lists.askmonty.org/pipermail/commits/2015-March/007543.html I don't expect you'll mind changes to the test itself, we discussed earlier that's how SHOW SLAVE STATUS should be handled; but please check if you're okay with the additions to show_slave_status.inc, or you would prefer a separate include file for SHOW ALL SLAVES STATUS instead. Thanks, Elena

1 0

[Maria-developers] rpl test with gtid as default
by Jonas Oreland 09 Mar '15

09 Mar '15

what do you think ? /Jonas

3 3

Re: [Maria-developers] a1c4620: MDEV-6858: enforce_storage_engine option
by Sergei Golubchik 09 Mar '15

09 Mar '15

Hi, Jan! On Mar 08, Jan Lindström wrote: > revision-id: a1c4620646d43eaff979c132a57ee2101070fa5e > parent(s): 1b74f32b1d94465f6d6e14e4aa7ba1f94c32e39e > committer: Jan Lindström > branch nick: 10.1-innodb > timestamp: 2015-03-08 17:23:05 +0200 > message: > > MDEV-6858: enforce_storage_engine option > > Merge from Percona Server enforced use of a specific storage engine > authored by Stewart Smith. > > Modified to be session variable and modifiable only by SUPER. Use > similar implementation as default_storage_engine. > > diff --git a/mysql-test/t/enforce_storage_engine.test b/mysql-test/t/enforce_storage_engine.test > new file mode 100644 > index 0000000..36231e6 > --- /dev/null > +++ b/mysql-test/t/enforce_storage_engine.test > @@ -0,0 +1,35 @@ ... > +--error ER_UNKNOWN_STORAGE_ENGINE > +CREATE TABLE t2 (c1 INT PRIMARY KEY AUTO_INCREMENT, c2 VARCHAR(10)) ENGINE=MyISAM; > + > +SET SESSION enforce_storage_engine=MyISAM; > +select @@session.enforce_storage_engine; > +select * from t1; > +drop table t1; > + > +--error 1286 > +SET SESSION enforce_storage_engine=FooBar; > + > +select @@session.enforce_storage_engine; Consider adding tests for "only modifiable by SUPER" (create a new user, connect with it, see that enforce_storage_engine cannot be changed), and for modified enforce_storage_engine (you tested that it can be changed, but didn't do any CREATE TABLE with the new value). > diff --git a/sql/handler.cc b/sql/handler.cc > index 12d7ffb..5d5ac30 100644 > --- a/sql/handler.cc > +++ b/sql/handler.cc > @@ -117,6 +117,15 @@ static plugin_ref ha_default_tmp_plugin(THD *thd) > return ha_default_plugin(thd); > } > > + > +static plugin_ref ha_enforced_plugin(THD *thd) > +{ > + if (thd->variables.enforced_table_plugin) > + return thd->variables.enforced_table_plugin; > + return NULL; > +} > + > + > /** @brief > Return the default storage engine handlerton for thread > > @@ -148,6 +157,25 @@ handlerton *ha_default_tmp_handlerton(THD *thd) > > > /** @brief > + Return the enforced storage engine handlerton for thread > + > + SYNOPSIS > + ha_enforce_handlerton(thd) > + thd current thread > + > + RETURN > + pointer to handlerton OR NULL > + > +*/ > +handlerton *ha_enforced_handlerton(THD *thd) > +{ > + plugin_ref plugin= ha_enforced_plugin(thd); > + if (plugin) > + return plugin_hton(plugin); > + return NULL; > +} > + > +/** @brief > Return the storage engine handlerton for the supplied name > > SYNOPSIS > diff --git a/sql/mysqld.cc b/sql/mysqld.cc > index b97742d..6966fd9 100644 > --- a/sql/mysqld.cc > +++ b/sql/mysqld.cc > @@ -4738,8 +4738,8 @@ static void add_file_to_crash_report(char *file) > #define init_default_storage_engine(X,Y) \ > init_default_storage_engine_impl(#X, X, &global_system_variables.Y) > > -static int init_default_storage_engine_impl(const char *opt_name, > - char *engine_name, plugin_ref *res) > +int init_default_storage_engine_impl(const char *opt_name, > + char *engine_name, plugin_ref *res) You've made init_default_storage_engine_impl() extern, but you never use it anywhere. And you've forgot to create a command line option for enforced_table_plugin. > { > if (!engine_name) > { > diff --git a/sql/sql_table.cc b/sql/sql_table.cc > index e9a1ae9..184c073 100644 > --- a/sql/sql_table.cc > +++ b/sql/sql_table.cc > @@ -9745,12 +9745,24 @@ static bool check_engine(THD *thd, const char *db_name, > DBUG_ENTER("check_engine"); > handlerton **new_engine= &create_info->db_type; > handlerton *req_engine= *new_engine; > + handlerton *enf_engine= ha_enforced_handlerton(thd); I'd write it directly as thd->variables.enforced_table_plugin ? plugin_hton(thd->variables.enforced_table_plugin) : NULL; Or as an inlined function here, in sql_table.cc. Not as two small but non-inlined functions in handler.cc, that are only used once here. > bool no_substitution= thd->variables.sql_mode & MODE_NO_ENGINE_SUBSTITUTION; > *new_engine= ha_checktype(thd, req_engine, no_substitution); > DBUG_ASSERT(*new_engine); > if (!*new_engine) > DBUG_RETURN(true); > > + if (enf_engine) > + { > + if (enf_engine != *new_engine && no_substitution) > + { > + const char *engine_name= ha_resolve_storage_engine_name(req_engine); > + my_error(ER_UNKNOWN_STORAGE_ENGINE, MYF(0), engine_name, engine_name); > + DBUG_RETURN(TRUE); > + } > + *new_engine= enf_engine; > + } > + > if (req_engine && req_engine != *new_engine) > { > push_warning_printf(thd, Sql_condition::WARN_LEVEL_NOTE, > diff --git a/sql/sys_vars.cc b/sql/sys_vars.cc > index 8371df0..cd53714 100644 > --- a/sql/sys_vars.cc > +++ b/sql/sys_vars.cc > @@ -3437,6 +3437,28 @@ static Sys_var_plugin Sys_default_tmp_storage_engine( > SESSION_VAR(tmp_table_plugin), NO_CMD_LINE, > MYSQL_STORAGE_ENGINE_PLUGIN, DEFAULT(&default_tmp_storage_engine)); > > +extern int init_default_storage_engine_impl(const char* opt_name, char *engine_name, plugin_ref *plugin); Not used below. > + > + > +static bool check_super_and_not_null(sys_var *self, THD*thd, set_var *var) I think that NULL value should be allowed, it means "don't enforce a storage engine". > +{ > +#ifndef NO_EMBEDDED_ACCESS_CHECKS > + if (!(thd->security_ctx->master_access & SUPER_ACL)) > + { > + my_error(ER_SPECIFIC_ACCESS_DENIED_ERROR, MYF(0), "SUPER"); > + return true; > + } > +#endif > + return var->value && var->value->is_null(); > +} > + > +static Sys_var_plugin Sys_enforce_storage_engine( > + "enforce_storage_engine", "Force the use of a storage engine for new " > + "tables", > + SESSION_VAR(enforced_table_plugin), > + NO_CMD_LINE, MYSQL_STORAGE_ENGINE_PLUGIN, > + DEFAULT(0), NO_MUTEX_GUARD, NOT_IN_BINLOG, ON_CHECK(check_super_and_not_null)); > + > #if defined(ENABLED_DEBUG_SYNC) > /* > Variable can be set for the session only. Regards, Sergei

1 0

[Maria-developers] [GSoC'15] MaxScale Project
by sriram patil 07 Mar '15

07 Mar '15

Hi Markus, I am Sriram, a Masters' student at IIIT Hyderabad, India. I have worked with MariaDB in GSoC-2014 on MDEV-5359 under Alexander Barkov. I have also documented my work on my blog <http://srirampatil.tumblr.com/>. I am interested to work on "MaxScale filter to capture incoming operations for consumption in external sources". I have downloaded and built MariaDB 10.1 source code from github. I was trying to build MaxScale but it resulted in some errors. I followed the installation instructions given on MaxScale github(develop branch). The error is as follows /home/sriram/gsoc/MaxScale/query_classifier/query_classifier.cc: In function ‘bool skygw_stmt_causes_implicit_commit(LEX*, int*)’: /home/sriram/gsoc/MaxScale/query_classifier/query_classifier.cc:848:40: error: ‘struct LEX’ has no member named ‘drop_temporary’ succp = !(lex->drop_temporary); ^ make[2]: *** [query_classifier/CMakeFiles/query_classifier.dir/query_classifier.cc.o] Error 1 make[1]: *** [query_classifier/CMakeFiles/query_classifier.dir/all] Error 2 I checked struct LEX from MariaDB source code and it does not have the mentioned field. So, is MaxScale expecting some other MariaDB source code? It would be great if you could give me some pointers regarding the same. Thanks, Sriram Patil

2 2

Re: [Maria-developers] GSoC 15 : MaxScale Project under Markus Makela
by Markus Mäkelä 06 Mar '15

06 Mar '15

Hi, The most up to date documentation for MaxScale can be found on the develop branch on GitHub: https://github.com/mariadb-corporation/MaxScale/blob/develop/Documentation/… I suggest using that rather than the PDF documentation since it's a lot more easy to use. The filter interface can be found in the server/include/filter.h file. All filters must implement these functions for them to work with MaxScale. The filters are located in the server/modules/filter folder. The best way to get a hang of the filters is to take a look at one and the regexfilter and qlafilter are the ones that show the filter interface in action without having much code in them. Other useful stuff to look at can be found in the server/include folder are buffer.h(MaxScale uses these to route queries though it), modutils.h(useful functions for SQL handling), housekeeper.h(for tasks that take a long time this can be used) and hashtable.h(a generic hashtable). All of the MaxScale team works around UTC+1. For quick questions ask them on #maxscale on Freenode but email will most likely be answered as fast as IRC. Markus > Hello guys, > > As per the official list > <https://mariadb.com/kb/en/mariadb/google-summer-of-code-2015/> of projects > by MariaDB, I'm interested in the project - MaxScale filter to capture > incoming operations for consumption in external sources ( Mentor : Markus > Makela ) > > *I've 3 requests/doubts : * > > 0.I've downloaded the MaxScale code from github > <https://github.com/mariadb-corporation/MaxScale> and have read the > documentation. > It says the filter configuration file is : MaxScale.ini > I didn't find this file in the codebase. > > Nor was I able to figure out the code-location related to the "filter" > module. > > 1.Kindly suggest for the further things I should do/try. > > Shall I install MariaDB & MaxScale? > What part of code-base should I look into & play around? > > 2.What is the UTC time when most of you guys are online on IRC-channel > (#maria)? > > Since I'm from India, I need to know the time when most of you guys are > online to get immediate help on IRC. > > > Thanks in advance. > > -- > Lokesh Walase -- Markus Mäkelä, Software Engineer MariaDB Corporation t: +358 40 7740484 | Skype: markus.j.makela

1 0

[Maria-developers] Comments/thoughts on patch
by Jonas Oreland 06 Mar '15

06 Mar '15

Hi Kristian, I wrote attached patch (so that i would understand what code actually does) and found myself while refactoring also doing a change, namely that i removed the check for server_id in a couple of places. it still passes all tests... could you comment on this, or write(outline) a testcase that fails with my patch. i think functionality that i might have removed would be if you write have 2 mysqld M1 & M2 and your replicate both way...and you *don't* use domain_id...and you write concurrently on both mysqld (but maybe disjoint data ??). feedback/comments appreciated. /Jonas ps. patch applies to f4f37533a09b9776e8d5ac3f3a27957f553c9043

2 9

[Maria-developers] MaxScale JIRA project
by Vangelis Katsikaros 05 Mar '15

05 Mar '15

Hi While browsing the GSOC projects, I noticed that the MX project, and the tickets mentioned, require signing in. I don't know if this intentional or not (the MDEV project doesn't require signin for viewing the project). Just thought I should mention it. Regards Vangelis

2 2

[Maria-developers] GSoC 15 : MaxScale Project under Markus Makela
by Lokesh Walase 04 Mar '15

04 Mar '15

Hello guys, As per the official list <https://mariadb.com/kb/en/mariadb/google-summer-of-code-2015/> of projects by MariaDB, I'm interested in the project - MaxScale filter to capture incoming operations for consumption in external sources ( Mentor : Markus Makela ) *I've 3 requests/doubts : * 0.I've downloaded the MaxScale code from github <https://github.com/mariadb-corporation/MaxScale> and have read the documentation. It says the filter configuration file is : MaxScale.ini I didn't find this file in the codebase. Nor was I able to figure out the code-location related to the "filter" module. 1.Kindly suggest for the further things I should do/try. Shall I install MariaDB & MaxScale? What part of code-base should I look into & play around? 2.What is the UTC time when most of you guys are online on IRC-channel (#maria)? Since I'm from India, I need to know the time when most of you guys are online to get immediate help on IRC. Thanks in advance. -- Lokesh Walase

1 0

[Maria-developers] MDEV-6838: Using too big key for internal temp tables
by Vicențiu Ciorbaru 03 Mar '15

03 Mar '15

Hi Sergei and Sergey! I've implemented the fix for MDEV-6838. I'd like a review from both of you regarding the patch. There are a couple of changes made. The main issues are outlined in the commit message. I've changed the way create_key_part_by_field gets called in order to decouple the KEY_PART_INFO creation process from any KEYINFO modifications and also documented what the method does. My other concern is that create_key_part_by_field does not seem to make any sense within the table class as it's basically a constructor for KEY_PART_INFO and makes no use of table fields. I suggest we move the code as either a constructor for KEY_PART_INFO, or an init() function. I think that would be better left as a separate patch, but the method itself is very easy to factor out in case you agree with the change. I'm looking forward for your input. Regards, Vicențiu ---------- Forwarded message ---------- From: <vicentiu(a)mariadb.org> Date: 28 February 2015 at 23:58 Subject: 45b6edb: MDEV-6838: Using too big key for internal temp tables To: commits(a)mariadb.org revision-id: 45b6edb158f8101d641f550179ee15df363f686f parent(s): fa87fc733d7855e0e5f9b959ca0bddf772ca57e5 committer: Vicențiu Ciorbaru branch nick: server timestamp: 2015-02-28 23:58:05 +0200 message: MDEV-6838: Using too big key for internal temp tables This bug manifests due to wrong computation and evaluation of keyinfo->key_length. The issues were: * Using table->file->max_key_length() as an absolute value that must not be reached for a key, while it represents the maximum number of bytes possible for a table key. * Incorrectly computing the keyinfo->key_length size during KEY_PART_INFO creation. The metadata information regarding the key such the field length (for strings) was added twice. --- mysql-test/r/table_keyinfo-6838.result | 12 ++++++++++++ mysql-test/t/table_keyinfo-6838.test | 18 ++++++++++++++++++ sql/sql_select.cc | 7 ++++--- sql/structs.h | 1 + sql/table.cc | 34 ++++++++++++++++++++++++++-------- sql/table.h | 2 +- 6 files changed, 62 insertions(+), 12 deletions(-) diff --git a/mysql-test/r/table_keyinfo-6838.result b/mysql-test/r/table_keyinfo-6838.result new file mode 100644 index 0000000..55b0350 --- /dev/null +++ b/mysql-test/r/table_keyinfo-6838.result @@ -0,0 +1,12 @@ +CREATE TABLE t1 (i INT, state VARCHAR(997)) ENGINE=MyISAM; +INSERT INTO t1 VALUES (2,'Louisiana'),(9,'Maine'); +CREATE TABLE t2 (state VARCHAR(997), j INT) ENGINE=MyISAM; +INSERT INTO t2 VALUES ('Louisiana',9),('Alaska',5); +INSERT INTO t2 SELECT t2.* FROM t2 JOIN t2 AS t3 JOIN t2 AS t4 JOIN t2 AS t5 JOIN t2 AS t6; +SET @@max_heap_table_size= 16384; +set @@optimizer_switch='derived_merge=OFF'; +set @@optimizer_switch='extended_keys=ON'; +SELECT * FROM t1 AS t1_1 LEFT JOIN ( t1 AS t1_2 INNER JOIN (SELECT * FROM t2) v2 ON t1_2.i = j ) ON t1_1.state = v2.state LIMIT 1; +i state i state state j +2 Louisiana 9 Maine Louisiana 9 +DROP TABLE t1, t2; diff --git a/mysql-test/t/table_keyinfo-6838.test b/mysql-test/t/table_keyinfo-6838.test new file mode 100644 index 0000000..5f4c9f4 --- /dev/null +++ b/mysql-test/t/table_keyinfo-6838.test @@ -0,0 +1,18 @@ +# Test case for MDEV-6838. +# Due to incorrect key_length computation, this usecase failed with the error +# Using too big key for internal temp table. + +CREATE TABLE t1 (i INT, state VARCHAR(997)) ENGINE=MyISAM; +INSERT INTO t1 VALUES (2,'Louisiana'),(9,'Maine'); + +CREATE TABLE t2 (state VARCHAR(997), j INT) ENGINE=MyISAM; +INSERT INTO t2 VALUES ('Louisiana',9),('Alaska',5); +INSERT INTO t2 SELECT t2.* FROM t2 JOIN t2 AS t3 JOIN t2 AS t4 JOIN t2 AS t5 JOIN t2 AS t6; + +SET @@max_heap_table_size= 16384; +set @@optimizer_switch='derived_merge=OFF'; +set @@optimizer_switch='extended_keys=ON'; + +SELECT * FROM t1 AS t1_1 LEFT JOIN ( t1 AS t1_2 INNER JOIN (SELECT * FROM t2) v2 ON t1_2.i = j ) ON t1_1.state = v2.state LIMIT 1; + +DROP TABLE t1, t2; diff --git a/sql/sql_select.cc b/sql/sql_select.cc index 387fdb3..dabe46e 100644 --- a/sql/sql_select.cc +++ b/sql/sql_select.cc @@ -7957,7 +7957,8 @@ static bool create_hj_key_for_table(JOIN *join, JOIN_TAB *join_tab, { Field *field= table->field[keyuse->keypart]; uint fieldnr= keyuse->keypart+1; - table->create_key_part_by_field(keyinfo, key_part_info, field, fieldnr); + table->create_key_part_by_field(key_part_info, field, fieldnr); + keyinfo->key_length += key_part_info->store_length; key_part_info++; } } @@ -15921,7 +15922,7 @@ bool create_internal_tmp_table(TABLE *table, KEY *keyinfo, goto err; bzero(seg, sizeof(*seg) * keyinfo->key_parts); - if (keyinfo->key_length >= table->file->max_key_length() || + if (keyinfo->key_length > table->file->max_key_length() || keyinfo->key_parts > table->file->max_key_parts() || share->uniques) { @@ -16107,7 +16108,7 @@ bool create_internal_tmp_table(TABLE *table, KEY *keyinfo, goto err; bzero(seg, sizeof(*seg) * keyinfo->key_parts); - if (keyinfo->key_length >= table->file->max_key_length() || + if (keyinfo->key_length > table->file->max_key_length() || keyinfo->key_parts > table->file->max_key_parts() || share->uniques) { diff --git a/sql/structs.h b/sql/structs.h index ae71819..9840cec 100644 --- a/sql/structs.h +++ b/sql/structs.h @@ -76,6 +76,7 @@ typedef struct st_key_part_info { /* Info about a key part */ */ uint16 store_length; uint16 key_type; + /* Fieldnr begins counting from 1 */ uint16 fieldnr; /* Fieldnum in UNIREG */ uint16 key_part_flag; /* 0 or HA_REVERSE_SORT */ uint8 type; diff --git a/sql/table.cc b/sql/table.cc index cfa0b6c..7df2ae6 100644 --- a/sql/table.cc +++ b/sql/table.cc @@ -5968,18 +5968,32 @@ bool TABLE::alloc_keys(uint key_count) } -void TABLE::create_key_part_by_field(KEY *keyinfo, - KEY_PART_INFO *key_part_info, +/** + @brief + Populate a KEY_PART_INFO structure with the data related to a field entry. + + @param key_part_info The structure to fill. + @param field The field entry that represents the key part. + @param fleldnr The number of the field, count starting from 1. + + TODO: This method does not make use of any table specific fields. It + could be refactored to act as a constructor for KEY_PART_INFO instead. +*/ + +void TABLE::create_key_part_by_field(KEY_PART_INFO *key_part_info, Field *field, uint fieldnr) -{ +{ key_part_info->null_bit= field->null_bit; key_part_info->null_offset= (uint) (field->null_ptr - (uchar*) record[0]); key_part_info->field= field; key_part_info->fieldnr= fieldnr; key_part_info->offset= field->offset(record[0]); - key_part_info->length= (uint16) field->pack_length(); - keyinfo->key_length+= key_part_info->length; + /* + field->key_length() accounts for the raw length of the field, excluding + any metadata such as length of field or the NULL flag. + */ + key_part_info->length= (uint16) field->key_length(); key_part_info->key_part_flag= 0; /* TODO: The below method of computing the key format length of the @@ -5991,17 +6005,20 @@ void TABLE::create_key_part_by_field(KEY *keyinfo, */ key_part_info->store_length= key_part_info->length; + /* + The total store length of the key part is the raw length of the field + + any metadata information, such as its length for strings and/or the null + flag. + */ if (field->real_maybe_null()) { key_part_info->store_length+= HA_KEY_NULL_LENGTH; - keyinfo->key_length+= HA_KEY_NULL_LENGTH; } if (field->type() == MYSQL_TYPE_BLOB || field->type() == MYSQL_TYPE_GEOMETRY || field->real_type() == MYSQL_TYPE_VARCHAR) { key_part_info->store_length+= HA_KEY_BLOB_LENGTH; - keyinfo->key_length+= HA_KEY_BLOB_LENGTH; // ??? key_part_info->key_part_flag|= field->type() == MYSQL_TYPE_BLOB ? HA_BLOB_PART: HA_VAR_LENGTH_PART; } @@ -6124,7 +6141,8 @@ bool TABLE::add_tmp_key(uint key, uint key_parts, if (key_start) (*reg_field)->key_start.set_bit(key); (*reg_field)->part_of_key.set_bit(key); - create_key_part_by_field(keyinfo, key_part_info, *reg_field, fld_idx+1); + create_key_part_by_field(key_part_info, *reg_field, fld_idx+1); + keyinfo->key_length += key_part_info->store_length; (*reg_field)->flags|= PART_KEY_FLAG; key_start= FALSE; key_part_info++; diff --git a/sql/table.h b/sql/table.h index 7a1e380..8324454 100644 --- a/sql/table.h +++ b/sql/table.h @@ -1269,7 +1269,7 @@ struct TABLE bool add_tmp_key(uint key, uint key_parts, uint (*next_field_no) (uchar *), uchar *arg, bool unique); - void create_key_part_by_field(KEY *keyinfo, KEY_PART_INFO *key_part_info, + void create_key_part_by_field(KEY_PART_INFO *key_part_info, Field *field, uint fieldnr); void use_index(int key_to_save); void set_table_map(table_map map_arg, uint tablenr_arg)

2 1

[Maria-developers] MDEV-6089 review
by Sergei Golubchik 03 Mar '15

03 Mar '15

Hi. Here're I'll be asking questions about the patches I'm looking at. 1. Is this mostly a merge of WL#7305 or mostly your own implementation? That is, most of the changes and design decisions are yours or "because MySQL did it"? 2. Do you work like I do (commit, push, fix failing tests, combine commits with "git rebase -i" to get a clean history) or you've just magically got 32bit rdiffs correctly in every commit? :) 3. Why "MDL objects cache won't be needed"? The comment said "On some systems (e.g. Windows XP) constructing/destructing MDL_lock objects can be fairly expensive. We use this cache to avoid these costs..." Why is this no longer an issue? 4. MDL_lock_strategy - this object is in 5.7 too, but you've done it very differently. Why? And I wonder why didn't you move the common part of two strategies to the base class. You wanted a pure interface class? 5. Added initializer callback to lf-hash. May be you should remove this hack with decreasing LF_HASH::element_size etc and move this functionality to a custom initializer? It was added in 6ba12f. In particular, see changes to lf* files and whether they can use your initializer instead. 6. Replaced hash with lock-free hash: test result changes. Is the order of rows stable? perhaps you should use --sort_results ? in MDL_lock constructor, I'd set m_strategy to 0 (ori to 1 or, may be, keep it uninitialized) as a safety measure, to ensure it's not used before MDL_lock::lf_hash_initializer(). Otherwise looks ok. Mostly a straightforward change. Regards, Sergei

2 3

[Maria-developers] passwordless mariadb root login with auht_socket in Debian
by Otto Kekäläinen 03 Mar '15

03 Mar '15

Hello Daniel! It seems there are some issues with the passwordless login patch you sent me some weeks ago (https://github.com/ottok/mariadb-10.0/pull/3) Can you help me debug the failing postinst script? I added 'set -x' to the mariadb-server-10.0.postinst script and when it is run during the configure phase it will exit with code 1 and make dpkg abort the installation. Last lines of execution: SET sql_log_bin=0; CREATE TABLE IF NOT EXISTS plugin (name char(64) COLLATE utf8_bin NOT NULL DEFAULT '\'''\'', dl char(128) COLLATE utf8_bin NOT NULL DEFAULT '\'''\'', PRIMARY KEY (name)) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_bin COMMENT='\''MySQL plugins'\'';' + logger -p daemon.err -t mysqld_safe -i + /usr/sbin/mysqld --bootstrap --user=mysql --skip-grant-tables --default-storage-engine=myisam --plugin-load-add=auth_socket + set -e + echo 'SET sql_log_bin=0; USE mysql; DELIMITER //; CREATE PROCEDURE debian_plugin_install(IN plugin_name CHAR(50), IN soname CHAR(50)) BEGIN DECLARE CONTINUE HANDLER FOR NOT FOUND EXECUTE inst_plug; set @plugin_name=plugin_name; set @soname=soname ;set @install_plugin=CONCAT("INSTALL PLUGIN ",@plugin_name," SONAME '\''", @soname, "'\''");PREPARE inst_plug FROM @install_plugin ; select PLUGIN_NAME INTO @a from information_schema.plugins where PLUGIN_NAME=@plugin_name AND PLUGIN_STATUS='\''ACTIVE'\'' AND PLUGIN_TYPE='\''AUTHENTICATION'\'' AND PLUGIN_LIBRARY LIKE concat(@soname,'\''%'\'' ); DEALLOCATE PREPARE inst_plug; END// CALL debian_plugin_install('\''unix_socket'\'', '\''auth_socket'\'') // DROP PROCEDURE debian_plugin_install//' + logger -p daemon.err -t mysqld_safe -i + /usr/sbin/mysqld --bootstrap --user=mysql --skip-grant-tables --default-storage-engine=myisam --plugin-load-add=auth_socket timo@debunstable:~$ echo $? 1 timo@debunstable:~$ echo $? 0 This implies that the exit code from the line '/usr/sbin/mysqld --bootstrap --user=mysql --skip-grant-tables --default-storage-engine=myisam --plugin-load-add=auth_socket' is 1. Any ideas why? If you want to test it yourself, add to you apt sources the test repo deb http://labs.seravo.fi/~otto/mariadb-repo/ mariabd-10.0-sid-amd64/

4 15

[Maria-developers] 10.0 is now on github
by Sergei Golubchik 02 Mar '15

02 Mar '15

Hi, Olivier, Kentoku. Since when we've moved 10.1 development to git (about a year ago), the plan always was to move other trees to git too. So, last month we've moved all our development to git, 5.5 and 10.0 trees are now on github, in the https://github.com/MariaDB/server repository. I still keep launchpad trees, lp:maria/10.0, in particular, to be able to merge 10.0-connect, 10.0-spider, and 10.0-mroonga. When needed, I merge these trees to lp:maria/10.0, and then merge from lp:maria/10.0 to a 10.0 branch in the git repository. But it would be great if you could also move the development to git and github. The easiest and safest approach would be as follows: 1. you tell me when to do it. 2. I do the last merge of your lp tree to 10.0 and then to git and push it to github. 3. you fork https://github.com/MariaDB/server under your github account and then you can create your branch and do anything you want there. 4. tell me the url of your forked tree and I'll configure buildbot to build it. this way you don't need to export your changes from bzr to git (which takes few hours) - I'll do it (it'll be a couple of minutes for me, because I've already run git-remote-bzr many times, and it has all the data cached). And the history will be safer, as we can be sure that all already exported commits won't be re-exported the second time. So, just tell me when you're ready :) By the way, if you haven't pushed anything since 10.0.17 - that is, there's nothing to merge - you skip the first two steps and fork https://github.com/MariaDB/server right away. Good luck! Regards, Sergei

1 0

[Maria-developers] MDEV-7286 TRIGGER: CREATE OR REPLACE, CREATE IF NOT EXISTS
by Alexander Barkov 27 Feb '15

27 Feb '15

Hi Serg, Please review a patch for MDEV-7286. Thanks.

4 5

[Maria-developers] MDEV-7281 EVENT: CREATE OR REPLACE
by Alexander Barkov 26 Feb '15

26 Feb '15

Hi Sergei, Thanks for your suggestions that you gave in the first review. Please review a new patch for MDEV-7281, which addresses your suggestions. >> diff --git a/mysql-test/r/create_drop_event.result b/mysql-test/ /create_drop_event.result > > again, same thing. I don't see a test that the command actually worked. > please use different event definitions in all three create variants. > and add show create event after every create to verify that > create if not exists did *not* modify the event, but create or replace > did modify it. > Done. >>> --- a/sql/event_db_repository.cc >>> +++ b/sql/event_db_repository.cc >>> @@ -684,18 +684,28 @@ Event_db_repository::create_event(THD *thd, Event_parse_data *parse_data, >>> DBUG_PRINT("info", ("check existance of an event with the same name")); >>> if (!find_named_event(parse_data->dbname, parse_data->name, table)) >>> { >>> - if (create_if_not) >>> + if (thd->lex->is_create_or_replace()) >>> + { >>> + if ((ret= table->file->ha_delete_row(table->record[0]))) >>> + { >>> + table->file->print_error(ret, MYF(0)); >>> + goto end; >>> + } >>> + } >>> + else if (create_if_not) >>> { >>> *event_already_exists= true; >>> push_warning_printf(thd, Sql_condition::WARN_LEVEL_NOTE, >>> ER_EVENT_ALREADY_EXISTS, ER(ER_EVENT_ALREADY_EXISTS), >>> parse_data->name.str); >>> ret= 0; >>> + goto end; >>> } >>> else >>> + { >>> my_error(ER_EVENT_ALREADY_EXISTS, MYF(0), parse_data->name.str); >>> - >>> - goto end; >>> + goto end; >>> + } >>> } else >>> *event_already_exists= false; >> >> I wonder whether this works as it should. On create-or-replace you >> delete the old event row and a new one is supposedly inserted. So, the >> table is fine. >> >> But event_already_exists will stay unset, undefined, actually, because >> the caller didn't set it either. If you set it to TRUE, then the caller >> won't update in-memory even list, so your new event won't be enabled. >> If you set it to FALSE, it will try to insert a new event without >> deleting the old one and I don't know what will happen (either both will >> stay active or the insertion fails). >> event_already_exists is now set to "false" on OR REPLACE. The in-memory list is updated in events.cc, in the method Events::create_event(...). This is the new code for this: if (thd->lex->create_info.or_replace() && event_queue) event_queue->drop_event(thd, parse_data->dbname, parse_data->name); >> Do you have a test to verify that after CREATE OR REPLACE EVENT the old >> event no longer exists and the new event does exist and is executed as >> expected? >> Added a test for this. >>> diff --git a/sql/sql_parse.cc b/sql/sql_parse.cc >>> index 922f0d7..648fd48 100644 >>> --- a/sql/sql_parse.cc >>> +++ b/sql/sql_parse.cc >>> @@ -3995,9 +3997,8 @@ end_with_restore_list: >>> switch (lex->sql_command) { >>> case SQLCOM_CREATE_EVENT: >>> { >>> - bool if_not_exists= (lex->create_info.options & >>> - HA_LEX_CREATE_IF_NOT_EXISTS); >>> - res= Events::create_event(thd, lex->event_parse_data, if_not_exists); >>> + res= Events::create_event(thd, lex->event_parse_data, >>> + lex->is_create_if_not_exists()); >> >> This doesn't seem to be particularly logical. >> You pass lex->is_create_if_not_exists)() here an argument, >> but you check thd->lex->is_create_or_replace() in the >> Event_db_repository::create_event anyway. I'd suggest not to pass >> lex->is_create_if_not_exists() as an argument, and check both in >> Event_db_repository::create_event(). The "if not exists" flag is not passed to Events::create_event() any more. Thanks.

2 1

Re: [Maria-developers] 26f48fa: MDEV-6997: SET STATEMENT last_insert_id FOR ... does not affect the value written to the binlog
by Sergei Golubchik 25 Feb '15

25 Feb '15

Hi, Sanja! On Nov 28, sanja(a)mariadb.com wrote: > revision-id: 26f48fa6c110efc62e17e1edfe6569445387e11d > parent(s): d0d6284cab70af148269d95744f615454cac24bb > committer: Oleksandr Byelkin > branch nick: work-maria-10.1-MDEV-6997-real > timestamp: 2014-11-28 13:52:59 +0100 > message: > > MDEV-6997: SET STATEMENT last_insert_id FOR ... does not affect the value written to the binlog > > Problem was in reinitialization of first_successful_insert_id_in_prev_stmt_for_binlog after setting variables. > > --- > mysql-test/r/set_statement.result | 2 -- > mysql-test/r/set_statement_notembedded_binlog.result | 8 ++++++++ > mysql-test/t/set_statement.test | 2 -- > mysql-test/t/set_statement_notembedded_binlog.test | 15 +++++++++++++++ > sql/sql_parse.cc | 4 ++++ > sql/sys_vars.cc | 2 +- > 6 files changed, 28 insertions(+), 5 deletions(-) > > diff --git a/mysql-test/r/set_statement_notembedded_binlog.result b/mysql-test/r/set_statement_notembedded_binlog.result > new file mode 100644 > index 0000000..bd50832 > --- /dev/null > +++ b/mysql-test/r/set_statement_notembedded_binlog.result > @@ -0,0 +1,8 @@ > +drop table if exists t1; > +drop view if exists t1; > +PURGE BINARY LOGS before now(); why not reset master? > +set statement last_insert_id = 112 for create table t1 as select last_insert_id(); > +show binlog events limit 4,1; > +Log_name Pos Event_type Server_id End_log_pos Info > +x x x x x use `test`; DROP TABLE IF EXISTS `t1` /* generated by server */ Ehm, and what does this test shows? > +drop table t1; > diff --git a/sql/sql_parse.cc b/sql/sql_parse.cc > index 5af700b..54abfd0 100644 > --- a/sql/sql_parse.cc > +++ b/sql/sql_parse.cc > @@ -2728,6 +2728,10 @@ static bool do_execute_sp(THD *thd, sp_head *sp) > my_error(ER_WRONG_ARGUMENTS, MYF(0), "SET"); > goto error; > } > + if (!thd->in_sub_stmt) /* stored functions and triggers are a special case */ > + { > + thd->stmt_depends_on_first_successful_insert_id_in_prev_stmt= 0; > + } Hmm. What does this fix? I'd think that stmt_depends_on_first_successful_insert_id_in_prev_stmt is already 0 here. Anyway, without a working test it's difficult to see whether your fix is doing anything. > } Regards, Sergei

2 1

[Maria-developers] [MDEV-7586] Merged derived tables/VIEWs increment created_tmp_tables
by Vicențiu Ciorbaru 25 Feb '15

25 Feb '15

Hi Sergey! As discussed on the mdev page, I've implemented the fix for incrementing the created_tmp_tables variable. I've attached the patch for review. The changes in the patch are explained within the mdev page. commit 53d78644b793f7b667e71863ccd0b1ba54c894f3 Author: Vicențiu Ciorbaru <cvicentiu(a)gmail.com> Date: Sun Feb 22 03:40:17 2015 +0200 MDEV-7586: Merged derived tables/VIEWs increment created_tmp_tables Temporary table count fix. The number of temporary tables was increased when the table is not actually created. (when do_not_open was passed as TRUE to create_tmp_table). Another issue fixed is that create_tmp_table was called incorrectly in create_result_table, having keep_row_order passed for the do_not_open parameter and keep_row_order always set to false. diff --git a/mysql-test/r/tmp_table_count-7586.result b/mysql-test/r/tmp_table_count-7586.result new file mode 100644 index 0000000..0c526e0 --- /dev/null +++ b/mysql-test/r/tmp_table_count-7586.result @@ -0,0 +1,83 @@ +create table t2 (a int); +insert into t2 values (1),(2),(3); +create view v2 as select a from t2; +flush status; +select * from v2; +a +1 +2 +3 +show status like '%Created_tmp%'; +Variable_name Value +Created_tmp_disk_tables 0 +Created_tmp_files 0 +Created_tmp_tables 0 +explain select * from v2; +id select_type table type possible_keys key key_len ref rows Extra +1 SIMPLE t2 ALL NULL NULL NULL NULL 3 +select * from (select * from t2) T1; +a +1 +2 +3 +show status like '%Created_tmp%'; +Variable_name Value +Created_tmp_disk_tables 0 +Created_tmp_files 0 +Created_tmp_tables 0 +explain select * from (select * from t2) T1; +id select_type table type possible_keys key key_len ref rows Extra +1 SIMPLE t2 ALL NULL NULL NULL NULL 3 +drop view v2; +drop table t2; +CREATE TABLE t1(a int); +INSERT INTO t1 values(1),(2); +CREATE TABLE t2(a int); +INSERT INTO t2 values(1),(2); +EXPLAIN SELECT * FROM t1 WHERE a IN (SELECT * FROM t2 GROUP BY a HAVING a > 1); +id select_type table type possible_keys key key_len ref rows Extra +1 PRIMARY t1 ALL NULL NULL NULL NULL 2 Using where +1 PRIMARY <subquery2> eq_ref distinct_key distinct_key 4 test.t1.a 1 +2 MATERIALIZED t2 ALL NULL NULL NULL NULL 2 Using temporary +truncate table performance_schema.events_statements_history_long; +flush status; +CREATE TABLE t3 SELECT * FROM t1 WHERE a IN (SELECT * FROM t2 GROUP BY a HAVING a > 1); +# Performance schema should be the same as "Created_tmp_tables" variable below +select sum(created_tmp_tables) from performance_schema.events_statements_history_long; +sum(created_tmp_tables) +2 +show status like '%Created_tmp%'; +Variable_name Value +Created_tmp_disk_tables 0 +Created_tmp_files 0 +Created_tmp_tables 2 +drop table t3; +EXPLAIN SELECT * FROM t1 WHERE a IN (SELECT * FROM t2 GROUP BY a); +id select_type table type possible_keys key key_len ref rows Extra +1 PRIMARY t1 ALL NULL NULL NULL NULL 2 +1 PRIMARY <subquery2> eq_ref distinct_key distinct_key 4 func 1 +2 MATERIALIZED t2 ALL NULL NULL NULL NULL 2 +truncate table performance_schema.events_statements_history_long; +flush status; +CREATE TABLE t3 SELECT * FROM t1 WHERE a IN (SELECT * FROM t2 GROUP BY a); +# Performance schema should be the same as "Created_tmp_tables" variable below +select sum(created_tmp_tables) from performance_schema.events_statements_history_long; +sum(created_tmp_tables) +1 +show status like '%Created_tmp%'; +Variable_name Value +Created_tmp_disk_tables 0 +Created_tmp_files 0 +Created_tmp_tables 1 +drop table t1,t2,t3; +truncate table performance_schema.events_statements_history_long; +flush status; +# Performance schema should be the same as "Created_tmp_tables" variable below +select sum(created_tmp_tables) from performance_schema.events_statements_history_long; +sum(created_tmp_tables) +0 +show status like '%Created_tmp%'; +Variable_name Value +Created_tmp_disk_tables 0 +Created_tmp_files 0 +Created_tmp_tables 0 diff --git a/mysql-test/t/tmp_table_count-7586.test b/mysql-test/t/tmp_table_count-7586.test new file mode 100644 index 0000000..756913e --- /dev/null +++ b/mysql-test/t/tmp_table_count-7586.test @@ -0,0 +1,53 @@ +# MDEV-7586 regression test. +# Test if created_tmp_tables status variable is correctly incremented. +--source include/have_perfschema.inc + +create table t2 (a int); +insert into t2 values (1),(2),(3); +create view v2 as select a from t2; + +flush status; +select * from v2; +show status like '%Created_tmp%'; + +explain select * from v2; + +select * from (select * from t2) T1; +show status like '%Created_tmp%'; + +explain select * from (select * from t2) T1; + + +drop view v2; +drop table t2; + + +CREATE TABLE t1(a int); +INSERT INTO t1 values(1),(2); +CREATE TABLE t2(a int); +INSERT INTO t2 values(1),(2); + +EXPLAIN SELECT * FROM t1 WHERE a IN (SELECT * FROM t2 GROUP BY a HAVING a > 1); +truncate table performance_schema.events_statements_history_long; +flush status; +CREATE TABLE t3 SELECT * FROM t1 WHERE a IN (SELECT * FROM t2 GROUP BY a HAVING a > 1); +--echo # Performance schema should be the same as "Created_tmp_tables" variable below +select sum(created_tmp_tables) from performance_schema.events_statements_history_long; +show status like '%Created_tmp%'; +drop table t3; + +EXPLAIN SELECT * FROM t1 WHERE a IN (SELECT * FROM t2 GROUP BY a); +truncate table performance_schema.events_statements_history_long; +flush status; +CREATE TABLE t3 SELECT * FROM t1 WHERE a IN (SELECT * FROM t2 GROUP BY a); +--echo # Performance schema should be the same as "Created_tmp_tables" variable below +select sum(created_tmp_tables) from performance_schema.events_statements_history_long; +show status like '%Created_tmp%'; + +drop table t1,t2,t3; + +truncate table performance_schema.events_statements_history_long; +flush status; +--echo # Performance schema should be the same as "Created_tmp_tables" variable below +select sum(created_tmp_tables) from performance_schema.events_statements_history_long; +show status like '%Created_tmp%'; diff --git a/sql/opt_subselect.cc b/sql/opt_subselect.cc index d5e3334..2122349 100644 --- a/sql/opt_subselect.cc +++ b/sql/opt_subselect.cc @@ -3890,7 +3890,6 @@ SJ_TMP_TABLE::create_sj_weedout_tmp_table(THD *thd) /* STEP 1: Get temporary table name */ - thd->inc_status_created_tmp_tables(); if (use_temp_pool && !(test_flags & TEST_KEEP_TMP_TABLES)) temp_pool_slot = bitmap_lock_set_next(&temp_pool); diff --git a/sql/sql_class.cc b/sql/sql_class.cc index 9cdf5ce..4463eb7 100644 --- a/sql/sql_class.cc +++ b/sql/sql_class.cc @@ -3676,7 +3676,7 @@ create_result_table(THD *thd_arg, List<Item> *column_types, if (! (table= create_tmp_table(thd_arg, &tmp_table_param, *column_types, (ORDER*) 0, is_union_distinct, 1, options, HA_POS_ERROR, (char*) table_alias, - keep_row_order))) + !create_table, keep_row_order))) return TRUE; col_stat= (Column_statistics*) table->in_use->alloc(table->s->fields * diff --git a/sql/sql_select.cc b/sql/sql_select.cc index 4393f02..bb381ff 100644 --- a/sql/sql_select.cc +++ b/sql/sql_select.cc @@ -15802,7 +15802,6 @@ create_tmp_table(THD *thd, TMP_TABLE_PARAM *param, List<Item> &fields, (int) distinct, (int) save_sum_fields, (ulong) rows_limit, MY_TEST(group))); - thd->inc_status_created_tmp_tables(); thd->query_plan_flags|= QPLAN_TMP_TABLE; if (use_temp_pool && !(test_flags & TEST_KEEP_TMP_TABLES)) @@ -16756,14 +16755,19 @@ bool open_tmp_table(TABLE *table) HA_OPEN_TMP_TABLE | HA_OPEN_INTERNAL_TABLE))) { - table->file->print_error(error,MYF(0)); /* purecov: inspected */ - table->db_stat=0; - return(1); + table->file->print_error(error, MYF(0)); /* purecov: inspected */ + table->db_stat= 0; + return 1; } table->db_stat= HA_OPEN_KEYFILE+HA_OPEN_RNDFILE; - (void) table->file->extra(HA_EXTRA_QUICK); /* Faster */ - table->created= TRUE; - return(0); + (void) table->file->extra(HA_EXTRA_QUICK); /* Faster */ + if (!table->created) + { + table->created= TRUE; + table->in_use->inc_status_created_tmp_tables(); + } + + return 0; } @@ -16780,7 +16784,7 @@ bool open_tmp_table(TABLE *table) options Option bits DESCRIPTION - Create an internal emporary table according to passed description. The is + Create an internal temporary table according to passed description. The is assumed to have one unique index or constraint. The passed array or TMP_ENGINE_COLUMNDEF structures must have this form: @@ -16799,7 +16803,7 @@ bool open_tmp_table(TABLE *table) */ -bool create_internal_tmp_table(TABLE *table, KEY *keyinfo, +bool create_internal_tmp_table(TABLE *table, KEY *keyinfo, TMP_ENGINE_COLUMNDEF *start_recinfo, TMP_ENGINE_COLUMNDEF **recinfo, ulonglong options) @@ -16930,8 +16934,10 @@ bool create_internal_tmp_table(TABLE *table, KEY *keyinfo, goto err; } table->in_use->inc_status_created_tmp_disk_tables(); + table->in_use->inc_status_created_tmp_tables(); table->in_use->query_plan_flags|= QPLAN_TMP_DISK; share->db_record_offset= 1; + table->created= TRUE; DBUG_RETURN(0); err: DBUG_RETURN(1); @@ -16971,7 +16977,7 @@ bool create_internal_tmp_table(TABLE *table, KEY *keyinfo, /* Create internal MyISAM temporary table */ -bool create_internal_tmp_table(TABLE *table, KEY *keyinfo, +bool create_internal_tmp_table(TABLE *table, KEY *keyinfo, TMP_ENGINE_COLUMNDEF *start_recinfo, TMP_ENGINE_COLUMNDEF **recinfo, ulonglong options) @@ -17076,6 +17082,7 @@ bool create_internal_tmp_table(TABLE *table, KEY *keyinfo, goto err; } table->in_use->inc_status_created_tmp_disk_tables(); + table->in_use->inc_status_created_tmp_tables(); table->in_use->query_plan_flags|= QPLAN_TMP_DISK; share->db_record_offset= 1; table->created= TRUE; @@ -17109,7 +17116,7 @@ create_internal_tmp_table_from_heap(THD *thd, TABLE *table, if (is_duplicate) *is_duplicate= FALSE; - if (table->s->db_type() != heap_hton || + if (table->s->db_type() != heap_hton || error != HA_ERR_RECORD_FILE_FULL) { /* @@ -17139,8 +17146,8 @@ create_internal_tmp_table_from_heap(THD *thd, TABLE *table, new_table.no_rows= table->no_rows; if (create_internal_tmp_table(&new_table, table->key_info, start_recinfo, recinfo, - thd->lex->select_lex.options | - thd->variables.option_bits)) + thd->lex->select_lex.options | + thd->variables.option_bits)) goto err2; if (open_tmp_table(&new_table)) goto err1; @@ -17203,7 +17210,7 @@ create_internal_tmp_table_from_heap(THD *thd, TABLE *table, new_table.s= table->s; // Keep old share *table= new_table; *table->s= share; - + table->file->change_table_ptr(table, table->s); table->use_all_columns(); if (save_proc_info) diff --git a/sql/sql_select.h b/sql/sql_select.h index 7d53731..9cad225 100644 --- a/sql/sql_select.h +++ b/sql/sql_select.h @@ -1788,10 +1788,6 @@ Field *create_tmp_field(THD *thd, TABLE *table,Item *item, Item::Type type, bool table_cant_handle_bit_fields, bool make_copy_field, uint convert_blob_length); -bool create_internal_tmp_table(TABLE *table, KEY *keyinfo, - TMP_ENGINE_COLUMNDEF *start_recinfo, - TMP_ENGINE_COLUMNDEF **recinfo, - ulonglong options, my_bool big_tables); /* General routine to change field->ptr of a NULL-terminated array of Field @@ -1879,20 +1875,20 @@ void make_possible_keys_line(TABLE *table, key_map possible_keys, String *line); #define RATIO_TO_PACK_ROWS 2 #define MIN_STRING_LENGTH_TO_PACK_ROWS 10 -TABLE *create_tmp_table(THD *thd,TMP_TABLE_PARAM *param,List<Item> &fields, - ORDER *group, bool distinct, bool save_sum_fields, - ulonglong select_options, ha_rows rows_limit, - const char* alias, bool do_not_open=FALSE, +TABLE *create_tmp_table(THD *thd, TMP_TABLE_PARAM *param, List<Item> &fields, + ORDER *group, bool distinct, bool save_sum_fields, + ulonglong select_options, ha_rows rows_limit, + const char* alias, bool do_not_open=FALSE, bool keep_row_order= FALSE); void free_tmp_table(THD *thd, TABLE *entry); bool create_internal_tmp_table_from_heap(THD *thd, TABLE *table, TMP_ENGINE_COLUMNDEF *start_recinfo, - TMP_ENGINE_COLUMNDEF **recinfo, + TMP_ENGINE_COLUMNDEF **recinfo, int error, bool ignore_last_dupp_key_error, bool *is_duplicate); -bool create_internal_tmp_table(TABLE *table, KEY *keyinfo, +bool create_internal_tmp_table(TABLE *table, KEY *keyinfo, TMP_ENGINE_COLUMNDEF *start_recinfo, - TMP_ENGINE_COLUMNDEF **recinfo, + TMP_ENGINE_COLUMNDEF **recinfo, ulonglong options); bool open_tmp_table(TABLE *table); void setup_tmp_table_column_bitmaps(TABLE *table, uchar *bitmaps); Regards, Vicențiu

2 2

[Maria-developers] MDEV-7628 Row-based replication of old DECIMAL to new DECIMAL
by Alexander Barkov 25 Feb '15

25 Feb '15

Hi Sergei, Kristian, all, Thanks to everyone who participated in the discussion on this topic. I tried to collect all ideas into a JIRA task: https://mariadb.atlassian.net/browse/MDEV-7628 Please have a look. Thanks.

1 0

[Maria-developers] Contributing in Support for GTID in mysqlbinlog for GSoC'15
by Rohit Kashyap 24 Feb '15

24 Feb '15

HI Kristian, I am Rohit Kashyap, I am currently working on MDEV-7569 and recently came over to your post on Support for GTID in mysqlbinlog [MDEV-4989] I would like to ask you for guidance and leads on this so that I can take it up as my GSoC'15 Project under MariaDB. -- Kind Regards, Rohit

2 3

[Maria-developers] auth_socket/unix_socket auth plugin naming incompatibility with Percona Server
by John Leach 23 Feb '15

23 Feb '15

Hi, I just tested a switch from Percona Server 5.6 to MariaDB 10.0.16 and hit a problem with socket authentication. I had it set up to auth root using the auth_socket plugin, but when I switched to MariaDB I could no longer login as root. I get this error from the client: ERROR 1524 (HY000): Plugin 'auth_socket' is not loaded Anyway, after I bunch of investigation, I found that the plugin used to expose itself as socket_peercred but was renamed due to a conflict and Percona and MariaDB chose different names to fix it. Percona renamed it to auth_socket, matching the actual library filename auth_socket.so and MariaDB chose unix_socket, but keeping the filename auth_socket.so: https://github.com/MariaDB/server/commit/b8b7b4eb6bef26d05257272be02674ae55… https://github.com/percona/percona-server/commit/91a2afcbcba8af28d2f10f8083… This isn't great, but it's made worse because the plugin is successfully loaded in both cases (because the filename is auth_socket.so) and no errors are reported in the logs, but the plugin just exposes itself internally using a different name so reports that auth_socket isn't loaded in MariaDB. I fixed it by starting mysql with skip-grant-tables and running: update mysql.user set plugin='unix_socket' where plugin='auth_socket'; but that alone doesn't work, because MariaDB doesn't know that unix_socket is provided by auth_socket.so: ERROR 1524 (HY000): Plugin 'unix_socket' is not loaded So I then added this to the config file, which fixed it: plugin-load=unix_socket=auth_socket.so Any thoughts on what to do here? Can the plugin be made to recognise both names, so other users don't end up in this situation? I presume there would need to be both auth_socket.so and unix_socket.so. I also presume Percona would need to provide the same, to help those moving from MariaDB to Percona. Thanks, John. -- https://www.brightbox.com

2 4

[Maria-developers] (no subject)
by Rohit Kashyap 22 Feb '15

22 Feb '15

HI Kristian, I am Rohit Kashyap, I am currently working on MDEV-7569 and recently came over to your post on Support for GTID in mysqlbinlog [MDEV-4989] I would like to ask you for guidance and leads on this so that I can take it up as my GSoC'15 Project under MariaDB. -- Kind Regards, Rohit

1 0

[Maria-developers] Suggestion for MariaDB Workbench
by Rohit Kashyap 22 Feb '15

22 Feb '15

Hi, Based on the following posts http://stackoverflow.com/questions/22616861/can-i-use-mysql-workbench-to-cr… https://mariadb.com/kb/en/mariadb/guiworkbench-for-mariadb-aria/ https://mariadb.com/kb/en/mariadb/graphical-clients/ Can we build a Standalone GUI Client for MariaDB. -- Kind Regards, Rohit

1 0

[Maria-developers] Google Summer of Code 2015
by Delveri chick 22 Feb '15

22 Feb '15

Hi sergii i have some working knowledge of ldap not much though but I think following your post, it will be great for MariaDb to incorporate an ldap plugin that could be used by organisations running ldap servers and mariadb. For example most large organisations like Google keep information about their workers in a directory service on the servers. Creating users tables to hold this info will take too much time. Managing this table is also a huge task which can also reduce performance due to overpopulation. The ldap plugin will serve as an interface between mariadb and the directory service such that info can be retrieved with ease. The cool stuff with ldap is that if a request is made to the directory service and the requested data is not there, a reference to another directory service that contains it is returned. The Ldap plugin if incorporated into mariadb can be used to authenticate users since in runs directly on the TCT/IP stack. Please request for more details if you need more explanation that will convince mariadb developpers to include this task at GSoc 2015. Best regards Chick

1 0

[Maria-developers] binlog_row_image
by Rasmus Johansson 19 Feb '15

19 Feb '15

Hi, Is anyone familiar with binlog_row_image in MySQL 5.6, would it make sense to add it to MariaDB? I received a request for it where they have a wide table with high write activity and only a few fields are updated for each DML. Apparently binlog_row_image would be efficient in this case. There is also a JIRA issue about binlog_row_image from the MySQL compatibility point of view: https://mariadb.atlassian.net/browse/MDEV-7471 Rasmus -- Rasmus Johansson, VP Engineering MariaDB | t: +358 50 499 9589 | Skype: ratzpo

2 1

Re: [Maria-developers] e91559c: MDEV-7006 MDEV-7007: SET STATEMENT and slow log
by Sergei Golubchik 16 Feb '15

16 Feb '15

Hi, Oleksandr! On Jan 28, Oleksandr Byelkin wrote: > revision-id: e91559cf7f2b19378e9e5d3f72c3a6794a09d1e0 > parent(s): cbc318fcf3680c01dca1320b87f625124a497528 > committer: Oleksandr Byelkin > branch nick: work-maria-10.1-MDEV-6997 > timestamp: 2014-11-26 11:54:29 +0100 > message: > > MDEV-7006 MDEV-7007: SET STATEMENT and slow log > fixed embedded server tests > > old values of SET STATENENT variables now saved in its own Query_arena and restored later Basically, I have two comments: 1. I think it might be kind of expensive to create MEM_ROOT for every SET STATEMENT. And perhaps a user will never need to set-statement these log variables. I don't see a good solution for that. Either we can drop the support for log related variables in SET STATEMENT, basically not using your patch. I wouldn't like that, from a user point of view this is a pretty artificial limitation, he doesn't care about memroots. Or we can only create memroot for log related variables and put other variables in the statement memroot, which also means restoring values in two steps. Seems like too complex for a solution. A compromise solution would be to do almost like in your patch, but not to free memroot after every statement. That is, a memroot is created once, for the first SET STATEMENT. At the end of the statement you only MY_MARK_BLOCKS_FREE or MY_KEEP_PREALLOC. Then the next SET STATEMENT wouldn't need to allocate that much. Not perfect, but might do as a tradeoff. Or, perhaps you can suggest a better solution. 2. I didn't quite like that you've added restore_set_statement_var, reset_arena_for_set_stmt, free_arena_for_set_stmt, etc everywhere. I'd prefer you to add them to one function which is called from all these places. Like, there must be some cleanup-after-statement function that you should be able to use. > diff --git a/mysql-test/t/set_statement.test b/mysql-test/t/set_statement.test > index a658071..108c1a3 100644 > --- a/mysql-test/t/set_statement.test > +++ b/mysql-test/t/set_statement.test > @@ -995,6 +985,57 @@ set global general_log=0; > set statement lock_wait_timeout=1 for select @@lock_wait_timeout; > set global general_log=@save_general_log; > > +--echo # MDEV-7006 MDEV-7007: SET statement and slow log > + > +set @save_long_query_time= @@long_query_time; > +set @save_slow_query_log= @@slow_query_log; > +set @save_log_output= @@log_output; > + > +set statement long_query_time=default for select @@long_query_time; > +set statement log_slow_filter=default for select @@log_slow_filter; > +set statement log_slow_verbosity=default for select @@log_slow_verbosity; > +set statement log_slow_rate_limit=default for select @@log_slow_rate_limit; > +set statement slow_query_log=default for select @@slow_query_log; > + > +truncate table mysql.slow_log; > +set slow_query_log= 1; > +set global log_output='TABLE'; > + > +select sql_text from mysql.slow_log; > +set @@long_query_time=0.01; > +--echo #should be written > +select sleep(0.1); > +set @@long_query_time=@save_long_query_time; > +select sql_text from mysql.slow_log; > +--echo #--- > +--echo #should be written > +set statement long_query_time=0.01 for select sleep(0.1); > +select sql_text from mysql.slow_log; Okay, I hope it'll work, but please do try it in a separate branch in buildbot. all these timing-related tests have proven to be very fragile, so I've learned to be very cautious about these stuff. even in test that looks safe to me :) > +--echo #--- > +set @@long_query_time=0.01; > +--echo #should NOT be written > +set statement slow_query_log=0 for select sleep(0.1); > +set @@long_query_time=@save_long_query_time; > +select sql_text from mysql.slow_log; > +--echo #--- > +--echo #should be NOT written > +set statement long_query_time=0.01,log_slow_filter='full_scan' for select sleep(0.1); > +select sql_text from mysql.slow_log; > +--echo #--- > +--echo #should be NOT written > +set statement long_query_time=0.01,log_slow_rate_limit=9999 for select sleep(0.1); > +select sql_text from mysql.slow_log; > +--echo #--- > +# > +# log_slow_verbosity is impossible to check because results are not written > +# in TABLE mode > +# > + > +set global log_output= @save_log_output; > +set @@slow_query_log= @save_slow_query_log; > +set @@long_query_time= @save_long_query_time; > + > + > # > # Prohibited Variables > # > diff --git a/sql/sql_lex.cc b/sql/sql_lex.cc > index 6432e19..08094ce 100644 > --- a/sql/sql_lex.cc > +++ b/sql/sql_lex.cc > @@ -4235,6 +4236,53 @@ int LEX::print_explain(select_result_sink *output, uint8 explain_flags, > return res; > } > > +void LEX::set_arena_for_set_stmt(Query_arena *backup) > +{ > + DBUG_ENTER("LEX::set_arena_for_set_stmt"); > + DBUG_ASSERT(arena_for_set_stmt== 0); > + MEM_ROOT *mem_root= new MEM_ROOT(); > + if (!(mem_root)) > + DBUG_VOID_RETURN; > + init_sql_alloc(mem_root, ALLOC_ROOT_MIN_BLOCK_SIZE, 0, > + MYF(MY_THREAD_SPECIFIC)); > + if (!(arena_for_set_stmt= new Query_arena(mem_root, > + Query_arena::STMT_INITIALIZED))) > + DBUG_VOID_RETURN; 1. allocate a new memroot for every SET STATEMENT? 2. no error reporting? > + DBUG_PRINT("info", ("mem_root: 0x%lx arena: 0x%lx", > + (ulong) mem_root, (ulong) arena_for_set_stmt)); > + thd->set_n_backup_active_arena(arena_for_set_stmt, backup); > + DBUG_VOID_RETURN; > +} > + > + > +void LEX::reset_arena_for_set_stmt(Query_arena *backup) > +{ > + DBUG_ENTER("LEX::reset_arena_for_set_stmt"); > + DBUG_ASSERT(arena_for_set_stmt); > + thd->restore_active_arena(arena_for_set_stmt, backup); > + DBUG_PRINT("info", ("mem_root: 0x%lx arena: 0x%lx", > + (ulong) arena_for_set_stmt->mem_root, > + (ulong) arena_for_set_stmt)); > + DBUG_VOID_RETURN; > +} > + > + > +void LEX::free_arena_for_set_stmt() > +{ > + DBUG_ENTER("LEX::free_arena_for_set_stmt"); > + if (!arena_for_set_stmt) > + return; > + DBUG_PRINT("info", ("mem_root: 0x%lx arena: 0x%lx", > + (ulong) arena_for_set_stmt->mem_root, > + (ulong) arena_for_set_stmt)); > + arena_for_set_stmt->free_items(); > + free_root(arena_for_set_stmt->mem_root, MYF(0)); > + delete(arena_for_set_stmt->mem_root); > + delete(arena_for_set_stmt); Eh... The arena is allocated in the memroot. I don't think you can free the memroot before the arena. Probably something like this: MEM_ROOT *memroot= arena_for_set_stmt->mem_root; delete arena_for_set_stmt; free_root(memroot, MYF(0)); delete memroot; > + arena_for_set_stmt= 0; > + DBUG_VOID_RETURN; > +} > + > void LEX::restore_set_statement_var() > { > DBUG_ENTER("LEX::restore_set_statement_var"); > @@ -4243,7 +4291,9 @@ void LEX::restore_set_statement_var() > DBUG_PRINT("info", ("vars: %d", old_var_list.elements)); > sql_set_variables(thd, &old_var_list, false); > old_var_list.empty(); > + free_arena_for_set_stmt(); > } > + DBUG_ASSERT(!is_arena_for_set_stmt()); the indentation is broken like this in many parts of the patch. May be that's because you've forwarded it with your MUA... > DBUG_VOID_RETURN; > } > > diff --git a/sql/sql_parse.cc b/sql/sql_parse.cc > index 5af700b..ae831d0 100644 > --- a/sql/sql_parse.cc > +++ b/sql/sql_parse.cc > @@ -1956,6 +1958,7 @@ bool dispatch_command(enum enum_server_command command, THD *thd, > > /* Check that some variables are reset properly */ > DBUG_ASSERT(thd->abort_on_warning == 0); > + thd->lex->restore_set_statement_var(); too bad you had to put these restore_set_statement_var, reset_arena_for_set_stmt, free_arena_for_set_stmt, etc everywhere explicitly. > DBUG_RETURN(error); > } > Regards, Sergei

2 2

[Maria-developers] Fwd: Contributing in MariaDB Project
by Rohit Kashyap 13 Feb '15

13 Feb '15

Hi, I am Rohit from NIT Patna, India. A Engineering Student and contributor in MySQL Community, approved by Oracle. You guys are doing great things at Team Maria DB. Kudos for that. I would like to start and contribute in the Maria project, and would like to ask for guiding in the same, with code base and introductory Bugs to prove myself. -- Kind Regards, Rohit Freenode: rkgudboy -- Kind Regards, Rohit

2 5

Re: [Maria-developers] Row-based replication of old DECIMAL to new DECIMAL
by Sergei Golubchik 13 Feb '15

13 Feb '15

Hi, Alexander! On Dec 22, Alexander Barkov wrote: > Hi Sergei, Kristian, > > I'm looking at: > > MDEV-7268 Column of table cannot be converted from type 'decimal(0,?)' > to type ' 'decimal(10,7)' > > This bug cannot be fixed in general case, because the old DECIMAL data > type does not write its metadata into the binary log. > > I suggest we don't fix replication of old DECIMAL > (neither binlogging, nor slave-side). I think that's reasonable. But then CHECK TABLE ... FOR UPGRADE should issue a warning for old DECIMAL columns. > Instead, we fix "ALTER TABLE ... FORCE" (and thus mysql_upgrade) > to force changing old DECIMAL to new DECIMAL. > Currently, "ALTER TABLE...FORCE" preserve old DECIMAL. > > Not sure which version is best to fix in. > The complainer had problems with 5.5. I don't know, I'd say 10.0. For the complainer there's a workaround - I'm sure you can come up with an ALTER TABLE that changes columns from old to new DECIMAL, may be something like ALTER TABLE xxx MODIFY COLUMN yyy DECIMAL(....) Regards, Sergei

7 15

[Maria-developers] Crash during configure without development SSL libraries installed
by Vicențiu Ciorbaru 12 Feb '15

12 Feb '15

Hi Sergei! The following error occurs while trying to configure MariaDB 10.0 using the command "cmake ." without the development SSL libraries installed. I've attached the patch to fix the errors. ERROR LOG: ---------------------- CMake Error at cmake/ssl.cmake:162 (FILE): file STRINGS file "/home/vicentiu/Workspace/MariaDB/10.0/OPENSSL_INCLUDE_DIR-NOTFOUND/openssl/opensslv.h" cannot be read. Call Stack (most recent call first): CMakeLists.txt:356 (MYSQL_CHECK_SSL) Regards, Vicentiu

2 1

[Maria-developers] Fwd: MDEV-6838 Using too big key for internal temp tables
by Vicențiu Ciorbaru 12 Feb '15

12 Feb '15

+ CC maria-developers Hi Sergei! I have attached the diff for MDEV-6838. Let me know your thoughts on it. The test case now fails without the patch and passes with the patch on both 5.5 and 10.0. Sergey Petrunia is alright with the patch. Regards, Vicențiu

2 2

[Maria-developers] Interest in Contributing to MariaDB and participation in GSoC 2015
by Tekang Check 11 Feb '15

11 Feb '15

Hello, I am Tekang Mengwe, a third year Accounting student at the University of Buea. I am interested in contributing code to MariaDB and participating in the 2015 round of the Google Summer of Code. While looking at the projects. I'm interested in Database replication and would love to do some work in this respect. I am proficient in C/C++ and code as a hobby. I will greatly appreciate any assistance in this respect. Regards, Tekang

6 7

[Maria-developers] Debian and reproducible builds errors for MariaDB
by Otto Kekäläinen 06 Feb '15

06 Feb '15

Hello! Can somebody please help me with this, preferrably send me a patch I can add to the Debian MariaDB 10.0 packaging tree? Debian has this new test suite for reproducible builds: https://reproducible.debian.net/reproducible.html MariaDB has notes while MySQL is OK (no notes): https://reproducible.debian.net/index_notes.html MariaDB should be at least as good as MySQL, right? MariaDB notes: https://reproducible.debian.net/rb-pkg/mariadb-10.0.html The errors in the build log https://reproducible.debian.net/rbuild/mariadb-10.0_10.0.16-1.rbuild.log look like this: /tmp/buildd/mariadb-10.0-10.0.16/storage/connect/ha_connect.cc:174:46: error: macro "__DATE__" might prevent reproducible builds [-Werror=date-time] char compver[]= "Version 1.03.0006 " __DATE__ " " __TIME__; ^ /tmp/buildd/mariadb-10.0-10.0.16/storage/connect/ha_connect.cc:174:60: error: macro "__TIME__" might prevent reproducible builds [-Werror=date-time] char compver[]= "Version 1.03.0006 " __DATE__ " " __TIME__; ^ Could somebody look into this and send me a patch that fixes it? (and commit the same patch upstream, ofcourse) PS. While searching the log for errors I also noticed these unrelated ones, how do we fix them?: /tmp/buildd/mariadb-10.0-10.0.16/builddir /tmp/buildd/mariadb-10.0-10.0.16/builddir/storage/perfschema /tmp/buildd/mariadb-10.0-10.0.16/builddir/storage/perfschema/CMakeFiles/perfschema.dir/DependInfo.cmake --color= troff: fatal error: can't find macro file m /tmp/buildd/mariadb-10.0-10.0.16/builddir /tmp/buildd/mariadb-10.0-10.0.16/builddir/storage/federatedx /tmp/buildd/mariadb-10.0-10.0.16/builddir/storage/federatedx/CMakeFiles/federatedx.dir/DependInfo.cmake --color= troff: fatal error: can't find macro file m

2 1

[Maria-developers] storage engine index usage
by Andras Szabo 06 Feb '15

06 Feb '15

Hi Folks, A new question has risen during the development of our storage engine. In short the engine is providing a read-only view to the server filesystem. One can issue (implemented via assisted discovery): CREATE TABLE etc ENGINE=fsview ROOT=‘/etc’; Then it’s possible to tell: SELECT name, fsize FROM etc ORDER BY fsize DESC LIMIT 10 resulting in: +-----------------+--------+ | name | fsize | +-----------------+--------+ | ld.so.cache | 129417 | | mailcap | 44732 | | fb.modes | 24501 | | mime.types | 24066 | | brltty.conf | 22478 | | services | 19558 | | devscripts.conf | 18037 | | ltrace.conf | 14867 | | login.defs | 10551 | | sensors3.conf | 10344 | +-----------------+--------+ So far so good (the listing is not recursive). An additional feature is to access subdirectories of the root (this is working via automated discovery): SELECT name, fsize FROM `etc/init.d` This is not terribly wrong, but we’re thinking about the possiblity of improving it somehow that: SELECT name, fsize FROM etc WHERE path = “init.d”; However we’re in a bit of trouble how we could improve the storage engine to be able to handle this. Of course we would not like to do a "deep enumeration" of the root directory (full table scan) . So somehow we would like to access/use the information from the where clause. Can anybody recommend a way to solve that? -- Andras Szabo Sent with Airmail

3 2

Re: [Maria-developers] ping
by Sergei Golubchik 05 Feb '15

05 Feb '15

Hi, Vicențiu! Next time, please, cc: (or to: ) your commit to commits(a)mariadb.org. Or use the post-commit trigger from http://bazaar.launchpad.net/~maria-captains/mariadb-tools/trunk/files/head:… > commit 4ede7cdbe51332d1b07def8830974877ebcee936 > Author: Vicențiu Ciorbaru <cvicentiu(a)gmail.com> > Date: Wed Feb 4 23:08:05 2015 +0000 > > MDEV-6918 Create a way to see a user's default role. > > Added an extra column to i_s_applicable_roles, named IS_DEFAULT. > The column displays which role is the default role for the user > querying the table. Looks good, thanks! A couple of minor comments, see below. Ok to push after that. > diff --git a/mysql-test/suite/roles/i_s_applicable_roles_is_default.test b/mysql-test/suite/roles/i_s_applicable_roles_is_default.test > new file mode 100644 > index 0000000..1782f7a > --- /dev/null > +++ b/mysql-test/suite/roles/i_s_applicable_roles_is_default.test > @@ -0,0 +1,61 @@ I'd suggest to start the test with --enable_connect_log it'll make the result file a bit easier to follow. > +create user foo; > +create role role1; > +create role role2; > +create role role3; > + > diff --git a/sql/sql_acl.cc b/sql/sql_acl.cc > index 96fff6d..45ce43d 100644 > --- a/sql/sql_acl.cc > +++ b/sql/sql_acl.cc > @@ -10349,6 +10349,17 @@ applicable_roles_insert(ACL_USER_BASE *grantee, ACL_ROLE *role, void *ptr) > else > table->field[2]->store(STRING_WITH_LEN("NO"), cs); > > + /* Default role is only valid when looking at a role granted to a user. */ > + if (!is_role) > + { > + if (!is_role && data->user->default_rolename.length && you don't need a second !is_role under if(!is_role). I suppose that's a remnant of the first version of your patch, when you returned NO for roles granted to roles :) > + !strcmp(data->user->default_rolename.str, role->user.str)) > + table->field[3]->store(STRING_WITH_LEN("YES"), cs); > + else > + table->field[3]->store(STRING_WITH_LEN("NO"), cs); > + table->field[3]->set_notnull(); > + } > + > if (schema_table_store_record(table->in_use, table)) > return -1; > return 0; Regards, Sergei

1 0

[Maria-developers] Simplify and stabilize build tests in downstream distributions?
by Otto Kekäläinen 05 Feb '15

05 Feb '15

Hello Elena, I asked you on IRC in December about if the mtr run, as it is now called from debian/rules, is optimal and sensible for in-distro build time testing? Canonical staff did these changes to MySQL 5.6, do you think we should follow suite? https://github.com/basak/debian-mysql/commit/6eb941e0f6fc9de35468dc59605208… - see only line 142 https://github.com/basak/debian-mysql/commit/4710f5a3b429c640176fd45b7c4b70… https://github.com/basak/debian-mysql/commit/78d6a4ac03f2e4d1553cc35574fc78…

2 2

Re: [Maria-developers] [Commits] 7320cbf: MDEV-7177 - Server hangs on shutdown after InnoDB error (main.plugin_loaderr
by Sergei Golubchik 04 Feb '15

04 Feb '15

Hi, Sergey! On Feb 02, svoj(a)mariadb.org wrote: > revision-id: 7320cbf85ddfaf7ef76c8f03718819377fd14318 > parent(s): 11deb416fe12ebfd65a1f089c7eb6087c192e2dd > committer: Sergey Vojtovich > branch nick: 10.1 > timestamp: 2015-02-02 15:41:49 +0400 > message: > > MDEV-7177 - Server hangs on shutdown after InnoDB error (main.plugin_loaderr > fails in buildbot) > > There was a race condition in timer functionality of query timeouts. > > Race was as following: > main thread: init_thr_timers() > timer handler thread: my_thread_init() > main thread: end_thr_timer()/timer_thread_state= ABORTING > timer handler thread: timer_thread_state= RUNNING, continue normal op > main thread: waits indefinitely for timer handler thread to go down > > The original idea of the fix is to set RUNNING state in main thread, before > starting timer handler thread. But it didn't survive further cleanups: > - removed "timer_thread_state" and used "thr_timer_inited" for this purpose > - removed unused "timer_thread_running" > - removed code responisble for "timer handler thread" shutdown synchronization, > use pthread_join() instead. That looks ok to me. The only concern is - in end_thr_timer(), the old code had a timeout waiting for a timer thread to die. pthread_join() doesn't. I don't know if that's a problem, perhaps not. Regards, Sergei

2 1

Re: [Maria-developers] [Commits] 11deb41: MDEV-7499 - System variables have broken default values on big endian
by Sergei Golubchik 04 Feb '15

04 Feb '15

Hi, Sergey! On Jan 30, svoj(a)mariadb.org wrote: > revision-id: 11deb416fe12ebfd65a1f089c7eb6087c192e2dd > parent(s): edf34f38ac4fad7996bf19cd9ac669d2a6825400 > committer: Sergey Vojtovich > branch nick: 10.1 > timestamp: 2015-01-30 16:13:28 +0400 > message: > > MDEV-7499 - System variables have broken default values on big endian > > INFORMATION_SCHEMA.SYSTEM_VARIABLES.DEFAULT_VALUE had broken values on > big endian. > > Default value is internally stored as longlong, while I_S references it's > pointer (longlong *) according to variable type (e.g. int, my_bool, etc). This > works well on little endian, but on big endian we always get 0 for such > variables. Ok to push, thanks! One suggestion below > diff --git a/sql/sql_plugin.cc b/sql/sql_plugin.cc > index 46849e2..1928df5 100644 > --- a/sql/sql_plugin.cc > +++ b/sql/sql_plugin.cc > @@ -3254,7 +3254,29 @@ static int pluginvar_sysvar_flags(const st_mysql_sys_var *p) > uchar* sys_var_pluginvar::real_value_ptr(THD *thd, enum_var_type type) > { > if (type == OPT_DEFAULT) > - return (uchar*)&option.def_value; > + { > + switch (plugin_var->flags & PLUGIN_VAR_TYPEMASK) { > + case PLUGIN_VAR_BOOL: > + thd->sys_var_tmp.my_bool_value= option.def_value; > + return (uchar*) &thd->sys_var_tmp.my_bool_value; > + case PLUGIN_VAR_INT: > + thd->sys_var_tmp.int_value= option.def_value; > + return (uchar*) &thd->sys_var_tmp.int_value; > + case PLUGIN_VAR_LONG: > + case PLUGIN_VAR_ENUM: > + thd->sys_var_tmp.long_value= option.def_value; > + return (uchar*) &thd->sys_var_tmp.long_value; > + case PLUGIN_VAR_LONGLONG: > + case PLUGIN_VAR_SET: > + case PLUGIN_VAR_DOUBLE: > + return (uchar*) &option.def_value; PLUGIN_VAR_LONGLONG and PLUGIN_VAR_SET - ok, but for PLUGIN_VAR_DOUBLE I'd either copy it to thd->sys_var_tmp too or add an assert, like this case PLUGIN_VAR_DOUBLE: compile_time_assert(sizeof(double) == sizeof(option.def_value)); /* fall through */ case PLUGIN_VAR_LONGLONG: case PLUGIN_VAR_SET: return (uchar*) &option.def_value; More for documentation purposes than for protection agains a wrong sizeof(double). It explains why you don't copy the value. > + case PLUGIN_VAR_STR: > + thd->sys_var_tmp.ptr_value= (void*) option.def_value; > + return (uchar*) &thd->sys_var_tmp.ptr_value; > + default: > + DBUG_ASSERT(0); > + } > + } > Regards, Sergei

2 1

Re: [Maria-developers] [Commits] 101d7eb: MDEV-4987: Sort by domain_id when list of GTIDs are output
by Kristian Nielsen 03 Feb '15

03 Feb '15

nirbhay(a)mariadb.com writes: > revision-id: 101d7eb963816514362da8a98fc7db7135181910 > committer: Nirbhay Choubey > timestamp: 2015-01-31 21:48:14 -0500 > message: > > MDEV-4987: Sort by domain_id when list of GTIDs are output > > Added logic to sort gtid list based on domain_id before > populating them in string. Added a test case. Thanks! The patch looks ok to push after fixing / considering below in-line comments: > diff --git a/mysql-test/suite/rpl/t/rpl_gtid_sort.test b/mysql-test/suite/rpl/t/rpl_gtid_sort.test > new file mode 100644 > index 0000000..d3b8c6d > --- /dev/null > +++ b/mysql-test/suite/rpl/t/rpl_gtid_sort.test > +--connection server_2 > +SHOW VARIABLES LIKE '%GTID%'; > +#SET GLOBAL gtid_slave_pos=""; This commented-out SET looks like some left-over, probably you meant to remove it? > diff --git a/sql/rpl_gtid.cc b/sql/rpl_gtid.cc > index e5620ec..a0471f3 100644 > --- a/sql/rpl_gtid.cc > +++ b/sql/rpl_gtid.cc > @@ -247,11 +247,13 @@ > { > my_hash_init(&hash, &my_charset_bin, 32, offsetof(element, domain_id), > sizeof(uint32), NULL, rpl_slave_state_free_element, HASH_UNIQUE); > + my_init_dynamic_array(&gtid_cache, sizeof(rpl_gtid), 8, 8, MYF(0)); The name 'cache' confused me slightly, as I think of a cache as something that stores some state between invocations. But it is actually just a temporary buffer used for sorting. A suggestion for a better name could be 'gtid_sort_array'. > rpl_slave_state::~rpl_slave_state() > { > + delete_dynamic(&gtid_cache); > } The other parts of rpl_slave_state are freed in rpl_slave_state::deinit(). I think this is because there is a global variable of this class, and destructors in global variables can cause various issues. So better put this delete_dynamic() into deinit() as well (for consistency, if nothing else). > @@ -705,7 +707,13 @@ class Gtid_db_intact : public Table_check_intact > return sub_id; > } > > +/* A callback used in sorting of gtid list based on domain_id. */ > +static int rpl_gtid_cmp_cb(const void *id1, const void *id2) > +{ > + return (((rpl_gtid *)id1)->domain_id - ((rpl_gtid *)id2)->domain_id); > +} I think this is wrong, you can get signed/unsigned integer overflow (eg. it will consider 0xffffffff as being less than 0x00000000). We might be able to do the subtraction in longlong with a cast (though that would still be susceptible to overflow when converting to the int return value, right?), but it's probably clearer to just do: uint32 d1= ((rpl_gtid *)id1)->domain_id; uint32 d2= ((rpl_gtid *)id2)->domain_id; if (d1 < d2) return -1; else if (d1 > f2) return 1; else return 0; You should probably also modify the test case to have a domain id > 0x80000000, just to check. Otherwise looks good, thanks! - Kristian.

2 2

[Maria-developers] custom storage backend rnd_next() implementation and caching questions
by Andras Szabo 03 Feb '15

03 Feb '15

Hi Everyone, I am developing a custom mariadb storage backend which is intended to give filesystem related information via mariadb tables. For example one could tell: CREATE TEMPORARY TABLE IF NOT EXISTS `/var/log` ENGINE=fsview; which would create a table (using the assisted discovery method) with the following schema: MariaDB [test]> describe `/var`; +-----------+--------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +-----------+--------------+------+-----+---------+-------+ | file_name | varchar(255) | YES | | NULL | | | contents | longblob | YES | | NULL | | +-----------+--------------+------+-----+---------+———+ However i have some implementation issues: 1. I am having trouble implementing rnd_next() in the backend: int ha_fsview::rnd_next(uchar *buf) { DBUG_ENTER(__PRETTY_FUNCTION__); if (!dirp) { DBUG_RETURN(HA_ERR_INTERNAL_ERROR); } if (struct dirent* entry = readdir(dirp)) { uchar *pos = buf; *pos = 0; ++pos; // file name (1bytes size + str) *pos = strlen(entry->d_name); ++pos; size_t bytes_to_copy = strlen(entry->d_name); memcpy(pos, entry->d_name, bytes_to_copy); pos+=bytes_to_copy; DBUG_PRINT("info", ("fsview filename: %s", entry->d_name)); // content (4 bytes size + content ptr) static const std::string fake_content = "hello world"; int s = fake_content.size(); memcpy(pos, &s, sizeof(s)); pos+=sizeof(s); const char* c = fake_content.c_str(); memcpy(pos, &c, sizeof(c)); pos+=sizeof(c); DBUG_RETURN(0); } else { DBUG_RETURN(HA_ERR_END_OF_FILE); } } My problem is that filenames (first field) is ok, but i cannot really see the expected content “hello world” in the second field. I am assuming that i am doing something wrong, but i cannot really find the reason. Could somebody help me, please? 2. If i am issuing: SELECT * FROM `/var`; twice, then the server is not initiating a table scan again on my backend (most likely results from the first run are stored in some cache). So if filesystem contents are changed between the two statements, then it won’t be reflected in the resultset. Of course if i do a: SELECT SQL_NO_CACHE * FROM `/var`; then everything is fine, however i would not like to put the burden on the user to specify the option to not cache the results. My question is whether it’s possible to specify in the storage backend that the results from the tables should not be cached. Thank for the help in advance -- Andras Szabo Sent with Airmail

3 7

Re: [Maria-developers] [Commits] Rev 4577: MDEV-5719: Wrong result with GROUP BY and LEFT OUTER JOIN in lp:~maria-captains/maria/10.0
by Sergey Petrunia 29 Jan '15

29 Jan '15

Hi Sergei, == TL;DR == I have debugged that LEFT JOIN problem. The fix is not pretty, but correct. I will now push it into the 10.0 tree. == Details == So, the commit changes - return (table->null_row ? TRUE : - null_ptr ? MY_TEST(null_ptr[row_offset] & null_bit) : 0); + return real_maybe_null() ? + MY_TEST(null_ptr[row_offset] & null_bit) : table->null_row; The patch changes the behavior for NULLable fields. Suppose * real_maybe_null()= TRUE * table->null_row==TRUE // the table has a NULL-complemented row * (null_ptr[row_offset] & null_bit) == FALSE // however the value // is not null. the old code will return TRUE the new code will return FALSE The new code may seem wrong at first glance (it's a NULL-complemented row, why we return FALSE from is_null()?) However, if we look where table->null_row is set to true, we will find that it is done here: inline void mark_as_null_row(TABLE *table) { table->null_row=1; table->status|=STATUS_NULL_ROW; bfill(table->null_flags,table->s->null_bytes,255); } The function also sets all NULL-bits, so normally one can assume that (table->null_row=TRUE) => ((null_ptr[..] & null_bit) == TRUE) However, there is a case when the above does not hold. It happens when we run GROUP BY query, and GROUP BY is satisfied by scanning an index which produces rows in group order. In that case, the following can occur: consider the query from the testcase: SELECT ... FROM table_one a LEFT JOIN table_two b ON a.oid=b.oid GROUP BY a.oid * we read from a row (1,3) * we make index lookup in table b and find row (3). * row combination a(1,3)-b(3) passes the WHERE clause, GROUP BY execution code records it as current row. * there are no other matches in table b, so we get back to reading table a. * we read row a(1,4) * we look for matches in table b. there are no matches, so we produce a NULL-complemented row b(NULLs) * row combination a(1,4)-b(NULLs) is passed to the GROUP BY runtime. The runtime sees that we are now in a different group, and figures that it should pass the row combination representing the previous group into query output. * It attempts to pass the row combination a(1,3)-b(3) (*) into query output. However, the current row for table b is already the NULL-complemented row. the row combination (*) is stored in the "result_field" objects (and not in "regular field" objects which refer to table->record[0]). It seems wrong that we look at result_field (which points to a row that's different from table->record[0]), and at the same time we are looking at table->null_row, which refers to the record in table->record[0]. However, "result_field" storage has no place for table->null_row bit. The fix basically means: "for NULL-able fields, field's NULL-bit should override table->null_row". Considering the above, it is correct (albeit ugly). Now, what about GROUP BY and NOT NULL fields? I discovered the following: if a table has table->maybe_null, then Field objects that refer to such table and are pointed by Item_field::result_field, are always NULLable. (code-wise, check out setup_copy_fields(), these calls: item->result_field=field->new_field(thd->mem_root,field->table, 1); item->result_field->move_field(copy->to_ptr,copy->to_null_ptr,1); result_field object of NOT NULL column is null-able. On Fri, Jan 23, 2015 at 08:25:59AM -0800, Nirbhay Choubey wrote: > At lp:~maria-captains/maria/10.0 > > ------------------------------------------------------------ > revno: 4577 > revision-id: nirbhay(a)mariadb.com-20150123162551-h88uxn780ivj5k2n > parent: sergii(a)pisem.net-20150119153032-fqrmxjr13z0p1ly0 > committer: Nirbhay Choubey <nirbhay(a)mariadb.com> > branch nick: b5719-10.0 > timestamp: Fri 2015-01-23 11:25:51 -0500 > message: > MDEV-5719: Wrong result with GROUP BY and LEFT OUTER JOIN > > Merged revision 5224 from mysql-5.6 and added a test case. > .. > revno: 5224 > committer: Sergey Glukhov <sergey.glukhov(a)oracle.com> > branch nick: mysql-5.6 > timestamp: Wed 2013-06-19 14:24:08 +0400 > message: > Bug#16620047 INCORRECT QUERY RESULT (AFTER SERVER UPGRADE) > .. > === modified file 'mysql-test/r/group_by_innodb.result' > --- a/mysql-test/r/group_by_innodb.result 2014-01-26 20:49:11 +0000 > +++ b/mysql-test/r/group_by_innodb.result 2015-01-23 16:25:51 +0000 > @@ -57,3 +57,26 @@ > NULL 11.1,22.2 > DROP TABLE t1; > End of 5.5 tests > +# > +# MDEV-5719: Wrong result with GROUP BY and LEFT OUTER JOIN > +# > +CREATE TABLE table_one (oidGroup INT, oid INT PRIMARY KEY)ENGINE=INNODB; > +INSERT INTO table_one VALUES (1,1),(1,2),(1,3),(1,4); > +CREATE TABLE table_two (oid INT PRIMARY KEY)ENGINE=INNODB; > +INSERT INTO table_two VALUES (3); > +SELECT a.oidGroup, a.oid, b.oid FROM table_one a LEFT JOIN table_two b ON > +a.oid=b.oid WHERE a.oidGroup=1; > +oidGroup oid oid > +1 1 NULL > +1 2 NULL > +1 3 3 > +1 4 NULL > +SELECT a.oidGroup, a.oid, b.oid FROM table_one a LEFT JOIN table_two b ON > +a.oid=b.oid WHERE a.oidGroup=1 GROUP BY a.oid; > +oidGroup oid oid > +1 1 NULL > +1 2 NULL > +1 3 3 > +1 4 NULL > +DROP TABLE table_one, table_two; > +# End of tests > > === modified file 'mysql-test/t/group_by_innodb.test' > --- a/mysql-test/t/group_by_innodb.test 2014-01-26 20:49:11 +0000 > +++ b/mysql-test/t/group_by_innodb.test 2015-01-23 16:25:51 +0000 > @@ -67,3 +67,22 @@ > > --echo End of 5.5 tests > > +--echo # > +--echo # MDEV-5719: Wrong result with GROUP BY and LEFT OUTER JOIN > +--echo # > +CREATE TABLE table_one (oidGroup INT, oid INT PRIMARY KEY)ENGINE=INNODB; > +INSERT INTO table_one VALUES (1,1),(1,2),(1,3),(1,4); > + > +CREATE TABLE table_two (oid INT PRIMARY KEY)ENGINE=INNODB; > +INSERT INTO table_two VALUES (3); > + > +# Returns a value > +SELECT a.oidGroup, a.oid, b.oid FROM table_one a LEFT JOIN table_two b ON > +a.oid=b.oid WHERE a.oidGroup=1; > + > +SELECT a.oidGroup, a.oid, b.oid FROM table_one a LEFT JOIN table_two b ON > +a.oid=b.oid WHERE a.oidGroup=1 GROUP BY a.oid; > + > +DROP TABLE table_one, table_two; > + > +--echo # End of tests > > === modified file 'sql/field.h' > --- a/sql/field.h 2014-10-20 12:42:00 +0000 > +++ b/sql/field.h 2015-01-23 16:25:51 +0000 > @@ -656,21 +656,28 @@ > inline bool is_null(my_ptrdiff_t row_offset= 0) const > { > /* > + If the field is NULLable, it returns NULLity based > + on null_ptr[row_offset] value. Otherwise it returns > + NULL flag depending on TABLE::null_row value. > + > The table may have been marked as containing only NULL values > for all fields if it is a NULL-complemented row of an OUTER JOIN > or if the query is an implicitly grouped query (has aggregate > functions but no GROUP BY clause) with no qualifying rows. If > - this is the case (in which TABLE::null_row is true), the field > - is considered to be NULL. > + this is the case (in which TABLE::null_row is true) and the > + field is not nullable, the field is considered to be NULL. > + > + Do not change the order of testing. Fields may be associated > + with a TABLE object without being part of the current row. > + For NULL value check to work for these fields, they must > + have a valid null_ptr, and this pointer must be checked before > + TABLE::null_row. > + > Note that if a table->null_row is set then also all null_bits are > set for the row. > - > - Otherwise, if the field is NULLable, it has a valid null_ptr > - pointer, and its NULLity is recorded in the "null_bit" bit of > - null_ptr[row_offset]. > */ > - return (table->null_row ? TRUE : > - null_ptr ? MY_TEST(null_ptr[row_offset] & null_bit) : 0); > + return real_maybe_null() ? > + MY_TEST(null_ptr[row_offset] & null_bit) : table->null_row; > } > inline bool is_real_null(my_ptrdiff_t row_offset= 0) const > { return null_ptr ? (null_ptr[row_offset] & null_bit ? 1 : 0) : 0; } > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits -- BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

2 2

[Maria-developers] Dynamic columns support in libmaria client library?
by Peter Laursen 28 Jan '15

28 Jan '15

This page https://mariadb.com/kb/en/mariadb/documentation/nosql/dynamic-columns-api/ does not mention anything about it. it just says *The API is a part of libmysql C client library. In order to use it, one needs to include this header file* *#include <mysql/ma_dyncol.h>* *and link against libmysql.* Now can ma_dyncol.h be linked against libmaria as well? There are 2 questions actually 1) technical: will it work with libmaria? 2) legal: can a client program using libmaria under LGPL license include ma_dyncol.h? -- Peter -- Webyog

3 5

Re: [Maria-developers] [Commits] 51fa027: Improve performance for calculating memory allocation
by Sergei Golubchik 25 Jan '15

25 Jan '15

Hi, Monty! On Jan 23, Michael Widenius wrote: > revision-id: 51fa0272febd782674b3d44828718e805c44c1a8 > parent(s): ab440b0fb7302d707ba0ba41382bf911404db1cb > committer: Michael Widenius > branch nick: maria-10.1 > timestamp: 2015-01-23 14:32:53 +0200 > message: > > Improve performance for calculating memory allocation See my comments below: > diff --git a/include/mysql/plugin.h b/include/mysql/plugin.h > index 16b0c6e..9ae0fe8 100644 > --- a/include/mysql/plugin.h > +++ b/include/mysql/plugin.h > @@ -190,6 +196,7 @@ struct st_mysql_show_var { > > #define SHOW_VAR_FUNC_BUFF_SIZE (256 * sizeof(void*)) > typedef int (*mysql_show_var_func)(MYSQL_THD, struct st_mysql_show_var*, char *); > +typedef int (*mysql_show_var_extended_func)(MYSQL_THD, struct st_mysql_show_var*, enum enum_var_type, char *); I'd put scope the last making mysql_show_var_extended_func backward compatible with mysql_show_var_func. this makes some code simpler, see below. > > /* > diff --git a/sql/sql_show.cc b/sql/sql_show.cc > index 044b60d..8eed7b9 100644 > --- a/sql/sql_show.cc > +++ b/sql/sql_show.cc > @@ -3115,9 +3115,16 @@ static bool show_status_array(THD *thd, const char *wild, > if var->type is SHOW_FUNC or SHOW_SIMPLE_FUNC, call the function. > Repeat as necessary, if new var is again one of the above > */ > - for (var=variables; var->type == SHOW_FUNC || > - var->type == SHOW_SIMPLE_FUNC; var= &tmp) > - ((mysql_show_var_func)(var->value))(thd, &tmp, buff); > + for (var=variables; > + var->type == SHOW_FUNC || var->type == SHOW_SIMPLE_FUNC || > + var->type == SHOW_EXTENDED_FUNC; > + var= &tmp) > + { > + if (var->type == SHOW_EXTENDED_FUNC) > + ((mysql_show_var_extended_func)(var->value))(thd, &tmp, scope, buff); > + else > + ((mysql_show_var_func)(var->value))(thd, &tmp, buff); You won't need this if() when scope is the last argument. In fact I'm still thinking whether it would be a good idea to add scope to mysql_show_var_func and SHOW_FUNC, without introducing SHOW_EXTENDED_FUNC at all. This wouldn't break any existing plugin, as it's perfectly binary compatible extension. Also, see below. > + } > > SHOW_TYPE show_type=var->type; > if (show_type == SHOW_ARRAY) > diff --git a/sql/sql_class.cc b/sql/sql_class.cc > index 0d49407..d200af6 100644 > --- a/sql/sql_class.cc > +++ b/sql/sql_class.cc > @@ -1747,6 +1749,16 @@ void add_to_status(STATUS_VAR *to_var, STATUS_VAR *from_var) > to_var->binlog_bytes_written+= from_var->binlog_bytes_written; > to_var->cpu_time+= from_var->cpu_time; > to_var->busy_time+= from_var->busy_time; > + to_var->local_memory_used+= from_var->local_memory_used; why? it's not additive, right? and global_status_var.local_memory_used is never shown anyway. > + > + /* > + Update global_memory_used. We have to do this with atomic_add as the > + global value can change outside of LOCK_status. > + */ > + // workaround for gcc 4.2.4-1ubuntu4 -fPIE (from DEB_BUILD_HARDENING=1) > + int64 volatile * volatile ptr= &to_var->global_memory_used; I'd rather remove this workaround. 4.2.4 is ancient, and quite probably this bug doesn't happen with MY_MEMORY_ORDER_RELAXED at all > + my_atomic_add64_explicit(ptr, from_var->global_memory_used, > + MY_MEMORY_ORDER_RELAXED); > } > > /* > @@ -1784,6 +1796,11 @@ void add_diff_to_status(STATUS_VAR *to_var, STATUS_VAR *from_var, > dec_var->binlog_bytes_written; > to_var->cpu_time+= from_var->cpu_time - dec_var->cpu_time; > to_var->busy_time+= from_var->busy_time - dec_var->busy_time; > + > + /* > + We don't need to accumulate memory_used as these are not after > + this function is called. Sorry, I cannot parse that :( > + */ > } > > #define SECONDS_TO_WAIT_FOR_KILL 2 > diff --git a/sql/mysqld.cc b/sql/mysqld.cc > index ea53e47..c4e1dc3 100644 > --- a/sql/mysqld.cc > +++ b/sql/mysqld.cc > @@ -3972,19 +3972,24 @@ static void my_malloc_size_cb_func(long long size, my_bool is_thread_specific) > if (thd) > { > DBUG_PRINT("info", ("memory_used: %lld size: %lld", > - (longlong) thd->status_var.memory_used, size)); > + (longlong) thd->status_var.local_memory_used, > + size)); > - thd->status_var.memory_used+= size; > + thd->status_var.local_memory_used+= size; > - DBUG_ASSERT((longlong) thd->status_var.memory_used >= 0); > + DBUG_ASSERT((longlong) thd->status_var.local_memory_used >= 0); > } > } > } > + else if (likely(thd)) > + thd->status_var.global_memory_used+= size; Hmm, ok. So, you think that current_thd is cheaper than updating a global variable? > + else > + { > // workaround for gcc 4.2.4-1ubuntu4 -fPIE (from DEB_BUILD_HARDENING=1) > - int64 volatile * volatile ptr=&global_status_var.memory_used; > + int64 volatile * volatile ptr=&global_status_var.global_memory_used; I would've removed this workaround too > my_atomic_add64_explicit(ptr, size, MY_MEMORY_ORDER_RELAXED); > + } > } > } > > - > static int init_common_variables() > { > umask(((~my_umask) & 0666)); > @@ -8028,6 +8033,21 @@ static int show_default_keycache(THD *thd, SHOW_VAR *var, char *buff) > return 0; > } > > + > +static int show_memory_used(THD *thd, SHOW_VAR *var, enum enum_var_type scope, > + char *buff) > +{ > + var->type= SHOW_LONGLONG; > + var->value= buff; > + if (scope == OPT_GLOBAL) > + *(longlong*) buff= (global_status_var.local_memory_used + > + global_status_var.global_memory_used); > + else > + *(longlong*) buff= thd->status_var.local_memory_used; > + return 0; I'm surprised you haven't added SHOW_EXTENDED_SIMPLE_FUNC :) That would've make a very confusing name though. May be we should indeed add scope as the last argument to mysql_show_var_func and reuse both SHOW_SIMPLE_FUNC and SHOW_FUNC? I'm starting to think it's a good thing to do. Makes the API simpler and the number of choices to understand - smaller. If you do that - please don't forget to increment MARIA_PLUGIN_INTERFACE_VERSION. > +} > + > + > #ifndef DBUG_OFF > static int debug_status_func(THD *thd, SHOW_VAR *var, char *buff) > { Regards, Sergei

1 0

[Maria-developers] Additions to GIS in 10.1.
by Alexey Botchkov 23 Jan '15

23 Jan '15

Hi all. What's new about GIS in 10.1. Related tasks in JIRA are: MDEV-4045 Missing OGC Spatial functions. MDEV-60 Support for Spatial Reference systems for the GIS data. MDEV-12 OpenGIS: create required tables: GeometryColumns, related views. Speaking shortly, the MariaDB GIS part is now OpenGIS compliant, and passes all the OpenGIS conformance tests To get that the next things were done: Missing functions were added: IsRing(ln) The return type is Integer, with a return value of 1 for TRUE, 0 for FALSE, and –1 for UNKNOWN corresponding to a function invocation on NULL arguments; return TRUE if c is a ring, i.e., if c is closed and simple. A simple Curve does not pass through the same Point more than once. PointOnSurface(s) The return type is POINT, returns a Point guaranteed to lie on the Surface. arguments: POLYGON or MYLTIPOLYGON. NumInteriorRing(p) The return type is Integer, returns the number of interiorRings arguments: POLYGON. Relate(g1 Geometry, g2 Geometry, patternMatrix String) The return type is Integer, with a return value of 1 for TRUE, 0 for FALSE, and –1 for UNKNOWN corresponding to a function invocation on NULL arguments; returns TRUE if the spatial relationship specified by the pattern Matrix holds. ConvexHull(g1 Geometry) The return is Geometry, returna a geometric object that is the convex hull of g1 arguments: GEOMETRY. The information about the GEOMETRY column's reference system (SRID) is now stored in the field's metadata. Syntax of the CREATE and ALTER was modified to specify the SRID for the geometry columns. With this information some views required by OpenGIS, were added: GEOMETRY_COLUMNS, SPATIAL_REF_SYS: These are links to the INFORMATION_SCHEMA.GEOMETRY_COLUMNS and INFORMATION_SCHEMA.SPATIAL_REF_SYS respectively. INFORMATION_SCHEMA.GEOMETRY_COLUMNS: describes the available feature tables and their Geometry properties. Fields: F_TABLE_CATALOG CHARACTER VARYING(256) NOT NULL, F_TABLE_SCHEMA CHARACTER VARYING(256) NOT NULL, F_TABLE_NAME CHARACTER VARYING(256) NOT NULL, the above 3 fields are the fully qualified name of the feature table containing the geometry column; F_GEOMETRY_COLUMN CHARACTER VARYING(256) NOT NULL, the name of the column in the feature table that is the Geometry Column. G_TABLE_CATALOG CHARACTER VARYING(256) NOT NULL, G_TABLE_SCHEMA CHARACTER VARYING(256) NOT NULL, G_TABLE_NAME CHARACTER VARYING(256) NOT NULL, the above 3 fields are the name of the geometry table and its schema and catalog. The geometry table implements the geometry column; STORAGE_TYPE INTEGER, always 1 in MariaDB (means binary geometry implementation) GEOMETRY_TYPE INTEGER, the type of geometry values stored in this column. 0 = GEOMETRY 1 = POINT 3 = LINESTRING 5 = POLYGON 7 = MULTIPOINT 9 = MULTILINESTRING 11 = MULTIPOLYGON COORD_DIMENSION INTEGER, the number of dimensions in the spatial reference system. Always 2 in MariaDB. MAX_PPR INTEGER, Always 0 in MariaDB. SRID INTEGER the ID of the Spatial Reference System used for the coordinate geometry in this table. It is a foreign key reference to the SPATIAL_REF_SYS table. INFORMATION_SCHEMA.SPATIAL_REF_SYS: stores information on each spatial reference system used in the database. Fields: SRID INTEGER NOT NULL PRIMARY KEY, an integer value that uniquely identifies each Spatial Reference System within a database. AUTH_NAME CHARACTER VARYING(256), the name of the standard or standards body that is being cited for this reference system. In most cases the value is "EPSG". AUTH_SRID INTEGER TEXT the Well-known Text Representation of the Spatial Reference System. Required stored procedures added: AddGeometryColumn(FEATURE_TABLE_CATALOG, FEATURE_TABLE_SCHEMA, FEATURE_TABLE_NAME, GEOMETRY_COLUMN_NAME, SRID) adds new column of spatial type to the specified table DropGeometryColumn(FEATURE_TABLE_CATALOG, FEATURE_TABLE_SCHEMA, FEATURE_TABLE_NAME, GEOMETRY_COLUMN_NAME) removes the spatial column from the table. That's it for now. Best regards. HF

1 0

[Maria-developers] Renaming Virtual Columns to Computed Columns
by Ian Gilfillan 19 Jan '15

19 Jan '15

There was a suggestion on the docs mailing list that we rename Virtual Columns to Computed Columns. I think it makes sense, but before we implement I want to make sure everyone is happy with the change, so am repeating it here as not everyone follows docs. Currently, we use the term Virtual Columns (see https://mariadb.com/kb/en/mariadb/documentation/sql-structure-and-commands/…) Oracle uses this term, while MS SQL and Firebird use the term Computed Columns to describe similar functionality. MariaDB's virtual column implementation has both persistent and virtual attributes. Talking of "virtual virtual" and "persistent virtual" can be confusing. Oracle's implementation, to my knowledge, only has virtual virtual, while MS SQL's Computed Columns offers both persistent ("persisted" in their case) and virtual. I think "Computed" is a slightly more descriptive term, and avoids the ambiguity of "persistent virtual" and "virtual virtual". The MariaDB error messages already use the term computed, for example "Constant expression in computed column function is not allowed". Unless anyone has a good reason to prefer the term virtual, we're going to rather use the term "computed column" in the docs. The article will still mention virtual column as an alternative etc, but computed will be the preferred term. ian

3 3

Re: [Maria-developers] [Commits] Rev 4562: MDEV-7402 'reset master' hangs, waits for signalled COND_xid_list in lp:~maria-captains/maria/10.0
by Kristian Nielsen 19 Jan '15

19 Jan '15

Sergei Golubchik <serg(a)mariadb.org> writes: > ------------------------------------------------------------ > revno: 4562 > revision-id: sergii(a)pisem.net-20150118130747-rs67yzzk8a7ebwdd > parent: sergii(a)pisem.net-20150118002113-snztyzwvzc500c2m > fixes bug: https://mariadb.atlassian.net/browse/MDEV-7402 > committer: Sergei Golubchik <sergii(a)pisem.net> > branch nick: 10.0 > timestamp: Sun 2015-01-18 14:07:47 +0100 > message: > MDEV-7402 'reset master' hangs, waits for signalled COND_xid_list > > Using a boolean flag for 'there is a RESET MASTER in progress' doesn't > work very well for multiple concurrent RESET MASTER statements. Ok to push with a testcase like the below, as we discussed. - Kristian. ----------------------------------------------------------------------- --echo *** MDEV-7402: 'reset master' hangs, waits for signalled COND_xid_list *** --source include/wait_for_binlog_checkpoint.inc connect(con3,localhost,root,,); # Make the binlog background thread wait before clearing the pending checkpoint. # The bug was that one RESET MASTER would clear the reset_master_pending # flag set by another RESET MASTER; this could cause the wakeup from the # binlog background thread not to be sent, and thus the second RESET MASTER # to wait infinitely. SET debug_sync="reset_logs_after_set_reset_master_pending SIGNAL reset_master_ready WAIT_FOR reset_master_cont"; send RESET MASTER; --connection default SET @old_dbug= @@global.DEBUG_DBUG; SET GLOBAL debug_dbug="+d,inject_binlog_background_thread_before_mark_xid_done"; SET debug_sync="now WAIT_FOR reset_master_ready"; RESET MASTER; SET debug_sync="now WAIT_FOR injected_binlog_background_thread"; SET GLOBAL debug_dbug=@old_dbug; SET debug_sync="now SIGNAL reset_master_cont"; --connection con3 REAP; --connection default SET debug_sync = 'reset';

1 0

Re: [Maria-developers] [MDEV-7362] ANALYZE TABLES crash with table-independent-statistics gathering
by Sergei Golubchik 17 Jan '15

17 Jan '15

Hi, Vicențiu! On Jan 17, Vicențiu Ciorbaru wrote: > > Here's the new diff, let me know if you have any more comments. Shall I > push this as is? Yes, just one small detail (file name and the comment), see below. Then please push, no need to ask for another review :) But... 1. Please don't paste patches into the email anymore. At least not with your mail client that wraps long lines around and sends everything as html. See https://mariadb.com/kb/en/using-git/#commit-emails 2. Why git? This is the patch for 10.0 bug, at the moment 10.0 is still in bzr. 3. This looks like a reverse diff, how did you manage to do that? :) > diff --git a/mysql-test/t/statistics_index_crash.test b/mysql-test/t/statistics_index_crash.test > deleted file mode 100644 > index dbbdca3..0000000 > --- a/mysql-test/t/statistics_index_crash.test if you create a new test file for one bug only (as opposite to adding to an existing test file), mention the bug number in the file name. Like statistics_index_crash-7362.test > +++ /dev/null > @@ -1,26 +0,0 @@ > ---source include/have_stat_tables.inc > ---source include/have_innodb.inc Start a test case for a bug with a comment that says the bug number, the synopsys, and optionally more information, if needed. The bare minimum looks like # # MDEV-7362 ANALYZE TABLES crash with table-independent-statistics gathering # MDEV-7380 engine-independent stats SEGV on ANALYZE TABLE (#2) # > - > -CREATE TABLE t1 (a longtext, FULLTEXT KEY (`a`)) ENGINE=InnoDB; > -INSERT INTO t1 VALUES (unhex('3E0D0A4141414142334E7A6143317963324541414141424977414141674541726D')); > -ANALYZE TABLE t1 PERSISTENT FOR ALL; > ---sorted_result > -SELECT * FROM mysql.index_stats WHERE index_name='a' AND table_name='t1'; > -DROP TABLE t1; > - > -CREATE TABLE t1 (a longtext, FULLTEXT KEY (`a`)) ENGINE=MyISAM; > -INSERT INTO t1 VALUES (unhex('3E0D0A4141414142334E7A6143317963324541414141424977414141674541726D')); > -ANALYZE TABLE t1 PERSISTENT FOR ALL; > ---sorted_result > -SELECT * FROM mysql.index_stats WHERE index_name='a' AND table_name='t1'; > -DROP TABLE t1; > - > -CREATE TABLE geom (g GEOMETRY NOT NULL, SPATIAL INDEX(g)) ENGINE=MyISAM; > -INSERT INTO geom VALUES > - (MultiPolygonFromText('MULTIPOLYGON(((28 26,28 0,84 0,84 42,28 26),(52 18,66 23,73 9,48 6,52 18)),((59 18,67 18,67 13,59 13,59 18)))')), > - (MPolyFromText('MULTIPOLYGON(((28 26,28 0,84 0,84 42,28 26),(52 18,66 23,73 9,48 6,52 18)),((59 18,67 18,67 13,59 13,59 18)))')), > - (MPolyFromWKB(AsWKB(MultiPolygon(Polygon(LineString(Point(0, 3), Point(3, 3), Point(3, 0), Point(0, 3))))))); > -ANALYZE TABLE geom PERSISTENT FOR ALL; > ---sorted_result > -SELECT * FROM mysql.index_stats WHERE index_name='g' AND table_name='geom'; > -DROP TABLE geom; Regards, Sergei

1 0

[Maria-developers] [MDEV-7362] ANALYZE TABLES crash with table-independent-statistics gathering
by Vicențiu Ciorbaru 17 Jan '15

17 Jan '15

Hi Sergei! I've implemented a fix for the bug described in MDEV-7362, that also fixes MDEV-7380. The test case also covers MDEV-7380. With the following fix,the I've spent a bit of time trying to figure out the architecture of things there. I am not confident my fix is complete, or if this is the way we want to go about fixing it. The fix does what we discussed over IRC, that we are not interested in computing statistics for a FULLTEXT index. What I think would be the root cause of the problem is calling ha_index_init() on an index that does not support this. By calling table->file->ha_index_init(), the table->field array gets corrupted and that ultimately leads to a crash in the function using it afterwards. Then again, this module is completely new to me, so any comments are appreciated. Here is the diff of the first changes. diff --git a/mysql-test/t/mdev-7362.test b/mysql-test/t/mdev-7362.test new file mode 100644 index 0000000..6551893 --- /dev/null +++ b/mysql-test/t/mdev-7362.test @@ -0,0 +1,14 @@ +CREATE TABLE t1 (a longtext, FULLTEXT KEY (`a`)) ENGINE=MyISAM; +insert into t1 values (unhex('3E0D0A4141414142334E7A6143317963324541414141424977414141674541726D')); +analyze table t1 persistent for all; +drop table t1; + +CREATE TABLE t1 (a longtext, FULLTEXT KEY (`a`)) ENGINE=MyISAM; +insert into t1 values (unhex('3E0D0A4141414142334E7A6143317963324541414141424977414141674541')); +analyze table t1 persistent for all; +drop table t1; + +CREATE TABLE t1 (f longtext NOT NULL, FULLTEXT KEY f (f)) ENGINE=MyISAM; +INSERT INTO t1 VALUES (REPEAT('a',27)),('foo'); +ANALYZE TABLE t1 PERSISTENT FOR ALL; +drop table t1; diff --git a/sql/sql_statistics.cc b/sql/sql_statistics.cc index d368145..2c1cccf 100644 --- a/sql/sql_statistics.cc +++ b/sql/sql_statistics.cc @@ -2367,6 +2367,9 @@ int collect_statistics_for_index(THD *thd, TABLE *table, uint index) DBUG_RETURN(rc); } + if (key_info->flags & HA_FULLTEXT) // skip if FULLTEXT index + DBUG_RETURN(rc); + table->key_read= 1; table->file->extra(HA_EXTRA_KEYREAD); Regards, Vicențiu

3 5

[Maria-developers] MDEV-7440 Build fails in libgroonga on Solaris
by Alexander Barkov 16 Jan '15

16 Jan '15

Hello Kentoku, Can you please have a look into this problem? https://mariadb.atlassian.net/browse/MDEV-7440 Can you fix please? Thanks.

1 0

Re: [Maria-developers] [Commits] Rev 4398: MDEV-7368 : SLES: Failed to start mysql.service: Unit
by Sergei Golubchik 15 Jan '15

15 Jan '15

Hi, Nirbhay! On Dec 23, Nirbhay Choubey wrote: > At lp:~maria-captains/maria/5.5 > > ------------------------------------------------------------ > revno: 4398 > revision-id: nirbhay(a)mariadb.com-20141224033207-8lh5i8d84qsjey79 > parent: elenst(a)montyprogram.com-20141223172123-pkfhjnv7oy1suxpa > committer: Nirbhay Choubey <nirbhay(a)mariadb.com> > branch nick: b7368-5.5 > timestamp: Tue 2014-12-23 22:32:07 -0500 > message: > MDEV-7368 : SLES: Failed to start mysql.service: Unit > mysql.service failed to load > > Added 'systemctl daemon-reload' command in postin and > postun rpm scripts. > === modified file 'support-files/rpm/server-postin.sh' > --- a/support-files/rpm/server-postin.sh 2013-11-20 11:18:46 +0000 > +++ b/support-files/rpm/server-postin.sh 2014-12-24 03:32:07 +0000 > @@ -1,6 +1,10 @@ > > # Make MySQL start/shutdown automatically when the machine does it. > if [ $1 = 1 ] ; then > + if [ -x /usr/bin/systemctl ] ; then > + /usr/bin/systemctl daemon-reload >/dev/null 2>&1 > + fi > + looks ok > if [ -x /sbin/chkconfig ] ; then > /sbin/chkconfig --add mysql > fi > @@ -51,30 +55,31 @@ > SETARGETDIR=/etc/selinux/targeted/src/policy > SEDOMPROG=$SETARGETDIR/domains/program > SECONPROG=$SETARGETDIR/file_contexts/program > +if [ -f /etc/redhat-release ] ; then > + if grep -q "Red Hat Enterprise Linux .. release 4" /etc/redhat-release \ > + || grep -q "CentOS release 4" /etc/redhat-release ; then I'd rather written if grep -Eqs '(Red Hat Enterprise Linux ..|CentOS) release 4' /etc/redhat-release; then or (which is more portable, but perhaps unnecessary so for an rpm postin script) if grep '$Red Hat Enterprise Linux ..\|CentOS$ release 4' /etc/redhat-release >/dev/null 2>&1; then but if you'd like you can push your variant. Regards, Sergei

2 1

Re: [Maria-developers] [Commits] Rev 4401: MDEV-7448 - mtr may leave stale mysqld in lp:maria/5.5
by Sergei Golubchik 14 Jan '15

14 Jan '15

Hi, Sergey! On Jan 14, Sergey Vojtovich wrote: > ------------------------------------------------------------ > revno: 4401 > revision-id: svoj(a)mariadb.org-20150114135038-jss2vvpkhihysfmj > parent: sergii(a)pisem.net-20150110130746-xlrofqddt4j8xkw6 > committer: Sergey Vojtovich <svoj(a)mariadb.org> > branch nick: 5.5 > timestamp: Wed 2015-01-14 17:50:38 +0400 > message: > MDEV-7448 - mtr may leave stale mysqld > > Let mtr handle SIGHUP the same way as SIGINT. This solves stale mysqld processes > left after broken/closed ssh connection. Looks ok. But please make sure that all tests pass before pushing it. mysqld uses SIGHUP to print various information to the error log. If there's a test that actually tests that by sending SIGHUP - your change might've broken it. Regards, Sergei

1 0

[Maria-developers] MySQL bug #68557 "sjis changes in 5.5 break date parsing" fixed in MariaDB >= 5.5.38
by Hartmut Holzgraefe 13 Jan '15

13 Jan '15

Any idea how that happened, as I can't find anything that seems to be related in the 5.5.38 changelog? http://bugs.mysql.com/bug.php?id=68557 -- Hartmut Holzgraefe, Principal Support Engineer (EMEA) MariaDB Corporation | http://www.mariadb.com/

2 1

[Maria-developers] Buildbot slow upload
by Kristian Nielsen 13 Jan '15

13 Jan '15

Hi Serg, Daniel, Some time ago there was discussion of slow Buildbot upload speed (sorry, don't have the original mail thread to reply to). I found some new information on this. Yesterday I had to do a custom build, but was not able to because my tarbake upload kept getting stuck. So I had to investigate. Turns out the real problem was that hasky:/media/backup/archive/pack was full. And apparently, the error handling in Buildbot upload step is broken, so the error is not caught properly but just causes the upload to hang. I have noticed this before, that upload gets stuck. I wonder if full disk on hasky has caused this before. Daniel: Do we have something that removes old builds from /archive/pack? I did not see anything in root or buildbot crontabs? With respect to slow uploads, while trying various things, one thing I did was increase the upload blocksize in Buildbot config. Now I see tarbake upload (50MB) taking just 1 minute, which seems fair (probably it can be more if the network or buildbot is busy). So I'm wondering if the slow upload that was seen was in fact a combination of too low default blocksize, and uploads occasionally getting _completely_ stuck? It's of course still fine to move to using rsync, as Serg proposed earlier; just wanted to mention what I found. And in either case, we still need a solution to prevent /archive/pack from running full in the future... - Kristian.

2 1

[Maria-developers] MDEV-7288 USER/ROLE: CREATE OR REPLACE, CREATE IF NOT EXISTS, DROP IF EXISTS
by Alexander Barkov 09 Jan '15

09 Jan '15

Hi Sergei, Please have a look into the patch for MDEV-7288. It consists of the USER/ROLE related part of the big GSoC patch from Sriram that you reviewed earlier, with review suggestions addressed. You earlier had the following suggestions/questions: >> diff --git a/sql/sql_acl.cc b/sql/sql_acl.cc >> index 5f077c2..1e156be 100644 >> --- a/sql/sql_acl.cc >> +++ b/sql/sql_acl.cc >> ... >> @@ -9215,10 +9252,26 @@ bool mysql_create_user(THD *thd, List <LEX_USER> &lis >> */ >> if (handle_grant_data(tables, 0, user_name, NULL)) >> { >> - append_user(thd, &wrong_users, user_name); >> - >> - result= TRUE; >> - continue; >> + if (thd->lex->is_create_or_replace()) >> + { >> + if (handle_grant_data(tables, 1, user_name, NULL) <= 0) >> + { >> + append_user(thd, &wrong_users, user_name); > > I don't understand that at all. If user creation failed, you try to drop > it, but don't create a user again, after the old one was dropped. How > could it work? > Yeah, it looks confusing, because handle_grant_data() can do different things depending on the arguments. The call for handle_grant_data(tables, 0, user_name, NULL) does not create any users. It only checks if the user already exists. The call for handle_grant_data(tables, 1, user_name, NULL) drops the user. And the call for replace_user_table() after this block is the one that actually creates the user. I have added some comments in the new code, to make it clearer. >>> @@ -9268,7 +9321,7 @@ bool mysql_create_user(THD *thd, List <LEX_USER> &list, >>> (handle_as_role) ? "CREATE ROLE" : "CREATE USER", >>> wrong_users.c_ptr_safe()); >>> >>> - if (some_users_created) >>> + if (some_users_created || !result) >> >> What is that for? To binlog CREATE IF NOT EXISTS? Yes, that was the reason. I renamed "some_users_created" to "binlog" and changed the logic slightly to set "binlog" to true under the block checking lex->create_info.if_not_exists(), instead of this additional test for "result". So now it should be easier to read: if (binlog) result |= write_bin_log(thd, FALSE, thd->query(), thd->query_length()); Thanks.

3 3

[Maria-developers] prefix matching for command-line options - how to disable?
by Sergei Golubchik 06 Jan '15

06 Jan '15

Hi, We've got a bit of a problem with command-line option prefixes in the test suite. As you know, in MariaDB a command line option (the same applies to my.cnf too) doesn't need to be specified completely, it's sufficient to use an unambigous prefix of it. For example, 'net-retry-count=5' in my.cnf can be abbreviated as 'net-ret=5'. But not as 'net-re=5', the latter will fail with an error mysqld: ambiguous option '--net-re' (net_read_timeout, net_retry_count) This is the historical behavior of MySQL since about 2002. But it creates problems for plugins. InnoDB is particularly badly affected - it has lots of I_S plugins that are prefixes of other I_S plugins. For example: INNODB_BUFFER_PAGE INNODB_BUFFER_PAGE_LRU ... INNODB_CMP INNODB_CMPMEM INNODB_CMPMEM_RESET INNODB_CMP_PER_INDEX INNODB_CMP_PER_INDEX_RESET INNODB_CMP_RESET ... INNODB_SYS_FOREIGN INNODB_SYS_FOREIGN_COLS ... INNODB_SYS_TABLES INNODB_SYS_TABLESPACES INNODB_SYS_TABLESTATS also, it has plugins that match other plugin variables, like INNODB_LOCKS and innodb_locks_unsafe_for_binlog. As a result one cannot see or predict what these options will do. For example --enable-innodb-buffer-page might enable INNODB_BUFFER_PAGE plugin. Or it might enable INNODB_BUFFER_PAGE_LRU plugin - depending on which plugin will loaded (and will see and consume the option) first. And one cannot possibly disable INNODB_BUFFER_PAGE plugin, if the server will try to load INNODB_BUFFER_PAGE_LRU first. I see few solutions for this issue: 1. support a special suffix to option names that will disable prefix matching, for example: mysqld --enable-innodb-buffer-page! this will not match innodb-buffer-page-lru, so it's unambiguous. Of course, any dedicated character can be used, --enable-innodb-buffer-page$, or even a prefix --enable-exact-innodb-buffer-page, --enable-strict-innodb-buffer-page 2. add a special command-line option to disable prefix matching: mysqld --disable-getopt-prefix-matching --enable-innodb-buffer-page 3. do like mysql-5.7 is doing and completely remove support for prefix matching. In 5.7 one has to specified option names in full. I suspect that this is likely to break many exising setups. Opinions? Anybody cares about this obscure issue at all? Regards, Sergei

3 3

[Maria-developers] Buildslave terrier2 disabled due to KVM problems, needs fixing to be re-enabled
by Kristian Nielsen 05 Jan '15

05 Jan '15

Jani, Daniel, I discovered that our buildslave terrier2 seems to have problems with its KVM. Builds are incredibly slow, and it appears that it is not using KVM at all. So it is apparently using just QEMU, and soft-emulating the CPU. This is too slow to be useful, I think. So for now, I've removed all usage of terrier2 from the Buildbot config. If one of you wants to fix terrier2 to be useful again, then the patch to revert (to re-enable terrier2) is this: revid:knielsen@knielsen-hq.org-20150105182519-7492d95v1blss8jf http://bazaar.launchpad.net/~maria-captains/mariadb-tools/trunk/revision/kn… Before re-enabling, KVM needs to be somehow fixed to work (eg. something like `time perl -le 'for (0..100000000){}'` needs to be tested to run at about the same speed inside a guest VM as directly on the host). Maybe it would be best to upgrade the OS on the machine, maybe to the same as we have on bb01/bb02? We have had a number of problems on terrier2 with various KVM issues over the time. - Kristian.

1 0

[Maria-developers] New buildbot? - debpkg101-sid from Otto's github
by Otto Kekäläinen 02 Jan '15

02 Jan '15

Hello Daniel! Thanks for setting up the current buildbots that build my branches of 5.5 and 10.0. Could you now please also set up one the build from my 10.1 branch as I have started working on it too? The buildbot should be a copy of http://buildbot.askmonty.org/buildbot/builders/debpkg-sid/ but so that it pulls in the branch 10.1 from the repository https://github.com/ottok/mariadb Thanks! 2014-09-01 11:25 GMT+03:00 Otto Kekäläinen <otto(a)seravo.fi>: > Hello! > > 2014-03-14 12:59 GMT+02:00 Kristian Nielsen <knielsen(a)knielsen-hq.org>: >> You can see the two builders here: >> >> http://buildbot.askmonty.org/buildbot/builders/debpkg-trusty >> http://buildbot.askmonty.org/buildbot/builders/debpkg-sid/ > > Thanks Kristian and Daniel for creating these in March. > > The trusty build testing is less relevant nowadays while testing 10.0 > has become relevant. > > Could you please delete the buildbot debpkg-trusty and create a copy > of debpkg-sid called debpkg-10.0-sid which would be exactly like > debpkg-sid, but follow commits in this repo: > https://github.com/ottok/mariadb-10.0 ? > > Thanks! > > PS. I already configured the webhook in Github to be called in the > mariadb-10.0 repo as it is done in the mariadb-5.5 repo.

1 2

Re: [Maria-developers] [Commits] b2f45ba: MDEV-7324 - Lock-free hash for table definition cache
by Sergei Golubchik 28 Dec '14

28 Dec '14

Hi, Sergey! On Dec 22, svoj(a)mariadb.org wrote: > revision-id: b2f45ba8280c97f8dc8b1adb3234cf2e28e48982 > parent(s): 4a469b0ed68bb341e1a1e1ba540a45a3b6ff3ef9 > committer: Sergey Vojtovich > branch nick: 10.1 > timestamp: 2014-12-22 19:56:48 +0400 > message: > > MDEV-7324 - Lock-free hash for table definition cache That looked good to me. I didn't find anything obviously wrong. There are mostly questions below. Some are style suggestions (like, renaming of methods, etc). The only place I'm not sure about is early lf_hash_search_unpin() in tdc_acquire_share(). > diff --git a/mysys/lf_hash.c b/mysys/lf_hash.c > --- a/mysys/lf_hash.c > +++ b/mysys/lf_hash.c > @@ -122,7 +122,7 @@ static int lfind(LF_SLIST * volatile *head, CHARSET_INFO *cs, uint32 hashnr, > { > if (unlikely(callback)) > { > - if (callback(cursor->curr + 1, (void*)key)) > + if (cur_hashnr & 1 && callback(cursor->curr + 1, (void*)key)) Sorry for this. > return 1; > } > else if (cur_hashnr >= hashnr) > diff --git a/sql/sql_class.cc b/sql/sql_class.cc > --- a/sql/sql_class.cc > +++ b/sql/sql_class.cc > @@ -912,17 +912,17 @@ bool Drop_table_error_handler::handle_condition(THD *thd, > #endif /* defined(ENABLED_DEBUG_SYNC) */ > wait_for_commit_ptr(0), > main_da(0, false, false), > - m_stmt_da(&main_da) > + m_stmt_da(&main_da), > #ifdef WITH_WSREP > - , > wsrep_applier(is_wsrep_applier), > wsrep_applier_closing(false), > wsrep_client_thread(false), > wsrep_apply_toi(false), > wsrep_po_handle(WSREP_PO_INITIALIZER), > wsrep_po_cnt(0), > - wsrep_apply_format(0) > + wsrep_apply_format(0), > #endif > + tdc_hash_pins(0) Eh... I actually wanted wsrep related data to the at the end of the THD. Just to keep the rest of the THD layout identical for wsrep and non-wsrep builds. In case a wsrep-enabled plugin is loaded into non-wsrep server or visa versa. That was an early change, though, I've done more cleanups after that and moved wsrep into a service, so perhaps (even, most probably) this is not needed anymore. But I'd better stay on the safe side and keep wsrep at the end of THD. > { > ulong tmp; > > @@ -1698,6 +1698,8 @@ void THD::cleanup(void) > > free_root(&main_mem_root, MYF(0)); > main_da.free_memory(); > + if (tdc_hash_pins) > + lf_hash_put_pins(tdc_hash_pins); no tdc_hash_pins=0 here? > if (status_var.memory_used != 0) > { > DBUG_PRINT("error", ("memory_used: %lld", status_var.memory_used)); > diff --git a/sql/table.h b/sql/table.h > --- a/sql/table.h > +++ b/sql/table.h > @@ -611,32 +612,7 @@ struct TABLE_SHARE > mysql_mutex_t LOCK_ha_data; /* To protect access to ha_data */ > mysql_mutex_t LOCK_share; /* To protect TABLE_SHARE */ > > - typedef I_P_List <TABLE, TABLE_share> TABLE_list; > - typedef I_P_List <TABLE, All_share_tables> All_share_tables_list; > - struct > - { > - /** > - Protects ref_count, m_flush_tickets, all_tables, free_tables, flushed, > - all_tables_refs. > - */ > - mysql_mutex_t LOCK_table_share; > - mysql_cond_t COND_release; > - TABLE_SHARE *next, **prev; /* Link to unused shares */ > - uint ref_count; /* How many TABLE objects uses this */ > - uint all_tables_refs; /* Number of refs to all_tables */ > - /** > - List of tickets representing threads waiting for the share to be flushed. > - */ > - Wait_for_flush_list m_flush_tickets; > - /* > - Doubly-linked (back-linked) lists of used and unused TABLE objects > - for this share. > - */ > - All_share_tables_list all_tables; > - TABLE_list free_tables; > - ulong version; > - bool flushed; > - } tdc; > + TDC_element *tdc; Hmm. Ok. I think I like this change - TABLE_SHARE now contains table metadata and everything related to tdc is moved to TDC_element. Right? > > LEX_CUSTRING tabledef_version; > > diff --git a/sql/table.cc b/sql/table.cc > --- a/sql/table.cc > +++ b/sql/table.cc > @@ -3845,11 +3844,11 @@ bool TABLE_SHARE::visit_subgraph(Wait_for_flush *wait_for_flush, > because we won't try to acquire tdc.LOCK_table_share while > holding a write-lock on MDL_lock::m_rwlock. > */ > - mysql_mutex_lock(&tdc.LOCK_table_share); > - tdc.all_tables_refs++; > - mysql_mutex_unlock(&tdc.LOCK_table_share); > + mysql_mutex_lock(&tdc->LOCK_table_share); > + tdc->all_tables_refs++; > + mysql_mutex_unlock(&tdc->LOCK_table_share); Do I understand it correctly that a TDC_element can never be deleted from LF_HASH as long as at least one TABLE_SHARE references it? > > - All_share_tables_list::Iterator tables_it(tdc.all_tables); > + TDC_element::All_share_tables_list::Iterator tables_it(tdc->all_tables); > > /* > In case of multiple searches running in parallel, avoid going > @@ -3946,21 +3945,10 @@ bool TABLE_SHARE::wait_for_old_version(THD *thd, struct timespec *abstime, > > mdl_context->done_waiting_for(); > > - mysql_mutex_lock(&tdc.LOCK_table_share); > - > - tdc.m_flush_tickets.remove(&ticket); > - > - if (tdc.m_flush_tickets.is_empty() && tdc.ref_count == 0) > - { > - /* > - If our thread was the last one using the share, > - we must destroy it here. > - */ > - mysql_mutex_unlock(&tdc.LOCK_table_share); > - destroy(); > - } > - else > - mysql_mutex_unlock(&tdc.LOCK_table_share); > + mysql_mutex_lock(&tdc->LOCK_table_share); > + tdc->m_flush_tickets.remove(&ticket); > + mysql_cond_broadcast(&tdc->COND_release); > + mysql_mutex_unlock(&tdc->LOCK_table_share); Why is the logic here different? > > > /* > diff --git a/sql/table_cache.h b/sql/table_cache.h > --- a/sql/table_cache.h > +++ b/sql/table_cache.h > @@ -16,6 +16,171 @@ > Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA */ > > > +extern PSI_mutex_key key_TABLE_SHARE_LOCK_table_share; > +extern PSI_cond_key key_TABLE_SHARE_COND_release; > + > +class TDC_element > +{ > +public: > + uchar m_key[NAME_LEN + 1 + NAME_LEN + 1]; > + uint m_key_length; > + ulong version; > + bool flushed; > + TABLE_SHARE *share; > + > + typedef I_P_List <TABLE, TABLE_share> TABLE_list; > + typedef I_P_List <TABLE, All_share_tables> All_share_tables_list; > + /** > + Protects ref_count, m_flush_tickets, all_tables, free_tables, flushed, > + all_tables_refs. > + */ > + mysql_mutex_t LOCK_table_share; > + mysql_cond_t COND_release; > + TDC_element *next, **prev; /* Link to unused shares */ > + uint ref_count; /* How many TABLE objects uses this */ > + uint all_tables_refs; /* Number of refs to all_tables */ > + /** > + List of tickets representing threads waiting for the share to be flushed. > + */ > + Wait_for_flush_list m_flush_tickets; > + /* > + Doubly-linked (back-linked) lists of used and unused TABLE objects > + for this share. > + */ > + All_share_tables_list all_tables; > + TABLE_list free_tables; > + > + TDC_element() {} > + > + TDC_element(const char *key, uint key_length) : m_key_length(key_length) > + { > + memcpy(m_key, key, key_length); > + } > + > + > + void dbug_release_assertions() Somewhat confusing name. If I hadn't see the code below I'd thought this function releases some assertions (how? why?) may be assert_correctness_at_release() would be less ambiguous? > + { > + DBUG_ASSERT(share == 0); > + DBUG_ASSERT(ref_count == 0); > + DBUG_ASSERT(m_flush_tickets.is_empty()); > + DBUG_ASSERT(all_tables.is_empty()); > + DBUG_ASSERT(free_tables.is_empty()); > + DBUG_ASSERT(all_tables_refs == 0); > + DBUG_ASSERT(next == 0); > + DBUG_ASSERT(prev == 0); > + } > + > + > + /** > + Acquire TABLE object from table cache. > + > + @pre share must be protected against removal. > + > + Acquired object cannot be evicted or acquired again. > + > + While locked: > + - pop object from TABLE_SHARE::tdc.free_tables > + > + While unlocked: > + - mark object used by thd "While locked" / "While unlocked" - what does that mean? > + > + @return TABLE object, or NULL if no unused objects. > + */ > + > + TABLE *acquire_table(THD *thd) > + { > + TABLE *table; > + > + mysql_mutex_lock(&LOCK_table_share); > + table= free_tables.pop_front(); > + if (table) > + { > + DBUG_ASSERT(!table->in_use); lots of asserts - good! > + table->in_use= thd; > + /* The ex-unused table must be fully functional. */ > + DBUG_ASSERT(table->db_stat && table->file); > + /* The children must be detached from the table. */ > + DBUG_ASSERT(!table->file->extra(HA_EXTRA_IS_ATTACHED_CHILDREN)); I don't even think of asking why it's HA_EXTRA_IS_ATTACHED_CHILDREN and not HA_EXTRA_HAS_ATTACHED_CHILDREN or, at least, HA_EXTRA_CHILDREN_ARE_ATTACHED. Ah, whatever... Horrors of the English language hidden in old MySQL code :) > + } > + mysql_mutex_unlock(&LOCK_table_share); > + return table; > + } > + > + > + /** > + Get last element of free_tables. > + */ > + > + TABLE *free_tables_back() Eh, I've thought for a few seconds what could it mean to "free tables back" until I read the function comment. You see, "free" is so often used as a verb that one absolutely doesn't see it that here it means "free_tables' back" even though it's also a common naming pattern (xxx_back for the list xxx). Rename the method, may be? > + { > + TABLE_list::Iterator it(share->tdc->free_tables); > + TABLE *entry, *last= 0; > + while ((entry= it++)) > + last= entry; > + return last; > + } > + > + > + /** > + Wait for MDL deadlock detector to complete traversing tdc.all_tables. > + > + Must be called before updating TABLE_SHARE::tdc.all_tables. > + */ > + > + void wait_for_mdl_deadlock_detector() > + { > + while (all_tables_refs) > + mysql_cond_wait(&COND_release, &LOCK_table_share); > + } > + > + > + /** > + Prepeare table share for use with table definition cache. > + */ > + > + static void lf_alloc_constructor(uchar *arg) > + { > + TDC_element *element= (TDC_element*) (arg + LF_HASH_OVERHEAD); > + DBUG_ENTER("tdc_init_share"); intentionally? or forgot to update DBUG_ENTER? > + mysql_mutex_init(key_TABLE_SHARE_LOCK_table_share, > + &element->LOCK_table_share, MY_MUTEX_INIT_FAST); > + mysql_cond_init(key_TABLE_SHARE_COND_release, &element->COND_release, 0); > + element->m_flush_tickets.empty(); > + element->all_tables.empty(); > + element->free_tables.empty(); > + element->all_tables_refs= 0; > + element->share= 0; > + element->ref_count= 0; > + element->next= 0; > + element->prev= 0; > + DBUG_VOID_RETURN; > + } > + > + > + /** > + Release table definition cache specific resources of table share. > + */ > + > + static void lf_alloc_destructor(uchar *arg) > + { > + TDC_element *element= (TDC_element*) (arg + LF_HASH_OVERHEAD); > + DBUG_ENTER("tdc_deinit_share"); intentionally? or forgot to update DBUG_ENTER? > + element->dbug_release_assertions(); > + mysql_cond_destroy(&element->COND_release); > + mysql_mutex_destroy(&element->LOCK_table_share); > + DBUG_VOID_RETURN; > + } > + > + > + static uchar *key(const TDC_element *element, size_t *length, > + my_bool not_used __attribute__((unused))) > + { > + *length= element->m_key_length; > + return (uchar*) element->m_key; > + } > +}; > + > + > enum enum_tdc_remove_table_type > { > TDC_RT_REMOVE_ALL, > diff --git a/sql/sql_base.cc b/sql/sql_base.cc > --- a/sql/sql_base.cc > +++ b/sql/sql_base.cc > @@ -270,58 +270,75 @@ uint get_table_def_key(const TABLE_LIST *table_list, const char **key) > # Pointer to list of names of open tables. > */ > > -OPEN_TABLE_LIST *list_open_tables(THD *thd, const char *db, const char *wild) > +struct list_open_tables_arg > { > + THD *thd; > + const char *db; > + const char *wild; > TABLE_LIST table_list; > OPEN_TABLE_LIST **start_list, *open_list; > - TABLE_SHARE *share; > - TDC_iterator tdc_it; > - DBUG_ENTER("list_open_tables"); > +}; > > - bzero((char*) &table_list,sizeof(table_list)); > - start_list= &open_list; > - open_list=0; > > - tdc_it.init(); > - while ((share= tdc_it.next())) > +static my_bool list_open_tables_callback(TDC_element *element, > + list_open_tables_arg *arg) > { > - if (db && my_strcasecmp(system_charset_info, db, share->db.str)) > - continue; > - if (wild && wild_compare(share->table_name.str, wild, 0)) > - continue; > + char *db= (char*) element->m_key; > + char *table_name= (char*) element->m_key + strlen((char*) element->m_key) + 1; > > + if (arg->db && my_strcasecmp(system_charset_info, arg->db, db)) > + return FALSE; > + if (arg->wild && wild_compare(table_name, arg->wild, 0)) > + return FALSE; > + > /* Check if user has SELECT privilege for any column in the table */ > - table_list.db= share->db.str; > - table_list.table_name= share->table_name.str; > - table_list.grant.privilege=0; > + arg->table_list.db= db; > + arg->table_list.table_name= table_name; > + arg->table_list.grant.privilege= 0; > > - if (check_table_access(thd,SELECT_ACL,&table_list, TRUE, 1, TRUE)) > - continue; > + if (check_table_access(arg->thd, SELECT_ACL, &arg->table_list, TRUE, 1, TRUE)) > + return FALSE; > > - if (!(*start_list = (OPEN_TABLE_LIST *) > - sql_alloc(sizeof(**start_list)+share->table_cache_key.length))) > - { > - open_list=0; // Out of memory > - break; > - } > + if (!(*arg->start_list= (OPEN_TABLE_LIST *) sql_alloc( when you know the thd, prefer thd->alloc() over sql_alloc() (which is, basically, current_thd->alloc()) > + sizeof(**arg->start_list) + element->m_key_length))) > + return TRUE; > + > - strmov((*start_list)->table= > - strmov(((*start_list)->db= (char*) ((*start_list)+1)), > - share->db.str)+1, > - share->table_name.str); > + strmov((*arg->start_list)->table= > + strmov(((*arg->start_list)->db= (char*) ((*arg->start_list) + 1)), > + db) + 1, table_name); > - (*start_list)->in_use= 0; > + (*arg->start_list)->in_use= 0; > - mysql_mutex_lock(&share->tdc.LOCK_table_share); > - TABLE_SHARE::All_share_tables_list::Iterator it(share->tdc.all_tables); > + > + mysql_mutex_lock(&element->LOCK_table_share); > + TDC_element::All_share_tables_list::Iterator it(element->all_tables); > TABLE *table; > while ((table= it++)) > if (table->in_use) > - ++(*start_list)->in_use; > + ++(*arg->start_list)->in_use; > - mysql_mutex_unlock(&share->tdc.LOCK_table_share); > + mysql_mutex_unlock(&element->LOCK_table_share); > - (*start_list)->locked= 0; /* Obsolete. */ > + (*arg->start_list)->locked= 0; /* Obsolete. */ > - start_list= &(*start_list)->next; > + arg->start_list= &(*arg->start_list)->next; > - *start_list=0; > + *arg->start_list= 0; > + return FALSE; > } > - tdc_it.deinit(); > - DBUG_RETURN(open_list); > + > + > +OPEN_TABLE_LIST *list_open_tables(THD *thd, const char *db, const char *wild) > +{ > + list_open_tables_arg argument; > + DBUG_ENTER("list_open_tables"); > + > + argument.thd= thd; > + argument.db= db; > + argument.wild= wild; > + bzero((char*) &argument.table_list, sizeof(argument.table_list)); > + argument.start_list= &argument.open_list; > + argument.open_list= 0; > + > + if (tdc_iterate(thd, (my_hash_walk_action) list_open_tables_callback, > + &argument, true)) > + DBUG_RETURN(0); > + > + DBUG_RETURN(argument.open_list); > } > > /***************************************************************************** > diff --git a/sql/table_cache.cc b/sql/table_cache.cc > --- a/sql/table_cache.cc > +++ b/sql/table_cache.cc > @@ -72,32 +76,23 @@ > /** > Protects unused shares list. > > - TABLE_SHARE::tdc.prev > - TABLE_SHARE::tdc.next > - oldest_unused_share > - end_of_unused_share > + TDC_element::prev > + TDC_element::next > + unused_shares > */ > > static mysql_mutex_t LOCK_unused_shares; > -static mysql_rwlock_t LOCK_tdc; /**< Protects tdc_hash. */ > my_atomic_rwlock_t LOCK_tdc_atomics; /**< Protects tdc_version. */ > > #ifdef HAVE_PSI_INTERFACE > -static PSI_mutex_key key_LOCK_unused_shares, key_TABLE_SHARE_LOCK_table_share; > +PSI_mutex_key key_LOCK_unused_shares, key_TABLE_SHARE_LOCK_table_share; really? not static? > static PSI_mutex_info all_tc_mutexes[]= > { > { &key_LOCK_unused_shares, "LOCK_unused_shares", PSI_FLAG_GLOBAL }, > { &key_TABLE_SHARE_LOCK_table_share, "TABLE_SHARE::tdc.LOCK_table_share", 0 } > }; > > -static PSI_rwlock_key key_rwlock_LOCK_tdc; > -static PSI_rwlock_info all_tc_rwlocks[]= > -{ > - { &key_rwlock_LOCK_tdc, "LOCK_tdc", PSI_FLAG_GLOBAL } > -}; > - > - > -static PSI_cond_key key_TABLE_SHARE_COND_release; > +PSI_cond_key key_TABLE_SHARE_COND_release; and here > static PSI_cond_info all_tc_conds[]= > { > { &key_TABLE_SHARE_COND_release, "TABLE_SHARE::tdc.COND_release", 0 } > @@ -265,85 +269,41 @@ void tc_add_table(THD *thd, TABLE *table) > > if (need_purge) > { > - TABLE_SHARE *purge_share= 0; > - TABLE_SHARE *share; > - TABLE *entry; > - ulonglong UNINIT_VAR(purge_time); > - TDC_iterator tdc_it; > + tc_add_table_arg argument; > + argument.purge_time= ULONGLONG_MAX; > + tdc_iterate(thd, (my_hash_walk_action) tc_add_table_callback, &argument); > > - tdc_it.init(); > - while ((share= tdc_it.next())) > + if (argument.purge_time != ULONGLONG_MAX) > { > - mysql_mutex_lock(&share->tdc.LOCK_table_share); > - if ((entry= tc_free_tables_back(share)) && > - (!purge_share || entry->tc_time < purge_time)) > + TDC_element *element= (TDC_element*) lf_hash_search(&tdc_hash, > + thd->tdc_hash_pins, > + argument.key, > + argument.key_length); > + if (element) > { > - purge_share= share; > - purge_time= entry->tc_time; > - } > - mysql_mutex_unlock(&share->tdc.LOCK_table_share); > - } > + TABLE *entry; > + mysql_mutex_lock(&element->LOCK_table_share); > + lf_hash_search_unpin(thd->tdc_hash_pins); > + element->wait_for_mdl_deadlock_detector(); you wait under a mutex? hmm... > > - if (purge_share) > - { > - mysql_mutex_lock(&purge_share->tdc.LOCK_table_share); > - tc_wait_for_mdl_deadlock_detector(purge_share); > - tdc_it.deinit(); > /* > It may happen that oldest table was acquired meanwhile. In this case > just go ahead, number of objects in table cache will normalize > eventually. > */ > - if ((entry= tc_free_tables_back(purge_share)) && > - entry->tc_time == purge_time) > + if ((entry= element->free_tables_back()) && > + entry->tc_time == argument.purge_time) > { > - entry->s->tdc.free_tables.remove(entry); > + element->free_tables.remove(entry); > tc_remove_table(entry); > - mysql_mutex_unlock(&purge_share->tdc.LOCK_table_share); > + mysql_mutex_unlock(&element->LOCK_table_share); > intern_close_table(entry); > } > else > - mysql_mutex_unlock(&purge_share->tdc.LOCK_table_share); > + mysql_mutex_unlock(&element->LOCK_table_share); > } > - else > - tdc_it.deinit(); > } > } > - > - > -/** > - Acquire TABLE object from table cache. > - > - @pre share must be protected against removal. > - > - Acquired object cannot be evicted or acquired again. > - > - While locked: > - - pop object from TABLE_SHARE::tdc.free_tables > - > - While unlocked: > - - mark object used by thd > - > - @return TABLE object, or NULL if no unused objects. > -*/ > - > -static TABLE *tc_acquire_table(THD *thd, TABLE_SHARE *share) > -{ > - TABLE *table; > - > - mysql_mutex_lock(&share->tdc.LOCK_table_share); > - table= share->tdc.free_tables.pop_front(); > - if (table) > - { > - DBUG_ASSERT(!table->in_use); > - table->in_use= thd; > - /* The ex-unused table must be fully functional. */ > - DBUG_ASSERT(table->db_stat && table->file); > - /* The children must be detached from the table. */ > - DBUG_ASSERT(!table->file->extra(HA_EXTRA_IS_ATTACHED_CHILDREN)); > - } ok, you've moved it to table_cache.h... > - mysql_mutex_unlock(&share->tdc.LOCK_table_share); > - return table; > } > > > @@ -397,88 +357,76 @@ bool tc_release_table(TABLE *table) > */ > table->in_use= 0; > /* Add table to the list of unused TABLE objects for this share. */ > - table->s->tdc.free_tables.push_front(table); > - mysql_mutex_unlock(&table->s->tdc.LOCK_table_share); > + table->s->tdc->free_tables.push_front(table); > + mysql_mutex_unlock(&table->s->tdc->LOCK_table_share); > return false; > > purge: > - tc_wait_for_mdl_deadlock_detector(table->s); > + table->s->tdc->wait_for_mdl_deadlock_detector(); > tc_remove_table(table); > - mysql_mutex_unlock(&table->s->tdc.LOCK_table_share); > + mysql_mutex_unlock(&table->s->tdc->LOCK_table_share); > table->in_use= 0; > intern_close_table(table); > return true; > } > > > -extern "C" uchar *tdc_key(const uchar *record, size_t *length, > - my_bool not_used __attribute__((unused))) > -{ > - TABLE_SHARE *entry= (TABLE_SHARE*) record; > - *length= entry->table_cache_key.length; > - return (uchar*) entry->table_cache_key.str; > -} > - > - > /** > Delete share from hash and free share object. > - > - @return > - @retval 0 Success > - @retval 1 Share is referenced > */ > > -static int tdc_delete_share_from_hash(TABLE_SHARE *share) > +static void tdc_delete_share_from_hash(TDC_element *element) > { > + THD *thd= current_thd; > + LF_PINS *pins; > + TABLE_SHARE *share; > DBUG_ENTER("tdc_delete_share_from_hash"); > - mysql_rwlock_wrlock(&LOCK_tdc); > - mysql_mutex_lock(&share->tdc.LOCK_table_share); > - if (--share->tdc.ref_count) > - { > - mysql_cond_broadcast(&share->tdc.COND_release); > - mysql_mutex_unlock(&share->tdc.LOCK_table_share); > - mysql_rwlock_unlock(&LOCK_tdc); > - DBUG_RETURN(1); > - } > - my_hash_delete(&tdc_hash, (uchar*) share); > + > + share= element->share; > + DBUG_ASSERT(share); > + element->share= 0; > /* Notify PFS early, while still locked. */ "while still locked" what? > PSI_CALL_release_table_share(share->m_psi); > share->m_psi= 0; > - mysql_rwlock_unlock(&LOCK_tdc); > > - if (share->tdc.m_flush_tickets.is_empty()) > + if (!element->m_flush_tickets.is_empty()) > { > - /* No threads are waiting for this share to be flushed, destroy it. */ > - mysql_mutex_unlock(&share->tdc.LOCK_table_share); > - free_table_share(share); > - } > - else > - { > - Wait_for_flush_list::Iterator it(share->tdc.m_flush_tickets); > + Wait_for_flush_list::Iterator it(element->m_flush_tickets); > Wait_for_flush *ticket; > while ((ticket= it++)) > (void) ticket->get_ctx()->m_wait.set_status(MDL_wait::GRANTED); > - /* > - If there are threads waiting for this share to be flushed, > - the last one to receive the notification will destroy the > - share. At this point the share is removed from the table > - definition cache, so is OK to proceed here without waiting > - for this thread to do the work. > - */ > - mysql_mutex_unlock(&share->tdc.LOCK_table_share); > + > + do > + { > + mysql_cond_wait(&element->COND_release, &element->LOCK_table_share); > + } while (!element->m_flush_tickets.is_empty()); > } > - DBUG_RETURN(0); > + > + mysql_mutex_unlock(&element->LOCK_table_share); Ok, so the caller must lock LOCK_table_share? please put mysql_mutex_assert_owner() in the beginning of the function > + > + if (thd) When thd is NULL here? On shutdown? > + { > + fix_thd_pins(thd); > + pins= thd->tdc_hash_pins; > } > + else > + pins= lf_hash_get_pins(&tdc_hash); > > + DBUG_ASSERT(pins); // What can we do about it? > + element->dbug_release_assertions(); > + lf_hash_delete(&tdc_hash, pins, element->m_key, element->m_key_length); > + if (!thd) > + lf_hash_put_pins(pins); > + free_table_share(share); > + DBUG_VOID_RETURN; > +} > > + > /** > Initialize table definition cache. > - > - @retval 0 Success > - @retval !0 Error > */ > > -int tdc_init(void) > +void tdc_init(void) > { > DBUG_ENTER("tdc_init"); > #ifdef HAVE_PSI_INTERFACE > @@ -642,26 +548,34 @@ void tdc_deinit_share(TABLE_SHARE *share) > Caller is expected to unlock table share with tdc_unlock_share(). > > @retval 0 Share not found > - @retval !0 Pointer to locked table share > + @retval MY_ERRPTR OOM > + @retval ptr Pointer to locked table share > */ > > -TABLE_SHARE *tdc_lock_share(const char *db, const char *table_name) > +TDC_element *tdc_lock_share(THD *thd, const char *db, const char *table_name) > { > + TDC_element *element; > char key[MAX_DBKEY_LENGTH]; > - uint key_length; > > DBUG_ENTER("tdc_lock_share"); > - key_length= tdc_create_key(key, db, table_name); > + if (fix_thd_pins(thd)) > + DBUG_RETURN((TDC_element*) MY_ERRPTR); I suppose you can do it in THD:init, if you'd like. A THD almost always will need to access table definition cache, so it'll need these pins. Then there will be no need to return MY_ERRPTR here and handle it in the caller > > - mysql_rwlock_rdlock(&LOCK_tdc); > - TABLE_SHARE* share= (TABLE_SHARE*) my_hash_search(&tdc_hash, > - (uchar*) key, key_length); > - if (share && !share->error) > - mysql_mutex_lock(&share->tdc.LOCK_table_share); > - else > - share= 0; > - mysql_rwlock_unlock(&LOCK_tdc); > - DBUG_RETURN(share); > + element= (TDC_element *) lf_hash_search(&tdc_hash, thd->tdc_hash_pins, > + (uchar*) key, > + tdc_create_key(key, db, table_name)); > + if (element) > + { > + mysql_mutex_lock(&element->LOCK_table_share); > + if (!element->share || element->share->error) > + { > + mysql_mutex_unlock(&element->LOCK_table_share); > + element= 0; > + } > + lf_hash_search_unpin(thd->tdc_hash_pins); > + } > + > + DBUG_RETURN(element); Looks good to me > } > > > @@ -702,60 +616,60 @@ TABLE_SHARE *tdc_acquire_share(THD *thd, const char *db, const char *table_name, > TABLE **out_table) > { > TABLE_SHARE *share; > + TDC_element *element; > bool was_unused; > DBUG_ENTER("tdc_acquire_share"); > > - mysql_rwlock_rdlock(&LOCK_tdc); > - share= (TABLE_SHARE*) my_hash_search_using_hash_value(&tdc_hash, hash_value, > - (uchar*) key, > - key_length); > - if (!share) > + if (fix_thd_pins(thd)) > + DBUG_RETURN(0); > + > +retry: > + while (!(element= (TDC_element*) lf_hash_search_using_hash_value(&tdc_hash, Was lf_hash_search_using_hash_value added in a separate patch? > + thd->tdc_hash_pins, hash_value, (uchar*) key, key_length))) > { > - TABLE_SHARE *new_share; > - mysql_rwlock_unlock(&LOCK_tdc); > + TDC_element tmp(key, key_length); > + int res= lf_hash_insert(&tdc_hash, thd->tdc_hash_pins, (uchar*) &tmp); > > - if (!(new_share= alloc_table_share(db, table_name, key, key_length))) > + if (res == -1) > DBUG_RETURN(0); > - new_share->error= OPEN_FRM_OPEN_ERROR; > + else if (res == 1) > + continue; > > - mysql_rwlock_wrlock(&LOCK_tdc); > - share= (TABLE_SHARE*) my_hash_search_using_hash_value(&tdc_hash, hash_value, > - (uchar*) key, > - key_length); > - if (!share) > - { > - bool need_purge; > + element= (TDC_element*) lf_hash_search_using_hash_value(&tdc_hash, > + thd->tdc_hash_pins, hash_value, (uchar*) key, key_length); > + lf_hash_search_unpin(thd->tdc_hash_pins); How can you be sure that nobody frees your element after you unpin it? > + DBUG_ASSERT(element); > + element->dbug_release_assertions(); > > - share= new_share; > - mysql_mutex_lock(&share->tdc.LOCK_table_share); > - if (my_hash_insert(&tdc_hash, (uchar*) share)) > + if (!(share= alloc_table_share(db, table_name, key, key_length))) > { > - mysql_mutex_unlock(&share->tdc.LOCK_table_share); > - mysql_rwlock_unlock(&LOCK_tdc); > - free_table_share(share); > + lf_hash_delete(&tdc_hash, thd->tdc_hash_pins, key, key_length); > DBUG_RETURN(0); > } > - need_purge= tdc_hash.records > tdc_size; > - mysql_rwlock_unlock(&LOCK_tdc); > > /* note that tdc_acquire_share() *always* uses discovery */ > open_table_def(thd, share, flags | GTS_USE_DISCOVERY); > - share->tdc.ref_count++; > - mysql_mutex_unlock(&share->tdc.LOCK_table_share); > > if (share->error) > { > - tdc_delete_share_from_hash(share); > + free_table_share(share); > + lf_hash_delete(&tdc_hash, thd->tdc_hash_pins, key, key_length); > DBUG_RETURN(0); > } > - else if (need_purge) > + > + mysql_mutex_lock(&element->LOCK_table_share); > + element->share= share; > + share->tdc= element; > + element->ref_count++; > + element->version= tdc_refresh_version(); > + element->flushed= false; > + mysql_mutex_unlock(&element->LOCK_table_share); > + > tdc_purge(false); > if (out_table) > *out_table= 0; > share->m_psi= PSI_CALL_get_table_share(false, share); > goto end; > - } > - free_table_share(new_share); > } > > /* cannot force discovery of a cached share */ > @@ -763,18 +677,25 @@ TABLE_SHARE *tdc_acquire_share(THD *thd, const char *db, const char *table_name, > > if (out_table && (flags & GTS_TABLE)) > { > - if ((*out_table= tc_acquire_table(thd, share))) > + if ((*out_table= element->acquire_table(thd))) > { > - mysql_rwlock_unlock(&LOCK_tdc); > + lf_hash_search_unpin(thd->tdc_hash_pins); what this lf_hash_search_unpin corresponds to? > DBUG_ASSERT(!(flags & GTS_NOLOCK)); > - DBUG_ASSERT(!share->error); > - DBUG_ASSERT(!share->is_view); > - DBUG_RETURN(share); > + DBUG_ASSERT(element->share); > + DBUG_ASSERT(!element->share->error); > + DBUG_ASSERT(!element->share->is_view); > + DBUG_RETURN(element->share); > } > } > > - mysql_mutex_lock(&share->tdc.LOCK_table_share); > - mysql_rwlock_unlock(&LOCK_tdc); > + mysql_mutex_lock(&element->LOCK_table_share); > + if (!(share= element->share)) > + { > + mysql_mutex_unlock(&element->LOCK_table_share); > + lf_hash_search_unpin(thd->tdc_hash_pins); and here > + goto retry; > + } > + lf_hash_search_unpin(thd->tdc_hash_pins); may be that first lf_hash_search_unpin was wrong? the one that goes directly after lf_hash_search. > > /* > We found an existing table definition. Return it if we didn't get > @@ -983,13 +864,36 @@ bool tdc_remove_table(THD *thd, enum_tdc_remove_table_type remove_type, > thd->mdl_context.is_lock_owner(MDL_key::TABLE, db, table_name, > MDL_EXCLUSIVE)); > > - if ((share= tdc_delete_share(db, table_name))) > + > + mysql_mutex_lock(&LOCK_unused_shares); > + if (!(element= tdc_lock_share(thd, db, table_name))) > { > - I_P_List <TABLE, TABLE_share> purge_tables; > - uint my_refs= 1; > + mysql_mutex_unlock(&LOCK_unused_shares); > + DBUG_ASSERT(remove_type != TDC_RT_REMOVE_NOT_OWN_KEEP_SHARE); > + DBUG_RETURN(false); > + } > > - mysql_mutex_lock(&share->tdc.LOCK_table_share); > - tc_wait_for_mdl_deadlock_detector(share); > + DBUG_ASSERT(element != MY_ERRPTR); // What can we do about it? allocate thd->tdc_hash_pins in THD::init > + > + if (!element->ref_count) > + { > + if (element->prev) > + { > + unused_shares.remove(element); > + /* Concurrent thread may start using share again, reset prev and next. */ What concurrent thread? element->ref_count is 0, you removed the the share from the unused_shares list. There can be no concurrent threads, as far as I understand. And indeed, you do remove the share from the hash (which does free_table_share) immediately below, which also proves that there can be no concurrent threads. > + element->prev= 0; > + element->next= 0; > + } > + mysql_mutex_unlock(&LOCK_unused_shares); > + > + tdc_delete_share_from_hash(element); > + DBUG_RETURN(true); > + } > + mysql_mutex_unlock(&LOCK_unused_shares); > + > + element->ref_count++; > + > + element->wait_for_mdl_deadlock_detector(); > /* > Mark share flushed in order to ensure that it gets > automatically deleted once it is no longer referenced. Regards, Sergei

2 4

[Maria-developers] ccache with mariadb?
by Otto Kekäläinen 28 Dec '14

28 Dec '14

Hello! Has somebody managed to configure ccache correctly so that it would work with MariaDB compilation? https://ccache.samba.org/ I don't seem to get it working as the build always seems to pick up the real gcc binary instead of any of the ccache symlink wrappers I've manually put in place to replace all of gcc, cc1, cc1plus etc.

2 3

Re: [Maria-developers] MDEV-7273 - 10.1 fails to start up during tc_log initializations on PPC64
by Sergei Golubchik 26 Dec '14

26 Dec '14

Hi, Sergey! On Dec 26, svoj(a)mariadb.org wrote: > revision-id: f65901eef28728212a350f61569dc884c747bffc > parent(s): 8c616cd347aa7b3fcc2d8defb5e172ad316f432c > committer: Sergey Vojtovich > branch nick: 10.1 > timestamp: 2014-12-26 23:38:45 +0400 > message: > > MDEV-7273 - 10.1 fails to start up during tc_log initializations on PPC64 > > log-tc-size is 24K by default. Page size is 64K on PPC64. But log-tc-size > must be at least 3 x page size. This is enforced by TC_LOG_MMAP::open() > with a comment: to guarantee non-empty pool. > > This all makes server not startable in default configuration on PPC64. > > Autosize log-tc-size, so that it's min value= page size * 3, default > value= page size * 6, block size= page size. > > diff --git a/sql/log.cc b/sql/log.cc > index 66e1426..52c85d6 100644 > --- a/sql/log.cc > +++ b/sql/log.cc > @@ -8780,6 +8779,7 @@ int TC_LOG_MMAP::sync() > int TC_LOG_MMAP::unlog(ulong cookie, my_xid xid) > { > pending_cookies *full_buffer= NULL; > + uint32 ncookies= tc_log_page_size / sizeof(my_xid); > DBUG_ASSERT(*(my_xid *)(data+cookie) == xid); > > /* > @@ -8793,7 +8793,7 @@ int TC_LOG_MMAP::unlog(ulong cookie, my_xid xid) > mysql_mutex_lock(&LOCK_pending_checkpoint); > if (pending_checkpoint == NULL) > { > - uint32 size= sizeof(*pending_checkpoint); > + uint32 size= sizeof(*pending_checkpoint) + sizeof(ulong) * (ncookies - 1); > if (!(pending_checkpoint= > (pending_cookies *)my_malloc(size, MYF(MY_ZEROFILL)))) > { > @@ -8804,8 +8804,7 @@ int TC_LOG_MMAP::unlog(ulong cookie, my_xid xid) > } > > pending_checkpoint->cookies[pending_checkpoint->count++]= cookie; > - if (pending_checkpoint->count == sizeof(pending_checkpoint->cookies) / > - sizeof(pending_checkpoint->cookies[0])) > + if (pending_checkpoint->count == ncookies) > { > full_buffer= pending_checkpoint; > pending_checkpoint= NULL; > @@ -8839,7 +8838,7 @@ int TC_LOG_MMAP::unlog(ulong cookie, my_xid xid) > if (count == 0) > { > uint i; > - for (i= 0; i < sizeof(pending->cookies)/sizeof(pending->cookies[0]); ++i) > + for (i= 0; i < tc_log_page_size / sizeof(my_xid); ++i) I wonder why you didn't use ncookies here :) > delete_entry(pending->cookies[i]); > my_free(pending); > } Regards, Sergei

2 1

Re: [Maria-developers] MDEV-7273 - 10.1 fails to start up during tc_log initializations on PPC64
by Sergei Golubchik 26 Dec '14

26 Dec '14

Hi, Sergey! Yes, that's what I had in mind, thanks! There are few rough edges, see below. Ok to push after you fix that, there's no need for another review round. On Dec 26, svoj(a)mariadb.org wrote: > revision-id: 72e58dc2b1a0bb859e918de0ff820d5a2a92dc32 > parent(s): db89dd3a8f7b0d868946d25ba98c6f88612d309a > committer: Sergey Vojtovich > branch nick: 10.1 > timestamp: 2014-12-26 17:03:45 +0400 > message: > > MDEV-7273 - 10.1 fails to start up during tc_log initializations on PPC64 > > log-tc-size is 24K by default. Page size is 64K on PPC64. But log-tc-size > must be at least 3 x page size. This is enforced by TC_LOG_MMAP::open() > with a comment: to guarantee non-empty pool. > > This all makes server not startable in default configuration on PPC64. > > Autosize log-tc-size, so that it's min value= page size * 6, default > value= page size * 6, block size= page size. > > --- > mysql-test/r/mysqld--help.result | 1 - > mysql-test/suite/sys_vars/r/log_tc_size_basic.result | 4 ++++ > .../suite/sys_vars/r/sysvars_server_notembedded.result | 14 ++++++++++++++ > mysql-test/suite/sys_vars/t/log_tc_size_basic.test | 5 +++++ > mysql-test/t/mysqld--help.test | 2 +- > sql/log.cc | 5 ++--- > sql/log.h | 3 +-- > sql/mysqld.cc | 7 +------ > sql/sys_vars.cc | 11 +++++++++++ > 9 files changed, 39 insertions(+), 13 deletions(-) > > diff --git a/mysql-test/suite/sys_vars/t/log_tc_size_basic.test b/mysql-test/suite/sys_vars/t/log_tc_size_basic.test > new file mode 100644 > index 0000000..1fce212 > --- /dev/null > +++ b/mysql-test/suite/sys_vars/t/log_tc_size_basic.test > @@ -0,0 +1,5 @@ > +--error ER_INCORRECT_GLOBAL_LOCAL_VAR,ER_UNKNOWN_SYSTEM_VARIABLE > +SET GLOBAL log_tc_size=1; > + > +--error ER_INCORRECT_GLOBAL_LOCAL_VAR,ER_UNKNOWN_SYSTEM_VARIABLE do we ever build on platforms without mmap? if the answer is "no" - you don't need ER_UNKNOWN_SYSTEM_VARIABLE here > +SET SESSION log_tc_size=1; > diff --git a/mysql-test/suite/sys_vars/r/sysvars_server_notembedded.result b/mysql-test/suite/sys_vars/r/sysvars_server_notembedded.result > index 006b9a9..5491169 100644 > --- a/mysql-test/suite/sys_vars/r/sysvars_server_notembedded.result > +++ b/mysql-test/suite/sys_vars/r/sysvars_server_notembedded.result > @@ -1757,6 +1757,20 @@ NUMERIC_BLOCK_SIZE NULL > ENUM_VALUE_LIST innodb,query_plan,explain > READ_ONLY NO > COMMAND_LINE_ARGUMENT REQUIRED > +VARIABLE_NAME LOG_TC_SIZE > +SESSION_VALUE NULL > +GLOBAL_VALUE 24576 > +GLOBAL_VALUE_ORIGIN AUTO > +DEFAULT_VALUE 24576 > +VARIABLE_SCOPE GLOBAL > +VARIABLE_TYPE BIGINT UNSIGNED > +VARIABLE_COMMENT Size of transaction coordinator log. > +NUMERIC_MIN_VALUE 24576 > +NUMERIC_MAX_VALUE 18446744073709551615 > +NUMERIC_BLOCK_SIZE 4096 > +ENUM_VALUE_LIST NULL > +READ_ONLY YES > +COMMAND_LINE_ARGUMENT REQUIRED if the answer is "yes" - this test will fail there. even if the answer is "no", the variable value is architecture dependent. But it's good to see all stable variable metadata (e.g. that GLOBAL_VALUE_ORIGIN is AUTO). > VARIABLE_NAME LOG_WARNINGS > SESSION_VALUE 1 > GLOBAL_VALUE 1 > diff --git a/sql/log.cc b/sql/log.cc > index 66e1426..6d6512e 100644 > --- a/sql/log.cc > +++ b/sql/log.cc > @@ -8793,7 +8792,7 @@ int TC_LOG_MMAP::unlog(ulong cookie, my_xid xid) > mysql_mutex_lock(&LOCK_pending_checkpoint); > if (pending_checkpoint == NULL) > { > - uint32 size= sizeof(*pending_checkpoint); > + uint32 size= sizeof(*pending_checkpoint) + tc_log_page_size / sizeof(my_xid); that's not enough. There are further sizeof's, like if (pending_checkpoint->count == sizeof(pending_checkpoint->cookies) / sizeof(pending_checkpoint->cookies[0])) and for (i= 0; i < sizeof(pending->cookies)/sizeof(pending->cookies[0]); ++i) I've found these two by looking for "sizeof" in the Kristian's patch that introduced pending_checkpoint. There can be more sizeofs added later. > if (!(pending_checkpoint= > (pending_cookies *)my_malloc(size, MYF(MY_ZEROFILL)))) > { > diff --git a/sql/sys_vars.cc b/sql/sys_vars.cc > index fb11377..bfaf7d7 100644 > --- a/sql/sys_vars.cc > +++ b/sql/sys_vars.cc > @@ -5202,3 +5202,14 @@ static Sys_var_mybool Sys_strict_password_validation( > "that cannot be validated (passwords specified as a hash)", > GLOBAL_VAR(strict_password_validation), > CMD_LINE(OPT_ARG), DEFAULT(TRUE), NO_MUTEX_GUARD, NOT_IN_BINLOG); > + > +#ifdef HAVE_MMAP > +static Sys_var_ulong Sys_log_tc_size( > + "log_tc_size", > + "Size of transaction coordinator log.", > + READ_ONLY GLOBAL_VAR(opt_tc_log_size), > + CMD_LINE(REQUIRED_ARG), > + VALID_RANGE(my_getpagesize() * 6, ULONG_MAX), > + DEFAULT(my_getpagesize() * 6), > + BLOCK_SIZE(my_getpagesize())); I think, min value could be my_getpagesize()*3 ? > +#endif Regards, Sergei

2 1

Re: [Maria-developers] MDEV-7273 - 10.1 fails to start up during tc_log initializations on PPC64
by Sergei Golubchik 25 Dec '14

25 Dec '14

Hi, Sergey! On Dec 17, svoj(a)mariadb.org wrote: > revision-id: 4a469b0ed68bb341e1a1e1ba540a45a3b6ff3ef9 > parent(s): ea8862aa9ffa1cfac9a99bbe0fe9b539f12b9a19 > committer: Sergey Vojtovich > branch nick: 10.1 > timestamp: 2014-12-17 13:55:01 +0400 > message: > > MDEV-7273 - 10.1 fails to start up during tc_log initializations on PPC64 > > log-tc-size is 24K by default. Page size is 64K on PPC64. But log-tc-size > must be at least 3 x page size. This is enforced by TC_LOG_MMAP::open() > with a comment: to guarantee non-empty pool. > > This all makes server not startable in default configuration on PPC64. > > When creating tc log, adjust it's size to max(log-tc-size, page size * 6). > > --- > sql/log.cc | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/sql/log.cc b/sql/log.cc > index 24e5283..a2fd70f 100644 > --- a/sql/log.cc > +++ b/sql/log.cc > @@ -8316,7 +8316,7 @@ int TC_LOG_MMAP::open(const char *opt_name) > O_RDWR, MYF(MY_WME))) < 0) > goto err; > inited=1; > - file_length= opt_tc_log_size; > + file_length= MY_MAX(tc_log_page_size * 2 * 3, opt_tc_log_size); > if (mysql_file_chsize(fd, file_length, 0, MYF(MY_WME))) > goto err; > } I don't particularly like silent adjustments of user-specified values. And anyway, as you found out, this fix is not sufficient. I'd suggest the following: 1. Create a sysvar for log-tc-size 2. invoke my_getpagesize() not from log.cc, but from mysqld.cc, and set min and default values for log-tc-size appropriately, don't forget to use SYSVAR_AUTOSIZE() 3. change the code not to depend on the TC_LOG_PAGE_SIZE constant. - it's basically used in only two places now: to set a default for TC_LOG_MIN_SIZE (but you'll fix it step 2) and in kristian's code for group commit, cookies[] array. - don't mind TC_LOG_PAGE_SIZE being used in TC_LOG_BINLOG, it means nothing there, any reasonable number will do, I suppose. - so basically you only need to remove TC_LOG_PAGE_SIZE from cookies[] array. That is, you can define it as 'ulong cookies[1]' and take care to fix all sizeof() in the code to add the real size there. 4. Then you can remove that assert. The tc log page will be of my_getpagesize() size and log-tc-size will be N*my_getpagesize() where N>3. Regards, Sergei

1 0

[Maria-developers] JIRA ping
by Jonas Oreland 23 Dec '14

23 Dec '14

Hi Kristian, I light of recent "I did see any of your comments until by accident just now", I now mail you directly that I updated both MDEV-162 and MDEV-7257. I want (review) comments. /Jonas

2 9

[Maria-developers] Review of Debian packaging file listing?
by Otto Kekäläinen 22 Dec '14

22 Dec '14

Hello! I recently updated my build scripts to automatically generate a file listing for each commit in in the Debian packaging "pool" I have. Here is a recent one: http://labs.seravo.fi/~otto/mariadb-repo/mariadb-10.0-sid-amd64/filelist-ec… It is now very easy to review and track what files are in what package. Could somebody please quickly review that file and tell me if you spot: - any files in wrong package and should be moved? - any package that has too few files and maybe something missing? - any file attributes that might be wrong or insecure? - should any of the MariaDB plugins deserve to be shipped as a separate package? (apart from current connect and oqgraph) -- Check out our blog at http://seravo.fi/blog and follow @ottokekalainen

4 5

[Maria-developers] Report from Debian/Ubuntu MySQL packaging sprint in London in December
by Otto Kekäläinen 21 Dec '14

21 Dec '14

Hello! I thought I'd brief the readers of this mailing list about the packaging sprint in London last week. It was organized by Canonical and in addition to Canonical staff and me there where also people from Oracle and Percona. We mostly worked on syncing the MySQL/MariaDB/Percona server packaging and a little bit also on creating a unified Galera package. We made good progress and all results have been pushed to public repos. I also recently updated the pages https://wiki.debian.org/Teams/MySQL/MariaDB and https://wiki.debian.org/Teams/MySQL/Galera to reflect the current status. One big technical change that was discussed and planned is the decoupling of the config and data directories. The plan is that in the future no MySQL variant in particular would anymore own the /etc/mysql/my.cnf file but instead each variant would register their own config file at installation time by using the Debian update-alternatives mechanism. So in future there would be /etc/mysql/mariadb.conf, mysql.conf, percona.conf etc and some more snippets in /etc/mysql/mariadb.conf.d/.. etc. The plan is not complete nor tested, but there are preliminary changes in the mysql-common package to provide the config file registration hook and in a preliminary Percona Debian package to use the new config file registration facility. As the config file defines the data directory, successful decoupling of config files would make it easy to also decouple the data directories so that if the user installs/removes/upgrades any of the MySQL variants they would not mess up their data accidentally and the installation scripts would help users to dump/migrate/copy/move their data in the common scenarios. -- Check out our blog at http://seravo.fi/blog and follow @ottokekalainen

2 1

[Maria-developers] CREATE USER gotcha
by Alexander Barkov 19 Dec '14

19 Dec '14

Hi Ian, I think there is a CREATE USER related gotcha which is not described in the manual. CREATE USER allows to create multiple users at the same time: CREATE USER user1, user2; A reader of https://mariadb.com/kb/en/mariadb/documentation/sql-commands/account-manage… might think that the above query is a short synonym for: CREATE USER user1; CREATE USER user2; But in fact it is not always true. The difference can happen during replication. A "CREATE USER" statement is binlogged as a single shot, but only if at least one user was really created. Suppose (on master) user1 already exists, while user2 does not. "CREATE USER user1, user2" will be binary logged as is, while two independent "CREATE USER user1" and "CREARE USER user2" statements will be logged only as "CREATE USER user2". The CREATE statement for user1 won't be logged, because it failed on master! Now, suppose non of the users exist on slave. In case of "CREATE USER user1, user2;" both will be created on slave. In case of two independent queries, only "user2" will be created on slave.

4 6

[Maria-developers] Syntax for parallel replication
by Kristian Nielsen 18 Dec '14

18 Dec '14

Hi Serg, Can you help me with suggestions/comments for the following proposal for how to do configuration for MDEV-6676, speculative parallel replication? I am not too happy about how things are in 10.0. There, there is a single option --slave-parallel-threads=N. If N>0, then parallel replication is enabled, else not. One problem is that this makes it not configurable per multi-source master connection. Another problem is that there are two possible mechanisms for parallelisation, group-commit based and domain_id based; currently, one can enable none of them or both, but not just one of them. MDEV-6676 will introduce at least one other mechanism, which seems to make it essential to make a better way to configure this. Now, ideally, there would not be any configuration at all. The server would just run things in parallel when possible and to the degree desirable. However, I think there are some reasons that we need to allow the user this configurability: - Parallel replication is still a somewhat experimental feature, so it seems too risky to enable it by default. Also, it doesn't really seem possible for the server to automatically set the best number of threads to use, with current implementation (or possibly any implementation). - When replicating with non-transactional updates, or in non-gtid mode, slave state is not crash safe. This is true in non-parallel replication also, but in parallel replication, the problem seems amplified, as there may be multiple transactions in progress at the time of a crash, complicating possible manual recovery. This also suggests that parallel replication must be configurable. - When using domain-based parallel replication, the user is responsible for ensuring that independent domains are non-conflicting and can be replicated out-of-order wrt. each other. So if replication domains are used, but this property is not guaranteed, then domain-based parallel replication need to be configurable, or parallel replication cannot be used at all. - The new speculative replication feature in MDEV-6676 is not always guaranteed to be a win - in some workloads, where there are many conflicts between successive transactions, excessive rollback could cause it to be less efficient than not using it. Again, this suggests it needs to be configurable. So given this, I came up with the following idea for syntax: CHANGE MASTER TO PARALLEL_MODE=(domain,groupcommit,transactional,waiting) Each of the four keywords in the parenthesis is optional. "domain" enables domain-based parallelisation, where each replication domain is treated independently. "groupcommit" enables the non-speculative mode, where only transactions that group-committed together on the master are applied in parallel on the slave. "transactional" enables the speculative mode, where all transactional DML is optimistically tried in parallel, and then in case of conflict a rollback and retry is done. "groupcommit" and "transactional" are mutually exclusive, at most one of them can be specified. The default would be (domain,groupcommit) to be back-wards compatible with 10.0. If slave_parallel_thread=0, then no parallel apply will happen even if PARALLEL_MODE is non-empty. If slave_parallel_thread>0 but PARALLEL_MODE is empty (PARALLEL_MODE=()), then again no parallel apply will be done. The "waiting" option is not essential to add, we could remove it, I put it in because there were already a number of options so it seemed to cause no harm. The idea is that on the master, we detect if transaction T2 had to do a row lock wait on transaction T1. If so, it seems likely that a similar conflict could occur on the slave, so we will not run T2 in parallel with T1; instead we will let T2 wait for T1 to commit before T2 is started. The "waiting" option could be enabled by a user to disable this check, enabling even more aggressive parallelisation. I am not sure if it is worth it to have this configurable though, comments welcome. I have not checked how hard it will be to implement the new syntax. We do not have any similar multi-option CHANGE MASTER elements, as far as I know, but it is similar to ENUM system variables. And we already have a IGNORE_SERVER_IDS syntax with comma-separated list within parenthesis. So hopefully not too hard to do, and somewhat consistent with existing syntax. So what do you think? Is it a reasonable syntax? Any comments, or suggestions for better way to do it? Thanks, - Kristian.

5 23

Re: [Maria-developers] Rev 4383: MDEV-6916:Upgrade from MySQL to MariaDB breaks already created views
by Sergei Golubchik 17 Dec '14

17 Dec '14

Hi, Sanja! Comments about the code, see below. Comments about the whole approach - see my other email. On Dec 07, sanja(a)askmonty.org wrote: > At file:///home/bell/maria/bzr/work-maria-5.5-MDEV-6916/ > > ------------------------------------------------------------ > revno: 4383 > revision-id: sanja(a)askmonty.org-20141207094825-tx163unk3mm4gkql > parent: sanja(a)askmonty.org-20141203234414-63zkpeq9eb0n9lc1 > committer: sanja(a)askmonty.org > branch nick: work-maria-5.5-MDEV-6916 > timestamp: Sun 2014-12-07 10:48:25 +0100 > message: > MDEV-6916:Upgrade from MySQL to MariaDB breaks already created views > (SQL-level commands to support view update) > === modified file 'mysql-test/r/view.result' > --- a/mysql-test/r/view.result 2014-12-03 22:37:59 +0000 > +++ b/mysql-test/r/view.result 2014-12-07 09:48:25 +0000 > @@ -5398,6 +5398,29 @@ DROP VIEW v1; > DROP TABLE t1, t2; > create view v1 as select 1; > drop view v1; > +# > +#MDEV-6916:Upgrade from MySQL to MariaDB breaks already created views > +# (SQL command for upgrade check) > +# > +create table t1 (a int); > +create algorithm=temptable view v1 as select a from t1; > +show create view v1; > +View Create View character_set_client collation_connection > +v1 CREATE ALGORITHM=TEMPTABLE DEFINER=`root`@`localhost` SQL SECURITY DEFINER VIEW `v1` AS select `t1`.`a` AS `a` from `t1` latin1 latin1_swedish_ci > +check table v1 force mariadb upgrade; > +Table Op Msg_type Msg_text > +test.v1 check note View algorithm swap > +show create view v1; > +View Create View character_set_client collation_connection > +v1 CREATE ALGORITHM=MERGE DEFINER=`root`@`localhost` SQL SECURITY DEFINER VIEW `v1` AS select `t1`.`a` AS `a` from `t1` latin1 latin1_swedish_ci No, I don't like that. FORCE or not, upgrading of a view should not blindly change the alorithm. This functionality is created to solve a specific problem. So when a view has mariadb-version field we know with certainty that the algorithm is correct and it should *not* be changed with or without FORCE. In the test you'll need to copy mysql-created view frm (from std_data). Addendum: ok, I've read till the end. See my comment in view_check() function. > +check table v1 force mariadb upgrade; Also, I don't like that CHECK changes the algorithm of the view. For tables CHECK only performs the *check* (and updates the version in the frm). It's REPAIR that does the change. To be consistent with the behavior for tables, CHECK ... FOR UPGRADE should only report an error, like "view needs upgrading" and a user should then use REPAIR to actually fix the view. > +Table Op Msg_type Msg_text > +test.v1 check note View algorithm swap > +show create view v1; > +View Create View character_set_client collation_connection > +v1 CREATE ALGORITHM=TEMPTABLE DEFINER=`root`@`localhost` SQL SECURITY DEFINER VIEW `v1` AS select `t1`.`a` AS `a` from `t1` latin1 latin1_swedish_ci > +drop view v1; > +drop table t1; > # ----------------------------------------------------------------- > # -- End of 5.5 tests. > # ----------------------------------------------------------------- > > === modified file 'sql/share/errmsg-utf8.txt' > --- a/sql/share/errmsg-utf8.txt 2014-05-27 06:45:01 +0000 > +++ b/sql/share/errmsg-utf8.txt 2014-12-07 09:48:25 +0000 > @@ -6565,3 +6565,5 @@ ER_QUERY_EXCEEDED_ROWS_EXAMINED_LIMIT > ER_NO_SUCH_TABLE_IN_ENGINE 42S02 > eng "Table '%-.192s.%-.192s' doesn't exist in engine" > swe "Det finns ingen tabell som heter '%-.192s.%-.192s' i handlern" > +ER_VIEW_UPGRADE_IS_DONE > + eng "View algorithm swap" No way, you cannot add a new error message in 5.5 because the next major branch (10.0) is already GA. > === modified file 'sql/sql_admin.cc' > --- a/sql/sql_admin.cc 2014-05-03 16:12:17 +0000 > +++ b/sql/sql_admin.cc 2014-12-07 09:48:25 +0000 > @@ -314,7 +314,8 @@ static bool mysql_admin_table(THD* thd, > HA_CHECK_OPT *), > int (handler::*operator_func)(THD *, > HA_CHECK_OPT *), > - int (view_operator_func)(THD *, TABLE_LIST*)) > + int (view_operator_func)(THD *, TABLE_LIST*, > + HA_CHECK_OPT *)) > { > TABLE_LIST *table; > SELECT_LEX *select= &thd->lex->select_lex; > @@ -521,7 +522,7 @@ static bool mysql_admin_table(THD* thd, > ER_CHECK_NO_SUCH_TABLE, ER(ER_CHECK_NO_SUCH_TABLE)); > /* if it was a view will check md5 sum */ > if (table->view && > - view_checksum(thd, table) == HA_ADMIN_WRONG_CHECKSUM) > + view_check(thd, table, check_opt) == HA_ADMIN_WRONG_CHECKSUM) > push_warning(thd, MYSQL_ERROR::WARN_LEVEL_WARN, > ER_VIEW_CHECKSUM, ER(ER_VIEW_CHECKSUM)); > if (thd->stmt_da->is_error() && > @@ -536,7 +537,7 @@ static bool mysql_admin_table(THD* thd, > if (table->view) > { > DBUG_PRINT("admin", ("calling view_operator_func")); > - result_code= (*view_operator_func)(thd, table); > + result_code= (*view_operator_func)(thd, table, check_opt); > goto send_result; > } > > @@ -849,6 +850,14 @@ send_result_message: > system_charset_info); > break; > } > + case HA_ADMIN_VIEW_UPGRADE_IS_DONE: > + { > + protocol->store(STRING_WITH_LEN("note"), system_charset_info); > + protocol->store(ER(ER_VIEW_UPGRADE_IS_DONE), > + strlen(ER(ER_VIEW_UPGRADE_IS_DONE)), > + system_charset_info); > + break; > + } Don't do that. First, you cannot have ER_VIEW_UPGRADE_IS_DONE. Second, it's simply unnecessary. Return HA_ADMIN_OK from the view_operator_func() and use push_warning inside view_operator_func() to say that the algorithm was changed. > case HA_ADMIN_NEEDS_UPGRADE: > case HA_ADMIN_NEEDS_ALTER: > @@ -1071,7 +1080,7 @@ bool Check_table_statement::execute(THD > > res= mysql_admin_table(thd, first_table, &m_lex->check_opt, "check", > lock_type, 0, 0, HA_OPEN_FOR_REPAIR, 0, > - &handler::ha_check, &view_checksum); > + &handler::ha_check, &view_check); > > m_lex->select_lex.table_list.first= first_table; > m_lex->query_tables= first_table; > > === modified file 'sql/sql_view.cc' > --- a/sql/sql_view.cc 2014-12-03 23:44:14 +0000 > +++ b/sql/sql_view.cc 2014-12-07 09:48:25 +0000 > @@ -729,9 +729,29 @@ err: > DBUG_RETURN(res || thd->is_error()); > } > > +static void make_view_filename(LEX_STRING *dir, char *dir_buff, > + size_t dir_buff_len, > + LEX_STRING *path, char *path_buff, > + size_t path_buff_len, > + LEX_STRING *file, > + TABLE_LIST *view) > +{ > + /* print file name */ > + dir->length= build_table_filename(dir_buff, dir_buff_len - 1, > + view->db, "", "", 0); > + dir->str= dir_buff; > + > + path->length= build_table_filename(path_buff, path_buff_len - 1, > + view->db, view->table_name, reg_ext, 0); > + path->str= path_buff; > + > + file->str= path->str + dir->length; > + file->length= path->length - dir->length; > +} > + > > /* number of required parameters for making view */ > -static const int required_view_parameters= 15; > +static const int required_view_parameters= 16; eh? why? > /* > table of VIEW .frm field descriptors > @@ -795,6 +815,65 @@ static File_option view_parameters[]= > static LEX_STRING view_file_type[]= {{(char*) STRING_WITH_LEN("VIEW") }}; > > > +int mariadb_fix_view(THD *thd, TABLE_LIST *view) > +{ > + char dir_buff[FN_REFLEN + 1], path_buff[FN_REFLEN + 1]; > + LEX_STRING dir, file, path; > + DBUG_ENTER("mariadb_fix_view"); > + > + if (view->algorithm == VIEW_ALGORITHM_UNDEFINED) > + DBUG_RETURN(HA_ADMIN_OK); > + > + make_view_filename(&dir, dir_buff, sizeof(dir_buff), > + &path, path_buff, sizeof(path_buff), > + &file, view); > + /* init timestamp */ > + if (!view->timestamp.str) > + view->timestamp.str= view->timestamp_buffer; > + > + /* check old .frm */ > + { > + char path_buff[FN_REFLEN]; > + LEX_STRING path; > + File_parser *parser; > + > + path.str= path_buff; > + fn_format(path_buff, file.str, dir.str, "", MY_UNPACK_FILENAME); > + path.length= strlen(path_buff); > + if (access(path.str, F_OK)) > + DBUG_RETURN(HA_ADMIN_INVALID); > + > + if (!(parser= sql_parse_prepare(&path, thd->mem_root, 0))) > + DBUG_RETURN(HA_ADMIN_INTERNAL_ERROR); > + > + if (!parser->ok() || !is_equal(&view_type, parser->type())) > + DBUG_RETURN(HA_ADMIN_INVALID); > + } > + > + DBUG_ASSERT(view->algorithm == VIEW_ALGORITHM_MERGE || > + view->algorithm == VIEW_ALGORITHM_TMPTABLE); if() instead of an assert, please. while unlikely, it's possible that the file would be changed between two opens. > + if (view->algorithm == VIEW_ALGORITHM_MERGE) > + view->algorithm= VIEW_ALGORITHM_TMPTABLE; > + else > + view->algorithm= VIEW_ALGORITHM_MERGE; > + > + if (sql_create_definition_file(&dir, &file, view_file_type, > + (uchar*)view, view_parameters)) > + { > + sql_print_error("View '%-.192s'.'%-.192s': algorithm swap error.", "View %`s.%`s: cannot change the algorithm from %s to %s" And push_warning_printf() with the same. Or, rather, only push_warning_printf without sql_print_information. > + view->db, view->table_name); > + DBUG_RETURN(HA_ADMIN_INTERNAL_ERROR); > + } > + sql_print_information("View '%-.192s'.'%-.192s': algorithm swapped to '%s'", "View %`s.%`s: algorithm changed from %s to %s" And push_warning_printf() with the same. Or, rather, only push_warning_printf without sql_print_information. > + view->db, view->table_name, > + (view->algorithm == VIEW_ALGORITHM_MERGE)? > + "MERGE":"TEMPTABLE"); > + > + > + DBUG_RETURN(HA_ADMIN_VIEW_UPGRADE_IS_DONE); > +} > + > + > /* > Register VIEW (write .frm & process .frm's history backups) > > @@ -931,16 +1010,9 @@ static int mysql_register_view(THD *thd, > } > loop_out: > /* print file name */ > - dir.length= build_table_filename(dir_buff, sizeof(dir_buff) - 1, > - view->db, "", "", 0); > - dir.str= dir_buff; > - > - path.length= build_table_filename(path_buff, sizeof(path_buff) - 1, > - view->db, view->table_name, reg_ext, 0); > - path.str= path_buff; > - > - file.str= path.str + dir.length; > - file.length= path.length - dir.length; > + make_view_filename(&dir, dir_buff, sizeof(dir_buff), > + &path, path_buff, sizeof(path_buff), > + &file, view); > > /* init timestamp */ > if (!view->timestamp.str) > @@ -1953,6 +2025,33 @@ int view_checksum(THD *thd, TABLE_LIST * > HA_ADMIN_OK); > } you can make view_checksum() static now. > > +/** > + Check view > + > + @param thd thread handle > + @param view view for check > + @param check_opt check options > + > + @retval HA_ADMIN_OK OK > + @retval HA_ADMIN_NOT_IMPLEMENTED it is not VIEW > + @retval HA_ADMIN_WRONG_CHECKSUM check sum is wrong > +*/ > +int view_check(THD *thd, TABLE_LIST *view, HA_CHECK_OPT *check_opt) > +{ > + int res; > + DBUG_ENTER("view_check"); > + if ((res= view_checksum(thd, view)) != HA_ADMIN_OK) > + DBUG_RETURN(res); > + if (((check_opt->sql_flags & TT_MARIADB_UPGRADE) && > + !view->mariadb_version) || > + ((check_opt->sql_flags & (TT_MARIADB_UPGRADE | TT_FORCE)))) What is the point? Why would one ever want to change the algorithm of the provably created-in-mariadb view and pretend that it's an "upgrade"? To change the algorithm one should use ALTER. Fixing the view after an upgrade is not altering it, it's *fixing after an upgrade*. And views that were created in mariadb don't need to be fixed after an upgrade. No FORCE please. > + { > + DBUG_PRINT("XXX", ("Upgrade")); please either remove this DBUG_PRINT or change it to use a reasonable keyword. > + DBUG_RETURN(mariadb_fix_view(thd, view)); > + } > + DBUG_RETURN(HA_ADMIN_OK); > +} > + > === modified file 'sql/sql_yacc.yy' > --- a/sql/sql_yacc.yy 2014-11-08 18:54:42 +0000 > +++ b/sql/sql_yacc.yy 2014-12-07 09:48:25 +0000 > @@ -7299,6 +7300,10 @@ mi_check_type: > | EXTENDED_SYM { Lex->check_opt.flags|= T_EXTEND; } > | CHANGED { Lex->check_opt.flags|= T_CHECK_ONLY_CHANGED; } > | FOR_SYM UPGRADE_SYM { Lex->check_opt.sql_flags|= TT_FOR_UPGRADE; } > + | MARIADB_SYM UPGRADE_SYM > + { Lex->check_opt.sql_flags|= TT_MARIADB_UPGRADE; } > + | FORCE_SYM MARIADB_SYM UPGRADE_SYM > + { Lex->check_opt.sql_flags|= (TT_MARIADB_UPGRADE | TT_FORCE); } > ; > > optimize: Regards, Sergei

1 0