- developers - lists.mariadb.org

[Maria-developers] Updated myisamchk to handle LARGE files
by Michael Widenius 10 May '13

10 May '13

Hi! Here is a patch that should fix your myisamchk issues. This is to be applied to MariaDB 5.5.30 source. Can you verify that this works so I can add it to MariaDB 10.0 ? Some notes: - If you run repair with two --force options, like this: myisamchk --repair --force --force ... Then myisamchk will use the full myisam_sort_buffer_size for keys. Other buffers will be allocated as needed. This means that with this option one will never get the "myisam_sort_buffer_size is too small" error. I also added the option --create-missing-keys, which will create keys just for any disabled keys. Other things: - Better error message for 'myisam_sort_buffer_size is too small' errors. - Somewhat larger buffers as default. - Changed a lot of variables from uint -> ulonglong to try to ensure that big files will not cause any problems. - Removed some checks that limited the sort_buffer_size to 4G. - Fixed code that would cause problems if sort_buffer_size would be > 8G - Fixed printing of longlong variables. - Fixed possible core dump if 'alloc_dynamic()' would fail. - Fixed some possible problems on system where uint would be <= ulong. Hope this helps! Regards, Monty

2 2

Re: [Maria-developers] GSoC projects related on data replication.
by Kristian Nielsen 09 May '13

09 May '13

QIU Shuang <qiush.summer(a)gmail.com> writes: > A quick question: are the codes related to replication in the path `sql/rpl_*' and `sql/repl_*'? Much of the replication code is in: sql/log_event.* sql/slave.* sql/log.* sql/sql_repl.* sql/rpl_* sql/repl* Hope this helps, - Kristian.

1 0

[Maria-developers] how to use TC_LOG_MMAP in MySQL
by nanyi607rao 09 May '13

09 May '13

hi , In http://bazaar.launchpad.net/~maria-captains/maria/10.0-mdev452/revision/3434 you make some change to the class TC_LOG_MMAP,and I want to debug those codes to see how it works.but i don't understand in what conditions will MySQL use the TC_LOG_MMAP.it only use TC_LOG_DUMMY or MYSQL_BIN_LOG depends on the value of opt_bin_log.so how will it use the TC_LOG_MMAP in MySQL. 2013-05-09 nanyi607rao

3 2

[Maria-developers] GTID and failovers with multi-domain replication
by Pavel Ivanov 08 May '13

08 May '13

Kristian, You've mentioned that in future implementation of parallel replication MariaDB will use multiple domains to replicate from one master to a slave. It means for me that I need to understand how multiple domains will work. And I realized that I don't know how the following situation can be handled. I don't know how parallel replication will work, so I'll explain situation in terms of several masters. Let's assume we have server S1 that is master working with domain_id=0, server S2 is master working with domain_id=1, servers S3 and S4 are slaves and replicate from both of these masters, i.e. they have both domains in their databases. Now let's say S1 has last GTID 0-1-100, S2 has last GTID 1-2-100. Before S3 and S4 were able to fully catch up with S1 and S2 power got cut out from S1 and S2. As replication from two masters goes independently it's possible that S3 will have last transactions 0-1-100, 1-2-99 while S4 will have last transactions 0-1-99, 1-2-100. As my masters are out I want either S3 or S4 to temporarily become master. But it looks like I won't be able to do so: S3 won't connect to S4 because S4 doesn't have 0-1-100 and S4 won't connect to S3 because S3 doesn't have 1-2-100. Ideally I'd want for S3 to replicate from S4 in domain 1 and S4 to replicate from S3 in domain 0, and when they are equal in their position I can declare one of them master for both domains. But it looks like there are no tools to do such operation. How would you suggest to resolve such situation? Thank you, Pavel

2 3

[Maria-developers] Documentation about GTID
by Giuseppe Maxia 08 May '13

08 May '13

Hi! I am testing MariaDB 10.0.2, and I am interested in GTID. The only documentation I found is the worklog. Without docs at hand, I tried something: slave1 [localhost] {msandbox} ((none)) > CHANGE MASTER TO master_gtid_pos=''; ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'master_gtid_pos=''' at line 1 slave1 [localhost] {msandbox} ((none)) > CHANGE MASTER TO master_gtid_pos=auto; ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'master_gtid_pos=auto' at line 1 This is supposed to work, according to a blog post and some worklogs, but it doesn't. As for GTID in practice, it seems to be active. slave1 [localhost] {msandbox} ((none)) > show variables like '%gtid%'; +----------------+--------+ | Variable_name | Value | +----------------+--------+ | gtid_domain_id | 0 | | gtid_pos | 0-1-11 | | gtid_seq_no | 0 | +----------------+--------+ 3 rows in set (0.00 sec) slave1 [localhost] {msandbox} ((none)) > use mysql Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Database changed slave1 [localhost] {msandbox} (mysql) > select * from rpl_slave_state; +-----------+--------+-----------+--------+ | domain_id | sub_id | server_id | seq_no | +-----------+--------+-----------+--------+ | 0 | 12 | 1 | 11 | +-----------+--------+-----------+--------+ However, when I run "show slave status", I get: Using_Gtid: 0 Could you provide a pointer to the docs of the expected behavior at this stage? Thanks Giuseppe

9 35

[Maria-developers] MYSQL_TYPE_DECIMAL type
by Zardosht Kasheff 08 May '13

08 May '13

Hello all, Under what circumstances can a table be created with MYSQL_TYPE_DECIMAL as being one of the columns? I thought MYSQL_TYPE_DECIMAL was deprecated pre 5.1. I thought all new tables use MYSQL_TYPE_NEWDECIMAL. We have a user reporting a table he wants to alter has that type. However, when I create that table, I don't get that type. As an aside, we don't support that type, mainly because I thought it was impossible to have a table created with it (I am apparently wrong). Is there a way to ensure that tables undergoing an alter convert that column to MYSQL_TYPE_NEWDECIMAL? Thanks -Zardosht

3 3

[Maria-developers] is this a bug?
by jhx1008 08 May '13

08 May '13

hi all: I have a problem about the type cast when I have read the source of Maria & MySQL I notice that when cast from string into int, Maria & MySQL do it like the C function atoi code: for (ul= 0 ; str < end9 && (ch= (uchar) (*str - '0')) < 10; str++) { ul= ul * 10 + ch; } but cast from hex into int, the code is: code: char *end=(char*) str_value.ptr()+str_value.length(), *ptr=end-min(str_value.length(),sizeof(longlong)); ulonglong value=0; for (; ptr != end ; ptr++) value=(value << 8)+ (ulonglong) (uchar) *ptr; so I do the test blew: create table t(id int auto_increment primary key, a int)engine=innodb; insert into t(a) values('1'); insert into t(a) values(0x31); everything is ok, I get the result: mysql> select * from t; +----+------+ | id | a | +----+------+ | 1 | 1 | | 2 | 49 | +----+------+ the value 0x31 is the ascii of the string '1', but we got the different result after inserting maybe it's not a problem, but in the replication environment, it maybe cause the data of slave not the same as master In the master, when we set the connection characterset into multi-bytes characterset and this characterset may escape with backslash is dangerous (like gbk, cp932) and then we use the prepared statement to do the insert, the string value in the insert must be cast into hex before write into the binlog, so we do the insert(insert into t(a) values('1')) on the master, but in the slave it does the another(insert into t(a) values(0x31)) , the data is not the same. how to reappear: master(binlog format=statement): create table t(id int auto_increment primay key, a int) engine=innodb; set names gbk; prepare stmt from 'insert into t(a) values(?)'; set @a='1'; execute stmt using @a; the result in master is: mysql> select * from t; +----+------+ | id | a | +----+------+ | 1 | 1 | +----+------+ and result in slave is: mysql> select * from t; +----+------+ | id | a | +----+------+ | 1 | 49 | +----+------+ I test the newest release version mariadb-10.0.2 and MySQL-5.6.10, the problem is still exist, is this a bug? 2013-04-28 jhx1008 @netease

4 7

[Maria-developers] MariaDB allows for slave to connect with non-existent GTID
by Pavel Ivanov 07 May '13

07 May '13

Kristian, I've realized that the way slaves are processed now on the master allows them to connect even if they request non-existent GTID. Is it "works as intended" and will be different in the "strict mode" or you didn't want for such things to happen even in non-strict mode? I've attached a test reproducing the problem. It sets up replication 1->2->3, executes transaction on 1, executes transaction on 2, disconnects 3, executes one more transaction on 1, fails over to 2, executes one more transaction on 2. Then when it tries to connect 3 to 1 it should fail because 1 doesn't have the last transaction that 3 has. I agree that this scenario is artificial, but in reality scenario can be e.g. like this: 2 is slave of 1, then it gets disconnected and executes one extra transaction, then backup is taken from 2, then 2 gets restarted and restored to the state of 1, then failover happens and 2 becomes master, then 3 restores from the backup taken earlier from 2 and if all this time 1 and then 2 as masters executed some transactions 3 will be able to connect to 2 even though its last transaction doesn't exist on 2... Pavel

2 3

[Maria-developers] Missing locking around THD::set_db() ?
by Kristian Nielsen 07 May '13

07 May '13

Hi, I was looking at a Valgrind warning in Buildbot (appended below). What happens seems to be this: Thread 1 is running SHOW PROCESSLIST, it grabs the pointer THD::db to the current database of thread 2. Thread 2 then does THD::set_db(), freeing the old THD::db pointer and allocating a new one with the new data. Thread 1 then resumes, doing strdup() of the _old_, now invalid, THD::db pointer, which reads garbage data (or could even segfault if we get really unlucky). This seems like a genuine bug. I see absolutely no locking protecting against this race :-( Any suggestions for how to deal with this? The only locking I see in mysqld_list_processes() is holding LOCK_thread_count, but taking that for every THD::set_db() sounds quite bad for scalability (for example, set_db() is run by every query replicated by a slave thread). Should we use LOCK_thd_data also for THD::db ? Or maybe we could have a fixed buffer in THD for the db? Seems we would need 193 bytes. Then SHOW PROCESSLIST could sometimes show a random mix of old and new db name if we race, but at least we would not read invalid memory or crash due to mummap() ... Any better ideas? - Kristian. ----------------------------------------------------------------------- Original valgrind warning, for reference: rpl.rpl_stop_slave 'mix,xtradb' w5 [ fail ] Found warnings/errors in server log file! Test ended at 2013-04-22 22:05:12 line ==9897== Thread 20: ==9897== Invalid read of size 1 ==9897== at 0x4C2826B: memcpy (mc_replace_strmem.c:878) ==9897== by 0xBAF9E8: strmake_root (my_alloc.c:424) ==9897== by 0x63C74E: mysqld_list_processes(THD*, char const*, bool) (sql_class.h:771) ==9897== by 0x5C8574: mysql_execute_command(THD*) (sql_parse.cc:3149) ==9897== by 0x5CC844: mysql_parse(THD*, char*, unsigned int, Parser_state*) (sql_parse.cc:5759) ==9897== by 0x5CDDC9: dispatch_command(enum_server_command, THD*, char*, unsigned int) (sql_parse.cc:1068) ==9897== by 0x5CE4BD: do_command(THD*) (sql_parse.cc:794) ==9897== by 0x6A4E23: do_handle_one_connection(THD*) (sql_connect.cc:1266) ==9897== by 0x6A4F23: handle_one_connection (sql_connect.cc:1181) ==9897== by 0x8F0027: pfs_spawn_thread (pfs.cc:1015) ==9897== by 0x4E3506F: start_thread (in /lib64/libpthread-2.9.so) ==9897== by 0x62F713C: clone (in /lib64/libc-2.9.so) ==9897== Address 0x1a06a973 is 3 bytes inside a block of size 5 free'd ==9897== at 0x4C257D8: free (vg_replace_malloc.c:446) ==9897== by 0xBB96BB: my_free (my_malloc.c:115) ==9897== by 0x838C86: Query_log_event::do_apply_event(Relay_log_info const*, char const*, unsigned int) (sql_class.h:2846) ==9897== by 0x53904C: apply_event_and_update_pos(Log_event*, THD*, Relay_log_info*) (log_event.h:1230) ==9897== by 0x5446DD: exec_relay_log_event(THD*, Relay_log_info*) (slave.cc:2766) ==9897== by 0x545BD9: handle_slave_sql (slave.cc:3627) ==9897== by 0x8F0027: pfs_spawn_thread (pfs.cc:1015) ==9897== by 0x4E3506F: start_thread (in /lib64/libpthread-2.9.so) ==9897== by 0x62F713C: clone (in /lib64/libc-2.9.so)

4 4

Re: [Maria-developers] [Gsoc] Regex enhancements Project
by Sergei Golubchik 04 May '13

04 May '13

Hi, Sudheera! On Apr 19, Sudheera Palihakkara wrote: > Hi, > I went through other threads on this topic. In one thread you mentioned to > choose a suitable regex library. > > *( Preliminary research - only about chosing a regex library to use in > MariaDB. You should be able to explain why we should use this library and > not some other one.) > > * > What do you mean by "choosing"? don't we have to enhance the exiting regex > library? Or choose from exiting already implemented libraries which are > free to use? sorry if it's a stupid question, but I'm confused. :O Enhancing our old regex library to support all modern features and multiple charsets is complex and bug-prone work. I don't see why we should bother doing it, when there are plenty of regex libraries available. There's PHP's mb_regex, there's prce, and many others too. We'd better just pick one that works better for MariaDB, and put it instead of Henry Spencer's library. Regards, Sergei P.S. Please, don't reply to me only, use reply-to-all, so that your mails appear on the mailing list.

3 9