- developers - lists.mariadb.org

Re: [Maria-developers] MDEV-8356 analysis and ideas
by Sergey Petrunia 29 Jun '15

29 Jun '15

Hi Diwas, Nice analysis. Please find my comments below. On Mon, Jun 29, 2015 at 10:01:56AM +0530, Diwas Joshi wrote: > hello sergey, so i was reading further on MDEV-8356 last night. There are > two main things here: type of sf and return type of sf. Right now the > create function... is giving errors at > > store_failed= store_failed || > table->field[MYSQL_PROC_MYSQL_TYPE]-> > store((longlong)type, TRUE); > > in sf_create_routine(), which is right since we have been been using > TYPE_ENUM_TABLE for m_type so far and mysql.proc takes > enum('FUNCTION','PROCEDURE') for type. This can be solved by either > changing this enum in mysql.proc definition and including TABLE, or storing > table_valued functions as FUNCTION as well. I think first option is better, > including TABLE in the enum, since this will help us in distinguishing > these functions while executing later if needed. It is not clear for me which option is better. On one hand, there is your argument that adding 'TABLE' into the enum will make it easier to tell table functions from regular function. On the other hand, try grepping for 'TYPE_ENUM_FUNCTION' in sql/*.{h,cc}. You find a lot of code that checks if m_type == TYPE_ENUM_FUNCTION. You will need to review each check to see what should be done for TYPE_ENUM_FUNCTION. If we can't decide towards one option or another, let's use whatever you prefer right now. If we discover that that works poorly, we will be able to reverse the decision later on. > The second thing is about return type of function. Right now > sp_returns_type() method is being used to store the return type of > functions. But in our case we have a list of create_fields: > List<Create_field> m_cols_list keeping list of attribute names and their > types for the table. We need to store all these, for this we can either > alter mysql.proc definition to make a field appropriate for storing this > list. The type of the field storing return type right now is longblob which > as i read further on it is a binary large object that can hold a variable > amount of data and are treated as binary strings (byte strings). Maybe we > can try to find a way to store all the attribute information in this field > only, maybe putting in some delimiters between different field types or > some other way since after all its a string. I am still not sure about this > problem. I tried creating this function: CREATE FUNCTION f2(a INT, b VARCHAR(11)) RETURNS VARCHAR(32) DETERMINISTIC BEGIN set @a=3; RETURN @a; END| and mysql.proc.returns has: returns: varchar(32) CHARSET latin1 So, it is a text defition of the return type. It seems natural if for table functions we Is it a problem to do similar thing for Table functions? e.g for a function CREATE FUNCTION f1(a INT, b VARCHAR(11)) RETURNS TABLE t1(id INT, name VARCHAR(11)) BEGIN set @a=3; END| store 'TABLE t1(id INT, name VARCHAR(11)' ? I guess it depends on where the value of mysql.proc.returns is used. Please research this. > > Also, another thing is to store the table alias m_table_alias, because > queries inside the function will be using it. There is no field present > specifically for this in mysql.proc. So either we can create one. There is > a field specific_name in it, as i saw in the code is right now storing the > same thing as name field. So maybe we can use this field to store the table > alias. I would prefer that m_table_alias was stored as part of the expression in mysql.proc.returns field. Can we try that option first? If we figure that can't work, we can think of other options. BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] MariaDB multi-source replication testing at Booking.com
by Karoly Nagy 29 Jun '15

29 Jun '15

Dear MariaDB developers, I'm Károly Nagy working for Booking.com currently testing multi-source replication functionality of MariaDB. Kristian suggested I should reach out to you on this mailing list regarding my questions. We're seeing very high and fluctuating mutex contentions while replicating from two sources (Oracle MySQL 5.6) to a single MariaDB slave. You can see that on the graphs below. The spin waits are relatively [1] aligned but the mutex rounds [2] are 5-10 times higher than it is on the two sources combined together and not consistent. The sources have a relatively constant pattern while the target has dips around 2.5k and spikes up to 8k. The os waits are in completely different order of magnitude [3]. The scenario where values were captured: * Multi-source target is replicating the full dataset of `source 2` and a subset of `source 1` (the hot data) - MariaDB 10.0.16 * Both sources are MySQL 5.6 being part of their replication chain as slaves with log_slave_updates * Source 2 is in normal mode - Oracle MySQL 5.6.17 * Source 1 is catching up from a 1 day replication delay - Oracle MySQL 5.6.24 * All the slaves are warm having the buffer pool fully populated Is this behavior expected? Could you give us some insights on why we could see these results? If there is any more information you need please let me know. Thank you for your help in advance! Every metric is on /10 seconds basis. [1] Mutex spin waits [2] Mutex rounds [3] Mutex OS waits Best regards, -- Károly Nagy System engineer Booking.com <http://booking.com/> BV Rembrandt Square Office, Herengracht 597, 1017 CE Amsterdam Direct +31 (0)20 715 8403

3 4

Re: [Maria-developers] [Commits] 132935a: MDEV-8339 - Server crash during table cache eviction
by Sergei Golubchik 27 Jun '15

27 Jun '15

Hi, Sergey! On Jun 25, svoj(a)mariadb.org wrote: > revision-id: 132935a3ef6578f5fabc765ff6ac966cd3798819 > parent(s): 1f4a89b10ca8df61ec51fcc87e1203dc72579759 > committer: Sergey Vojtovich > branch nick: mariadb > timestamp: 2015-06-25 14:21:16 +0400 > message: > > MDEV-8339 - Server crash during table cache eviction > > TDC_element::free_tables_back() had pre-lfhash leftover code, which referenced > TDC_element::free_tables via TDC_element::share. This is not correct as share > may be NULL (newly inserted or to be removed), instead access free_tables > directly. ok to push Regards, Sergei

1 0

[Maria-developers] MDEV-8214 Asian MB2 charsets: compare broken bytes as "greater than any non-broken character"
by Alexander Barkov 26 Jun '15

26 Jun '15

Hi Sergei, For simplicity I decided to split: MDEV-8036 Fix all collations to compare broken bytes as "greater than any non-broken character" into sub-tasks. Please review a patch for the first sub-task MDEV-8214. The overall plan is: - MDEV-8214, for the Asian charsets with mbmaxlen=2 - MDEV-8215, for the Asian charsets with mbmaxlen=3, based on MDEV-8214 - MDEV-8036, the rest (Unicode character sets) Thanks.

2 4

[Maria-developers] MDEV-7502 Automatic provisioning of slave - error reporting
by Martin Kaluznik 26 Jun '15

26 Jun '15

Hello Kristian, with general idea working (sending of database / table structures + table data), my next plan is to improve / introduce error handling in existing code along with error handling on slave side (ddl events not from provisioning, ignore errors when updating not yet existing rows, ...). What are general guidelines for introduction of new error codes? Is there anything else I need to do except adding it to /sql/share/errmsg-utf8.txt ? I would create only one new error code - ER_PROVISIONING_FAILED - and place specific messages inside formatted text - same way as ER_MASTER_FATAL_ERROR_READING_BINLOG works. It doesn't go well with translations, but... you can decide. Maybe create different error codes at least for errors, where we are expecting user to fix them - table without index, statement mode. Also, how to tell slave from master that provisioning finished? Is introduction of new error code ER_PROVISIONING_DONE acceptable solution? The error itself would never produce error message in logs. Alternatively I can reuse / create similar event to Stop_log_event, note - provisioning is completed after SQL thread processes whole relay log, not when IO thread receives this event, so even reuse would require additional code. And side question, is there any log event other than Query_log_event which can produce ddl statement? Martin

2 1

Re: [Maria-developers] [Commits] 2cf747e: MDEV-8219: enforce_storage_engine cannot be set globally
by Sergei Golubchik 26 Jun '15

26 Jun '15

Hi, Jan! On Jun 26, Jan Lindström wrote: > revision-id: 2cf747e885603122b2adbefc131516cd12dc953c > parent(s): d3b7eb7b99ef7dae79c20b5beb63acdfdf6d8046 > committer: Jan Lindström > branch nick: 10.1-bugs > timestamp: 2015-06-26 08:56:50 +0300 > message: > > MDEV-8219: enforce_storage_engine cannot be set globally > > Change session only variable enforce_storage_engine to be > session variable and make sure that also global value > is used if session variable is not set. Ok to push, thanks! Regards, Sergei

1 0

[Maria-developers] Please review: MDEV-7824 [Bug #68041] Zero date can be inserted in strict no-zero mode through a default value
by Alexander Barkov 25 Jun '15

25 Jun '15

Hi Sergei, Please review a patch for mdev-7824. It's based on a MySQL patch for http://bugs.mysql.com/bug.php?id=68041 and is a blocker for: MDEV-3929 Add full support for auto-initialized/updated timestamp and datetime Thanks.

2 9

[Maria-developers] MDEV-6877 - binlog_row_image implementation advice
by Vicențiu Ciorbaru 25 Jun '15

25 Jun '15

Hi Sergei! This past week I've done quite a bit of reading regarding the replication code and how the binlog is used by both mysqlbinlog and the slaves in order to do replication. I think I now have a good enough understanding of the flow of the code for both MySQL and MariaDB to actually start coding the feature into MariaDB. I do have a couple of unknowns still. I don't quite understand what's the use for the TABLE_MAP event. My intuition tells me its some sort of id we assign to a table name (test.t1 for example) that gets used afterwards in the binlog row events. We seem to be using a bitmap of columns that we want to log into the binlog. MySQL does extend the TABLE class to flip the bits according to the binlog_row_image variable. That seems like the first place to start with the addition. I could not find any other "hidden" logic that we use to choose how we log row operations in the binlog. What I would like to know is if there are any other pitfalls that I am supposed to watch out for, or if there is anything else I may be missing regarding the logging part. When it comes to actually doing the replication, I concluded that Log_event::do_apply_event is the place where we begin translating the event from the binlog to an SQL statement. Another concern that I have is how to test the implementation. Right now, I run the test, get the binlog file and print it with mysqlbinlog and see if the output is what I expect. Do we have something within mysql-test-run that automates this? Regards, Vicentiu

2 7

Re: [Maria-developers] [Commits] 9f00950: MDEV-7631 : Invalid WSREP_SST rows appear in mysqld-bin.index file
by Sergei Golubchik 25 Jun '15

25 Jun '15

Hi, Nirbhay! On Jun 24, Nirbhay Choubey wrote: > revision-id: 9f00950d2ad9be5533e66a5fa87a795a9574585f > parent(s): 0f4478105f5027ce589a0f08c6019d906d632029 > committer: Nirbhay Choubey > branch nick: 10.0-galera.b7631 > timestamp: 2015-06-24 23:25:22 -0400 > message: > > MDEV-7631 : Invalid WSREP_SST rows appear in mysqld-bin.index file > > Try not to release 0 (STDIN_FILENO) file descriptor as it can > incorrectly get reused by streams opened later. This really deserved a more descriptive comment. There's nothing wrong when fd 0 is reused by other streams. The problem was that because of a bug (arguably) in old glibc in CentOS6/RHEL6 freopen remapped stdout/stderr to fd 0 and 1. Which is, on itself, is not a problem either, but when mysqld forked other processes, these children had no access to stdout and stderr. it's too late to recommit now and, luckily, MDEV-7631 has all the information about this issue. Regards, Sergei

1 0

Re: [Maria-developers] [Commits] aff5606: Add close-on-exec flag to open(), socket(), accept() & fopen().
by Sergei Golubchik 24 Jun '15

24 Jun '15

Hi, Nirbhay! On Jun 24, Nirbhay Choubey wrote: > revision-id: aff560696cac5e081d19f4460ccd9c906ddc9d84 > parent(s): 70714d3597ec9ffa742cc9e173d4ad32f294cc51 > committer: Nirbhay Choubey > branch nick: 10.0-galera.b7631 > timestamp: 2015-06-24 11:58:11 -0400 > message: > > Add close-on-exec flag to open(), socket(), accept() & fopen(). > > @@ -1026,7 +1026,8 @@ static inline void inline_mysql_socket_register( > (&state, socket_listen.m_psi, PSI_SOCKET_CONNECT, (size_t)0, src_file, src_line); > > /* Instrumented code */ > - socket_accept.fd= accept(socket_listen.fd, addr, &addr_length); > + socket_accept.fd= accept4(socket_listen.fd, addr, &addr_length, > + SOCK_CLOEXEC); Note that, according to the manpage, "accept4() is a nonstandard Linux extension". It needs a configure-time check and ifdef. Alternatively, don't use it at all, do fcntl(FD_CLOEXEC) instead. > /* Instrumentation end */ > if (locker != NULL) > @@ -1036,7 +1037,8 @@ static inline void inline_mysql_socket_register( > #endif > { > /* Non instrumented code */ > - socket_accept.fd= accept(socket_listen.fd, addr, &addr_length); > + socket_accept.fd= accept4(socket_listen.fd, addr, &addr_length, > + SOCK_CLOEXEC); > } Regards, Sergei

2 1