- developers - lists.mariadb.org

Re: [Maria-developers] MDEV-6179: dynamic columns functions/cast()/convert() doesn't play nice with CREATE/ALTER TABLE in file:///home/bell/maria/bzr/work-maria-5.5-MDEV-6179/
by Sergei Golubchik 08 Nov '14

08 Nov '14

Hi, Sanja! On Nov 02, sanja(a)askmonty.org wrote: > At file:///home/bell/maria/bzr/work-maria-5.5-MDEV-6179/ > > ------------------------------------------------------------ > revno: 4341 > revision-id: sanja(a)askmonty.org-20141102133114-v46li6r2goyzhhp7 > parent: sanja(a)askmonty.org-20141031130729-dya4hci39x6zrv89 > committer: sanja(a)askmonty.org > branch nick: work-maria-5.5-MDEV-6179 > timestamp: Sun 2014-11-02 14:31:14 +0100 > message: > MDEV-6179: dynamic columns functions/cast()/convert() doesn't play > nice with CREATE/ALTER TABLE > > Type attributes (length, dec, charset) which parsed to global > variables now saved and restored during prsing types which can > appeared in a expression. Why did you create a stack in LEX, when bison generated parser already has a stack of states? Also, there's no need to store that in LEX, it doesn't need to live longer than the field_spec rule. By coincidence I've spent a couple of last days rewriting this very part of the parser. I've also fixed this bug without knowing it :) But in 10.1, and the patch is too big to put it in 10.0, let alone 5.5, so it's useless here. Still, I tried to create a smaller patch for this bug in 5.5. Please, review, it's on the commit list. Regards, Sergei

1 0

Re: [Maria-developers] MariaDB 10.0.14 got rid of replication safety warning without warning in release notes?
by Pavel Ivanov 08 Nov '14

08 Nov '14

And here's response from Monty to my last email. And this response together with the whole thread makes me really worried. It alludes to the fact that all MySQL and MariaDB developers before making any changes should read the whole codebase first and then make sure that their changes work the same way and that they conform to the intentions of original code developers even though there's no way they can talk to those original developers. AFAIK that's not how development on big projects with hundreds of participants works. And it really raises the question for me: what kind of quality the product that is developed with such principles in mind can have? Sergei Golubchik, I believe you are the technical leader of MariaDB project. Could you please tell me what you think about this issue? On Tue, Nov 4, 2014 at 6:32 AM, Michael Widenius <michael.widenius(a)gmail.com> wrote: > > Hi! > >>>>>> "Pavel" == Pavel Ivanov <pivanof(a)google.com> writes: > > Pavel> On Sun, Nov 2, 2014 at 3:27 PM, Michael Widenius <monty(a)mariadb.org> wrote: >>>>> This is the same order that keys are presented in SHOW CREATE TABLE >>> > Pavel> Why is it safe to assume that keys have the same order on master and on slave? >>> >>> Primary and Unique keys should always be ordered the same way on >>> master and slave. This is handled by the 'sort_keys()' function that >>> you can find at: sql/sql_table.cc, line 2770. > > Pavel> This function seem to sort only by "type" of the key. If several > Pavel> unique keys have the same "type" according to its classification then > Pavel> they will be in the "original order" as the comment in the function > Pavel> says. And the original order may be different. > > This function ensures that the table will be identical on slave and > master as long as one of the following requirements are fulfilled: > > - You are using the same create table and alter table statements on > both machines. > - You are using the same CREATE TABLE statement as 'show create' shows > on the master. > - You are adding unique keys in the same order on master and slave. > > This is always the case when you are using the replication that MySQL > and MariaDB uses. > > If you are creating tables differently on the slave than on the > master, then you are on your own. > > So in all normal and most abnormal cases, the table will be created > identically on the master and slave, even if the slave is using a > different storage engine, have different columns or have additional > keys than on the master. > > <cut> > >>> I added a test for this to: >>> mysql-test/t/insert.test >>> >>> --echo # >>> --echo # MDEV-5168: Ensure that we can disable duplicate key warnings >>> --echo # from INSERT IGNORE >>> --echo # > > Pavel> This test doesn't check which duplicate key is asserted to execute ON > Pavel> DUPLICATE KEY action. > > This test verifies that we don't get warnings for the tests. > Elinas test verifies which key is used. > >>> Elena added another test for this in >>> mysql-test/suite/storage_engine/insert_with_keys.test >>> >>> See line 142, that start with: >>> --let $create_definition = a $int_indexed_col UNIQUE KEY, b $int_indexed_col UNIQUE KEY, c $int_col > > Pavel> This test seem to check that UPDATE is executed on the correct > Pavel> duplicate key, but it's not executed during regular mtr run. How and > Pavel> when it's executed in MariaDB? > > The storage engine tests are to be used by people that implements > their own storage engine for MariaDB. > > We run the extended tests from time to time to verify storage engines. > We will also run it on some buildbot servers when running releae builds. > The storage engine tests is not something that has to be run for > normal builds. > >>> What is important is that all storage engine should work the same >>> way. If one returns something different compared to any of the >>> existing engines then it's a bug. >>> >>> In this case the implicit documentation is how MyISAM, Aria and InnoDB >>> works. All of them returns the first conflicting key. > > Pavel> I'm sorry to say, but you've explained exactly the reason why this > Pavel> behavior is not guaranteed and can be changed at any time. > > It can only be changed if someone adds new code that breaks the > storage engine interface, as defined by existing engines and one > doesn't check it with the storage engine test suite. > > Pavel> Yes, I > Pavel> understand that when you wrote the interface you had some particular > Pavel> requirements in mind. But ultimately it's up to people actively > Pavel> working on the code (who I believe are counted by hundreds now) to > Pavel> decide whether to maintain the particular behavior or change > it. > > It's the MariaDB captains that are responsible to maintain the > interface. The only way one can change something is to propose a > change and get it accepted. > > It's not free for anyone to just change old functionality and expect > that to be accepted in the standard version. > > In all cases, including this, it's an absolute requirement that > engines MUST work exactly as existing engines, so that a user can > switch engines and expect them to work identically. > > (The only exceptions are things that are impossible to fix, like > the differences between transactional and not transactional tables and > that there is no defined row order in tables). > > If engine writes don't ensure this, then you will get data corruption > on the server by just running with different engines. > > For case on insert on duplicate key, there is no acceptable reason > for engines to work differently. > > Pavel> And > Pavel> as nobody knows the original requirements. > > The original requirements was well defined. Just that non one bother > to ask doesn't mean that the requirement didn't exist. > > And the fact that all existing engines in MariaDB ensures this shows > that the requirements has been understood. > > In the case when we notice that two engines works differently, it's up > to the MariaDB captains to fix this! > The fix is NOT to extend the API and allow things to work differently. > > > Pavel> and the reasons why they > Pavel> can't be changed, and when the behavior is not tested anywhere > > We add new tests when we notice that they are needed, as we did in > this case. > For example, the case with INSERT ... ON DUPLICATE KEY is now tested. > > Pavel> and > Pavel> doesn't have any documentation even in the weakest form of comments or > Pavel> asserts, the decision is usually very easy -- change at will, nobody > Pavel> should ever depend on implementation details. > > Yes, no one should depend in implementation details, except when the > implementation detail is different from other engines and it results > in different data. > > If that is the case then the engine that does things differently than > the existing engines must be changed. > > Pavel> Even if you promise to > Pavel> fix storage engines that change this behavior, by the time you notice > Pavel> the change you'll likely won't be able to fix it (as e.g. in case of > Pavel> https://mariadb.atlassian.net/browse/MDEV-6916) or someone will > Pavel> already experience a data corruption. > > In most cases we can fix this. What was unfortunate with the views > code was that we didn't store the MariaDB version number in the view > definition. This is to be fixed in 10.1 so that this will never happen > again! > > Pavel> I'd strongly recommend to revert this change. > > Sorry, but I can't agree to revert the change because: > > a) Something that can't ever happen in MariaDB (when used correctly > with the including engines). > b) Allowing a behavior that could for the same command result in > different data sets, depending on the storage engine. > c) I have not seen a real world > > One of my main jobs is to ensure that the storage engine API is > evolved in a resonable manner and if there is a conflict between how > storage engine works ensure that things are resolved so that the user > can expect the same results independent on the storage engine. > > Reverting the change would go directly agains the above and also > against what I belive is the future of the storage engine API. > > Regards, > Monty

2 1

Re: [Maria-developers] MariaDB 10.0.14 got rid of replication safety warning without warning in release notes?
by Pavel Ivanov 07 Nov '14

07 Nov '14

Monty, can you please subscribe all your email address to maria-develoeprs@ ? People reading the list cannot see your emails. I'm forwarding here the unaltered message from Monty that I was answering to in my last email. On Sun, Nov 2, 2014 at 3:27 PM, Michael Widenius <monty(a)mariadb.org> wrote: > > Hi! > > I have again to apologize for a late reply. > After my last reply to your topic I traveled to China and noticed > that the firewall there has got much better; You can't access google, > twitter, facebook, blogspot etc :( I tried to connect over a VPN, but > that didn't work for me either. > > It did feel a bit strange to not be able to do computer work for more > than a week, but the lovely people in China who took care of me did > compensate for that. > >>>>>> "Pavel" == Pavel Ivanov <pivanof(a)google.com> writes: > > Pavel> Which key is "the first duplicate key". >>> >>> The first key that has a duplicate value in the order of keys in the KEY >>> structure that MariaDB provides for the storage engine. > > Pavel> I'm definitely not that familiar with the relevant MariaDB code, but I > Pavel> didn't find KEY structure to be passed to any insert-related function. > Pavel> Can you elaborate (preferably with code references) how this > Pavel> restriction is supposed to work? > > The following code examples are from 10.0. > > All handler functions get access to the TABLE object, which defines > the record format and the keys. > > The keys are stored in TABLE->key_info (see table.h, line 1062) > > When one reads a key, one refers to keys according to the number in > the key_info structure. For example: > > A set of reads by key is done by calling: > > handler->ha_index_init(key_number,..); > read_until_no_more_rows > handler->ha_index_read_map(). > handler->ha->index_end() > > When one calls handler->info() to ask for a which key caused a > duplicate error, it's supposed to store the first duplicate key (key > with smallest number) in handler->errkey. > >>> This is the same order that keys are presented in SHOW CREATE TABLE > > Pavel> Why is it safe to assume that keys have the same order on master and on slave? > > Primary and Unique keys should always be ordered the same way on > master and slave. This is handled by the 'sort_keys()' function that > you can find at: sql/sql_table.cc, line 2770. > > The only case when this is not the case, is if you add an unique key > on the slave that was not on the master and there is conflicts when > you insert things because of this key. > > However, if you do this, I would say that you are on your own. > >>> To help engine writes we are constantly adding new test in the storage >>> engine test suite, to make it easier for storage engine vendors to >>> test that their storage engine follows the interface. >>> We have now added a test to check for the correct behaviour for the >>> case of INSERT ... ON DUPLICATE KEY. > > Pavel> This is very interesting. Can you say which test it is and what > Pavel> exactly does it test? Is it present in 10.0.14 or it's only at the > Pavel> latest 10.0 branch? > > I added a test for this to: > mysql-test/t/insert.test > > --echo # > --echo # MDEV-5168: Ensure that we can disable duplicate key warnings > --echo # from INSERT IGNORE > --echo # > > Elena added another test for this in > mysql-test/suite/storage_engine/insert_with_keys.test > > See line 142, that start with: > --let $create_definition = a $int_indexed_col UNIQUE KEY, b $int_indexed_col UNIQUE KEY, c $int_col > >>> Do you have or use a storage engine that doesn't provide the expected value >>> for the conflicting key ? > > Pavel> I don't have a specific example of storage engine that detects > Pavel> duplicate key in different order, but I'm not convinced and don't have > Pavel> hard proof that such engine cannot exist and that InnoDB itself is not > Pavel> susceptible to this problem. I can easily imagine though that even > Pavel> given the particular order of keys in table definition the key > Pavel> reported as having duplicate can depend on the way storage engine > Pavel> stores those keys. > > As key order is defined on the upper level, above InnoDB, this should > be 100 % safe (at least for InnoDB). > > For other store engines, this is safe for any engine that test for > duplicate key in key order. > > If we ever find a case where it's not safe with any storage engine > that we deliver with MariaDB, we will fix the storage engine. > > It's ok if they do thing differently so that they can detect duplicate > key for other keys faster. > > However, it has been required from all storage engines from start that > if one calls 'handler->info()' to request which key was duplicated, > they must return the first key. > > If not, they are breaking the storage engine interface (both for MySQL > and MariaDB) as there is no way the upper level can guarantee that > things will work identically between storage engines. > > Pavel> BTW, can you point to MySQL documentation and/or comment in the code > Pavel> that indicates this requirement to storage engine implementation? > > Most of the storage engine interface is not 100 % documented. This is > one of the case. > However, I wrote the code related to errkey and this is how it was > supposed to work. > In general if something is not documented, then it's up to the people > that created the interface or are actively working on it to define how > it should work. > > What is important is that all storage engine should work the same > way. If one returns something different compared to any of the > existing engines then it's a bug. > > In this case the implicit documentation is how MyISAM, Aria and InnoDB > works. All of them returns the first conflicting key. > > Pavel> On the related note Release Notes mention the error log flood > Pavel> protection. Why it's not behind flag? How should I disable the feature > Pavel> and get back to the old behavior? > >>> We already had in MariaDB flood protection for some error >>> messages. > > Pavel> Can you say exactly which error messages are suppressed already? Is > Pavel> there a documentation link about that? > > All 'unsafe logging' errors are suppressed. > See enum enum_binlog_stmt_unsafe in sql_lex.h > (This is what I know). > >>> I am happy to look into adding a flag that is usable for any kind of >>> suppressed messages. >>> >>> A couple of ways to do this: >>> >>> a) Not suppress anything if you give --log-warnings a value > 2 >>> b) make LIMIT_UNSAFE_WARNING_ACTIVATION_TIMEOUT a configurable >>> variable, where 0 would mean no suppression. >>> >>> Which of the above, or both, would you prefer? >>> Or do you have any other suggestion of how to do this? > > Pavel> Reading documentation on --log-warnings I don't see how it can be > Pavel> related to the error suppression. So I think making > Pavel> LIMIT_UNSAFE_WARNING_ACTIVATION_TIMEOUT configurable would be much > Pavel> better, though it should apply to all possible error suppressions. > > Currently we only have suppression for a limited set of 'unsafe' > errors, but I agree that when we add a flag we should name this with > a general name to be able to reuse it for all possible future > suppressions. > > In 10.1 we are looking at adding a feature that one can suppress any > error from the error log, but that's a different topic. > > Regards, > Monty

1 0

[Maria-developers] Maybe a new information schema table for replication?
by Roberto Spadim 06 Nov '14

06 Nov '14

Hi guys, i got one question these days and i don't know how to solve... the question is: "How to see the queries executing by replication thread?" i'm thinking about a information schema plugin to see whats is 'queued' in replication thread, at "master" (write to binlog) and "slave" (executing from binlog) side any idea if this is possible? i didn't checked or read any part of binlog/replication code yet thanks guys! :) -- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

1 0

[Maria-developers] pcre.h should be installed if MariaDB is built with bundled pcre ?
by Kazuhiko Shiozaki 06 Nov '14

06 Nov '14

Hi, pcre.h is not installed by 'make install' even if MariaDB is built with bundled pcre. But pcre.h is referred in one of installed header files like : include/mysql/private/item_cmpfunc.h:#include "pcre.h" /* pcre header file */ So I think that it is better to install pcre.h as well if MariaDB is built with bundled pcre. What do you think ? -- Kazuhiko Shiozaki, Nexedi SA Senior Consultant Nexedi: Consulting and Development of Free / Open Source Software http://www.nexedi.com/ ERP5: Full Featured High End Open Source ERP http://www.erp5.com/

2 1

[Maria-developers] Fwd: Is it soon 'end of this year'?
by Peter Laursen 04 Nov '14

04 Nov '14

Forwarding to MariaDB developers mailing list. It may be a matter for somebody else than Sergei (Georg, for instance) (and "here" means"Webyog"). Abishek detailed errors in a chat with me "symbols like Snippet mysql_client_find_plugin , Snippet mysql_client_register_plugin are not found" ..exactly like in the MySQL bug report. Do you need a JIRA report with a full test case? -- Peter ---------- Forwarded message ---------- From: Abhishek Kumar Pandey <abhishek.pandey(a)webyog.com> Date: Tue, Nov 4, 2014 at 12:51 PM Subject: Re: [Maria-developers] Is it soon 'end of this year'? To: Peter Laursen <peter_laursen(a)webyog.com> Cc: "team. sqlyog" <team.sqlyog(a)webyog.com>, serg(a)mariadb.org, "team.monyog" <team.monyog(a)webyog.com> I am the developer here working with this. I am facing same problem as reported here- http://bugs.mysql.com/bug.php?id=69204 <http://www.google.com/url?q=http%3A%2F%2Fbugs.mysql.com%2Fbug.php%3Fid%3D69…> . Is the same fix needed in the mariadb library? On Tue, Oct 14, 2014 at 6:58 PM, Peter Laursen <peter_laursen(a)webyog.com> wrote: > You will get to this page: https://edelivery.oracle.com/ > > I think you should use a non-webyog email address. Not sure what Mahesh > did. He downloaded frequently in order to check latest MEM. > > > --Peter > > On Tue, Oct 14, 2014 at 3:26 PM, Peter Laursen <peter_laursen(a)webyog.com> > wrote: > >> You could also try to add the code as described by Sergei. First step is >> if it will compile and connect to servers not usinglain text/LDAP. >> >> -- Peter >> >> On Tue, Oct 14, 2014 at 3:12 PM, Supratik Goswami < >> supratik.goswami(a)webyog.com> wrote: >> >>> On Tue, Oct 14, 2014 at 4:58 PM, Sudhindra Bhat < >>> sudhindra.bhat(a)webyog.com> wrote: >>> >>>> +supratik >>>> >>>> We will check this once we have access to MySQL Enterprise server from >>>> Oracle. >>>> >>>> @supratik, >>>> Do we have the binary for MySQL Enterprise server? A Trial is also >>>> sufficient. >>>> >>> >>> No, I think. You may download it. >>> >>>> >>>> On Tue, Oct 14, 2014 at 2:52 PM, Peter Laursen < >>>> peter_laursen(a)webyog.com> wrote: >>>> >>>>> We need to try this again. I don't rememebr who gave up supporting >>>>> 'clear text plugin' with MariaDB API back in April 2013. Probably it was >>>>> not Sudhindra, Likely Shubhansh or Vishal. @Vishal: do you remember >>>>> anything? >>>>> >>>>> I remember one detail from the old discussion we had with MariaDB. >>>>> They told something like "Once enabled nothing special need to be done to >>>>> support clear text. The client wll automatically detect when the server >>>>> requests the password sent as clear text". This mean as far as I can >>>>> understand that we will - unlke early 11.x version - not need a checkbox in >>>>> the connection manager. >>>>> >>>>> We also need access to a commercial server (TRIAL) from Oracle. Mahesh >>>>> could arrange that. Who can today? >>>>> >>>>> >>>>> -- Peter >>>>> >>>>> >>>>> ---------- Forwarded message ---------- >>>>> From: Sergei Golubchik <serg(a)mariadb.org> >>>>> Date: Mon, Oct 13, 2014 at 11:24 PM >>>>> Subject: Re: [Maria-developers] Is it soon 'end of this year'? >>>>> To: Peter Laursen <peter_laursen(a)webyog.com> >>>>> >>>>> >>>>> Hi, Peter! >>>>> >>>>> On Oct 13, Peter Laursen wrote: >>>>> > Thanks .. you probabaly know that I am not a developer rmyself (I >>>>> basically >>>>> > stopped being so when ALGOL died 35 years ago!). I have shared all >>>>> > information with SQLyog developer team and we will look into this >>>>> again. >>>>> >>>>> Sure! Don't hesitate to tell if something won't work for you. >>>>> >>>>> Regards, >>>>> Sergei >>>>> >>>>> >>>> >>>> >>>> -- >>>> Regards, >>>> Sudhindra Bhat >>>> >>> >>> >> >

2 1

Re: [Maria-developers] MariaDB 10.0.14 got rid of replication safety warning without warning in release notes?
by Pavel Ivanov 03 Nov '14

03 Nov '14

On Sun, Nov 2, 2014 at 3:27 PM, Michael Widenius <monty(a)mariadb.org> wrote: >>> This is the same order that keys are presented in SHOW CREATE TABLE > > Pavel> Why is it safe to assume that keys have the same order on master and on slave? > > Primary and Unique keys should always be ordered the same way on > master and slave. This is handled by the 'sort_keys()' function that > you can find at: sql/sql_table.cc, line 2770. This function seem to sort only by "type" of the key. If several unique keys have the same "type" according to its classification then they will be in the "original order" as the comment in the function says. And the original order may be different. > The only case when this is not the case, is if you add an unique key > on the slave that was not on the master and there is conflicts when > you insert things because of this key. > > However, if you do this, I would say that you are on your own. > >>> To help engine writes we are constantly adding new test in the storage >>> engine test suite, to make it easier for storage engine vendors to >>> test that their storage engine follows the interface. >>> We have now added a test to check for the correct behaviour for the >>> case of INSERT ... ON DUPLICATE KEY. > > Pavel> This is very interesting. Can you say which test it is and what > Pavel> exactly does it test? Is it present in 10.0.14 or it's only at the > Pavel> latest 10.0 branch? > > I added a test for this to: > mysql-test/t/insert.test > > --echo # > --echo # MDEV-5168: Ensure that we can disable duplicate key warnings > --echo # from INSERT IGNORE > --echo # This test doesn't check which duplicate key is asserted to execute ON DUPLICATE KEY action. > Elena added another test for this in > mysql-test/suite/storage_engine/insert_with_keys.test > > See line 142, that start with: > --let $create_definition = a $int_indexed_col UNIQUE KEY, b $int_indexed_col UNIQUE KEY, c $int_col This test seem to check that UPDATE is executed on the correct duplicate key, but it's not executed during regular mtr run. How and when it's executed in MariaDB? >>> Do you have or use a storage engine that doesn't provide the expected value >>> for the conflicting key ? > > Pavel> I don't have a specific example of storage engine that detects > Pavel> duplicate key in different order, but I'm not convinced and don't have > Pavel> hard proof that such engine cannot exist and that InnoDB itself is not > Pavel> susceptible to this problem. I can easily imagine though that even > Pavel> given the particular order of keys in table definition the key > Pavel> reported as having duplicate can depend on the way storage engine > Pavel> stores those keys. > > As key order is defined on the upper level, above InnoDB, this should > be 100 % safe (at least for InnoDB). > > For other store engines, this is safe for any engine that test for > duplicate key in key order. > > If we ever find a case where it's not safe with any storage engine > that we deliver with MariaDB, we will fix the storage engine. > > It's ok if they do thing differently so that they can detect duplicate > key for other keys faster. > > However, it has been required from all storage engines from start that > if one calls 'handler->info()' to request which key was duplicated, > they must return the first key. > > If not, they are breaking the storage engine interface (both for MySQL > and MariaDB) as there is no way the upper level can guarantee that > things will work identically between storage engines. > > Pavel> BTW, can you point to MySQL documentation and/or comment in the code > Pavel> that indicates this requirement to storage engine implementation? > > Most of the storage engine interface is not 100 % documented. This is > one of the case. > However, I wrote the code related to errkey and this is how it was > supposed to work. > In general if something is not documented, then it's up to the people > that created the interface or are actively working on it to define how > it should work. > > What is important is that all storage engine should work the same > way. If one returns something different compared to any of the > existing engines then it's a bug. > > In this case the implicit documentation is how MyISAM, Aria and InnoDB > works. All of them returns the first conflicting key. I'm sorry to say, but you've explained exactly the reason why this behavior is not guaranteed and can be changed at any time. Yes, I understand that when you wrote the interface you had some particular requirements in mind. But ultimately it's up to people actively working on the code (who I believe are counted by hundreds now) to decide whether to maintain the particular behavior or change it. And as nobody knows the original requirements and the reasons why they can't be changed, and when the behavior is not tested anywhere and doesn't have any documentation even in the weakest form of comments or asserts, the decision is usually very easy -- change at will, nobody should ever depend on implementation details. Even if you promise to fix storage engines that change this behavior, by the time you notice the change you'll likely won't be able to fix it (as e.g. in case of https://mariadb.atlassian.net/browse/MDEV-6916) or someone will already experience a data corruption. I'd strongly recommend to revert this change. Thank you, Pavel

1 0

[Maria-developers] MDEV-5528 Command line variable to choose MariaDB-5.3 vs MySQL-5.6 temporal data formats
by Alexander Barkov 03 Nov '14

03 Nov '14

Hello Ian, I've pushed MDEV-5528 into 10.1. I guess it will need some documenting. Can you please have a look? Thanks.

1 0

[Maria-developers] nginx sources in mroonga
by Alexander Barkov 03 Nov '14

03 Nov '14

Hi Kentoku, Sergei Golubchik is asking why we have complete nginx sources inside mroonga? Can you remove them please? Thanks.

3 2

[Maria-developers] Cross-compiling MariaDB galera cluster
by Sylvain Raybaud 30 Oct '14

30 Oct '14

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Dear MariaDB community, Recently I've been trying to cross-compile MariaDB and I've been running in some problems. I'd like to have your opinion on them. 1) Determining stack direction 2) Building Percona XtraDB 3) Building jemalloc Let me first tell that I'm trying to build MariaDB galera version 10.0.12. I'm compiling for Arm but may be compiling for something else in the unpredictable future. 1) Determining stack direction I started by passing the following options to cmake (via the buildroot system): - -DCMAKE_CROSSCOMPILING=1 -DWITH_WSREP=1 -DWITH_INNODB_DISALLOW_WRITES=1 The first problem I ran into was cmake complaining: STACK_DIRECTION is not defined. Please specify -DSTACK_DIRECTION=1 or -DSTACK_DIRECTION=-1 when calling cmake. It seems that STACK_DIRECTION is not automatically determined when cross-compiling. I may determine the correct value for a given target. But if I need a portable solution (and I need one in order to create a buildroot package so that I can build for many different target without too much hassle) I'm not sure what my options are. It seems that the toolchain is aware of this value since stack direction (or more precisely STACK_GROWS_DOWNWARD) is set in ./output/host/opt/ext-toolchain/lib/gcc/arm-none-linux-gnueabi/4.8.1/plugin/include/config/arm/arm.h Maybe there is a way to use this somehow? What do you think? 2) Building Percona XtraDB For the sake of trying I explicitly set -DSTACK_DIRECTION=1 just to see what happens. Building goes on until it reaches the xtradb module. Then it fails with: CMake Error at storage/xtradb/CMakeLists.txt:434 (MESSAGE): Percona XtraDB is not supported on this platform (Here I'll just copy-paste something I wrote on maria-discuss list a few days ago) This error is triggered in storage/xtradb/CMakeLists.txt when XTRADB_OK is not set; it seems to me that XTRADB_OK is set in either one of the following cases: a) CMAKE_CROSSCOMPILING is NOT set and some tests are successful (thus setting HAVE_IB_GCC_ATOMIC_BUILTINS) or b) We're compiling for MSVC or CMAKE_SYSTEM_NAME is "SunOS" and some other conditions are met. Since I'm not compiling for MSVC or SunOS and I'm crosscompiling indeed, XTRADB_OK is not set and the error is triggered. Here I'm stuck. I've got the feeling that as such one cannot cross-compile xtradb because XTRADB_OK will never be set; or maybe with proper options passed to cmake the tests would be made elsewhere and HAVE_IB_GCC_ATOMIC_BUILTINS would be set beforehand? Or maybe I should patch this CMakeLists.txt? Or maybe I should just compile without xtradb altogether, if that's possible? Finally I just passed -DXTRADB_OK just to see what would happen, again (obviously there is little chance that it would result in anything usable). And reached problem #3: 3) Building jemalloc This one fails at configure time with: checking whether we are cross compiling... configure: error: in `/home/sylvain/DEPOTS/buildroot/output/build/mariadb-galera-10.0.12/extra/jemalloc/build': configure: error: cannot run C compiled programs. If you meant to cross compile, use `--host'. Somehow I was expecting that having passed -DCMAKE_CROSSCOMPILING=1 and the host being set by buildroot it would propagate down to the modules... Am I missing something? At this point I'm out of dirty hacks, even to "see what happens" ;) Cheers, - -- Sylvain Raybaud www.green-communications.fr -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBAgAGBQJT4ke/AAoJEEkkwl4JtJ9yJW0QALBz3wj0VsfMtUBUabP13q74 ++y1AiCLZ0js+UGCqxB9PVz4yvhhnMAz8QnPpH37XJtDm+VaJBQTEZL65CHF3nlr dHe46/q/wswlbdhNsQu6JmMe3JoOe2TFbjDenQMiUvW9mOilt86ioveHVLGYbpnq 5LYMrPlYXUBpDcxEYXXbTpIHsAoHUrOE1Gj5AsxwbUvQuHDDr9M/Ws01iIXLbzT8 0pImhgBPYUwqaj9tJnPkls+9LzoOb2m269n9TCbmWk0ixakpHjeNMn5lnRDhOwNh AKkHrGdMUomSBca4G0kQoeNxwC2rCjO8+bRxjh7v6GTdME/sgsgn3yOv2EznldUr tFaO/JfraHSUgmiRyVo41U3uTESxZKRJYOV9IqUhl4HiS85D5oa/bD6l84rhx95J 52/hUoApoStr8tSk0tzXk1/UWJEQ3fUZ7XoXqgWq+c1uMgbgZftAAqAw7Xvmc88t tf69FqBzw5dpf8Kg0I7iAggBpBLIWGJwnlVvcmLY4UUE1cDFfZstMBSXaS155LQF VClK+ytE2Ef8oAB9/nlMXgg4b3WXlD0PRQDRTNIwCAzdvsuu+S0q64xLtw/JYnLl KPcLWfwXy+kRhT+vtUWtqMjjkSQ3e6rJDybK11vgYIAWm/Xfk5s8i0g7Bgmnf3wp JGYUAR285gY0XMqL6LmU =vBo4 -----END PGP SIGNATURE-----

2 4

[Maria-developers] Valgrind failure in Buildbot
by Kristian Nielsen 30 Oct '14

30 Oct '14

Hi Sergey, We briefly discussed the new valgrind error in buildbot after your push on IRC: https://buildbot.askmonty.org/buildbot/builders/work-amd64-valgrind/builds/… ==13161== Conditional jump or move depends on uninitialised value(s) ==13161== at 0x4C280AA: memcpy (mc_replace_strmem.c:115) ==13161== by 0x723F6B: Field_varstring::get_key_image(unsigned char*, unsigned int, Field::imagetype) (field.cc:6932) ==13161== by 0x818D5B: key_copy(unsigned char*, unsigned char*, st_key*, unsigned int, bool) (key.cc:146) ==13161== by 0x6DD6FE: Mrr_ordered_index_reader::interrupt_read() (multi_range_read.cc:448) ==13161== by 0x6DE605: Mrr_ordered_rndpos_reader::refill_from_index_reader() (multi_range_read.cc:688) ==13161== by 0x6DE7D2: Mrr_ordered_rndpos_reader::refill_buffer(bool) (multi_range_read.cc:618) ==13161== by 0x6DFB27: DsMrr_impl::dsmrr_init(handler*, st_range_seq_if*, void*, unsigned int, unsigned int, st_handler_buffer*) (multi_range_read.cc:966) ==13161== by 0x91AAE9: ha_innobase::multi_range_read_init(st_range_seq_if*, void*, unsigned int, unsigned int, st_handler_buffer*) (ha_innodb.cc:13961) ==13161== by 0x6DA60A: JOIN_TAB_SCAN_MRR::open() (sql_join_cache.cc:3867) ==13161== by 0x6D7363: JOIN_CACHE::join_matching_records(bool) (sql_join_cache.cc:2249) ==13161== by 0x6D5647: JOIN_CACHE::join_records(bool) (sql_join_cache.cc:2086) ==13161== by 0x5F8259: sub_select_cache(JOIN*, st_join_table*, bool) (sql_select.cc:16717) ==13161== by 0x6D71BA: JOIN_CACHE::generate_full_extensions(unsigned char*) (sql_join_cache.cc:2397) ==13161== by 0x6D74E6: JOIN_CACHE::join_matching_records(bool) (sql_join_cache.cc:2289) ==13161== by 0x6D5647: JOIN_CACHE::join_records(bool) (sql_join_cache.cc:2086) ==13161== by 0x5F8219: sub_select_cache(JOIN*, st_join_table*, bool) (sql_select.cc:16697) I took a quick look. The problem seems to be that the code is copying out the fields of a key from table->record[0] (key_copy()), however, the first length byte of the value is uninitialised. I think it's the field col_varchar_key in the table, testcase is appended below. I checked a bit back up the callstack. The field col_varchar_key is reported as uninitialised already at the start of Mrr_ordered_rndpos_reader::refill_from_index_reader(), but I'm not really familiar with when and how table->record[0] is supposed to be initialised during multi-range read. As you said on IRC, your recent patch is related to interrupt_read(), so probably best if you take a look and see if you can figure out something. Incidentally, I noticed something that looks wrong in key_copy(to_key, from_record, key_info, key_length, with_zerofill) The comments say that the key is copied from `from_record', however actually the function only works if the from_record passed in is table->record[0] (or more precisely, the same record as is installed in key_info->key_part->field). Because it checks the null bit in from_record, but copies the data from table->record[0]. However, this is not the problem in this particular failure (as in this case from_record == table->record[0]), just struck me as something that could easily cause surprises later... Below is a reduced test case that triggers the valgrind error. It did not trigger for me with a debug build, but _did_ trigger (in 5.5) with this build command: EXTRA_FLAGS="-O3 -fno-omit-frame-pointer -Wno-uninitialized" AM_EXTRA_MAKEFLAGS="VERBOSE=1" BUILD/compile-amd64-valgrind-max - Kristian. ----------------------------------------------------------------------- -- source include/have_xtradb.inc set optimizer_switch='mrr=on,mrr_sort_keys=on,index_condition_pushdown=on'; CREATE TABLE t1 ( pk int(11) NOT NULL AUTO_INCREMENT, col_int_key int(11) NOT NULL, col_varchar_key varchar(1) NOT NULL, col_varchar_nokey varchar(1) NOT NULL, PRIMARY KEY (pk), KEY col_int_key (col_int_key), KEY col_varchar_key (col_varchar_key,col_int_key) ) ENGINE=InnoDB; INSERT INTO t1 VALUES (10,8,'v','v'), (11,8,'f','f'), (12,5,'v','v'), (13,8,'s','s'), (14,8,'a','a'), (15,6,'p','p'), (16,7,'z','z'), (17,2,'a','a'), (18,5,'h','h'), (19,7,'h','h'), (20,2,'v','v'), (21,9,'v','v'), (22,142,'b','b'), (23,3,'y','y'), (24,0,'v','v'), (25,3,'m','m'), (26,5,'z','z'), (27,9,'n','n'), (28,1,'d','d'), (29,107,'a','a'); CREATE TABLE t2 ( pk int(11) NOT NULL AUTO_INCREMENT, col_int_key int(11) NOT NULL, col_varchar_key varchar(1) NOT NULL, col_varchar_nokey varchar(1) NOT NULL, PRIMARY KEY (pk), KEY col_int_key (col_int_key), KEY col_varchar_key (col_varchar_key,col_int_key) ) ENGINE=InnoDB; INSERT INTO t2 VALUES (10,3,'i','i'), (11,186,'x','x'), (12,1,'g','g'), (13,8,'q','q'), (14,226,'m','m'), (15,133,'p','p'), (16,6,'e','e'), (17,3,'t','t'), (18,8,'j','j'), (19,5,'h','h'), (20,7,'w','w'); set join_cache_level=6; set join_buffer_size=1536; SELECT count(*), sum(table1.col_int_key*table2.pk) FROM t2 AS table1, t1 AS table2, t2 AS table3 WHERE table3.col_varchar_nokey = table2.col_varchar_key AND table3.pk > table2.col_varchar_nokey ; -----------------------------------------------------------------------

1 0

Re: [Maria-developers] Rev 4323: Set proper alternative names on RHEL7: they should match Fedora names
by Sergei Golubchik 30 Oct '14

30 Oct '14

Hi, Sergey! I cannot verify whether rhel7 alternative names should match Fedora names or they should be something different. Assuming they should match, your patch is ok. On Oct 09, Sergey Vojtovich wrote: > ------------------------------------------------------------ > revno: 4323 > revision-id: svoj(a)mariadb.org-20141009112748-eq3xwoyobgnz0cxj > parent: svoj(a)mariadb.org-20141008141031-xv2x7vrq269202vf > committer: Sergey Vojtovich <svoj(a)mariadb.org> > branch nick: 5.5 > timestamp: Thu 2014-10-09 15:27:48 +0400 > message: > Set proper alternative names on RHEL7: they should match Fedora names. > === modified file 'cmake/cpack_rpm.cmake' > --- a/cmake/cpack_rpm.cmake 2014-10-08 07:35:00 +0000 > +++ b/cmake/cpack_rpm.cmake 2014-10-09 11:27:48 +0000 > @@ -182,7 +182,7 @@ IF(RPM MATCHES "(rhel|centos)5") > ELSEIF(RPM MATCHES "(rhel|centos)6") > ALTERNATIVE_NAME("client" "mysql") > ALTERNATIVE_NAME("shared" "mysql-libs") > -ELSEIF(RPM MATCHES "fedora") > +ELSEIF(RPM MATCHES "fedora" OR RPM MATCHES "(rhel|centos)7") > SET(epoch 1) # this is fedora > ALTERNATIVE_NAME("client" "mariadb") > ALTERNATIVE_NAME("client" "mysql") Regards, Sergei

2 2

[Maria-developers] Fwd: [debian-mysql] Backporting the mysql_no_login plugin
by Colin Charles 30 Oct '14

30 Oct '14

This seems to get a +1 for backporting by at least Honza (RH), so I am wondering if we do this in the 5.5 branch too, that is shipping in many distributions. Begin forwarded message: > From: "Norvald H. Ryeng" <norvald.ryeng(a)oracle.com> > Subject: [debian-mysql] Backporting the mysql_no_login plugin > Date: 24 October 2014 15:49:34 GMT+8 > To: "pkg-mysql-maint(a)lists.alioth.debian.org" <pkg-mysql-maint(a)lists.alioth.debian.org>, "Honza Horak" <hhorak(a)redhat.com>, "Roman Drahtmueller" <draht(a)suse.de> > > Hi package maintainers, > > We have a new plugin in MySQL 5.7 that makes it possible to have > accounts that can't log in: > > CREATE USER foo@localhost IDENTIFIED WITH 'mysql_no_login'; > > The mysql_no_login plugin simply denies all login attempts. This is > useful for users that are created, e.g., to serve as proxy users, or > as owners of stored programs/functions, views or events. > > This new plugin doesn't fix known security defects in the server, but > does provide new and better means to harden security. Best practices > for security include application of least-required privileges, and in > some cases, that means no client connections for privileged > accounts. This new plugin provides means to implement such > restrictions in a standard way. > > Because of the security benefits, we'd like to discuss backporting it > to 5.6. Like you, we don't like big changes to GA releases, but this > time we think it has a good use case, it's safe and has a very low > risk of regressions: > > - Since this is a plugin, it doesn't touch server code > - All new code is in a plugin that must be enabled explicitly by the > DBA > - The code itself is very simple. It's only one line of "real" code > (unconditionally return authentication failure), plus necessary > plugin plumbing to fill out the plugin API. > > If we backport this to 5.6, there are multiple ways to avoid it: > > - Apply a patch from us to remove the plugin > - Don't build it > - Build it, but don't ship it > - Build and ship it, but don't use it (in any case, the DBA has to > enable it and alter the user accounts to use it) > > So what do you think about backporting this? The only thing you'll > notice is one more file in the plugins directory. > > Regards, > > Norvald > > _______________________________________________ > pkg-mysql-maint mailing list > pkg-mysql-maint(a)lists.alioth.debian.org > http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-mysql-maint -- Colin Charles, Chief Evangelist, MariaDB Corporation blog: http://bytebot.net/blog/| t: +6-012-204-3201 | Skype: colincharles

2 1

Re: [Maria-developers] [Commits] b28e6bf: MDEV-6945 MDEV-6948 MDEV-6956 MDEV-6957 MDEV-6953 MDEV-6954 MDEV-6955
by Sergei Golubchik 29 Oct '14

29 Oct '14

Hi, Sanja! On Oct 29, sanja(a)mariadb.com wrote: > --- > mysql-test/r/set_statement.result | 14 ++++++++++++++ > mysql-test/r/set_statement_profiling.result | 4 ++++ > mysql-test/t/set_statement.test | 18 ++++++++++++++++++ > mysql-test/t/set_statement_profiling.test | 9 +++++++++ Did you put set_statement_debug.test in a separate commit? That's good. > sql/set_var.h | 4 +++- > sql/share/errmsg-utf8.txt | 2 ++ > sql/sql_parse.cc | 5 +++++ > sql/sys_vars.cc | 20 +++++++++++--------- > sql/sys_vars.h | 1 + > 9 files changed, 67 insertions(+), 10 deletions(-) > > diff --git a/sql/share/errmsg-utf8.txt b/sql/share/errmsg-utf8.txt > index bb42807..1633169 100644 > --- a/sql/share/errmsg-utf8.txt > +++ b/sql/share/errmsg-utf8.txt > @@ -7113,3 +7113,5 @@ ER_STATEMENT_TIMEOUT 70100 > eng "Query execution was interrupted (max_statement_time exceeded)" > ER_SUBQUERIES_NOT_SUPPORTED 42000 > eng "%s does not support subqueries or stored functions." > +ER_SET_STATEMENT_NOT_SUPPORTED 42000 > + eng "Variable '%s' is not supported in SET STATEMENT." Use the text eng "The system variable %.200s cannot be set in SET STATEMENT." > diff --git a/sql/sys_vars.cc b/sql/sys_vars.cc > index 6252e89..fd77e5f 100644 > --- a/sql/sys_vars.cc > +++ b/sql/sys_vars.cc > @@ -997,7 +997,7 @@ static bool check_master_connection(sys_var *self, THD *thd, set_var *var) > static Sys_var_session_lexstring Sys_default_master_connection( > "default_master_connection", > "Master connection to use for all slave variables and slave commands", > - SESSION_ONLY(default_master_connection), > + NO_SET_STMT SESSION_ONLY(default_master_connection), Why should this be prohibited? I'd rather have it supported. Use case: setting many connection variables for a single master, like in SET STATEMENT default_master_connection=foobar SET max_relay_log_size=10000000, sql_slave_skip_counter=5; Admittedly, this use case, while valid, isn't a strong argument at the moment - we don't have many per-master variables yet. When we'll have more, this will become more important. So, if fixing default_master_connection is not easy - keep it NO_SET_STMT, it can be enabled later. Why, exactly, does it fail? > NO_CMD_LINE, IN_SYSTEM_CHARSET, > DEFAULT(""), MAX_CONNECTION_NAME, ON_CHECK(check_master_connection)); > #endif > @@ -3527,12 +3528,12 @@ static Sys_var_bit Sys_unique_checks( > #ifdef ENABLED_PROFILING > static Sys_var_bit Sys_profiling( > "profiling", "profiling", > - SESSION_VAR(option_bits), NO_CMD_LINE, OPTION_PROFILING, > + NO_SET_STMT SESSION_VAR(option_bits), NO_CMD_LINE, OPTION_PROFILING, Why should this be prohibited? I'd rather have it supported. Use case: one usually needs to profile a single statement, not everything. > DEFAULT(FALSE)); > > static Sys_var_ulong Sys_profiling_history_size( > "profiling_history_size", "Limit of query profiling memory", > - SESSION_VAR(profiling_history_size), CMD_LINE(REQUIRED_ARG), > + NO_SET_STMT SESSION_VAR(profiling_history_size), CMD_LINE(REQUIRED_ARG), > VALID_RANGE(0, 100), DEFAULT(15), BLOCK_SIZE(1)); > #endif > > @@ -3566,7 +3567,8 @@ static bool check_skip_replication(sys_var *self, THD *thd, set_var *var) > > static Sys_var_bit Sys_skip_replication( > "skip_replication", "skip_replication", > - SESSION_ONLY(option_bits), NO_CMD_LINE, OPTION_SKIP_REPLICATION, > + NO_SET_STMT SESSION_ONLY(option_bits), > + NO_CMD_LINE, OPTION_SKIP_REPLICATION, Why should this be prohibited? I'd rather have it supported. Use case: skip replication of a single statement. A generic version of FLUSH NO_WRITE_TO_BINLOG TABLE REPAIR NO_WRITE_TO_BINLOG TABLE ANALYZE NO_WRITE_TO_BINLOG TABLE OPTIMIZE NO_WRITE_TO_BINLOG TABLE and all ALTER TABLE ... PARTITION variants: ALTER TABLE ... REBUILD PARTITION NO_WRITE_TO_BINLOG ALTER TABLE ... OPTIMIZE PARTITION NO_WRITE_TO_BINLOG ALTER TABLE ... ANALYZE PARTITION NO_WRITE_TO_BINLOG ALTER TABLE ... REPAIR PARTITION NO_WRITE_TO_BINLOG ALTER TABLE ... COALESCE PARTITION NO_WRITE_TO_BINLOG ALTER TABLE ... ADD PARTITION NO_WRITE_TO_BINLOG ALTER TABLE ... REORGANIZE PARTITION NO_WRITE_TO_BINLOG > DEFAULT(FALSE), NO_MUTEX_GUARD, NOT_IN_BINLOG, > ON_CHECK(check_skip_replication)); > > @@ -3688,7 +3690,7 @@ static ulonglong read_rand_seed(THD *thd) > static Sys_var_session_special Sys_rand_seed1( > "rand_seed1", "Sets the internal state of the RAND() " > "generator for replication purposes", > - sys_var::ONLY_SESSION, NO_CMD_LINE, > + NO_SET_STMT sys_var::ONLY_SESSION, NO_CMD_LINE, > VALID_RANGE(0, ULONG_MAX), BLOCK_SIZE(1), > NO_MUTEX_GUARD, IN_BINLOG, ON_CHECK(0), > ON_UPDATE(update_rand_seed1), ON_READ(read_rand_seed)); > @@ -3706,7 +3708,7 @@ static bool update_rand_seed2(THD *thd, set_var *var) > static Sys_var_session_special Sys_rand_seed2( > "rand_seed2", "Sets the internal state of the RAND() " > "generator for replication purposes", > - sys_var::ONLY_SESSION, NO_CMD_LINE, > + NO_SET_STMT sys_var::ONLY_SESSION, NO_CMD_LINE, > VALID_RANGE(0, ULONG_MAX), BLOCK_SIZE(1), > NO_MUTEX_GUARD, IN_BINLOG, ON_CHECK(0), > ON_UPDATE(update_rand_seed2), ON_READ(read_rand_seed)); I'd rather fix rand_seed1/rand_seed2 variables not to be special anymore. Like, move them to SV structure, remove special behavior and so on. Regards, Sergei

1 0

Re: [Maria-developers] [Commits] b18ddd2: Added 'mariadb-galera-server' metapackage to install all
by Sergei Golubchik 28 Oct '14

28 Oct '14

Hi, Nirbhay! On Oct 23, nirbhay(a)skysql.com wrote: > > Added 'mariadb-galera-server' metapackage to install all > necessary Galera-related dependencies. > > --- > debian/dist/Debian/control | 13 ++++++++++--- > debian/dist/Ubuntu/control | 13 ++++++++++--- > 2 files changed, 20 insertions(+), 6 deletions(-) > > diff --git a/debian/dist/Debian/control b/debian/dist/Debian/control > index e68f027..bc48dbc 100644 > --- a/debian/dist/Debian/control > +++ b/debian/dist/Debian/control > @@ -194,9 +194,7 @@ Recommends: libhtml-template-perl > Pre-Depends: mariadb-common, adduser (>= 3.40), debconf > Depends: mariadb-client-10.1 (>= ${source:Version}), libdbi-perl, > perl (>= 5.6), ${shlibs:Depends}, ${misc:Depends}, psmisc, passwd, > - lsb-base (>= 3.0-10), mariadb-server-core-10.1 (>= ${binary:Version}), > - galera (>=25.2), rsync, lsof, socat | netcat, grep, gawk, iproute, > - coreutils, findutils > + lsb-base (>= 3.0-10), mariadb-server-core-10.1 (>= ${binary:Version}) > Provides: mariadb-server, mysql-server, virtual-mysql-server > Conflicts: mariadb-server (<< ${source:Version}), mysql-server (<< ${source:Version}), > mysql-server-4.1, mysql-server-5.0, mysql-server-5.1, mysql-server-5.5, Looks ok. Few comments: 1. What about RPM? 2. Did you compare with, say, non-galera 10.0 - just to make sure all dependencies you removed are, indeed, only for galera and the server doesn't need any of that? 3. I'd also add /etc/mysql/conf.d/galera.cnf (/etc/my.cnf.d/galera.cnf in rpms) that enables wsrep, and may be sets other reasonable defaults for galera. Regards, Sergei

1 0

[Maria-developers] DisconnectEx() problem
by Delveri chick 28 Oct '14

28 Oct '14

Hi i was tracing why mysql_socket_shutdown(...) won't call DisconnectEx(...) as reported in the TODO list and realised DisconnectEx(...) cannot be called in include/mysql/psi/mysql_socket.h. Possible solution would be to add it here or include the *.h where it it defined. Where is it defined please? Regards Delveri

2 1

Re: [Maria-developers] [Commits] 18a7470: MDEV-6939 : Dots in file names of configuration files
by Sergei Golubchik 28 Oct '14

28 Oct '14

Hi, Nirbhay! 1. Why 10.1? 1.0 would be fine too, may be even 5.5 2. In cases like this I usually make two commits, one for a cleanup and one for a bug fix. Anyway, the fix is ok to push, of course. On Oct 24, nirbhay(a)mariadb.com wrote: > revision-id: 18a7470a563c30bffbfe787e4e5406b687597a44 > parent(s): cc656e4d8538326d85e84adbe48d0a3739d55674 > committer: Nirbhay Choubey > branch nick: server > timestamp: 2014-10-24 16:34:54 -0400 > message: > > MDEV-6939 : Dots in file names of configuration files > > Use fn_ext2() to get the file extension from last occurrence > of FN_EXTCHAR ('.') instead. Also made some cosmetic changes > in mysys/mf_fn_ext.c. > > --- > mysys/mf_fn_ext.c | 11 ++++++----- > mysys/my_default.c | 2 +- > 2 files changed, 7 insertions(+), 6 deletions(-) > Regards, Sergei

1 0

[Maria-developers] DATE and GROUP_MIN_MAX optimization
by Alexander Barkov 28 Oct '14

28 Oct '14

Hello Sergey, I'm working on pluggable data types and moving some pieces of the code into new classes for data type handlers. I noticed that GROUP_MIN_MAX optimization works fine with a TIME column with temporal and string literals but does not work for an integer literal: # This script creates demo data: DROP TABLE IF EXISTS t1; CREATE TABLE t1 (id INT, b TIME, KEY(id,b)) ENGINE=MyISAM; INSERT INTO t1 VALUES (1,10),(1,11),(1,12),(1,13),(1,14),(1,15),(1,16),(2,10),(2,11),(2,12),(2,13),(2,14),(2,15),(2,16); SELECT * FROM t1; # Now three EXPLAIN queries: explain SELECT id, MIN(b),MAX(b) FROM t1 WHERE b<TIME'10:10:10' GROUP BY id; explain SELECT id, MIN(b),MAX(b) FROM t1 WHERE b<'10:10:10' GROUP BY id; explain SELECT id, MIN(b),MAX(b) FROM t1 WHERE b<10 GROUP BY id; The first and the second EXPLAIN queries tell that 8 rows are to be examined; tracing the code in debugger shows that group_min_max optimization is active: > mysql> explain SELECT id, MIN(b),MAX(b) FROM t1 WHERE b<TIME'10:10:10' GROUP BY id; > +------+-------------+-------+-------+---------------+------+---------+------+------+---------------------------------------+ > | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra | > +------+-------------+-------+-------+---------------+------+---------+------+------+---------------------------------------+ > | 1 | SIMPLE | t1 | range | NULL | id | 9 | NULL | 8 | Using where; Using index for group-by | > +------+-------------+-------+-------+---------------+------+---------+------+------+---------------------------------------+ > 1 row in set (0.00 sec) > > mysql> explain SELECT id, MIN(b),MAX(b) FROM t1 WHERE b<'10:10:10' GROUP BY id; > +------+-------------+-------+-------+---------------+------+---------+------+------+---------------------------------------+ > | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra | > +------+-------------+-------+-------+---------------+------+---------+------+------+---------------------------------------+ > | 1 | SIMPLE | t1 | range | NULL | id | 9 | NULL | 8 | Using where; Using index for group-by | > +------+-------------+-------+-------+---------------+------+---------+------+------+---------------------------------------+ > 1 row in set (0.00 sec) > The third EXPLAIN query tells that all 14 rows is to be examined, and tracing in debugging shows that group_min_max is not enabled: > mysql> explain SELECT id, MIN(b),MAX(b) FROM t1 WHERE b<10 GROUP BY id; > +------+-------------+-------+-------+---------------+------+---------+------+------+--------------------------+ > | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra | > +------+-------------+-------+-------+---------------+------+---------+------+------+--------------------------+ > | 1 | SIMPLE | t1 | index | NULL | id | 9 | NULL | 14 | Using where; Using index | > +------+-------------+-------+-------+---------------+------+---------+------+------+--------------------------+ > 1 row in set (0.01 sec) Any reasons not to use group_min_max for TIME column in a combination with an integer literal? I have a feeling that it should work. What about decimal and real literals? Thanks.

1 0

[Maria-developers] Problem with KVM version in Buildbot?
by Kristian Nielsen 24 Oct '14

24 Oct '14

Hi Daniel, We have a problem with a build in Buildbot: https://internal.askmonty.org/buildbot/builders/kvm-rpm-centos7_0-x86_64/bu… + qemu-img create -b /kvm/vms/vm-centos7_0-x86_64-build.qcow2 -f qcow2 /dev/shm/vm-tmp-2302.qcow2 qemu-img: '' uses a qcow2 feature which is not supported by this qemu version: QCOW version 3 qemu-img: Could not open '/kvm/vms/vm-centos7_0-x86_64-build.qcow2' Failed to clone base image, aborting This failure happens on the build host `terrier'. I assume the problem is that the image was created on a machine with a newer KVM? The host terrier seems to have a rather old one: knielsen@terrier:~> kvm --version QEMU emulator version 0.13.0 (qemu-kvm-devel), Copyright (c) 2003-2008 Fabrice Bellard How do you think we should solve the problem? Should we upgrade terrier? Or should we alternatively make sure to create new images in a backward-compatible version (and re-create affected images) ? Also, another issue related to VM images. Our old images were created with rather small disk size. I noticed that some new images are created much larger, for example: $ qemu-img info /kvm/vms/vm-sid-amd64-build.qcow2 image: /kvm/vms/vm-sid-amd64-build.qcow2 file format: qcow2 virtual size: 40G (42949672960 bytes) disk size: 3.1G cluster_size: 65536 Format specific information: compat: 0.10 That is fine, I think, it is indeed annoying when we run out of space in a build VM. However, the reason I noticed this is because host bb02 ran out of space in /run/shm/ due to a 16 GB copy of this image. So when we use larger VM images, we cannot use the optimisation to have the temporary copies in ramdisk, lest we get disk full errors. If the virtual size is larger to (ram disk size) / (max_builds), then we need to disable using ramdisk for buildbot for that image. I will try to fix the config to not use ramdisk for existing images that have too large virtual size. But it is something to keep in mind for the future. - Kristian.

3 3

Re: [Maria-developers] SQL Accelerator FPGA/ASIC
by Nasser Ghoseiri 24 Oct '14

24 Oct '14

Dear Jan, Thank you for your response. I have background in SQL (MS, Sybase, MySQL, Postgre), so luckily I'm not a beginner in query writing, however, I have absolutely no background in how databases work on background (other than hash-table). The idea is not really SIMD here, but a specially designed hardware to perform database functions much faster than the conventional DISK<->RAM<->CACHE<->CPU model. What I'm struggling to figure is that the latency caused by a enterprise-grade SQL script (not very correct and advanced code, but not very low-quality either), is Memory-Related or Arithmetic related or both? Here is what can be made for example: 1) A card with 16 DDR3 Chips on board and a Central FPGA, allowing very fast access to data, and like 100 small CPUs integrated in the FPGA to process a query. (a query that was written to take advantage of these features however) 2) A card with XDR Memory chips and like 32GB of Flash Chips to achieve very high-speed storage, retrieval (SSD<->RAM) and an FPGA that has many cores inside. A query is executed by FPGA pulling large chunk of data from Flashes, putting them in XDR RAM chips and processes them in a parallel manner 3) A card with XDR Memory chips and an FPGA on board, connected to a PCI Express 8x which receives all the data it needs from the host computer, and does the parallel processing inside itself, returning it back to the host. 4) A card with an FPGA inside, having 100 small CPUs and PCI Express to take commands from host computer and process in a parallel manner 5) A card with an FPGA, not having small processors inside, but special circuitry that does either lookup, sorting, etc in a very fast manner These are a few examples that I gave. The difference among them is the notion of having RAM, having a cold-storage, having an FPGA with many CPUs inside or an FPGA with special dedicated circuitry that performs special functions in parallel that is difficult for a CPU to handle (imaging adding 300 numbers to another 300 numbers in a single cycle, where a CPU with SIMD would take much longer, or maybe a string processor that processes many string columns with PATINDEX, etc in parallel where a CPU would be slow as it can't handle them in parallel). So what I'm researching is whether beneath the SQL pyramid, there is a memory intensive operation going on or logic-intensive. What to look for and what to aim for... Thanks in advance for your help, Nasser On Friday, October 24, 2014 7:49 PM, Jan Lindström <jan.lindstrom(a)mariadb.com> wrote: Hi, This idea of SIMD (single instruction multiple data) processing is not totally new one, similarly the idea to perform SQL-operations inside GPU or GPGA is not new. In traditional relational databases problematic is the fact that e.g. in your example TableX contains several columns, picking columns A and B from pages that reside first on disk, then on main-memory and finally on L1-L3 cache is not cheap, and then they are not on continuous memory. This is because page in cache would contain values for other columns that we not even need. In columnar database architecture this would be a lot easier, you just feed column containing values for A and B directly to SIMD operation and every page in main memory would contain a lot more values to process compared to traditional relational database where page would contain also values for columns that we really do not even need to process the result set of query. Anyway, I find the proposal interesting and challenging. R: Jan Lindström On Thu, Oct 23, 2014 at 3:39 PM, Nasser Ghoseiri <cdmcsd(a)yahoo.com> wrote: Dear Serg, > > >Following our little chat in IRC, I'm writing this email to explain in more detail what the idea is. My name is Nasser GHOSEIR (Founder, CTO of Butterfly labs), which I must note that this project is not from Butterfly Labs, >but will be a new company in Europe. > > >Our idea is to find a way to accelerate SQL query processing by either: > > >1) Creating an FPGA solution (Which will later evolve into ASIC), that has like 400 processors in itself, allow distribute calculation of some kind >2) Creating an FPGA/ASIC solution that performs large number of unrelated tasks (such as addition, multiplication, etc) in parallel >3) Creating an FPGA/ASIC solution that allows very high-speed access to data with some PRE-PROCESSING involved to accelerate the calculation >4) Creating a very high-speed storage solution, but no pre-processing. > > >An example to give is for 200,000 rows, imagine: "SELECT A+B, C FROM TableX WHERE C > 0". If there are 200,000 records, the processor has to perform 200,000 additions (A+B). >A CPU will handle these additions one-by-one (or maybe few-by-few if SSE, etc is used). However, an FPGA/ASIC solution can perform 1000 additions in a single-cycle. This results in acceleration. > > >Now, to what extend this can be effective, or what other solutions (maybe string processing?) can be implemented to accelerate the SQL processing is a question to me right now. But we do have experience in >making extremely fast processors (our Butterfly Labs Monarch chip performs 400 billion double-SHA256 hashes in 1 second, around 20,000 times faster than the best intel XEON processor can do only twenty >million hashes per second). > > >The project is in brain-storming phase, and we are aware that this solution will be useful to large enterprises, or companies that have to deal with 200,000 rows in a single "SELECT" query. The idea is to integrate >some features in hardware, and then re-write a portion of MariaDB to take advantage of the new resources. Also, it is possible that users/companies will need to re-write their query to make it compliant. > > >You can reach my by mail at cdmcsd(a)yahoo.com or by phone at +33 6 72 17 26 19 (France). > > > > > > >Best Regards, >Nasser GHOSEIRI > > > > > > > > >_______________________________________________ >Mailing list: https://launchpad.net/~maria-developers >Post to : maria-developers(a)lists.launchpad.net >Unsubscribe : https://launchpad.net/~maria-developers >More help : https://help.launchpad.net/ListHelp > >

1 0

[Maria-developers] Please review: MDEV-5528 Command line variable to choose MariaDB-5.3 vs MySQL-5.6 temporal data formats
by Alexander Barkov 24 Oct '14

24 Oct '14

Hi, Please review my patch for MDEV-5528. It also fixes a bug: MDEV-6649 Different warnings for TIME and TIME(N) when @@old_mode=zero_date_time_cast Thanks.

2 8

[Maria-developers] doubt about mariadb dialog plugin at server and client side
by Roberto Spadim 23 Oct '14

23 Oct '14

Hi guys I'm with a doubt at dialog plugin I created a user at mariadb server: ---SQL INSTALL PLUGIN three_attempts SONAME 'dialog_examples.so'; CREATE USER test_dialog IDENTIFIED VIA three_attempts USING 'SECRET'; FLUSH PRIVILEGES; --- now i'm trying to connect as user test_dialog: ---SHELL $mysql *--plugin-dir=/home/rspadim/mariadb/mariadb-10.0.14/plugin/auth_dialog* --user=test_dialog --host=spd1.spadim.com.br --password Enter password: (i just hit enter) [mariadb] Password, please: (enter again) [mariadb] Password, please: (SECRET enter) Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 49243 Server version: 5.5.5-10.0.14-MariaDB-log Source distribution Copyright (c) 2000, 2012, Oracle, Monty Program Ab and others. Reading history-file /home/beto/.mysql_history Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. MariaDB [(none)]> quit Writing history-file /home/beto/.mysql_history --- ---SHELL $ls /home/rspadim/mariadb/mariadb-10.0.14/plugin/auth_dialog CMakeFiles cmake_install.cmake CMakeLists.txt CTestTestfile.cmake dialog.c *dialog.so* Makefile --- ---MYSQL.USER table || *Host* || *User* || *Password* || *Select_priv* || *Insert_priv* || *Update_priv* || *Delete_priv* || *Create_priv* || *Drop_priv* || *Reload_priv* || *Shutdown_priv* || *Process_priv* || *File_priv* || *Grant_priv* || *References_priv* || *Index_priv* || *Alter_priv* || *Show_db_priv* || *Super_priv* || *Create_tmp_table_priv* || *Lock_tables_priv* || *Execute_priv* || *Repl_slave_priv* || *Repl_client_priv* || *Create_view_priv* || *Show_view_priv* || *Create_routine_priv* || *Alter_routine_priv* || *Create_user_priv* || *Event_priv* || *Trigger_priv* || *Create_tablespace_priv* || *ssl_type* || *ssl_cipher* || *x509_issuer* || *x509_subject* || *max_questions* || *max_updates* || *max_connections* || *max_user_connections* || *plugin* || *auth_string* || || % || test_dialog || >>>CHECK NO PASSWORD!<<< || N || N || N || N || N || N || N || N || N || N || N || N || N || N || N || N || N || N || N || N || N || N || N || N || N || N || N || N || N || || || || || 0 || 0 || 0 || 0 || three_attempts || SECRET || --- the doubts: 1) why it accept my login with password? i *don't* have a password at mysql.user schema 2) why i didn't received a dialog like in heidisql? 3) there's a auth trace or auth verbose option at client side? or any other command to check if i loaded the dialog plugin? -- Roberto Spadim

2 3

[Maria-developers] SQL Accelerator FPGA/ASIC
by Nasser Ghoseiri 23 Oct '14

23 Oct '14

Dear Serg, Following our little chat in IRC, I'm writing this email to explain in more detail what the idea is. My name is Nasser GHOSEIR (Founder, CTO of Butterfly labs), which I must note that this project is not from Butterfly Labs, but will be a new company in Europe. Our idea is to find a way to accelerate SQL query processing by either: 1) Creating an FPGA solution (Which will later evolve into ASIC), that has like 400 processors in itself, allow distribute calculation of some kind 2) Creating an FPGA/ASIC solution that performs large number of unrelated tasks (such as addition, multiplication, etc) in parallel 3) Creating an FPGA/ASIC solution that allows very high-speed access to data with some PRE-PROCESSING involved to accelerate the calculation 4) Creating a very high-speed storage solution, but no pre-processing. An example to give is for 200,000 rows, imagine: "SELECT A+B, C FROM TableX WHERE C > 0". If there are 200,000 records, the processor has to perform 200,000 additions (A+B). A CPU will handle these additions one-by-one (or maybe few-by-few if SSE, etc is used). However, an FPGA/ASIC solution can perform 1000 additions in a single-cycle. This results in acceleration. Now, to what extend this can be effective, or what other solutions (maybe string processing?) can be implemented to accelerate the SQL processing is a question to me right now. But we do have experience in making extremely fast processors (our Butterfly Labs Monarch chip performs 400 billion double-SHA256 hashes in 1 second, around 20,000 times faster than the best intel XEON processor can do only twenty million hashes per second). The project is in brain-storming phase, and we are aware that this solution will be useful to large enterprises, or companies that have to deal with 200,000 rows in a single "SELECT" query. The idea is to integrate some features in hardware, and then re-write a portion of MariaDB to take advantage of the new resources. Also, it is possible that users/companies will need to re-write their query to make it compliant. You can reach my by mail at cdmcsd(a)yahoo.com or by phone at +33 6 72 17 26 19 (France). Best Regards, Nasser GHOSEIRI

1 0

[Maria-developers] COMPOUND STATEMENT fails on Windows.
by Peter Laursen 22 Oct '14

22 Oct '14

I tried to comment on this Blog https://blog.mariadb.org/mariadb-10-1-1-compound-statements/, but it won't let me use my (corporate) gmail address for authencticaton. So I send this mail instead: I tried the example provided on Windows (both server and client are Win7 64 bit) using the command line client shipped with MariaDB 10.1: Enter password: ******** Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 5 Server version: 10.1.1-MariaDB mariadb.org binary distribution Copyright (c) 2000, 2014, Oracle, SkySQL Ab and others. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. MariaDB [(none)]> use test; Database changed MariaDB [test]> IF @have_csv = 'YES' THEN -> CREATE TABLE IF NOT EXISTS general_log ( -> event_time TIMESTAMP(6) NOT NULL, -> user_host MEDIUMTEXT NOT NULL, -> thread_id BIGINT(21) UNSIGNED NOT NULL, -> server_id INTEGER UNSIGNED NOT NULL, -> command_type VARCHAR(64) NOT NULL, -> argument MEDIUMTEXT NOT NULL -> ) engine=CSV CHARACTER SET utf8 comment="General log"; ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '' a t line 9 MariaDB [test]> END IF; Obviously the SEMICOLON after comment="General log" disturbs. But this is no better: MariaDB [test]> IF @have_csv = 'YES' THEN -> CREATE TABLE IF NOT EXISTS general_log ( -> event_time TIMESTAMP(6) NOT NULL, -> user_host MEDIUMTEXT NOT NULL, -> thread_id BIGINT(21) UNSIGNED NOT NULL, -> server_id INTEGER UNSIGNED NOT NULL, -> command_type VARCHAR(64) NOT NULL, -> argument MEDIUMTEXT NOT NULL -> ) engine=CSV CHARACTER SET utf8 comment="General log" -> END IF; ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'END IF' at line 10 MariaDB [test]> What is the problem here? Is it someWindows-specific bug/issue? BTW it is the same in SQLyog (compiled with MariaDB C-API) as in commandline. -- Peter -- Webyog

2 2

Re: [Maria-developers] [Commits] 28fcccc: MDEV-5231: Per query variables from Percona Server (rewritten)
by Sergei Golubchik 22 Oct '14

22 Oct '14

Hi, Sanja! On Oct 13, sanja(a)mariadb.com wrote: > revision-id: 28fccccad32fa8a47dc572b968613e18ff51b634 > parent(s): fec5ab5a56cb9a45c621207620cc85079cddf537 > committer: Oleksandr Byelkin > branch nick: work-maria-10.1-sysvar2 > timestamp: 2014-10-13 22:35:40 +0200 > message: > > MDEV-5231: Per query variables from Percona Server (rewritten) This looks a lot better that the previous patch. Thanks! See my comments below. Some parts of the patch are not used (left-overs from the old implementation?). Some others, I think, are not needed. And some of the old tests don't make much sense, your new tests made them redundant and useless. A couple of comments after reading my previous review: * please add a test for any sysvar with non-trivial implementations of sys_var::session_update(), that is, for a sysvar where old approach doesn't work. Unless, of course, your timestamp tests are exactly that :) * add tests with @@default_engine. For example: SET @@default_engine=MyISAM; SET STATEMENT @@default_engine=MEMORY CREATE TABLE t1 (a int); SHOW CREATE TABLE t1; -- verify the engine SET STATEMENT @@default_engine=MyISAM CREATE TABLE t2 (a int); etc. See other comments below: > diff --git a/mysql-test/r/set_statement.result b/mysql-test/r/set_statement.result > new file mode 100644 > index 0000000..fed937e > --- /dev/null > +++ b/mysql-test/r/set_statement.result > @@ -0,0 +1,941 @@ > +'# SET STATEMENT ..... FOR .... TEST' > +DROP TABLE IF EXISTS t1; > +DROP FUNCTION IF EXISTS myProc; > +DROP PROCEDURE IF EXISTS p1; > +DROP PROCEDURE IF EXISTS p2; > +DROP PROCEDURE IF EXISTS p3; > +DROP PROCEDURE IF EXISTS p4; > +DROP PROCEDURE IF EXISTS p5; > +DROP TABLE IF EXISTS STATEMENT; > +'# Setup database' > +CREATE TABLE t1 (v1 INT, v2 INT); > +INSERT INTO t1 VALUES (1,2); > +INSERT INTO t1 VALUES (3,4); > +'' > +'#------------------ STATEMENT Test 1 -----------------------#' > +'# Initialize variables to known setting' > +SET SESSION sort_buffer_size=100000; > +'' > +'# Pre-STATEMENT variable value' > +SHOW SESSION VARIABLES LIKE 'sort_buffer_size'; > +Variable_name Value > +sort_buffer_size 100000 > +SET STATEMENT sort_buffer_size=150000 FOR SELECT *,@@sort_buffer_size FROM t1; > +v1 v2 @@sort_buffer_size > +1 2 150000 > +3 4 150000 > +'' > +'# Post-STATEMENT variable value' > +SHOW SESSION VARIABLES LIKE 'sort_buffer_size'; > +Variable_name Value > +sort_buffer_size 100000 > +'' > +'#------------------ STATEMENT Test 2 -----------------------#' > +'# Initialize variables to known setting' > +SET SESSION binlog_format=mixed; > +SET SESSION sort_buffer_size=100000; > +'# Pre-STATEMENT variable value' > +SHOW SESSION VARIABLES LIKE 'sort_buffer_size'; > +Variable_name Value > +sort_buffer_size 100000 > +SHOW SESSION VARIABLES LIKE 'binlog_format'; > +Variable_name Value > +binlog_format MIXED > +SET STATEMENT sort_buffer_size=150000, binlog_format=row > +FOR SELECT *,@@sort_buffer_size,@@binlog_format FROM t1; > +v1 v2 @@sort_buffer_size @@binlog_format > +1 2 150000 ROW > +3 4 150000 ROW > +'# Post-STATEMENT variable value' > +SHOW SESSION VARIABLES LIKE 'sort_buffer_size'; > +Variable_name Value > +sort_buffer_size 100000 > +SHOW SESSION VARIABLES LIKE 'binlog_format'; > +Variable_name Value > +binlog_format MIXED > +'' > +'#------------------ STATEMENT Test 3 -----------------------#' > +'# set initial variable value, make prepared statement > +SET SESSION binlog_format=row; > +PREPARE stmt1 FROM 'SET STATEMENT binlog_format=row FOR SELECT *,@@binlog_format FROM t1'; > +'' > +'# Change variable setting' > +SET SESSION binlog_format=mixed; > +'' > +'# Pre-STATEMENT variable value' > +'' > +SHOW SESSION VARIABLES LIKE 'binlog_format'; > +Variable_name Value > +binlog_format MIXED > +'' > +EXECUTE stmt1; > +v1 v2 @@binlog_format > +1 2 ROW > +3 4 ROW > +'' > +'# Post-STATEMENT variable value' > +SHOW SESSION VARIABLES LIKE 'binlog_format'; > +Variable_name Value > +binlog_format MIXED > +'' > +DEALLOCATE PREPARE stmt1; > +'#------------------ STATEMENT Test 4 -----------------------#' > +'# set initial variable value, make prepared statement > +SET SESSION myisam_sort_buffer_size=500000, myisam_repair_threads=1; > +'' > +'# Pre-STATEMENT variable value' > +SHOW SESSION VARIABLES LIKE 'myisam_sort_buffer_size'; > +Variable_name Value > +myisam_sort_buffer_size 500000 > +SHOW SESSION VARIABLES LIKE 'myisam_repair_threads'; > +Variable_name Value > +myisam_repair_threads 1 > +'' > +SET STATEMENT myisam_sort_buffer_size=800000, > +myisam_repair_threads=2 FOR OPTIMIZE TABLE t1; 1. this is not "make prepared statement" as the comment says 2. what does it actually test? you don't verify whether new values had any effect. > +Table Op Msg_type Msg_text > +test.t1 optimize status OK > +'' > +'# Post-STATEMENT variable value' > +SHOW SESSION VARIABLES LIKE 'myisam_sort_buffer_size'; > +Variable_name Value > +myisam_sort_buffer_size 500000 > +SHOW SESSION VARIABLES LIKE 'myisam_repair_threads'; > +Variable_name Value > +myisam_repair_threads 1 > +'' > +'#------------------ STATEMENT Test 5 -----------------------#' > +'# Initialize variables to known setting' > +SET SESSION sort_buffer_size=100000; > +'' > +'# Pre-STATEMENT variable value' > +SHOW SESSION VARIABLES LIKE 'sort_buffer_size'; > +Variable_name Value > +sort_buffer_size 100000 > +'' > +SET STATEMENT sort_buffer_size=150000 FOR SELECT * FROM t2; > +ERROR 42S02: Table 'test.t2' doesn't exist > +'' > +'# Post-STATEMENT variable value' > +SHOW SESSION VARIABLES LIKE 'sort_buffer_size'; > +Variable_name Value > +sort_buffer_size 150000 Looks wrong. The value is not restored in case of an error. > +'' > +'#------------------ STATEMENT Test 6 -----------------------#' > +'# Initialize variables to known setting' > +SET SESSION keep_files_on_create=ON; > +'' > +'# Pre-STATEMENT variable value' > +SHOW SESSION VARIABLES LIKE 'keep_files_on_create'; > +Variable_name Value > +keep_files_on_create ON > +'' > +SET STATEMENT keep_files_on_create=OFF FOR SELECT *,@@keep_files_on_create FROM t1; > +v1 v2 @@keep_files_on_create > +1 2 0 > +3 4 0 How is this test different from other tests above? > +'' > +'# Post-STATEMENT variable value' > +SHOW SESSION VARIABLES LIKE 'keep_files_on_create'; > +Variable_name Value > +keep_files_on_create ON > +'' > +'#------------------ STATEMENT Test 7 -----------------------#' > +'# Initialize variables to known setting' > +SET SESSION max_join_size=2222220000000; > +'' > +'# Pre-STATEMENT variable value' > +SHOW SESSION VARIABLES LIKE 'max_join_size'; > +Variable_name Value > +max_join_size 2222220000000 > +'' > +SET STATEMENT max_join_size=1000000000000 FOR SELECT *,@@max_join_size FROM t1; > +v1 v2 @@max_join_size > +1 2 1000000000000 > +3 4 1000000000000 Same question - how is this test different from other tests above? > +'' > +'# Post-STATEMENT variable value' > +SHOW SESSION VARIABLES LIKE 'max_join_size'; > +Variable_name Value > +max_join_size 2222220000000 > +'' > +'#------------------Test 8-----------------------#' > +'# Initialize test variables' > +SET SESSION myisam_sort_buffer_size=500000, > +myisam_repair_threads=1, > +sort_buffer_size = 200000, > +max_join_size=2222220000000, > +keep_files_on_create=ON; > +'' > +'# LONG ' > +SHOW SESSION VARIABLES LIKE 'sort_buffer_size'; > +Variable_name Value > +sort_buffer_size 200000 > +SET STATEMENT sort_buffer_size = 100000 > +FOR SHOW SESSION VARIABLES LIKE 'sort_buffer_size'; > +Variable_name Value > +sort_buffer_size 100000 > +SHOW SESSION VARIABLES LIKE 'sort_buffer_size'; > +Variable_name Value > +sort_buffer_size 200000 > +'' > +'# MY_BOOL ' > +SHOW SESSION VARIABLES LIKE 'keep_files_on_create'; > +Variable_name Value > +keep_files_on_create ON > +SET STATEMENT keep_files_on_create=OFF > +FOR SHOW SESSION VARIABLES LIKE 'keep_files_on_create'; > +Variable_name Value > +keep_files_on_create OFF > +SHOW SESSION VARIABLES LIKE 'keep_files_on_create'; > +Variable_name Value > +keep_files_on_create ON > +'' > +'# INT/LONG ' > +SHOW SESSION VARIABLES LIKE 'myisam_repair_threads'; > +Variable_name Value > +myisam_repair_threads 1 > +SET STATEMENT myisam_repair_threads=2 > +FOR SHOW SESSION VARIABLES LIKE 'myisam_repair_threads'; > +Variable_name Value > +myisam_repair_threads 2 > +SHOW SESSION VARIABLES LIKE 'myisam_repair_threads'; > +Variable_name Value > +myisam_repair_threads 1 > +'' > +'# ULONGLONG ' > +SHOW SESSION VARIABLES LIKE 'max_join_size'; > +Variable_name Value > +max_join_size 2222220000000 > +SET STATEMENT max_join_size=2000000000000 > +FOR SHOW SESSION VARIABLES LIKE 'max_join_size'; > +Variable_name Value > +max_join_size 2000000000000 > +SHOW SESSION VARIABLES LIKE 'max_join_size'; > +Variable_name Value > +max_join_size 2222220000000 > +'' > +'#------------------Test 9-----------------------#' > +'# set initial variable values > +SET SESSION myisam_sort_buffer_size=500000, > +myisam_repair_threads=1, > +sort_buffer_size=100000, > +binlog_format=mixed, > +keep_files_on_create=ON, > +max_join_size=2222220000000; > +'' > +'' > +'# Pre-STATEMENT variable value > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +CREATE FUNCTION myProc (cost DECIMAL(10,2)) > +RETURNS DECIMAL(10,2) > +SQL SECURITY DEFINER > +tax: BEGIN > +DECLARE order_tax DECIMAL(10,2); > +SET order_tax = cost * .05; > +RETURN order_tax; > +END| > +'' > +'# During Execution values > +SET STATEMENT myisam_sort_buffer_size=400000, > +myisam_repair_threads=2, > +sort_buffer_size=200000, > +binlog_format=row, > +keep_files_on_create=OFF, > +max_join_size=4444440000000 FOR > +SELECT myProc(123.45); > +myProc(123.45) > +6.17 How does this verify that SET STATEMENT had any effect? > +'' > +'# Post-STATEMENT No 1 variable value Pre-STATEMENT for No 2' > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +SET STATEMENT myisam_sort_buffer_size=400000, > +myisam_repair_threads=2, > +sort_buffer_size=200000, > +binlog_format=row, > +keep_files_on_create=OFF, > +max_join_size=4444440000000 FOR > +DROP FUNCTION myProc; How does this verify that SET STATEMENT had any effect? > +'' > +'# Post-STATEMENT No 2 variable value > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'#------------------Test 10-----------------------#' > +'# set initial variable values > +SET SESSION myisam_sort_buffer_size=500000, > +myisam_repair_threads=1, > +sort_buffer_size=100000, > +binlog_format=mixed, > +keep_files_on_create=ON, > +max_join_size=2222220000000; > +'' > +'# Pre-STATEMENT variable value > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +SET STATEMENT myisam_sort_buffer_size=400000, > +myisam_repair_threads=2, > +sort_buffer_size=200000, > +binlog_format=row, > +keep_files_on_create=OFF, > +max_join_size=4444440000000 FOR > +PREPARE stmt2 > +FROM 'SELECT * FROM t1'; > +'' > +'Test No 1 Post Value & Test 2 Pre values' > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +SET STATEMENT myisam_sort_buffer_size=400000, > +myisam_repair_threads=2, > +sort_buffer_size=200000, > +binlog_format=row, > +keep_files_on_create=OFF, > +max_join_size=4444440000000 FOR > +EXECUTE stmt2; > +v1 v2 > +1 2 > +3 4 How does this verify that SET STATEMENT had any effect? Add @@binlog_format and @@sort_buffer_size to the SELECT and use different values for @@sort_buffer_size in SET STATEMENT for PREPARE and EXECUTE. Ok, I see that you do it below. Then how is this test useful? > +'' > +'# Post-STATEMENT No 2 > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +DEALLOCATE PREPARE stmt2; > +'' > +'#------------------Test 11-----------------------#' > +'# set initial variable values > +SET SESSION myisam_sort_buffer_size=500000, > +myisam_repair_threads=1, > +sort_buffer_size=100000, > +binlog_format=mixed, > +keep_files_on_create=ON, > +max_join_size=2222220000000; > +'' > +'' > +'# Pre-STATEMENT variable value > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +SET STATEMENT myisam_sort_buffer_size=400000, > +myisam_repair_threads=2, > +sort_buffer_size=200000, > +keep_files_on_create=OFF, > +max_join_size=4444440000000 FOR > +PREPARE stmt1 FROM > +'SET STATEMENT binlog_format=row FOR SELECT *,@@binlog_format FROM t1'; please also add @@sort_buffer_size to this SELECT > +'' > +'Test No 1 Post Value & Test 2 Pre values' > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +SET STATEMENT myisam_sort_buffer_size=400000, > +myisam_repair_threads=2, > +sort_buffer_size=200000, > +keep_files_on_create=OFF, > +max_join_size=4444440000000 FOR > +EXECUTE stmt1; > +v1 v2 @@binlog_format > +1 2 ROW > +3 4 ROW > +'' > +'# Post-STATEMENT No 2 > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +'#------------------Test 12-----------------------#' > +'# set initial variable values > +SET SESSION myisam_sort_buffer_size=500000, > +myisam_repair_threads=1, > +sort_buffer_size=100000, > +binlog_format=mixed, > +keep_files_on_create=ON, > +max_join_size=2222220000000; > +'' > +'' > +'# Pre-STATEMENT variable value > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +SET STATEMENT myisam_sort_buffer_size=400000, > +myisam_repair_threads=2, > +sort_buffer_size=200000, > +binlog_format=row, > +keep_files_on_create=OFF, > +max_join_size=4444440000000 FOR > +CREATE PROCEDURE p1() BEGIN > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +END| > +'' > +'Test No 1 Post Value & Test 2 Pre values' > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +SET STATEMENT myisam_sort_buffer_size=400000, > +myisam_repair_threads=2, > +sort_buffer_size=200000, > +binlog_format=row, > +keep_files_on_create=OFF, > +max_join_size=4444440000000 FOR Use different values here, not same as in the previous SET STATEMENT > +CALL p1(); > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +400000 2 200000 ROW 0 4444440000000 > +'' > +'# Post-STATEMENT No 2 > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +'#------------------Test 13-----------------------#' > +'# set initial variable values > +SET SESSION myisam_sort_buffer_size=500000, > +myisam_repair_threads=1, > +sort_buffer_size=100000, > +binlog_format=mixed, > +keep_files_on_create=ON, > +max_join_size=2222220000000; > +'' > +'' > +CREATE PROCEDURE p2() BEGIN > +SET STATEMENT myisam_sort_buffer_size=400000, > +myisam_repair_threads=3, > +sort_buffer_size=300000, > +binlog_format=mixed, > +keep_files_on_create=OFF, > +max_join_size=3333330000000 FOR > +CALL p1(); > +END| > +'' > +'# Pre-STATEMENT variable value > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +SET STATEMENT myisam_sort_buffer_size=400000, > +myisam_repair_threads=2, > +sort_buffer_size=200000, > +binlog_format=row, > +keep_files_on_create=OFF, > +max_join_size=4444440000000 FOR > +CALL p2(); > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +400000 3 300000 MIXED 0 3333330000000 > +'' > +'# Post-STATEMENT > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +'#------------------Test 14-----------------------#' > +'# set initial variable values > +SET SESSION myisam_sort_buffer_size=500000, > +myisam_repair_threads=1, > +sort_buffer_size=100000, > +binlog_format=mixed, > +keep_files_on_create=ON, > +max_join_size=2222220000000; > +'' > +'' > +CREATE PROCEDURE p3() BEGIN > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +SET STATEMENT myisam_sort_buffer_size=320000, > +myisam_repair_threads=2, > +sort_buffer_size=220022, > +binlog_format=row, > +keep_files_on_create=ON, > +max_join_size=2222220000000 FOR > +CALL p2(); > +END| > +'' > +'# Pre-STATEMENT variable value > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +SET STATEMENT myisam_sort_buffer_size=400000, > +myisam_repair_threads=2, > +sort_buffer_size=200000, > +binlog_format=row, > +keep_files_on_create=OFF, > +max_join_size=4444440000000 FOR > +CALL p3(); > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +400000 2 200000 ROW 0 4444440000000 > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +400000 3 300000 MIXED 0 3333330000000 > +'' > +'# Post-STATEMENT > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +'' > +'' > +'#------------------Test 15-----------------------#' > +'# set initial variable values > +SET SESSION myisam_sort_buffer_size=500000, > +myisam_repair_threads=1, > +sort_buffer_size=100000, > +binlog_format=mixed, > +keep_files_on_create=ON, > +max_join_size=2222220000000; > +'' > +'' > +CREATE PROCEDURE p4() BEGIN > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +SET STATEMENT myisam_sort_buffer_size=320000, > +myisam_repair_threads=2, > +sort_buffer_size=220022, > +binlog_format=row, > +keep_files_on_create=ON, > +max_join_size=2222220000000 FOR > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +SET STATEMENT myisam_sort_buffer_size=320000, > +myisam_repair_threads=2, > +sort_buffer_size=220022, > +binlog_format=row, > +keep_files_on_create=ON, > +max_join_size=2222220000000 FOR > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +SET STATEMENT myisam_sort_buffer_size=320000, > +myisam_repair_threads=2, > +sort_buffer_size=220022, > +binlog_format=row, > +keep_files_on_create=ON, > +max_join_size=2222220000000 FOR > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +END| > +'' > +'# Pre-STATEMENT variable value > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +SET STATEMENT myisam_sort_buffer_size=400000, > +myisam_repair_threads=2, > +sort_buffer_size=200000, > +binlog_format=row, > +keep_files_on_create=OFF, > +max_join_size=4444440000000 FOR > +CALL p4(); > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +400000 2 200000 ROW 0 4444440000000 > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +320000 2 220022 ROW 1 2222220000000 > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +320000 2 220022 ROW 1 2222220000000 > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +320000 2 220022 ROW 1 2222220000000 > +'' > +'# Post-STATEMENT > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +'#------------------Test 16-----------------------#' > +'' > +'# Pre-STATEMENT variable value > +SELECT @@sql_mode; > +@@sql_mode > + > +'' > +'' > +SET STATEMENT sql_mode='ansi' FOR PREPARE stmt FROM 'SELECT "t1".* FROM t1'; > +execute stmt; > +v1 v2 > +1 2 > +3 4 Ok, that's getting interesting. Now before deallocating, try to trigger automatic reprepare. e.g. ALTER TABLE t1 ADD COLUMN v3 int; and then execute it again. also do the same with SP. I mean, set statement sql_mode=ansi create procedure this procedure uses ansi quotes. verify that the correct sql mode (ansi) is recorded in mysql.proc table. > +deallocate prepare stmt; > +'' > +'# Post-STATEMENT > +SELECT @@sql_mode; > +@@sql_mode > + > +'' > +'' > +'#------------------Test 17-----------------------#' > +'# set initial variable values > +SET SESSION myisam_sort_buffer_size=500000, > +myisam_repair_threads=1, > +sort_buffer_size=100000, > +binlog_format=mixed, > +keep_files_on_create=ON, > +max_join_size=2222220000000; > +'' > +'# Pre-STATEMENT variable value > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +SET STATEMENT myisam_sort_buffer_size=320000, > +myisam_repair_threads=2, > +sort_buffer_size=220022, > +binlog_format=row, > +keep_files_on_create=ON, > +max_join_size=2222220000000 > +FOR SET SESSION > +myisam_sort_buffer_size=260000, > +myisam_repair_threads=3, > +sort_buffer_size=230013, > +binlog_format=row, > +keep_files_on_create=ON, > +max_join_size=2323230000000; Add a couple of other variables to SET SESSION. The point - have some variables in SET SESSION that are *not* present in SET STATEMENT. Verify that their values were correctly set. Ok, I see that you do it below. Then how is this test useful? > +'' > +'# Post-STATEMENT > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +'#------------------Test 18-----------------------#' > +'# set initial variable values > +SET SESSION myisam_sort_buffer_size=500000, > +myisam_repair_threads=1, > +sort_buffer_size=100000, > +binlog_format=mixed, > +keep_files_on_create=ON, > +max_join_size=2222220000000; > +'' > +'# Pre-STATEMENT variable value > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +CREATE PROCEDURE p5() BEGIN > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +SET SESSION > +myisam_sort_buffer_size=260000, > +myisam_repair_threads=3, > +sort_buffer_size=230013, > +binlog_format=row, > +keep_files_on_create=ON, > +max_join_size=2323230000000; > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +END| How is this test different from other tests above? > +'' > +'' > +SET STATEMENT myisam_sort_buffer_size=400000, > +myisam_repair_threads=2, > +sort_buffer_size=200000, > +binlog_format=row, > +keep_files_on_create=OFF, > +max_join_size=4444440000000 FOR > +CALL p5(); > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +400000 2 200000 ROW 0 4444440000000 > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +260000 3 230013 ROW 1 2323230000000 > +'' > +'# Post-STATEMENT > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +'#------------------Test 19-----------------------#' > +SET STATEMENT max_error_count=100 FOR; > +ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '' at line 1 > +SET STATEMENT max_error_count=100 INSERT t1 VALUES (1,2); > +ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'INSERT t1 VALUES (1,2)' at line 1 > +SET STATEMENT FOR INSERT INTO t1 VALUES (1,2); > +ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'FOR INSERT INTO t1 VALUES (1,2)' at line 1 > +SET max_error_count=100 FOR INSERT INTO t1 VALUES (1,2); > +ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'FOR INSERT INTO t1 VALUES (1,2)' at line 1 > +SET max_error_count=100 FOR INSERT INTO t1 VALUES (1,2); Same erroneous statement tested twice? > +ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'FOR INSERT INTO t1 VALUES (1,2)' at line 1 > +SET STATEMENT GLOBAL max_error_count=100 FOR INSERT INTO t1 VALUES (1,2); > +ERROR HY000: Unknown system variable 'GLOBAL' > +SET STATEMENT @@global.max_error_count=100 FOR INSERT INTO t1 VALUES (1,2); > +ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '@@global.max_error_count=100 FOR INSERT INTO t1 VALUES (1,2)' at line 1 > +'' > +'' > +'#------------------Test 20-----------------------#' > +SET STATEMENT connect_timeout=100 FOR INSERT INTO t1 VALUES (1,2); > +ERROR HY000: Variable 'connect_timeout' is a GLOBAL variable and should be set with SET GLOBAL > +'' > +'' > +'#------------------Test 21-----------------------#' > +SELECT @@myisam_sort_buffer_size, @@sort_buffer_size; > +@@myisam_sort_buffer_size @@sort_buffer_size > +500000 100000 > +SET STATEMENT myisam_sort_buffer_size = 700000, sort_buffer_size = 3000000 > +FOR SET STATEMENT myisam_sort_buffer_size=200000 > +FOR SELECT @@myisam_sort_buffer_size, @@sort_buffer_size; > +@@myisam_sort_buffer_size @@sort_buffer_size > +200000 100000 > +SELECT @@myisam_sort_buffer_size, @@sort_buffer_size; > +@@myisam_sort_buffer_size @@sort_buffer_size > +500000 100000 > +'' > +'' > +'#------------------Test 22-----------------------#' > +CREATE TABLE STATEMENT(a INT); > +DROP TABLE STATEMENT; > +'' > +'# Cleanup' > +DROP TABLE t1; > +DROP PROCEDURE p1; > +DROP PROCEDURE p2; > +DROP PROCEDURE p3; > +DROP PROCEDURE p4; > +DROP PROCEDURE p5; > +CREATE TABLE t1 (v1 INT, v2 INT); > +PREPARE stmt > +FROM 'SELECT * FROM t1'; > +SET STATEMENT myisam_sort_buffer_size=400000, > +myisam_repair_threads=(select max(v1) from t1), > +sort_buffer_size=200000, > +binlog_format=row, > +keep_files_on_create=OFF, > +max_join_size=4444440000000 FOR > +EXECUTE stmt; > +ERROR HY000: SET STATEMENT does not support using tables in the assignment variables expressions Why do you need a prepared statement to test that error? > +deallocate prepare stmt; > +drop table t1; > +set session binlog_format=mixed; > +PREPARE stmt1 FROM 'SELECT @@binlog_format'; > +execute stmt1; > +@@binlog_format > +MIXED > +set statement binlog_format=row for execute stmt1; > +@@binlog_format > +ROW > +execute stmt1; > +@@binlog_format > +MIXED > +deallocate prepare stmt1; > +set statement binlog_format=row for PREPARE stmt1 FROM 'SELECT @@binlog_format'; > +execute stmt1; > +@@binlog_format > +MIXED > +execute stmt1; > +@@binlog_format > +MIXED > +deallocate prepare stmt1; > +PREPARE stmt1 FROM 'set statement binlog_format=row for SELECT @@binlog_format'; > +execute stmt1; > +@@binlog_format > +ROW > +execute stmt1; > +@@binlog_format > +ROW > +deallocate prepare stmt1; > +set session binlog_format=default; > +set session binlog_format=mixed; > +SET STATEMENT sort_buffer_size=150000 FOR set session binlog_format=row; > +SELECT @@binlog_format; > +@@binlog_format > +ROW those are good tests, and they make many of the tests above unnecessary > +#Correctly set timestamp > +set session timestamp=4646464; > +select @@timestamp != 4646464; > +@@timestamp != 4646464 > +0 > +select @@timestamp != 4646464; > +@@timestamp != 4646464 > +0 > +#Correctly returned normal behaviour > +set session timestamp=default; > +select @@timestamp != 4646464; > +@@timestamp != 4646464 > +1 > +select @@timestamp != 4646464; > +@@timestamp != 4646464 > +1 > +#here timestamp should be set only for the statement then restored default > +set statement timestamp=4646464 for select @@timestamp; > +@@timestamp > +4646464.000000 > +set @save_tm=@@timestamp; > +select @@timestamp != 4646464; > +@@timestamp != 4646464 > +1 > +select @@timestamp != 4646464; > +@@timestamp != 4646464 > +1 > +select @@timestamp != @save_tm; > +@@timestamp != @save_tm > +1 > diff --git a/sql/set_var.cc b/sql/set_var.cc > index bae6511..20e3040 100644 > --- a/sql/set_var.cc > +++ b/sql/set_var.cc > @@ -147,7 +147,7 @@ void sys_var_end() > flags(flags_arg), show_val_type(show_val_type_arg), > guard(lock), offset(off), on_check(on_check_func), on_update(on_update_func), > deprecation_substitute(substitute), > - is_os_charset(FALSE) > + is_os_charset(FALSE), default_val(TRUE) I've asked below why is that needed. > { > /* > There is a limitation in handle_options() related to short options: > @@ -695,7 +695,8 @@ int sql_set_variables(THD *thd, List<set_var_base> *var_list) > } > > err: > - free_underlaid_joins(thd, &thd->lex->select_lex); > + if (free) > + free_underlaid_joins(thd, &thd->lex->select_lex); Ok, so in normal SET you want to free_underlaid_joins, and in SET STATEMENT you don't. Right? > DBUG_RETURN(error); > } > > diff --git a/sql/item_func.cc b/sql/item_func.cc > index 2b89aa0..651e0b1 100644 > --- a/sql/item_func.cc > +++ b/sql/item_func.cc > @@ -5556,7 +5556,7 @@ longlong Item_func_get_user_var::val_int() > tmp_var_list.push_back(new set_var_user(new Item_func_set_user_var(name, > new Item_null()))); > /* Create the variable */ > - if (sql_set_variables(thd, &tmp_var_list)) > + if (sql_set_variables(thd, &tmp_var_list, true)) Make if 'false', not 'true' here. It doesn't make any practical difference (there are no joins, the variable is set to Item_null, explicitly), but makes the logic a bit clearer (standalone SET statement = free joins, a SET that is part of another statement = don't free joins). Alternatively (and better) approach would be to remove sql_set_variables() here and use the same code that Item_func_set_user_var does. And then sql_set_variables() will be only used in SET variants, where a sysvar can be used. Here sysvars aren't possible, so sql_set_variables() is misleading. > { > thd->lex= sav_lex; > goto err; > diff --git a/sql/set_var.h b/sql/set_var.h > index e48f394..9ee914b 100644 > --- a/sql/set_var.h > +++ b/sql/set_var.h > @@ -118,6 +119,9 @@ class sys_var > */ > bool set_default(THD *thd, set_var *var); > bool update(THD *thd, set_var *var); > + void stmt_update(THD *thd) { > + on_update && on_update(this, thd, OPT_SESSION); > + } Doesn't seem to be used anywhere in the patch > > String *val_str_nolock(String *str, THD *thd, const uchar *value); > longlong val_int(bool *is_null, THD *thd, enum_var_type type, const LEX_STRING *base); > diff --git a/sql/share/errmsg-utf8.txt b/sql/share/errmsg-utf8.txt > index 233bb83..4072af4 100644 > --- a/sql/share/errmsg-utf8.txt > +++ b/sql/share/errmsg-utf8.txt > @@ -7110,3 +7110,5 @@ ER_SLAVE_SKIP_NOT_IN_GTID > eng "When using GTID, @@sql_slave_skip_counter can not be used. Instead, setting @@gtid_slave_pos explicitly can be used to skip to after a given GTID position." > ER_STATEMENT_TIMEOUT 70100 > eng "Query execution was interrupted (max_statement_time exceeded)" > +ER_SET_STATEMENT_TABLES_IS_NOT_SUPPORTED > + eng "SET STATEMENT does not support using tables in the assignment variables expressions" Better write it as ER_SUBQUERIES_NOT_SUPPORTED eng "%s does not support subqueries or stored functions." and use it for SET STATEMENT: my_error(ER_SUBQUERIES_NOT_SUPPORTED, MYF(0), "SET STATEMENT"); and also use it for KILL: - my_error(ER_NOT_SUPPORTED_YET, MYF(0), "Usage of subqueries or stored " - "function calls as part of this statement"); + my_error(ER_SUBQUERIES_NOT_SUPPORTED, MYF(0), "KILL"); > diff --git a/sql/sql_class.h b/sql/sql_class.h > index d7bbfc3..0f6b39e 100644 > --- a/sql/sql_class.h > +++ b/sql/sql_class.h > @@ -3711,6 +3711,10 @@ class THD :public Statement, > void rgi_unlock_temporary_tables(); > bool rgi_have_temporary_tables(); > public: > + inline MEM_ROOT *get_execution_mem_root() > + { > + return &main_mem_root; > + } Doesn't seem to be used anywhere in this patch > /* > Flag, mutex and condition for a thread to wait for a signal from another > thread. > diff --git a/sql/sql_lex.h b/sql/sql_lex.h > index 5e9c7b9..03861d1 100644 > --- a/sql/sql_lex.h > +++ b/sql/sql_lex.h > @@ -2379,6 +2381,8 @@ struct LEX: public Query_tables_list > required a local context, the parser pops the top-most context. > */ > List<Name_resolution_context> context_stack; > + /* true if SET STATEMENT ... FOR ... statement is use, false otherwise */ > + bool set_statement; This doesn't seem to be needed, I've checked all places where set_statement is used below. > > SQL_I_List<ORDER> proc_list; > SQL_I_List<TABLE_LIST> auxiliary_table_list, save_list; > diff --git a/sql/sql_parse.cc b/sql/sql_parse.cc > index d8906b2..eff0e08 100644 > --- a/sql/sql_parse.cc > +++ b/sql/sql_parse.cc > @@ -2637,6 +2637,98 @@ static bool do_execute_sp(THD *thd, sp_head *sp) > thd->get_binlog_format(&orig_binlog_format, > &orig_current_stmt_binlog_format); > > + if (lex->set_statement && !lex->stmt_var_list.is_empty()) Why do you check lex->set_statement? I'd think that if (!lex->stmt_var_list.is_empty()) is quite enough. > + { > + DBUG_PRINT("info", ("SET STATEMENT %d vars", lex->stmt_var_list.elements)); > + > + lex->old_var_list.empty(); > + List_iterator_fast<set_var_base> it(lex->stmt_var_list); > + set_var_base *var; > + while ((var=it++)) > + { > + DBUG_ASSERT(var->is_system()); > + set_var *o= NULL, *v= (set_var*)var; > + if (v->var->is_default()) > + o= new set_var(v->type, v->var, &v->base, NULL); Why do you treat default values differently? > + else > + { > + switch (v->var->option.var_type) > + { > + case GET_BOOL: > + case GET_INT: > + case GET_LONG: > + case GET_LL: > + { > + bool null_value; > + longlong val= v->var->val_int(&null_value, thd, v->type, &v->base); > + o= new set_var(v->type, v->var, &v->base, > + (null_value ? > + (Item *)new Item_null() : > + (Item *)new Item_int(val))); > + } > + break; > + case GET_UINT: > + case GET_ULONG: > + case GET_ULL: > + { > + bool null_value; > + ulonglong val= v->var->val_int(&null_value, thd, v->type, &v->base); > + o= new set_var(v->type, v->var, &v->base, > + (null_value ? > + (Item *)new Item_null() : > + (Item *)new Item_uint(val))); > + } > + break; > + case GET_DOUBLE: > + { > + bool null_value; > + double val= v->var->val_real(&null_value, thd, v->type, &v->base); > + o= new set_var(v->type, v->var, &v->base, > + (null_value ? > + (Item *)new Item_null() : > + (Item *)new Item_float(val, 1))); > + } > + break; > + default: > + case GET_NO_ARG: > + case GET_DISABLED: > + DBUG_ASSERT(0); > + case 0: > + case GET_FLAGSET: > + case GET_ASK_ADDR: > + case GET_TYPE_MASK: > + case GET_ENUM: > + case GET_SET: > + case GET_STR: > + case GET_STR_ALLOC: > + { > + char buff[STRING_BUFFER_USUAL_SIZE]; > + String tmp(buff, sizeof(buff), v->var->charset(thd)),*val; > + val= v->var->val_str(&tmp, thd, v->type, &v->base); > + if (val) > + { > + Item_string *str= new Item_string(v->var->charset(thd)); > + str->set_str_with_copy(val->ptr(), val->length()); > + o= new set_var(v->type, v->var, &v->base, str); > + } > + else > + o= new set_var(v->type, v->var, &v->base, new Item_null()); > + } > + break; > + } > + } > + DBUG_ASSERT(o); > + lex->old_var_list.push_back(o); > + } > + if (thd->is_error() || > + (res= sql_set_variables(thd, &lex->stmt_var_list, false))) > + { > + if (!thd->is_error()) > + my_error(ER_WRONG_ARGUMENTS, MYF(0), "SET"); > + goto error; > + } > + } > + > /* > Force statement logging for DDL commands to allow us to update > privilege, system or statistic tables directly without the updates > @@ -2812,6 +2904,12 @@ static bool do_execute_sp(THD *thd, sp_head *sp) > } > case SQLCOM_EXECUTE: > { > + /* > + Clean up free_list here from Items used in setting statement variables > + to present clear item list to Prepared_statement::execute_loop which > + need it empty because switch arena (and so item list) on statement arena. > + */ > + //free_items(thd->free_list); thd->free_list= NULL; Eh? Forgot to remove it? > mysql_sql_stmt_execute(thd); > break; > } > diff --git a/sql/sql_plugin.cc b/sql/sql_plugin.cc > index 9ae3d79..4b54989 100644 > --- a/sql/sql_plugin.cc > +++ b/sql/sql_plugin.cc > @@ -4078,3 +4078,48 @@ static void restore_ptr_backup(uint n, st_ptr_backup *backup) > (backup++)->restore(); > } > > + > +/** > + Create deep copy of system_variables instance. > +*/ > +struct system_variables * > +copy_system_variables(const struct system_variables *src) Doesn't seem to be used anywhere in the patch > +{ > + struct system_variables *dst; > + > + DBUG_ASSERT(src); > + > + dst= (struct system_variables *) > + sql_alloc(sizeof(struct system_variables)); > + if (!dst) > + return NULL; > + *dst= *src; > + > + if (dst->dynamic_variables_ptr) > + { > + if (!(dst->dynamic_variables_ptr= > + (char *)my_malloc(dst->dynamic_variables_size, MYF(MY_WME | MY_FAE)))) > + return NULL; > + memcpy(dst->dynamic_variables_ptr, > + src->dynamic_variables_ptr, > + src->dynamic_variables_size); > + } > + > + mysql_mutex_lock(&LOCK_plugin); > + dst->table_plugin= > + intern_plugin_lock(NULL, src->table_plugin); > + mysql_mutex_unlock(&LOCK_plugin); > + > + return dst; > +} > + > +void free_system_variables(struct system_variables *v) Doesn't seem to be used anywhere in the patch > +{ > + DBUG_ASSERT(v); > + > + mysql_mutex_lock(&LOCK_plugin); > + intern_plugin_unlock(NULL, v->table_plugin); > + mysql_mutex_unlock(&LOCK_plugin); > + > + my_free(v->dynamic_variables_ptr); > +} > diff --git a/sql/sql_yacc.yy b/sql/sql_yacc.yy > index e7fcdfb..d140597 100644 > --- a/sql/sql_yacc.yy > +++ b/sql/sql_yacc.yy > @@ -14416,8 +14420,52 @@ set: > } > start_option_value_list > {} > + | SET STATEMENT_SYM > + { > + LEX *lex= Lex; > + mysql_init_select(lex); > + lex->sql_command= SQLCOM_SET_OPTION; > + /* Don't clear var_list in the case of recursive statement */ What's "recursive set statement"? SET STATEMENT a=1 FOR SET STATEMENT ? I'd simply return a syntax error here. Like MYSQL_YYABORT_UNLESS(lex->stmt_var_list.is_empty()); > + if (!lex->set_statement) > + { > + lex->var_list.empty(); > + lex->stmt_var_list.empty(); Why here, and not in lex_start() ? > + } > + lex->autocommit= 0; > + lex->set_statement= true; > + sp_head *sp= lex->sphead; > + if (sp && !sp->is_invoked()) > + { > + sp->m_param_begin= ((yychar == YYEMPTY) ? YYLIP->get_ptr() : YYLIP->get_tok_start()); > + sp->m_param_end= ((yychar == YYEMPTY) ? YYLIP->get_ptr() : YYLIP->get_tok_end()); what's that? > + } > + } > + set_stmt_option_value_following_option_type_list > + { > + LEX *lex= Lex; > + if (lex->query_tables) > + { > + my_error(ER_SET_STATEMENT_TABLES_IS_NOT_SUPPORTED, MYF(0)); why? what about stored functions that use tables internally? > + MYSQL_YYABORT; > + } > + //if (lex->set_statement && lex->stmt_var_list.is_empty()) > + { > + lex->stmt_var_list= lex->var_list; > + lex->var_list.empty(); > + } > + } > + FOR_SYM statement > + {} > ; > > +set_stmt_option_value_following_option_type_list: > + /* > + Only system variables can be used here. If this condition is changed > + please check careful code under lex->option_type == OPT_STATEMENT > + condition on wrong type casts. > + */ > + option_value_following_option_type > + | set_stmt_option_value_following_option_type_list ',' option_value_following_option_type > > // Start of option value list > start_option_value_list: > @@ -14522,6 +14570,14 @@ option_value_following_option_type: > { > LEX *lex= Lex; > > + /* > + Ignore SET STATEMENT variables list on slaves because system > + variables are not replicated except certain variables set the > + values of whose are written to binlog event header and nothing > + additional is required to set them. > + */ > + if (!(thd->slave_thread && lex->set_statement)) > + { Hmm, okay. Alternatively, you can simply make sure that set statement is not written into binlog. Or don't execute assignments in the slave thread. > if ($1.var && $1.var != trg_new_row_fake_var) > { > /* It is a system variable. */ Regards, Sergei

2 3

Re: [Maria-developers] [Commits] Rev 4445: MDEV-6878: Use of uninitialized saved_primary_key in Mrr_ordered_index_reader::resume_read() in file:///home/psergey/dev2/10.0/
by Jeremy Cole 20 Oct '14

20 Oct '14

Sergey, Nice test case. Would you mind cleaning up the end-space in your patch? Also, is there a reason to put the entire body of Mrr_ordered_index_reader::resume_read in the if() instead of an early return? Regards, Jeremy On Thu, Oct 16, 2014 at 11:58 AM, Sergey Petrunya <psergey(a)askmonty.org> wrote: > At file:///home/psergey/dev2/10.0/ > > ------------------------------------------------------------ > revno: 4445 > revision-id: psergey(a)askmonty.org-20141016135713-md92qr4g9fzhitxh > parent: bar(a)mnogosearch.org-20141013083155-j3j31hqhfc8i95y1 > committer: Sergey Petrunya <psergey(a)askmonty.org> > branch nick: 10.0 > timestamp: Thu 2014-10-16 17:57:13 +0400 > message: > MDEV-6878: Use of uninitialized saved_primary_key in > Mrr_ordered_index_reader::resume_read() > > - Make Mrr_ordered_index_reader::resume_read() restore index position > only if it was saved before with > Mrr_ordered_index_reader::interrupt_read(). > === modified file 'mysql-test/r/innodb_mrr_cpk.result' > --- a/mysql-test/r/innodb_mrr_cpk.result 2013-09-26 18:20:15 +0000 > +++ b/mysql-test/r/innodb_mrr_cpk.result 2014-10-16 13:57:13 +0000 > @@ -194,3 +194,41 @@ id select_type table type possible_keys > 2 DERIVED t2 ALL NULL NULL NULL NULL # > set join_cache_level= @tmp_mdev5037; > drop table t0,t1,t2; > +# > +# MDEV-6878: Use of uninitialized saved_primary_key in > Mrr_ordered_index_reader::resume_read() > +# > +create table t0(a int); > +insert into t0 values (0),(1),(2),(3),(4),(5),(6),(7),(8),(9); > +create table t1 ( > +pk varchar(32) character set utf8 primary key, > +kp1 char(32) not null, > +col1 varchar(32), > +key (kp1) > +) engine=innodb; > +insert into t1 > +select > +concat('pk-', 1000 +A.a), > +concat('kp1-', 1000 +A.a), > +concat('val-', 1000 +A.a) > +from test.t0 A ; > +create table t2 as select kp1 as a from t1; > +set join_cache_level=8; > +set optimizer_switch='mrr=on,mrr_sort_keys=on'; > +explain > +select * from t2 straight_join t1 force index(kp1) where t1.kp1=t2.a; > +id select_type table type possible_keys key key_len > ref rows Extra > +1 SIMPLE t2 ALL NULL NULL NULL NULL 10 > +1 SIMPLE t1 ref kp1 kp1 32 test.t2.a 1 > Using join buffer (flat, BKAH join); Key-ordered Rowid-ordered scan > +select * from t2 straight_join t1 force index(kp1) where t1.kp1=t2.a; > +a pk kp1 col1 > +kp1-1000 pk-1000 kp1-1000 val-1000 > +kp1-1001 pk-1001 kp1-1001 val-1001 > +kp1-1002 pk-1002 kp1-1002 val-1002 > +kp1-1003 pk-1003 kp1-1003 val-1003 > +kp1-1004 pk-1004 kp1-1004 val-1004 > +kp1-1005 pk-1005 kp1-1005 val-1005 > +kp1-1006 pk-1006 kp1-1006 val-1006 > +kp1-1007 pk-1007 kp1-1007 val-1007 > +kp1-1008 pk-1008 kp1-1008 val-1008 > +kp1-1009 pk-1009 kp1-1009 val-1009 > +drop table t0,t1,t2; > > === modified file 'mysql-test/t/innodb_mrr_cpk.test' > --- a/mysql-test/t/innodb_mrr_cpk.test 2013-09-26 18:20:15 +0000 > +++ b/mysql-test/t/innodb_mrr_cpk.test 2014-10-16 13:57:13 +0000 > @@ -192,3 +192,35 @@ explain SELECT 1 FROM (SELECT url, id FR > set join_cache_level= @tmp_mdev5037; > > drop table t0,t1,t2; > + > +--echo # > +--echo # MDEV-6878: Use of uninitialized saved_primary_key in > Mrr_ordered_index_reader::resume_read() > +--echo # > +create table t0(a int); > +insert into t0 values (0),(1),(2),(3),(4),(5),(6),(7),(8),(9); > + > +create table t1 ( > + pk varchar(32) character set utf8 primary key, > + kp1 char(32) not null, > + col1 varchar(32), > + key (kp1) > +) engine=innodb; > + > +insert into t1 > +select > + concat('pk-', 1000 +A.a), > + concat('kp1-', 1000 +A.a), > + concat('val-', 1000 +A.a) > +from test.t0 A ; > + > +create table t2 as select kp1 as a from t1; > + > +set join_cache_level=8; > +set optimizer_switch='mrr=on,mrr_sort_keys=on'; > +explain > +select * from t2 straight_join t1 force index(kp1) where t1.kp1=t2.a; > +select * from t2 straight_join t1 force index(kp1) where t1.kp1=t2.a; > + > +drop table t0,t1,t2; > + > + > > === modified file 'sql/multi_range_read.cc' > --- a/sql/multi_range_read.cc 2014-06-06 17:28:42 +0000 > +++ b/sql/multi_range_read.cc 2014-10-16 13:57:13 +0000 > @@ -426,6 +426,7 @@ bool Mrr_ordered_index_reader::set_inter > *space_start += key_len; > > have_saved_rowid= FALSE; > + read_was_interrupted= FALSE; > return FALSE; > } > > @@ -434,6 +435,7 @@ void Mrr_ordered_index_reader::set_no_in > support_scan_interruptions= FALSE; > saved_key_tuple= saved_rowid= saved_primary_key= NULL; /* safety */ > have_saved_rowid= FALSE; > + read_was_interrupted= FALSE; > } > > void Mrr_ordered_index_reader::interrupt_read() > @@ -451,6 +453,7 @@ void Mrr_ordered_index_reader::interrupt > &table->key_info[table->s->primary_key], > table->key_info[table->s->primary_key].key_length); > } > + read_was_interrupted= TRUE; > > /* Save the last rowid */ > memcpy(saved_rowid, file->ref, file->ref_length); > @@ -468,14 +471,17 @@ void Mrr_ordered_index_reader::position( > void Mrr_ordered_index_reader::resume_read() > { > TABLE *table= file->get_table(); > - KEY *used_index= &table->key_info[file->active_index]; > - key_restore(table->record[0], saved_key_tuple, > - used_index, used_index->key_length); > - if (saved_primary_key) > + if (read_was_interrupted) > { > - key_restore(table->record[0], saved_primary_key, > - &table->key_info[table->s->primary_key], > - table->key_info[table->s->primary_key].key_length); > + KEY *used_index= &table->key_info[file->active_index]; > + key_restore(table->record[0], saved_key_tuple, > + used_index, used_index->key_length); > + if (saved_primary_key) > + { > + key_restore(table->record[0], saved_primary_key, > + &table->key_info[table->s->primary_key], > + table->key_info[table->s->primary_key].key_length); > + } > } > } > > @@ -557,8 +563,7 @@ int Mrr_ordered_index_reader::init(handl > is_mrr_assoc= !MY_TEST(mode & HA_MRR_NO_ASSOCIATION); > mrr_funcs= *seq_funcs; > source_exhausted= FALSE; > - if (support_scan_interruptions) > - bzero(saved_key_tuple, key_info->key_length); > + read_was_interrupted= false; > have_saved_rowid= FALSE; > return 0; > } > > === modified file 'sql/multi_range_read.h' > --- a/sql/multi_range_read.h 2013-09-26 18:20:15 +0000 > +++ b/sql/multi_range_read.h 2014-10-16 13:57:13 +0000 > @@ -339,6 +339,12 @@ class Mrr_ordered_index_reader : public > > uchar *saved_key_tuple; /* Saved current key tuple */ > uchar *saved_primary_key; /* Saved current primary key tuple */ > + > + /* > + TRUE<=> saved_key_tuple (and saved_primary_key when applicable) have > + valid values. > + */ > + bool read_was_interrupted; > > static int compare_keys(void* arg, uchar* key1, uchar* key2); > static int compare_keys_reverse(void* arg, uchar* key1, uchar* key2); > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits >

2 1

Re: [Maria-developers] MariaDB 10.0.14 got rid of replication safety warning without warning in release notes?
by Pavel Ivanov 20 Oct '14

20 Oct '14

> Pavel> Which key is "the first duplicate key". > > The first key that has a duplicate value in the order of keys in the KEY > structure that MariaDB provides for the storage engine. I'm definitely not that familiar with the relevant MariaDB code, but I didn't find KEY structure to be passed to any insert-related function. Can you elaborate (preferably with code references) how this restriction is supposed to work? > This is the same order that keys are presented in SHOW CREATE TABLE Why is it safe to assume that keys have the same order on master and on slave? > To help engine writes we are constantly adding new test in the storage > engine test suite, to make it easier for storage engine vendors to > test that their storage engine follows the interface. > We have now added a test to check for the correct behaviour for the > case of INSERT ... ON DUPLICATE KEY. This is very interesting. Can you say which test it is and what exactly does it test? Is it present in 10.0.14 or it's only at the latest 10.0 branch? > Do you have or use a storage engine that doesn't provide the expected value > for the conflicting key ? I don't have a specific example of storage engine that detects duplicate key in different order, but I'm not convinced and don't have hard proof that such engine cannot exist and that InnoDB itself is not susceptible to this problem. I can easily imagine though that even given the particular order of keys in table definition the key reported as having duplicate can depend on the way storage engine stores those keys. BTW, can you point to MySQL documentation and/or comment in the code that indicates this requirement to storage engine implementation? > Pavel> On the related note Release Notes mention the error log flood > Pavel> protection. Why it's not behind flag? How should I disable the feature > Pavel> and get back to the old behavior? > > We already had in MariaDB flood protection for some error > messages. Can you say exactly which error messages are suppressed already? Is there a documentation link about that? > I am happy to look into adding a flag that is usable for any kind of > suppressed messages. > > A couple of ways to do this: > > a) Not suppress anything if you give --log-warnings a value > 2 > b) make LIMIT_UNSAFE_WARNING_ACTIVATION_TIMEOUT a configurable > variable, where 0 would mean no suppression. > > Which of the above, or both, would you prefer? > Or do you have any other suggestion of how to do this? Reading documentation on --log-warnings I don't see how it can be related to the error suppression. So I think making LIMIT_UNSAFE_WARNING_ACTIVATION_TIMEOUT configurable would be much better, though it should apply to all possible error suppressions. Thank you, Pavel On Fri, Oct 17, 2014 at 1:43 PM, Michael Widenius <michael.widenius(a)gmail.com> wrote: > > Hi! > > First sorry for the delayed reply; I am at a conference in Oxford and > didn't have an reliable internet connection until now. > >>>>>> "Pavel" == Pavel Ivanov <pivanof(a)google.com> writes: > > Pavel> Hi, > Pavel> I see that in MariaDB 10.0.14 Michael disabled replication safety > Pavel> warning for the case of INSERT ON DUPLICATE KEY UPDATE with several > Pavel> unique keys with a comment "We should assume that the store engine > Pavel> will report the first duplicate key for this case." > Pavel> http://bazaar.launchpad.net/~maria-captains/maria/10.0/revision/4400.1.3 > > Pavel> What does that mean? > > That we don't anymore give a warning for things that can't happen in > the distributed MariaDB server. > > Pabel> Which key is "the first duplicate key". > > The first key that has a duplicate value in the order of keys in the KEY > structure that MariaDB provides for the storage engine. > > This is the same order that keys are presented in SHOW CREATE TABLE > > Pavel> and why > Pavel> MariaDB relies on an engine behavior which AFAIK is not guaranteed? > > This is guaranteed for all storage engines shipped with MariaDB and > should also be guaranteed for any storage engine that wants to follow > the MariaDB and even the original MySQL storage engine interface. > > The change is also in line with how I designed the part of the storage > engine interface that gives the SQL layer the information about which > key that was conflicted. As we use this for things like INSERT ... ON > DUPLICATE KEY, it's critical that all storage engine provides the same > information in case of conflicts. > > To help engine writes we are constantly adding new test in the storage > engine test suite, to make it easier for storage engine vendors to > test that their storage engine follows the interface. > We have now added a test to check for the correct behaviour for the > case of INSERT ... ON DUPLICATE KEY. > > What is important when it comes to the storage engine interface is > that in all possible cases, a query should work identically against 2 > different storage engines. > > The only 'allowed difference's that I can think of for the moment are: > > - Different storage engines may retrieve rows in different order. > - In case of errors, changes to non transactional tables will not roll > back. > - Automatically generated key values may be different between storage engines. > (but preferably they should not be that). However when things are > replicated the slave, it should get the same values as the master. > > In the case of INSERT ... ON DUPLICATE KEY, it was clear that someone > had broke the storage engine interface, for no good reason. Instead > of users having to depend on 'unsafe, different and undocumented > behavior' for which row as updated, it's better to fix this issue once > and for all, especially as fixing this is trivial. > > Do you have or use a storage engine that doesn't provide the expected value > for the conflicting key ? > > If yes, there is a couple of ways to fix this. I am happy to talk with > you to find a solution that will satisfy your needs and ours. > > I underatand that in many cases it's not critical which unique key the > storage engine reports as duplicate. However in some cases, it's > critical to get the first key. > > I could add a new extra call to inform the engine that we now require > the first duplicate key and not just any conflicting key. This would > be used for example for INSERT ... ON DUPLICATE KEY ... > > I would prefer to have the check for which key was duplicated in the > storage engine. However, if critically needed, and assuming there is more > than one storage engine that needs this, we could add a check on the > handler level that will do a lookup on all unique keys to find the > first duplicate one. > > What solution would you prefer? > > Pavel> And why this serious change in behavior wasn't mentioned in the > Pavel> Release Notes? > > I didn't see it as a a serious change as it's only about removing a > warning that has no value for any normal MariaDB user (who only use > storage engines supplied with MariadB) and which actually caused > serious problems for some MariaDB users. > > That said, I agree it was a mistake from my part to not notice that > this was missing in the release notes. I will try to do better in the > future and check that the release notes correctly reflects any changes > that I have done. > > I appologize for missing this in this case. > > Pavel> On the related note Release Notes mention the error log flood > Pavel> protection. Why it's not behind flag? How should I disable the feature > Pavel> and get back to the old behavior? > > We already had in MariaDB flood protection for some error > messages. What I did was to make it more general and extend it to also > include other error messages, like the one we are discussing. As > there wasn't a flag for the old suppression, I didn't see a need for > adding a flag for the new suppression. > > I am happy to look into adding a flag that is usable for any kind of > suppressed messages. > > A couple of ways to do this: > > a) Not suppress anything if you give --log-warnings a value > 2 > b) make LIMIT_UNSAFE_WARNING_ACTIVATION_TIMEOUT a configurable > variable, where 0 would mean no suppression. > > Which of the above, or both, would you prefer? > Or do you have any other suggestion of how to do this? > > For 10.1 we will probably add a mechanism where you can permanently > suppress any specific error message to be written to the error log. > (They would still be given to the client). > > Any suggestion or wishes regarding this? > > Regards, > Monty

1 0

[Maria-developers] Igor please review: [Commits] Rev 4446: MDEV-6879: Dereference of NULL primary_file->table in DsMrr_impl::get_disk_sweep_mrr_cost()
by Sergey Petrunia 20 Oct '14

20 Oct '14

Hello Igor, Could you please review the below: At file:///home/psergey/dev2/10.0-cp2/ ------------------------------------------------------------ revno: 4446 revision-id: psergey at askmonty.org-20141020175620-vodbkx9ugmdhh8c1 parent: psergey at askmonty.org-20141016135713-md92qr4g9fzhitxh committer: Sergey Petrunya <psergey at askmonty.org> branch nick: 10.0-cp2 timestamp: Mon 2014-10-20 21:56:20 +0400 message: MDEV-6879: Dereference of NULL primary_file->table in DsMrr_impl::get_disk_sweep_mrr_cost() (Attempt #2) - Don't attempt to use BKA for materialized derived tables. The table is neither filled nor fully opened yet, so attempt to call handler->multi_range_read_info() causes crash. === modified file 'mysql-test/r/derived_opt.result' --- a/mysql-test/r/derived_opt.result 2013-06-18 15:21:00 +0000 +++ b/mysql-test/r/derived_opt.result 2014-10-20 17:56:20 +0000 @@ -352,4 +352,33 @@ pk pk 72 72 80 80 drop table t1, t2, t3, t4; +# +# MDEV-6879: Dereference of NULL primary_file->table in DsMrr_impl::get_disk_sweep_mrr_cost() +# +create table t1(a int); +insert into t1 values (0),(1),(2),(3),(4),(5),(6),(7),(8),(9); +create table t2 (a int, b int, c text); +insert into t2 +select +A.a + B.a* 10, +A.a + B.a* 10, +'blob-data' +from t1 A, t1 B; +set @tmp_jcl= @@join_cache_level; +set @tmp_os= @@optimizer_switch; +set join_cache_level=6; +set @@optimizer_switch='derived_merge=on,derived_with_keys=on,mrr=on'; +explain +select * from +t1 join +(select * from t2 order by a limit 1000) as D1 +where +D1.a= t1.a; +id select_type table type possible_keys key key_len ref rows Extra +1 PRIMARY t1 ALL NULL NULL NULL NULL 10 Using where +1 PRIMARY <derived2> ref key0 key0 5 test.t1.a 10 +2 DERIVED t2 ALL NULL NULL NULL NULL 100 Using filesort +set join_cache_level=@tmp_jcl; +set optimizer_switch=@tmp_os; +drop table t1, t2; set optimizer_switch=@exit_optimizer_switch; === modified file 'mysql-test/t/derived_opt.test' --- a/mysql-test/t/derived_opt.test 2012-11-19 16:17:46 +0000 +++ b/mysql-test/t/derived_opt.test 2014-10-20 17:56:20 +0000 @@ -272,5 +272,34 @@ limit 10; drop table t1, t2, t3, t4; +--echo # +--echo # MDEV-6879: Dereference of NULL primary_file->table in DsMrr_impl::get_disk_sweep_mrr_cost() +--echo # +create table t1(a int); +insert into t1 values (0),(1),(2),(3),(4),(5),(6),(7),(8),(9); + +create table t2 (a int, b int, c text); +insert into t2 +select + A.a + B.a* 10, + A.a + B.a* 10, + 'blob-data' +from t1 A, t1 B; + +set @tmp_jcl= @@join_cache_level; +set @tmp_os= @@optimizer_switch; +set join_cache_level=6; +set @@optimizer_switch='derived_merge=on,derived_with_keys=on,mrr=on'; +explain +select * from + t1 join + (select * from t2 order by a limit 1000) as D1 +where + D1.a= t1.a; + +set join_cache_level=@tmp_jcl; +set optimizer_switch=@tmp_os; +drop table t1, t2; + # The following command must be the last one the file set optimizer_switch=@exit_optimizer_switch; === modified file 'sql/sql_select.cc' --- a/sql/sql_select.cc 2014-10-09 08:30:11 +0000 +++ b/sql/sql_select.cc 2014-10-20 17:56:20 +0000 @@ -10550,6 +10550,19 @@ uint check_join_cache_usage(JOIN_TAB *ta } /* + Don't use BKA for materialized tables. We could actually have a + meaningful use of BKA when linked join buffers are used. + + The problem is, the temp.table is not filled (actually not even opened + properly) yet, and this doesn't let us call + handler->multi_range_read_info(). It is possible to come up with + estimates, etc. without acessing the table, but it seems not to worth the + effort now. + */ + if (tab->table->pos_in_table_list->is_materialized_derived()) + no_bka_cache= true; + + /* Don't use join buffering if we're dictated not to by no_jbuf_after (This is not meaningfully used currently) */ @@ -10615,7 +10628,7 @@ uint check_join_cache_usage(JOIN_TAB *ta if (tab->ref.is_access_triggered()) goto no_join_cache; - if (!tab->is_ref_for_hash_join()) + if (!tab->is_ref_for_hash_join() && !no_bka_cache) { flags= HA_MRR_NO_NULL_ENDPOINTS | HA_MRR_SINGLE_POINT; if (tab->table->covering_keys.is_set(tab->ref.key)) BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] review MDEV 6631
by Roberto Spadim 18 Oct '14

18 Oct '14

hi guys could review MDEV-6631? there's a pull request, maybe should include some test files, but I don't know how to implement profiler check with mysql test tools -- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

1 0

Re: [Maria-developers] MariaDB 10.0.14 got rid of replication safety warning without warning in release notes?
by Pavel Ivanov 18 Oct '14

18 Oct '14

As I have reports of people not seeing this email on maria-developers@ list (probably because Monty sent it from the wrong address) I'm forwarding it to the list. I'll respond to it in detail later. On Fri, Oct 17, 2014 at 1:43 PM, Michael Widenius <michael.widenius(a)gmail.com> wrote: > > Hi! > > First sorry for the delayed reply; I am at a conference in Oxford and > didn't have an reliable internet connection until now. > >>>>>> "Pavel" == Pavel Ivanov <pivanof(a)google.com> writes: > > Pavel> Hi, > Pavel> I see that in MariaDB 10.0.14 Michael disabled replication safety > Pavel> warning for the case of INSERT ON DUPLICATE KEY UPDATE with several > Pavel> unique keys with a comment "We should assume that the store engine > Pavel> will report the first duplicate key for this case." > Pavel> http://bazaar.launchpad.net/~maria-captains/maria/10.0/revision/4400.1.3 > > Pavel> What does that mean? > > That we don't anymore give a warning for things that can't happen in > the distributed MariaDB server. > > Pabel> Which key is "the first duplicate key". > > The first key that has a duplicate value in the order of keys in the KEY > structure that MariaDB provides for the storage engine. > > This is the same order that keys are presented in SHOW CREATE TABLE > > Pavel> and why > Pavel> MariaDB relies on an engine behavior which AFAIK is not guaranteed? > > This is guaranteed for all storage engines shipped with MariaDB and > should also be guaranteed for any storage engine that wants to follow > the MariaDB and even the original MySQL storage engine interface. > > The change is also in line with how I designed the part of the storage > engine interface that gives the SQL layer the information about which > key that was conflicted. As we use this for things like INSERT ... ON > DUPLICATE KEY, it's critical that all storage engine provides the same > information in case of conflicts. > > To help engine writes we are constantly adding new test in the storage > engine test suite, to make it easier for storage engine vendors to > test that their storage engine follows the interface. > We have now added a test to check for the correct behaviour for the > case of INSERT ... ON DUPLICATE KEY. > > What is important when it comes to the storage engine interface is > that in all possible cases, a query should work identically against 2 > different storage engines. > > The only 'allowed difference's that I can think of for the moment are: > > - Different storage engines may retrieve rows in different order. > - In case of errors, changes to non transactional tables will not roll > back. > - Automatically generated key values may be different between storage engines. > (but preferably they should not be that). However when things are > replicated the slave, it should get the same values as the master. > > In the case of INSERT ... ON DUPLICATE KEY, it was clear that someone > had broke the storage engine interface, for no good reason. Instead > of users having to depend on 'unsafe, different and undocumented > behavior' for which row as updated, it's better to fix this issue once > and for all, especially as fixing this is trivial. > > Do you have or use a storage engine that doesn't provide the expected value > for the conflicting key ? > > If yes, there is a couple of ways to fix this. I am happy to talk with > you to find a solution that will satisfy your needs and ours. > > I underatand that in many cases it's not critical which unique key the > storage engine reports as duplicate. However in some cases, it's > critical to get the first key. > > I could add a new extra call to inform the engine that we now require > the first duplicate key and not just any conflicting key. This would > be used for example for INSERT ... ON DUPLICATE KEY ... > > I would prefer to have the check for which key was duplicated in the > storage engine. However, if critically needed, and assuming there is more > than one storage engine that needs this, we could add a check on the > handler level that will do a lookup on all unique keys to find the > first duplicate one. > > What solution would you prefer? > > Pavel> And why this serious change in behavior wasn't mentioned in the > Pavel> Release Notes? > > I didn't see it as a a serious change as it's only about removing a > warning that has no value for any normal MariaDB user (who only use > storage engines supplied with MariadB) and which actually caused > serious problems for some MariaDB users. > > That said, I agree it was a mistake from my part to not notice that > this was missing in the release notes. I will try to do better in the > future and check that the release notes correctly reflects any changes > that I have done. > > I appologize for missing this in this case. > > Pavel> On the related note Release Notes mention the error log flood > Pavel> protection. Why it's not behind flag? How should I disable the feature > Pavel> and get back to the old behavior? > > We already had in MariaDB flood protection for some error > messages. What I did was to make it more general and extend it to also > include other error messages, like the one we are discussing. As > there wasn't a flag for the old suppression, I didn't see a need for > adding a flag for the new suppression. > > I am happy to look into adding a flag that is usable for any kind of > suppressed messages. > > A couple of ways to do this: > > a) Not suppress anything if you give --log-warnings a value > 2 > b) make LIMIT_UNSAFE_WARNING_ACTIVATION_TIMEOUT a configurable > variable, where 0 would mean no suppression. > > Which of the above, or both, would you prefer? > Or do you have any other suggestion of how to do this? > > For 10.1 we will probably add a mechanism where you can permanently > suppress any specific error message to be written to the error log. > (They would still be given to the client). > > Any suggestion or wishes regarding this? > > Regards, > Monty

1 0

Re: [Maria-developers] [Commits] Rev 3900: Fix for test failures on 64-bit platform. in lp:~maria-captains/maria/10.0-galera-6593
by Kristian Nielsen 17 Oct '14

17 Oct '14

Nirbhay Choubey <nirbhay(a)mariadb.com> writes: > message: > Fix for test failures on 64-bit platform. > === modified file 'sql/rpl_mi.cc' > --- a/sql/rpl_mi.cc 2014-10-10 23:06:40 +0000 > +++ b/sql/rpl_mi.cc 2014-10-17 02:08:55 +0000 > @@ -1466,7 +1466,7 @@ > { > for (int i= DO_DOMAIN_IDS; i <= IGNORE_DOMAIN_IDS; i ++) > { > - my_init_dynamic_array(&m_domain_ids[i], sizeof(uint32), 16, 16, MYF(0)); > + my_init_dynamic_array(&m_domain_ids[i], sizeof(ulong), 16, 16, MYF(0)); > } > } > Why do you need to use ulong? Domain ids should always be uint32, also on 64-bit. What is the real problem? Eg. why is there a test failure? I'd expect that the right fix was to convert to uint32 in the places where a domain_id might come into the code as ulong... > @@ -1483,7 +1483,7 @@ > domain ids list. DO_DOMAIN_IDS list is only looked-up is both (do & ignore) > list are non-empty. > */ > -void Domain_id_filter::do_filter(uint32 domain_id) > +void Domain_id_filter::do_filter(ulong domain_id) > { > DYNAMIC_ARRAY *do_domain_ids= &m_domain_ids[DO_DOMAIN_IDS]; > DYNAMIC_ARRAY *ignore_domain_ids= &m_domain_ids[IGNORE_DOMAIN_IDS]; > @@ -1491,21 +1491,21 @@ > if (do_domain_ids->elements > 0) > { > if (likely(do_domain_ids->elements == 1)) > - m_filter= ((* (uint32*) dynamic_array_ptr(do_domain_ids, 0)) > + m_filter= ((* (ulong *) dynamic_array_ptr(do_domain_ids, 0)) > != domain_id); > else > - m_filter= (bsearch((const uint32 *) &domain_id, do_domain_ids->buffer, > - do_domain_ids->elements, sizeof(uint32), > + m_filter= (bsearch((const ulong *) &domain_id, do_domain_ids->buffer, > + do_domain_ids->elements, sizeof(ulong), Why is a cast needed here? If it's because of const, could you avoid the cast just by declaring domain_id const in the argument of the function? Thanks, - Kristian.

2 1

[Maria-developers] doubts about show ___ any information table plugin
by Roberto Spadim 17 Oct '14

17 Oct '14

hi guys ! :) i was reading this: https://mariadb.com/kb/en/mariadb/development/mariadb-internals-documentati… and i'm with some doubts... automatically parser translate "_" to space? for example table A_B, i could use SHOW A B ? about the FLUSH what it does? any example of what it should do? like... i know we can reset some information counters, it's a 'shortcut' to avoid something like "SET pluginvariable=0"? if yes, maybe we could add some example at KB page, i'm with doubts, maybe a dba without knowledge about plugin internal will not understand thanks :) -- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

2 1

[Maria-developers] help with auth plugins
by Roberto Spadim 17 Oct '14

17 Oct '14

Hi guys! :) i need some help... i got the auth_dialog and i'm including some features to implement otp (one time password) my question is... what ".h" file should i include to allow read/write from any_table from mysql schema? i checked some read/write from selftunning optimizer ( https://github.com/igniting/server/blob/3d101751d27215ccf88c6c9701ca113de4a… ) but i didn't understand what should be included i'm getting messages like: In file included from /home/rspadim/mariadb/mariadb-10.1/server/sql/opt_range.h:36:0, from /home/rspadim/mariadb/mariadb-10.1/server/plugin/auth_otp/otp_auth.cc:26: --- my current source code (i don't know if .h order is ok): INCLUDES: #include "unireg.h" <- line 20 #include "sql_base.h" #include "lex.h" #include "sql_table.h" #include "sql_priv.h" #include "sql_class.h" #include "opt_range.h" <- line 26 #include "records.h" #include <mysql/plugin_auth.h> #include <string.h> #include <stdio.h> #include <stdlib.h> #include <mysql/auth_dialog_client.h> -- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

1 0

[Maria-developers] Two bugs in MRR in need of review/comment
by Jeremy Cole 16 Oct '14

16 Oct '14

Hello, I filed these two bugs against multi-range read (MRR) in MariaDB 10.0.12 which affected us and could use some review of the patch for correctness, along with development of a proper test case: MDEV-6878 <https://mariadb.atlassian.net/browse/MDEV-6878>: Use of uninitialized saved_primary_key in Mrr_ordered_index_reader::resume_read() MDEV-6879 <https://mariadb.atlassian.net/browse/MDEV-6879>: Dereference of NULL primary_file->table in DsMrr_impl::get_disk_sweep_mrr_cost() Could someone please take a look at them? Regards, Jeremy

1 0

Re: [Maria-developers] MDEV-5359 CREATE OR REPLACE, CREATE IF NOT EXISTS, DROP IF EXISTS
by Sergei Golubchik 15 Oct '14

15 Oct '14

Hi, Alexander! This is all pretty good. Great work, thanks. I didn't have many comments. Few about small stylistic changes, and a couple of more serious ones - about tests and about events. See below. On Oct 02, Alexander Barkov wrote: > diff --git a/mysql-test/r/create_drop_db.result b/mysql-test/r/create_drop_db.result > new file mode 100644 > index 0000000..e7d4c18 > --- /dev/null > +++ b/mysql-test/r/create_drop_db.result > @@ -0,0 +1,17 @@ > +CREATE DATABASE db1; > +CREATE DATABASE IF NOT EXISTS db1; > +Warnings: > +Note 1007 Can't create database 'db1'; database exists > +CREATE OR REPLACE DATABASE db1; > +CREATE OR REPLACE DATABASE IF NOT EXISTS db2; > +ERROR HY000: Incorrect usage of OR REPLACE and IF NOT EXISTS > +DROP DATABASE db1; > +DROP DATABASE IF EXISTS db1; > +Warnings: > +Note 1008 Can't drop database 'db1'; database doesn't exist > +DROP DATABASE db1; > +ERROR HY000: Can't drop database 'db1'; database doesn't exist > +CREATE OR REPLACE DATABASE db1; > +CREATE DATABASE db1; > +ERROR HY000: Can't create database 'db1'; database exists > +DROP DATABASE IF EXISTS db1; I don't see where this test verifies that CREATE OR REPLACE actually replaces a database. Add the following: create database; create table; create or replace database; the table must not exist now; and add a test for create if not exists when a database does not exist. you can combine it with a previous test, if you'd like. > diff --git a/mysql-test/r/create_drop_event.result b/mysql-test/r/create_drop_event.result > new file mode 100644 > index 0000000..40d16e6 > --- /dev/null > +++ b/mysql-test/r/create_drop_event.result > @@ -0,0 +1,31 @@ > +SET GLOBAL event_scheduler=off; > +CREATE DATABASE db1; > +CREATE EVENT event_1 ON SCHEDULE EVERY 1 SECOND DO DROP DATABASE db1; > +SHOW DATABASES LIKE 'db1'; > +Database (db1) > +db1 > +SET GLOBAL event_scheduler=on; > +SHOW DATABASES LIKE 'db1'; > +Database (db1) > +SET GLOBAL event_scheduler=off; > +DROP DATABASE db1; > +ERROR HY000: Can't drop database 'db1'; database doesn't exist > +CREATE EVENT event_1 ON SCHEDULE EVERY 1 SECOND DO DROP DATABASE db1; > +ERROR HY000: Event 'event_1' already exists > +CREATE EVENT IF NOT EXISTS event_1 ON SCHEDULE EVERY 1 SECOND DO DROP DATABASE db1; > +Warnings: > +Note 1537 Event 'event_1' already exists > +CREATE OR REPLACE EVENT IF NOT EXISTS event_1 ON SCHEDULE EVERY 1 SECOND DO DROP DATABASE db1; > +ERROR HY000: Incorrect usage of OR REPLACE and IF NOT EXISTS > +CREATE OR REPLACE EVENT event_1 ON SCHEDULE EVERY 1 SECOND DO DROP DATABASE db1; again, same thing. I don't see a test that the command actually worked. please use different event definitions in all three create variants. and add show create event after every create to verify that create if not exists did *not* modify the event, but create or replace did modify it. > +CREATE DATABASE db1; > +SHOW DATABASES LIKE 'db1'; > +Database (db1) > +db1 > +SET GLOBAL event_scheduler=on; > +SHOW DATABASES LIKE 'db1'; > +Database (db1) > +SET GLOBAL event_scheduler=off; > +DROP DATABASE db1; > +ERROR HY000: Can't drop database 'db1'; database doesn't exist > +DROP EVENT IF EXISTS event_1; > diff --git a/mysql-test/r/create_drop_function.result b/mysql-test/r/create_drop_function.result > new file mode 100644 > index 0000000..750d179 > --- /dev/null > +++ b/mysql-test/r/create_drop_function.result > @@ -0,0 +1,34 @@ > +SET timestamp=UNIX_TIMESTAMP('2014-09-30 08:00:00'); > +CREATE FUNCTION f1(str char(20)) > +RETURNS CHAR(100) > +RETURN CONCAT('Hello, ', str, '!'); > +SELECT * FROM mysql.proc WHERE name like 'f1'; > +db name type specific_name language sql_data_access is_deterministic security_type param_list returns body definer created modified sql_mode comment character_set_client collation_connection db_collation body_utf8 > +test f1 FUNCTION f1 SQL CONTAINS_SQL NO DEFINER str char(20) char(100) CHARSET latin1 RETURN CONCAT('Hello, ', str, '!') root@localhost 2014-09-30 08:00:00 2014-09-30 08:00:00 latin1 latin1_swedish_ci latin1_swedish_ci RETURN CONCAT('Hello, ', str, '!') > +SELECT f1('world'); > +f1('world') > +Hello, world! > +CREATE FUNCTION f1(str char(20)) > +RETURNS TEXT > +RETURN CONCAT('Hello, ', str, '!'); > +ERROR 42000: FUNCTION f1 already exists > +CREATE FUNCTION IF NOT EXISTS f1(str char(20)) > +RETURNS CHAR(100) > +RETURN CONCAT('Hello, ', str, '!'); > +Warnings: > +Note 1304 FUNCTION f1 already exists same. please *different* definitions in different create variants. and some verification that the definition changed (or didn't change) - like show create or select f1(); > +CREATE OR REPLACE FUNCTION IF NOT EXISTS f1(str char(20)) > +RETURNS CHAR(100) > +RETURN CONCAT('Hello, ', str, '!'); > +ERROR HY000: Incorrect usage of OR REPLACE and IF NOT EXISTS > +CREATE OR REPLACE FUNCTION f1(str char(20)) > +RETURNS CHAR(100) > +RETURN CONCAT('Hello, ', str, '!'); > +DROP FUNCTION f1; > +CREATE FUNCTION IF NOT EXISTS f1(str char(20)) > +RETURNS CHAR(100) > +RETURN CONCAT('Hello, ', str, '!'); > +SELECT f1('world'); > +f1('world') > +Hello, world! > +DROP FUNCTION f1; > diff --git a/mysql-test/r/create_drop_index.result b/mysql-test/r/create_drop_index.result > new file mode 100644 > index 0000000..23f6388 > --- /dev/null > +++ b/mysql-test/r/create_drop_index.result > @@ -0,0 +1,34 @@ > +CREATE TABLE t1(id INT); > +CREATE INDEX idx ON t1(id); > +ALTER TABLE t1 ADD INDEX idx(id); > +ERROR 42000: Duplicate key name 'idx' > +ALTER TABLE t1 ADD INDEX IF NOT EXISTS idx(id); > +Warnings: > +Note 1061 Duplicate key name 'idx' > +CREATE OR REPLACE INDEX idx2 ON t1(id); same comment here (and everywhere below) > +Warnings: > +Note 1831 Duplicate index 'idx2' defined on the table 'test.t1'. This is deprecated and will be disallowed in a future release. > +ALTER TABLE t1 ADD INDEX IF NOT EXISTS idx2(id); > +Warnings: > +Note 1061 Duplicate key name 'idx2' > +CREATE OR REPLACE UNIQUE INDEX idx ON t1(id); > +CREATE INDEX idx ON t1(id); > +ERROR 42000: Duplicate key name 'idx' > +CREATE TABLE t2(col INT); > +CREATE OR REPLACE INDEX idx ON t2(col); > +CREATE INDEX idx ON t2(col); > +ERROR 42000: Duplicate key name 'idx' > +# Is this correct behaviour? > +ALTER TABLE t2 ADD FOREIGN KEY fk(col) REFERENCES t1(id); > +CREATE INDEX fk ON t2(col); > +Warnings: > +Note 1831 Duplicate index 'fk' defined on the table 'test.t2'. This is deprecated and will be disallowed in a future release. > +# Clearing up > +DROP INDEX idx ON t1; > +DROP TABLE t1; > +ALTER TABLE t1 DROP INDEX idx2; > +ERROR 42S02: Table 'test.t1' doesn't exist > +ALTER TABLE t2 DROP INDEX idx; > +ALTER TABLE t2 DROP INDEX fk; > +ALTER TABLE t2 DROP FOREIGN KEY fk; > +DROP TABLE t2; > diff --git a/mysql-test/r/proc_func.result b/mysql-test/r/proc_func.result > new file mode 100644 > index 0000000..1105ded > --- /dev/null > +++ b/mysql-test/r/proc_func.result > @@ -0,0 +1,63 @@ > +# > +# Testing CREATE PROCEDURE IF NOT EXISTS > +# > +DROP TABLE IF EXISTS t1; > +DROP PROCEDURE IF EXISTS proc1; > +CREATE TABLE t1 (id INT); > +CREATE PROCEDURE proc1 (OUT cnt INT) BEGIN SELECT COUNT(*) INTO cnt FROM t1; END$$ how is this test (and other tests below) different from create_drop_xxx tests above? > +CALL proc1(@cnt); > +SELECT @cnt; > +@cnt > +0 > +INSERT INTO t1 VALUES (1), (2), (3); > +CALL proc1(@cnt); > +SELECT @cnt; > +@cnt > +3 > +CREATE PROCEDURE proc1 (OUT cnt INT) BEGIN SELECT COUNT(*) INTO cnt FROM t1; END$$ > diff --git a/sql/event_db_repository.cc b/sql/event_db_repository.cc > index b20269b..01cdabb 100644 > --- a/sql/event_db_repository.cc > +++ b/sql/event_db_repository.cc > @@ -684,18 +684,28 @@ Event_db_repository::create_event(THD *thd, Event_parse_data *parse_data, > DBUG_PRINT("info", ("check existance of an event with the same name")); > if (!find_named_event(parse_data->dbname, parse_data->name, table)) > { > - if (create_if_not) > + if (thd->lex->is_create_or_replace()) > + { > + if ((ret= table->file->ha_delete_row(table->record[0]))) > + { > + table->file->print_error(ret, MYF(0)); > + goto end; > + } > + } > + else if (create_if_not) > { > *event_already_exists= true; > push_warning_printf(thd, Sql_condition::WARN_LEVEL_NOTE, > ER_EVENT_ALREADY_EXISTS, ER(ER_EVENT_ALREADY_EXISTS), > parse_data->name.str); > ret= 0; > + goto end; > } > else > + { > my_error(ER_EVENT_ALREADY_EXISTS, MYF(0), parse_data->name.str); > - > - goto end; > + goto end; > + } > } else > *event_already_exists= false; I wonder whether this works as it should. On create-or-replace you delete the old event row and a new one is supposedly inserted. So, the table is fine. But event_already_exists will stay unset, undefined, actually, because the caller didn't set it either. If you set it to TRUE, then the caller won't update in-memory even list, so your new event won't be enabled. If you set it to FALSE, it will try to insert a new event without deleting the old one and I don't know what will happen (either both will stay active or the insertion fails). Do you have a test to verify that after CREATE OR REPLACE EVENT the old event no longer exists and the new event does exist and is executed as expected? > diff --git a/sql/share/errmsg-utf8.txt b/sql/share/errmsg-utf8.txt > index ed7976e..1cfaaa4 100644 > --- a/sql/share/errmsg-utf8.txt > +++ b/sql/share/errmsg-utf8.txt > @@ -7110,3 +7110,11 @@ ER_IT_IS_A_VIEW 42S02 > eng "'%-.192s' is a view" > ER_SLAVE_SKIP_NOT_IN_GTID > eng "When using GTID, @@sql_slave_skip_counter can not be used. Instead, setting @@gtid_slave_pos explicitly can be used to skip to after a given GTID position." > +ER_USER_CREATE_EXISTS > + eng "Can't create user '%-.64s'@'%-.64s'; it already exists" > +ER_USER_DROP_EXISTS > + eng "Can't drop user '%-.64s'@'%-.64s'; it doesn't exist" > +ER_ROLE_CREATE_EXISTS > + eng "Can't create role '%-.64s'; it already exists" > +ER_ROLE_DROP_EXISTS > + eng "Can't drop role '%-.64s'; it doesn't exist" just a reminder - when merging (preferrably - rebasing) don't forget to put these errror messages after all 10.0 error messages. > diff --git a/sql/sp.cc b/sql/sp.cc > index 188b311..058a617 100644 > --- a/sql/sp.cc > +++ b/sql/sp.cc > @@ -835,6 +835,8 @@ db_load_routine(THD *thd, stored_procedure_type type, > > thd->lex= &newlex; > newlex.current_select= NULL; > + // Resetting all the flags in create_info > + newlex.create_info.options= 0; Isn't it safer to reset options in the parser, after CREATE clause? Like | CREATE { Lex->create_info.options= 0; } opt_or_replace ... Ok, see my very last comment in this review. > defstr.set_charset(creation_ctx->get_client_cs()); > > @@ -1015,6 +1063,47 @@ sp_create_routine(THD *thd, stored_procedure_type type, sp_head *sp) > ret= SP_OPEN_TABLE_FAILED; > else > { > + /* Checking if the routine already exists */ > + if (db_find_routine_aux(thd, type, lex->spname, table) == SP_OK) > + { > + if (lex->is_create_or_replace()) > + { > + if ((ret= sp_drop_routine_internal(thd, type, lex->spname, table))) > + goto done; > + } > + else if (lex->is_create_if_not_exists()) > + { > + push_warning_printf(thd, Sql_condition::WARN_LEVEL_NOTE, > + ER_SP_ALREADY_EXISTS, ER(ER_SP_ALREADY_EXISTS), > + type == TYPE_ENUM_FUNCTION ? > + "FUNCTION" : "PROCEDURE", > + lex->spname->m_name.str); > + > + ret= SP_OK; > + > + // Setting retstr as it is used for logging. > + if (sp->m_type == TYPE_ENUM_FUNCTION) > + { > + sp_returns_type(thd, retstr, sp); this is ok > + store_failed= store_failed || > + table->field[MYSQL_PROC_FIELD_RETURNS]-> > + store(retstr.ptr(), retstr.length(), system_charset_info); but why do you need to update table->field[MYSQL_PROC_FIELD_RETURNS] ? > + } > + > + if (store_failed) > + { > + ret= SP_FLD_STORE_FAILED; > + goto done; > + } > + goto log; > + } > + else > + { > + ret= SP_WRITE_ROW_FAILED; > + goto done; > + } > + } > + > restore_record(table, s->default_values); // Get default values for fields > > /* NOTE: all needed privilege checks have been already done. */ > diff --git a/sql/sql_acl.cc b/sql/sql_acl.cc > index 5f077c2..1e156be 100644 > --- a/sql/sql_acl.cc > +++ b/sql/sql_acl.cc > @@ -9148,6 +9148,42 @@ end: > DBUG_RETURN(result); > } > > + > +class Warning_user > +{ > +private: > + bool m_handle_as_role; > + void push_warning_role(THD *thd, uint err, const LEX_USER *user) > + { > + push_warning_printf(thd, Sql_condition::WARN_LEVEL_NOTE, > + err, ER(err), user->user.str); > + } > + void push_warning_user(THD *thd, uint err, const LEX_USER *user) > + { > + push_warning_printf(thd, Sql_condition::WARN_LEVEL_NOTE, > + err, ER(err), user->user.str, user->host.str); > + } > +public: > + Warning_user(bool handle_as_role) > + :m_handle_as_role(handle_as_role) > + { } > + void push_create_exists(THD *thd, const LEX_USER *user) > + { > + if (m_handle_as_role) > + push_warning_role(thd, ER_ROLE_CREATE_EXISTS, user); > + else > + push_warning_user(thd, ER_USER_CREATE_EXISTS, user); > + } > + void push_drop_doesnt_exist(THD *thd, const LEX_USER *user) > + { > + if (m_handle_as_role) > + push_warning_role(thd, ER_ROLE_DROP_EXISTS, user); > + else > + push_warning_user(thd, ER_USER_DROP_EXISTS, user); > + } > +}; > + > + > /* > Create a list of users. > > @@ -9215,10 +9252,26 @@ bool mysql_create_user(THD *thd, List <LEX_USER> &list, bool handle_as_role) > */ > if (handle_grant_data(tables, 0, user_name, NULL)) > { > - append_user(thd, &wrong_users, user_name); > - > - result= TRUE; > - continue; > + if (thd->lex->is_create_or_replace()) > + { > + if (handle_grant_data(tables, 1, user_name, NULL) <= 0) > + { > + append_user(thd, &wrong_users, user_name); I don't understand that at all. If user creation failed, you try to drop it, but don't create a user again, after the old one was dropped. How could it work? > + result= TRUE; > + continue; > + } > + } > + else if (thd->lex->is_create_if_not_exists()) > + { > + Warning_user(handle_as_role).push_create_exists(thd, user_name); Eh? Why do you need a class for that? Especially as it's temporarily instantiated here, and just to push a warning. Please, remove it. > + continue; > + } > + else > + { > + append_user(thd, &wrong_users, user_name); > + result= TRUE; > + continue; > + } > } > > some_users_created= TRUE; > @@ -9268,7 +9321,7 @@ bool mysql_create_user(THD *thd, List <LEX_USER> &list, bool handle_as_role) > (handle_as_role) ? "CREATE ROLE" : "CREATE USER", > wrong_users.c_ptr_safe()); > > - if (some_users_created) > + if (some_users_created || !result) What is that for? To binlog CREATE IF NOT EXISTS? > result |= write_bin_log(thd, FALSE, thd->query(), thd->query_length()); > > mysql_rwlock_unlock(&LOCK_grant); > diff --git a/sql/sql_parse.cc b/sql/sql_parse.cc > index 922f0d7..648fd48 100644 > --- a/sql/sql_parse.cc > +++ b/sql/sql_parse.cc > @@ -3855,7 +3855,9 @@ end_with_restore_list: > } > } > #endif > - if (check_access(thd, CREATE_ACL, lex->name.str, NULL, NULL, 1, 0)) > + if (check_access(thd, lex->is_create_or_replace() ? > + CREATE_ACL | DROP_ACL : CREATE_ACL, good > + lex->name.str, NULL, NULL, 1, 0)) > break; > res= mysql_create_db(thd, lex->name.str, &create_info, 0); > break; > @@ -3995,9 +3997,8 @@ end_with_restore_list: > switch (lex->sql_command) { > case SQLCOM_CREATE_EVENT: > { > - bool if_not_exists= (lex->create_info.options & > - HA_LEX_CREATE_IF_NOT_EXISTS); > - res= Events::create_event(thd, lex->event_parse_data, if_not_exists); > + res= Events::create_event(thd, lex->event_parse_data, > + lex->is_create_if_not_exists()); This doesn't seem to be particularly logical. You pass lex->is_create_if_not_exists)() here an argument, but you check thd->lex->is_create_or_replace() in the Event_db_repository::create_event anyway. I'd suggest not to pass lex->is_create_if_not_exists() as an argument, and check both in Event_db_repository::create_event(). > break; > } > case SQLCOM_ALTER_EVENT: > @@ -6514,7 +6520,11 @@ bool add_field_to_list(THD *thd, LEX_STRING *field_name, enum_field_types type, > lex->col_list.push_back(new Key_part_spec(*field_name, 0)); > key= new Key(Key::PRIMARY, null_lex_str, > &default_key_create_info, > - 0, lex->col_list, NULL, lex->check_exists); > + 0, lex->col_list, NULL); > + > + if (lex->check_exists) > + key->key_create_info.options = HA_LEX_CREATE_IF_NOT_EXISTS; Hmm, is it for ALTER TABLE t1 ADD COLUMN IF NOT EXISTS foo INT PRIMARY KEY ? Do you have test cases for it? What does it do if the column exists but the key doesn't? What if the key exists and the column doens't? > + > lex->alter_info.key_list.push_back(key); > lex->col_list.empty(); > } > @@ -6524,7 +6534,11 @@ bool add_field_to_list(THD *thd, LEX_STRING *field_name, enum_field_types type, > lex->col_list.push_back(new Key_part_spec(*field_name, 0)); > key= new Key(Key::UNIQUE, null_lex_str, > &default_key_create_info, 0, > - lex->col_list, NULL, lex->check_exists); > + lex->col_list, NULL); > + > + if (lex->check_exists) > + key->key_create_info.options = HA_LEX_CREATE_IF_NOT_EXISTS; Same, needs tests (if there aren't). > + > lex->alter_info.key_list.push_back(key); > lex->col_list.empty(); > } > diff --git a/sql/sql_table.cc b/sql/sql_table.cc > index 8af77d9..b10c6d7 100644 > --- a/sql/sql_table.cc > +++ b/sql/sql_table.cc > @@ -5846,6 +5846,8 @@ handle_if_exists_options(THD *thd, TABLE *table, Alter_info *alter_info) > } > if (remove_key) > { take care when merging. "if (remove_key)" is not in the current 10.1, instead there's "remove_key:" label and goto. > + if (key->is_create_if_not_exists()) > + { > push_warning_printf(thd, Sql_condition::WARN_LEVEL_NOTE, > ER_DUP_KEYNAME, ER(ER_DUP_KEYNAME), keyname); > key_it.remove(); > diff --git a/sql/sql_trigger.cc b/sql/sql_trigger.cc > index 70ac626..b337451 100644 > --- a/sql/sql_trigger.cc > +++ b/sql/sql_trigger.cc > @@ -607,6 +607,89 @@ end: > DBUG_RETURN(result); > } > > +/** > + Build stmt_query to write it in the bin-log. > + > + @param thd current thread context (including trigger definition in > + LEX) > + @param tables table list containing one open table for which the > + trigger is created. > + @param[out] stmt_query after successful return, this string contains > + well-formed statement for creation this trigger. > + > + @note > + - Assumes that trigger name is fully qualified. > + - NULL-string means the following LEX_STRING instance: > + { str = 0; length = 0 }. > + - In other words, definer_user and definer_host should contain > + simultaneously NULL-strings (non-SUID/old trigger) or valid strings > + (SUID/new trigger). > + > + @retval > + False success > + @retval > + True error > +*/ > +static bool build_trig_stmt_query(THD *thd, TABLE_LIST *tables, > + String *stmt_query, > + LEX_STRING *trg_definer, > + char trg_definer_holder[]) > +{ > + LEX *lex = thd->lex; > + LEX_STRING definer_user; > + LEX_STRING definer_host; > + > + if (trg_definer == NULL && > + !(trg_definer= alloc_lex_string(&tables->table->mem_root))) > + return true; > + > + if (lex->sphead->m_chistics->suid != SP_IS_NOT_SUID) > + { > + /* SUID trigger. */ > + definer_user= lex->definer->user; > + definer_host= lex->definer->host; > + lex->definer->set_lex_string(trg_definer, trg_definer_holder); > + } > + else > + { > + /* non-SUID trigger. */ > + definer_user.str= 0; > + definer_user.length= 0; > + > + definer_host.str= 0; > + definer_host.length= 0; > + > + trg_definer->str= (char*) ""; > + trg_definer->length= 0; Why do you bother setting definer here? You can simply do below (see below): > + } > + > + /* > + Create well-formed trigger definition query. Original query is not > + appropriated, because definer-clause can be not truncated. > + */ > + > + stmt_query->append(STRING_WITH_LEN("CREATE ")); > + > + if (lex->is_create_or_replace()) > + stmt_query->append(STRING_WITH_LEN("OR REPLACE ")); > + > + if (lex->sphead->m_chistics->suid != SP_IS_NOT_SUID) > + { > + /* > + Append definer-clause if the trigger is SUID (a usual trigger in > + new MySQL versions). > + */ > + append_definer(thd, stmt_query, &definer_user, &definer_host); here: append_definer(thd, stmt_query, &lex->definer->user, &lex->definer->host); and set trg_definer: lex->definer->set_lex_string(trg_definer, trg_definer_holder); } else trg_definer= empty_lex_string; > + } > + > + LEX_STRING stmt_definition; > + stmt_definition.str= (char*) thd->lex->stmt_definition_begin; > + stmt_definition.length= thd->lex->stmt_definition_end - > + thd->lex->stmt_definition_begin; > + trim_whitespace(thd->charset(), & stmt_definition); > + stmt_query->append(stmt_definition.str, stmt_definition.length); > + return false; > +} > > /** > Create trigger for table. > @@ -639,16 +722,16 @@ bool Table_triggers_list::create_trigger(THD *thd, TABLE_LIST *tables, > char file_buff[FN_REFLEN], trigname_buff[FN_REFLEN]; > LEX_STRING file, trigname_file; > LEX_STRING *trg_def; > - LEX_STRING definer_user; > - LEX_STRING definer_host; > ulonglong *trg_sql_mode; > char trg_definer_holder[USER_HOST_BUFF_SIZE]; > - LEX_STRING *trg_definer; > + LEX_STRING *trg_definer= NULL; > Item_trigger_field *trg_field; > struct st_trigname trigname; > LEX_STRING *trg_client_cs_name; > LEX_STRING *trg_connection_cl_name; > LEX_STRING *trg_db_cl_name; > + sp_head *trg_head= bodies[lex->trg_chistics.event] > + [lex->trg_chistics.action_time]; trg_head as an element in a bodies array? May be trg_body instead? > > if (check_for_broken_triggers()) > return true; > diff --git a/sql/sql_view.cc b/sql/sql_view.cc > index a18193c..52b534e 100644 > --- a/sql/sql_view.cc > +++ b/sql/sql_view.cc > @@ -858,6 +863,14 @@ static int mysql_register_view(THD *thd, TABLE_LIST *view, > view->definer.host= lex->definer->host; > view->view_suid= lex->create_view_suid; > view->with_check= lex->create_view_check; > + > + DBUG_EXECUTE_IF("simulate_register_view_failure", > + my_error(ER_OUT_OF_RESOURCES, MYF(0));); > + DBUG_EXECUTE_IF("simulate_register_view_failure", > + error= -1;); > + DBUG_EXECUTE_IF("simulate_register_view_failure", > + goto err;); combine these in one DBUG_EXECUTE_IF: DBUG_EXECUTE_IF("simulate_register_view_failure", { my_error(ER_OUT_OF_RESOURCES, MYF(0)); error= -1; goto err; }); > + > if ((view->updatable_view= (can_be_merged && > view->algorithm != VIEW_ALGORITHM_TMPTABLE))) > { > diff --git a/sql/sql_yacc.yy b/sql/sql_yacc.yy > index d6b3fa4..7da047f 100644 > --- a/sql/sql_yacc.yy > +++ b/sql/sql_yacc.yy > @@ -752,8 +752,18 @@ bool setup_select_in_parentheses(LEX *lex) > return FALSE; > } > > -static bool add_create_index_prepare (LEX *lex, Table_ident *table) > +static bool add_create_index_prepare (LEX *lex, Table_ident *table, > + uint create_or_replace, > + uint create_if_not_exists) > { > + if (create_or_replace && create_if_not_exists) > + { > + my_error(ER_WRONG_USAGE, MYF(0), "OR REPLACE", "IF NOT EXISTS"); > + return TRUE; > + } I'd rather see this check in the opt_if_not_exists rule, in only one place, that is. For this to work, though, the result of create_or_replace rule must be stored somewhere where opt_if_not_exists can access it. For example, in Lex->create_info.options. > + > + lex->key_create_info.options = (create_or_replace | create_if_not_exists); > + > lex->sql_command= SQLCOM_CREATE_INDEX; > if (!lex->current_select->add_table_to_list(lex->thd, table, NULL, > TL_OPTION_UPDATING, Regards, Sergei

1 0

[Maria-developers] MariaDB 10.0 on POWER
by Colin Charles 14 Oct '14

14 Oct '14

Hi, I'm sure you have read: https://www.flamingspork.com/blog/2014/10/08/mariadb-10-0-on-power/ For XL: There’s obviously still some work in progress going on, but my initial results show performance within around 10% of MySQL, so with a bit of work we will hopefully see MariaDB reach performance parity. For Serg: One interesting find was the code to account for thread memory usage uses a single atomic variable: this does not scale and does end up showing up on profiles. It isn't clear 10% loss to which MySQL, but comparing to 5.6 or 5.7, this has to "just work better" Thanks -- Colin Charles, Chief Evangelist, MariaDB Corporation blog: http://bytebot.net/blog/| t: +6-012-204-3201 | Skype: colincharles

2 1

Re: [Maria-developers] [Commits] 47fea67: MDEV-12 OpenGIS: create required tables: GeometryColumns, related views.
by Sergei Golubchik 14 Oct '14

14 Oct '14

Hi, Alexey! On Sep 02, Alexey Botchkov wrote: > revision-id: 47fea6798f0f37b8a42ad047d7daf6ec9bb52813 > parent(s): e44751b65f4760067d15f8a526e8f97f84810c29 > committer: Alexey Botchkov > branch nick: > timestamp: 2014-09-02 11:31:26 +0500 > message: > > MDEV-12 OpenGIS: create required tables: GeometryColumns, related views. > GEOMETRY_COLUMNS and SPATIAL_REF_SYS tables added to the INFORMATION_SCHEMA. > > diff --git a/mysql-test/r/information_schema.result > b/mysql-test/r/information_schema.result > index 6df5b89..93c9e90 100644 > --- a/mysql-test/r/information_schema.result > +++ b/mysql-test/r/information_schema.result > @@ -43,7 +43,8 @@ create view v1 (c) as > SELECT table_name FROM information_schema.TABLES > WHERE table_schema IN ('mysql', 'INFORMATION_SCHEMA', 'test', > 'mysqltest') AND > table_name not like 'ndb_%' AND table_name not like 'innodb_%' AND > -table_name not like 'xtradb_%'; > +table_name not like 'xtradb_%' AND > +table_name not like 'geometry%' AND table_name not like 'spatial%'; Why did you filter these tables out? Normally, only plugin-created tables are filtered out, because these plugins might be disabled in the build. But we don't ever build without geometry, do we? I'd keep these tables in the result. > select * from v1; > c > ALL_PLUGINS > diff --git a/mysql-test/t/gis.test b/mysql-test/t/gis.test > index c387069..1a07a33 100644 > --- a/mysql-test/t/gis.test > +++ b/mysql-test/t/gis.test > @@ -1461,3 +1461,7 @@ insert into t1 values(Geomfromtext('POLYGON((1 1, > 2 2, 2 1, 1 1))')); > drop table t1; > > --echo End of 5.5 tests > + > +SHOW CREATE TABLE information_schema.geometry_columns; > +SHOW CREATE TABLE information_schema.spatial_ref_sys; Please, add tests for the content of these tables. Then ok to push (assuming these new tests will work fine). Regards, Sergei

1 0

[Maria-developers] Is it soon 'end of this year'?
by Peter Laursen 13 Oct '14

13 Oct '14

Around ½ year ago I discussed with some people here (Sergei G and/or Georg and/or Wlad) about support for Oracle-style plugins in the MariaDB C-API. The plugins include 1 Windows authentication plugin 2) Clear text plugin. At that time I was told that it could be considered 'by end end of this year'. 1) Windows authentication plugin. Not very important, but Sergei told that he had been playing wiht it and it could probably be done in ~1 day. 2) This is used by Oracle for setting up LDAP.authentication. I am very well aware that there is criticism of Oracle's implementation. But this does not change the fact hat people use it with servers from Oracle and expect clients to work with it. We get request(s) for it almost every week. So this is most important (for us). Any chance that those can soon be prioritized? (if you feel your self mis-quoted then I apologize).. -- Peter --Webyog

3 12

[Maria-developers] doubt about MDL
by Roberto Spadim 13 Oct '14

13 Oct '14

hi guys, i'm reading https://github.com/MariaDB/server/blob/10.1/sql/mdl.h i'm with a doubt... today we have the metadata lock plugin to check what mdl we have... my doubt is, how to expose how many threads are sharing the mdl key, and how many threads are waiting the mdl key, should it be possible or only one threrad could use one mdl? -- Roberto Spadim

2 3

Re: [Maria-developers] Page encryption exception handling.
by Sergei Golubchik 13 Oct '14

13 Oct '14

Hi, Clemens! On Oct 13, Clemens Dörrhöfer wrote: > Hi Sergei, > > I included an error, which was triggered by a missing key in the > keyfile. The server will see a wrong space ID, since it has not been > decrypted, which from my point of view is the reason for mariadb to > crash. > Basically we run into a problem when trying to decrypt a page using > fil_decrypt_page theoretically we can catch this exception much > sooner, since all we need to know is the key id we need for decryption > / encryption and throw an exception as soon as we know that we cannot > handle a key in best case before we trigger any aio. > > fil_decrypt_page() at fil0pageencryption.cc:391 0x555555feeb91 > os_file_read_func() at os0file.cc:3.156 0x555555e22648 > pfs_os_file_read_func() at os0file.ic:288 0x555555fdb4fa > fil_read_first_page() at fil0fil.cc:2.156 0x555555fe1eb0 > fil_validate_single_table_tablespace() at fil0fil.cc:4.303 0x555555fe5fd0 > fil_load_single_table_tablespace() at fil0fil.cc:4.443 0x555555fe64af > fil_load_single_table_tablespaces() at fil0fil.cc:4.806 0x555555fe6d1a > recv_init_crash_recovery() at log0recv.cc:3.046 0x555555e14447 > recv_recovery_from_checkpoint_start_func() at log0recv.cc:3.360 0x555555e14c69 > innobase_start_or_create_for_mysql() at srv0start.cc:2.471 0x555555eeea65 This still doesn't show where it crashes, only where you detect the missing key. But anyway, I agree, if you can detect a missing key sooner, you'd better do that, instead of returning invalid page to XtraDB, because it used to crash the server in case of severe errors like that. Regards, Sergei

1 0

[Maria-developers] Isn't thsi a bug?
by Peter Laursen 13 Oct '14

13 Oct '14

Refer: http://bugs.mysql.com/bug.php?id=74238 here I prefer the 'relaxed parsing' in Maria DB. But look at this: CREATE TABLE `d` ( `rec_id` INT(10) UNSIGNED NOT NULL AUTO_INCREMENT, `datetime` DATETIME(4) DEFAULT NULL, `timestamp` TIMESTAMP(4) NOT NULL DEFAULT CURRENT_TIMESTAMP(5) ON UPDATE CURRENT_TIMESTAMP(4), PRIMARY KEY (`rec_id`) ) ENGINE=INNODB; -- it actually creates the table SHOW CREATE TABLE d; /* returns CREATE TABLE `d` ( `rec_id` int(10) unsigned NOT NULL AUTO_INCREMENT, `datetime` datetime(4) DEFAULT NULL, `timestamp` timestamp(4) NOT NULL DEFAULT CURRENT_TIMESTAMP(4) ON UPDATE CURRENT_TIMESTAMP(4), PRIMARY KEY (`rec_id`) ) ENGINE=InnoDB DEFAULT CHARSET=latin1 */ I don't think DEFAULT CURRENT_TIMESTAMP(5) should be accepted for the datatype DATETIME(4). What say? -- Peter -- Webyog

2 2

[Maria-developers] Fwd: [JIRA] (MDEV-6566) Different INSERT behaviour on bad bytes with and without character set conversion
by Alexander Barkov 13 Oct '14

13 Oct '14

Hi Sergei, I suggest to fix this starting from 10.1. -------- Original Message -------- Subject: [JIRA] (MDEV-6566) Different INSERT behaviour on bad bytes with and without character set conversion Date: Thu, 9 Oct 2014 21:03:51 +0300 (EEST) From: Sergei Golubchik (JIRA) <jira(a)mariadb.atlassian.net> To: bar(a)mariadb.org [ https://mariadb.atlassian.net/browse/MDEV-6566?page=com.atlassian.jira.plug… ] Sergei Golubchik updated MDEV-6566: ----------------------------------- Fix Version/s: (was: 5.5.40) 5.5.41 > Different INSERT behaviour on bad bytes with and without character set conversion > --------------------------------------------------------------------------------- > > Key: MDEV-6566 > URL: https://mariadb.atlassian.net/browse/MDEV-6566 > Project: MariaDB Server > Issue Type: Bug > Affects Versions: 5.3.12, 5.5.39, 10.0.13 > Reporter: Alexander Barkov > Assignee: Alexander Barkov > Fix For: 10.0.15, 5.5.41 > > > If I run this SQL script in a utf8 client: > {noformat} > SET NAMES utf8; > DROP TABLE IF EXISTS t1; > CREATE TABLE t1 ( > a VARCHAR(10) CHARACTER SET ucs2, > b VARCHAR(10) CHARACTER SET utf8 > ); > INSERT INTO t1 SELECT 'a 😁 b', 'a 😁 b'; > SHOW WARNINGS; > SELECT * FROM t1; > {noformat} > It displays the following warnings: > {noformat} > +---------+------+----------------------------------------------------------------------+ > | Level | Code | Message | > +---------+------+----------------------------------------------------------------------+ > | Warning | 1300 | Invalid utf8 character string: '\xF0\x9F\x98\x81 b' | > | Warning | 1300 | Invalid utf8 character string: '\xF0\x9F\x98\x81 b' | > | Warning | 1366 | Incorrect string value: '\xF0\x9F\x98\x81 b' for column 'a' at row 1 | > | Warning | 1265 | Data truncated for column 'b' at row 1 | > +---------+------+----------------------------------------------------------------------+{noformat} > with the following results: > {noformat} > +----------+------+ > | a | b | > +----------+------+ > | a ???? b | a | > +----------+------+ > {noformat} > Notice, the character '😁' is "U+1F601 GRINNING FACE WITH SMILING EYES". It is outside of the BMP range supported by MariaDB character sets "utf8" and "ucs2". > The two columns worked differently: > - The ucs2 column correctly treated '😁' as an invalid sequence of four bytes, > converted it into four question marks, and appended the character 'b' after them. > - The utf8 column just stopped on this character and lost the trailing 'b'. > The warnings are also different for the two columns. > As "utf8" and "ucs2" are equitable character sets that support exactly the > same character range and repertoire (just with a different encoding), > it would be fair to expect the same behaviour of the two columns. > The column "a" exposes a better behaviour: it preserves as much > information as possible. > The column "b" should be fixed to work in the same way with "a". > The reason of the failure is that the conversion code that is activated > for the column "a" (with conversion) uses an mb_wc()..mb_wc() loop, while the > code activated for the column "b" (without conversion) copies only a well formed > prefix. > The code without conversion should not stop on a bad byte sequence > and try to copy as much data as possible, like the conversion code does. -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003)

2 1

[Maria-developers] Page encryption exception handling.
by Clemens Dörrhöfer 13 Oct '14

13 Oct '14

Hi community, I am working on implementing page encryption for XtraDB. Encryption itself works fine, now I am focusing on exception handling. My problem is that whenever page encryption is misconfigured it will crash the entire server, which is no acceptable behaviour. Example I encrypted a table with key id 42, I delete the key from the keypool thus I can no longer decrypt the table, because the appropriate key is missing. Ideally whenever a client tries to query data from an encrypted table, which cannot be decrypted, it will receive an error message informing the client that Table X could not be decrypted because key 42 is missing. Can you point me out to how to accomplish such a behaviour. Greetings Clemens Dörrhöfer

2 1

Re: [Maria-developers] [Commits] [svoj@mariadb.org: Rev 4437: Fixed mroonga build failure on Power8: define generic gcc version of in lp:maria/10.0]
by Kouhei Sutou 09 Oct '14

09 Oct '14

Hi Sergey, Thanks for the fix! I've merged your change into Groonga: https://github.com/groonga/groonga/commit/b3d38c5b9863cb7a9e93b9461c8b93fbf… Thanks, -- kou In <20141009083146.GA3231@june> "[Commits] [svoj(a)mariadb.org: Rev 4437: Fixed mroonga build failure on Power8: define generic gcc version of in lp:maria/10.0]" on Thu, 9 Oct 2014 12:31:47 +0400, Sergey Vojtovich <svoj(a)mariadb.org> wrote: > Hi Kentoku, > > FYI: I just pushed this fix to 10.0. There're also some test failures on Power8 > in mroonga/storage suite, mroonga/wrapper was 100% successful. > > Regards, > Sergey > > ----- Forwarded message from Sergey Vojtovich <svoj(a)mariadb.org> ----- > > Date: Thu, 9 Oct 2014 08:25:58 +0000 (UTC) > From: Sergey Vojtovich <svoj(a)mariadb.org> > To: commits(a)mariadb.org > Subject: [Commits] Rev 4437: Fixed mroonga build failure on Power8: define generic gcc version of in lp:maria/10.0 > > At lp:maria/10.0 > > ------------------------------------------------------------ > revno: 4437 > revision-id: svoj(a)mariadb.org-20141009082545-nt183b7y1gjihfjp > parent: psergey(a)askmonty.org-20141006112922-45pcytb993xcwh89 > committer: Sergey Vojtovich <svoj(a)mariadb.org> > branch nick: 10.0-mroonga > timestamp: Thu 2014-10-09 12:25:45 +0400 > message: > Fixed mroonga build failure on Power8: define generic gcc version of > GRN_SET_64BIT. > === modified file 'storage/mroonga/vendor/groonga/lib/groonga_in.h' > --- a/storage/mroonga/vendor/groonga/lib/groonga_in.h 2014-09-20 15:33:45 +0000 > +++ b/storage/mroonga/vendor/groonga/lib/groonga_in.h 2014-10-09 08:25:45 +0000 > @@ -505,6 +505,9 @@ typedef int grn_cond; > /* todo */ > # define GRN_SET_64BIT(p,v) \ > (void)atomic_swap_64(p, v) > +# elif defined(__ATOMIC_SEQ_CST) /* GCC atomic builtins */ > +# define GRN_SET_64BIT(p,v) \ > + __atomic_store_n(p, v, __ATOMIC_SEQ_CST) > # endif /* ATOMIC 64BIT SET */ > > # ifdef HAVE_MKOSTEMP > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits > > ----- End forwarded message ----- > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits

2 2

[Maria-developers] INET6, data type plugins
by Alexander Barkov 09 Oct '14

09 Oct '14

Hello all, I have mostly implemented the server side part of the fixed length 16-byte INET6 data type (only some clean-ups left). MDEV-274 The data type for IPv6/IPv4 addresses in MariaDB https://mariadb.atlassian.net/browse/MDEV-274 I made it in a very new style, which is ready to data type plugins, as described in: MDEV-4912 Add a plugin to field types (column types) https://mariadb.atlassian.net/browse/MDEV-4912 It is done by introducing a new abstract super-class Type_handler with a number of virtual public methods (around 50 public methods at this point). To implement a new data type one just needs to implement an instantiable class derived from Type_handler and register the new data type in a special global registry. Everything start to work automatically, including (but not limited to): - correct comparison - hybrid type functions like COALESCE, LEAST - aggregate functions MAX() and MIN() - CAST to the new type - ORDER BY, both for field and for items - conversion to the standard data types - indexing and index search The definition of the instantiable class that actually implements INET6 looks like this: class Type_handler_inet6: public Type_handler { private: class Field_inet6: public Field { ... }; class Arg_comparator_inet6: public Arg_comparator { ... }; class Item_typecast_inet6: public Item_func { ... }; public: // various access methods }; It looks very nice for me so far :) I have some questions about the remaining things: 1. In my current patch the new data type INET6 is not loaded dynamically from a shared library. It's implemented inside mysqld. Is it Okey? Or would we prefer to make it as a plugin from the beginning? (in this case there's some more work left: the dynamic loader code, client-server protocol, PS protocol). 2. In case of pluggable data types we'll need some changes in the client-server protocol, to transfer data type names (and probably some more metadata) to the client side in some textual form. Any ideas on that? 3. PS protocol obviously needs two ways to bind data: in text format and in binary format. For example, in case of INET6 it would be nice to be able to bind parameters and fetch values using: - text representation "ffff::ffff" - and binary representation 0xFFFF000000000000000000000000FFFF. I can see some ways to do this: a. Every pluggable data type uses its own type code in MYSQL_BIND: char ip6[16]= "\xff\xff...\xff"; MYSQL_BIND bind; bind.buffer_type= 128; // This is INET6 bind.buffer= ip6; bind.length= sizeof(ip6); but the code (128) may vary between servers, depending on how the plugin is installed. b. Or we could introduce a singe new type code MYSQL_TYPE_RAW: char ip6[16]= "\xff\xff...\xff"; MYSQL_BIND bind; bind.buffer_type= MYSQL_TYPE_RAW; bind.buffer= ip6; bind.length= sizeof(ip6); The server should be able to treat a single type code with different data types without any problems, because it knows the parameter context. But this will not allow to mix values of different types though, for example insert an INET4 value into an INET6 column (or vice verse) won't be possible. c. Or we could support both "native" type codes and MYSQL_TYPE_RAW. 4. Do we need the INET4 data type (fixed length, 4 bytes)? 5. Do we need a PostgreSQL-alike INET data type which is able to store both INET4 and INET6 values (I guess this one should be variable length: 4-bytes for INET4 and 16 bytes for INET6 values. Any feedback would be appreciated. The current patch (attached) will be polished. but the general idea should already be clear from the current version. Thanks.

3 4

Re: [Maria-developers] [Commits] Rev 4391: MDEV-6676: Speculative parallel replication. in http://bazaar.launchpad.net/~maria-captains/maria/10.0
by Jonas Oreland 08 Oct '14

08 Oct '14

how about also adding a mode = 'auto' which currently can be == transaction, but in future could be extended to based on statistics flip actual modes back and forth ? /Jonas On Tue, Oct 7, 2014 at 4:20 PM, <knielsen(a)knielsen-hq.org> wrote: > At http://bazaar.launchpad.net/~maria-captains/maria/10.0 > > ------------------------------------------------------------ > revno: 4391 > revision-id: knielsen(a)knielsen-hq.org-20141007142037-y85xq099ltdiqno6 > parent: knielsen(a)knielsen-hq.org-20141003094323-nosx0y7vv0kjv20e > committer: Kristian Nielsen <knielsen(a)knielsen-hq.org> > branch nick: work-10.0-mdev6676 > timestamp: Tue 2014-10-07 16:20:37 +0200 > message: > MDEV-6676: Speculative parallel replication. > > Better syntax for configuring the parallel mode. Now it is a CHANGE > MASTER > option instead of a system variable, which makes it possible to configure > differently for different multi-source replication masters. > > Now also the domain-based replication mode can be disabled. > === modified file 'mysql-test/include/check-testcase.test' > --- a/mysql-test/include/check-testcase.test 2013-12-16 12:02:21 +0000 > +++ b/mysql-test/include/check-testcase.test 2014-10-07 14:20:37 +0000 > @@ -64,6 +64,7 @@ if ($tmp) > --echo Master_SSL_Crlpath # > --echo Using_Gtid No > --echo Gtid_IO_Pos # > + --echo Parallel_Mode domain,groupcommit > } > if (!$tmp) { > # Note: after WL#5177, fields 13-18 shall not be filtered-out. > > === modified file 'mysql-test/r/mysqld--help.result' > --- a/mysql-test/r/mysqld--help.result 2014-09-19 13:25:37 +0000 > +++ b/mysql-test/r/mysqld--help.result 2014-10-07 14:20:37 +0000 > @@ -915,17 +915,6 @@ > parallel replication thread when reading ahead in the > relay log looking for opportunities for parallel > replication. Only used when --slave-parallel-threads > 0. > - --slave-parallel-mode=name > - Controls what transactions are applied in parallel when > - using --slave-parallel-threads. Syntax: > - slave_paralle_mode=value[,value...], where "value" could > - be one or more of: "domain", to apply different > - replication domains in parallel; "groupcommit", to apply > - in parallel transactions that group-committed together on > - the master; "transactional", to optimistically try to > - apply all transactional DML in parallel; and "waiting" to > - extend "transactional" to even transactions that had to > - wait on the master. > --slave-parallel-threads=# > If non-zero, number of threads to spawn to apply in > parallel events on the slave that were group-committed on > @@ -1320,7 +1309,6 @@ slave-exec-mode STRICT > slave-max-allowed-packet 1073741824 > slave-net-timeout 3600 > slave-parallel-max-queued 131072 > -slave-parallel-mode domain,groupcommit > slave-parallel-threads 0 > slave-skip-errors (No default value) > slave-sql-verify-checksum TRUE > > === modified file 'mysql-test/suite/rpl/r/rpl_parallel.result' > --- a/mysql-test/suite/rpl/r/rpl_parallel.result 2014-08-15 > 09:31:13 +0000 > +++ b/mysql-test/suite/rpl/r/rpl_parallel.result 2014-10-07 > 14:20:37 +0000 > @@ -1,5 +1,5 @@ > -include/rpl_init.inc [topology=1->2] > -SET @old_parallel_threads=@@GLOBAL.slave_parallel_threads; > +include/master-slave.inc > +[connection master] > SET GLOBAL slave_parallel_threads=10; > ERROR HY000: This operation cannot be performed as you have a running > slave ''; run STOP SLAVE '' first > include/stop_slave.inc > @@ -923,8 +923,79 @@ SELECT * FROM t2 WHERE a >= 30 ORDER BY > 32 > 33 > 34 > +*** MDEV-6676 - test syntax of CHANGE MASTER TO PARALLEL_MODE=xxx *** > +Parallel_Mode = 'domain,groupcommit' > include/stop_slave.inc > -SET GLOBAL slave_parallel_threads=@old_parallel_threads; > +CHANGE MASTER TO parallel_mode=(,domain,groupcommit); > +ERROR 42000: You have an error in your SQL syntax; check the manual that > corresponds to your MariaDB server version for the right syntax to use near > 'domain,groupcommit)' at line 1 > +CHANGE MASTER TO parallel_mode=(domain,groupcommit,transactional); > +ERROR HY000: Invalid use of 'transactional' option for PARALLEL_MODE in > CHANGE MASTER TO > +CHANGE MASTER TO parallel_mode=(waiting,domain,transactional,waiting); > +ERROR HY000: Invalid use of 'waiting' option for PARALLEL_MODE in CHANGE > MASTER TO > +CHANGE MASTER TO parallel_mode=(domain,domain); > +ERROR HY000: Invalid use of 'domain' option for PARALLEL_MODE in CHANGE > MASTER TO > +CHANGE MASTER TO parallel_mode=(waiting,transactional,domain); > +Parallel_Mode = 'domain,transactional,waiting' > +CHANGE MASTER TO parallel_mode=(domain,groupcommit); > +Parallel_Mode = 'domain,groupcommit' > +*** MDEV-6676 - test that empty parallel_mode does not replicate in > parallel *** > +INSERT INTO t2 VALUES (40); > +include/save_master_gtid.inc > +CHANGE MASTER TO parallel_mode=(); > +SET @old_dbug= @@GLOBAL.debug_dbug; > +SET GLOBAL debug_dbug="+d,slave_crash_if_parallel_apply"; > +include/start_slave.inc > +include/sync_with_master_gtid.inc > +SELECT * FROM t2 WHERE a >= 40 ORDER BY a; > +a > +40 > +include/stop_slave.inc > +SET GLOBAL debug_dbug=@old_dbug; > +*** MDEV-6676 - test disabling domain-based parallel replication *** > +SET gtid_domain_id = 1; > +INSERT INTO t2 VALUES (41); > +INSERT INTO t2 VALUES (42); > +INSERT INTO t2 VALUES (43); > +INSERT INTO t2 VALUES (44); > +INSERT INTO t2 VALUES (45); > +INSERT INTO t2 VALUES (46); > +DELETE FROM t2 WHERE a >= 41; > +SET gtid_domain_id = 2; > +INSERT INTO t2 VALUES (41); > +INSERT INTO t2 VALUES (42); > +INSERT INTO t2 VALUES (43); > +INSERT INTO t2 VALUES (44); > +INSERT INTO t2 VALUES (45); > +INSERT INTO t2 VALUES (46); > +SET gtid_domain_id = 0; > +include/save_master_gtid.inc > +CHANGE MASTER TO parallel_mode=(groupcommit); > +include/start_slave.inc > +include/sync_with_master_gtid.inc > +SELECT * FROM t2 WHERE a >= 40 ORDER BY a; > +a > +40 > +41 > +42 > +43 > +44 > +45 > +46 > +include/stop_slave.inc > +CHANGE MASTER TO parallel_mode=(domain,groupcommit); > +include/start_slave.inc > +*** MDEV-6676 - test that parallel mode is saved correctly in master.info > across server restart *** > +Parallel_Mode = 'domain,groupcommit' > +include/stop_slave.inc > +CHANGE MASTER TO parallel_mode=(transactional,waiting); > +Parallel_Mode = 'transactional,waiting' > +include/start_slave.inc > +include/rpl_restart_server.inc [server_number=2] > +Parallel_Mode = 'transactional,waiting' > +CHANGE MASTER TO parallel_mode=(domain,groupcommit); > +include/start_slave.inc > +include/stop_slave.inc > +SET GLOBAL slave_parallel_threads=0; > include/start_slave.inc > SET DEBUG_SYNC= 'RESET'; > DROP function foo; > > === modified file 'mysql-test/suite/rpl/r/rpl_parallel_optimistic.result' > --- a/mysql-test/suite/rpl/r/rpl_parallel_optimistic.result 2014-09-24 > 19:25:10 +0000 > +++ b/mysql-test/suite/rpl/r/rpl_parallel_optimistic.result 2014-10-07 > 14:20:37 +0000 > @@ -2,11 +2,9 @@ include/rpl_init.inc [topology=1->2] > ALTER TABLE mysql.gtid_slave_pos ENGINE=InnoDB; > CREATE TABLE t1 (a int PRIMARY KEY, b INT) ENGINE=InnoDB; > SET @old_parallel_threads=@@GLOBAL.slave_parallel_threads; > -SET @old_mode=@@GLOBAL.slave_parallel_mode; > include/stop_slave.inc > SET GLOBAL slave_parallel_threads=10; > -SET GLOBAL slave_parallel_mode="domain,transactional"; > -CHANGE MASTER TO master_use_gtid=slave_pos; > +CHANGE MASTER TO master_use_gtid=slave_pos, > parallel_mode=(domain,transactional); > INSERT INTO t1 VALUES(1,1); > BEGIN; > INSERT INTO t1 VALUES(2,1); > @@ -288,8 +286,8 @@ SET GLOBAL binlog_format= @old_format; > SET GLOBAL tx_isolation= @old_isolation; > include/start_slave.inc > include/stop_slave.inc > +CHANGE MASTER TO parallel_mode=(domain,groupcommit); > SET GLOBAL slave_parallel_threads=@old_parallel_threads; > -SET GLOBAL slave_parallel_mode=@old_mode; > include/start_slave.inc > DROP TABLE t1, t2, t3; > include/rpl_end.inc > > === modified file > 'mysql-test/suite/rpl/r/rpl_parallel_optimistic_nobinlog.result' > --- a/mysql-test/suite/rpl/r/rpl_parallel_optimistic_nobinlog.result > 2014-09-24 19:25:10 +0000 > +++ b/mysql-test/suite/rpl/r/rpl_parallel_optimistic_nobinlog.result > 2014-10-07 14:20:37 +0000 > @@ -6,12 +6,10 @@ INSERT INTO t1 VALUES (1,0), (2,0), (3,0 > INSERT INTO t2 VALUES (1,0), (2,0); > SET @old_isolation= @@GLOBAL.tx_isolation; > SET @old_parallel_threads=@@GLOBAL.slave_parallel_threads; > -SET @old_mode=@@GLOBAL.slave_parallel_mode; > include/stop_slave.inc > SET GLOBAL TRANSACTION ISOLATION LEVEL READ COMMITTED; > SET GLOBAL slave_parallel_threads=10; > -SET GLOBAL slave_parallel_mode="domain,transactional,waiting"; > -CHANGE MASTER TO master_use_gtid=slave_pos; > +CHANGE MASTER TO master_use_gtid=slave_pos, > parallel_mode=(domain,transactional,waiting); > *** Test that we replicate correctly when using READ COMMITTED and > --log-slave-updates=0 on the slave *** > INSERT INTO t1 SELECT 4, COUNT(*) FROM t2; > INSERT INTO t2 SELECT 4, COUNT(*) FROM t1; > @@ -78,8 +76,8 @@ a b > 10 10 > include/stop_slave.inc > SET GLOBAL tx_isolation= @old_isolation; > +CHANGE MASTER TO parallel_mode=(domain,groupcommit); > SET GLOBAL slave_parallel_threads=@old_parallel_threads; > -SET GLOBAL slave_parallel_mode=@old_mode; > include/start_slave.inc > DROP TABLE t1, t2; > include/rpl_end.inc > > === modified file 'mysql-test/suite/rpl/t/rpl_parallel.test' > --- a/mysql-test/suite/rpl/t/rpl_parallel.test 2014-08-15 09:31:13 +0000 > +++ b/mysql-test/suite/rpl/t/rpl_parallel.test 2014-10-07 14:20:37 +0000 > @@ -1,13 +1,12 @@ > --source include/have_innodb.inc > --source include/have_debug.inc > --source include/have_debug_sync.inc > ---let $rpl_topology=1->2 > ---source include/rpl_init.inc > +--source include/master-slave.inc > > # Test various aspects of parallel replication. > > --connection server_2 > -SET @old_parallel_threads=@@GLOBAL.slave_parallel_threads; > +--let $old_parallel_threads=`SELECT @@GLOBAL.slave_parallel_threads` > --error ER_SLAVE_MUST_STOP > SET GLOBAL slave_parallel_threads=10; > --source include/stop_slave.inc > @@ -1466,9 +1465,100 @@ SET sql_slave_skip_counter= 1; > SELECT * FROM t2 WHERE a >= 30 ORDER BY a; > > > +--echo *** MDEV-6676 - test syntax of CHANGE MASTER TO PARALLEL_MODE=xxx > *** > +--connection server_2 > + > +--let $status_items= Parallel_Mode > +--source include/show_slave_status.inc > +--source include/stop_slave.inc > +--error ER_PARSE_ERROR > +CHANGE MASTER TO parallel_mode=(,domain,groupcommit); > +--error ER_INVALID_SLAVE_PARALLEL_MODE > +CHANGE MASTER TO parallel_mode=(domain,groupcommit,transactional); > +--error ER_INVALID_SLAVE_PARALLEL_MODE > +CHANGE MASTER TO parallel_mode=(waiting,domain,transactional,waiting); > +--error ER_INVALID_SLAVE_PARALLEL_MODE > +CHANGE MASTER TO parallel_mode=(domain,domain); > +CHANGE MASTER TO parallel_mode=(waiting,transactional,domain); > +--let $status_items= Parallel_Mode > +--source include/show_slave_status.inc > +CHANGE MASTER TO parallel_mode=(domain,groupcommit); > +--let $status_items= Parallel_Mode > +--source include/show_slave_status.inc > + > + > +--echo *** MDEV-6676 - test that empty parallel_mode does not replicate > in parallel *** > +--connection server_1 > +INSERT INTO t2 VALUES (40); > +--source include/save_master_gtid.inc > + > +--connection server_2 > +CHANGE MASTER TO parallel_mode=(); > +# Test that we do not use parallel apply, by injecting an unconditional > +# crash in the parallel apply code. > +SET @old_dbug= @@GLOBAL.debug_dbug; > +SET GLOBAL debug_dbug="+d,slave_crash_if_parallel_apply"; > +--source include/start_slave.inc > +--source include/sync_with_master_gtid.inc > +SELECT * FROM t2 WHERE a >= 40 ORDER BY a; > +--source include/stop_slave.inc > +SET GLOBAL debug_dbug=@old_dbug; > + > + > +--echo *** MDEV-6676 - test disabling domain-based parallel replication > *** > +--connection server_1 > +# Let's do a bunch of transactions that will conflict if run out-of-order > in > +# domain-based parallel replication mode. > +SET gtid_domain_id = 1; > +INSERT INTO t2 VALUES (41); > +INSERT INTO t2 VALUES (42); > +INSERT INTO t2 VALUES (43); > +INSERT INTO t2 VALUES (44); > +INSERT INTO t2 VALUES (45); > +INSERT INTO t2 VALUES (46); > +DELETE FROM t2 WHERE a >= 41; > +SET gtid_domain_id = 2; > +INSERT INTO t2 VALUES (41); > +INSERT INTO t2 VALUES (42); > +INSERT INTO t2 VALUES (43); > +INSERT INTO t2 VALUES (44); > +INSERT INTO t2 VALUES (45); > +INSERT INTO t2 VALUES (46); > +SET gtid_domain_id = 0; > +--source include/save_master_gtid.inc > +--connection server_2 > +CHANGE MASTER TO parallel_mode=(groupcommit); > +--source include/start_slave.inc > +--source include/sync_with_master_gtid.inc > +SELECT * FROM t2 WHERE a >= 40 ORDER BY a; > +--source include/stop_slave.inc > +CHANGE MASTER TO parallel_mode=(domain,groupcommit); > +--source include/start_slave.inc > + > + > +--echo *** MDEV-6676 - test that parallel mode is saved correctly in > master.info across server restart *** > +--connection server_2 > +--let $status_items= Parallel_Mode > +--source include/show_slave_status.inc > +--source include/stop_slave.inc > +CHANGE MASTER TO parallel_mode=(transactional,waiting); > +--let $status_items= Parallel_Mode > +--source include/show_slave_status.inc > +--source include/start_slave.inc > + > +--let $rpl_server_number= 2 > +--source include/rpl_restart_server.inc > + > +--connection server_2 > +--let $status_items= Parallel_Mode > +--source include/show_slave_status.inc > +CHANGE MASTER TO parallel_mode=(domain,groupcommit); > +--source include/start_slave.inc > + > + > --connection server_2 > --source include/stop_slave.inc > -SET GLOBAL slave_parallel_threads=@old_parallel_threads; > +eval SET GLOBAL slave_parallel_threads=$old_parallel_threads; > --source include/start_slave.inc > SET DEBUG_SYNC= 'RESET'; > > > === modified file 'mysql-test/suite/rpl/t/rpl_parallel_optimistic.test' > --- a/mysql-test/suite/rpl/t/rpl_parallel_optimistic.test 2014-09-24 > 19:25:10 +0000 > +++ b/mysql-test/suite/rpl/t/rpl_parallel_optimistic.test 2014-10-07 > 14:20:37 +0000 > @@ -11,11 +11,9 @@ CREATE TABLE t1 (a int PRIMARY KEY, b IN > --connection server_2 > --sync_with_master > SET @old_parallel_threads=@@GLOBAL.slave_parallel_threads; > -SET @old_mode=@@GLOBAL.slave_parallel_mode; > --source include/stop_slave.inc > SET GLOBAL slave_parallel_threads=10; > -SET GLOBAL slave_parallel_mode="domain,transactional"; > -CHANGE MASTER TO master_use_gtid=slave_pos; > +CHANGE MASTER TO master_use_gtid=slave_pos, > parallel_mode=(domain,transactional); > > > --connection server_1 > @@ -309,8 +307,8 @@ SET GLOBAL tx_isolation= @old_isolation; > > --connection server_2 > --source include/stop_slave.inc > +CHANGE MASTER TO parallel_mode=(domain,groupcommit); > SET GLOBAL slave_parallel_threads=@old_parallel_threads; > -SET GLOBAL slave_parallel_mode=@old_mode; > --source include/start_slave.inc > > --connection server_1 > > === modified file > 'mysql-test/suite/rpl/t/rpl_parallel_optimistic_nobinlog.test' > --- a/mysql-test/suite/rpl/t/rpl_parallel_optimistic_nobinlog.test > 2014-09-24 19:25:10 +0000 > +++ b/mysql-test/suite/rpl/t/rpl_parallel_optimistic_nobinlog.test > 2014-10-07 14:20:37 +0000 > @@ -16,12 +16,10 @@ INSERT INTO t2 VALUES (1,0), (2,0); > --sync_with_master > SET @old_isolation= @@GLOBAL.tx_isolation; > SET @old_parallel_threads=@@GLOBAL.slave_parallel_threads; > -SET @old_mode=@@GLOBAL.slave_parallel_mode; > --source include/stop_slave.inc > SET GLOBAL TRANSACTION ISOLATION LEVEL READ COMMITTED; > SET GLOBAL slave_parallel_threads=10; > -SET GLOBAL slave_parallel_mode="domain,transactional,waiting"; > -CHANGE MASTER TO master_use_gtid=slave_pos; > +CHANGE MASTER TO master_use_gtid=slave_pos, > parallel_mode=(domain,transactional,waiting); > > > --echo *** Test that we replicate correctly when using READ COMMITTED and > --log-slave-updates=0 on the slave *** > @@ -63,8 +61,8 @@ SELECT * FROM t2 ORDER BY a; > --connection server_2 > --source include/stop_slave.inc > SET GLOBAL tx_isolation= @old_isolation; > +CHANGE MASTER TO parallel_mode=(domain,groupcommit); > SET GLOBAL slave_parallel_threads=@old_parallel_threads; > -SET GLOBAL slave_parallel_mode=@old_mode; > --source include/start_slave.inc > > --connection server_1 > > === modified file 'sql/lex.h' > --- a/sql/lex.h 2014-02-02 09:00:36 +0000 > +++ b/sql/lex.h 2014-10-07 14:20:37 +0000 > @@ -194,6 +194,7 @@ static SYMBOL symbols[] = { > { "DISTINCTROW", SYM(DISTINCT)}, /* Access likes this */ > { "DIV", SYM(DIV_SYM)}, > { "DO", SYM(DO_SYM)}, > + { "DOMAIN", SYM(DOMAIN_SYM)}, > { "DOUBLE", SYM(DOUBLE_SYM)}, > { "DROP", SYM(DROP)}, > { "DUAL", SYM(DUAL_SYM)}, > @@ -257,6 +258,7 @@ static SYMBOL symbols[] = { > { "GRANT", SYM(GRANT)}, > { "GRANTS", SYM(GRANTS)}, > { "GROUP", SYM(GROUP_SYM)}, > + { "GROUPCOMMIT", SYM(GROUPCOMMIT_SYM)}, > { "HANDLER", SYM(HANDLER_SYM)}, > { "HARD", SYM(HARD_SYM)}, > { "HASH", SYM(HASH_SYM)}, > @@ -429,6 +431,7 @@ static SYMBOL symbols[] = { > { "PACK_KEYS", SYM(PACK_KEYS_SYM)}, > { "PAGE", SYM(PAGE_SYM)}, > { "PAGE_CHECKSUM", SYM(PAGE_CHECKSUM_SYM)}, > + { "PARALLEL_MODE", SYM(PARALLEL_MODE_SYM)}, > { "PARSER", SYM(PARSER_SYM)}, > { "PARSE_VCOL_EXPR", SYM(PARSE_VCOL_EXPR_SYM)}, > { "PARTIAL", SYM(PARTIAL)}, > @@ -650,6 +653,7 @@ static SYMBOL symbols[] = { > { "VIEW", SYM(VIEW_SYM)}, > { "VIRTUAL", SYM(VIRTUAL_SYM)}, > { "WAIT", SYM(WAIT_SYM)}, > + { "WAITING", SYM(WAITING_SYM)}, > { "WARNINGS", SYM(WARNINGS)}, > { "WEEK", SYM(WEEK_SYM)}, > { "WEIGHT_STRING", SYM(WEIGHT_STRING_SYM)}, > > === modified file 'sql/log_event.cc' > --- a/sql/log_event.cc 2014-09-25 13:47:58 +0000 > +++ b/sql/log_event.cc 2014-10-07 14:20:37 +0000 > @@ -12728,7 +12728,7 @@ bool rpl_get_position_info(const char ** > return FALSE; > #else > const Relay_log_info *rli= &(active_mi->rli); > - if (opt_slave_parallel_threads == 0) > + if (!rli->mi->using_parallel()) > { > *log_file_name= rli->group_master_log_name; > *log_pos= rli->group_master_log_pos + > > === modified file 'sql/mysqld.cc' > --- a/sql/mysqld.cc 2014-09-26 10:37:19 +0000 > +++ b/sql/mysqld.cc 2014-10-07 14:20:37 +0000 > @@ -562,8 +562,6 @@ ulong stored_program_cache_size= 0; > > ulong opt_slave_parallel_threads= 0; > ulong opt_slave_domain_parallel_threads= 0; > -ulonglong opt_slave_parallel_mode= > - SLAVE_PARALLEL_DOMAIN | SLAVE_PARALLEL_GROUPCOMMIT; > ulong opt_binlog_commit_wait_count= 0; > ulong opt_binlog_commit_wait_usec= 0; > ulong opt_slave_parallel_max_queued= 131072; > > === modified file 'sql/mysqld.h' > --- a/sql/mysqld.h 2014-09-26 10:37:19 +0000 > +++ b/sql/mysqld.h 2014-10-07 14:20:37 +0000 > @@ -187,7 +187,6 @@ extern ulong stored_program_cache_size; > extern ulong opt_slave_parallel_threads; > extern ulong opt_slave_domain_parallel_threads; > extern ulong opt_slave_parallel_max_queued; > -extern ulonglong opt_slave_parallel_mode; > extern ulong opt_binlog_commit_wait_count; > extern ulong opt_binlog_commit_wait_usec; > extern my_bool opt_gtid_ignore_duplicates; > > === modified file 'sql/rpl_mi.cc' > --- a/sql/rpl_mi.cc 2014-09-02 12:07:01 +0000 > +++ b/sql/rpl_mi.cc 2014-10-07 14:20:37 +0000 > @@ -40,7 +40,8 @@ Master_info::Master_info(LEX_STRING *con > heartbeat_period(0), received_heartbeats(0), master_id(0), > prev_master_id(0), > using_gtid(USE_GTID_NO), events_queued_since_last_gtid(0), > - gtid_reconnect_event_skip_count(0), gtid_event_seen(false) > + gtid_reconnect_event_skip_count(0), gtid_event_seen(false), > + parallel_mode(SLAVE_PARALLEL_DOMAIN|SLAVE_PARALLEL_GROUPCOMMIT) > { > host[0] = 0; user[0] = 0; password[0] = 0; > ssl_ca[0]= 0; ssl_capath[0]= 0; ssl_cert[0]= 0; > @@ -178,6 +179,7 @@ void init_master_log_pos(Master_info* mi > mi->events_queued_since_last_gtid= 0; > mi->gtid_reconnect_event_skip_count= 0; > mi->gtid_event_seen= false; > + mi->parallel_mode= SLAVE_PARALLEL_DOMAIN|SLAVE_PARALLEL_GROUPCOMMIT; > > /* Intentionally init ssl_verify_server_cert to 0, no option available > */ > mi->ssl_verify_server_cert= 0; > @@ -514,6 +516,22 @@ file '%s')", fname); > else > mi->using_gtid= Master_info::USE_GTID_NO; > } > + else if (!strncmp(buf, STRING_WITH_LEN("parallel_mode="))) > + { > + int val= atoi(buf + (sizeof("parallel_mode=") - 1)); > + mi->parallel_mode= val & (SLAVE_PARALLEL_DOMAIN | > + SLAVE_PARALLEL_GROUPCOMMIT | > + SLAVE_PARALLEL_TRX | > + SLAVE_PARALLEL_WAITING); > + } > + else if (!strncmp(buf, STRING_WITH_LEN("END_MARKER"))) > + { > + /* > + Guard agaist extra left-overs at the end of file, in case a > later > + update causes the file to shrink compared to earlier > contents. > + */ > + break; > + } > } > } > } > @@ -657,7 +675,9 @@ int flush_master_info(Master_info* mi, > my_b_printf(file, > > "%u\n%s\n%s\n%s\n%s\n%s\n%d\n%d\n%d\n%s\n%s\n%s\n%s\n%s\n%d\n%s\n%s\n%s\n%s\n%d\n%s\n%s\n" > "\n\n\n\n\n\n\n\n\n\n\n" > - "using_gtid=%d\n", > + "using_gtid=%d\n" > + "parallel_mode=%u\n" > + "END_MARKER\n", > LINES_IN_MASTER_INFO, > mi->master_log_name, llstr(mi->master_log_pos, lbuf), > mi->host, mi->user, > @@ -666,7 +686,7 @@ int flush_master_info(Master_info* mi, > mi->ssl_cipher, mi->ssl_key, mi->ssl_verify_server_cert, > heartbeat_buf, "", ignore_server_ids_buf, > "", 0, > - mi->ssl_crl, mi->ssl_crlpath, mi->using_gtid); > + mi->ssl_crl, mi->ssl_crlpath, mi->using_gtid, > mi->parallel_mode); > my_free(ignore_server_ids_buf); > err= flush_io_cache(file); > if (sync_masterinfo_period && !err && > > === modified file 'sql/rpl_mi.h' > --- a/sql/rpl_mi.h 2014-09-02 12:07:01 +0000 > +++ b/sql/rpl_mi.h 2014-10-07 14:20:37 +0000 > @@ -75,6 +75,10 @@ class Master_info : public Slave_reporti > return connection_name.str == 0; > } > static const char *using_gtid_astext(enum enum_using_gtid arg); > + bool using_parallel() > + { > + return opt_slave_parallel_threads > 0 && parallel_mode != 0; > + } > > /* the variables below are needed because we can change masters on the > fly */ > char master_log_name[FN_REFLEN+6]; /* Room for multi-*/ > @@ -178,6 +182,12 @@ class Master_info : public Slave_reporti > uint64 gtid_reconnect_event_skip_count; > /* gtid_event_seen is false until we receive first GTID event from > master. */ > bool gtid_event_seen; > + /* > + The parallel replication modes, if any. A combination (binary OR) of > any of > + SLAVE_PARALLEL_DOMAIN, SLAVE_PARALLEL_GROUPCOMMIT, > SLAVE_PARALLEL_TRX, and > + SLAVE_PARALLEL_WAITING. > + */ > + uint32 parallel_mode; > }; > int init_master_info(Master_info* mi, const char* master_info_fname, > const char* slave_info_fname, > > === modified file 'sql/rpl_parallel.cc' > --- a/sql/rpl_parallel.cc 2014-10-03 09:43:23 +0000 > +++ b/sql/rpl_parallel.cc 2014-10-07 14:20:37 +0000 > @@ -1891,6 +1891,7 @@ rpl_parallel::do_event(rpl_group_info *s > bool did_enter_cond= false; > PSI_stage_info old_stage; > > + DBUG_EXECUTE_IF("slave_crash_if_parallel_apply", DBUG_SUICIDE();); > /* Handle master log name change, seen in Rotate_log_event. */ > typ= ev->get_type_code(); > if (unlikely(typ == ROTATE_EVENT)) > @@ -1971,7 +1972,8 @@ rpl_parallel::do_event(rpl_group_info *s > if (typ == GTID_EVENT) > { > Gtid_log_event *gtid_ev= static_cast<Gtid_log_event *>(ev); > - uint32 domain_id= (rli->mi->using_gtid == Master_info::USE_GTID_NO ? > + uint32 domain_id= (rli->mi->using_gtid == Master_info::USE_GTID_NO || > + !(rli->mi->parallel_mode & SLAVE_PARALLEL_DOMAIN) ? > 0 : gtid_ev->domain_id); > if (!(e= find(domain_id))) > { > @@ -2012,7 +2014,7 @@ rpl_parallel::do_event(rpl_group_info *s > { > Gtid_log_event *gtid_ev= static_cast<Gtid_log_event *>(ev); > bool new_gco; > - ulonglong mode= opt_slave_parallel_mode; > + ulonglong mode= rli->mi->parallel_mode; > uchar gtid_flags= gtid_ev->flags2; > group_commit_orderer *gco; > uint8 force_switch_flag; > > === modified file 'sql/rpl_parallel.h' > --- a/sql/rpl_parallel.h 2014-10-03 09:43:23 +0000 > +++ b/sql/rpl_parallel.h 2014-10-07 14:20:37 +0000 > @@ -12,7 +12,11 @@ class Relay_log_info; > struct inuse_relaylog; > > > -/* Bit masks for the values in --slave-parallel-mode. */ > +/* > + Bit masks for the values in --slave-parallel-mode. > + Note that these values cannot be changed - they are stored in > master.info, > + so need to be possible to read back in a different version of the > server. > +*/ > #define SLAVE_PARALLEL_DOMAIN (1ULL << 0) > #define SLAVE_PARALLEL_GROUPCOMMIT (1ULL << 1) > #define SLAVE_PARALLEL_TRX (1ULL << 2) > > === modified file 'sql/share/errmsg-utf8.txt' > --- a/sql/share/errmsg-utf8.txt 2014-07-08 17:38:26 +0000 > +++ b/sql/share/errmsg-utf8.txt 2014-10-07 14:20:37 +0000 > @@ -7111,3 +7111,5 @@ ER_SLAVE_SKIP_NOT_IN_GTID > eng "When using GTID, @@sql_slave_skip_counter can not be used. > Instead, setting @@gtid_slave_pos explicitly can be used to skip to after a > given GTID position." > ER_TABLE_DEFINITION_TOO_BIG > eng "The definition for table %`s is too big" > +ER_INVALID_SLAVE_PARALLEL_MODE > + eng "Invalid use of '%s' option for PARALLEL_MODE in CHANGE > MASTER TO" > > === modified file 'sql/slave.cc' > --- a/sql/slave.cc 2014-09-02 12:07:01 +0000 > +++ b/sql/slave.cc 2014-10-07 14:20:37 +0000 > @@ -625,8 +625,7 @@ int terminate_slave_threads(Master_info* > if (thread_mask & (SLAVE_SQL|SLAVE_FORCE_ALL)) > { > DBUG_PRINT("info",("Terminating SQL thread")); > - if (opt_slave_parallel_threads > 0 && > - mi->rli.abort_slave && mi->rli.stop_for_until) > + if (mi->using_parallel() && mi->rli.abort_slave && > mi->rli.stop_for_until) > { > mi->rli.stop_for_until= false; > mi->rli.parallel.stop_during_until(); > @@ -2576,6 +2575,8 @@ static bool send_show_master_info_header > field_list.push_back(new Item_empty_string("Using_Gtid", > sizeof("Current_Pos")-1)); > field_list.push_back(new Item_empty_string("Gtid_IO_Pos", 30)); > + field_list.push_back(new Item_empty_string("Parallel_Mode", > + sizeof("domain,groupcommit,transactional,waiting")-1)); > if (full) > { > field_list.push_back(new Item_return_int("Retried_transactions", > @@ -2708,8 +2709,7 @@ static bool send_show_master_info_data(T > else > { > idle= mi->rli.sql_thread_caught_up; > - if (opt_slave_parallel_threads > 0 && idle && > - !mi->rli.parallel.workers_idle()) > + if (mi->using_parallel() && idle && > !mi->rli.parallel.workers_idle()) > idle= false; > } > if (idle) > @@ -2786,13 +2786,34 @@ static bool send_show_master_info_data(T > protocol->store(mi->ssl_ca, &my_charset_bin); > // Master_Ssl_Crlpath > protocol->store(mi->ssl_capath, &my_charset_bin); > + // Using_Gtid > protocol->store(mi->using_gtid_astext(mi->using_gtid), > &my_charset_bin); > + // Gtid_IO_Pos > { > char buff[30]; > String tmp(buff, sizeof(buff), system_charset_info); > mi->gtid_current_pos.to_string(&tmp); > protocol->store(tmp.ptr(), tmp.length(), &my_charset_bin); > } > + // Parallel_Mode > + { > + /* Note how sizeof("domain") has room for "domain," due to traling > 0. */ > + char buf[sizeof("domain") + sizeof("groupcommit") + > + sizeof("transactional") + sizeof("waiting") + 1]; > + char *p= buf; > + uint32 mode= mi->parallel_mode; > + if (mode & SLAVE_PARALLEL_DOMAIN) > + p= strmov(p, "domain,"); > + if (mode & SLAVE_PARALLEL_GROUPCOMMIT) > + p= strmov(p, "groupcommit,"); > + if (mode & SLAVE_PARALLEL_TRX) > + p= strmov(p, "transactional,"); > + if (mode & SLAVE_PARALLEL_WAITING) > + p= strmov(p, "waiting,"); > + if (p != buf) > + --p; // Discard last ',' > + protocol->store(buf, p-buf, &my_charset_bin); > + } > if (full) > { > protocol->store((uint32) mi->rli.retried_trans); > @@ -3521,7 +3542,7 @@ static int exec_relay_log_event(THD* thd > > update_state_of_relay_log(rli, ev); > > - if (opt_slave_parallel_threads > 0) > + if (rli->mi->using_parallel()) > { > int res= rli->parallel.do_event(serial_rgi, ev, event_size); > if (res >= 0) > @@ -4667,7 +4688,7 @@ log '%s' at position %s, relay log '%s' > } > } > > - if (opt_slave_parallel_threads > 0) > + if (mi->using_parallel()) > rli->parallel.wait_for_done(thd, rli); > > /* Thread stopped. Print the current replication position to the log */ > @@ -4693,7 +4714,7 @@ log '%s' at position %s, relay log '%s' > (We want the first one to be before the printout of stop position to > get the correct position printed.) > */ > - if (opt_slave_parallel_threads > 0) > + if (mi->using_parallel()) > rli->parallel.wait_for_done(thd, rli); > > /* > @@ -6314,7 +6335,7 @@ static Log_event* next_event(rpl_group_i > llstr(my_b_tell(cur_log),llbuf1), > llstr(rli->event_relay_log_pos,llbuf2))); > DBUG_ASSERT(my_b_tell(cur_log) >= BIN_LOG_HEADER_SIZE); > - DBUG_ASSERT(opt_slave_parallel_threads > 0 || > + DBUG_ASSERT(rli->mi->using_parallel() || > my_b_tell(cur_log) == rli->event_relay_log_pos); > } > #endif > > === modified file 'sql/sql_lex.h' > --- a/sql/sql_lex.h 2014-08-07 16:06:56 +0000 > +++ b/sql/sql_lex.h 2014-10-07 14:20:37 +0000 > @@ -218,11 +218,18 @@ struct LEX_MASTER_INFO > uint port, connect_retry; > float heartbeat_period; > /* > + Modes of parallel replication enabled, if any. A combination (binary > OR) of > + any of SLAVE_PARALLEL_DOMAIN, SLAVE_PARALLEL_GROUPCOMMIT, > + SLAVE_PARALLEL_TRX, and SLAVE_PARALLEL_WAITING. > + */ > + uint32 repl_parallel_mode; > + /* > Enum is used for making it possible to detect if the user > changed variable or if it should be left at old value > */ > enum {LEX_MI_UNCHANGED, LEX_MI_DISABLE, LEX_MI_ENABLE} > - ssl, ssl_verify_server_cert, heartbeat_opt, > repl_ignore_server_ids_opt; > + ssl, ssl_verify_server_cert, heartbeat_opt, > repl_ignore_server_ids_opt, > + repl_parallel_mode_opt; > enum { > LEX_GTID_UNCHANGED, LEX_GTID_NO, LEX_GTID_CURRENT_POS, > LEX_GTID_SLAVE_POS > } use_gtid_opt; > @@ -241,10 +248,11 @@ struct LEX_MASTER_INFO > pos= relay_log_pos= server_id= port= connect_retry= 0; > heartbeat_period= 0; > ssl= ssl_verify_server_cert= heartbeat_opt= > - repl_ignore_server_ids_opt= LEX_MI_UNCHANGED; > + repl_ignore_server_ids_opt= repl_parallel_mode_opt= > LEX_MI_UNCHANGED; > gtid_pos_str.length= 0; > gtid_pos_str.str= NULL; > use_gtid_opt= LEX_GTID_UNCHANGED; > + repl_parallel_mode= 0; > } > }; > > > === modified file 'sql/sql_repl.cc' > --- a/sql/sql_repl.cc 2014-08-07 16:06:56 +0000 > +++ b/sql/sql_repl.cc 2014-10-07 14:20:37 +0000 > @@ -3434,6 +3434,9 @@ bool change_master(THD* thd, Master_info > lex_mi->relay_log_name || lex_mi->relay_log_pos) > mi->using_gtid= Master_info::USE_GTID_NO; > > + if (lex_mi->repl_parallel_mode_opt != LEX_MASTER_INFO::LEX_MI_UNCHANGED) > + mi->parallel_mode= lex_mi->repl_parallel_mode; > + > /* > If user did specify neither host nor port nor any log name nor any log > pos, i.e. he specified only user/password/master_connect_retry, he > probably > > === modified file 'sql/sql_yacc.yy' > --- a/sql/sql_yacc.yy 2014-09-03 14:24:31 +0000 > +++ b/sql/sql_yacc.yy 2014-10-07 14:20:37 +0000 > @@ -1130,6 +1130,7 @@ bool my_yyoverflow(short **a, YYSTYPE ** > %token DIV_SYM > %token DOUBLE_SYM /* SQL-2003-R */ > %token DO_SYM > +%token DOMAIN_SYM > %token DROP /* SQL-2003-R */ > %token DUAL_SYM > %token DUMPFILE > @@ -1194,6 +1195,7 @@ bool my_yyoverflow(short **a, YYSTYPE ** > %token GRANT /* SQL-2003-R */ > %token GRANTS > %token GROUP_SYM /* SQL-2003-R */ > +%token GROUPCOMMIT_SYM > %token GROUP_CONCAT_SYM > %token GT_SYM /* OPERATOR */ > %token HANDLER_SYM > @@ -1376,6 +1378,7 @@ bool my_yyoverflow(short **a, YYSTYPE ** > %token PACK_KEYS_SYM > %token PAGE_SYM > %token PAGE_CHECKSUM_SYM > +%token PARALLEL_MODE_SYM > %token PARAM_MARKER > %token PARSER_SYM > %token PARSE_VCOL_EXPR_SYM > @@ -1604,6 +1607,7 @@ bool my_yyoverflow(short **a, YYSTYPE ** > %token VIEW_SYM /* SQL-2003-N */ > %token VIRTUAL_SYM > %token WAIT_SYM > +%token WAITING_SYM > %token WARNINGS > %token WEEK_SYM > %token WEIGHT_STRING_SYM > @@ -2244,6 +2248,14 @@ rule: <-- starts at col 1 > { > Lex->mi.repl_ignore_server_ids_opt= > LEX_MASTER_INFO::LEX_MI_ENABLE; > } > + | PARALLEL_MODE_SYM EQ '(' ')' > + { > + Lex->mi.repl_parallel_mode_opt= > LEX_MASTER_INFO::LEX_MI_DISABLE; > + } > + | PARALLEL_MODE_SYM EQ '(' parallel_mode_list ')' > + { > + Lex->mi.repl_parallel_mode_opt= > LEX_MASTER_INFO::LEX_MI_ENABLE; > + } > | > master_file_def > ; > @@ -2260,6 +2272,52 @@ rule: <-- starts at col 1 > insert_dynamic(&Lex->mi.repl_ignore_server_ids, (uchar*) > &($1)); > } > > +parallel_mode_list: > + parallel_mode_option > + | parallel_mode_list ',' parallel_mode_option > + ; > + > +parallel_mode_option: > + DOMAIN_SYM > + { > + if (Lex->mi.repl_parallel_mode & SLAVE_PARALLEL_DOMAIN) > + { > + my_error(ER_INVALID_SLAVE_PARALLEL_MODE, MYF(0), > "domain"); > + MYSQL_YYABORT; > + } > + Lex->mi.repl_parallel_mode|= SLAVE_PARALLEL_DOMAIN; > + } > + | GROUPCOMMIT_SYM > + { > + if (Lex->mi.repl_parallel_mode & > + (SLAVE_PARALLEL_GROUPCOMMIT|SLAVE_PARALLEL_TRX)) > + { > + my_error(ER_INVALID_SLAVE_PARALLEL_MODE, MYF(0), > "groupcommit"); > + MYSQL_YYABORT; > + } > + Lex->mi.repl_parallel_mode|= SLAVE_PARALLEL_GROUPCOMMIT; > + } > + | TRANSACTIONAL_SYM > + { > + if (Lex->mi.repl_parallel_mode & > + (SLAVE_PARALLEL_GROUPCOMMIT|SLAVE_PARALLEL_TRX)) > + { > + my_error(ER_INVALID_SLAVE_PARALLEL_MODE, MYF(0), > "transactional"); > + MYSQL_YYABORT; > + } > + Lex->mi.repl_parallel_mode|= SLAVE_PARALLEL_TRX; > + } > + | WAITING_SYM > + { > + if (Lex->mi.repl_parallel_mode & SLAVE_PARALLEL_WAITING) > + { > + my_error(ER_INVALID_SLAVE_PARALLEL_MODE, MYF(0), > "waiting"); > + MYSQL_YYABORT; > + } > + Lex->mi.repl_parallel_mode|= SLAVE_PARALLEL_WAITING; > + } > + > + > master_file_def: > MASTER_LOG_FILE_SYM EQ TEXT_STRING_sys > { > @@ -14098,6 +14156,7 @@ user: user_maybe_role > | DISABLE_SYM {} > | DISCARD {} > | DISK_SYM {} > + | DOMAIN_SYM {} > | DUMPFILE {} > | DUPLICATE_SYM {} > | DYNAMIC_SYM {} > @@ -14131,6 +14190,7 @@ user: user_maybe_role > | GET_FORMAT {} > | GRANTS {} > | GLOBAL_SYM {} > + | GROUPCOMMIT_SYM {} > | HASH_SYM {} > | HARD_SYM {} > | HOSTS_SYM {} > @@ -14221,6 +14281,7 @@ user: user_maybe_role > | ONLY_SYM {} > | PACK_KEYS_SYM {} > | PAGE_SYM {} > + | PARALLEL_MODE_SYM {} > | PARTIAL {} > | PARTITIONING_SYM {} > | PARTITIONS_SYM {} > @@ -14336,6 +14397,7 @@ user: user_maybe_role > | VALUE_SYM {} > | WARNINGS {} > | WAIT_SYM {} > + | WAITING_SYM {} > | WEEK_SYM {} > | WEIGHT_STRING_SYM {} > | WORK_SYM {} > > === modified file 'sql/sys_vars.cc' > --- a/sql/sys_vars.cc 2014-09-19 13:25:37 +0000 > +++ b/sql/sys_vars.cc 2014-10-07 14:20:37 +0000 > @@ -1834,31 +1834,11 @@ static Sys_var_ulong Sys_slave_parallel_ > VALID_RANGE(0,2147483647), DEFAULT(131072), BLOCK_SIZE(1)); > > > -static const char *slave_parallel_mode_names[] = { > - "domain", "groupcommit", "transactional", "waiting", NULL > -}; > - > -static Sys_var_set Sys_slave_parallel_mode( > - "slave_parallel_mode", > - "Controls what transactions are applied in parallel when using " > - "--slave-parallel-threads. Syntax: > slave_paralle_mode=value[,value...], " > - "where \"value\" could be one or more of: \"domain\", to apply " > - "different replication domains in parallel; \"groupcommit\", to > apply " > - "in parallel transactions that group-committed together on the > master; " > - "\"transactional\", to optimistically try to apply all > transactional " > - "DML in parallel; and \"waiting\" to extend \"transactional\" to " > - "even transactions that had to wait on the master.", > - GLOBAL_VAR(opt_slave_parallel_mode), CMD_LINE(REQUIRED_ARG), > - slave_parallel_mode_names, > - DEFAULT(SLAVE_PARALLEL_DOMAIN | > - SLAVE_PARALLEL_GROUPCOMMIT)); > - > - > static Sys_var_bit Sys_replicate_allow_parallel( > "replicate_allow_parallel", > "If set when a transaction is written to the binlog, that > transaction " > - "is allowed to replicate in parallel on a slave where " > - "slave_parallel_mode is set to \"transactional\". Can be cleared > for " > + "is allowed to replicate in parallel on a slave where > parallel_mode " > + "is set to \"transactional\" (in CHANGE MASTER). Can be cleared > for " > "transactions that are likely to cause a conflict if replicated in > " > "parallel, to avoid unnecessary rollback and retry.", > SESSION_ONLY(option_bits), NO_CMD_LINE, OPTION_RPL_ALLOW_PARALLEL, > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits >

2 1

[Maria-developers] MDEV-6429 (storage engine API for parallel replication) possibly a requirement for MDEV-6676 (speculative parallel replication)
by Kristian Nielsen 06 Oct '14

06 Oct '14

Hi Serg, Earlier this year we discussed MDEV-6429, which is to implement a proper storage engine API for a feature used in parallel replication. This is where InnoDB reports to the replication layer whenever transaction T1 needs to wait for T2; parallel replication uses this to detect and resolve deadlocks caused by enforced commit order. In 10.0, we decided to not do a "proper" API, instead just modify InnoDB to call into the server layer directly. In 10.0, the need for this API/feature is somewhat InnoDB-specific, it is not known whether any of the other transactional storage engines would need to do anything similar for parallel replication to work. If we implement MDEV-6676, however, then the need for the API becomes more critical. Because MDEV-6676 absolutely only works if we have a mechanism to detect conflicts that cause a deadlock with commit order, as was implemented for InnoDB. If we were to attempt to enable speculative parallel replication for eg. Tokutek transactions, without implementing the necessary check in Tokutek, then the slave would quickly end up hanging. Thinking more on this, I suppose what we really need is what you requested also during the original discussion: A true optional feature for storage engines to implement this additional check. And a way for the server layer to see if a storage engine has the implementation. If it does, then we can try to run its transactions in parallel as per MDEV-6676. If not, we will not try, and things will still work correctly (just without the parallelisation optimisation). Maybe for the first version, the best thing to do is to just hard-code a check for the InnoDB storage engine. This will allow us to get real-world testing to see if the new speculative parallel replication feature is interesting to any users. Then if it is, and there turns out to be interest from other storage engine venders (such as Tokutek) to also support speculative parallel replication, we can work with them to implement a proper API. This approach would also allow to consider the needs of a different storage engine when designing the API, which makes it much more likely that the API will be of general use and not too tied to InnoDB internals. What do you think? - Kristian.

1 0

[Maria-developers] MariaDB Debian packaging: pending merge to upstream
by Otto Kekäläinen 03 Oct '14

03 Oct '14

Hello! Now when the rehearsal is done using simple spelling error fix patches, I now posted the first big merge request about my 5.5/debian upon the upstream 5.5 branch. Issue at https://mariadb.atlassian.net/browse/MDEV-6284 Merge request at https://code.launchpad.net/~otto/maria/maria-fix-debpkg-5.5/+merge/221576 I also have these old merge request still open: https://code.launchpad.net/~otto/maria/maria-fix-debian-copyright-trunk/+me… https://code.launchpad.net/~otto/maria/maria-fix-debian-copyright-5.5/+merg… The sooner somebody has can merge these the less likely it is that there will be merge conflicts.. Thanks! -- Check out our blog at http://seravo.fi/blog and follow @ottokekalainen

2 6

Re: [Maria-developers] Fwd: [Commits] Rev 3897: bzr merge -r4123..4144 codership/5.6 in lp:~maria-captains/maria/maria-10.0-galera
by Philip Stoev 01 Oct '14

01 Oct '14

Hello, Thanks for taking the time to review this. Your concerns are spot-on. == modified file 'mysql-test/mysql-test-run.pl' --- a/mysql-test/mysql-test-run.pl 2014-09-29 00:43:56 +0000 +++ b/mysql-test/mysql-test-run.pl 2014-09-30 22:06:15 +0000 @@ -3189,6 +3189,9 @@ mtr_report(" - adding wsrep, galera to default test suites"); push @DEFAULT_SUITES, qw(wsrep galera); + # ADD scripts to $PATH to that wsrep_sst_* can be found + $ENV{'PATH'} = $ENV{'PATH'}.':'.$basedir.'/ scripts'; > Is this the right place to do this ? Serg already commented about the fact > that we should not "temper" mtr globally, instead only when galera tests > are run, I just wonder because at scripts directory contain other than > wsrep scripts, does this affect existing tests... Yes, you are right, I reverted this change and moved it into suite/galera/galera2nodes.cnf, which now looks like this: [ENV] PATH=/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:@ENV.MYSQL_BINDIR/scripts Unfortunately, in the context of .cnf files, MTR does not allow to simply append to the existing $PATH, so I have to reinitialize it from scratch. === modified file 'mysql-test/suite/galera/ galera_2nodes.cnf' --- a/mysql-test/suite/galera/galera_2nodes.cnf 2014-08-26 20:23:56 +0000 +++ b/mysql-test/suite/galera/galera_2nodes.cnf 2014-09-30 22:06:15 +0000 @@ -2,22 +2,36 @@ !include include/default_mysqld.cnf [mysqld.1] +innodb_file_per_table binlog-format=row +innodb_file_format=Barracuda +performance_schema=OFF + wsrep_provider=(a)ENV.WSREP_PROVIDER wsrep_cluster_address='gcomm://' wsrep_provider_options='base_port=(a)mysqld.1.#galera_port' wsrep_sst_receive_address='127.0.0.1:@mysqld.1.#sst_port' # enforce read-committed characteristics across the cluster wsrep_causal_reads=ON +wsrep_sync_wait = 7 +wsrep_node_address=127.0.0.1 +wsrep_node_incoming_address=127.0.0.1 > Why we need to have innodb_file_per_table=1, innodb_file_format=Barracuda > and performance_schema=OFF ? I have removed those options. performance_schema was OFF because there was an assertion that affected all the tests. Secondly, why wsrep_sync_wait=7 (can't these different numbers have more meaningful values i.e. enum ?) This is required to ensure that all statements see the latest state of the database, allowing for deterministic tests. With respect to using a bitmask vs. an enum, please feel free to open an issue in LaunchPad. Philip Stoev

1 0

Re: [Maria-developers] [Commits] Rev 3897: bzr merge -r4123..4144 codership/5.6 in lp:~maria-captains/maria/maria-10.0-galera
by Sergei Golubchik 01 Oct '14

01 Oct '14

Hi, Jan! On Oct 01, Jan Lindström wrote: > Hi Nirbhay, Philip > > First of all, it is good to see that we have more galera tests. Few > questions: > > == modified file 'mysql-test/mysql-test-run.pl' > --- a/mysql-test/mysql-test-run.pl 2014-09-29 00:43:56 +0000 > +++ b/mysql-test/mysql-test-run.pl 2014-09-30 22:06:15 +0000 > @@ -3189,6 +3189,9 @@ > mtr_report(" - adding wsrep, galera to default test suites"); > push @DEFAULT_SUITES, qw(wsrep galera); > > + # ADD scripts to $PATH to that wsrep_sst_* can be found > + $ENV{'PATH'} = $ENV{'PATH'}.':'.$basedir.'/scripts'; > > Is this the right place to do this ? Serg already commented about the fact > that we should not "temper" mtr globally, instead only when galera tests > are run, I just wonder because at scripts directory contain other than > wsrep scripts, does this affect existing tests... And, just to note, in 10.1 I did it somewhat differently: my ($path) = grep { -f "$_/wsrep_sst_rsync"; } "$::bindir/scripts", $::path_client_bindir; return "No SST scripts" unless $path; $ENV{PATH}="$path:$ENV{PATH}"; Note the differences: - I check whether a script really exists - I look in two locations - same paths where mtr looks for mysqlhotcopy - the path is added to the beginning of the PATH, because we want to find our scripts before any other similar scripts that might happen to be in the PATH because of the existing MariaDB-Galera installation Regards, Sergei

1 0

[Maria-developers] Aria bug with query cache - wrong result and wrong table invalidation
by Roberto Spadim 30 Sep '14

30 Sep '14

Hi guys, could someone take a look at MDEV-6817? i'm testing a stress test with aria and row_format=page the problem is, running many php>while(1){ mysql>select max(column) from table; php> get the value, sum+1 mysql>insert into table (the value of column+1) php>} i got some problems... the value of the first max(column) sometimes return an old value in others words, the query isn't consistent, or the table invalidation code is wrong i got a value 296 at query cache if i change the autocommit from 1 to 0, i got a value of 297, i could use SQL_NO_CACHE and got the same 297 value the problem only occur with row_format=page -- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

1 0

Re: [Maria-developers] Patch: CREATE [[NO] FORCE] VIEW Links
by James Briggs 30 Sep '14

30 Sep '14

Hi folks. Thank you Elena for commenting on MDEV-6709 "Disambiguating "CREATE FORCE VIEW view1 AS SELECT * FROM TABLE1" "As I understand, your patch hasn't been accepted/merged into the main tree yet. Why don't you apply the bugfix in your tree and resubmit the patch?" 1) MDEV-6709 is a related but separate issue from my FORCE patch, applying to MDEV-356 and others. I think it should be a separate patch. 2) What would you like me to do next to get my original patch accepted? 3) Who is the current maintainer for views? I feel that the view features are only 50% implemented in MariaDB and need some design effort. Thanks, James Briggs. -- Cassandra/MySQL DBA. Available in San Jose area or remote. ________________________________ From: James Briggs <actionmessage(a)yahoo.com> To: Maria-developers <maria-developers(a)lists.launchpad.net> Sent: Monday, September 8, 2014 2:52 PM Subject: Patch: CREATE [[NO] FORCE] VIEW Links Hi List. I mailed two patches to the list to implement CREATE [[NO] FORCE] plus tests this weekend. It looks like the mailing list software filters out long posts, so here are links to the list archives: Patch: CREATE [[NO] FORCE] VIEW Options https://lists.launchpad.net/maria-developers/msg07681.html Patch: CREATE VIEW test program https://lists.launchpad.net/maria-developers/msg07682.htm Thanks, James Briggs. -- Available for MySQL/NoSQL DBA/Programming in San Jose area or remote.

1 0

[Maria-developers] [GSoC2014] Self-tuning cost coefficients
by Anshu Avinash 28 Sep '14

28 Sep '14

Hi all, I have written a documentation for framework which I have created for gathering and calculating cost coefficients for optimizer, as part of my GSoC project. It is available at http://igniting.in/gsoc2014/2014/08/20/documentation/. Any suggestions/reviews are welcome. Regards Anshu Avinash

3 3

[Maria-developers] MariaDB 10.0.14 got rid of replication safety warning without warning in release notes?
by Pavel Ivanov 26 Sep '14

26 Sep '14

Hi, I see that in MariaDB 10.0.14 Michael disabled replication safety warning for the case of INSERT ON DUPLICATE KEY UPDATE with several unique keys with a comment "We should assume that the store engine will report the first duplicate key for this case." http://bazaar.launchpad.net/~maria-captains/maria/10.0/revision/4400.1.3 What does that mean? Which key is "the first duplicate key" and why MariaDB relies on an engine behavior which AFAIK is not guaranteed? And why this serious change in behavior wasn't mentioned in the Release Notes? On the related note Release Notes mention the error log flood protection. Why it's not behind flag? How should I disable the feature and get back to the old behavior? Thank you, Pavel

2 2

Re: [Maria-developers] [Commits] 236cc6c: wsrep-related changes: removed some unnecessary files & minor modifications.
by Sergei Golubchik 26 Sep '14

26 Sep '14

Hi, Nirbhay! On Sep 25, nirbhay(a)skysql.com wrote: > revision-id: 236cc6cd49f250a43f0f6a794d5052207b2d9348 > parent(s): a756ac61900e28448776427e6ebbb0ef6f48d5cc > committer: Nirbhay Choubey > branch nick: server > timestamp: 2014-09-25 20:59:15 -0400 > message: > > wsrep-related changes: removed some unnecessary files & minor modifications. Nirbhay, I've ask your few times on irc and email *not* to do that. I already have all these changes in my tree, now I'll need to merge that. It adds no value, only wasted double work and wasted time for useless merges. Please don't do changes that I've already done! Regards, Sergei

1 0

Re: [Maria-developers] More upgrades needed for work-amd64-valgrind
by Sergey Petrunia 25 Sep '14

25 Sep '14

Hi Jani, On Thu, Sep 25, 2014 at 02:22:48PM +0300, Jani Tolonen wrote: > Hi Sergey, > > cmake version was just one minor version too old. I upgraded it and > libstdc++ and according to cmake documentation the error should now be > gone. Please let me know if there are still problems. It seems like not - valgrind builder has has built 10.1 tree and is now running tests (for the first time in ... :). Thank you for your help! BR Sergei -- Sergei Petrunia, Software Developer MariaDB | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] More upgrades needed for work-amd64-valgrind
by Sergey Petrunia 25 Sep '14

25 Sep '14

Hi Jani, Thanks. Please let me know when cmake upgrade is finished. On Wed, Sep 24, 2014 at 04:07:51PM +0300, Jani Tolonen wrote: > Hi Sergei, > > Ok, I will check if I can get cmake to work a bit later tonight. > > BR, > Jani > > On 24 September 2014 14:48, Sergey Petrunia <sergey(a)mariadb.com> wrote: > > > Hello Jani, > > > > Thanks for getting git to work for work-amd64-valgrind host. > > > > I am looking at > > http://buildbot.askmonty.org/buildbot/grid?branch=10.1 > > > > and see that "git update" step succeeds. However, the next step fails: > > > > > > http://buildbot.askmonty.org/buildbot/builders/work-amd64-valgrind/builds/6… > > > > it is probably because of cmake being too old. > > > > Could you please update cmake on work-amd64-valgrind, also? > > > > BR > > Sergei > > -- > > Sergei Petrunia, Software Developer > > MariaDB | Skype: sergefp | Blog: http://s.petrunia.net/blog > > > > > > -- BR Sergei -- Sergei Petrunia, Software Developer MariaDB | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] BUG - MDEV 6781
by Roberto Spadim 25 Sep '14

25 Sep '14

Hi guys, i found a bug at query cache and sent a patch to solve it https://mariadb.atlassian.net/browse/MDEV-6781 https://github.com/rspadim/server/commit/5a5e47bf4141ebc3584ed365173dde87b7… there's a pull request at 10.1 github (that's only a line removing a +1 at database name length) please review the patch that was only found cause i'm using qc_info plugin from mdev-4682 thanks :) -- Roberto Spadim

1 0

[Maria-developers] OQGRAPH/others engine and query cache
by Roberto Spadim 24 Sep '14

24 Sep '14

Hi guys i was checking OQGRAPH limitations with query cache it can't use query cache cause we don't know when data base change, that's ok right? i think that's ok too some others engines have this 'problem' too the first answer is, since oqgraph can't know when data table changes, we can't cache oqgraph tables... i think that's right BUT... query cache 'save' table used in query cache, for example: select * from oq_graph this will include "select * from oq_graph" at query cache using table 'oq_graph' instead of only using 'oq_graph' we could include 'oq_graph' AND 'data_table' even when we 'only' use oq_graph at FROM part this is more interesting than turning query cache off to oq_graph, and this remove the problem of "the data table don't tell oqgraph when we need to invalidate query cache" and that's right! but since data_table changed it will invalidate all queries using it, including oq_graph queries =) check if i'm not wrong, i can try a patch with it if we don't have others query cache related problems (for example query cache don't support tables outside query) -- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

1 0

[Maria-developers] More upgrades needed for work-amd64-valgrind
by Sergey Petrunia 24 Sep '14

24 Sep '14

Hello Jani, Thanks for getting git to work for work-amd64-valgrind host. I am looking at http://buildbot.askmonty.org/buildbot/grid?branch=10.1 and see that "git update" step succeeds. However, the next step fails: http://buildbot.askmonty.org/buildbot/builders/work-amd64-valgrind/builds/6… it is probably because of cmake being too old. Could you please update cmake on work-amd64-valgrind, also? BR Sergei -- Sergei Petrunia, Software Developer MariaDB | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] [Maria-discuss] MariaDB encryption
by Jonas Oreland 23 Sep '14

23 Sep '14

Hi again, > What is the type of license of your code? I asked internally about license, and it seems like we releasing dual gpl2/apache licensed code. > I would like to know, which interfaces from maria-DB you are using. I don't 100% understand the question. We didn't write any actual encryption code, but used the one provided in openssl. Other than that, we didn't really "use interfaces", but rather added/modified functionality/interfaces here and there. Can you be more specific ? /Jonas On Sat, Jun 7, 2014 at 11:20 PM, Elmar Eperiesi-Beck <elmar(a)eperiesi-beck.de > wrote: > Hi! > We (eperi) would be glad to do a joined work with Google. > Our solution works with MS-SQL, Oracle and other DBs and we are currently > porting it to MariaDB - and - as Monty said - its never to late to put some > sources together and make the best for the open source community. > > What is the type of license of your code? > > Jonas, I am looking forward to connect to you directly. > > Regards > Elmar > > Hi! > > > Hi Jonas, > > (same Jonas we know from NDBCLUSTER? :-) Good to see you again) > > > > On 6 Jun 2014, at 02:31, Jonas Oreland <jonaso(a)google.com> wrote: > > > >> Hi there, > >> I read this blog post > >> > http://monty-says.blogspot.com/2014/05/for-your-eyes-only-or-adding-better.… > >> and wanted to inform you that we at Google has developed > on-disk/block-level encryption for Innodb, aria (as used by temporary > tables), binlogs and temp-files. > >> The code is not yet published, but we expect it to be within a few > weeks or so. > >> We (of course?) think that it would be better if you instead of > developing new code > >> spent the time testing/reviewing ours. > > We are out course happy to do this! > > >> I'm happy to answer questions on the topic, > >> and will let you know once we've published it. > > The main question I have about the Innodb encryption is if it based on the > compression code we did for fusion-io? > The idea we had on our side was that by using the new compression hooks we > could add encryption with very little changes to the Innodb code. > Looking forward to when you are ready to publish the code so we can > discuss your changes in detail. > > > This is great news! > > > > From what I gather, from Monty's blog post (and a 1:1 we had some time > back), this is something done by a partner/external company that has a > mostly OSS solution, that we should integrate into 10.1 > > Yes, that's correct. It I would have known that Google was working on > encryption I would have included them in my discussions with eperi. > Fortunately it's not yet too late to do this. > I am sure eperi would like to work on the Google code as a base! > > > That said, Google's release of something that works for InnoDB, Aria, > binlogs, temp files (and presumably not too hard to add for MyISAM) is > something we should definitely review and target for 10.1 > > Yes! > > Regards, > Monty > > >

3 21

[Maria-developers] MDEV-4682
by Roberto Spadim 23 Sep '14

23 Sep '14

Hi guys, could check MDEV-4682? it's done to review thanks -- Roberto Spadim

2 2

[Maria-developers] Switching on use of components in Jira
by Rasmus Johansson 23 Sep '14

23 Sep '14

Hello all, I will later today switch on the usage of Components in Jira. Components are a way of grouping issues inside a project. The idea is to map every issue in Jira, being it a bug or a new feature to a component which describes to what area of MariaDB the issue belongs. For example a bug in Aria would be marked with the component "Storage Engine - Aria". Currently there are 37 components defined in Jira that should cover many areas, but I'm sure there are a few missing. There is one generic to which you can assign issues that doesn't fall into any other category, called OTHER. I'll include the list of components here in the end of this email. * What do you need to do? * The Component field will be a mandatory field when closing an issue. Therefore if a component (or several components) haven't been specified earlier for an issue, you'll have to choose at least one when closing an issue. The field has autocompletion so if you start typing for example "Galera" you'll be presented with the alternatives "Galera", "Galera Arbitrator garbd", and "Galera SST". If you don't find a suitable component then choose the component called "OTHER". * What is the benefit of using components? * It's a way of organizing issues inside a project, so that you easily can pull out all issues related to a certain area of MariaDB, e.g. if I wanted to know all bug fixes for replication I would click on the Replication component and get a list. There is the possibility to set up automatic assignees based on components and components can be used in filters and reports throughout Jira. I hope this won't be a burden. All feedback is of course welcome. BR, Rasmus * Components in the MariaDB Server project in JIRA * Data Definition - Alter Table Data Definition - Procedure Data Definition - Temporary Data Manipulation - Insert Delayed Data Manipulation - Subquery Dynamic Columns Events Full-text Search Galera Galera Arbitrator garbd Galera SST GIS Locale Settings OTHER Platform Debian Platform FreeBSD Platform Power Platform Windows Plugin - Audit Plugins Query Cache Replication Show Profile sql_error_log SSL Storage Engine - Aria Storage Engine - Cassandra Storage Engine - Connect Storage Engine - InnoDB Storage Engine - OQGRAPH Storage Engine - Spider Storage Engine - XtraDB Storage Enging - TokuDB Time zones Triggers Views XML Functions

3 4

[Maria-developers] Close some MDEV
by Roberto Spadim 22 Sep '14

22 Sep '14

Hi guys, could close these mdev: 1)MDEV-4599 (it will be implemented at mdev-4584) 2)MDEV-4581 (it will be implemented at mdev-4682) Please check this mdev-4682 too, i sent a patch via github, there's an option that should be implemented at compile time to turn stats on/off i didn't checked how to implement it yet, if anyone could help me :) it must set a macro #define QUERY_CACHE_QC_INFO_PLUGIN 1 at "configure" time, maybe via cmake gui or something like it maybe i should change QUERY_CACHE_QC_INFO_PLUGIN to QUERY_CACHE_INCLUDE_STATS to easier understand of macro definitions, help is wellcome here -- Roberto Spadim

1 0

Re: [Maria-developers] [Commits] Rev 4376: MDEV-6676: Speculative parallel replication in http://bazaar.launchpad.net/~maria-captains/maria/10.0
by Jonas Oreland 22 Sep '14

22 Sep '14

> > Intermediate commit. Patch is far from complete, but this small patch was > nevertheless sufficient to be able to sysbench-0.4 OLTP with full > parallelisation. > "full parallelisation" does that mean that X threads on master make slave achieve k*X higher throughput ? /Jonas

4 16

[Maria-developers] Need suggestions for benchmarking MDEV-6676 parallel replication
by Kristian Nielsen 22 Sep '14

22 Sep '14

Hi Axel, I'd like to ask your advice on some benchmarking of MDEV-6676. MDEV-6676 is about extending in-order parallel replication to be significantly more aggressive, and potentially get a _lot_ more parallelism on the slave. The basic idea is to just try to run _everything_ in parallel. If there is a conflict, the code will roll back the erroneous transaction and retry it, this time not in parallel. I have run a basic sysbench 0.5 OLTP on my laptop, and results were promising. However, I need something better than that. My main problem is that often when I run write-load benchmarks, the bottleneck is InnoDB log write or purge, or something like that. So the load does not really scale with multiple threads, even on the master, and parallel replication is not really that useful even if perfectly implemented. So do you have a suggestion for a write-load benchmark that scales well for medium concurrency, like 8-32 threads or something like that? And for a machine on which to run it? The best benchmark I think would be one that is I/O bound on a working set that is not in memory, on a machine with a disk subsystem that has good throughput at high concurrency. Because this should resemble the real-world use cases where parallel replication is most needed. Do you have any experience on running such benchmarks, and do we have any machines that could be used? Often the benchmarks I see in blogs are in-memory... Or do you have another suggestion? Thanks, - Kristian.

1 0

[Maria-developers] MDEV-4682 revision
by Roberto Spadim 20 Sep '14

20 Sep '14

hi guys, i sent some patch about qc_info plugin ( https://mariadb.atlassian.net/browse/MDEV-4682) via pull request at github (https://github.com/MariaDB/server/pull/6) could anyone check if it's ok? i included test file changes there's only one thing todo, that's include/not include statistics inside query cache, since it increase memory use, i think could be nice at runtime change this variable and include statistics on/off (more info inside mdev) -- Roberto Spadim

1 0

[Maria-developers] doubt about tests files
by Roberto Spadim 20 Sep '14

20 Sep '14

hi guys! i git cloned the last verison of mariadb from github, but i didn't found test files is it outside mariadb github source? -- Roberto Spadim

1 1

Re: [Maria-developers] [Commits] c8496e4: %B
by Roberto Spadim 19 Sep '14

19 Sep '14

working =P 2014-09-19 18:04 GMT-03:00 Igor Babaev <igor(a)askmonty.org>: > revision-id: c8496e48c4b7c35e9702f98daa19cdb4ef190097 > parent(s): 0486b3b796ad7ec597bff775da248096731c58d5 > committer: Igor Babaev > branch nick: 10.1-test-commit > timestamp: 2014-09-19 13:47:43 -0700 > message: > > %B > --- > sql/opt_range.cc | 2 ++ > 1 files changed, 2 insertions(+), 0 deletions(-) > > diff --git a/sql/opt_range.cc b/sql/opt_range.cc > index 27e20c0..c35bde6 100644 > --- a/sql/opt_range.cc > +++ b/sql/opt_range.cc > @@ -14,6 +14,8 @@ > along with this program; if not, write to the Free Software > Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 > USA */ > // test commit in git > +test 2 > + > /* > TODO: > Fix that MAYBE_KEY are stored in the tree so that we can detect use > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits > -- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

1 0

[Maria-developers] Free Software Foundation address
by Matej Mužila 17 Sep '14

17 Sep '14

Hi, the address of the Free Software Foundation has changed and I think itshould be changed in sources too. Nowdays the correct one is: Free Software Foundation 51 Franklin Street, Fifth Floor Boston, MA 02110-1301 USA http://www.fsf.org/about/contact/ Regards Matej Mužila

1 0

[Maria-developers] master.info and relay-log.info location
by Tim Callaghan 15 Sep '14

15 Sep '14

Has any work been done in MariaDB 10.0 or 10.1 to move them from their file-based location to within XtraDB (and thus potentially TokuDB)? The knowledge base article at https://mariadb.com/kb/en/mariadb/development/general-development-informati… lists the following as a roadmap feature: - Transactional storage of slave state, rather than file-based master.info and relay-log.info . So the slave can recover consistently after a crash.

2 1

[Maria-developers] contribution procedure
by Jonas Oreland 12 Sep '14

12 Sep '14

Hi Serg, hope all is well. I created https://mariadb.atlassian.net/browse/MDEV-6712 for a plugin feature I wrote, and now I humbly ponder on how I get this review/triaged/something. What would be your best advice? /Jonas

2 1

[Maria-developers] How to get all fields in a given index?
by Rongrong 10 Sep '14

10 Sep '14

Hi, I'm trying to implement some feature that requires to get the list of fields for a given index. I'm trying to follow the optimizer code to find some clue but am not successful so far. Can anyone give me some pointers? Thanks! Rongrong

2 2

[Maria-developers] Total breakdown of development procedures :-(
by Kristian Nielsen 10 Sep '14

10 Sep '14

Today, I learned that a large change (the galera merge) has been pushed into 10.1 on August 27. At that point, there was not a _single_ builder in buildbot that was green! As a result, we now have an extremely serious regression in the server caused by this push, easily seen in the testsuite. This is a disaster! Someone (or more likely everyone) have forgotten the basic rules, rules that have been in place since the beginning of MariaDB, and which are essential to ensuring the necessary quality of a product used in production-critical environments every day: - No pushes into a tree with known test failures in Buildbot. - No pushes of larger changes without first seeing them green in Buildbot in a feature tree. - No pushes without review. - No pushes of new features that are not of beta quality. Must be feature complete, must have passed tests and review, must have documentation ready. I have asked Jan to revert the Galera merge until it can be tested green in Buildbot and passed RQG replication tests by Elena. But the problem here is clearly larger than any single patch. Things have degenerated far beyond what is even remotely acceptable. We need to step back and start following the rules to the letter to recover from this. - Kristian.

1 0

Re: [Maria-developers] [Commits] Rev 4385: MDEV-6459 max_relay_log_size and sql_slave_skip_counter misbehave on PPC64 in lp:~maria-captains/maria/10.0
by Sergey Vojtovich 09 Sep '14

09 Sep '14

Hi Sergei, just curious: any special reason to make these variables ulonglong? Both seem to fit into uint. Thanks, Sergey On Tue, Sep 09, 2014 at 04:52:12PM +0200, Sergei Golubchik wrote: > At lp:~maria-captains/maria/10.0 > > ------------------------------------------------------------ > revno: 4385 > revision-id: sergii(a)pisem.net-20140909145212-62txhaq57k301s16 > parent: sergii(a)pisem.net-20140909130235-h2sy95zwu55701l0 > fixes bug: https://mariadb.atlassian.net/browse/MDEV-6459 > committer: Sergei Golubchik <sergii(a)pisem.net> > branch nick: 10.0 > timestamp: Tue 2014-09-09 16:52:12 +0200 > message: > MDEV-6459 max_relay_log_size and sql_slave_skip_counter misbehave on PPC64 > > make slave_skip_counter and max_relay_log_size ulonglong > (sysvars should generally never be ulong) > === modified file 'sql/rpl_rli.h' > --- a/sql/rpl_rli.h 2014-09-08 10:59:57 +0000 > +++ b/sql/rpl_rli.h 2014-09-09 14:52:12 +0000 > @@ -251,10 +251,11 @@ class Relay_log_info : public Slave_repo > errors, and have been manually applied by DBA already. > Must be ulong as it's refered to from set_var.cc > */ > - volatile ulong slave_skip_counter; > + volatile ulonglong slave_skip_counter; > + ulonglong max_relay_log_size; > + > volatile ulong abort_pos_wait; /* Incremented on change master */ > volatile ulong slave_run_id; /* Incremented on slave start */ > - ulong max_relay_log_size; > mysql_mutex_t log_space_lock; > mysql_cond_t log_space_cond; > /* > > === modified file 'sql/sql_class.h' > --- a/sql/sql_class.h 2014-07-31 09:08:56 +0000 > +++ b/sql/sql_class.h 2014-09-09 14:52:12 +0000 > @@ -519,6 +519,14 @@ typedef struct system_variables > ulonglong sortbuff_size; > ulonglong group_concat_max_len; > ulonglong default_regex_flags; > + > + /** > + Place holders to store Multi-source variables in sys_var.cc during > + update and show of variables. > + */ > + ulonglong slave_skip_counter; > + ulonglong max_relay_log_size; > + > ha_rows select_limit; > ha_rows max_join_size; > ha_rows expensive_subquery_limit; > @@ -587,12 +595,6 @@ typedef struct system_variables > */ > uint32 gtid_domain_id; > uint64 gtid_seq_no; > - /** > - Place holders to store Multi-source variables in sys_var.cc during > - update and show of variables. > - */ > - ulong slave_skip_counter; > - ulong max_relay_log_size; > > /** > Default transaction access mode. READ ONLY (true) or READ WRITE (false). > > === modified file 'sql/sys_vars.cc' > --- a/sql/sys_vars.cc 2014-08-07 16:06:56 +0000 > +++ b/sql/sys_vars.cc 2014-09-09 14:52:12 +0000 > @@ -4248,11 +4248,11 @@ static Sys_var_uint Sys_slave_net_timeou > Return 0 + warning if it doesn't exist > */ > > -ulong Sys_var_multi_source_ulong:: > -get_master_info_ulong_value(THD *thd, ptrdiff_t offset) > +ulonglong Sys_var_multi_source_ulonglong:: > +get_master_info_ulonglong_value(THD *thd, ptrdiff_t offset) > { > Master_info *mi; > - ulong res= 0; // Default value > + ulonglong res= 0; // Default value > mysql_mutex_unlock(&LOCK_global_system_variables); > mysql_mutex_lock(&LOCK_active_mi); > mi= master_info_index-> > @@ -4261,7 +4261,7 @@ get_master_info_ulong_value(THD *thd, pt > if (mi) > { > mysql_mutex_lock(&mi->rli.data_lock); > - res= *((ulong*) (((uchar*) mi) + master_info_offset)); > + res= *((ulonglong*) (((uchar*) mi) + master_info_offset)); > mysql_mutex_unlock(&mi->rli.data_lock); > } > mysql_mutex_unlock(&LOCK_active_mi); > @@ -4273,7 +4273,7 @@ get_master_info_ulong_value(THD *thd, pt > bool update_multi_source_variable(sys_var *self_var, THD *thd, > enum_var_type type) > { > - Sys_var_multi_source_ulong *self= (Sys_var_multi_source_ulong*) self_var; > + Sys_var_multi_source_ulonglong *self= (Sys_var_multi_source_ulonglong*) self_var; > bool result= true; > Master_info *mi; > > @@ -4310,16 +4310,12 @@ static bool update_slave_skip_counter(sy > return false; > } > > - > -static Sys_var_multi_source_ulong > -Sys_slave_skip_counter("sql_slave_skip_counter", > - "Skip the next N events from the master log", > - SESSION_VAR(slave_skip_counter), > - NO_CMD_LINE, > - my_offsetof(Master_info, rli.slave_skip_counter), > - VALID_RANGE(0, UINT_MAX), DEFAULT(0), BLOCK_SIZE(1), > - ON_UPDATE(update_slave_skip_counter)); > - > +static Sys_var_multi_source_ulonglong Sys_slave_skip_counter( > + "sql_slave_skip_counter", "Skip the next N events from the master log", > + SESSION_VAR(slave_skip_counter), NO_CMD_LINE, > + MASTER_INFO_VAR(rli.slave_skip_counter), > + VALID_RANGE(0, UINT_MAX), DEFAULT(0), BLOCK_SIZE(1), > + ON_UPDATE(update_slave_skip_counter)); > > static bool update_max_relay_log_size(sys_var *self, THD *thd, Master_info *mi) > { > @@ -4328,17 +4324,14 @@ static bool update_max_relay_log_size(sy > return false; > } > > -static Sys_var_multi_source_ulong > -Sys_max_relay_log_size( "max_relay_log_size", > - "relay log will be rotated automatically when the " > - "size exceeds this value. If 0 are startup, it's " > - "set to max_binlog_size", > - SESSION_VAR(max_relay_log_size), > - CMD_LINE(REQUIRED_ARG), > - my_offsetof(Master_info, rli.max_relay_log_size), > - VALID_RANGE(0, 1024L*1024*1024), DEFAULT(0), > - BLOCK_SIZE(IO_SIZE), > - ON_UPDATE(update_max_relay_log_size)); > +static Sys_var_multi_source_ulonglong Sys_max_relay_log_size( > + "max_relay_log_size", > + "relay log will be rotated automatically when the size exceeds this " > + "value. If 0 are startup, it's set to max_binlog_size", > + SESSION_VAR(max_relay_log_size), CMD_LINE(REQUIRED_ARG), > + MASTER_INFO_VAR(rli.max_relay_log_size), > + VALID_RANGE(0, 1024L*1024*1024), DEFAULT(0), BLOCK_SIZE(IO_SIZE), > + ON_UPDATE(update_max_relay_log_size)); > > static Sys_var_charptr Sys_slave_skip_errors( > "slave_skip_errors", "Tells the slave thread to continue " > > === modified file 'sql/sys_vars.h' > --- a/sql/sys_vars.h 2014-07-18 15:45:21 +0000 > +++ b/sql/sys_vars.h 2014-09-09 14:52:12 +0000 > @@ -1993,7 +1993,8 @@ class Sys_var_replicate_events_marked_fo > like sql_slave_skip_counter are GLOBAL. > */ > > -class Sys_var_multi_source_ulong; > +#define MASTER_INFO_VAR(X) my_offsetof(Master_info, X), sizeof(Master_info::X) > +class Sys_var_multi_source_ulonglong; > class Master_info; > > typedef bool (*on_multi_source_update_function)(sys_var *self, THD *thd, > @@ -2002,31 +2003,27 @@ bool update_multi_source_variable(sys_va > THD *thd, enum_var_type type); > > > -class Sys_var_multi_source_ulong :public Sys_var_ulong > +class Sys_var_multi_source_ulonglong :public Sys_var_ulonglong > { > ptrdiff_t master_info_offset; > on_multi_source_update_function update_multi_source_variable_func; > public: > - Sys_var_multi_source_ulong(const char *name_arg, > + Sys_var_multi_source_ulonglong(const char *name_arg, > const char *comment, int flag_args, > ptrdiff_t off, size_t size, > CMD_LINE getopt, > ptrdiff_t master_info_offset_arg, > - ulong min_val, ulong max_val, ulong def_val, > - uint block_size, > + size_t master_info_arg_size, > + ulonglong min_val, ulonglong max_val, > + ulonglong def_val, uint block_size, > on_multi_source_update_function on_update_func) > - :Sys_var_ulong(name_arg, comment, flag_args, off, size, > - getopt, min_val, max_val, def_val, block_size, > - 0, VARIABLE_NOT_IN_BINLOG, 0, update_multi_source_variable), > + :Sys_var_ulonglong(name_arg, comment, flag_args, off, size, > + getopt, min_val, max_val, def_val, block_size, > + 0, VARIABLE_NOT_IN_BINLOG, 0, update_multi_source_variable), > master_info_offset(master_info_offset_arg), > update_multi_source_variable_func(on_update_func) > { > - } > - bool session_update(THD *thd, set_var *var) > - { > - session_var(thd, ulong)= (ulong) (var->save_result.ulonglong_value); > - /* Value should be moved to multi_master in on_update_func */ > - return false; > + SYSVAR_ASSERT(master_info_arg_size == size); > } > bool global_update(THD *thd, set_var *var) > { > @@ -2039,9 +2036,9 @@ class Sys_var_multi_source_ulong :public > } > uchar *session_value_ptr(THD *thd,LEX_STRING *base) > { > - ulong *tmp, res; > - tmp= (ulong*) (((uchar*)&(thd->variables)) + offset); > - res= get_master_info_ulong_value(thd, master_info_offset); > + ulonglong *tmp, res; > + tmp= (ulonglong*) (((uchar*)&(thd->variables)) + offset); > + res= get_master_info_ulonglong_value(thd, master_info_offset); > *tmp= res; > return (uchar*) tmp; > } > @@ -2049,7 +2046,7 @@ class Sys_var_multi_source_ulong :public > { > return session_value_ptr(thd, base); > } > - ulong get_master_info_ulong_value(THD *thd, ptrdiff_t offset); > + ulonglong get_master_info_ulonglong_value(THD *thd, ptrdiff_t offset); > bool update_variable(THD *thd, Master_info *mi) > { > return update_multi_source_variable_func(this, thd, mi); > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits

2 1

[Maria-developers] Patch: CREATE VIEW test program
by James Briggs 08 Sep '14

08 Sep '14

Hi List. Below is tests/view.pl for testing CREATE VIEW options. It tests several thousand permutations and was used to find MDEV-6709 Thanks, James Briggs. -- Available for MySQL/NoSQL DBA/Programming in San Jose area or remote. tests/view.pl: #!/usr/bin/perl # Program: view.pl # Licence: GPL version 2 # Author: James Briggs # Date: 2014 09 07 # Env: Perl 5 # Note: requires CREATE [[NO] FORCE] option # # Testing of views. # use strict; use warnings; use DBI; use Getopt::Long; $|=1; use vars qw($opt_help $opt_Information $opt_force $opt_debug $opt_verbose $opt_root_user $opt_root_password $opt_user $opt_password $opt_database $opt_host $opt_silent); $opt_help = $opt_Information = $opt_force = $opt_debug = $opt_verbose = $opt_silent = 0; $opt_host = "localhost", $opt_root_user = "root"; $opt_root_password = ""; $opt_user = "view_user"; $opt_password = "view_user"; $opt_database = "view_test"; $opt_force = 1; my $version = "1.0"; my $opt_table="table1"; my $opt_user2 = $opt_user . '2'; GetOptions("Information","help","server=s","root-user=s","root-password=s","user","password=s","database=s","force","host=s","debug","verbose","silent") || usage(); usage() if ($opt_help || $opt_Information); # magic constants use constant N_TEST_RECORDS => 3; use constant HAS_VIEW_GRANTS => 0; use constant NO_VIEW_GRANTS => 1; my @cols = qw[id col1 col2 col3 col4]; my $tmp_table="/tmp/mysql-view.test"; unlink($tmp_table); # %roles array offets use constant U_DBH => 0; use constant U_PRIV_LEVEL => 1; use constant U_USER => 2; use constant U_HOST => 3; use constant U_PASSWORD => 4; # run tests with multiple user accounts from SUPER to very basic privs my %roles = ( # [ U_DBH, U_PRIV_LEVEL, U_USER, U_HOST, U_PASSWORD ] root => [ undef, HAS_VIEW_GRANTS, $opt_root_user, $opt_host, $opt_root_password ], power => [ undef, HAS_VIEW_GRANTS, $opt_user, $opt_host, $opt_password ], crud => [ undef, NO_VIEW_GRANTS, $opt_user2, $opt_host, $opt_password], # use the non-root user account again, this time with less privs ); if (!$opt_force) { print_info(); } # # setup test database # my $dbh = user_connect($opt_root_user,$opt_root_password, 0, 'test'); # $opt_database may not exist yet, so connect with 'test' $roles{'root'}->[U_DBH] = $dbh; test_query('root', "drop database if exists $opt_database"); # drop database to quickly drop any tables and views test_query('root', "create database $opt_database"); test_query('root', "use $opt_database"); # # setup test table # test_query('root', "create table $opt_table (id int primary key auto_increment, col1 int, col2 int, col3 int, col4 int)"); my @c = @cols; # column names for test database shift @c; # remove first column (id) my $cols = join ',', @c; # squash array into a string for my $i (1..N_TEST_RECORDS) { test_query('root', "insert into $opt_table ($cols) values (2, 3, 4, 5)"); } test_query('root', "grant select, insert, update, delete, create, drop, create view, show view on $opt_database.* to '$opt_user'\@'$opt_host' identified by '$opt_password'"); my $dbh_power = user_connect($opt_user, $opt_password, 0); test_query('root', "grant select, insert, update, delete on $opt_database.* to '$opt_user2'\@'$opt_host' identified by '$opt_password'"); my $dbh_crud = user_connect($opt_user2, $opt_password, 0); $roles{'power'}->[0] = $dbh_power; $roles{'crud'}->[0] = $dbh_crud; # # test views # # query array offsets use constant Q_QRY => 0; use constant Q_HI_PRIV => 1; use constant Q_LO_PRIV => 2; use constant Q_OUTPUT => 3; use constant Q_COMMENT => 4; my @t0 = ( # [ query, ignore_failure_hi_priv, ignore_failure_lo_priv, result, comment ] # [ Q_QRY, Q_HI_PRIV, Q_LO_PRIV, Q_OUTPUT, Q_COMMENT ] [ "create view $opt_table as select * from $opt_table", 1, 1, undef, "should fail - duplicate object name" ], [ "create view view1 as select * from ${opt_table}2", 1, 1, undef, "should fail - no base table found" ], [ "create view view1 as select * from $opt_table", 0, 1, undef, "" ], [ "select count(*) from view1", 0, 1, N_TEST_RECORDS, "" ], [ "create definer = current_user() sql security invoker view v1 as select 1", 0, 1, undef, "" ], [ "drop view view1", 0, 1, undef, "" ], [ "drop view v1", 0, 1, undef, "" ], ); test_driver('original create view commands', \%roles, \@t0); # # test new CREATE NO FORCE VIEW view options in 10.1.x # # CREATE FORCE VIEW should work the same as default (omitted) my @t1 = ( [ "create no force view $opt_table as select * from $opt_table", 1, 1, undef, "should fail - duplicate object name" ], [ "create no force view view1 as select * from $opt_table", 0, 1, undef, "" ], [ "create no force view view1 as select * from $opt_table", 1, 1, undef, "should fail - duplicate object name" ], [ "select count(*) from view1", 0, 1, N_TEST_RECORDS, "" ], [ "drop view view1", 0, 1, undef, "" ], ); test_driver('new CREATE NO FORCE VIEW options in 10.1.x', \%roles, \@t1); # # test new CREATE FORCE VIEW options in 10.1.x # # Similar to Oracle Enterprise, behavior of CREATE FORCE VIEW: # # - no base table needs exist at creation time # - thus no table or column access rights need exist # # - however, CREATE VIEW and SHOW VIEW should be enforced my @t2 = ( [ "create force view $opt_table as select id, col1 from $opt_table", 1, 1, undef, "should fail - duplicate object name" ], [ "create force view view1 as select id, col1 from $opt_table", 0, 1, undef, "" ], [ "create force view view1 as select id, col1 from $opt_table", 1, 1, undef, "should fail - duplicate object name" ], [ "create force view view2 as select id, col1 from ${opt_table}2", 0, 1, undef, "" ], # failed on "CREATE ALGORITHM=UNDEFINED DEFINER=`root`@`localhost` SQL SECURITY DEFINER VIEW `view1` AS select `*` AS `*` from `table1`" # note two AS symbols # [ "select count(*) from view1", 0, 1, N_TEST_RECORDS, "" ], [ "drop view view1, view2", 0, 1, undef, "" ], ); test_driver('new CREATE VIEW FORCE view options in 10.1.x', \%roles, \@t2); # test views by automatically generating thousands of permutation of CREATE VIEW gen_permutations(); # # Clean up # unlink($tmp_table); test_query('root', "revoke all privileges, grant option from '$opt_user'\@'%'"); test_query('root', "revoke all privileges, grant option from '$opt_user2'\@'%'"); test_query('root', "drop database if exists $opt_database"); print "end of test\n"; exit 0; # # do permuted tests # # Todo: # # - derived tables # - I_S tables # - views of views # - invalid syntax sub gen_permutations { # CREATE # [OR REPLACE] # [ALGORITHM = {UNDEFINED | MERGE | TEMPTABLE}] # [DEFINER = { user | CURRENT_USER }] # [SQL SECURITY { DEFINER | INVOKER }] # [[NO] FORCE] # VIEW view_name [(column_list)] # AS select_statement # [WITH [CASCADED | LOCAL] CHECK OPTION] use constant P_CREATE => 0; use constant P_ALGORITHM => 1; use constant P_DEFINER => 2; use constant P_SQL_SECURITY => 3; use constant P_FORCE => 4; use constant P_WITH => 5; # insert a blank string to make the option skippable my @p = ( [ 'create', 'create or replace'], [ '', 'undefined', 'merge', 'temptable' ], [ '', 'user', 'current_user'], [ '', 'definer', 'invoker' ], [ '', 'force', 'no force'], [ '', 'cascaded', 'local'], ); my $permutations = 0; my $sql = ''; my $quiet = 1; print "Trying create view permutations\n\n"; for my $role (sort { $b cmp $a } keys %roles) { for my $create (@{$p[P_CREATE]}) { for my $algorithm (@{$p[P_ALGORITHM]}) { for my $definer (@{$p[P_DEFINER]}) { for my $sql_security (@{$p[P_SQL_SECURITY]}) { for my $force (@{$p[P_FORCE]}) { for my $col_list (('', $cols)) { for my $columns (('*', $cols)) { for my $with (@{$p[P_WITH]}) { next if ($columns eq '*' && $force eq 'force'); # ambiguous view definition $sql = $create . ' '; $sql .= "ALGORITHM = $algorithm " if $algorithm ne ''; if ($definer ne '') { if ($definer eq 'user') { $sql .= "DEFINER = '$roles{$role}->[U_USER]'\@'$roles{$role}->[U_HOST]' "; } else { $sql .= "DEFINER = $definer "; } } $sql .= "$force " if $force ne ''; $sql .= "VIEW "; $sql .= "view1 "; $sql .= "($col_list) " if $col_list ne '' and $col_list eq $columns; # if you use $col_list, the number of columns must match $columns $sql .= "AS SELECT $columns FROM $opt_table "; $sql .= "WITH $with CHECK OPTION" if $with ne ''; print "$permutations:$role: $sql\n"; my $will_error = 0; $will_error = 1 if ($role eq 'crud') # no grants || ($algorithm eq 'temptable' and $with ne '') # always an error to CHECK a TEMPTABLE # || ($definer eq 'user' and $role ne 'root') # super priv needed for DEFINER user ; test_query('root', 'drop view if exists view1', 0, $quiet); test_query($role, $sql, $will_error, $quiet); test_query($role, 'select count(*) from view1', $will_error, $quiet); if (!$will_error && $roles{$role}->[U_PRIV_LEVEL] == HAS_VIEW_GRANTS) { if (db_cmp_count($roles{'root'}->[U_DBH], 'select count(*) from view1', N_TEST_RECORDS )) { die "error: wrong row count for '$sql'"; } } $permutations++; } } } } } } } } } print "total permutations = $permutations\n"; return 0; } sub test_driver { my ($heading, $r_roles, $r_qry) = @_; print $heading, "\n\n"; for my $role (sort { $b cmp $a } keys %$r_roles) { test_query('root', "drop view if exists view1"); for my $q (@$r_qry) { my $flag_fail = ($r_roles->{$role}->[U_PRIV_LEVEL] == HAS_VIEW_GRANTS) ? $q->[Q_HI_PRIV] : $q->[Q_LO_PRIV]; print "$role '$q->[Q_QRY]'", ($flag_fail ? ' should fail' : ' should pass'), "\n"; test_query($role, $q->[Q_QRY], $flag_fail, 0); if (defined $q->[Q_OUTPUT]) { my $ret = db_cmp_count($r_roles->{'root'}->[U_DBH], $q->[Q_QRY], ($flag_fail ? undef : $q->[Q_OUTPUT])); die if $ret; } } } print "\n\n"; } sub usage { print <<EOF; $0 Ver $version This program tests that the VIEW commands works by creating a temporary database ($opt_database) and users ($opt_user, $opt_user2). Options: --database (Default $opt_database) In which database the test tables are created. --force Don''t ask any question before starting this test. --host='host name' (Default $opt_host) Host name where the database server is located. --Information --help Print this help --root-password Password for root-user. --user (Default $opt_user) A non-existing user on which we will test view commands --password Password for non-root-user. --verbose Write all queries when we are execute them. --root-user='user name' (Default $opt_root_user) superuser for creating tables and grants EOF exit(0); } sub print_info { my $tmp; print <<EOF; This test will do view statements against the $opt_database database ! the $opt_database database and $opt_user user will be created and deleted ! EOF while (1) { print "Start test (yes/no) ? "; $tmp=<STDIN>; chomp($tmp); $tmp=lc($tmp); last if ($tmp =~ /^yes$/i); exit 1 if ($tmp =~ /^n/i); print "\n"; } } sub user_connect { my ($user, $password, $ignore_error, $db) = @_; $db = $opt_database if not defined $db or $db eq ''; print "Connecting $user\n" if ($opt_verbose); my $dbh =DBI->connect("DBI:mysql:$db:$opt_host",$user, $password, { PrintError => 0}); if (!$dbh) { if ($opt_verbose || !$ignore_error) { print "error on connect: $DBI::errstr\n"; } if (!$ignore_error) { die "The above should not have failed!"; } } elsif ($ignore_error) { die "Connect succeeded when it shouldn't have !\n"; } else { return $dbh; } } sub test_query { my ($role, $query, $ignore_error, $quiet) = @_; my ($package, $filename, $line) = caller; my $dbh = $roles{$role}->[U_DBH]; if (defined $dbh && !$dbh->ping) { $roles{$role}->[U_DBH] = user_connect($roles{$role}->[U_USER], $roles{$role}->[U_PASSWORD], $ignore_error); $dbh = $roles{$role}->[U_DBH]; } elsif (not defined $dbh) { $roles{$role}->[U_DBH] = user_connect($roles{$role}->[U_USER], $roles{$role}->[U_PASSWORD], $ignore_error); $dbh = $roles{$role}->[U_DBH]; } if (do_query($dbh, $query, $ignore_error, $quiet)) { if (!defined($ignore_error)) { print "error:$line: This query should not have failed: '$query', do SHOW CREATE VIEW VIEW_TEST.VIEW1 to troubleshoot.\n"; exit 1; } } elsif (defined($ignore_error) && $ignore_error == 1) { print "error:$line: This query should not have succeeded: '$query'\n"; exit 1; } } sub do_query { my ($my_dbh, $query, $ignore_error, $quiet) = @_; my ($sth, $row, $fatal_error); print "$query\n" if ($opt_debug || $opt_verbose); if (!($sth= $my_dbh->prepare($query))) { print "error in prepare: $DBI::errstr\n"; return 1; } if (!$sth->execute) { $fatal_error= ($DBI::errstr =~ /parse error/); if (!$ignore_error || ($opt_verbose && $ignore_error != 3) || $fatal_error) { print "error in execute: $DBI::errstr\n"; { my $cmd = "show create view $opt_database.view1"; my $out = `mysql -h $opt_host -u root -p$opt_root_password -e '$cmd'`; print "$cmd: $out\n"; print '"*** You have the SQL parser bug: AS select `*` AS `*` ***' . "\n" if $out =~ / AS SELECT .* AS /i; } } die if ($fatal_error); $sth->finish; return 1; } if (!$opt_silent and !$quiet) { my $found = 0; while (($row=$sth->fetchrow_arrayref)) { $found = 1; my $tab = ''; for my $col (@$row) { print $tab; print defined($col) ? $col : "NULL"; $tab="\t"; } print "\n"; } print "\n" if $found; } $sth->finish; return 0; } # Note: cmp_tmp_table is not currently used, but available for future use again sub cmp_tmp_table { my ($s) = @_; if (not defined $s) { if (-e $tmp_table) { return 1; } else { return 0; } } $s =~ s/\n+$//g; # remove trailing blanks open X, "<", $tmp_table or return 2; local($/)=''; my $t = <X>; close X; unlink($tmp_table) or warn "debug: cannot unlink tmp table"; $t =~ s/\n+$//g; if ($s ne $t) { return 1; } return 0; } sub db_cmp_count { my ($dbh, $q, $s) = @_; my $sth = $dbh->prepare($q); my $ret = $sth->execute() || do { return 0 if not defined $s; return 2; }; my $out = ''; if (my (@row) = $sth->fetchrow_array()) { $out = $row[0]; } $sth->finish; return 1 if $out != $s; return 0; } # The End.

1 0

[Maria-developers] mariadb native client -- how to call RPMs?
by Honza Horak 05 Sep '14

05 Sep '14

Hi, in Fedora, we're about to package mariadb-native-client, which we'd like to utilize in Fedora as much as possible since using this piece of software we can break the dependency on whole mariadb server package and thus make the base system more compact. However, since mariadb.org does not provide RPMs yet, I'm wondering what would be the best name for such RPM package. The tar ball is called mariadb_client.tar.gz but the project is called mariadb-native-client on launchpad, which I like more btw. because it differs more from mariadb-client package (which is actually a subpackage of mariadb). So, is mariadb-native-client a good choice or is there a better one? Cheers, Honza

3 3

[Maria-developers] no information about references - mysql bug 73757
by Roberto Spadim 04 Sep '14

04 Sep '14

Hi guys, there's a bug report about no precise information to solve a problem: http://bugs.mysql.com/bug.php?id=73757 test case: create table t11 (f1 integer primary key) engine innodb; alter table t11 add constraint c1 foreign key (f1) references t1(f1); /* Erro SQL (1005): Can't create table `teste`.`#sql-3f90_25c30` (errno: 150 "Foreign key constraint is incorrectly formed") */ should be better explain that reference table t1 don't exists or something like it should i report at jira, or just mysql bug database is ok? -- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

2 2

[Maria-developers] Updating MySQL Client Protocol: Possible and still be backward compatible?
by Adam Scott 04 Sep '14

04 Sep '14

Is it possible to have mysql client protocol return the OS username and allow the protocol to backward compatible? Looking to see if reasonable to provide same functionality as Oracle does with select sys_context( 'userenv', 'os_user' ) from dual;

2 1

[Maria-developers] doubt about mdev 4682 - tests
by Roberto Spadim 04 Sep '14

04 Sep '14

Guys, i'm with time and was looking an old patch about query cache information schema https://mariadb.atlassian.net/browse/MDEV-4682 it add an statistic about query cache hits my doubt is... i was trying to understand how could i create tests, since the result of statistic isn't deterministic how should i execute a test case like this? columns that i have doubt: columns ... QUERY_ROWS <- deterministic SELECT_ROWS_READ <- deterministic QUERY_HITS <- deterministic QUERY_HITS_PERIOD_LOW <- non deterministic QUERY_HITS_PERIOD_HIGH <- non deterministic QUERY_HITS_PERIOD_OUTLIERS <- non deterministic QUERY_HITS_TOTAL_TIME_US <- non deterministic QUERY_HITS_MEAN_PERIOD_US <- non deterministic QUERY_HITS_MEAN_PERIOD_LOW_US <- non deterministic QUERY_HITS_MEAN_PERIOD_HIGH_US <- non deterministic QUERY_INSERT_TIME <- probably deterministic (or not?) QUERY_LAST_HIT_TIME <- probably deterministic (or not?) SELECT_EXPEND_TIME_US <- non deterministic SELECT_LOCK_TIME_US <- non deterministic ... -- Roberto Spadim

2 1

[Maria-developers] Query in size
by Kavneet Kaur 04 Sep '14

04 Sep '14

Hi all, I just when through a revision~maria-captains/mariadb-java-client/trunk : /src/test/java/org/mariadb/jdbc/BigQueryTest.java (revision 516)Big Query activated in this code contains a character array of size 20000000. This is something that can be modified. All big queries possible cannot occupy this space. This is sort of leading to more of wastage of memory. Can anyone help me understand why this size has been selected when the same could be done at a size (50-90)% lesser. Regards Kavneet Kaur

2 1

Re: [Maria-developers] Interest in contributing to MariaDB
by Djimeli Konrad 03 Sep '14

03 Sep '14

Thanks for the reply Roberto, i was beginning to get confused because the code i got from github was on c/c++. On 3 September 2014 21:13, Djimeli Konrad <djkonro35(a)gmail.com> wrote: > Thanks for the reply Roberto, i was beginning to get confused because the > code i got from github was on c/c++. > > > On 3 September 2014 20:39, Roberto Spadim <roberto(a)spadim.com.br> wrote: > >> ???? java? no guy, mariadb database source code is C/C++ check : >> >> github : https://github.com/MariaDB/server >> and issue tracker: https://mariadb.atlassian.net >> there's mariadb knowledge: https://mariadb.com/kb/en/ >> there're some book about mariadb/mysql internall and mysql internall >> documentation: http://dev.mysql.com/doc/internals/en/ >> >> java is a connect, you can check more at mariadb page: >> https://mariadb.com/products/connectors-plugins >> >> >> 2014-09-03 17:15 GMT-03:00 Kavneet Kaur <kavneetk(a)outlook.com>: >> >> Hi Djimeli, >>> >>> As far as I have studied about the code it's in java. If you wish to >>> begin it from basic of database, you can go for installation. You should go >>> through documentations also. Explore your interest and surf as much as you >>> can. >>> >>> All the best! >>> >>> > Date: Wed, 3 Sep 2014 15:35:24 +0000 >>> > From: djkonro35(a)gmail.com >>> > To: maria-developers(a)lists.launchpad.net >>> > Subject: [Maria-developers] Interest in contributing to MariaDB >>> >>> > >>> > Hello >>> > >>> > My name is Djimeli Konrad and i am primarily a c/c++ >>> > programmer with some experience in php and mysql. I am new to open >>> > source and would like to contribute to mariadb so i wish i could get >>> > some pointers on how to begin. >>> > >>> > Thanks >>> > konrad >>> > >>> > _______________________________________________ >>> > Mailing list: https://launchpad.net/~maria-developers >>> > Post to : maria-developers(a)lists.launchpad.net >>> > Unsubscribe : https://launchpad.net/~maria-developers >>> > More help : https://help.launchpad.net/ListHelp >>> >>> _______________________________________________ >>> Mailing list: https://launchpad.net/~maria-developers >>> Post to : maria-developers(a)lists.launchpad.net >>> Unsubscribe : https://launchpad.net/~maria-developers >>> More help : https://help.launchpad.net/ListHelp >>> >>> >> >> >> -- >> Roberto Spadim >> SPAEmpresarial >> Eng. Automação e Controle >> > >

1 0

[Maria-developers] Interest in contributing to MariaDB
by Djimeli Konrad 03 Sep '14

03 Sep '14

Hello My name is Djimeli Konrad and i am primarily a c/c++ programmer with some experience in php and mysql. I am new to open source and would like to contribute to mariadb so i wish i could get some pointers on how to begin. Thanks konrad

3 2

[Maria-developers] New MariaDB builds: regular trusty and special kvm-deb-sid-pbuilder-amd and (Re: MariaDB)
by Otto Kekäläinen 01 Sep '14

01 Sep '14

Hello! 2014-02-04 20:41 GMT+02:00 Daniel Bartholomew <dbart(a)mariadb.com>: > On Tue, Feb 4, 2014 at 5:39 AM, Kristian Nielsen > <knielsen(a)knielsen-hq.org> wrote: >>> And while you are at it, please also consider setting up an instance >>> 'kvm-deb-sid-pbuilder-amd64' that provides a Debian unstable >> >> Daniel, can you prepare new sets of VMs for Ubuntu trusty and Debian sid? >> Then I can try to setup Buildbot to do these builds. > > Ok, will do. I should have them up and running before the end of the week. Any updates on this?

3 22

Re: [Maria-developers] [Commits] eecac19: MDEV-6513 deprecate engine_condition_pushdown value of the @@optimizer_switch
by Oleksandr Byelkin 28 Aug '14

28 Aug '14

Hi! Everything is OK. Except one small note in two places. But it is OK to push as is. 26.08.2014 17:17, serg(a)mariadb.org пишет: > revision-id: eecac1976469d513062a0741005afc357b897871 > parent(s): f1c1c04a34f39bcd6622b9886fb089dd41e51103 > committer: Sergei Golubchik > branch nick: maria > timestamp: 2014-08-25 23:13:37 +0200 > message: > > MDEV-6513 deprecate engine_condition_pushdown value of the @@optimizer_switch > > * ignore the OPTIMIZER_SWITCH_ENGINE_CONDITION_PUSHDOWN bit > * issue a deprecation warning on 'engine_condition_pushdown=on' > * remove unused remains of the old pre-5.5 engine_condition_pushdown variable > > --- [skip] > --- a/sql/handler.h > +++ b/sql/handler.h > @@ -197,17 +197,11 @@ enum enum_alter_inplace_result { > #define HA_RECORD_MUST_BE_CLEAN_ON_WRITE (1ULL << 41) > > /* > - Table condition pushdown must be performed regardless of > - 'engine_condition_pushdown' setting. > - > - This flag is aimed at storage engines that come with "special" predicates > - that can only be evaluated inside the storage engine. > - For example, when one does > - select * from sphinx_table where query='{fulltext_query}' > - then the "query=..." condition must be always pushed down into storage > - engine. > + This storage engine supports condition pushdown > */ > -#define HA_MUST_USE_TABLE_CONDITION_PUSHDOWN (1ULL << 42) > +#define HA_CAN_TABLE_CONDITION_PUSHDOWN (1ULL << 42) > +/* old name for the same flag */ > +#define HA_MUST_USE_TABLE_CONDITION_PUSHDOWN HA_CAN_TABLE_CONDITION_PUSHDOWN > > /** > The handler supports read before write removal optimization > diff --git a/sql/mysqld.cc b/sql/mysqld.cc > index 57d2cc6..fe52a85 100644 > --- a/sql/mysqld.cc > +++ b/sql/mysqld.cc > @@ -8574,18 +8574,6 @@ mysqld_get_one_option(int optid, > } > break; > #endif /* defined(ENABLED_DEBUG_SYNC) */ > - case OPT_ENGINE_CONDITION_PUSHDOWN: > - /* > - The last of --engine-condition-pushdown and --optimizer_switch on > - command line wins (see get_options(). > - */ > - if (global_system_variables.engine_condition_pushdown) > - global_system_variables.optimizer_switch|= > - OPTIMIZER_SWITCH_ENGINE_CONDITION_PUSHDOWN; > - else > - global_system_variables.optimizer_switch&= > - ~OPTIMIZER_SWITCH_ENGINE_CONDITION_PUSHDOWN; > - break; > case OPT_LOG_ERROR: > /* > "No --log-error" == "write errors to stderr", > @@ -8964,10 +8952,6 @@ static int get_options(int *argc_ptr, char ***argv_ptr) > &extra_connection_count); > #endif > > - global_system_variables.engine_condition_pushdown= > - MY_TEST(global_system_variables.optimizer_switch & > - OPTIMIZER_SWITCH_ENGINE_CONDITION_PUSHDOWN); > - > opt_readonly= read_only; > > /* > diff --git a/sql/mysqld.h b/sql/mysqld.h > index 21c020b..7a0a75c 100644 > --- a/sql/mysqld.h > +++ b/sql/mysqld.h > @@ -554,7 +554,6 @@ enum options_mysqld > OPT_DEBUG_SYNC_TIMEOUT, > OPT_DELAY_KEY_WRITE_ALL, > OPT_DEPRECATED_OPTION, > - OPT_ENGINE_CONDITION_PUSHDOWN, > OPT_IGNORE_DB_DIRECTORY, > OPT_ISAM_LOG, > OPT_KEY_BUFFER_SIZE, > diff --git a/sql/records.cc b/sql/records.cc > index 1b230c4..cbcdb36 100644 > --- a/sql/records.cc > +++ b/sql/records.cc > @@ -287,8 +287,8 @@ bool init_read_record(READ_RECORD *info,THD *thd, TABLE *table, > thd->variables.read_buff_size); > } > /* Condition pushdown to storage engine */ > - if ((thd->variables.optimizer_switch & > - OPTIMIZER_SWITCH_ENGINE_CONDITION_PUSHDOWN) && > + if ((table->file->ha_table_flags() & > + HA_MUST_USE_TABLE_CONDITION_PUSHDOWN) && > select && select->cond && > (select->cond->used_tables() & table->map) && > !table->file->pushed_cond) > diff --git a/sql/sql_class.h b/sql/sql_class.h > index f96110e..c42a87a 100644 > --- a/sql/sql_class.h > +++ b/sql/sql_class.h > @@ -597,7 +597,6 @@ typedef struct system_variables > my_bool tx_read_only; > my_bool low_priority_updates; > my_bool query_cache_wlock_invalidate; > - my_bool engine_condition_pushdown; > my_bool keep_files_on_create; > > my_bool old_mode; > diff --git a/sql/sql_priv.h b/sql/sql_priv.h > index 5dc1918..9578c2f 100644 > --- a/sql/sql_priv.h > +++ b/sql/sql_priv.h > @@ -199,7 +199,7 @@ template <class T> bool valid_buffer_range(T jump, > #define OPTIMIZER_SWITCH_INDEX_MERGE_SORT_UNION (1ULL << 2) > #define OPTIMIZER_SWITCH_INDEX_MERGE_INTERSECT (1ULL << 3) > #define OPTIMIZER_SWITCH_INDEX_MERGE_SORT_INTERSECT (1ULL << 4) > -#define OPTIMIZER_SWITCH_ENGINE_CONDITION_PUSHDOWN (1ULL << 5) > +#define deprecated_ENGINE_CONDITION_PUSHDOWN (1ULL << 5) > #define OPTIMIZER_SWITCH_INDEX_COND_PUSHDOWN (1ULL << 6) > #define OPTIMIZER_SWITCH_DERIVED_MERGE (1ULL << 7) > #define OPTIMIZER_SWITCH_DERIVED_WITH_KEYS (1ULL << 8) > diff --git a/sql/sql_select.cc b/sql/sql_select.cc > index 05c88a5..f061b51 100644 > --- a/sql/sql_select.cc > +++ b/sql/sql_select.cc > @@ -9561,10 +9561,8 @@ make_join_select(JOIN *join,SQL_SELECT *select,COND *cond) > if (tab->table) > { > tab->table->file->pushed_cond= NULL; > - if (((thd->variables.optimizer_switch & > - OPTIMIZER_SWITCH_ENGINE_CONDITION_PUSHDOWN) || > - (tab->table->file->ha_table_flags() & > - HA_MUST_USE_TABLE_CONDITION_PUSHDOWN)) && > + if ((tab->table->file->ha_table_flags() & > + HA_MUST_USE_TABLE_CONDITION_PUSHDOWN) && It might be that it is better to use 'new' name for the flag in the lines we have changed. > !first_inner_tab) > { > COND *push_cond= > @@ -23598,10 +23596,8 @@ void JOIN_TAB::save_explain_data(Explain_table_access *eta, table_map prefix_tab > { > const COND *pushed_cond= tab->table->file->pushed_cond; > > - if (((thd->variables.optimizer_switch & > - OPTIMIZER_SWITCH_ENGINE_CONDITION_PUSHDOWN) || > - (tab->table->file->ha_table_flags() & > - HA_MUST_USE_TABLE_CONDITION_PUSHDOWN)) && > + if ((tab->table->file->ha_table_flags() & > + HA_MUST_USE_TABLE_CONDITION_PUSHDOWN) && The same. > pushed_cond) > { > eta->push_extra(ET_USING_WHERE_WITH_PUSHED_CONDITION); > diff --git a/sql/sys_vars.cc b/sql/sys_vars.cc > index 7f7c725..db764d7 100644 > --- a/sql/sys_vars.cc > +++ b/sql/sys_vars.cc > @@ -2181,13 +2181,15 @@ export const char *optimizer_switch_names[]= > "exists_to_in", > "default", NullS > }; > -/** propagates changes to @@engine_condition_pushdown */ > static bool fix_optimizer_switch(sys_var *self, THD *thd, > enum_var_type type) > { > SV *sv= (type == OPT_GLOBAL) ? &global_system_variables : &thd->variables; > - sv->engine_condition_pushdown= > - MY_TEST(sv->optimizer_switch & OPTIMIZER_SWITCH_ENGINE_CONDITION_PUSHDOWN); > + if (sv->optimizer_switch & deprecated_ENGINE_CONDITION_PUSHDOWN) > + push_warning_printf(current_thd, Sql_condition::WARN_LEVEL_WARN, > + ER_WARN_DEPRECATED_SYNTAX_NO_REPLACEMENT, > + ER(ER_WARN_DEPRECATED_SYNTAX_NO_REPLACEMENT), > + "engine_condition_pushdown=on"); > return false; > } > static Sys_var_flagset Sys_optimizer_switch( > diff --git a/storage/spider/mysql-test/spider/bg/r/spider_fixes.result b/storage/spider/mysql-test/spider/bg/r/spider_fixes.result > index 6db61fa..3033586 100644 > [skip]

2 1

Re: [Maria-developers] Learning about MariaDB, Getting Started
by Kavneet Kaur 26 Aug '14

26 Aug '14

Hi, James, I am aiming to become am interested in data security and reducing query time by increasing efficiency and computational speed. Coding and optimizing codes are also cup of my tea. How can I contribute this in MariaDB as it's the only open source database software that attracted me. Regards,Kavneet Kaur Date: Fri, 22 Aug 2014 04:44:45 -0700 From: actionmessage(a)yahoo.com Subject: Re: [Maria-developers] Learning about MariaDB, Getting Started To: kavneetk(a)outlook.com; maria-developers(a)lists.launchpad.net; ashish1294(a)gmail.com Hi Kavneet and Ashish. Depends on what you're interested in working on, but here's some suggestions.Direct link to currently open worklogs:https://mariadb.atlassian.net/browse/MDEV-6629?filter=-4Three easy MDEVs and some other areas to work on:- MDEV-6584 - not using tcp_fastopen- MDEV-6555 - sql_log_bin should be SESSION_ONLY- MDEV-5696 - Graceful MariaDB shutdown at Windows restart- test and see if fixes are needed for SSL, Galera, Cassandra, GTID, IPv6 or partition features. "Howto Add a New Command to the MySQL Server" Tutorial http://www.jebriggs.com/blog/2013/05/howto-add-a-new-command-to-the-mysql-s…, James. From: Kavneet Kaur <kavneetk(a)outlook.com> To: "maria-developers(a)lists.launchpad.net" <maria-developers(a)lists.launchpad.net> Sent: Thursday, August 21, 2014 11:49 PM Subject: [Maria-developers] Learning about MariaDB Hi! I am highly interested in subject of database. I am also interested in becoming a research developer. Worked on MS SQL. Want to try MariaDB. May I get the guidance about this product as i'm totally new to it. Is there any specific way to begin? _______________________________________________Mailing list: https://launchpad.net/~maria-developersPost to : maria-developers(a)lists.launchpad.netUnsubscribe : https://launchpad.net/~maria-developersMore help : https://help.launchpad.net/ListHelp

3 6

[Maria-developers] Please review: 10% performance improvement for INSERT INTO t1 (blob) VALUES ('long value')
by Alexander Barkov 26 Aug '14

26 Aug '14

Hi, I think I found and fixed a bottleneck in how string literals are processed. The attached patch improves INSERT performance for strings, and especially for long blob and text values. Insertion time for 10MByte blob 1024 times (10GByte of data total) reduced on my laptop from 300 seconds to 274 seconds, which gives near 10% pure time improvement. What the patch changes: Before the patch : 1. The function get_text() is sql_lex.cc allocated an unescaped copy of every string literal (unescaping backslashes and double quotes, if any). Strangely, copying happened even if there were no really any escapes in the string. 2, The syntax parser in sql_yacc.yy created Item_string using the new unescaped buffer. Furthermore, in case of a multi-byte connection character set, the constructor for Item_string made another loop on the unescaped buffer, to calculate length in characters, which is needed to set max_length properly. After the patch: Item is now created using directly the SQL fragment, without making a copy. Even for escaped values!!! Length in characters is also calculated during the very first pass in get_text(), without any additional loops in the Item constructors. Unescaping is now done in the very end, when the value is actually needed: - Either in Field::store(). This happens when the string value is used in: INSERT INTO t1 VALUES('string'); Unescaping is done directly to the Field buffer, without any intermediary temporary storage. - Or in val_str(), if the string value is used elsewhere (in SELECT list, functions, operators, etc). The unescaped value is cached, to make sure that unescaping is done only one time, even if the value is needed for multiple rows, like in: SELECT * FROM t1 WHERE a='string with backslash or quote escapes'; The patch also fixes a few bugs related to string conversion/copying (some of them were reported on JIRA earlier, and some of them I found while working on the patch). Please also find "a.c" attached. I used it to test the improvement. I used a RelWithDebInfo compiled binary for tests. Note, "perf" and "valgrind --tool=cachegrind" also demonstrate a good improvement (reduced CPU and memory usage). I can send their statistics in a separate letter, if anyone is interested. Please review the patch. Thanks.

1 0

[Maria-developers] List affecting CVEs at mariadb.com
by Otto Kekäläinen 26 Aug '14

26 Aug '14

Hello Daniel (and others), The usual changelogs[1] and relese notes[2] don't seem to contain CVE identifiers, or even a separate section about fixed security issues For the downstream security teams if would be reassuring if the CVE information would be easily available. For example if the security teams follow the CVE news and they for example know or suspect that CVE-2014-4260 affects MariaDB, it would be nice to see if it is already fixed or what version it was fixed in, so downstream security teams can organize and prioritize their patching and release work. Do you have any suggestion how to address this? Should we maybe have a separate wiki page, e.g. https://mariadb.com/kb/en/mariadb/cve/ that would have a table of CVEs and MariaDB 5.5/10.0/Galera versions where they are fixed? Or should just each release notes include a subsection "Security" with these details? Something else? Of course we need to consider timing issues, e.g. a security issue fixed in MariaDB might get publicity and a CVE only later when Oracle releases it, and in those cases old release notes need to be upgraded to include the CVE identifiers. [1] https://mariadb.com/kb/en/mariadb-10013-changelog/ [2] https://mariadb.com/kb/en/mariadb-10013-release-notes/ (To be exact, googling for 'mariadb cve' does give one hit at mariadb.com in the 5.3.12 release notes)

3 3

Re: [Maria-developers] [Commits] Rev 4271: MDEV-6601 Assertion `!thd->in_active_multi_stmt_transa ction() || thd->in_multi_stmt_transaction_mode()' failed on executing a stored procedure with commit in lp:~maria-captains/maria/5.5
by Alexey Botchkov 25 Aug '14

25 Aug '14

Hi, Sergei. I'd add SERVER_STATUS_LAST_ROW_SENT to the status_backup_mask. It's not necessary as I don't see it affecting anything, but just to make a sign that we don't want this status to go higher out of the call. Best regards. HF 25.08.2014 10:49, Sergei Golubchik wrote: > At lp:~maria-captains/maria/5.5 > > ------------------------------------------------------------ > revno: 4271 > revision-id: sergii(a)pisem.net-20140825064904-h8isjhv963kbxlzy > parent: knielsen(a)knielsen-hq.org-20140813134639-wk760plnzg5wu4x8 > fixes bug: https://mariadb.atlassian.net/browse/MDEV-6601 > committer: Sergei Golubchik <sergii(a)pisem.net> > branch nick: 5.5 > timestamp: Mon 2014-08-25 08:49:04 +0200 > message: > MDEV-6601 Assertion `!thd->in_active_multi_stmt_transa ction() || thd->in_multi_stmt_transaction_mode()' failed on executing a stored procedure with commit > > Don't restore the whole of thd->server_status after a routine invocation, > only restore SERVER_STATUS_CURSOR_EXISTS, as --ps --embedded needs. > In particular, don't restore SERVER_STATUS_IN_TRANS. > === modified file 'mysql-test/r/sp-bugs.result' > --- a/mysql-test/r/sp-bugs.result 2014-01-24 12:50:18 +0000 > +++ b/mysql-test/r/sp-bugs.result 2014-08-25 06:49:04 +0000 > @@ -268,3 +268,9 @@ END $$ > CALL test_5531(1); > DROP PROCEDURE test_5531; > DROP TABLE t1; > +create procedure sp() begin > +commit; > +end| > +start transaction; > +call sp(); > +drop procedure sp; > > === modified file 'mysql-test/t/sp-bugs.test' > --- a/mysql-test/t/sp-bugs.test 2014-01-24 12:50:18 +0000 > +++ b/mysql-test/t/sp-bugs.test 2014-08-25 06:49:04 +0000 > @@ -285,3 +285,16 @@ DELIMITER ;$$ > CALL test_5531(1); > DROP PROCEDURE test_5531; > DROP TABLE t1; > + > +# > +# MDEV-6601 Assertion `!thd->in_active_multi_stmt_transa ction() || thd->in_multi_stmt_transaction_mode()' failed on executing a stored procedure with commit > +# > +delimiter |; > +create procedure sp() begin > + commit; > +end| > +delimiter ;| > +start transaction; > +call sp(); > +drop procedure sp; > + > > === modified file 'sql/sp_head.cc' > --- a/sql/sp_head.cc 2014-04-15 13:17:47 +0000 > +++ b/sql/sp_head.cc 2014-08-25 06:49:04 +0000 > @@ -1224,6 +1224,7 @@ sp_head::execute(THD *thd, bool merge_da > Item_change_list old_change_list; > String old_packet; > uint old_server_status; > + const uint status_backup_mask= SERVER_STATUS_CURSOR_EXISTS; > Reprepare_observer *save_reprepare_observer= thd->m_reprepare_observer; > Object_creation_ctx *saved_creation_ctx; > Warning_info *saved_warning_info; > @@ -1358,7 +1359,7 @@ sp_head::execute(THD *thd, bool merge_da > It is probably safe to use same thd->convert_buff everywhere. > */ > old_packet.swap(thd->packet); > - old_server_status= thd->server_status; > + old_server_status= thd->server_status & status_backup_mask; > > /* > Switch to per-instruction arena here. We can do it since we cleanup > @@ -1488,7 +1489,7 @@ sp_head::execute(THD *thd, bool merge_da > thd->spcont->pop_all_cursors(); // To avoid memory leaks after an error > > /* Restore all saved */ > - thd->server_status= old_server_status; > + thd->server_status= (thd->server_status & ~status_backup_mask) | old_server_status; > old_packet.swap(thd->packet); > DBUG_ASSERT(thd->change_list.is_empty()); > old_change_list.move_elements_to(&thd->change_list); > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits

1 0

[Maria-developers] Getting Started
by Ashish Kedia 23 Aug '14

23 Aug '14

Hi all, I am 3rd Year undergraduate student majoring in Information Technology from India. I am new to MariaDB however I wish to contribute. I am well acquainted with C++ and other tools required. Can someone help me find small tasks/bugs/projects to get started with ? Regards, Ashish Kedia, 3rd Year Undergraduate Student, Dept. of Information Technology, National Institute of Technology Karnataka, Surathkal

3 2

[Maria-developers] Closing bugs as Incomplete/Cannot reproduce requires FixVersion?
by Sergey Petrunia 23 Aug '14

23 Aug '14

Hi Serg, (writing to you because I think it was you who set Jira's rules for Fix Version field). I'm trying to close MDEV-6032 as "Incomplete" ("Cannot reproduce" would also work). This resolution is meant to say that the issue is not really resolved. And Jira's "Close Issue" dialog gives me this error: Field Fix Version/s with actual value does not match regular expression \[([0-9]+\.[0-9]+\.[0-9]+.*|N/A)\] I think it doesn't make sense. I even think that "Fix Version" must be empty for bugs that are closed with resolution other than "Fixed" or "Duplicate". Do you agree with this reasoning? If yes, is it possible to fix Jira to follow this logic? BR Sergei -- Sergei Petrunia, Software Developer MariaDB | Skype: sergefp | Blog: http://s.petrunia.net/blog

2 1

[Maria-developers] Learning about MariaDB
by Kavneet Kaur 22 Aug '14

22 Aug '14

Hi! I am highly interested in subject of database. I am also interested in becoming a research developer. Worked on MS SQL. Want to try MariaDB. May I get the guidance about this product as i'm totally new to it. Is there any specific way to begin?

1 0

[Maria-developers] [GSoC] Pencils down
by sriram patil 21 Aug '14

21 Aug '14

Hi All, Following is the link to my pencils down blog post. You can go through it even if you have not followed my blog posts before. http://goo.gl/DqBgDe Thanks a lot for all your feedback and suggestions. Sriram.

3 3

[Maria-developers] [GSOC2014] UseServerPrepStmts - Connector/J
by Puneet Dewan 20 Aug '14

20 Aug '14

Hi Everyone, I have pushed the code and it can be found on the following link: https://code.launchpad.net/~team-java-connector/mariadb-java-client/mariadb… Thanks for the support. :) Please review and give suggestions. Regards Puneet.

1 0

Re: [Maria-developers] [GSoC] Optimize mysql-test-runs - Results of new strategy
by Pablo Estrada 19 Aug '14

19 Aug '14

Hello Elena and all, First, addressing the previous email: Looking at the dump, I see it can also happen that the dump contains > several records for a pair platform/bbnum. I am not sure why it happens, I > think it shouldn't, might be a bug in buildbot and/or configuration, or > environmental problems. Anyway, due to the way we store output files, they > can well override each other in this case, thus for several platform/bbnum > record you will have only one file. I suppose that's what was hard to > resolve, sorry about that. > No worries ; ). There are several cases where platform and build number are the same. The system just names the files as follows: <platform>_<build_id>-log-test_1-stdio <platform>_<build_id>-log-test_2-stdio ..... <platform>_<build_id>-log-test_5-stdio These files seem to correspond temporarily with the test runs (*test_1-stdio) belongs to the first test_run of the same plt-bnum, and so on. Unfortunately, there are some cases where there are more test_runs on the dump than files available, and this means that it's impossible to be sure which file belongs to which test_run exactly. > You should consider skipped tests, at least for now. Your logic that they > are skipped because they can't be run is generally correct; unfortunately, > MTR first produces the *full* list of tests to run, and determines whether > a test can be run or not on a later stage, when it starts running the > tests. Your tool will receive the initial test list, and I'm not sure it's > realistic to re-write MTR so that it takes into account limitations that > cause skipping tests before creating the list. > I see. Okay then, duly noted. Possibly it's better to skip a test run altogether if there is no input > list for it; it would be definitely the best if there were 5K (or whatever > slice you are currently using) of continuous test runs with input lists; if > it so happens that there are lists for some branches but not others, you > can skip the branch entirely. > This doesn't seem like a good option. Recall drops seriously, and the test_runs that have a corresponding file don't seem to have a special pattern, and tend to have long spaces between them, so the information becomes irrelevant, and seemingly, not useful. > The core module should take as parameters > - list of tests to choose from, > - size of the running set (%), > - branch/platform (if we use them in the end), > and produce a new list of tests of the size of the running set. > > The wrapper module should > - read the list of tests from the outside world (for now, from a file), > - receive branch/platform as command-line options, > - have the running set size set as an easily changeable constant or as a > configuration parameter, > > and return the list of tests -- lets say for now, in the form of <test > suite>.<test name>, blank-separated, e.g. > main.select innondb.create-index ... > >> >> I am almost done 'translating' the code into a solution that divides it in 'core' and 'wrapper'. There are a few bugs that I still haven't figured out, but I believe I can iron those out pretty soon. I will also incorporate the percentage rather than fixed running_set. Now, regarding the state of the project (and the recall numbers that I am able to achieve so far), here are some observations: - Unfortunately, I am running out of ideas to try to improve recall. I tried tuning some parameters, giving more weight to ones or others, etc. I still wasn't able to push recall beyond ~87% on the strategy that uses file correlations. For what I've seen, some failures are just extremely hard to predict. - The strategy that uses only a weighted average of the failure frequency achieves a higher recall, but for a shorter time. The recall decays quickly afterwards. I may try to add some file-correlations to this strategy, to see if the recall can be sustained for a longer term. - There is one problem that I see regarding the data and the potential real-world implementation of the program: By verifying the recall with the historical data, we run the possibility of 'expecting' overfitting... so the results regarding the errors found when comparing to the historical data, and the results that could have been obtained by a real-world implementation are potentially different. A possible way to address that issue would require modifying the buildbot to gather more data over a longer term. So having said that, I am looking for some advice in the following regards: - I will try to take a step back from the new strategy, and see how I can adapt the original strategy to prevent the recall function from declining so sharply with time. - I will also spend some time keeping a codebase that adjusts better to the model that we need for the implementation. I will upload code soon. All suggestions are welcome. - Nonetheless, I feel that more data would allow to improve the algorithm greatly. Is it possible to prepare logging into the buildbot that would allow for more precise data collection? A slower, more iterative process, working closer with the buildbot and doing more detailed data collection might deliver better results. (I understand that this would probably influence the time-scope of the project) Let me know what you think about my suggestions. Regards Pablo

2 22

[Maria-developers] [GSoC] Week Twelve!
by sriram patil 14 Aug '14

14 Aug '14

Hi All, Following is the link to my twelfth weeks blog post. http://goo.gl/Zkr86e Suggestions and/or reviews are welcome! Thanks, Sriram

3 2

[Maria-developers] Fwd: I have a replication patch, please accept it.
by Colin Charles 12 Aug '14

12 Aug '14

Xu Liang, the best place to post it is maria-developers I've done that via a FWD -- Kristian, what do you think? Begin forwarded message: > From: "Xu,Liang(OP)" <xuliang02(a)baidu.com> > Subject: I have a replication patch,please accept it. > Date: 6 August 2014 15:31:27 GMT+8 > To: "kb-admins(a)askmonty.org" <kb-admins(a)askmonty.org> > > http://bugs.mysql.com/bug.php?id=73479 > > > > > > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > Baidu DBA 徐良 > 分机：86912826 > 手机：18689220441 > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -- Colin Charles, Chief Evangelist, SkySQL - The MariaDB Company blog: http://bytebot.net/blog/| t: +6-012-204-3201 | Skype: colincharles

1 0

[Maria-developers] MariaDB 10.0.13 Release Prep
by Daniel Bartholomew 11 Aug '14

11 Aug '14

I've begun the prep work for the MariaDB 10.0.13 release. Draft changelog and release notes are here: - https://mariadb.com/kb/en/mariadb-10013-release-notes/ - https://mariadb.com/kb/en/mariadb-10013-changelog/ A big thanks to those that have already contributed content! As always, please let me know ASAP if anything else needs to be added/changed/etc..., or jump in and do it yourself. Thanks! -- Daniel Bartholomew, MariaDB Release Manager MariaDB | http://mariadb.com

1 0

[Maria-developers] innodb-buffer-pool-instances confusion
by Daniel Bartholomew 07 Aug '14

07 Aug '14

I was chatting with some folks on IRC and someone pointed this out to me. On the "Upgrading From MariaDB 5.5 to MariaDB 10.0" page, it says that the new default value for innodb-buffer-pool-instances is '0': https://mariadb.com/kb/en/upgrading-from-mariadb-55-to-mariadb-100/ And when I run the following, it says the default is indeed '0': mysqld --no-defaults --verbose --help | grep innodb-buffer-pool-instances ... innodb-buffer-pool-instances 0 But when I'm in the client, I get the following: MariaDB [(none)]> select @@innodb_buffer_pool_instances; +--------------------------------+ | @@innodb_buffer_pool_instances | +--------------------------------+ | 8 | +--------------------------------+ So when innodb-buffer-pool-instances is set to '0', does that mean the actual value is dynamic or calculated in some way based on my other settings? Just trying to understand what's going on, and once I do, use it to update the XtraDB/InnoDB buffer-pool page (which doesn't appear to have been updated for 10.0). https://mariadb.com/kb/en/xtradbinnodb-buffer-pool/ Thanks! -- Daniel Bartholomew, MariaDB Release Manager MariaDB | http://mariadb.com

4 3

[Maria-developers] CONNECT system variables
by Ian Gilfillan 07 Aug '14

07 Aug '14

Hi I am trying to document the CONNECT system variables at https://mariadb.com/kb/en/mariadb/documentation/storage-engines/connect/con… but I can't seem to find much documentation in the code. I managed to put something together from Olivier's notes in JIRA for some of them, but can anyone help fill in the gaps and send me details on the other variables? thanks, ian

1 0