June 2013 - developers - lists.mariadb.org

[Maria-developers] Group commit comparison between mysql and mariadb
by Sergei Petrunia 11 Jun '13

11 Jun '13

Hi Kristian, I'm wondering, do we have any comparison of MariaDB's binlog group commit implementation vs MySQL's binlog's group commit? I recall Mats writing a blog about the differences last year, but he didn't have the code back then. I'm wondering if Oracle had implemented what he described, or did something else... I understand that in order to do benchmark we need to finish merging InnoDB from MySQL 5.6 into MariaDB 10.0. One can't benchmark group commit per se, it is tied to innodb. Do we still have any clue, before the merge happens? Btw, what should the merge do - take group commit from MariaDB or from MySQL? BR Sergei -- Sergei Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

4 3

Re: [Maria-developers] Buildbot question
by Kristian Nielsen 11 Jun '13

11 Jun '13

Sergey Vojtovich <svoj(a)mariadb.org> writes: > I'm observing pretty odd buildbot behavior with some of my pushes (probably > merges). Bar and Daniel were unable to answer my questions and suggested to > ask you instead. The problem seems to be that Buildbot has become confused about revisions. Look at the top of the page you linked, at the revisions for each column. You will notice that they are not in order. The 3rd column is the highest revision, which is probably why spider suite is enabled there (I assume you pushed something to add the suite). I see your current top revision in the tree is 3760, while the Buildbot page has a newer revision 3767. It also looks like the Buildbot history contains _another_ earlier revision that was also number 3760, maybe that is the cause of the confusion (Buildbot perhaps thinks it is rebuilding the old 3760). You can see your revision sorted as the first columns in this link: http://buildbot.askmonty.org/buildbot/grid?branch=10.0-spider&width=20 This confusion usually happens because the tree was overwritten and revisions became renumbered. This can happen due to bzr push --overwrite, but in your case it is probably because you merged in the opposite direction. When merging a tree A with a tree B and pushing to B, it is generally best to merge A into B (cd to B and bzr merge A). This keeps the existing revision numbers of B intact. In the main trees we have protection that enforces this, but not in feature trees. (I would be nice if we could fix Buildbot to prevent this confusion, but so far noone has taken it upon themselves. At one point I asked the Launchpad/bzr developers to make the revision id (as opposed to revision numbers) available so that Buildbot could avoid using revision numbers completely, but the answer was that "hiding revision ids was a feature" ...). Usually the confusion is sorted out by itself on a following push, or you can use the trick above adding width=N to the link to see the column of the revision you want. Hope this helps, - Kristian.

2 1

[Maria-developers] MariaDB 10.0.3 release prep
by Daniel Bartholomew 11 Jun '13

11 Jun '13

All, I've begun the prep work for the MariaDB 10.0.3 alpha release. The packages are uploaded and the mirrors should pick them up soon (if they haven't already). The draft changelog and release notes are here: - https://kb.askmonty.org/en/mariadb-1003-release-notes/ - https://kb.askmonty.org/en/mariadb-1003-changelog/ Please let me know ASAP if there's anything that needs to be mentioned/highlighted/removed/etc... Assuming all goes well, the plan is to activate the release tomorrow (11 Jun 2013) at around 10am US Eastern Time. Thanks. -- Daniel Bartholomew Google+ - http://gplus.to/dbart Twitter - http://twitter.com/daniel_bart MariaDB: An Enhanced Drop-in Replacement for MySQL Website - http://mariadb.org Twitter - http://twitter.com/mariadb Google+ - http://google.com/+mariadb Facebook - http://fb.com/MariaDB.dbms Knowledgebase - http://kb.askmonty.org

1 0

[Maria-developers] audit plugin help with MDEV-575 (fail2ban like)
by Roberto Spadim 09 Jun '13

09 Jun '13

hi guys i want get users / ip that try to connect to server with bad password/user and retry very frequently (normally a brute force attack) some things i need: 1)get information about connection when it fail or not (maybe an audit plugin could do this?), information that i need: 1.1)protocol (tcp/unix/others) 1.2)user 1.3)remote ip (when tcp only) it must be executed in every login 2) log this information maybe we could use CSV,aria, myisam, to log bad logins and put it in a log table: PROTOCOL, USER, REMOTE IP, DATATIME 3) add columns to user BAD_LOGIN_MAX_TRIES, BAD_LOGIN_LOG_TIME, BAD_LOGIN_MIN_RETRY_TIME 4) add server global bad_login_tcp_log_time, bad_login_tcp_max_errors 5) implement checks before connection: when openned a connection to server via TCP execute something like: SELECT COUNT(*) FROM bad_login_log WHERE remote_ip="remote_ip" AND datetime>NOW() - @@bad_login_tcp_log_time if COUNT(*) > @@bad_login_tcp_max_erros disconnect 6) implement user checks: SELECT COUNT(*),(SELECT BAD_LOGIN_MAX_TRIES FROM users WHERE user="user") FROM bad_login_log WHERE user="user" AND datetime>NOW() - (SELECT BAD_LOGIN_LOG_TIME FROM users WHERE user="user") if count(*) > bad_login_max_tries form that user disconnect that's all -- Roberto Spadim SPAEmpresarial

1 0

[Maria-developers] Help with MDEV-4419, ENUM optimization
by Roberto Spadim 09 Jun '13

09 Jun '13

Hi guys, i want to optimize fields with ENUM() in item comparation the point is: SELECT COUNT(*) FROM table WHERE enum_field!='x' column 'enum_field' is ENUM('x,'y','z') NOT NULL DEFAULT 'x' i want to rewrite the query in a way that we can execute it with index: SELECT COUNT(*)FROM table WHERE enum_field IN ('y','z','') the "" is not allowed values ('a' for example) the feature is 'easy' to understand but i don't know where to implement it will read possible values of enum fields, and rewrite "!=","<","<=",">=",">" operators to IN operator with fields that match the condition of the primary operator some think like: column "operator" const value if(operator in ('!=','<','<=','>=','>' AND column is ENUM){ get all possible values from column copy all values to a return variable execute the "operator" in all values with const values if operator return false remove the value from the return variable change the "operator" and "const value" to : "IN" and return variable comma separated }else{ no optimization leave operation as it is } example: column = enum('x','y','z'), operator = ">" const value = "3" if(column is ENUM and operator = ">"){ possible values => "x","y","z","" (check that "" is used when insert into, value = "a" for example) return variable = "x","y","z", "" for each value.... "x" > "3" ? yes, ASCII 120 > ASCII 51 -> do nothing "y" > "3" ? yes, ASCII 121 > ASCII 51 -> do nothing "z" > "3" ? yes, ASCII 122 > ASCII 51 -> do nothing "" > "3" ? no, "" < ASCII 51 -> remove the value "" from return variable end for now the return variable is : "x","y","z" the "" was removed rewrite the operator to: "column IN ('x','y','z')" } ----------------------- this will optimize a lot: using != in a big table (446577 rows) result = Using where, 446577 rows, type= ALL using IN() in a big table: result = Using where, 28 rows, key = index, type = range in other words, a lot of optimization with ENUM fields, since they are a pseudo "index" of what we can found in the column =) thank guys any help is wellcome to make a patch -- Roberto Spadim SPAEmpresarial

1 0

[Maria-developers] GSoC MariaDB Kerberos Authentication Plugin Proposal
by QIU Shuang 06 Jun '13

06 Jun '13

Hi Sergei, Thank you and the community for accepting my proposal of this year's GSoC: Kerberize MariaDB -- add kerberos authentication support to MariaDB. I'm Shuang Qiu, a second year graduate student from Shanghai Jiao Tong University, Shanghai, China. My field of research is distributed computing, especially distributed storage. I just get in touch with my project mentor Mr. Vaintroub and join the #maria IRC channel at freenode.net, where I can be found as shuang. Thank you again and hope we can have a happy and productive summer! Cheers! Sincerely, Shuang

1 0

Re: [Maria-developers] [Maria-discuss] Clear-text plugin
by Peter Laursen 06 Jun '13

06 Jun '13

"no support for dlopen()" . How does this apply to Windows clients? We always build first on Windows and once we have no errors of any kind we build on Linux if the application exists for that platform (what MONyog does and SQLyog does not) and fix what issues may occur there. This issue has occurred with SQLyog (and as such for a Windows client). I am not coder/developer myself, but on Windows I think we have no other option than statically linking? We could of could compile the API as a .dll, but we don't (because we had a lot of problems with it earlier). -- Peter On Thu, Jun 6, 2013 at 3:25 PM, Sergei Golubchik <serg(a)mariadb.org> wrote: > Hi, Peter! > > On Jun 04, Peter Laursen wrote: > > We have the same problem. We cannot test either. But we have a few user > > reports. We will discuss what we can do. > > > > Is there any parameter or compile-time option we should be aware of? Any > > specific documentation somewhere? Anything? > > Not really. If the cleartext plugin is built and installed, it'll be > used when the server requests it. > > I've just checked that this plugin is present both in rpm and deb > packages. I didn't check binary tarballs, but it should be there too. > > So, there's nothing to configure at run-time or compile-time, everything > should work automatically and transparently. > > Hmm, unless you build a completely static client library with no support > for dlopen(). > > Regards, > Sergei >

2 1

Re: [Maria-developers] [Maria-discuss] Clear-text plugin
by Peter Laursen 04 Jun '13

04 Jun '13

We have the same problem. We cannot test either. But we have a few user reports. We will discuss what we can do. Is there any parameter or compile-time option we should be aware of? Any specific documentation somewhere? Anything? -- Peter On Tue, Jun 4, 2013 at 2:19 PM, Sergei Golubchik <serg(a)mariadb.org> wrote: > Hi, Peter! > > On Jun 04, Peter Laursen wrote: > > For clarity: a 'dummy' parameter doing nothing does not solve our > > problem. We actually want to build clients that can (also) connect to > > Oracle MySQL Enterprise servers configured to make use of Oracle's PAM > > implementation. > > We build and distribute a cleartext plugin. And it can be used if the > server requests. In a sense, in MariaDB a cleartext plugin is always > enabled. So a dummy option is needed only to restore the binary protocol > compatibility. That's my understanding, at least, but I don't have > Oracle's PAM plugin, so I cannot test it. > > But if you have verified that your client with our C connector can not > connect to Oracle MySQL Enterprise server with the Oracle's PAM > implementation - please submit a bug report, and we will fix it. > > Regards, > Sergei > >

1 0

Re: [Maria-developers] [Maria-discuss] Clear-text plugin
by Peter Laursen 04 Jun '13

04 Jun '13

For clarity: a 'dummy' parameter doing nothing does not solve our problem. We actually want to build clients that can (also) connect to Oracle MySQL Enterprise servers configured to make use of Oracle's PAM implementation. -- Peter On Tue, Jun 4, 2013 at 11:05 AM, Peter Laursen <peter_laursen(a)webyog.com>wrote: > yes .. in the Connector/C. We use C/C++ for both SQLyog and MONyog. > > -- Peter > > > On Tue, Jun 4, 2013 at 10:17 AM, Sergei Golubchik <serg(a)mariadb.org>wrote: > >> Hi, Peter! >> >> On Jun 04, Peter Laursen wrote: >> > This is a continuation of >> > https://mariadb.atlassian.net/browse/CONC-25. We requested support for >> > the clear-text plugin (or a workaround of any kind) that would allow a >> > client compiled with the MariaDB connector to use PAM authentication >> > with MySQL Enterprise servers from Oracle. >> >> Another related issue is https://mariadb.atlassian.net/browse/MDEV-4574 >> which should be fixed in 5.5.32, next 5.5 release. >> >> But you, probably, want it in Connector/C, don't you? >> >> Regards, >> Sergei >> >> >

1 0

[Maria-developers] Clear-text plugin
by Peter Laursen 04 Jun '13

04 Jun '13

This is a continuation of https://mariadb.atlassian.net/browse/CONC-25. We requested support for the clear-text plugin (or a workaround of any kind) that would allow a client compiled with the MariaDB connector to use PAM authentication with MySQL Enterprise servers from Oracle. I will not dispute Sergei's opinions about Oracle's implementation. But I believe it misses the point. The question is if the MariaDB connector should be a replacement for the C-IPA from Oracle. And the question is nothing else than that. Not the quality of solutions bundled with Enterprise servers from Oracle. I would like to hear opinions not only from 'core developers' like Sergei, but also from wider circles in MariaDB organization (management, marketing) as well as the user community. For us the problem is like this: We introduced support for the clear-text plugin in SQLyog 11.1. In 11.12 beta2 we replaced Oracles connector with MariaDB (and had to removed it again). We had a few crashes and memory leaks with mariaDB Connector but that is getting in control now (ther were bugs in both the API and in oru code). We probably could patch the MariaDB code without much problems to support the full-text plugin - but we don't have a MySQL Enterprise server to test with and we cannot release without proper testing. We have already started getting complaints from users/customers about that what we introduced in 11.1 has been removed in current development tree. It seems that we have quite a lot of customers using Oracle's Enterprise servers wiht Oracle's PAM authentication. A solution/workaround would very much be appreciated. Regards Peter Laursen Webyog

1 0