- developers - lists.mariadb.org

Re: [Maria-developers] Deadlock in Parallel Replication with FLUSH TABLE WITH READ LOCK.
by Kristian Nielsen 20 Oct '15

20 Oct '15

> I enabled // replication on the slave that has MyISAM tables. > The first results are good, but I ended up with what looks like a deadlock. > I opened support # 10289 for "Deadlock in Parallel Replication with > FLUSH TABLE WITH READ LOCK". There is a patch for this: https://lists.launchpad.net/maria-developers/msg08675.html But no reply on the review request so far. - Kristian.

1 0

[Maria-developers] Code review process
by Otto Kekäläinen 19 Oct '15

19 Oct '15

Hello! During the developer meetup there was a discussion about Gerrit. OpenStack has quite a nice documentation how to contribute, including the git review command and gerrit workflow: http://docs.openstack.org/infra/manual/developers.html -- Otto Kekäläinen MariaDB Foundation

1 0

Re: [Maria-developers] [Maria-discuss] Moving from Launchpad lists to something else... ?
by Tom Worster 19 Oct '15

19 Oct '15

On 10/15/15, 5:03 AM, "Maria-discuss on behalf of Colin Charles" <maria-discuss-bounces+fsb=thefsb.org(a)lists.launchpad.net on behalf of colin(a)mariadb.org> wrote: >> On 15 Oct 2015, at 05:35, Jean Weisbuch <jean(a)phpnet.org> wrote: >> >> To be able to give the best answer we must first know what is >>motivating the move from LP. > >So the reason we were thinking of ditching launchpad for mailing lists: >1. When there was a MariaDB-Manager google group, that group got a lot of >user posts about MariaDB Server. That group now no longer exists, but it >gave us a hint that people are interested in a Google group > >2. We moved everything from Launchpad (bugs to Jira, code to Github) and >the only thing remaining is mailing lists > >3. The MariaDB Knowledgebase: https://mariadb.com/kb/en/ used to have an >OpenID login that would be useful if you had a LP account, but OpenID is >apparently going away (or gone), so there’s less use for LP > >4. LaunchPad itself seems to be “hard” for people to signup (see #1). Good points. Making it easy for people to get involved is important, but if that's the idea then moving to Google Groups isn't a great solution. Apart from it being old and clunky owing to Google's neglect, the insurmountable problem is that it's still an email list at core. Personally, I like email lists best but I'm very old and have been in computing since the 70s, and I accept that most of the rest of the world has moved on. Another problem the Maria community might consider together with the discuss email list is the low quality and participation in the web Q&A of the Knowledge Base. That sometimes makes me sad. There is now a decent open-source system that can function as email list replacement and provide web and mobile Q&A, as well as other stuff: https://www.discourse.org I've used it at New Relic and it seems pretty good. It's got a lot of good stuff and I want to highlight just two. 1. "Mailing List Support - Opt into a special mode where all messages are sent to you via email, exactly like a mailing list. Start new topics via email." 2. You can host it yourself or pay someone you trust to do so. It's not a "free" cloud service where you pay by feeding the internet advertising/surveillance economy. I don't know if Discourse is right for Maria but I wouldn't mention it if I didn't think it might help build a bigger and more active support community, which I long for. Tom

2 1

[Maria-developers] JSON in MariaDB 10.2
by Colin Charles 14 Oct '15

14 Oct '15

Hi all, We just had a session here at the MariaDB Server developer’s meeting in Amsterdam on JSON in MariaDB 10.2. Before forgetting while things are still fresh in the mind, here are the notes from it (in Markdown format). cheers, -colin # JSON in MariaDB * Virtual columns with functional indexes (i.e. generated columns like in 5.7) can be implemented in 2-3 weeks (Sergei) * Eric: Concern about using non-standard function names (functions in 5.7 are `JSN_INSERT` whereas the standard uses `JSON_QUERY`) * Suggestion: Use standard names when available, create UDF to match MySQL names that can be loaded as desired. (or vice versa -- do what MySQL 5.7 does and have UDFs to follow the standard) * Come up with Usable JSON Test suite * Employee DB in JSON? * Good tests exist in DocStore (from the 5.6 facebook tree) * Monty believes we should have more strictly typed (DocStore for example says `unique key doc_c(doc.address.zipcode as int, c)`). Monty thinks it should be optional to keep compatibility with 5.7. This is also similar to the JSON proposed standard (where you have to choose between lax or strict mode) * Colin: Might want to convert JSON document to a relational table (This is part of the standard as `JSON_TABLE`) * Colin: Do we support something similar to what FB has or what oracle has? * SergeyP: We need to follow oracle to maintain compatibility. * Colin: We need to do this soon (i.e. since 5.7 is coming and its going to be a problem for 10.1 not having it; so 10.2 needs it as soon as possible). Everyone generally agrees this is a good idea. # References * [SQL SUPPORT FOR JSON](http://jtc1bigdatasg.nist.gov/_workshop/08_SQL_Support_for_JSON_abstr… * [WL#8132: JSON datatype and binary storage format](https://dev.mysql.com/worklog/task/?id=8132) * [WL#7909: Server side JSON functions](https://dev.mysql.com/worklog/task/?id=7909) * [WL#8249: JSON comparator](https://dev.mysql.com/worklog/task/?id=8249) * [WL#8170: Expression analyzer for GC](https://dev.mysql.com/worklog/task/?id=8170) * [WL#8149: B-tree Index Support on non-materialized virtual columns](https://dev.mysql.com/worklog/task/?id=8149) * [WL#8227: Support SEs to create index on virtual generated columns](https://dev.mysql.com/worklog/task/?id=8227) * [WL#8114: Don't store virtual generated columns in database](https://dev.mysql.com/worklog/task/?id=8114) * [JSON Labs Release: Native JSON Data Type and Binary Format](http://mysqlserverteam.com/json-labs-release-native-json-data-type-… * [JSON Labs Release: JSON Functions, Part 1 — Manipulation JSON Data](http://mysqlserverteam.com/json-labs-release-json-functions-part-1-ma… * [JSON Labs Release: JSON Functions, Part 2 — Querying JSON Data](http://mysqlserverteam.com/mysql-5-7-lab-release-json-functions-part-… * [JSON Labs Release: Effective Functional Indexes in InnoDB](http://mysqlserverteam.com/json-labs-release-effective-functional-i… * [11.6 The JSON Data Type](https://dev.mysql.com/doc/refman/5.7/en/json.html) * [DocStore: Document Database for MySQL at Facebook](https://www.percona.com/live/mysql-conference-2015/sites/default/… * [JSON Support in MySQL](https://www.percona.com/live/mysql-conference-2015/sites/default/fil… # Thanks Thanks to Bryan Alsdorf for writing some of the notes during the session -- Colin Charles, Chief Evangelist, MariaDB Corporation blog: http://bytebot.net/blog/ | t: +1-347-903-3201 | Skype: colincharles | Twitter: @bytebot

1 0

Re: [Maria-developers] [Commits] 5f461a9: MDEV-7640: CHANGE MASTER TO doesn't work with prepared statements
by Oleksandr Byelkin 11 Oct '15

11 Oct '15

Hi! On 17.09.15 00:10, Nirbhay Choubey wrote: > revision-id: 5f461a94a5aa2e1a20fa559ca612d96b53201e4e > parent(s): 29ac245dd04c5416d57a90b0ab3c4d08cbc4d723 > committer: Nirbhay Choubey > branch nick: 5.5 > timestamp: 2015-09-16 18:10:06 -0400 > message: > > MDEV-7640: CHANGE MASTER TO doesn't work with prepared statements > > When CHANGE MASTER was executed as a PS, its attributes were wrongly > getting reset toward the end of PREPARE. As a result, the subsequent > executions had no effect. Fixed by making sure that the CHANGE MASTER > attributes are preserved during the lifetime of the PS. It is OK to push but after expanding the comment about what exactly reset in the persistent statement memory (as we agreed) [skip]

1 0

Re: [Maria-developers] [Commits] 06e3c05: MDEV-8842 add group support to pam_user_map module.
by Sergei Golubchik 07 Oct '15

07 Oct '15

Hi, Holyfoot! On Oct 07, holyfoot(a)askmonty.org wrote: > revision-id: 06e3c05f1ad3156b5a3e0a775b07fb13b9ebb1a8 (mariadb-10.1.7-68-g06e3c05) > parent(s): 8afe96f011eb8037a92b4b3aab16118b0771ad50 > committer: Alexey Botchkov > timestamp: 2015-10-07 15:52:26 +0500 > message: > > MDEV-8842 add group support to pam_user_map module. > Added to the pam_user_map module. Great, thanks! Ok to push. I assume you've tested that it still works after all the changes. Regards, Sergei

1 0

Re: [Maria-developers] [Commits] 50dc8c0: MDEV-8842 add group support to pam_user_map module.
by Sergei Golubchik 07 Oct '15

07 Oct '15

Hi, Holyfoot! On Oct 07, holyfoot(a)askmonty.org wrote: > revision-id: 50dc8c0e8a27d18a3d75ff43a87975fcd5e0e7f6 (mariadb-10.1.7-67-g50dc8c0) > parent(s): bed4e847950eef50930b44632eea43416e7b37d1 > committer: Alexey Botchkov > timestamp: 2015-10-07 00:51:33 +0500 > message: > > MDEV-8842 add group support to pam_user_map module. > Added to the pam_user_map module. Looks pretty much ok. Just one suggestion: I'd keep groups in the pam_sm_authenticate and passed the pointer down to user_in_group. Indeed, there may many @group entries in the mapping file, seems like a waste do repopulate group array every time. > --- > plugin/auth_pam/mapper/pam_user_map.c | 59 +++++++++++++++++++++++++++++++++-- > 1 file changed, 57 insertions(+), 2 deletions(-) > > diff --git a/plugin/auth_pam/mapper/pam_user_map.c b/plugin/auth_pam/mapper/pam_user_map.c > index e73ab6d..a3008bd 100644 > --- a/plugin/auth_pam/mapper/pam_user_map.c > +++ b/plugin/auth_pam/mapper/pam_user_map.c > @@ -13,22 +13,71 @@ auth required pam_user_map.so > > And create /etc/security/user_map.conf with the desired mapping > in the format: orig_user_name: mapped_user_name > + @user's_group_name: mapped_user_name > ========================================================= > -#comments and emty lines are ignored > +#comments and emtpy lines are ignored > john: jack > bob: admin > top: accounting > +@group_ro: readonly > ========================================================= > > */ > > +#include <stdlib.h> > #include <stdio.h> > #include <syslog.h> > +#include <grp.h> > +#include <pwd.h> > +#include <limits.h> > #include <security/pam_modules.h> > > #define FILENAME "/etc/security/user_map.conf" > #define skip(what) while (*s && (what)) s++ > > +#define GROUP_BUFFER_SIZE 100 > +int user_in_group(const char *user, const char *group) > +{ > + gid_t group_buffer[GROUP_BUFFER_SIZE]; > + gid_t *groups= group_buffer; > + gid_t group_id; > + gid_t user_group_id; > + int ng, i; > + > + { > + struct passwd *pw= getpwnam(user); > + struct group *g= getgrnam(group); > + if (pw == NULL || g == NULL) > + return 0; > + user_group_id= pw->pw_gid; > + group_id= g->gr_gid; > + } > + > + ng= GROUP_BUFFER_SIZE; > + if (getgrouplist(user, user_group_id, groups, &ng) < 0) > + { > + /* The rare case when the user is present in more than */ > + /* GROUP_BUFFER_SIZE groups. */ > + groups= (gid_t *) malloc(ng * sizeof (gid_t)); > + if (!groups) > + return 0; > + > + (void) getgrouplist(user, user_group_id, groups, &ng); > + } > + > + for (i= 0; i < ng; i++) > + { > + if (groups[i] == group_id) > + break; > + } > + > + if (groups != group_buffer) > + free(groups); > + > + return i < ng; > +} > + > + > int pam_sm_authenticate(pam_handle_t *pamh, int flags, > int argc, const char *argv[]) > { > @@ -51,10 +100,14 @@ int pam_sm_authenticate(pam_handle_t *pamh, int flags, > while (fgets(buf, sizeof(buf), f) != NULL) > { > char *s= buf, *from, *to, *end_from, *end_to; > + int check_group; > + > line++; > > skip(isspace(*s)); > if (*s == '#' || *s == 0) continue; > + if ((check_group= *s == '@')) > + s++; > from= s; > skip(isalnum(*s) || (*s == '_')); > end_from= s; > @@ -67,7 +120,9 @@ int pam_sm_authenticate(pam_handle_t *pamh, int flags, > if (end_to == to) goto syntax_error; > > *end_from= *end_to= 0; > - if (strcmp(username, from) == 0) > + if (check_group ? > + user_in_group(username, from) : > + (strcmp(username, from) == 0)) > { > pam_err= pam_set_item(pamh, PAM_USER, to); > goto ret; Regards, Sergei

2 1

Re: [Maria-developers] [Commits] c934b5e: MDEV-8842 add group support to pam_user_map module.
by Sergei Golubchik 06 Oct '15

06 Oct '15

Hi, Holyfoot! On Oct 06, holyfoot(a)askmonty.org wrote: > revision-id: c934b5e661b19b7934d1a9af40afc70b727c2273 (mariadb-10.1.7-64-gc934b5e) > parent(s): 90f2c822469a4f88f882ba8974e790f0fb0b2702 > committer: Alexey Botchkov > timestamp: 2015-10-06 14:02:29 +0500 > message: > > MDEV-8842 add group support to pam_user_map module. > Added to the pam_user_map module. Right, thanks. A couple of comments, see below. > diff --git a/plugin/auth_pam/mapper/pam_user_map.c b/plugin/auth_pam/mapper/pam_user_map.c > index e73ab6d..0787b1c 100644 > --- a/plugin/auth_pam/mapper/pam_user_map.c > +++ b/plugin/auth_pam/mapper/pam_user_map.c ... > + /* > + So below we get the buffer of the NGROUPS_MAX size > + which can take 256K on some systems. > + It supposed to be working differently - we'd get > + the 'ng' from the getgrouplist() and alloc the respective > + size, but the problem is a bug in getgrouplist(). > + The glibc 2.3.2 implementation of this function is broken: > + it overwrites memory when the actual number of groups is larger > + than *ngroups. > + */ Forget 2.3.2, use it as it was supposed to be used. Our oldest builder uses glibc-2.5 > + (void) getgrouplist(user, user_group_id, groups, &ng); > + for (i= 0; i < ng; i++) > + { > + struct group *g; > + if ((g= getgrgid(groups[i])) == NULL) > + return 0; > + if (g->gr_gid == group_id) > + return 1; Eh? Why don't you compare group_id with groups[i] directly ? Regards, Sergei

2 1

Re: [Maria-developers] [Commits] 7ea735b: MDEV-8431 Feedback plugin needs an option for http proxy.
by Sergei Golubchik 06 Oct '15

06 Oct '15

Hi, Holyfoot! On Oct 05, holyfoot(a)askmonty.org wrote: > revision-id: 7ea735baa2df4f4577cf951bc69ccf87fd91f436 (mariadb-10.1.7-40-g7ea735b) > parent(s): ba0b6685510f4a271da8d386ff75bfd83d3156c2 > committer: Alexey Botchkov > timestamp: 2015-10-05 14:46:12 +0500 > message: > > MDEV-8431 Feedback plugin needs an option for http proxy. > 'feedback_http_proxy' system variable added to specify the > proxy server as host:port. Not a dynamic one. That's a bit more complex that I thought it'll be, but ok. Still - did you test it? How? Regards, Sergei

2 2

Re: [Maria-developers] [Commits] ee656a4: MDEV-7846: postreview fix
by Sergei Golubchik 06 Oct '15

06 Oct '15

Hi, Sanja! I have no idea how this postreview fix relates to any of my review comments. Let me repeat them again: 1. Add comments 2. Add a test case with row items 3. In your test case - if you remove GROUP BY you get a crash again. This might be a different bug, though. On Aug 05, sanja(a)mariadb.com wrote: > revision-id: ee656a4aabb5ebbe973b7e4c99be83eb5dc8c0ce > parent(s): 9f8a458fb2d07298810bb5d9824ce728017bfb32 > committer: Oleksandr Byelkin > branch nick: server > timestamp: 2015-08-05 11:57:35 +0200 > message: > > MDEV-7846: postreview fix > > diff --git a/sql/item_cmpfunc.cc b/sql/item_cmpfunc.cc > index 29fff8e..e96d1bc 100644 > --- a/sql/item_cmpfunc.cc > +++ b/sql/item_cmpfunc.cc > @@ -1455,12 +1455,13 @@ bool Item_in_optimizer::fix_left(THD *thd, Item **ref) > next execution we need to copy args[1]->left_expr again. > */ > ref0= &(((Item_in_subselect *)args[1])->left_expr); > + args[0]= ((Item_in_subselect *)args[1])->left_expr; > } > if ((!(*ref0)->fixed && (*ref0)->fix_fields(thd, ref0)) || > (!cache && !(cache= Item_cache::get_cache(*ref0)))) > DBUG_RETURN(1); > if (args[0] != (*ref0)) > - current_thd->change_item_tree(args, (*ref0)); > + args[0]= (*ref0); > DBUG_PRINT("info", ("actual fix fields")); Regards, Sergei

2 1