- developers - lists.mariadb.org

[Maria-developers] Please review a fix for MDEV-10780 (and likely for MDEV-10806 and MDEV-10910)
by Alexander Barkov 10 Nov '16

10 Nov '16

Hello Sanja, Elena, Wlad, Sanja, please review a patch for MDEV-10780. The patch is for 10.1. I could not reproduce it in 10.0. But it should be safe to push it into 10.0 anyway. This patch is most likely fixing MDEV-10806 (assigned to Sanja) and MDEV-10910 (assigned to Elena). Thanks!

3 6

[Maria-developers] Building MariaDB 10.3 on windows
by jerome brauge 07 Nov '16

07 Nov '16

Hi, I'm new on this mailing list, so sorry if it's not the right place. I'm trying to build MariaDB 10.3 on Windows 10 with Visual Studio Express 2014. Compiling failed with many errors "fatal error C1189: #error: Macro definition of snprintf conflicts with Standard Library function declaration". To correct this, in ma_global.h, i replaced line #define snprintf _snprintf by #ifdef _MSC_VER #if _MSC_VER< 1900 #define snprintf _snprintf #endif #else #define snprintf _snprintf #endif and build success. Best regards. J.Brauge -----Message d'origine----- De : Maria-developers [mailto:maria-developers-bounces+j.brauge=qualiac.com@lists.launchpad.net] De la part de Sergei Golubchik Envoyé : vendredi 4 novembre 2016 11:41 À : maria-developers(a)lists.launchpad.net Objet : [Maria-developers] security spring cleaning in MariaDB org on github Hi, Now when Github has per-user ownership rights and suggests to migrate away from the legacy admin team https://help.github.com/articles/migrating-your-previous-admin-teams-to-the… we're performing some spring cleaning in this area. The legacy admin team (named "Core") is removed. Most of its members lost admin access to the org. Currently only the MariaDB Foundation CEO and few board members (those, who actually have used admin access recently) retained their admin rights. Everyone who was in the Core team should still have write access to repositories, if you've found that it's not the case, please complain asap. If you think you need admin access, please request it (again). Note that - only members of the organization can have it, being in the Developers group is a plus - 2FA is required for all admins (and highly recommended for all other members) Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org _______________________________________________ Mailing list: https://launchpad.net/~maria-developers Post to : maria-developers(a)lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-developers More help : https://help.launchpad.net/ListHelp

2 1

[Maria-developers] Please review MDEV-11245 Move prepare_create_field and sp_prepare_create_field() as methods to Column_definition
by Alexander Barkov 07 Nov '16

07 Nov '16

Hello Monty, I'm done with: MDEV-10577 sql_mode=ORACLE: %TYPE in variable declarations But I'd like to move a certain part of MDEV-10577 into a separate patch. I created an MDEV entry for this: MDEV-11245 Move prepare_create_field and sp_prepare_create_field() as methods to Column_definition Can you please review it? See attached. There's also a copy here: hasky.askmonty.org:/tmp/MDEV-11245.diff The patch is for 10.3. Thanks.

1 0

[Maria-developers] Community Involvement - GSoC Mentor Summit impressions
by Vicențiu Ciorbaru 07 Nov '16

07 Nov '16

Hi everyone! Since MariaDB has completed another Google Summer of Code program this year, with some great projects as well, the MariaDB Foundation was able to send 2 mentors to the summit in California this year. I was one of the people that had the privilege to go. There were a lot of talks there about community involvement and how to encourage people to contribute. I've picked the ideas that I've found to be most interesting and useful and have blogged [1] about them. I intend to try and apply some of them if possible. Since this is a community matter (albeit more GSoC related), it makes sense to ask for feedback from the community. I think some of the ideas are at least partly implemented but we can always do better. If you're interested, lets have a discussion on this. Vicențiu [1] http://vicentiu.ciorbaru.io/community-involvement-gsoc-mentor-summit-impres…

1 0

[Maria-developers] security spring cleaning in MariaDB org on github
by Sergei Golubchik 07 Nov '16

07 Nov '16

Hi, Now when Github has per-user ownership rights and suggests to migrate away from the legacy admin team https://help.github.com/articles/migrating-your-previous-admin-teams-to-the… we're performing some spring cleaning in this area. The legacy admin team (named "Core") is removed. Most of its members lost admin access to the org. Currently only the MariaDB Foundation CEO and few board members (those, who actually have used admin access recently) retained their admin rights. Everyone who was in the Core team should still have write access to repositories, if you've found that it's not the case, please complain asap. If you think you need admin access, please request it (again). Note that - only members of the organization can have it, being in the Developers group is a plus - 2FA is required for all admins (and highly recommended for all other members) Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

2 5

[Maria-developers] Please review: support for --force-restart in MTR
by Sergey Petrunia 07 Nov '16

07 Nov '16

Hi Sergei, Elena I'm not sure who maintains mysql-test-run.pl but you two seem to have contributed to it in the past, this is why I'm addressing this to you. I'm going to get "MTR v2" to support --force-restart, just like MTR v1 did; http://lists.askmonty.org/pipermail/commits/2016-November/010062.html http://lists.askmonty.org/pipermail/commits/2016-November/010063.html Since I don't really understand this MTRv1 vs MTRv2 change (and why some things are in v1 but not in v2, etc), I wanted to check this change with you. Thanks, Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] [MariaDB/server] MDEV-11065 - Compressed binary log (#247)
by Kristian Nielsen 03 Nov '16

03 Nov '16

vinchen <notifications(a)github.com> writes: > The new code is here: > https://github.com/vinchen/server/commits/GCSAdmin-10.2-binlog-compressed2-2 > And added two fixed: > 1.Avoid overflowing buffers in case of corrupt events > 2.Check the compressed algorithm. Looks fine, thanks for fixing this. I have now merged and pushed this to MariaDB 10.2. > > Rows_log_event::write_data_body(). I *think* the reason for that is that > > the > > SQL thread never sees the compressed events (they are uncompressed by the > > IO > > thread), but I would like your confirmation that my understanding is > > correct. > > 3. Did you think about testing that BINLOG statements with compressed > As mentioned above, the compressed events would uncompressed in constructor > in SQL thread. Right, thanks, I understand now. So the BINLOG statements in the output of mysqlbinlog have the uncompressed data. This makes sense, just like the SQL queries are uncompressed before being output by mysqlbinlog. So in fact, one change I suggested is wrong, to accept compressed event types in BINLOG statements. So I reverted this change again, and also added some test cases: https://github.com/MariaDB/server/commit/56a041cde657e5618c519a3c50e8075136… BTW, I think this is recently merged code (from the delayed replication feature) that would not have appeared in your original patch. Thanks for the explanation to help me understand this. I also fixed a .result file - this is a failure that would only show up when running the test suite with --embedded: https://github.com/MariaDB/server/commit/3c0ff6153f75bb8e63c08ff3c828235512… So I think everything should be fine now and this should appear in MariaDB 10.2.3, if I understand correctly. Once again thanks for the patch. It was a pleasure to see a replication addition that was so well done, and with so much attention to detail. - Kristian.

1 0

[Maria-developers] MDEV-11219 main.null fails in buldbot and outside with ps-protocol
by Alexander Barkov 03 Nov '16

03 Nov '16

Hello Sergei, Please review a patch for MDEV-11219. Thanks!

1 0

[Maria-developers] Please review MDEV-10811 Change design from "Item is Type_handler" to "Item has Type_handler"
by Alexander Barkov 03 Nov '16

03 Nov '16

Hello Alexey, Please review a patch for 10.3 for MDEV-10811 Change design from "Item is Type_handler" to "Item has Type_handler Thanks.

1 0

Re: [Maria-developers] [MariaDB/server] Compressed binary log (#247)
by Kristian Nielsen 03 Nov '16

03 Nov '16

GCSAdmin <notifications(a)github.com> writes: > We add new event types to support compress the binlog as follow: > QUERY_COMPRESSED_EVENT, > WRITE_ROWS_COMPRESSED_EVENT_V1, > UPDATE_ROWS_COMPRESSED_EVENT_V1, > DELETE_POWS_COMPRESSED_EVENT_V1, > WRITE_ROWS_COMPRESSED_EVENT, > UPDATE_ROWS_COMPRESSED_EVENT, > DELETE_POWS_COMPRESSED_EVENT > Thanks for the patch! Overall, I'm much impressed with the quality, I see a lot of attention to detail. Below, I have a number of comments and suggestions, but they are all minor for a patch of this complexity. A number of the comments are style fixes or simple changes, and I found it easier to just do the changes to the source for you to review. I hope that is ok. I rebased the series without intermediate merge to simplify review to a single patch. The rebase with my suggested changes on top is here: https://github.com/knielsen/server/commits/GCSAdmin-10.2-binlog-compressed2… Please check the changes I made and let me know if you disagree with anything or I misunderstood something. Changes are mostly: - A bunch of code style (indentation, lines <= 80 chars, clarify comments, and so on). - Change LOG_EVENT_IS_QUERY() etc. from macros to static inline functions. - check_event_type() updated (new code merged into 10.2 recently). - Minor .result file update, also from code merged into 10.2 recently. I also have the following questions/suggestions: 1. I think the code should sanity-check the compressed event header, to check that it does not access outside the event buffer in corrupt events. Eg. if lenlen is 7 and there is less than 7 bytes available in the event. I know that replication code in general is not robust to corrupt events, but it seems best that new code avoids overflowing buffers in case of corrupt event data. 2. There are some places where the compressed event types are not added to switch() or if() statements, mainly related to minimal binlog images, I think: Rows_log_event::read_write_bitmaps_cmp(), Rows_log_event::get_data_size(), Rows_log_event::do_apply_event(), Rows_log_event::write_data_body(). I *think* the reason for that is that the SQL thread never sees the compressed events (they are uncompressed by the IO thread), but I would like your confirmation that my understanding is correct. 3. Did you think about testing that BINLOG statements with compressed events can execute correctly? This happens with mysqlbinlog | mysql, when there are (compressed) row-based events in the binlog. I'm wondering if this could expose compressed events to code that normally runs in the SQL thread and expects to have events uncompressed for it by the IO thread? A few detailed comments on the patch below (most comments are done as changes in the above-linked git branch). Otherwise, after answer to above 3 questions, I think the patch looks good to go into 10.2. - Kristian. ----------------------------------------------------------------------- > +#define BINLOG_COMPRESSED_HEADER_LEN 1 > +#define BINLOG_COMPRESSED_ORIGINAL_LENGTH_MAX_BYTES 4 > +/** > + Compressed Record > + Record Header: 1 Byte > + 0 Bit: Always 1, mean compressed; > + 1-3 Bit: Reversed, compressed algorithm??Always 0, means zlib > + 4-7 Bit: Bytes of "Record Original Length" > + Record Original Length: 1-4 Bytes > + Compressed Buf: I did not understand this. Why "reversed"? It seems to be bits 0-2 that have the bytes of "Record Original Length" and bit 7 that has the '1' bit meaning compression enabled? Maybe this can be clarified. > +int query_event_uncompress(const Format_description_log_event *description_event, bool contain_checksum, > + const char *src, char* buf, ulong buf_size, bool* is_malloc, > + char **dst, ulong *newlen) > +{ > + ulong len = uint4korr(src + EVENT_LEN_OFFSET); > + const char *tmp = src; > + > + DBUG_ASSERT((uchar)src[EVENT_TYPE_OFFSET] == QUERY_COMPRESSED_EVENT); > + > + uint8 common_header_len= description_event->common_header_len; > + uint8 post_header_len= description_event->post_header_len[QUERY_COMPRESSED_EVENT-1]; > + > + tmp += common_header_len; > + > + uint db_len = (uint)tmp[Q_DB_LEN_OFFSET]; > + uint16 status_vars_len= uint2korr(tmp + Q_STATUS_VARS_LEN_OFFSET); > + > + tmp += post_header_len + status_vars_len + db_len + 1; > + > + uint32 un_len = binlog_get_uncompress_len(tmp); > + *newlen = (tmp - src) + un_len; This is one place I would like to see a check on the data in the event. Maybe just that 'len' is larger than 1+(tmp[0]&7). Or maybe simply that len is at least 10, the minimal size for compressed events. > +int row_log_event_uncompress(const Format_description_log_event *description_event, bool contain_checksum, > + const char *src, char* buf, ulong buf_size, bool* is_malloc, > + char **dst, ulong *newlen) > +{ > + uint32 un_len = binlog_get_uncompress_len(tmp); > + *newlen = (tmp - src) + un_len; Another place where I'd like a check against looking outside the event buffer. > +int binlog_buf_uncompress(const char *src, char *dst, uint32 len, uint32 *newlen) > +{ > + if((src[0] & 0x80) == 0) > + { > + return 1; > + } > + > + uint32 lenlen = src[0] & 0x07; > + uLongf buflen = *newlen; > + if(uncompress((Bytef *)dst, &buflen, (const Bytef*)src + 1 + lenlen, len) != Z_OK) Again check on length.

5 12