December 2016 - developers - lists.mariadb.org

[Maria-developers] Windows line endings in MariaDB sources again (
by Otto Kekäläinen 23 Jul '19

23 Jul '19

Hello! While trying to release 10.0.26 for Debian and Ubuntu I noticed git chokes again on Windows line endings. As our .gitattributes states that by default git should clean up the line endings, why do we keep getting new Windows line endings in new code? mariadb-10.0$ cat .gitattributes # Set the default behavior, in case people don't have core.autocrlf set. * text=auto ... storage/connect/mysql-test/connect/std_data/*.txt -text mariadb-10.0.26$ find . | xargs file | grep CRLF ./storage/connect/mysql-test/connect/std_data/boyswin.txt: ASCII text, with CRLF line terminators ./storage/connect/mysql-test/connect/std_data/expenses.txt: ASCII text, with CRLF line terminators ./storage/connect/mysql-test/connect/std_data/emp.txt: ASCII text, with CRLF line terminators ./mysql-test/std_data/loaddata7.dat: ASCII text, with CRLF line terminators ./mysql-test/r/loadxml.result: ASCII text, with CRLF, LF line terminators ./mysql-test/r/perror-win.result: ASCII text, with CRLF, LF line terminators ./mysql-test/r/mysql_binary_mode.result: ASCII English text, with CRLF, LF line terminators ./mysql-test/r/func_regexp_pcre.result: UTF-8 Unicode C++ program text, with CRLF, CR, LF line terminators ./pcre/testdata/grepoutputN: ASCII text, with CRLF, CR, LF line terminators ./pcre/testdata/greppatN4: ASCII text, with CRLF line terminators This leads for me to: dpkg-source: info: building mariadb-10.0 using existing ./mariadb-10.0_10.0.26.orig.tar.gz dpkg-source: info: local changes detected, the modified files are: mariadb-10.0/storage/connect/mysql-test/connect/std_data/boyswin.txt mariadb-10.0/storage/connect/mysql-test/connect/std_data/emp.txt mariadb-10.0/storage/connect/mysql-test/connect/std_data/expenses.txt dpkg-source: error: aborting due to unexpected upstream changes, see /tmp/mariadb-10.0_10.0.26-1.diff.Yzwwhw Which means I have to do extra work with importing the upstream sources.. - Otto

2 4

[Maria-developers] Please review MDEV-11292 Item_func_sp::fix_length_and_dec() does not set collation derivation properly
by Alexander Barkov 20 Nov '17

20 Nov '17

Hello Alexey, Please review a patch for MDEV-11292 (for 10.3). Thanks.

2 2

Re: [Maria-developers] 0310b44: MDEV-10418 Assertion `m_extra_cache' failed in ha_partition::late_extra_cache(uint)
by Sergei Golubchik 21 Feb '17

21 Feb '17

Hi, Alexey! On Dec 20, Alexey Botchkov wrote: > revision-id: 0310b444dba1b9cbd6921963dd3bff689ca23a24 (mariadb-5.5.53-34-g0310b44) > parent(s): f23b41b9b8a30e0e54a1ec7a8923057b0e57e0f5 > committer: Alexey Botchkov > timestamp: 2016-12-20 00:24:20 +0400 > message: > > MDEV-10418 Assertion `m_extra_cache' failed in ha_partition::late_extra_cache(uint) > > The m_extra_prepare_for_update parameter should be set to FALSE > as the query ends. This seems fairly obvious, that it should. But why MySQL doesn't have this bug (despite not resetting m_extra_prepare_for_update)? > diff --git a/sql/ha_partition.cc b/sql/ha_partition.cc > index 15fa7d1..4374354 100644 > --- a/sql/ha_partition.cc > +++ b/sql/ha_partition.cc > @@ -6667,6 +6667,7 @@ int ha_partition::reset(void) > DBUG_ENTER("ha_partition::reset"); > if (m_part_info) > bitmap_set_all(&m_part_info->used_partitions); > + m_extra_prepare_for_update= FALSE; > file= m_file; > do > { Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

2 2

[Maria-developers] MDEV-10142 - bb-10.2-compatibility
by jerome brauge 01 Feb '17

01 Feb '17

Hello, To build this branch on windows (Visual Studio Express 2015), I have to add "#pragma warning( disable : 4099 )" in sql_lex.h. But I don't know if it's the right solution. I have some questions: - I read the comment of Michael Widenius on subtask MDEV-10574. I think that it's important to change behavior of string functions whose returns a string (like rtrim, ltrim, substring, concat, ...) to return null instead of ''. Without this, we have to use UDF instead native functions and it's never good from a performance point of view. Have you made a decision ? - On Oracle, we can easily disable/enable triggers. I found an old request for this on jira : MDEV-7579. As you already manage order_action now, perhaps you can simply use negative orders to indicate disabled triggers. - do you have planning to implement operator || to concat strings ? Best regards, J. Brauge

3 16

[Maria-developers] Please review MDEV-11692 Comparison data type aggregation for pluggable data types
by Alexander Barkov 31 Jan '17

31 Jan '17

Hello Vicențiu, can you please review a patch for MDEV-11692. The patch is for bb-10.2-ext. Thanks!

2 1

[Maria-developers] Please review MDEV-11134 Assertion `fixed' failed in Item::const_charset_converter(THD*, CHARSET_INFO*, bool, const char*)
by Alexander Barkov 23 Jan '17

23 Jan '17

Hello Sergei, can you please review a fix for MDEV-11134. I made it the easiest way, just fixed the assert to cover this special case when Item_param::safe_charset_converter() is called from mysql_prepare_create_table(). But perhaps it can be done in different ways: 1. Do call fix_fields() - Fix mysql_prepare_create_table() to call fix_fields. - Fix Item_param::cleanup() not to set fixed to false. or 2. Sync Item_param::fixed with Item_param::basic_const_item() - Fix Item_param::set_xxx() to set both state=XXX_VALUE and fixed=true. - Fix Item_param::cleanup() not to set fixed to false (like in #1) Thanks!

2 6

[Maria-developers] Please review MDEV-9522 Split sql_select.cc:can_change_cond_ref_to_const into virtual methods in Type_handler
by Alexander Barkov 13 Jan '17

13 Jan '17

Hello Vicențiu, can you please review a patch for MDEV-9522 ? Thanks!

2 3

Re: [Maria-developers] Review for MDEV-10332
by Sergei Golubchik 29 Dec '16

29 Dec '16

Hi, Georg! First, I've looked a bit at EVP_CIPHER_CTX_new/etc. With the CRYPTO_set_mem_functions() one can overwrite malloc/realloc/free functions that OpenSSL is using. We absolutely should do it. At the very least, OpenSSL should use my_malloc/my_realloc/my_free. But a more interesting approach would be to allocate a buffer on the stack and let our openssl m/r/f helpers take the memory from there. And print a warning with a fallback to malloc if that buffer isn't enough. This way openssl won't need mallocs is most cases. Still, I'd suggest to do this in a separate commit. In your main "OpenSSL 1.1 support" commit, just use CRYPTO_set_mem_functions to put OpenSSL through my_malloc/my_realloc/my_free. On Nov 15, Georg Richter wrote: > From 6103b311c4c5740998d342afbba9ac70f8ec5731 Mon Sep 17 00:00:00 2001 > From: Georg Richter <georg(a)mariadb.com> > Date: Tue, 15 Nov 2016 18:39:54 +0100 > Subject: [PATCH] MDEV-10332: Added support for OpenSSL 1.1 > > diff --git a/mysql-test/r/openssl_6975,tlsv10.result b/mysql-test/r/openssl_6975,tlsv10.result > index a65167f..e216ec2 100644 > --- a/mysql-test/r/openssl_6975,tlsv10.result > +++ b/mysql-test/r/openssl_6975,tlsv10.result > @@ -1,11 +1,11 @@ > create user ssl_sslv3@localhost; > -grant select on test.* to ssl_sslv3@localhost require cipher "RC4-SHA"; > +grant select on test.* to ssl_sslv3@localhost require cipher "DES-CBC3-SHA"; did you verify that after your changes all tests also work with older openssl versions and with yassl? > create user ssl_tls12@localhost; > grant select on test.* to ssl_tls12@localhost require cipher "AES128-SHA256"; > TLS1.2 ciphers: user is ok with any cipher > ERROR 2026 (HY000): SSL connection error: sslv3 alert handshake failure > diff --git a/mysys_ssl/my_crypt.cc b/mysys_ssl/my_crypt.cc > index 49bd9af..d72bc72 100644 > --- a/mysys_ssl/my_crypt.cc > +++ b/mysys_ssl/my_crypt.cc > @@ -36,9 +36,20 @@ > class MyCTX > { > public: > - EVP_CIPHER_CTX ctx; > - MyCTX() { EVP_CIPHER_CTX_init(&ctx); } > - virtual ~MyCTX() { EVP_CIPHER_CTX_cleanup(&ctx); ERR_remove_state(0); } > + EVP_CIPHER_CTX *ctx; > +#if defined(HAVE_YASSL) > + MyCTX() { ctx= new EVP_CIPHER_CTX; EVP_CIPHER_CTX_init(ctx); } > + virtual ~MyCTX() { EVP_CIPHER_CTX_cleanup(ctx); delete ctx; ERR_remove_state(0); } > +#else > + MyCTX() {ctx= EVP_CIPHER_CTX_new(); EVP_CIPHER_CTX_init(ctx); } > + virtual ~MyCTX() { EVP_CIPHER_CTX_free(ctx); > +#if OPENSSL_VERSION_NUMBER < 0x10100000L > + ERR_remove_state(0); > +#else > + ERR_clear_error(); > +#endif > + } > +#endif No #ifdef's here, please. YaSSL-specific code from mysys_ssl/yassl.cc makes yassl to look sufficiently like openssl, so that mysys_ssl/my_crypt.cc wouldn't need any ifdefs. Please, keep it that way. And, certainly, don't put any #if's that check openssl version in the code, keep them in the headers or in the beginning of the file (there is, already #define ERR_remove_state(X) there) > virtual int init(const EVP_CIPHER *cipher, int encrypt, const uchar *key, > uint klen, const uchar *iv, uint ivlen) > @@ -72,7 +83,10 @@ class MyCTX_nopad : public MyCTX > { > public: > const uchar *key; > + const uchar *src; > int klen; > + uchar oiv[90]; 90 bytes for an IV? > + uint extra_bytes; > > MyCTX_nopad() : MyCTX() { } > ~MyCTX_nopad() { } > @@ -285,7 +310,11 @@ int my_random_bytes(uchar *buf, int num) > instead of whatever random engine is currently set in OpenSSL. That way > we are guaranteed to have a non-blocking random. > */ > +#if OPENSSL_VERSION_NUMBER < 0x10100000L > RAND_METHOD *rand = RAND_SSLeay(); > +#else > + RAND_METHOD *rand = RAND_OpenSSL(); > +#endif no #if's in the code, please > if (rand == NULL || rand->bytes(buf, num) != 1) > return MY_AES_OPENSSL_ERROR; > return MY_AES_OK; > diff --git a/mysys_ssl/my_md5.cc b/mysys_ssl/my_md5.cc > index 7139ea9..4885033 100644 > --- a/mysys_ssl/my_md5.cc > +++ b/mysys_ssl/my_md5.cc > @@ -58,10 +57,11 @@ static void md5_result(MD5_CONTEXT *context, uchar digest[MD5_HASH_SIZE]) > > #elif defined(HAVE_OPENSSL) > #include <openssl/evp.h> > -typedef EVP_MD_CTX MD5_CONTEXT; > +typedef void MD5_CONTEXT; > > -static void md5_init(MD5_CONTEXT *context) > +static void md5_init(MD5_CONTEXT *ctx) > { > + EVP_MD_CTX *context= (EVP_MD_CTX *)ctx; > EVP_MD_CTX_init(context); > #ifdef EVP_MD_CTX_FLAG_NON_FIPS_ALLOW > /* Ok to ignore FIPS: MD5 is not used for crypto here */ > @@ -123,21 +143,43 @@ void my_md5_multi(uchar *digest, ...) > { > va_list args; > va_start(args, digest); > - > - MD5_CONTEXT md5_context; > +#ifdef HAVE_YASSL > + MD5_CONTEXT *md5_context= new MD5_CONTEXT; > +#else > +#if OPENSSL_VERSION_NUMBER < 0x10100000L > + EVP_MD_CTX *md5_context= (EVP_MD_CTX *)alloca(sizeof(EVP_MD_CTX)); > +#else > + EVP_MD_CTX *md5_context= EVP_MD_CTX_new(); > +#endif > +#endif > const uchar *str; > > - md5_init_fast(&md5_context); > + md5_init_fast(md5_context); > for (str= va_arg(args, const uchar*); str; str= va_arg(args, const uchar*)) > - md5_input(&md5_context, str, va_arg(args, size_t)); > + md5_input(md5_context, str, va_arg(args, size_t)); > > - md5_result(&md5_context, digest); > + md5_result(md5_context, digest); > va_end(args); > +#if defined(HAVE_YASSL) > + delete md5_context; > +#else > +#if OPENSSL_VERSION_NUMBER >= 0x10100000L > + EVP_MD_CTX_free(md5_context); > +#endif > +#endif > } > > size_t my_md5_context_size() > { > +#if defined(HAVE_YASSL) > return sizeof(MD5_CONTEXT); > +#elif OPENSSL_VERSION_NUMBER < 0x10100000L > + return sizeof(EVP_MD_CTX); > +#else > + /* Since OpenSSL 1.1 we can't determine the length of EVP_MD_CTX anymore, > + so we need to return a number (48 bytes + 16 bytes reserved) */ > + return 64; How can you be sure that it's enough? I'd just return sizeof(void*) and in md5_init stored a return value of EVP_MD_CTX_new() there. Until we do that optimization that I mentioned in the beginning of the email. And get rid of #if's please. > +#endif > } Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] TRUE LOCK=NONE for slave
by Sachin Setiya 29 Dec '16

29 Dec '16

Hello everyone! I am thinking of implementing true lock=none for slaves. Currently Alter command is only executed on slave when master commits it. I(thanks marko for suggesting this idea) am thinking of writing BEGIN_DDL_EVENT in binlog after master hits ha_prepare_inplace_alter_table. Then master will write QUERY_EVENT on binlog with actual alter query . On commit/rollback master will write COMMIT_DDL_EVENT/ROLLBACK_DDL_EVENT. On slave there will be pool of threads(configurable global variable), which will apply these DDLs. On receiving BEGIN_DDL_EVENT slave thread will pass the QUERY_EVENT to one of the worker thread. Worker thread will execute until ha_inplace_alter_table. Actual commit_inplace_alter will be called by sql thread. If sql thread recieve some kind of rollback event , then it will somehow signal worker thread to stop executing alter. If none of the worker threads are available then event will be en-queued, then If we received rollback event the we will simply discard event from queue, If we received commit event then SQL thread will synchronously process DDL event. Please let me know what you think. Credits for idea Marko and Lixun. Regards sachin

3 3

Re: [Maria-developers] [Commits] d158f55659c: MDEV-11679 Remove redundant function fsp_header_get_crypt_offset()
by Jan Lindström 29 Dec '16

29 Dec '16

On Wed, Dec 28, 2016 at 5:16 PM, <marko.makela(a)mariadb.com> wrote: > > diff --git a/storage/innobase/fil/fil0crypt.cc b/storage/innobase/fil/ > fil0crypt.cc > index 75efcdfdab0..095b9f706c0 100644 > --- a/storage/innobase/fil/fil0crypt.cc > +++ b/storage/innobase/fil/fil0crypt.cc > @@ -1179,10 +1179,11 @@ fil_crypt_start_encrypting_space( > > /* 3 - compute location to store crypt data */ > byte* frame = buf_block_get_frame(block); > - ulint maxsize = 0; > ut_ad(crypt_data); > - crypt_data->page0_offset = > - fsp_header_get_crypt_offset(page_size, &maxsize); > + crypt_data->page0_offset = FSP_HEADER_OFFSET > + + fsp_header_get_encryption_offset(page_size); > + const ulint maxsize = page_size.logical() > + - crypt_data->page0_offset - FIL_PAGE_DATA_END; > > This I do not like, why fsp_header_get_encryption_offset does not calculate the actual offset so that there is no need for adjustments? Now every place where you call this you need to add FSP_HEADER_OFFSET to returned offset. Similarly for maxsize actually meaning free space on the page. This is naturally technically correct, but is it maintainable? R: Jan

1 0