- developers - lists.mariadb.org

[Maria-developers] Query in size
by Kavneet Kaur 04 Sep '14

04 Sep '14

Hi all, I just when through a revision~maria-captains/mariadb-java-client/trunk : /src/test/java/org/mariadb/jdbc/BigQueryTest.java (revision 516)Big Query activated in this code contains a character array of size 20000000. This is something that can be modified. All big queries possible cannot occupy this space. This is sort of leading to more of wastage of memory. Can anyone help me understand why this size has been selected when the same could be done at a size (50-90)% lesser. Regards Kavneet Kaur

2 1

Re: [Maria-developers] Interest in contributing to MariaDB
by Djimeli Konrad 03 Sep '14

03 Sep '14

Thanks for the reply Roberto, i was beginning to get confused because the code i got from github was on c/c++. On 3 September 2014 21:13, Djimeli Konrad <djkonro35(a)gmail.com> wrote: > Thanks for the reply Roberto, i was beginning to get confused because the > code i got from github was on c/c++. > > > On 3 September 2014 20:39, Roberto Spadim <roberto(a)spadim.com.br> wrote: > >> ???? java? no guy, mariadb database source code is C/C++ check : >> >> github : https://github.com/MariaDB/server >> and issue tracker: https://mariadb.atlassian.net >> there's mariadb knowledge: https://mariadb.com/kb/en/ >> there're some book about mariadb/mysql internall and mysql internall >> documentation: http://dev.mysql.com/doc/internals/en/ >> >> java is a connect, you can check more at mariadb page: >> https://mariadb.com/products/connectors-plugins >> >> >> 2014-09-03 17:15 GMT-03:00 Kavneet Kaur <kavneetk(a)outlook.com>: >> >> Hi Djimeli, >>> >>> As far as I have studied about the code it's in java. If you wish to >>> begin it from basic of database, you can go for installation. You should go >>> through documentations also. Explore your interest and surf as much as you >>> can. >>> >>> All the best! >>> >>> > Date: Wed, 3 Sep 2014 15:35:24 +0000 >>> > From: djkonro35(a)gmail.com >>> > To: maria-developers(a)lists.launchpad.net >>> > Subject: [Maria-developers] Interest in contributing to MariaDB >>> >>> > >>> > Hello >>> > >>> > My name is Djimeli Konrad and i am primarily a c/c++ >>> > programmer with some experience in php and mysql. I am new to open >>> > source and would like to contribute to mariadb so i wish i could get >>> > some pointers on how to begin. >>> > >>> > Thanks >>> > konrad >>> > >>> > _______________________________________________ >>> > Mailing list: https://launchpad.net/~maria-developers >>> > Post to : maria-developers(a)lists.launchpad.net >>> > Unsubscribe : https://launchpad.net/~maria-developers >>> > More help : https://help.launchpad.net/ListHelp >>> >>> _______________________________________________ >>> Mailing list: https://launchpad.net/~maria-developers >>> Post to : maria-developers(a)lists.launchpad.net >>> Unsubscribe : https://launchpad.net/~maria-developers >>> More help : https://help.launchpad.net/ListHelp >>> >>> >> >> >> -- >> Roberto Spadim >> SPAEmpresarial >> Eng. Automação e Controle >> > >

1 0

[Maria-developers] Interest in contributing to MariaDB
by Djimeli Konrad 03 Sep '14

03 Sep '14

Hello My name is Djimeli Konrad and i am primarily a c/c++ programmer with some experience in php and mysql. I am new to open source and would like to contribute to mariadb so i wish i could get some pointers on how to begin. Thanks konrad

3 2

[Maria-developers] New MariaDB builds: regular trusty and special kvm-deb-sid-pbuilder-amd and (Re: MariaDB)
by Otto Kekäläinen 01 Sep '14

01 Sep '14

Hello! 2014-02-04 20:41 GMT+02:00 Daniel Bartholomew <dbart(a)mariadb.com>: > On Tue, Feb 4, 2014 at 5:39 AM, Kristian Nielsen > <knielsen(a)knielsen-hq.org> wrote: >>> And while you are at it, please also consider setting up an instance >>> 'kvm-deb-sid-pbuilder-amd64' that provides a Debian unstable >> >> Daniel, can you prepare new sets of VMs for Ubuntu trusty and Debian sid? >> Then I can try to setup Buildbot to do these builds. > > Ok, will do. I should have them up and running before the end of the week. Any updates on this?

3 22

Re: [Maria-developers] [Commits] eecac19: MDEV-6513 deprecate engine_condition_pushdown value of the @@optimizer_switch
by Oleksandr Byelkin 28 Aug '14

28 Aug '14

Hi! Everything is OK. Except one small note in two places. But it is OK to push as is. 26.08.2014 17:17, serg(a)mariadb.org пишет: > revision-id: eecac1976469d513062a0741005afc357b897871 > parent(s): f1c1c04a34f39bcd6622b9886fb089dd41e51103 > committer: Sergei Golubchik > branch nick: maria > timestamp: 2014-08-25 23:13:37 +0200 > message: > > MDEV-6513 deprecate engine_condition_pushdown value of the @@optimizer_switch > > * ignore the OPTIMIZER_SWITCH_ENGINE_CONDITION_PUSHDOWN bit > * issue a deprecation warning on 'engine_condition_pushdown=on' > * remove unused remains of the old pre-5.5 engine_condition_pushdown variable > > --- [skip] > --- a/sql/handler.h > +++ b/sql/handler.h > @@ -197,17 +197,11 @@ enum enum_alter_inplace_result { > #define HA_RECORD_MUST_BE_CLEAN_ON_WRITE (1ULL << 41) > > /* > - Table condition pushdown must be performed regardless of > - 'engine_condition_pushdown' setting. > - > - This flag is aimed at storage engines that come with "special" predicates > - that can only be evaluated inside the storage engine. > - For example, when one does > - select * from sphinx_table where query='{fulltext_query}' > - then the "query=..." condition must be always pushed down into storage > - engine. > + This storage engine supports condition pushdown > */ > -#define HA_MUST_USE_TABLE_CONDITION_PUSHDOWN (1ULL << 42) > +#define HA_CAN_TABLE_CONDITION_PUSHDOWN (1ULL << 42) > +/* old name for the same flag */ > +#define HA_MUST_USE_TABLE_CONDITION_PUSHDOWN HA_CAN_TABLE_CONDITION_PUSHDOWN > > /** > The handler supports read before write removal optimization > diff --git a/sql/mysqld.cc b/sql/mysqld.cc > index 57d2cc6..fe52a85 100644 > --- a/sql/mysqld.cc > +++ b/sql/mysqld.cc > @@ -8574,18 +8574,6 @@ mysqld_get_one_option(int optid, > } > break; > #endif /* defined(ENABLED_DEBUG_SYNC) */ > - case OPT_ENGINE_CONDITION_PUSHDOWN: > - /* > - The last of --engine-condition-pushdown and --optimizer_switch on > - command line wins (see get_options(). > - */ > - if (global_system_variables.engine_condition_pushdown) > - global_system_variables.optimizer_switch|= > - OPTIMIZER_SWITCH_ENGINE_CONDITION_PUSHDOWN; > - else > - global_system_variables.optimizer_switch&= > - ~OPTIMIZER_SWITCH_ENGINE_CONDITION_PUSHDOWN; > - break; > case OPT_LOG_ERROR: > /* > "No --log-error" == "write errors to stderr", > @@ -8964,10 +8952,6 @@ static int get_options(int *argc_ptr, char ***argv_ptr) > &extra_connection_count); > #endif > > - global_system_variables.engine_condition_pushdown= > - MY_TEST(global_system_variables.optimizer_switch & > - OPTIMIZER_SWITCH_ENGINE_CONDITION_PUSHDOWN); > - > opt_readonly= read_only; > > /* > diff --git a/sql/mysqld.h b/sql/mysqld.h > index 21c020b..7a0a75c 100644 > --- a/sql/mysqld.h > +++ b/sql/mysqld.h > @@ -554,7 +554,6 @@ enum options_mysqld > OPT_DEBUG_SYNC_TIMEOUT, > OPT_DELAY_KEY_WRITE_ALL, > OPT_DEPRECATED_OPTION, > - OPT_ENGINE_CONDITION_PUSHDOWN, > OPT_IGNORE_DB_DIRECTORY, > OPT_ISAM_LOG, > OPT_KEY_BUFFER_SIZE, > diff --git a/sql/records.cc b/sql/records.cc > index 1b230c4..cbcdb36 100644 > --- a/sql/records.cc > +++ b/sql/records.cc > @@ -287,8 +287,8 @@ bool init_read_record(READ_RECORD *info,THD *thd, TABLE *table, > thd->variables.read_buff_size); > } > /* Condition pushdown to storage engine */ > - if ((thd->variables.optimizer_switch & > - OPTIMIZER_SWITCH_ENGINE_CONDITION_PUSHDOWN) && > + if ((table->file->ha_table_flags() & > + HA_MUST_USE_TABLE_CONDITION_PUSHDOWN) && > select && select->cond && > (select->cond->used_tables() & table->map) && > !table->file->pushed_cond) > diff --git a/sql/sql_class.h b/sql/sql_class.h > index f96110e..c42a87a 100644 > --- a/sql/sql_class.h > +++ b/sql/sql_class.h > @@ -597,7 +597,6 @@ typedef struct system_variables > my_bool tx_read_only; > my_bool low_priority_updates; > my_bool query_cache_wlock_invalidate; > - my_bool engine_condition_pushdown; > my_bool keep_files_on_create; > > my_bool old_mode; > diff --git a/sql/sql_priv.h b/sql/sql_priv.h > index 5dc1918..9578c2f 100644 > --- a/sql/sql_priv.h > +++ b/sql/sql_priv.h > @@ -199,7 +199,7 @@ template <class T> bool valid_buffer_range(T jump, > #define OPTIMIZER_SWITCH_INDEX_MERGE_SORT_UNION (1ULL << 2) > #define OPTIMIZER_SWITCH_INDEX_MERGE_INTERSECT (1ULL << 3) > #define OPTIMIZER_SWITCH_INDEX_MERGE_SORT_INTERSECT (1ULL << 4) > -#define OPTIMIZER_SWITCH_ENGINE_CONDITION_PUSHDOWN (1ULL << 5) > +#define deprecated_ENGINE_CONDITION_PUSHDOWN (1ULL << 5) > #define OPTIMIZER_SWITCH_INDEX_COND_PUSHDOWN (1ULL << 6) > #define OPTIMIZER_SWITCH_DERIVED_MERGE (1ULL << 7) > #define OPTIMIZER_SWITCH_DERIVED_WITH_KEYS (1ULL << 8) > diff --git a/sql/sql_select.cc b/sql/sql_select.cc > index 05c88a5..f061b51 100644 > --- a/sql/sql_select.cc > +++ b/sql/sql_select.cc > @@ -9561,10 +9561,8 @@ make_join_select(JOIN *join,SQL_SELECT *select,COND *cond) > if (tab->table) > { > tab->table->file->pushed_cond= NULL; > - if (((thd->variables.optimizer_switch & > - OPTIMIZER_SWITCH_ENGINE_CONDITION_PUSHDOWN) || > - (tab->table->file->ha_table_flags() & > - HA_MUST_USE_TABLE_CONDITION_PUSHDOWN)) && > + if ((tab->table->file->ha_table_flags() & > + HA_MUST_USE_TABLE_CONDITION_PUSHDOWN) && It might be that it is better to use 'new' name for the flag in the lines we have changed. > !first_inner_tab) > { > COND *push_cond= > @@ -23598,10 +23596,8 @@ void JOIN_TAB::save_explain_data(Explain_table_access *eta, table_map prefix_tab > { > const COND *pushed_cond= tab->table->file->pushed_cond; > > - if (((thd->variables.optimizer_switch & > - OPTIMIZER_SWITCH_ENGINE_CONDITION_PUSHDOWN) || > - (tab->table->file->ha_table_flags() & > - HA_MUST_USE_TABLE_CONDITION_PUSHDOWN)) && > + if ((tab->table->file->ha_table_flags() & > + HA_MUST_USE_TABLE_CONDITION_PUSHDOWN) && The same. > pushed_cond) > { > eta->push_extra(ET_USING_WHERE_WITH_PUSHED_CONDITION); > diff --git a/sql/sys_vars.cc b/sql/sys_vars.cc > index 7f7c725..db764d7 100644 > --- a/sql/sys_vars.cc > +++ b/sql/sys_vars.cc > @@ -2181,13 +2181,15 @@ export const char *optimizer_switch_names[]= > "exists_to_in", > "default", NullS > }; > -/** propagates changes to @@engine_condition_pushdown */ > static bool fix_optimizer_switch(sys_var *self, THD *thd, > enum_var_type type) > { > SV *sv= (type == OPT_GLOBAL) ? &global_system_variables : &thd->variables; > - sv->engine_condition_pushdown= > - MY_TEST(sv->optimizer_switch & OPTIMIZER_SWITCH_ENGINE_CONDITION_PUSHDOWN); > + if (sv->optimizer_switch & deprecated_ENGINE_CONDITION_PUSHDOWN) > + push_warning_printf(current_thd, Sql_condition::WARN_LEVEL_WARN, > + ER_WARN_DEPRECATED_SYNTAX_NO_REPLACEMENT, > + ER(ER_WARN_DEPRECATED_SYNTAX_NO_REPLACEMENT), > + "engine_condition_pushdown=on"); > return false; > } > static Sys_var_flagset Sys_optimizer_switch( > diff --git a/storage/spider/mysql-test/spider/bg/r/spider_fixes.result b/storage/spider/mysql-test/spider/bg/r/spider_fixes.result > index 6db61fa..3033586 100644 > [skip]

2 1

Re: [Maria-developers] Learning about MariaDB, Getting Started
by Kavneet Kaur 26 Aug '14

26 Aug '14

Hi, James, I am aiming to become am interested in data security and reducing query time by increasing efficiency and computational speed. Coding and optimizing codes are also cup of my tea. How can I contribute this in MariaDB as it's the only open source database software that attracted me. Regards,Kavneet Kaur Date: Fri, 22 Aug 2014 04:44:45 -0700 From: actionmessage(a)yahoo.com Subject: Re: [Maria-developers] Learning about MariaDB, Getting Started To: kavneetk(a)outlook.com; maria-developers(a)lists.launchpad.net; ashish1294(a)gmail.com Hi Kavneet and Ashish. Depends on what you're interested in working on, but here's some suggestions.Direct link to currently open worklogs:https://mariadb.atlassian.net/browse/MDEV-6629?filter=-4Three easy MDEVs and some other areas to work on:- MDEV-6584 - not using tcp_fastopen- MDEV-6555 - sql_log_bin should be SESSION_ONLY- MDEV-5696 - Graceful MariaDB shutdown at Windows restart- test and see if fixes are needed for SSL, Galera, Cassandra, GTID, IPv6 or partition features. "Howto Add a New Command to the MySQL Server" Tutorial http://www.jebriggs.com/blog/2013/05/howto-add-a-new-command-to-the-mysql-s…, James. From: Kavneet Kaur <kavneetk(a)outlook.com> To: "maria-developers(a)lists.launchpad.net" <maria-developers(a)lists.launchpad.net> Sent: Thursday, August 21, 2014 11:49 PM Subject: [Maria-developers] Learning about MariaDB Hi! I am highly interested in subject of database. I am also interested in becoming a research developer. Worked on MS SQL. Want to try MariaDB. May I get the guidance about this product as i'm totally new to it. Is there any specific way to begin? _______________________________________________Mailing list: https://launchpad.net/~maria-developersPost to : maria-developers(a)lists.launchpad.netUnsubscribe : https://launchpad.net/~maria-developersMore help : https://help.launchpad.net/ListHelp

3 6

[Maria-developers] Please review: 10% performance improvement for INSERT INTO t1 (blob) VALUES ('long value')
by Alexander Barkov 26 Aug '14

26 Aug '14

Hi, I think I found and fixed a bottleneck in how string literals are processed. The attached patch improves INSERT performance for strings, and especially for long blob and text values. Insertion time for 10MByte blob 1024 times (10GByte of data total) reduced on my laptop from 300 seconds to 274 seconds, which gives near 10% pure time improvement. What the patch changes: Before the patch : 1. The function get_text() is sql_lex.cc allocated an unescaped copy of every string literal (unescaping backslashes and double quotes, if any). Strangely, copying happened even if there were no really any escapes in the string. 2, The syntax parser in sql_yacc.yy created Item_string using the new unescaped buffer. Furthermore, in case of a multi-byte connection character set, the constructor for Item_string made another loop on the unescaped buffer, to calculate length in characters, which is needed to set max_length properly. After the patch: Item is now created using directly the SQL fragment, without making a copy. Even for escaped values!!! Length in characters is also calculated during the very first pass in get_text(), without any additional loops in the Item constructors. Unescaping is now done in the very end, when the value is actually needed: - Either in Field::store(). This happens when the string value is used in: INSERT INTO t1 VALUES('string'); Unescaping is done directly to the Field buffer, without any intermediary temporary storage. - Or in val_str(), if the string value is used elsewhere (in SELECT list, functions, operators, etc). The unescaped value is cached, to make sure that unescaping is done only one time, even if the value is needed for multiple rows, like in: SELECT * FROM t1 WHERE a='string with backslash or quote escapes'; The patch also fixes a few bugs related to string conversion/copying (some of them were reported on JIRA earlier, and some of them I found while working on the patch). Please also find "a.c" attached. I used it to test the improvement. I used a RelWithDebInfo compiled binary for tests. Note, "perf" and "valgrind --tool=cachegrind" also demonstrate a good improvement (reduced CPU and memory usage). I can send their statistics in a separate letter, if anyone is interested. Please review the patch. Thanks.

1 0

[Maria-developers] List affecting CVEs at mariadb.com
by Otto Kekäläinen 26 Aug '14

26 Aug '14

Hello Daniel (and others), The usual changelogs[1] and relese notes[2] don't seem to contain CVE identifiers, or even a separate section about fixed security issues For the downstream security teams if would be reassuring if the CVE information would be easily available. For example if the security teams follow the CVE news and they for example know or suspect that CVE-2014-4260 affects MariaDB, it would be nice to see if it is already fixed or what version it was fixed in, so downstream security teams can organize and prioritize their patching and release work. Do you have any suggestion how to address this? Should we maybe have a separate wiki page, e.g. https://mariadb.com/kb/en/mariadb/cve/ that would have a table of CVEs and MariaDB 5.5/10.0/Galera versions where they are fixed? Or should just each release notes include a subsection "Security" with these details? Something else? Of course we need to consider timing issues, e.g. a security issue fixed in MariaDB might get publicity and a CVE only later when Oracle releases it, and in those cases old release notes need to be upgraded to include the CVE identifiers. [1] https://mariadb.com/kb/en/mariadb-10013-changelog/ [2] https://mariadb.com/kb/en/mariadb-10013-release-notes/ (To be exact, googling for 'mariadb cve' does give one hit at mariadb.com in the 5.3.12 release notes)

3 3

Re: [Maria-developers] [Commits] Rev 4271: MDEV-6601 Assertion `!thd->in_active_multi_stmt_transa ction() || thd->in_multi_stmt_transaction_mode()' failed on executing a stored procedure with commit in lp:~maria-captains/maria/5.5
by Alexey Botchkov 25 Aug '14

25 Aug '14

Hi, Sergei. I'd add SERVER_STATUS_LAST_ROW_SENT to the status_backup_mask. It's not necessary as I don't see it affecting anything, but just to make a sign that we don't want this status to go higher out of the call. Best regards. HF 25.08.2014 10:49, Sergei Golubchik wrote: > At lp:~maria-captains/maria/5.5 > > ------------------------------------------------------------ > revno: 4271 > revision-id: sergii(a)pisem.net-20140825064904-h8isjhv963kbxlzy > parent: knielsen(a)knielsen-hq.org-20140813134639-wk760plnzg5wu4x8 > fixes bug: https://mariadb.atlassian.net/browse/MDEV-6601 > committer: Sergei Golubchik <sergii(a)pisem.net> > branch nick: 5.5 > timestamp: Mon 2014-08-25 08:49:04 +0200 > message: > MDEV-6601 Assertion `!thd->in_active_multi_stmt_transa ction() || thd->in_multi_stmt_transaction_mode()' failed on executing a stored procedure with commit > > Don't restore the whole of thd->server_status after a routine invocation, > only restore SERVER_STATUS_CURSOR_EXISTS, as --ps --embedded needs. > In particular, don't restore SERVER_STATUS_IN_TRANS. > === modified file 'mysql-test/r/sp-bugs.result' > --- a/mysql-test/r/sp-bugs.result 2014-01-24 12:50:18 +0000 > +++ b/mysql-test/r/sp-bugs.result 2014-08-25 06:49:04 +0000 > @@ -268,3 +268,9 @@ END $$ > CALL test_5531(1); > DROP PROCEDURE test_5531; > DROP TABLE t1; > +create procedure sp() begin > +commit; > +end| > +start transaction; > +call sp(); > +drop procedure sp; > > === modified file 'mysql-test/t/sp-bugs.test' > --- a/mysql-test/t/sp-bugs.test 2014-01-24 12:50:18 +0000 > +++ b/mysql-test/t/sp-bugs.test 2014-08-25 06:49:04 +0000 > @@ -285,3 +285,16 @@ DELIMITER ;$$ > CALL test_5531(1); > DROP PROCEDURE test_5531; > DROP TABLE t1; > + > +# > +# MDEV-6601 Assertion `!thd->in_active_multi_stmt_transa ction() || thd->in_multi_stmt_transaction_mode()' failed on executing a stored procedure with commit > +# > +delimiter |; > +create procedure sp() begin > + commit; > +end| > +delimiter ;| > +start transaction; > +call sp(); > +drop procedure sp; > + > > === modified file 'sql/sp_head.cc' > --- a/sql/sp_head.cc 2014-04-15 13:17:47 +0000 > +++ b/sql/sp_head.cc 2014-08-25 06:49:04 +0000 > @@ -1224,6 +1224,7 @@ sp_head::execute(THD *thd, bool merge_da > Item_change_list old_change_list; > String old_packet; > uint old_server_status; > + const uint status_backup_mask= SERVER_STATUS_CURSOR_EXISTS; > Reprepare_observer *save_reprepare_observer= thd->m_reprepare_observer; > Object_creation_ctx *saved_creation_ctx; > Warning_info *saved_warning_info; > @@ -1358,7 +1359,7 @@ sp_head::execute(THD *thd, bool merge_da > It is probably safe to use same thd->convert_buff everywhere. > */ > old_packet.swap(thd->packet); > - old_server_status= thd->server_status; > + old_server_status= thd->server_status & status_backup_mask; > > /* > Switch to per-instruction arena here. We can do it since we cleanup > @@ -1488,7 +1489,7 @@ sp_head::execute(THD *thd, bool merge_da > thd->spcont->pop_all_cursors(); // To avoid memory leaks after an error > > /* Restore all saved */ > - thd->server_status= old_server_status; > + thd->server_status= (thd->server_status & ~status_backup_mask) | old_server_status; > old_packet.swap(thd->packet); > DBUG_ASSERT(thd->change_list.is_empty()); > old_change_list.move_elements_to(&thd->change_list); > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits

1 0

[Maria-developers] Getting Started
by Ashish Kedia 23 Aug '14

23 Aug '14

Hi all, I am 3rd Year undergraduate student majoring in Information Technology from India. I am new to MariaDB however I wish to contribute. I am well acquainted with C++ and other tools required. Can someone help me find small tasks/bugs/projects to get started with ? Regards, Ashish Kedia, 3rd Year Undergraduate Student, Dept. of Information Technology, National Institute of Technology Karnataka, Surathkal

3 2