- developers - lists.mariadb.org

[Maria-developers] I wonder what elements add to CMakeLists.txt
by kyung nam Kim 23 Mar '15

23 Mar '15

Dears. I'm a newer in maria develper community. I'm develping a simple plugin for monitoring. If I access from other module to my plugin interface, I wonder how and what elements add to CMakeLists.txt. for example, My plugin is daemon type and myplugin.h has a definition of public methodA. I want to use methodA in my plugin from sql/sql_parse.cc . I think that information of library is needed at build time. ( -L, -l option) I don't know how to add that elements to CMakeLists.txt. plz,let me know. Thanks in advance.

2 5

Re: [Maria-developers] [server] Execute sync_with_master before SELECT in rpl_gtid_basic.test (#31)
by Kristian Nielsen 23 Mar '15

23 Mar '15

f4rnham <notifications(a)github.com> writes: > Usually slave synchronized on its own before select, but not always. Those two lines were probably swapped by accident > You can view, comment on, or merge this pull request online at: Indeed, silly mistake... This was already fixed in 10.0, so it will be merged to 10.1 that way. But thanks for looking into this anyway, it's good to be aware of these kinds of races in replication tests, they are quite common. - Kristian.

1 0

Re: [Maria-developers] [JIRA] (MDEV-7557) Option to put into slow log any transaction that was retried in parallel replication
by Kristian Nielsen 23 Mar '15

23 Mar '15

"Sergei Golubchik (JIRA)" <jira(a)mariadb.atlassian.net> writes: >> Key: MDEV-7557 >> URL: https://mariadb.atlassian.net/browse/MDEV-7557 >> In optimistic parallel replication (MDEV-6676), we speculatively apply >> transactions in parallel, even if they might conflict. If too many conflicts >> happen, excessive retry might degrade performance. >> The DBA needs some way to monitor for such problems. One good way would seem >> to have an option to add to the slow query log any transactions that were >> retried due to parallel replication conflicts. Possibly with extra information >> such as duration of transaction and similar. > I'm not sure that slow log is a good match for this kind of events Right. So do you have a suggestion for a better way? I was thinking on --log-queries-not-using-indexes, which similarly uses the slow query log to expose certain kinds of problematic queries. So queries that cause deadlocks could be another useful filter for the slow log (it should then probably be for all queries that deadlock, not just replication queries). In non-parallel replication, we put deadlock errors as warnings in the error log. But I disabled that for parallel replication. In non-parallel replication, you normally do not expect any deadlocks (since there is only one thread, how can it deadlock?). But in parallel replication, deadlocks are generally unavoidable, and not harmful. And DBAs do not like noise in the error logs that does not indicate a problem and cannot be removed. One option might be to log deadlocks as INFO level in the error log, and then users can temporarily raise verbosity level to see them? Another option could be to add an option to enable/disable logging deadlocks as warnings in parallel replication. (Though probably you will think as me, that this feels a bit like too ad-hoc an option to add.) Any other ideas? - Kristian.

2 1

[Maria-developers] GSoC 2015: Indexes on Virtual Columns and Array Based UDFs
by Richa Sehgal 20 Mar '15

20 Mar '15

Dear mentors, I am currently pursuing Master's in University of Illinois at Urbana Champaign, USA and completed by B.Tech from Indian Institute of Technology - Delhi (IIT- Delhi) This is regarding GSoC 2015. I am really interested in databases, and I was very excited to see all these projects listed here. The exciting part was that some of the projects are really “hard” as in they have challenged the database community since a long time, and thus it would be very interesting to solve some of these challenges as part of GSoC. I want to discuss 2 projects: A. Indexes on virtual columns Materialization gives us two things: 1. A name to the column which we can use in queries 2. A formal "regular" column which is stored and indexed in the regular fashion - Disadvantage: Extra memory requirements for the materialized column. My initial thoughts on this project are the following: We do need the name of the column which can be used to query. So maybe we can expose a command such as: create virtual_index <name> on <column_name> <expression> What this would do would run a regular query which evaluates expressions (like in WHERE clause) and the feed the result into the indexer. This index can then be stored in the regular fashion. B. Having UDFs returning an array/set There are three approaches that I can think of: 1. Supporting array/set as native datatype inside MariaDB (like int64, double, etc) - This might be hard and touches all levels of stack. 2. Have the array/set pass in serialized form to the above node of query execution and have appropriate deserializer when we want to interpret the result - Coming up with ser/deser strategy might be tough and this would be expensive too. 3. The query execution would be in a Tree structure where each node must be exposing functions like init(), next(), read(int col_index), etc. Maybe we can use this to emulate the evaluation of UDF against row. I think this is the suggestion that is listed in the project. I would like to get some direction on this from the mentors. I would like to discuss these and then decide on one of them. Am I approaching this in the right direction? Can you please point me to the next steps? Thanks Richa

3 9

[Maria-developers] Tasks for newcomer
by Martin Kaluznik 19 Mar '15

19 Mar '15

Hello, I am student interested in GSoC project Automatic provisioning of slave ( https://mariadb.atlassian.net/browse/MDEV-7502 ). Who would like to contribute to codebase. I will appreciate if somebody could suggest me something to work on. I found something really small for myself (https://mariadb.atlassian.net/browse/MDEV-7780 ) and implemented it in most basic way. https://github.com/f4rnham/server/commits/MDEV-7780 Config entry for case specified in report would be fake_version=5.5.25 It will probably require test case too, and I am not sure if this feature will be actually safe, it may create some inconsistencies. I have never worked with bzr before, but if it will be required, I will look into that. Thanks for feedback, Martin Kaluznik ( martin.kaluznik(a)gmail.com ). I can be also found on IRC under nick Farnham

2 1

Re: [Maria-developers] 1cda762: MDEV-6069: Remove old logic for 3.23-to-higher upgrades from upgrade SQL scripts
by Sergei Golubchik 19 Mar '15

19 Mar '15

Hi, Nirbhay! On Mar 17, Nirbhay Choubey wrote: > revision-id: 1cda762d3f1132e6696e1c451435746db0a1420a > parent(s): b0542b78c866fb32383f759914b10f060d0e14cd > committer: Nirbhay Choubey > branch nick: server > timestamp: 2015-03-17 11:20:49 -0400 > message: > > MDEV-6069: Remove old logic for 3.23-to-higher upgrades from upgrade SQL scripts > > Fix for failing tests. > > * Update mysql_system_tables_fix.sql to makeup the differences in system > tables in 5.1.17 (main.system_mysql_db_fix50117) I don't quite understand that. Why these ALTER TABLEs were not needed before? > * Removed system_mysql_db tests for versions 5.0.30 & 4.1.23. ok Regards, Sergei

3 3

[Maria-developers] [GSoC] MDEV-7502 Automatic provisioning of slave
by Martin Kaluznik 18 Mar '15

18 Mar '15

Hello, I am student interested in GSoC project Automatic provisioning of slave ( https://mariadb.atlassian.net/browse/MDEV-7502 ). I have read current code and put together my idea of how it could be implemented. Please correct me if there is better solution for some steps or I just misunderstood something. Slave starts regular sql thread and something really similar to current io thread (maybe best to begin with will be copy of handle_slave_io and related functions, it may still end with some duplicate code, which at the end can be moved to new common functions). Slave will receive gtids of current master as response to copy-modified version of COM_BINLOG_DUMP, they will mark position from which binlogs will be sent to slave. Master proceeds with building of list containing what needs to be sent to slave. Here exact implementation will depend on whether the master itself will block any DDL queries during provisioning process or not. If he will (probably won’t – it’s not necessary and it will just slow him down), we will have guaranteed that internal containers with databases/tables won’t change and they can be just iterated through safely (not true for procedure/trigger containers?). In other case, it will be probably required to prefetch list of databases/tables to copy to slave (it can be done lazily with tables/triggers/procedures/…, holding only list of tables/… for database which is currently being processed). In the second case, slave will have to detect DDL event and interrupt connection. The range scans on primary key mentioned in description, I assume it would look like SELECT * FROM tbl WHERE pk > @lastChunkKey ORDER BY pk LIMIT @chunkSize; just done with internal functions more efficiently, and the result will be converted to Rows_log_event which will be sent to slave. To restrictions I would add at least one more, provisioning cannot be continued if master is restarted during process (or can we actually store information about provisioning process in database?). Another possible problem could be slave reconnect. I found kill_zombie_dump_threads function, in this case, new thread could take information about provisioning progress from old thread and with help of fake gtids bundled with chunks somehow figure out which chunk was last successfully received by slave before disconnect. It would require additional memory to store few (or more?) most recently sent chunks and wouldn't guarantee success. Great way to solve this would be, to be able to map gtid sequence number to exact chunk of data, but I can’t see how it can be done without assuming something about maximum amount of rows in table / number of tables / triggers / … Thanks for feedback, Martin Kaluznik ( martin.kaluznik [at] gmail.com ). I can be also found on IRC under nick Farnham

2 1

Re: [Maria-developers] [Commits] 2bdbfd3: Fix assertion failure seen on Buildbot win32-debug
by Sergei Golubchik 18 Mar '15

18 Mar '15

Hi, Jan! On Mar 18, Jan Lindström wrote: > revision-id: 2bdbfd334bba93f75b2952e8e76f20ed29fb2bd4 > parent(s): c14d9c21f0a6974d9909e00a23522114caad3bfc > committer: Jan Lindström > branch nick: 10.0-git > timestamp: 2015-03-18 12:18:39 +0200 > message: > > Fix assertion failure seen on Buildbot win32-debug > > There is a bug in Visual Studio 2010 > Visual Studio has a feature "Checked Iterators". In a debug build, every > iterator operation is checked at runtime for errors, e g, out of range. > Disable this "Checked Iterators" for Windows and Debug if defined. This comment, > +/* There is a bug in Visual Studio 2010 > +Visual Studio has a feature "Checked Iterators". In a debug build, every > +iterator operation is checked at runtime for errors, e g, out of range. > +Disable this "Checked Iterators" for Windows and Debug if defined. > +*/ and this comment don't explain what bug is it. They create an impression that Visual Studio has run-time checks for an iterator running out of range. And that our code fails this check (because the iterator runs out of range). And that your "fix" is to disable the check instead of fixing the buggy out-of-range iterator. I'm not saying that's what happens, I'm saying that your comment suggests that this happens, because you didn't describe what was the actual bug. A better comment should say that our iterator is correct, but a buggy check erroneously marks it as out-of-range. It should explain what iterator is it, exactly, why it's correct, etc. Could you please fix the comment please? (or the iterator, whatever should be fixed) > +#ifdef UNIV_DEBUG > +#ifdef __WIN__ > +#ifdef _ITERATOR_DEBUG_LEVEL > +#define _ITERATOR_DEBUG_LEVEL 0 this will produce a warning "macro _ITERATOR_DEBUG_LEVEL is redefined" > +#endif /* _ITERATOR_DEBUG_LEVEL */ > +#endif /* __WIN__*/ > +#endif /* UNIV_DEBUG */ Regards, Sergei

1 0

Re: [Maria-developers] [Commits] 97037da: Replace static usage of AES_CTR with current encryption algorithm.
by Jeremy Cole 18 Mar '15

18 Mar '15

Howdy Jan, > But the whole encryption_algorithm stuff seems not well thought out in any >> case. >> > > This was introduced because we can't assume that all installations will > install openSSL especially on Windows, thus encryption should work > with bundled yaSSL also. Do you have some better suggestion ? > Yes, I assumed so, but the current implementation is lazy and user-hostile. And, to me, having a single global variable controlling "the" encryption algorithm (for what, exactly?) is weird. Presumably if it makes sense to have it changeable, it also makes sense to set different algorithms for different objects. At a minimum, IMHO: store the encryption_algorithm selection with each encrypted object so that data can be migrated in the future. Then, implement sufficient support in the various necessary places so that it can be safely changed. Currently as far as I can tell it is globally set once, and impossible to change in the future. This could be done via stealing some bits from either the stored encryption scheme (1 byte) or key version (4 bytes) or adding a new encryption algorithm field which is stored alongside those everywhere (this is probably the best strategy IMHO, but takes a bit of thought). Alternatively as an absolute (and even lower) minimum: ensure that changing it once the database is initialized produces a loud and very clear error message to the user. Regards, Jeremy

1 0

[Maria-developers] Fwd: Port InnoDB memcached interface to MariaDB (GSOC 2015)
by Aalekh Nigam 18 Mar '15

18 Mar '15

Hello, I am Aalekh Nigam, 3rd Year Undergraduate student at JIIT Noida, India. I would like to propose my participation for Port InnoDB Memcache Interface to MariaDB (https://mariadb.atlassian.net/browse/MDEV-4674) as GSOC 2015 Student. Although I have been continuously involved with discussion regarding the project with the assigned mentor (Colin Charles), but would also like to propose my raw architecture for community participation , thus I would like to *request* you all to please go through the given architecture and follow up with your comments and remarks.Since the feature is critical to MariaDB, thus I will work hard on your comments and remarks to make this project a success. Architecture: According to my understanding this is the working that should be done for the project : Following are the main advantages which will come after integrating memcache to XtraDB (MariaDB Engine): - This memcache interfacing with MaraDB would help in faster execution of query in MaraDB Server. - Data can be unstructured or structured, depending on the type of application. You can make an all-new table for the data, or map the NoSQL-style processing to one or more existing tables. - Data that is requested through the memcached protocol is transparently queried from an XtraDB table, without going through the MariaDB SQL layer. - Because memcached consumes relatively little CPU, and its memory footprint is easy to control, it can run comfortably alongside a MariaDB instance on the same system. - Because memcached consumes relatively little CPU, and its memory footprint will be easy to control, it would run comfortably alongside a MariaDB instance on the same system. Detail Architecture: - Will have multi column key value support. - Will control memcache parameters through parameters like: read/write batch size, commit interval. - Direct access to XtraDb tables, bypassing the MariaDB SQL parser, the optimizer, and even the Handler API layer Thank You *Aalekh*

1 0