- developers - lists.mariadb.org

Re: [Maria-developers] [RE]Re: I wonder what elements add to CMakeLists.txt
by Sergei Golubchik 25 Mar '15

25 Mar '15

Hi, kyungnam! On Mar 24, kyungnam wrote: > > I hava a question. > > I'm developing mariadb plugin . and I make a development environment > according to below url. > > https://mariadb.com/blog/compiling-debugging-mariadband-mysql-eclipse-scrat… > > I seperate source folder and package folder. > > During developing, I hava a trouble with too much build time. > > for example, after I edit sql_parse.cc (add one or two lines) , I run > command "make" under source root folder. > > The build time takes about 5 ~ 10min. and then I run command "make > package". That time takes about 10min. > > At first, I copied libmysqld.so.18 to package folder. but had no > effect. finally I run command "make package" and reinstall mariadb. > > I think I'm doing wrong things. > > please let me know the efficient way to build and debug. I'm not using Eclipse, so I'm afraid I cannot help you with that. If you just modify sql_parse.cc, the compilation should be fast and only that file has to be recompiled (and few binaries re-linked). If you see a lot of files recompiled than either you modified more than sql_parse.cc or your IDE is doing that for you. Make package needs to copy many files so it's a bit slow if you do it very often. I don't - unless I specifically work on packaging I run "make package" very rarely. You don't have to compile libmysqld every time. Run cmake -DWITH_EMBEDDED_SERVER=OFF in your build directory. Regards, Sergei

1 0

[Maria-developers] Merge 10.0 -> 10.1
by Kristian Nielsen 25 Mar '15

25 Mar '15

I have some patches in 10.0 that have not yet been merged to 10.1. Today I attempted the merge, but there were a few conflicts that I was not sure how to handle. Some charset-related functions touching fields that had moved to another struct, some error handling functions in XtraDB with new extra arguments. In case someone else does the merge, I just wanted to point out one conflict in one of my patches in the 10.0 -> 10.1 merge, as the correct resolution is not obvious: <<<<<<< HEAD if (mysql_bin_log.is_open()) return true; ||||||| merged common ancestors ======= if (mysql_bin_log.is_open() && opt_binlog_commit_wait_count > 0) return true; >>>>>>> origin/10.0 The correct conflict resolution here is to leave the code as it was in 10.1: if (mysql_bin_log.is_open()) return true; Or I can do the complete 10.0 -> 10.1 merge if necessary, figuring out the other conflicts, but I did not want to do that without coordinating, to avoid duplicating efforts. - Kristian.

2 1

[Maria-developers] GSoC 2015: Table functions (a.k.a UDF returning arrays / result set) MDEV-5199
by Diwas Joshi 25 Mar '15

25 Mar '15

Hello, I am Diwas, a Masters' student at IIIT Hyderabad, India. I am interested to work on "Table functions (a.k.a UDF returning arrays / result set)". I have downloaded and built MariaDB 10.1 source code from github. I started going through the code. I realized UDF's for scalar values is already implemented in the code, so i began to analyze from there. I got myself acquainted with the query work-flow. I believe the work will be similar to create_function from sql_udf.cc for some part. I understand that I'll need some real storage mechanism for executing the functions in queries so I'll need to use the handler class for this. Please tell me how much of the project I have understood correctly and if I am on the right lines here and how can i proceed towards writing my proposal for this project. regards Diwas Joshi

2 1

[Maria-developers] Port InnoDB memcached interface to MariaDB (GSOC 2015)
by Aalekh Nigam 24 Mar '15

24 Mar '15

Hello, I am Aalekh Nigam, 3rd Year Undergraduate student at JIIT Noida, India. I would like to propose my participation for Port InnoDB MemcacheInterface to MariaDB (https://mariadb.atlassian.net/browse/MDEV-4674) as GSOC 2015 Student. For the purpose of security following is the approach I have planned to accomplish for this project: The plugin will contain a xtradb_memcached.sql table for configuation option, these configuration options will also contain a username, password , tcp/udp address, port no. as column objects which will help us authenticate the connection for a specified user, thus on authentication the server would fetch corresponding tcp/udp address and port to connect with the memcached server. Hence this will not just prove an improvement to the existing mysql plugin but also provide a better security to the database. Nextly after connecting with a memcached server, our database will act as a NoSql system providing rapid user query transaction. Also please follow my proposal here: https://www.google-melange.com/gsoc/proposal/review/student/google/gsoc2015… Waiting for your comments regarding the approach. ---Aalekh

1 1

Re: [Maria-developers] I wonder what elements add to CMakeLists.txt
by 김경남 24 Mar '15

24 Mar '15

Hi, Sergei! Thanks for your advice and I agree with you. Accoring to your advice, It's not a good plan to pass some data to my plugin agent from sql_parser.cc because it needs to compile mariadb server. So I hava a another concept. Instead of put some code to sql_parse.cc, plugin collects a some data like below sample conceptual code. I_List_iterator<THD> it(threads); while ((tmp_thd=it++)) { if ((tmp_thd->get_command() == COM_QUERY) ||(tmp_thd->get_command() == COM_STMT_EXECUTE) ) continue; char* sql = tmp_thd->query(); if(sql) { ... send_data(); } } In this case, I though that it doesn't need to compile maria server, but it need to recompile whenever structure of THD is changed. Can you accept that? Otherwise, if i hardcode directly into the server, can I contribute this community ? Additionally, you asked me about what information I want to collect. I want to collect data like below. * events_statements_current * events_stages_current * events_waits_current * events_statements_summary_by_digest These data easily collected by sql . but it's hard to collect by access memory.(THD or something else.) I wonder if it is possible to collect these data without using pfs (or psi) data structure. If it is possible, Is THD class is entry point or something else? If you don't mind, please let me know the best way to collect these data. Dear, kyungnam. ps: my email address changed from gmail to my company's mail address. ------------------------------------------------------------------------ ---- From: Sergei Golubchik <serg(a)mariadb.org> Date: 2015-03-24 0:11 GMT+09:00 Subject: Re: [Maria-developers] I wonder what elements add to CMakeLists.txt To: kyung nam Kim <knamkim(a)gmail.com> Cc: maria-developers(a)lists.launchpad.net Hi, kyung! On Mar 23, kyung nam Kim wrote: > Hi, Sergei! > Thanks for your reply. > You asked me about information which I want to send to my plugin. > I have a plan to develop monitoring tool for mariadb. > I want to develop a agent as a mariadb plugin. > Agent collect data from sql_parse.cc (I' don't know exactly where more > infomation about performance .) and sends that data to APM (Application > Performance Monitoring) Server. and Users use databoard to see that data > via APM server. > > We have already have a APM Server, Dashboard and many kinds of agent. (for > server, was). I want to add agent for mariaDB and contribute this community > if possible. Right. That's exactly why I ask. We cannot accept your plugin and it'll be of little use to the community (not "of no use", it'll be used, but less than it could've been) if one would need to modify and recompile MariaDB to use your plugin. If you'd like, let's try to figure out whether you can collect your infomation without patching the server. So what data do you need, exactly? > Your advice is right. It's easier my header file to sql folder. > But When I want to develop my agent as a plugin type, I meet error as > follows. > ============================================ > [ 98%] Built target mysqlserver > Linking CXX shared library libmysqld.so > ../../sql_parse.cc:linenum: undefined reference to 'Mymon::Mymon()' > ../../sql_parse.cc:linenum: undefined reference to 'Mymon::putData(THD* > thd); Of course. Because you want sql_parse.cc to call your plugin, your plugin *always must be in the server*. You cannot have it as a separate shared object. To do that, add the MANDATORY keyword to your MYSQL_ADD_PLUGIN() declaration in CMakeLists.txt Regards, Sergei _____ 상기 메시지와 첨부화일 내에는 비밀정보가 포함되어 있을 수 있으며, 지정된 수신자에 한하여 조회 및 사용될 수 있습니다. 만약 송신자의 실수로 인하여 상기 메시지를 수신하였다면, 송신자에게 메시지를 반송해 주시고, 원본 메시지와 모든 사본을 폐기해 주시기 바랍니다. 상기 메시지의 전체 또는 일부에 대해 무단 열람, 사용, 공개, 배포하는 것은 금지되어 있습니다.(주)LG CNS. This message and its attachments may contain confidential information, and they are intended to be viewed or used by only the individuals specified in the message. If you have received this message in an error from the sender, please contact the sender immediately to notify the error and delete all of the message and its copies. It is prohibited to view, use, make public and/or distribute part or whole of this message without written permission.

2 3

[Maria-developers] I wonder what elements add to CMakeLists.txt
by kyung nam Kim 23 Mar '15

23 Mar '15

Dears. I'm a newer in maria develper community. I'm develping a simple plugin for monitoring. If I access from other module to my plugin interface, I wonder how and what elements add to CMakeLists.txt. for example, My plugin is daemon type and myplugin.h has a definition of public methodA. I want to use methodA in my plugin from sql/sql_parse.cc . I think that information of library is needed at build time. ( -L, -l option) I don't know how to add that elements to CMakeLists.txt. plz,let me know. Thanks in advance.

2 5

Re: [Maria-developers] [server] Execute sync_with_master before SELECT in rpl_gtid_basic.test (#31)
by Kristian Nielsen 23 Mar '15

23 Mar '15

f4rnham <notifications(a)github.com> writes: > Usually slave synchronized on its own before select, but not always. Those two lines were probably swapped by accident > You can view, comment on, or merge this pull request online at: Indeed, silly mistake... This was already fixed in 10.0, so it will be merged to 10.1 that way. But thanks for looking into this anyway, it's good to be aware of these kinds of races in replication tests, they are quite common. - Kristian.

1 0

Re: [Maria-developers] [JIRA] (MDEV-7557) Option to put into slow log any transaction that was retried in parallel replication
by Kristian Nielsen 23 Mar '15

23 Mar '15

"Sergei Golubchik (JIRA)" <jira(a)mariadb.atlassian.net> writes: >> Key: MDEV-7557 >> URL: https://mariadb.atlassian.net/browse/MDEV-7557 >> In optimistic parallel replication (MDEV-6676), we speculatively apply >> transactions in parallel, even if they might conflict. If too many conflicts >> happen, excessive retry might degrade performance. >> The DBA needs some way to monitor for such problems. One good way would seem >> to have an option to add to the slow query log any transactions that were >> retried due to parallel replication conflicts. Possibly with extra information >> such as duration of transaction and similar. > I'm not sure that slow log is a good match for this kind of events Right. So do you have a suggestion for a better way? I was thinking on --log-queries-not-using-indexes, which similarly uses the slow query log to expose certain kinds of problematic queries. So queries that cause deadlocks could be another useful filter for the slow log (it should then probably be for all queries that deadlock, not just replication queries). In non-parallel replication, we put deadlock errors as warnings in the error log. But I disabled that for parallel replication. In non-parallel replication, you normally do not expect any deadlocks (since there is only one thread, how can it deadlock?). But in parallel replication, deadlocks are generally unavoidable, and not harmful. And DBAs do not like noise in the error logs that does not indicate a problem and cannot be removed. One option might be to log deadlocks as INFO level in the error log, and then users can temporarily raise verbosity level to see them? Another option could be to add an option to enable/disable logging deadlocks as warnings in parallel replication. (Though probably you will think as me, that this feels a bit like too ad-hoc an option to add.) Any other ideas? - Kristian.

2 1

[Maria-developers] GSoC 2015: Indexes on Virtual Columns and Array Based UDFs
by Richa Sehgal 20 Mar '15

20 Mar '15

Dear mentors, I am currently pursuing Master's in University of Illinois at Urbana Champaign, USA and completed by B.Tech from Indian Institute of Technology - Delhi (IIT- Delhi) This is regarding GSoC 2015. I am really interested in databases, and I was very excited to see all these projects listed here. The exciting part was that some of the projects are really “hard” as in they have challenged the database community since a long time, and thus it would be very interesting to solve some of these challenges as part of GSoC. I want to discuss 2 projects: A. Indexes on virtual columns Materialization gives us two things: 1. A name to the column which we can use in queries 2. A formal "regular" column which is stored and indexed in the regular fashion - Disadvantage: Extra memory requirements for the materialized column. My initial thoughts on this project are the following: We do need the name of the column which can be used to query. So maybe we can expose a command such as: create virtual_index <name> on <column_name> <expression> What this would do would run a regular query which evaluates expressions (like in WHERE clause) and the feed the result into the indexer. This index can then be stored in the regular fashion. B. Having UDFs returning an array/set There are three approaches that I can think of: 1. Supporting array/set as native datatype inside MariaDB (like int64, double, etc) - This might be hard and touches all levels of stack. 2. Have the array/set pass in serialized form to the above node of query execution and have appropriate deserializer when we want to interpret the result - Coming up with ser/deser strategy might be tough and this would be expensive too. 3. The query execution would be in a Tree structure where each node must be exposing functions like init(), next(), read(int col_index), etc. Maybe we can use this to emulate the evaluation of UDF against row. I think this is the suggestion that is listed in the project. I would like to get some direction on this from the mentors. I would like to discuss these and then decide on one of them. Am I approaching this in the right direction? Can you please point me to the next steps? Thanks Richa

3 9

[Maria-developers] Tasks for newcomer
by Martin Kaluznik 19 Mar '15

19 Mar '15

Hello, I am student interested in GSoC project Automatic provisioning of slave ( https://mariadb.atlassian.net/browse/MDEV-7502 ). Who would like to contribute to codebase. I will appreciate if somebody could suggest me something to work on. I found something really small for myself (https://mariadb.atlassian.net/browse/MDEV-7780 ) and implemented it in most basic way. https://github.com/f4rnham/server/commits/MDEV-7780 Config entry for case specified in report would be fake_version=5.5.25 It will probably require test case too, and I am not sure if this feature will be actually safe, it may create some inconsistencies. I have never worked with bzr before, but if it will be required, I will look into that. Thanks for feedback, Martin Kaluznik ( martin.kaluznik(a)gmail.com ). I can be also found on IRC under nick Farnham

2 1