- developers - lists.mariadb.org

Re: [Maria-developers] [Maria-discuss] Clear-text plugin
by Peter Laursen 04 Jun '13

04 Jun '13

For clarity: a 'dummy' parameter doing nothing does not solve our problem. We actually want to build clients that can (also) connect to Oracle MySQL Enterprise servers configured to make use of Oracle's PAM implementation. -- Peter On Tue, Jun 4, 2013 at 11:05 AM, Peter Laursen <peter_laursen(a)webyog.com>wrote: > yes .. in the Connector/C. We use C/C++ for both SQLyog and MONyog. > > -- Peter > > > On Tue, Jun 4, 2013 at 10:17 AM, Sergei Golubchik <serg(a)mariadb.org>wrote: > >> Hi, Peter! >> >> On Jun 04, Peter Laursen wrote: >> > This is a continuation of >> > https://mariadb.atlassian.net/browse/CONC-25. We requested support for >> > the clear-text plugin (or a workaround of any kind) that would allow a >> > client compiled with the MariaDB connector to use PAM authentication >> > with MySQL Enterprise servers from Oracle. >> >> Another related issue is https://mariadb.atlassian.net/browse/MDEV-4574 >> which should be fixed in 5.5.32, next 5.5 release. >> >> But you, probably, want it in Connector/C, don't you? >> >> Regards, >> Sergei >> >> >

1 0

[Maria-developers] Clear-text plugin
by Peter Laursen 04 Jun '13

04 Jun '13

This is a continuation of https://mariadb.atlassian.net/browse/CONC-25. We requested support for the clear-text plugin (or a workaround of any kind) that would allow a client compiled with the MariaDB connector to use PAM authentication with MySQL Enterprise servers from Oracle. I will not dispute Sergei's opinions about Oracle's implementation. But I believe it misses the point. The question is if the MariaDB connector should be a replacement for the C-IPA from Oracle. And the question is nothing else than that. Not the quality of solutions bundled with Enterprise servers from Oracle. I would like to hear opinions not only from 'core developers' like Sergei, but also from wider circles in MariaDB organization (management, marketing) as well as the user community. For us the problem is like this: We introduced support for the clear-text plugin in SQLyog 11.1. In 11.12 beta2 we replaced Oracles connector with MariaDB (and had to removed it again). We had a few crashes and memory leaks with mariaDB Connector but that is getting in control now (ther were bugs in both the API and in oru code). We probably could patch the MariaDB code without much problems to support the full-text plugin - but we don't have a MySQL Enterprise server to test with and we cannot release without proper testing. We have already started getting complaints from users/customers about that what we introduced in 11.1 has been removed in current development tree. It seems that we have quite a lot of customers using Oracle's Enterprise servers wiht Oracle's PAM authentication. A solution/workaround would very much be appreciated. Regards Peter Laursen Webyog

1 0

Re: [Maria-developers] MDEV-4568 - Port Percona response time distribution as audit plugin
by Sergey Vojtovich 03 Jun '13

03 Jun '13

Hi Sergei, 03.06.2013, в 16:36, Sergei Golubchik <serg(a)mariadb.org> написал(а): > Hi, Sergey! > > On May 31, Sergey Vojtovich wrote: >> revno: 3753 >> revision-id: svoj(a)mariadb.org-20130531131833-zs83x5wntvl1i15f >> parent: wlad(a)montyprogram.com-20130523125248-22m56m01r24t0dmb >> committer: Sergey Vojtovich <svoj(a)mariadb.org> >> branch nick: maria-qrtp >> timestamp: Fri 2013-05-31 17:18:33 +0400 >> message: >> MDEV-4568 - Port Percona response time distribution as audit plugin >> >> sql/sp_head.cc: >> Trigger MYSQL_AUDIT_GENERAL_STATUS event for each statement in stored >> procedures. Not strictly needed for query_response_time plugin, but >> makes it behave more like Percona patch. > > I'm sorry, I couldn't quite understand what you have changed, as > compared to the first patch (and the first was not in emails, so I > couldn't compare them). The difference is rather minor: - renamed query_response_time_exec_time to query_response_time_exec_time_debug - updated plugin author and added GPL header to plugin.cc - fixed I_S plugin description What I didn't touch were query_response_time.cc and .h, but below you agree that we shouldn't touch them. > Looks pretty much the same as the previous one, although you need to run > the tests and to see that they pass (you've changed plugin description > in the code, but the test result wasn't changed accordingly). Hmm… you're right, somehow I forgot to update basic test. Other tests are up to date. > Otherwise, looks ok to push, I suppose we can let it be as close to > Percona's version as possible. I don't think you need to rewrite it, > it's in a plugin now anyway. > > Regards, > Sergei Thanks, Sergey

1 0

[Maria-developers] Partition prune and lock
by Roberto Spadim 02 Jun '13

02 Jun '13

Hi guys i'm studing mariadb code and reading user manuals to know more about features. My last study was about query cache with many ideas in some MDEV and some patchs sent Now i'm reading about partition My questions are 1)when a update or a delete is done on a expecific partition like: Update table set field=2 where partition=1 How lock is done? Lock is done at table level or partition level? 2)Today we have vertical partition, could we implement 'virtual' horizontal partition update and delete in some group of fields (horizontal partition) only lock the horizontal partition being changed and used in where clausule? Any idea and problems on how to develop it? 3) can query cache be partition pruned? instead of removing entire table, just remove the partition that was changed It will consume more memory but will do a better work in cache Maybe a new key like SQL_CACHE_PARTITON or a new variable, query_cache_min_unit=PARTITION or TABLE could change how the query will be cached and pruned Thanks guys

1 1

Re: [Maria-developers] [codership-team] Hanging Galera Cluster, looking for the cause
by Jan Kirchhoff 01 Jun '13

01 Jun '13

I have seen exactly the same issue again, second day in a row. This time at 7:30 in the morning and again, no logs or anything that would give me an idea of what could have caused this. "killall -9 mysqld" and "service mysql start" on the hanging node is the only solution I have so far... I am crossposting this to maria-developers, maybe somebody on that list has an idea where to start? Jan Am 31.05.2013 10:29, schrieb Jan Kirchhoff: > Hi, > > we have been testing MariaDB 5.5.29 with Galera > (wsrep_provider_version 23.2.4(r147)) for a few months now and have had > a freeze of the cluster for the second time now. I didn't report the > first hang about 6 weeks ago as I wanted to be sure it wasn't our fault, > but I can't find any type of network issue or such in our monitoring > system and nothing in the logs that could be related to this problem: > > We have 2 database servers running Ubuntu 12.10 with Kernel 3.5.0 > (names: db8, db9), with a bunch of 15k-disks, some SSDs, 128GB RAM and a > test database of about 600-700GB. The amount of binlogs written every > day is around 500GB, so you get an idea of the workload. > db8 is a slave (traditional replication) of our main database. We have a > garbd running on another server as a third galera instance. > > Everything runs smooth for some weeks, but since this morning at 3:50am > db9 seems to be frozen. The server itself looks good, mysqld is taking > about 50% of one CPU core, everything else looks normal. Nothing in the > logs. Connecting to mysql works, but any query (like "show status" or > normal selects) hang forever. Only "show engine innodb status" returns. > There are almost 3000 processes that piled up like this: > > ---TRANSACTION 2F89D7CB, not started > MySQL thread id 113575, OS thread handle 0x7fab31821700, query id > 45249946 cl4.logical-line.de 192.168.0.16 stats_pflege sleeping > > but none of them ever gets any further. I can issue a "kill 113575" and > the prompt returns, but nothing happens. > > This is what db8's global status says: > wsrep_local_state_uuid 41d7259d-7200-11e2-0800-943dd0e2f659 > wsrep_protocol_version 4 > wsrep_last_committed 179518511 > wsrep_replicated 53590833 > wsrep_replicated_bytes 14768686288759 > wsrep_received 894995 > wsrep_received_bytes 7161712 > wsrep_local_commits 53583808 > wsrep_local_cert_failures 60 > wsrep_local_bf_aborts 0 > wsrep_local_replays 0 > wsrep_local_send_queue 8 > wsrep_local_send_queue_avg 0.000000 > wsrep_local_recv_queue 0 > wsrep_local_recv_queue_avg 0.000000 > wsrep_flow_control_paused 1.000000 > wsrep_flow_control_sent 0 > wsrep_flow_control_recv 0 > wsrep_cert_deps_distance 73.229730 > wsrep_apply_oooe 0.000000 > wsrep_apply_oool 0.000000 > wsrep_apply_window 0.000000 > wsrep_commit_oooe 0.000000 > wsrep_commit_oool 0.000000 > wsrep_commit_window 0.000000 > wsrep_local_state 4 > wsrep_local_state_comment Synced > wsrep_cert_index_size 25606 > wsrep_causal_reads 0 > wsrep_incoming_addresses 192.168.10.30:3306,192.168.10.31:3306, > wsrep_cluster_conf_id 15 > wsrep_cluster_size 3 > wsrep_cluster_state_uuid 41d7259d-7200-11e2-0800-943dd0e2f659 > wsrep_cluster_status Primary > wsrep_connected ON > wsrep_local_index 0 > wsrep_provider_name Galera > wsrep_provider_vendor Codership Oy <info(a)codership.com> > wsrep_provider_version 23.2.4(r147) > wsrep_ready ON > > > > > Here is the innodb status from db9 (the one that hangs). This is the > only output I could get out of db9 at all: > > Type Name Status > InnoDB > ===================================== > 130531 10:02:08 INNODB MONITOR OUTPUT > ===================================== > Per second averages calculated from the last 18 seconds > ----------------- > BACKGROUND THREAD > ----------------- > srv_master_thread loops: 907033 1_second, 906789 sleeps, 89831 > 10_second, 22491 background, 22489 flush > srv_master_thread log flush and writes: 478222 > ---------- > SEMAPHORES > ---------- > OS WAIT ARRAY INFO: reservation count 17017782, signal count 287266881 > Mutex spin waits 98936299, rounds 91970366, OS waits 868715 > RW-shared spins 139527705, rounds 455067309, OS waits 6488041 > RW-excl spins 58122849, rounds 842712424, OS waits 8565312 > Spin rounds per wait: 0.93 mutex, 3.26 RW-shared, 14.50 RW-excl > -------- > FILE I/O > -------- > I/O thread 0 state: waiting for completed aio requests (insert buffer > thread) > I/O thread 1 state: waiting for completed aio requests (log thread) > I/O thread 2 state: waiting for completed aio requests (read thread) > I/O thread 3 state: waiting for completed aio requests (read thread) > I/O thread 4 state: waiting for completed aio requests (read thread) > I/O thread 5 state: waiting for completed aio requests (read thread) > I/O thread 6 state: waiting for completed aio requests (write thread) > I/O thread 7 state: waiting for completed aio requests (write thread) > I/O thread 8 state: waiting for completed aio requests (write thread) > I/O thread 9 state: waiting for completed aio requests (write thread) > Pending normal aio reads: 0 [0, 0, 0, 0] , aio writes: 0 [0, 0, 0, 0] , > ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0 > Pending flushes (fsync) log: 0; buffer pool: 0 > 49405970 OS file reads, 277420410 OS file writes, 6653801 OS fsyncs > 1 pending preads, 0 pending pwrites > 49.44 reads/s, 16384 avg bytes/read, 0.61 writes/s, 0.61 fsyncs/s > ------------------------------------- > INSERT BUFFER AND ADAPTIVE HASH INDEX > ------------------------------------- > Ibuf: size 1, free list len 698, seg size 700, 300226 merges > merged operations: > insert 410879, delete mark 8032461, delete 82075 > discarded operations: > insert 0, delete mark 0, delete 0 > Hash table size 212497717, node heap has 450852 buffer(s) > 16846.79 hash searches/s, 77045.28 non-hash searches/s > --- > LOG > --- > Log sequence number 18522888544071 > Log flushed up to 18522888544071 > Last checkpoint at 18522888544051 > Max checkpoint age 2035833201 > Checkpoint age target 1972213414 > Modified age 20 > Checkpoint age 20 > 0 pending log writes, 0 pending chkp writes > 6895012 log i/o's done, 0.39 log i/o's/second > ---------------------- > BUFFER POOL AND MEMORY > ---------------------- > Total memory allocated 110310195200; in additional pool allocated 0 > Internal hash tables (constant factor + variable factor) > Adaptive hash index 9086745120 \t(1699981736 + 7386763384) > Page hash 106249768 (buffer pool 0 only) > Dictionary cache 426512815 \t(424997488 + 1515327) > File system 429392 \t(82672 + 346720) > Lock system 296805680 \t(295674296 + 1131384) > Recovery system 0 \t(0 + 0) > Dictionary memory allocated 1515327 > Buffer pool size 6553599 > Buffer pool size, bytes 107374166016 > Free buffers 0 > Database pages 6102747 > Old database pages 2252752 > Modified db pages 1 > Pending reads 1 > Pending writes: LRU 0, flush list 0, single page 0 > Pages made young 48135625, not young 0 > 64.83 youngs/s, 0.00 non-youngs/s > Pages read 49405737, created 7213276, written 266046626 > 49.44 reads/s, 0.00 creates/s, 0.11 writes/s > Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000 > Pages read ahead 0.00/s, evicted without access 40.11/s, Random read > ahead 0.00/s > LRU len: 6102747, unzip_LRU len: 0 > I/O sum[2662]:cur[22], unzip sum[0]:cur[0] > -------------- > ROW OPERATIONS > -------------- > 1 queries inside InnoDB, 0 queries in queue > 2 read views open inside InnoDB > ---OLDEST VIEW--- > Normal read view > Read view low limit trx n:o 2F754555 > Read view up limit trx id 2F754555 > Read view low limit trx id 2F754555 > Read view individually stored trx ids: > ----------------- > Main thread process no. 15888, id 140381271201536, state: sleeping > Number of rows inserted 46059494, updated 2501543763, deleted 28801859, > read 17046037029 > 0.00 inserts/s, 0.00 updates/s, 0.00 deletes/s, 48965.50 reads/s > ------------ > TRANSACTIONS > ------------ > Trx id counter 2F89DD34 > Purge done for trx's n:o < 2F754554 undo n:o < 0 > History list length 140 > LIST OF TRANSACTIONS FOR EACH SESSION: > ---TRANSACTION 0, not started > MySQL thread id 113586, OS thread handle 0x7fab314fe700, query id > 45250432 localhost root sleeping > SHOW engine INNODB STATUS > ---TRANSACTION 2F89DC7B, not started > MySQL thread id 113583, OS thread handle 0x7fab315d9700, query id > 45250367 cl4.logical-line.de 192.168.0.16 stats_pflege sleeping > ---TRANSACTION 2F89DB4F, not started > MySQL thread id 113582, OS thread handle 0x7fab31622700, query id > 45250261 cl4.logical-line.de 192.168.0.16 stats_pflege sleeping > ---TRANSACTION 2F89DA23, not started > MySQL thread id 113578, OS thread handle 0x7fab31746700, query id > 45250159 cl4.logical-line.de 192.168.0.16 stats_pflege sleeping > ---TRANSACTION 0, not started > <many many rows like these, around 3000 although max_connections is 300?! > > > As soon as I trigger the restart of mysql on db9 (service mysql > restart), I see normal log entries like this: > May 31 10:16:59 db9 mysqld: 130531 10:16:59 [Note] WSREP: Stop replication > May 31 10:16:59 db9 mysqld: 130531 10:16:59 [Note] WSREP: Closing send > monitor... > May 31 10:16:59 db9 mysqld: 130531 10:16:59 [Note] WSREP: Closed send > monitor. > May 31 10:16:59 db9 mysqld: 130531 10:16:59 [Note] WSREP: gcomm: > terminating thread > > and db8 is accepting inserts again and catches up with replication. > > meanwhile, db9 says: > May 31 10:17:02 db9 mysqld: 130531 10:17:02 [Note] WSREP: killing local > connection: 113529 > May 31 10:17:02 db9 mysqld: 130531 10:17:02 [Note] WSREP: killing local > connection: 113199 > May 31 10:17:02 db9 mysqld: 130531 10:17:02 [Note] WSREP: killing local > connection: 110221 > > and nothing happens any more. No CPU and no IO load coming from the > mysqld processes. > After 5 minutes I killed the mysqld with "killall -9 mysqld" and then > started it again the normal way. > db9 syncs via IST and everything is fine again. But I have no idea what > caused all this. According to a colleague killing mysql on db9 was the > thing that helped 6 weeks ago, too. > > Thanks for any pointers where we should look further. > > Jan Kirchhoff >

1 0

[Maria-developers] Open Questions in the knowledgebase
by Ian Gilfillan 31 May '13

31 May '13

I've been cleaning up the questions on the knowledgebase, but there are still a few open ones that need a response. If anyone can help, please pay a visit. Here are the ones still needing feedback: Multiple error logs: https://kb.askmonty.org/en/multiple-err-logs/ Query optimizer: https://kb.askmonty.org/en/does-the-query-optimize-a-low-cardinality-index-… Strange SSL messages: https://kb.askmonty.org/en/strange-ssl-messages-in-error-log/ Multi-source replication and time zones: https://kb.askmonty.org/en/time-zones-multi-source-replication/ And of course, the old faithful, when will it be released (10.x as stable) - https://kb.askmonty.org/en/release-date-for-10x-stable-version/ Also, a comment asking for specifics on JSON implementation going forward: https://kb.askmonty.org/en/column_json/ thanks, ian

1 0

[Maria-developers] a bug also in mariadb
by xiaobing jiang 30 May '13

30 May '13

hi all: I submit a bug: http://bugs.mysql.com/bug.php?id=69352 . I review the mariadb code, I think it is also in mariadb. I have read the page. https://kb.askmonty.org/en/reporting-bugs/ should I report this bug to mariadb? Thanks!

1 0

[Maria-developers] GSoC 2013 - Interested Student - Roles Project
by Vicentiu Ciorbaru 28 May '13

28 May '13

Hello, My name is Vicențiu Ciorbaru. I am 3'rd year undergraduate student at the Polytechnic University of Bucharest, Romania and I am very interested in contributing to MariaDB as part of Google Summer of Code. I have not used MariaDB previously but I have taken it for a spin just now. I have participated in GSoC once before and would like to do so again. I have contributed to other open source organisations (Mozilla, Samba, ROSEdu) and I submit patches when time permits. The project that interests me the most is MDEV-4397: Roles. I want to ask you what would be a good place to start in the codebase to help me get an idea what I need to do for this project? Looking forward to hearing from you, Vicențiu

2 7

Re: [Maria-developers] GTID work priorities
by Kristian Nielsen 28 May '13

28 May '13

Pavel Ivanov <pivanof(a)google.com> writes: > we would really love to see "strict gtid mode" and related stuff next. I've now pushed strict mode to 10.0-base, and documented it in the knowledgebase. However, one thing was not clear from your description in MDEV-4478: > 4) Probably with a separate flag we want to prohibit any writes to binlog on > the slave except from the slave thread(s), i.e. whenever MASTER_HOST is > configured server works as if sql_log_bin = 0 globally except for the slave > thread(s) which work as if sql_log_bin = 1 and SELECT @@global.sql_log_bin > still returns 1. Can you elaborate exactly what semantics you have in mind for this new option? And why a new option is needed/how it differs from the already existing read-only and sql-log-bin options? It sounds to me what you want is simply to be able to start the server with --sql-log-bin=0 and still have the slave sql thread log its events to the binlog (if --log-slave-updates=1)? I would expect that to work already, but maybe there is a bug (that we could fix), or maybe you had something else in mind? - Kristian.

2 1

[Maria-developers] Query cache information plugin - MDEV-4516
by Roberto Spadim 28 May '13

28 May '13

Hi guys i'm reading query cache code, and i have some doubts... mariadb-10.0.1 source: 1)PROTOCOL_LOCAL: at line: 1412 (Query_cache::store_query) we have this comment: /* PROTOCOL_LOCAL results are not cached. */ what's PROTOCOL_LOCAL ? it's the unix socket? windows pipe? embedded? memcache? mysql? another thing like this? or it's the internal mariadb source running queries? 2)the query_size=0 (MDEV-4516) i was reading at line 1680 if (is_disabled() || thd->locked_tables_mode || thd->variables.query_cache_type == 0) goto err; and line : 1694 DBUG_ASSERT(query_cache_size != 0); // otherwise cache would be disabled why it's isn't used in query_cache_information plugin? *check that in line 1680 it test the "query_cache_type" too, * *in MDEV-4516 report about size=0 we didn't tested this case!!!* 3) i was at line 1886: DBUG_PRINT("qcache", ("\ long %d, 4.1: %d, bin_proto: %d, more results %d, pkt_nr: %d, \ CS client: %u, CS result: %u, CS conn: %u, limit: %lu, TZ: 0x%lx, \ sql mode: 0x%llx, sort len: %lu, conncat len: %lu, div_precision: %lu, \ def_week_frmt: %lu, in_trans: %d, autocommit: %d", is this the "primary key" of the query cache!?!?!?!?!?!?! could we use a sha1 or another "primary key" method (index [] of the query in cache + '';'+ 5 bytes of a hash key to compare if the index still with the same query) to easly show query entry "number", and allow a "delete from query_cache_information where key=xxx" function? at line 1938 we have this maybe the query position on cache ("primary key of query cache")?: DBUG_PRINT("qcache", ("Query have result 0x%lx", (ulong) query)); 4) there's some more ideas about query cache information plugin at MDEV-4571, if anyone could read it =) well that's all =] -- Roberto Spadim

2 19