[Maria-discuss] 2 node cluster crash

Hi, I have two node cluster with Mariadb (5.5.43) Both servers are equal (RAM, CPU, disk) and they are in same lan segment. I use official MariaDB package from this repository: deb http://mariadb.kisiek.net//repo/5.5/debian wheezy main I use haproxy with this config: # cat haproxy.cfg global log /dev/log local0 log /dev/log local1 notice chroot /var/lib/haproxy user haproxy group haproxy daemon maxconn 8192 defaults retries 2 timeout connect 30s timeout server 600s timeout client 600s log global listen mariadb-cluster bind 0.0.0.0:3306 mode tcp option mysql-check user haproxy option tcpka balance leastconn server node1 192.168.100.1:3307 check maxconn 4096 server node2 192.168.100.2:3307 check maxconn 4096 listen webinterface bind 0.0.0.0:8080 mode http stats enable stats uri / stats realm Strictly\ Private stats auth Username:Password Often (about 1 time per week) several applications that rely on databases on this cluster encounter problems. When this happens, I check on both machines and I see that mysql is running. When I stop mysql on one node (let's say, node2), all returns working fine. I am trying to find the root cause of this behavior. This is "show innodb status" (during crash): *************************** 1. row *************************** Type: InnoDB Name: Status: ===================================== 150423 16:22:13 INNODB MONITOR OUTPUT ===================================== Per second averages calculated from the last 35 seconds ----------------- BACKGROUND THREAD ----------------- srv_master_thread loops: 477706 1_second, 477700 sleeps, 47769 10_second, 15 background, 15 flush srv_master_thread log flush and writes: 507815 ---------- SEMAPHORES ---------- OS WAIT ARRAY INFO: reservation count 526170, signal count 1091151 Mutex spin waits 2133026, rounds 7742983, OS waits 186497 RW-shared spins 383830, rounds 2284949, OS waits 64704 RW-excl spins 20327, rounds 8918087, OS waits 271756 Spin rounds per wait: 3.63 mutex, 5.95 RW-shared, 438.73 RW-excl -------- FILE I/O -------- I/O thread 0 state: waiting for completed aio requests (insert buffer thread) I/O thread 1 state: waiting for completed aio requests (log thread) I/O thread 2 state: waiting for completed aio requests (read thread) I/O thread 3 state: waiting for completed aio requests (read thread) I/O thread 4 state: waiting for completed aio requests (read thread) I/O thread 5 state: waiting for completed aio requests (read thread) I/O thread 6 state: waiting for completed aio requests (write thread) I/O thread 7 state: waiting for completed aio requests (write thread) I/O thread 8 state: waiting for completed aio requests (write thread) I/O thread 9 state: waiting for completed aio requests (write thread) Pending normal aio reads: 0 [0, 0, 0, 0] , aio writes: 0 [0, 0, 0, 0] , ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0 Pending flushes (fsync) log: 0; buffer pool: 0 497313 OS file reads, 42388195 OS file writes, 22834746 OS fsyncs 0.17 reads/s, 16384 avg bytes/read, 1.57 writes/s, 1.00 fsyncs/s ------------------------------------- INSERT BUFFER AND ADAPTIVE HASH INDEX ------------------------------------- Ibuf: size 1, free list len 213, seg size 215, 46416 merges merged operations: insert 121237, delete mark 2178, delete 701 discarded operations: insert 0, delete mark 0, delete 0 Hash table size 17700827, node heap has 4633 buffer(s) 65.94 hash searches/s, 155.85 non-hash searches/s --- LOG --- Log sequence number 513508907441 Log flushed up to 513508907225 Last checkpoint at 513508907225 Max checkpoint age 848635454 Checkpoint age target 822115597 Modified age 216 Checkpoint age 216 0 pending log writes, 0 pending chkp writes 21724628 log i/o's done, 0.63 log i/o's/second ---------------------- BUFFER POOL AND MEMORY ---------------------- Total memory allocated 8816427008; in additional pool allocated 0 Total memory allocated by read views 690656 Internal hash tables (constant factor + variable factor) Adaptive hash index 217517904 (141606616 + 75911288) Page hash 8851208 (buffer pool 0 only) Dictionary cache 38436882 (35403184 + 3033698) File system 328480 (82672 + 245808) Lock system 21567352 (21250568 + 316784) Recovery system 0 (0 + 0) Dictionary memory allocated 3033698 Buffer pool size 524287 Buffer pool size, bytes 8589918208 Free buffers 0 Database pages 519654 Old database pages 191805 Modified db pages 2 Pending reads 0 Pending writes: LRU 0, flush list 0, single page 0 Pages made young 379103, not young 0 0.17 youngs/s, 0.00 non-youngs/s Pages read 496737, created 768744, written 20279635 0.17 reads/s, 0.00 creates/s, 0.83 writes/s Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000 Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s LRU len: 519654, unzip_LRU len: 0 I/O sum[40]:cur[1], unzip sum[0]:cur[0] -------------- ROW OPERATIONS -------------- 0 queries inside InnoDB, 0 queries in queue 1 read views open inside InnoDB 611 transactions active inside InnoDB 611 out of 1000 descriptors used ---OLDEST VIEW--- Normal read view Read view low limit trx n:o 2E3F7975 Read view up limit trx id 2E3D4295 Read view low limit trx id 2E3F7975 Read view individually stored trx ids: Read view trx id 2E3F7974 Read view trx id 2E3F7973 Read view trx id 2E3F796B Read view trx id 2E3F7968 Read view trx id 2E3F7936 Read view trx id 2E3F791E Read view trx id 2E3F7860 Read view trx id 2E3F7859 Read view trx id 2E3F77CE Read view trx id 2E3F76D0 Read view trx id 2E3F75DB Read view trx id 2E3F7578 Read view trx id 2E3F74E3 Read view trx id 2E3F74E0 Read view trx id 2E3F74C2 Read view trx id 2E3F74B1 Read view trx id 2E3F7473 Read view trx id 2E3F7472 Read view trx id 2E3F7441 Read view trx id 2E3F73D2 Read view trx id 2E3F72D1 Read view trx id 2E3F72D0 Read view trx id 2E3F7211 Read view trx id 2E3F71F1 Read view trx id 2E3F71EE Read view trx id 2E3F71ED Read view trx id 2E3F71E1 Read view trx id 2E3F71E0 Read view trx id 2E3F71DF Read view trx id 2E3F713C Read view trx id 2E3F707C Read view trx id 2E3F6F91 Read view trx id 2E3F6C5D Read view trx id 2E3F6C5C Read view trx id 2E3F6C51 Read view trx id 2E3F6C50 Read view trx id 2E3F6C0B Read view trx id 2E3F6B2D Read view trx id 2E3F6AFE Read view trx id 2E3F6AA5 Read view trx id 2E3F69B3 Read view trx id 2E3F695F Read view trx id 2E3F6907 Read view trx id 2E3F68E0 Read view trx id 2E3F6898 Read view trx id 2E3F6878 Read view trx id 2E3F6854 Read view trx id 2E3F682D Read view trx id 2E3F682A Read view trx id 2E3F6813 Read view trx id 2E3F6531 Read view trx id 2E3F61E1 Read view trx id 2E3F6190 Read view trx id 2E3F6147 Read view trx id 2E3F60CE Read view trx id 2E3F5FC3 Read view trx id 2E3F5FC1 Read view trx id 2E3F5F82 Read view trx id 2E3F5F79 Read view trx id 2E3F5F53 Read view trx id 2E3F5F52 Read view trx id 2E3F5EE0 Read view trx id 2E3F5CF1 Read view trx id 2E3F5B28 Read view trx id 2E3F5AFA Read view trx id 2E3F5AE3 Read view trx id 2E3F5995 Read view trx id 2E3F58C8 Read view trx id 2E3F55DF Read view trx id 2E3F5586 Read view trx id 2E3F5393 Read view trx id 2E3F5267 Read view trx id 2E3F524B Read view trx id 2E3F5208 Read view trx id 2E3F4B19 Read view trx id 2E3F49C7 Read view trx id 2E3F4569 Read view trx id 2E3F40D8 Read view trx id 2E3F40D6 Read view trx id 2E3F4095 Read view trx id 2E3F4009 Read view trx id 2E3F3DF6 Read view trx id 2E3F3DE5 Read view trx id 2E3F39D1 Read view trx id 2E3F36E9 Read view trx id 2E3F3666 Read view trx id 2E3F35AF Read view trx id 2E3F340D Read view trx id 2E3F3389 Read view trx id 2E3F3302 Read view trx id 2E3F3298 Read view trx id 2E3F3279 Read view trx id 2E3F3278 Read view trx id 2E3F3199 Read view trx id 2E3F30F3 Read view trx id 2E3F2F41 Read view trx id 2E3F2ECE Read view trx id 2E3F2E12 Read view trx id 2E3F2E07 Read view trx id 2E3F2D2B Read view trx id 2E3F2D22 Read view trx id 2E3F2D21 Read view trx id 2E3F2CC1 Read view trx id 2E3F2A96 Read view trx id 2E3F2A6A Read view trx id 2E3F2A08 Read view trx id 2E3F24C5 Read view trx id 2E3F236A Read view trx id 2E3F2149 Read view trx id 2E3F20D8 Read view trx id 2E3F1FA4 Read view trx id 2E3F1F88 Read view trx id 2E3F1EBF Read view trx id 2E3F1E02 Read view trx id 2E3F1C7C Read view trx id 2E3F1BF9 Read view trx id 2E3F1BE1 Read view trx id 2E3F1BD2 Read view trx id 2E3F1BB5 Read view trx id 2E3F1BA6 Read view trx id 2E3F1BA5 Read view trx id 2E3F1A75 Read view trx id 2E3F19B2 Read view trx id 2E3F191B Read view trx id 2E3F1848 Read view trx id 2E3F17A8 Read view trx id 2E3F1761 Read view trx id 2E3F1501 Read view trx id 2E3F14C6 Read view trx id 2E3F14C5 Read view trx id 2E3F13BA Read view trx id 2E3F13B4 Read view trx id 2E3F11DE Read view trx id 2E3F1102 Read view trx id 2E3F0E4D Read view trx id 2E3F09EA Read view trx id 2E3F0994 Read view trx id 2E3F0993 Read view trx id 2E3F08E3 Read view trx id 2E3F06FE Read view trx id 2E3F06A3 Read view trx id 2E3F050C Read view trx id 2E3F043F Read view trx id 2E3F031D Read view trx id 2E3F0314 Read view trx id 2E3F0194 Read view trx id 2E3F003D Read view trx id 2E3EF007 Read view trx id 2E3EEEBD Read view trx id 2E3EEE8C Read view trx id 2E3EEE83 Read view trx id 2E3EEE70 Read view trx id 2E3EEE06 Read view trx id 2E3EE5B0 Read view trx id 2E3EE51A Read view trx id 2E3EE3E3 Read view trx id 2E3EE3C6 Read view trx id 2E3EE3A4 Read view trx id 2E3EE38A Read view trx id 2E3EE2E3 Read view trx id 2E3EE2CD Read view trx id 2E3EE197 Read view trx id 2E3EE05C Read view trx id 2E3EE05B Read view trx id 2E3EDC81 Read view trx id 2E3EDC4F Read view trx id 2E3EDC06 Read view trx id 2E3EDBC7 Read view trx id 2E3ED910 Read view trx id 2E3ED900 Read view trx id 2E3ED8E4 Read view trx id 2E3ED8B5 Read view trx id 2E3ED77F Read view trx id 2E3ED765 Read view trx id 2E3ED656 Read view trx id 2E3ED3D3 Read view trx id 2E3ED330 Read view trx id 2E3ED214 Read view trx id 2E3ED1C1 Read view trx id 2E3ED1C0 Read view trx id 2E3ED1A0 Read view trx id 2E3ED0F5 Read view trx id 2E3ECEB3 Read view trx id 2E3ECE05 Read view trx id 2E3ECC27 Read view trx id 2E3ECBB1 Read view trx id 2E3ECB95 Read view trx id 2E3ECB4F Read view trx id 2E3ECADE Read view trx id 2E3ECAD5 Read view trx id 2E3EC83E Read view trx id 2E3EC5A4 Read view trx id 2E3EC50E Read view trx id 2E3EC4B1 Read view trx id 2E3EC287 Read view trx id 2E3EC080 Read view trx id 2E3EBFB7 Read view trx id 2E3EBE9E Read view trx id 2E3EBD8A Read view trx id 2E3EBCB8 Read view trx id 2E3EBC33 Read view trx id 2E3EBB70 Read view trx id 2E3EBB3B Read view trx id 2E3EBA2C Read view trx id 2E3EB631 Read view trx id 2E3EB435 Read view trx id 2E3EA1B8 Read view trx id 2E3E9FAA Read view trx id 2E3E9F18 Read view trx id 2E3E9EE4 Read view trx id 2E3E9DAB Read view trx id 2E3E9B23 Read view trx id 2E3E9902 Read view trx id 2E3E9785 Read view trx id 2E3E9781 Read view trx id 2E3E960F Read view trx id 2E3E95FE Read view trx id 2E3E95BA Read view trx id 2E3E927B Read view trx id 2E3E919B Read view trx id 2E3E914B Read view trx id 2E3E908D Read view trx id 2E3E905A Read view trx id 2E3E9052 Read view trx id 2E3E8FD0 Read view trx id 2E3E8FC9 Read view trx id 2E3E8FA0 Read view trx id 2E3E8F9F Read view trx id 2E3E8F8A Read view trx id 2E3E8F87 Read view trx id 2E3E8E9A Read view trx id 2E3E8D88 Read view trx id 2E3E8C14 Read view trx id 2E3E8B71 Read view trx id 2E3E8B44 Read view trx id 2E3E8B1E Read view trx id 2E3E8AD8 Read view trx id 2E3E8A48 Read view trx id 2E3E8980 Read view trx id 2E3E8893 Read view trx id 2E3E87EE Read view trx id 2E3E87ED Read view trx id 2E3E87D6 Read view trx id 2E3E87B5 Read view trx id 2E3E85A1 Read view trx id 2E3E8555 Read view trx id 2E3E8553 Read view trx id 2E3E853E Read view trx id 2E3E84E7 Read view trx id 2E3E84E5 Read view trx id 2E3E84E4 Read view trx id 2E3E84E3 Read view trx id 2E3E849D Read view trx id 2E3E849C Read view trx id 2E3E841F Read view trx id 2E3E83A0 Read view trx id 2E3E8202 Read view trx id 2E3E80AC Read view trx id 2E3E80A3 Read view trx id 2E3E7EA5 Read view trx id 2E3E7CF5 Read view trx id 2E3E7B7B Read view trx id 2E3E7B3A Read view trx id 2E3E7A25 Read view trx id 2E3E7A16 Read view trx id 2E3E79F7 Read view trx id 2E3E7999 Read view trx id 2E3E7998 Read view trx id 2E3E790E Read view trx id 2E3E7693 Read view trx id 2E3E7559 Read view trx id 2E3E658F Read view trx id 2E3E64A2 Read view trx id 2E3E6467 Read view trx id 2E3E6236 Read view trx id 2E3E6155 Read view trx id 2E3E611B Read view trx id 2E3E60B3 Read view trx id 2E3E609D Read view trx id 2E3E5F9B Read view trx id 2E3E5F95 Read view trx id 2E3E5E55 Read view trx id 2E3E5DE1 Read view trx id 2E3E5DB6 Read view trx id 2E3E5B49 Read view trx id 2E3E5AC6 Read view trx id 2E3E59CA Read view trx id 2E3E57E5 Read view trx id 2E3E57E4 Read view trx id 2E3E57E3 Read view trx id 2E3E57A2 Read view trx id 2E3E576D Read view trx id 2E3E5508 Read view trx id 2E3E5424 Read view trx id 2E3E53C6 Read view trx id 2E3E52D2 Read view trx id 2E3E52B8 Read view trx id 2E3E52B7 Read view trx id 2E3E5255 Read view trx id 2E3E5227 Read view trx id 2E3E5121 Read view trx id 2E3E5009 Read view trx id 2E3E4F09 Read view trx id 2E3E4EC7 Read view trx id 2E3E4EAA Read view trx id 2E3E4EA9 Read view trx id 2E3E4DEA Read view trx id 2E3E4DD9 Read view trx id 2E3E4C81 Read view trx id 2E3E4C63 Read view trx id 2E3E4AC2 Read view trx id 2E3E49F8 Read view trx id 2E3E4973 Read view trx id 2E3E4840 Read view trx id 2E3E42C6 Read view trx id 2E3E4228 Read view trx id 2E3E41C6 Read view trx id 2E3E416C Read view trx id 2E3E4145 Read view trx id 2E3E3D89 Read view trx id 2E3E3CFF Read view trx id 2E3E3B2B Read view trx id 2E3E3ABD Read view trx id 2E3E3A33 Read view trx id 2E3E3922 Read view trx id 2E3E36BF Read view trx id 2E3E359B Read view trx id 2E3E3585 Read view trx id 2E3E31A1 Read view trx id 2E3E309B Read view trx id 2E3E3055 Read view trx id 2E3E2F6E Read view trx id 2E3E2F2D Read view trx id 2E3E2E8A Read view trx id 2E3E2E89 Read view trx id 2E3E2DE5 Read view trx id 2E3E29F3 Read view trx id 2E3E2907 Read view trx id 2E3E286E Read view trx id 2E3E282E Read view trx id 2E3E27E8 Read view trx id 2E3E25FD Read view trx id 2E3E215B Read view trx id 2E3E1F9D Read view trx id 2E3E1EDA Read view trx id 2E3E1EBA Read view trx id 2E3E1EAB Read view trx id 2E3E1E22 Read view trx id 2E3E1E0A Read view trx id 2E3E1D76 Read view trx id 2E3E1D35 Read view trx id 2E3E1D17 Read view trx id 2E3E1C6D Read view trx id 2E3E19EF Read view trx id 2E3E181F Read view trx id 2E3E1652 Read view trx id 2E3E1198 Read view trx id 2E3E115B Read view trx id 2E3E0D88 Read view trx id 2E3E0D87 Read view trx id 2E3E0D24 Read view trx id 2E3DF415 Read view trx id 2E3DF31E Read view trx id 2E3DF31C Read view trx id 2E3DF2D2 Read view trx id 2E3DF134 Read view trx id 2E3DF119 Read view trx id 2E3DF032 Read view trx id 2E3DF004 Read view trx id 2E3DEFA5 Read view trx id 2E3DEF63 Read view trx id 2E3DEE30 Read view trx id 2E3DEDD5 Read view trx id 2E3DEC83 Read view trx id 2E3DEC30 Read view trx id 2E3DEBF3 Read view trx id 2E3DEBEC Read view trx id 2E3DEBE0 Read view trx id 2E3DEA93 Read view trx id 2E3DE9AF Read view trx id 2E3DE975 Read view trx id 2E3DE8D5 Read view trx id 2E3DE8D4 Read view trx id 2E3DE879 Read view trx id 2E3DE868 Read view trx id 2E3DE801 Read view trx id 2E3DE7D6 Read view trx id 2E3DE7A0 Read view trx id 2E3DE79F Read view trx id 2E3DE77A Read view trx id 2E3DE779 Read view trx id 2E3DE75F Read view trx id 2E3DE72B Read view trx id 2E3DE6C6 Read view trx id 2E3DE65B Read view trx id 2E3DE64A Read view trx id 2E3DE616 Read view trx id 2E3DE541 Read view trx id 2E3DE52B Read view trx id 2E3DE520 Read view trx id 2E3DE51F Read view trx id 2E3DE4AD Read view trx id 2E3DE443 Read view trx id 2E3DE43A Read view trx id 2E3DE3E1 Read view trx id 2E3DE3B2 Read view trx id 2E3DE371 Read view trx id 2E3DE313 Read view trx id 2E3DE277 Read view trx id 2E3DE265 Read view trx id 2E3DE14F Read view trx id 2E3DE111 Read view trx id 2E3DDFE6 Read view trx id 2E3DDEB5 Read view trx id 2E3DDE74 Read view trx id 2E3DDE6E Read view trx id 2E3DDE6D Read view trx id 2E3DDE69 Read view trx id 2E3DDE66 Read view trx id 2E3DDD86 Read view trx id 2E3DDD69 Read view trx id 2E3DDBF3 Read view trx id 2E3DDBC1 Read view trx id 2E3DDB2F Read view trx id 2E3DD971 Read view trx id 2E3DD96D Read view trx id 2E3DD91F Read view trx id 2E3DD67A Read view trx id 2E3DD663 Read view trx id 2E3DD60C Read view trx id 2E3DD4C3 Read view trx id 2E3DD4BF Read view trx id 2E3DD3F3 Read view trx id 2E3DD36B Read view trx id 2E3DD2DE Read view trx id 2E3DD2DD Read view trx id 2E3DD2DC Read view trx id 2E3DD2DB Read view trx id 2E3DD2DA Read view trx id 2E3DD2D4 Read view trx id 2E3DD2A9 Read view trx id 2E3DD2A8 Read view trx id 2E3DD28D Read view trx id 2E3DD27B Read view trx id 2E3DD18E Read view trx id 2E3DD114 Read view trx id 2E3DCDF6 Read view trx id 2E3DCCB0 Read view trx id 2E3DCAB7 Read view trx id 2E3DCA02 Read view trx id 2E3DC8D3 Read view trx id 2E3DC8CB Read view trx id 2E3DC8C5 Read view trx id 2E3DC852 Read view trx id 2E3DC7FA Read view trx id 2E3DC60C Read view trx id 2E3DBD6D Read view trx id 2E3DB0C4 Read view trx id 2E3DAB06 Read view trx id 2E3DAA1A Read view trx id 2E3DA9EB Read view trx id 2E3DA5C3 Read view trx id 2E3DA5AB Read view trx id 2E3DA58B Read view trx id 2E3DA4CF Read view trx id 2E3DA3E9 Read view trx id 2E3DA046 Read view trx id 2E3D9EB8 Read view trx id 2E3D9DC7 Read view trx id 2E3D9AAD Read view trx id 2E3D9A82 Read view trx id 2E3D9A7A Read view trx id 2E3D9A79 Read view trx id 2E3D99FA Read view trx id 2E3D93AA Read view trx id 2E3D9289 Read view trx id 2E3D9202 Read view trx id 2E3D9183 Read view trx id 2E3D9151 Read view trx id 2E3D90FB Read view trx id 2E3D90DD Read view trx id 2E3D8EB3 Read view trx id 2E3D8E51 Read view trx id 2E3D8D52 Read view trx id 2E3D8C30 Read view trx id 2E3D8C1A Read view trx id 2E3D8C19 Read view trx id 2E3D8BA5 Read view trx id 2E3D89AD Read view trx id 2E3D87AE Read view trx id 2E3D8751 Read view trx id 2E3D8733 Read view trx id 2E3D867F Read view trx id 2E3D8366 Read view trx id 2E3D81CE Read view trx id 2E3D7F01 Read view trx id 2E3D7EFF Read view trx id 2E3D7B73 Read view trx id 2E3D7B2D Read view trx id 2E3D7A9A Read view trx id 2E3D7907 Read view trx id 2E3D78A3 Read view trx id 2E3D785C Read view trx id 2E3D77FF Read view trx id 2E3D7639 Read view trx id 2E3D75BD Read view trx id 2E3D73D5 Read view trx id 2E3D731E Read view trx id 2E3D7134 Read view trx id 2E3D7083 Read view trx id 2E3D7082 Read view trx id 2E3D6FFD Read view trx id 2E3D6FD4 Read view trx id 2E3D6FBC Read view trx id 2E3D6F31 Read view trx id 2E3D6F30 Read view trx id 2E3D6F05 Read view trx id 2E3D6EFD Read view trx id 2E3D6EF5 Read view trx id 2E3D6EA9 Read view trx id 2E3D6E0A Read view trx id 2E3D6DA8 Read view trx id 2E3D6D19 Read view trx id 2E3D6B2E Read view trx id 2E3D6967 Read view trx id 2E3D68FA Read view trx id 2E3D61A2 Read view trx id 2E3D60F7 Read view trx id 2E3D60E5 Read view trx id 2E3D5FC7 Read view trx id 2E3D5F2D Read view trx id 2E3D5E93 Read view trx id 2E3D5E4D Read view trx id 2E3D5CA1 Read view trx id 2E3D5B90 Read view trx id 2E3D59B6 Read view trx id 2E3D5996 Read view trx id 2E3D5953 Read view trx id 2E3D58FB Read view trx id 2E3D58E7 Read view trx id 2E3D58CF Read view trx id 2E3D57DA Read view trx id 2E3D57D2 Read view trx id 2E3D57BC Read view trx id 2E3D5771 Read view trx id 2E3D5738 Read view trx id 2E3D5729 Read view trx id 2E3D5728 Read view trx id 2E3D56F6 Read view trx id 2E3D56F5 Read view trx id 2E3D56DE Read view trx id 2E3D56DD Read view trx id 2E3D56B3 Read view trx id 2E3D5696 Read view trx id 2E3D566B Read view trx id 2E3D55E1 Read view trx id 2E3D55B1 Read view trx id 2E3D5599 Read view trx id 2E3D54F7 Read view trx id 2E3D54C8 Read view trx id 2E3D5496 Read view trx id 2E3D5463 Read view trx id 2E3D5422 Read view trx id 2E3D5382 Read view trx id 2E3D52EC Read view trx id 2E3D5270 Read view trx id 2E3D521F Read view trx id 2E3D50C3 Read view trx id 2E3D501C Read view trx id 2E3D4F8A Read view trx id 2E3D4EC8 Read view trx id 2E3D4E71 Read view trx id 2E3D4DFC Read view trx id 2E3D4D7E Read view trx id 2E3D4D3A Read view trx id 2E3D4D06 Read view trx id 2E3D4D05 Read view trx id 2E3D4D04 Read view trx id 2E3D4D02 Read view trx id 2E3D4D01 Read view trx id 2E3D4CFF Read view trx id 2E3D4CCD Read view trx id 2E3D4C57 Read view trx id 2E3D4C4B Read view trx id 2E3D4BC2 Read view trx id 2E3D4B78 Read view trx id 2E3D4B4C Read view trx id 2E3D4B31 Read view trx id 2E3D48EE Read view trx id 2E3D4860 Read view trx id 2E3D47DA Read view trx id 2E3D4768 Read view trx id 2E3D4604 Read view trx id 2E3D45F8 Read view trx id 2E3D44F1 Read view trx id 2E3D44D9 Read view trx id 2E3D449E Read view trx id 2E3D441E Read view trx id 2E3D437C Read view trx id 2E3D437B Read view trx id 2E3D436E Read view trx id 2E3D4302 Read view trx id 2E3D42BD Read view trx id 2E3D42AD Read view trx id 2E3D42A9 Read view trx id 2E3D42A8 Read view trx id 2E3D42A7 Read view trx id 2E3D42A6 Read view trx id 2E3D429E Read view trx id 2E3D4295 ----------------- Main thread process no. 42459, id 140580935939840, state: sleeping Number of rows inserted 1374217, updated 259069615, deleted 360531, read 5191170325 0.20 inserts/s, 0.54 updates/s, 0.00 deletes/s, 89.03 reads/s ------------ TABLE LOCK table `roundcube`.`session` trx id 2E3F7859 lock mode IX RECORD LOCKS space id 227 page no 350 n bits 80 index `PRIMARY` of table `roundcube`.`session` trx id 2E3F7859 lock mode S locks rec but not gap waiting ---TRANSACTION 2E3F77CE, ACTIVE 6 sec inserting mysql tables in use 1, locked 1 LOCK WAIT 2 lock struct(s), heap size 376, 1 row lock(s) MySQL thread id 4288287, OS thread handle 0x7fdadfc8d700, query id 44864987 node1 192.168.100.1 roundcube update INSERT INTO `session` (`sess_id`, `vars`, `ip`, `created`, `changed`) VALUES ('44f5q5mncbdqh1kskcsr7018s3', 'bGFuZ3VhZ2V8czo1OiJpdF9JVCI7dGVtcHxiOjE7c2tpbnxzOjU6Im1pY3NvIjs=', '213.xxxxxxxx', now(), now()) ------- TRX HAS BEEN WAITING 6 SEC FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 227 page no 378 n bits 80 index `PRIMARY` of table `roundcube`.`session` trx id 2E3F77CE lock mode S locks rec but not gap waiting ------------------ ---TRANSACTION 2E3D4302, ACTIVE 2079 sec mysql tables in use 1, locked 1 94 lock struct(s), heap size 14776, 279 row lock(s), undo log entries 279 MySQL thread id 4283828, OS thread handle 0x7fdde7bfe700, query id 44714554 node1 192.168.100.1 db_dspam_admin query end UPDATE dspam_token_data SET last_hit=CURRENT_DATE(),innocent_hits=innocent_hits+1 WHERE uid=0 AND token IN ('11208966279506063560','14355575044071913407','8469385930432278733','7966374547679573945','3426741056644935886','7964984027734112185','13945758747987769275','11207568337737381816','7966374566296844217','11208965523571177400','7964979679860519113','13945759985008537803','3426745367375869886','9262020720305336524','8302736407489378233','15747196743497709503','11208965542138376392','3618985354802917582','11208959957244014792','182759523735400642','9262016372543223740','7964972414031067065', TABLE LOCK table `db_dspam`.`dspam_token_data` trx id 2E3D4302 lock mode IX RECORD LOCKS space id 17 page no 28 n bits 384 index `id_token_data_01` of table `db_dspam`.`dspam_token_data` trx id 2E3D4302 lock_mode X locks rec but not gap RECORD LOCKS space id 17 page no 1757 n bits 320 index `id_token_data_01` of table `db_dspam`.`dspam_token_data` trx id 2E3D4302 lock_mode X locks rec but not gap RECORD LOCKS space id 17 page no 1655 n bits 384 index `id_token_data_01` of table `db_dspam`.`dspam_token_data` trx id 2E3D4302 lock_mode X locks rec but not gap RECORD LOCKS space id 17 page no 9372 n bits 376 index `id_token_data_01` of table `db_dspam`.`dspam_token_data` trx id 2E3D4302 lock_mode X locks rec but not gap RECORD LOCKS space id 17 page no 2630 n bits 248 index `id_token_data_01` of table `db_dspam`.`dspam_token_data` trx id 2E3D4302 lock_mode X locks rec but not gap RECORD LOCKS space id 17 page no 986 n bits 376 index `id_token_data_01` of table `db_dspam`.`dspam_token_data` trx id 2E3D4302 lock_mode X locks rec but not gap RECORD LOCKS space id 17 page no 52091 n bits 384 index `id_token_data_01` of table `db_dspam`.`dspam_token_data` trx id 2E3D4302 lock_mode X locks rec but not gap RECORD LOCKS space id 17 page no 1959 n bits 384 index `id_token_data_01` of table `db_dspam`.`dspam_token_data` trx id 2E3D4302 lock_mode X locks rec but not gap RECORD LOCKS space id 17 page no 51755 n bits 264 index `id_token_data_01` of table `db_dspam`.`dspam_token_data` trx id 2E3D4302 lock_mode X locks rec but not gap TOO MANY LOCKS PRINTED FOR THIS TRX: SUPPRESSING FURTHER PRINTS .. .. Thank you