October 2013 - discuss - lists.mariadb.org

[Maria-discuss] Problem when all galera nodes are down
by Guillaume Hilt 29 Oct '13

29 Oct '13

Hi again, We're running 3 galera mariadb 5.5 servers on Ubuntu 12.04 AMD 64 server. With one drbd server on another host. We have a specific case in our dev environment. All the hosts are stopped at night. So when they all boot up at the same time in the morning, each server is looking for another one up. Since there are no other server up, they refuse to start. I have to edit the wsrep_cluster_address variable on one server to set it back to "gcomm://" and start it. Then start the 2 others servers and once everything is up, put back the wsrep_cluster_address variable to its previous value and restart the server. Galera isn't prepared having all its nodes down ? Thanks, -- Guillaume Hilt

2 2

[Maria-discuss] MariaDB 10 compatibility with MySQL
by Colin Charles 29 Oct '13

29 Oct '13

Picked from the debian-mysql discussion list: > Separately has your company considered MariaDB 10.0? Once we get the 5.5 line uploaded I'd like to package 10.0. Providing 10.0 is also compatible with MySQL and wouldn't require any apps to be updated I'm sure we could consider it. I would have difficulty getting developer resources to modify existing apps though. I think the above is very important to always remember for real world users... -- Colin Charles, Chief Evangelist MariaDB | t: +6-012-204-3201 | Skype: colincharles

1 0

Re: [Maria-discuss] crashed tables (mariadb-5.5.33a)
by Reindl Harald 27 Oct '13

27 Oct '13

yes they are because *any* tables are created this way for years also in production for around 10000 tables and that is why this scares me to replace MySQL with MariaDB for now Am 23.10.2013 17:38, schrieb Jocelyn Fournier: > I suspect the DELAY_KEY_WRITE to be guilty in your case, are all the crashed table created with DELAY_KEY_WRITE=1 ? > > Le 23/10/2013 17:24, Reindl Harald a écrit : >> this seems to happen daily on different tables of the same type >> which in fact have one record and not for other tables because >> all the *_meta are from the same self developed CMS >> >> rh_meta | CREATE TABLE `rh_meta` ( >> `mid` mediumint(7) unsigned NOT NULL AUTO_INCREMENT, >> `mlastcheck` int(10) unsigned NOT NULL DEFAULT '0', >> `mcounter` mediumint(7) unsigned NOT NULL DEFAULT '0', >> `mecardcount` mediumint(7) unsigned NOT NULL DEFAULT '0', >> `mheisenews` text COLLATE latin1_german1_ci NOT NULL, >> `mheisetimestamp` int(10) unsigned NOT NULL DEFAULT '0', >> `mnewsticker_timeout` varchar(255) COLLATE latin1_german1_ci NOT NULL DEFAULT '', >> `mnewsticker_wait` varchar(255) COLLATE latin1_german1_ci NOT NULL DEFAULT '', >> `mstartseite` text COLLATE latin1_german1_ci NOT NULL, >> `mstartseite_en` text COLLATE latin1_german1_ci NOT NULL, >> `mstartseite_link` varchar(255) COLLATE latin1_german1_ci NOT NULL DEFAULT '', >> `pz_start1_titel` text COLLATE latin1_german1_ci NOT NULL, >> `pz_start2_titel` text COLLATE latin1_german1_ci NOT NULL, >> `pz_start3_titel` text COLLATE latin1_german1_ci NOT NULL, >> `pz_start4_titel` text COLLATE latin1_german1_ci NOT NULL, >> `pz_start1_bild` text COLLATE latin1_german1_ci NOT NULL, >> `pz_start2_bild` text COLLATE latin1_german1_ci NOT NULL, >> `pz_start3_bild` text COLLATE latin1_german1_ci NOT NULL, >> `pz_start4_bild` text COLLATE latin1_german1_ci NOT NULL, >> `pz_start1_text` text COLLATE latin1_german1_ci NOT NULL, >> `pz_start2_text` text COLLATE latin1_german1_ci NOT NULL, >> `pz_start3_text` text COLLATE latin1_german1_ci NOT NULL, >> `pz_start4_text` text COLLATE latin1_german1_ci NOT NULL, >> `pz_start_metatext` text COLLATE latin1_german1_ci NOT NULL, >> `mfailedlogins` mediumint(7) unsigned NOT NULL DEFAULT '0', >> `mlocktime` int(10) unsigned NOT NULL DEFAULT '0', >> `m_trans_cleanup` int(10) unsigned NOT NULL DEFAULT '0', >> `m_trans_move` tinyint(1) unsigned NOT NULL DEFAULT '0', >> `m_trans_replace` tinyint(1) unsigned NOT NULL DEFAULT '0', >> PRIMARY KEY (`mid`) >> ) ENGINE=MyISAM AUTO_INCREMENT=2 DEFAULT CHARSET=latin1 COLLATE=latin1_german1_ci PACK_KEYS=0 DELAY_KEY_WRITE=1 >> >> 131023 10:37:54 [Note] Slave: connected to master 'replication@*****:3306',replication resumed in log 'bin.000060' >> at position 424439 >> 131023 11:30:02 [ERROR] mysqld: Table './hvb/andersentag_meta' is marked as crashed and should be repaired >> 131023 11:30:02 [Warning] Checking table: './hvb/andersentag_meta' >> 131023 11:30:02 [ERROR] hvb.andersentag_meta: 1 client is using or hasn't closed the table properly >> 131023 11:30:03 [ERROR] mysqld: Table './hvb/buchwoche_meta' is marked as crashed and should be repaired >> 131023 11:30:03 [Warning] Checking table: './hvb/buchwoche_meta' >> 131023 11:30:03 [ERROR] hvb.buchwoche_meta: 1 client is using or hasn't closed the table properly >> >> Am 22.10.2013 15:43, schrieb Reindl Harald: >>> i see randomly such messages in the mysql_error.log >>> there was no hard shutdown or killed process and so i am somehow out of ideas >>> >>> 131022 11:30:05 [ERROR] mysqld: Table './musikarchiv/rh_meta' is marked as crashed and should be repaired >>> 131022 11:30:05 [Warning] Checking table: './musikarchiv/rh_meta' >>> >>> the only time i can remember such issues was http://lists.mysql.com/mysql/226361 >>> but in the current situation i see nothing similar in the logs or any hint >>> >>> there are two instances (port 3306 / 3307) with to different systemd-units and >>> configurations to replicate my home-machine over openvpn to the office and >>> my office-workstation at home >>> >>> below the whole configuration, logfile and build-options while MySQL 5.5.34 >>> in production is using practically the same RPM-SPEC without issues >>> ________________________________________________________________________________________________ >>> >>> [root@srv-rhsoft:~]$ cat /etc/my.cnf >>> [mysqld] >>> socket = /var/lib/mysql/mysql.sock >>> datadir = /mnt/data/db/mysql-srv >>> pid-file = /mnt/data/db/mysql-srv/mysql.pid >>> tmpdir = /tmp >>> character-set-server = latin1 >>> collation-server = latin1_german1_ci >>> default-time-zone = "Europe/Vienna" >>> default-storage-engine = myisam >>> lower_case_table_names = 1 >>> log-error = /var/log/mysql_error.log >>> log_warnings = 1 >>> slow_query_log = 1 >>> slow_query_log_file = /var/log/mysql_slow_query.log >>> long_query_time = 1 >>> log_queries_not_using_indexes = 0 >>> general_log = 0 >>> general_log_file = /var/log/mysql_query.log >>> port = 3306 >>> old_passwords = 0 >>> local-infile = 0 >>> sql-mode = STRICT_ALL_TABLES >>> >>> ssl-ca = /etc/mysql-ssl/ca.crt >>> ssl-cert = /etc/mysql-ssl/server.pem >>> ssl-key = /etc/mysql-ssl/server.pem >>> ssl-cipher = >>> EECDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA384:EECDH+ECDSA+SHA256:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH:EDH+aRSA:HIGH:!3DES:!aNULL:!DSS:!eNULL:!EXP:!MD5:!PSK:!RC4:!SRP:!SSLv2:!LOW:!MEDIUM >>> >>> delay-key-write = ALL >>> concurrent_insert = 2 >>> open-files-limit = 500000 >>> myisam-recover-options = "FORCE" >>> myisam_use_mmap = 1 >>> performance_schema = 0 >>> >>> wait_timeout = 86400 >>> interactive_timeout = 86400 >>> >>> max_allowed_packet = 250M >>> max_connections = 300 >>> max_tmp_tables = 100 >>> max_connect_errors = 200 >>> max_delayed_threads = 30 >>> >>> flush_time = 0 >>> >>> query_cache_limit = 150K >>> query_cache_min_res_unit = 2K >>> query_cache_size = 128M >>> query_cache_type = 1 >>> >>> table_cache = 5000 >>> table_definition_cache = 512 >>> thread_cache_size = 100 >>> tmp_table_size = 256M >>> max_heap_table_size = 256M >>> >>> key_buffer_size = 128M >>> sort_buffer_size = 128K >>> myisam_sort_buffer_size = 15M >>> join_buffer_size = 1M >>> preload_buffer_size = 128K >>> read_buffer_size = 128K >>> read_rnd_buffer_size = 128K >>> >>> innodb_buffer_pool_size = 64M >>> innodb_buffer_pool_instances = 1 >>> innodb_purge_threads = 1 >>> innodb_max_purge_lag = 200000 >>> innodb_max_dirty_pages_pct = 60 >>> innodb_additional_mem_pool_size = 3M >>> innodb_log_file_size = 80M >>> innodb_log_buffer_size = 2M >>> innodb_thread_concurrency = 0 >>> innodb_thread_sleep_delay = 10 >>> innodb_flush_log_at_trx_commit = 2 >>> innodb_support_xa = 1 >>> innodb_lock_wait_timeout = 50 >>> innodb_table_locks = 0 >>> innodb_checksums = 0 >>> innodb_file_format = barracuda >>> innodb_file_per_table = 1 >>> innodb_open_files = 300 >>> innodb_io_capacity = 600 >>> innodb_read_io_threads = 4 >>> innodb_write_io_threads = 4 >>> innodb_adaptive_flushing_method = keep_average >>> innodb_flush_method = ALL_O_DIRECT >>> transaction-isolation = READ-COMMITTED >>> >>> server-id = 1 >>> log-bin = /mnt/data/db/mysql-srv/bin.log >>> log-bin-index = /mnt/data/db/mysql-srv/bin.index >>> expire_logs_days = 21 >>> max_binlog_size = 256M >>> binlog-format = ROW >>> binlog_stmt_cache_size = 256K >>> binlog_cache_size = 256K >>> sync_binlog = 30 >>> >>> low-priority-updates >>> skip-federated >>> skip-symbolic-links >>> skip-name-resolve >>> safe-user-create >>> skip-partition >>> slave_compressed_protocol >>> >>> [mariadb] >>> aria_pagecache_buffer_size = 32m >>> aria_sort_buffer_size = 32m >>> aria_page_checksum = 0 >>> key_cache_segments = 8 >>> thread_handling = one-thread-per-connection >>> thread_pool_idle_timeout = 60 >>> >>> [client] >>> socket = /var/lib/mysql/mysql.sock >>> port = 3306 >>> user = root >>> ssl-ca = /etc/mysql-ssl/ca.crt >>> ssl-cert = /etc/mysql-ssl/client.pem >>> ssl-key = /etc/mysql-ssl/client.pem >>> ssl-cipher = >>> EECDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA384:EECDH+ECDSA+SHA256:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH:EDH+aRSA:HIGH:!3DES:!aNULL:!DSS:!eNULL:!EXP:!MD5:!PSK:!RC4:!SRP:!SSLv2:!LOW:!MEDIUM >>> >>> [mysqladmin] >>> socket = /var/lib/mysql/mysql.sock >>> port = 3306 >>> user = root >>> >>> [mysqld_safe] >>> socket = /var/lib/mysql/mysql.sock >>> port = 3306 >>> log-error = /var/log/mysql_error.log >>> ________________________________________________________________________________________________ >>> >>> 31022 9:02:20 [Note] Event Scheduler: Purging the queue. 0 events >>> 131022 9:02:20 [Note] Error reading relay log event: slave SQL thread was killed >>> 131022 9:02:20 [Note] Slave I/O thread killed during or after a reconnect done to recover from failed read >>> 131022 9:02:20 [Note] Slave I/O thread exiting, read up to log 'bin.000058', position 952251 >>> 131022 9:02:21 InnoDB: Starting shutdown... >>> 131022 9:02:22 InnoDB: Shutdown completed; log sequence number 2638210237 >>> 131022 9:02:22 [Note] /usr/libexec/mysqld: Shutdown complete >>> 131022 9:02:27 [Note] /usr/libexec/mysqld: Normal shutdown >>> 131022 9:02:27 [Note] Event Scheduler: Purging the queue. 0 events >>> 131022 9:02:28 InnoDB: Starting shutdown... >>> 131022 9:02:29 InnoDB: Shutdown completed; log sequence number 14738980420 >>> 131022 9:02:29 [Note] /usr/libexec/mysqld: Shutdown complete >>> 131022 9:03:01 InnoDB: The InnoDB memory heap is disabled >>> 131022 9:03:01 InnoDB: Mutexes and rw_locks use GCC atomic builtins >>> 131022 9:03:01 InnoDB: Compressed tables use zlib 1.2.7 >>> 131022 9:03:01 InnoDB: Using Linux native AIO >>> 131022 9:03:01 InnoDB: Initializing buffer pool, size = 50.0M >>> 131022 9:03:01 InnoDB: Completed initialization of buffer pool >>> 131022 9:03:02 InnoDB: highest supported file format is Barracuda. >>> 131022 9:03:02 InnoDB: The InnoDB memory heap is disabled >>> 131022 9:03:02 InnoDB: Mutexes and rw_locks use GCC atomic builtins >>> 131022 9:03:02 InnoDB: Compressed tables use zlib 1.2.7 >>> 131022 9:03:02 InnoDB: Using Linux native AIO >>> 131022 9:03:02 InnoDB: Initializing buffer pool, size = 64.0M >>> 131022 9:03:02 InnoDB: Completed initialization of buffer pool >>> 131022 9:03:02 InnoDB: highest supported file format is Barracuda. >>> 131022 9:03:11 InnoDB: Waiting for the background threads to start >>> 131022 9:03:11 InnoDB: Waiting for the background threads to start >>> 131022 9:03:12 Percona XtraDB (http://www.percona.com) 5.5.33a-MariaDB-31.1 started; log sequence number 2638210237 >>> 131022 9:03:12 [Note] Plugin 'FEDERATED' is disabled. >>> 131022 9:03:12 [Note] Plugin 'FEEDBACK' is disabled. >>> 131022 9:03:12 [Note] Plugin 'partition' is disabled. >>> 131022 9:03:12 [Note] Server socket created on IP: '127.0.0.1'. >>> 131022 9:03:12 Percona XtraDB (http://www.percona.com) 5.5.33a-MariaDB-31.1 started; log sequence number 14738980420 >>> 131022 9:03:12 [Note] Plugin 'FEDERATED' is disabled. >>> 131022 9:03:12 [Note] Plugin 'FEEDBACK' is disabled. >>> 131022 9:03:12 [Note] Plugin 'partition' is disabled. >>> 131022 9:03:14 [Note] Slave SQL thread initialized, starting replication in log 'bin.000058' at position 952251, >>> relay log './mysql-relay-bin.000140' position: 952529 >>> 131022 9:03:14 [Note] Server socket created on IP: '0.0.0.0'. >>> 131022 9:03:14 [Note] Slave I/O thread: connected to master 'replication@10.0.0.99:3306',replication started in >>> log 'bin.000058' at position 952251 >>> 131022 9:03:14 [Note] Event Scheduler: Loaded 0 events >>> 131022 9:03:14 [Note] /usr/libexec/mysqld: ready for connections. >>> Version: '5.5.33a-MariaDB' socket: '/var/lib/mysql/mysql_replication.sock' port: 3307 thelounge >>> 131022 9:03:14 [Note] Event Scheduler: Loaded 0 events >>> 131022 9:03:14 [Note] /usr/libexec/mysqld: ready for connections. >>> Version: '5.5.33a-MariaDB-log' socket: '/var/lib/mysql/mysql.sock' port: 3306 thelounge >>> 131022 11:30:05 [ERROR] mysqld: Table './musikarchiv/rh_meta' is marked as crashed and should be repaired >>> 131022 11:30:05 [Warning] Checking table: './musikarchiv/rh_meta' >>> 131022 11:30:05 [ERROR] musikarchiv.rh_meta: 1 client is using or hasn't closed the table properly >>> 131022 11:30:07 [ERROR] mysqld: Table './lounge_panel/cms1_meta' is marked as crashed and should be repaired >>> 131022 11:30:07 [Warning] Checking table: './lounge_panel/cms1_meta' >>> 131022 11:30:07 [ERROR] lounge_panel.cms1_meta: 1 client is using or hasn't closed the table properly >>> ________________________________________________________________________________________________ >>> >>> [builduser@testserver:~]$ cat .rpmrc >>> optflags: x86_64 -m64 -O2 -march=corei7 -mtune=corei7 -fopenmp -mmmx -msse2 -msse3 -msse4.1 -msse4.2 -maes >>> -mfpmath=sse -pipe -fomit-frame-pointer -finline-functions -finline-limit=60 -fexceptions -fstack-protector-strong >>> --param=ssp-buffer-size=8 -D_FORTIFY_SOURCE=2 >>> >>> export CFLAGS="%{optflags} -O3 -fno-stack-protector -fstack-protector --param=ssp-buffer-size=8 -fPIC >>> -fomit-frame-pointer -fno-exceptions -ffixed-ebp -fwrapv -fno-strict-aliasing -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 >>> -D_LARGEFILE_SOURCE" >>> export CXXFLAGS="$CFLAGS" >>> export FFLAGS="$CFLAGS" >>> export FCFLAGS="$CFLAGS" >>> export LDFLAGS="-Wl,-z,now -Wl,-z,relro,-z,noexecstack -pie" >>> export SH_LDFLAGS="$LDFLAGS" >>> >>> cmake . -DBUILD_CONFIG=mysql_release \ >>> -DFEATURE_SET="community" \ >>> -DCMAKE_INSTALL_PREFIX="%{_prefix}" \ >>> -DINSTALL_INCLUDEDIR=include/mysql \ >>> -DINSTALL_LAYOUT=RPM \ >>> -DINSTALL_LIBDIR="%{_lib}/mysql" \ >>> -DINSTALL_MANDIR=share/man \ >>> -DINSTALL_MYSQLSHAREDIR=share/mysql \ >>> -DINSTALL_MYSQLTESTDIR=share/mysql-test \ >>> -DINSTALL_PLUGINDIR="%{_lib}/mysql/plugin" \ >>> -DINSTALL_SBINDIR=libexec \ >>> -DINSTALL_SCRIPTDIR=bin \ >>> -DINSTALL_SQLBENCHDIR= \ >>> -DINSTALL_SUPPORTFILESDIR=share/mysql \ >>> -DMYSQL_DATADIR="%{_sharedstatedir}/mysql" \ >>> -DMYSQL_UNIX_ADDR="%{_sharedstatedir}/mysql/mysql.sock" \ >>> -DWITH_XTRADB_STORAGE_ENGINE=ON \ >>> -DWITH_PARTITION_STORAGE_ENGINE=ON \ >>> -DWITH_ARCHIVE_STORAGE_ENGINE=OFF \ >>> -DWITH_BLACKHOLE_STORAGE_ENGINE=OFF \ >>> -DWITH_SPHINX_STORAGE_ENGINE=OFF \ >>> -DENABLED_PROFILING=OFF \ >>> -DENABLE_DTRACE=OFF \ >>> -DENABLE_DEBUG_SYNC=OFF \ >>> -DWITHOUT_TOKUDB=ON \ >>> -DWITH_VALGRIND=OFF \ >>> -DWITH_SAFEMALLOC=OFF \ >>> -DWITH_EMBEDDED_SERVER=OFF \ >>> -DWITH_FEEDBACK=OFF \ >>> -DWITH_LIBWRAP=OFF \ >>> -DWITH_QUERY_CACHE_INFO=OFF \ >>> -DWITH_READLINE=OFF \ >>> -DUSE_NEW_READLINE_INTERFACE=ON \ >>> -DENABLED_LOCAL_INFILE=ON \ >>> -DNOT_FOR_DISTRIBUTION=ON \ >>> -DWITH_FAST_MUTEXES=ON \ >>> -DWITH_ATOMIC_OPS=smp \ >>> -DWITHOUT_DYNAMIC_PLUGINS=ON \ >>> -DWITH_PIC=ON \ >>> -DWITH_JEMALLOC=system \ >>> -DWITH_SSL=system \ >>> -DWITH_ZLIB=system \ >>> -DCOMPILATION_COMMENT="thelounge"

2 2

[Maria-discuss] New Question: MariaDB Partitioning and Stored Procedure Support
by AskMonty KB 26 Oct '13

26 Oct '13

Hello, A new question has been asked in "MariaDB FAQ" by htaylor: -------------------------------- Does MariaDB support the use of stored procedures with partitioning of tables. -------------------------------- To view or answer this question please visit: http://mariadb.com/kb/en/mariadb-partitioning-and-stored-procedure-support/

1 0

[Maria-discuss] Synchronization down after reboot
by Guillaume Hilt 25 Oct '13

25 Oct '13

Hello, I'm using MariaDB with Galera replication on 3 Ubuntu 12.04 servers : Server version: 5.5.33a-MariaDB-1~precise mariadb.org binary distribution, wsrep_23.7.6.rXXXX They're all development servers, in the same local network, so there's no firewall between them. They were all working fine until a reboot. Now node 2 and 3 aren't in sync anymore and mysql refuse to start. Replication is done using xtrabackup. I can start over by doing a mysqlhotcopy on node1 and put the data on node2 and node3 but i'd like to understand why it didn't survived a reboot. Here's my nodes : * Node 1 : dev-sql-001 / 192.168.3.121 * Node 2 : dev-sql-002 / 192.168.3.221 * Node 3 : dev-sql-003 / 192.168.3.222 Here's the log on node 1 when I try to start node 2 : Oct 22 10:19:29 dev-sql-001 mysqld: 131022 10:19:29 [Note] WSREP: declaring abbffcb5-3af2-11e3-a043-5b38dfb689cf stable Oct 22 10:19:29 dev-sql-001 mysqld: 131022 10:19:29 [Note] WSREP: Node f743192f-3aee-11e3-a47b-97626a725dde state prim Oct 22 10:19:29 dev-sql-001 mysqld: 131022 10:19:29 [Note] WSREP: view(view_id(PRIM,abbffcb5-3af2-11e3-a043-5b38dfb689cf,6) memb { Oct 22 10:19:29 dev-sql-001 mysqld: #011abbffcb5-3af2-11e3-a043-5b38dfb689cf, Oct 22 10:19:29 dev-sql-001 mysqld: #011f743192f-3aee-11e3-a47b-97626a725dde, Oct 22 10:19:29 dev-sql-001 mysqld: } joined { Oct 22 10:19:29 dev-sql-001 mysqld: } left { Oct 22 10:19:29 dev-sql-001 mysqld: } partitioned { Oct 22 10:19:29 dev-sql-001 mysqld: }) Oct 22 10:19:29 dev-sql-001 mysqld: 131022 10:19:29 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 1, memb_num = 2 Oct 22 10:19:29 dev-sql-001 mysqld: 131022 10:19:29 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID. Oct 22 10:19:29 dev-sql-001 mysqld: 131022 10:19:29 [Note] WSREP: STATE EXCHANGE: sent state msg: ac5a1fa8-3af2-11e3-ba64-7e36060f57f6 Oct 22 10:19:29 dev-sql-001 mysqld: 131022 10:19:29 [Note] WSREP: STATE EXCHANGE: got state msg: ac5a1fa8-3af2-11e3-ba64-7e36060f57f6 from 0 (dev-sql-002) Oct 22 10:19:29 dev-sql-001 mysqld: 131022 10:19:29 [Note] WSREP: STATE EXCHANGE: got state msg: ac5a1fa8-3af2-11e3-ba64-7e36060f57f6 from 1 (dev-sql-001) Oct 22 10:19:29 dev-sql-001 mysqld: 131022 10:19:29 [Note] WSREP: Quorum results: Oct 22 10:19:29 dev-sql-001 mysqld: #011version = 2, Oct 22 10:19:29 dev-sql-001 mysqld: #011component = PRIMARY, Oct 22 10:19:29 dev-sql-001 mysqld: #011conf_id = 5, Oct 22 10:19:29 dev-sql-001 mysqld: #011members = 1/2 (joined/total), Oct 22 10:19:29 dev-sql-001 mysqld: #011act_id = 968, Oct 22 10:19:29 dev-sql-001 mysqld: #011last_appl. = 0, Oct 22 10:19:29 dev-sql-001 mysqld: #011protocols = 0/4/2 (gcs/repl/appl), Oct 22 10:19:29 dev-sql-001 mysqld: #011group UUID = 2896facc-2110-11e3-0800-717f6d7ec2af Oct 22 10:19:29 dev-sql-001 mysqld: 131022 10:19:29 [Note] WSREP: Flow-control interval: [23, 23] Oct 22 10:19:29 dev-sql-001 mysqld: 131022 10:19:29 [Note] WSREP: New cluster view: global state: 2896facc-2110-11e3-0800-717f6d7ec2af:968, view# 6: Primary, number of nodes: 2, my index: 1, protocol version 2 Oct 22 10:19:29 dev-sql-001 mysqld: 131022 10:19:29 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. Oct 22 10:19:29 dev-sql-001 mysqld: 131022 10:19:29 [Note] WSREP: Assign initial position for certification: 968, protocol version: 2 Oct 22 10:19:32 dev-sql-001 mysqld: 131022 10:19:32 [Note] WSREP: Node 0 (dev-sql-002) requested state transfer from '*any*'. Selected 1 (dev-sql-001)(SYNCED) as donor. Oct 22 10:19:32 dev-sql-001 mysqld: 131022 10:19:32 [Note] WSREP: Shifting SYNCED -> DONOR/DESYNCED (TO: 968) Oct 22 10:19:32 dev-sql-001 mysqld: 131022 10:19:32 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. Oct 22 10:19:32 dev-sql-001 mysqld: 131022 10:19:32 [Note] WSREP: Running: 'wsrep_sst_xtrabackup --role 'donor' --address '192.168.3.221:4444/xtrabackup_sst' --auth 'root:test' --socket '/var/run/mysqld/mysqld.sock' --datadir '/var/lib/mysql/' --defaults-file '/etc/mysql/my.cnf' --gtid '2896facc-2110-11e3-0800-717f6d7ec2af:968'' Oct 22 10:19:32 dev-sql-001 mysqld: 131022 10:19:32 [Note] WSREP: sst_donor_thread signaled with 0 Oct 22 10:19:32 dev-sql-001 mysqld: WSREP_SST: [INFO] Streaming with tar (20131022 10:19:32.294) Oct 22 10:19:32 dev-sql-001 mysqld: WSREP_SST: [INFO] Note: Advanced xtrabackup features - encryption,compression etc. not available with tar. (20131022 10:19:32.301) Oct 22 10:19:32 dev-sql-001 mysqld: WSREP_SST: [INFO] Streaming the backup to joiner at 192.168.3.221 4444 (20131022 10:19:32.330) Oct 22 10:19:32 dev-sql-001 mysqld: WSREP_SST: [ERROR] innobackupex finished with error: 1. Check /var/lib/mysql//innobackup.backup.log (20131022 10:19:32.662) Oct 22 10:19:32 dev-sql-001 mysqld: WSREP_SST: [ERROR] Cleanup after exit with status:22 (20131022 10:19:32.668) Oct 22 10:19:32 dev-sql-001 mysqld: 131022 10:19:32 [ERROR] WSREP: Failed to read from: wsrep_sst_xtrabackup --role 'donor' --address '192.168.3.221:4444/xtrabackup_sst' --auth 'root:test' --socket '/var/run/mysqld/mysqld.sock' --datadir '/var/lib/mysql/' --defaults-file '/etc/mysql/my.cnf' --gtid '2896facc-2110-11e3-0800-717f6d7ec2af:968' Oct 22 10:19:32 dev-sql-001 mysqld: 131022 10:19:32 [ERROR] WSREP: Process completed with error: wsrep_sst_xtrabackup --role 'donor' --address '192.168.3.221:4444/xtrabackup_sst' --auth 'root:test' --socket '/var/run/mysqld/mysqld.sock' --datadir '/var/lib/mysql/' --defaults-file '/etc/mysql/my.cnf' --gtid '2896facc-2110-11e3-0800-717f6d7ec2af:968': 22 (Invalid argument) Oct 22 10:19:32 dev-sql-001 mysqld: 131022 10:19:32 [Warning] WSREP: 1 (dev-sql-001): State transfer to 0 (dev-sql-002) failed: -1 (Operation not permitted) Oct 22 10:19:32 dev-sql-001 mysqld: 131022 10:19:32 [Note] WSREP: Shifting DONOR/DESYNCED -> JOINED (TO: 968) Oct 22 10:19:33 dev-sql-001 mysqld: 131022 10:19:33 [Note] WSREP: Member 1 (dev-sql-001) synced with group. Oct 22 10:19:33 dev-sql-001 mysqld: 131022 10:19:33 [Note] WSREP: Shifting JOINED -> SYNCED (TO: 968) Oct 22 10:19:33 dev-sql-001 mysqld: 131022 10:19:33 [Note] WSREP: Synchronized with group, ready for connections Oct 22 10:19:33 dev-sql-001 mysqld: 131022 10:19:33 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. Oct 22 10:19:33 dev-sql-001 mysqld: 131022 10:19:33 [Note] WSREP: Node f743192f-3aee-11e3-a47b-97626a725dde state prim Oct 22 10:19:33 dev-sql-001 mysqld: 131022 10:19:33 [Note] WSREP: view(view_id(PRIM,f743192f-3aee-11e3-a47b-97626a725dde,7) memb { Oct 22 10:19:33 dev-sql-001 mysqld: #011f743192f-3aee-11e3-a47b-97626a725dde, Oct 22 10:19:33 dev-sql-001 mysqld: } joined { Oct 22 10:19:33 dev-sql-001 mysqld: } left { Oct 22 10:19:33 dev-sql-001 mysqld: } partitioned { Oct 22 10:19:33 dev-sql-001 mysqld: #011abbffcb5-3af2-11e3-a043-5b38dfb689cf, Oct 22 10:19:33 dev-sql-001 mysqld: }) Oct 22 10:19:33 dev-sql-001 mysqld: 131022 10:19:33 [Note] WSREP: forgetting abbffcb5-3af2-11e3-a043-5b38dfb689cf (tcp://192.168.3.221:4567) Oct 22 10:19:33 dev-sql-001 mysqld: 131022 10:19:33 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 0, memb_num = 1 Oct 22 10:19:33 dev-sql-001 mysqld: 131022 10:19:33 [Note] WSREP: STATE_EXCHANGE: sent state UUID: ae9636c9-3af2-11e3-b69e-0fef55014e34 Oct 22 10:19:33 dev-sql-001 mysqld: 131022 10:19:33 [Note] WSREP: STATE EXCHANGE: sent state msg: ae9636c9-3af2-11e3-b69e-0fef55014e34 Oct 22 10:19:33 dev-sql-001 mysqld: 131022 10:19:33 [Note] WSREP: STATE EXCHANGE: got state msg: ae9636c9-3af2-11e3-b69e-0fef55014e34 from 0 (dev-sql-001) Oct 22 10:19:33 dev-sql-001 mysqld: 131022 10:19:33 [Note] WSREP: Quorum results: Oct 22 10:19:33 dev-sql-001 mysqld: #011version = 2, Oct 22 10:19:33 dev-sql-001 mysqld: #011component = PRIMARY, Oct 22 10:19:33 dev-sql-001 mysqld: #011conf_id = 6, Oct 22 10:19:33 dev-sql-001 mysqld: #011members = 1/1 (joined/total), Oct 22 10:19:33 dev-sql-001 mysqld: #011act_id = 968, Oct 22 10:19:33 dev-sql-001 mysqld: #011last_appl. = 0, Oct 22 10:19:33 dev-sql-001 mysqld: #011protocols = 0/4/2 (gcs/repl/appl), Oct 22 10:19:33 dev-sql-001 mysqld: #011group UUID = 2896facc-2110-11e3-0800-717f6d7ec2af Oct 22 10:19:33 dev-sql-001 mysqld: 131022 10:19:33 [Note] WSREP: Flow-control interval: [16, 16] Oct 22 10:19:33 dev-sql-001 mysqld: 131022 10:19:33 [Note] WSREP: New cluster view: global state: 2896facc-2110-11e3-0800-717f6d7ec2af:968, view# 7: Primary, number of nodes: 1, my index: 0, protocol version 2 Oct 22 10:19:33 dev-sql-001 mysqld: 131022 10:19:33 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. Oct 22 10:19:33 dev-sql-001 mysqld: 131022 10:19:33 [Note] WSREP: Assign initial position for certification: 968, protocol version: 2 Oct 22 10:19:38 dev-sql-001 mysqld: 131022 10:19:38 [Note] WSREP: cleaning up abbffcb5-3af2-11e3-a043-5b38dfb689cf (tcp://192.168.3.221:4567) /var/lib/mysql//innobackup.backup.log on node1 : 131022 10:19:32 innobackupex: Connecting to MySQL server with DSN 'dbi:mysql:;mysql_read_default_file=/etc/mysql/my.cnf;mysql_read_default_group=xtrabackup;mysql_socket=/var/run/mysqld/mysqld.sock' as 'root' (using password: YES). innobackupex: Error: Failed to connect to MySQL server: DBI connect(';mysql_read_default_file=/etc/mysql/my.cnf;mysql_read_default_group=xtrabackup;mysql_socket=/var/run/mysqld/mysqld.sock','root',...) failed: Access denied for user 'root'@'localhost' (using password: YES) at /usr//bin/innobackupex line 2902 As for node2 logs : Oct 22 10:19:25 dev-sql-002 mysqld_safe: Starting mysqld daemon with databases from /var/lib/mysql Oct 22 10:19:25 dev-sql-002 mysqld_safe: WSREP: Running position recovery with --log_error=/tmp/tmp.gp3R8iSBdK --pid-file=/var/lib/mysql/dev-sql-002-recover.pid Oct 22 10:19:25 dev-sql-002 mysqld: 131022 10:19:25 [Warning] option 'table_cache': unsigned value 2097152 adjusted to 524288 Oct 22 10:19:28 dev-sql-002 mysqld_safe: WSREP: Recovered position 2896facc-2110-11e3-0800-717f6d7ec2af:370 Oct 22 10:19:28 dev-sql-002 mysqld: 131022 10:19:28 [Warning] option 'table_cache': unsigned value 2097152 adjusted to 524288 Oct 22 10:19:28 dev-sql-002 mysqld: 131022 10:19:28 [Note] WSREP: wsrep_start_position var submitted: '2896facc-2110-11e3-0800-717f6d7ec2af:370' Oct 22 10:19:28 dev-sql-002 mysqld: 131022 10:19:28 [Note] WSREP: Read nil XID from storage engines, skipping position init Oct 22 10:19:28 dev-sql-002 mysqld: 131022 10:19:28 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib/galera/libgalera_smm.so' Oct 22 10:19:28 dev-sql-002 mysqld: 131022 10:19:28 [Note] WSREP: wsrep_load(): Galera 23.2.7-wheezy(r) by Codership Oy <info(a)codership.com> loaded succesfully. Oct 22 10:19:28 dev-sql-002 mysqld: 131022 10:19:28 [Note] WSREP: Found saved state: 00000000-0000-0000-0000-000000000000:-1 Oct 22 10:19:28 dev-sql-002 mysqld: 131022 10:19:28 [Note] WSREP: Reusing existing '/var/lib/mysql//galera.cache'. Oct 22 10:19:28 dev-sql-002 mysqld: 131022 10:19:28 [Note] WSREP: Passing config to GCS: base_host = 192.168.3.221; base_port = 4567; cert.log_conflicts = no; gcache.dir = /var/lib/mysql/; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = /var/lib/mysql//galera.cache; gcache.page_size = 128M; gcache.size = 128M; gcs.fc_debug = 0; gcs.fc_factor = 1; gcs.fc_limit = 16; gcs.fc_master_slave = NO; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = NO; replicator.causal_read_timeout = PT30S; replicator.commit_order = 3 Oct 22 10:19:28 dev-sql-002 mysqld: 131022 10:19:28 [Note] WSREP: Assign initial position for certification: -1, protocol version: -1 Oct 22 10:19:28 dev-sql-002 mysqld: 131022 10:19:28 [Note] WSREP: wsrep_sst_grab() Oct 22 10:19:28 dev-sql-002 mysqld: 131022 10:19:28 [Note] WSREP: Start replication Oct 22 10:19:28 dev-sql-002 mysqld: 131022 10:19:28 [Note] WSREP: Setting initial position to 00000000-0000-0000-0000-000000000000:-1 Oct 22 10:19:28 dev-sql-002 mysqld: 131022 10:19:28 [Note] WSREP: protonet asio version 0 Oct 22 10:19:28 dev-sql-002 mysqld: 131022 10:19:28 [Note] WSREP: backend: asio Oct 22 10:19:28 dev-sql-002 mysqld: 131022 10:19:28 [Note] WSREP: GMCast version 0 Oct 22 10:19:28 dev-sql-002 mysqld: 131022 10:19:28 [Note] WSREP: (abbffcb5-3af2-11e3-a043-5b38dfb689cf, 'tcp://0.0.0.0:4567') listening at tcp://0.0.0.0:4567 Oct 22 10:19:28 dev-sql-002 mysqld: 131022 10:19:28 [Note] WSREP: (abbffcb5-3af2-11e3-a043-5b38dfb689cf, 'tcp://0.0.0.0:4567') multicast: , ttl: 1 Oct 22 10:19:28 dev-sql-002 mysqld: 131022 10:19:28 [Note] WSREP: EVS version 0 Oct 22 10:19:28 dev-sql-002 mysqld: 131022 10:19:28 [Note] WSREP: PC version 0 Oct 22 10:19:28 dev-sql-002 mysqld: 131022 10:19:28 [Note] WSREP: gcomm: connecting to group 'dev-sql', peer 'dev-sql-001:,dev-sql-002:' Oct 22 10:19:28 dev-sql-002 mysqld: 131022 10:19:28 [Warning] WSREP: (abbffcb5-3af2-11e3-a043-5b38dfb689cf, 'tcp://0.0.0.0:4567') address 'tcp://192.168.3.221:4567' points to own listening address, blacklisting Oct 22 10:19:28 dev-sql-002 mysqld: 131022 10:19:28 [Note] WSREP: (abbffcb5-3af2-11e3-a043-5b38dfb689cf, 'tcp://0.0.0.0:4567') address 'tcp://192.168.3.221:4567' pointing to uuid abbffcb5-3af2-11e3-a043-5b38dfb689cf is blacklisted, skipping Oct 22 10:19:29 dev-sql-002 mysqld: 131022 10:19:29 [Note] WSREP: declaring f743192f-3aee-11e3-a47b-97626a725dde stable Oct 22 10:19:29 dev-sql-002 mysqld: 131022 10:19:29 [Note] WSREP: Node f743192f-3aee-11e3-a47b-97626a725dde state prim Oct 22 10:19:29 dev-sql-002 mysqld: 131022 10:19:29 [Note] WSREP: view(view_id(PRIM,abbffcb5-3af2-11e3-a043-5b38dfb689cf,6) memb { Oct 22 10:19:29 dev-sql-002 mysqld: #011abbffcb5-3af2-11e3-a043-5b38dfb689cf, Oct 22 10:19:29 dev-sql-002 mysqld: #011f743192f-3aee-11e3-a47b-97626a725dde, Oct 22 10:19:29 dev-sql-002 mysqld: } joined { Oct 22 10:19:29 dev-sql-002 mysqld: } left { Oct 22 10:19:29 dev-sql-002 mysqld: } partitioned { Oct 22 10:19:29 dev-sql-002 mysqld: }) Oct 22 10:19:29 dev-sql-002 mysqld: 131022 10:19:29 [Note] WSREP: gcomm: connected Oct 22 10:19:29 dev-sql-002 mysqld: 131022 10:19:29 [Note] WSREP: Changing maximum packet size to 64500, resulting msg size: 32636 Oct 22 10:19:29 dev-sql-002 mysqld: 131022 10:19:29 [Note] WSREP: Shifting CLOSED -> OPEN (TO: 0) Oct 22 10:19:29 dev-sql-002 mysqld: 131022 10:19:29 [Note] WSREP: Opened channel 'dev-sql' Oct 22 10:19:29 dev-sql-002 mysqld: 131022 10:19:29 [Note] WSREP: Waiting for SST to complete. Oct 22 10:19:29 dev-sql-002 mysqld: 131022 10:19:29 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 0, memb_num = 2 Oct 22 10:19:29 dev-sql-002 mysqld: 131022 10:19:29 [Note] WSREP: STATE_EXCHANGE: sent state UUID: ac5a1fa8-3af2-11e3-ba64-7e36060f57f6 Oct 22 10:19:29 dev-sql-002 mysqld: 131022 10:19:29 [Note] WSREP: STATE EXCHANGE: sent state msg: ac5a1fa8-3af2-11e3-ba64-7e36060f57f6 Oct 22 10:19:29 dev-sql-002 mysqld: 131022 10:19:29 [Note] WSREP: STATE EXCHANGE: got state msg: ac5a1fa8-3af2-11e3-ba64-7e36060f57f6 from 0 (dev-sql-002) Oct 22 10:19:29 dev-sql-002 mysqld: 131022 10:19:29 [Note] WSREP: STATE EXCHANGE: got state msg: ac5a1fa8-3af2-11e3-ba64-7e36060f57f6 from 1 (dev-sql-001) Oct 22 10:19:29 dev-sql-002 mysqld: 131022 10:19:29 [Note] WSREP: Quorum results: Oct 22 10:19:29 dev-sql-002 mysqld: #011version = 2, Oct 22 10:19:29 dev-sql-002 mysqld: #011component = PRIMARY, Oct 22 10:19:29 dev-sql-002 mysqld: #011conf_id = 5, Oct 22 10:19:29 dev-sql-002 mysqld: #011members = 1/2 (joined/total), Oct 22 10:19:29 dev-sql-002 mysqld: #011act_id = 968, Oct 22 10:19:29 dev-sql-002 mysqld: #011last_appl. = -1, Oct 22 10:19:29 dev-sql-002 mysqld: #011protocols = 0/4/2 (gcs/repl/appl), Oct 22 10:19:29 dev-sql-002 mysqld: #011group UUID = 2896facc-2110-11e3-0800-717f6d7ec2af Oct 22 10:19:29 dev-sql-002 mysqld: 131022 10:19:29 [Note] WSREP: Flow-control interval: [23, 23] Oct 22 10:19:29 dev-sql-002 mysqld: 131022 10:19:29 [Note] WSREP: Shifting OPEN -> PRIMARY (TO: 968) Oct 22 10:19:29 dev-sql-002 mysqld: 131022 10:19:29 [Note] WSREP: State transfer required: Oct 22 10:19:29 dev-sql-002 mysqld: #011Group state: 2896facc-2110-11e3-0800-717f6d7ec2af:968 Oct 22 10:19:29 dev-sql-002 mysqld: #011Local state: 00000000-0000-0000-0000-000000000000:-1 Oct 22 10:19:29 dev-sql-002 mysqld: 131022 10:19:29 [Note] WSREP: New cluster view: global state: 2896facc-2110-11e3-0800-717f6d7ec2af:968, view# 6: Primary, number of nodes: 2, my index: 0, protocol version 2 Oct 22 10:19:29 dev-sql-002 mysqld: 131022 10:19:29 [Warning] WSREP: Gap in state sequence. Need state transfer. Oct 22 10:19:31 dev-sql-002 mysqld: 131022 10:19:31 [Note] WSREP: Running: 'wsrep_sst_xtrabackup --role 'joiner' --address '192.168.3.221' --auth 'root:test' --datadir '/var/lib/mysql/' --defaults-file '/etc/mysql/my.cnf' --parent '13950'' Oct 22 10:19:31 dev-sql-002 mysqld: WSREP_SST: [INFO] Streaming with tar (20131022 10:19:31.981) Oct 22 10:19:31 dev-sql-002 mysqld: WSREP_SST: [INFO] Note: Advanced xtrabackup features - encryption,compression etc. not available with tar. (20131022 10:19:31.988) Oct 22 10:19:31 dev-sql-002 mysqld: WSREP_SST: [INFO] However, xbstream requires manual cleanup of data directory before SST - lp:1193240. (20131022 10:19:31.994) Oct 22 10:19:32 dev-sql-002 mysqld: 131022 10:19:32 [Note] WSREP: Prepared SST request: xtrabackup|192.168.3.221:4444/xtrabackup_sst Oct 22 10:19:32 dev-sql-002 mysqld: 131022 10:19:32 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. Oct 22 10:19:32 dev-sql-002 mysqld: 131022 10:19:32 [Note] WSREP: Assign initial position for certification: 968, protocol version: 2 Oct 22 10:19:32 dev-sql-002 mysqld: 131022 10:19:32 [Warning] WSREP: Failed to prepare for incremental state transfer: Local state UUID (00000000-0000-0000-0000-000000000000) does not match group state UUID (2896facc-2110-11e3-0800-717f6d7ec2af): 1 (Operation not permitted) Oct 22 10:19:32 dev-sql-002 mysqld: #011 at galera/src/replicator_str.cpp:prepare_for_IST():436. IST will be unavailable. Oct 22 10:19:32 dev-sql-002 mysqld: 131022 10:19:32 [Note] WSREP: Node 0 (dev-sql-002) requested state transfer from '*any*'. Selected 1 (dev-sql-001)(SYNCED) as donor. Oct 22 10:19:32 dev-sql-002 mysqld: 131022 10:19:32 [Note] WSREP: Shifting PRIMARY -> JOINER (TO: 968) Oct 22 10:19:32 dev-sql-002 mysqld: 131022 10:19:32 [Note] WSREP: Requesting state transfer: success, donor: 1 Oct 22 10:19:32 dev-sql-002 mysqld: tar: This does not look like a tar archive Oct 22 10:19:32 dev-sql-002 mysqld: tar: Exiting with failure status due to previous errors Oct 22 10:19:32 dev-sql-002 mysqld: WSREP_SST: [ERROR] Error while getting data from donor node: exit codes: 0 2 (20131022 10:19:32.659) Oct 22 10:19:32 dev-sql-002 mysqld: WSREP_SST: [ERROR] Data directory /var/lib/mysql/ needs to be empty for SST: Manual intervention required in that case (20131022 10:19:32.666) Oct 22 10:19:32 dev-sql-002 mysqld: WSREP_SST: [ERROR] Cleanup after exit with status:32 (20131022 10:19:32.672) Oct 22 10:19:32 dev-sql-002 mysqld: 131022 10:19:32 [Warning] WSREP: 1 (dev-sql-001): State transfer to 0 (dev-sql-002) failed: -1 (Operation not permitted) Oct 22 10:19:32 dev-sql-002 mysqld: 131022 10:19:32 [ERROR] WSREP: gcs/src/gcs_group.c:gcs_group_handle_join_msg():719: Will never receive state. Need to abort. Oct 22 10:19:32 dev-sql-002 mysqld: 131022 10:19:32 [Note] WSREP: gcomm: terminating thread Oct 22 10:19:32 dev-sql-002 mysqld: 131022 10:19:32 [Note] WSREP: gcomm: joining thread Oct 22 10:19:32 dev-sql-002 mysqld: 131022 10:19:32 [Note] WSREP: gcomm: closing backend Oct 22 10:19:32 dev-sql-002 mysqld: WSREP_SST: [ERROR] Killing nc pid (20131022 10:19:32.707) Oct 22 10:19:32 dev-sql-002 mysqld: WSREP_SST: [INFO] Removing the sst_in_progress file (20131022 10:19:32.725) Oct 22 10:19:32 dev-sql-002 mysqld: 131022 10:19:32 [ERROR] WSREP: Process completed with error: wsrep_sst_xtrabackup --role 'joiner' --address '192.168.3.221' --auth 'root:test' --datadir '/var/lib/mysql/' --defaults-file '/etc/mysql/my.cnf' --parent '13950': 32 (Broken pipe) Oct 22 10:19:32 dev-sql-002 mysqld: 131022 10:19:32 [ERROR] WSREP: Failed to read uuid:seqno from joiner script. Oct 22 10:19:32 dev-sql-002 mysqld: 131022 10:19:32 [ERROR] WSREP: SST failed: 32 (Broken pipe) Oct 22 10:19:32 dev-sql-002 mysqld: 131022 10:19:32 [ERROR] Aborting Oct 22 10:19:32 dev-sql-002 mysqld: Oct 22 10:19:33 dev-sql-002 mysqld: 131022 10:19:33 [Note] WSREP: view(view_id(NON_PRIM,abbffcb5-3af2-11e3-a043-5b38dfb689cf,6) memb { Oct 22 10:19:33 dev-sql-002 mysqld: #011abbffcb5-3af2-11e3-a043-5b38dfb689cf, Oct 22 10:19:33 dev-sql-002 mysqld: } joined { Oct 22 10:19:33 dev-sql-002 mysqld: } left { Oct 22 10:19:33 dev-sql-002 mysqld: } partitioned { Oct 22 10:19:33 dev-sql-002 mysqld: #011f743192f-3aee-11e3-a47b-97626a725dde, Oct 22 10:19:33 dev-sql-002 mysqld: }) Oct 22 10:19:33 dev-sql-002 mysqld: 131022 10:19:33 [Note] WSREP: view((empty)) Oct 22 10:19:33 dev-sql-002 mysqld: 131022 10:19:33 [Note] WSREP: gcomm: closed Oct 22 10:19:33 dev-sql-002 mysqld: 131022 10:19:33 [Note] WSREP: /usr/sbin/mysqld: Terminated. Oct 22 10:19:33 dev-sql-002 mysqld_safe: mysqld from pid file /var/run/mysqld/mysqld.pid ended Oct 22 10:19:55 dev-sql-002 /etc/init.d/mysql[14428]: 0 processes alive and '/usr/bin/mysqladmin --defaults-file=/etc/mysql/debian.cnf ping' resulted in Oct 22 10:19:55 dev-sql-002 /etc/init.d/mysql[14428]: #007/usr/bin/mysqladmin: connect to server at 'localhost' failed Oct 22 10:19:55 dev-sql-002 /etc/init.d/mysql[14428]: error: 'Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (111)' Oct 22 10:19:55 dev-sql-002 /etc/init.d/mysql[14428]: Check that mysqld is running and that the socket: '/var/run/mysqld/mysqld.sock' exists! Oct 22 10:19:55 dev-sql-002 /etc/init.d/mysql[14428]: WSREP status on node1 : MariaDB [(none)]> SHOW STATUS LIKE 'wsrep_%' -> ; +----------------------------+--------------------------------------+ | Variable_name | Value | +----------------------------+--------------------------------------+ | wsrep_local_state_uuid | 2896facc-2110-11e3-0800-717f6d7ec2af | | wsrep_protocol_version | 4 | | wsrep_last_committed | 968 | | wsrep_replicated | 0 | | wsrep_replicated_bytes | 0 | | wsrep_received | 12 | | wsrep_received_bytes | 1126 | | wsrep_local_commits | 0 | | wsrep_local_cert_failures | 0 | | wsrep_local_bf_aborts | 0 | | wsrep_local_replays | 0 | | wsrep_local_send_queue | 0 | | wsrep_local_send_queue_avg | 0.500000 | | wsrep_local_recv_queue | 0 | | wsrep_local_recv_queue_avg | 0.000000 | | wsrep_flow_control_paused | 0.000000 | | wsrep_flow_control_sent | 0 | | wsrep_flow_control_recv | 0 | | wsrep_cert_deps_distance | 0.000000 | | wsrep_apply_oooe | 0.000000 | | wsrep_apply_oool | 0.000000 | | wsrep_apply_window | 0.000000 | | wsrep_commit_oooe | 0.000000 | | wsrep_commit_oool | 0.000000 | | wsrep_commit_window | 0.000000 | | wsrep_local_state | 4 | | wsrep_local_state_comment | Synced | | wsrep_cert_index_size | 0 | | wsrep_causal_reads | 0 | | wsrep_incoming_addresses | 192.168.3.121:3306 | | wsrep_cluster_conf_id | 5 | | wsrep_cluster_size | 1 | | wsrep_cluster_state_uuid | 2896facc-2110-11e3-0800-717f6d7ec2af | | wsrep_cluster_status | Primary | | wsrep_connected | ON | | wsrep_local_index | 0 | | wsrep_provider_name | Galera | | wsrep_provider_vendor | Codership Oy <info(a)codership.com> | | wsrep_provider_version | 23.2.7-wheezy(r) | | wsrep_ready | ON | +----------------------------+--------------------------------------+ 40 rows in set (0.00 sec) Thanks, -- Guillaume Hilt

2 3

[Maria-discuss] Parallel build support for v10?
by jen142＠promessage.com 24 Oct '13

24 Oct '13

Hi, I build MariaDB v10 from src on linux/64. If, after `cmake` configure I attempt a parallel build, make -j4 VERBOSE=1 I get numerous/sequential build failures. ALL of the related failures vanish if I simple single-thread the build, make -j1 VERBOSE=1 *IS* MarbiDB v10 currently intended to be parallel-buildable? jen

2 2

[Maria-discuss] How to disable TokuDB libtokuportability.so check in 10.0.4/r3863 source build?
by jen142＠promessage.com 24 Oct '13

24 Oct '13

I'm building v10.0.4/r3863 on opensuse/64. This latest-version build newly fails @ cd /usr/local/src/mariadb/bld/storage/tokudb/ft-index/ft && ./logformat . ./logformat: error while loading shared libraries: libtokuportability.so: cannot open shared object file: No such file or directory This is changed behavior. I'm currently not interested in TokuDB. Reading @ https://mariadb.com/kb/en/how-to-enable-tokudb-in-mariadb/#enabling-tokudb it's not supported on my platform anyway. There's info on how to ENABLE it, but not how to DISABLE it -- so the build doesn't complain. How do I correctly DISABLE it in my build? Jen

2 1

[Maria-discuss] maria certifications?
by Federico Razzoli 24 Oct '13

24 Oct '13

I found a site about MariaDB certifications: http://www.mariadbcertification.org/ However, it looks like a sort of work in progress. Also, I don't understand if it is something official, or at least recommended by the Foundation. And, if it is not, is any official certification planned? Ciao Federico

2 2

[Maria-discuss] New Question: understanding Unkwnon error when connection is aborted
by AskMonty KB 24 Oct '13

24 Oct '13

Hello, A new question has been asked in "MariaDB FAQ" by jaihind213: -------------------------------- [Warning] Aborted connection 290 to db: 'test_db' user: 'dev' host: 'host.stage.com' (Unknown error) We have been getting this error in the logs. would like to know when this happens. thanks -------------------------------- To view or answer this question please visit: http://mariadb.com/kb/en/understanding-unkwnon-error-when-connection-is-abo…

1 0

[Maria-discuss] crashed tables (mariadb-5.5.33a)
by Reindl Harald 23 Oct '13

23 Oct '13

i see randomly such messages in the mysql_error.log there was no hard shutdown or killed process and so i am somehow out of ideas 131022 11:30:05 [ERROR] mysqld: Table './musikarchiv/rh_meta' is marked as crashed and should be repaired 131022 11:30:05 [Warning] Checking table: './musikarchiv/rh_meta' the only time i can remember such issues was http://lists.mysql.com/mysql/226361 but in the current situation i see nothing similar in the logs or any hint there are two instances (port 3306 / 3307) with to different systemd-units and configurations to replicate my home-machine over openvpn to the office and my office-workstation at home below the whole configuration, logfile and build-options while MySQL 5.5.34 in production is using practically the same RPM-SPEC without issues ________________________________________________________________________________________________ [root@srv-rhsoft:~]$ cat /etc/my.cnf [mysqld] socket = /var/lib/mysql/mysql.sock datadir = /mnt/data/db/mysql-srv pid-file = /mnt/data/db/mysql-srv/mysql.pid tmpdir = /tmp character-set-server = latin1 collation-server = latin1_german1_ci default-time-zone = "Europe/Vienna" default-storage-engine = myisam lower_case_table_names = 1 log-error = /var/log/mysql_error.log log_warnings = 1 slow_query_log = 1 slow_query_log_file = /var/log/mysql_slow_query.log long_query_time = 1 log_queries_not_using_indexes = 0 general_log = 0 general_log_file = /var/log/mysql_query.log port = 3306 old_passwords = 0 local-infile = 0 sql-mode = STRICT_ALL_TABLES ssl-ca = /etc/mysql-ssl/ca.crt ssl-cert = /etc/mysql-ssl/server.pem ssl-key = /etc/mysql-ssl/server.pem ssl-cipher = EECDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA384:EECDH+ECDSA+SHA256:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH:EDH+aRSA:HIGH:!3DES:!aNULL:!DSS:!eNULL:!EXP:!MD5:!PSK:!RC4:!SRP:!SSLv2:!LOW:!MEDIUM delay-key-write = ALL concurrent_insert = 2 open-files-limit = 500000 myisam-recover-options = "FORCE" myisam_use_mmap = 1 performance_schema = 0 wait_timeout = 86400 interactive_timeout = 86400 max_allowed_packet = 250M max_connections = 300 max_tmp_tables = 100 max_connect_errors = 200 max_delayed_threads = 30 flush_time = 0 query_cache_limit = 150K query_cache_min_res_unit = 2K query_cache_size = 128M query_cache_type = 1 table_cache = 5000 table_definition_cache = 512 thread_cache_size = 100 tmp_table_size = 256M max_heap_table_size = 256M key_buffer_size = 128M sort_buffer_size = 128K myisam_sort_buffer_size = 15M join_buffer_size = 1M preload_buffer_size = 128K read_buffer_size = 128K read_rnd_buffer_size = 128K innodb_buffer_pool_size = 64M innodb_buffer_pool_instances = 1 innodb_purge_threads = 1 innodb_max_purge_lag = 200000 innodb_max_dirty_pages_pct = 60 innodb_additional_mem_pool_size = 3M innodb_log_file_size = 80M innodb_log_buffer_size = 2M innodb_thread_concurrency = 0 innodb_thread_sleep_delay = 10 innodb_flush_log_at_trx_commit = 2 innodb_support_xa = 1 innodb_lock_wait_timeout = 50 innodb_table_locks = 0 innodb_checksums = 0 innodb_file_format = barracuda innodb_file_per_table = 1 innodb_open_files = 300 innodb_io_capacity = 600 innodb_read_io_threads = 4 innodb_write_io_threads = 4 innodb_adaptive_flushing_method = keep_average innodb_flush_method = ALL_O_DIRECT transaction-isolation = READ-COMMITTED server-id = 1 log-bin = /mnt/data/db/mysql-srv/bin.log log-bin-index = /mnt/data/db/mysql-srv/bin.index expire_logs_days = 21 max_binlog_size = 256M binlog-format = ROW binlog_stmt_cache_size = 256K binlog_cache_size = 256K sync_binlog = 30 low-priority-updates skip-federated skip-symbolic-links skip-name-resolve safe-user-create skip-partition slave_compressed_protocol [mariadb] aria_pagecache_buffer_size = 32m aria_sort_buffer_size = 32m aria_page_checksum = 0 key_cache_segments = 8 thread_handling = one-thread-per-connection thread_pool_idle_timeout = 60 [client] socket = /var/lib/mysql/mysql.sock port = 3306 user = root ssl-ca = /etc/mysql-ssl/ca.crt ssl-cert = /etc/mysql-ssl/client.pem ssl-key = /etc/mysql-ssl/client.pem ssl-cipher = EECDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA384:EECDH+ECDSA+SHA256:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH:EDH+aRSA:HIGH:!3DES:!aNULL:!DSS:!eNULL:!EXP:!MD5:!PSK:!RC4:!SRP:!SSLv2:!LOW:!MEDIUM [mysqladmin] socket = /var/lib/mysql/mysql.sock port = 3306 user = root [mysqld_safe] socket = /var/lib/mysql/mysql.sock port = 3306 log-error = /var/log/mysql_error.log ________________________________________________________________________________________________ 31022 9:02:20 [Note] Event Scheduler: Purging the queue. 0 events 131022 9:02:20 [Note] Error reading relay log event: slave SQL thread was killed 131022 9:02:20 [Note] Slave I/O thread killed during or after a reconnect done to recover from failed read 131022 9:02:20 [Note] Slave I/O thread exiting, read up to log 'bin.000058', position 952251 131022 9:02:21 InnoDB: Starting shutdown... 131022 9:02:22 InnoDB: Shutdown completed; log sequence number 2638210237 131022 9:02:22 [Note] /usr/libexec/mysqld: Shutdown complete 131022 9:02:27 [Note] /usr/libexec/mysqld: Normal shutdown 131022 9:02:27 [Note] Event Scheduler: Purging the queue. 0 events 131022 9:02:28 InnoDB: Starting shutdown... 131022 9:02:29 InnoDB: Shutdown completed; log sequence number 14738980420 131022 9:02:29 [Note] /usr/libexec/mysqld: Shutdown complete 131022 9:03:01 InnoDB: The InnoDB memory heap is disabled 131022 9:03:01 InnoDB: Mutexes and rw_locks use GCC atomic builtins 131022 9:03:01 InnoDB: Compressed tables use zlib 1.2.7 131022 9:03:01 InnoDB: Using Linux native AIO 131022 9:03:01 InnoDB: Initializing buffer pool, size = 50.0M 131022 9:03:01 InnoDB: Completed initialization of buffer pool 131022 9:03:02 InnoDB: highest supported file format is Barracuda. 131022 9:03:02 InnoDB: The InnoDB memory heap is disabled 131022 9:03:02 InnoDB: Mutexes and rw_locks use GCC atomic builtins 131022 9:03:02 InnoDB: Compressed tables use zlib 1.2.7 131022 9:03:02 InnoDB: Using Linux native AIO 131022 9:03:02 InnoDB: Initializing buffer pool, size = 64.0M 131022 9:03:02 InnoDB: Completed initialization of buffer pool 131022 9:03:02 InnoDB: highest supported file format is Barracuda. 131022 9:03:11 InnoDB: Waiting for the background threads to start 131022 9:03:11 InnoDB: Waiting for the background threads to start 131022 9:03:12 Percona XtraDB (http://www.percona.com) 5.5.33a-MariaDB-31.1 started; log sequence number 2638210237 131022 9:03:12 [Note] Plugin 'FEDERATED' is disabled. 131022 9:03:12 [Note] Plugin 'FEEDBACK' is disabled. 131022 9:03:12 [Note] Plugin 'partition' is disabled. 131022 9:03:12 [Note] Server socket created on IP: '127.0.0.1'. 131022 9:03:12 Percona XtraDB (http://www.percona.com) 5.5.33a-MariaDB-31.1 started; log sequence number 14738980420 131022 9:03:12 [Note] Plugin 'FEDERATED' is disabled. 131022 9:03:12 [Note] Plugin 'FEEDBACK' is disabled. 131022 9:03:12 [Note] Plugin 'partition' is disabled. 131022 9:03:14 [Note] Slave SQL thread initialized, starting replication in log 'bin.000058' at position 952251, relay log './mysql-relay-bin.000140' position: 952529 131022 9:03:14 [Note] Server socket created on IP: '0.0.0.0'. 131022 9:03:14 [Note] Slave I/O thread: connected to master 'replication@10.0.0.99:3306',replication started in log 'bin.000058' at position 952251 131022 9:03:14 [Note] Event Scheduler: Loaded 0 events 131022 9:03:14 [Note] /usr/libexec/mysqld: ready for connections. Version: '5.5.33a-MariaDB' socket: '/var/lib/mysql/mysql_replication.sock' port: 3307 thelounge 131022 9:03:14 [Note] Event Scheduler: Loaded 0 events 131022 9:03:14 [Note] /usr/libexec/mysqld: ready for connections. Version: '5.5.33a-MariaDB-log' socket: '/var/lib/mysql/mysql.sock' port: 3306 thelounge 131022 11:30:05 [ERROR] mysqld: Table './musikarchiv/rh_meta' is marked as crashed and should be repaired 131022 11:30:05 [Warning] Checking table: './musikarchiv/rh_meta' 131022 11:30:05 [ERROR] musikarchiv.rh_meta: 1 client is using or hasn't closed the table properly 131022 11:30:07 [ERROR] mysqld: Table './lounge_panel/cms1_meta' is marked as crashed and should be repaired 131022 11:30:07 [Warning] Checking table: './lounge_panel/cms1_meta' 131022 11:30:07 [ERROR] lounge_panel.cms1_meta: 1 client is using or hasn't closed the table properly ________________________________________________________________________________________________ [builduser@testserver:~]$ cat .rpmrc optflags: x86_64 -m64 -O2 -march=corei7 -mtune=corei7 -fopenmp -mmmx -msse2 -msse3 -msse4.1 -msse4.2 -maes -mfpmath=sse -pipe -fomit-frame-pointer -finline-functions -finline-limit=60 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=8 -D_FORTIFY_SOURCE=2 export CFLAGS="%{optflags} -O3 -fno-stack-protector -fstack-protector --param=ssp-buffer-size=8 -fPIC -fomit-frame-pointer -fno-exceptions -ffixed-ebp -fwrapv -fno-strict-aliasing -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE" export CXXFLAGS="$CFLAGS" export FFLAGS="$CFLAGS" export FCFLAGS="$CFLAGS" export LDFLAGS="-Wl,-z,now -Wl,-z,relro,-z,noexecstack -pie" export SH_LDFLAGS="$LDFLAGS" cmake . -DBUILD_CONFIG=mysql_release \ -DFEATURE_SET="community" \ -DCMAKE_INSTALL_PREFIX="%{_prefix}" \ -DINSTALL_INCLUDEDIR=include/mysql \ -DINSTALL_LAYOUT=RPM \ -DINSTALL_LIBDIR="%{_lib}/mysql" \ -DINSTALL_MANDIR=share/man \ -DINSTALL_MYSQLSHAREDIR=share/mysql \ -DINSTALL_MYSQLTESTDIR=share/mysql-test \ -DINSTALL_PLUGINDIR="%{_lib}/mysql/plugin" \ -DINSTALL_SBINDIR=libexec \ -DINSTALL_SCRIPTDIR=bin \ -DINSTALL_SQLBENCHDIR= \ -DINSTALL_SUPPORTFILESDIR=share/mysql \ -DMYSQL_DATADIR="%{_sharedstatedir}/mysql" \ -DMYSQL_UNIX_ADDR="%{_sharedstatedir}/mysql/mysql.sock" \ -DWITH_XTRADB_STORAGE_ENGINE=ON \ -DWITH_PARTITION_STORAGE_ENGINE=ON \ -DWITH_ARCHIVE_STORAGE_ENGINE=OFF \ -DWITH_BLACKHOLE_STORAGE_ENGINE=OFF \ -DWITH_SPHINX_STORAGE_ENGINE=OFF \ -DENABLED_PROFILING=OFF \ -DENABLE_DTRACE=OFF \ -DENABLE_DEBUG_SYNC=OFF \ -DWITHOUT_TOKUDB=ON \ -DWITH_VALGRIND=OFF \ -DWITH_SAFEMALLOC=OFF \ -DWITH_EMBEDDED_SERVER=OFF \ -DWITH_FEEDBACK=OFF \ -DWITH_LIBWRAP=OFF \ -DWITH_QUERY_CACHE_INFO=OFF \ -DWITH_READLINE=OFF \ -DUSE_NEW_READLINE_INTERFACE=ON \ -DENABLED_LOCAL_INFILE=ON \ -DNOT_FOR_DISTRIBUTION=ON \ -DWITH_FAST_MUTEXES=ON \ -DWITH_ATOMIC_OPS=smp \ -DWITHOUT_DYNAMIC_PLUGINS=ON \ -DWITH_PIC=ON \ -DWITH_JEMALLOC=system \ -DWITH_SSL=system \ -DWITH_ZLIB=system \ -DCOMPILATION_COMMENT="thelounge"

1 1