Hi! On Mon, Sep 29, 2014 at 4:40 PM, Carlos Raúl Laguna <carlosla1987@gmail.com> wrote:
The log is actually just the output of executing mysqld i just focus in this part,
address 'tcp://172.17.1.2:4567' points to own listening address, blacklisting,
However i do manage to start one node, now the problem it seems to be in the other to, here is what it show when y try to add the second node to the cluster.
140929 16:24:41 [Note] WSREP: Service thread queue flushed. 140929 16:24:41 [Note] WSREP: Assign initial position for certification: -1, protocol version: -1 140929 16:24:41 [Note] WSREP: wsrep_sst_grab() 140929 16:24:41 [Note] WSREP: Start replication 140929 16:24:41 [Note] WSREP: Setting initial position to 00000000-0000-0000-0000- 000000000000:-1 140929 16:24:41 [Note] WSREP: protonet asio version 0 140929 16:24:41 [Note] WSREP: Using CRC-32C (optimized) for message checksums. 140929 16:24:41 [Note] WSREP: backend: asio 140929 16:24:41 [Note] WSREP: GMCast version 0 140929 16:24:41 [Note] WSREP: (a47be000-4816-11e4-b0fd-43eda06d6eff, 'tcp://0.0.0.0:4567') listening at tcp://0.0.0.0:4567 140929 16:24:41 [Note] WSREP: (a47be000-4816-11e4-b0fd-43eda06d6eff, 'tcp://0.0.0.0:4567') multicast: , ttl: 1 140929 16:24:41 [Note] WSREP: EVS version 0 140929 16:24:41 [Note] WSREP: PC version 0 140929 16:24:41 [Note] WSREP: gcomm: connecting to group 'baruwa_cluster', peer 'baruwadb1:' 140929 16:24:41 [Note] WSREP: (a47be000-4816-11e4-b0fd-43eda06d6eff, 'tcp://0.0.0.0:4567') turning message relay requesting on, nonlive peers: tcp://172.17.1.4:4567 140929 16:24:41 [Note] WSREP: declaring 498ac792-480e-11e4-994f-533085d0ff8b stable 140929 16:24:41 [Note] WSREP: declaring ef83de45-4807-11e4-8802-0203dc2e072c stable 140929 16:24:41 [Note] WSREP: Node 498ac792-480e-11e4-994f-533085d0ff8b state prim 140929 16:24:41 [Note] WSREP: (a47be000-4816-11e4-b0fd-43eda06d6eff, 'tcp://0.0.0.0:4567') turning message relay requesting off 140929 16:24:41 [Note] WSREP: view(view_id(PRIM,498ac792-480e-11e4-994f-533085d0ff8b,3) memb { 498ac792-480e-11e4-994f-533085d0ff8b,0 a47be000-4816-11e4-b0fd-43eda06d6eff,0 ef83de45-4807-11e4-8802-0203dc2e072c,0 } joined { } left { } partitioned { }) 140929 16:24:41 [Note] WSREP: gcomm: connected 140929 16:24:41 [Note] WSREP: Changing maximum packet size to 64500, resulting msg size: 32636 140929 16:24:41 [Note] WSREP: Shifting CLOSED -> OPEN (TO: 0) 140929 16:24:41 [Note] WSREP: Opened channel 'baruwa_cluster' 140929 16:24:41 [Note] WSREP: Waiting for SST to complete. 140929 16:24:41 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 1, memb_num = 3 140929 16:24:41 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID. 140929 16:24:41 [Note] WSREP: STATE EXCHANGE: sent state msg: a49fe848-4816-11e4-a8c7-b70f254d3249 140929 16:24:41 [Note] WSREP: STATE EXCHANGE: got state msg: a49fe848-4816-11e4-a8c7-b70f254d3249 from 0 (baruwadb2) 140929 16:24:41 [Note] WSREP: STATE EXCHANGE: got state msg: a49fe848-4816-11e4-a8c7-b70f254d3249 from 2 (baruwadb1) 140929 16:24:41 [Note] WSREP: STATE EXCHANGE: got state msg: a49fe848-4816-11e4-a8c7-b70f254d3249 from 1 (baruwadb3) 140929 16:24:41 [Note] WSREP: Quorum results: version = 3, component = PRIMARY, conf_id = 2, members = 2/3 (joined/total), act_id = 1, last_appl. = -1, protocols = 0/5/3 (gcs/repl/appl), group UUID = 0c63fe44-47fe-11e4-8d91-d3636a34c2fc 140929 16:24:41 [Note] WSREP: Flow-control interval: [28, 28] 140929 16:24:41 [Note] WSREP: Shifting OPEN -> PRIMARY (TO: 1) 140929 16:24:41 [Note] WSREP: State transfer required: Group state: 0c63fe44-47fe-11e4-8d91-d3636a34c2fc:1 Local state: 00000000-0000-0000-0000-000000000000:-1 140929 16:24:41 [Note] WSREP: New cluster view: global state: 0c63fe44-47fe-11e4-8d91-d3636a34c2fc:1, view# 3: Primary, number of nodes: 3, my index: 1, protocol version 3 140929 16:24:41 [Warning] WSREP: Gap in state sequence. Need state transfer. 140929 16:24:43 [Note] WSREP: Running: 'wsrep_sst_rsync --role 'joiner' --address '172.17.1.12' --auth '' --datadir '/var/lib/mysql/' --defaults-file '/etc/mysql/my.cnf' --parent '22299' '' ' 140929 16:24:44 [Note] WSREP: Prepared SST request: rsync| 172.17.1.12:4444/rsync_sst 140929 16:24:44 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 140929 16:24:44 [Note] WSREP: REPL Protocols: 5 (3, 1) 140929 16:24:44 [Note] WSREP: Service thread queue flushed. 140929 16:24:44 [Note] WSREP: Assign initial position for certification: 1, protocol version: 3 140929 16:24:44 [Note] WSREP: Service thread queue flushed. 140929 16:24:44 [Warning]* WSREP: Failed to prepare for incremental state transfer: Local state UUID (00000000-0000-0000-0000-000000000000) does not match group state UUID (0c63fe44-47fe-11e4-8d91-d3636a34c2fc): 1 (Operation not permitted)* at galera/src/replicator_str.cpp:prepare_for_IST():447. IST will be unavailable. 140929 16:24:44 [Note] WSREP: Member 1.0 (baruwadb3) requested state transfer from '*any*'. Selected 0.0 (baruwadb2)(SYNCED) as donor. 140929 16:24:44 [Note] WSREP: Shifting PRIMARY -> JOINER (TO: 1) 140929 16:24:44 [Note] WSREP: Requesting state transfer: success, donor: 0
There should be more after this. Besides, you should also inspect donor logs for any issues or errors. Best, Nirbhay Any ideas ?
2014-09-29 15:55 GMT-04:00 Nirbhay Choubey <nirbhay@mariadb.com>:
Hi,
On Mon, Sep 29, 2014 at 1:11 PM, Carlos Raúl Laguna < carlosla1987@gmail.com> wrote:
I have a 3node galera-mariadb10 cluster working without issues until today in ubuntu 14.04, all node stop, all i can grasp from mysqld is this:
The provided log seem to be incomplete? I do not see any error that would have caused the node to stop or abort.
[Warning] option 'table_cache': unsigned value 2097152 adjusted to 524288 [Note] WSREP: Read nil XID from storage engines, skipping position init [Note] WSREP: wsrep_load(): loading provider library '/usr/lib/galera/libgalera_smm.so' [Note] WSREP: wsrep_load(): Galera 25.3.5-wheezy(rXXXX) by Codership Oy <info@codership.com> loaded successfully. [Note] WSREP: CRC-32C: using hardware acceleration. [Note] WSREP: Found saved state: 193c65da-169f-11e4-a58c-ba3055701076:-1 [Note] WSREP: Passing config to GCS: base_host = 172.17.1.2; base_port = 4567; cert.log_conflicts = no; debug = no; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT15S; evs.join_retrans_period = PT1S; evs.max_install_timeouts = 1; evs.send_window = 4; evs.stats_report_period = PT1M; evs.suspect_timeout = PT5S; evs.user_send_window = 2; evs.view_forget_timeout = PT24H; gcache.dir = /var/lib/mysql/; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = /var/lib/mysql//galera.cache; gcache.page_size = 128M; gcache.size = 128M; gcs.fc_debug = 0; gcs.fc_factor = 1.0; gcs.fc_limit = 16; gcs.fc_master_slave = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = no; gmcast.segment = 0; gmcast.version = 0; pc.announce_timeout = PT3S; pc.checksum = false; pc.ignore_quorum = false; pc.ignore_sb = false; pc.npvo = false; pc.version = 0; pc.wait_prim = true; pc.wait_prim_timeout = P30S; pc.weight = 1; protonet. Note] WSREP: Service thread queue flushed. [Note] WSREP: Assign initial position for certification: -1, protocol version: -1 [Note] WSREP: wsrep_sst_grab() [Note] WSREP: Start replication [Note] WSREP: Setting initial position to 00000000-0000-0000-0000-000000000000:-1 [Note] WSREP: protonet asio version 0 [Note] WSREP: Using CRC-32C (optimized) for message checksums. [Note] WSREP: backend: asio [Note] WSREP: GMCast version 0 [Note] WSREP: (d35fbdc5-47fa-11e4-88ce-ebeacde19e39, 'tcp:// 0.0.0.0:4567') listening at tcp://0.0.0.0:4567 140929 13:05:34 [Note] WSREP: (d35fbdc5-47fa-11e4-88ce-ebeacde19e39, 'tcp://0.0.0.0:4567') multicast: , ttl: 1 [Note] WSREP: EVS version 0 [Note] WSREP: PC version 0 [Note] WSREP: gcomm: connecting to group 'baruwa_cluster', peer 'baruwadb1:,baruwadb2:,baruwadb3:' [Warning] WSREP: (d35fbdc5-47fa-11e4-88ce-ebeacde19e39, 'tcp:// 0.0.0.0:4567') address 'tcp://172.17.1.2:4567' points to own listening address, blacklisting [Warning] WSREP: no nodes coming from prim view, prim not possible [Note] WSREP: view(view_id(NON_PRIM,d35fbdc5-47fa-11e4-88ce-ebeacde19e39,1) memb { d35fbdc5-47fa-11e4-88ce-ebeacde19e39,0 } joined { } left { } partitioned { }) [Note] WSREP: gcomm: connected 7 [Note] WSREP: Changing maximum packet size to 64500, resulting msg size: 32636 [Note] WSREP: Shifting CLOSED -> OPEN (TO: 0) [Note] WSREP: Opened channel 'baruwa_cluster' [Note] WSREP: Waiting for SST to complete. [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1 [Note] WSREP: Received NON-PRIMARY. [Note] WSREP: New cluster view: global state: :-1, view# -1: non-Primary, number of nodes: 1, my index: 0, protocol version -1 1 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. [Warning] WSREP: last inactive check more than PT1.5S ago (PT3.5018S), skipping check
my.cnf: #galera settings wsrep_provider=/usr/lib/galera/libgalera_smm.so
wsrep_cluster_address="gcomm://baruwadb1,baruwadb2,baruwadb3?pc.wait_prim=no" wsrep_sst_method=rsync wsrep_cluster_name="baruwa_cluster"
Perhaps you need to bootstrap the cluster.
Best, Nirbhay
Any help will be appreciated. Thanks and regards
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp