[Maria-discuss] rsync failure?
Have the rsync failure (see the trace below): 2016-11-08 15:17:34 139959897945856 [ERROR] WSREP: Failed to read 'ready <addr>' from: wsrep_sst_rsync --role 'joiner' --address 'node3.frame.mesos' --datadir '/var/lib/mysql/' --parent '114' '' Read: '(null)' 2016-11-08 15:17:34 139959897945856 [ERROR] WSREP: Process completed with error: wsrep_sst_rsync --role 'joiner' --address 'node3.frame.mesos' --datadir '/var/lib/mysql/' --parent '114' '' : 1 (Operation not permitted) 2016-11-08 15:17:34 139960239241984 [ERROR] WSREP: Failed to prepare for 'rsync' SST. Unrecoverable. 2016-11-08 15:17:34 139960239241984 [ERROR] Aborting Can someone with better understanding of replication process explain the nature of this problem? Can it be caused by hostname resolution failure, or would it be storage related? Thank you,
On Tue, Nov 8, 2016 at 10:50 AM, l vic <lvic4594@gmail.com> wrote:
Have the rsync failure (see the trace below):
2016-11-08 15:17:34 139959897945856 [ERROR] WSREP: Failed to read 'ready <addr>' from: wsrep_sst_rsync --role 'joiner' --address 'node3.frame.mesos' --datadir '/var/lib/mysql/' --parent '114' '' Read: '(null)' 2016-11-08 15:17:34 139959897945856 [ERROR] WSREP: Process completed with error: wsrep_sst_rsync --role 'joiner' --address 'node3.frame.mesos' --datadir '/var/lib/mysql/' --parent '114' '' : 1 (Operation not permitted) 2016-11-08 15:17:34 139960239241984 [ERROR] WSREP: Failed to prepare for 'rsync' SST. Unrecoverable. 2016-11-08 15:17:34 139960239241984 [ERROR] Aborting
Can someone with better understanding of replication process explain the nature of this problem? Can it be caused by hostname resolution failure, or would it be storage related?
Therer is a possibility that something could have gone wrong on the donor node. Checkout its error log to see whether that's the case. Best, Nirbhay
Thank you,
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
How can i determine the donor node? Would that be seed node? Thank you On Tue, Nov 8, 2016 at 11:04 AM, Nirbhay Choubey <nirbhay@mariadb.com> wrote:
On Tue, Nov 8, 2016 at 10:50 AM, l vic <lvic4594@gmail.com> wrote:
Have the rsync failure (see the trace below):
2016-11-08 15:17:34 139959897945856 [ERROR] WSREP: Failed to read 'ready <addr>' from: wsrep_sst_rsync --role 'joiner' --address 'node3.frame.mesos' --datadir '/var/lib/mysql/' --parent '114' '' Read: '(null)' 2016-11-08 15:17:34 139959897945856 [ERROR] WSREP: Process completed with error: wsrep_sst_rsync --role 'joiner' --address 'node3.frame.mesos' --datadir '/var/lib/mysql/' --parent '114' '' : 1 (Operation not permitted) 2016-11-08 15:17:34 139960239241984 [ERROR] WSREP: Failed to prepare for 'rsync' SST. Unrecoverable. 2016-11-08 15:17:34 139960239241984 [ERROR] Aborting
Can someone with better understanding of replication process explain the nature of this problem? Can it be caused by hostname resolution failure, or would it be storage related?
Therer is a possibility that something could have gone wrong on the donor node. Checkout its error log to see whether that's the case.
Best, Nirbhay
Thank you,
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
On Tue, Nov 8, 2016 at 11:32 AM, l vic <lvic4594@gmail.com> wrote:
How can i determine the donor node? Would that be seed node?
On the joiner you will see the following: 2016-11-08 12:13:57 140589576218368 [Note] WSREP: Member 1.0 (node2) requested state transfer from '*any*'. Selected 0.0 (node1)(SYNCED) as donor. In my installation, I have used wsrep_node_name to assign names to the nodes. Best, Nirbhay
Thank you
On Tue, Nov 8, 2016 at 11:04 AM, Nirbhay Choubey <nirbhay@mariadb.com> wrote:
On Tue, Nov 8, 2016 at 10:50 AM, l vic <lvic4594@gmail.com> wrote:
Have the rsync failure (see the trace below):
2016-11-08 15:17:34 139959897945856 [ERROR] WSREP: Failed to read 'ready <addr>' from: wsrep_sst_rsync --role 'joiner' --address 'node3.frame.mesos' --datadir '/var/lib/mysql/' --parent '114' '' Read: '(null)' 2016-11-08 15:17:34 139959897945856 [ERROR] WSREP: Process completed with error: wsrep_sst_rsync --role 'joiner' --address 'node3.frame.mesos' --datadir '/var/lib/mysql/' --parent '114' '' : 1 (Operation not permitted) 2016-11-08 15:17:34 139960239241984 [ERROR] WSREP: Failed to prepare for 'rsync' SST. Unrecoverable. 2016-11-08 15:17:34 139960239241984 [ERROR] Aborting
Can someone with better understanding of replication process explain the nature of this problem? Can it be caused by hostname resolution failure, or would it be storage related?
Therer is a possibility that something could have gone wrong on the donor node. Checkout its error log to see whether that's the case.
Best, Nirbhay
Thank you,
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
From your previous answer about using node-name vs. node-address to identify the node:
use wsrep-node-address; wsrep-node-name, OTOH, could be used to assign a name to the node. It helps identify the node in the error logs and can be used in wsrep-sst-donor On Tue, Nov 8, 2016 at 12:54 PM, Nirbhay Choubey <nirbhay@mariadb.com> wrote:
On Tue, Nov 8, 2016 at 11:32 AM, l vic <lvic4594@gmail.com> wrote:
How can i determine the donor node? Would that be seed node?
On the joiner you will see the following:
2016-11-08 12:13:57 140589576218368 [Note] WSREP: Member 1.0 (node2) requested state transfer from '*any*'. Selected 0.0 (node1)(SYNCED) as donor.
In my installation, I have used wsrep_node_name to assign names to the nodes.
Best, Nirbhay
Thank you
On Tue, Nov 8, 2016 at 11:04 AM, Nirbhay Choubey <nirbhay@mariadb.com> wrote:
On Tue, Nov 8, 2016 at 10:50 AM, l vic <lvic4594@gmail.com> wrote:
Have the rsync failure (see the trace below):
2016-11-08 15:17:34 139959897945856 [ERROR] WSREP: Failed to read 'ready <addr>' from: wsrep_sst_rsync --role 'joiner' --address 'node3.frame.mesos' --datadir '/var/lib/mysql/' --parent '114' '' Read: '(null)' 2016-11-08 15:17:34 139959897945856 [ERROR] WSREP: Process completed with error: wsrep_sst_rsync --role 'joiner' --address 'node3.frame.mesos' --datadir '/var/lib/mysql/' --parent '114' '' : 1 (Operation not permitted) 2016-11-08 15:17:34 139960239241984 [ERROR] WSREP: Failed to prepare for 'rsync' SST. Unrecoverable. 2016-11-08 15:17:34 139960239241984 [ERROR] Aborting
Can someone with better understanding of replication process explain the nature of this problem? Can it be caused by hostname resolution failure, or would it be storage related?
Therer is a possibility that something could have gone wrong on the donor node. Checkout its error log to see whether that's the case.
Best, Nirbhay
Thank you,
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
on the joiner i see: WSREP: Failed to prepare for incremental state transfer: Local state UUID (00000000-0000-0000-0000-000000000000) does not match group state UUID (f74ffa7e-a5f2-11e6-b2a3-1fead9a6118d): 1 (Operation not permitted) at galera/src/replicator_str.cpp:prepare_for_IST():482. IST will be unavailable. 2016-11-08 20:38:23 139911323707136 [Note] WSREP: Member 0.0 () requested state transfer from '*any*'. Selected 1.0 ()(SYNCED) as donor. 2016-11-08 20:38:23 139911323707136 [Note] WSREP: Shifting PRIMARY -> JOINER (TO: 4) 2016-11-08 20:38:23 139911628454656 [Note] WSREP: Requesting state transfer: success, donor: 1 2016-11-08 20:38:24 139911323707136 [Warning] WSREP: 1.0 (): State transfer to 0.0 () failed: -255 (Unknown error 255) On Tue, Nov 8, 2016 at 12:54 PM, Nirbhay Choubey <nirbhay@mariadb.com> wrote:
On Tue, Nov 8, 2016 at 11:32 AM, l vic <lvic4594@gmail.com> wrote:
How can i determine the donor node? Would that be seed node?
On the joiner you will see the following:
2016-11-08 12:13:57 140589576218368 [Note] WSREP: Member 1.0 (node2) requested state transfer from '*any*'. Selected 0.0 (node1)(SYNCED) as donor.
In my installation, I have used wsrep_node_name to assign names to the nodes.
Best, Nirbhay
Thank you
On Tue, Nov 8, 2016 at 11:04 AM, Nirbhay Choubey <nirbhay@mariadb.com> wrote:
On Tue, Nov 8, 2016 at 10:50 AM, l vic <lvic4594@gmail.com> wrote:
Have the rsync failure (see the trace below):
2016-11-08 15:17:34 139959897945856 [ERROR] WSREP: Failed to read 'ready <addr>' from: wsrep_sst_rsync --role 'joiner' --address 'node3.frame.mesos' --datadir '/var/lib/mysql/' --parent '114' '' Read: '(null)' 2016-11-08 15:17:34 139959897945856 [ERROR] WSREP: Process completed with error: wsrep_sst_rsync --role 'joiner' --address 'node3.frame.mesos' --datadir '/var/lib/mysql/' --parent '114' '' : 1 (Operation not permitted) 2016-11-08 15:17:34 139960239241984 [ERROR] WSREP: Failed to prepare for 'rsync' SST. Unrecoverable. 2016-11-08 15:17:34 139960239241984 [ERROR] Aborting
Can someone with better understanding of replication process explain the nature of this problem? Can it be caused by hostname resolution failure, or would it be storage related?
Therer is a possibility that something could have gone wrong on the donor node. Checkout its error log to see whether that's the case.
Best, Nirbhay
Thank you,
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
I don't see any log entries pertaining to node3 join on the donor nodes On Tue, Nov 8, 2016 at 11:04 AM, Nirbhay Choubey <nirbhay@mariadb.com> wrote:
On Tue, Nov 8, 2016 at 10:50 AM, l vic <lvic4594@gmail.com> wrote:
Have the rsync failure (see the trace below):
2016-11-08 15:17:34 139959897945856 [ERROR] WSREP: Failed to read 'ready <addr>' from: wsrep_sst_rsync --role 'joiner' --address 'node3.frame.mesos' --datadir '/var/lib/mysql/' --parent '114' '' Read: '(null)' 2016-11-08 15:17:34 139959897945856 [ERROR] WSREP: Process completed with error: wsrep_sst_rsync --role 'joiner' --address 'node3.frame.mesos' --datadir '/var/lib/mysql/' --parent '114' '' : 1 (Operation not permitted) 2016-11-08 15:17:34 139960239241984 [ERROR] WSREP: Failed to prepare for 'rsync' SST. Unrecoverable. 2016-11-08 15:17:34 139960239241984 [ERROR] Aborting
Can someone with better understanding of replication process explain the nature of this problem? Can it be caused by hostname resolution failure, or would it be storage related?
Therer is a possibility that something could have gone wrong on the donor node. Checkout its error log to see whether that's the case.
Best, Nirbhay
Thank you,
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
participants (2)
-
l vic -
Nirbhay Choubey