I saw that, but please note that this only happens 45 minutes after I start the mariadb service while attempting an SST (because the IST failed?). All that time I see one mysqld thread spinning, but nothing in the logs. To be sure I increased RAM on the donor to 32G so there shouldn't be any issues allocating memory.



On Fri, Jun 28, 2019 at 2:07 AM Daniel Black <daniel@linux.ibm.com> wrote:
On Thu, 20 Jun 2019 17:59:26 +0200
Tiemen Ruiten <t.ruiten@tech-lab.io> wrote:
> On the donor, I see:
>
> ...
> 2019-06-20 16:24:48 0 [ERROR] WSREP: Failed to read from:
> wsrep_sst_rsync --role 'donor' --address
> '10.100.130.24:4444/rsync_sst' --socket '/var/lib/mysql/mysql.sock'
> --datadir '/var/lib/mysql/'     '' --gtid
> '6a1fd0aa-4739-11e5-af88-42756e861245:7403368279' --gtid-domain-id
> '0' --bypass 2019-06-20 16:24:48 0 [ERROR] WSREP: Process completed
> with error: wsrep_sst_rsync --role 'donor' --address
> '10.100.130.24:4444/rsync_sst' --socket '/var/lib/mysql/mysql.sock'
> --datadir '/var/lib/mysql/'     '' --gtid
> '6a1fd0aa-4739-11e5-af88-42756e861245:7403368279' --gtid-domain-id
> '0' --bypass: 12 (Cannot allocate memory) 2019-06-20 16:24:48 0
> [ERROR] WSREP: Command did not run: wsrep_sst_rsync --role 'donor'
> --address '10.100.130.24:4444/rsync_sst' --socket
> '/var/lib/mysql/mysql.sock' --datadir '/var/lib/mysql/'     '' --gtid
> '6a1fd0aa-4739-11e5-af88-42756e861245:7403368279' --gtid-domain-id
> '0' --bypass 2019-06-20 16:24:48 0 [Warning] WSREP: Could not find
> peer: ddb4f4da-935f-11e9-bcff-8209857266f0
> 2019-06-20 16:24:48 0 [Warning] WSREP: 1.0 (mariadb2): State transfer
> to -1.-1 (left the group) failed: -12 (Cannot allocate memory)
>
> What is going on?

The last lines are the biggest hint - Cannot allocate memory on the
donor.