[Maria-discuss] apparent master shutdown
HI Folks, For the last several years, actually ever since MySQL went to pieces (I mean went to Oracle) I have been running five MariaDB databases in five Australian cities all replicating via a daisy chain. This system has never let me down. I have had times when hardware or network failures have stopped or broken the replication but recovery has always been simple easy and quick. Over the years our IT Infrastructure team have replaced and upgraded several of the servers and I've simply installed MariaDB on the new servers and set the ini files and copied the data and that's all I've had to do. Today the Infrastructure team replaced one of the servers. The old server was running WIN SERVER 2008 and the new server is running WIN SERVER 2012. I've just installed Maria DB on the new server and reset the ini file, loaded the data and restarted the replication. Everything went perfect as expected. (I've learnt to expect nothing less from MariaDB) The databases are replicating perfectly and all user interfaces are connecting and running perfectly. HOWEVER. (AND HERE IS THE BAD NEWS):- Every six seconds I'm getting the following two log entries on the new server. 141206 21:56:30 [Note] Slave: received end packet from server, apparent master shutdown: 141206 21:56:30 [Note] Slave I/O thread: Failed reading log event, reconnecting to retry, log 'repbin.000071' at position 12487 Six seconds later ... 141206 21:56:36 [Note] Slave: received end packet from server, apparent master shutdown: 141206 21:56:36 [Note] Slave I/O thread: Failed reading log event, reconnecting to retry, log 'repbin.000071' at position 12487 And so on ... There are no log entries on this slave's master indicating any outage or stoppage. Note:- the rep_log_pos changes whenever there has been a replication activity as you would expect. Now I know I'm running an old version of MariaDB (version 5.3.4) but I've never had a problem since I first installed and I suppose I'm a victim of the "if it ain't broken don't fix it" philosophy but I haven't, until now been able to justify the expense and time of upgrading what is working perfectly. OK, for that, shoot me down in flames if you like but has anyone got any ideas what may be causing this? The database size is 8.6 GB with 156 tables (all are Innodb), 32 views, 98 functions, 292 procedures and 3 triggers. It drives around 100 users across the five replicated sites. Kind Regards, Bruce Carlson.
Check that the server-id's are unique -- <http://dk.linkedin.com/in/jesperstaunhansen> On Sat, Dec 6, 2014 at 12:40 PM, Bruce Carlson <Bruce.Carlson@nepean.com> wrote:
HI Folks,
For the last several years, actually ever since MySQL went to pieces (I mean went to Oracle) I have been running five MariaDB databases in five Australian cities all replicating via a daisy chain.
This system has never let me down.
I have had times when hardware or network failures have stopped or broken the replication but recovery has always been simple easy and quick.
Over the years our IT Infrastructure team have replaced and upgraded several of the servers and I’ve simply installed MariaDB on the new servers and set the ini files and copied the data and that’s all I’ve had to do.
Today the Infrastructure team replaced one of the servers. The old server was running WIN SERVER 2008 and the new server is running WIN SERVER 2012.
I’ve just installed Maria DB on the new server and reset the ini file, loaded the data and restarted the replication. Everything went perfect as expected. (I’ve learnt to expect nothing less from MariaDB)
The databases are replicating perfectly and all user interfaces are connecting and running perfectly.
HOWEVER. (AND HERE IS THE BAD NEWS):-
Every six seconds I’m getting the following two log entries on the new server.
141206 21:56:30 [Note] Slave: received end packet from server, apparent master shutdown:
141206 21:56:30 [Note] Slave I/O thread: Failed reading log event, reconnecting to retry, log 'repbin.000071' at position 12487
Six seconds later …
141206 21:56:36 [Note] Slave: received end packet from server, apparent master shutdown:
141206 21:56:36 [Note] Slave I/O thread: Failed reading log event, reconnecting to retry, log 'repbin.000071' at position 12487
And so on …
There are no log entries on this slave’s master indicating any outage or stoppage.
Note:- the rep_log_pos changes whenever there has been a replication activity as you would expect.
Now I know I’m running an old version of MariaDB (version 5.3.4) but I’ve never had a problem since I first installed and I suppose I’m a victim of the “if it ain’t broken don’t fix it” philosophy but I haven’t, until now been able to justify the expense and time of upgrading what is working perfectly.
OK, for that, shoot me down in flames if you like but has anyone got any ideas what may be causing this?
The database size is 8.6 GB with 156 tables (all are Innodb), 32 views, 98 functions, 292 procedures and 3 triggers.
It drives around 100 users across the five replicated sites.
Kind Regards,
Bruce Carlson.
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
Thanks for the tip Jesper, however I just checked all five servers and they all have unique server ID’s. Any other suggestions? Cheers, Bruce Carlson From: Jesper Staun Hansen [mailto:jesper@staunhansen.dk] Sent: Sunday, 7 December 2014 1:21 AM To: Bruce Carlson Cc: Maria Discuss Subject: Re: [Maria-discuss] apparent master shutdown Check that the server-id's are unique -- [http://www.linkedin.com/img/webpromo/btn_viewmy_160x33.png]<http://dk.linkedin.com/in/jesperstaunhansen> On Sat, Dec 6, 2014 at 12:40 PM, Bruce Carlson <Bruce.Carlson@nepean.com<mailto:Bruce.Carlson@nepean.com>> wrote: HI Folks, For the last several years, actually ever since MySQL went to pieces (I mean went to Oracle) I have been running five MariaDB databases in five Australian cities all replicating via a daisy chain. This system has never let me down. I have had times when hardware or network failures have stopped or broken the replication but recovery has always been simple easy and quick. Over the years our IT Infrastructure team have replaced and upgraded several of the servers and I’ve simply installed MariaDB on the new servers and set the ini files and copied the data and that’s all I’ve had to do. Today the Infrastructure team replaced one of the servers. The old server was running WIN SERVER 2008 and the new server is running WIN SERVER 2012. I’ve just installed Maria DB on the new server and reset the ini file, loaded the data and restarted the replication. Everything went perfect as expected. (I’ve learnt to expect nothing less from MariaDB) The databases are replicating perfectly and all user interfaces are connecting and running perfectly. HOWEVER. (AND HERE IS THE BAD NEWS):- Every six seconds I’m getting the following two log entries on the new server. 141206 21:56:30 [Note] Slave: received end packet from server, apparent master shutdown: 141206 21:56:30 [Note] Slave I/O thread: Failed reading log event, reconnecting to retry, log 'repbin.000071' at position 12487 Six seconds later … 141206 21:56:36 [Note] Slave: received end packet from server, apparent master shutdown: 141206 21:56:36 [Note] Slave I/O thread: Failed reading log event, reconnecting to retry, log 'repbin.000071' at position 12487 And so on … There are no log entries on this slave’s master indicating any outage or stoppage. Note:- the rep_log_pos changes whenever there has been a replication activity as you would expect. Now I know I’m running an old version of MariaDB (version 5.3.4) but I’ve never had a problem since I first installed and I suppose I’m a victim of the “if it ain’t broken don’t fix it” philosophy but I haven’t, until now been able to justify the expense and time of upgrading what is working perfectly. OK, for that, shoot me down in flames if you like but has anyone got any ideas what may be causing this? The database size is 8.6 GB with 156 tables (all are Innodb), 32 views, 98 functions, 292 procedures and 3 triggers. It drives around 100 users across the five replicated sites. Kind Regards, Bruce Carlson. _______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net<mailto:maria-discuss@lists.launchpad.net> Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
Try to set setting slave_compressed_protocol=1 And restarting your replication thread Dikirim dari Yahoo Mail pada Android
HI, Thanks for all the help everyone. I found the problem. IT Support guys restarted the old server with a new IP address after migrating everything to the new hardware and of cause it restarted the MariaDB (mysql) service with the same server ID as the new hardware. Stopped and disabled the old server and all is good. Once again thanks to everyone. All your advice at least pointed me in the right direction. It was a duplicated server ID but not on any of the “in service” machines Kind Regards, Bruce Carlson MariaDB …. Still the best DB on the planet. (& so are the people) From: Irwanto [mailto:irwan_forum@yahoo.co.id] Sent: Sunday, 7 December 2014 3:58 AM To: Bruce Carlson; Jesper Staun Hansen Cc: Maria Discuss Subject: Bls: Re: [Maria-discuss] apparent master shutdown Try to set setting slave_compressed_protocol=1 And restarting your replication thread Dikirim dari Yahoo Mail pada Android<https://overview.mail.yahoo.com/mobile/?.src=Android> Dari:"Bruce Carlson" <Bruce.Carlson@nepean.com<mailto:Bruce.Carlson@nepean.com>> Tanggal:Sab, 6 Des 2014 pada 23:18 Judul:Re: [Maria-discuss] apparent master shutdown Thanks for the tip Jesper, however I just checked all five servers and they all have unique server ID’s. Any other suggestions? Cheers, Bruce Carlson From: Jesper Staun Hansen [mailto:jesper@staunhansen.dk] Sent: Sunday, 7 December 2014 1:21 AM To: Bruce Carlson Cc: Maria Discuss Subject: Re: [Maria-discuss] apparent master shutdown Check that the server-id's are unique -- [http://www.linkedin.com/img/webpromo/btn_viewmy_160x33.png]<http://dk.linkedin.com/in/jesperstaunhansen> On Sat, Dec 6, 2014 at 12:40 PM, Bruce Carlson <Bruce.Carlson@nepean.com<javascript:return>> wrote: HI Folks, For the last several years, actually ever since MySQL went to pieces (I mean went to Oracle) I have been running five MariaDB databases in five Australian cities all replicating via a daisy chain. This system has never let me down. I have had times when hardware or network failures have stopped or broken the replication but recovery has always been simple easy and quick. Over the years our IT Infrastructure team have replaced and upgraded several of the servers and I’ve simply installed MariaDB on the new servers and set the ini files and copied the data and that’s all I’ve had to do. Today the Infrastructure team replaced one of the servers. The old server was running WIN SERVER 2008 and the new server is running WIN SERVER 2012. I’ve just installed Maria DB on the new server and reset the ini file, loaded the data and restarted the replication. Everything went perfect as expected. (I’ve learnt to expect nothing less from MariaDB) The databases are replicating perfectly and all user interfaces are connecting and running perfectly. HOWEVER. (AND HERE IS THE BAD NEWS):- Every six seconds I’m getting the following two log entries on the new server. 141206 21:56:30 [Note] Slave: received end packet from server, apparent master shutdown: 141206 21:56:30 [Note] Slave I/O thread: Failed reading log event, reconnecting to retry, log 'repbin.000071' at position 12487 Six seconds later … 141206 21:56:36 [Note] Slave: received end packet from server, apparent master shutdown: 141206 21:56:36 [Note] Slave I/O thread: Failed reading log event, reconnecting to retry, log 'repbin.000071' at position 12487 And so on … There are no log entries on this slave’s master indicating any outage or stoppage. Note:- the rep_log_pos changes whenever there has been a replication activity as you would expect. Now I know I’m running an old version of MariaDB (version 5.3.4) but I’ve never had a problem since I first installed and I suppose I’m a victim of the “if it ain’t broken don’t fix it” philosophy but I haven’t, until now been able to justify the expense and time of upgrading what is working perfectly. OK, for that, shoot me down in flames if you like but has anyone got any ideas what may be causing this? The database size is 8.6 GB with 156 tables (all are Innodb), 32 views, 98 functions, 292 procedures and 3 triggers. It drives around 100 users across the five replicated sites. Kind Regards, Bruce Carlson. _______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net<javascript:return> Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
participants (3)
-
Bruce Carlson -
Irwanto -
Jesper Staun Hansen