[Maria-discuss] Lost connection to MySQL server at, 'reading authorization packet', system error: 104 (or 0)
Hi all, I'm very inexperienced mariadb/mysql/sql user/admin, but not totally clueless, been maintaining a tiny mysql DB for postfixadmin for years, then converted it to mariadb. We recently started using SOGo (groupware). I hired the SOGo guys to help with the install/config, and have been using it trouble-free for a few months now (love it for the most part), until now. Everything seems to be working fine, but since the upgrade to SOGo 2.2.4 on 6/12, which I am told increases the number of connections it uses to the DB, I have been getting very occasional errors like this (the system error is always either 104 or 0): 2014-06-12 03:30:15.530 sogo-tool[31674] ERROR: could not open MySQL4 connection to database 'sogo': Lost connection to MySQL server at 'reading authorization packet', system error: 104 <0x0x236c140[GCSChannelManager]> could not open channel <MySQL4Channel[0x0x22241b0] connection=0x(null)> for URL: mysql://user:password@myhost.com:3306/sogo/sogo_sessions_folder <0x0x236c140[GCSChannelManager]> will prevent opening of this channel 5 seconds after 2014-06-12 03:30:01 -0400 2014-06-12 03:30:15.531 sogo-tool[31674] Can't aquire channel But it only happened once per day after the upgrade, then once it happened twice (about 12 hours apart), and it happened at times when the load would be the lowest (ie, 3:30am for the one above). Googling reveals a lot of reports about this error over the years, some seemingly to be bugs, others DB tuning issues. Should I be looking at tuning: max_connections max_user_connections ? If not, does anyone know where I should start looking? Is there some kind of extra logging I could enable that wouldn't cause serious performance problems? Or... maybe this is simply not something I should be worried about? Thanks for any help/suggestions...
Am 18.06.2014 13:30, schrieb Tanstaafl:
I'm very inexperienced mariadb/mysql/sql user/admin, but not totally clueless, been maintaining a tiny mysql DB for postfixadmin for years, then converted it to mariadb.
We recently started using SOGo (groupware). I hired the SOGo guys to help with the install/config, and have been using it trouble-free for a few months now (love it for the most part), until now.
Everything seems to be working fine, but since the upgrade to SOGo 2.2.4 on 6/12, which I am told increases the number of connections it uses to the DB, I have been getting very occasional errors like this (the system error is always either 104 or 0):
2014-06-12 03:30:15.530 sogo-tool[31674] ERROR: could not open MySQL4 connection to database 'sogo'
the client seems to use the very outdated mysql4 protocol which is not compatible - no idea why it does that only sometimes - maybe a completly wrong error message from the software!
which I am told increases the number of connections
Should I be looking at tuning:
max_connections max_user_connections
surely, the have to tell you *how many connections* and not only "increases", there must be a upper limit controlled by the software and multiply it by the users tells you how many connections you need to allow
On 6/18/2014 7:53 AM, Reindl Harald <h.reindl@thelounge.net> wrote:
the client seems to use the very outdated mysql4 protocol which is not compatible
When you say 'not compatible' - do you mean that mariadb simply will (would) not work with that protocol if the software attempts to use it?
- no idea why it does that only sometimes - maybe a completly wrong error message from the software!
So, either for some reason the software really does attempt to use the mysql4 protocl during this transaction, and mariadb doesn't support it so the connection fails, or, the software itself is reporting the error incorrectly? I just want to understand your response before I go back the the SOGo list with this. Oh - I don't have debug logging enabled on mariadb... what would be the best way to get more information about this event on my mariadb server without flooding the logs with irrelevant stuff and/or impacting performance? Thanks!
Am 19.06.2014 13:17, schrieb Tanstaafl:
On 6/18/2014 7:53 AM, Reindl Harald <h.reindl@thelounge.net> wrote:
the client seems to use the very outdated mysql4 protocol which is not compatible
When you say 'not compatible' - do you mean that mariadb simply will (would) not work with that protocol if the software attempts to use it?
that's not a matter of MariaDB it's a matter that MySQL4 passed away nearly 10 years please read about the changes between MySQL4 and MySQL5 if it comes to authentication, how passwords are stored and so on...... at least the the message reads like authentication problems
- no idea why it does that only sometimes - maybe a completly wrong error message from the software!
So, either for some reason the software really does attempt to use the mysql4 protocl during this transaction, and mariadb doesn't support it so the connection fails,
or,
the software itself is reporting the error incorrectly?
yes
I just want to understand your response before I go back the the SOGo list with this.
Oh - I don't have debug logging enabled on mariadb... what would be the best way to get more information about this event on my mariadb server without flooding the logs with irrelevant stuff and/or impacting performance?
if the client really refuses connection due handshake most likely nothing on the server side - ask the vednor, you said it worked before you upgraded the client software - so why do you search the solution on the server side first?
On 6/19/2014 7:34 AM, Reindl Harald <h.reindl@thelounge.net> wrote:
Am 19.06.2014 13:17, schrieb Tanstaafl:
On 6/18/2014 7:53 AM, Reindl Harald <h.reindl@thelounge.net> wrote:
the client seems to use the very outdated mysql4 protocol which is not compatible
When you say 'not compatible' - do you mean that mariadb simply will (would) not work with that protocol if the software attempts to use it?
that's not a matter of MariaDB it's a matter that MySQL4 passed away nearly 10 years
So, just to be clear... What should this be saying? Something like: "... ERROR: could not open MySQL5 connection to database..." ? In other words, should the mysql4 be mysql5? Still trying to figure out a way to adjust logging to better assist troubleshooting this. I agree this is most likely a client (SOGo) problem since it only appeared after the last update, but the devs say otherwise (or mostly just remained silent)... so I'd like some additional ammunition before going back to them and opening up a bug... Thanks
Reindl/anyone? On 6/22/2014 12:01 PM, Tanstaafl <tanstaafl@libertytrek.org> wrote:
On 6/19/2014 7:34 AM, Reindl Harald <h.reindl@thelounge.net> wrote:
Am 19.06.2014 13:17, schrieb Tanstaafl:
On 6/18/2014 7:53 AM, Reindl Harald <h.reindl@thelounge.net> wrote:
the client seems to use the very outdated mysql4 protocol which is not compatible
When you say 'not compatible' - do you mean that mariadb simply will (would) not work with that protocol if the software attempts to use it?
that's not a matter of MariaDB it's a matter that MySQL4 passed away nearly 10 years
So, just to be clear...
What should this be saying? Something like:
"... ERROR: could not open MySQL5 connection to database..." ?
In other words, should the mysql4 be mysql5?
Still trying to figure out a way to adjust logging to better assist troubleshooting this.
I agree this is most likely a client (SOGo) problem since it only appeared after the last update, but the devs say otherwise (or mostly just remained silent)... so I'd like some additional ammunition before going back to them and opening up a bug...
Am 30.06.2014 13:59, schrieb Tanstaafl:
Reindl/anyone?
comlain at the client developers what else nobody else has such problems you even state before update the client there was no problem if you update software B and things break why seek the problem in software A at all?
On 6/22/2014 12:01 PM, Tanstaafl <tanstaafl@libertytrek.org> wrote:
On 6/19/2014 7:34 AM, Reindl Harald <h.reindl@thelounge.net> wrote:
Am 19.06.2014 13:17, schrieb Tanstaafl:
On 6/18/2014 7:53 AM, Reindl Harald <h.reindl@thelounge.net> wrote:
the client seems to use the very outdated mysql4 protocol which is not compatible
When you say 'not compatible' - do you mean that mariadb simply will (would) not work with that protocol if the software attempts to use it?
that's not a matter of MariaDB it's a matter that MySQL4 passed away nearly 10 years
So, just to be clear...
What should this be saying? Something like:
"... ERROR: could not open MySQL5 connection to database..." ?
In other words, should the mysql4 be mysql5?
Still trying to figure out a way to adjust logging to better assist troubleshooting this.
I agree this is most likely a client (SOGo) problem since it only appeared after the last update, but the devs say otherwise (or mostly just remained silent)... so I'd like some additional ammunition before going back to them and opening up a bug...
Well, I guess I should expect no less from you Reindl... occasionally you're helpful, but more often than not you simply ignore very specific questions like the one in this email) and just have to flex your superiority complex. On 6/30/2014 8:01 AM, Reindl Harald <h.reindl@thelounge.net> wrote:
Am 30.06.2014 13:59, schrieb Tanstaafl:
Reindl/anyone?
comlain at the client developers
what else
nobody else has such problems you even state before update the client there was no problem if you update software B and things break why seek the problem in software A at all?
On 6/22/2014 12:01 PM, Tanstaafl <tanstaafl@libertytrek.org> wrote:
On 6/19/2014 7:34 AM, Reindl Harald <h.reindl@thelounge.net> wrote:
Am 19.06.2014 13:17, schrieb Tanstaafl:
On 6/18/2014 7:53 AM, Reindl Harald <h.reindl@thelounge.net> wrote:
the client seems to use the very outdated mysql4 protocol which is not compatible
When you say 'not compatible' - do you mean that mariadb simply will (would) not work with that protocol if the software attempts to use it?
that's not a matter of MariaDB it's a matter that MySQL4 passed away nearly 10 years
So, just to be clear...
What should this be saying? Something like:
"... ERROR: could not open MySQL5 connection to database..." ?
In other words, should the mysql4 be mysql5?
Still trying to figure out a way to adjust logging to better assist troubleshooting this.
I agree this is most likely a client (SOGo) problem since it only appeared after the last update, but the devs say otherwise (or mostly just remained silent)... so I'd like some additional ammunition before going back to them and opening up a bug...
that's not a matter of superiority solve problems at the root-cause if a update of 3rd party software introduces problems it is very clear where the root-cause is - roll back that update or if that is not possible for whatever reason escalate the problem at the 3rd party which introduced it what other help do you expect if nobody else has a similar problem and there is for days no repsonse it becomes even more clear that you seek at the wrong end of the chain Am 30.06.2014 21:41, schrieb Tanstaafl:
Well, I guess I should expect no less from you Reindl... occasionally you're helpful, but more often than not you simply ignore very specific questions like the one in this email) and just have to flex your superiority complex.
On 6/30/2014 8:01 AM, Reindl Harald <h.reindl@thelounge.net> wrote:
Am 30.06.2014 13:59, schrieb Tanstaafl:
Reindl/anyone?
comlain at the client developers
what else
nobody else has such problems you even state before update the client there was no problem if you update software B and things break why seek the problem in software A at all?
On 6/22/2014 12:01 PM, Tanstaafl <tanstaafl@libertytrek.org> wrote:
On 6/19/2014 7:34 AM, Reindl Harald <h.reindl@thelounge.net> wrote:
Am 19.06.2014 13:17, schrieb Tanstaafl:
On 6/18/2014 7:53 AM, Reindl Harald <h.reindl@thelounge.net> wrote: > the client seems to use the very outdated mysql4 protocol > which is not compatible
When you say 'not compatible' - do you mean that mariadb simply will (would) not work with that protocol if the software attempts to use it?
that's not a matter of MariaDB it's a matter that MySQL4 passed away nearly 10 years
So, just to be clear...
What should this be saying? Something like:
"... ERROR: could not open MySQL5 connection to database..." ?
In other words, should the mysql4 be mysql5?
Still trying to figure out a way to adjust logging to better assist troubleshooting this.
I agree this is most likely a client (SOGo) problem since it only appeared after the last update, but the devs say otherwise (or mostly just remained silent)... so I'd like some additional ammunition before going back to them and opening up a bug...
On 6/30/2014 3:48 PM, Reindl Harald <h.reindl@thelounge.net> wrote:
solve problems at the root-cause
Absolutely agree...
if a update of 3rd party software introduces problems it is very clear where the root-cause is
Not necessarily. Have you never heard of a case where updating one software reveals a bug in some other related software that was not apparent before? Happens all the time. Of course, it is very likely that you are correct, but...
- roll back that update or if that is not possible for whatever reason escalate the problem at the 3rd party which introduced it
what other help do you expect
An answer to my very simple and specific GENERAL question that will help me to better troubleshoot this to ascertain precisely where the problem is.
if nobody else has a similar problem and there is for days no repsonse it becomes even more clear that you seek at the wrong end of the chain
The questions I asked and re-asked for a response to were GENERAL questions, ie: 1. What should this (error message) be saying? Something like: "... ERROR: could not open MySQL5 connection to database..." ? In other words, should the mysql4 be mysql5 in the error message? No amount of googling has answered this, and no amount of googling has revealed if there is even a 'mysql4' vs 'mysql5' 'connection protocol', or if this error message is purely created by the reporting (3rd party) software. Then my other question about maybe getting a little more debug logging without going into full debug logging mode. I have little experience troubleshooting these kinds of problems, which is why I am asking (I thought that would be obvious).
Am 02.07.2014 14:13, schrieb Tanstaafl:
An answer to my very simple and specific GENERAL question that will help me to better troubleshoot this to ascertain precisely where the problem is.
if nobody else has a similar problem and there is for days no repsonse it becomes even more clear that you seek at the wrong end of the chain
The questions I asked and re-asked for a response to were GENERAL questions, ie:
1. What should this (error message) be saying? Something like:
"... ERROR: could not open MySQL5 connection to database..." ?
In other words, should the mysql4 be mysql5 in the error message?
you refuse to understand that this message comes from the *client* and not from MySQL/MariaDB - so guess why nobody here gave an answer by logical thinking it's clear that it can only come from the client because by the fact there was no connection established how could the server give any answer?
No amount of googling has answered this, and no amount of googling has revealed if there is even a 'mysql4' vs 'mysql5' 'connection protocol', or if this error message is purely created by the reporting (3rd party) software
if nobody including Google ever faced that messages guess from where it comes -> the 3rd party software - you are wasting your own time because you ask at the wrong mailing-list no idea how i could have expressed this more clear....
On 7/2/2014 8:30 AM, Reindl Harald <h.reindl@thelounge.net> wrote:
you refuse to understand that this message comes from the*client* and not from MySQL/MariaDB - so guess why nobody here gave an answer
Yes, the error I posted comes from the client, but apparently the server is seeing errors from the client too... I just realized that when I responded to this thread again on 6/30, I neglected to include some new additional/important info regarding this issue that happened a couple of weeks after this initial report (about a week ago) that leads me to believe that I should be able to troubleshoot this, at least to an extent, on the mysql server level... Per my initial email, these errors were happening randomly for a couple of weeks, and only 1 to 3 or 4 times per day... But a couple of weeks later, my mysql server actually blocked the SOGo server completely: 2014-06-29 19:49:01.986 sogo-tool[7566] ERROR: could not open MySQL4 connection to database 'sogo': Host 'sogo.example.com' is blocked because of many connection errors; unblock with 'mysqladmin flush-hosts' This clearly shows that the mysql server was seeing too many errors from the client. I had to log into the mysql server and issue the flush-hosts command to restore communication between the two. But, I cannot find any evidence of any connection errors in /var/log/mysql/mysqld.err or any other logs on my mysql server. So... where should I find these errors that were occurring that caused mariadb to block all connections to my SOGo host until I issued the flush-hosts command? The only things in /var/log/mysql/mysqld.err on both sides of the date/time that I updated SOGo are:
140201 9:59:40 InnoDB: The InnoDB memory heap is disabled 140201 9:59:40 InnoDB: Mutexes and rw_locks use GCC atomic builtins 140201 9:59:40 InnoDB: Compressed tables use zlib 1.2.7 140201 9:59:40 InnoDB: Using Linux native AIO 140201 9:59:40 InnoDB: Initializing buffer pool, size = 16.0M 140201 9:59:40 InnoDB: Completed initialization of buffer pool 140201 9:59:40 InnoDB: highest supported file format is Barracuda. 140201 9:59:40 InnoDB: Waiting for the background threads to start 140201 9:59:41 Percona XtraDB (http://www.percona.com) 5.5.30-MariaDB-30.1 started; log sequence number 3480883 140201 9:59:41 [Warning] /usr/sbin/mysqld: unknown option '--loose-federated' 140201 9:59:41 [Note] Event Scheduler: Loaded 0 events 140201 9:59:41 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.30-MariaDB-log' socket: '/var/run/mysqld/mysqld.sock' port: 3306 Source distribution 140614 14:24:40 [Note] /usr/sbin/mysqld: Normal shutdown
140614 14:24:40 [Note] Event Scheduler: Purging the queue. 0 events 140614 14:24:40 InnoDB: Starting shutdown... 140614 14:24:41 InnoDB: Shutdown completed; log sequence number 219455836 140614 14:24:41 [Note] /usr/sbin/mysqld: Shutdown complete
140614 14:24:42 [Warning] No argument was provided to --log-bin and neither --log-basename or --log-bin-index where used; This may cause repliction to break when this server acts as a master and has its h ostname changed! Please use '--log-basename=myhost' or '--log-bin=mysqld-bin' to avoid this problem. 140614 14:24:42 InnoDB: The InnoDB memory heap is disabled 140614 14:24:42 InnoDB: Mutexes and rw_locks use GCC atomic builtins 140614 14:24:42 InnoDB: Compressed tables use zlib 1.2.8 140614 14:24:42 InnoDB: Using Linux native AIO 140614 14:24:42 InnoDB: Initializing buffer pool, size = 16.0M 140614 14:24:42 InnoDB: Completed initialization of buffer pool 140614 14:24:42 InnoDB: highest supported file format is Barracuda. 140614 14:24:42 InnoDB: Waiting for the background threads to start 140614 14:24:43 Percona XtraDB (http://www.percona.com) 5.5.37-MariaDB-34.0 started; log sequence number 219455836 140614 14:24:43 [Warning] /usr/sbin/mysqld: unknown option '--loose-federated' 140614 14:24:43 [Note] Server socket created on IP: '0.0.0.0'. 140614 14:24:43 [Note] Event Scheduler: Loaded 0 events 140614 14:24:43 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.37-MariaDB-log' socket: '/var/run/mysqld/mysqld.sock' port: 3306 Source distribution 140615 10:36:21 [Note] /usr/sbin/mysqld: Normal shutdown
140615 10:36:21 [Note] Event Scheduler: Purging the queue. 0 events 140615 10:36:21 InnoDB: Starting shutdown... 140615 10:36:22 InnoDB: Shutdown completed; log sequence number 219486293 140615 10:36:22 [Note] /usr/sbin/mysqld: Shutdown complete
140615 10:36:22 [Warning] No argument was provided to --log-bin and neither --log-basename or --log-bin-index where used; This may cause repliction to break when this server acts as a master and has its h ostname changed! Please use '--log-basename=myhost' or '--log-bin=mysqld-bin' to avoid this problem. 140615 10:36:22 InnoDB: The InnoDB memory heap is disabled 140615 10:36:22 InnoDB: Mutexes and rw_locks use GCC atomic builtins 140615 10:36:22 InnoDB: Compressed tables use zlib 1.2.8 140615 10:36:22 InnoDB: Using Linux native AIO 140615 10:36:22 InnoDB: Initializing buffer pool, size = 16.0M 140615 10:36:22 InnoDB: Completed initialization of buffer pool 140615 10:36:22 InnoDB: highest supported file format is Barracuda. 140615 10:36:22 InnoDB: Waiting for the background threads to start 140615 10:36:23 Percona XtraDB (http://www.percona.com) 5.5.37-MariaDB-34.0 started; log sequence number 219486293 140615 10:36:23 [Warning] /usr/sbin/mysqld: unknown option '--loose-federated' 140615 10:36:23 [Note] Server socket created on IP: '0.0.0.0'. 140615 10:36:23 [Note] Event Scheduler: Loaded 0 events 140615 10:36:23 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.37-MariaDB-log' socket: '/var/run/mysqld/mysqld.sock' port: 3306 Source distribution 140701 5:56:47 [Note] /usr/sbin/mysqld: Normal shutdown
140701 5:56:47 [Note] Event Scheduler: Purging the queue. 0 events 140701 5:56:47 InnoDB: Starting shutdown... 140701 5:56:48 InnoDB: Shutdown completed; log sequence number 240645473 140701 5:56:48 [Note] /usr/sbin/mysqld: Shutdown complete
140701 5:56:48 [Warning] No argument was provided to --log-bin and neither --log-basename or --log-bin-index where used; This may cause repliction to break when this server acts as a master and has its hostname changed! Please use '--log-basename=myhost' or '--log-bin=mysqld-bin' to avoid this problem. 140701 5:56:48 InnoDB: The InnoDB memory heap is disabled 140701 5:56:48 InnoDB: Mutexes and rw_locks use GCC atomic builtins 140701 5:56:48 InnoDB: Compressed tables use zlib 1.2.8 140701 5:56:48 InnoDB: Using Linux native AIO 140701 5:56:48 InnoDB: Initializing buffer pool, size = 16.0M 140701 5:56:48 InnoDB: Completed initialization of buffer pool 140701 5:56:48 InnoDB: highest supported file format is Barracuda. 140701 5:56:48 InnoDB: Waiting for the background threads to start 140701 5:56:49 Percona XtraDB (http://www.percona.com) 5.5.37-MariaDB-34.0 started; log sequence number 240645473 140701 5:56:49 [Warning] /usr/sbin/mysqld: unknown option '--loose-federated' 140701 5:56:49 [Note] Server socket created on IP: '0.0.0.0'. 140701 5:56:49 [Note] Event Scheduler: Loaded 0 events 140701 5:56:49 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.37-MariaDB-log' socket: '/var/run/mysqld/mysqld.sock' port: 3306 Source distribution
Anyone have any ideas on how to proceed troubleshooting this? It is most likely just a matter of time until the block happens again, and I'd prefer to try to find the problem and fix it, and I'll have better luck fixing this on the client side if I can give more details of errors the server is seeing from them. Thanks
you refuse to understand that this message comes from
I know nothing about SOGo (this is a MariaDB list), but I searched the proper mailing list for you, and it seems that someone already had your problem: http://www.mail-archive.com/users%40sogo.nu/msg18460.html Regards, Federico -------------------------------------------- Mer 9/7/14, Tanstaafl <tanstaafl@libertytrek.org> ha scritto: Oggetto: Re: [Maria-discuss] Lost connection to MySQL server at, 'reading authorization packet', system error: 104 (or 0) A: maria-discuss@lists.launchpad.net Data: Mercoledì 9 luglio 2014, 17:45 On 7/2/2014 8:30 AM, Reindl Harald <h.reindl@thelounge.net> wrote: the*client*
and not from MySQL/MariaDB - so guess why nobody here gave an answer
140201 9:59:40 InnoDB: The InnoDB memory heap is disabled 140201 9:59:40 InnoDB: Mutexes and rw_locks use GCC atomic builtins 140201 9:59:40 InnoDB: Compressed tables use zlib 1.2.7 140201 9:59:40 InnoDB: Using Linux native AIO 140201 9:59:40 InnoDB: Initializing buffer pool, size = 16.0M 140201 9:59:40 InnoDB: Completed initialization of buffer pool 140201 9:59:40 InnoDB: highest supported file
140201 9:59:40 InnoDB: Waiting for the background threads to start 140201 9:59:41 Percona XtraDB (http://www.percona.com) 5.5.30-MariaDB-30.1 started; log sequence number 3480883 140201 9:59:41 [Warning] /usr/sbin/mysqld: unknown option '--loose-federated' 140201 9:59:41 [Note] Event Scheduler: Loaded 0 events 140201 9:59:41 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.30-MariaDB-log' socket: '/var/run/mysqld/mysqld.sock' port: 3306 Source distribution 140614 14:24:40 [Note] /usr/sbin/mysqld: Normal shutdown
140614 14:24:40 [Note] Event Scheduler: Purging the queue. 0 events 140614 14:24:40 InnoDB: Starting shutdown... 140614 14:24:41 InnoDB: Shutdown completed; log sequence number 219455836 140614 14:24:41 [Note] /usr/sbin/mysqld: Shutdown complete
140614 14:24:42 [Warning] No argument was provided to --log-bin and neither --log-basename or --log-bin-index where used; This may cause repliction to break when
ostname changed! Please use '--log-basename=myhost' or '--log-bin=mysqld-bin' to avoid this problem. 140614 14:24:42 InnoDB: The InnoDB memory heap is disabled 140614 14:24:42 InnoDB: Mutexes and rw_locks use GCC atomic builtins 140614 14:24:42 InnoDB: Compressed tables use zlib 1.2.8 140614 14:24:42 InnoDB: Using Linux native AIO 140614 14:24:42 InnoDB: Initializing buffer pool, size = 16.0M 140614 14:24:42 InnoDB: Completed initialization of buffer pool 140614 14:24:42 InnoDB: highest supported file format is Barracuda. 140614 14:24:42 InnoDB: Waiting for the background threads to start 140614 14:24:43 Percona XtraDB (http://www.percona.com) 5.5.37-MariaDB-34.0 started; log sequence number 219455836 140614 14:24:43 [Warning] /usr/sbin/mysqld: unknown
140614 14:24:43 [Note] Server socket created on IP: '0.0.0.0'. 140614 14:24:43 [Note] Event Scheduler: Loaded 0 events 140614 14:24:43 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.37-MariaDB-log' socket: '/var/run/mysqld/mysqld.sock' port: 3306 Source distribution 140615 10:36:21 [Note] /usr/sbin/mysqld: Normal shutdown
140615 10:36:21 [Note] Event Scheduler: Purging the queue. 0 events 140615 10:36:21 InnoDB: Starting shutdown... 140615 10:36:22 InnoDB: Shutdown completed; log sequence number 219486293 140615 10:36:22 [Note] /usr/sbin/mysqld: Shutdown complete
140615 10:36:22 [Warning] No argument was provided to --log-bin and neither --log-basename or --log-bin-index where used; This may cause repliction to break when
ostname changed! Please use '--log-basename=myhost' or '--log-bin=mysqld-bin' to avoid this problem. 140615 10:36:22 InnoDB: The InnoDB memory heap is disabled 140615 10:36:22 InnoDB: Mutexes and rw_locks use GCC atomic builtins 140615 10:36:22 InnoDB: Compressed tables use zlib 1.2.8 140615 10:36:22 InnoDB: Using Linux native AIO 140615 10:36:22 InnoDB: Initializing buffer pool, size = 16.0M 140615 10:36:22 InnoDB: Completed initialization of buffer pool 140615 10:36:22 InnoDB: highest supported file format is Barracuda. 140615 10:36:22 InnoDB: Waiting for the background threads to start 140615 10:36:23 Percona XtraDB (http://www.percona.com) 5.5.37-MariaDB-34.0 started; log sequence number 219486293 140615 10:36:23 [Warning] /usr/sbin/mysqld: unknown
140615 10:36:23 [Note] Server socket created on IP: '0.0.0.0'. 140615 10:36:23 [Note] Event Scheduler: Loaded 0 events 140615 10:36:23 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.37-MariaDB-log' socket: '/var/run/mysqld/mysqld.sock' port: 3306 Source distribution 140701 5:56:47 [Note] /usr/sbin/mysqld: Normal shutdown
140701 5:56:47 [Note] Event Scheduler: Purging
140701 5:56:47 InnoDB: Starting shutdown... 140701 5:56:48 InnoDB: Shutdown completed; log sequence number 240645473 140701 5:56:48 [Note] /usr/sbin/mysqld: Shutdown complete
140701 5:56:48 [Warning] No argument was provided to --log-bin and neither --log-basename or --log-bin-index where used; This may cause repliction to break when
140701 5:56:48 InnoDB: The InnoDB memory heap is disabled 140701 5:56:48 InnoDB: Mutexes and rw_locks use GCC atomic builtins 140701 5:56:48 InnoDB: Compressed tables use zlib 1.2.8 140701 5:56:48 InnoDB: Using Linux native AIO 140701 5:56:48 InnoDB: Initializing buffer pool, size = 16.0M 140701 5:56:48 InnoDB: Completed initialization of buffer pool 140701 5:56:48 InnoDB: highest supported file
Yes, the error I posted comes from the client, but apparently the server is seeing errors from the client too... I just realized that when I responded to this thread again on 6/30, I neglected to include some new additional/important info regarding this issue that happened a couple of weeks after this initial report (about a week ago) that leads me to believe that I should be able to troubleshoot this, at least to an extent, on the mysql server level... Per my initial email, these errors were happening randomly for a couple of weeks, and only 1 to 3 or 4 times per day... But a couple of weeks later, my mysql server actually blocked the SOGo server completely: 2014-06-29 19:49:01.986 sogo-tool[7566] ERROR: could not open MySQL4 connection to database 'sogo': Host 'sogo.example.com' is blocked because of many connection errors; unblock with 'mysqladmin flush-hosts' This clearly shows that the mysql server was seeing too many errors from the client. I had to log into the mysql server and issue the flush-hosts command to restore communication between the two. But, I cannot find any evidence of any connection errors in /var/log/mysql/mysqld.err or any other logs on my mysql server. So... where should I find these errors that were occurring that caused mariadb to block all connections to my SOGo host until I issued the flush-hosts command? The only things in /var/log/mysql/mysqld.err on both sides of the date/time that I updated SOGo are: format is Barracuda. this server acts as a master and has its h option '--loose-federated' this server acts as a master and has its h option '--loose-federated' the queue. 0 events this server acts as a master and has its hostname changed! Please use '--log-basename=myhost' or '--log-bin=mysqld-bin' to avoid this problem. format is Barracuda.
140701 5:56:48 InnoDB: Waiting for the background threads to start 140701 5:56:49 Percona XtraDB (http://www.percona.com) 5.5.37-MariaDB-34.0 started; log sequence number 240645473 140701 5:56:49 [Warning] /usr/sbin/mysqld: unknown option '--loose-federated' 140701 5:56:49 [Note] Server socket created on IP: '0.0.0.0'. 140701 5:56:49 [Note] Event Scheduler: Loaded 0 events 140701 5:56:49 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.37-MariaDB-log' socket: '/var/run/mysqld/mysqld.sock' port: 3306 Source distribution
Anyone have any ideas on how to proceed troubleshooting this? It is most likely just a matter of time until the block happens again, and I'd prefer to try to find the problem and fix it, and I'll have better luck fixing this on the client side if I can give more details of errors the server is seeing from them. Thanks _______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
On 7/9/2014 12:24 PM, Federico Razzoli <federico_raz@yahoo.it> wrote:
I know nothing about SOGo (this is a MariaDB list), but I searched the proper mailing list for you, and it seems that someone already had your problem:
That is me from another account - still trying to fix this.
Am 09.07.2014 17:45, schrieb Tanstaafl:
On 7/2/2014 8:30 AM, Reindl Harald <h.reindl@thelounge.net> wrote:
you refuse to understand that this message comes from the*client* and not from MySQL/MariaDB - so guess why nobody here gave an answer
Yes, the error I posted comes from the client, but apparently the server is seeing errors from the client too...
I just realized that when I responded to this thread again on 6/30, I neglected to include some new additional/important info regarding this issue that happened a couple of weeks after this initial report (about a week ago) that leads me to believe that I should be able to troubleshoot this, at least to an extent, on the mysql server level...
Per my initial email, these errors were happening randomly for a couple of weeks, and only 1 to 3 or 4 times per day...
But a couple of weeks later, my mysql server actually blocked the SOGo server completely:
2014-06-29 19:49:01.986 sogo-tool[7566] ERROR: could not open MySQL4 connection to database 'sogo': Host 'sogo.example.com' is blocked because of many connection errors; unblock with 'mysqladmin flush-hosts'
because your server has enough from that broken client http://mysqlblog.fivefarmers.com/2013/08/08/understanding-max_connect_errors...
On 7/9/2014 12:52 PM, Reindl Harald <h.reindl@thelounge.net> wrote:
Am 09.07.2014 17:45, schrieb Tanstaafl:
But a couple of weeks later, my mysql server actually blocked the SOGo server completely:
2014-06-29 19:49:01.986 sogo-tool[7566] ERROR: could not open MySQL4 connection to database 'sogo': Host 'sogo.example.com' is blocked because of many connection errors; unblock with 'mysqladmin flush-hosts'
because your server has enough from that broken client http://mysqlblog.fivefarmers.com/2013/08/08/understanding-max_connect_errors...
But my point is shouldn't these errors that the server is seeing show in the (mysql) logs somewhere?
On 7/9/2014 1:29 PM, Tanstaafl <tanstaafl@libertytrek.org> wrote:
On 7/9/2014 12:52 PM, Reindl Harald <h.reindl@thelounge.net> wrote:
Am 09.07.2014 17:45, schrieb Tanstaafl:
But a couple of weeks later, my mysql server actually blocked the SOGo server completely:
2014-06-29 19:49:01.986 sogo-tool[7566] ERROR: could not open MySQL4 connection to database 'sogo': Host 'sogo.example.com' is blocked because of many connection errors; unblock with 'mysqladmin flush-hosts'
because your server has enough from that broken client http://mysqlblog.fivefarmers.com/2013/08/08/understanding-max_connect_errors...
But my point is shouldn't these errors that the server is seeing show in the (mysql) logs somewhere?
So I can then have a little more ammunition to bring to the devs in charge of the connecting Client?
Am 09.07.2014 19:31, schrieb Tanstaafl:
On 7/9/2014 1:29 PM, Tanstaafl <tanstaafl@libertytrek.org> wrote:
On 7/9/2014 12:52 PM, Reindl Harald <h.reindl@thelounge.net> wrote:
Am 09.07.2014 17:45, schrieb Tanstaafl:
But a couple of weeks later, my mysql server actually blocked the SOGo server completely:
2014-06-29 19:49:01.986 sogo-tool[7566] ERROR: could not open MySQL4 connection to database 'sogo': Host 'sogo.example.com' is blocked because of many connection errors; unblock with 'mysqladmin flush-hosts'
because your server has enough from that broken client http://mysqlblog.fivefarmers.com/2013/08/08/understanding-max_connect_errors...
But my point is shouldn't these errors that the server is seeing show in the (mysql) logs somewhere?
So I can then have a little more ammunition to bring to the devs in charge of the connecting Client?
the client fails - mysql is only the messenger * the client did not fail before upgrade the client * now it fails it's the same state as in your fist post: the developers of the client software are responsible treat them to fix that or downgrade that software
Am 09.07.2014 19:29, schrieb Tanstaafl:
On 7/9/2014 12:52 PM, Reindl Harald <h.reindl@thelounge.net> wrote:
Am 09.07.2014 17:45, schrieb Tanstaafl:
But a couple of weeks later, my mysql server actually blocked the SOGo server completely:
2014-06-29 19:49:01.986 sogo-tool[7566] ERROR: could not open MySQL4 connection to database 'sogo': Host 'sogo.example.com' is blocked because of many connection errors; unblock with 'mysqladmin flush-hosts'
because your server has enough from that broken client http://mysqlblog.fivefarmers.com/2013/08/08/understanding-max_connect_errors...
But my point is shouldn't these errors that the server is seeing show in the (mysql) logs somewhere?
why should they? if they would and i can access your database server it takes me one minute to DOS your machine by write the disk full with "is blocked because of many connection errors" messages to log everything unconditional is naive and if you ever had a real and serious distributed DOS to any of your servers you know why
On 7/9/2014 1:55 PM, Reindl Harald <h.reindl@thelounge.net> wrote:
Am 09.07.2014 19:29, schrieb Tanstaafl:
But my point is shouldn't these errors that the server is seeing show in the (mysql) logs somewhere?
why should they?
if they would and i can access your database server it takes me one minute to DOS your machine by write the disk full with "is blocked because of many connection errors" messages
Sometimes I think you are being intentionally obtuse. I mean THE ERRORS THAT THE SERVER SEES THAT CAUSES THE HOST TO BE BLOCKED. Not the subsequent errors saying that it is blocked.
Am 09.07.2014 20:35, schrieb Tanstaafl:
On 7/9/2014 1:55 PM, Reindl Harald <h.reindl@thelounge.net> wrote:
Am 09.07.2014 19:29, schrieb Tanstaafl:
But my point is shouldn't these errors that the server is seeing show in the (mysql) logs somewhere?
why should they?
if they would and i can access your database server it takes me one minute to DOS your machine by write the disk full with "is blocked because of many connection errors" messages
Sometimes I think you are being intentionally obtuse.
no - the only one who is obtuse in that thread is you because you ignore over a long time that this is a client issue nobody on the server side can help you downgrade the client or treat it's developers - period
I mean THE ERRORS THAT THE SERVER SEES THAT CAUSES THE HOST TO BE BLOCKED. Not the subsequent errors saying that it is blocked
the server don't need to see specific errors so stop calling others obtuse while you are lacking depper knowldege try it out, just connect in a loop to a mysqld server followed by a disconnect and you will trigger it too - any connection not ending in a successfull login counts so if cour broken client connects, don't like the repsone and decides to close the connection there is not much else than add +1 to the count for the server side
provide at least a test case, without test cases we can't help you with good answers and solve the problem
Hi, Tanstaafl! On Jun 18, Tanstaafl wrote:
Hi all,
I'm very inexperienced mariadb/mysql/sql user/admin, but not totally clueless, been maintaining a tiny mysql DB for postfixadmin for years, then converted it to mariadb.
We recently started using SOGo (groupware). I hired the SOGo guys to help with the install/config, and have been using it trouble-free for a few months now (love it for the most part), until now.
Everything seems to be working fine, but since the upgrade to SOGo 2.2.4 on 6/12, which I am told increases the number of connections it uses to the DB, I have been getting very occasional errors like this (the system error is always either 104 or 0):
2014-06-12 03:30:15.530 sogo-tool[31674] ERROR: could not open MySQL4 connection to database 'sogo': Lost connection to MySQL server at 'reading authorization packet', system error: 104 <0x0x236c140[GCSChannelManager]> could not open channel <MySQL4Channel[0x0x22241b0] connection=0x(null)> for URL: mysql://user:password@myhost.com:3306/sogo/sogo_sessions_folder <0x0x236c140[GCSChannelManager]> will prevent opening of this channel 5 seconds after 2014-06-12 03:30:01 -0400 2014-06-12 03:30:15.531 sogo-tool[31674] Can't aquire channel
But it only happened once per day after the upgrade, then once it happened twice (about 12 hours apart), and it happened at times when the load would be the lowest (ie, 3:30am for the one above).
MySQL4 - what does it mean? Using the protocol as it existed in MySQL version 4.x? The connector - does SOGo use libmysqlclient of some specific version? Does it have its own implementation of MySQL protocol? Regards, Sergei
On 6/18/2014 7:30 AM, Tanstaafl <tanstaafl@libertytrek.org> wrote:
Googling reveals a lot of reports about this error over the years, some seemingly to be bugs, others DB tuning issues.
Should I be looking at tuning:
max_connections max_user_connections ?
If not, does anyone know where I should start looking? Is there some kind of extra logging I could enable that wouldn't cause serious performance problems?
Or... maybe this is simply not something I should be worried about?
Thanks for any help/suggestions...
Anyone have a suggestion how best to adjust logging so I can maybe see what could be causing these? Thanks
participants (5)
-
Federico Razzoli -
Reindl Harald -
Roberto Spadim -
Sergei Golubchik -
Tanstaafl