After upgrade from 10.11.11 to 10.11.13, Mariadb will not start with the existing SSL certs.

Hello, (Not reporting as a bug immediately; looking for confirmation and advice before doing so as it feels like this would be more widely reported if others are affected.) Several of our Mariadb servers failed to return after regular patching this morning after Mariadb upgraded from 10.11.11 to 10.11.13. Looks like, after being happy with the SSL keys for over six months, Mariadb suddenly doesn't like them. This looks like something has changed in Mariadb in this version that is stopping it reading our SSL keys, but I cannot see anything in the release notes relating to changes to SSL for .12 or .13 https://mariadb.com/kb/en/mariadb-10-11-13-changelog/ Workings: Patching immediately before updated from 10.11.11to 10.11.13. No other updates applied apart from zabbix-agent). VMs are running Rocky 9. Upgrading: MariaDB-client x86_64 10.11.13-1.el9 mariadb_10.11_r9 9.3 M MariaDB-common x86_64 10.11.13-1.el9 mariadb_10.11_r9 88 k MariaDB-server x86_64 10.11.13-1.el9 mariadb_10.11_r9 18 M MariaDB-shared x86_64 10.11.13-1.el9 mariadb_10.11_r9 131 k Before patching, mariadb was using one-sided SSL with self-signed SSL certs. Everything working as expected, been in place for over six months. After Maria upgraded from 10.11.11 to .13, Mariadb will not start. 250526 6:38:03 server_audit: logging started to the syslog. SSL error: Unable to get private key from '/etc/my.cnf.d/ssl/server-key.pem' 2025-05-26 6:38:03 0 [ERROR] Failed to setup SSL 2025-05-26 6:38:03 0 [ERROR] SSL error: Unable to get private key 2025-05-26 6:38:03 0 [ERROR] Aborting server-key.pem looks okay, starts "-----BEGIN RSA PRIVATE KEY-----" - privs are 700, owned by mysql user, turning off selinux doesn't fix it. File date six months old (it's a year long key with checks to renew at -30d), and openssl likes it; # openssl rsa -in server-key.pem -check RSA key ok writing RSA key -----BEGIN RSA PRIVATE KEY----- My temporary fix is removing this from the config and then mariadb will restart. Fortunately are clients don't require SSL. [mysqld] #ssl-ca=/etc/my.cnf.d/ssl/ca-cert.pem #ssl-cert=/etc/my.cnf.d/ssl/server-cert.pem #ssl-key=/etc/my.cnf.d/ssl/server-key.pem Thank you

Additional: Curiously, my servers running Rocky 8 have not been affected by this little problem when upgrading MariaDb from 10.11.11 to 10.11.13 and are running fine.

Hi, Simon, The only remotely related change I could think of was a fix for MDEV-36229 - in 10.11.11 MariaDB had CAP_DAC_OVERRIDE capability, basically ignoring filesystem level access privileges. I know you wrote "privs are 700, owned by mysql user". But as this is the only thing I was able to find - double-check, perhaps? May be sudo mysql and try to read the file, or strace mariadbd startup if possible. Regards, Sergei Chief Architect, MariaDB Server and security@mariadb.org On May 27, Simon Avery via discuss wrote:
Hello,
(Not reporting as a bug immediately; looking for confirmation and advice before doing so as it feels like this would be more widely reported if others are affected.)
Several of our Mariadb servers failed to return after regular patching this morning after Mariadb upgraded from 10.11.11 to 10.11.13. Looks like, after being happy with the SSL keys for over six months, Mariadb suddenly doesn't like them.
This looks like something has changed in Mariadb in this version that is stopping it reading our SSL keys, but I cannot see anything in the release notes relating to changes to SSL for .12 or .13 https://mariadb.com/kb/en/mariadb-10-11-13-changelog/
Workings:
Patching immediately before updated from 10.11.11to 10.11.13. No other updates applied apart from zabbix-agent). VMs are running Rocky 9.
Upgrading: MariaDB-client x86_64 10.11.13-1.el9 mariadb_10.11_r9 9.3 M MariaDB-common x86_64 10.11.13-1.el9 mariadb_10.11_r9 88 k MariaDB-server x86_64 10.11.13-1.el9 mariadb_10.11_r9 18 M MariaDB-shared x86_64 10.11.13-1.el9 mariadb_10.11_r9 131 k
Before patching, mariadb was using one-sided SSL with self-signed SSL certs. Everything working as expected, been in place for over six months.
After Maria upgraded from 10.11.11 to .13, Mariadb will not start.
250526 6:38:03 server_audit: logging started to the syslog. SSL error: Unable to get private key from '/etc/my.cnf.d/ssl/server-key.pem' 2025-05-26 6:38:03 0 [ERROR] Failed to setup SSL 2025-05-26 6:38:03 0 [ERROR] SSL error: Unable to get private key 2025-05-26 6:38:03 0 [ERROR] Aborting
server-key.pem looks okay, starts "-----BEGIN RSA PRIVATE KEY-----" - privs are 700, owned by mysql user, turning off selinux doesn't fix it. File date six months old (it's a year long key with checks to renew at -30d), and openssl likes it;
# openssl rsa -in server-key.pem -check RSA key ok writing RSA key -----BEGIN RSA PRIVATE KEY-----
My temporary fix is removing this from the config and then mariadb will restart. Fortunately are clients don't require SSL.
[mysqld] #ssl-ca=/etc/my.cnf.d/ssl/ca-cert.pem #ssl-cert=/etc/my.cnf.d/ssl/server-cert.pem #ssl-key=/etc/my.cnf.d/ssl/server-key.pem

Sergie, I again owe you my thanks. The prompt to re-examine the file ownership was spot-on. Somehow, they had become owned by root - possibly some time before, and not mysql: Thus the 600 mode was preventing maria from reading the file and the error message was not hinting at an incompatibility, but was accurate - it could not open the file. Chowning the ssl files back to mysql: allowed Maria to restart. I'm relieved the issue is entirely local! Thanks again, Simon -----Original Message----- From: Sergei Golubchik <serg@mariadb.org> Sent: 27 May 2025 12:38 To: Simon Avery <Simon.Avery@atass-sports.co.uk> Cc: discuss@lists.mariadb.org Subject: Re: [MariaDB discuss] After upgrade from 10.11.11 to 10.11.13, Mariadb will not start with the existing SSL certs. Hi, Simon, The only remotely related change I could think of was a fix for MDEV-36229 - in 10.11.11 MariaDB had CAP_DAC_OVERRIDE capability, basically ignoring filesystem level access privileges. I know you wrote "privs are 700, owned by mysql user". But as this is the only thing I was able to find - double-check, perhaps? May be sudo mysql and try to read the file, or strace mariadbd startup if possible. Regards, Sergei Chief Architect, MariaDB Server and security@mariadb.org
participants (2)
-
Sergei Golubchik
-
Simon Avery