Hi, I think that the two implementations can coexist and my suggestion in MDEV-15935 <https://jira.mariadb.org/browse/MDEV-15935> about making the error packet version a "forceful" redirection would allow that. I believe that a reasonable way of doing a very graceful redirection would be as follows: * Turn on the redirection to another server (SET GLOBAL server_redirect_mode=ON). This then causes an advisory redirection to be added to all OK packets as well as EOF packets if the deprecate EOF protocol is in use. Newer client libraries would use this to migrate over to another server once their transactions are over and older clients that either don't implement the deprecate EOF protocol or don't know about redirection would remain oblivious to this. * After some grace period, the forceful redirection would be turned on (SET GLOBAL server_redirect_mode=FORCE). This would turn the advisory redirection into the currently proposed forceful redirection that sends an error packet. This amounts to the same as lowering max_connections to prevent new connections from taking place except that you get a different error message. * Once forceful redirection is in effect, you would wait for open transactions to complete, possibly after turn on read_only, and then shut down the server. As for the redirection information, a URL would be a more flexible way of connecting and would allow things like UNIX domain sockets and named pipes to be used as redirection targets. I definitely agree that a simple host and port approach (correctly formatted to support IPv6 of course) must be accepted as a redirection by all client libraries but some form of standardization for the redirection format string is needed or at least how client libraries should interpret unknown values. The forceful redirection has one minor problem that the advisory redirection does not have: only one redirection target can be given in the current form. The reason for multi-host redirections would be to prevent sudden spikes of new connections from happening on the server where the redirection is being done. The advisory redirection, as mentioned in MDEV-15935, could use SQL to set a session-level value which in turn opens up limitless possibilities of using it for load balanced redirection. This is probably a very minor in the grand scheme of things and having one host to redirect to is probably OK in most cases if the advisory redirection is implemented on top of the forceful one: the former would shift most of the connections away from the server and the latter would only be the final step that prevents new ones from being created. For things like JDBC connection pools that support multiple hosts, redirections are technically already implemented: I believe they just pick another server when they get a Too many connections error. The benefits of a forceful redirection are thus minimal for these use-cases and the advisory one would be much more effective in avoiding new work from being placed on servers that are about to shut down. However even in this case the redirection error could be interpreted as an instruction to avoid using the given server for some period of time. Markus On 6/29/23 22:30, Lenski, Daniel via developers wrote:

Hi Sergei,

On Thu, 2023-06-29 at 15:58 +0200, Sergei Golubchik wrote:

...

On Jun 28, Lenski, Daniel via developers wrote:

...

I would like to propose a new feature in the MariaDB client-server protocol<https://mariadb.com/kb/en/clientserver-protocol>: application-layer redirection of client connections.

We want the MariaDB server to be able to tell clients connecting to it, “Sorry, this server is unavailable. Connect to an alternate server instead.” This mechanism is inspired by HTTP 302 (“temporary redirect”) mechanism familiar to all developers of web applications, and is intended to have similar semantics and security properties, since these have now been widely deployed and tested for decades.

I have submitted a minimal but viable implementation of this at:

*https://github.com/MariaDB/server/pull/2681, server-side implementation *https://github.com/mariadb-corporation/mariadb-connector-c/pull/226, MariaDB Connector/C implementation I'm afraid MDEV-15935 is a bit confusing. There are lots of discussions and ideas in the comments, like, using a specially modified OK packet, using the error message, etc. Right! I got the sense from the Jira discussion (https://jira.mariadb.org/browse/MDEV-15935) that there's a large and varied "wish lish" of different capabilities for this feature — not all of which are practical or compatible.

It has also been open for over 5 years without an accepted implementation.

This is why I thought it would make a lot of sense to implement redirection in a very MINIMAL and SIMPLE form — but one that would already be useful in its initial form, and with a design that's amentable to future extensibility. (Many thanks to Otto for encouraging and guiding this implementation!)

...

But the final consensus was to use session tracking feature, it allows for most flexibility and user control over how redirect is happening. To quote here:

We'll add a new global/session variable, say, redirect_url. The value should be an empty string or a connection string in the conventional format (in the style of a connection url of C/J or C/NodeJS or C/C or FederatedX).

This variable is appended to the default value of session_track_system_variables variable.

When this variable is changed, the existing SESSION_TRACK_SYSTEM_VARIABLES implementation will send its new value to the client as a part of the OK packet. It's up to the client or to the connector to use this value or to ignore it. I read through this proposal as well, but I didn't realize that it had been in any way chosen as the desired implementation. I am surprised, because there seems to be 2 key problems with it…

(1) Redirection targets in "URL" format will be extremely difficult to support in consistent and secure ways across client implementations.

The Connector/C library currently has no ability to parse connection "URLs" in either the Connector/J (https://mariadb.com/kb/en/about-mariadb-connector-j) or the MySQL formats ( https://dev.mysql.com/doc/connector-j/8.0/en/connector-j-reference-jdbc-url-...), which are partially but fully compatible with each other, and to my knowledge there is no formal definition of either. As far as I know, Connector/C is the basis for most higher-level languages’ connector libraries (e.g. Connector/Python).

If the redirection target can contain all the different kinds of connection information that MariaDB Connector/J URLs support (very long list of optional parameters! https://mariadb.com/kb/en/about-mariadb-connector-j/#optional-url-parameters), it will be very difficult to ensure that all languages’ Connector/X libraries handle all of these parameters in a consistent and secure way when redirecting.

For example, if a *server* sends a redirect URL containing the optional `keyStorePassword` field, should a *client* library trust or attempt to use this value? I think not. Or what if Connector/J observes the `timezone` parameter when redirecting to a new server, but Connector/C ignores it? This would be a source of inconsistent behavior and could frustrate users of the redirection feature.

Precisely in order to avoid this level of complexity, my proposal includes a *much simpler* format for the redirection target: `host[:port]`. This should be simple enough for any and all Connector/X client libraries to implement in a uniform fashion. At the same time, it is designed to be extensible; if there are clear use cases for it, it could be extended in the future to include more complex formats for the redirection target, such as Connector/J-style and/or MySQL-style connection URLs.

(2) Using OK packets and session-tracking, the server would have no way to enforce redirection.

As you wrote, "It's up to the client or to the connector to use this value [of redirect_url] or to ignore it."

In this form, redirection instructions sent by the server would be merely "advisory", and the server would have no way to ensure that clients stop connecting to it.

- Clients using OLD Connector libraries will continue to be oblivious to this value, and to ignore it. As all of us have experienced, old/forgotten/abandoned/unpatched client software is often used for DECADES beyond its originally-intended lifespan. - Clients using NEW Connector libraries can be configured to ignore the redirection value. Quite likely many of them WILL do this, since connecting to a new server is more difficult and more error- prone than just continuing to send queries to the already-connected server.

This "advisory" redirection would seem to be greatly limited in the kind of use cases it supports. If servers sending redirection information can't RELY ON clients disconnecting from them, then they can't use redirection as a prelude to shutting down the server.

This is why my proposal instead includes the redirection information in an ERROR packet: supporting clients will be able to follow the redirection (or simply disconnect if they don't want to follow redirects, e.g. `mariadb --no-follow-redirects`), while older clients will be disconnected with a comprehensible error message. In this way, a MariaDB server will be able to RELY ON clients disconnecting from it, which will simplify and clarify the usage of redirection for server shutdown. Note that my initial proposal only involves sending ER_SERVER_REDIRECT error packets to *newly- connecting* clients; in the future, it could also be extended to send similar packets to *already-existing* clients, for example for load-balancing purposes.

Thanks,

Daniel Lenski Amazon RDS MySQL/MariaDB engine team _______________________________________________ developers mailing list --developers@lists.mariadb.org To unsubscribe send an email todevelopers-leave@lists.mariadb.org

-- Markus Mäkelä, Senior Software Engineer MariaDB Corporation

On Fri, 2023-06-30 at 13:29 +0200, Sergei Golubchik wrote:

There was an original pull request from ~5 years ago, but then the contributor disappeared and I didn't want to do anything until they'd at least confirm that the new suggested approach works for them Are you referring to this Connector/J PR, which attempted to add client-side parsing of a redirection string in a URL-ish format? https://github.com/mariadb-corporation/mariadb-connector-j/pull/134/files#di...

...

(1) Redirection targets in "URL" format will be extremely difficult to support in consistent and secure ways across client implementations.

MDEV-15935 itself (the server part) does not specify what the URL should look like, it's only about passing basically an arbitrary string to the client.

Giving the server the ability to passing "an arbitrary string" to the client will create many more problems than it will solve, if there is no agreement for how the client software should interpret that string. If the server can send a redirect string that looks something like "mysql://UserB@other-server.company.com:3306/my_database?ttl=10&user=UserA" (as proposed in the stale Connector/J PR), but Connector/C ignores the query parameter fields including the username, and Connector/J misinterprets UserB as part of the hostname… that’s chaos, not a solution.

Although the original PR used URLs, Federated uses URLs, Connector/J uses URLs - it's de facto very common already. It might be good to standardize the URL format and use it everywhere.

Agreed. However, this is a complex undertaking, and older/non-standard URL formats WILL continue to proliferate if existing clients and servers continue to accept them or silently ignore them.

...

The Connector/C library currently has no ability to parse connection "URLs" in either the Connector/J (https://mariadb.com/kb/en/about-mariadb-connector-j) or the MySQL formats ( https://dev.mysql.com/doc/connector-j/8.0/en/connector-j-reference-jdbc-url-...), which are partially but fully compatible with each other, and to my knowledge there is no formal definition of either. As far as I know, Connector/C is the basis for most higher-level languages’ connector libraries (e.g. Connector/Python).

Not quite. Connector/C can parse both DSN format (key=value pairs, semicolon-separated) and URI format, See https://github.com/mariadb-corporation/mariadb-connector-c/blob/5af90f00ff/l... and https://github.com/mariadb-corporation/mariadb-connector-c/blob/5af90f00ff/l...

although the latter cannot be used for normal connections yet, which is good, becayse without standard URI syntax it would've definitely been incompatible with everything.

All that the second snippet appears to show is that Connector/C can look for "://" in a string; as you say, it "cannot be used for normal connections yet." This does not mean that it can parse a string like "jdbc:mysql://mariadb.server.com:3306/defaultDB?characterEncoding=latin1" and set all of its relevant internal state in a way that's equivalent to what Connector/J does. Because of this, it would be inadvisably premature to ALLOW servers to SEND a JDBC URL like that as a redirection target, since we cannot expect various client software/libraries to be able handle that URL in a consistent way. Compare this with HTTP 3xx redirection, which has very well-defined semantics in terms of caching, HTTP methods, etc (summarized in https://en.wikipedia.org/wiki/URL_redirection#HTTP_status_codes_3xx) . Because of that servers can generally rely on clients to follow redirection instructions in a consistent way.

Right, that's why I moved URI topic out of MDEV-15935, making the latter only about "how to pass <arbitrary string> to the client"

There are two common approaches to a connection string - let's call them "JDBC way" (that's URI) and "ODBC way" (semicolon separated key=value pairs). mysqljs uses URI, Perl DBI uses DSN, PHP PDO uses DSN, Federated engine uses URI, etc.

I'd really rather avoid introducing a new connection string format.

Agreed, not introducing a new format makes sense.

MDEV-15935 says that we "might" optionally implement URI validation in the server. Without it one can literally send an arbitrary string. This might be error-prone. With the server side URI validation we can limit redirection URIs to "mysql://host[:port]" which basically is what you wanted and postpones all the URI complexity till we have some standard specs for it.

I'd be happy to modify my PRs to only allow SERVER_REDIRECT_TARGET of the form "mysql://host[:port]", to avoid introducing a new format.

...

This "advisory" redirection would seem to be greatly limited in the kind of use cases it supports. If servers sending redirection information can't RELY ON clients disconnecting from them, then they can't use redirection as a prelude to shutting down the server.

Of course, they can. The idea is to tell clients to connect to another server and to disconnect them after that. If redirection is forced, then the server will send a "redirection error" and disconnect. If redirection is advisory, the server will send a redirection info and disconnect few seconds later when it shuts down.

That would be substantially more complex to implement. If the server can simply send an error packet that says "Go somewhere else" and then disconnect the client, it doesn't need to track any further information on the connection. If the server needs to send an advisory packet that says "Please go somewhere else or I'll disconnect you in 15 seconds", and then continue handling that connection as normal, and then interrupt and disconnect it 15 seconds later… that requires a lot more state tracking. (What happens if the client starts a long-running query, or does some non-transactional DDL, or does anything with a non- transactional, non-crash-safe engine?)

Anyway, in the original proposal (from ~5 years ago) it was advisory, apparently it's what the user needed.

But WHY did the user need that? What are the use cases for "advisory-ONLY" redirection? Does it make sense to design and implement this feature be based on what one user thought would be useful 5 years ago, rather than basing it on a stateless redirection model that has been successfully used in HTTP for 25+ years? Thanks, Daniel Lenski Amazon RDS MySQL/MariaDB engine team

Hi Georg, On Wed, 2023-07-05 at 20:15 +0200, Georg Richter wrote: Do you agree it makes sense to have a new error that can tell clients to go to another server (similar to how HTTP 301 redirects work)? I would very much like to hear your opinion on this one. If you are in agreement, Daniel can spend more time polishing the feature and also taking care of implementation details such as preventing redirect loops and other corner cases. If we want to implement redirection via error message, then the client needs an option… Done. 1. Added to the Connector/C PR in https://github.com/mariadb-corporation/mariadb-connector-c/pull/226/commits/... 2. Added to the mariadb CLI as part of the MariaDB/Server PR, in https://github.com/MariaDB/server/pull/2681/commits/b43f5909605c8b31cef22156... … an option to enable redirecting, it should be disabled by default. Why does it make sense for it to be disabled by default? Again, I'm trying to follow the semantics of HTTP 3xx redirects here. Most HTTP(S) clients (e.g. web browsers) do follow redirections by default, and only disable them in specific cases where the server configuration is broken. For example, if the client encounters a redirect loop, or gets redirected to a server with an invalid certificate; the latter case has nothing to do with redirection per se. The current PR introduces a new vulnerability, since the client redirects automatically under the hood. Why should automatic redirection be considered a vulnerability? If a client, C1, trusts a MariaDB database server, S1, to handle its queries correctly, then presumably it should also trust that server to tell it to go redirect to another server, S2, and issues its queries to that alternate server instead. This is precisely the logic behind automatic redirection in HTTPS. If I trust "https://www.search-engine.com/?q=search+terms", then I shouldn't mind if the server tells me "go issue this same query to our secondary server at https://www2.search-engine.com/?q=search+terms". If redirection offered an opportunity for untrusted third parties to redirect to attacker-controlled servers, that would be a vulnerability. And that is precisely why it is crucial to plug the pre-existing https://jira.mariadb.org/browse/CONC-648 vulnerability while implementing redirection. Thanks, Daniel

On Sat, 2023-07-08 at 12:42 -0700, Antony T Curtis wrote:

In my opinion, disabled by default would be the correct choice except where using x509 certificate and requires server cert verification where it would be okay to enable redirection because an untrusted server should fail the verification.

Okay. A connection without TLS [1] or with TLS but without server cert verification [2] is already technically easy to MITM or silently downgrade to plaintext [3], but it is true that the risk is magnified by redirection, because a server that's on a trusted/local network may redirect to a server outside this local network, thus magnifying the practical concern. I will modify the Connector/C PR (https://github.com/mariadb-corporation/mariadb-connector-c/commit/54886ac29d...) such that… * If, and only if, the `ssl-verify-server-cert` or `ssl-fp-list` options are enabled, then Connector/C will enable `follow-server-redirects` by default. * Otherwise, `follow-server-redirects` will be disabled by default (but it will still be possible to enable it by explicitly specifying this option). Thanks, Daniel Lenski Amazon RDS MySQL/MariaDB engine team [1] `--ssl=OFF` [2] e.g. `--ssl` but not `--ssl-verify-server-cert` for the command-line client [3] Due to https://jira.mariadb.org/browse/MDEV-28634

Hi, Andrey, On Jul 19, andrey.sidorov--- via developers wrote:

Hi Daniel,

Have you considered using AUTH_SWITCH_REQUEST for that purpose? That would allow redirect to happen after switch to TLS and client/server certificate validation.

One of the main use cases at the moment is to redirect clients to another server when this current server is being shut down. For that we need to be able to send the redirect information in the middle of the session, not only when a connection is being established. Both session tracking and error message approach allow that. Regards, Sergei VP of MariaDB Server Engineering and security@mariadb.org

...

Have you considered using AUTH_SWITCH_REQUEST for that purpose? That would allow redirect to happen after switch to TLS and client/server certificate validation.

One of the main use cases at the moment is to redirect clients to another server when this current server is being shut down.

For that we need to be able to send the redirect information in the middle of the session, not only when a connection is being established. Both session tracking and error message approach allow that.

Not strictly so - we don't *need* to send the redirect information in the middle of the session. The session can just be dropped, and when clients automatically reconnect they will get the error message (if old client software) or do an automatic redirect to the new server (if https://github.com/MariaDB/server/pull/2681 is merged). This is similar to how HTTP 301/302 redirects work, and pretty sweet as it allows to gracefully drain the old server and for clients to fall back on tried-and-tested semantics on what is done on connection interrupts. Consider this scenario: 1. Server A is getting 1 new connection per second, has 100 existing active connections, which of 10 are actually doing a query and 2 of them are long-running transactions. 2. Admin prepares server B to replicate server A data, and once replica lag is low runs on server A: SET GLOBAL SERVER_REDIRECT_TARGET=sever-b.example.com 3. Server A redirects all new connections, server B gets 1 new connection per second but does not yet serve traffic, clients re-try connections/queries 4. Server A drops 98 connections, 2 continue with long-running transaction 5. After defined delay, server A also drops the long-running transaction connections to avoid being stuck on them 6. All traffic is hitting server B, which catching up in replication, clients keep re-connecting 7. Server B has caught up, get's promoted to primary, and starts accepting connections 8. Server B serves at this point all traffic, and clients/applications continue to work without errors as long as the total switchover delay was shorter than the configured re-try delay of clients 9. Server A serves no queries, all clients connecting to it get redirected to server B - after some delay server A can be completely shut down 10. If any of the clients had an old version of the MariaDB connector, they will stop working but error logs will clearly show that a redirect happened. No extra proxy software is necessary nor pre-distributed configuration. This redirect can be done even in an emergency without prior preparations if server A for example is stalled on a deadlock or if server A was overloaded and a failover to a new server with better hardware is needed.

Hi!

...

Not strictly so - we don't *need* to send the redirect information in the middle of the session. The session can just be dropped, and when clients automatically reconnect they will get the error message (if old client software) or do an automatic redirect to the new server (if https://github.com/MariaDB/server/pull/2681 is merged).

Automatic reconnect is disabled by default, so applications would just stop unless the reconnect option was enabled or they execute reconnect. (MySQL 8.1 already deprecated automatic reconnect).

All apps that run 24/7 have automatic reconnect, and will always have. It would not make sense for an app to stop running just because a database connection once got dropped for whatever reason. MySQL 8.1 is indeed removing the transparent automatic reconnect in their connector C [1], and it has been deprecated already in latest 8.0.34, but that isn't because the pattern of automatic reconnect goes away, they just want to make it always visible to the layer above managing the db connection (typically some ORM or programming language MySQL library, connection pool etc). The point still stands that changing the server from saying "sorry, can't do your query, try again" to "sorry, can't take your query, try again at this address" is nicely backwards compatible and has known behavior implications on all apps and connectors using the database, and that this would be a clean and robust way to do redirects and achieve e.g. near zero second failover. [1] https://dev.mysql.com/doc/c-api/8.0/en/c-api-auto-reconnect.html

Hi,

On Jul 23, Otto Kekäläinen wrote:

...

...

...

Have you considered using AUTH_SWITCH_REQUEST for that purpose? That would allow redirect to happen after switch to TLS and client/server certificate validation.

One of the main use cases at the moment is to redirect clients to another server when this current server is being shut down.

For that we need to be able to send the redirect information in the middle of the session, not only when a connection is being established. Both session tracking and error message approach allow that.

Not strictly so - we don't *need* to send the redirect information in the middle of the session. The session can just be dropped, and when clients automatically reconnect they will get the error message (if old client software) or do an automatic redirect to the new server (if https://github.com/MariaDB/server/pull/2681 is merged).

First, this requires the client to perform one extra connection attempt, just to get the redirection information. This sounds like a waste of time. And also this is not guaranteed to work - if the server shuts down, the socket might be already closed, so the client simply won't be able to connect again.

It is not a waste of time as the time it takes to attempt a connection and get an error+redirect reply is minimal. The actual delay of how much time is "wasted" in a switchover situation comes from how quickly the server A is able to drain all connections and server B to sync up and get promoted to primary, and that is the time the design should focus to optimize. It is guaranteed to work as long as the DBA keeps the server running and redirecting - just like with HTTP 301 redirects.

Second, it aborts the session in the middle where the client might not expect it. For example, in the middle of the transaction. Some clients might be able to replay it, others might not. That's why I suggest to let the client decide when it can reconnect.

The switchover is initiated and decided by the DBA controlling the server. You cannot possibly expect it to be controlled by the client - and in particular to be decided by the *slowest* client. The server can allow some time for the draining of existing connections to happen, but eventually it needs to tell old client connections to stop, and only after the redirect has started for new connections + all existing connections have been dropped can the actual switchover proceed by promoting a new primary.

...

This is similar to how HTTP 301/302 redirects work, and pretty sweet as it allows to gracefully drain the old server and for clients to fall back on tried-and-tested semantics on what is done on connection interrupts.

HTTP is stateless, what it does cannot always be blindly applied to the very much stateful MariaDB client-server protocol.

And, really, I don't understand how phrases "session can just be dropped" and "gracefully drain the old server" can be used in adjacent statements. There's nothing graceful about forcefully dropping the connection.

Perhaps not directly graceful, but at least all production grade clients have some mechanisms to cope with network interruptions and such. If you introduce a 'recommendation' based direct, it will lead clients to implement completely novel logic which in worst case leads to clients competing on which one is the last to use the old server to minimize per-client downtime and making system level total downtime in switchover worse for all other clients. Thus introducing a redirection that is based on server denying new connections will nicely align with existing error handling logic in clients.

Hello Sergei and Yuchen! I was looking at the code Yuchen wrote this week [1]. Thinking back about practical scenarios on how the redirect would be used to gracefully decommission server A and have all existing and new connections user server B instead (as laid out in this thread at end of July [2]) I fail to see how it would work. Consider this scenario with session track variable redirect_url: 1. Server A is getting 1 new connection per second, has 100 existing active connections, of which 10 are actually doing a query and 2 of them are long-running transactions. 2. Admin prepares server B to replicate server A data, and once replica lag is low runs on server A: SET GLOBAL redirect_url=mariadb://server-b.example.com 3. Server B gets 100 new connections immediately from clients that follow the session tracked variable. Server A continues to authenticate and accept all new connections, which leads to server B getting one *new* per second but does not yet serve traffic, clients re-try connections/queries. 4. Server A goes into READ_ONLY mode and any client that did not follow the redirect will get errors on INSERT/UPDATE/DELETE 5. All clients are reaching server B, which is catching up on replication, clients keep re-connecting in attempt to get queries fulfilled 6. Server B has caught up, gets promoted to primary, and starts accepting connections 7. Server B serves at this point all traffic, and clients/applications continue to work without errors as long as the total switchover delay was shorter than the configured re-try delay of clients 8. If at this point there are any clients that don't support the session tracked redirect_url, or even it they support it, but do not obey it, they will be server stale data from server A 9. Server A shuts down, servers no redirects anymore nor stale data 9. If any of the clients had an old version of the MariaDB connector, they will stop working and error logs will simply say that connection to server A failed I find it a bit concerning that in this model old clients, instead of getting an error with a human readable explanation of what happened, continue to read stale data. And only way to prevent that is to shut down server A quickly, whereafter the old clients only get an error about server not being reachable and nothing about the old client not supporting redirects. Hence, I still feel that Daniel's submission [3,4] would be a better design, being more robust and safer. An error+redirect is easier to reason about regarding integrity of data and error modes than the voluntary maybe-redirect that is now in the works by Yuchen. [1] https://github.com/MariaDB/server/commit/04b99a30a3f.patch). [2] https://lists.mariadb.org/hyperkitty/list/developers@lists.mariadb.org/threa... [3] https://github.com/MariaDB/server/pull/2681 [4] https://github.com/mariadb-corporation/mariadb-connector-c/pull/22

Hi, I wrote down an idea for out-of-band warning messages in MDEV-31839 <https://jira.mariadb.org/browse/MDEV-31839> where by reserving one "system variable" for warnings, we can inject them to the initial OK packet that the client receives. This way clients that understand the server variable changes are able to display them to the users. The MariaDB Connector/C is already able to extract the system variable change information even if it's in the first OK packet sent after authentication. I found this sort of accidentally when I was testing some code for MDEV-31609 <https://jira.mariadb.org/browse/MDEV-31609>. I would not be surprised if after MDEV-31609 this could be implemented by just adding a dummy server_motd global variable that when set is sent to the clients. Markus On 8/17/23 14:13, Marko Mäkelä via developers wrote:

Hi Otto, all,

On Wed, Aug 16, 2023 at 4:00 PM Otto Kekäläinen via developers <developers@lists.mariadb.org> wrote:

...

I find it a bit concerning that in this model old clients, instead of getting an error with a human readable explanation of what happened, continue to read stale data. And only way to prevent that is to shut down server A quickly, whereafter the old clients only get an error about server not being reachable and nothing about the old client not supporting redirects. This reminds me of a recent true story of a DBaaS provider that did not give adequate warning to users when decommissioning a server:

https://community.influxdata.com/t/getting-weird-results-from-gcp-europe-wes... https://news.ycombinator.com/item?id=36657829

It was not anything MySQL or MariaDB compatible, and they made matters worse by not only taking the server offline, but also deleting the data immediately.

For some users, the only notification was via some web based dashboard, which they might never have accessed.

I think that what might be useful is something analogous to the *nix /etc/motd, which is typically displayed to users when they log in. In the case of the MySQL or MariaDB client protocol, that would be some "out of band" data. It would not necessarily have to prevent new connections (at first), merely cause newer clients to output the "warning" message to some log. An example would be: "This server will be decommissioned on 2023-12-13. Please contact https://sales.example.com." Closer to the deadline, attempts to connect would lead to a hard error, along with the message.

...

Hence, I still feel that Daniel's submission [3,4] would be a better design, being more robust and safer. An error+redirect is easier to reason about regarding integrity of data and error modes than the voluntary maybe-redirect that is now in the works by Yuchen.

[1]https://github.com/MariaDB/server/commit/04b99a30a3f.patch). [2]https://lists.mariadb.org/hyperkitty/list/developers@lists.mariadb.org/threa... [3]https://github.com/MariaDB/server/pull/2681 [4]https://github.com/mariadb-corporation/mariadb-connector-c/pull/22 Could any of these approaches be extended to support a soft or hard "this database will be shut down" notice?

Marko

-- Markus Mäkelä, Senior Software Engineer MariaDB Corporation

Hi!

Yes, if the client doesn't support redirects, it'll work as before - that is, the client will be kicked out when the server is shut down and then the client will connect to a new server because, of course, if there's failover, there must be some way of telling clients what server to connect to. The error will be ER_SERVER_SHUTDOWN which is what old clients were always getting in this case. So, basically, for old clients there will be no changes whatsoever.

..which is not ideal. Some backwards compatibility would make the user experience much better.

...

Hence, I still feel that Daniel's submission [3,4] would be a better design, being more robust and safer. An error+redirect is easier to reason about regarding integrity of data and error modes than the voluntary maybe-redirect that is now in the works by Yuchen.

Hm. Technically, the server can check whether redirect_url is set, and send ER_SERVER_REDIRECT instead of ER_SERVER_SHUTDOWN. I'm not sure it'll be much of an improvement, but it's doable.

That would make sense from the onset to ensure the rollout of the redirect feature is successful and users (and connector authors) can discover what happened.