Hi, Jean-Francois! On Apr 17, Jean-Francois B. Gagne wrote:

Hi Sergei,

(sorry for cross-posting to MariaDB.dev and MariaDB.discuss: I think this interest both audiences.)

Sergei Golubchik commented on MDEV-12499:

...

-----------------------------------------

This only happens if you created an ARCHIVE table and never used it before the upgrade. No INSERT, no SELECT, not even {{SELECT * FROM INFORMATION_SCHEMA.TABLES}} — nothing.

ARCHIVE engine supports table discovery, that is, the engine is always the authoritative source of table metadata, while {{.frm}} files are merely a metadata cache. When you do {{CREATE TABLE}} the server does not know whether the engine wants the table metadata to be cached. This happens when the table is opened for the first time.

If there is no {{.frm}} file and the ARCHIVE engine is not loaded, the server has no way of knowing that the ARCHIVE table exists. That's why it's not shown anywhere.

If you do {{SELECT * FROM test.a1}} before the upgrade, the engine will tell the server to create the {{.frm}} file and on the upgrade the plugin will be loaded. In a practical use case, it's enough to have at least one ARCHIVE table that was accessed at least once after it was created, and the ARCHIVE plugin will be loaded on upgrade.

Thanks for the detailed info above. In this, you write ".frm files are merely a metadata cache", is there any sort of documentation about when .frm files are useful, useless, and might get out of sync ? This probably deserves a page in the KB (and Google does not know much on "mariadb frm file site:mariadb.com").

Good point, thanks. I've added a paragraph about it to the https://mariadb.com/kb/en/mariadb/table-discovery/ page.

However, your comment above raises the following question: if "ARCHIVE engine supports table discovery, that is, the engine is always the authoritative source of table metadata", where is this information stored? I guess that unless the plugin is loaded, this information is not accessible. This causes a chicken and egg problem which MDEV-12499 was trying to solve: how can mysql_upgrade knows if it should add BLACKHOLE and ARCHIVE to the list of plugin to load (see MDEV-11942).

Correct. The information is stored in the engine - for Archive tables, in the .ARZ file - and it is not accessible when the plugin is not loaded.

Finally, you marked MDEV-12499 as "Won't Fix", do you have another solution for ARCHIVE and MDEV-11942 ?

I think it is not an issue in this particular case. This problem will only show up when one created Archive tables and never ever accessed a single one of them before the upgrade. I don't think it's a realistic scenario. Note, that in my tests I simply started "mysql" client and that was enough for .frm files to be created (because mysql client queries table metadata on startup). And it can hardly be called a compatibility issue :) A "broken compatibility" refers to something that worked before the upgrade, but stopped working after. If one never accessed Archive tables before the upgrade, one can still complain that they are not accessible after the upgrade, but one cannot claim it to be a compatibility problem, I think. Regards, Sergei Chief Architect MariaDB and security@mariadb.org