Hi Sergei, (sorry for cross-posting to MariaDB.dev and MariaDB.discuss: I think this interest both audiences.) Sergei Golubchik commented on MDEV-12499:
-----------------------------------------
This only happens if you created an ARCHIVE table and never used it before the upgrade. No INSERT, no SELECT, not even {{SELECT * FROM INFORMATION_SCHEMA.TABLES}} — nothing.
ARCHIVE engine supports table discovery, that is, the engine is always the authoritative source of table metadata, while {{.frm}} files are merely a metadata cache. When you do {{CREATE TABLE}} the server does not know whether the engine wants the table metadata to be cached. This happens when the table is opened for the first time.
If there is no {{.frm}} file and the ARCHIVE engine is not loaded, the server has no way of knowing that the ARCHIVE table exists. That's why it's not shown anywhere.
If you do {{SELECT * FROM test.a1}} before the upgrade, the engine will tell the server to create the {{.frm}} file and on the upgrade the plugin will be loaded. In a practical use case, it's enough to have at least one ARCHIVE table that was accessed at least once after it was created, and the ARCHIVE plugin will be loaded on upgrade.
Thanks for the detailed info above. In this, you write ".frm files are merely a metadata cache", is there any sort of documentation about when .frm files are useful, useless, and might get out of sync ? This probably deserves a page in the KB (and Google does not know much on "mariadb frm file site:mariadb.com"). However, your comment above raises the following question: if "ARCHIVE engine supports table discovery, that is, the engine is always the authoritative source of table metadata", where is this information stored ? I guess that unless the plugin is loaded, this information is not accessible. This causes a chicken and egg problem which MDEV-12499 was trying to solve: how can mysql_upgrade knows if it should add BLACKHOLE and ARCHIVE to the list of plugin to load (see MDEV-11942). Finally, you marked MDEV-12499 as "Won't Fix", do you have another solution for ARCHIVE and MDEV-11942 ? Many thanks, JFG