Hello All,
Sometimes, TokuDB gets left in a state where it thinks that parts of a table exist (as observed in TokuDB's file map), but not all of the underlying files (the TokuDB data files and/or the frm file) exist. This probably occurs due to the interaction between the non-transactional DDL that MySQL/MariaDB uses to create or drop tables, and the transactional TokuDB engine underneath. There is some bug there.
So, what can one do to remove broken TokuDB file maps? There is an internal fractal tree called the tokudb.directory that names high level data names to an underlying file that the data is stored in. The tokudb_file_map information schema dumps out the contents of this tree. The key to this tree is the high level data name (called a dname). In TokuDB, this data name consists of the database name, the table name, and the index name. There is also a tree for the table (called status) that contains table wide metadata.
I added the tokudb_delete_dnames_from_directory tool to the
https://github.com/prohaska7/mariadb-server/tree/fix-tokudb-filemap branch. This standalone tool can be used to delete dnames from the tokudb.directory. Standalone means that MariaDB must have been previously shutdown cleanly (which puts the tokudb recovery logs in a nice clean state so that the tool does not have to run crash recovery).
The input to the tool is the MariaDB data directory with nice and clean tokudb recovery logs along with a sequence of dname's. The tool transactionally deletes rows from the tokudb.directory fractal tree, which updates the recovery logs.
Building the tool requires a MariaDB build environment a simple compile sequence something like this:
gcc -g -o tokudb_delete_dnames_from_directory tokudb_delete_dnames_from_directory.cc -I$TFT/buildheader -L$TFT/src -ltokufractaltree
Running the tool looks something like this:
Get the dnames of interest from the tokudb_file_map. In this case, they are ./test/s-main and ./test/s-status.
shutdown MariaDB
./tokudb_delete_dnames_from_directory ~/projects/mariadb-server-install/data ./test/s-main ./test/s-status