[Maria-developers] storage engine index usage
Hi Folks, A new question has risen during the development of our storage engine. In short the engine is providing a read-only view to the server filesystem. One can issue (implemented via assisted discovery): CREATE TABLE etc ENGINE=fsview ROOT=‘/etc’; Then it’s possible to tell: SELECT name, fsize FROM etc ORDER BY fsize DESC LIMIT 10 resulting in: +-----------------+--------+ | name | fsize | +-----------------+--------+ | ld.so.cache | 129417 | | mailcap | 44732 | | fb.modes | 24501 | | mime.types | 24066 | | brltty.conf | 22478 | | services | 19558 | | devscripts.conf | 18037 | | ltrace.conf | 14867 | | login.defs | 10551 | | sensors3.conf | 10344 | +-----------------+--------+ So far so good (the listing is not recursive). An additional feature is to access subdirectories of the root (this is working via automated discovery): SELECT name, fsize FROM `etc/init.d` This is not terribly wrong, but we’re thinking about the possiblity of improving it somehow that: SELECT name, fsize FROM etc WHERE path = “init.d”; However we’re in a bit of trouble how we could improve the storage engine to be able to handle this. Of course we would not like to do a "deep enumeration" of the root directory (full table scan) . So somehow we would like to access/use the information from the where clause. Can anybody recommend a way to solve that? -- Andras Szabo Sent with Airmail
well, i'm not a expert , and it's not comment about storage engine plugin, but i will give one example with information schema plugin, there's parameters to know what you could optimize with where clausule, maybe this can help: http://dev.mysql.com/doc/refman/5.7/en/writing-information-schema-plugins.ht... static int simple_fill_table(THD *thd, TABLE_LIST *tables, Item *cond) http://dev.mysql.com/tech-resources/articles/mysql_i_s_plugins_part2.html The TABLE_LIST *tables argument provides the handle to the information schema table that is being filled, and the COND *cond argument represents the WHERE condition of the SQL statement that is currently being handled, allowing the fill_table function to directly filter rows (instead of relying on the query execution engine to do that). As such, these arguments are occupied with the actual delivery of rows of data to the server. 2015-02-06 14:08 GMT-02:00 Andras Szabo <andrei.hu@gmail.com>:
Hi Folks,
A new question has risen during the development of our storage engine.
In short the engine is providing a read-only view to the server filesystem.
One can issue (implemented via assisted discovery):
CREATE TABLE etc ENGINE=fsview ROOT=‘/etc’;
Then it’s possible to tell:
SELECT name, fsize FROM etc ORDER BY fsize DESC LIMIT 10
resulting in:
+-----------------+--------+ | name | fsize | +-----------------+--------+ | ld.so.cache | 129417 | | mailcap | 44732 | | fb.modes | 24501 | | mime.types | 24066 | | brltty.conf | 22478 | | services | 19558 | | devscripts.conf | 18037 | | ltrace.conf | 14867 | | login.defs | 10551 | | sensors3.conf | 10344 | +-----------------+--------+
So far so good (the listing is not recursive).
An additional feature is to access subdirectories of the root (this is working via automated discovery):
SELECT name, fsize FROM `etc/init.d`
This is not terribly wrong, but we’re thinking about the possiblity of improving it somehow that:
SELECT name, fsize FROM etc WHERE path = “init.d”;
However we’re in a bit of trouble how we could improve the storage engine to be able to handle this.
Of course we would not like to do a "deep enumeration" of the root directory (full table scan) . So somehow we would like to access/use the information from the where clause.
Can anybody recommend a way to solve that?
-- Andras Szabo Sent with Airmail
_______________________________________________ Mailing list: https://launchpad.net/~maria-developers Post to : maria-developers@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-developers More help : https://help.launchpad.net/ListHelp
-- Roberto Spadim SPAEmpresarial - Software ERP Eng. Automação e Controle
Hi, Andras! On Feb 06, Andras Szabo wrote:
SELECT name, fsize FROM etc WHERE path = “init.d”;
However we’re in a bit of trouble how we could improve the storage engine to be able to handle this.
Of course we would not like to do a "deep enumeration" of the root directory (full table scan) . So somehow we would like to access/use the information from the where clause.
Can anybody recommend a way to solve that?
Supporting indexes would be a natural way to do it. That's not very difficult. See any existing engine, a simple one, preferrably. Or check my book https://www.packtpub.com/big-data-and-business-intelligence/mysql-51-plugin-... the last chapter is about exactly that. Regards, Sergei
participants (3)
-
Andras Szabo
-
Roberto Spadim
-
Sergei Golubchik