Hello, A MariaDB query optimizer member here. I think it would be nice to get VIDEX for MariaDB. As far I'm informed, the basics of interaction between the query optimizer and storage engine are the same between MariaDB and MySQL: - ha_engine::info() call - ha_engine::records_in_range() call - extended keys feature Execution part (index/table read methods) is also very similar although there are some cosmetic differences. There are bigger differences in how partitioning is handled, data dictionary and column histograms. I think it's feasible to work around those. To sum up, at first glance adoption to MariaDB seems doable. I'd like to study VINDEX further and see where/how we MariaDB devs could lend a hand. Best Regards, -- Sergei Petrunia Query Optimizer team MariaDB Corporation On Wed, Apr 16, 2025 at 09:41:16PM -0500, 康荣 via discuss wrote:
Dear MariaDB Community,
I hope this message finds you well. We'd like to introduce VIDEX (Virtual Index for MySQL) and humbly seek your guidance and assistance in adapting it for MariaDB compatibility.
We've developed a virtual index storage engine (plugin) called VIDEX. It can simulate the impact of indexes on query plan costs without requiring real data. VIDEX forwards the statistical information, ndv (records_per_key), and cardinality information (records_in_range) needed for making query plans via a RESTful approach, with responses provided by an "algorithm service." This disaggregated design is particularly well-suited for integrating various AI-boosted cardinality and ndv algorithms. VIDEX has been successfully deployed at ByteDance, serving thousands of MySQL instances daily and over millions of SQL queries for index optimization tasks.
We implemented VIDEX based on MySQL 8.0 and now, thanks to Federico Razzoli for his suggestion (LinkedIn post 1, LinkedIn post 2, , Github Issue), we'd like to adapt it to MariaDB. However, we've discovered significant differences between MySQL 8.0 and the latest MariaDB codebase. Therefore, we're seeking help from the MariaDB community, and would greatly appreciate collaboration with experienced developers to modify the code. We're very willing to cooperate in this effort. In the future, we also hope to have the opportunity to add VIDEX to the official MariaDB plugin list.
About VIDEX VIDEX GitHub repository: https://github.com/bytedance/videx/
Virtual index technology is already available in PostgreSQL, Oracle, and IBM DB2, and would provide MariaDB users with valuable capabilities for query optimization and database research. VIDEX offers key features that would benefit MariaDB users:
• What-if analysis for index strategies without production impact • Support for integration of custom or AI-based estimation algorithms • Accurate simulation of query plans, verified with complex benchmarks like TPC-H and JOB
If you're interested in collaborating or have guidance to offer on adapting our code to MariaDB, we'd greatly appreciate hearing from you.
Thank you for your consideration.
Best regards, Rong ByteBrain Team, Bytedance
_______________________________________________ discuss mailing list -- discuss@lists.mariadb.org To unsubscribe send an email to discuss-leave@lists.mariadb.org