Hi, Rong, Could you please explain why did you chose this particular architecture? What is the use case? You refer to PostgreSQL, Oracle, and IBM DB2, but neither of them involves taking via REST API to a separate server and neither mentions "AI-boosted cardinality and ndv algorithms". So, what is "AI-boosted cardinality and ndv algorithms" and why a separate server? Regards, Sergei Chief Architect, MariaDB Server and security@mariadb.org On Apr 16, Rong Kang via discuss wrote:
We've developed a virtual index storage engine (plugin) called VIDEX. It can simulate the impact of indexes on query plan costs without requiring real data. VIDEX forwards the statistical information, ndv (records_per_key), and cardinality information (records_in_range) needed for making query plans via a RESTful approach, with responses provided by an "algorithm service." This disaggregated design is particularly well-suited for integrating various AI-boosted cardinality and ndv algorithms. VIDEX has been successfully deployed at ByteDance, serving thousands of MySQL instances daily and over millions of SQL queries for index optimization tasks.
*About VIDEX* VIDEX GitHub repository: https://github.com/bytedance/videx/
Virtual index technology is already available in PostgreSQL, Oracle, and IBM DB2, and would provide MariaDB users with valuable capabilities for query optimization and database research. VIDEX offers key features that would benefit MariaDB users:
- What-if analysis for index strategies without production impact - Support for integration of custom or AI-based estimation algorithms - Accurate simulation of query plans, verified with complex benchmarks like TPC-H and JOB
If you're interested in collaborating or have guidance to offer on adapting our code to MariaDB, we'd greatly appreciate hearing from you.
Thank you for your consideration.