Can you be more specific about your performance issues? We want to learn what can be made better in MyRocks and RocksDB.

Level compaction is enabled by default. To confirm what is being used look for this in $datadir/.rocksdb/LOG
2017/06/01-09:18:41.867035 7fba2196ea00 Options.compaction_style: kCompactionStyleLevel

The most important options to change are rocksdb_block_cache_size and rocksdb_max_background_compactions. There are more that can be set, but it would be great if changing these were sufficient to resolve your problems. When compression is enabled I set rocksdb_block_cache_size to 1/3 of RAM. I usually set rocksdb_max_background_compactions to a value between 1/4 and 1/2 of the CPUs on the server.

What kind of storage do you have -- disk array, disk, fast SSD, less-fast SSD (TLC NAND)?

On Thu, Jun 1, 2017 at 2:54 AM, Julien Muchembled <jm@nexedi.com> wrote:

Hello,

We have started to evaluate MyRocks 3 months ago, our DBs were initially empty and now they have reached 500 GB on-disk. We experience severe performance issues, and we have the feelings it could be better.

https://github.com/facebook/rocksdb/wiki/RocksDB-Tuning-Guide is interesting, especially the "Universal Compaction" section:
« Universal compaction has a size limitation. Please be careful when your DB (or column family) size is over 100GB. »

I don't think that write amplification is a bottleneck for our application, so my obvious first question is:
What compaction style is used in MariaDB and how to make sure that Level Style Compaction is used ?

Also, we haven't set any 'rocksdb_*' option. The default configuration may not be appropriate in our case. For example, rocksdb_block_cache_size (default: 512MB) seems to be one of the most important options, similarly to innodb_buffer_pool_size and tokudb_cache_size, although I have never understood why (see my unanswered question at https://lists.launchpad.net/maria-discuss/msg02683.html ). Although the DB is much bigger than RAM (128 GB), RAM should be big enough to contain the active working set.

Julien

_______________________________________________
Mailing list: https://launchpad.net/~maria-discuss
Post to : maria-discuss@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-discuss
More help : https://help.launchpad.net/ListHelp

Mark Callaghan
mdcallag@gmail.com