Hi Marko,
Le 2 févr. 2018 à 12:27, Marko Mäkelä <marko.makela@mariadb.com> a écrit :
I would like to redesign the InnoDB redo log format for better performance in MariaDB 10.4. Part of this would involve minimizing write amplification and optimizing for journaled file systems.
Jun Su from Microsoft suggested to me that regular writes (as opposed to appends) could cause some write amplification inside journaled file systems.
InnoDB traditionally pre-allocates both data and log files. Maybe it was a good idea in 1994 when the code was initially conceived. But we have had journaled or copy-on-write file systems and also SSDs for quite some time now.
I wrote two test programs that write a 2GiB file in 2KiB blocks, either pre-allocating the file upfront, or appending to the file. On the two SSDs that I tested (with ext4fs), appending was always faster. The programs are attached to https://jira.mariadb.org/browse/MDEV-14425
I would appreciate it if someone can provide a counterexample where writing to a preallocated file would be faster than appending, on a modern file system. Also, I would like to see how HDDs would perform.
I've just tested your program on an XFS filesystem, the preallocate performance is almost the same than append: time ./append foo1&time ./append foo2&time ./append foo3&time ./append foo4 <12:57:26 [1] 38171 [2] 38172 [3] 38174 ./append foo3 0.03s user 1.86s system 30% cpu 6.297 total ./append foo1 0.03s user 1.81s system 28% cpu 6.554 total ./append foo2 0.02s user 1.99s system 30% cpu 6.610 total ./append foo4 0.02s user 1.84s system 27% cpu 6.711 total time ./preallocate foo1&time ./preallocate foo2&time ./preallocate foo3&time ./preallocate foo4 <12:59:09 [1] 39461 [2] 39462 [3] 39464 ./preallocate foo4 0.02s user 1.98s system 32% cpu 6.119 total ./preallocate foo1 0.03s user 2.01s system 33% cpu 6.120 total ./preallocate foo2 0.02s user 2.16s system 34% cpu 6.396 total <12:59:16 ./preallocate foo3 0.04s user 1.88s system 28% cpu 6.687 total <12:59:17 Disk are 4 Intel SSD 520 (480GB) in Raid5 configuration on controller DELL PERC H710P (tested several times) I've also tested it with 4GB files, and on a slower system (still Intel SSD disks in Raid5 with XFS), append is also performing quite the same than preallocate. HTH, -- Jocelyn Fournier Founder M : +33 6 51 21 54 10 https://www.softizy.com Softizy - At your side to Optimize your PHP / MySQL applications