On 14/08/2018 22:17, Federico Razzoli wrote:
Hi,
Just a small correction: ZoL does not support O_DIRECT, but FreeBSD ZFS does. Probably other distributions also do.
Regards, Federico Razzoli
Hi, I just tried on FreeBSD 11.x a small C program with O_DIRECT support [1] and it really seems O_DIRECT is ignored: writes go into ARC and are served from it when data is read. ZFS compression for the dataset it off. This do not surprise me: O_DIRECT implies zero-memory-copy and/or DMA from main memory to the disk themselves. While with standard filesystem this should be possible, with CoW+checksum (and anything which transforms data when they flow, ie: compression) this become very difficult. Back to main point... anyone with some insights on doublewrite and ZFS? # Before running the test program: ARC Size: 0.09% 1.14 MiB Target Size: (Adaptive) 100.00% 1.20 GiB Min Size (Hard Limit): 12.50% 153.30 MiB Max Size (High Water): 8:1 1.20 GiB # After running it: ARC Size: 48.65% 596.61 MiB Target Size: (Adaptive) 100.00% 1.20 GiB Min Size (Hard Limit): 12.50% 153.30 MiB Max Size (High Water): 8:1 1.20 GiB # Reading the just-written file shows data are server by ARC (ie: too fast for coming from the disk) root@freebsd:~ # dd if=/tank/test.img of=/dev/null bs=1M 512+0 records in 512+0 records out 536870912 bytes transferred in 0.188852 secs (2842809718 bytes/sec) [1] Test program: root@freebsd:~ # cat test.c #define _GNU_SOURCE #include <string.h> #include <stdlib.h> #include <fcntl.h> #include <stdio.h> #include <unistd.h> #define BLOCKSIZE 128*1024 int main() { void *buffer; int i = 0; int w = 0; buffer = malloc(BLOCKSIZE); buffer = memset(buffer, 48, BLOCKSIZE); int f = open("/tank/test.img", O_CREAT|O_TRUNC|O_WRONLY|O_DIRECT); for (i=0; i<512*8; i++) { w = write(f, buffer, BLOCKSIZE); } close(f); free(buffer); return 0; } -- Danti Gionatan Supporto Tecnico Assyoma S.r.l. - www.assyoma.it email: g.danti@assyoma.it - info@assyoma.it GPG public key ID: FF5F32A8