The big O_ DIRECT

The Linux kernel provides a page cache, an intermediary resting place for data travelling between applications and block devices, which is used for both reading and writing. The page cache improves read speeds – it enables the kernel to pre-fetch blocks and also makes repeated access to the same data much quicker. For writing, the page cache means that applications don’t have to wait for their data to be written to the block device – the kernel flushes writes to the block device when it’s convenient.

It’s possible to request that the kernel avoids using its page cache. Opening a file using an O_DIRECT option means applications can bypass the page cache so their reads and writes happen directly between the block device and the application’s own internal buffers.

In general, this degrades performance and applications should not do it. But some applications, most notably databases, implement their own caching mechanisms and therefore need to interact directly with block devices.

Note in particular that direct doesn’t imply faster and often the reverse is true. Furthermore, some filesystems, most notably ZFS, do not support O_DIRECT . Linus Torvads even comments (see man 2 open) that: “The thing that has always disturbed me about O_DIRECT is that the whole interface is just stupid, and was probably designed by a deranged monkey on some serious mindcontrolling substances.” 1 It’s a good idea to flush the page cache between tests; use echo 3 >

/proc/sys/vm/drop_caches to do so.