O_DIRECT|O_SYNC semantics?

[Jeremy Chadwick]

On Wed, Jul 03, 2013 at 09:05:40PM +0200, Zoltan Arnold NAGY wrote:
> Could someone have a look here:
> http://serverfault.com/questions/520141/please-explain-my-fio-results-is-o-synco-direct-misbehaving-on-linux
> 
> Basically, I'm seeing wastly different results on Linux and on FreeBSD 9.1.
> Either FreeBSD's not honoring O_SYNC properly, or Linux does something
> wicked.
> 
> I've been at it for a few days, without any real progress.
> 
> I do realize that since I'm operating at a block device level not with any
> filesystem it's strange to ask on -fs, but I came to this results while
> experimenting with the SSD as a ZIL device, and was surprised at the low
> numbers.

Block devices on FreeBSD are ***always*** O_DIRECT.  There is no
"caching mechanism" with such.

Block devices on Linux result in caching, unless O_DIRECT is used.

Because you're asking about some underlying kernel behaviour, I
might recommend this be discussed on the -hackers list, where many of
the low-level folks hang out.

I can assure you that you're going to be asked to provide "dmesg" (on
FreeBSD) from the system you're testing with, so you'd best have that
ready.

-- 
| Jeremy Chadwick                                   jdc at koitsu.org |
| UNIX Systems Administrator                http://jdc.koitsu.org/ |
| Making life hard for others since 1977.             PGP 4BD6C0CB |