How to disable hard disk write cache?

[Alexander Motin]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

On 21.09.2014 12:15, Xin Li wrote:
> On 9/17/14 11:04 AM, Xu Zhe wrote:
>> I have googled about the "tagged command" but only found
>> something related to TCQ, which mainly talks about the queueing
>> but not anything related to cache sync. Any more hints?
> 
> Hrm I don't have much thing off hand, but these are defined by the 
> standards.  Alexander knows much more than I do in this area.
> 
>> I saw some pages that mentioned about SATA FUA command support
>> on Linux (Which I guess is what I am looking for):
> 
>> https://www.kernel.org/doc/Documentation/block/writeback_cache_control.txt
>
>>  However, I have not found useful information related to
>> Freebsd. :(
> 
> Probably g_bio(9) but the documentation do not have much detail on 
> driver implementation, and callers of the API expects the driver to
> do all the right things.  This is if you want to implement a file
> system, where you would go.
> 
> But back to your question, if you want to know e.g., "how to
> disable write cache on an AHCI connected drive", you would go to
> the individual driver's manual, e.g. ada(4).

I see there was several different topics touched in this threads, so
let me start from the beginning. There are several methods to control
disk caches, and respectively several approaches to maintaining the
data consistency:

 - Caches can be controlled globally
SCSI disks allow to control write cache via the Caching mode page. You
may do it with `camcontrol mode` command. ATA disks has specific
commands for that, and easiest way to do it is using sysctls/tunables
documented in ada(4) manual page. Disabling caches globally heavily
affects performance, and in most cases is overkill. It means that
_every_ request will go to the media before the operation complete.
For disks without command queuing (like legacy ATA) that usually meant
_one_ I/O request per platter revolution. Do you want disk doing 120
IOPS peak? If you write huge file in 128K chunks, you will get limited
by 120/s * 128K = 15MB/s! Command queuing (NCQ for SATA) significantly
improved this situation since OS can now send more operations down to
the disk to give it more flexibility, in significant part compensating
disabled cache. But number of simultaneously active tags in disk, HBA
or application can be limited, creating delays.

 - Caches can be controlled per-command
SCSI disks may support FUA and DPO flags, that allow to do above cache
control on per-command basis. SATA disks got FUA flag just recently.
This approach has all properties of above, just can be controller per
data type or application. Those flags are equivalent of IO_SYNC and
IO_DIRECT flags on VFS layer of FreeBSD. Though FreeBSD block layer
never had their equivalents. I've heard that Windows NTFS used this
technique to keep metadata consistency up to some point, but moved
away from it.

 - Caches can be flushed to media with SYNCHRONIZE CACHE commands
Both SATA and SCSI disks provide ways to flush write caches to the
media on request. It allows disks to do many writes in any order they
prefer within one transaction group, but then reliably push everything
to the media before when the group being committed. FreeBSD supports
that on both VFS (fflush/VOP_FSYNC) and block layers (BIO_FLUSH). ZFS
on FreeBSD relies on it to keep transaction groups atomicity and so
metadata correctness. I've heard, this is what Windows NTFS is doing
now too.

I hope that answered most of questions.

- -- 
Alexander Motin
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQF8BAEBCgBmBQJUHqAeXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w
ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRFOThDRjNDNEU2OUNDM0NEMEU1NzlENTU4
MzE4QzM5NTVCQUIyMjdGAAoJEIMYw5VbqyJ/E6UH+QF5AU3KNKjBLyNsgGTOn+UV
SZAj3V3JLUNM4Z+8Z+4rxY/26/nbfNPoEJD4SaOt74oEUA574XjVLkHmqZ5JKygV
dzOE2m8t0gyDIPurGx2CfRyG7UdJKbVsJ7Ebxafd8lRbwHGoObySUmew8t8WSIHA
zBsI3ZiRYzBX7NnejPlJ8UPh498PBl78U+Ak08q/0scdPWsCneCjqHHn0Rx2MEFp
7sllphFh+EBXJXqetFRJfuWmm7yfIrzjO5UJ1mTjq5dCSeXrsIxBMBTSMEG34Yv6
9PqPWYPdVWQKgluKcaTKgrzPVTBgAqefx4gHRm/460a+7qohloCelMsgAsttYuA=
=De3m
-----END PGP SIGNATURE-----