Disable read/write caching to disk?
Eric Anderson
anderson at centtech.com
Thu May 26 19:50:24 PDT 2005
Scott Long wrote:
> Eric Anderson wrote:
>
>> Scott Long wrote:
>>
>>> Eric Anderson wrote:
>>>
>>>> Bjoern Koenig wrote:
>>>>
>>>>> Bjoern Koenig wrote:
>>>>>
>>>>>> Eric Anderson wrote:
>>>>>>
>>>>>>> Is it possible to disable all read and write caching to a disk?
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> You can disable write the cache by adding the line
>>>>>>
>>>>>> hw.ata.wc="0"
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> I assumed that you use ATA. If you use SCSI devices then read at
>>>>> least the manpages da(4) and camcontrol(8).
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Thanks.. I've just read (quickly) both man pages. It seems as
>>>> though you are suggesting disabling the physical disk caching, which
>>>> should not make a difference in my case. The disk would report
>>>> whatever it needs to report to either host, and those should be in
>>>> sync.
>>>>
>>>> When I mount the filesystem on host B ro, it shows me the filesystem
>>>> as of the time that I mounted it ro. Any subsequent changes on host
>>>> A (which has it mounted rw) are not seem on host B unless I unmount
>>>> and mount again on host B. This seems like a FreeBSD feature and
>>>> not a general scsi feature.
>>>>
>>>> Eric
>>>>
>>>>
>>>>
>>>>
>>>
>>> You simply cannot disable OS caching in FreeBSD. It's a fundamental
>>> part of the block I/O and VM layers. There are filesystems like GFS
>>> that deal with the issue of directly connecting more than one computer
>>> to a disk or set of disks, and there are distributed filesystems like
>>> AFS and Coda that deal with making the storage on multiple computers
>>> appear as a single network filesystem. Unfortunately, no port of GFS
>>> has been done yet, and I estimate that such a port would take 4-6
>>> months.
>>
>>
>>
>> Thanks Scott. I know of GFS, and would *love* to see it ported to
>> FreeBSD. I wonder if there is a group of developers that would be
>> willing to do that?
>
>
> I'd love to do it sometime. Some diligence would have to be done on the
> license and possible patents, though. I'm not sure what RedHat's stance
> is on sharing in this regard.
If you are serious about working on it, I could look into the RedHat
side of things. This is something I really think is important for
FreeBSD (in the future). If GFS isn't right, maybe another clustering
shared file system, or a Global UFS (I don't even know if that's possible).
>> I understand what I am doing is 'illegal', but I'm wondering why the
>> ro mount only sees the changes from the time of ro mount. Mounting rw
>> also shows the same thing.
>>
>> Do you think it's a caching issue, or something with UFS that makes it
>> work this way?
>>
>
> Even on a read-only mount, reads are cached. Cached blocks are only
> evicted when there is memory pressure or when the OS specifically
> invalidates them, and it's rather unpredictable when this will happen.
I see. So most likely the lack of updates being visible on host B is due
to read caches.
>> I'm in no way advocating doing this, nor am I saying 'it should work'
>> - I'm trying to learn more, understand it, and maybe use it as a
>> failover mechanism.
>
>
> The only thing that is moderately workable is to have all client
> machines mount the FS read-only, and have none of them do any writing
> to it.
>
>>
>> Does anyone know the real dangers of forcing an unclean UFS filesystem
>> mounted rw and skipping the fsck?
>
>
> Assuming that the previous shutdown allowed all cached metadata in the
> disk to get to the platters in the proper order (which is not terribly
> true with ATA), the main inconsistency that you'll have is that deleted
> files might still have allocated inodes and thus will consume space on
> the filesystem even though they aren't accessible. This is the premise
> that background fsck operates on, btw.
>
> However, there is a real possibility for there being other
> inconsistencies that are not safe to run with.
So it sounds dangerous, but not disastrous.. Sounds like soft-updates
would help this alot, so I'll turn them back on for this filesystem (I
typically do use it).
At a minimum, it would be awesome to even have a way to do one host rw
and several doing ro. Think of the case of a web server farm, where
it's nearly all reads.
Thanks for the details and information!
Eric
--
------------------------------------------------------------------------
Eric Anderson Sr. Systems Administrator Centaur Technology
A lost ounce of gold may be found, a lost moment of time never.
------------------------------------------------------------------------
More information about the freebsd-current
mailing list