QMail and SoftUpdates

[Don Lewis]

On 18 May, Julian Elischer wrote:
> 
> 
> On Mon, 17 May 2004, Don Lewis wrote:
> 
>> On 17 May, Nikita Danilov wrote:
>> > Xin LI writes:
>> >  > On Mon, May 17, 2004 at 01:18:15PM -0400, Mikhail Teterin wrote:
>> >  > > The link at
>> >  > > 
>> >  > > 	http://cr.yp.to/qmail/faq/reliability.html#filesystems
>> >  > > 
>> >  > > claims, using SoftUpdates for mailqueue is dangerous. Is that still
>> >  > > true? Thanks!
>> >  > 
>> >  > Yes, it is dangerous. Same is true for any journalling file systems,
>> >  > which essentially does the same thing: delayed write of data/metadata.
>> >  > 
>> >  > Delayed write will make it possible for the Operating System to group
>> >  > several writes together and write them once, or at least, in a better
>> >  > order in order to improve performance. However, for the mail case, once
>> >  > it responds "250", then the remote peer is allowed to remove the message
>> >  > from its queue. If the system crashes, and the data was not written into
>> >  > disk, then your message is lost.
>> > 
>> > Unless mail-server did fsync(2) which is guaranteed to return only after
>> > data reached stable storage. If file-system doesn't provide such
>> > guarantee it's broken, if mail server doesn't call fsync, or
>> > fdatasync---it is. Even without any journalling involved.
>> 
>> Based on the information I found using Google, it appears that qmail
>> relies on link(2) being synchronous to let it know that a queued message
>> is safely on the disk with a known file name before it issues the "250"
>> response. I believe this was true without softupdates, but with
>> softupdates enabled it is definitely not true.
>> 
> 
> An fsync will sync ALL directory entries pointing to the file

I haven't looked at how qmail works, but my suspicion is that it
fsync()s the file and then creates a link (and probably unlinks the old
name) to mark the queue file as valid and is not partially written. I
think this would work with softupdates if the file were fsync()ed again
after the link() call.  I won't comment about why this change is
unlikely to make it into the code.