Disk sync at shutdown and fusefs filesystems

[Csaba Henk]

On 2007-12-12, Alejandro Pulver <alepulver at FreeBSD.org> wrote:
> On Wed, 12 Dec 2007 03:00:07 +0100
> Csaba Henk <csaba at beastie.creo.hu> wrote:
>
>> On 2007-12-11, Alejandro Pulver <alepulver at FreeBSD.org> wrote:
>> > The problem with NTFS-3G (and all other FUSE based drivers maybe) is
>> > that it doesn't flush the cache data to the disk at shutdown, but it
>> > does when unmounted (and I guess this doesn't happen automatically). I
>> > noticed this when files I write before manually unmounting persist, and
>> > otherwise sometimes they don't.
>>
>> I just happen to discuss this issue with Szaka (ntfs-3g developer) and
>> Miklos Szeredi (FUSE developer). At least, we're discussing something
>> which might have a relevance here.
>>
>> They have already discovered issues with system shutdown on Linux, and
>> Miklos has implemented a solution for this dubbed as "synchronous
>> umount". According to this, the protocol is enhanced with a new message
>> called DESTROY. Upon unmounting the fs, the kernel sends a DESTROY to
>> the daemon and waits for answer. That is, unmount(2) won't complete
>> until the fs says to the kernel "OK, I'm done".
>>
>> This was introduced in the following commit (as seen in my HG mirror):
>>
>>   http://mercurial.creo.hu/repos/fuse-hg/?rev/a5df6fb4a0e6
>>
>> and it's already included in the current sysutils/fusefs-libs port.
>>
>> And it wouldn't be hard to add kernel side support for FreeBSD. There
>> are some questions though:
>>
>>  - Do you think it could be actually useful for solving the shutdown
>>    issue on FreeBSD?
>>
>
> Hmm, I don't know much of this, but isn't the Linux problem related to
> flushing its own block device cache? In FreeBSD it doesn't exist (i.e.
> ublio is only user-space), so I wonder if just unmounting before
> shutdown solves the issue. I mean, does the kernel still keep
> information after a FUSE filesystem is unmounted?
>
> Please correct me if I'm wrong.
>
> At least the currently discussed trick only works because it waits a few
> seconds after unmounting to let it flush the cache (but I think it's a
> common fact that filesystems get registered/unregistered with a small
> delay, and may not be related to that).

The point in synch umount is that you don't need to wait for an ad hoc
amount of time in order to have the various caches flushed / media
sync'd -- it enables the filesystem daemon itself to notify the umount
procedure that it's done and the world can go on.

The exact nature of caches (userspace / in-kernel) doesn't really make a
difference from this POV. (Implementing the appropriate synchronization
mechanisms is up to the fs daemon.)

>>  - Some "got hung in unmount" issues are to be sorted out (these
>>    appeared on Linux, and they might or might not appear on FreeBSD).
>>
>
> Never seen this, but also never unmounted at shutdown before. I have a
> patch for it (see thread). Then we could easily see if it get stalled
> at shutdown (or when manually stopping the rc.d script).

Of course you've never seen this! -- these have appeared as consequences
of the synchronous umount (ie., the DESTROY message) which is not yet
implemented on FreeBSD.

The actual question is whether it is worth to implement it. For me it
seems "yes", but I don't know the ins and outs of the FreeBSD
init/shutdown system, that's why I'd like to hear the opinion of people
like you about this before I go and code it.

Whether hangs occur or not if fuse4bsd does sync umount is not that
important. I mean, first I would code a basic implementation of DESTROY
(that's pretty simple to do!) and we'd see how well that works and if we
see problems I try to tune the implementation. That's just business as
usual.

The next issue is more important...

>>  - Security issue: with synch unmount, any user who can mount (w/ synch
>>    unmount), is capable of making the unmount stuck (which is easy to
>>    fix when the system is up -- just kill the fs daemon -- but can
>>    make the shutdown process hopelessly stuck). So we'd have to
>>    decide who/when shall be able to do mounts for which the unmount is
>>    synchronous. (The current criteria for this on Linux -- ie.,
>>    is the fuseblk fs variant being used? -- is N/A to FreeBSD for
>>    reasons which are OT here. However, Miklos decided to
>>    change this so that sych unmount will be tied to the "allow_other"
>>    option, which is tied to root privileges, and does make sense
>>    on FreeBSD, too. I'd be happy to hear more suitable criteria.
>>
>
> This would depend on the previous point.

It's more important to specify a suitable security policy -- who/when
should be capable of mounting an fs in a way so that the umount will
be synchronous? (Short term for this: "mounting with synch umount").

I think it's quite a "standalone" question, it depends on nothing else.
If a FUSE fs is mounted with synch umount its daemon can block the umount
command/syscall, and therefore it can probably block the whole shutdown
sequence (killing the process or umounting with -f might help, but these
probably won't happen during the normal shutdown course).

A possible compromise is letting only the superuser to mount with synch
umount. That would mean only superuser mounted ntfs fs-es will be
cleanly unmounted during shutdown (I mean, without resorting to hacks in
the shutdown scripts). ("Tying synch umount to 'allow_other'", as
mentioned above, is practically the same choice.) 

This of course can be refined, eg. make it configurable with a sysctl,
etc.

Either case, it's a design issue and not just an implementation detail
so it would be clever to try to make up our mind about this.

Csaba