touch(1) not working on directories in an msdosfs(5) envirement
Bruce Evans
brde at optusnet.com.au
Sat Aug 20 08:59:08 UTC 2011
On Fri, 19 Aug 2011, Rick Macklem wrote:
> Alexander Best wrote:
>> can somebody confirm this issue? is it already known?
>>
>> otaku% ll|grep HELL
>> drwxr-xr-x 1 arundel arundel 16384 19 Aug 19:57 HELLO
>> -rw-r--r-- 1 arundel arundel 0 19 Aug 20:13 HELLO2
>> otaku% touch HELLO*
>> otaku% ll|grep HELL
>> drwxr-xr-x 1 arundel arundel 16384 19 Aug 19:57 HELLO
>> -rw-r--r-- 1 arundel arundel 0 19 Aug 20:55 HELLO2
This is fixed (hacked around to keep the diffs small) in my version:
% Index: msdosfs_vnops.c
% ===================================================================
% RCS file: /home/ncvs/src/sys/fs/msdosfs/msdosfs_vnops.c,v
% retrieving revision 1.147
% diff -u -2 -r1.147 msdosfs_vnops.c
% --- msdosfs_vnops.c 4 Feb 2004 21:52:53 -0000 1.147
% +++ msdosfs_vnops.c 12 Nov 2007 21:47:48 -0000
% @@ -457,5 +457,7 @@
% (error = VOP_ACCESS(ap->a_vp, VWRITE, cred, ap->a_td))))
% return (error);
% +#if 0
% if (vp->v_type != VDIR) {
% +#endif
% if ((pmp->pm_flags & MSDOSFSMNT_NOWIN95) == 0 &&
% vap->va_atime.tv_sec != VNOVAL) {
The main part of the fix is to just remove the special case for directories
which just breaks utimes() on directories. Even DOS and Windows never
had this brokenness. What DOS and Windows do specially for directories
is not update their modification time when their contents is changed.
FreeBSD is compatible with this, and the above special case is apparently
the result of trying too hard to be compatible with DOS and Windows.
% @@ -463,4 +465,7 @@
% unix2dostime(&vap->va_atime, &dep->de_ADate,
% NULL, NULL);
% + if (vp->v_type != VDIR)
% + dep->de_Attributes |= ATTR_ARCHIVE;
% + dep->de_flag |= DE_MODIFIED;
% }
% if (vap->va_mtime.tv_sec != VNOVAL) {
Now that setting times on directories is unbroken, we have to be more
careful with the archive bit. In DOS and Windows, the archive bit is
meaningless for FAT* directories and is not set by any syscall that I
know of (unlike for ffs IIRC). The above avoids setting it for directories.
Not setting DE_MODIFIED is an unrelated micro-optimization (try harder not
to set it when we didn't change anything).
% @@ -468,8 +473,11 @@
% unix2dostime(&vap->va_mtime, &dep->de_MDate,
% &dep->de_MTime, NULL);
% + if (vp->v_type != VDIR)
% + dep->de_Attributes |= ATTR_ARCHIVE;
% + dep->de_flag |= DE_MODIFIED;
Similarly for the mtime.
% }
% - dep->de_Attributes |= ATTR_ARCHIVE;
% - dep->de_flag |= DE_MODIFIED;
This was moved early.
% +#if 0
% }
% +#endif
Finish hacking way the special case for directories.
% }
% /*
% @@ -494,5 +502,5 @@
% }
% }
% - return (deupdat(dep, 1));
% + return (deupdat(dep, 0));
Remove an unrelated pessimization (a synchronous update where even an
asynchronous update is more than what is needed). Even ffs in Net/2
didn't have the full pessimization here -- it pessimized SETATTR on
times but not on the more important ownership and permission attributes.
ffs still had the pessimization for times in 4.4BSD-Lite2, but FreeBSD
fixed it in 1998 (ufs_vnops.c 1.79; the fix was buried in a mega-commit
with a content-free log message :-(), and I fixed it in my version of
ffs long before that. msdosfs_setattr() is a little different from
ufs_setattr(); in particular, it does the (previous synchronous) update
for all successful calls while ufs_setattr() only ever did it for times,
so the pessimization has a wider scope in msdosfs.
% }
The above is only the least serious of the bugs in msdosfs_setattr() :-(.
With the above fix, plain touch works as well as possible -- it cannot
work perfectly since setting of atimes is not always supported. But
touch -r and more importantly, cp -p only work as well as possible for
root, since they use utimes() without the null timeptr arg that allows
plain touch to work. A non-null timeptr arg ends up normally requiring
root permissions for msdosfs where it normally doesn't require extra
permissions for ffs, because ownership requirements for the non-null case
cannot be satisfied by file systems that don't really support ownerships.
We fudge the ownerships and use weak checks on them in most places, but
for utimes() we use strict checks that almost always fail: from my old
version:
% if (vap->va_flags != VNOVAL) {
% if (vp->v_mount->mnt_flag & MNT_RDONLY)
% return (EROFS);
% if (cred->cr_uid != pmp->pm_uid &&
% (error = suser_cred(cred, PRISON_ROOT)))
% return (error);
The implementation of this check has changed significantly in -current,
but its semantics and result havven't. The file must be owned by
someone (pmp->pm_uid), and no one else except root has permission for
utimes() with a non-null timeptr. This works right in ffs because the
file can normally be owned by its rightful owner, but in msdosfs the
owner is faked and there can be only one owner for the whole file
systems. I use owner root and group msdosfs for all msdosfs file
systems. The group permissions allow non-root users in group msdosfs
to do almost everything except this unimportant utimes() operation
to almost all msdosfs files.
% /*
% * We are very inconsistent about handling unsupported
% * attributes. We ignored the access time and the
% * read and execute bits. We were strict for the other
% * attributes.
% *
% * Here we are strict, stricter than ufs in not allowing
% * users to attempt to set SF_SETTABLE bits or anyone to
% * set unsupported bits. However, we ignore attempts to
% * set ATTR_ARCHIVE for directories `cp -pr' from a more
% * sensible filesystem attempts it a lot.
% */
This comment is partly about the problem as it affects non-time attributes.
There is a problem with cp -p for almost all attributes, since msdosfs
doesn't really support them so cp -p from another file system that supports
more of them of them must either fail, or the failures must be silently or
unsilently ignored. cp -p unsilently ignores EPERM errors for *chown(),
but doesn't ignore any (?) other error (exit status != 0). This gives the
rather silly handling for typical errors for cp -p to msdosfs: chown()
usually fails at the syscall level but succeeds with a warning at the
shell level, but the less important utimes() usually fails at both levels.
There is a related problem with file time granularity. It is the usual
case that the file system has a different granularity than the system
and other file systems. When the target has more granularity than the
source, it is usually impossible to duplicate the times. Having utimes()
fail when the times cannot be duplicated would be too strict in general,
but sometimes you would like to be strict. POSIX has only recently
started addressing this problem. (Old?) FAT with its 2-second granularity
has always been unable to represent the default 1-second granularity, and
has always handled this by silently truncating to the granularity that it
supports.
My test directory for testing this mail shows another granularity problem:
(the file system is FAT32 with Win95 long names): after mkdir of it, a
stat utility on it gives:
% file=z
% ...
% atime=Sat Aug 20 00:00:00 2011 (1313762400.0)
% ctime=Sat Aug 20 16:14:29 2011 (1313820869.740000000)
% mtime=Sat Aug 20 16:14:28 2011 (1313820868.0)
This has the expected 2-second granularity for the mtime, but the other
times are strange:
- the atime is far in the past, and according to other tests has a
granularity of at least 200 seconds
- the ctime has a granularity of 100 msec. This differs significantly
from the mtime's granularity, so the ctime is up to 1.99 seconds in
advance of the mtime. This is probably a local bug -- I probably
don't have the fix for confusion between the ctime and the creation
time (birthtime). msdosfs only has a creation time so the ctime must
be faked and should usually be the same as the mtime. But how does
the creation time have more precision?
In other tests, creat() of a file sets the mtime and ctime reasonably,
but the atime remains with a fixed value far in the past. touch
advances the mtime correctly, but doesn't update the ctime. This is
consistent with displayed ctime actually being the creation time.
> Yes, FAT file systems do not maintain a directory modify
> time.
Er, yes they do...
> (The original FAT12,16 structure didn't even have a
> modify time for the root dir.)
... except for the root directory, they don't, and this doesn't depend on
the the version -- there is no directory entry for the root file system,
so the modification time can't be stored in the usual place for root
directories, only.
> Just like Windows.
I normally use cygwin for managing file times on Windows, and touch and
cp -p work reasonably well with it. In particular, tuch updates directory
times. 15-25 years ago, I used DOS utitities to manage file times and
don't remember any problems with touch. Even DOS 2.1 (?) has a syscall
like utimes().
> This causes issues when a FAT fs is exported via NFS and
> someone was going to experiment with an "in memory only"
> modify time for dirs, to minimize caching issues, but I
> haven't heard back from them lately.
"Memory only" times must never escape to userland. Linux has (had?)
file times in vfs, which makes many things easy, but old versions of
Linux did let them escape to userland. I ran fussy POSIX conformance
tests for times. The tests would succeed for non-POSIX file systems,
but only due to the times being in memory, and then unly while the
files were cached in memory.
> Apparently Mac OS X chooses to update the modify time that
> exists on FAT32 file systems, but that isn't Windows compatible.
Yes, it's a bug in Mac OS to be incompatile. However, I sometimes
wish for a mount option to control this. Similarly for weakening
of checking for attributes that cannot be set. Also, for file
times, there is another annoying problem which might be best handled
by a mount option: msdosfs file time change twice a year with daylight
saving. I sometimes back up msdosfs files to ffs where their times
don't change like this, and would like an easy way to stop the changes.
Moving across timezones might cause even more frequent changes, but this
doesn't affect me.
Bruce
More information about the freebsd-fs
mailing list