touch(1) not working on directories in an msdosfs(5) envirement
Alexander Best
arundel at freebsd.org
Sat Aug 20 09:42:54 UTC 2011
On Sat Aug 20 11, Bruce Evans wrote:
> On Fri, 19 Aug 2011, Rick Macklem wrote:
>
> >Alexander Best wrote:
> >>can somebody confirm this issue? is it already known?
> >>
> >>otaku% ll|grep HELL
> >>drwxr-xr-x 1 arundel arundel 16384 19 Aug 19:57 HELLO
> >>-rw-r--r-- 1 arundel arundel 0 19 Aug 20:13 HELLO2
> >>otaku% touch HELLO*
> >>otaku% ll|grep HELL
> >>drwxr-xr-x 1 arundel arundel 16384 19 Aug 19:57 HELLO
> >>-rw-r--r-- 1 arundel arundel 0 19 Aug 20:55 HELLO2
>
> This is fixed (hacked around to keep the diffs small) in my version:
WOW! such a lot of detailed info on the subject and useful lines of code. you
should really get back to being an active committer, so we all get the
benefit of your code. ;)
>
> % Index: msdosfs_vnops.c
> % ===================================================================
> % RCS file: /home/ncvs/src/sys/fs/msdosfs/msdosfs_vnops.c,v
> % retrieving revision 1.147
> % diff -u -2 -r1.147 msdosfs_vnops.c
> % --- msdosfs_vnops.c 4 Feb 2004 21:52:53 -0000 1.147
> % +++ msdosfs_vnops.c 12 Nov 2007 21:47:48 -0000
> % @@ -457,5 +457,7 @@
> % (error = VOP_ACCESS(ap->a_vp, VWRITE, cred, ap->a_td))))
> % return (error);
> % +#if 0
> % if (vp->v_type != VDIR) {
> % +#endif
> % if ((pmp->pm_flags & MSDOSFSMNT_NOWIN95) == 0 &&
> % vap->va_atime.tv_sec != VNOVAL) {
>
> The main part of the fix is to just remove the special case for directories
> which just breaks utimes() on directories. Even DOS and Windows never
> had this brokenness. What DOS and Windows do specially for directories
> is not update their modification time when their contents is changed.
> FreeBSD is compatible with this, and the above special case is apparently
> the result of trying too hard to be compatible with DOS and Windows.
>
> % @@ -463,4 +465,7 @@
> % unix2dostime(&vap->va_atime, &dep->de_ADate,
> % NULL, NULL);
> % + if (vp->v_type != VDIR)
> % + dep->de_Attributes |= ATTR_ARCHIVE;
> % + dep->de_flag |= DE_MODIFIED;
> % }
> % if (vap->va_mtime.tv_sec != VNOVAL) {
>
> Now that setting times on directories is unbroken, we have to be more
> careful with the archive bit. In DOS and Windows, the archive bit is
> meaningless for FAT* directories and is not set by any syscall that I
> know of (unlike for ffs IIRC). The above avoids setting it for directories.
> Not setting DE_MODIFIED is an unrelated micro-optimization (try harder not
> to set it when we didn't change anything).
>
> % @@ -468,8 +473,11 @@
> % unix2dostime(&vap->va_mtime, &dep->de_MDate,
> % &dep->de_MTime, NULL);
> % + if (vp->v_type != VDIR)
> % + dep->de_Attributes |= ATTR_ARCHIVE;
> % + dep->de_flag |= DE_MODIFIED;
>
> Similarly for the mtime.
>
> % }
> % - dep->de_Attributes |= ATTR_ARCHIVE;
> % - dep->de_flag |= DE_MODIFIED;
>
> This was moved early.
>
> % +#if 0
> % }
> % +#endif
>
> Finish hacking way the special case for directories.
>
> % }
> % /*
> % @@ -494,5 +502,5 @@
> % }
> % }
> % - return (deupdat(dep, 1));
> % + return (deupdat(dep, 0));
>
> Remove an unrelated pessimization (a synchronous update where even an
> asynchronous update is more than what is needed). Even ffs in Net/2
> didn't have the full pessimization here -- it pessimized SETATTR on
> times but not on the more important ownership and permission attributes.
> ffs still had the pessimization for times in 4.4BSD-Lite2, but FreeBSD
> fixed it in 1998 (ufs_vnops.c 1.79; the fix was buried in a mega-commit
> with a content-free log message :-(), and I fixed it in my version of
> ffs long before that. msdosfs_setattr() is a little different from
> ufs_setattr(); in particular, it does the (previous synchronous) update
> for all successful calls while ufs_setattr() only ever did it for times,
> so the pessimization has a wider scope in msdosfs.
>
> % }
>
> The above is only the least serious of the bugs in msdosfs_setattr() :-(.
> With the above fix, plain touch works as well as possible -- it cannot
> work perfectly since setting of atimes is not always supported. But
> touch -r and more importantly, cp -p only work as well as possible for
> root, since they use utimes() without the null timeptr arg that allows
> plain touch to work. A non-null timeptr arg ends up normally requiring
> root permissions for msdosfs where it normally doesn't require extra
> permissions for ffs, because ownership requirements for the non-null case
> cannot be satisfied by file systems that don't really support ownerships.
> We fudge the ownerships and use weak checks on them in most places, but
> for utimes() we use strict checks that almost always fail: from my old
> version:
>
> % if (vap->va_flags != VNOVAL) {
> % if (vp->v_mount->mnt_flag & MNT_RDONLY)
> % return (EROFS);
> % if (cred->cr_uid != pmp->pm_uid &&
> % (error = suser_cred(cred, PRISON_ROOT)))
> % return (error);
>
> The implementation of this check has changed significantly in -current,
> but its semantics and result havven't. The file must be owned by
> someone (pmp->pm_uid), and no one else except root has permission for
> utimes() with a non-null timeptr. This works right in ffs because the
> file can normally be owned by its rightful owner, but in msdosfs the
> owner is faked and there can be only one owner for the whole file
> systems. I use owner root and group msdosfs for all msdosfs file
> systems. The group permissions allow non-root users in group msdosfs
> to do almost everything except this unimportant utimes() operation
> to almost all msdosfs files.
>
> % /*
> % * We are very inconsistent about handling unsupported
> % * attributes. We ignored the access time and the
> % * read and execute bits. We were strict for the other
> % * attributes.
> % *
> % * Here we are strict, stricter than ufs in not allowing
> % * users to attempt to set SF_SETTABLE bits or anyone to
> % * set unsupported bits. However, we ignore attempts to
> % * set ATTR_ARCHIVE for directories `cp -pr' from a more
> % * sensible filesystem attempts it a lot.
> % */
>
> This comment is partly about the problem as it affects non-time attributes.
> There is a problem with cp -p for almost all attributes, since msdosfs
> doesn't really support them so cp -p from another file system that supports
> more of them of them must either fail, or the failures must be silently or
> unsilently ignored. cp -p unsilently ignores EPERM errors for *chown(),
> but doesn't ignore any (?) other error (exit status != 0). This gives the
> rather silly handling for typical errors for cp -p to msdosfs: chown()
> usually fails at the syscall level but succeeds with a warning at the
> shell level, but the less important utimes() usually fails at both levels.
>
> There is a related problem with file time granularity. It is the usual
> case that the file system has a different granularity than the system
> and other file systems. When the target has more granularity than the
> source, it is usually impossible to duplicate the times. Having utimes()
> fail when the times cannot be duplicated would be too strict in general,
> but sometimes you would like to be strict. POSIX has only recently
> started addressing this problem. (Old?) FAT with its 2-second granularity
> has always been unable to represent the default 1-second granularity, and
> has always handled this by silently truncating to the granularity that it
> supports.
>
> My test directory for testing this mail shows another granularity problem:
> (the file system is FAT32 with Win95 long names): after mkdir of it, a
> stat utility on it gives:
>
> % file=z
> % ...
> % atime=Sat Aug 20 00:00:00 2011 (1313762400.0)
> % ctime=Sat Aug 20 16:14:29 2011 (1313820869.740000000)
> % mtime=Sat Aug 20 16:14:28 2011 (1313820868.0)
>
> This has the expected 2-second granularity for the mtime, but the other
> times are strange:
> - the atime is far in the past, and according to other tests has a
> granularity of at least 200 seconds
> - the ctime has a granularity of 100 msec. This differs significantly
> from the mtime's granularity, so the ctime is up to 1.99 seconds in
> advance of the mtime. This is probably a local bug -- I probably
> don't have the fix for confusion between the ctime and the creation
> time (birthtime). msdosfs only has a creation time so the ctime must
> be faked and should usually be the same as the mtime. But how does
> the creation time have more precision?
> In other tests, creat() of a file sets the mtime and ctime reasonably,
> but the atime remains with a fixed value far in the past. touch
> advances the mtime correctly, but doesn't update the ctime. This is
> consistent with displayed ctime actually being the creation time.
>
> >Yes, FAT file systems do not maintain a directory modify
> >time.
>
> Er, yes they do...
>
> >(The original FAT12,16 structure didn't even have a
> >modify time for the root dir.)
>
> ... except for the root directory, they don't, and this doesn't depend on
> the the version -- there is no directory entry for the root file system,
> so the modification time can't be stored in the usual place for root
> directories, only.
>
> >Just like Windows.
>
> I normally use cygwin for managing file times on Windows, and touch and
> cp -p work reasonably well with it. In particular, tuch updates directory
> times. 15-25 years ago, I used DOS utitities to manage file times and
> don't remember any problems with touch. Even DOS 2.1 (?) has a syscall
> like utimes().
>
> >This causes issues when a FAT fs is exported via NFS and
> >someone was going to experiment with an "in memory only"
> >modify time for dirs, to minimize caching issues, but I
> >haven't heard back from them lately.
>
> "Memory only" times must never escape to userland. Linux has (had?)
> file times in vfs, which makes many things easy, but old versions of
> Linux did let them escape to userland. I ran fussy POSIX conformance
> tests for times. The tests would succeed for non-POSIX file systems,
> but only due to the times being in memory, and then unly while the
> files were cached in memory.
>
> >Apparently Mac OS X chooses to update the modify time that
> >exists on FAT32 file systems, but that isn't Windows compatible.
>
> Yes, it's a bug in Mac OS to be incompatile. However, I sometimes
> wish for a mount option to control this. Similarly for weakening
> of checking for attributes that cannot be set. Also, for file
> times, there is another annoying problem which might be best handled
> by a mount option: msdosfs file time change twice a year with daylight
> saving. I sometimes back up msdosfs files to ffs where their times
> don't change like this, and would like an easy way to stop the changes.
> Moving across timezones might cause even more frequent changes, but this
> doesn't affect me.
>
> Bruce
More information about the freebsd-fs
mailing list