Re: fsck -C -p: NO WRITE ACCESS

From: Andriy Gapon <avg_at_FreeBSD.org>
Date: Thu, 10 Feb 2022 07:05:53 UTC
On 09/02/2022 20:33, Kirk McKusick wrote:
> Thanks for this report. I am trying to track down several issues that
> it raises.

Thank you very much, Kirk.

> First, I am trying to figure out why you are getting a filesystem
> that does not have check-hashes. The newfs(8) on 13 has always created
> them by default. Did you bring this filesystem over from a 12 system?
> Or did you grab this 13 image from the periodic 13-STABLE builds
> (found on download.freebsd.org/ftp/snapshots/VM-IMAGES/13.0-STABLE)?
> Apparently those images are built on a 12 system, so their filesystems
> do not have the check hashes.

This is a quite old filesystem.  I think I created it in 2016 or so.

> I have found the cause of the unaligned write and am working on a fix
> for it. It would be helpful if you could run another experiment for me.
> Rerun the `fsck/dev/zvol/.../vault.eli' but answer no to the first
> question (SAVE DATA TO FIND ALTERNATE SUPERBLOCKS?) but then answer y
> to the next question (ADD CYLINDER GROUP CHECK-HASH PROTECTION?). It
> will want to do aligned writes and I want to know if they work.

Yes!  Those writes worked and fsck no longer asks about those things.
Only the alternate superblock question remains:

# fsck /dev/zvol/.../vault.eli
** /dev/zvol/.../vault.eli

SAVE DATA TO FIND ALTERNATE SUPERBLOCKS? [yn] n


ADD CYLINDER GROUP CHECK-HASH PROTECTION? [yn] y


ADD SUPERBLOCK CHECK-HASH PROTECTION? [yn] y


ADD INODE CHECK-HASH PROTECTION? [yn] y

** Last Mounted on /usr/home/avg/secret
** Phase 1 - Check Blocks and Sizes
** Phase 2 - Check Pathnames
** Phase 3 - Check Connectivity
** Phase 4 - Check Reference Counts
** Phase 5 - Check Cyl groups
136 files, 371 used, 253491 free (35 frags, 31682 blocks, 0.0% fragmentation)

***** FILE SYSTEM IS CLEAN *****

***** FILE SYSTEM WAS MODIFIED *****


# fsck /dev/zvol/.../vault.eli
** /dev/zvol/.../vault.eli

SAVE DATA TO FIND ALTERNATE SUPERBLOCKS? [yn] n

** Last Mounted on /usr/home/avg/secret
** Phase 1 - Check Blocks and Sizes
** Phase 2 - Check Pathnames
** Phase 3 - Check Connectivity
** Phase 4 - Check Reference Counts
** Phase 5 - Check Cyl groups
136 files, 371 used, 253491 free (35 frags, 31682 blocks, 0.0% fragmentation)

***** FILE SYSTEM IS CLEAN *****

> I am still working to understand your original problem, and will let
> you know when I get an answer to that problem.

I've just thought of using ktrace with fsck -p -C and this is what I see:
    674 fsck_ufs CALL  fstatat(AT_FDCWD,0x7fffffffe617,0x7fffffffd858,0)
    674 fsck_ufs NAMI  "/dev/zvol/.../vault.eli"
    674 fsck_ufs STRU  struct stat {dev=1895890688, ino=447, mode=020640, 
nlink=1, uid=0, gid=5, rdev=447, atime=1644476165, mtime=1644439708, 
ctime=1644439708, birthtime=-1, size=0, blksize=4096, blocks=0, flags=0x0 }
    674 fsck_ufs RET   fstatat 0

    674 fsck_ufs CALL  fstatat(AT_FDCWD,0x7fffffffe617,0x7fffffffd858,0)
    674 fsck_ufs NAMI  "/dev/zvol/.../vault.eli"
    674 fsck_ufs STRU  struct stat {dev=1895890688, ino=447, mode=020640, 
nlink=1, uid=0, gid=5, rdev=447, atime=1644476165, mtime=1644439708, 
ctime=1644439708, birthtime=-1, size=0, blksize=4096, blocks=0, flags=0x0 }
    674 fsck_ufs RET   fstatat 0

    674 fsck_ufs CALL  openat(AT_FDCWD,0x7fffffffe617,0<O_RDONLY>)
    674 fsck_ufs NAMI  "/dev/zvol/.../vault.eli"
    674 fsck_ufs RET   openat 3

    674 fsck_ufs CALL  pread(0x3,0x800a8e000,0x2000,0x10000)
    674 fsck_ufs GIO   fd 3 read 4096 bytes
    674 fsck_ufs RET   pread 8192/0x2000
    674 fsck_ufs CALL  pread(0x3,0x800aae000,0x1000,0x828000)
    674 fsck_ufs GIO   fd 3 read 4096 bytes
    674 fsck_ufs RET   pread 4096/0x1000
    674 fsck_ufs CALL  pread(0x3,0x800a8c000,0x1000,0x30018000)
    674 fsck_ufs GIO   fd 3 read 4096 bytes
    674 fsck_ufs RET   pread 4096/0x1000

    674 fsck_ufs CALL  openat(AT_FDCWD,0x7fffffffe617,0x1<O_WRONLY>)
    674 fsck_ufs NAMI  "/dev/zvol/.../vault.eli"
    674 fsck_ufs RET   openat 4

    674 fsck_ufs CALL  close(0x3)
    674 fsck_ufs RET   close 0
    674 fsck_ufs CALL  close(0x4)
    674 fsck_ufs RET   close 0

    674 fsck_ufs CALL  fstatat(AT_FDCWD,0x7fffffffe617,0x7fffffffd840,0)
    674 fsck_ufs NAMI  "/dev/zvol/.../vault.eli"
    674 fsck_ufs STRU  struct stat {dev=1895890688, ino=447, mode=020640, 
nlink=1, uid=0, gid=5, rdev=447, atime=1644476334, mtime=1644439708, 
ctime=1644439708, birthtime=-1, size=0, blksize=4096, blocks=0, flags=0x0 }
    674 fsck_ufs RET   fstatat 0

    674 fsck_ufs CALL  openat(AT_FDCWD,0x7fffffffe617,0<O_RDONLY>)
    674 fsck_ufs NAMI  "/dev/zvol/.../vault.eli"
    674 fsck_ufs RET   openat 3

    674 fsck_ufs CALL  openat(AT_FDCWD,0x7fffffffe617,0x1<O_WRONLY>)
    674 fsck_ufs NAMI  "/dev/zvol/.../vault.eli"
    674 fsck_ufs RET   openat -1 errno 1 Operation not permitted

Interesting... we have this sequence of opens: O_RDONLY, O_WRONLY, O_RDONLY, 
O_WRONLY.  And for some reason the second O_WRONLY open fails.
Perhaps something gets confused on the GEOM / GELI side.


-- 
Andriy Gapon