[Bug 271296] sysutils/fusefs-sshfs: hard linking files gives input/ouput errors

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=271296

            Bug ID: 271296
           Summary: sysutils/fusefs-sshfs: hard linking files gives
                    input/ouput errors
           Product: Ports & Packages
           Version: Latest
          Hardware: amd64
                OS: Any
            Status: New
          Severity: Affects Only Me
          Priority: ---
         Component: Individual Port(s)
          Assignee: bofh@freebsd.org
          Reporter: gkroon@maelstrom.ninja
          Assignee: bofh@freebsd.org
             Flags: maintainer-feedback?(bofh@freebsd.org)

Hi all, I just upgraded my server (let's call it "server A") from FreeBSD 13.1
to FreeBSD 13.2.
But now my system produces input/output errors when I try to make hard links in
an SSHFS mounted directory.

# Summary

Example to reproduce on server A:

```
$ cd ~/Maildir  # this is the SSHFS mount, on "server B"
$ echo "foo" > foo
$ ln foo bar
ln: bar: Input/output error
```

It seems to create a new file with a new inode instead (see first column):

```
771 -rw-r--r--  1 user  user  4 May  3 10:58 /home/user/Maildir/bar
774 -rw-r--r--  1 user  user  4 May  3 10:58 /home/user/Maildir/foo
```

Server A is using FreeBSD 13.2, on which I executed the above commands.
Server B is using TrueNAS-13.0-U4 (based on FreeBSD 13.1-RELEASE-p7), on which
the remote directory exists I'm mounting on the client.

This is the /etc/fstab entry on server A, which worked using FreeBSD 13.1:

```
user@server-b.lan:/mnt/tank/home/user/Maildir /home/user/Maildir fusefs
rw,allow_other,mountprog=/usr/local/bin/sshfs,port=1116,IdentityFile=/root/.ssh/id_ed25519,reconnect,ServerAliveInterval=15,ServerAliveCountMax=3,late
0 0
```

Creating hard links works on both the servers' own filesystems (ZFS), but not
anymore when I upgraded server A to FreeBSD 13.2, using the same SSHFS mounted
directory.

## Setup

SSHFS is installed on my system:

```
$ pkg info | grep fusefs-sshfs
fusefs-sshfs-3.7.3_1           Mount remote directories over ssh
```

I'm using the following entry in /boot/loader.conf:

```
$ grep fuse /boot/loader.conf
fusefs_load="YES"
```

I can also see the FUSE kernel module being loaded (at boot):

```
$ kldstat | grep fuse
 2    1 0xffffffff8213f000    262f0 fusefs.ko
```

And the SSHFS mount is still being mounted (at boot):

```
$ mount | grep home
zroot/usr/home on /usr/home (zfs, local, noatime, nfsv4acls)
user@server-b.lan:/mnt/tank/home/user/Maildir on /usr/home/user/Maildir
(fusefs.sshfs)
```

Please note that /home is of course a symlink to /usr/home:

```
$ ls -ld /home
lrwxr-xr-x  1 root  wheel  8 Dec  5 12:58 /home@ -> usr/home
```

## Background

I found out the above when my MDA (Dovecot) on server A produced the following
errors in the mail logs:

```
May  3 10:35:08 mail dovecot[3349]: imap(user)<73981><YPoBCcP6eJXAqLIB>: Error:
Mailbox INBOX:
link(/home/user/Maildir/cur/1683098837.M45130P81809.mail,S=44503,W=45217:2,,
/home/user/Maildir/.Trash/tmp/1683102908.M640256P73981.mail) failed:
Input/output error
```

These errors popped up after the upgrade to FreeBSD 13.2, and it is making it
impossible to move mails around in my MUAs (e.g. Thunderbird).
Interestingly, my MTA (Postfix) can still receive and store mails on the SSHFS
mount.
No problem at all.
So it seems only Dovecot is hard linking files, which is now causing
input/output errors when I upgraded to FreeBSD 13.2.

Also, the Dovecot user still has permissions to perform read and write
operations on the SSHFS mount.
But hard linking is still troublesome:

```
# sudo -u dovecot touch /home/user/Maildir/foo
# sudo -u dovecot rm /home/user/Maildir/foo
# sudo -u dovecot touch /home/user/Maildir/foo
# sudo -u dovecot ln /home/user/Maildir/foo /home/user/Maildir/bar
ln: /home/user/Maildir/bar: Input/output error
```

Can anyone reproduce this by trying to make a hard link on an SSHFS mount using
FreeBSD 13.2?

Thanks in advance!

P.S. I copied the above from my original post on
[Reddit](https://www.reddit.com/r/freebsd/comments/136ewci/upgraded_to_freebsd_132_using_sshfs_hard_linking/).
I have also since given up on SSHFS, and replaced it with NFS over WireGuard.

-- 
You are receiving this mail because:
You are the assignee for the bug.