[Bug 300130] nfscl: EXCHANGE_ID verifier bump on every recovery causes client-replacement churn against Linux nfsd

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=300130

            Bug ID: 300130
           Summary: nfscl: EXCHANGE_ID verifier bump on every recovery
                    causes client-replacement churn against Linux nfsd
           Product: Base System
           Version: 15.1-RELEASE
          Hardware: amd64
                OS: Any
            Status: New
          Severity: Affects Only Me
          Priority: ---
         Component: kern
          Assignee: bugs@FreeBSD.org
          Reporter: e@4779.name

## Summary

nfsrpc_setclient() bumps clp->nfsc_rev and sends co_verf = (nfs_exchangeboot,
nfsc_rev) on EVERY setclient (sys/fs/nfsclient/nfs_clrpcops.c:1113 for v4.1,
:1185 for v4.0; verifier built at :5541-5542). Per RFC 5661 s18.35.4 a changed
verifier with the same owner means 'client rebooted': the server discards ALL
prior state (sessions, stateids). When recovery is initiated not by a real
reboot but by a racing operation failing NFSERR_BADSESSION/STALE_STATEID
(nfscl_initiate_recovery -> nfscl_recover -> setclient), the client destroys
its own valid server-side state. Every in-flight op then fails
(BADSESSION on old session slots, BADSTATEID for old open stateids), each
failure initiates recovery again, and the loop self-sustains for as long as
streaming traffic races the replacement.

## Observed on 15.1-RELEASE-p5 (amd64), NFSv4.2 mount, Linux nfsd 7.2.9 server

mount options: nfsv4,minorversion=2,tcp,resvport,nconnect=4,hard,cto

- Server tracepoints (nfsd_clid_confirmed / nfsd_cb_shutdown): full
  EXCHANGE_ID+CREATE_SESSION cycle every ~1-20 s around the clock under load;
  clientid cl_id counter +22 per 30 s; each predecessor session immediately
  nfsd_cb_shutdown state=DOWN.
- nfsd_compound_status(status!=0): OP_SEQUENCE status=10052 (BADSESSION)
  ~1/4 s and OP_READ status=10025 (BADSTATEID) ~2/s during bursts.
- Client narration (vfs.nfs.debuglevel=1): endless 'aft exch=0 / aft
  createsess=0 / aft reclaimcomp=0' full-recovery cycles; intermittent
  'Got badsession' / 'Marked defunct' / 'Badsession looping'.
- Collateral: close(2)-triggered COMMIT that lands inside a replacement window
  blocks forever (hard mount, rc_timeout=-1) — observed as userspace threads
  stuck in nfsrpc_commit -> clnt_reconnect_call for tens of minutes while the
  TCP connection stays ESTABLISHED and no retransmit SYN is ever sent.
- Verified identical code in main and stable/15 (2026-10-26).

## Related

- PR 299221 — recovery storm after BADSESSION (CREATE_SESSION retry vs
  EXCHANGE_ID fallback); fixed by patch, but the fallback path still bumps the
  verifier, so the loop below survives that fix.
- PR 299225 — struct nfsclds leak per recovery (same recovery machinery).
- Commit 5ab48fb9f7ac re-arms recovery on every BADSESSION (cited in 299221).

## Suggested direction

Do not change the verifier for recovery initiated from
nfscl_initiate_recovery()/nfscl_recover() when the clientid is still valid
server-side: keeping co_verf stable lets CREATE_SESSION on the extant clientid
succeed (the code already tries that first), so no state is destroyed and
racing ops simply retry on the new session. Bump nfsc_rev only when the client
truly lost its lease (nfscl_hasexpired path) or at first setclient.

## Repro

FreeBSD 15.1 client, Linux nfsd server, NFSv4.1+ hard mount with active
streaming reads (seeding) plus writes; induce any recovery (e.g. server-side
lease expiry) and observe the churn via server nfsd tracepoints.

-- 
You are receiving this mail because:
You are the assignee for the bug.