Re: double used hostuuids - Re: NFS issue - newnfs_request: Wrong session srvslot=1 slot=0, freeing free slot!!

Ronald Klop wrote:
> Van: Pete French <pete@twisted.org.uk>
> Datum: 28 augustus 2022 10:16
> Aan: stable@freebsd.org
> Onderwerp: Re: double used hostuuids - Re: NFS issue - newnfs_request: Wrong session srvslot=1 slot=0, freeing free slot!!
> 
> On 27/08/2022 16:18, Rick Macklem wrote:
> > Ronald Klop <ronald-lists@klop.ws> wrote:
> >> On 8/27/22 00:17, Rick Macklem wrote:
> >>> Ganbold Tsagaankhuu <ganbold@gmail.com> wrote:
> >>>>> Rick,
> >>>>>
> >>>>> On Fri, Aug 26, 2022 at 11:18 AM Rick Macklem > ><rmacklem@uoguelph.ca<mailto:rmacklem@uoguelph.ca>> wrote:
> >>> Ganbold Tsagaankhuu <ganbold@gmail.com<mailto:ganbold@gmail.com>> wrote:
> >>>>>> Hi,
> >>>>>>
> >>>>>> We are having trouble with NFS running on STABLE:
> >>>>>>
> >>>>>> Aug 26 02:21:42 iron2 kernel: newnfs_request: Wrong session srvslot=1 slot=0
> >>> [stuff snipped]
> >>>>>> Aug 26 02:22:46 iron2 kernel: newnfs_request: Wrong session srvslot=1 slot=0
> >>>>>> Aug 26 02:22:46 iron2 kernel: freeing free slot!!
> >>>>>>
> >>>>>> We are running FreeBSD 13.1-STABLE #3 stable/13-n252198-c1434fd2dea: Fri Aug 26 01:51:53 UTC 2022 and mount options are:
> >>>>>>
> >>>>>> rw,nfsv4,minorversion=1,bg,soft,timeo=20,retrans=5,retrycnt=5
> >>>>>> ro,nfsv4,minorversion=1,bg,soft,timeo=20,retrans=5,retrycnt=5
> >>>>>>
> >>>>>> Is there any fix for this issue?
> >>> Oh, and one more thing. If you have multiple clients mounting the
> >>> NFSv4 server, make sure they all have unique hostids.
> >>> Check /etc/hostid and "sysctl kern.hostuuid". If two clients have the
> >>> same kern.hostuuid, there will be lots of trouble.
> >>>
> >>> rick
> >>
> >> Just a thought. Is it possible/easy to warn about double used hostuuids >from different client IP addresses?
> >> Although that will not help this person using Netapp as a server.
> > I don't think so. Same hostuuid implies same system, so how does a
> > server know they are two different systems?
> > - A client could have multiple IP host addresses, so different client
> >    host IP addresses for a TCP connection does not imply different systems.
> >
> > I can, however, modify the console message the server generates when
> > it sees a session has been replaced to include "check clients have
> > unique hostuuids", which might help.
> >
> > I also plan on adding a sentence to "man mount_nfs" about this,
> > since I just had an email discussion with someone else where the
> > problem turned out to be "same hostuuids for multiple clients"
> > and the loss of sessions on the FreeBSD server was the hint that
> > clued me in.
> >
> > At least I now know this configuration issue exists.
> >
> > rick
> >
> > Regards,
> > Ronald.
> >
> 
> It well worth adding this I think. I didnt realise this about NFSv4, and I do a lot with cloud machines, where I > simply clone the discs, and thus ended up with many machines with the same hostid. Took me a while to
> work out why my NFS was havign issues...
I have already committed a change for the server console message to main and it will be MFC'd
in a couple of weeks.

I will do a man page update soon, as well.

> -pete.

> It might help this case if the nfs client combined hostid+ip as a client id. Or include mac address. People
> tend to change the mac after a clone.
Well, in the past I have thought about this...
The problem is that, ideally, the string used by the NFSv4 client mount should be
invariant over time (including client reboot cycles).
Depending upon the situation, a machine's IP addresses can change over time (dynamically
assigned via dhcp, for example). They can also end up as addresses like 192.168.1.n sitting
behind a nat gateway, where the IP could be duplicated on other subnets.
As for MAC, if it is taken from a hardware card, then that hardware card gets replaced,
the MAC changes.

I think /etc/hostid (or whatever is used to set "kern.hostuuid") seems the best bet
for something unique that remains invariant for the life of the system.
I think all cloners need to do is remove /etc/hostid from the master being
cloned and then each clone will generate their own /etc/hostid upon first boot.

rick

Regards,
Ronald