nfs v2/v3 and diskless boot problem

[Danny Braniss]

> Danny Braniss wrote:
> >> Danny Braniss wrote:
> >>> there is an undocumented option:
> >>> 	boot-nfsroot-options
> >>> that the diskeless boot can use. I tried 
> >>> 	boot-nfsroot-options = "nfsv3"
> >>> since the pxeboot does the initial mount via nfsv2, and this has at least
> >>> one problem: removing a file from the readonly / will hang the system.
> >>>
> >>> so, the remount to v3 works in the case that the root is served by a Freebsd
> >>> nfs server, but fails if it's NetAPP. The reason is that the v2 filehandle
> >>> is 32 bytes, and when switching to V3 it becomes 28bytes - sizeof(fhandle_t).
> >>> This is not liked by the NetApp, which correctly gives error 1001: BADHANDLE 
> >>> :-)
> >>>
> >>> While I'm trying to come up with a solution, I am wondering if someone
> >>> can shed some light:
> >>>  - is sizeof(fhandle_t) == 28 bytes is mystical, or changing it to
> >>>    32 bytes will start WW3?
> >>
> >> NFSv3 file handles (by spec) can be up to 64bytes.
> > 
> > true, but in freebsd, look at sys/nfs/nfsproto.h
> > #define	NFSX_V2FH	32
> > #define NFSX_V3FH		(sizeof (fhandle_t))
> > #define NFSX_V4FH		128
> > 
> > so for v3 it's 28 bytes. (fhandle_t is defined in sys/mount.h)
> > 
> > 
> >> I'm not 100% sure what is happening, but it sounds like the file handle 
> >> for the mount point or maybe one of the directories is not getting reset 
> >> on remount.
> >>
> >> When do you get the BADHANDLE error?  Can you capture a 
> >> tshark/wireshark/tcpdump of the remount and error?
> > 
> > I did, and if you look in sys/nfsclient/nfs_vfsops.c, nfs_convert_diskless is responsible
> > for chopping off the 4 extra bytes. BTW, I tried to change the bcopy count to NFSX_V2FH/32, and
> > it panics the kernel :-(
> > 
> > danny
> 
> 
> oh - looks like this says it all:
> http://fxr.googlebit.com/source/sys/nfsclient/nfsdiskless.h?v=8-CURRENT#L51
> 
that's where the boot-nfsroot-options comes from:-)
if you notice, the filehandle for v3 is 64 bytes, but
only 28 are used.

but as I mentioned initially, this ONLY works when the server is FreeBSD, and
breaks for other servers, ie NetAPP. AND the initial question stands:
what's in a filehandle, or can it be > 28bytes.

danny