Linux (client) - FreeBSD (server) NFS4 interoperability problem

[Tomáš Drbohlav]

On 5.12.2010 15:17, Rick Macklem wrote:
>>>>
>>>> Although I am not an nfs expert, when I opened the dump in
>>>> WireShark
>>>> it
>>>> seems to me, that server does not respond to packet #83, Compound
>>>> Call
>>>> containing OPEN opcode. That is probably the thing which disturbes
>>>> Linux
>>>> client. But why? It is beyond my capabilities.
>>>>
>>>> What can I do to make i work? Is it me or server, who si broken?
>>>>
>>> I'll take a look at the packet dump when I get home in a couple of
>>> days.
>>
>> Thank, we appreciate any help!
>>
>>> The main thing I'd suggest is trying a non-Kerberized NFSv4
>>> mount from Linux->FreeBSD and see how that does?
>>
>> Looks almost the same to me. Attaching dump/
>
> Ok, I took a look at the dump and it's kinda interesting. Here's
> what happens:
> - Linux client does a SetclientID (#51) specifying a callback
>    via TCP at port 37372.
> - When client does an Open at #53, the server tries to establish
>    a TCP connection for the callback. (See SYN packets #56, 58-64, 72)
> - The Linux client does not respond to these SYNs.
> - At #70, client gives up waiting for the Open reply (which hasn't
>    happened yet because the server hasn't timed out on the TCP connect
>    for the callback) and disconnects the main TCP NFS connection.

thanks, that sounds reasonable, the last disconnect can be unmount on 
client side, which we made at the end... in previous test we let it wait 
more then few minutes (which should enough for timeout I guess)

> So, I can think of a couple of ways this can be worked around.
> 1 - Figure out why the client isn't responding too the SYNs for
>      the callback path and fix it so it connects. (Is port 37372
>      somehow blocked for the client?)

traffic may be blocked, but we do not want server to make callbacks 
anyway (we are exporting data which are being modified by Samba or 
locally so no delegations and server is going to be access through NAT)

> 2 - Disable the server from attempting to do a callback connection
>      even when the client advertises one. (At the moment this can
>      only be done for Kerberized mounts by setting nfsrv_nogsscallback == 1
>      in sys/fs/nfsserver/nfs_nfsdstate.c. (Maybe I should add a sysctl
>      to disable callback attempts?)

I have try it and it solved our other problem (small delay after first 
file open after connection), probably connected with server trying to 
establish callback. I will check with Linux client and let you know, but 
this si what we needed anyway.

Sysctl is a good idea, btw man nfsv4 mentions 
vfs.newnfs.issue_delegations ... what is the purpose of callback other 
than delegation?

> 3 - Reconfiguring the client so that it doesn't advertise a callback path.
>      (I have no idea how to do this for Linux, but there is a
>       nfsv4 at linux-nfs.org email list where someone might be able to help
>       with doing this?)

will try, but it is not good for us anyway, as we do not control all the 
clients so we want be resistant against discussed lockup.

Thank for now, I will let you know about the tests.

  Drb

> Basically, it's hard to say if the bug is the server trying to establish
> the callback TCP connection for too long or the client not waiting long
> enough for the Open Reply. (The server will eventually give up on trying
> to establish the callback TCP connection and will then reply to the
> client. It could also be argued that the server should reply NFS4ERR_DELAY
> to the Open request while it is trying to establish the TCP connection,
> if it doesn't happen quickly.) I'll look at the timeout for the callback
> TCP connect and see if I can make it smaller easily. Not having a
> callback path doesn't break NFSv4, it just disables issuing of delegations,
> which aren't currently enabled by default in the FreeBSD server at this
> time anyhow.
>
> If I can come up with a simple patch to reduce the timeout on the TCP
> connect for the callback, I'll email you that for testing.
>
> rick
>