strange NFS failure

[Marcus Reid]

On Wed, Jun 08, 2005 at 09:00:38AM -0400, Robin P. Blanchard wrote:
> 
> > just upgraded two systems to RELENG_5 this evening and NFS 
> > client has collapsed.  is this known issue?  apologies -- i 
> > realise i should really be tracking stable list.
> > 
> > the error is
> > 
> >      [udp] 10.12.12.66:/usr/cabinet: RPCPROG_NFS: RPC: Port 
> > mapper failure -
> > RPC: Unable to recieve
> > 
> > and yes rpcbind is running OK.  rpcinfo of the system 
> > produces normal output.
> > 
> > thanks for any pointers.
> 
> Sounds related to:
> http://lists.freebsd.org/mailman/htdig/freebsd-stable/2005-June/015726.html
> 
> Using "broken" kernel also yields "broken" nfs...

I'm having this problem as well, and the culprit last time was
sys/kern/uipc_socket.c which has been recently changed.  mount_nfs
closes the socket before the server can send a packet back.

Hope this kdump output helps track it down.

Marcus


  1380 mount_nfs 0.041369 CALL  socket(0x2,0x2,0x11)
  1380 mount_nfs 0.041389 RET   socket 4
  1380 mount_nfs 0.041417 CALL  getsockname(0x4,0xbfbfe2f0,0xbfbfe2ec)
  1380 mount_nfs 0.041425 RET   getsockname 0
  1380 mount_nfs 0.041449 CALL  getsockopt(0x4,0xffff,0x1008,0xbfbfe2e8,0xbfbfe2ec)
  1380 mount_nfs 0.041455 RET   getsockopt 0
  1380 mount_nfs 0.041481 CALL  getsockname(0x4,0xbfbfe2d0,0xbfbfe2cc)
  1380 mount_nfs 0.041486 RET   getsockname 0
  1380 mount_nfs 0.041506 CALL  getsockopt(0x4,0,0x13,0xbfbfe2c4,0xbfbfe2c8)
  1380 mount_nfs 0.041510 RET   getsockopt 0
  1380 mount_nfs 0.041532 CALL  setsockopt(0x4,0,0x13,0xbfbfe2c0,0x4)
  1380 mount_nfs 0.041537 RET   setsockopt 0
  1380 mount_nfs 0.041558 CALL  bind(0x4,0xbfbfe2d0,0x10)
  1380 mount_nfs 0.041572 RET   bind 0
  1380 mount_nfs 0.041609 CALL  getrlimit(0x8,0xbfbfe2c0)
  1380 mount_nfs 0.041613 RET   getrlimit 0
  1380 mount_nfs 0.041806 CALL  getsockname(0x4,0xbfbfe240,0xbfbfe23c)
  1380 mount_nfs 0.041812 RET   getsockname 0
  1380 mount_nfs 0.041833 CALL  getsockopt(0x4,0xffff,0x1008,0xbfbfe238,0xbfbfe23c)
  1380 mount_nfs 0.041837 RET   getsockopt 0
  1380 mount_nfs 0.041914 CALL  gettimeofday(0xbfbfe308,0)
  1380 mount_nfs 0.041918 RET   gettimeofday 0
  1380 mount_nfs 0.041939 CALL  getpid
  1380 mount_nfs 0.041942 RET   getpid 1380/0x564
  1380 mount_nfs 0.042007 CALL  ioctl(0x4,FIONBIO,0xbfbfe304)
  1380 mount_nfs 0.042016 RET   ioctl 0
  1380 mount_nfs 0.042108 CALL  gettimeofday(0xbfbfe440,0)
  1380 mount_nfs 0.042112 RET   gettimeofday 0
  1380 mount_nfs 0.042132 CALL  kqueue
  1380 mount_nfs 0.042142 RET   kqueue 5
  1380 mount_nfs 0.042166 CALL  sendto(0x4,0x805f354,0x38,0,0x805d008,0x10)
  1380 mount_nfs 0.042284 GIO   fd 4 wrote 56 bytes
       0x0000 42a1 77fc 0000 0000 0000 0002 0001 86a0  |B.w.............|
       0x0010 0000 0002 0000 0003 0000 0000 0000 0000  |................|
       0x0020 0000 0000 0000 0000 0001 86a3 0000 0003  |................|
       0x0030 0000 0011 0000 0000                      |........|

  1380 mount_nfs 0.042286 RET   sendto 56/0x38
  1380 mount_nfs 0.042317 CALL  kevent(0x5,0x805d0dc,0x1,0xbfbfe470,0x1,0xbfbfe418)
  1380 mount_nfs 0.042339 RET   kevent 1
  1380 mount_nfs 0.042359 CALL  close(0x5)
  1380 mount_nfs 0.042372 RET   close 0
  1380 mount_nfs 0.042398 CALL  close(0x4)
  1380 mount_nfs 0.042418 RET   close 0
  1380 mount_nfs 0.042499 CALL  write(0x2,0xbfbfe010,0x53)
  1380 mount_nfs 0.042537 GIO   fd 2 wrote 83 bytes
       "[udp] ops:/usr/src: RPCPROG_NFS: RPC: Port mapper failure - RPC: Unabl\
        e to receive
       "