Non-Blocking write over NFS fails

[Heath Nielson]

Hi all.  It's been some time since posted here but I hope someone can see if
I'm doing something stupid or if I really have found a problem.

On my FreeBSD 7 client (192.168.0.50) I mounted an NFS-share from a FreeBSD 7
server (192.168.0.5).  When I try to write a large file from the client to the
share in non-blocking mode, the write() call always returns EAGAIN.  I whipped
up a small little test program to illustrate the behavior:


#include <fcntl.h>
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <sys/types.h>
#include <sys/uio.h>
#include <unistd.h>


int main(int argc, char* argv[]) {
  int bufSz = 1172767;
  fprintf(stderr, "Creating file: %s\n", argv[1]);
  int fd = open(argv[1], O_CREAT|O_RDWR);
  if(fd < 0) perror(0);
  int ret = fcntl(fd, F_SETFL, O_NONBLOCK);
  if(ret < 0) {
     perror(0);
  }
  char* buf = (char*)malloc(bufSz);
  memset(buf, 0, bufSz);
  ret = -1;
  while(ret < 0) {
    ret = write(fd, buf, bufSz);
    fprintf(stderr, "write: %d\n", ret);
    if(ret < 0) perror(0);
    sleep(1);
  }
  free(buf);
  close(fd);
  return 0;
}

If the size is one byte less (1172766) the write succeeds so I guess I'm
hitting some buffer limit.  The mount in an NFSv3 mount and the error occurs
both over UDP and TCP.  When I do a tcpdump from the client, I see
(NFS over TCP):

01:47:18.308353 IP 192.168.0.50.835170573 > 192.168.0.5.nfs: 112
lookup fh 1073,399698/4993024 "t.txt"
01:47:18.308548 IP 192.168.0.5.nfs > 192.168.0.50.835170573: reply ok
120 lookup ERROR: No such file or directory
01:47:18.308575 IP 192.168.0.50.835170574 > 192.168.0.5.nfs: 144
create fh 1073,399698/4993024 "t.txt"
01:47:18.308856 IP 192.168.0.5.nfs > 192.168.0.50.835170574: reply ok
272 create fh 1073,399698/4993038
01:47:18.409715 IP 192.168.0.50.859 > 192.168.0.5.nfsd: . ack
2113663444 win 16588 <nop,nop,timestamp 16900146 554159454>

None of the data appears to be sent over the wire.  The program loops
indefinitely returning EAGAIN with the write() call.  When I run the program
with a smaller buffer size I get:

01:50:31.624967 IP 192.168.0.50.835170584 > 192.168.0.5.nfs: 112
lookup fh 1073,399698/4993024 "t.txt"
01:50:31.625116 IP 192.168.0.5.nfs > 192.168.0.50.835170584: reply ok
120 lookup ERROR: No such file or directory
01:50:31.625140 IP 192.168.0.50.835170585 > 192.168.0.5.nfs: 144
create fh 1073,399698/4993024 "t.txt"
01:50:31.625411 IP 192.168.0.5.nfs > 192.168.0.50.835170585: reply ok
272 create fh 1073,399698/4993039
01:50:31.626582 IP 192.168.0.50.835170586 > 192.168.0.5.nfs: 1448
write fh 1073,399698/4993039 32768 (32768) bytes @ 0
01:50:31.626592 IP 192.168.0.50.0 > 192.168.0.5.nfs: 1448 null
01:50:31.626598 IP 192.168.0.50.0 > 192.168.0.5.nfs: 1448 null
01:50:31.626605 IP 192.168.0.50.0 > 192.168.0.5.nfs: 1448 null
01:50:31.626612 IP 192.168.0.50.0 > 192.168.0.5.nfs: 1448 null
01:50:31.626625 IP 192.168.0.50.0 > 192.168.0.5.nfs: 1448 null
01:50:31.626633 IP 192.168.0.50.0 > 192.168.0.5.nfs: 1448 null
01:50:31.627062 IP 192.168.0.5.nfsd > 192.168.0.50.859: . ack
175520546 win 28946 <nop,nop,timestamp 554352775 17089867>
[....]
01:50:31.726744 IP 192.168.0.5.nfsd > 192.168.0.50.859: . ack 1172801
win 28946 <nop,nop,timestamp 554352874 17089948>
01:50:31.727005 IP 192.168.0.5.nfs > 192.168.0.50.835170621: reply ok
164 write [|nfs]

Since I'm not familiar with the NFS code does anyone know if this should work
(or not)?  If it is a problem, where should I go looking for the source of the
problem?  I'm assuming somewhere in src/sys/nfsclient.


Thanks,
Heath
(Please CC me since I'm not on the list)