No buffer space available

[Jeremy Chadwick]

On Sat, Apr 07, 2007 at 08:20:58PM -0300, Marc G. Fournier wrote:
> - --On Saturday, April 07, 2007 20:12:00 +0100 Chris <chrcoluk at gmail.com> wrote:
> > Also to add I now have a 2nd box using 6.2 STABLE few days old code,
> > had to use it because of broadcom 5755 nic card, I plan to use large
> > tcp window sizes so will be interesting to see if this also suffers
> > from the problem.
> 
> I've got 8 servers on the same network, 3 are almost identical, but one of them 
> (the one with the problem) is using software RAID vs hardware ... but, if you 
> are seeing it without using software RAID, then that is obviously not the 
> culprit :(

May be a red herring...

I'm able to reproduce the "No buffer space available" message when
setting net.inet.tcp.(send|recv)space to non-default values.  All I've
tried is the following, with a kernel dated 2007/04/22:

  # sysctl net.inet.tcp.sendspace=131072
  # sysctl net.inet.tcp.recvspace=262144

Example session:

$ su2
# sysctl net.inet.tcp.sendspace=131072
net.inet.tcp.sendspace: 32768 -> 131072
# sysctl net.inet.tcp.recvspace=262144
net.inet.tcp.recvspace: 65536 -> 262144
# logout
$ ssh medusa
socket: No buffer space available
ssh: connect to host medusa port 22: No buffer space available
$ su2
# sysctl net.inet.tcp.sendspace=32768
net.inet.tcp.sendspace: 131072 -> 32768
# sysctl net.inet.tcp.recvspace=65536
net.inet.tcp.recvspace: 262144 -> 65536
# logout
$ ssh medusa
Last login: Mon Apr 23 03:45:45 2007 from ...

I assume this is because the maximum size of a TCP datagram is 65536
bytes, but as I'm not familiar enough with TCP on such a low level,
this may be speculation on my part.

Just something worth checking/tinkering with.

-- 
| Jeremy Chadwick                                    jdc at parodius.com |
| Parodius Networking                           http://www.parodius.com/ |
| UNIX Systems Administrator                      Mountain View, CA, USA |
| Making life hard for others since 1977.                  PGP: 4BD6C0CB |