RELENG_5 and FAST_IPSEC limits

[Sam Leffler]

Mike Tancsa wrote:
> Hi,
> 
> We are running into a case where there are too many SAs, and doing a 
> setkey -D would fail with a
> 
> "recv: Resource temporarily unavailable"
> 
> after displaying most of the associations.
> 
> Is there a way to get around this, or is there a hard limit ?
> 
> # setkey -D | grep ^172 | wc
>      186     372    5096
> 
> When the remotes are renegotiating, and there are a lot of tunnels in 
> the state of mature and dying, this number can go up to 341, but not 
> higher.  This also seems to send racoon into a hung state that we then 
> need to kill off and restart.
> 
> It was suggested in a post that /usr/src/sys/net/raw_cb.h get changed from
> 
> 
> #define RAWSNDQ 8192
> #define RAWRCVQ 8192
> 
> to something larger like
> 
> #define RAWSNDQ 24576
> #define RAWRCVQ 24576
> 
> If this is the underlying issue, will it work on its own, or are there 
> other values that need to be tuned ?  Will I need to recompile any 
> userland apps (e.g. racoon, setkey) and are there any other values I 
> would need to adjust

Looks like you're hitting the limit on returning status information 
through a PF_KEY socket.  FWIW this is not related to FAST_IPSEC; it's 
an issue with PF_KEY and is common to both IPsec implementations.

Upping the raw socket buffer sizes should permit more information to be 
returned but you may always exceed this limit as you can create more 
SA's than can be reported in a single msg.  Some groups have dealt with 
this by changing the PF_KEY api, e.g. to report an incomplete msg so the 
user-mode app can retrieve more data with additional reads.  If upping 
the socket buffer limits doesn't help then you might search for patches.

	Sam