RELENG_5 and FAST_IPSEC limits

[Mike Tancsa]

At 04:23 PM 15/03/2005, Sam Leffler wrote:
>Mike Tancsa wrote:
>>Hi,
>>We are running into a case where there are too many SAs, and doing a 
>>setkey -D would fail with a
>>"recv: Resource temporarily unavailable"
>>after displaying most of the associations.
>>Is there a way to get around this, or is there a hard limit ?
>># setkey -D | grep ^172 | wc
>>      186     372    5096
>>When the remotes are renegotiating, and there are a lot of tunnels in the 
>>state of mature and dying, this number can go up to 341, but not 
>>higher.  This also seems to send racoon into a hung state that we then 
>>need to kill off and restart.
>>It was suggested in a post that /usr/src/sys/net/raw_cb.h get changed from
>>
>>#define RAWSNDQ 8192
>>#define RAWRCVQ 8192
>>to something larger like
>>#define RAWSNDQ 24576
>>#define RAWRCVQ 24576
>>If this is the underlying issue, will it work on its own, or are there 
>>other values that need to be tuned ?  Will I need to recompile any 
>>userland apps (e.g. racoon, setkey) and are there any other values I 
>>would need to adjust
>
>Looks like you're hitting the limit on returning status information 
>through a PF_KEY socket.  FWIW this is not related to FAST_IPSEC; it's an 
>issue with PF_KEY and is common to both IPsec implementations.
>
>Upping the raw socket buffer sizes should permit more information to be 
>returned but you may always exceed this limit as you can create more SA's 
>than can be reported in a single msg.  Some groups have dealt with this by 
>changing the PF_KEY api, e.g. to report an incomplete msg so the user-mode 
>app can retrieve more data with additional reads.  If upping the socket 
>buffer limits doesn't help then you might search for patches.


Hi and Thanks for the response! We will start by increasing the 2 
values.  Are there any potential negative side effects of doing so that we 
need to be aware of ? i.e. anything else that would need to be changed as a 
result ?

I am pretty sure racoon was getting hung up in pfkey.c in this loop

        while (1) {
                 if (msg)
                         racoon_free(msg);
                 msg = pk_recv(s, &len);
                 if (msg == NULL) {
                         if (len < 0)
                                 goto done;
                         else
                                 continue;
                 }

                 if (msg->sadb_msg_type != SADB_DUMP || msg->sadb_msg_pid 
!= pid)
                         continue;

                 ml = msg->sadb_msg_len << 3;
                 bl = buf ? buf->l : 0;
                 buf = vrealloc(buf, bl + ml);
                 if (buf == NULL) {
                         plog(LLV_ERROR, LOCATION, NULL,
                                 "failed to reallocate buffer to dump.\n");
                         goto fail;
                 }
                 memcpy(buf->v + bl, msg, ml);

                 if (msg->sadb_msg_seq == 0)
                         break;
         }
         goto done;


Should have core dumped him at the time :( Oh well, thats what happens when 
you do 5am maint!  If we plan for the worst case scenario for msg size, I 
guess this loop should not be an issue.

         ---Mike