connect(): Operation not permitted

[Kian Mohageri]

On Wed, Jul 2, 2008 at 5:39 PM, Stef <stef-list at memberwebs.com> wrote:
> Kian Mohageri wrote:
>> On Sun, May 18, 2008 at 3:33 AM, Johan Ström <johan at stromnet.se> wrote:
>>> On May 18, 2008, at 9:19 AM, Matthew Seaman wrote:
>>>
>>>> Johan Ström wrote:
>>>>
>>>>> drop all traffic)? A check with pfctl -vsr reveals that the actual rule
>>>>> inserted is "pass on lo0 inet from 123.123.123.123 to 123.123.123.123 flags
>>>>> S/SA keep state". Where did that "keep state" come from?
>>>> 'flags S/SA keep state' is the default now for tcp filter rules -- that
>>>> was new in 7.0 reflecting the upstream changes made between the 4.0 and
>>>> 4.1
>>>> releases of OpenBSD.  If you want a stateless rule, append 'no state'.
>>>>
>>>> http://www.openbsd.org/faq/pf/filter.html#state
>>> Thanks! I was actually looking around in the pf.conf manpage but failed to
>>> find it yesterday, but looking closer today I now saw it.
>>> Applied the no state (and quick) to the rule, and now no state is created.
>>> And the problem I had in the first place seems to have been resolved too
>>> now, even though it didn't look like a state problem.. (started to deny new
>>> connections much earlier than the states was full, altough maybee i wasnt
>>> looking for updates fast enough or something).
>>>
>>
>> I'd be willing to bet it's because you're reusing the source port on a
>> new connection before the old state expires.
>>
>> You'll know if you check the state-mismatch counter.
>>
>> Anyway, glad you found a resolution.
>
> I've been experiencing this "Operation not permitted" too. I've been
> trying to track down the problem for many months, but due to the
> complexity of my firewalls (scores of jails each with scores of rules),
> I wasn't brave enough to ask for help :)
>
> As a work around we started creating rules without state, whenever we
> would run into the problem.
>
> Thanks for the pointer about state-mismatch. The state-mismatch counter
> does is in fact high in my case (see below). How would I go about
> getting the pf state timeout and the reuse of ports for outbound
> connections to match? Or is this an intractable problem, that just needs
> to be worked around?
>

Make sure your state-mismatch counter is increasing at the same times
you experience the problem (and isn't just high from some unrelated
issue).

A similar/related problem was addressed in OpenBSD 4.3
(http://www.openbsd.org/plus43.html).

  * In pf(4), allow state reuse if both sides are in FIN_WAIT_2 and a
new SYN arrives.

I'm not sure if it's been imported yet.  If not, you could try tuning
your timeout values (see pf.conf(5)).

The specific issue I was experienced was solved by shortening
tcp.closed, IIRC.  It's been a while though.

-Kian