[patch] Source entries removing is awfully slow.

[Ermal Luçi]

On Sat, Mar 9, 2013 at 2:37 PM, Kajetan Staszkiewicz
<vegeta at tuxpowered.net>wrote:

> Dnia sobota, 9 marca 2013 o 13:14:16 Ermal Luçi napisał(a):
> > On Fri, Mar 8, 2013 at 9:51 PM, Kajetan Staszkiewicz
> >
> > <vegeta at tuxpowered.net>wrote:
> > > Dnia piątek, 8 marca 2013 o 21:11:43 Ermal Luçi napisał(a):
> > > > Is this FreeBSD 9.x or HEAD?
> > >
> > > I found the problem and developed the patch on 9.1.
> > >
> > Can you please test this more 'beautiful' patch.
>
> Oh, somehow I did not notice an existing implementation for doubly linked
> list.
> I'm quite new to kernel programming.
>
> > Its similar to yours but also delays src state removal to the proper
> purge
> > thread.
>
> I'll try it right after the weekend.
>
> > Though the src node removal option through pfctl -K does a lot of job to
> > cleanup things
> > Still need to undertand why it takes so much time for you to loop through
> > 500K states.
>
> That is because the loop will not be called just once.
>
> `pfctl -K 0.0.0.0/0 -K ip.of.internal.server.behind.this.loadbalancer`
> will
> match multiple Source entries, up to a thousand of them in normal
> conditions
> ("normal" for my loadbalancers) and many many more when under a DDoS
> attack.
>
>
I would expect from a proper software to kill states from those clients and
then kill the srcnode for the backend server.
It does not make proper sense to not kill state before src nodes since that
is what will impact your connectivity.

Though the patch improves your use case a lot still would be better to even
kill those states during this step, with an extra option,
since otherwise you'd have to create for each of those client a separate
request.

Do you control the application to test an extra addition to this patch to
allow killing the linked states as well?


> > The purge thread does that every tick by partitioning it to a few per
> time
> > slot but still minutes is way loong.
> >
> > Can you please try to give a top -SH view of the time when this happens
> and
> > a pfctl -vvsa output?
>
> I'll try on Monday, although as far as I remember the machine was quite
> frozen
> during this operation.
>
> --
> | pozdrawiam / greetings | powered by Debian, CentOS and FreeBSD |
> |  Kajetan Staszkiewicz  | jabber,email: vegeta()tuxpowered net  |
> |        Vegeta          | www: http://vegeta.tuxpowered.net     |
> `------------------------^---------------------------------------'
>



-- 
Ermal