Flowtables -- any tuning hints?

[Kip Macy]

On Sat, Jul 11, 2009 at 10:24 AM, Scott Ullrich<sullrich at gmail.com> wrote:
> Hello Freebsd-current@ folks,
>
> I see with the commit "svn commit: r191259 - head/sys/netinet"
> flowtables have been added..  Cool!
>
> Does anyone have any tuning hints for this addition -- specifically
> how much memory does the hash table consume?   Or better yet does any
> documentation exist for this newly added feature?
>
> Looking for an easy way to calculate max flows for the amount of
> memory installed in a FreeBSD machine.


You want to avoid hash collisions. So, generally speaking you want the
hash table to be sized 2x larger than the number of unique connection
destinations.  You want the maximum number of flows to be as large as
the maximum number of unique destinations x number of cores. When you
get to the case of hundreds of thousands of unique destinations as in
the case of a small ISP doing IP forwarding, you're probably better
off disabling the flowtable. For most other workloads its likely to be
a clear win. Running a process on an 8-core system with 8 threads each
calling sendto(...) with 10 bytes I can push 3.5 - 4Mpps (with cxgb -
you won't get this with most cards) with the flowtable enabled. With
the flowtable disabled lock contention causes performance to degrade
to 330kpps with the aforementioned workload.

Let me know if you have any issues.

Cheers,
Kip