Freebsd IP Forwarding performance (question,
and some info) [7-stable, current, em, smp]
paul at gtcomm.net
Wed Jul 2 08:50:35 UTC 2008
SMP DISABLED on my Opteron 2212 (ULE, Preemption on)
Yields ~750kpps in em0 and out em1 (one direction)
I am miffed why this yields more pps than
a) with all 4 cpus running and b) 4 cpus with lagg load balanced over 3
incoming connections so 3 taskq threads
I would be willing to set up test equipment (several servers plugged
into a switch) with ipkvm and power port access
if someone or a group of people want to figure out ways to improve the
routing process, ipfw, and lagg.
Maximum PPS with one ipfw rule on UP:
tops out about 570Kpps.. almost 200kpps lower ? (frown)
I'm going to drop in a 3ghz opteron instead of the 2ghz 2212 that's in
here and see how that scales, using UP same kernel etc I have now.
Julian Elischer wrote:
> Paul wrote:
>> ULE without PREEMPTION is now yeilding better results.
>> input (em0) output
>> packets errs bytes packets errs bytes colls
>> 571595 40639 34564108 1 0 226 0
>> 577892 48865 34941908 1 0 178 0
>> 545240 84744 32966404 1 0 178 0
>> 587661 44691 35534512 1 0 178 0
>> 587839 38073 35544904 1 0 178 0
>> 587787 43556 35540360 1 0 178 0
>> 540786 39492 32712746 1 0 178 0
>> 572071 55797 34595650 1 0 178 0
>> *OUCH, IPFW HURTS..
>> loading ipfw, and adding one ipfw rule allow ip from any to any drops
>> 100Kpps off :/ what's up with THAT?
>> unloaded ipfw module and back 100kpps more again, that's not right
>> with ONE rule.. :/
> ipfw need sto gain a lock on hte firewall before running,
> and is quite complex.. I can believe it..
> in FreeBSD 4.8 I was able to use ipfw and filter 1Gb between two
> interfaces (bridged) but I think it has slowed down since then due to
> the SMP locking.
>> em0 taskq is still jumping cpus.. is there any way to lock it to one
>> cpu or is this just a function of ULE
>> running a tar czpvf all.tgz * and seeing if pps changes..
>> negligible.. guess scheduler is doing it's job at least..
>> Hmm. even when it's getting 50-60k errors per second on the interface
>> I can still SCP a file through that interface although it's not
>> fast.. 3-4MB/s..
>> You know, I wouldn't care if it added 5ms latency to the packets when
>> it was doing 1mpps as long as it didn't drop any.. Why can't it do
>> that? Queue them up and do them in bigggg chunks so none are
>> 32 bit system is compiling now.. won't do > 400kpps with GENERIC
>> kernel, as with 64 bit did 450k with GENERIC, although that could be
>> the difference between opteron 270 and opteron 2212..
>> freebsd-net at freebsd.org mailing list
>> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
More information about the freebsd-net