Unstable local network throughput
Ben RUBSON
ben.rubson at gmail.com
Mon Aug 15 14:49:24 UTC 2016
> On 12 Aug 2016, at 00:52, Adrian Chadd <adrian.chadd at gmail.com> wrote:
>
> Which ones of these hit the line rate comfortably?
So Adrian, I ran tests again using FreeBSD 11-RC1.
I put iperf throughput in result files (so that we can classify them), as well as top -P ALL and pcm-memory.x.
iperf results : columns 3&4 are for srv1->srv2, columns 5&6 are for srv2->srv1 (both flows running at the same time).
Results, expected throughput (best first) :
11, 01, 05, 07, 06
Results, bad (best first) :
04, 02, 09, 03
Results, worst (best first) :
10, 08
00) Idle system
http://pastebin.com/raw/K1iMVHVF
01) No pinning
http://pastebin.com/raw/7J3HibX0
02) numactl -l fixed-domain-rr -m 0 -c 0
http://pastebin.com/raw/Yt7yYr0K
03) numactl -l fixed-domain-rr -m 0 -c 0
+ cpuset -l <0-11> -x <IRQ>
http://pastebin.com/raw/1FAgDUSU
04) numactl -l fixed-domain-rr -m 0 -c 0
+ cpuset -l <12-23> -x <IRQ>
http://pastebin.com/raw/fTAxrzBb
05) numactl -l fixed-domain-rr -m 1 -c 1
http://pastebin.com/raw/kuAHzKu2
06) numactl -l fixed-domain-rr -m 1 -c 1
+ cpuset -l <0-11> -x <IRQ>
http://pastebin.com/raw/tgtaZgwb
07) numactl -l fixed-domain-rr -m 1 -c 1
+ cpuset -l <12-23> -x <IRQ>
http://pastebin.com/raw/16ReuGFF
08) No pinning, default kernel (no NUMA option)
http://pastebin.com/raw/Ah74fKRx
09) default kernel (no NUMA option)
+ cpuset -l <0-11>
+ cpuset -l <0-11> -x <IRQ>
http://pastebin.com/raw/YE0PxEu8
10) default kernel (no NUMA option)
+ cpuset -l <12-23>
+ cpuset -l <12-23> -x <IRQ>
http://pastebin.com/raw/RPh8aM49
11) No pinning, default kernel (no NUMA option), NUMA BIOS disabled
http://pastebin.com/raw/LyGcLKDd
Ben
More information about the freebsd-net
mailing list