kern/187718: UDP bad performance and out-of-order packet with iperf3 and igb(4) drivers
Olivier Cochard-Labbe
olivier at cochatrd.me
Wed Mar 19 00:50:01 UTC 2014
>Number: 187718
>Category: kern
>Synopsis: UDP bad performance and out-of-order packet with iperf3 and igb(4) drivers
>Confidential: no
>Severity: non-critical
>Priority: low
>Responsible: freebsd-bugs
>State: open
>Quarter:
>Keywords:
>Date-Required:
>Class: sw-bug
>Submitter-Id: current-users
>Arrival-Date: Wed Mar 19 00:50:00 UTC 2014
>Closed-Date:
>Last-Modified:
>Originator: Olivier Cochard-Labbe
>Release: -current
>Organization:
BSD Router project
>Environment:
FreeBSD R1 11.0-CURRENT FreeBSD 11.0-CURRENT #0 r263118M: Fri Mar 14 00:02:53 CET 2014 root at orange.bsdrp.net:/usr/obj/BSDRPcur.amd64/usr/local/BSDRP/BSDRPcur/FreeBSD/src/sys/amd64 amd64
>Description:
Testing UDP throughput with iperf and iperf3 give totally different value:
With iperf:
[root at R1]~# iperf -u -c 99.99.99.100 -w65536 -l1460 -t30 -i4 -b1000m -P1
------------------------------------------------------------
Client connecting to 99.99.99.100, UDP port 5001
Sending 1460 byte datagrams
UDP buffer size: 64.0 KByte
------------------------------------------------------------
[ 3] local 99.99.99.99 port 64273 connected with 99.99.99.100 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0- 4.0 sec 373 MBytes 783 Mbits/sec
[ 3] 4.0- 8.0 sec 374 MBytes 784 Mbits/sec
[ 3] 8.0-12.0 sec 372 MBytes 780 Mbits/sec
[ 3] 12.0-16.0 sec 373 MBytes 783 Mbits/sec
[ 3] 16.0-20.0 sec 373 MBytes 783 Mbits/sec
[ 3] 20.0-24.0 sec 373 MBytes 782 Mbits/sec
[ 3] 24.0-28.0 sec 373 MBytes 783 Mbits/sec
[ 3] 0.0-30.0 sec 2.73 GBytes 783 Mbits/sec
[ 3] Sent 2010212 datagrams
[ 3] Server Report:
[ 3] 0.0-30.0 sec 2.73 GBytes 781 Mbits/sec 0.010 ms 4953/2010211 (0.25%)
[ 3] 0.0-30.0 sec 954425 datagrams received out-of-order
with iperf3.
[root at R1]~# iperf3 -u -c 99.99.99.100 -w65536 -l1460 -t30 -i4 -b1000m -P1
Connecting to host 99.99.99.100, port 5201
[ 4] local 99.99.99.99 port 43694 connected to 99.99.99.100 port 5201
[ ID] Interval Transfer Bandwidth Total Datagrams
[ 4] 0.00-4.00 sec 48.7 MBytes 102 Mbits/sec 1295380
[ 4] 4.00-8.00 sec 28.2 MBytes 59.2 Mbits/sec 1334970
[ 4] 8.00-12.00 sec 31.6 MBytes 66.4 Mbits/sec 1334995
[ 4] 12.00-16.00 sec 32.0 MBytes 67.1 Mbits/sec 1330605
[ 4] 16.00-20.00 sec 47.3 MBytes 99.2 Mbits/sec 1335097
[ 4] 20.00-24.00 sec 30.6 MBytes 64.1 Mbits/sec 1335121
[ 4] 24.00-28.00 sec 35.3 MBytes 74.0 Mbits/sec 1331542
[ 4] 28.00-30.00 sec 19.3 MBytes 81.0 Mbits/sec 663836
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 4] 0.00-30.00 sec 273 MBytes 76.4 Mbits/sec 7.755 ms 9381619/9954141 (94%)
[ 4] Sent 9954141 datagrams
>How-To-Repeat:
Load on the generator during the iperf3 test is:
[root at R1]~# top -nCHSIzs1
last pid: 14453; load averages: 0.37, 0.19, 0.09 up 4+17:17:33 01:26:07
160 processes: 6 running, 104 sleeping, 50 waiting
Mem: 4192K Active, 26M Inact, 751M Wired, 27M Buf, 15G Free
Swap:
PID USERNAME PRI NICE SIZE RES STATE C TIME CPU COMMAND
14452 root 100 0 14300K 1880K CPU1 1 0:26 96.39% iperf3
11 root -92 - 0K 800K WAIT 1 0:07 0.88% intr{irq279: igb2:que}
11 root -92 - 0K 800K WAIT 2 0:13 0.78% intr{irq280: igb2:que}
11 root -92 - 0K 800K WAIT 3 0:05 0.68% intr{irq281: igb2:que}
11 root -92 - 0K 800K WAIT 0 0:11 0.49% intr{irq278: igb2:que}
and an extract of the iperf receiver during this test:
iperf3: OUT OF ORDER - incoming packet = 9972872 and received packet = 9983051 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 9950852 and received packet = 9983052 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 9947731 and received packet = 9983052 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 9972873 and received packet = 9983052 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 9950853 and received packet = 9983053 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 9947732 and received packet = 9983053 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 9972874 and received packet = 9983053 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 9950854 and received packet = 9983054 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 9947733 and received packet = 9983054 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 9972875 and received packet = 9983054 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 9950855 and received packet = 9983055 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 9947734 and received packet = 9983055 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 9972876 and received packet = 9983055 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 9950856 and received packet = 9983056 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 9947735 and received packet = 9983056 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 9972877 and received packet = 9983056 AND SP = 5
iperf3: OUT OF ORDER - incoming packet = 9950857 and received packet = 9983057 AND SP = 5
[ 5] 30.00-30.22 sec 25.6 MBytes 957 Mbits/sec 55.317 ms 41962/46554 (90%)
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 5] 0.00-30.22 sec 318 MBytes 88.3 Mbits/sec 55.317 ms 9425274/9983057 (94%)
[SUM] 0.0-30.2 sec 9425274 datagrams received out-of-order
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
>Fix:
>Release-Note:
>Audit-Trail:
>Unformatted:
More information about the freebsd-bugs
mailing list