5.4 network performance

[John Baldwin]

On Tuesday 31 May 2005 01:37 pm, Singh, Vijay wrote:
> Hello. I am trying to benchmark 5.4 performance for a company project.
> I've got:
>
> FreeBSD 5.4-RELEASE #0: Fri May 27 20:52:58 PDT 2005
>     admin at netpc13.lab.netapp.com:/usr/obj/usr/src/sys/SMP
> Timecounter "i8254" frequency 1193182 Hz quality 0
> CPU: Intel(R) Xeon(TM) CPU 2.40GHz (2395.92-MHz 686-class CPU)
>   Origin = "GenuineIntel"  Id = 0xf27  Stepping = 7
>
> Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE
> ,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,S
> SE2,SS,HTT,TM,PBE>
>   Hyperthreading: 2 logical CPUs
> real memory  = 1073676288 (1023 MB)
> avail memory = 1041121280 (992 MB)
> ACPI APIC Table: <RCC    GCHE    >
> FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
>  cpu0 (BSP): APIC ID:  0
>  cpu1 (AP): APIC ID:  1
>  cpu2 (AP): APIC ID:  2
>  cpu3 (AP): APIC ID:  3
>
> em0: <Intel(R) PRO/1000 Network Connection, Version - 1.7.35> port
> 0xc800-0xc83f mem 0xfe8c0000-0xfe8dffff irq 18 at device 2.0 on pci1
> em1: <Intel(R) PRO/1000 Network Connection, Version - 1.7.35> port
> 0xd000-0xd03f mem 0xfe8e0000-0xfe8fffff irq 19 at device 2.1 on pci1
>
> The UP version of the builds is able to deliver close to line rate on
> these 2 interfaces. However the SMP build (with WITNESS and INVARIANTS
> disabled) gives me half the line rate on them. I am using netperf.
>
> /opt/netperf/netperf -H x.x.x.x -f m -l 120
>
> Socket Socket  Message  Elapsed
> Size   Size    Size     Time     Throughput
> bytes  bytes   bytes    secs.    10^6bits/sec
>
> 65536  16384  16384    120.00    429.14
> 65536  16384  16384    120.00    501.74
>
> Is there something I can do to make the system scale? I cannot move to
> -CURRENT, but I can try pulling patches.

Does UP with 'device apic' also show poor performance?  Try disabling USB 
support in the kernel as some Intel server motherboards have a "feature" that 
our APIC code trips over that can hurt performance by adding lots of stray 
interrupts on one of the USB controllers.

-- 
John Baldwin <jhb at FreeBSD.org>  <><  http://www.FreeBSD.org/~jhb/
"Power Users Use the Power to Serve"  =  http://www.FreeBSD.org