5.4 network performance

[John Baldwin]

On Tuesday 31 May 2005 05:19 pm, Singh, Vijay wrote:
> > On Tuesday 31 May 2005 01:37 pm, Singh, Vijay wrote:
> > > Hello. I am trying to benchmark 5.4 performance for a
> >
> > company project.
> >
> > > I've got:
> > >
> > > FreeBSD 5.4-RELEASE #0: Fri May 27 20:52:58 PDT 2005
> > >     admin at netpc13.lab.netapp.com:/usr/obj/usr/src/sys/SMP
> > > Timecounter "i8254" frequency 1193182 Hz quality 0
> > > CPU: Intel(R) Xeon(TM) CPU 2.40GHz (2395.92-MHz 686-class CPU)
> > >   Origin = "GenuineIntel"  Id = 0xf27  Stepping = 7
> >
> > Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,P
> >
> > > GE ,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,S
> > > SE2,SS,HTT,TM,PBE>
> > >   Hyperthreading: 2 logical CPUs
> > > real memory  = 1073676288 (1023 MB)
> > > avail memory = 1041121280 (992 MB)
> > > ACPI APIC Table: <RCC    GCHE    >
> > > FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs  cpu0
> >
> > (BSP): APIC
> >
> > > ID:  0
> > >  cpu1 (AP): APIC ID:  1
> > >  cpu2 (AP): APIC ID:  2
> > >  cpu3 (AP): APIC ID:  3
> > >
> > > em0: <Intel(R) PRO/1000 Network Connection, Version - 1.7.35> port
> > > 0xc800-0xc83f mem 0xfe8c0000-0xfe8dffff irq 18 at device 2.0 on pci1
> > > em1: <Intel(R) PRO/1000 Network Connection, Version - 1.7.35> port
> > > 0xd000-0xd03f mem 0xfe8e0000-0xfe8fffff irq 19 at device 2.1 on pci1
> > >
> > > The UP version of the builds is able to deliver close to
> >
> > line rate on
> >
> > > these 2 interfaces. However the SMP build (with WITNESS and
> >
> > INVARIANTS
> >
> > > disabled) gives me half the line rate on them. I am using netperf.
> > >
> > > /opt/netperf/netperf -H x.x.x.x -f m -l 120
> > >
> > > Socket Socket  Message  Elapsed
> > > Size   Size    Size     Time     Throughput
> > > bytes  bytes   bytes    secs.    10^6bits/sec
> > >
> > > 65536  16384  16384    120.00    429.14
> > > 65536  16384  16384    120.00    501.74
> > >
> > > Is there something I can do to make the system scale? I
> >
> > cannot move to
> >
> > > -CURRENT, but I can try pulling patches.
> >
> > Does UP with 'device apic' also show poor performance?  Try
> > disabling USB support in the kernel as some Intel server
> > motherboards have a "feature" that our APIC code trips over
> > that can hurt performance by adding lots of stray interrupts
> > on one of the USB controllers.
>
> Thanks. I have the following in GENERIC, do I need to comment out all?
> Is the problem you mention responsible for the "Interrupt storm
> detected" messages? Also I see some ACPI messages (below). Could that
> also be a problem?
>
> # USB support
> device          uhci            # UHCI PCI->USB interface
> device          ohci            # OHCI PCI->USB interface
> #device         ehci            # EHCI PCI->USB interface (USB 2.0)
> device          usb             # USB Bus (required)
> #device         udbp            # USB Double Bulk Pipe devices
> device          ugen            # Generic
> device          uhid            # "Human Interface Devices"
> device          ukbd            # Keyboard
> device          ulpt            # Printer
> device          umass           # Disks/Mass storage - Requires scbus
> and da
> device          ums             # Mouse
> device          urio            # Diamond Rio 500 MP3 player
> device          uscanner        # Scanners
> # USB Ethernet, requires mii
> device          aue             # ADMtek USB Ethernet
> device          axe             # ASIX Electronics USB Ethernet
> device          cdce            # Generic USB over Ethernet
> device          cue             # CATC USB Ethernet
> device          kue             # Kawasaki LSI USB Ethernet
> device          rue             # RealTek RTL8150 USB Ethernet

Yes, comment all of that out.  Disabling USB should solve your interrupt 
storm, yes.

>     ACPI-0698: *** Warning: Type override - [DEB_] had invalid type
> (Integer) for Scope operator, changed to (Scope)
>     ACPI-0698: *** Warning: Type override - [MLIB] had invalid type
> (Integer) for Scope operator, changed to (Scope)
> 	[...]
>     ACPI-0698: *** Warning: Type override - [KBC_] had invalid type
> (Integer) for Scope operator, changed to (Scope)
>     ACPI-0698: *** Warning: Type override - [OEM_] had invalid type
> (Integer) for Scope operator, changed to (Scope)

I don't think you need to worry about these.

-- 
John Baldwin <jhb at FreeBSD.org>  <><  http://www.FreeBSD.org/~jhb/
"Power Users Use the Power to Serve"  =  http://www.FreeBSD.org