5.4 network performance

[Singh, Vijay]

Hi, sorry for the trouble, but even after commenting the USB lines from
GENERIC, and a "make buildkernel" I see the following in dmesg:

usb0: OHCI version 1.0, legacy support
usb0: SMM does not respond, resetting
usb0: <OHCI (generic) USB controller> on ohci0
usb0: USB revision 1.0
uhub0: (0x1166) OHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub0: 4 ports with 4 removable, self powered

Is there something else I need to do to disable USB support in the
kernel?

Br
vijay

> -----Original Message-----
> From: John Baldwin [mailto:jhb at FreeBSD.org] 
> Sent: Tuesday, May 31, 2005 2:46 PM
> To: freebsd-smp at FreeBSD.org
> Cc: Singh, Vijay
> Subject: Re: 5.4 network performance
> 
> On Tuesday 31 May 2005 05:19 pm, Singh, Vijay wrote:
> > > On Tuesday 31 May 2005 01:37 pm, Singh, Vijay wrote:
> > > > Hello. I am trying to benchmark 5.4 performance for a
> > >
> > > company project.
> > >
> > > > I've got:
> > > >
> > > > FreeBSD 5.4-RELEASE #0: Fri May 27 20:52:58 PDT 2005
> > > >     admin at netpc13.lab.netapp.com:/usr/obj/usr/src/sys/SMP
> > > > Timecounter "i8254" frequency 1193182 Hz quality 0
> > > > CPU: Intel(R) Xeon(TM) CPU 2.40GHz (2395.92-MHz 686-class CPU)
> > > >   Origin = "GenuineIntel"  Id = 0xf27  Stepping = 7
> > >
> > > 
> Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR
> > > ,P
> > >
> > > > GE ,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,S
> > > > SE2,SS,HTT,TM,PBE>
> > > >   Hyperthreading: 2 logical CPUs
> > > > real memory  = 1073676288 (1023 MB) avail memory = 
> 1041121280 (992 
> > > > MB)
> > > > ACPI APIC Table: <RCC    GCHE    >
> > > > FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs  cpu0
> > >
> > > (BSP): APIC
> > >
> > > > ID:  0
> > > >  cpu1 (AP): APIC ID:  1
> > > >  cpu2 (AP): APIC ID:  2
> > > >  cpu3 (AP): APIC ID:  3
> > > >
> > > > em0: <Intel(R) PRO/1000 Network Connection, Version - 
> 1.7.35> port 
> > > > 0xc800-0xc83f mem 0xfe8c0000-0xfe8dffff irq 18 at device 2.0 on 
> > > > pci1
> > > > em1: <Intel(R) PRO/1000 Network Connection, Version - 
> 1.7.35> port 
> > > > 0xd000-0xd03f mem 0xfe8e0000-0xfe8fffff irq 19 at device 2.1 on 
> > > > pci1
> > > >
> > > > The UP version of the builds is able to deliver close to
> > >
> > > line rate on
> > >
> > > > these 2 interfaces. However the SMP build (with WITNESS and
> > >
> > > INVARIANTS
> > >
> > > > disabled) gives me half the line rate on them. I am 
> using netperf.
> > > >
> > > > /opt/netperf/netperf -H x.x.x.x -f m -l 120
> > > >
> > > > Socket Socket  Message  Elapsed
> > > > Size   Size    Size     Time     Throughput
> > > > bytes  bytes   bytes    secs.    10^6bits/sec
> > > >
> > > > 65536  16384  16384    120.00    429.14
> > > > 65536  16384  16384    120.00    501.74
> > > >
> > > > Is there something I can do to make the system scale? I
> > >
> > > cannot move to
> > >
> > > > -CURRENT, but I can try pulling patches.
> > >
> > > Does UP with 'device apic' also show poor performance?  Try 
> > > disabling USB support in the kernel as some Intel server 
> > > motherboards have a "feature" that our APIC code trips 
> over that can 
> > > hurt performance by adding lots of stray interrupts on one of the 
> > > USB controllers.
> >
> > Thanks. I have the following in GENERIC, do I need to 
> comment out all?
> > Is the problem you mention responsible for the "Interrupt storm 
> > detected" messages? Also I see some ACPI messages (below). 
> Could that 
> > also be a problem?
> >
> > # USB support
> > device          uhci            # UHCI PCI->USB interface
> > device          ohci            # OHCI PCI->USB interface
> > #device         ehci            # EHCI PCI->USB interface (USB 2.0)
> > device          usb             # USB Bus (required)
> > #device         udbp            # USB Double Bulk Pipe devices
> > device          ugen            # Generic
> > device          uhid            # "Human Interface Devices"
> > device          ukbd            # Keyboard
> > device          ulpt            # Printer
> > device          umass           # Disks/Mass storage - 
> Requires scbus
> > and da
> > device          ums             # Mouse
> > device          urio            # Diamond Rio 500 MP3 player
> > device          uscanner        # Scanners
> > # USB Ethernet, requires mii
> > device          aue             # ADMtek USB Ethernet
> > device          axe             # ASIX Electronics USB Ethernet
> > device          cdce            # Generic USB over Ethernet
> > device          cue             # CATC USB Ethernet
> > device          kue             # Kawasaki LSI USB Ethernet
> > device          rue             # RealTek RTL8150 USB Ethernet
> 
> Yes, comment all of that out.  Disabling USB should solve 
> your interrupt storm, yes.
> 
> >     ACPI-0698: *** Warning: Type override - [DEB_] had invalid type
> > (Integer) for Scope operator, changed to (Scope)
> >     ACPI-0698: *** Warning: Type override - [MLIB] had invalid type
> > (Integer) for Scope operator, changed to (Scope)
> > 	[...]
> >     ACPI-0698: *** Warning: Type override - [KBC_] had invalid type
> > (Integer) for Scope operator, changed to (Scope)
> >     ACPI-0698: *** Warning: Type override - [OEM_] had invalid type
> > (Integer) for Scope operator, changed to (Scope)
> 
> I don't think you need to worry about these.
> 
> --
> John Baldwin <jhb at FreeBSD.org>  <><  
> http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to 
> Serve"  =  http://www.FreeBSD.org
>