USB bus setup error with 7.1 + PAE

[Charles Owens]

Hans Petter Selasky wrote:
> On Wednesday 02 December 2009 18:08:47 Charles Owens wrote:
>> [originally posted to -hardware]
>>
>> Hello,
>>
>> I'm having problems with the functioning of USB keyboards with an IBM
>> System x3550 M2 (new quad core Xeon, 6 GB RAM).  With a FreeBSD 7.1
>> PAE-enabled kernel, initialization of some of the USB buses fails.  From
>> an external perspective, this results in the rear USB port being
>> non-functional, while the two front USB ports work fine.  For some
>> reason, though, when booting the same PAE kernel from a CD none of the
>> USB ports function.  It may be worth noting that we have four other
>> hardware platforms (different IBM, HP, and OEM Intel) that do not
>> exhibit this behavior (ie. PAE and USB seem okay with each other).
>>
>> In previous posting (
>> http://docs.freebsd.org/cgi/mid.cgi?4B0D14CB.1010000 ) I included output
>> of usbdevs and boot log when running the PAE-enabled kernel.   Note the
>> "root hub error" messages in boot log.
>>
>> Below I'll repeat the same info when running a non-PAE kernel.
>>
>> Any thoughts as to what the story is here... and suggestions for
>> resolution?   Thanks in advance.  (We expect in next few days to test
>> with 8.0 for comparison, but at this point much prefer to see matter
>> resolved in 7.x)
>>
>>
>> [root at dmz55 /usr/home/muck]# usbdevs -v
>> Controller /dev/usb0:
>> addr 1: full speed, self powered, config 1, UHCI root hub(0x0000),
>> Intel(0x0000), rev 1.00
>>  port 1 powered
>>  port 2 addr 2: low speed, power 100 mA, config 1, Comfort Curve
>> Keyboard 2000(0x00dd), Microsoft(0x045e), rev 1.73
>> Controller /dev/usb1:
>> addr 1: full speed, self powered, config 1, UHCI root hub(0x0000),
>> Intel(0x0000), rev 1.00
>>  port 1 powered
>>  port 2 powered
>> Controller /dev/usb2:
>> addr 1: high speed, self powered, config 1, EHCI root hub(0x0000),
>> Intel(0x0000), rev 1.00
>>  port 1 powered
>>  port 2 powered
>>  port 3 powered
>>  port 4 powered
>>  port 5 powered
>>  port 6 powered
>> Controller /dev/usb3:
>> addr 1: full speed, self powered, config 1, UHCI root hub(0x0000),
>> Intel(0x0000), rev 1.00
>>  port 1 powered
>>  port 2 addr 2: full speed, power 100 mA, config 1, RNDIS/CDC
>> ETHER(0x4010), IBM(0x04b3), rev 2.15
>> Controller /dev/usb4:
>> addr 1: full speed, self powered, config 1, UHCI root hub(0x0000),
>> Intel(0x0000), rev 1.00
>>  port 1 powered
>>  port 2 powered
>> Controller /dev/usb5:
>> addr 1: full speed, self powered, config 1, UHCI root hub(0x0000),
>> Intel(0x0000), rev 1.00
>>  port 1 powered
>>  port 2 powered
>> Controller /dev/usb6:
>> addr 1: high speed, self powered, config 1, EHCI root hub(0x0000),
>> Intel(0x0000), rev 1.00
>>  port 1 addr 2: high speed, power 2 mA, config 2, IBM Composite
>> Device-0(0x4012), IBM(0x04b3), rev 0.00
>>  port 2 powered
>>  port 3 powered
>>  port 4 powered
>>  port 5 powered
>>  port 6 addr 3: high speed, power 450 mA, config 1, product
>> 0x1a00(0x1a00), vendor 0x2001(0x2001), rev 10.01
>>
>>
>>
>>
>> Copyright (c) 1992-2009 The FreeBSD Project.
>> Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
>>         The Regents of the University of California. All rights reserved.
>> FreeBSD is a registered trademark of The FreeBSD Foundation.
>> FreeBSD 7.1-RELEASE-p8 #0: Wed Nov 18 11:22:57 EST 2009
>>     muck at dmz55.greatbaysoftware.com:/usr/obj/usr/src/sys/DMZ55
>> Timecounter "i8254" frequency 1193182 Hz quality 0
>> CPU: Intel(R) Xeon(R) CPU           E5504  @ 2.00GHz (2000.08-MHz
>> 686-class CPU)
>>   Origin = "GenuineIntel"  Id = 0x106a5  Stepping = 5
>>
>> Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MC
>> A,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
>>
>> Features2=0x9ce3bd<SSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,
>> DCA,SSE4.1,SSE4.2,POPCNT> AMD Features=0x28100000<NX,RDTSCP,LM>
>>   AMD Features2=0x1<LAHF>
>>   Cores per package: 8
>>   Logical CPUs per core: 2
>> real memory  = 2137583616 (2038 MB)
>> avail memory = 2081480704 (1985 MB)
>> ACPI APIC Table: <IBM    THURLEY >
>> FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
>>  cpu0 (BSP): APIC ID:  0
>>  cpu1 (AP): APIC ID:  2
>>  cpu2 (AP): APIC ID:  4
>>  cpu3 (AP): APIC ID:  6
>> ioapic0 <Version 2.0> irqs 0-23 on motherboard
>> ioapic1 <Version 2.0> irqs 24-47 on motherboard
>> kbd1 at kbdmux0
>> ath_hal: 0.9.20.3 (AR5210, AR5211, AR5212, RF5111, RF5112, RF2413, RF5413)
>> acpi0: <IBM THURLEY> on motherboard
>> acpi0: [ITHREAD]
>> acpi0: Power Button (fixed)
>> Timecounter "HPET" frequency 14318180 Hz quality 900
>> Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
>> acpi_timer0: <24-bit timer at 3.579545MHz> port 0x588-0x58b on acpi0
>> pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
>> pci0: <ACPI PCI bus> on pcib0
>> pcib1: <ACPI PCI-PCI bridge> irq 28 at device 1.0 on pci0
>> pci11: <ACPI PCI bus> on pcib1
>> pci11: <network, ethernet> at device 0.0 (no driver attached)
>> pci11: <network, ethernet> at device 0.1 (no driver attached)
>> pcib2: <PCI-PCI bridge> irq 29 at device 2.0 on pci0
>> pci16: <PCI bus> on pcib2
>> pci16: <network, ethernet> at device 0.0 (no driver attached)
>> pci16: <network, ethernet> at device 0.1 (no driver attached)
>> pcib3: <ACPI PCI-PCI bridge> irq 24 at device 3.0 on pci0
>> pci21: <ACPI PCI bus> on pcib3
>> pcib4: <ACPI PCI-PCI bridge> irq 30 at device 7.0 on pci0
>> pci26: <ACPI PCI bus> on pcib4
>> pci0: <base peripheral, interrupt controller> at device 16.0 (no driver
>> attached)
>> pci0: <base peripheral, interrupt controller> at device 16.1 (no driver
>> attached)
>> pci0: <base peripheral, interrupt controller> at device 17.0 (no driver
>> attached)
>> pci0: <base peripheral, interrupt controller> at device 17.1 (no driver
>> attached)
>> pci0: <base peripheral, interrupt controller> at device 20.0 (no driver
>> attached)
>> pci0: <base peripheral, interrupt controller> at device 20.1 (no driver
>> attached)
>> pci0: <base peripheral, interrupt controller> at device 20.2 (no driver
>> attached)
>> pci0: <base peripheral, interrupt controller> at device 20.3 (no driver
>> attached)
>> pci0: <base peripheral> at device 22.0 (no driver attached)
>> pci0: <base peripheral> at device 22.1 (no driver attached)
>> pci0: <base peripheral> at device 22.2 (no driver attached)
>> pci0: <base peripheral> at device 22.3 (no driver attached)
>> pci0: <base peripheral> at device 22.4 (no driver attached)
>> pci0: <base peripheral> at device 22.5 (no driver attached)
>> pci0: <base peripheral> at device 22.6 (no driver attached)
>> pci0: <base peripheral> at device 22.7 (no driver attached)
>> uhci0: <UHCI (generic) USB controller> port 0x20a0-0x20bf irq 17 at
>> device 26.0 on pci0
>> uhci0: [GIANT-LOCKED]
>> uhci0: [ITHREAD]
>> usb0: <UHCI (generic) USB controller> on uhci0
>> usb0: USB revision 1.0
>> uhub0: <Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb0
>> uhub0: 2 ports with 2 removable, self powered
>> uhci1: <UHCI (generic) USB controller> port 0x2080-0x209f irq 18 at
>> device 26.1 on pci0
>> uhci1: [GIANT-LOCKED]
>> uhci1: [ITHREAD]
>> usb1: <UHCI (generic) USB controller> on uhci1
>> usb1: USB revision 1.0
>> uhub1: <Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb1
>> uhub1: 2 ports with 2 removable, self powered
>> ehci0: <EHCI (generic) USB 2.0 controller> mem 0x9ba21400-0x9ba217ff irq
>> 19 at device 26.7 on pci0
>> ehci0: [GIANT-LOCKED]
>> ehci0: [ITHREAD]
>> usb2: EHCI version 1.0
>> usb2: wrong number of companions (3 != 2)
>> usb2: companion controllers, 2 ports each: usb0 usb1
>> usb2: <EHCI (generic) USB 2.0 controller> on ehci0
>> usb2: USB revision 2.0
>> uhub2: <Intel EHCI root hub, class 9/0, rev 2.00/1.00, addr 1> on usb2
>> uhub2: 6 ports with 6 removable, self powered
>> pcib5: <PCI-PCI bridge> irq 16 at device 28.0 on pci0
>> pci1: <PCI bus> on pcib5
>> mpt0: <LSILogic SAS/SATA Adapter> port 0x1000-0x10ff mem
>> 0x9b910000-0x9b913fff,0x9b900000-0x9b90ffff irq 16 at device 0.0 on pci1
>> mpt0: [ITHREAD]
>> mpt0: MPI Version=1.5.20.0
>> mpt0: Capabilities: ( RAID-0 RAID-1E RAID-1 )
>> mpt0: 0 Active Volumes (2 Max)
>> mpt0: 0 Hidden Drive Members (14 Max)
>> pcib6: <PCI-PCI bridge> irq 16 at device 28.4 on pci0
>> pci6: <PCI bus> on pcib6
>> pcib7: <PCI-PCI bridge> irq 16 at device 0.0 on pci6
>> pci7: <PCI bus> on pcib7
>> vgapci0: <VGA-compatible display> mem
>> 0x9a000000-0x9affffff,0x9b800000-0x9b803fff,0x9b000000-0x9b7fffff irq 16
>> at device 0.0 on pci7
>> uhci2: <UHCI (generic) USB controller> port 0x2060-0x207f irq 17 at
>> device 29.0 on pci0
>> uhci2: [GIANT-LOCKED]
>> uhci2: [ITHREAD]
>> usb3: <UHCI (generic) USB controller> on uhci2
>> usb3: USB revision 1.0
>> uhub3: <Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb3
>> uhub3: 2 ports with 2 removable, self powered
>> uhci3: <UHCI (generic) USB controller> port 0x2040-0x205f irq 18 at
>> device 29.1 on pci0
>> uhci3: [GIANT-LOCKED]
>> uhci3: [ITHREAD]
>> usb4: <UHCI (generic) USB controller> on uhci3
>> usb4: USB revision 1.0
>> uhub4: <Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb4
>> uhub4: 2 ports with 2 removable, self powered
>> uhci4: <UHCI (generic) USB controller> port 0x2020-0x203f irq 19 at
>> device 29.2 on pci0
>> uhci4: [GIANT-LOCKED]
>> uhci4: [ITHREAD]
>> usb5: <UHCI (generic) USB controller> on uhci4
>> usb5: USB revision 1.0
>> uhub5: <Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb5
>> uhub5: 2 ports with 2 removable, self powered
>> ehci1: <EHCI (generic) USB 2.0 controller> mem 0x9ba21000-0x9ba213ff irq
>> 17 at device 29.7 on pci0
>> ehci1: [GIANT-LOCKED]
>> ehci1: [ITHREAD]
>> usb6: EHCI version 1.0
>> usb6: companion controllers, 2 ports each: usb3 usb4 usb5
>> usb6: <EHCI (generic) USB 2.0 controller> on ehci1
>> usb6: USB revision 2.0
>> uhub6: <Intel EHCI root hub, class 9/0, rev 2.00/1.00, addr 1> on usb6
>> uhub6: 6 ports with 6 removable, self powered
>> ukbd0: <IBM IBM Composite Device-0, class 0/0, rev 2.00/0.00, addr 2> on
>> uhub6
>> kbd2 at ukbd0
>> ums0: <IBM IBM Composite Device-0, class 0/0, rev 2.00/0.00, addr 2> on
>> uhub6
>> ums0: X report 0x0002 not supported
>> device_attach: ums0 attach returned 6
>> ums0: <IBM IBM Composite Device-0, class 0/0, rev 2.00/0.00, addr 2> on
>> uhub6
>> ums0: 3 buttons and Z dir.
>> umass0: <IBM IBM Composite Device-0, class 0/0, rev 2.00/0.00, addr 2>
>> on uhub6
>> umass1: <IBM IBM Composite Device-0, class 0/0, rev 2.00/0.00, addr 2>
>> on uhub6
>> axe0: <vendor 0x2001 product 0x1a00, class 2/0, rev 2.00/10.01, addr 3>
>> on uhub6
>> axe0: AX88172, bufsz 1536, boundary 64
>> miibus0: <MII bus> on axe0
>> rlphy0: <IC Plus 10/100 PHY> PHY 3 on miibus0
>> rlphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
>> axe0: WARNING: using obsoleted if_watchdog interface
>> axe0: WARNING: using obsoleted IFF_NEEDSGIANT flag
>> axe0: Ethernet address: 00:80:c8:38:15:ff
>> pcib8: <PCI-PCI bridge> at device 30.0 on pci0
>> pci31: <PCI bus> on pcib8
>> isab0: <PCI-ISA bridge> at device 31.0 on pci0
>> isa0: <ISA bus> on isab0
>> atapci0: <Intel ATA controller> port
>> 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x20f0-0x20ff,0x20e0-0x20ef irq 16
>> at device 31.2 on pci0
>> ata0: <ATA channel 0> on atapci0
>> ata0: [ITHREAD]
>> ata1: <ATA channel 1> on atapci0
>> ata1: [ITHREAD]
>> pci0: <serial bus, SMBus> at device 31.3 (no driver attached)
>> atapci1: <Intel ATA controller> port
>> 0x2108-0x210f,0x2124-0x2127,0x2100-0x2107,0x2120-0x2123,0x20d0-0x20df,0x20c
>> 0-0x20cf irq 21 at device 31.5 on pci0
>> atapci1: [ITHREAD]
>> ata2: <ATA channel 0> on atapci1
>> ata2: [ITHREAD]
>> ata3: <ATA channel 1> on atapci1
>> ata3: [ITHREAD]
>> sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on
>> acpi0
>> sio0: type 16550A, console
>> sio0: [FILTER]
>> sio1: <16550A-compatible COM port> port 0x2f8-0x2ff irq 3 on acpi0
>> sio1: type 16550A
>> sio1: [FILTER]
>> cpu0: <ACPI CPU> on acpi0
>> est0: <Enhanced SpeedStep Frequency Control> on cpu0
>> p4tcc0: <CPU Frequency Thermal Control> on cpu0
>> cpu1: <ACPI CPU> on acpi0
>> est1: <Enhanced SpeedStep Frequency Control> on cpu1
>> p4tcc1: <CPU Frequency Thermal Control> on cpu1
>> cpu2: <ACPI CPU> on acpi0
>> est2: <Enhanced SpeedStep Frequency Control> on cpu2
>> p4tcc2: <CPU Frequency Thermal Control> on cpu2
>> cpu3: <ACPI CPU> on acpi0
>> est3: <Enhanced SpeedStep Frequency Control> on cpu3
>> p4tcc3: <CPU Frequency Thermal Control> on cpu3
>> pmtimer0 on isa0
>> orm0: <ISA Option ROM> at iomem 0xc0000-0xc7fff pnpid ORM0000 on isa0
>> atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
>> atkbd0: <AT Keyboard> irq 1 on atkbdc0
>> kbd0 at atkbd0
>> atkbd0: [GIANT-LOCKED]
>> atkbd0: [ITHREAD]
>> ppc0: parallel port not found.
>> sc0: <System console> at flags 0x100 on isa0
>> sc0: VGA <16 virtual consoles, flags=0x100>
>> vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
>> ukbd1: <Microsoft Comfort Curve Keyboard 2000, class 0/0, rev 2.00/1.73,
>> addr 2> on uhub0
>> kbd3 at ukbd1
>> uhid0: <Microsoft Comfort Curve Keyboard 2000, class 0/0, rev 2.00/1.73,
>> addr 2> on uhub0
>> cdce0: <IBM RNDIS/CDC ETHER, class 2/0, rev 2.00/2.15, addr 2> on uhub3
>> cdce0: faking MAC address
>> cdce0: WARNING: using obsoleted IFF_NEEDSGIANT flag
>> cdce0: Ethernet address: 2a:00:00:00:00:00
>> Timecounters tick every 1.000 msec
>> acd0: CDRW <HL-DT-STCD-RW/DVD DRIVE GCC-T20N/1.00> at ata0-master UDMA33
>> da0 at mpt0 bus 0 target 1 lun 0
>> da0: <ATA WD3000BLFS-23YBU 4V04> Fixed Direct Access SCSI-5 device
>> da0: 300.000MB/s transfers
>> da0: Command Queueing Enabled
>> da0: 286102MB (585937500 512 byte sectors: 255H 63S/T 36472C)
>> da1 at mpt0 bus 0 target 2 lun 0
>> da1: <ATA WD3000BLFS-23YBU 4V04> Fixed Direct Access SCSI-5 device
>> da1: 300.000MB/s transfers
>> da1: Command Queueing Enabled
>> da1: 286102MB (585937500 512 byte sectors: 255H 63S/T 36472C)
>> SMP: AP CPU #1 Launched!
>> SMP: AP CPU #2 Launched!
>> SMP: AP CPU #3 Launched!
>> da2 at umass-sim1 bus 1 target 0 lun 0
>> da2: <Linux Virtual Floppy 0316> Removable Direct Access SCSI-0 device
>> da2: 40.000MB/s transfers
>> da2: Attempt to query device size failed: NOT READY, Medium not present
>> cd0 at umass-sim0 bus 0 target 0 lun 0
>> cd0: <Linux Virtual CD/DVD 0316> Removable CD-ROM SCSI-0 device
>> cd0: 40.000MB/s transfers
>> da3 at umass-sim1 bus 1 target 0 lun 1
>> da3: <Linux Virtual Floppy 0316> Removable Direct Access SCSI-0 device
>> da3: 40.000MB/s transfers
>> da3: Attempt to query device size failed: NOT READY, Medium not present
>> Trying to mount root from ufs:/dev/da0s1a
>>
>>
> 
> Hi,
> 
> The USB stack in 8.0-RELEASE has power save enabled by default. Maybe 
> something in that area is not wired correctly in the hardware.
> 
> You can try:
> 
> usbconfig -u X -a Y power_on
> 
> For all your HUBs and root HUBs.
> 
> --HPS

We've tested now with 8.0 and 8.0 does NOT have the problem (USB
keyboard functions properly with PAE enabled).

The question now, is what can be done to get this to fly with 7.1.  A
few more details regarding behavior with 7.1:

* Only _sometimes_ does the keyboard work on some USB ports.  After most
reboots it doesn't work at all on any ports.
* Weirdly, a 'axe' based USB NIC appears to always work just fine. (!)
Could that be a clue?

Thanks,

Charles