pci buses missing from suspend/resume powerstate

[J.R. Oldroyd]

On Sat, 2 Apr 2011 11:29:58 -0400, "J.R. Oldroyd" <fbsd at opal.com> wrote:
>
> [suspend/resume of network cards]
> 
>     pcib0
>       pci0
>         hostb0
>         pcib1
>           pci1
>             vgapci0
>               vgapm0
>                 scpm0
>               acpi_video0
>               drm0
>             hdac1
>               pcm2
>         pcib2
>           pci2
>             ath0
>         pcib3
>           pci3
>             re0
>               miibus0
>                 rgephy0
>         ...

I had been working from this:

on suspend:
kernel: pci0:1:5:0: Transition from D0 to D3		# vgapci
kernel: pci0:1:5:1: Transition from D0 to D3		# hdac1
kernel: pci0:2:0:0: Transition from D0 to D3		# ath0
kernel: pci0:3:0:0: Transition from D0 to D3		# re0
kernel: pci0:0:18:2: Transition from D0 to D3		# ohci0
kernel: pci0:0:19:2: Transition from D0 to D3		# ohci1
kernel: pci0:0:20:2: Transition from D0 to D3		# hdac0

but on resume, only:
kernel: pci0:0:18:2: Transition from D3 to D0
kernel: pci0:0:19:2: Transition from D3 to D0

which had led me to think the buses were not being resumed.

Looking further at this, two things come to light:
	- they ARE being suspended using set_powerstate to D3, but
	  after the resume, their powerstate still shows as D0, so
	  set_powerstate is not being re-invoked to actually set D0
	  again
	- the PCI-PCI bridges pcib1, pcib2, pcib3 have hdrtype==1 and
	  so are also excluded from the set_powerstate (don't know if
	  that's OK or not)

But, overriding those checks and forcing set_powerstate D0 to be invoked
for all devices still doesn't get ath0 or re0 working after the resume.

I have an older but similar laptop model with the same configuration of
ath0 and re0 also on secondary PCI buses pci2 and pci3 and that one does
suspend/resume properly.  But on the newer laptop:

kernel: ath0: unable to reset hardware; hal status 3
kernel: re0: reset never completed!
kernel: re0: PHY write failed
last message repeated 4 times

So this problem is specific to this newer hardware.  Perhaps related to
the PCI-PCI bridges?  pcib2 and pcib3 are both devid=96xx1022... perhaps
some odd quirk handling is needed here?

	-jr