svn commit: r232403 - in head/sys: dev/acpica dev/cardbus dev/pci powerpc/ofw sparc64/pci

[John Baldwin]

On Friday, March 02, 2012 5:38:07 pm Marius Strobl wrote:
> On Fri, Mar 02, 2012 at 05:18:37PM -0500, John Baldwin wrote:
> > On Friday, March 02, 2012 4:45:52 pm Marius Strobl wrote:
> > > On Fri, Mar 02, 2012 at 08:38:05PM +0000, John Baldwin wrote:
> > > > Author: jhb
> > > > Date: Fri Mar  2 20:38:04 2012
> > > > New Revision: 232403
> > > > URL: http://svn.freebsd.org/changeset/base/232403
> > > > 
> > > > Log:
> > > >   - Add a bus_dma tag to each PCI bus that is a child of a Host-PCI bridge.
> > > >     The tag enforces a single restriction that all DMA transactions must not
> > > >     cross a 4GB boundary.  Note that while this restriction technically only
> > > >     applies to PCI-express, this change applies it to all PCI devices as it
> > > >     is simpler to implement that way and errs on the side of caution.
> > > 
> > > Hrm, wouldn't it have been more appropriate to implement this in the
> > > Host-PCI bridges instead? Probably for anything but x86 this would
> > > allow to easily distinguish at least between PCI and PCI-Express capable
> > > bus hierarchies and would move the PAE workaround to the MD bits to
> > > where it belongs.
> > 
> > The PAE workaround is dying as I've already fixed HEAD to make boundary arguments
> > use a bus_addr_t.  However, I committed it in this fashion so it can be MFC'd
> > since I can't merge the boundary KBI change.
> > 
> > The reason I did not do this in the Host-PCI bridge drivers is that we would have
> > to do this in umpteen different drivers, and all for a change that is mandated by
> > the PCI spec and not specific to the various specs the Host-PCI bridge drivers
> > all cater to (OFW, ACPI, MPTable, various and sundry embedded platforms, etc.).
> > 
> 
> Well, on !x86 we don't have generic Host-PCI bridge drivers based on
> what you call specs here, but rather specific drivers for individual
> Host-PCI, Host-PCI-X and Host-PCI-Express chips. Whether the PCI*
> buses beneath these are enumerated using FDT, OFW etc. doesn't really
> matter in this context.

Yes, but the 4GB limit is also not specific to the things that make psycho(4)
differ from schizo(4).  We could at least have a common helper routine to
create the tag, but we'd still have a lot of duplicated code to implement
N bus_get_dma_tag methods, etc.  That type of setup is just begging to have
some random Host-PCI bridge driver forget to include the right logic.  It
seems simpler and safer to do it in one central place, so it is always
correct.

Also, while PCI-express is the only type of PCI whose specification enforces
a 4GB boundary, we have seen non-PCI-e PCI devices also need a 4GB boundary
(e.g. bge(4) devices behind an AMD HyperTransport PCI-X bridge).

Given all that, I think having older, non-PCI-e devices possibly use an
extra S/G element in the rare case that a transaction crosses a 4GB boundary
(which won't even happen with the 32-bit DVMA on sparc64 IIUC), is an ok
tradeoff to ensure this constraint is correctly defined for all platforms,
including future chipsets.

-- 
John Baldwin