svn commit: r205221 - head/sys/dev/bge

[Bruce Evans]

On Wed, 17 Mar 2010, Andrew Gallatin wrote:

> Pyun YongHyeon wrote:
>
>>   Revert r205090.
>>   It's hard to know when the mail box register write will get flushed to
>>   the hardware and it may take longer.
>>     Pointed out by:	scottl
>
> I may be mis-reading the code, but it looks like the mailbox
> register is in memory space, which should be flushed immediately
> unless write-combining is enabled on the region.  The bge
> driver does not seem to be setting up write combining.
> Is the concern that something may enable write combining
> behind your back?  In that case, a wmb() could act as a
> serializing instruction and flush the WC buffers.

We want writes to the PCI bus to be efficient.  Normally (?) writes
to bge registers appear to be several times faster than reads.  I don't
know if this depends on write combining but think it depends on write
buffering which can delay the write to the hardware by about the
difference between the read time and the time to write to the bufer.
Any forcing of serialization or timing would presumably lose the
benefits of the buffer.

> Or is it something completely different? Eg, maybe the chip
> polls the mailboxes at some regular interval, and it doesn't
> notice a write immediately. So writing earlier gives a better chance
> that it will see the new value sooner.

The old and restored strategy is to write early and then read.  The
read forces the write to the hardware, so it gives a 100% chance that
the hardware sees the write before the read (and before everything
that follows the read; accesses to the status block in fact follow the
read).  Probably these reads take even longer than most PCI reads since
they have to wait for the write that was just done, but not much can
be done about that except moving the write even earlier and/or moving
stuff that doesn't need to be serialized in between the write and the
read.

Bruce