Intel 10Gb

[Andrew Gallatin]

Alexander Sack wrote:

>> To use DCA you need:
>>
>> - A DCA driver to talk to the IOATDMA/DCA pcie device, and obtain the tag
>>        table
>> - An interface that a client device (eg, NIC driver) can use to obtain
>>        either the tag table, or at least the correct tag for the CPU
>>        that the interrupt handler is bound to.  The basic support in
>>        a NIC driver boils down to something like:
>>
>> nic_interrupt_handler()
>> {
>>  if (sc->dca.enabled && (curcpu != sc->dca.last_cpu)) {
>>     sc->dca.last_cpu = curcpu;
>>     tag = dca_get_tag(curcpu);
>>     WRITE_REG(sc, DCA_TAG, tag);
>>  }
>> }
> 
> Drew, at least in the Intel documentation, it seems the NIC uses the
> LAPIC id to tell the PCIe TLPs where to put inbound NIC I/O (in the
> TLP the DCA info is stored) to the appropriate core's cache.  i.e. the
> heuristic you gave above is more granular than what I think Intel

The pseudo-code above was intended to be the MSI-X interrupt handler
for a single queue, not some dispatcher for multiple queues.
Sorry that wasn't clear.  So yes, the DCA tag value may be different
per queue.

> does.  I could be wrong, maybe Jack can chime in and correct me.  But
> it seems with Intel chipsets it is a per queue parameter which allows
> you to bind a core cache's to a queue via DCA.  The added piece to
> this for at least bpf(4) consumers is to have bpf(4) subscribe to
> these queues AND to allow an interface for libpcap applications to
> know where what queue is on what core and THEN bind to it.

Yes, everything associated with a queue must be bound to the same
core (or at least to cores which share a cache).

Drew