igb(4) Pondering a bind to cpu patch

[Juli Mallett]

On Tue, Apr 24, 2012 at 17:11, Sean Bruno <seanbru at yahoo-inc.com> wrote:
> http://people.freebsd.org/~sbruno/if_igb.c.txt
>
> 8 core machine
> 2 igb(4) interfaces
> set num_queues=4
>
> igb0:0 --> cpu0
> [...]
> igb1:0 --> cpu0
> [...]
>
> I suspect, that we need a static global to keep track of what cpu last
> was last bound to a queue.  My patch does do this, but I don't know if
> its the right thing.
>
> Since I'm doing multiple interfaces, I need to make sure I don't
> schedule a queue to a non existent cpu, so take a modulo of the counter
> and the number of cpus in the box.

This seems like a plausible approach, and certainly much more DWIM
than what I've done in the past, which is to use cpuset with -x to
bind each IRQ to a core by hand.  If there were a way for interfaces
to export queue information including any relevant IRQs, it would be
easy to make a frontend that would use cpuset in a usable way, but
right now that's the solution I've found most tenable and uninvasive.
The question here is what behavior to assume the user wants when they
restrict the number of queues, which is why putting the policy bits
into userland would be preferable to this sort of scheme, I suppose.