two NIC on 2 core system (scheduling problem)

[Alexander Motin]

Bartosz Giza wrote:
> On other router based on the same hardware and software i have something 
> like that:
> 
>    10 root       1 171 ki31     0K     8K RUN    1 235.4H 78.66% idle: cpu1
>    11 root       1 171 ki31     0K     8K RUN    0 185.2H 72.12% idle: cpu0
>    20 root       1 -68    -     0K     8K -      0  48.7H 23.00% em0 taskq
>    23 root       1 -68    -     0K     8K WAIT   0  19.2H  9.67% irq16: fxp1
>    21 root       1 -68    -     0K     8K WAIT   1  28.2H  8.01% irq17: bge0
> 
> I don't know why on this router system balance over two cores. One 
> difference is that on this router i have another fxp card (3 total)

In verbose boot messages system shows that different IRQs assigned to
different APICs in round-robin fashion. So I may assume that this
IRQ->CPU mapping is static. em0's taskqueue at the same time able to
migrate CPUs as any usual process.

> Another question is why em0 taskq is eating so much cpu ? BGE interface is 
> actually one that pushes 2 times more packets than em0 and it uses about 
> half cpu comparing to em0. Is that not strange ?
> Could someone tell my why is this happening ? BGE is faster ? or maybe i can 
> tune some

The CPU time you see there includes much more then just a card handling
itself. It also includes CPU time of the most parts of network stack
used to process received packet. So if you have NAT, big firewall,
netgraph or any other CPU-hungry actions done with packets incoming via
em0 you will see such results.
Even more interesting is that if bge0 or fxp0 cards will require much
CPU time to send packet, this time will also be accounted to em0 process. :)

-- 
Alexander Motin