[Bug 254514] vnet: /sbin/ifconfig epair10b vnet $name getting stuck if one CPU is busy

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254514

crest@rlwinm.de changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |crest@rlwinm.de

--- Comment #7 from crest@rlwinm.de ---
While adding multi-queue support to my under development if_pair(4) driver (the
layer 3 alternative to if_epair(4)) I encountered a worse version of this,
because I used a thread pool with one pinned thread per CPU. The problem was
that that it's possible for lower priority threads on other CPUs to keep an
interface queue permanently busy by producing mbufs faster than the single high
priority consumer thread dedicated to the queue can consume them. This starves
all lower priority threads on the same CPU. This includes all userspace threads
and kernel threads with a priority lower than device/network like those
updating the tcp timestamps. Since in a GENERIC kernel per epair interface
thread isn't pinned it will eventually unwedge, but that makes it even harder
to debug.

My solution was to have the high priority thread voluntarily yield the CPU with
kern_yield() at userspace priority. After yielding the thread has to restore
its original priority. I decided to yield every 64 packets and if the tick
counter was incremented since the last the the thread yielded the CPU
voluntarily. Both are very cheap to check and protect against different
variants of the starvation problem (processing to many packets at once between
yielding and packet processing being too slow per packet). Expect a slightly
reduced throughput with only one or two flows, but the fairness makes the
system feel more responsive and it scales far better with multiple threads.

-- 
You are receiving this mail because:
You are on the CC list for the bug.