Issue with epoch_drain_callbacks and unloading iavf(4) [using iflib]

[Eric Joyner]

On Wed, Jan 29, 2020 at 1:41 PM Hans Petter Selasky <hps at selasky.org> wrote:

> On 2020-01-29 22:30, Eric Joyner wrote:
> > Hi freebsd-net,
> >
> > We've encountered an issue with unloading the iavf(4) driver on FreeBSD
> > 12.1 (and stable). On a VM with two iavf(4) interfaces, if we send heavy
> > traffic to iavf1 and try to kldunload the driver, the kldunload process
> > hangs on iavf0 until iavf1 stops receiving traffic.
> >
> > After some debugging, it looks like epoch_drain_callbacks() [via
> > if_detach_internal()] tries to switch CPUs to run on one that iavf1 is
> > using for RX processing, but since iavf1 is busy, it can't make the
> switch,
> > so cpu_switch() just hangs and nothing happens until iavf1's RX thread
> > stops being busy.
> >
> > I can work around this by inserting a kern_yield(PRI_USER) somewhere in
> one
> > of the iavf txrx functions that iflib calls into (e.g.
> > iavf_isc_rxd_available), but that's not a proper fix. Does anyone know
> what
> > to do to prevent this from happening?
> >
> > Wildly guessing, does maybe epoch_drain_callbacks() need a higher
> priority
> > than the PI_SOFT used in the group taskqueues used in iflib's RX
> processing?
> >
>
> Hi,
>
> Which scheduler is this? ULE or BSD?
>
> EPOCH(9) expects some level of round-robin scheduling on the same
> priority level. Setting a higher priority on EPOCH(9) might cause epoch
> to start spinning w/o letting the lower priority thread which holds the
> EPOCH() section to finish.
>
> --HPS
>
>
Hi Hans,

kern.sched.name gives me "ULE"

- Eric