Can net.iflib.min_tx_latency=0 introduce an unbounded amount of latency?

[Ryan Stone]

Thanks for the analysis.  I haven't had much of an opportunity to get
into the guts of iflib and I'm not very successful at following the
round trips through mpring yet.

On Mon, Mar 23, 2020 at 12:50 AM Patrick Kelsey <pkelsey at freebsd.org> wrote:
>
>
>
> On Sun, Mar 22, 2020 at 2:31 PM Ryan Stone <rysto32 at gmail.com> wrote:
>>
>> I've been tracking down a bug at work that appears to be due to
>> excessive latency introduced on the TX side of a TCP connection.  In
>> looking through the iflib code, I noticed the tunable
>> net.iflib.min_tx_latency.  My reading of the iflib code is that if a
>> packet is enqueued to a tx ring but not posted (when
>> min_tx_latency=0), then iflib can introduce an unbounded amount of
>> latency if the ring is idle.  Is my reading of the code correct?
>>
>
> Let's see...in the case described above, the next iflib_timer() invocation (max 500ms later) should find that txq->ift_db_pending is non-zero, which will cause it to enqueue the TX task, which will invoke _task_fn_tx(), which will then enqueue the special marker (void **)&txq, and one way or another, iflib_txq_drain() will be called.
>
> Looking at iflib_txq_drain() it appears that the first call to iflib_txd_db_check() should post the pending packets either because a non-zero number of slots were just reclaimed, or the check for txq->ift_db_pending >= TXQ_MAX_DB_DEFERRED(...) is satisfied, unless there are too many posted packets in the hardware queue that aren't being drained by the hardware (this would result in the in_use parameter remaining elevated such that a small number of pending packets would not exceed the threshold, and also no slot reclaim occurring).
>
> -Patrick