[Bug 227100] [epair] epair interface stops working when it reaches the hardware queue limit

[bugzilla-noreply at freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=227100

            Bug ID: 227100
           Summary: [epair] epair interface stops working when it reaches
                    the hardware queue limit
           Product: Base System
           Version: CURRENT
          Hardware: amd64
                OS: Any
            Status: New
          Severity: Affects Only Me
          Priority: ---
         Component: kern
          Assignee: freebsd-bugs at FreeBSD.org
          Reporter: reshadpatuck1 at gmail.com

Created attachment 191964
  --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=191964&action=edit
the output of netstat and dtrace

When the epair interface reaches the hardware queue limit, epairs stop
transferring data.

This bug refers to this mailing list conversation
https://lists.freebsd.org/pipermail/freebsd-net/2018-March/050077.html

So far using the patch/if_epair source file attached to this bug we can tell
that the error occurs in this block of code

```
        if ((epair_dpcpu->epair_drv_flags & IFF_DRV_OACTIVE) != 0) {
                /*
                 * Our hardware queue is full, try to fall back
                 * queuing to the ifq but do not call ifp->if_start.
                 * Either we are lucky or the packet is gone.
                 */
                IFQ_ENQUEUE(&ifp->if_snd, m, error);
                if (!error)
                        (void)epair_add_ifp_for_draining(ifp);

                SDT_PROBE3(if_epair, transmit, epair_transmit_locked, enqueued,
                                ifp, m, error);
                return (error);
        }
```

Where the value of the 'error' is 55.

Setting 'net.link.epair.netisr_maxqlen' to a very small value makes this occur
faster.

This issue seems to be happening in the wild only on one of my servers.
Other servers under more load in different environments do not seem to exhibit
this behaviour.

@Kristof please chime in if I have missed something out.

Attached:
- commands.txt
- epair-sdt-diff.patch 
- epair_transmit_locked:enqueued-error-code.d
- if_epair.c

-- 
You are receiving this mail because:
You are the assignee for the bug.