pfsync errors

[Bill Marquette]

On 8/28/07, Max Laier <max at love2party.net> wrote:
> > I'm going to toy with some settings for the em(4) driver in
> > loader.conf and see if I can raise the txd and rxd descriptors since
> > we're not running on 82542 or 82543 chipped hardware we can go above
> > 256 descriptors.

Raised it to 4096 for both txd and rxd with no impact (may help
something else, who knows).

> No that's the internal work deferral queue.  Try something like the
> following in contrib/pf/net/if_pfsync.c:
>
> @@ -229,7 +229,7 @@
>         callout_init(&sc->sc_bulk_tmo, NET_CALLOUT_MPSAFE);
>         callout_init(&sc->sc_bulkfail_tmo, NET_CALLOUT_MPSAFE);
>         callout_init(&sc->sc_send_tmo, NET_CALLOUT_MPSAFE);
> -       sc->sc_ifq.ifq_maxlen = ifqmaxlen;
> +       sc->sc_ifq.ifq_maxlen = 2 * ifqmaxlen;
>         mtx_init(&sc->sc_ifq.ifq_mtx, ifp->if_xname, "pfsync send queue",
>             MTX_DEF);
>         if_attach(ifp);
>
> But there might be other reasons like timing wrt the locks.  I'll have to
> check for details.  It might also be a good idea to MFC the taskqueue
> approach from CURRENT, rather than using the callout ... that's a bit of
> work however.

That seems to have halved the error rate, not sure if it's not just
the time of day though (we're now past our peak traffic load by a few
hours).

$ netstat -s -ppfsync && sleep 1 && netstat -s -ppfsync
pfsync:
        383169 packets received (IPv4)
        0 packets received (IPv6)
                0 packets discarded for bad interface
                0 packets discarded for bad ttl
                0 packets shorter than header
                0 packets discarded for bad version
                0 packets discarded for bad HMAC
                0 packets discarded for bad action
                0 packets discarded for short packet
                0 states discarded for bad values
                0 stale states
                35622 failed state lookup/inserts
        23635 packets sent (IPv4)
        0 packets sent (IPv6)
                0 send failed due to mbuf memory error
                5580 send error
pfsync:
        387782 packets received (IPv4)
        0 packets received (IPv6)
                0 packets discarded for bad interface
                0 packets discarded for bad ttl
                0 packets shorter than header
                0 packets discarded for bad version
                0 packets discarded for bad HMAC
                0 packets discarded for bad action
                0 packets discarded for short packet
                0 states discarded for bad values
                0 stale states
                35711 failed state lookup/inserts
        23876 packets sent (IPv4)
        0 packets sent (IPv6)
                0 send failed due to mbuf memory error
                5676 send error

--Bill