terrible if_vmx / vmxnet3 rx performance with lro (post iflib)

[Josh Paetzel]

On Wed, Feb 19, 2020, at 7:17 AM, Andriy Gapon wrote:
> On 18/02/2020 16:09, Andriy Gapon wrote:
> > My general experience with post-iflib vmxnet3 is that vmxnet3 has some
> > peculiarities that result in a certain "impedance mismatch" with iflib.
> > Although we now have a bit less code and it is a bit more regular, there are a
> > few significant (for us, at least) problems:
> > - https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=243126
> > - https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=240608
> 
> By the way, we (Panzura) use these changes to fix or work around the above two
> problems: https://people.freebsd.org/~avg/iflib-vmx.pz.diff
> 
> Questions / comments are welcome.
> Especially from people who worked on iflib.
> 
> > - https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=243392
> > - the problem described above
> > - a couple of issues that we already fixed or worked around
> > 
> > We are contemplating locally reverting to the pre-iflib vmxnet3 and we are
> > wondering if the conversion was really worth it in general.
> 
> 
> -- 
> Andriy Gapon
> _______________________________________________
> freebsd-net at freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
>

I'd like to follow this up just to make it 100% clear.  The problem is a ~4x regression in RX performance.  It affects stock FreeBSD, including 12.1-RELEASE.

In my 40Gbps connected lab single thread iperf receive went from 9Gbps to 2.5Gbps.

If this can't be fixed or looked at I'd heavily suggest looking at reverting "iflib"ing change in stock FreeBSD.

-- 

Thanks,

Josh Paetzel