svn commit: r365449 - head/sbin/rcorder

[Olivier Cochard-Labbé]

On Tue, Sep 8, 2020 at 12:36 PM Andrey V. Elsukov <ae at freebsd.org> wrote:

> Author: ae
> Date: Tue Sep  8 10:36:11 2020
> New Revision: 365449
> URL: https://svnweb.freebsd.org/changeset/base/365449
>
> Log:
>   Add a few features to rcorder:
>
>
>
Hi Andrey,

I've spent some time bisecting an IPSec gateway performance regression on
-head that points to this commit.

So my bench uses a simple static IPSec aes-gcm-16 setup, and their results
vary a lot between those 2 revisions:

   - r365448: Estimated Equilibrium Ethernet throughput= 1413 Mb/s (maximum
   value seen: 1419 Mb/s)
   - r365449: Estimated Equilibrium Ethernet throughput= 469 Mb/s (maximum
   value seen: 469 Mb/s)

How could a modification to the rcoder be the source cause of that ?
My rc.conf contains this:
kld_list="aesni"
ipsec_enable="YES"

So My first thought was that the aesni module wasn't loaded anymore:
Before upgrade:
[root at hp]~# uname -a
FreeBSD hp 13.0-CURRENT FreeBSD 13.0-CURRENT #15 r365448M: Thu Sep 17
18:17:58 CEST 2020     olivier at lame4.bsdrp.net:/usr/src/amd64.amd64/sys/amd64
 amd64
[root at hp]~# kldstat
Id Refs Address                Size Name
 1   11 0xffffffff80200000  1ee58b0 kernel
 2    1 0xffffffff82319000     34c8 fdescfs.ko
 3    1 0xffffffff8231d000     a240 aesni.ko
 4    1 0xffffffff82328000     8c98 ioat.ko
 5    1 0xffffffff82331000     e350 ipsec.ko

Then after upgrade:

[root at hp]~# uname -a
FreeBSD hp 13.0-CURRENT FreeBSD 13.0-CURRENT #14 r365449M: Thu Sep 17
17:01:35 CEST 2020     olivier at lame4.bsdrp.net:/usr/src/amd64.amd64/sys/amd64
 amd64
[root at hp]~# kldstat
Id Refs Address                Size Name
 1   11 0xffffffff80200000  1ee58b0 kernel
 2    1 0xffffffff82319000     34c8 fdescfs.ko
 3    1 0xffffffff8231d000     e350 ipsec.ko
 4    1 0xffffffff8232c000     a240 aesni.ko
 5    1 0xffffffff82337000     8c98 ioat.ko

=> aesni.ko is correctly loaded, so it is not the problem, but notice the
order of the kernel modules that have changed.
Could be this the source of the problem ? Let's try:

[root at hp]~# service ipsec stop
Clearing ipsec manual keys/policies.
[root at hp]~# kld
kldconfig kldload   kldstat   kldunload kldxref
[root at hp]~# kldunload ioat
[root at hp]~# kldunload aesni
[root at hp]~# kldunload ipsec
[root at hp]~# kldload aesni
[root at hp]~# kldload ipsec
[root at hp]~# service ipsec start
Installing ipsec manual keys/policies.
[root at hp]~# kldstat
Id Refs Address                Size Name
 1   11 0xffffffff80200000  1ee58b0 kernel
 2    1 0xffffffff82319000     34c8 fdescfs.ko
 5    1 0xffffffff82337000     8c98 ioat.ko
 6    1 0xffffffff8231d000     a240 aesni.ko
 7    1 0xffffffff82328000     e350 ipsec.ko

And after that the IPSec bench results are back to their previous value :-)
So rcorder needs to load aesni before ipsec.

Regards,

Olivier