svn commit: r334054 - in head: sys/kern sys/netipsec tools/tools/crypto usr.bin/netstat

[Mateusz Guzik]

On Tue, May 22, 2018 at 5:54 PM, Fabien Thomas <fabient at freebsd.org> wrote:

> Author: fabient
> Date: Tue May 22 15:54:25 2018
> New Revision: 334054
> URL: https://svnweb.freebsd.org/changeset/base/334054
>
> Log:
>   Add a SPD cache to speed up lookups.
>
>   When large SPDs are used, we face two problems:
>
>   - too many CPU cycles are spent during the linear searches in the SPD
>     for each packet
>   - too much contention on multi socket systems, since we use a single
>     shared lock.
>
>
>  void
> +spdcache_init(void)
> +{
> +       int i;
> +
> +       TUNABLE_INT_FETCH("net.key.spdcache.maxentries",
> +           &V_key_spdcache_maxentries);
> +       TUNABLE_INT_FETCH("net.key.spdcache.threshold",
> +           &V_key_spdcache_threshold);
> +
> +       if (V_key_spdcache_maxentries) {
> +               V_key_spdcache_maxentries = MAX(V_key_spdcache_maxentries,
> +                   SPDCACHE_MAX_ENTRIES_PER_HASH);
> +               V_spdcachehashtbl = hashinit(V_key_spdcache_maxentries /
> +                   SPDCACHE_MAX_ENTRIES_PER_HASH,
> +                   M_IPSEC_SPDCACHE, &V_spdcachehash_mask);
> +               V_key_spdcache_maxentries = (V_spdcachehash_mask + 1)
> +                   * SPDCACHE_MAX_ENTRIES_PER_HASH;
> +
> +               V_spdcache_lock = malloc(sizeof(struct mtx) *
> +                   (V_spdcachehash_mask + 1),
> +                   M_IPSEC_SPDCACHE, M_WAITOK|M_ZERO);
> +
> +               for (i = 0; i < V_spdcachehash_mask + 1; ++i)
> +                       SPDCACHE_LOCK_INIT(i);
> +       }
> +}
> +
>

This ends up putting two locks per cacheline and sharing bucket heads
across other lines.

Unless you got a good reason not to, you should define a struct a which has
both the lock and the list.

An example of this can be found in kern/kern_lockf.c

-- 
Mateusz Guzik <mjguzik gmail.com>