KASAN port

[Mark Johnston]

Hi,

I ported the KASAN implementation from NetBSD to FreeBSD.  This is a
testing and debugging tool that leverages compiler instrumentation to
maintain a kernel "shadow" map which stores information about which
addresses in the main kernel are safe to access.  If you've been paying
attention to recent kernel commits you may have noticed that several
bugs have been found and fixed using this tool already; there are
several more that I'm aiming to have fixed in 13.0.

There was a GSOC project by Costin Carabas and andrew@ which did an
initial port of KASAN and several other debugging facilities; I reused a
few pieces of that work but this was mostly a direct port.

The instrumentation and validity checking introduces a fairly
substantial performance hit.  I think a 2-3x slowdown is pretty typical,
but it could be more for workloads which execute a lot of kernel code.
It's best used in conjunction with test suites that exercise a lot of
kernel functionality, like the regression test suite, stress2 or
syzkaller.

KASAN is currently only implemented for amd64.  It would be a useful and
probably relatively small project to port it to platforms like arm64 and
riscv.  If anyone is interested in this, please contact me.

I posted reviews for various pieces of the port here:
https://reviews.freebsd.org/D29454: Add a KASAN option to the kernel build
https://reviews.freebsd.org/D29416: Add the KASAN runtime
https://reviews.freebsd.org/D29417: amd64: Implement a KASAN shadow map
https://reviews.freebsd.org/D29455: amd64: Add MD bits for KASAN
https://reviews.freebsd.org/D29456: uma: Add KASAN state transitions
https://reviews.freebsd.org/D29457: kstack: Add KASAN state transitions
https://reviews.freebsd.org/D29458: kmem: Add KASAN state transitions
https://reviews.freebsd.org/D29459: vfs: Add KASAN state transitions for vnodes
https://reviews.freebsd.org/D29460: execve: Mark exec argument buffers
https://reviews.freebsd.org/D29461: malloc: Add state transitions for KASAN

A couple of small LLVM changes are also required:
https://reviews.llvm.org/D98285
https://reviews.llvm.org/D98286

Please ask questions and provide review feedback.  To test the port,
grab https://github.com/markjdb/freebsd/tree/ff/kasan and:

$ make kernel-toolchain WITHOUT_SYSTEM_COMPILER=
$ make buildkernel KERNCONF=GENERIC-KASAN

There are a few limitations of the current implementation, especially
from the fact that we don't have a way to disable all uses of the direct
map.  However, we have a way to reduce usage of that map by kernel
memory allocators and that's enough to find non-trivial bugs, so it
seems worthwhile to commit it now and continue to refine it.