[Bug 297427] linuxulator: implement pkey_alloc/pkey_free/pkey_mprotect (x86 memory protection keys)

From: <bugzilla-noreply_at_freebsd.org>
Date: Tue, 11 Aug 2026 04:51:04 UTC
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=297427

            Bug ID: 297427
           Summary: linuxulator: implement
                    pkey_alloc/pkey_free/pkey_mprotect (x86 memory
                    protection keys)
           Product: Base System
           Version: CURRENT
          Hardware: amd64
                OS: Any
            Status: New
          Severity: Affects Some People
          Priority: ---
         Component: kern
          Assignee: bugs@FreeBSD.org
          Reporter: dteske@FreeBSD.org

The Linuxulator does not implement the Linux protection-key syscalls
pkey_alloc(2), pkey_free(2), and pkey_mprotect(2); all three are stubs
returning ENOSYS:

  linux: jid 0 pid 11616 (brave): syscall pkey_alloc not implemented

Modern Linux software probes these at startup.  Chromium-based
browsers (found via www/linux-brave) use memory protection keys for
V8's heap and JIT sandboxing; glibc >= 2.27 (linux_base-rl9 ships
2.34) exposes pkey_alloc/pkey_free/pkey_mprotect/pkey_get/pkey_set.
On hardware without PKU, Linux returns ENOSPC from pkey_alloc and
applications fall back gracefully, but ENOSYS is a different signal
and forecloses the hardening on hardware that has the feature.

FreeBSD already supports Intel MPK natively on amd64: PKRU is managed
as XSAVE state, pmap_pkru_set()/pmap_pkru_clear() tag page ranges,
sysarch(2) exposes AMD64_SET_PKRU/AMD64_CLEAR_PKRU, and protection-key
faults deliver SIGSEGV with si_code SEGV_PKUERR.  Implementing the
Linux syscalls is a matter of bridging them to this existing
infrastructure plus per-process allocation bookkeeping:

  - pkey_alloc: allocate from a per-process key bitmap (key 0
    implicitly allocated, as Linux's mm_pkey_allocation_map; ENOSPC
    when keys 1..15 are exhausted or PKU is absent), and apply the
    requested initial access rights to the calling thread's PKRU.
  - pkey_free: bookkeeping only; like Linux, freeing neither untags
    pages nor updates PKRU.
  - pkey_mprotect: mprotect, then tag the range via pmap_pkru_set()
    (pkey -1 degrades to plain mprotect, as on Linux).
  - The allocation map is inherited on fork and reset on exec.
  - At exec, PKRU is initialized to 0x55555554 (Linux's init_pkru
    default: access disabled for keys 1..15), so memory tagged with a
    not-yet-allocated key is inaccessible to threads that have not
    been granted rights -- the property V8's thread isolation relies
    on.

I will attach a litmus program exercising the full surface: allocation
in 1..15, initial deny-all PKRU, page tagging, access/write denial
faults (verifying SEGV_PKUERR), initial rights applied at alloc, fork
inheritance of the allocation map, pkey -1 passthrough, double-free
EINVAL, and exhaustion at 15 keys then ENOSPC.

On a stock kernel the litmus reports ENOSYS.  With the proposed patch
(review to follow on Phabricator) all checks pass on 16.0-CURRENT
amd64 (13th Gen Intel Core i3-1315U, PKU-capable), and www/linux-brave
starts and renders with no pkey_alloc complaints, taking the
PKU-backed sandboxing path instead of the fallback.

Expected litmus output on a patched system ends with "VERDICT: PASS";
on a stock kernel it exits with a note that the kernel lacks the pkey
syscalls, and on PKU-less hardware it reports ENOSPC (matching Linux
behavior there -- nothing to fix on such machines).

-- 
You are receiving this mail because:
You are the assignee for the bug.