Re: 5 gigs of contiguos kernel VM shared with userspace

On Mon, 10 Aug 2026 11:22:29 -0400
Mark Johnston <markj@freebsd.org> wrote:

> On Mon, Aug 10, 2026 at 05:12:49AM +0300, Vadim Goncharov wrote:
> > On Sun, 9 Aug 2026 19:15:59 -0400
> > Mark Johnston <markj@freebsd.org> wrote:
> >   
> > > On Mon, Aug 10, 2026 at 01:32:02AM +0300, Vadim Goncharov wrote:  
> > > > Hello,
> > > > 
> > > > Are there ways to allocate in kernel a range of contiguous virtual
> > > > address space, share it with userspace, and preferably run memory
> > > > allocator inside it which has it's pointers relative to base address?
> > > > Same as processes cooperating via mmap(), but for kernel<->user.
> > > > 
> > > > I'd like just run UMA inside it, but seems UMA zones cannot be told to
> > > > take from particular range, or am I missing something? On user side,
> > > > jemalloc mentions something like arenas, but seems it is too
> > > > unprepared for even mmap() between processes case (may be adapting
> > > > some TLSF allocator [*] is possible, but that's a premature question
> > > > before VM one).    
> > > 
> > > UMA is really two fairly distinct things: a slab allocator, and a cache.
> > > UMA zones provide per-CPU and per-NUMA domain caches, and UMA kegs
> > > allocate and manage slabs.
> > > 
> > > You can have a zone without a keg: if you have some collection of
> > > objects you want to allocate to consumers, you can use
> > > uma_zcache_create() to create a "cache zone" which provides the usual
> > > per-CPU etc. caches to store objects returned by a custom uz_import
> > > callback that you provide when creating the zone.  For instance, if you
> > > have some static array of objects (i.e., there is no need to subdivide a
> > > slab) and just want to slap a scalable cache in front of it,
> > > uma_zcache_create() is handy.
> > > 
> > > The other thing you could do is provide a custom slab allocator for the
> > > keg via uma_zone_set_allocf().  There, you provide some page allocator,
> > > and you can impose whatever constraints you want.  uma_small_alloc() is
> > > used by default for page-sized slabs, and page_alloc() is used by  
> > 
> > Is that all ported/portable to userspace?  
> 
> No.  Most of it probably wouldn't be too bad.  I'd move some parts of it
> to a uma_kern.c (e.g., uma_small_alloc() and page_alloc(), referenced
> above, would live there). 

Have jemalloc similar thing to slabs, even if only for some limited set of
sizes? I expect many of my objects being 32-byte size.

> But you'd also need to decide how per-CPU
> caches should be implemented in userspace.

Oh, I did not think about that at all - so far, I presumed single-threaded
userspace process.
 
> > > default for multi-page slabs.  Your implementation could return pages
> > > from some pre-allocated, contiguous range of memory.  
> > 
> > That's the main [prerequisite] question: how to do in kernel thing similar
> > to mmap(0, 0x5000000, MAP_SHARED|MAP_ALIGNED_SUPER, ...) and share this
> > region with userspace process(es).  
> 
> It depends, I guess you want the region to be demand-paged? 

I don't know for sure. Likely, I'll need it wired, or both variants for
different setups. The question is because being swapped is simpler?

> If so, something like:
> 1. kmem_subinit() to find some region of the kernel address space it can
>    use to map the area.
> 2. Allocate a VM object.
> 3. Use vm_map_find() to allocate the kernel virtual address region from
>    the submap, and to associate that region with your VM object.
> 4. Map that VM object into userspace as desired, e.g., by returning the
>    VM object from some device node's d_mmap_single implementation.
> 5. Now the kernel and participating userspace programs can fault on
>    their respective mappings of that VM object.

I haven't found man page for kmem_subinit, where can I read more background on it?

-- 
WBR, @nuclight