_msgbufp vs msgbufp in the kernel - no symbols for dmesg -N ?

From: Marcin Cieslak <saper_at_saper.info>
Date: Wed, 09 Sep 2026 12:56:00 UTC
I have noticed that most of the tools like dmesg, vmcore, ps
do not seem to work on my amd64 minidump kernel core dumps anymore.

I noticed that those tools expect C-mangled symbol names ("_msgbufp")
but they are simply not available:

> dmesg -N /boot/kernel/kernel -M /big/crash/ieee80211_sta_join/vmcore.7
dmesg: /boot/kernel/kernel: msgbufp not found

(the message is misleading, dmesg looks for _msgbufp)

> nm /boot/kernel/kernel |grep msgbufp 
ffffffff8210b1d0 B msgbufp

I have written a shor test program which gives the following
for my 16.0-CURRENT amd64 laptop:

> sudo ./kvmtest /big/crash/ieee80211_sta_join/vmcore.7
Password:
kvm_nlist2: from corefile invalid_entries=0
kvm_nlist2: "msgbufp" from corefile type=0x09 addr=0xffffffff8210b1d0
kvm_nlist2: "_msgbufp" from corefile type=0x00 addr=0x00000000
kvm_nlist2: from live invalid_entries=0
kvm_nlist2: "msgbufp" from live type=0x04 addr=0xffffffff8210b1d0
kvm_nlist2: "_msgbufp" from live type=0x04 addr=0xffffffff8210b1d0

What is going on here ? How should the symbols look like?
I am using the same /boot/kernel/kernel that is live and was used to generate
the dump.

The test program below or git clone https://repo.or.cz/kvmtest.git

----------------8<-------------

#include <fcntl.h>
#include <kvm.h>
#include <nlist.h>
#include <stdio.h>
#include <err.h>

int
main(int argc, char *argv[])
{
 	struct {
 		char *disp_name;
 		char *core_name;
 	} kernels[] = {
 		{ "corefile", argv[1] },
 		{ "live", "/dev/mem" },
 	};

 	if (argc < 2)
 		errx(1, "usage: %s kernelcorefile", argv[0]);

 	int i, rc;

 	for (i = 0; i < 2; i++) {
 		struct kvm_nlist mylist[3] = {
 			{ "msgbufp", 0, 0},
 			{ "_msgbufp", 0, 0},
 			{ NULL, 0, 0},
 		};
 		kvm_t *kd = kvm_open(
 			"/boot/kernel/kernel",
 			kernels[i].core_name, NULL, O_RDONLY,
 			kernels[i].disp_name
 		);
 		if (kd == NULL) goto next;
 		rc = kvm_nlist2(kd, mylist);
 		printf("kvm_nlist2: from %s invalid_entries=%d\n",
 			kernels[i].disp_name,
 			rc);
 		printf("kvm_nlist2: \"%s\" from %s type=0x%02x addr=0x%08lx\n",
 			mylist[0].n_name,
 			kernels[i].disp_name,
 			mylist[0].n_type,
 			mylist[0].n_value
 		);
 		printf("kvm_nlist2: \"%s\" from %s type=0x%02x addr=0x%08lx\n",
 			mylist[1].n_name,
 			kernels[i].disp_name,
 			mylist[1].n_type,
 			mylist[1].n_value
 		);
 	next:
 		if (kd)
 			kvm_close(kd);
 	}
 	return 0;
}