nmdm issues with bhyve

From: Polarian <polarian_at_polarian.dev>
Date: Tue, 16 Jul 2024 14:38:51 UTC
Hello,

Over the last few days I have been trying to get nmdm working with
bhyve. I have discussed it within #bhyve over on libera.chat however
none of the suggestions so far have fixed the problem, therefore I am
bringing it to the mailing list for further support.

A rundown what I have done so far, I have a shell script with the
following contents:

#!/bin/sh

bhyve -c 1 -m 1G -u -H \
        -s 0,amd_hostbridge \
        -s 3,ahci-hd,/path/to/install.img \
        -s 4,virtio-blk,/dev/zvol/zpool-storage/dns \
        -s 5,virtio-net,tap0 \
        -s 31,lpc -l com1,/dev/nmdm3A \
        -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
        dns

To verify the virtual machine was in fact booting, I did test stdio,
and the install medium for OpenBSD does successfully boot.

The problem, well there is many. Following the FAQ entry [1], the
following flag is suggested:

-s 31,lpc -l com1,nmdm0A

(where 0 can be substituted to prevent collisions)

Enter problem #1, when I try this by executing the script (with doas,
doas sh dns.sh) I get the following:

Unable to initialize backend 'nmdm3A' for LPC device com1
Device emulation initialization error: No such file or directory

Now, both vmm and nmdm have been loaded, checked with the following
commands:

kldstat | grep vmm
kldstat | grep nmdm

And these have been entered into /boot/loader.conf see below:

vmm_load="YES"
nmdm_load="YES"

So they have been loaded into the kernel.

When looking into the problem I found an article on the FreeBSD forums
which has the same error [2], which would suggest the example in the
FAQ is either wrong, or deprecated and no longer works.

The solution from this article is to ensure to specify the /dev path, as
seen in the full script at the beginning of the email, /dev/nmdm0A,
this DOES work and the vm does startup.

Problem #2, the device doesn't show up, even though the vm is, in fact,
running.

ls /dev | grep nmdm

The above returns nothing, there is no nmdm device. I have tested it to
ensure nmdm is in fact working, using cu to open both A and B and
verifying that the data is being passes between them. So nmdm is
working!

AllanJude on IRC suggested that I change from 0 to some other number in
case of conflicts, and so I did (as seen in the script at the beginning
of the email) changing 0 --> 3, same issue remains.

I can find no further information on this issue, and I am out of ideas,
so if anyone has nmdm successfully working, could you give me a hand in
getting it to work on my server?

The FAQ [1] also suggested pty can be used, or a terminal multiplexer
such as tmux, however I am yet to find any examples on how this could
be done to substitute nmdm. If anyone has some useful links to some
examples for these, that would be great!

Thank you,
-- 
Polarian
GPG signature: 0770E5312238C760
Jabber/XMPP: polarian@icebound.dev

[1]
https://wiki.freebsd.org/bhyve#Q:_Does_bhyve_have_an_.22out-of-band.22.2F.22lights-out_management.22_serial_console.3F
[2]
https://forums.freebsd.org/threads/unable-to-initialize-backend-nmdm0a-for-lpc-device-com1.82177/