svn commit: r359950 - head/usr.sbin/bhyve

[Conrad Meyer]

Thanks!  I believe we're running into VM_MAX_MEMMAPS (4) due to:

1. Low memory (0 through PCI hole)
2. High memory (4GB+)
3. Framebuffer
4. VMgenid's segment
5. EFI firmware's segment

As a temporary workaround, if you do not need any of: >3GB RAM,
framebuffer, or EFI boot, turning off any one of these should free a
segment and allow boot.  Unless there is an ABI problem, I think we
should probably go ahead and bump VM_MAX_MEMMAPS, or make it
dynamically sized.  And as a userspace workaround for the issue that
doesn't require a reboot (or reload of vmm.ko), we could add a knob to
bhyve(8) to disable vmgenid.

Best,
Conrad

On Sun, Apr 19, 2020 at 4:14 PM Shawn Webb <shawn.webb at hardenedbsd.org> wrote:
>
> This is the full output from bhyve:
>
> fbuf frame buffer base: 0x69191a00000 [sz 16777216]
> bhyve: bootrom_alloc: vm_mmap_mapseg: No space left on device
> bhyve: vmgenc_init: bootrom_alloc
>
> Thanks,
>
> --
> Shawn Webb
> Cofounder / Security Engineer
> HardenedBSD
>
> GPG Key ID:          0xFF2E67A277F8E1FA
> GPG Key Fingerprint: D206 BB45 15E0 9C49 0CF9  3633 C85B 0AF8 AB23 0FB2
> https://git-01.md.hardenedbsd.org/HardenedBSD/pubkeys/src/branch/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc
>
> On Sun, Apr 19, 2020 at 04:04:16PM -0700, Conrad Meyer wrote:
> > Hey Shawn,
> >
> > I will take a look. Thanks for the report and especially the repro example.
> > What sort of bad symptoms are you observing (or will it be super obvious
> > when I try this)?
> >
> > Thanks,
> > Conrad
> >
> > On Sun, Apr 19, 2020 at 15:53 Shawn Webb <shawn.webb at hardenedbsd.org> wrote:
> >
> > > On Wed, Apr 15, 2020 at 02:00:18AM +0000, Conrad Meyer wrote:
> > > > Author: cem
> > > > Date: Wed Apr 15 02:00:17 2020
> > > > New Revision: 359950
> > > > URL: https://svnweb.freebsd.org/changeset/base/359950
> > > >
> > > > Log:
> > > >   bhyve(8): Add VM Generation Counter ACPI device
> > > >
> > > >   Add an implementatation of the 'Virtual Machine Generation ID' spec to
> > > >   Bhyve.  The spec provides a randomly generated GUID (at bhyve start) in
> > > >   device memory, along with an ACPI device with _CID VM_Gen_Counter and
> > > ADDR
> > > >   evaluating to a Package pointing at that GUID.
> > > >
> > > >   A GPE is defined which Notifies the ACPI Device when the generation
> > > changes
> > > >   (such as when a snapshot is rolled back).  At this time, Bhyve does not
> > > >   support snapshotting, so the GPE is never actually raised.
> > > >
> > > >   Suggested by:       rpokala
> > > >   Discussed with:     grehan
> > > >   Differential Revision:      https://reviews.freebsd.org/D23165
> > > >
> > > > Added:
> > > >   head/usr.sbin/bhyve/vmgenc.c   (contents, props changed)
> > > >   head/usr.sbin/bhyve/vmgenc.h   (contents, props changed)
> > > > Modified:
> > > >   head/usr.sbin/bhyve/Makefile
> > > >   head/usr.sbin/bhyve/acpi.c
> > > >   head/usr.sbin/bhyve/acpi.h
> > > >   head/usr.sbin/bhyve/bhyverun.c
> > > >   head/usr.sbin/bhyve/pm.c
> > >
> > > Hey Conrad,
> > >
> > > Something about this commit broke bhyve in UEFI mode. Reverting this
> > > specific change caused bhyve to work again. Here's a sample command:
> > >
> > > /usr/obj/usr/src/amd64.amd64/usr.sbin/bhyve/bhyve \
> > >     -c 4 \
> > >     -m 16g \
> > >     -H \
> > >     -A \
> > >     -P \
> > >     -S \
> > >     -g 0 \
> > >     -s 0:0,hostbridge \
> > >     -s 1:0,lpc \
> > >     -s 29,fbuf,tcp=127.0.0.1:5910,w=1024,h=768,wait \
> > >     -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
> > >     -s 2:0,virtio-net,tap1 \
> > >     -s 3:0,virtio-blk,/dev/zvol/rpool/bhyve/hbsd-cross-dso-cfi-01/disk-01 \
> > >     -l com1,/dev/nmdm-hbsd-cross-dso-cfi-01-A \
> > >     -s 31:0,ahci-cd,/ISO/HardenedBSD/12-stable_amd64/2020-04-19_disc1.iso \
> > >     hbsd-cdcfi-01
> > >
> > > Thanks,
> > >
> > > --
> > > Shawn Webb
> > > Cofounder / Security Engineer
> > > HardenedBSD
> > >
> > > GPG Key ID:          0xFF2E67A277F8E1FA
> > > GPG Key Fingerprint: D206 BB45 15E0 9C49 0CF9  3633 C85B 0AF8 AB23 0FB2
> > >
> > > https://git-01.md.hardenedbsd.org/HardenedBSD/pubkeys/src/branch/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc
> > >