[Bug 276463] Linux guest uses only 503G of memory when using UEFI loader

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=276463

            Bug ID: 276463
           Summary: Linux guest uses only 503G of memory when using UEFI
                    loader
           Product: Base System
           Version: 13.2-RELEASE
          Hardware: amd64
                OS: Any
            Status: New
          Keywords: bhyve
          Severity: Affects Only Me
          Priority: ---
         Component: bhyve
          Assignee: virtualization@FreeBSD.org
          Reporter: antranigv@freebsd.am

Created attachment 247783
  --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=247783&action=edit
Information about the Linux Guest

We have a massive system (details below) which runs a massive VM. The host runs
on FreeBSD while the VM guest is Ubuntu Linux.

```
Host Information:
root@genomic:~ # sysctl hw.model
hw.model: AMD EPYC 7702 64-Core Processor                
root@genomic:~ # sysctl hw.ncpu
hw.ncpu: 256
root@genomic:~ # sysctl hw.physmem
hw.physmem: 2185595944960
root@genomic:~ # sysctl hw.realmem
hw.realmem: 2200361238528
```

The VM uses 1.5TB of RAM, here's the VM configuration:
```
# vm info comp0
------------------------
Virtual Machine: comp0
------------------------
  state: stopped
  datastore: default
  loader: uefi
  uuid: 2893e12c-637d-11ee-af94-7cc255269cb8
  cpu: 240
  cpu-topology: sockets=2, cores=60, threads=2
  memory: 1536G

  network-interface
    number: 0
    emulation: virtio-net
    virtual-switch: public
    fixed-mac-address: 58:9c:fc:0e:4b:00
    fixed-device: -

  virtual-disk
    number: 0
    device-type: file
    emulation: virtio-blk
    options: -
    system-path: /usr/local/vm/comp0/disk0.img
    bytes-size: 216895848448 (202.000G)
    bytes-used: 190077076480 (177.023G)
```

and here's the bhyve arguments according to vm-bhyve:

```
Jan 19 15:40:32:  [bhyve options: -c 240,sockets=2,cores=60,threads=2 -m 1536G
-Hwl bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd -U
2893e12c-637d-11ee-af94-7cc255269cb8 -u -S]
Jan 19 15:40:32:  [bhyve devices: -s 0,hostbridge -s 31,lpc -s
4:0,virtio-blk,/usr/local/vm/comp0/disk0.img -s
5:0,virtio-blk,/dev/zvol/zscratch/scratch -s
6:0,virtio-net,tap0,mac=58:9c:fc:0e:4b:00 -s 7:0,passthru,68/0/81 -s
8:0,fbuf,tcp=127.0.0.1:5900,wait]
```

We've been using the GRUB loader for a while and decided to move to the UEFI
loader. After the migration we noticed that Linux shows only 503GB of RAM from
the 1.5TB allocated. We didn't have such issue when using the grub loader.

I'm attaching a tarball which has information of the Linux system, including
the following files.

linux_uefi_iomem.txt — /proc/iomem
linux_uefi_meminfo.txt — /proc/meminfo
linux_uefi_vmstat.txt — /proc/vmstat
linux_uefi_dmidecode.txt — dmidecode
linux_uefi_free.txt — free
linux_uefi_dmesg.txt — dmesg
linux_uefi_cpuinfo.txt — /proc/cpuinfo


We've also noticed that inside of TianoCore's UEFI menu, it shows the 1.5TB of
RAM properly (check screenshot at: https://notes.bsd.am/bhyve_uefi_menu.png )

Not sure if this is an issue with Linux or bhyve's UEFI loader. any tips are
appreciated.

-- 
You are receiving this mail because:
You are the assignee for the bug.