Re: CPU hot-plug and RAM hot-add in virtual machines

On Wed, Jul 21, 2021 at 02:06:07PM +0200, Miroslav Lachman wrote:
> I don't need that badly but it is another problem our customers have 
> with FreeBSD over Linux. One of them has about 50 VM, mostly Linux, some 
> Windows and 8 FreeBSD.


I'm also a cloud provider. Again, I've maybe had one customer ever request/use it.
(Although I'm much further away from the front lines now-a-days). 

There's a reason it is disabled by default, in that enabling it also
disables vNUMA, which gives up to an 8% CPU hit accross the board in all
your VMs that you have Hot-Add enabled on. Also, hot-add memory really trashes
the page tables on OSs that support it. I would not be surprised that
memory utilization efficiency becomes much worse when hot-adding memory because they
have to suddenly deal with more than they started with. (I couldn't find
studies on this, probably because it is so little utilized). 

So, if people want it in the first place, they are going to have to
shut down the VM to enable it. And then it will have the CPU hit when enabled.
(It also does some weird stuff with pass-through USB connections, but that
probably isn't too much an issue in a cloud environment).

Surely, they have regular patch windows that CPU/memory can be added in
those windows?
I wouldn't tout a feature that virtually nobody uses as a plus for an OS,
especially with downsides such as it'll make it run slower in general for
the same level of CPU.