FreeBSD 12.1 i386 on Hyper-V Fails to Mount Root

[Konstantin Belousov]

On Mon, May 17, 2021 at 04:12:20AM +0000, Wei Hu wrote:
> Copy Kib  who was the author of PTI and other relate patches.
> 
> This looks to be caused by Meltdown and Spectre mitigation patches which were 
> checked into head couple years ago. See this link for details about these patches:
> https://wiki.freebsd.org/SpeculativeExecutionVulnerabilities
> 
> The amd64  patches also caused problems
> on Hyper-V (triple fault) when vmbus and other HyperV drivers (netvsc, storvsc etc)
> were loaded separately. The workaround for this on amd64 is either build these
> Hyper-V modules into the kernel or completely disable mitigation by
> setting vm.pmap.pti="0" in /boot/loader.conf in the affect FreeBSD guests.
> 
For amd64 I posted the patch to supposedly fix the issue, you did not
responded.

> It looks the problem on i386 guests is different. While multiple patches have been 
> checked in for amd64, there was only one patch r332489 for i386.  I am not sure
> whether there is similar vm.pmap.pti option to disable the patch for i386 or not.
> 
> I am not familiar to these patches. 
> 
> @ Kib, will you be able to comment on this?
> 
> Thanks,
> Wei
> 
> 
> 
> > -----Original Message-----
> > From: Harry Schmalzbauer <freebsd at omnilan.de>
> > Sent: Sunday, May 16, 2021 2:55 AM
> > To: Wei Hu <weh at microsoft.com>; Chris Knight <stryqx at gmail.com>; freebsd-
> > virtualization at freebsd.org
> > Subject: Re: FreeBSD 12.1 i386 on Hyper-V Fails to Mount Root
> > 
> > Am 14.05.2021 um 18:44 schrieb Harry Schmalzbauer:
> > > Am 16.10.2020 um 11:35 schrieb Wei Hu via freebsd-virtualization:
> > >>> :
> > >>>
> > >> Did you try 12.0? Maybe something changed in 12.x causes failure.
> > >> I386 is not well tested on Hyper-V. I will try to reproduce and take
> > >> a look.
> > >
> > > The regression happened before 12.0.
> > > Today I stumbled across this problem too and tried 11.4 (/May 8th,
> > > 2020) /sucessfully, while 12.0 (/Oct 19th, 2018)/ is failing with i386
> > > releases.
> > >
> > > One commit which is in 12 but not in 11.4 is r332489.
> > >
> > :
> > :
> > :
> > > Unfortunately beyond my skills.
> > 
> > In sys/x86/x86/local_apic.c, I added this debug printf:
> >          for (idx = IPI_DYN_FIRST; idx <= IPI_DYN_LAST; idx++) {
> > printf("DEBUG: native_lapic_ipi_alloc(), idx = %i\n", idx);
> >                  ip = &idt[idx];
> >                  func = (ip->gd_hioffset << 16) | ip->gd_looffset;
> >                  if ((!pti && func == (uintptr_t)&IDTVEC(rsvd)) ||
> >                      (pti && func == (uintptr_t)&IDTVEC(rsvd_pti))) {
> >                          vector = idx;
> >                          setidt(vector, ipifunc, SDT_APIC, SEL_KPL, GSEL_APIC);
> >                          break;
> >                  }
> >          }
> > 
> > This is my latest attempt to see if at least the origin of my fault tracing matches.
> > It does.
> > 
> >  From #define IPI_DYN_FIRST   (APIC_IPI_INTS + 9) in sys/x86/include/apicvar.h
> > I expected to see 252, 253 and 254, which I sucessfully verified.
> > 
> > Now comes the part I cannot read: (uintptr_t)&IDTVEC(rsvd)
> > IDTVEC(rsvd) seems to be a assembler makro from sys/i386/i386/exception.s
> > 
> > But now it doesn't make any sense for me to dig further, I simply do not
> > understand it.
> > 
> > Unfortunately having broken i386 support on Hyper-V is not a cosmetic issue.
> > Removing "device hyperv" from kernel gives access to the v-hdd via legacy
> > IDE/ada, but then I have no NIC.
> > Choosing between missing HDD and missing network support is hard
> > nowadays ;-)
> > 
> > Any help getting i386 up and running back on Hyper-V highly appreciated!  11.4
> > is the latest working version, all i386-releases starting with 12.0 don't work!
> > 
> > Thanks,
> > -harry
>