Linux domU only works with xen_platform_pci=0 ?

[Roger Pau Monné]

On Mon, May 14, 2018 at 08:34:54PM -0600, Nathan Friess wrote:
> On 2018-05-14 07:04 AM, Roger Pau Monné wrote:
> > On Sun, May 13, 2018 at 07:33:03PM +0200, Kai Otto wrote:
> > > On 13/05/18 17:16, Roger Pau Monné wrote:
> > > > On Sun, May 13, 2018 at 03:51:36PM +0200, Kai Otto wrote:
> > > > > Hello,
> > > > > 
> > > > > I'm trying to set up a FreeBSD 11.1 system as Xen virtualization host.
> > > > > Following a combination of handbook [1] and wiki [2], I was able to get
> > > > > get a FreeBSD dom0 in PVH mode, and FreeBSD domU in HVM mode running,
> > > > > including installation to disk and networking.
> > > > > 
> > > > > It seemed to me as if the switch 'xen_platform_pci' doesn't have an
> > > > > effect on FreeBSD domU's, as I see e.g. a xenpci0 and xbd0 in dmesg no
> > > > > matter if I set it to 1 or 0.
> > > > > 
> > > > > Do I understand it correctly, that this switch makes the difference
> > > > > between HVM and PVHVM mode?
> > > > > According to xl.cfg(5), it 'enables a guest Operating System [...] to
> > > > > make use of paravirtualization features such as disk and network devices'.
> > > > > 
> > > > > 
> > > > > Afterwards, I tried to create a Linux domU. Both Centos 7 and Alpine
> > > > > Linux only detected the harddisk with xen_platform_pci=0.
> > > > > 
> > > > > For Alpine Linux, with xen_platform_pci=1, I get the following messages
> > > > > on the console:
> > > > > 
> > > > > vbd vbd-5632: 19 xenbus_dev_probe on device/vbd/5632
> > > > That's ENODEV IIRC, I think there's something wrong with FreeBSD disk
> > > > backend.
> > > > 
> > > > > vbd vbd-5632: failed to write error node for device/vbd/5632 (19
> > > > > xenbus_dev_probe on device/vbd/5632)
> > > > > 
> > > > > After waiting for a couple minutes it boots, but doesn't detect the disk.
> 
> I had similar issues with Linux domUs being unable to detect their disks
> when FreeBSD 11.1-RELEASE is the backend:
> 
> https://lists.freebsd.org/pipermail/freebsd-xen/2016-December/002924.html
> 
> 
> What seems to be happening is that on my system the frontend and backend may
> go from state InitWait to Initialised in different orders and so either end
> may end up getting stuck waiting for the other side to change state when the
> other side already has done so.
> 
> I have been running with the attached patch since my last message above and
> Linux domUs have been working perfectly since then.  I realize that the call
> to pause() is not the correct solution but it demonstrates that some fine
> tuning of how the states are handled will help.

Can you please give a try to the patch at:

https://lists.freebsd.org/pipermail/freebsd-xen/2018-May/003132.html

I think this is the proper way to solve the issue, and I would like to
commit it ASAP in order to MFC it to stable-11 before 11.2 is
released, but it could benefit from some more testing.

Thanks, Roger.