11-RELEASE acting as vbd backend

[Nathan Friess]

On 12/15/2016 10:34 AM, Roger Pau Monné wrote:
> OK, I finally have this ready, you will need the following patches for FreeBSD,
> which I pushed to my git repo on top of current HEAD:
>
> git://xenbits.xen.org/people/royger/freebsd.git add_watches_xenstore
>
> You can either compile a kernel from this branch or pickup the 4 top patches
> and apply them to your local tree (I don't think you are going to find any
> conflicts even if the tree is stable/11 or 11.0-RELEASE).
>
> You should also update your xen-tools package to the latest version, I've just
> updated it to contain the xenstore device path fix:
>
> https://svnweb.freebsd.org/ports?view=revision&revision=428628
>
> After this is done, it should just be a matter of launching xendriverdomain at
> system boot and it should work out of the box. Let me know how this goes.

Hi Roger,

Thank you!  With your patches I've made some progress but not quite 
there yet.  I've been trying to trace through xl devd but I haven't been 
able to pinpoint the underlying issue.  Here is where I'm at.

I applied the 4 patches from your add_watches_xenstore branch to the 
11.0-RELEASE kernel and built xen-tools from svn.

The xendriverdomain rc script now starts xl devd successfully.  However, 
when the other domU starts and the backend should be set up in FreeBSD, 
xl devd attempts to run /etc/xen/scripts/block, which is in /usr/local 
instead.  My guess is that dom0 is generating that path and because dom0 
is Linux the paths don't match.  (I'm not specifying the script in the 
domU config directly.)  Adding a symlink in /etc to /usr/local/etc/xen 
gets around this for now.

Secondly, when /etc/xen/scripts/block runs, it doesn't have 
/usr/local/bin or sbin in its PATH, so it cannot execute xenstore-read 
or write.  That was also easy enough to fix.

So, now the block script runs but after that nothing seems to happen. 
This is where I started to trace the code in libxl.  I think what needs 
to happen is that xl devd (or the block script?) needs to set 
local/domain/(freebsd)/backend/vbd/(linuxpv)/51713/sectors, info, and 
sector-size.  The .../state variable is also still set to 3 after the 
block script runs.

It looks like the Linux hotplug scripts don't set those variables 
directly either, so I'm not sure where that should be happening.  (Aside 
from that, I did see that Linux sets .../physical-device to a 
major:minor and some feature- variables that FreeBSD does not set.)  The 
xldevd.log doesn't show anything interesting and running with -vvv only 
shows some cleanup being done after the script has finished.

Just to test, I set the above variables in xenstore and then set state 
to 4 manually, and that did kick the domU into trying to set up the vbd 
frontend but the kernel crashed in blkfront_gather_backend_features, 
which is the first thing that happens after reading the sectors, 
sector-size, and info variables.  I was copying directly from what 10.3 
was doing for the sectors and sector-size so they should be the same so 
I'm not sure what I did wrong.


Sorry for the long email but hopefully that gives you some idea of where 
to look next.  If nothing seems obvious, then if you could point me in 
the direction of what should be happening in xl after the block script 
runs I can try to trace things further.

Nathan