breakage in blkfront with ring_pages > 1

[Justin T. Gibbs]

On 7/5/11 7:14 PM, Colin Percival wrote:
>  On 06/10/11 13:30, Justin T. Gibbs wrote:
> > On 6/9/11 9:26 PM, Colin Percival wrote:
> >> Has anyone seen anything like this? Is it possible that there's a bug
> >> in how our blkfront negotiates the request ring? Does anyone have
> >> ring_pages > 1 in use?
> >
> > The only backend driver I know of that can support more than one ring 
page
> > is FreeBSD. So, the problem likely is that blkfront is getting the
> > negotiation wrong and only 1 page is in use.
>
>  Turns out that the linux backend in question really does support multiple
>  pages.

Can you provide a source or spec reference for this driver?  There was a
proposal (in 2009?) to add multiple page support to the XenSource
provided blkfront/back, but it didn't get committed to the Xen repository.
It also doesn't use the same XenStore variables that you list.  Is this
something Amazon developed in house?

>  There's also an inconsistency about how multiple rings are negotiated:
>  We set:
>  * ring-pages to the number of pages blkfront wants to use
>  * ring-ref to the reference for the first page
>  * ring-refXX to the references for other pages
>  while linux sets:
>  * num-ring-pages to the number of pages blkfront wants to use
>  * ring-refXX to the page references.

You forgot to mention that the FreeBSD driver allows request size to
be negotiated and explicitly supports "chained" requests to break the
44k request size limit imposed by the original interface.  The 2009
proposal did not allow larger requests which is why I didn't follow
it when I implemented the FreeBSD extension.  Allowing larger requests
both improves performance and removes the need to do costly split/coalesce
operations on both sides of the blkif interface.

>  It seems to be impossible to be compatible with both, since Linux 
interprets
>  having a value set for ring-ref to indicate that the single-ring 
protocol is
>  being used and doesn't check anything else.

It's not "Linux".  There is no multi-page support in either the kernel.org
hosted drivers or those in the Linux tree hosted in the official Xen
repository.

>  Is there any official source which defines the blkback/front protocol?

Only the source code in the Xen repository.  Both the FreeBSD extensions
that I did and those for the Linux system you are using are likely backwards
compatible to the original interface, but unfortunately are not compatible
with each other.

>  Maybe
>  the right option is to have a loader tunable dev.xn.linuxback to 
control which
>  version of the protocol is used?

What a mess.

--
Justin