Linux kernel compatability

Alexander Kabaev kabaev at gmail.com
Wed Jan 5 17:40:56 UTC 2011


On Tue, 4 Jan 2011 10:53:30 -1000 (HST)
Jeff Roberson <jroberson at jroberson.net> wrote:

> On Tue, 4 Jan 2011, Alexander Kabaev wrote:
> 
> > On Mon, 3 Jan 2011 19:03:01 -1000 (HST)
> > Jeff Roberson <jroberson at jroberson.net> wrote:
<SKIP>
> >>>
> >>> This probably will go against popular opinion here, but having 10k
> >>> linux emulation layer that _almost_ work in the tree will be an
> >>> unfortunate event and will do more damage to FreeBSD as a platform
> >>> than good in the long run. I would rather see this code never hit
> >>> main repository.
> >>
> >> I would argue that the layer works very well for infiniband.  Much
> >> better than almost.  It is only almost complete in that there is no
> >> need for me to implement features that we're not using.
> >>
> >> I am interested in hearing your other concerns however.
> >>
> >> Thanks,
> >> Jeff
> >>
> >
> 
> Alexander, let me first start out by saying I have a great deal of
> respect for you and I hear your concerns.  I see that this is a
> somewhat heated issue and I can really only address the technical
> points.  The more existential questions about FreeBSD will have to be
> left to others.
> 

Well, my objection was leaning more on the existential side of the
argument rather than technical point. I will be the last person out
there doubting that you will get all the technically challenges right.

> > The considerations are simple enough. First, we do not have many IB
> > users of FreeBSD in the wild and those that we have (Isilon) seem
> > to be perfectly capable of managing the IB stack out of the tree,
> > without dumping the thousands of lines of the code into the base.
> > If they had the stack before, but were not willing/capable to
> > provide adequate care for it in the past, there is no reason to
> > expect things to change with second stack, which now will rot in
> > our tree instead of theirs.
> 
> They provided adequate care for it to keep their product running on
> old versions of FreeBSD.  Unfortunately it is a large stack and there
> are a great number of people and organizations working on improving
> and advancing it on Linux via OFED and having a private stack does
> not give you the benefit of their work.  The motivation for making
> the wrapper layer was entirely to keep pace with this development and
> make it less likely that what is in the tree will rot.

I fail to see what having the code in the tree is set to achieve. If we
have hordes of potential customers waiting for Infiniband drivers to
hit the tree, they kept surprisingly low profile to date. All ten of
them. You said so yourself, the codebase is HUGE and who is going to
feed and care for it once your relationship to this work is over? None
of our big customers do really care about ongoing FreeBSD development
in -current and instead center their interest around specific version
of stable they happen to base their products on. I do not see that
changing. We have much less complicated pieces of software in the tree
that are used by many, many more people than Infiniband ever will and
yet their development has stagnated in the absence of active maintainer
(smbfs *cough*). With Infiniband, where minuscule percentage of FreeBSD
developers have access to the hardware and specs the rot is not only
likely possibility, but guaranteed certainty.

> >
> > Second, semi-complete Linux compat layer in kernel will have the
> > same effect as linuxulator in userland - we do have some vendors
> > still trying to bother with FreeBSD drivers for their hardware now
> > and we will have none after we provide the possibility to hack
> > their Linux code to run somewhat stably on top of Linux compat
> > layer. Due to intentional fluidity of Linux kAPI, our shims will
> > never quite walk and quack like their original implementation in
> > Linux kernel and combined result will always be lees stable than
> > native Linux linux drivers in Linux kernel.
> 
> I have heard this argument about the linuxulator and what we're
> really talking about is slipping FreeBSD marketshare.  I don't share
> the view that the linuxulator futhered this slip but rather my view
> is that it allows us to stay relevant in areas where companies can
> not justify an independent FreeBSD effort.  Adobe is a good example
> of this.
> 

It compounded the Adobe's reluctance to work on portable flash player.

> Let's talk nuts and bolts about what this thing does.  In the vast 
> majority of cases it simply shuffles arguments and function names
> around where there is a 1:1 correlation between linux api and FreeBSD
> API.  Think about things like atomics, callouts, locks, jiffies vs
> ticks, etc.  In these areas the systems are trivially different.  In
> a very small number of areas where this wasn't the case I did a
> direct port and noted it with an #ifdef.

There are also less beautiful things like struct task in td_retval[1],
etc. The layer does look complete enough to compel device writers to
start relying on ii. But it is written to cover OFED needs only, so any
other work will likely require extending your wrapper some more
again and again. Before you know it we are in an arms race against Linux
kernel which we cannot win.

> This works specifically in the infiniband case because it is its own 
> middle layer.  You can't write a scsi driver for linux and use it on
> BSD with this.  You can't write a network driver even.  But if you do
> bring in code from linux you don't have to worry about changing every
> kmalloc to malloc and every printk to printf so diffs can be reduced
> in trivial cases.  I thought given your work on XFS for FreeBSD that
> would make sense to you.

Well, XFS I worked on had clear separation on OS-specific and core
parts with minor incursions of linuxisms in core. This probably has
changed with SGI demise and consequent death of Irix.

> Our options are, to leave FreeBSD users without infiniband, which I
> can tell you has cost us more market share as I know of specific
> cases we have lost due to it.  To maintain our own stack
> independently, which no one has the budget for.  Or to try to
> integrate with OFED.  Do you see some other approach?

Maintain OFED-compatible stack out of the tree. Make it downloadable
in source form to anyone who cares, there won't be many in absolute
numbers. NVidia has managed this + complication of binary blob, multiple
commercial vendors on Linux have managed this and if parties interested
in Infiniband on FreeBSD can not, what makes anyone think that driver in
the tree will fare any better than shared stack maintained elsewhere?

-- 
Alexander Kabaev
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 188 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-arch/attachments/20110105/5e69a61f/signature.pgp


More information about the freebsd-arch mailing list