Network device driver KPI/ABI and TOE

[Jack Vogel]

On Jan 6, 2008 5:47 AM, Robert Watson <rwatson at freebsd.org> wrote:
...

> My proposal, and this is really a proposal to drive discussion as much as a
> proposal for a policy, is that the internal TCP data structures exported via
> the TOE interfaces and accessed by TOE device drivers *not* be considered
> ABI/KPI-stable in -STABLE branches.  While I think we shouldn't intentionally
> change them to break TOE, it's unrealistic to expect that these network stack
> internals won't change as part of normal maintenance and feature development
> that take place in -STABLE branches.
>
> For those who aren't involved in those day-to-day internals, a comparable
> situation might be if a CAM SCSI storage driver was dependent not only on
> there being no changes made to the on-disk layout of UFS (even backwards
> compatible ones), but also the in-memory data structures of soft updates. Any
> significant changes to soft updates internals would break such device drivers
> due to a requirement for forward compatibility.  In some ways this isn't a
> perfect comparison, as soft updates isn't under active development, but from a
> layering and abstraction perspective, it's quite similar.
>
> We don't yet ship TOE in a -STABLE branch, but I believe Kip hopes to MFC TOE
> support, and with other device driver vendors starting to take a look, I think
> we want out thoughts on the table regarding this matter.  I presume that we'll
> see the TOE interfaces continue to evolve over the next 6-18 months, and we
> should make sure that we know whether or not third party device driver authors
> can expect ABI/KPI stability before, rather than after, it hits a -STABLE
> branch.  On a similar note, these necessary changes to network stack internals
> will result in modifications to in-tree device drivers, so device driver
> authors who implement TOE should expect to see the TOE parts of their drivers
> being significantly modified as development occurs on those other parts of the
> stack.
>
> There's also the opportunity to think about whether it's possible to harden
> things in such a ways as to not give up our flexibility to keep maintaining
> and improving TCP (and other related subsystems), yet improving the quality of
> life for a third party TOE driver maintainer.  For example, might we provide
> accessor routines for certain data structures, or attempt to structure things
> to hide more of TCP locking from a TOE implementation?  Should we suggest that
> non-native TOE implementations rely less on our TCP code and provide there own
> where the hardware doesn't provide a complete implementation, in order to
> avoid building dependency on things that we know will change?

I agree Robert, I have hit minor KPI changes during the 6.X evolution and
found them annoying.

On the other hand, I know what its like when a company has hardware and
wants support for it :)

Is it perhaps a possible compromise to put in support but leave it defined
off by default?

Happy New Year BTW :)

Jack