svn commit: r225458 - in stable/8/sys: dev/usb dev/usb/quirk dev/usb/storage sys

[Robert N. M. Watson]

On 10 Sep 2011, at 12:35, Robert N. M. Watson wrote:

>> I understand your point. The change is not breaking any KPIs towards 
>> userspace. The structure in question is only used within the kernel and kernel 
>> modules.
> 
> Right -- exactly my point. If this change breaks third-party compiled USB device drivers, then our current approach to device driver KBIs does not allow it to be MFC'd in this form. Are there ways you can reformulate the change to avoid breaking those drivers? Sometimes this can be done by adding new symbols, rather than replacing currently symbols, although mileage varies.

FYI, you can find our rules for modifications to many network stack data structures here:

  http://wiki.freebsd.org/NetKPIKBI

We're willing to make rather extensive changes to in-kernel data structures between major releases, but between minor releases we modify kernel structures frequently used by device drivers only according to specific rules.

Some are essentially immutable -- for example, struct mbuf, as assumptions about their size, layout, and interpretation are embedded in hundreds of kernel modules. Others, such as struct ipq, are only allocated by base kernel components, so you can safely extend them without breaking device drivers that incorporate earlier assumptions about layout (they just don't see new fields, which is often fine). Some contain explicit padding put in before the .0 release to allow new features to be added subject to constrained rules -- for example, we often put additional padding fields into struct inpcb so that we can add new well-defined features after the release, but can't grow the structure due to it being embedded.

Over time, we've been trying to make the ABI/KBI more robust by...

(1) Attempting to divorce kernel data structures from user data structures as much as possible, allowing kernel data structures to be changed without requiring application modification.

(2) Attempting to document which data structures can change and in what ways (vis the above wiki page) so that there's a mutual understanding about what can be done.

(3) Think about what features will be added early in major release cycles -- hence announcements of a "padding day" earlier in the 9.0 cycle, and explicit commits to add necessary padding.

(4) Avoid embedding base kernel data structures within module data structures or allowing use of stack allocation of those structures. For example, historically, BSD device drivers would embed "struct ifnet" inside their per-drive softc. Now, the base kernel is responsible for allocating struct ifnet's, and per-driver softc's have a pointer to the ifnet allocated by the kernel. This adds more indirection in some cases, but can significantly improve robustness.

(5) In more extreme cases, disallow direct field access on data structures entirely, requiring modules to treat them as opaque and access fields via accessor functions. We don't use this approach all that much, but have (for example) used it in inpcb locking, as it allowed us to migrate from mutexes to rwlocks transparently, as well as change other aspects of locking strategy. We use this in the MAC Framework and a number of other places to achieve higher levels of abstraction that will allow us to change (for example) label allocation strategies between minor versions. Apple takes this much further than us: in XNU, all access to mbuf fields is done via accessor functions, allowing them to change the layout of mbufs without breaking device drivers (such that device drivers can work with multiple major Mac OS X versions). This presumably comes at a measurable cost that they deem appropriate given the specifics of their end-user requirements.

Another advantage to documenting KPIs/KBIs in a formal way is that it actually gives us more flexibility to break other interfaces -- they aren't on the list we advertise as being stable. We would benefit from doing a much better job at this, but have been trying to be more structured in the network stack, especially, due to a high demand for fundamental code changes in support of new features.

Robert