Re: Current ULE state and plans (was Re: HMP scheduling on FreeBSD)

Hi,

> > Code is not free.  As I've just said, the majority of the burden when maintaining schedulers is testing with different scenarios, possibly also working on tools to automate what we can, which however currently seem far from being able to model the whole reality.  Who is going to perform that?  Are we going to continue to ship under-tested code, creating among users a false sense of real support?
> 
> And do you claim FreeBSD is doing the above now for ULE?

I am absolutely doing that, at least with the means I have.

For example, I did that before and after the runqueue changes in ULE to support really distinct 32 priority levels for realtime and idletime.  I spent a lot of time analyzing the effect of nice values, polishing Jeff's initial `late` testing program to be able to simulate more scenarios, actually obtaining traces of real workloads at some companies to produce synthetic tests, and also having them report their metrics of interest before and after the changes.

Is that enough?  Probably not, as workloads can be varied and schedulers' behaviors sometimes complex.  And here, how much a scheduler is widespread is simply an advantage for its overall quality (provided people report problems, that problems are handled, etc.; as usual).

I did not do that for 4BSD, just basic smoke tests and that's all.
 
> To be honest, the NFS code in FreeBSD is "free" (at least in
> the sense that no one spends any $$$ on it) and is undertested.
> (Personally, I would never claim it is supported. Want software
> that is supported, then pay someone or some company $$$ to
> support it. Imho, testing/support occurs when companies like
> Netflix invest in those resources and FreeBSD benefits as a
> side effect. However, for large parts of FreeBSD, there is no
> one paying $$$ to test/support it.)
> --> I honestly believe most of the code in FreeBSD qualifies
>       as under-tested.

I rather think NFS is a particular beast.  Besides the fact you're essentially the only person working on it, it is a big code base (big files, and big, entangled functions, but also at a higher-level multiple components with cross-dependencies) with lots of special cases.  It is supposed to interoperate with completely alien implementations that have their own quirks.  It also relies too much on VFS internals to my taste, potentially causing problems as they change.  I'm glad you're still maintaining it, as when you stop, it may be difficult to find someone to pass the baton to, and even if we find someone, he will have to spend a lot of time just to get up to speed.

We certainly have under-tested and under-maintained code, but I don't think most of our "useful" code is like that, "useful" in the sense of exercized by a large fraction of FreeBSD users, or even by a small fraction but that eagerly depends on it.

Involving companies that are willing to do some testing is very important, yes.

> Long ago when I did run FreeBSD NFS server code in a
> production (or near-production) environment, I always
> found 4BSD worked better that ULE and always used it
> for those production kernels.
> But that was a long time ago, rick

"That was long ago", coupled with "ULE has changed significantly" in the last decade, and "ULE is going to change soon again", is exactly my point.

Thanks and regards.

-- 
Olivier Certner