Optimizations.

[Poul-Henning Kamp]

In message <20030516002105.K40030-100000 at haldjas.folklore.ee>, Narvi writes:
>On Thu, 15 May 2003, Marcel Moolenaar wrote:
>> On Thu, May 15, 2003 at 02:30:33PM +0200, Pawel Jakub Dawidek wrote:
>> > Maybe is time to think about some 'optimiztion team' creation?
>> I think I don't want to see this happen based on professional
>> experience.
>Well, supposedly any such team would need to start by creating a set of
>tools and benchmarks [...]

If I have to be honest, I think this is the wrong way to approach the
subject, if on no other ground than on the 3. rule of optimizations
("Don't do it yet").

While it would be nice to have a set of "blessed benchmarks" canned
and ready to run, we should learn from the lmbench fiasco in Linux
that such benchmarks can easier mislead than lead.

My personal professional experience with optimizations or "Performance
management" as it was called, is that you want some very rigid
_functional_ testcases, which must pass at any one time so you don't
unnoticed loose functionality to optimizations.

We also know that the main performance issue is Giant, Giant and Giant.

So I really think the band of merry men we are talking about, if they
can be interested, would do much more good if they would start out
building functional and regression tests for our most critical
facilities.

I can't speak for the other heavy-duty guys in the project, but I
would personally be _really_ _REALLY_ grateful if I could "cd
/usr/src ; make test" and know that a significant fraction of our
functionality worked if it returned a zero exit code.


-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.