Re: [RFC] An idea for general kernel post-processing automation in FreeBSD

On Tue, May 23, 2023 at 6:39 PM John Baldwin <jhb@freebsd.org> wrote:
>
> On 5/23/23 6:30 AM, Mark Johnston wrote:
> > On Tue, May 23, 2023 at 08:00:53AM +0200, Hans Petter Selasky wrote:
> >> Hi Warner,
> >>
> >> When you make systems and use them, you get bound by them. If a system makes
> >> a certain solution advantageous, it gets chosen, independent if it is a good
> >> solution or not.
> >>
> >> The reason for our disagreement is simply this:
> >>
> >> You are thinking like a politican and want to be popular in the current
> >> system of FreeBSD.
> >>
> >> I'm thinking like a mathematican regardless of what makes me popular in the
> >> current system of FreeBSD.
> >>
> >>
> >> Why not state that from the start? Implementing Quick Sort in qsort() and
> >> using that everywhere is a political decision.
> >>
> >> This is not a technical fight, it is a political fight.
> >
> > I don't agree.  From my point of view, Warner's position is the
> > pragmatic one.  We have perhaps 2% Linux's number of active
> > developers[*], but we are relatively much larger in terms of
> > performance, code complexity, size, etc..
> >
> > Layering and simplicity of design are some of the main tools we have to
> > counteract this imbalance.  It helps reduce the amount of time
> > developers spend on bugs that aren't directly related to what they are
> > doing.  It helps us think about and predict the behaviour of the system
> > using only intuition.  This tradeoff can mean that we do not provide the
> > best possible performance in all cases, but that's often a reasonable
> > tradeoff in this rather non-mathematical world where we do not have
> > infinite resources.
> >
> > The existing SYSINIT bubblesort is a good example of this tradeoff.  At
> > the time it was written, it made sense to choose a simple, "good enough"
> > solution and move on.  Even now, this simple solution just works and is
> > perfectly acceptable on the vast majority of systems where FreeBSD is
> > deployed.
> >
> > I see this attitude reflected in Warner's and others' replies, and I
> > agree with it.  I suspect that Warner, rather than wanting to be popular
> > per se, is replying in his own self-interest, which is to spend zero
> > time debugging anything that might break if we start doing extra work at
> > compile time to sort linker sets.
> >
> > It could be that some specific use-case will make your proposal more
> > attractive.  I don't mean to suggest that the topic should be closed
> > forever.  So far though, having read the thread and D40193, I'm not
> > really sold.
> >
> > [*] I'm sure this number can vary wildly depending on how you define it;
> > my impression from reading Linux lists for a while is just that we have
> > way fewer people who understand core pieces of the system.
>
> +1.
>
> Also, the concern over the runtime for the potential worst-case for qsort
> seems _way_ overblown compared to the actual unsorted arrays one will get
> out of the linker.  The only possible downside of qsort that matters in
> this case is the stack usage.
>

IMO sorting at runtime is a minor feature by itself, too. I mentioned
in one of the reviews somewhere, we have in the past have had bugs due
to poor specification of sysinit order within a subsystem -- I'm
recalling a specific one that wasn't even that long ago, maybe three
or four years, that manu@ had hit because he did or didn't include
some device in his config that ended up shifting link order of other
objects and reversing two sysinits into an order that wasn't actually
functional. We can still add some bits to test that (preferably a
tunable to reverse order within a subsystem rather than having to
re-link the kernel) even with sorting at link-time, but it sure does
look a lot less fragile if we have to sort it anyways and we just
reverse one criteria.

Thanks,

Kyle Evans