Re: [RFC] An idea for general kernel post-processing automation in FreeBSD

On Tue, 23 May 2023 18:53:47 -0500
Kyle Evans <kevans@freebsd.org> wrote:

> On Tue, May 23, 2023 at 6:39?PM John Baldwin <jhb@freebsd.org> wrote:
> >
> > On 5/23/23 6:30 AM, Mark Johnston wrote:
> > > On Tue, May 23, 2023 at 08:00:53AM +0200, Hans Petter Selasky wrote:
> > >> Hi Warner,
> > >>
> > >> When you make systems and use them, you get bound by them. If a system makes
> > >> a certain solution advantageous, it gets chosen, independent if it is a good
> > >> solution or not.
> > >>
> > >> The reason for our disagreement is simply this:
> > >>
> > >> You are thinking like a politican and want to be popular in the current
> > >> system of FreeBSD.
> > >>
> > >> I'm thinking like a mathematican regardless of what makes me popular in the
> > >> current system of FreeBSD.
> > >>
> > >>
> > >> Why not state that from the start? Implementing Quick Sort in qsort() and
> > >> using that everywhere is a political decision.
> > >>
> > >> This is not a technical fight, it is a political fight.
> > >
> > > I don't agree.  From my point of view, Warner's position is the
> > > pragmatic one.  We have perhaps 2% Linux's number of active
> > > developers[*], but we are relatively much larger in terms of
> > > performance, code complexity, size, etc..
> > >
> > > Layering and simplicity of design are some of the main tools we have to
> > > counteract this imbalance.  It helps reduce the amount of time
> > > developers spend on bugs that aren't directly related to what they are
> > > doing.  It helps us think about and predict the behaviour of the system
> > > using only intuition.  This tradeoff can mean that we do not provide the
> > > best possible performance in all cases, but that's often a reasonable
> > > tradeoff in this rather non-mathematical world where we do not have
> > > infinite resources.
> > >
> > > The existing SYSINIT bubblesort is a good example of this tradeoff.  At
> > > the time it was written, it made sense to choose a simple, "good enough"
> > > solution and move on.  Even now, this simple solution just works and is
> > > perfectly acceptable on the vast majority of systems where FreeBSD is
> > > deployed.
> > >
> > > I see this attitude reflected in Warner's and others' replies, and I
> > > agree with it.  I suspect that Warner, rather than wanting to be popular
> > > per se, is replying in his own self-interest, which is to spend zero
> > > time debugging anything that might break if we start doing extra work at
> > > compile time to sort linker sets.
> > >
> > > It could be that some specific use-case will make your proposal more
> > > attractive.  I don't mean to suggest that the topic should be closed
> > > forever.  So far though, having read the thread and D40193, I'm not
> > > really sold.
> > >
> > > [*] I'm sure this number can vary wildly depending on how you define it;
> > > my impression from reading Linux lists for a while is just that we have
> > > way fewer people who understand core pieces of the system.
> >
> > +1.
> >
> > Also, the concern over the runtime for the potential worst-case for qsort
> > seems _way_ overblown compared to the actual unsorted arrays one will get
> > out of the linker.  The only possible downside of qsort that matters in
> > this case is the stack usage.
> >
> 
> IMO sorting at runtime is a minor feature by itself, too. I mentioned
> in one of the reviews somewhere, we have in the past have had bugs due
> to poor specification of sysinit order within a subsystem -- I'm
> recalling a specific one that wasn't even that long ago, maybe three
> or four years, that manu@ had hit because he did or didn't include
> some device in his config that ended up shifting link order of other
> objects and reversing two sysinits into an order that wasn't actually
> functional.

 Yes this problem was a pain to debug (I mean just to understand the
actual problem), and iirc it was never solved, I only had the issue on
one machine that I don't use anymore and the problem went away and
re-apears from time to time when sysinits where added.

> We can still add some bits to test that (preferably a
> tunable to reverse order within a subsystem rather than having to
> re-link the kernel) even with sorting at link-time, but it sure does
> look a lot less fragile if we have to sort it anyways and we just
> reverse one criteria.
> 
> Thanks,
> 
> Kyle Evans
> 


-- 
Emmanuel Vadot <manu@bidouilliste.com> <manu@freebsd.org>