Re: git: 40b287054521 - main - mi_startup: Instrument the bubblesort with TSLOG

On Fri, 19 May 2023, 07:07 Hans Petter Selasky, <hps@selasky.org> wrote:

> On 5/19/23 15:49, Colin Percival wrote:
> > The branch main has been updated by cperciva:
> >
> > URL:
> https://cgit.FreeBSD.org/src/commit/?id=40b287054521f0a92e5ae9a26e6a87d17ee85eea
> >
> > commit 40b287054521f0a92e5ae9a26e6a87d17ee85eea
> > Author:     Colin Percival <cperciva@FreeBSD.org>
> > AuthorDate: 2023-05-19 13:46:42 +0000
> > Commit:     Colin Percival <cperciva@FreeBSD.org>
> > CommitDate: 2023-05-19 13:46:42 +0000
> >
> >      mi_startup: Instrument the bubblesort with TSLOG
> >
> >      The bubblesort of SYSINITs is currently responsible for 7% of the
> >      kernel boot time when booting a 1 CPU / 128 MB VM under Firecracker.
> >
> >      It needs to be replaced with a faster sort, but until that happens
> >      at least instrumenting it with TSLOG makes it show up in
> flamecharts.
> > ---
> >   sys/kern/init_main.c | 2 ++
> >   1 file changed, 2 insertions(+)
> >
> > diff --git a/sys/kern/init_main.c b/sys/kern/init_main.c
> > index 1974c4e68ce4..e4cb501bc57b 100644
> > --- a/sys/kern/init_main.c
> > +++ b/sys/kern/init_main.c
> > @@ -255,6 +255,7 @@ restart:
> >        * Perform a bubble sort of the system initialization objects by
> >        * their subsystem (primary key) and order (secondary key).
> >        */
> > +     TSENTER2("bubblesort");
> >       for (sipp = sysinit; sipp < sysinit_end; sipp++) {
> >               for (xipp = sipp + 1; xipp < sysinit_end; xipp++) {
> >                       if ((*sipp)->subsystem < (*xipp)->subsystem ||
> > @@ -266,6 +267,7 @@ restart:
> >                       *xipp = save;
> >               }
> >       }
> > +     TSEXIT2("bubblesort");
> >
> >       last = SI_SUB_COPYRIGHT;
> >   #if defined(VERBOSE_SYSINIT)
> >
>
> Hi Colin,
>
> If all kernel modules and the kernel could sort their SYSINIT() and
> SYSUNINIT() data at compile time, then all you need to do, is to merge
> two sorted lists, when loading new modules.
>
> Maybe this even could be part of the compiler's existing __constructor
> attribute. In FreeBSD we have an example of build boot loader modules,
> and statically sorting all sysinit data at compile time. See the tool I
> made many years ago for this purpose:
>
> stand/usb/tools/sysinit.c
>
> What do you think?
>
> --HPS
>


A somewhat minimal improvement can be seen here:
https://reviews.freebsd.org/D39916. I noticed the same slowness when
booting on qemu with tracing enabled.

Sorting at compile time would be ideal and in theory the priority argument
in the constructor attribute should work.

>