Re: Kernel modules not loading on 15-prerelease

On Sep 4, 2025, at 07:14, bob prohaska <fbsd@www.zefox.net> wrote:

> On Wed, Sep 03, 2025 at 06:18:03PM -0700, Mark Millard wrote:
>> On Sep 3, 2025, at 17:39, bob prohaska <fbsd@www.zefox.net> wrote:
>> 
>>> Most list  depends on kernel.1500061 (1500061,1500061), perhaps
>>> 10% list 63 and one lists some of each (!)
>>> /boot/kernel/usb.ko
>>> depends on kernel.1500061 (1500061,1500061)
>>> depends on kernel.1500061 (1500061,1500061)
>>> depends on kernel.1500061 (1500061,1500061)
>>> depends on kernel.1500061 (1500061,1500061)
>>> depends on kernel.1500061 (1500061,1500061)
>>> depends on kernel.1500061 (1500061,1500061)
>>> depends on kernel.1500061 (1500061,1500061)
>>> depends on kernel.1500061 (1500061,1500061)
>>> depends on kernel.1500063 (1500063,1500063)
>>> depends on kernel.1500063 (1500063,1500063)
>>> The above looks wrong to me.....
>> 
>> Lack of META_MODE (with filemon working) and
>> WITHOUT_CLEAN tends to mean more things are
>> not rebuilt. Likely you are seeing the
>> distinction between rebuilt things and
>> not-rebuilt things.
>> 
>> /boot/kernel/usb.ko is certainly interesting.
>> Apparently, different parts of it have their
>> own dependencies on the kernel and only some
>> parts were rebuilt before being linked
>> together. (I do not know the details.)
>> 
>>>> Going in a different direction, I'll remind:
>>>> 
>>>>    The environment of make(1) for the build can be controlled via the
>>>>    SRC_ENV_CONF variable, which defaults to /etc/src-env.conf.  Some
>>>>    examples that may only be set in this file are WITH_DIRDEPS_BUILD, and
>>>>    WITH_META_MODE, and MAKEOBJDIRPREFIX as they are environment-only
>>>>    variables.
>>>> 
>>>> I'll note that I use a env prefix before a make
>>>> command (in a script):
>>>> 
>>>> env __MAKE_CONF="/usr/home/root/src.configs/make.conf" \
>>>> SRCCONF="/dev/null" SRC_ENV_CONF="/usr/home/root/src.configs/src.conf.aarch64-nodbg-clang.aarch64-host" \
>>>> MAKEOBJDIRPREFIX="/usr/obj/BUILDs/main-aarch64-usr_src-nodbg-clang" \
>>>> WITH_META_MODE=yes \
>>>> time -l make $*
>>>> 
>>>> You reference elsewhere using: -DWITH_META_MODE
>>>> That looks wrong to me. For make:
>>>> 
>>>>    -D variable
>>>>            Define variable to be 1, in the global scope.
>>>> 
>>>> is defining a make variable, not an environment variable.
>>>> I'm not saying that you should use -e but, as evidence,
>>>> note the wording:
>>>> 
>>>>    -e      Let environment variables override global variables within
>>>>            makefiles.
>>>> 
>>>> So: global variables are not environment variables.
>>>> 
>>>> As far as I can tell, you have not been using META_MODE
>>>> based on what you report doing.
> 
> For the most recent (past week) experiments that's correct.

I'm saying more than that: Any time you used just
-DWITH_META_MODE you were not correctly specifying
that META_MODE would be used: that specification
has to be via an environment variable called
WITH_META_MODE, not via a global make variable
called WITH_META_MODE .

>>>> 
>>>> Absent META_MODE use, the system's recent change to using
>>>> WITHOUT_CLEAN by default for buildworld and buildkernel
>>>> can lead to more lack of updates of files that should be
>>>> updated.
>>>> 
>>>>> At the same time, I see
>>>>> # uname -K
>>>>> 1500063
>>>>> # uname -U
>>>>> 1500061
>>>> 
>>>> -U output is a separate issue from the kernel
>>>> and module version requirement mismatches as
>>>> far as I can tell.
>>>> 
>>>> uname gets the -U putput text via:
>>>> 
>>>> static void                                      
>>>> native_uservers(void)
>>>> {                                                
>>>>       static char buf[128];
>>>> 
>>>>       snprintf(buf, sizeof(buf), "%d", __FreeBSD_version);
>>>>       uservers = buf;
>>>> }
>>>> 
>>>> Recently WITHOUT_CLEAN became the default. So
>>>> its status for your build depends on the timing
>>>> of the commit that you used.
>>>> 
>>>> A WITHOUT_CLEAN (non-META_MODE) build might not
>>>> have rebuilt uname. What does the following report:
>>>> 
>>>> # file /usr/obj/usr/src/arm.armv7/usr.bin/uname/uname
>>>> 
>>>> My paths are different, but here is an example (I've not
>>>> built for armv7 in a long time):
>>>> 
>>>> # file /usr/obj/BUILDs/main-CA7-nodbg-clang/usr/main-src/arm.armv7/usr.bin/uname/uname
>>>> /usr/obj/BUILDs/main-CA7-nodbg-clang/usr/main-src/arm.armv7/usr.bin/uname/uname: ELF 32-bit LSB executable, ARM, EABI5 version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1, FreeBSD-style, for FreeBSD 15.0 (1500048), not stripped
>>>> 
>>>> Note that "(1500048)". Which number does your build
>>>> of uname show for what is in your build tree (not
>>>> what is installed)?
>>>> 
>>> # file /usr/obj/usr/src/arm.armv7/usr.bin/uname/uname
>>> /usr/obj/usr/src/arm.armv7/usr.bin/uname/uname: ELF 32-bit LSB executable, ARM, EABI5 version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1, FreeBSD-style, for FreeBSD 15.0 (1500063), not stripped
>>> #
>> 
>> Hmm. I'm surprised at the 1500063 when uname -U reported
>> 1500061. I wonder if it has since been rebuilt.
>> 
>>>>> How can the kernel and modules get out of sync?
>>>> 
>>>> WITHOUT_CLEAN (recently by default) use without META_MODE
>>>> to cause updates based on better dependency information.
>>>> 
>>>>> Even then, /usr/src was updated  followed by an immediate
>>>>> buildworld/buildkernel.
>>>> 
>>>> You do not mention installations explicitly, just
>>>> builds. Was the world installed too?
>>> Yes, sorry for the ambiguity.
>>> 
>>>> 
>>>>> In that case, how did world and kernel
>>>>> become mismatched and remain so for a week or more?
>>>> 
>>>> See above about correctly providing META_MODE and
>>>> about the recent change to have WITHOUT_CLEAN by
>>>> default (less rebuilds by default).
>>>> 
>>>>> The suggestion to delete /usr/obj is looking unavoidable, are
>>>>> additional steps warranted?
>>>> 
>>>> Given that you seem to have been doing doing
>>>> builds where META_MODE was not helping to
>>>> track dependencies, I'd recommend a from-scratch
>>>> META_MODE based build to get the tracking
>>>> information fully in place for use in later
>>>> builds.
>>>> 
>>>> If you are to do META_MODE builds, builds that
>>>> disable META_MODE mess up its dependency tracking
>>>> information by not updating it. Thus, only if
>>>> you stop using META_MODE overall should to avoid
>>>> using META_MODE for a specific build.
>>>> 
>>> 
>>> When filemon stopped loading I ran a few build/install 
>>> cycles without meta mode. I thought that would be harmless, 
>>> but maybe not. Right now meta mode is running with the 
>>> NO_FILEMON option.
>> 
>> Good point, META_MODE does not do as much when
>> filemon is not operational.  Once filemon is
>> operational, you will likely need to synchronize
>> META_MODE again so that it gets all the information.
>> 
> Ahh, that is a significant detail. I expected "no filemon"
> to perhaps be slower, but to be functionally equivalent.

filemon provides information that is otherwise not
available. But META_MODE still does more than not
using META_MODE, even without filemon.

>>> I'll let it run till it either finishes or crashes, then
>>> decide whether to remove /usr/obj after rebooting.
>>> 
>>> One possible culprit is of course me: From time to time
>>> the machine seemingly stalls, with no response to keyboard
>>> nor debugger escape. I've routinely power-cycles the machine
>>> to recover, confident that internal checks would catch any
>>> inconsistencies that might be introduced by ungraceful
>>> shutdown. Might my confidence be misplaced?
>> 
>> I do not know that you have many alternatives.
> 
> Might there be some program that can independently
> audit installed binaries against those in /usr/obj?

One could install a buildworld into a separate 
(non-nested?) directory tree and do a "diff -rq"
and look for expected vs. unexpected files with
differences. (The new tree would not have tailored
configuration files, /usr/local/ , and such, for
example.)

But failures during a build leave an incomplete,
possibly corrupted build. Comparing the pre-build
live system to the partial-build is not expected
to match well. So not a likely-useful path for
such a failure timing.

Do you ever have such a failures during
installworld ? buildworld ?

>> One thing using pkgbase style installs does is leave checksum
>> data around for everything pkg installs. "pkg check -sa",
>> if it is executable, will try to checksum everything handled
>> via pkg and report on mismatches, both base packages and port
>> packages. There is no equivalent for any other installation
>> style that I know of.
> 
> It looks like pkgbase is a subject of much discussion and at
> least some contention. Is there a tutorial somewhere of how
> to use it in a simple self-hosted system? From the page at
> https://wiki.freebsd.org/action/show/pkgbase?action=show&redirect=PkgBase
> it seems a good deal more involved than "make installworld" 8-)
> 

Having an rpi2 build the packages as an extra step might have
its own issues with resources or time. I've not experimented
with building packages (yet?) --so I've not experimented with
installing my own builds of such. I've experimented with
installing and using official pkgbase builds. Personal kernel
and world builds I still build like I did before pkgbase.
installkernel to distinct kernel naming and installworld just
to a directory tree for use with chroot or the like, still
booting the pkgbase'd world.


===
Mark Millard
marklmi at yahoo.com