Re: generated sources Was: git: 408957613bc0 - main - Regen

On Thu, May 23, 2024 at 11:20:08AM -0700, Gleb Smirnoff wrote:
>   Hi,
> 
> sorry for slightly diverting the discussion, but looks like this thread is the
> right place...
> 
> On Wed, May 22, 2024 at 12:55:24PM -0700, John Baldwin wrote:
> J> I don't think it's worth reverting at this point.  For `make sysent` I would
> J> suggest 'sysent: Regen' as a commit log in the future.  We typically use
> J> 'src.conf.5: Regen' after running `make makeman`.
> 
> On Wed, May 22, 2024 at 05:05:14PM +0000, Brooks Davis wrote:
> B> This is more or less standard practice for generated files.  These
> B> commits are of basically no interest.  Historically they had to exist
> B> because the source file $FreeBSD$ entries were embedded in the generated
> B> files and thus had to be final.  That isn't true any more so I often just
> B> include them in the origional commit unless I'm doing a lot of minor
> B> changes to the source files.
> 
> Is it possible to generate the files during normal kernel build? That would
> require lua packages installed, but we already almost require git installed to
> work with FreeBSD sources. Absence of lua can be checked in Makefile and a
> verbose error message would be printed. Or may be we can use the small lua that
> we have in the loader? Sorry if the very last suggestion was stupid - really
> don't have any knowledge in the area.

TL;DR: No one likes the current state of affairs, but it's complicated
to fix or we'd have done it long ago.

We'd like to generate these files at build time, but at least
sys/sys/syscall.mk is difficult to handle as it's required to build libc
so I belive we'd need to arrange to create it before entering lib/libc.
The others seem straightforward, but right now we've got a single script
to generate everything and we'd want to be selective about generating
things so we did it in the right place.  I know there's some work in
progress to turn sys/tools/makesyscalls.lua into something more
library-like, but it's not done.  (I've not been helping by adding
things to makesyscalls.lua and thus moving the goalposts...)

> After the xz/liblzma backdoor all the generated code committed manually makes
> me concerned, as it has 0% chance to get reviewed by community.

This is a valid, but not very practical concern.  We check that these
are in sync routinely in github CI and could easily factor those checks
out to do in jenkins if we wanted more immediate post-commit feedback.

> Just if my suggestion sounds stupid and there is a good reason to have what we
> have - an automatic generation followed by a manual commit - then please
> explain this reason.

Not stupid, just not easy to change.

-- Brooks