Re: generated sources Was: git: 408957613bc0 - main - Regen

On 5/23/24 2:37 PM, Brooks Davis wrote:
> On Thu, May 23, 2024 at 11:20:08AM -0700, Gleb Smirnoff wrote:
>>    Hi,
>>
>> sorry for slightly diverting the discussion, but looks like this thread is the
>> right place...
>>
>> On Wed, May 22, 2024 at 12:55:24PM -0700, John Baldwin wrote:
>> J> I don't think it's worth reverting at this point.  For `make sysent` I would
>> J> suggest 'sysent: Regen' as a commit log in the future.  We typically use
>> J> 'src.conf.5: Regen' after running `make makeman`.
>>
>> On Wed, May 22, 2024 at 05:05:14PM +0000, Brooks Davis wrote:
>> B> This is more or less standard practice for generated files.  These
>> B> commits are of basically no interest.  Historically they had to exist
>> B> because the source file $FreeBSD$ entries were embedded in the generated
>> B> files and thus had to be final.  That isn't true any more so I often just
>> B> include them in the origional commit unless I'm doing a lot of minor
>> B> changes to the source files.
>>
>> Is it possible to generate the files during normal kernel build? That would
>> require lua packages installed, but we already almost require git installed to
>> work with FreeBSD sources. Absence of lua can be checked in Makefile and a
>> verbose error message would be printed. Or may be we can use the small lua that
>> we have in the loader? Sorry if the very last suggestion was stupid - really
>> don't have any knowledge in the area.
> 
> TL;DR: No one likes the current state of affairs, but it's complicated
> to fix or we'd have done it long ago.
> 
> We'd like to generate these files at build time, but at least
> sys/sys/syscall.mk is difficult to handle as it's required to build libc
> so I belive we'd need to arrange to create it before entering lib/libc.
> The others seem straightforward, but right now we've got a single script
> to generate everything and we'd want to be selective about generating
> things so we did it in the right place.  I know there's some work in
> progress to turn sys/tools/makesyscalls.lua into something more
> library-like, but it's not done.  (I've not been helping by adding
> things to makesyscalls.lua and thus moving the goalposts...)
> 
>> After the xz/liblzma backdoor all the generated code committed manually makes
>> me concerned, as it has 0% chance to get reviewed by community.
> 
> This is a valid, but not very practical concern.  We check that these
> are in sync routinely in github CI and could easily factor those checks
> out to do in jenkins if we wanted more immediate post-commit feedback.

Also, FWIW, the discussion I've seen on some other lists (e.g. for GDB/binutils)
is actually favoring checking in auto-generated files like configure since it's
easier to validate what files are used to generate a release tarball as a counter
to the xz case where one of the issues is that the release tarball was justn't
a straight tarball of a git tag which gave an opportunity for the actor to inject
other changes into the release tarball.

-- 
John Baldwin