Re: port binary dumping core on recent head in poudriere [tmpfs corruptions involving blocks of zeros that should not be all zeros]

On 26 Nov 2024, at 13:32, Dimitry Andric <dim@FreeBSD.org> wrote:
> 
> On 26 Nov 2024, at 11:19, Dag-Erling Smørgrav <des@FreeBSD.org> wrote:
>> 
>> Mark Millard <marklmi@yahoo.com> writes:
>>> From inside a bulk -i where I did a manual make command
>>> after it built and installed libsass.so.1.0.0 . The
>>> manual make produced a /wrkdirs/ :
>>> [...]
>>> So the original creation looks okay. But . . .
>>> [...]
>>> So: The later, staged copy is a bad copy. Both are in the
>>> tmpfs. So copying to the staging area makes a corrupted
>>> copy inside the same tmpfs. After that, further copies of
>>> staging's bad copy can be expected to be messed up.
>> 
>> This and the fact that it happens on 14 and 15 but not on 13 strongly
>> suggests an issue wth `copy_file_range(2)`, since `install(1)` in 14 and
>> 15 (but not in 13) now uses `copy_file_range(2)` if at all possible.
>> 
>> My educated guess is that hole detection doesn't work reliably for files
>> that have had holes filled while memory-mapped, so `copy_file_range(2)`
>> thinks there is a hole where there isn't one and skips some of the data
>> when `install(1)` uses it to copy the library from `${WRKSRC}` to
>> `${STAGEDIR}`.  This may or may not be specific to tmpfs.
>> 
>> You may want to try applying the attached patch to your FreeBSD 14 and
>> 15 jails.  It prevents `cp(1)` and `install(1)` from trying to use
>> `copy_file_range(2)`.
> 
> Yes, tmpfs is indeed the culprit (or at least involved). I have had USE_TMPFS=localbase in my poudriere.conf for a long time, since otherwise my build machine would run out of memory very quickly, so I didn't encounter any issues.
> 
> Now I changed it to USE_TMPFS=yes, rebuilt only textproc/libsass and textproc/sassc, and then after reinstalling those packages:
> 
> $ /usr/local/bin/sassc
> Segmentation fault

And after applying Dag-Erling's patch to disable copy_file_range for cp and install, it works correctly again.

-Dimitry