Re: Security Run Output

Gerard E. Seibert wrote:
> For quite some time now, I have been receiving a warning message of
> 1025 packages with mismatched checksums in the daily "Security Run
> Output" email. They are all prefixed with "py39-"
> 
Because Python packages that build using the older method of directly 
executing setup.py, aka distutils, have not yet been switched to not 
compile bytecode during the build. The trigger to compile/remove 
bytecode after all pkg(8) transactions complete had been reverted due to 
an overreaction and opportunity to make the process more resilient. 
These particular checksum mismatches are completely harmless.

tl;dr https://wiki.freebsd.org/Python/CompiledPackages

Basically, bytecode is a CPython implementation detail that, due to how 
CPython is implemented in general, is not deterministic despite all the 
verification methods available. At the baseline, the comparison is based 
on mtime of the .py files and magic number of the CPython interpreter on 
the *target* system. If any one of them does not match between an 
existing bytecode file, the referenced .py and interpreter's magic 
number, the bytecode is invalidated and recompiled on import, defeating 
the purpose of compiling and including them as part of the package. 
Bytecode exists to cache the results of interpreted instructions for 
faster execution, but become filesystem pollutants when not used.

If a module is imported where the executing user (usually root if 
installed via our pkg(8)) has write privileges to the module's location, 
and the existing bytecode is invalidated, guess what, new bytecode is 
(over-)written to the same location (following PEP-3147) and a checksum 
mismatch ensues anyway.

The "new" method of compiling bytecode after all pkg(8) transactions 
complete is to match what Python's own packaging tooling (pip et al) do. 
System-type package managers have built and included bytecode as part of 
the packages as a crutch.

-- 
Charlie Li
…nope, still don't have an exit line.