TCL and Unicode

[Shawn Webb]

On Mon, Mar 22, 2021 at 06:32:29PM +0100, Dimitry Andric wrote:
> On 22 Mar 2021, at 14:40, Shawn Webb <shawn.webb at hardenedbsd.org> wrote:
> > 
> > I'm tracking down a regression in ports regarding TCL and unicode. The
> > primary victim of the problem is databases/sqlite3. Note that I use
> > freebsd-ports on github as my upstream, so I'll be using git commit
> > hashes from that repo.
> > 
> > The sqlite3 build failure can be seen at [1].
> > 
> > If I revert commit 787aad81fc79d441fb0c9a750e6e33b6c0ea7ac6, sqlite3
> > builds fine. I noticed a few key changes from that commit:
> > 
> > The build of sqlite3 depends on TCL: instead of using the distfile
> > that has the autoconf artifacts pre-generated, the distfile without
> > the autoconf artifacts is used (changing from
> > sqlite-autoconf-3340100.tar.gz to sqlite-src-3340100.zip). This
> > means that the TCL-based autoconf artifacts must be generated
> > locally. At least, partially. It seems that the only part of the
> > build that depends on TCL is the sqlite3_analyzer.
> > 
> > Admittedly, this change is somewhat confusing to me. I'm having
> > somewhat of a hard time knowing whether it's TCL or sqlite3 itself
> > as the main culprit.
> > 
> > Any guidance is appreciated.
> > 
> > [1]: http://ci-08.md.hardenedbsd.org/data/hardenedbsd-current_amd64-local/2021-03-21_13h53m43s/logs/errors/sqlite3-3.34.1,1.log
> 
> Hi Shawn,
> 
> It builds fine for me locally on 14.0-CURRENT (as of ~2 days ago), and
> indeed the sqlite3.c file is now dynamically generated by tcl.
> 
> In your CI failure case, it looks like something is inserting blobs of
> zero bytes into the resulting file, though? So either the file system
> is going bad, or tcl is outputting nonsense, for some reason. At least,
> I think you'll have to do some investigations in that direction...

Looks like compiling lang/tcl86 with the following CFLAGS causes tcl
to output invalid unicode for some reason:

-ftrivial-auto-var-init=zero
-enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang

Thanks for the hint!

-- 
Shawn Webb
Cofounder / Security Engineer
HardenedBSD

https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.freebsd.org/pipermail/freebsd-ports/attachments/20210322/4c80bf54/attachment.sig>