Re: git: 4bd01d6ae016 - main - fts: refactor to use fd-relative operations internally
Date: Mon, 10 Aug 2026 06:49:25 UTC
On 10 Aug, Herbert J. Skuhra wrote: > On Mon, 10 Aug 2026, at 07:08, Don Lewis wrote: >> On 7 Aug, Don Lewis wrote: >>> On 4 Aug, Herbert J. Skuhra wrote: >>>> On Mon, 03 Aug 2026 21:13:08 +0200, Alan Somers wrote: >>>>> >>>>> The branch main has been updated by asomers: >>>>> >>>>> URL: https://cgit.FreeBSD.org/src/commit/?id=4bd01d6ae01632501b63438b8d9a401db9744a78 >>>>> >>>>> commit 4bd01d6ae01632501b63438b8d9a401db9744a78 >>>>> Author: Jitendra Bhati <bhatijitendra2022@gmail.com> >>>>> AuthorDate: 2026-06-12 17:07:55 +0000 >>>>> Commit: Alan Somers <asomers@FreeBSD.org> >>>>> CommitDate: 2026-08-03 19:12:28 +0000 >>>>> >>>>> fts: refactor to use fd-relative operations internally >>>>> >>>>> Replace all _open() calls with _openat() in __fts_open(), fts_read(), >>>>> and fts_children(). Replace statfs() with _fstatfs(). >>>>> >>>>> Add fts_dirfd to struct _ftsent, set to the file descriptor of the >>>>> parent directory. Callers can use openat(ent->fts_dirfd, ent->fts_name, >>>>> ...) to access files safely without relying on fts_accpath, which >>>>> enables programs in capability mode to open the files described by >>>>> _ftsent. >>>>> >>>>> This is a preparatory change for fts_openat() which will allow callers >>>>> to provide a pre-opened directory fd, enabling fts(3) traversal inside >>>>> Capsicum capability mode. >>>>> >>>>> Mirror all fts_open() changes to fts_open_b(). >>>>> >>>>> As a result of expanding _ftsend, publish new ELF symbol versions for >>>>> fts_openat and related functions. >>>>> >>>>> Sponsored by: Google LLC (GSoC 2026) >>>>> Reviewed by: asomers >>>>> Pull Request: https://github.com/freebsd/freebsd-src/pull/2303 >>>>> --- >>>>> include/fts.h | 2 + >>>>> lib/libc/gen/Makefile.inc | 1 + >>>>> lib/libc/gen/Symbol.map | 18 +- >>>>> lib/libc/gen/fts-compat15.c | 1355 +++++++++++++++++++++++++++++++++++++++++++ >>>>> lib/libc/gen/fts-compat15.h | 97 ++++ >>>>> lib/libc/gen/fts.3 | 18 +- >>>>> lib/libc/gen/fts.c | 49 +- >>>>> 7 files changed, 1519 insertions(+), 21 deletions(-) >>>> >>>> This change seems to break poudriere: >>>> >>>> [00:00:04] Recording filesystem state for prepkg...mtree: ./bin: No such file or directory >>>> [ERROR] Unhandled error! >>>> [00:00:04] Cleaning up >>>> [00:00:04] Unmounting file systems >>>> Exiting with status 1 >>> >>> I am seeing a different error with poudriere-devel: >>> >>> [00:00:01] Starting jail CURRENTamd64-default >>> Updating /var/run/os-release done. >>> [00:00:02] Will build as nobody:nobody (65534:65534) >>> [00:00:03] Ports supports: FLAVORS SUBPACKAGES SELECTED_OPTIONS >>> [00:00:03] Acquiring build logs lock for CURRENTamd64-default... done >>> [00:00:03] Logs: /var/poudriere/data/logs/bulk/CURRENTamd64-default/2026-08-07_15h29m37s >>> [00:00:03] Loading MOVED for /var/poudriere/data/.m/CURRENTamd64-default/ref/usr/ports >>> [00:00:03] Gathering ports metadata >>> [00:00:03] Warning: (misc/freebsd-release-manifests): jexec: jail_attach(41): Operation not permitted >>> [00:00:03] Warning: (misc/freebsd-release-manifests): Error: Error looking up dependencies for misc/freebsd-release-manifests >>> [00:00:03] Warning: (ports-mgmt/pkg): jexec: jail_attach(41): Operation not permitted >>> [00:00:03] Warning: (ports-mgmt/portconfig): jexec: jail_attach(41): Operation not permitted >>> [00:00:03] Warning: (ports-mgmt/pkg): Error: Error looking up dependencies for ports-mgmt/pkg >>> [00:00:03] Warning: (ports-mgmt/portconfig): Error: Error looking up dependencies for ports-mgmt/portconfig >>> [00:00:03] Error: /usr/local/share/poudriere/bulk.sh:gather_port_vars:184:Fatal errors encountered gathering ports metadata >>> [CURRENTamd64-default] [2026-08-07_15h29m37s] [crashed] Time: 00:00:01 >>> [00:00:03] Logs: /var/poudriere/data/logs/bulk/CURRENTamd64-default/2026-08-07_15h29m37s >>> >>> >>> When I upgraded my pkg build machine from >>> 3abd3d3cf3f034cb3d4250a95a8d6c41e72971ef to >>> 30ccf2f48c11e54fc0540510dcec7cd006a2c366, I found that poudriere-devel >>> started having jexec failures. I suspected a kernel problem and started >>> to set things up to track down the offending commit. I was surprised to >>> find that using the old world 3abd3d3cf3f034cb3d4250a95a8d6c41e72971ef >>> with the new 30ccf2f48c11e54fc0540510dcec7cd006a2c366 kernel did not >>> have this problem, which makes this look like a userland problem. >>> >>> I did a git bisect to find the problematic commit by doing: >>> make buildworld >>> make installworld >>> cd /usr/ports/ports-mgmt/poudriere-devel && make clean reinstall >>> poudriere bulk -j CURRENTamd64 -C ports-mgmt/poudriere-devel >>> at each step. I did the poudriere-devel build and reinstallation so >>> that it is propertly built against and linked to the newly installed >>> world. The result of the final poudriere bulk determines whether the >>> iteration is a pass or a fail. Any failures of the earlier steps are >>> handled as skips. >>> >>> When I ran git bisect, I got this result: >>> 4bd01d6ae01632501b63438b8d9a401db9744a78 is the first 'bad' commit >>> commit 4bd01d6ae01632501b63438b8d9a401db9744a78 >>> Author: Jitendra Bhati <bhatijitendra2022@gmail.com> >>> Date: Fri Jun 12 22:37:55 2026 +0530 >>> >>> fts: refactor to use fd-relative operations internally >>> >>> poudriere-devel runs jexec on the host and not in the jail. I think the >>> permission error should only happen if jexec is run by a user other than >>> root. I added some debug output to the poudriere internal scripts and >>> it looks like the uid is root. >>> >>> I don't know why jexec would be sensitive to fts, since jexec does not >>> use it: >>> %ldd /usr/sbin/jexec >>> /usr/sbin/jexec: >>> libjail.so.1 => /lib/libjail.so.1 (0x3704d89e7000) >>> libutil.so.10 => /lib/libutil.so.10 (0x3704d8ae3000) >>> libc.so.7 => /lib/libc.so.7 (0x3704d9af8000) >>> libsys.so.7 => /lib/libsys.so.7 (0x3704da328000) >>> [vdso] (0x3704d7516000) >>> >>> I see that a fix to fts was committed, but uupgrading to >>> commit 2ae8976a3f6a7e4deb1905326145f5c2575264f4 >>> Author: Gleb Smirnoff <glebius@FreeBSD.org> >>> Date: Fri Aug 7 13:46:22 2026 -0700 >>> >>> ocs_fc: return correct error code from ocs_hw_set_persistent_topology() >>> did not make a difference. >>> >>> I'm stumped. >> >> If I go back one commit to 9590878fca68e62c63d607da73139698e204d0f0 on >> both the host and vm, and rebuild poudriere-devel to use the old fts, >> poudriere-devel is able to build ports. >> >> If I then, update the host world to >> 4bd01d6ae01632501b63438b8d9a401db9744a78 (there are no kernel changes >> at that step), poudriere-devel still works. If I then rebuild and >> reinstall poudriere-devel so that it picks up the new fts API, I start >> seeing jexec errors again. >> >> I added some debug fprintfs to jail exec and its inputs look OK: >> [00:00:02] Loading MOVED for >> /var/poudriere/data/.m/CURRENTamd64-default/ref/usr/ports >> [00:00:03] Gathering ports metadata >> [00:00:03] Warning: (ports-mgmt/poudriere-devel): jexec >> CURRENTamd64-default jid=11 uid=0 euid=0 >> [00:00:03] Warning: (misc/freebsd-release-manifests): jexec >> CURRENTamd64-default jid=11 uid=0 euid=0 >> [00:00:03] Warning: (misc/freebsd-release-manifests): jexec >> jail_attach() errno=1 >> [00:00:03] Warning: (misc/freebsd-release-manifests): jexec: >> jail_attach(11): Operation not permitted >> [00:00:03] Warning: (ports-mgmt/pkg): jexec CURRENTamd64-default jid=11 >> uid=0 euid=0 >> [00:00:03] Warning: (misc/freebsd-release-manifests): Error: Error >> looking up dependencies for misc/freebsd-release-manifests >> [00:00:03] Warning: (ports-mgmt/pkg): jexec jail_attach() errno=1 >> [00:00:03] Warning: (ports-mgmt/portconfig): jexec CURRENTamd64-default >> jid=11 uid=0 euid=0 >> [00:00:03] Warning: (ports-mgmt/pkg): jexec: jail_attach(11): Operation >> not permitted >> [00:00:03] Warning: (ports-mgmt/portconfig): jexec jail_attach() errno=1 >> [00:00:03] Warning: (ports-mgmt/portconfig): jexec: jail_attach(11): >> Operation not permitted >> [00:00:03] Warning: (ports-mgmt/pkg): Error: Error looking up >> dependencies for ports-mgmt/pkg >> [00:00:03] Warning: (ports-mgmt/portconfig): Error: Error looking up >> dependencies for ports-mgmt/portconfig >> [00:00:03] Error: >> /usr/local/share/poudriere/bulk.sh:gather_port_vars:184:Fatal errors >> encountered gathering ports metadata >> >> but the call fo jail_attach() fails. > > But does this also happen with 74bd6fb1eb028d15df407efb62628253dab90aa5? I tried with 2ae8976a3f6a7e4deb1905326145f5c2575264f4, which is newer than 74bd6fb1eb028d15df407efb62628253dab90aa5 and still had the same problem. > Have you tried a clean build? > > The problem with the "too many open files" still exists. And now the main-weekly pkgbase is also affected. >