From nobody Wed Sep 07 15:59:39 2022 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4MN6Qc4VQDz4bWfJ for ; Wed, 7 Sep 2022 15:59:52 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-qt1-f175.google.com (mail-qt1-f175.google.com [209.85.160.175]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4MN6Qb5JHQz3v2K for ; Wed, 7 Sep 2022 15:59:51 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-qt1-f175.google.com with SMTP id cr9so10736303qtb.13 for ; Wed, 07 Sep 2022 08:59:51 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:subject:message-id:date:from:in-reply-to:references:mime-version :x-gm-message-state:from:to:cc:subject:date; bh=7OPAYTtszQhb468ZWEYWXzb6gXEpm9MUAz6LMijkHrY=; b=x/xFgEDhDfBXaSEmacwUJZoCFTuTPjbMkNAO9dQ7czjETIBNbldb4ADllsReVfw4t7 Ht8/cmgPpOti28XZlsLw31WPzR/iqHYjDB2Ju7Vw2i915VpEbmUeSi5ZqTB1p4+ideER tfoSDiiFvNW0jNH5bYE8MFySPv22he4yS3N4/xdQ4I+649RrzfgpIFUihFvKvYM7egUf xQg7sJY0UP2mgiWwhElVVcjd3kyCuKxoJd8Yitub0vrt5K9mnDJuOBOJdWOxPLiS/5Tc NwRnaJOsAIpkSjqFI8s3RsJg3I6gB6y+jIYvtT7qIqaHvcoyjzJNg3/LSScUwO26ix+9 /vjA== X-Gm-Message-State: ACgBeo2nXFJ8XI99piLSD3oxosetMF9frEth2jwJMLFN2yRafV6sxerL MlJOaAfU6QTG2D4jsThCZ+mOFiqcfPjuXwfQ6UM= X-Received: by 2002:ac8:5f4e:0:b0:345:45d:3701 with SMTP id y14-20020ac85f4e000000b00345045d3701mt3838152qta.139.1662566390150; Wed, 07 Sep 2022 08:59:50 -0700 (PDT) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 References: <20220907145524.7C9881FB@slippy.cwsent.com> In-Reply-To: <20220907145524.7C9881FB@slippy.cwsent.com> From: Alan Somers Date: Wed, 7 Sep 2022 09:59:39 -0600 Message-ID: Subject: Re: Header symbols that shouldn't be visible to ports? Cc: Alan Somers , Konstantin Belousov , FreeBSD CURRENT Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4MN6Qb5JHQz3v2K X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of asomers@gmail.com designates 209.85.160.175 as permitted sender) smtp.mailfrom=asomers@gmail.com X-Spamd-Result: default: False [0.00 / 15.00]; MISSING_TO(2.00)[]; SUBJECT_ENDS_QUESTION(1.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; FORGED_SENDER(0.30)[asomers@freebsd.org,asomers@gmail.com]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17:c]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; DMARC_NA(0.00)[freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[209.85.160.175:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.85.160.175:from]; RCPT_COUNT_THREE(0.00)[3]; FREEFALL_USER(0.00)[asomers]; RCVD_COUNT_TWO(0.00)[2]; R_DKIM_NA(0.00)[]; RCVD_TLS_LAST(0.00)[]; TO_DN_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; FROM_NEQ_ENVFROM(0.00)[asomers@freebsd.org,asomers@gmail.com]; MIME_TRACE(0.00)[0:+]; ARC_NA(0.00)[]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FREEMAIL_CC(0.00)[freebsd.org,gmail.com] X-ThisMailContainsUnwantedMimeParts: N On Wed, Sep 7, 2022 at 8:55 AM Cy Schubert wrote: > > In message om> > , Alan Somers writes: > > On Sat, Sep 3, 2022 at 11:10 PM Konstantin Belousov wro > > te: > > > > > > On Sat, Sep 03, 2022 at 10:19:12AM -0600, Alan Somers wrote: > > > > Our /usr/include headers define a lot of symbols that are used by > > > > critical utilities in the base system like ps and ifconfig, but aren't > > > > stable across major releases. Since they aren't stable, utilities > > > > built for older releases won't run correctly on newer ones. Would it > > > > make sense to guard these symbols so they can't be used by programs in > > > > the ports tree? There is some precedent for that, for example > > > > _WANT_SOCKET and _WANT_MNTOPTNAMES. > > > _WANT_SOCKET is clearly about exposing parts of the kernel definitions > > > for userspace code that wants to dig into kernel structures. Similarly > > > for _WANT_MNTOPTNAMES, but in fact this thing is quite stable. The > > > definitions are guarded by additional defines not due to their instability, > > > but because using them in userspace requires (much) more preparation from > > > userspace environment, which is either not trivial (_WANT_SOCKET) or > > > contradicts to standartized use of the header (_WANT_MNTOPTNAMES + > > > sys/mount.h). > > > > > > > > > > > I'm particular, I'm thinking about symbols like the following: > > > > MINCORE_SUPER > > > Why this symbol should be hidden? It is implementation-defined and > > > intended to be exposed to userspace. All MINCORE_* not only MINCORE_SUPER > > > are under BSD_VISIBLE braces, because POSIX does not define the symbols. > > > > Because it isn't stable. It changed for example in rev 847ab36bf22 > > for 13.0. Programs using the older value (including virtually every > > Rust program) won't work on 13.0 and later. > > > > > > > > > TDF_* > > > These symbols coming from non-standard header sys/proc.h. If userspace > > > includes the header, it is already outside any formal standard, and I > > > do not see a reason to make the implementation more convoluted there. > > > > > > > PRI_MAX* > > > > PRI_MIN* > > > > PI_*, PRIBIO, PVFS, etc > > > > IFCAP_* > > > These are all implementation-specific and come from non-standard headers, > > > unless I am mistaken, then please correct me. > > > > > > > RLIM_NLIMITS > > > > IFF_* > > > Same. > > > > > > > *_MAXID > > > This is too broad. > > > > I'm talking about symbols like IPV6CTL_MAXID, which record the size of > > sysctl lists. Obviously, these symbols can't be stable, and probably > > aren't useful outside of the base system. > > > > > > > > > > > > > Clearly delineating private symbols like this would ease the > > > > maintenance burden on languages that rely on FFI, like Ruby and Rust. > > > > FFI basically assumes that symbols once defined will never change. > > > > > > Why e.g. sys/proc.h is ever consumed by FFI wrappers? > > > > I should add a little detail. Rust uses FFI to access C functions, > > and #define'd constants are redefined in the Rust bindings. For most > > Rust programs, the build process doesn't check the contents of > > /usr/include in any way. Instead, all of that stuff is hard-coded in > > the Rust bindings. That makes cross-compiling a breeze! But it does > > cause problems when the C library changes. Adding a new symbol, like > > copy_file_range, isn't so bad. If your Rust program doesn't use it, > > then the Rust binding will become an unused symbol and get eliminated > > by the linker. If your Rust program does use it OTOH, then it will be > > resolved by the dynamic linker at runtime - if you're running on > > FreeBSD 13 or newer. Otherwise, your program will fail to run. A > > bigger problem is with symbols that change. For example, the 64-bit > > inode stuff. Rust programs still use a FreeBSD 11 ABI (we're working > > on that). But other symbols change more frequently. Things like > > PRI_MAX_REALTIME can change between any two releases. That creates a > > big maintenance burden to keep track of them in the FFI bindings. And > > they also aren't very useful in cross-compiled programs targeting a > > FreeBSD 11 ABI. Instead, they really need to have bindings > > automatically generated at build time. That's possible, but it's not > > the default. > > This is exactly what happened with DMD D. When 64-bit statfs was introduced > all DMD D compiled programs failed to run and recompiling didn't help. The > DMD upstream failed to understand the problem. Eventually the port had to > be removed. Ouch. Does DMD not use ELF symbol versioning? That's what Rust does. So all Rust programs are still using the FreeBSD 11 version of "struct statfs", and the libc function they link to is "statfs@FBSD_1.0" instead of "statfs". > > > > > So what the Rust community really needs is a way to know which symbols > > will be stable across releases, and which might vary. Are you > > suggesting that anything from a non-POSIX header file should be > > considered variable? > > > > Rust and every other community. > > > -- > Cheers, > Cy Schubert > FreeBSD UNIX: Web: http://www.FreeBSD.org > NTP: Web: https://nwtime.org > > e^(i*pi)+1=0 > >