Re: git: 160b4b922b60 - main - Add real sched.h

From: Konstantin Belousov <kostikbel_at_gmail.com>
Date: Sat, 13 Nov 2021 12:21:01 UTC
On Sat, Nov 13, 2021 at 08:48:36AM +0000, Antoine Brodin wrote:
> On Thu, Nov 11, 2021 at 8:06 AM Konstantin Belousov <kostikbel@gmail.com> wrote:
> >
> > On Thu, Nov 11, 2021 at 07:29:19AM +0000, Antoine Brodin wrote:
> > > On Wed, Nov 10, 2021 at 7:34 PM Konstantin Belousov <kib@freebsd.org> wrote:
> > > >
> > > > The branch main has been updated by kib:
> > > >
> > > > URL: https://cgit.FreeBSD.org/src/commit/?id=160b4b922b6021848b6b48afc894d16b879b7af2
> > > >
> > > > commit 160b4b922b6021848b6b48afc894d16b879b7af2
> > > > Author:     Konstantin Belousov <kib@FreeBSD.org>
> > > > AuthorDate: 2021-10-22 15:17:21 +0000
> > > > Commit:     Konstantin Belousov <kib@FreeBSD.org>
> > > > CommitDate: 2021-11-10 19:18:53 +0000
> > > >
> > > >     Add real sched.h
> > > >
> > > >     It is required by IEEE Std 1003.1-2008 AKA POSIX.
> > > >
> > > >     Put some Linux compatibility stuff under BSD_VISIBLE namespace, in
> > > >     particular, sys/cpuset.h definitions.  Also, if user really want
> > > >     Linux compatibility, she can request cpu_set_t typedef with
> > > >     _WITH_CPU_SET_T define.
> > > >
> > > >     Reviewed by:    jhb
> > > >     Sponsored by:   The FreeBSD Foundation
> > > >     MFC after:      1 week
> > > >     Differential revision:  https://reviews.freebsd.org/D32901
> > > > ---
> > > >  include/Makefile |  5 +++--
> > > >  include/sched.h  | 46 ++++++++++++++++++++++++++++++++++++++++++++++
> > > >  2 files changed, 49 insertions(+), 2 deletions(-)
> > >
> > > Hi,
> > >
> > > It seems that this broke some ports:
> > > http://gohan03.nyi.freebsd.org/data/main-amd64-default-baseline/p913ccecf2a92_s448bcd01dc/logs/errors/ninja-1.10.2,2.log
> > > http://gohan03.nyi.freebsd.org/data/main-amd64-default-baseline/p913ccecf2a92_s448bcd01dc/logs/errors/ghc-8.10.7.log
> > > http://gohan03.nyi.freebsd.org/data/main-amd64-default-baseline/p913ccecf2a92_s448bcd01dc/logs/errors/openpgm-5.2.122_6.log
> > > http://gohan03.nyi.freebsd.org/data/main-amd64-default-baseline/p913ccecf2a92_s448bcd01dc/logs/errors/guile2-2.2.7_1.log
> > > http://gohan03.nyi.freebsd.org/data/main-amd64-default-baseline/p913ccecf2a92_s448bcd01dc/logs/errors/liburcu-0.12.1.log
> > > http://gohan03.nyi.freebsd.org/data/main-amd64-default-baseline/p913ccecf2a92_s448bcd01dc/logs/errors/guile-3.0.7.log
> > > http://gohan03.nyi.freebsd.org/data/main-amd64-default-baseline/p913ccecf2a92_s448bcd01dc/logs/errors/squid-4.15.log
> > > ...
> >
> > Could you please try this?  My intent was that cpu_set_t would be requested
> > as needed by consumers with _WITH_CPU_SET_T.
> >
> > commit b6bb08f09dc7cf2796c35015f5aee93dedf33018
> > Author: Konstantin Belousov <kib@FreeBSD.org>
> > Date:   Thu Nov 11 10:01:54 2021 +0200
> >
> >     sched.h: Make cpu_set_t compat typedef visible unconditionally
> >
> >     or rather, visible if BSD namespace is enabled, instead of requiring
> >     _WITH_CPU_SET_T define to expose it.  It seems a lot of code assumes
> >     that presence of sched_getaffinity() implies existence of cpu_set_t.
> >
> >     Reported by:    portmgr (antoine)
> >     Sponsored by:   The FreeBSD Foundation
> >     MFC after:      1 week
> 
> Hello,
> 
> With this patch,  this is better (less broken ports).
> But this seems to expose problems with other headers ( sys/bitset.h
> for instance)
> 
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/xplanet-1.3.1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/mixxx22-2.2.4_7.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/Sorcer-lv2-1.1.3_8.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/polygraph-4.13.0_2.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/stress-ng-0.13.06.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/abyss-2.3.1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/gcta-1.26.0_3.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/gemma-0.98.3_1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/iqtree-2.0.6_2.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/ncbi-blast+-2.12.0_1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/ncbi-cxx-toolkit-25.2.0.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/rna-STAR-2.7.3.a_1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/mysql57-server-5.7.36.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/mysql80-server-8.0.26_1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/mysqlwsrep57-server-5.7.34.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/percona57-pam-for-mysql-5.7.34.37.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/percona57-server-5.7.34.37.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/maliit-framework-2.1.1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/electron12-12.2.2.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/kdevelop-pg-qt-2.2.1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/libvirt-7.9.0.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/util-linux-2.36_2.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/tea-60.5.1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/jzintv-1.0.20200712,1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/libc6-shim-20210627.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/mame-0.226_1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/mess-0.226_1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/keeperrl-0.0.23_5.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/RetroArch-1.9.7.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/stonesoup-sdl-0.27.1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/stonesoup-0.27.1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/mypaint-2.0.0_3.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/gcc11-11.2.0.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/gcc11-devel-11.2.1.s20211009.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/gcc12-devel-12.0.0.s20211010.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/mono-5.10.1.57_3.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/mono5.10-5.10.1.57_3.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/mono5.20-5.20.1.34.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/mono6.8-6.8.0.123.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/exim-4.94.2_1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/exim-ldap2-4.94.2_1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/exim-monitor-4.94.2_1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/exim-mysql-4.94.2_1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/exim-postgresql-4.94.2_1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/exim-sa-exim-4.94.2+4.2.1_1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/exim-sqlite-4.94.2_1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/giacxcas-1.6.0.7_3.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/libnormaliz-3.9.0_1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/libmediainfo-21.09.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/pipewire-0.3.36_1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/aircrack-ng-1.5.2_2.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/ettercap-0.8.3.1,1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/glusterfs-8.4.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/nepenthes-0.2.2_16.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/relayd-5.5.20140810_8.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/yate-6.4.0_1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/miktex-21.8.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/bagel-1.2.2_6.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/qbox-1.73.3.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/openssh-portable-hpn-8.8.p1_1,1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/openssh-portable-8.8.p1_1,1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/flock-2.30.1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/intel-pcm-202109.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/intel-pcm-devel-g20211013,1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/zbackup-1.4.4_19.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/apt-cacher-ng-3.2_2.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/node10-10.24.1_2.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/proxygen-2021.11.01.00.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/squid-4.15.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/squid-devel-5.0.6.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/vdr-plugin-live-0.3.0_5.log

Sigh.  So ut cannot work this way.

Of course I can (and probably should) implement missing Linux CPU_* macros,
but BIT_* macros have incompatible API.  There is no other way than requiring
each program to opt-in by defining _WITH_CPU_SET_T, instead of trying to
be fully Linux-compatible there.

I will commit the following patch after buildworld finishes.  I am not sure
if it makes sense to do dedicated exp-run to see if I missed something, or
just wait for regular run to finish with updated world.  In any case, please
ping me.

commit 49c92416389668ab1b4c7cb37d4bdf4545369a58
Author: Konstantin Belousov <kib@FreeBSD.org>
Date:   Thu Nov 11 10:01:54 2021 +0200

    sched.h: Hide all Linux compat sched_* functions under _WITH_CPU_SET_T
    
    Instead of only hiding cpu_set_t compat typedef.
    Too many software packages assumes that sched_getaffinity() presence
    implies full source compatibility with glibc.  We can (and should)
    handle missing CPU_* macros, but there are incompatible BIT_* uses
    which cannot be fixed in src/.
    
    So hide everything under _WITH_CPU_SET_T, in particular, do not expose
    sched_get/setaffinity(), as well as CPU_* and BIT_* macros.  Consumers
    that want sched* functions must opt-in.
    
    Reported by:    portmgr (antoine)
    Sponsored by:   The FreeBSD Foundation
    MFC after:      1 week

diff --git a/include/sched.h b/include/sched.h
index 1c41cda79046..460d3bfed154 100644
--- a/include/sched.h
+++ b/include/sched.h
@@ -32,12 +32,9 @@
 #include <sys/cdefs.h>
 #include <sys/types.h>
 #include <sys/sched.h>
-#if __BSD_VISIBLE
-#include <sys/cpuset.h>
-#endif /* __BSD_VISIBLE */
-
 #if __BSD_VISIBLE
 #ifdef _WITH_CPU_SET_T
+#include <sys/cpuset.h>
 struct _cpuset;
 typedef struct _cpuset cpu_set_t;
 #endif /* _WITH_CPU_SET_T */
@@ -45,9 +42,11 @@ typedef struct _cpuset cpu_set_t;
 
 __BEGIN_DECLS
 #if __BSD_VISIBLE
+#ifdef _WITH_CPU_SET_T
 int sched_getaffinity(pid_t pid, size_t cpusetsz, cpuset_t *cpuset);
 int sched_setaffinity(int pid, size_t cpusetsz, const cpuset_t *cpuset);
 int sched_getcpu(void);
+#endif /* _WITH_CPU_SET_T */
 #endif /* __BSD_VISIBLE */
 __END_DECLS
 
diff --git a/lib/libc/gen/sched_getaffinity.c b/lib/libc/gen/sched_getaffinity.c
index 191374a3c5e3..5557d3d93b47 100644
--- a/lib/libc/gen/sched_getaffinity.c
+++ b/lib/libc/gen/sched_getaffinity.c
@@ -26,6 +26,7 @@
  * SUCH DAMAGE.
  */
 
+#define	_WITH_CPU_SET_T
 #include <sched.h>
 
 int
diff --git a/lib/libc/gen/sched_getcpu_gen.c b/lib/libc/gen/sched_getcpu_gen.c
index ef1bebc2915a..704075c0ae53 100644
--- a/lib/libc/gen/sched_getcpu_gen.c
+++ b/lib/libc/gen/sched_getcpu_gen.c
@@ -26,6 +26,7 @@
  * SUCH DAMAGE.
  */
 
+#define	_WITH_CPU_SET_T
 #include <sched.h>
 #include "libc_private.h"
 
diff --git a/lib/libc/gen/sched_setaffinity.c b/lib/libc/gen/sched_setaffinity.c
index 9368b1752b74..ad775b5dbce5 100644
--- a/lib/libc/gen/sched_setaffinity.c
+++ b/lib/libc/gen/sched_setaffinity.c
@@ -26,6 +26,7 @@
  * SUCH DAMAGE.
  */
 
+#define	_WITH_CPU_SET_T
 #include <sched.h>
 
 int
diff --git a/lib/libc/x86/sys/sched_getcpu_x86.c b/lib/libc/x86/sys/sched_getcpu_x86.c
index 13ba18ef5e36..5e3bd5311734 100644
--- a/lib/libc/x86/sys/sched_getcpu_x86.c
+++ b/lib/libc/x86/sys/sched_getcpu_x86.c
@@ -32,6 +32,7 @@
 #include <machine/sysarch.h>
 #include <x86/ifunc.h>
 #include <errno.h>
+#define	_WITH_CPU_SET_T
 #include <sched.h>
 #include "libc_private.h"