Re: git: 160b4b922b60 - main - Add real sched.h
- In reply to: Antoine Brodin : "Re: git: 160b4b922b60 - main - Add real sched.h"
- Go to: [ bottom of page ] [ top of archives ] [ this month ]
Date: Sat, 13 Nov 2021 12:21:01 UTC
On Sat, Nov 13, 2021 at 08:48:36AM +0000, Antoine Brodin wrote:
> On Thu, Nov 11, 2021 at 8:06 AM Konstantin Belousov <kostikbel@gmail.com> wrote:
> >
> > On Thu, Nov 11, 2021 at 07:29:19AM +0000, Antoine Brodin wrote:
> > > On Wed, Nov 10, 2021 at 7:34 PM Konstantin Belousov <kib@freebsd.org> wrote:
> > > >
> > > > The branch main has been updated by kib:
> > > >
> > > > URL: https://cgit.FreeBSD.org/src/commit/?id=160b4b922b6021848b6b48afc894d16b879b7af2
> > > >
> > > > commit 160b4b922b6021848b6b48afc894d16b879b7af2
> > > > Author: Konstantin Belousov <kib@FreeBSD.org>
> > > > AuthorDate: 2021-10-22 15:17:21 +0000
> > > > Commit: Konstantin Belousov <kib@FreeBSD.org>
> > > > CommitDate: 2021-11-10 19:18:53 +0000
> > > >
> > > > Add real sched.h
> > > >
> > > > It is required by IEEE Std 1003.1-2008 AKA POSIX.
> > > >
> > > > Put some Linux compatibility stuff under BSD_VISIBLE namespace, in
> > > > particular, sys/cpuset.h definitions. Also, if user really want
> > > > Linux compatibility, she can request cpu_set_t typedef with
> > > > _WITH_CPU_SET_T define.
> > > >
> > > > Reviewed by: jhb
> > > > Sponsored by: The FreeBSD Foundation
> > > > MFC after: 1 week
> > > > Differential revision: https://reviews.freebsd.org/D32901
> > > > ---
> > > > include/Makefile | 5 +++--
> > > > include/sched.h | 46 ++++++++++++++++++++++++++++++++++++++++++++++
> > > > 2 files changed, 49 insertions(+), 2 deletions(-)
> > >
> > > Hi,
> > >
> > > It seems that this broke some ports:
> > > http://gohan03.nyi.freebsd.org/data/main-amd64-default-baseline/p913ccecf2a92_s448bcd01dc/logs/errors/ninja-1.10.2,2.log
> > > http://gohan03.nyi.freebsd.org/data/main-amd64-default-baseline/p913ccecf2a92_s448bcd01dc/logs/errors/ghc-8.10.7.log
> > > http://gohan03.nyi.freebsd.org/data/main-amd64-default-baseline/p913ccecf2a92_s448bcd01dc/logs/errors/openpgm-5.2.122_6.log
> > > http://gohan03.nyi.freebsd.org/data/main-amd64-default-baseline/p913ccecf2a92_s448bcd01dc/logs/errors/guile2-2.2.7_1.log
> > > http://gohan03.nyi.freebsd.org/data/main-amd64-default-baseline/p913ccecf2a92_s448bcd01dc/logs/errors/liburcu-0.12.1.log
> > > http://gohan03.nyi.freebsd.org/data/main-amd64-default-baseline/p913ccecf2a92_s448bcd01dc/logs/errors/guile-3.0.7.log
> > > http://gohan03.nyi.freebsd.org/data/main-amd64-default-baseline/p913ccecf2a92_s448bcd01dc/logs/errors/squid-4.15.log
> > > ...
> >
> > Could you please try this? My intent was that cpu_set_t would be requested
> > as needed by consumers with _WITH_CPU_SET_T.
> >
> > commit b6bb08f09dc7cf2796c35015f5aee93dedf33018
> > Author: Konstantin Belousov <kib@FreeBSD.org>
> > Date: Thu Nov 11 10:01:54 2021 +0200
> >
> > sched.h: Make cpu_set_t compat typedef visible unconditionally
> >
> > or rather, visible if BSD namespace is enabled, instead of requiring
> > _WITH_CPU_SET_T define to expose it. It seems a lot of code assumes
> > that presence of sched_getaffinity() implies existence of cpu_set_t.
> >
> > Reported by: portmgr (antoine)
> > Sponsored by: The FreeBSD Foundation
> > MFC after: 1 week
>
> Hello,
>
> With this patch, this is better (less broken ports).
> But this seems to expose problems with other headers ( sys/bitset.h
> for instance)
>
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/xplanet-1.3.1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/mixxx22-2.2.4_7.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/Sorcer-lv2-1.1.3_8.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/polygraph-4.13.0_2.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/stress-ng-0.13.06.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/abyss-2.3.1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/gcta-1.26.0_3.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/gemma-0.98.3_1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/iqtree-2.0.6_2.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/ncbi-blast+-2.12.0_1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/ncbi-cxx-toolkit-25.2.0.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/rna-STAR-2.7.3.a_1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/mysql57-server-5.7.36.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/mysql80-server-8.0.26_1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/mysqlwsrep57-server-5.7.34.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/percona57-pam-for-mysql-5.7.34.37.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/percona57-server-5.7.34.37.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/maliit-framework-2.1.1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/electron12-12.2.2.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/kdevelop-pg-qt-2.2.1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/libvirt-7.9.0.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/util-linux-2.36_2.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/tea-60.5.1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/jzintv-1.0.20200712,1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/libc6-shim-20210627.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/mame-0.226_1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/mess-0.226_1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/keeperrl-0.0.23_5.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/RetroArch-1.9.7.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/stonesoup-sdl-0.27.1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/stonesoup-0.27.1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/mypaint-2.0.0_3.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/gcc11-11.2.0.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/gcc11-devel-11.2.1.s20211009.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/gcc12-devel-12.0.0.s20211010.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/mono-5.10.1.57_3.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/mono5.10-5.10.1.57_3.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/mono5.20-5.20.1.34.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/mono6.8-6.8.0.123.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/exim-4.94.2_1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/exim-ldap2-4.94.2_1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/exim-monitor-4.94.2_1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/exim-mysql-4.94.2_1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/exim-postgresql-4.94.2_1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/exim-sa-exim-4.94.2+4.2.1_1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/exim-sqlite-4.94.2_1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/giacxcas-1.6.0.7_3.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/libnormaliz-3.9.0_1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/libmediainfo-21.09.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/pipewire-0.3.36_1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/aircrack-ng-1.5.2_2.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/ettercap-0.8.3.1,1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/glusterfs-8.4.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/nepenthes-0.2.2_16.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/relayd-5.5.20140810_8.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/yate-6.4.0_1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/miktex-21.8.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/bagel-1.2.2_6.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/qbox-1.73.3.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/openssh-portable-hpn-8.8.p1_1,1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/openssh-portable-8.8.p1_1,1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/flock-2.30.1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/intel-pcm-202109.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/intel-pcm-devel-g20211013,1.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/zbackup-1.4.4_19.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/apt-cacher-ng-3.2_2.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/node10-10.24.1_2.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/proxygen-2021.11.01.00.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/squid-4.15.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/squid-devel-5.0.6.log
> http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/vdr-plugin-live-0.3.0_5.log
Sigh. So ut cannot work this way.
Of course I can (and probably should) implement missing Linux CPU_* macros,
but BIT_* macros have incompatible API. There is no other way than requiring
each program to opt-in by defining _WITH_CPU_SET_T, instead of trying to
be fully Linux-compatible there.
I will commit the following patch after buildworld finishes. I am not sure
if it makes sense to do dedicated exp-run to see if I missed something, or
just wait for regular run to finish with updated world. In any case, please
ping me.
commit 49c92416389668ab1b4c7cb37d4bdf4545369a58
Author: Konstantin Belousov <kib@FreeBSD.org>
Date: Thu Nov 11 10:01:54 2021 +0200
sched.h: Hide all Linux compat sched_* functions under _WITH_CPU_SET_T
Instead of only hiding cpu_set_t compat typedef.
Too many software packages assumes that sched_getaffinity() presence
implies full source compatibility with glibc. We can (and should)
handle missing CPU_* macros, but there are incompatible BIT_* uses
which cannot be fixed in src/.
So hide everything under _WITH_CPU_SET_T, in particular, do not expose
sched_get/setaffinity(), as well as CPU_* and BIT_* macros. Consumers
that want sched* functions must opt-in.
Reported by: portmgr (antoine)
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
diff --git a/include/sched.h b/include/sched.h
index 1c41cda79046..460d3bfed154 100644
--- a/include/sched.h
+++ b/include/sched.h
@@ -32,12 +32,9 @@
#include <sys/cdefs.h>
#include <sys/types.h>
#include <sys/sched.h>
-#if __BSD_VISIBLE
-#include <sys/cpuset.h>
-#endif /* __BSD_VISIBLE */
-
#if __BSD_VISIBLE
#ifdef _WITH_CPU_SET_T
+#include <sys/cpuset.h>
struct _cpuset;
typedef struct _cpuset cpu_set_t;
#endif /* _WITH_CPU_SET_T */
@@ -45,9 +42,11 @@ typedef struct _cpuset cpu_set_t;
__BEGIN_DECLS
#if __BSD_VISIBLE
+#ifdef _WITH_CPU_SET_T
int sched_getaffinity(pid_t pid, size_t cpusetsz, cpuset_t *cpuset);
int sched_setaffinity(int pid, size_t cpusetsz, const cpuset_t *cpuset);
int sched_getcpu(void);
+#endif /* _WITH_CPU_SET_T */
#endif /* __BSD_VISIBLE */
__END_DECLS
diff --git a/lib/libc/gen/sched_getaffinity.c b/lib/libc/gen/sched_getaffinity.c
index 191374a3c5e3..5557d3d93b47 100644
--- a/lib/libc/gen/sched_getaffinity.c
+++ b/lib/libc/gen/sched_getaffinity.c
@@ -26,6 +26,7 @@
* SUCH DAMAGE.
*/
+#define _WITH_CPU_SET_T
#include <sched.h>
int
diff --git a/lib/libc/gen/sched_getcpu_gen.c b/lib/libc/gen/sched_getcpu_gen.c
index ef1bebc2915a..704075c0ae53 100644
--- a/lib/libc/gen/sched_getcpu_gen.c
+++ b/lib/libc/gen/sched_getcpu_gen.c
@@ -26,6 +26,7 @@
* SUCH DAMAGE.
*/
+#define _WITH_CPU_SET_T
#include <sched.h>
#include "libc_private.h"
diff --git a/lib/libc/gen/sched_setaffinity.c b/lib/libc/gen/sched_setaffinity.c
index 9368b1752b74..ad775b5dbce5 100644
--- a/lib/libc/gen/sched_setaffinity.c
+++ b/lib/libc/gen/sched_setaffinity.c
@@ -26,6 +26,7 @@
* SUCH DAMAGE.
*/
+#define _WITH_CPU_SET_T
#include <sched.h>
int
diff --git a/lib/libc/x86/sys/sched_getcpu_x86.c b/lib/libc/x86/sys/sched_getcpu_x86.c
index 13ba18ef5e36..5e3bd5311734 100644
--- a/lib/libc/x86/sys/sched_getcpu_x86.c
+++ b/lib/libc/x86/sys/sched_getcpu_x86.c
@@ -32,6 +32,7 @@
#include <machine/sysarch.h>
#include <x86/ifunc.h>
#include <errno.h>
+#define _WITH_CPU_SET_T
#include <sched.h>
#include "libc_private.h"