git: 67f3eac87e41 - main - callout: optionally spread callouts to all CPUs

From: Andrew Gallatin <gallatin_at_FreeBSD.org>
Date: Fri, 13 Nov 2026 17:54:40 UTC
The branch main has been updated by gallatin:

URL: https://cgit.FreeBSD.org/src/commit/?id=67f3eac87e41339ce7244e5234f142626972ee82

commit 67f3eac87e41339ce7244e5234f142626972ee82
Author:     Andrew Gallatin <gallatin@FreeBSD.org>
AuthorDate: 2026-11-13 17:12:02 +0000
Commit:     Andrew Gallatin <gallatin@FreeBSD.org>
CommitDate: 2026-11-13 17:54:35 +0000

    callout: optionally spread callouts to all CPUs
    
    By default, if the caller has not used one of the _on() or _cpucpu()
    variants of the callout API, the system assigns the callout to CPU0.
    
    Having most callouts running on CPU0 can be problematic in terms of
    the amount of CPU time spent executing callouts can have an unfair
    impact to CPU0, and a critical callout (like lacp_tick) is certain
    to land on the same CPU as everything else, so it is more likely
    to be delayed.
    
    This change introduces a change to callout_init* where if
    kern.callout_spread is non-zero, callouts initialized after SMP startup
    are distributed among CPUs.
    
    This is left as disabled by default, as it may reveal any latent
    synchronization bugs that were previously hidden, e.g., two callout handlers
    which were always accidentally serialized against each other before may now
    run concurrently.  It also may cause a slight increase in power consumption
    for mobile users.
    
    Reviewed by: markj, (+ mjg who reviewed the first version and suggested
                       the simpler mechanism used in this commit)
    Sponsored by: Netflix
    Differential Revision: https://reviews.freebsd.org/D61361
---
 share/man/man9/callout.9 | 10 ++++++++--
 sys/kern/kern_timeout.c  |  9 +++++++--
 2 files changed, 15 insertions(+), 4 deletions(-)

diff --git a/share/man/man9/callout.9 b/share/man/man9/callout.9
index d90d896a81e1..fe5f538d0abd 100644
--- a/share/man/man9/callout.9
+++ b/share/man/man9/callout.9
@@ -27,7 +27,7 @@
 .\" ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
 .\" POSSIBILITY OF SUCH DAMAGE.
 .\"
-.Dd September 18, 2026
+.Dd November 13, 2026
 .Dt CALLOUT 9
 .Os
 .Sh NAME
@@ -455,8 +455,14 @@ functions can be used.
 The callout subsystem provides a softclock thread for each CPU in the system.
 Callouts are assigned to a single CPU and are executed by the softclock thread
 for that CPU.
-Initially,
+By default,
 callouts are assigned to CPU 0.
+If the
+.Va kern.callout_spread
+sysctl is set to a non-zero value,
+callouts initialized after SMP startup are distributed among CPUs.
+This sysctl is also a loader tunable and defaults to zero.
+Changing it does not reassign existing callouts.
 The
 .Fn callout_reset_on ,
 .Fn callout_reset_sbt_on ,
diff --git a/sys/kern/kern_timeout.c b/sys/kern/kern_timeout.c
index 05856921df1d..effa20a34ce3 100644
--- a/sys/kern/kern_timeout.c
+++ b/sys/kern/kern_timeout.c
@@ -110,6 +110,11 @@ static int ncallout;
 SYSCTL_INT(_kern, OID_AUTO, ncallout, CTLFLAG_RDTUN | CTLFLAG_NOFETCH, &ncallout, 0,
     "Number of entries in callwheel and size of timeout() preallocation");
 
+static bool __read_mostly callout_spread;
+SYSCTL_BOOL(_kern, OID_AUTO, callout_spread, CTLFLAG_RWTUN,
+    &callout_spread, 0,
+    "Distribute initial callout CPU assignments after SMP startup");
+
 #ifdef	RSS
 static int pin_default_swi = 1;
 static int pin_pcpu_swi = 1;
@@ -1357,7 +1362,7 @@ callout_init(struct callout *c, int mpsafe)
 		c->c_lock = &Giant.lock_object;
 		c->c_iflags = 0;
 	}
-	c->c_cpu = cc_default_cpu;
+	c->c_cpu = callout_spread ? curcpu : cc_default_cpu;
 }
 
 void
@@ -1377,7 +1382,7 @@ _callout_init_lock(struct callout *c, struct lock_object *lock, int flags)
 	*c = (struct callout ){
 		.c_lock = lock,
 		.c_iflags = flags,
-		.c_cpu = cc_default_cpu,
+		.c_cpu = callout_spread ? curcpu : cc_default_cpu,
 	};
 }