From nobody Thu Oct 22 03:31:25 2026 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4j9BWx3DCSz6x45P for ; Thu, 22 Oct 2026 03:31:25 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4j9BWx1x1Qz4NLP for ; Thu, 22 Oct 2026 03:31:25 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1792639885; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=t4m2C6VcVhK1yqHY/DbQtgUc8SCvhM4rk8+5s5E4wKo=; b=fA+KHaTMcp7FHvrtpkeUfTuZ135arSvxp/HFJy/OmCydOPgYwsNXxfq6LixO+Lgvu2L5nD s5fKu1iWhoa6NX92PtQtud/wee6Nh86XwGnmxtm2tCkFEibROYMbn8JJIJBjG9z8IUvE1o 0FVipkFmxo0x4eLAazAw9wt4KcAuK2mgmmbBRavGHHle8vL94L0aEut7yrimRflnVqHizl F6xWuljp1d1FfAEo4LzY336CoYXNMepBn8sO7t+WuPtbj+ScsfSL/Dr2wbW/N2iRmFb7Ew H88BkJ+gNEtn6L0CLrdeSmlDJBEHfbNOWYwS0dUouvkn7XeqcdokmXAesJT1Jg== ARC-Seal: i=1; a=rsa-sha256; d=freebsd.org; s=dkim; cv=none; t=1792639885; b=nt7JMML0jMxAGIrpyPEqejpZzIlgT/26US4bxvkvBjyYhgRW5/L8wnWplsgCleblTkna3l 3RPDmIzjbycA3v/jAdKdRvn7MQQbeVz01ilsaaC87q3N7UbIXq2rklghYwTis8kgJzNpf6 NS0GQWT3aWy4VMwwu4Zjx+ze4V5MO5pdzR043+fgNtfmBbFWVCEnc9kyOGRys2qw92YRcK dItxz7lPtNnPj9Cc8CbdLm9ZQ5r2lDhClRp/ONdkJMUsUPQhUtm6VfN2Cvszi8A4OPzLaj HEl+AajnBaZcenFoH/zjXaA3sScKMCpf1vR98/tH+EfQ1NETuGi+zFPUik5sdw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1792639885; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=t4m2C6VcVhK1yqHY/DbQtgUc8SCvhM4rk8+5s5E4wKo=; b=DFnqa6KsN08OQr9zQBX6x7XRAhcVvPu1y8xQxVTD3TdJMheXAcH+ZFTcHUGHyVu19C0LEp V0GY1H/kPFupew2kH6OSEE71f62hgrk6SITAoIb7RfTASK9CgmxvW4rOC/lV9Wav5+3Gys iwBmZ5yhoEL4HmyMeOMsFneLlHMlxN+rGm06U0UGkpYoZBhjzpuGI9frj1hC/u5BQ95L/0 XlDebUN0VVvKvKx71/e/9A7WBX+uPAdMVDf20qYPPJP4SXJuQZff/TmAcm1l+Uuwq3flRz 6arcpvX6tkg5WHMFC1yVI0OxqEaTvMKPpJSu/0yYaZEL8LhiQHBtJmHpJoFyLQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4j9BWx1J0JzN18 for ; Thu, 22 Oct 2026 03:31:25 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 21448 by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Thu, 22 Oct 2026 03:31:25 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Gleb Smirnoff Subject: git: 7146ad04f18e - main - tcp: refactor tcp_autosndbuf() and associated sysctls List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org List-Id: Commit messages for the main branch of the src repository List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: glebius X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 7146ad04f18e7f0767a4d44cff43c1c67ea98cec Auto-Submitted: auto-generated Date: Thu, 22 Oct 2026 03:31:25 +0000 Message-Id: <6ad9838d.21448.54c59970@gitrepo.freebsd.org> The branch main has been updated by glebius: URL: https://cgit.FreeBSD.org/src/commit/?id=7146ad04f18e7f0767a4d44cff43c1c67ea98cec commit 7146ad04f18e7f0767a4d44cff43c1c67ea98cec Author: Gleb Smirnoff AuthorDate: 2026-10-22 03:27:47 +0000 Commit: Gleb Smirnoff CommitDate: 2026-10-22 03:27:47 +0000 tcp: refactor tcp_autosndbuf() and associated sysctls Rename tcp_sndbuf_autoscale() to tcp_autosndbuf(). This matches complementary tcp_autorcvbuf() that was factored out as a separate function a few months before tcp_sndbuf_autoscale() was. Move the large comment to the header of the function. Use unsigned types for all sizing sysctls and variables and use bool for the switching sysctl. Add a comment above net.inet.tcp.sendbuf_auto_lowat (default to off) providing a reference to its origins. Add DTrace probe tcp::send-autoresize, complementary to the existing tcp::receive-autoresize. Rack has its own function, just enable the DTrace probe there. Reviewed by: tuexen Differential Revision: https://reviews.freebsd.org/D60732 --- sys/netinet/in_kdtrace.c | 4 ++ sys/netinet/in_kdtrace.h | 1 + sys/netinet/tcp_output.c | 142 +++++++++++++++++++++--------------------- sys/netinet/tcp_stacks/bbr.c | 2 +- sys/netinet/tcp_stacks/rack.c | 5 +- sys/netinet/tcp_var.h | 10 +-- 6 files changed, 85 insertions(+), 79 deletions(-) diff --git a/sys/netinet/in_kdtrace.c b/sys/netinet/in_kdtrace.c index a44b4c8ed476..063ca3205aee 100644 --- a/sys/netinet/in_kdtrace.c +++ b/sys/netinet/in_kdtrace.c @@ -543,6 +543,10 @@ SDT_PROBE_DEFINE2_XLATE(tcp, , , receive__autoresize, "struct tcpcb *", "tcpsinfo_t *" , "u_int", "u_int"); +SDT_PROBE_DEFINE2_XLATE(tcp, , , send__autoresize, + "struct tcpcb *", "tcpsinfo_t *" , + "u_int", "u_int"); + SDT_PROBE_DEFINE5_XLATE(udp, , , receive, "void *", "pktinfo_t *", "struct inpcb *", "csinfo_t *", diff --git a/sys/netinet/in_kdtrace.h b/sys/netinet/in_kdtrace.h index 08fab3949b5c..01e6ed731982 100644 --- a/sys/netinet/in_kdtrace.h +++ b/sys/netinet/in_kdtrace.h @@ -442,6 +442,7 @@ SDT_PROBE_DECLARE(tcp, , , debug__output); SDT_PROBE_DECLARE(tcp, , , debug__user); SDT_PROBE_DECLARE(tcp, , , debug__drop); SDT_PROBE_DECLARE(tcp, , , receive__autoresize); +SDT_PROBE_DECLARE(tcp, , , send__autoresize); SDT_PROBE_DECLARE(udp, , , receive); SDT_PROBE_DECLARE(udp, , , send); diff --git a/sys/netinet/tcp_output.c b/sys/netinet/tcp_output.c index 09295544855f..d11a2b745382 100644 --- a/sys/netinet/tcp_output.c +++ b/sys/netinet/tcp_output.c @@ -107,31 +107,31 @@ SYSCTL_INT(_net_inet_tcp, OID_AUTO, tso, CTLFLAG_VNET | CTLFLAG_RW, &VNET_NAME(tcp_do_tso), 0, "Enable TCP Segmentation Offload"); -VNET_DEFINE(int, tcp_sendspace) = 1024*32; +VNET_DEFINE(u_int, tcp_sendspace) = 1024*32; #define V_tcp_sendspace VNET(tcp_sendspace) -SYSCTL_INT(_net_inet_tcp, TCPCTL_SENDSPACE, sendspace, CTLFLAG_VNET | CTLFLAG_RW, - &VNET_NAME(tcp_sendspace), 0, "Initial send socket buffer size"); +SYSCTL_UINT(_net_inet_tcp, TCPCTL_SENDSPACE, sendspace, + CTLFLAG_VNET | CTLFLAG_RW, &VNET_NAME(tcp_sendspace), 0, + "Initial send socket buffer size"); -VNET_DEFINE(int, tcp_do_autosndbuf) = 1; -SYSCTL_INT(_net_inet_tcp, OID_AUTO, sendbuf_auto, CTLFLAG_VNET | CTLFLAG_RW, - &VNET_NAME(tcp_do_autosndbuf), 0, - "Enable automatic send buffer sizing"); +VNET_DEFINE(bool, tcp_do_autosndbuf) = true; +SYSCTL_BOOL(_net_inet_tcp, OID_AUTO, sendbuf_auto, CTLFLAG_VNET | CTLFLAG_RW, + &VNET_NAME(tcp_do_autosndbuf), 0, "Enable automatic send buffer sizing"); -VNET_DEFINE(int, tcp_autosndbuf_inc) = 8*1024; -SYSCTL_INT(_net_inet_tcp, OID_AUTO, sendbuf_inc, CTLFLAG_VNET | CTLFLAG_RW, - &VNET_NAME(tcp_autosndbuf_inc), 0, - "Incrementor step size of automatic send buffer"); +VNET_DEFINE(u_int, tcp_autosndbuf_inc) = 8*1024; +SYSCTL_UINT(_net_inet_tcp, OID_AUTO, sendbuf_inc, CTLFLAG_VNET | CTLFLAG_RW, + &VNET_NAME(tcp_autosndbuf_inc), 0, + "Incrementor step size of automatic send buffer"); -VNET_DEFINE(int, tcp_autosndbuf_max) = 8*1024*1024; -SYSCTL_INT(_net_inet_tcp, OID_AUTO, sendbuf_max, CTLFLAG_VNET | CTLFLAG_RW, - &VNET_NAME(tcp_autosndbuf_max), 0, - "Max size of automatic send buffer"); +VNET_DEFINE(u_int, tcp_autosndbuf_max) = 8*1024*1024; +SYSCTL_UINT(_net_inet_tcp, OID_AUTO, sendbuf_max, CTLFLAG_VNET | CTLFLAG_RW, + &VNET_NAME(tcp_autosndbuf_max), 0, "Max size of automatic send buffer"); -VNET_DEFINE(int, tcp_sendbuf_auto_lowat) = 0; +/* See D11016, used by LimeLight. */ +VNET_DEFINE_STATIC(bool, tcp_sendbuf_auto_lowat) = false; #define V_tcp_sendbuf_auto_lowat VNET(tcp_sendbuf_auto_lowat) -SYSCTL_INT(_net_inet_tcp, OID_AUTO, sendbuf_auto_lowat, CTLFLAG_VNET | CTLFLAG_RW, - &VNET_NAME(tcp_sendbuf_auto_lowat), 0, - "Modify threshold for auto send buffer growth to account for SO_SNDLOWAT"); +SYSCTL_BOOL(_net_inet_tcp, OID_AUTO, sendbuf_auto_lowat, + CTLFLAG_VNET | CTLFLAG_RW, &VNET_NAME(tcp_sendbuf_auto_lowat), 0, + "Modify threshold for auto send buffer growth to account for SO_SNDLOWAT"); /* * Make sure that either retransmit or persist timer is set for SYN, FIN and @@ -508,7 +508,7 @@ after_sack_rexmit: /* len will be >= 0 after this point. */ KASSERT(len >= 0, ("[%s:%d]: len < 0", __func__, __LINE__)); - tcp_sndbuf_autoscale(tp, so, sendwin); + tcp_autosndbuf(tp, sendwin); /* * Decide if we can use TCP Segmentation Offloading (if supported by @@ -2119,61 +2119,59 @@ nospace: return (NULL); } +/* + * Automatic sizing of send socket buffer. Often the send buffer size is not + * optimally adjusted to the actual network conditions at hand (delay bandwidth + * product). Setting the buffer size too small limits throughput on links with + * high bandwidth and high delay (eg. trans-continental/oceanic links). + * Setting the buffer size too big consumes too much real kernel memory, + * especially with many connections on busy servers. + * + * The criteria to step up the send buffer one notch are: + * 1. receive window of remote host is larger than send buffer + * (with a fudge factor of 5/4th); + * 2. send buffer is filled to 7/8th with data (so we actually + * have data to make use of it); + * 3. send buffer fill has not hit maximal automatic size; + * 4. our send window (slow start and cogestion controlled) is + * larger than sent but unacknowledged data in send buffer. + * + * The remote host receive window scaling factor may limit the growing of the + * send buffer before it reaches its allowed maximum. + * + * It scales directly with slow start or congestion window and does at most one + * step per received ACK. This fast scaling has the drawback of growing the + * send buffer beyond what is strictly necessary to make full use of a given + * delay*bandwidth product. However testing has shown this not to be much of + * an problem. At worst we are trading wasting of available bandwidth (the + * non-use of it) for wasting some socket buffer memory. + * + * TODO: Shrink send buffer during idle periods together with congestion window. + */ void -tcp_sndbuf_autoscale(struct tcpcb *tp, struct socket *so, uint32_t sendwin) +tcp_autosndbuf(struct tcpcb *tp, u_int sendwin) { + struct socket *so = tptosocket(tp); + u_int lowat, newsize; - /* - * Automatic sizing of send socket buffer. Often the send buffer - * size is not optimally adjusted to the actual network conditions - * at hand (delay bandwidth product). Setting the buffer size too - * small limits throughput on links with high bandwidth and high - * delay (eg. trans-continental/oceanic links). Setting the - * buffer size too big consumes too much real kernel memory, - * especially with many connections on busy servers. - * - * The criteria to step up the send buffer one notch are: - * 1. receive window of remote host is larger than send buffer - * (with a fudge factor of 5/4th); - * 2. send buffer is filled to 7/8th with data (so we actually - * have data to make use of it); - * 3. send buffer fill has not hit maximal automatic size; - * 4. our send window (slow start and cogestion controlled) is - * larger than sent but unacknowledged data in send buffer. - * - * The remote host receive window scaling factor may limit the - * growing of the send buffer before it reaches its allowed - * maximum. - * - * It scales directly with slow start or congestion window - * and does at most one step per received ACK. This fast - * scaling has the drawback of growing the send buffer beyond - * what is strictly necessary to make full use of a given - * delay*bandwidth product. However testing has shown this not - * to be much of an problem. At worst we are trading wasting - * of available bandwidth (the non-use of it) for wasting some - * socket buffer memory. - * - * TODO: Shrink send buffer during idle periods together - * with congestion window. Requires another timer. Has to - * wait for upcoming tcp timer rewrite. - * - * XXXGL: should there be used sbused() or sbavail()? - */ - if (V_tcp_do_autosndbuf && so->so_snd.sb_flags & SB_AUTOSIZE) { - int lowat; - - lowat = V_tcp_sendbuf_auto_lowat ? so->so_snd.sb_lowat : 0; - if ((tp->snd_wnd / 4 * 5) >= so->so_snd.sb_hiwat - lowat && - sbused(&so->so_snd) >= - (so->so_snd.sb_hiwat / 8 * 7) - lowat && - sbused(&so->so_snd) < V_tcp_autosndbuf_max && - sendwin >= (sbused(&so->so_snd) - - (tp->snd_nxt - tp->snd_una))) { - if (!sbreserve_locked(so, SO_SND, - min(so->so_snd.sb_hiwat + V_tcp_autosndbuf_inc, - V_tcp_autosndbuf_max), curthread)) - so->so_snd.sb_flags &= ~SB_AUTOSIZE; + SOCK_RECVBUF_LOCK_ASSERT(so); + + if (!V_tcp_do_autosndbuf || (so->so_snd.sb_flags & SB_AUTOSIZE) == 0) + return; + + + lowat = V_tcp_sendbuf_auto_lowat ? so->so_snd.sb_lowat : 0; + if ((tp->snd_wnd / 4 * 5) >= so->so_snd.sb_hiwat - lowat && + sbused(&so->so_snd) >= + (so->so_snd.sb_hiwat / 8 * 7) - lowat && + sbused(&so->so_snd) < V_tcp_autosndbuf_max && + sendwin >= (sbused(&so->so_snd) - (tp->snd_nxt - tp->snd_una))) { + newsize = min(so->so_snd.sb_hiwat + V_tcp_autosndbuf_inc, + V_tcp_autosndbuf_max); + if (sbreserve_locked(so, SO_SND, newsize, curthread)) { + TCP_PROBE2(send__autoresize, tp, newsize); + } else { + so->so_snd.sb_flags &= ~SB_AUTOSIZE; } } } diff --git a/sys/netinet/tcp_stacks/bbr.c b/sys/netinet/tcp_stacks/bbr.c index a055f6d756ac..5169b6104393 100644 --- a/sys/netinet/tcp_stacks/bbr.c +++ b/sys/netinet/tcp_stacks/bbr.c @@ -12510,7 +12510,7 @@ recheck_resend: } /* len will be >= 0 after this point. */ KASSERT(len >= 0, ("[%s:%d]: len < 0", __func__, __LINE__)); - tcp_sndbuf_autoscale(tp, so, sendwin); + tcp_autosndbuf(tp, sendwin); /* * */ diff --git a/sys/netinet/tcp_stacks/rack.c b/sys/netinet/tcp_stacks/rack.c index 227c765b1cee..d817e97e4261 100644 --- a/sys/netinet/tcp_stacks/rack.c +++ b/sys/netinet/tcp_stacks/rack.c @@ -18915,8 +18915,11 @@ rack_sndbuf_autoscale(struct tcp_rack *rack) scaleup += so->so_snd.sb_hiwat; if (scaleup > V_tcp_autosndbuf_max) scaleup = V_tcp_autosndbuf_max; - if (!sbreserve_locked(so, SO_SND, scaleup, curthread)) + if (sbreserve_locked(so, SO_SND, scaleup, curthread)) { + TCP_PROBE2(send__autoresize, tp, scaleup); + } else { so->so_snd.sb_flags &= ~SB_AUTOSIZE; + } } } } diff --git a/sys/netinet/tcp_var.h b/sys/netinet/tcp_var.h index 75efd8eb92e0..daca017905ed 100644 --- a/sys/netinet/tcp_var.h +++ b/sys/netinet/tcp_var.h @@ -1304,12 +1304,12 @@ VNET_DECLARE(int, tcp_abc_l_var); VNET_DECLARE(uint32_t, tcp_ack_war_cnt); VNET_DECLARE(uint32_t, tcp_ack_war_time_window); VNET_DECLARE(u_int, tcp_autorcvbuf_max); -VNET_DECLARE(int, tcp_autosndbuf_inc); -VNET_DECLARE(int, tcp_autosndbuf_max); +VNET_DECLARE(u_int, tcp_autosndbuf_inc); +VNET_DECLARE(u_int, tcp_autosndbuf_max); VNET_DECLARE(int, tcp_bind_all_fibs); VNET_DECLARE(int, tcp_delack_enabled); -VNET_DECLARE(int, tcp_do_autosndbuf); VNET_DECLARE(bool, tcp_do_autorcvbuf); +VNET_DECLARE(bool, tcp_do_autosndbuf); VNET_DECLARE(int, tcp_do_ecn); VNET_DECLARE(int, tcp_do_lrd); VNET_DECLARE(int, tcp_do_prr); @@ -1343,7 +1343,7 @@ VNET_DECLARE(int, tcp_sack_globalmaxholes); VNET_DECLARE(int, tcp_sack_maxholes); VNET_DECLARE(int, tcp_sack_tso); VNET_DECLARE(int, tcp_sc_rst_sock_fail); -VNET_DECLARE(int, tcp_sendspace); +VNET_DECLARE(u_int, tcp_sendspace); VNET_DECLARE(int, tcp_udp_tunneling_overhead); VNET_DECLARE(int, tcp_udp_tunneling_port); VNET_DECLARE(struct inpcbinfo, tcbinfo); @@ -1404,6 +1404,7 @@ VNET_DECLARE(struct hhook_head *, tcp_hhh[HHOOK_TCP_LAST + 1]); #endif void tcp_autorcvbuf(struct tcpcb *, u_int); +void tcp_autosndbuf(struct tcpcb *, u_int); void tcp_account_for_send(struct tcpcb *, uint32_t, uint8_t, uint8_t, bool); int tcp_addoptions(struct tcpopt *, u_char *); struct tcpcb * @@ -1553,7 +1554,6 @@ void tcp_sack_lost_retransmission(struct tcpcb *, struct tcphdr *); int tcp_newreno(struct tcpcb *, struct tcphdr *); int tcp_compute_pipe(struct tcpcb *); uint32_t tcp_compute_initwnd(uint32_t); -void tcp_sndbuf_autoscale(struct tcpcb *, struct socket *, uint32_t); int tcp_stats_sample_rollthedice(struct tcpcb *tp, void *seed_bytes, size_t seed_len); int tcp_can_enable_pacing(void);