From nobody Wed Nov 24 18:39:36 2021 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 564D2189DF67; Wed, 24 Nov 2021 18:39:38 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4HzqYN5Btsz3qcf; Wed, 24 Nov 2021 18:39:36 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 86F686E99; Wed, 24 Nov 2021 18:39:36 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 1AOIda5Q095089; Wed, 24 Nov 2021 18:39:36 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 1AOIdaY5095088; Wed, 24 Nov 2021 18:39:36 GMT (envelope-from git) Date: Wed, 24 Nov 2021 18:39:36 GMT Message-Id: <202111241839.1AOIdaY5095088@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Mark Johnston Subject: git: ecbbe831445a - main - netinet: Deduplicate most in_cksum() implementations List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: markj X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: ecbbe831445a4b94279e5932828d60076e4ca444 Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1637779176; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=MXIiSzmuUJ5q/mE9HA5iQlxKGNnBNYRMNM9/amHTMcA=; b=fWWCGsrOafsW8Psf7iRZIaICl61IeRr/rIGGhcODLvu/I89FR1konxQQSBR5NEu71xdZB+ RErYaW6U6v+uxxpdoZRMBAG9DnlvqutaAsFxAJLP02MHS5lAfxV97nTZv2lPorlcpxD5sj 76mioNuCRHEUzt/wTB1GZceoeeYNpj18OmAPJO4isAN4nyfv2NAeMMPPsSbsw6HcTKEPHU 3XReijCwPeRiUhHUj7RsXwJwDT2Tzi7/uWV8tDkbdrKuordIyWnikRDlRv7eqdK0aCcFI1 imqLsynT5BsiOLRswqzWsVO2J/vZVo8YzKXtbSZsBYnmZFryQ9dlo9S5bi+2Qw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1637779176; a=rsa-sha256; cv=none; b=L1p1Kf8i+om6YaQ78/mRGM0Tw12gGi0AKByERtdnsvq2yXqUWhhsaQuA2qT3bEvzrsjN3R uxJGigTqUjmOD1WtACNdrjD7hjL8Y59WP4fUFbyOmq9DIJy1fdWIn0Yt6lgQgAhenq2oBI qR11VfEal4zzlqmNL2pr+PfykYH5HLF1mIfBfYDlaiY29ZRoibrTZLmPEJMxYb4KtYnP4l 3UNcqj0AHvN9Z3Fu2dWdgX5/6NrhAFj5/aXRf+eAWdieaDDsheaZDfZ+W+1bqgH4SkcJ0U pxHmeno+zu0IiRCh8r/Qdgm2dhhr6DorktkG8fc3npQIVhC0gS0OV5ekyKCYYA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by markj: URL: https://cgit.FreeBSD.org/src/commit/?id=ecbbe831445a4b94279e5932828d60076e4ca444 commit ecbbe831445a4b94279e5932828d60076e4ca444 Author: Mark Johnston AuthorDate: 2021-11-24 18:19:44 +0000 Commit: Mark Johnston CommitDate: 2021-11-24 18:31:16 +0000 netinet: Deduplicate most in_cksum() implementations in_cksum() and related routines are implemented separately for each platform, but only i386 and arm have optimized versions. Other platforms' copies of in_cksum.c are identical except for style differences and support for big-endian CPUs. Deduplicate the implementations for the rest of the platforms. This will make it easier to implement in_cksum() for unmapped mbufs. On arm and i386, define HAVE_MD_IN_CKSUM to mean that the MI implementation is not to be compiled. No functional change intended. Reviewed by: kp, glebius MFC after: 1 week Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D33095 --- sys/amd64/amd64/in_cksum.c | 243 ---------------------- sys/arm/arm/{in_cksum.c => in_cksum_machdep.c} | 0 sys/arm/include/in_cksum.h | 3 + sys/arm64/arm64/in_cksum.c | 241 --------------------- sys/conf/files | 1 + sys/conf/files.amd64 | 1 - sys/conf/files.arm | 2 +- sys/conf/files.arm64 | 1 - sys/conf/files.i386 | 2 +- sys/conf/files.mips | 1 - sys/conf/files.powerpc | 1 - sys/conf/files.riscv | 1 - sys/i386/i386/{in_cksum.c => in_cksum_machdep.c} | 0 sys/i386/include/in_cksum.h | 3 + sys/{mips/mips => netinet}/in_cksum.c | 7 + sys/powerpc/powerpc/in_cksum.c | 253 ----------------------- sys/riscv/riscv/in_cksum.c | 243 ---------------------- 17 files changed, 16 insertions(+), 987 deletions(-) diff --git a/sys/amd64/amd64/in_cksum.c b/sys/amd64/amd64/in_cksum.c deleted file mode 100644 index 30e90f043cc3..000000000000 --- a/sys/amd64/amd64/in_cksum.c +++ /dev/null @@ -1,243 +0,0 @@ -/* $NetBSD: in_cksum.c,v 1.7 1997/09/02 13:18:15 thorpej Exp $ */ - -/*- - * SPDX-License-Identifier: BSD-4-Clause - * - * Copyright (c) 1988, 1992, 1993 - * The Regents of the University of California. All rights reserved. - * Copyright (c) 1996 - * Matt Thomas - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions - * are met: - * 1. Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * 2. Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. - * 3. All advertising materials mentioning features or use of this software - * must display the following acknowledgement: - * This product includes software developed by the University of - * California, Berkeley and its contributors. - * 4. Neither the name of the University nor the names of its contributors - * may be used to endorse or promote products derived from this software - * without specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE - * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE - * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS - * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) - * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT - * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY - * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF - * SUCH DAMAGE. - * - * @(#)in_cksum.c 8.1 (Berkeley) 6/10/93 - */ - -#include /* RCS ID & Copyright macro defns */ -__FBSDID("$FreeBSD$"); - -#include -#include -#include -#include -#include -#include -#include - -/* - * Checksum routine for Internet Protocol family headers - * (Portable Alpha version). - * - * This routine is very heavily used in the network - * code and should be modified for each CPU to be as fast as possible. - */ - -#define ADDCARRY(x) (x > 65535 ? x -= 65535 : x) -#define REDUCE32 \ - { \ - q_util.q = sum; \ - sum = q_util.s[0] + q_util.s[1] + q_util.s[2] + q_util.s[3]; \ - } -#define REDUCE16 \ - { \ - q_util.q = sum; \ - l_util.l = q_util.s[0] + q_util.s[1] + q_util.s[2] + q_util.s[3]; \ - sum = l_util.s[0] + l_util.s[1]; \ - ADDCARRY(sum); \ - } - -static const u_int32_t in_masks[] = { - /*0 bytes*/ /*1 byte*/ /*2 bytes*/ /*3 bytes*/ - 0x00000000, 0x000000FF, 0x0000FFFF, 0x00FFFFFF, /* offset 0 */ - 0x00000000, 0x0000FF00, 0x00FFFF00, 0xFFFFFF00, /* offset 1 */ - 0x00000000, 0x00FF0000, 0xFFFF0000, 0xFFFF0000, /* offset 2 */ - 0x00000000, 0xFF000000, 0xFF000000, 0xFF000000, /* offset 3 */ -}; - -union l_util { - u_int16_t s[2]; - u_int32_t l; -}; -union q_util { - u_int16_t s[4]; - u_int32_t l[2]; - u_int64_t q; -}; - -static u_int64_t -in_cksumdata(const void *buf, int len) -{ - const u_int32_t *lw = (const u_int32_t *) buf; - u_int64_t sum = 0; - u_int64_t prefilled; - int offset; - union q_util q_util; - - if ((3 & (long) lw) == 0 && len == 20) { - sum = (u_int64_t) lw[0] + lw[1] + lw[2] + lw[3] + lw[4]; - REDUCE32; - return sum; - } - - if ((offset = 3 & (long) lw) != 0) { - const u_int32_t *masks = in_masks + (offset << 2); - lw = (u_int32_t *) (((long) lw) - offset); - sum = *lw++ & masks[len >= 3 ? 3 : len]; - len -= 4 - offset; - if (len <= 0) { - REDUCE32; - return sum; - } - } -#if 0 - /* - * Force to cache line boundary. - */ - offset = 32 - (0x1f & (long) lw); - if (offset < 32 && len > offset) { - len -= offset; - if (4 & offset) { - sum += (u_int64_t) lw[0]; - lw += 1; - } - if (8 & offset) { - sum += (u_int64_t) lw[0] + lw[1]; - lw += 2; - } - if (16 & offset) { - sum += (u_int64_t) lw[0] + lw[1] + lw[2] + lw[3]; - lw += 4; - } - } -#endif - /* - * access prefilling to start load of next cache line. - * then add current cache line - * save result of prefilling for loop iteration. - */ - prefilled = lw[0]; - while ((len -= 32) >= 4) { - u_int64_t prefilling = lw[8]; - sum += prefilled + lw[1] + lw[2] + lw[3] - + lw[4] + lw[5] + lw[6] + lw[7]; - lw += 8; - prefilled = prefilling; - } - if (len >= 0) { - sum += prefilled + lw[1] + lw[2] + lw[3] - + lw[4] + lw[5] + lw[6] + lw[7]; - lw += 8; - } else { - len += 32; - } - while ((len -= 16) >= 0) { - sum += (u_int64_t) lw[0] + lw[1] + lw[2] + lw[3]; - lw += 4; - } - len += 16; - while ((len -= 4) >= 0) { - sum += (u_int64_t) *lw++; - } - len += 4; - if (len > 0) - sum += (u_int64_t) (in_masks[len] & *lw); - REDUCE32; - return sum; -} - -u_short -in_addword(u_short a, u_short b) -{ - u_int64_t sum = a + b; - - ADDCARRY(sum); - return (sum); -} - -u_short -in_pseudo(u_int32_t a, u_int32_t b, u_int32_t c) -{ - u_int64_t sum; - union q_util q_util; - union l_util l_util; - - sum = (u_int64_t) a + b + c; - REDUCE16; - return (sum); -} - -u_short -in_cksum_skip(struct mbuf *m, int len, int skip) -{ - u_int64_t sum = 0; - int mlen = 0; - int clen = 0; - caddr_t addr; - union q_util q_util; - union l_util l_util; - - len -= skip; - for (; skip && m; m = m->m_next) { - if (m->m_len > skip) { - mlen = m->m_len - skip; - addr = mtod(m, caddr_t) + skip; - goto skip_start; - } else { - skip -= m->m_len; - } - } - - for (; m && len; m = m->m_next) { - if (m->m_len == 0) - continue; - mlen = m->m_len; - addr = mtod(m, caddr_t); -skip_start: - if (len < mlen) - mlen = len; - if ((clen ^ (long) addr) & 1) - sum += in_cksumdata(addr, mlen) << 8; - else - sum += in_cksumdata(addr, mlen); - - clen += mlen; - len -= mlen; - } - REDUCE16; - return (~sum & 0xffff); -} - -u_int in_cksum_hdr(const struct ip *ip) -{ - u_int64_t sum = in_cksumdata(ip, sizeof(struct ip)); - union q_util q_util; - union l_util l_util; - REDUCE16; - return (~sum & 0xffff); -} diff --git a/sys/arm/arm/in_cksum.c b/sys/arm/arm/in_cksum_machdep.c similarity index 100% rename from sys/arm/arm/in_cksum.c rename to sys/arm/arm/in_cksum_machdep.c diff --git a/sys/arm/include/in_cksum.h b/sys/arm/include/in_cksum.h index a75433830d73..352d46f07e7b 100644 --- a/sys/arm/include/in_cksum.h +++ b/sys/arm/include/in_cksum.h @@ -40,6 +40,9 @@ #include #ifdef _KERNEL + +#define HAVE_MD_IN_CKSUM + u_short in_cksum(struct mbuf *m, int len); u_short in_addword(u_short sum, u_short b); u_short in_cksum_skip(struct mbuf *m, int len, int skip); diff --git a/sys/arm64/arm64/in_cksum.c b/sys/arm64/arm64/in_cksum.c deleted file mode 100644 index ae02e91d9203..000000000000 --- a/sys/arm64/arm64/in_cksum.c +++ /dev/null @@ -1,241 +0,0 @@ -/* $NetBSD: in_cksum.c,v 1.7 1997/09/02 13:18:15 thorpej Exp $ */ - -/*- - * Copyright (c) 1988, 1992, 1993 - * The Regents of the University of California. All rights reserved. - * Copyright (c) 1996 - * Matt Thomas - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions - * are met: - * 1. Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * 2. Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. - * 3. All advertising materials mentioning features or use of this software - * must display the following acknowledgement: - * This product includes software developed by the University of - * California, Berkeley and its contributors. - * 4. Neither the name of the University nor the names of its contributors - * may be used to endorse or promote products derived from this software - * without specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE - * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE - * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS - * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) - * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT - * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY - * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF - * SUCH DAMAGE. - * - * @(#)in_cksum.c 8.1 (Berkeley) 6/10/93 - */ - -#include /* RCS ID & Copyright macro defns */ -__FBSDID("$FreeBSD$"); - -#include -#include -#include -#include -#include -#include -#include - -/* - * Checksum routine for Internet Protocol family headers - * (Portable Alpha version). - * - * This routine is very heavily used in the network - * code and should be modified for each CPU to be as fast as possible. - */ - -#define ADDCARRY(x) (x > 65535 ? x -= 65535 : x) -#define REDUCE32 \ - { \ - q_util.q = sum; \ - sum = q_util.s[0] + q_util.s[1] + q_util.s[2] + q_util.s[3]; \ - } -#define REDUCE16 \ - { \ - q_util.q = sum; \ - l_util.l = q_util.s[0] + q_util.s[1] + q_util.s[2] + q_util.s[3]; \ - sum = l_util.s[0] + l_util.s[1]; \ - ADDCARRY(sum); \ - } - -static const u_int32_t in_masks[] = { - /*0 bytes*/ /*1 byte*/ /*2 bytes*/ /*3 bytes*/ - 0x00000000, 0x000000FF, 0x0000FFFF, 0x00FFFFFF, /* offset 0 */ - 0x00000000, 0x0000FF00, 0x00FFFF00, 0xFFFFFF00, /* offset 1 */ - 0x00000000, 0x00FF0000, 0xFFFF0000, 0xFFFF0000, /* offset 2 */ - 0x00000000, 0xFF000000, 0xFF000000, 0xFF000000, /* offset 3 */ -}; - -union l_util { - u_int16_t s[2]; - u_int32_t l; -}; -union q_util { - u_int16_t s[4]; - u_int32_t l[2]; - u_int64_t q; -}; - -static u_int64_t -in_cksumdata(const void *buf, int len) -{ - const u_int32_t *lw = (const u_int32_t *) buf; - u_int64_t sum = 0; - u_int64_t prefilled; - int offset; - union q_util q_util; - - if ((3 & (long) lw) == 0 && len == 20) { - sum = (u_int64_t) lw[0] + lw[1] + lw[2] + lw[3] + lw[4]; - REDUCE32; - return sum; - } - - if ((offset = 3 & (long) lw) != 0) { - const u_int32_t *masks = in_masks + (offset << 2); - lw = (u_int32_t *) (((long) lw) - offset); - sum = *lw++ & masks[len >= 3 ? 3 : len]; - len -= 4 - offset; - if (len <= 0) { - REDUCE32; - return sum; - } - } -#if 0 - /* - * Force to cache line boundary. - */ - offset = 32 - (0x1f & (long) lw); - if (offset < 32 && len > offset) { - len -= offset; - if (4 & offset) { - sum += (u_int64_t) lw[0]; - lw += 1; - } - if (8 & offset) { - sum += (u_int64_t) lw[0] + lw[1]; - lw += 2; - } - if (16 & offset) { - sum += (u_int64_t) lw[0] + lw[1] + lw[2] + lw[3]; - lw += 4; - } - } -#endif - /* - * access prefilling to start load of next cache line. - * then add current cache line - * save result of prefilling for loop iteration. - */ - prefilled = lw[0]; - while ((len -= 32) >= 4) { - u_int64_t prefilling = lw[8]; - sum += prefilled + lw[1] + lw[2] + lw[3] - + lw[4] + lw[5] + lw[6] + lw[7]; - lw += 8; - prefilled = prefilling; - } - if (len >= 0) { - sum += prefilled + lw[1] + lw[2] + lw[3] - + lw[4] + lw[5] + lw[6] + lw[7]; - lw += 8; - } else { - len += 32; - } - while ((len -= 16) >= 0) { - sum += (u_int64_t) lw[0] + lw[1] + lw[2] + lw[3]; - lw += 4; - } - len += 16; - while ((len -= 4) >= 0) { - sum += (u_int64_t) *lw++; - } - len += 4; - if (len > 0) - sum += (u_int64_t) (in_masks[len] & *lw); - REDUCE32; - return sum; -} - -u_short -in_addword(u_short a, u_short b) -{ - u_int64_t sum = a + b; - - ADDCARRY(sum); - return (sum); -} - -u_short -in_pseudo(u_int32_t a, u_int32_t b, u_int32_t c) -{ - u_int64_t sum; - union q_util q_util; - union l_util l_util; - - sum = (u_int64_t) a + b + c; - REDUCE16; - return (sum); -} - -u_short -in_cksum_skip(struct mbuf *m, int len, int skip) -{ - u_int64_t sum = 0; - int mlen = 0; - int clen = 0; - caddr_t addr; - union q_util q_util; - union l_util l_util; - - len -= skip; - for (; skip && m; m = m->m_next) { - if (m->m_len > skip) { - mlen = m->m_len - skip; - addr = mtod(m, caddr_t) + skip; - goto skip_start; - } else { - skip -= m->m_len; - } - } - - for (; m && len; m = m->m_next) { - if (m->m_len == 0) - continue; - mlen = m->m_len; - addr = mtod(m, caddr_t); -skip_start: - if (len < mlen) - mlen = len; - if ((clen ^ (long) addr) & 1) - sum += in_cksumdata(addr, mlen) << 8; - else - sum += in_cksumdata(addr, mlen); - - clen += mlen; - len -= mlen; - } - REDUCE16; - return (~sum & 0xffff); -} - -u_int in_cksum_hdr(const struct ip *ip) -{ - u_int64_t sum = in_cksumdata(ip, sizeof(struct ip)); - union q_util q_util; - union l_util l_util; - REDUCE16; - return (~sum & 0xffff); -} diff --git a/sys/conf/files b/sys/conf/files index f25508f1fded..8936cdaf5bd3 100644 --- a/sys/conf/files +++ b/sys/conf/files @@ -4333,6 +4333,7 @@ netinet/accf_http.c optional accept_filter_http inet netinet/if_ether.c optional inet ether netinet/igmp.c optional inet netinet/in.c optional inet +netinet/in_cksum.c optional inet | inet6 netinet/in_debug.c optional inet ddb netinet/in_kdtrace.c optional inet | inet6 netinet/ip_carp.c optional inet carp | inet6 carp diff --git a/sys/conf/files.amd64 b/sys/conf/files.amd64 index 6b51c1a5a55d..9a741906c868 100644 --- a/sys/conf/files.amd64 +++ b/sys/conf/files.amd64 @@ -62,7 +62,6 @@ amd64/amd64/exception.S standard amd64/amd64/exec_machdep.c standard amd64/amd64/fpu.c standard amd64/amd64/gdb_machdep.c optional gdb -amd64/amd64/in_cksum.c optional inet | inet6 amd64/amd64/initcpu.c standard amd64/amd64/io.c optional io amd64/amd64/locore.S standard no-obj diff --git a/sys/conf/files.arm b/sys/conf/files.arm index ce9c4933a024..4a5e9cc0bac9 100644 --- a/sys/conf/files.arm +++ b/sys/conf/files.arm @@ -35,8 +35,8 @@ arm/arm/gic.c optional gic arm/arm/gic_fdt.c optional gic fdt arm/arm/gic_if.m optional gic arm/arm/identcpu-v6.c standard -arm/arm/in_cksum.c optional inet | inet6 arm/arm/in_cksum_arm.S optional inet | inet6 +arm/arm/in_cksum_machdep.c optional inet | inet6 kern/subr_intr.c standard arm/arm/locore.S standard no-obj arm/arm/hypervisor-stub.S standard diff --git a/sys/conf/files.arm64 b/sys/conf/files.arm64 index 2b7c8bc569c7..722a33da812d 100644 --- a/sys/conf/files.arm64 +++ b/sys/conf/files.arm64 @@ -56,7 +56,6 @@ arm64/arm64/gic_v3.c standard arm64/arm64/gic_v3_acpi.c optional acpi arm64/arm64/gic_v3_fdt.c optional fdt arm64/arm64/identcpu.c standard -arm64/arm64/in_cksum.c optional inet | inet6 arm64/arm64/locore.S standard no-obj arm64/arm64/machdep.c standard arm64/arm64/machdep_boot.c standard diff --git a/sys/conf/files.i386 b/sys/conf/files.i386 index 0c681d6a84a0..bca5f04aeb1d 100644 --- a/sys/conf/files.i386 +++ b/sys/conf/files.i386 @@ -108,7 +108,7 @@ i386/i386/elf_machdep.c standard i386/i386/exception.s standard i386/i386/gdb_machdep.c optional gdb i386/i386/geode.c optional cpu_geode -i386/i386/in_cksum.c optional inet | inet6 +i386/i386/in_cksum_machdep.c optional inet | inet6 i386/i386/initcpu.c standard i386/i386/io.c optional io i386/i386/k6_mem.c optional mem diff --git a/sys/conf/files.mips b/sys/conf/files.mips index 6211c90f4e7e..777d4af57c06 100644 --- a/sys/conf/files.mips +++ b/sys/conf/files.mips @@ -20,7 +20,6 @@ mips/mips/exception.S standard mips/mips/fp.S standard mips/mips/freebsd32_machdep.c optional compat_freebsd32 mips/mips/gdb_machdep.c standard -mips/mips/in_cksum.c optional inet mips/mips/libkern_machdep.c standard mips/mips/locore.S standard no-obj mips/mips/machdep.c standard diff --git a/sys/conf/files.powerpc b/sys/conf/files.powerpc index 5d10740ce41d..25fdfea388b9 100644 --- a/sys/conf/files.powerpc +++ b/sys/conf/files.powerpc @@ -254,7 +254,6 @@ powerpc/powerpc/elf64_machdep.c optional powerpc64 | powerpc64le powerpc/powerpc/exec_machdep.c standard powerpc/powerpc/fpu.c standard powerpc/powerpc/gdb_machdep.c optional gdb -powerpc/powerpc/in_cksum.c optional inet | inet6 powerpc/powerpc/interrupt.c standard powerpc/powerpc/intr_machdep.c standard powerpc/powerpc/iommu_if.m standard diff --git a/sys/conf/files.riscv b/sys/conf/files.riscv index 72939b42d67d..e343b9b7f9b3 100644 --- a/sys/conf/files.riscv +++ b/sys/conf/files.riscv @@ -47,7 +47,6 @@ riscv/riscv/elf_machdep.c standard riscv/riscv/exception.S standard riscv/riscv/exec_machdep.c standard riscv/riscv/intr_machdep.c standard -riscv/riscv/in_cksum.c optional inet | inet6 riscv/riscv/identcpu.c standard riscv/riscv/locore.S standard no-obj riscv/riscv/machdep.c standard diff --git a/sys/i386/i386/in_cksum.c b/sys/i386/i386/in_cksum_machdep.c similarity index 100% rename from sys/i386/i386/in_cksum.c rename to sys/i386/i386/in_cksum_machdep.c diff --git a/sys/i386/include/in_cksum.h b/sys/i386/include/in_cksum.h index a8214e97cd9a..6b9c62942561 100644 --- a/sys/i386/include/in_cksum.h +++ b/sys/i386/include/in_cksum.h @@ -118,6 +118,9 @@ in_pseudo(u_int sum, u_int b, u_int c) #endif #ifdef _KERNEL + +#define HAVE_MD_IN_CKSUM + #if !defined(__GNUCLIKE_ASM) #if defined(IPVERSION) && (IPVERSION == 4) u_int in_cksum_hdr(const struct ip *ip); diff --git a/sys/mips/mips/in_cksum.c b/sys/netinet/in_cksum.c similarity index 98% rename from sys/mips/mips/in_cksum.c rename to sys/netinet/in_cksum.c index 8f9a2d9de106..581950c8afa4 100644 --- a/sys/mips/mips/in_cksum.c +++ b/sys/netinet/in_cksum.c @@ -48,6 +48,11 @@ __FBSDID("$FreeBSD$"); #include #include +/* + * These implementations may be overridden on a per-platform basis. + */ +#ifndef HAVE_MD_IN_CKSUM + /* * Checksum routine for Internet Protocol family headers * (Portable Alpha version). @@ -248,3 +253,5 @@ u_int in_cksum_hdr(const struct ip *ip) REDUCE16; return (~sum & 0xffff); } + +#endif /* !HAVE_MD_IN_CKSUM */ diff --git a/sys/powerpc/powerpc/in_cksum.c b/sys/powerpc/powerpc/in_cksum.c deleted file mode 100644 index 906dc7e8d9a3..000000000000 --- a/sys/powerpc/powerpc/in_cksum.c +++ /dev/null @@ -1,253 +0,0 @@ -/* $FreeBSD$ */ -/* $NetBSD: in_cksum.c,v 1.7 1997/09/02 13:18:15 thorpej Exp $ */ - -/*- - * SPDX-License-Identifier: BSD-4-Clause - * - * Copyright (c) 1988, 1992, 1993 - * The Regents of the University of California. All rights reserved. - * Copyright (c) 1996 - * Matt Thomas - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions - * are met: - * 1. Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * 2. Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. - * 3. All advertising materials mentioning features or use of this software - * must display the following acknowledgement: - * This product includes software developed by the University of - * California, Berkeley and its contributors. - * 4. Neither the name of the University nor the names of its contributors - * may be used to endorse or promote products derived from this software - * without specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE - * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE - * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS - * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) - * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT - * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY - * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF - * SUCH DAMAGE. - * - * @(#)in_cksum.c 8.1 (Berkeley) 6/10/93 - */ - -#include /* RCS ID & Copyright macro defns */ - -#include -#include -#include -#include -#include -#include -#include -#include - -/* - * Checksum routine for Internet Protocol family headers - * (Portable Alpha version). - * - * This routine is very heavily used in the network - * code and should be modified for each CPU to be as fast as possible. - */ - -#define ADDCARRY(x) (x > 65535 ? x -= 65535 : x) -#define REDUCE32 \ - { \ - q_util.q = sum; \ - sum = q_util.s[0] + q_util.s[1] + q_util.s[2] + q_util.s[3]; \ - } -#define REDUCE16 \ - { \ - q_util.q = sum; \ - l_util.l = q_util.s[0] + q_util.s[1] + q_util.s[2] + q_util.s[3]; \ - sum = l_util.s[0] + l_util.s[1]; \ - ADDCARRY(sum); \ - } - -static const u_int32_t in_masks[] = { -#if _BYTE_ORDER == _LITTLE_ENDIAN - /*0 bytes*/ /*1 byte*/ /*2 bytes*/ /*3 bytes*/ - 0x00000000, 0x000000FF, 0x0000FFFF, 0x00FFFFFF, /* offset 0 */ - 0x00000000, 0x0000FF00, 0x00FFFF00, 0xFFFFFF00, /* offset 1 */ - 0x00000000, 0x00FF0000, 0xFFFF0000, 0xFFFF0000, /* offset 2 */ - 0x00000000, 0xFF000000, 0xFF000000, 0xFF000000, /* offset 3 */ -#else - /*0 bytes*/ /*1 byte*/ /*2 bytes*/ /*3 bytes*/ - 0x00000000, 0xFF000000, 0xFFFF0000, 0xFFFFFF00, /* offset 0 */ - 0x00000000, 0x00FF0000, 0x00FFFF00, 0x00FFFFFF, /* offset 1 */ - 0x00000000, 0x0000FF00, 0x0000FFFF, 0x0000FFFF, /* offset 2 */ - 0x00000000, 0x000000FF, 0x000000FF, 0x000000FF, /* offset 3 */ -#endif -}; - -union l_util { - u_int16_t s[2]; - u_int32_t l; -}; -union q_util { - u_int16_t s[4]; - u_int32_t l[2]; - u_int64_t q; -}; - -static u_int64_t -in_cksumdata(const void *buf, int len) -{ - const u_int32_t *lw = (const u_int32_t *) buf; - u_int64_t sum = 0; - u_int64_t prefilled; - int offset; - union q_util q_util; - - if ((3 & (long) lw) == 0 && len == 20) { - sum = (u_int64_t) lw[0] + lw[1] + lw[2] + lw[3] + lw[4]; - REDUCE32; - return sum; - } - - if ((offset = 3 & (long) lw) != 0) { - const u_int32_t *masks = in_masks + (offset << 2); - lw = (u_int32_t *) (((long) lw) - offset); - sum = *lw++ & masks[len >= 3 ? 3 : len]; - len -= 4 - offset; - if (len <= 0) { - REDUCE32; - return sum; - } - } -#if 0 - /* - * Force to cache line boundary. - */ - offset = 32 - (0x1f & (long) lw); - if (offset < 32 && len > offset) { - len -= offset; - if (4 & offset) { - sum += (u_int64_t) lw[0]; - lw += 1; - } - if (8 & offset) { - sum += (u_int64_t) lw[0] + lw[1]; - lw += 2; - } - if (16 & offset) { - sum += (u_int64_t) lw[0] + lw[1] + lw[2] + lw[3]; - lw += 4; - } - } -#endif - /* - * access prefilling to start load of next cache line. - * then add current cache line - * save result of prefilling for loop iteration. - */ - prefilled = lw[0]; - while ((len -= 32) >= 4) { - u_int64_t prefilling = lw[8]; - sum += prefilled + lw[1] + lw[2] + lw[3] - + lw[4] + lw[5] + lw[6] + lw[7]; - lw += 8; - prefilled = prefilling; - } - if (len >= 0) { - sum += prefilled + lw[1] + lw[2] + lw[3] - + lw[4] + lw[5] + lw[6] + lw[7]; - lw += 8; - } else { - len += 32; - } - while ((len -= 16) >= 0) { - sum += (u_int64_t) lw[0] + lw[1] + lw[2] + lw[3]; - lw += 4; - } - len += 16; - while ((len -= 4) >= 0) { - sum += (u_int64_t) *lw++; - } - len += 4; - if (len > 0) - sum += (u_int64_t) (in_masks[len] & *lw); - REDUCE32; - return sum; -} - -u_short -in_addword(u_short a, u_short b) -{ - u_int64_t sum = a + b; - - ADDCARRY(sum); - return (sum); -} - -u_short -in_pseudo(u_int32_t a, u_int32_t b, u_int32_t c) -{ - u_int64_t sum; - union q_util q_util; - union l_util l_util; - - sum = (u_int64_t) a + b + c; - REDUCE16; - return (sum); -} - -u_short -in_cksum_skip(struct mbuf *m, int len, int skip) -{ - u_int64_t sum = 0; - int mlen = 0; - int clen = 0; - caddr_t addr; - union q_util q_util; - union l_util l_util; - - len -= skip; - for (; skip && m; m = m->m_next) { - if (m->m_len > skip) { - mlen = m->m_len - skip; - addr = mtod(m, caddr_t) + skip; - goto skip_start; - } else { - skip -= m->m_len; - } - } - - for (; m && len; m = m->m_next) { - if (m->m_len == 0) - continue; - mlen = m->m_len; - addr = mtod(m, caddr_t); -skip_start: - if (len < mlen) - mlen = len; - - if ((clen ^ (long) addr) & 1) - sum += in_cksumdata(addr, mlen) << 8; - else - sum += in_cksumdata(addr, mlen); - - clen += mlen; - len -= mlen; - } - REDUCE16; - return (~sum & 0xffff); -} - -u_int in_cksum_hdr(const struct ip *ip) -{ - u_int64_t sum = in_cksumdata(ip, sizeof(struct ip)); - union q_util q_util; - union l_util l_util; - REDUCE16; - return (~sum & 0xffff); -} diff --git a/sys/riscv/riscv/in_cksum.c b/sys/riscv/riscv/in_cksum.c deleted file mode 100644 index 30e90f043cc3..000000000000 --- a/sys/riscv/riscv/in_cksum.c +++ /dev/null @@ -1,243 +0,0 @@ -/* $NetBSD: in_cksum.c,v 1.7 1997/09/02 13:18:15 thorpej Exp $ */ - -/*- - * SPDX-License-Identifier: BSD-4-Clause - * - * Copyright (c) 1988, 1992, 1993 - * The Regents of the University of California. All rights reserved. - * Copyright (c) 1996 - * Matt Thomas - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions - * are met: - * 1. Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * 2. Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. *** 225 LINES SKIPPED ***